CrypoTestingSummit2006

Session Title

Crypto related testing issues with Bob Lord and Co

Session Leader

Marcia Knous, Tim Riley, Bob Lord

Summary

Overview of crypto preferences for FFx and Tbird and how to test them better

Agenda

We are thinking about a out-of-band meeting at 2:15 or 3:30pm Thursday in Building K. Contact Tim Riley (IRC timr, IM tim_riley@yahoo, email timr@mozilla.com) or Marcia Knous. for details.

• Crypto overview, understanding the intricacies of certificates in Firefox and Tbird.
  • Ways we can improve the automated testing of certs in Thunderbird and other ways automated testing may help?
• Improving our manual test of Security in Litmus.
  • Currently many tests are just visual confirmation that the UI is there.
  • Which areas are the most important for us to test?
• These security experts plan to join us: Bob Lord, Bob Relyea, Kai Engert, and Chandra Kannan

Interested Attendees

• Tim Riley
• Marcia Knous
• Alice Nodelman
• Bob Lord
• Bob Relyea
• Kai Engert
• Chandra Kannan
• Robert Sayre
• Chris Cooper
• Nelson Bolyard
• robcee
• Wan-Teh Chang
• Juan Becerra

Meeting Notes

Crypo testing 11/15/06

Attendees

Bob Lord, Bob Relyea, Bob Clary, Nelson Bolyard, Wan-Teh Chang, Chandra Kannan, Rob Campbel, Juan Becerra, Tracy Walker, Chris Cooper, Alice Nodelman, Tim Riley,

Review Agenda

• Crypto problems found in Firefox 1.5.0.7
  • RSA signature issue
    • found by very creative cryptographers - may or may not be a real vulnerability
  • 7 bad certificates
• Desire to do better manual testing of Security/Crypo IU's
• Interest in using existing automated test suites

Some recent work

• NSS run nightly
  • what branches? Trunk and NSS 3.11 branch
  • doesn't crash and doesn't leak (anymore!)
  • 2Million tests and still problems found
    • test certs Generated by university in Finland
    • 4-5 CDs - ton of certs
  • Can these get out of date? [robcee]
    • [Nelson] then might
    • RSA public keys
    • Now elliptic curve certs
    • certs don't get out of date, but new technologies come along
• Tests run from tinderbox
  • SSL
  • SMIME

• Agreement by Mozilla to take updates
  • AI: Rob's team to check who made this commitment

Testing the UI

• SMIME
• TSL/ECC - This is main area. need to normalize this one
• Open SSL
• Test matrix between platforms and browsers
• Have seen regressions in:
  • Client Auth
  • RSA Keygen

MoCo QA doesn't know how to test encryption UI

• Tracy: don't understand how test UI
  • Seems well tested before we (MoCo QA) gets it
  • Chandra is a guru on PKI

Smartcard testing in Firefox

• Bob R
  • Get MoCo some USB smartcards
  • There is smartcard support in FF1.5
  • Bob R added hooks for registering when smartcard is inserted
  • Could have a special page where if smartcard is inserted it will take you to a special page
  • Need to setup public servers for testing (MoCo, Sec Test)
  • Setup automation to capture info about TLS session (see Bob L's demo)
    • See Chandra for ideas
  • MoCo, Sec Team to collaboration

PSM Testing

• Better testing [Nelson]
  • PSM - core Firefox component
    • Configuring PKI
    • QA on PSM??
  • Great opportunity for MoCO to create unit tests
    • What is the use case?? [juan]
    • Talk to Kai E and Chandra << AI Who??

SMIME

• SMIME
  • Automated tests?
  • Nelson seen many regressions in mail
    • signed mail get reported as invalid signatures
    • More trouble with IMAP
      • Setup messages on IMAP server
        • check for valid messages and attachments
        • try different IMAP servers
        • Setup canned set of messages (on CD, public server)
        • Lots of energy here!!

Setup Test Servers

• Need follow-up with Kai
  • Has a server with a ton of tests
  • Setup meeting with him <<== timr

Misc

• Shopping/SSL testing is a good starting point
• IE trashing
  • Warning: SSL is about to be used
  • requiring certs - sign by default even if you don't have a cert!
    • Then the message is rejected because no cert