General Firefox 1.5.0.x Test Plan

Bug Verifications

Identify bugs to be verified

• Triage fixed bugs and categorize
  • Focus on security and critical bugs first
  • Add "testcase" keyword for those bugs that have reproducible steps and/or attached testcase(s)
  • Identify difficult to reproduce bugs that do not have a testcase
    • add [need testcase] in status whiteboard.
    • request more info from reporter, developer, or anyone interested in the bug
  • Spot check "no verification necessary" bugs
    • Usually internal architecture changes with no externally visible effects
    • Quick code inspection and checkin verification with bonsai

Confidence metrics and sign-off requirements

• Security bug verifications: 100%
• Critical bug verifications: 100%
• Non-critical bug verifications will be done on a case by case basis, depending on developer feedback and community testing.

QA resources

• MoCo QA is responsible for:
  • Security and critical bug verifications
  • Some non-critical bugs and special test cases
  • Providing bug lists to the community and pointing people to the wiki page
• Mozilla Community tasks:
  • Test and help verify fixes for non-security bugs
• Target markets and Mozilla partners:
  • IBM for enterprise deployment functionality
  • Mozilla Japan for jp-critical issues

Regression Testing

• Smoke Tests
• BFT on all three platforms
• Java Script
• Top Sites (See FFT)
• Key extensions
• Performance
  • Gross memory usage
• Security Test Library tests

tomcats results

Plattform: Windows 2000 SP4 / XP SP2

• Smoketests
  • PASS for 1508 en-US Firefox Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.8) Gecko/20061023 Firefox/1.5.0.8, all tests passed, results in litmus

• • PASS for Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.8) Gecko/20061024 Thunderbird/1.5.0.7 ID:2006102408

• • PASS for version 1.5.0.8 (20061025) Tbird

Install Testing

• Test en-US after initial packaging.
• Test locales after locale packaing.

L10n Testing

• More in depth testing of 5 locals using Litmus L10n tests (AKA Sara's checklist)
• basic install/version id checks
• L10n test matrix (add link here!)
• Axel's external L10n status and owner info

Extension Testing

• Top 10 AMO extensions according to download numbers.
• Recommended sites
• http://developer.mozilla.org/contests/extendfirefox/

Update Testing

This is the testing of the partial and full update .mar files for the various locales once the main downloads are released.

Community Testing

• Mozilla QA Community Wiki - Please go here for all community related updates and activities.

Configuration Testing

We need to describe our configuration testing approach better. For now, we are generally testing these OS's:

• Windows XP
• MacOS 10.3, 10.4
• Linux Ferdora Core 4

But we need to consider our system requirements statements. I have seen at least three versions. Here is one:

• mozilla.com->Products Tab -> Firefox -> System Requirements link