Firefox:1.5.0.2:Test Plan

From MozillaWiki
Jump to: navigation, search

DRAFT - IN PROGRESS

Firefox 1.5.0.2 Test Plan

Bug Fix Verification

  • Identify list of bugs
    • Triage fixed bugs and categorize
      • ready for testing
      • test case needed
        • request more info from interested parties
      • no verification necessary
        • usually internal architecture changes with no externally visible effects
  • Set threshold for which bug fixes must be verified
    • 60% of the Firefox specific bugs (Example: not seamonkey, non-core platform, or Thunderbird specific fixes)
    • All critical security relevant bugs
  • Verify security fixes and some non-security fixes in house
  • Get community to help verify fixes for non-security bugs
  • Get members of target market to help verify strategic marketing changes
    • IBM for enterprise deployment functionality
    • Mozilla Japan for jp-critical issues

Regression Testing

  • Smoke Tests
  • BFT on all three platforms
  • Java Script
  • Top Sites (See FFT)
  • Key extensions
  • Performance
    • Gross memory usage
  • Security Test Library tests

Install Testing

  • Test en-US after initial packaging.
  • Test locales after locale packaing.

L10n Testing

Extension Testing

Update Testing

This is the testing of the partial and full update .mar files for the various locales once the main downloads are released.

  • Firefox 1.5.0.1 will use the following urls to determine if there is an update available:
  • Replace "release" in the url with the channel name to test other channels

Staging and all.html testing

  • This is consolidated with the FFx 1.0.8 testing in the 1.0.8 test plan. This is currently an internal wiki but is can be moved to the public wiki. For now there is too many other things going on. If you want to see this, just ping me (timr at mozilla.com) and I can fix that.

Community Testing

See internal and external results in each of these pages

Location of Downloads

  • Nightlies

Windows: ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-aviary1.0.1/firefox-1.0.8.en-US.win32.installer.exe MacOS: ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-aviary1.0.1/firefox-1.0.8.en-US.mac.dmg Linux: ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-aviary1.0.1/firefox-1.0.8.en-US.linux-i686.installer.tar.gz

  • Location of RC builds:

TBD

Configuration Testing

We need to describe our configuration testing approach better. For now, we are generally testing these OS's:

  • Windows XP
  • MacOS 10.3, 10.4
  • Linux Ferdora Core 4

But we need to consider our system requirements statements. I have seen at least three versions. Here is one:

mozilla.com->Products Tab -> Firefox -> System Requirements link


Mac OS X Universal Builds

  • Done:
    • Verification of bugs on ppc-only build [done]
    • Smoke test on ppc-only build [done]
    • Smoke test of universal build on ppc 10.2, 10.3, and 10.4
    • Smoke test of universal build running native x86 [done]
    • Smoke test of universal build running ppc-emulated on x86 mac [done]
    • focused testing of universal build issues
    • Flash on 10.4.6 works native (as version 8,0,24,0)
    • Flash on 10.4.6 works under rosetta (as version 8,0,27,0)
    • Flash on 10.4.4 (clean install) works native (as version 8,0,22,0)
    • Flash on 10.4.4 (clean install) works under rosetta (as version 8,0,22,0)
  • TODO:
    • bft on universal build on x86 (native and emulated)
      • Can do now: 2 x 4hrs
      • anyone with x86 mac
        • Marcia: native: PASS
        • Timr/marcia: under Rosetta: in progress - marcia wrapping up (4/11/06)
    • NEW: check talkback symbols when UB crashes on ppc mac
      • Can do now: 10 minutes - PASS - Davel 4/11/06
    • NEW: BFT of UB on ppc mac running 10.2 and 10.4.x
      • Can do now: have PPC Mac with these OS level in the QA lab. ~ 2 x 4 hrs
      • Who: marcia, tim, jay, davel?
        • Jay: 10.2 PASS
        • tracy: 10.4.x PASS
    • NEW: validate l10n of UB (new platform issues)
      • Need build. Builds can be fragile. 1-3 days depending on if issues come up
      • Testing: davel can use a diff tool. Then 4-8 hrs depending on what diffs say. Can be split up by 1-4 people
    • NEW: validate software update of UB (new platorm issues)
      • partial mar applies correctly
        • Need to builds mars. This process can be fragile. 1-3 days depending on issues
        • testing: davel- can tool help here?
      • aus serves correct pointers to updates
        • Easy to do once partials area available
        • Who: mark, josh?
      • spot check of builds and update paths
        • When updates are available.
        • Who: 1-4 people running through the rvarious update paths depending on what automated testing can check
        • 4-8 hrs depending on if tool available
    • Do we need an RC? If this is the primary download then we better be happy with it. IWC no RC needed.

Overall I think this adds up to 2-6 days depending on if anything goes wrong and how focused Build and QA can be. Two days could happen but that is probably not realistic.


Note: NEW = new as of the UB meeting on 2006.04.06

Test Results

Retrieved from "https://wiki.mozilla.org/index.php?title=Firefox:1.5.0.2:Test_Plan&oldid=23977"