QA/Firefox3/TestPlan/MajorUpdate

From MozillaWiki
< QA‎ | Firefox3‎ | TestPlan
Jump to: navigation, search

Major Update Test Plan: Fx 2.0x - Fx 3.0x


Overview

We are offering major updates to Firefox 2.0.0.x users so they can upgrade to Firefox 3. There has been one successful Major Update so far, from 2.0.0.16 to 3.0.1. The current plan is to offer a Major Update from 2.0.0.18 to 3.0.4 during the next few weeks and then a second one from 2.0.0.19 to 3.0.5, which will hopefully be the final MU.

Scope

We are going to test basic major update functionality in some P1 locales, CJKT locales, and partner builds. We want to make sure basic functionality in the resulting build is the same as a fresh Fx3 install, and we must ensure user preferences are preserved.

Most of this testing will be manually executed. However the partner distribution builds will, in addition, be tested with some automation. The major update test plan for partner distributions is here.

We will need to test MU for CJKT builds as these have special preferences that need to be preserved (?) after the major update. For refence see: CJKT Matrix for Fx1.5x - Fx2 MU.

Schedule

The Major Update will probably ship in the week following Thanksgiving, December 1 through 5. This is also the time period when the first Firefox 2.0.0.19 and 3.0.5 builds will become available so testing needs to be finished for the MU before then.

Current schedule for QA is:

  • Begin QA in Betatest channel: 11/18
  • Release to Beta: 11/20
  • Testing in Releasetest channel: 12/1
  • Release: 12/2

We need to test the major update scenario in the betatest channel, release it to the beta channel and then do releastest and release following this.

Basic Tests

At a basic level we should make sure:

  • Updates are presented
  • In-line content for Details and EULA is localized
  • Update does not break basic functionality for the user
  • User data and preferences are preserved

Extended Tests

The extended tests take into account rich user profiles. For this, we will start with an Fx2 build with bookmarks, history, saved form data, saved passwords, feeds, extensions and themes (mix of compatible and incompatible with Fx3), plugins, default search engine preferences, and any other customizations a user may have for a typical Fx200x installation.

During the major update the extensions and themes should be checked for compatibility.

After going through the major update all (most) of the user's preferences should be preserved.

Instructions

  • Download Firefox 2.0.0.20
  • Install it, shut down and change the update channel to betatest
    • it's in <app dir>/defaults/pref/channel-prefs.js
  • Restart the browser and select "Check for Updates" from the Help menu

Current Results

We will track the 2.0.0.20 -> 3.0.10 Major Update testing results in the Fx20020-Fx3010MU Test Plan.

Previous Results

For previous iterations of MU testing results see the following links as they are produced:

Reference (legacy)

A detailed list of scenarios QA must watch out for is here: http://wiki.mozilla.org/Firefox3/UpgradePolicy

For the 1.5x to 2.0x major updates testing we created a number of tests in Litmus. They are in the 1.5x - 2.0x Major Update testgroup and FX 1.5-2.0 Major Update - Basic Tests and FX 1.5-2.0 Major Update - Extended Tests subgroups in the Firefox 2.0 Catchall test run.

The major update dialog will have an explanatory page, and it will be localized. For example, the 1.5x - 2.0x MU had dialogs that look like this: http://www.chevrel.org/mozco/20migration/display.php

The following is a list of tests taken from the "1.5x - 2.0x Major Update" test group in Litmus, as explained above. 

check Details and EULA description
check that you can only install if you accept license
check eulas and descriptions are localized (Paul K., send a list of those that are ready)
check major update works on different locales
check interface elements work, buttons, links, text selection, scrolling, closing
check "fallback" major updates (should there be a failure in the update download for example)
check uninstalling and going back to previous build
check only one type of update is presented at a time minor/major
check major updates if you have multiple profiles
check extension compatibility check
check theme compatibility check
check what tabs are default after you update.
check these start pages are localized as before.
check Windows bits are signed after major update.
post: check minors are presented to the upgraded 2.0.0.1 users
post: check major updates are presented only to those using the latest 150x
post: check minor updates are presented for those users who said never
post: check 200x users are not presented a major
post: check majors a presented when user had said never, then manually checked for updates again (assuming major are once again available)
check to see what happens when one type of update has been downloaded and not applied, and another type of update is made available.
check to see if it is possible to download and say later and then regret and then install something better, like a major update.
check what is left over after a "diff" (currently some crumbs are left for Win and Mac and it has been captured in a bug)
check the state of the registry after a major update (and uninstall)
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Firefox3/TestPlan/MajorUpdate&oldid=143856"