« QA/Firefox3.1/TestPlan

Firefox 3.1 Beta1 Test Results

The covered testing for this release will focus on smoketests, bfts, l10n and spotchecks. This will only be on the en-US locale.

Related Pages

Firefox 3.1 Tracking Site

• Beta 1 features

Contact Info

PM: Mike Beltzner (beltzner)
Build: John O'Duinn (joduinn)
QA: Tony / others

Schedule

Code Freeze

9/30

Builds hand to QA

10/8

QA test signoff

10/14

• Smoketests
• BFTs
• L10n
• Updates
• Spotchecks

Build Location and IDs

Download:

• http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/3.1b1-candidates/build1
• ftp://ftp.mozilla.org/pub/firefox/nightly/3.1b1-candidates/build1/

Builds:

• Windows
  • Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1
    • 20081007144708
• Mac
  • Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1 (Full Build ID is: 20081007125523)
  • Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1
  • Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1
• Linux: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1

Test Coverage

• Areas Covered:
  • Smoketests
  • BFTs
  • L10n
  • Updates
  • Spotchecks & Webpages
  • Bug Fixes

• Areas Not Covered:
  • FFTs
  • Topsites
  • Accessibility

Test Results

Types of Tests	Windows XP	Mac 10.5	Linux
Smoketests	juanb - PASS	marcia - PASS	tomcat - PASS
BFTs	tracy - FAIL	marcia - PASS	Joel - PASS
L10n	Tony - PASS	Tony - PASS	Tony - PASS
Updates	tracy Betatest - PASS releasetest - PASS Beta - result	Tony Betatest - PASS releasetest - PASS Beta - result	Tony / (Tomcat for beta) Betatest - PASS releasetest - PASS Beta - result
Live Spotchecks & Webpages	juanb - Pass	marcia - result	Tomcat - result

Smoketest Testing

Lite verification tests (or smoketests) will be first ran when the builds are ready. Tests are available on litmus under the Firefox 3.1 test suite. In any case the Smoketests fail, we will raise blockers asap before proceeding with the bfts.

Please record your detailed results in the section below.

BFT Testing

BFTs are a more comprehensive set of litmus tests that cover the build testing after smoketests pass. Tests are available on litmus under the Firefox 3.1 test suite.

Please record your detailed results in the section below.

Updates Testing

We will be doing updates of all Alpha users to Beta1. We will be checking complete updates, and failover scenarios.

Will be available on betatest, releasetest, and beta channels.

Testcases and Results are tracked here.

L10N

Visual inspection: Go through the P1 list of locales, and run a spotcheck to see if everything visually looks fine. Refer to L10N Requirements for more specifics.

Testcases and Results are tracked here

• bug 459156 - BLOCKER :zhCN beta 1 builds show different search plugins per OS filed. A respin of zhCN only will be made on 3 platforms.
  • Update - Issue fixed on respin.

Live Spotchecks & Webpages

1) Verify the following webpages that gets pushed to staging and then to the live sites. Look for content messaging, spelling mistakes, broken links & imnages, etc..

• Firstrun page
• Whatsnew Page
• Releasenotes page

2) Check live site download links on all platforms. Please cover en-US, and 2 or 3 additional Tier 1 locales as well. Pay attention to the following areas:

• Check for signed installer and applications (windows only)
• Firstrun page, whatsnew page, Home page, Search plugin
• Install, uninstall
• General spotchecks and browser functionality
• Confirm Build IDs

Fixed Bugs Verifications

The following bugs are fixed for Beta 1 and need verifications:

• Platform 1.9.1 bugs: http://tinyurl.com/4c4qwb
• Firefox 3.1 bugs: http://tinyurl.com/3tw7zw

Testday

A Beta1 testday will be held and call for the community support to assist with testing. Activities include executing litmus testcases, bug verifications, and focus on the new feature areas that are being implemented. More can be found on QMO.

Detailed Results

Smoketests Results

Windows XP [juanb]

• Windows XP SP3 (vm)
• Smoketests Results in Litmus - PASS
• Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1
  • 20081007144708
  • bug 459082 - (filed) migration wizard not coming up
  • bug 458579 - (existing) Feed tab missing in page info

Mac [marcia]

• 10.5 Results, PASS, results in Litmus
  • Build ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1
  • Bugs Filed
    • Bug 458968 - Firefox Mac Beta 1 candidate build is branded "Minefield"

Linux [tomcat on Fedora F9]

• PASS - Results in Litmus
  • Build ID: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1

BFT Results

Windows XP [tracy]

• Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1 (20081007144708)

Failed form manager test case at www.google.com. Filed bug 459399 also seeing: bug 459157 Exception in (NS_ERROR_INVALID_POINTER) [nsIPermissionManager.add] failure blocking images from Page Info

Mac [marcia]

• Intel Mac 10.5, PASS, results in Litmus
• Build ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1
• New Bugs Filed:
  • Bug 459111 Crash in [@ nsNavHistoryResult::OnClearHistory() ] when deleting individual history entry
  • Bug 459157 Exception in (NS_ERROR_INVALID_POINTER) [nsIPermissionManager.add] failure blocking images from Page Info
  • Bug 459256 Error: [Exception... "'Error: %82' when calling method: [nsIDOMEventListener::handleEvent]
  • Bug 459406 "Save" button in query search is misaligned
• Existing Bugs Seen:
  • • Bug 428602

Linux [jmaher]

• Ubuntu 8.04 (vm) - FAIL
• Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1
• Bugs
  • New:
    • [bug 459282] session restore dialog appears after a system reboot when it shouldn't
    • [bug 459289] when firefox abnormally terminates and a download is active, it continues to download if you start a new session
    • [bug 459108] doing an auto search from the location bar doesn't work for numbers
  • Existing:
    • Bug 459157] Exception in (NS_ERROR_INVALID_POINTER) [nsIPermissionManager.add] failure blocking images from Page Info
    • [bug 374189] [Linux/SunOS] Ctrl+Y for "Downloads" doesn't work when focus is in a textbox (conflicts with "Redo")