« Firefox 18 | Firefox 19 | Firefox 20 »

Firefox 19 Test Plan

Summary

Milestone	Firefox 19
Status	Currently in Beta -- merging to Release on February 19th, 2013
Lead	Anthony Hughes (irc: ashughes)
2nd	Juan Becerra (irc: juanb)

The following is the test plan for Firefox 19 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 19. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Details

The following is a description of the tasks which need to be completed throughout this release.

2012-10-08: [DONE] Nightly Merge
2012-11-23: [DONE] Aurora Merge
2013-01-07: [DONE] Pre-Beta feature sign-offs
2013-01-10: [DONE] Beta 1
2013-01-17: [ON TRACK] Beta 2
2013-01-24: {{|}} Beta 3
2013-01-31: {{|}} Beta 4
2013-02-07: {{|}} Beta 5
2013-02-14: {{|}} Beta 6
2013-02-15: {{|}} Release Candidate
2013-02-19: {{|}} Final

Nightly

Merge

Functional Automation - result

Update Spotchecks - result

Features

See the Feature Release Tracking wiki for a general overview

Features

See the Feature Release Tracking wiki for a general overview

Feature	Owner	Status	Tests
Panel-based Download Manager	Simona Badau	Delayed to Firefox 20
Tab Strip Visual Redesign	Virgil Dicu	Delayed
Windows Plugin Hang UI	Manuela Muntean	Delayed
Show PDF inline	Mihaela Velimiroviciu	On Track
Awesomebar Search Provider Reset		On Track

Endurance Automation

The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard.

What to Do

Check the Endurance Charts
If you see an anomalous event (ie. spike or dip), get the report for the specific event
If the event happens twice in a row, file a bug under Mozmill Tests
Notify Anthony Hughes of the issue
Record the event in this wiki below

Current Status (Mihaela Velimiroviciu) - Last Updated - Nov 1st, 2012

Full Query

ID	Summary	Priority	Status
807338	Memory increase discovered by mozmill endurance tests - Mac 10.7&10.8 since Oct25	--	RESOLVED

1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);

Functional Automation

The purpose of this section is to track any potential functional regressions via results reported in the dashboard.

What to Do

Check the testrun reports
If you see any failing tests, check that there are not any bugs on file for it already
If not, file a bug under Mozmill Tests
Notify Anthony Hughes of the issue
Record the event in this wiki below

Current Status (Vlad Maniac / Alex Lakatos) - Last Updated

Bug Triage

The purpose of this section is to triage UNCONFIRMED incoming bugs.

What to Do

Check the UNCONFIRMED bugs
Test the bugs in the latest Firefox Nightly
If you can't reproduce the bug, ask for the reporter to provide more information
If you reproduce the bug, add your testing information and mark the bug NEW
Notify Anthony Hughes of the NEW issue

Current Status (SoftVision) - Last Updated

Result Spreadsheet

Aurora Merge


Builds	nightlies
Changeset	tip (pushlog)

Checklist

[DONE] Automated Testing
[DONE] Update Testing on auroratest
[DONE] Peer Sign-off
[DONE] Final sign-off
[DONE] Update Testing on aurora

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	PASS: report
Windows XP	PASS - 2012-11-23
Windows Vista	PASS - 2012-11-23
Windows 7 32-bit	PASS - 2012-11-23
Windows 7 64-bit	PASS - 2012-11-23
Windows 8 32-bit	PASS - 2012-11-23
Windows 8 64-bit	PASS - 2012-11-23
Mac OS X 10.6	PASS - 2012-11-23
Mac OS X 10.7	PASS - 2012-11-23
Mac OS X 10.8	PASS - 2012-11-23
Ubuntu 12.04 32-bit	PASS - 2012-11-23
Ubuntu 12.04 64-bit	PASS - 2012-11-23

Test Failures (known bugs)

Update Testing

Spotcheck updates from a previous Aurora to Aurora 19.

Platform	Aurora channel
Windows 7 64-bit	PASS - 2012-11-26	Firefox 17.0a2 2012-10-11 -> Firefox 19.0a2 2012-11-26
Windows 8 64-bit	PASS - 2012-11-26	Firefox 18.0a2 2012-11-15 -> Firefox 19.0a2 2012-11-26
Mac OS X 10.7	PASS - 2012-11-26	Firefox 18.0a2 2012-11-19 -> Firefox 19.0a2 2012-11-26
Ubuntu 12.04 64-bit	PASS - 2012-11-26	Firefox 17.0a2 2012-10-01 -> Firefox 19.0a2 2012-11-26

Issues

Bug Triage

VERIFYME: verify the fix and do some exploratory testing around the issue
FIXED: triage these bugs and test to ensure the fix is effective
QAWANTED: triage these bugs and test to fulfill the QA request
UNCONFIRMED: triage these bugs for potential new regressions

QAWANTED

Bugs requiring QA's assistance in debugging (steps, regression range, exploratory testing, verification, etc)

Status as of 2012-12-20:

bug 820805 crash in js::ion::IonBailoutIterator::IonBailoutIterator or js::ion::OperatorIn
- WORKSFORME - Unclear if qawanted is still neeeded (marcia)
bug 818626 crash in nsStyleContext::~nsStyleContext
- ASSIGNED - Untracked due to failed attemtps to reproduce, no more leads
bug 823119 crash in mozilla::gfx::DrawTargetD2D::CopySurface with PDF Viewer
- NEW - Reproduced many stability issues but not the specific signature (ashughes)
bug 668583 crash @ js::gc::ScanRope
- NEW - Pending feedback from Engineering about leads for QA to test (ashughes)
bug 698296 Firefox Crash Report @ je_free | js::gc::Arena::finalize
- NEW - So far unreproducible, any other leads? (ashughes)

Pre-beta Feature Sign-off

Firefox 19.0a2 Feature Status

Feature	Owner	Status
In-line PDF Viewer	MihaelaV (SV)	Ready for Beta
Awesomebar Search Provider Reset (bug 718088)	Paul Silaghi (SV), Manuela Muntean (SV)	Will be tested via Testday
Mac OSX 10.8 Notification Center Support (bug 728106, bug 777409)	unassigned	Moved to Firefox 20
Panel-based Download Manager	Simona Bidau (SV)	Moved to Firefox 20
Windows Plugin Hang UI	Manuela Muntean (SV)	Moved to Firefox 20
Tab Strip Visual Redesign	Virgil Dicu (SV)	Moved to Firefox 20

Beta Checklist

The following checklist should be completed before signing off for Release.

Beta 1

[DONE] Breakpad End-to-End functionality
[DONE] Mac OSX 10.8 Signing
[DONE] Add-on compatibility (top AMO, top third-party, toolbars, yandex, avg, norton, yahoo, bandoo, conduit, etc)
[DONE] Exploratory testing and bug triage against the flagship feature
Platforms: Windows 7 32-bit, Windows 8 64-bit, Mac OSX 10.8, Ubuntu 12.04 32-bit

Beta 2

[DONE] Breakpad End-to-End functionality
[DONE] Stop accepting MD5 as a hash algorithm in signatures
[DONE] Plug-in compatibility (Flash, Java, Silverlight, Realplayer, Quicktime, Shockwave, Unity [Mac])
[DONE] Firefox Beta testday stressing flagship features and bug triage
[DONE] Exploratory testing and bug triage against the flagship feature
Platforms: Windows 7 64-bit, Windows 8 32-bit, Mac OSX 10.7, Ubuntu 12.04 64-bit

Beta 3

[DONE] Breakpad End-to-End functionality
[DONE] Mac OSX 32-bit mode
[DONE] Web compatibility (search, webmail, banking, video, audio, gaming, news, cms, etc.)
[DONE] Exploratory testing and bug triage against the flagship feature
Platforms: Windows 7 32-bit, Windows Vista, Mac OSX 10.6, Ubuntu 12.04 32-bit

Beta 4

[DONE] Breakpad End-to-End functionality
[DONE] Verify update applies with root owned files
[DONE] Add-on compatibility (top AMO, top third-party, toolbars, yandex, avg, norton, yahoo, bandoo, conduit, etc)
[DONE] Exploratory testing and bug triage against the flagship feature
[DONE] Firefox Beta testday stressing flagship features and bug triage
Platforms: Windows 7 64-bit, Windows XP, Mac OSX 10.8, Ubuntu 12.10 32-bit

Beta 5

[DONE] Breakpad End-to-End functionality
[DONE] What's New Page on Firefox update
[DONE] Plug-in compatibility (Flash, Java, Silverlight, Realplayer, Quicktime, Shockwave, Unity [Mac])
[DONE] Exploratory testing and bug triage against the flagship feature
Platforms: Windows 7 32-bit, Windows 8 64-bit, Mac OSX 10.7, Ubuntu 12.04 32-bit

Beta 6

{{ |}} Breakpad End-to-End functionality
{{ |}} Mac OSX 10.5 and Windows 2000 de-support (install and update)
{{ |}} Web compatibility (search, webmail, banking, video, audio, gaming, news, cms, etc.)
{{ |}} Exploratory testing and bug triage against the flagship feature
Platforms: Windows 7 64-bit, Windows 8 32-bit, Mac OSX 10.6, Ubuntu 12.04 64-bit, Mac OSX 10.5, Windows 2000

Beta 1

Builds	FTP, HTTP
ID	20130109111322
Changeset	222e6877be4b (pushlog)

Checklist

[DONE] Functional Automation
[DONE] Manual Testing
[DONE] Bug Triage
[DONE] Update Automation on betatest channel
[DONE] Update Automation on releasetest channel
[DONE] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Functional Results	Restart Results
Windows XP	[DONE] 0 Fails	[DONE] 0 Fails
Windows Vista	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.6	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.7	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.8	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 64-bit	[DONE] 0 Fails	[DONE] 0 Fails

Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Focus Area	Windows 7	Windows 8	Mac OSX 10.7	Mac OSX 10.8	Ubuntu 12.04
Regression Tests: Run the Firefox 19.0b1 Regression Tests in MozTrap. Do some exploratory testing around the testcases to try to exercise some edge-cases.	[DONE] Manuela 1 Bugs	[DONE] Simona 0 Bugs	[DONE] Bogdan 0 Bugs	[DONE] Paul 0 Bugs	[DONE] Alexandra 0 Bugs
PDF.js: Test the new built-in PDF reader to check for issues with viewing and printing PDFs. Check some of the behaviour with other PDF readers side-loaded (ie. Adobe Reader, Foxit Reader, etc).	[DONE] Manuela 0 Bugs	[DONE] Simona 1 Bugs	[DONE] Bogdan 0 Bugs	[DONE] Paul 0 Bugs	[DONE] Cornel 0 Bugs
Add-on Compatibility: Test the top add-ons for stability issues and functional issues. Test featured add-ons from AMO and Third-party toolbars like Yahoo, Yandex, AVG, Amazon, Norton, etc.	[DONE] Manuela 1 Bugs	[DONE] Simona 1 Bugs	[DONE] Bogdan 0 Bugs	[DONE] Paul 0 Bugs	[DONE] Cornel 0 Bugs
Plug-in Compatibility: Test the latest plug-ins for stability issues and visual glitches. Test plugins like Flash, Reader, Java, Quicktime, Realplayer, Windows Media Player, Silverlight, etc.	[DONE] Manuela 0 Bugs	[DONE] Simona 0 Bugs	[DONE] Bogdan 0 Bugs	[DONE] Paul 0 Bugs	[DONE] Cornel 0 Bugs
Web Compatibility: Test some of the top visited websites to look for stability issues and UI glitches. Some suggestions include WebMail (including built in messengers), Social Networks, Video sites, Media streaming websites, News websites, Content Management systems, Web banking, HTML5 video, WebGL/MapsGL, User forums	[DONE] Manuela 0 Bugs	[DONE] Simona 0 Bugs	[DONE] Bogdan 0 Bugs	[DONE] Paul 0 Bugs	[DONE] Cornel 0 Bugs

Update Testing

Results of running the Mozmill update automation.
18.0b6 should get a partial update, <=18.0b5 should get a complete update.

Platform	Betatest (2013-01-10)	Releasetest (2013-01-10)	Beta (2013-01-10)
Windows XP	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows Vista	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 7 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 7 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 8 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 8 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.6	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.7	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.8	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Ubuntu 12.04 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Ubuntu 12.04 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails

Bug Triage

QAWANTED

Bugs: Test to find steps to reproduce, regression range, or testcases	Status
bug 820506 Problem selecting image node in SVG when used with clip-path in certain condition	TESTING (ashughes)
bug 668583 Crash @ js::gc::ScanRope	NEEDINFO: Need more information from developer
bug 698296 Firefox Crash Report @ je_free js::gc::Arena::finalize	NEEDINFO: Need more information from developer
bug 814200 Xt plugins cannot be resized larger in FF17	VERIFYME: To be verified in 19b2
bug 815542 Crash in IncrementalCollectSlice	DROPPED: Nothing left for QA to do

NORMAL FIXES

Bugs: Verify these fixes in the latest Beta candidate	Status
bug 828507 Firefox 19b1 Beta updates fail to apply	TESTING (ashughes)
bug 802476 Printing crashes with a local file	VERIFIED FIXED (Cornel)
bug 789933 Crash in mozilla::gfx::DrawTargetCairo::Stroke	VERIFIED FIXED (Bogdan)
bug 818276 Scrollbar on popoups disappeared after updating	VERIFIED FIXED (Paul)
bug 812828 "Open All in Tabs" fails if Library window is open with no background windows	VERIFIED FIXED (Ioana)
bug 810586 Collapsible table not rendered correctly	NEEDINFO: Bug still reproduces under certain conditions (Paul)
bug 810531 Invalidation artifacts when scrolling w3.org working draft pages	VERIFIED FIXED (Manuela)
bug 799900 Wrong position of East Asian input method candidate box in HiDPI mode

SECURITY FIXES

Bugs: Verify these fixes in the latest Beta candidate (requires privileged access)	Status
bug 814839 WebGL conformance test hang [@mozilla::WebGLContext::Clear]	VERIFIED FIXED (mwobensmith)
bug 756581 JS OOM Testing: Assertion failure	DROPPED: Needs custom test script, reporter to verify (mwobensmith)
bug 812341 Opt-only Crash [@ SuppressDeletedPropertyHelper]	DROPPED: Needs custom shell, reporter to verify (mwobensmith)
bug 814407 WebGL conformance crash with Mesa 8.0 after calling texImage2D	DROPPED: Needs custom hardware, reporter to verify (mwobensmith)
bug 814716 WebGL conformance crash (NVIDIA Noveau Linux driver)	DROPPED: Needs custom hardware, reporter to verify (mwobensmith)
bug 780770 Content can prevent chrome buttons from working	VERIFIED FIXED (mnoorenberghe)

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current Beta

N/A for Beta 1 due to lack of crash data

UNCONFIRMED

Bugs: Triage unconfirmed bugs for potential incoming regressions.	Status
bug 806137 Private browsing wipes regular browsing session	CONFIRMED (Ioana)
bug 803840 keyboard shortcuts are different on each platform	CONFIRMED (Ioana)
bug 800032 Twitter website F5 key not working	NEEDINFO (Ioana)
bug 823722 "Flash Video Downloader - Youtube Downloader" stopped working with update	RESOLVED WORKSFORME (Ioana)
bug 810131 Status bar appears when switching loading tab to new tab	NEEDINFO (Ioana)
bug 812871 Highlighted context menu items font rendering issues on Windows XP	NEEDINFO (Ioana)
bug 809805 GameDesire Pool doesn't open in Nightly	NEEDINFO (Ioana)
bug 818863 Pinned tabs do not work with "don't load tabs until selected" feature	RESOLVED DUPE of bug 713190 (Ioana)
bug 812888 Issue with viewing YouTube videos.	RESOLVED WORKSFORME (Ioana)
bug 805426 Navigator error pops up when clicked on picture	RESOLVED INVALID (Ioana)

Issues Found

Bugs: New issues found during testing the Beta candidate	Status
bug 828372 Yandex Elements toolbar buttons are missplaced	RESOLVED INVALID: Yandex Elements bug
bug 828473 Yahoo Toolbar changes position after Yandex bar is installed	RESOLVED INVALID: Yahoo Toolbar bug
bug 828507 Firefox 19b1 Beta updates fail to apply	RESOLVED FIXED (Simona)
bug 828898 Firefox fails to display all PDF handling options	RESOLVED INVALID: Foxit Reader bug

Items in bold are BLOCKER bugs

Beta 2

Builds	FTP, HTTP
ID	20130116072953
Changeset	8b833c1150b5 (pushlog)

Checklist

[DONE] Functional Automation
[DONE] Manual Testing
[DONE] Bug Triage
[DONE] Update Automation on betatest channel
[DONE] Update Automation on releasetest channel
[DONE] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Functional Results	Restart Results
Windows XP	[DONE] 0 Fails	[DONE] 0 Fails
Windows Vista	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.6	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.7	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.8	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 64-bit	[DONE] 0 Fails	[DONE] 0 Fails

Test Failures (known bugs)

Manual Testing

Focus testing around bug fixes landed between Beta 1 and Beta 2 (see Bug Triage section below).

Update Testing

Results of running the Mozmill update automation.
19.0b1 should get a partial update, <=18.0b6 should get a complete update.

Platform	Betatest (2013-01-16)	Releasetest (2013-01-17)	Beta 2013-01-17)
Windows XP	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows Vista	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 7 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 7 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 8 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 8 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.6	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.7	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.8	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Ubuntu 12.04 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Ubuntu 12.04 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails

Bug Triage

NORMAL FIXES

Verify these fixes in the latest Beta candidate.

[DONE] VERIFIED bug 828202 System proxy settings issue with Firefox 18 [1] (Manuela)
[DONE] VERIFIED bug 814200 Xt plugins cannot be resized larger in FF17 (Virgil)
[DONE] VERIFIED bug 828632 System Proxy Configuration Causes Memory Leak, syslogd load [2] (Simona)
[DONE] VERIFIED bug 828236 issue with NTLM and some SSL Proxies [3] (reporter)
[DONE] VERIFIED bug 829646 Proxy Auto Config calling myIPAddress() in the top level fails (Alexandra)
[DONE] VERIFIED bug 818468 [Linux] Langpacks bundled in distribution/extensions are registered but disabled (reporter)
[DONE] VERIFIED bug 814434 [hidpi/retina] url suggestion box pops up on wrong screen (mwobensmith)
[DONE] VERIFIED bug 830303 Fix plugin-container spawning on BSD after bug 753046 (reporter)
[MISSED] NEEDS CRASHSTATS bug 828184 startup crash in nsPrefBranch::GetIntPref with QIPCAP.dll
[MISSED] UNVERIFIABLE bug 828205 crash in js::DebugScopeProxy::has with Firebug
- Relies on having Ubuntu-provided Firefox 19.0b2 builds for Ubuntu 10.04 which do not exist

These will require setting up a proxy and having Firefox connect to the internet through that proxy. 1. "This only impacts system proxy pac file settings on all platforms. file:// uris are the only type known to be effected. uris pointing to non existant files, using strange syntax, etc..."

2. "This is an OSX specific issue and the patch is OSX specific at compile time, so don't waste time with other platforms. Configure a system proxy using a file:// pac file and you should see the logging entries syslog (using 18.0.0) and verify they are gone with 18.0.1 while verifying that the pac file is honored. The fix generally impacts the pac file handling for os x so some general testing could be redone."

3. "Setup squid to do integrated windows authentication (i.e. NTLM) to access the proxy and then configure proxy use for http and https traffic. Honestly, that's a giant pain in the butt to do but that's what the interop problem was - specifically with accessing https through the proxy. Manual configuration is fine in this case."

4. "This bug is pretty self descriptive and the patch for it includes an automated regression test. The issue involves using the myIPAddress() function in PAC file outside the scope of the FindProxyForURL() function - the test in the patch illustrates such a case."

SECURITY FIXES

Verify these fixes in the latest Beta candidate (requires privileged access)

[DONE] VERIFIED bug 775228 (CVE-2012-5830) use-after-free when loading html file on osx (mwobensmith)
[DONE] VERIFIED bug 812341 Opt-only Crash [@ SuppressDeletedPropertyHelper] (mwobensmith)
[DONE] VERIFIED bug 785734 (CVE-2012-5833) Mesa crashes on certain texImage2D calls (reporter)

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current Beta

QAWANTED

Test the following bugs to find steps to reproduce, regression range, or testcases.

[AT RISK] NEEDS ENGR bug 668583 crash @ js::gc::ScanRope (ashughes)
[AT RISK] NEEDS ENGR bug 698296 crash in js::SweepBackgroundThings @ js::gc::Arena::finalize
[ON TRACK] VERIFYME bug 818468 Langpacks bundled in distribution/extensions are registered but disabled even if shown enabled
[ON TRACK] VERIFYME bug 828184 startup crash in nsPrefBranch::GetIntPref with QIPCAP.dll (Websense Endpoint)
UNASSIGNED bug 783369 Norton Confidential 2012/2013 crashes (couictlr.dll, coffplgn.dll)

UNCONFIRMED

Issues Found

Items in bold are BLOCKER bugs

Known Issues

Full Query

ID	Summary	Priority	Status
747909	Youtube.com In Full Screen mode - copy actions are not allowed on HTML5 videos	--	RESOLVED
797116	[Win8] Flash hang in F_1152915508 while playing Knight Hunters	P3	RESOLVED

2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);

Beta 3

Builds	FTP, HTTP
ID	20130123083802
Changeset	19124ba6cd20 (pushlog)

Checklist

[DONE] Functional Automation
[DONE] Manual Testing
[DONE] Bug Triage
[DONE] Update Automation on releasetest channel
[ON TRACK] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Functional Results	Restart Results
Windows XP	PASS	PASS
Windows Vista	PASS	PASS
Windows 7 32-bit	PASS	PASS
Windows 7 64-bit	PASS	PASS
Windows 8 32-bit	PASS	PASS
Windows 8 64-bit	PASS	PASS
Mac OS X 10.6	PASS	PASS
Mac OS X 10.7	PASS	PASS
Mac OS X 10.8	PASS	PASS
Ubuntu 12.04 32-bit	PASS	PASS
Ubuntu 12.04 64-bit	PASS	PASS

Test Failures (known bugs)

Manual Testing

Focus testing around bug fixes landed between Beta 2 and Beta 3 (see Bug Triage section below).

Update Testing

Results of running the Mozmill update automation.
19.0bx should get a partial update, <=xx.x should get a complete update.

Platform	[link Betatest] (2013-01-24)	[link Releasetest] (2013-01-24)	[link Beta] (2013-01-24)
Windows XP	Skipped	http://mozmill-ondemand.blargon7.com/#/update/reports?branch=19.0&platform=All&from=2013-01-24&to=2013-01-25	status: number of fails
Windows Vista	Skipped	http://mozmill-ondemand.blargon7.com/#/update/reports?branch=19.0&platform=All&from=2013-01-24&to=2013-01-25	status: number of fails
Windows 7 32-bit	Skipped	PASS: results	status: number of fails
Windows 7 64-bit	Skipped	PASS: results	status: number of fails
Windows 8 32-bit	Skipped	PASS: results	status: number of fails
Windows 8 64-bit	Skipped	PASS: results	status: number of fails
Mac OS X 10.6	Skipped	PASS: results	status: number of fails
Mac OS X 10.7	Skipped	PASS: results	status: number of fails
Mac OS X 10.8	Skipped	PASS: results	status: number of fails
Ubuntu 12.04 32-bit	Skipped	PASS: results	status: number of fails
Ubuntu 12.04 64-bit	Skipped	PASS: results	status: number of fails

Bug Triage

NORMAL FIXES

Verify these fixes in the latest Beta candidate. (change link above for beta 3)

[DONE] VERIFIED bug 828286 center of svg animation is incorrect on industrial demo at zamek.pmmf.hu, ever since SVG display lists became enabled (Ioana)
[DONE] VERIFIED bug 829921 click events not delivered with trivial CSS transform, margin, overflow: hidden (Ioana)
[DONE] VERIFIED bug 827217 webgl.can-lose-context-in-foreground=false crashes in desktop Firefox (Ioana)
[DONE] NEEDINFO bug 826115 warnings about rollups too noisy (Manuela)
[DONE] VERIFIED bug 818023 IonMonkey: function.caller does not always work (and in interpreter and JM) (Paul)
[DONE] [QA-] bug 829858 Null dereference in nsHttpChannel::InitOfflineCacheEntry() (Paul)
[DONE] NEEDINFO bug 789975 Extremely long pause while collecting telemetry information on the main thread (Paul)
[DONE] VERIFIED bug 830066 Drag & drop links into toolkit downloads manager to download files fails (TypeError: aInitiatingDocument is undefined) (Cornel)
[DONE] DROPPED bug 804605 Kerberos authentication does not work with CNAME (Cornel)
- Asked the reporter to verify.
[DONE] NEEDINFO bug 812647 Startup error: Could not open any libc. (osfile_unix_allthreads.jsm) (Virgil)
[DONE] VERIFIED bug 827225 "Assertion failure: obj->isTypedArray()" with WebGL readPixels (Virgil)
[DONE] VERIFIED bug 825544 Clicking on a duckduckgo result link while the page is loading prevents the result page from being recorded (Alexandra)
[DONE] DROPPED bug 821329 Input widgets don't accept character input with MacOS's build-in handwriting feature (Bogdan)
- Asked the reporter to verify.
[DONE] DROPPED bug 825620 Regression: Flash is triggering the discrete GPU (Bogdan)
- Asked the reporter to verify.
[DONE] VERIFIED bug 831780 scrollWidth, scrollHeight of DIV containing IMG reported as 40x40px until the image's actual size is known, regardless of CSS sizing of the IMG (Simona)
[DONE] REOPENED bug 830214 crash when printing selection of a PDF with many pages in pdf.js (Simona)
[DONE] VERIFIED bug 829435 Japanese characters appear completely garbled in PDF viewer (Mihaela)

SECURITY FIXES

Verify these fixes in the latest Beta candidate (requires privileged access) (change link above for beta 3)

[DONE] VERIFIED bug 831095 Use-After-Free crash @xul!nsImageLoadingContent::OnStopContainer (mwobensmith)
[DONE] VERIFIED bug 827070 Heap-buffer-overflow WRITE in nsSaveAsCharset::DoCharsetConversion (mwobensmith)
[DONE] DROPPED bug 690970 cycle collector can be made to Unlink live objects (by using weak maps or watchpoints) (mwobensmith)
- Verification of fix not possible and developer feels issue is theoretical

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current Beta (change link above for beta 3)

[DONE] VERIFIED bug 819888 crash in nsWindow::DealWithPopups (Paul)
[DONE] DROPPED bug 803568 Nightly crashed when see pdf file in fullscreen (presentation mode) or zooming (Cornel)
- Verification impeded by bug 728571.
[DONE] VERIFIED bug 806206 crash in JSObject::global (Ioana)
[DONE] VERIFIED bug 810618 crash in mozilla::dom::EventHandlerNonNull::`.* deleting destructor' (Alexandra)
[DONE] VERIFIED bug 803125 crash in imgStatusTrackerObserver::OnStartDecode (Alexandra)

QAWANTED

Test the following bugs to find steps to reproduce, regression range, or testcases.

{{|}} VERIFIED bug bugID Bug description (verifier)

UNCONFIRMED

[DONE] UNCONFIRMED bug 834014 Edit menu greyed when ("select all" or "copy" is available) (Manuela - changed the component)
[DONE] UNCONFIRMED bug 834237 pdf.js: Chapter headlines in some PDFs are ugly because one headline has an overlap with another. (Manuela - changed the component)
[DONE] RESOLVED INCOMPLETE bug 829540 Firefox would not load any pages in normal mode. Had to restrst in safe mode with extensions and add-ons disabled (Manuela)
[DONE] UNCONFIRMED bug 807991 FireFTP causes manu js sinc reflows (Simona - changed the component)
[DONE] RESOLVED DUPLICATE bug 829414 Clear List button broken in Beta 19 (Virgil)

Issues Found

Bugs: New issues found during testing the Beta candidate	Status
bug 834256 Firefox crashes while viewing PDF when many PDF documents are opened	NEW

Items in bold are BLOCKER bugs

Beta 4

Builds	FTP, HTTP
ID	20130130080006
Changeset	74d5479be250 (pushlog)

Checklist

[DONE] Functional Automation
[ON TRACK] Manual Testing
[ON TRACK] Bug Triage
[ON TRACK] Update Automation on betatest channel
[ON TRACK] Update Automation on releasetest channel
[ON TRACK] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Functional Results	Restart Results
Windows XP	[DONE] 0 Fails	[DONE] 0 Fails
Windows Vista	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.6	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.7	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.8	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 32-bit	[DONE] 0 Fails	[DONE] 1 Fails
Ubuntu 12.04 64-bit	[DONE] 0 Fails	[DONE] 1 Fails

Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Focus Area	Windows 7	Windows 8	Mac OSX 10.7	Mac OSX 10.8	Ubuntu 12.04
Regression Tests: Run the Firefox 19.0b4 Regression Tests in MozTrap. Do some exploratory testing around the testcases to try to exercise some edge-cases.	[DONE] Alexandra 0 Bugs	[DONE] Bogdan 0 Bugs	[DONE] Manuela 0 Bugs	[DONE] Virgil 0 Bugs	[DONE] Simona 0 Bugs
PDF.js: Test the new built-in PDF reader to check for issues with viewing and printing PDFs. Check some of the behaviour with other PDF readers side-loaded (ie. Adobe Reader, Foxit Reader, etc). Several bugs were marked for tracking in 19 for this feature. We may disable it. Pay close attention to this.	[DONE] Alexandra 1 Bugs	[DONE] Bogdan 1 Bugs	[DONE] Manuela 1 Bugs	[DONE] Virgil 0 Bugs	[DONE] Simona 0 Bugs

Update Testing

Results of running the Mozmill update automation.
19.0b3 should get a partial update, <= 19.0b2 should get a complete update.

Platform	Betatest (2013-01-30)	Releasetest (2013-01-30)	Beta (2013-01-30)
Windows XP	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Windows Vista	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Windows 7 32-bit	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Windows 7 64-bit	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Windows 8 32-bit	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Windows 8 64-bit	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Mac OS X 10.6	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Mac OS X 10.7	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Mac OS X 10.8	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Ubuntu 12.04 32-bit	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Ubuntu 12.04 64-bit	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails

Bug Triage

QAWANTED

NORMAL FIXES

[DONE] VERIFIED bug 832998 web page is MUCH slower in FF than in Chrome (Ioana)
[DONE] VERIFIED bug 828205 crash in js::DebugScopeProxy::has with Firebug (Ioana)
[DONE] DROPPED bug 835090 Where possible xpcshell tests should run on xulrunner (Ioana)
- Automated tests changes to run on XULrunner.
[DONE] VERIFIED bug 832576 WebGL uniform setters got 3-4x slower due to refactoring (Ioana)
[DONE] VERIFIED bug 809846 Click to Play doesn't play music on jango.com or other sites using the SoundManager2 library (Paul)
[DONE] VERIFIED bug 820497 click-to-play: asynchronously adding different plugins won't update the popup notification (Paul)
[DONE] VERIFIED bug 744745 Click to play overlay not appearing on vimeo.com (Paul)
[DONE] VERIFIED bug 819871 Nightly: can not login at Mail.com (Paul)
[DONE] VERIFIED bug 813550 Location bar's placeholder in an RTL chrome should be aligned on the right (Paul)
[DONE] VERIFIED bug 830086 NullPointerException in my webapp in Firefox 18 (Virgil)

SECURITY FIXES

[DONE] VERIFIED bug 761448 "ASSERTION: We should never be reusing a shared inner window" with pagehide event (mwobensmith)
[DONE] VERIFIED bug 812893 Heap-use-after-free in nsOverflowContinuationTracker::Finish, with -moz-columns (mwobensmith)
[DONE] DROPPED bug 814716 WebGL conformance crash (NVIDIA Noveau Linux driver) (mwobensmith)
- Needs custom hardware, reporter to verify

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current Beta

[DONE] UPDATED bug 812776 crash in nsCSSRendering::PrepareBackgroundLayer @ nsIFrame::GetStyleVisibility (Ioana)
Recent crashes available in Socorro.
[DONE] UPDATED bug 813489 crash in nsHttpTransaction::SetSecurityCallbacks (Ioana)
Recent crashes available in Socorro.
[DONE] UPDATED bug 812203 crash in nsHttpConnection::SetSecurityCallbacks (Ioana)
Recent crashes available in Socorro.
[DONE] VERIFIED bug 814953 crash in nsWindow::ProcessMessage (Ioana)
[DONE] VERIFIED bug 803568 Nightly crashed when see pdf file in fullscreen (presentation mode) or zooming (Ioana)

UNCONFIRMED

Issues Found

Items in bold are BLOCKER bugs PDF Viewer

[DONE] NEW bug 836732 Buttons in pdf.js remain in hover state after selection (Virgil)
[DONE] NEW bug 836736 Selected text font changes to default when firefox is left in the background in pdf.js (Alexandra)
[DONE] NEW bug 836748 Loading stuck on a very large PDF file (Bogdan)

Beta 5

Builds	FTP, HTTP
ID	20130206083616
Changeset	3e382bb14817 (pushlog)

Checklist

{{ |}} Functional Automation
{{ |}} Manual Testing
{{ |}} Bug Triage
{{ |}} Update Automation on betatest channel
{{ |}} Update Automation on releasetest channel
{{ |}} Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Functional Results	Restart Results
Windows XP	[DONE] 0 Fails	[DONE] 0 Fails
Windows Vista	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.6	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.7	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.8	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 64-bit	[DONE] 0 Fails	[DONE] 0 Fails

Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Regression Tests

Run the following regression tests from the above linked testrun. Do some exploratory testing and try to exercise some edge-cases around these scenarios.

Breakpad End-to-End functionality
- [DONE] Windows 7 (Bogdan)
- [DONE] Mac OSX 10.8 (Alexandra)
- [DONE] Ubuntu 12.04 (Manuela)
What's New Page on Firefox update
- [DONE] Windows 7 (Bogdan)
- [DONE] Mac OSX 10.8 (Alexandra)
- [DONE] Ubuntu 12.04 (Manuela)

PDF.js (if time permits)

Test the new built-in PDF reader to check for issues with viewing and printing PDFs. Check some of the behaviour with other PDF readers side-loaded (ie. Adobe Reader, Foxit Reader, etc). Several bugs were marked for tracking in 19 for this feature. We may disable it. Pay close attention to this.

[DONE] Windows 7 (Bogdan)
[DONE] Mac OSX 10.8 (Alexandra)
[DONE] Ubuntu 12.04 (Manuela)

PDF Viewer Blocker Triage

The following bugs need triaging for potentially blocking PDF.js from shipping in Firefox 19. Try to reproduce these bugs in Firefox 19, set the tracking-firefox19:? flag if it reproduces and you believe it should block. Add your justification in the bug and in this wiki below.

Add a bullet-item below for any bugs you think are potential feature blockers. Be sure to include a reason you believe it should block. Easily reproducible crashes, hangs, or inability to load or download a PDF during a common user scenario would be examples of potential blockers.

Sign-off Status

QA is okay with a conditional sign-off for Firefox 19 considering we act on the following initiatives:

show the infobar more aggressively (maybe even always) and 3)
relnote that some PDF documents may not be displayed correctly but people can use an external reader
have a SUMO article ready
have a plan to uplift P1 and Critical fixes to Firefox 20
be prepared to hotfix disable PDF.js in Firefox 19 if negative feedback is explosive (bug 839239)
PDF.js Blocker Shortlist
- bug 830278 Printing selections in pdf.js don't work and can cause OOM errors in large PDFs
- bug 738967 pdf.js doesn't work for embedded PDFs
- bug 839096 Searching for consecutive words is broken in pdf.js
- bug 838549 PDF renders incorrectly if HWA turned off while document loaded
- bug 839548 Excessive memory while loading PDF thumbnail view

Update Testing

Results of running the Mozmill update automation.
19.0b3 should get a partial update, <= 19.0b2 should get a complete update.

Platform	Betatest	Releasetest	Beta
Windows XP	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows Vista	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 7 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 7 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 8 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 8 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.6	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.7	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.8	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Ubuntu 12.04 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Ubuntu 12.04 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails

Bug Fix Verifications

The following fixes require verification before sign-off.

[DONE] COMMENTED bug 824162 Strange focus switching with chat windows (Virgil)
- QA: Unable to verify fixed because fixes need to be implemented on Facebook's end
- RelMan: did not find any regressions so further testing does not block
[AT RISK] REOPENED bug 830278 Printing selections in pdf.js don't work and can cause OOM errors in large PDFs (Simona)
- QA: Still crashing in Firefox 19.0b5
- RelMan: retest with Beta, Aurora, and Nightly; reopen and update flags as appropriate
- QA: not fixed, all branches still crash with empty signature.
[DONE] VERIFIED bug 834075 PDF url in the address bar becomes wrong in a certain situation (Alexandra)
[DONE] VERIFIEDbug 836887 Zoom-In button using wrong icon on HiDPI screens (mwobensmith, juanb)
[DONE] VERIFIED bug 797288 Pdf.js download behavior inconsistent (Manuela)
[DONE] COMMENTED bug 837371 Crash in nsContainerFrame::BuildDisplayListForNonBlockChildren (AMD Radeon HD 6xxx) (Bogdan)
- QA: Unable to reproduce these crashes but were able to crash with null signatures
- RelMan: doesn't seem related so doesn't block ship, will need to watch crashstats post-release
[DONE] VERIFIED bug 825035 Blocklisted click-to-play notification only fades in/out once per window per session (Paul)
[DONE] VERIFIED bug 836463 PDF.js Broken Forms (Bogdan)
[DONE] VERIFIED bug 835814 Heap-use-after-free in nsAsyncDOMEvent::Run (ashughes)
[DONE] VERIFIED bug 837845 Multi-GB memory spike when using regular expressions (Virgil)
[DONE] VERIFIED bug 838001 Input Method can't be opened to type to input fields in a panel on Firefox 19+ (pzhang)
[DONE] VERIFIED bug 799900 Wrong position of East Asian input method candidate box in HiDPI mode (haozhun)

Issues Found

The following are new issues found while testing the Beta candidate. Items in bold are BLOCKER bugs.

[DONE] NEW bug 839074 With pdf.js enabled, selected text doesn't stay selected after zoom (Manuela)
[DONE] INVALID bug 839092 Clicking on titles from Show Document Outline does not always bring up on top of the page the title selected (Bogdan)
[DONE] FIXED bug 839089 Moving with arrow keys left and right changes page on a pdf zoomed page (Bogdan)
[DONE] NEW bug 839096 Searching for consecutive words is broken in pdf.js (Alexandra)

Beta 6

Builds	FTP, HTTP
ID	20130212082553
Changeset	258ce028c3be (pushlog)

Checklist

[DONE] Functional Automation
[DONE] Manual Testing
[DONE] Bug Triage
[DONE] Update Automation on betatest channel
[DONE] Update Automation on releasetest channel
[DONE] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Functional Results	Restart Results
Windows XP	[DONE] 2 Fails	[DONE] 0 Fails
Windows Vista	[DONE] 1 Fails	[DONE] 0 Fails
Windows 7 32-bit	[DONE] 1 Fails	[DONE] 0 Fails
Windows 7 64-bit	[DONE] 1 Fails	[DONE] 0 Fails
Windows 8 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 64-bit	[DONE] 1 Fails	[DONE] 0 Fails
Mac OS X 10.6	[DONE] 1 Fails	[DONE] 0 Fails
Mac OS X 10.7	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.8	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 64-bit	[DONE] 0 Fails	[DONE] 0 Fails

Test Failures (known bugs)

/testToolbar/testStopReloadButtons.js -> bug 832180
/testSecurity/testSafeBrowsingNotificationBar.js -> bug 840810
/testToolbar/testBackForwardButtons.js -> bug 840022

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Regression Tests

Run the following regression tests from the above linked testrun. Do some exploratory testing and try to exercise some edge-cases around these scenarios.

Breakpad End-to-End functionality

[DONE] PASS Windows 7 64-bit (Paul)
[DONE] PASS Windows 8 32-bit (Alexandra)
[DONE] PASS Mac OSX 10.6 (Manuela)
[DONE] PASS Ubuntu 12.04 64-bit (Virgil)

PDF.js

Test the new integrated PDF viewer. Pay particular attention to PDFs served by financial institutions (bank statements, bill statements, etc).

[DONE] PASS Windows 7 64-bit (Paul)
[DONE] PASS Windows 8 32-bit (Alexandra)
[DONE] PASS Mac OSX 10.6 (Manuela)
[DONE] PASS Ubuntu 12.04 64-bit (Virgil)

Proxy regressions from Fx18

See bug 828202#c25, bug 828632, bug 829646 and double check these issues remain resolved.

[DONE] PASS Windows 7 64-bit (Paul)
[DONE] PASS Windows 8 32-bit (Alexandra)
[DONE] PASS Mac OSX 10.6 (Manuela)
[DONE] PASS Ubuntu 12.04 64-bit (Virgil)

Mozmill-fails Regression Checking

Spotcheck verify the Stop, Reload, Back, and Forward buttons work under various conditions. Spotcheck verify the safe browsing notification bar works as expected.

[DONE] PASS Windows 7 64-bit (Paul)
[DONE] PASS Windows 8 32-bit (Alexandra)
[DONE] PASS Mac OSX 10.6 (Manuela)
[DONE] PASS Ubuntu 12.04 64-bit (Virgil)

Web compatibility (search, webmail, banking, video, audio, gaming, news, cms, etc.)

[DONE] PASS Windows 7 64-bit (Paul)
[DONE] PASS Windows 8 32-bit (Alexandra)
[DONE] PASS Mac OSX 10.6 (Manuela)
[DONE] PASS Ubuntu 12.04 64-bit (Virgil)

Mac OSX 10.5 and Windows 2000 de-support (install and update)

[DONE] PASS Windows 2000 (Simona)
[DONE] PASS Mac OSX 10.5 (Simona)

PDF Viewer Blocker Triage

The following bugs need triaging for potentially blocking PDF.js from shipping in Firefox 19. Try to reproduce these bugs in Firefox 19, set the tracking-firefox19:? flag if it reproduces and you believe it should block. Add your justification in the bug and in this wiki below.

Add a bullet-item below for any bugs you think are potential feature blockers. Be sure to include a reason you believe it should block. Easily reproducible crashes, hangs, or inability to load or download a PDF during a common user scenario would be examples of potential blockers.

bug 738967 pdf.js doesn't work for embedded PDFs
bug 838549 PDF renders incorrectly if HWA turned off while document loaded
bug 839548 Excessive memory while loading PDF thumbnail view
bug 829954 OOM crash in mozilla::gfx::AlphaBoxBlur::Blur
bug 799695 Background too dark
Nominated by community for UX:
- bug 738952 "Save as..." File menu entry or Ctrl+S produces unexpected results
- bug 761539 pdf.js doesn't display and print some characters (e.g. German umlauts)

QA is okay with a conditional sign-off for Firefox 19 considering we act on the following initiatives:

show the infobar more aggressively (maybe even always)
relnote that some PDF documents may not be displayed correctly but people can use an external reader
have a SUMO article ready
have a plan to uplift P1 and Critical fixes to Firefox 20
be prepared to hotfix disable PDF.js in Firefox 19 if negative feedback is explosive (bug 839239)

Update Testing

Results of running the Mozmill update automation.

Partial: 19.0b5
Complete: 19.0b4 and earlier

Platform	Betatest	Releasetest	Beta
Windows XP	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows Vista	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 7 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 7 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 8 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 8 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.6	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.7	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.8	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Ubuntu 12.04 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Ubuntu 12.04 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails

Bug Fix Verifications

The following fixes require verification before sign-off.

[DONE] VERIFIED bug 830278 Printing selections in pdf.js cause OOM errors in large PDFs (simona)
[DONE] VERIFIED bug 837461 Black squares appear in some PDFs using PDF.js (contributor)
[DONE] VERIFIED bug 838864 keyword.URL reset interferes poorly with some partner add-ons (Bogdan)
[DONE] CONDITIONAL bug 839270 crash in nsStyleContext::nsStyleContext mainly with AMD Radeon HD 6250/6310/6320 (juanb)
- Was hard to reproduce, will depend on post-release crashstats to verify
[DONE] VERIFIED STAGING bug 840161 Re-block AMD Radeon HD 6290/6300/6310/6320 for Direct3D 9/DIRECT2D (juanb)

Issues Found

The following are new issues found while testing the Beta candidate. Items in bold are BLOCKER bugs.

Social API

bug 841007 - Unreadable text on notification Facebook panel

Session Restore

NEW bug 791195 Session is not restored after Firefox crash

Release Candidate 1

Builds	FTP, HTTP
ID	20130215130331
Changeset	20238b786063 (pushlog)

Checklist

[DONE] Functional Automation
[DONE] Manual Testing
[DONE] Bug Fix Verifications
[DONE] Updates qualified on betatest channel
[DONE] Conditional sign-off for push to mirrors
[DONE] Updates qualified on releasetest channel
[DONE] Final sign-off for push to release
[DONE] Updates qualified on release channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Functional Results	Restart Results
Windows XP	[DONE] 1 Fails	[DONE] 0 Fails
Windows Vista	[DONE] 1 Fails	[DONE] 0 Fails
Windows 7 32-bit	[DONE] 1 Fails	[DONE] 0 Fails
Windows 7 64-bit	[DONE] 1 Fails	[DONE] 0 Fails
Windows 8 32-bit	[DONE] 1 Fails	[DONE] 0 Fails
Windows 8 64-bit	[DONE] 1 Fails	[DONE] 0 Fails
Mac OS X 10.6	[DONE] 1 Fails	[DONE] 0 Fails
Mac OS X 10.7	[DONE] 1 Fails	[DONE] 0 Fails
Mac OS X 10.8	[DONE] 1 Fails	[DONE] 0 Fails
Ubuntu 12.04 32-bit	[DONE] 1 Fails	[DONE] 0 Fails
Ubuntu 12.04 64-bit	[DONE] 1 Fails	[DONE] 0 Fails

Test Failures (known bugs)

/testSecurity/testEncryptedPageWarning.js (bug 810820)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Regression Tests

Run the following regression tests from the above linked testrun. Do some exploratory testing and try to exercise some edge-cases around these scenarios.

Breakpad End-to-End functionality

[DONE] PASS Windows 7 64-bit (Manuela)
[DONE] PASS Windows 8 32-bit (Virgil)
[DONE] PASS Mac OSX 10.6 (Simona)
[DONE] PASS Ubuntu 12.04 64-bit (Mihaela)

PDF.js

Test the new integrated PDF viewer. Pay particular attention to PDFs served by financial institutions (bank statements, bill statements, etc).

[DONE] PASS Windows 7 64-bit (Manuela)
[DONE] PASS Windows 8 32-bit (Virgil)
[DONE] PASS Mac OSX 10.6 (Simona)
[DONE] PASS Ubuntu 12.04 64-bit (Mihaela)

Update Testing

Results of running the Mozmill update automation.

Partial: 18.0.2, 17.0.1, 16.0.2
Complete: 18.0.1, 18.0, 17.0, 16.0.1, <=16.0

Platform	betatest report	releasetest: report	release: report
Windows XP	[DONE] 0 Fails	[DONE] 0 Fails	[DONE] 0 Fails
Windows Vista	[DONE] 0 Fails	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 32-bit	[DONE] 0 Fails	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 64-bit	[DONE] 0 Fails	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 32-bit	[DONE] 0 Fails	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 64-bit	[DONE] 0 Fails	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.6	[DONE] 0 Fails	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.7	[DONE] 0 Fails	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.8	[DONE] 0 Fails	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 32-bit	[DONE] 0 Fails	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 64-bit	[DONE] 0 Fails	[DONE] 0 Fails	[DONE] 0 Fails

Bug Fix Verifications

The following fixes require verification before sign-off.

[DONE] VERIFIED bug 817790 Stretched Facebook icon briefly shows over entire toolbar during window startup (Simona)
[DONE] VERIFIED bug 815359 Bing Maps v7 Ajax Pushpin Incompatibility (Manuela)
[DONE] VERIFIED bug 800363 (CVE-2012-5837) XSS in Web Developer Toolbar's chrome privilege page (Paul)
[DONE] VERIFIED bug 827979 Add Growl support back (Cornel)
[DONE] COMMENTED bug 829710 Firefox "slants" on Mac 10.6 (Simona)
[DONE] COMMENTED bug 834325 SVG graphic rendering broken when embedded (Bogdan)
[DONE] COMMENTED bug 822620 disable Notification Center support (Cornel)
[DONE] COMMENTED bug 821577 DMD hang on Mac at start-up (Virgil)
[DONE] VERIFIED bug 816603 background-color does not print in certain site (Ioana)
[DONE] VERIFIED bug 813851 Canvas gets confused about its path/transform state (Manuela)
TRACKING bug 839270 crash in nsStyleContext::nsStyleContext mainly with AMD Radeon HD 6250/6310/6320
- We think the crash explosiveness has been mitigated but have failed to verify 100% due to lack of user feedback

Issues Found

The following are new issues found while testing the Beta candidate. Items in bold are BLOCKER bugs.

19.0.1

The following release is Windows 8 only

Builds	FTP, HTTP
ID	20130226172142
Changeset	668544a286c1 (pushlog)

Checklist

[DONE] Functional Automation
[DONE] Manual Testing
[DONE] Bug Fix Verifications
[DONE] Conditional sign-off for push to mirrors
[DONE] Updates qualified on releasetest channel (19.0.1: Win8-only, 19.0: all else)
[DONE] Final sign-off for push to release
[DONE] Updates qualified on release channel (19.0.1: Win8-only, 19.0: all else)
{{ |}} Confirm download links on download.mozilla.org (19.0.1: Win8-only, 19.0: all else)

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform (report)	Functional Results	Restart Results
Windows XP	0 Failures	0 Failures
Windows Vista	0 Failures	0 Failures
Windows 7 32-bit	0 Failures	0 Failures
Windows 7 64-bit	0 Failures	0 Failures
Windows 8 32-bit	0 Failures	0 Failures
Windows 8 64-bit	0 Failures	0 Failures
Mac OS X 10.6	0 Failures	0 Failures
Mac OS X 10.7	0 Failures	0 Failures
Mac OS X 10.8	0 Failures	0 Failures
Ubuntu 12.04 32-bit	0 Failures	0 Failures
Ubuntu 12.04 64-bit	0 Failures	0 Failures

Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla Run the following regression tests and do some exploratory testing to exercise some possible edge-cases. Report any bugs found in Bugzilla and list them in the Issues Found section below.

Breakpad End-to-End functionality (testcase)

[DONE] Windows 8 32-bit (Bogdan)
[DONE] Windows 8 64-bit (Mihaela)

Confirm uninstallable on other Windows

At 7:30 a.m Pacific Time, updates were not available on betatest channel for any of the Windows versions (Win XP, Win 7, Win 8, Win Vista)

FAIL Windows 7 32-bit (Mihaela)
- can install
FAIL Windows Vista (Bogdan)
- can install
FAIL Windows XP (Mihaela)
- can install

Exploratory testing on AMD Radeon hardware

Do some exploratory testing on various Radeon GPUs and Catalyst driver versions to try and crash Firefox. Common areas of instability are Flash, Video, Facebook, WebGL, and Canvas.

PASS Mihaela, Windows 8 x64, AMD Radeon HD 7700 Series, Catalyst 12.10
- Sites and scenarios:
  - facebook: timeline navigation, view photos, status update, play games
  - gmail: compose and send email, read and navigate through emails
  - yahoo mail: read/navigate through emails
  - youtube: search, watch video, view videos from playlist, interact with video
  - google maps WebGL enabled: search, zoom, street view
  - vimeo: play video, interact with video
  - http://www.ebizmba.com/articles/best-flash-sites: interact with some of the websites
FAIL Mihaela, Windows 8 x64, AMD Radeon HD 7700 Series, Catalyst 12.10
- Scenario:
  - Have Catalyst 12.10
  - Open some pages in firefox (mail, facebook, youtube, flash games)
  - Update Catalyst to 13.1
  - Result: Firefox crashes while update is installing (crash report)
PASS Mihaela, Windows 8 x64, AMD Radeon HD 7700 Series, Catalyst 13.1
- Sites and scenarios:
  - facebook: timeline navigation, view photos, status update, play games
  - gmail: compose and send email, read and navigate through emails
  - yahoo mail: read/navigate through emails
  - youtube: search, watch video, view videos from playlist, interact with video
  - google maps WebGL enabled: search, zoom, street view
  - vimeo: play video, interact with video
  - http://www.ebizmba.com/articles/best-flash-sites: interact with some of the websites
PASS Bogdan, Windows 8 x32, AMD Radeon HD 6450, Catalyst 13.1
- Sites and scenarios:
  - https://www.facebook.com/ - Chat, posted comments, opened images, played videos and games.
  - https://mail.google.com/ - Compose/Send/Reply/Forward mails.
  - http://www.youtube.com/ - Loaded several videos and played them at the same time, changed the video from fullscreen mode to large player and small player, switched between video quality.
PASS Bogdan, Windows 8 x32, AMD Radeon HD 6450, Catalyst 13.1
- Flash tests:
  - http://jimcarrey.com/ http://www.syfy.com/tinman/oz/ http://wechoosethemoon.org/ http://cloudsovercuba.com/ http://waterlife.nfb.ca/ - Entered different areas of these sites that contain flash animations.
PASS Bogdan, Windows 8 x32, AMD Radeon HD 6450, Catalyst 13.1
- WebGL and Canvas tests:
  - http://www.ibiblio.org/e-notes/webgl/waves/vortex.html - Click on the image and change the direction of the image to stress the page, zoom in and out with mouse scroll
  - http://www.zygotebody.com/#nav=6.18,84.78,250 - Move the scrolling bar from the left side of the page through different stages of human anatomy, zoom in and out several times and then click the home button from the top left of the page.
  - http://www.smashcat.org/av/canvas_test/ - Run this test page.
  - http://www.kevs3d.co.uk/dev/canvask3d/k3d_test.html - Click on images to activate them then click again to deactivate them.
  - http://gyu.que.jp/jscloth/ - Go with the cursor of the mouse inside of the image and move the cursor in different directions and click several times.
PASS Bogdan, Windows 8 x32, AMD Radeon HD 6450, Catalyst 11.12
- Update driver test:
  - Updated the GPU driver from version 11.12 to 13.1 while videos were playing and Canvas test was running.
result Tester, Windows Version, GPU, Catalyst version
- list of sites, scenarios tested

QAWANTED Bugs

bug 806071 crash in js::analyze::ScriptAnalysis::analyzeLifetimes
- See if it can be reproduced with AMD Radeon HD 6250/6290/6310/6320/6400M on Windows 7 in Firefox 19.0 & 19.0.1

Update Testing

*Results provided via Mozmill automation*
Platform	Target	releasetest (reports: 19.0.1, 19.0)	release (reports: 19.0.1, 19.0)
Windows XP	Firefox 19.0	0 Failures	0 Failures
Windows Vista	Firefox 19.0	0 Failures	0 Failures
Windows 7 32-bit	Firefox 19.0	0 Failures	0 Failures
Windows 7 64-bit	Firefox 19.0	0 Failures	0 Failures
Windows 8 32-bit	Firefox 19.0.1	0 Failures	0 Failures
Windows 8 64-bit	Firefox 19.0.1	0 Failures	0 Failures
Mac OS X 10.6	Firefox 19.0	0 Failures	0 Failures
Mac OS X 10.7	Firefox 19.0	0 Failures	0 Failures
Mac OS X 10.8	Firefox 19.0	0 Failures	0 Failures
Ubuntu 12.04 32-bit	Firefox 19.0	0 Failures	0 Failures
Ubuntu 12.04 64-bit	Firefox 19.0	0 Failures	0 Failures

Bugs Found

[DONE] VERIFIED FIXED bug 845947 Firefox <=19.0 updates on betatest do not go passed Firefox 19.0

Unthrottling via bug 845447

Test results for unthrottling 18.0.2 updates on Windows 8; 19.0 updates on all other platforms.

Platform	Source	Target	Result
Mac OSX 10.7	Firefox 18.0.2 en-US	Firefox 19.0 en-US	PASS
Mac OSX 10.7	Firefox 18.0 de	Firefox 19.0 de	PASS
Ubuntu 12.04	Firefox 17.0 en-US	Firefox 19.0 en-US	PASS
Ubuntu 12.04	Firefox 18.0.1 fr	Firefox 19.0 fr	PASS
Windows 7	Firefox 18.0 en-US	Firefox 19.0 en-US	PASS
Windows 7	Firefox 17.0.1 it	Firefox 19.0 it	PASS
Windows 8	Firefox 18.0.2 en-US	no update	PASS
	Firefox 18.0.2 es-ES	no update	PASS
	Firefox 18.0 en-US	Firefox 18.0.2 en-US	PASS
	Firefox 17.0 zh-TW	Firefox 18.0.2 zh-TW	PASS

Bug Fix Verifications

Bugs	QA Contact	Status
bug 844053 Update only win8 users to 19.0.1	Anthony Hughes	[DONE] VERIFIED FIXED
bug 830531 [Win8] crash in XPC_WN_Helper_NewResolve	Anthony Hughes	[DONE] Verified crashes are mitigated
bug 845447 Please unthrottle updates for non-Win8 users	Anthony Hughes	[DONE] VERIFIED FIXED

Issues Found

The following are new issues found while testing the Beta candidate. Items in bold are BLOCKER bugs.

Bug Triage

QAWANTED

These bugs require help finding steps to reproduce, regression ranges, and/or test cases.

Bug	QA Contact	Status
bug 806071 crash in js::analyze::ScriptAnalysis::analyzeLifetimes
bug 829954 OOM crash in mozilla::gfx::AlphaBoxBlur::Blur	Juan Becerra	[DROPPED] Unable to find steps to reproduce
bug 839239 Prepare a hotfix to disabled PDF.js in Firefox 19	Anthony Hughes	[DROPPED] RESOLVED INVALID
bug 836236 Crash in _ZThn184_N22nsComboboxControlFrame15GetRollupWidgetEv	Juan Becerra	[DROPPED] RESOLVED WONTFIX
bug 838549 Loaded document disappears in some PDFs	Anthony Hughes	[DROPPED] RESOLVED WONTFIX
bug 820379 crash in mozilla::dom::CanvasRenderingContext2D::GetImageDataArray	Anthony Hughes	[DROPPED] RESOLVED WONTFIX
bug 839232 Move pre-release CTP blocks to Firefox 20 and up	Anthony Hughes	[DONE] VERIFIED FIXED
bug 797288 Pdf.js download behavior inconsistent	Manuela (Softvision)	[DONE] VERIFIED FIXED

CRITICAL FIXES

These bugs need their fixes test and verified. These bugs should be tested before any other fixed bugs. Add a bullet item for any bugs you test.

[DONE] VERIFIED bug 823119 crash in mozilla::gfx::DrawTargetD2D::CopySurface with PDF Viewer (ioana)
[DONE] UPDATED bug 828184 startup crash in nsPrefBranch::GetIntPref with QIPCAP.dll (Websense Endpoint) (ioana)
- recent crashes available in Socorro.

NORMAL FIXES

These bugs need their fixes tested and verified. Add a bullet item for any bugs you test.

[DONE] VERIFIED bug 808466 Changing selection sometimes doesn't repaint immediately (Cornel)
[DONE] VERIFIED bug 685202 Using a Windows non-admin user account, applying an automatic Firefox update causes the program to not clean up after itself, leaving the Apply Update button incorrectly enabled and Firefox unable to recieve futher automatic updates. (Cornel)
[DONE] VERIFIED bug 770252 HTML Video / Audio Playback does not playback through Headphones (Cornel)
[DONE] VERIFIED bug 816458 Removing an inline css transform on a table applies twice the resulting computed transform (Cornel)
[DONE] VERIFIED bug 800799 http-on-modify-request now called after JS that started the load is no longer on the stack (Bogdan)
[DONE] COMMENTED bug 834325 SVG graphic rendering broken when embedded (Bogdan)
[DONE] VERIFIED bug 815593 large image within div with overflow-y: scroll disappears for a few seconds when scrolling (Bogdan)
[DONE] VERIFIED bug 795701 WebGL Renderer : no information in about:support (Manuela)
[DONE] VERIFIED bug 809178 bad drawing on a page with SVG embedded in iframes (Manuela)

SECURITY FIXES

These bugs need their fixes tested and verified. This will require elevated Bugzilla privileges. Add a bullet item for any bugs you test.

TOPCRASH FIXES

These bugs need to be tested to confirm the crash no longer reproduces. You will need to confirm you can reproduce the crash in a known broken build before verifying it's fixed in the latest build. If the crash is not reproducible for you in a known broken build, check crashstats for instances of the crash signature in the latest Firefox versions. Add a bullet item for any bugs you test.

UNCONFIRMED

These bugs need confirmation that they reproduce and need to be moved into an appropriate Bugzilla component. Add a bullet item for any bugs you test.

Known Issues

Full Query

ID	Summary	Priority	Status
791195	Session is not restored after Firefox crash	--	RESOLVED
803456	Highlighting from Find within a PDF remains after closing Find bar	P2	RESOLVED
803468	Highlighting doesn't follow rotation	P3	RESOLVED
828991	In a page with wide content (triggering shrink-to-fit), "print selection" prints very tiny output, even if the wide stuff isn't selected	--	NEW

4 Total; 1 Open (25%); 3 Resolved (75%); 0 Verified (0%);

19.0.2

Builds	FTP, HTTP
ID	20130307023931
Changeset	6ffe3e9da8a8 (pushlog)

Checklist

[DONE] Functional Automation
[DONE] Manual Testing
[DONE] Bug Fix Verifications
[DONE] Conditional sign-off for push to mirrors
[DONE] Updates qualified on releasetest channel
[DONE] Final sign-off for push to release
[DONE] Updates qualified on release channel
[DONE] Confirm download links on download.mozilla.org

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Functional [report]	Remote [report]	Addon [report]
Windows XP	0 Failures	0 Failures	0 Failures
Windows Vista	0 Failures	0 Failures	0 Failures
Windows 7 32-bit	0 Failures	0 Failures	0 Failures
Windows 7 64-bit	0 Failures	0 Failures	0 Failures
Windows 8 32-bit	0 Failures	0 Failures	0 Failures
Windows 8 64-bit	0 Failures	0 Failures	0 Failures
Mac OS X 10.6	0 Failures	0 Failures	0 Failures
Mac OS X 10.7	0 Failures	0 Failures	0 Failures
Mac OS X 10.8	0 Failures	0 Failures	0 Failures
Ubuntu 12.04 32-bit	0 Failures	0 Failures	0 Failures
Ubuntu 12.04 64-bit	0 Failures	0 Failures	0 Failures

Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla Run the following regression tests and do some exploratory testing to exercise some possible edge-cases. Report any bugs found in Bugzilla and list them in the Issues Found section below.

Breakpad End-to-End functionality (testcase)

[DONE] PASS Windows 8 32-bit (Manuela)
[DONE] PASS Windows 7 64-bit (Simona)
[DONE] PASS Ubuntu 12.10 32-bit (Mihaela)
[DONE] PASS Mac OSX 10.8 (Paul)

Smoke Testing (basic functionality and web compatibility, particularly websites with built in text editors)

[DONE] PASS Windows 8 32-bit (Manuela)
[DONE] PASS Windows 7 64-bit (Simona)
[DONE] PASS Ubuntu 12.10 32-bit (Mihaela)
[DONE] PASS Mac OSX 10.8 (Paul)

Detailed results in Etherpad

Update Testing

*Partial: 16.0.2, 17.0.1, 18.0.2, 19.0, 19.0.1*
*Complete: <=16.0.1, 17.0, 18.0, 18.0.1*
*Locales: en-US, ...*
Platform	Target	betatest [report]	releasetest [report]	release [report]
Windows XP	Firefox 19.0.2	0 Failures	0 Failures	0 Failures
Windows Vista	Firefox 19.0.2	0 Failures	0 Failures	0 Failures
Windows 7 32-bit	Firefox 19.0.2	0 Failures	0 Failures	0 Failures
Windows 7 64-bit	Firefox 19.0.2	0 Failures	0 Failures	0 Failures
Windows 8 32-bit	Firefox 19.0.2	0 Failures	0 Failures	0 Failures
Windows 8 64-bit	Firefox 19.0.2	0 Failures	0 Failures	0 Failures
Mac OS X 10.6	Firefox 19.0.2	0 Failures	0 Failures	0 Failures
Mac OS X 10.7	Firefox 19.0.2	0 Failures	0 Failures	0 Failures
Mac OS X 10.8	Firefox 19.0.2	0 Failures	0 Failures	0 Failures
Ubuntu 12.04 32-bit	Firefox 19.0.2	0 Failures	0 Failures	0 Failures
Ubuntu 12.04 64-bit	Firefox 19.0.2	0 Failures	0 Failures	0 Failures

Bugs Found

Bug Fix Verifications

[DONE] bug 848644 ZDI CanSecWest 2013 bug (softvision)
[DONE] bug 830531 [Win8] crash in XPC_WN_Helper_NewResolve with AMD Radeon (ashughes)
[DONE] bug 839270 crash in nsStyleContext::nsStyleContext with AMD Radeon (bsmedberg via VNC)
[DONE] bug 846391 back out bug 846388 as part of next desktop firefox release (blocks Win8 release updates)

Issues Found

The following are new issues found while testing the Beta candidate. Items in bold are BLOCKER bugs.

Post-mortem

Successes

The beta checklist developed while Firefox 19 was in beta has reduced the time spent on superfluous testing.
- Great, is it enough freed up time to start moving our strategy in a proactive direction?
- Definitely. A first step might be creating beta test runs as part of the community test run proposal.

Failures

PDF.js UX concerns
- Could we have done more to put pressure on the team to fix or disable?
- Initial negative feedback has died down significantly, suspected early detractors
- How do we feel about using our Release early adopter audience as betatesters?
- Should we be doing more to contribute to Beta audience growth?
AMD concerns
- Could we have done more to test various hardware before release?
- Should we consider out/crowdsource testing a lot of hardware once before release?
l10n blind-spot
- Aurora l10n win32 builds were missing and QA didn't catch it in time
- Propose adding l10n to Aurora/Beta/Release sign-off checklist, to consist of:
  - functional testing via mozmill-ci (currently en-US & fr via pulse)
  - update testing via mozmill-ondemand (Beta/Release), manual spotcheck (Aurora)
  - verify download links via manual spotcheck
  - minimum of one random locale per platform
  - more proactive triaging of dashboards (daily, weekly?)
  - longterm: look at ways to improve cross-team awareness and automation coverage
- Auto-Dev:
  - Can we expand locale coverage for mozmill-ci functional? update?
  - Can we expand locale coverage for mozmill-ondemand update?
  - Can we expand locale coverage for mozmill-ci update?
  - What will it take to bring online more nodes?
  - What would it take to minimally qualify all locales? (time to execute, number of nodes)
  - Should we add auroratest to the mozmill-ci update jobs?
- Web QA
  - Do they have tests to verify the download links?
  - If not, lets get some in place
  - If so, are they auto or manual trigger? where do the results go?
- L10n Team
  - Do they have verification mechanisms in place?
  - If not, should they bear some verification responsibility?
  - If so, how can QA be better informed of the l10n state before sign-off?
- Build Team
  - Do they have scripts to verify build/update availability?
  - If not, should they bear some verification responsibility?
  - If so, what can be done to make QA more aware?
- Dev/Product ownership Team
  - Feature pages are not kept up-to-date, ergo we can't track down which features are targeted for which release. This means QA is starting the work on most features much later that necessary.
  - Some developers never answer feature-related emails, or they reply very late.
- QA Team
  - The bugs with no tracking flags for 19, that have been fixed on Firefox 19 while it was in nightly, should have been included in the bug verifications query
  - Crash bugs are verified up to some level by the team monitoring crash stats. How much value did our verifying them add?

Action Items

[DONE] [ioana] Create a community oriented testrun to spur participation in releases
[DONE] [ashughes] Work with auto-dev to expand automation coverage to en-US, fr, de, and it
[DONE] [juanb] Work with Asa to improve clarity of feature development
[DONE] [ioana] Escalate instances of unresponsive developers to Juan and Anthony
[ON TRACK] [ashughes] Talk with development team leads to find more effective forms of communication
[DONE] [ashughes] Update bug queries to include Firefox version reported and milestone, not just tracking/status flags
[ON TRACK] [ashughes] Talk to stability team to identify value add of crash verifications, improve processes

Releases/Firefox 19/Test Plan

Contents

Firefox 19 Test Plan

Summary

Details

Nightly

Merge

Features

Features

Endurance Automation

Functional Automation

Bug Triage

Aurora Merge

Automated Testing

Update Testing

Bug Triage

Pre-beta Feature Sign-off

Beta Checklist

Beta 1

Automated Testing

Manual Testing

Update Testing

Bug Triage

Issues Found

Beta 2

Automated Testing

Manual Testing

Update Testing

Bug Triage

Issues Found

Known Issues

Beta 3

Automated Testing

Manual Testing

Update Testing

Bug Triage

Issues Found

Beta 4

Automated Testing

Manual Testing

Update Testing

Bug Triage

Issues Found

Beta 5

Automated Testing

Manual Testing

PDF Viewer Blocker Triage

Update Testing

Bug Fix Verifications

Issues Found

Beta 6

Automated Testing

Manual Testing

PDF Viewer Blocker Triage

Update Testing

Bug Fix Verifications

Issues Found

Release Candidate 1

Automated Testing

Manual Testing

Update Testing

Bug Fix Verifications

Issues Found

19.0.1

Automated Testing

Manual Testing

Update Testing

Unthrottling via bug 845447

Bug Fix Verifications

Issues Found

Bug Triage

QAWANTED

CRITICAL FIXES

NORMAL FIXES

SECURITY FIXES

TOPCRASH FIXES

UNCONFIRMED

Known Issues

19.0.2

Automated Testing