Releases/Firefox 18/Test Plan
« Firefox 17 | Firefox 18 | Firefox 19 »
Contents
Firefox 18 Test Plan
Summary
Milestone | Firefox 18 |
Status | Currently in Aurora -- merging to Beta on November 19, 2012 |
Lead | Juan Becerra (irc: juanb) |
2nd | Anthony Hughes (irc: ashughes) |
Builds | [ftp] [http] [web] |
The following is the test plan for Firefox 18 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 18. After the release this document will be used as an archive of what was done to validate this release.
If you have some free time, please pick a task below and get in touch with one of the leads.
Details
The following is a description of the tasks which need to be completed throughout this release.
- 2012-08-27: [DONE] Nightly Merge
- 2012-10-08: [DONE] Aurora Merge
- 2012-11-16: [DONE] Pre-Beta feature sign-offs
- 2012-11-22: {{|}} Beta 1
- 2012-11-29: {{|}} Beta 2
- 2012-12-06: {{|}} Beta 3
- 2012-12-11: {{|}} Beta 4
- 2012-12-18: {{|}} Beta 5
- TBA {{|}} Beta 6
- TBA {{|}} Release Candidate
- TBA {{|}} Final
Nightly
Merge
- Functional Automation - report - PASS (2012-08-28)
- Update Spotchecks - report - PASS (2012-08-28)
Features
See the Feature Release Tracking wiki for a general overview
Features
See the Feature Release Tracking wiki for a general overview
Feature | Owner | Status | Tests |
Panel-based Download Manager | Simona Badau | On Track | |
Show PDF inline | Mihaela Velimiroviciu | On Track | |
Modern JIT Compiler for JavaScript (IonMonkey) | No QA needed here. |
Endurance Automation
The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard.
- What to Do
- Check the Charts
- If you see an anomalous event (ie. spike or dip), get the report for the specific event
- If the event happens twice in a row, file a bug under Mozmill Tests
- Notify Anthony Hughes of the issue
- Record the event in this wiki below
- Current Status (Mihaela Velimiroviciu) - Last Updated
Bugzilla query error
Functional Automation
The purpose of this section is to track any potential functional regressions via results reported in the dashboard.
- What to Do
- Check the testrun reports
- If you see any failing tests, check that there are not any bugs on file for it already
- If not, file a bug under Mozmill Tests
- Notify Anthony Hughes of the issue
- Record the event in this wiki below
- Current Status (Vlad Maniac / Alex Lakatos) - Last Updated
Bug Triage
The purpose of this section is to triage UNCONFIRMED incoming bugs.
- What to Do
- Check the UNCONFIRMED bugs
- Test the bugs in the latest Firefox Nightly
- If you can't reproduce the bug, ask for the reporter to provide more information
- If you reproduce the bug, add your testing information and mark the bug NEW
- Notify Anthony Hughes of the NEW issue
- Current Status (SoftVision) - Last Updated
Aurora Merge Sign-off
Automated Tests
Results of the Mozmill automation testruns.
Functional Tests
Platform | Result: report | Failures: known bugs |
Windows XP | result | |
Windows Vista | result | |
Windows 7 32-bit | result | |
Windows 7 64-bit | result | |
Mac OS X 10.6 | result | |
Mac OS X 10.7 | result | |
Ubuntu 11.10 32-bit | result | |
Ubuntu 11.10 -bit | result | |
Bugs | ||
Endurance Tests
Platform | Result: report | Max Memory |
Windows XP | result | |
Windows Vista | result | |
Windows 7 32-bit | result | |
Windows 7 64-bit | result | |
Mac OS X 10.6 | result | |
Mac OS X 10.7 | result | |
Ubuntu 11.10 32-bit | result | |
Ubuntu 11.10 -bit | result | |
Bugs | ||
Feature Sign-offs
- Log in to moztrap.mozilla.org
- Select Run Tests
- Select Product: Desktop Firefox
- Select Versions: 18
- Select Runs: Firefox 18 Aurora Merge Feature sign off
Feature | Owner | Win32 | Mac | Linux |
Panel-based Download Manager | Simona Badau | disabled-2012-10-12 | disabled-2012-10-12 | disabled-2012-10-12 |
Show PDF inline | Mihaela Velimiroviciu | PASS-2012-10-12 | PASS-2012-10-12 | PASS-2012-10-12 |
Modern JIT Compiler for JavaScript (IonMonkey) | No need for QA owner | NA | NA | NA |
- Details
- Panel Based Download Manager - the feature is disabled on Firefox 18 according to https://bugzilla.mozilla.org/show_bug.cgi?id=799436
Pre-Beta Feature Sign-offs
- Log in to moztrap.mozilla.org
- Select Run Tests
- Select Product: Desktop Firefox
- Select Versions: 18
- Select Runs: Firefox 18 Aurora Merge Feature sign off
Feature | Owner | Win32 | Mac | Linux |
Panel-based Download Manager | Simona Badau | disabled | disabled | disabled |
Show PDF inline | Mihaela Velimiroviciu | PASS | PASS | PASS |
Per-Site Third-Party Cookie Setting | Ioana Budnar | PASS | PASS | PASS |
Modern JIT Compiler for JavaScript (IonMonkey) | No need for QA owner | NA | NA | NA |
- Details
- Panel Based Download Manager - the feature is re-targeted for Firefox 19 and is disabled on Firefox 18 according to https://bugzilla.mozilla.org/show_bug.cgi?id=799436
Beta 1
Builds | Build 1: FTP, HTTP |
ID | 20121121075611 |
Changeset | 1f1083953e62 (pushlog) |
- Checklist
- [DONE] Automated Testing
- [DONE] Manual Testing
- [DONE] Update Testing
- [DONE] Bug Triage
- [DONE] Peer Sign-off
- [DONE] Final sign-off sent and updates qualified on Beta channel
Automated Testing
The following are the detailed results of the automated Mozmill test runs.
Platform | Build #1 Results (report)2012-11-22 |
Windows XP | PASS - 2012-11-22 |
Windows Vista | PASS - 2012-11-22 |
Windows 7 32-bit | PASS - 2012-11-22 |
Windows 7 64-bit | PASS - 2012-11-22 |
Windows 8 32-bit | PASS - 2012-11-22 |
Windows 8 64-bit | PASS - 2012-11-22 |
Mac OS X 10.6 | PASS - 2012-11-22 |
Mac OS X 10.7 | PASS - 2012-11-22 |
Mac OS X 10.8 | PASS - 2012-11-22 |
Ubuntu 12.04 32-bit | PASS - 2012-11-22 |
Ubuntu 12.04 64-bit | PASS - 2012-11-22 |
- Test Failures (known bugs)
Manual Testing
Run the Desktop Firefox Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.
Platform | Tester | Result |
Windows 7 | Virgil | PASS - 2012-11-23 |
Windows 8 | Ioana | PASS - 2012-11-23 |
Mac OSX 10.7 | Mihaela | PASS - 2012-11-23 |
Mac OSX 10.8 | Simona | PASS - 2012-11-23 |
Ubuntu 12.04 | Paul | PASS - 2012-11-23 |
Update Testing
- Partial: 17.0b6
- Complete: <= 17.0b5
Platform | Releasetest (report) | Beta (report) |
Windows XP | PASS - 2012-11-22 | PASS - 2012-11-22 |
Windows Vista | PASS - 2012-11-22 | PASS - 2012-11-22 |
Windows 7 32-bit | PASS - 2012-11-22 | PASS - 2012-11-22 |
Windows 7 64-bit | PASS - 2012-11-22 | PASS - 2012-11-22 |
Windows 8 32-bit | PASS - 2012-11-22 | PASS - 2012-11-22 |
Windows 8 64-bit | PASS - 2012-11-22 | PASS - 2012-11-22 |
Mac OS X 10.6 | PASS - 2012-11-22 | PASS - 2012-11-22 |
Mac OS X 10.7 | PASS - 2012-11-22 | PASS - 2012-11-22 |
Mac OS X 10.8 | PASS - 2012-11-22 | PASS - 2012-11-22 |
Ubuntu 12.04 32-bit | PASS - 2012-11-22 | PASS - 2012-11-22 |
Ubuntu 12.04 64-bit | PASS - 2012-11-22 | PASS - 2012-11-22 |
- Issues
Feature Sign-off
Feature owners to test and sign-off overall quality of their features following uplift to Beta.
Feature | Owner | Win32 | Mac | Linux |
Show PDF inline | Mihaela Velimiroviciu | PASS - 2012-11-23 | PASS - 2012-11-23 | PASS - 2012-11-23 |
Per-Site Third-Party Cookie Setting | Ioana Budnar | PASS - 2012-11-22 | PASS - 2012-11-23 | PASS - 2012-11-22 |
Beta 2
Builds | Build 1: FTP, HTTP |
ID | TBD |
Changeset | e8b696e40f36 (pushlog) |
- Checklist
- [DONE] Automated Testing
- [DONE] Manual Testing
- [ON TRACK] Update Testing
- [ON TRACK] Bug Triage
- [ON TRACK] Final sign-off sent and updates qualified on Beta channel
Automated Testing
The following are the detailed results of the automated Mozmill test runs.
Platform | Build #1 Results (report)2012-11-28 |
Windows XP | PASS - 2012-11-28 |
Windows Vista | PASS - 2012-11-28 |
Windows 7 32-bit | PASS - 2012-11-28 |
Windows 7 64-bit | PASS - 2012-11-28 |
Windows 8 32-bit | PASS - 2012-11-28 |
Windows 8 64-bit | PASS - 2012-11-28 |
Mac OS X 10.6 | PASS - 2012-11-28 |
Mac OS X 10.7 | PASS - 2012-11-28 |
Mac OS X 10.8 | PASS - 2012-11-28 |
Ubuntu 12.04 32-bit | PASS - 2012-11-28 |
Ubuntu 12.04 64-bit | PASS - 2012-11-28 |
- Test Failures (known bugs)
Manual Testing
Run the Desktop Firefox Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.
Platform | Tester | Result |
Windows 7 | Mihaela | PASS - 2012-11-29 |
Windows 8 | Ioana | PASS - 2012-11-29 |
Mac OSX 10.7 | Manuela | PASS - 2012-11-29 |
Mac OSX 10.8 | Virgil | PASS - 2012-11-29 |
Ubuntu 12.04 | Simona | PASS - 2012-11-29 |
Update Testing
- Partial: 17.0b6, 17.0b5
- Complete: <= 16.0b4
Platform | Releasetest (report) | Beta (report) |
Windows XP | PASS - 2012-11-28 | RESULT - 2012-11-28 |
Windows Vista | PASS - 2012-11-28 | RESULT - 2012-11-28 |
Windows 7 32-bit | PASS - 2012-11-28 | RESULT - 2012-11-28 |
Windows 7 64-bit | PASS - 2012-11-28 | RESULT - 2012-11-28 |
Windows 8 32-bit | PASS - 2012-11-28 | RESULT - 2012-11-28 |
Windows 8 64-bit | PASS - 2012-11-28 | RESULT - 2012-11-28 |
Mac OS X 10.6 | PASS - 2012-11-28 | RESULT - 2012-11-28 |
Mac OS X 10.7 | PASS - 2012-11-28 | RESULT - 2012-11-28 |
Mac OS X 10.8 | PASS - 2012-11-28 | RESULT - 2012-11-28 |
Ubuntu 12.04 32-bit | PASS - 2012-11-28 | RESULT - 2012-11-28 |
Ubuntu 12.04 64-bit | PASS - 2012-11-28 | RESULT - 2012-11-28 |
- Issues
Beta 3
Builds | Build 1: FTP, HTTP |
ID | 20121205060959 |
Changeset | 68cb786721c7 (pushlog) |
- Checklist
- [ON TRACK] Automated Testing
- [DONE] Manual Testing
- [ON TRACK] Update Testing
- [ON TRACK] Bug Triage
- [ON TRACK] Peer Sign-off
- [ON TRACK] Final sign-off sent and updates qualified on Beta channel
Automated Testing
The following are the detailed results of the automated Mozmill test runs.
Platform | Build #1 Results (report)2012-12-05 |
Windows XP | PASS - 2012-12-05 |
Windows Vista | PASS - 2012-12-05 |
Windows 7 32-bit | PASS - 2012-12-05 |
Windows 7 64-bit | PASS - 2012-12-05 |
Windows 8 32-bit | PASS - 2012-12-05 |
Windows 8 64-bit | PASS - 2012-12-05 |
Mac OS X 10.6 | PASS - 2012-12-05 |
Mac OS X 10.7 | PASS - 2012-12-05 |
Mac OS X 10.8 | PASS - 2012-12-05 |
Ubuntu 12.04 32-bit | PASS - 2012-12-05 |
Ubuntu 12.04 64-bit | PASS - 2012-12-05 |
- Test Failures (known bugs)
Manual Testing
Run the Desktop Firefox Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.
Platform | Tester | Result |
Windows 7 | Manuela | PASS - 2012-12-06 |
Windows 8 | Virgil | PASS - 2012-12-06 |
Mac OSX 10.7 | Simona | PASS - 2012-12-06 |
Mac OSX 10.8 | Paul | PASS - 2012-12-06 |
Ubuntu 12.04 | Ioana | PASS - 2012-12-06 |
Exploratory testing around beta 3 changes- etherpad
Update Testing
- Partial: 18.0b2, 18.0b1
- Complete: <= 17.0b6
Platform | Releasetest (report) | Beta (report) |
Windows XP | PASS - 2012-12-05 | RESULT - 2012-12-05 |
Windows Vista | PASS - 2012-12-05 | RESULT - 2012-12-05 |
Windows 7 32-bit | PASS - 2012-12-05 | RESULT - 2012-12-05 |
Windows 7 64-bit | PASS - 2012-12-05 | RESULT - 2012-12-05 |
Windows 8 32-bit | PASS - 2012-12-05 | RESULT - 2012-12-05 |
Windows 8 64-bit | PASS - 2012-12-05 | RESULT - 2012-12-05 |
Mac OS X 10.6 | PASS - 2012-12-05 | RESULT - 2012-12-05 |
Mac OS X 10.7 | PASS - 2012-12-05 | RESULT - 2012-12-05 |
Mac OS X 10.8 | PASS - 2012-12-05 | RESULT - 2012-12-05 |
Ubuntu 12.04 32-bit | PASS - 2012-12-05 | RESULT - 2012-12-05 |
Ubuntu 12.04 64-bit | PASS - 2012-12-05 | RESULT - 2012-12-05 |
- Issues
Beta 4
Builds | Build 1: FTP, HTTP |
ID | TBD |
Changeset | e904ee016207 (pushlog) |
- Checklist
- [DONE] Automated Testing
- [DONE] Manual Testing
- [DONE] Update Testing
- [DONE] Multi-Window Changes
- [ON TRACK] Bug Triage
- [ON TRACK] Final sign-off sent and updates qualified on Beta channel
Automated Testing
The following are the detailed results of the automated Mozmill test runs.
Platform | Build #1 Results (report)2012-12-12 |
Windows XP | 2 Fails - 2012-12-12 |
Windows Vista | PASS - 2012-12-12 |
Windows 7 32-bit | PASS - 2012-12-12 |
Windows 7 64-bit | PASS - 2012-12-12 |
Windows 8 32-bit | PASS - 2012-12-12 |
Windows 8 64-bit | PASS - 2012-12-12 |
Mac OS X 10.6 | PASS - 2012-12-12 |
Mac OS X 10.7 | PASS - 2012-12-12 |
Mac OS X 10.8 | PASS - 2012-12-12 |
Ubuntu 12.04 32-bit | PASS - 2012-12-12 |
Ubuntu 12.04 64-bit | PASS - 2012-12-12 |
- Test Failures (known bugs)
Manual Testing
Run the Desktop Firefox Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.
Platform | Tester | Result |
Windows 7 | Manuela | PASS - 2012-12-13 |
Windows 8 | Simona | PASS - 2012-12-13 |
Mac OSX 10.7 | Paul | PASS - 2012-12-13 |
Mac OSX 10.8 | Virgil | PASS - 2012-12-13 |
Ubuntu 12.04 | Virgil | PASS - 2012-12-13 |
Update Testing
- Partial: 18.0b3, 18.0b2
- Complete: <= 17.0b6
Platform | Releasetest (report) | Beta (report) |
Windows XP | PASS - 2012-12-13 | PASS - 2012-12-13 |
Windows Vista | PASS - 2012-12-13 | PASS - 2012-12-13 |
Windows 7 32-bit | PASS - 2012-12-13 | PASS - 2012-12-13 |
Windows 7 64-bit | PASS - 2012-12-13 | PASS - 2012-12-13 |
Windows 8 32-bit | PASS - 2012-12-13 | PASS - 2012-12-13 |
Windows 8 64-bit | PASS - 2012-12-13 | PASS - 2012-12-13 |
Mac OS X 10.6 | PASS - 2012-12-13 | PASS - 2012-12-13 |
Mac OS X 10.7 | PASS - 2012-12-13 | PASS - 2012-12-13 |
Mac OS X 10.8 | PASS - 2012-12-13 | PASS - 2012-12-13 |
Ubuntu 12.04 32-bit | PASS - 2012-12-13 | PASS - 2012-12-13 |
Ubuntu 12.04 64-bit | PASS - 2012-12-13 | PASS - 2012-12-13 |
- Issues
Beta 5
Builds | Build 1: FTP, HTTP |
ID | 20121219074241 |
Changeset | 05ecd95e85da (pushlog) |
- Checklist
- [DONE] Automated Testing
- [ON TRACK] Manual Testing
- [ON TRACK] Update Testing
- [ON TRACK] Bug Triage
- [ON TRACK] Final sign-off sent and updates qualified on Beta channel
Automated Testing
The following are the detailed results of the automated Mozmill test runs.
Platform | Build #1 Results (report) 2012-12-19 |
Windows XP | PASS - 2012-12-19 |
Windows Vista | PASS - 2012-12-19 |
Windows 7 32-bit | PASS - 2012-12-19 |
Windows 7 64-bit | PASS - 2012-12-19 |
Windows 8 32-bit | PASS - 2012-12-19 |
Windows 8 64-bit | PASS - 2012-12-19 |
Mac OS X 10.6 | PASS - 2012-12-19 |
Mac OS X 10.7 | PASS - 2012-12-19 |
Mac OS X 10.8 | PASS - 2012-12-19 |
Ubuntu 12.04 32-bit | PASS - 2012-12-19 |
Ubuntu 12.04 64-bit | PASS - 2012-12-18 |
- Test Failures (known bugs)
Manual Testing
Run the Desktop Firefox Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.
Platform | Tester | Result |
Windows 7 | Mihaela | PASS - 2012-12-20 |
Windows 8 | Ioana | PASS - 2012-12-20 |
Mac OSX 10.7 | Paul | PASS - 2012-12-20 |
Mac OSX 10.8 | Simona | PASS - 2012-12-20 |
Ubuntu 12.04 | Manuela | PASS - 2012-12-20 |
Mac OSX 10.6 (32bit mode) | Manuela | PASS - 2012-12-20 |
Update Testing
- Partial: 18.0b4, 18.0b3
- Complete: <= 17.0b6
Platform | Releasetest (report) | Beta (report) |
Windows XP | PASS - 2012-12-19 | RESULT - 2012-12-18 |
Windows Vista | PASS - 2012-12-19 | RESULT - 2012-12-18 |
Windows 7 32-bit | PASS - 2012-12-19 | RESULT - 2012-12-18 |
Windows 7 64-bit | PASS - 2012-12-19 | RESULT - 2012-12-18 |
Windows 8 32-bit | PASS - 2012-12-19 | RESULT - 2012-12-18 |
Windows 8 64-bit | PASS - 2012-12-19 | RESULT - 2012-12-18 |
Mac OS X 10.6 | PASS - 2012-12-19 | RESULT - 2012-12-18 |
Mac OS X 10.7 | PASS - 2012-12-19 | RESULT - 2012-12-18 |
Mac OS X 10.8 | PASS - 2012-12-19 | RESULT - 2012-12-18 |
Ubuntu 12.04 32-bit | PASS - 2012-12-19 | RESULT - 2012-12-18 |
Ubuntu 12.04 64-bit | PASS - 2012-12-19 | RESULT - 2012-12-18 |
SVG Testing
There were a couple of bug fixes in this cycle which require a little bit of testing around. Of the two, bug 820061 dealing with SVG animations would be more worrisome. Please verify both bugs and test SVG animation with a few samples from the latest test suite cited in:
http://www.w3.org/Graphics/SVG/WG/wiki/Test_Suite_Overview
- (Virgil)Compared test cases from 18b5(with SVG related patch) with previous builds (18b3-without patch):
-http://www.w3.org/Graphics/SVG/Test/20061213/htmlEmbedHarness/basic-index.html - """PASS""" - Ubuntu 12.04, Windows 7, Mac OS 10.8 -http://www.w3.org/Graphics/SVG/Test/20080912/htmlEmbedHarness/index.html - """PASS""" - Ubuntu 12.04, Windows 7, Mac OS 10.8
It will be sufficient to verify bug 813531 with the existing test case.
- (Virgil) bug 813531 verified
- Issues
Beta 6
Builds | Build 1: FTP, HTTP |
ID | 20121227074833 |
Changeset | e904ee016207 (pushlog) |
- Checklist
- [DONE] Automated Testing - to be manually triggered
- [DONE] Manual Testing
- [ON TRACK] Update Testing - manual
- [ON TRACK] Bug Triage
- [ON TRACK] Final sign-off sent and updates qualified on Beta channel
Automated Testing
Due to bug 807099 the Mozmill CI will not run for this cycle. However, we will try to run test runs, triggered manually, for Windows 7 64bit, Mac 10.7.x, and Linux 64bit.
Platform | Build #1 Results (report) 2012-12-28 |
Windows 7 64-bit | RESULT - 2012-12-28 |
Mac OS X 10.7 | RESULT - 2012-12-28 |
Ubuntu 12.04 64-bit | RESULT - 2012-12-28 |
Manual Testing
Run the Desktop Firefox Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.
In addition to manual spot-checks on all OS's below, we should run a breadth of tests on Windows 7 64bit, Mac OS X 10.7, Linux 64bit, that cover the most common use cases across critical areas of user interaction including video playback (Flash, Silverlight), private browsing, tabbed browsing, session restore - scenarios etherpad.
Platform | Tester | Result |
Windows 7 spotcheck + in-depth manual testing | Bogdan, Ioana | PASS - 2012-12-28 |
Windows 8 | Cornel | PASS - 2012-12-28 |
Mac OSX 10.7 spotcheck + in-depth manual testing | Paul, Ioana | PASS - 2012-12-28 |
Mac OSX 10.8 | Simona | PASS - 2012-12-28 |
Ubuntu 12.04 spotcheck + in-depth manual testing | Alexandra, Virgil | PASS - 2012-12-28 |
Update Testing
Updates automation will not be run for this cycle due to bug 807099 so we will run a series of manual tests using a pair-wise approach that cover at least one type of scenario among the three main types of OS's.
- Partial: 18.0b5, 18.0b4
- Complete: <= 17.0b6
Platform | Releasetest (report) | Beta (report) |
Windows XP | RESULT - 2012-12-28 | RESULT - 2012-12-28 |
Windows Vista | RESULT - 2012-12-28 | RESULT - 2012-12-28 |
Windows 7 32-bit | RESULT - 2012-12-28 | RESULT - 2012-12-28 |
Windows 7 64-bit | RESULT - 2012-12-28 | RESULT - 2012-12-28 |
Windows 8 32-bit | RESULT - 2012-12-28 | RESULT - 2012-12-28 |
Windows 8 64-bit | RESULT - 2012-12-28 | RESULT - 2012-12-28 |
Mac OS X 10.6 | RESULT - 2012-12-28 | RESULT - 2012-12-28 |
Mac OS X 10.7 | RESULT - 2012-12-28 | RESULT - 2012-12-28 |
Mac OS X 10.8 | RESULT - 2012-12-28 | RESULT - 2012-12-28 |
Ubuntu 12.04 32-bit | RESULT - 2012-12-28 | RESULT - 2012-12-28 |
Ubuntu 12.04 64-bit | RESULT - 2012-12-28 | RESULT - 2012-12-28 |
- Issues
Beta 7
Builds | Build 1: FTP, HTTP |
ID | 20121231071231 |
Changeset | d5935d0242bc (pushlog) |
- Checklist
- [DONE] Automated Testing
- [DONE] Manual Testing
- [DONE] Update Testing - releasetest
- [ON TRACK] Bug Triage
- [ON TRACK] Final sign-off sent and updates qualified on Beta channel
Automated Testing
Due to bug 807099 the Mozmill CI will not run for this cycle. However, we will try to run test runs, triggered manually, for Windows 7, Mac 10.7.x, and Linux (any).
Platform | Build #1 Results (report) 2013-01-02 |
Windows 7 64-bit | PASS - 2013-01-02 |
Mac OS X 10.7 | PASS - 2013-01-02 |
Ubuntu 12.04 64-bit | PASS - 2013-01-02 |
Manual Testing
The scope of this beta is three bugs:
- IN PROGRESS - bug 825022 (mwobensmith) - pick up NSS roots module version 1.93
- [DONE] - bug 815795 (juanb) - stack buffer overflow with canvas
- [DONE] - bug 822682 (ashughes) - Firefox freezes on smartcard re-insertion
These need verification.
The rest is a manual spotcheck to make sure nothing is obviously broken.
Run the Desktop Firefox Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.
Platform | Tester | Result |
Windows 7 spotcheck | juanb | PASS - 2013-01-02 |
Windows 8 | ashughes | PASS - 2013-01-02 |
Mac OSX 10.7 spotcheck | juanb | PASS - 2013-01-02 |
Mac OSX 10.8 | mwobensmith | PASS - 2013-01-02 |
Ubuntu 12.04 spotcheck | juanb | PASS - 2013-01-02 |
Update Testing
- Partial: 18.0b6, 18.0b5
- Complete: <= 17.0b6
Platform | Releasetest (report) | Beta (report) |
Windows XP | PASS - 2013-01-02 | RESULT - 2013-01-02 |
Windows Vista | PASS - 2013-01-02 | RESULT - 2013-01-02 |
Windows 7 32-bit | PASS - 2013-01-02 | RESULT - 2013-01-02 |
Windows 7 64-bit | PASS - 2013-01-02 | RESULT - 2013-01-02 |
Windows 8 32-bit | PASS - 2013-01-02 | RESULT - 2013-01-02 |
Windows 8 64-bit | PASS - 2013-01-02 | RESULT - 2013-01-02 |
Mac OS X 10.6 | PASS - 2013-01-02 | RESULT - 2013-01-02 |
Mac OS X 10.7 | PASS - 2013-01-02 | RESULT - 2013-01-02 |
Mac OS X 10.8 | PASS - 2013-01-02 | RESULT - 2013-01-02 |
Ubuntu 12.04 32-bit | PASS - 2013-01-02 | RESULT - 2013-01-02 |
Ubuntu 12.04 64-bit | PASS - 2013-01-02 | RESULT - 2013-01-02 |
- Issues
Release Candidate
Builds | Build 1: FTP, HTTP |
ID | 20130104151925 |
Changeset | 650ead2dacc4 |
- Checklist
- [ON TRACK] Automated Testing
- [ON TRACK] Update Testing
- [ON TRACK] Bug Triage
- [ON TRACK] Final sign-off sent and updates qualified on release channel
Automated Testing
Platform | Build #1 Results (report) 2013-01-06 |
Windows 7 64-bit | RESULT - 2013-01-06 |
Mac OS X 10.7 | RESULT - 2013-01-06 |
Ubuntu 12.04 64-bit | RESULT - 2013-01-06 |
Manual Testing
Run the Desktop Firefox Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.
Platform | Tester | Result |
Windows 7 spotcheck | Paul | FAIL - 2013-01-07 |
Windows 8 | Alexandra | PASS - 2013-01-07 |
Mac OSX 10.7 spotcheck | Manuela | PASS - 2013-01-07 |
Mac OSX 10.8 | Simona | 'PASS - 2013-01-07 |
Ubuntu 12.04 spotcheck | Ioana | PASS - 2013-01-07 |
Etherpads
Update Testing
- Partial: 17.0.1
- Complete: <= 16.0.2
Platform | Releasetest (report) | Release (report) |
Windows XP | RESULT - 2013-01-07 | RESULT - 2013-01-08 |
Windows Vista | RESULT - 2013-01-07 | RESULT - 2013-01-08 |
Windows 7 32-bit | RESULT - 2013-01-07 | RESULT - 2013-01-08 |
Windows 7 64-bit | RESULT - 2013-01-07 | RESULT - 2013-01-08 |
Windows 8 32-bit | RESULT - 2013-01-07 | RESULT - 2013-01-08 |
Windows 8 64-bit | RESULT - 2013-01-07 | RESULT - 2013-01-08 |
Mac OS X 10.6 | RESULT - 2013-01-07 | RESULT - 2013-01-08 |
Mac OS X 10.7 | RESULT - 2013-01-07 | RESULT - 2013-01-08 |
Mac OS X 10.8 | RESULT - 2013-01-07 | RESULT - 2013-01-08 |
Ubuntu 12.04 32-bit | RESULT - 2013-01-07 | RESULT - 2013-01-08 |
Ubuntu 12.04 64-bit | RESULT - 2013-01-07 | RESULT - 2013-01-08 |
- Issues
Firefox 18.0.1
Builds | Build 1: FTP, HTTP |
ID | 20130116073211 |
Changeset | f64a3223c828 |
Pushlog | Pushlog Changes |
- Checklist
- [DONE] Automated Testing
- [ON TRACK] Update Testing
- [DONE] Bug Triage
- [ON TRACK] Final sign-off sent and updates qualified on release channel
Automated Testing
Platform | Build #1 Results (report) 2013-01-16 |
Windows 7 64-bit | PASS - 2013-01-16 |
Mac OS X 10.7 | PASS - 2013-01-16 |
Ubuntu 12.04 64-bit | PASS - 2013-01-16 |
Manual Testing
Run the Desktop Firefox Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.
Platform | Tester | Result |
Windows 7 spotcheck | Manuela | PASS - 2013-01-17 |
Windows 8 | Ioana | PASS - 2013-01-18 |
Mac OSX 10.7 spotcheck | Alexandra | PASS - 2013-01-17 |
Mac OSX 10.8 | Simona | PASS - 2013-01-18 |
Ubuntu 12.04 spotcheck | Virgil | PASS - 2013-01-17 |
Bug Verifications for 18.0.1
Fx18.0.1 Drivers:
- VERIFIED FIXED bug 828632 - System Proxy Configuration Causes Memory Leak, syslogd load (raymond)
- VERIFIED FIXED bug 828202 - System proxy settings issue with Firefox 18 (Manuela)
- VERIFIED FIXED bug 828236 - issue with NTLM and some SSL Proxies (reporter)
- VERIFIED FIXED bug 829646 - Proxy Auto Config - calling myIPAddress() in the top level (outside of FindProxyForURL) fails (Alexandra)
Ride-alongs:
- VERIFIED FIXED bug 828954 - Firefox 18 breaks Unity content on OS X (juanb)
- VERIFIED FIXED bug 818468 - Langpacks bundled in distribution/extensions are registered but disabled even if shown enabled (reporter)
- VERIFIED FIXED bug 814434 - hidpi/retina url suggestion box pops up on wrong screen (mwobensmith)
- VERIFIED FIXED bug 830303 - Fix plugin-container spawning on BSD after bug 753046 (reporter)
- NEEDS CRASHSTATS bug 828184 - startup crash in nsPrefBranch::GetIntPref with QIPCAP.dll (Websense Endpoint) (juanb)
- UNVERIFIABLE bug 828205 - crash in js:ebugScopeProxy::has with Firebug (Ioana)
Update Testing
- Partial: 17.0.1, 16.0.2
- Complete: < 16.0.2
Platform | Betatest (report) | Releasetest (report) | Release (report) |
Windows XP | PASS - 2013-01-17 | RESULT - 2013-01-17 | RESULT - 2013-01-17 |
Windows Vista | PASS - 2013-01-17 | RESULT - 2013-01-17 | RESULT - 2013-01-17 |
Windows 7 32-bit | PASS - 2013-01-17 | RESULT - 2013-01-17 | RESULT - 2013-01-17 |
Windows 7 64-bit | PASS - 2013-01-17 | RESULT - 2013-01-17 | RESULT - 2013-01-17 |
Windows 8 32-bit | PASS - 2013-01-17 | RESULT - 2013-01-17 | RESULT - 2013-01-17 |
Windows 8 64-bit | PASS - 2013-01-17 | RESULT - 2013-01-17 | RESULT - 2013-01-17 |
Mac OS X 10.6 | PASS - 2013-01-17 | RESULT - 2013-01-17 | RESULT - 2013-01-17 |
Mac OS X 10.7 | PASS - 2013-01-17 | RESULT - 2013-01-17 | RESULT - 2013-01-17 |
Mac OS X 10.8 | PASS - 2013-01-17 | RESULT - 2013-01-17 | RESULT - 2013-01-17 |
Ubuntu 12.04 32-bit | PASS - 2013-01-17 | RESULT - 2013-01-17 | RESULT - 2013-01-17 |
Ubuntu 12.04 64-bit | PASS - 2013-01-17 | RESULT - 2013-01-17 | RESULT - 2013-01-17 |
- Issues
18.0.2
Builds | Build 1: FTP, HTTP |
ID | 20130201065344 |
Changeset | c4fe2b219941 |
Pushlog | Changes |
- Checklist
- [DONE] Automated Testing
- [DONE] Manual Testing
- [DONE] Update Testing - betatest
- [DONE] Bug Fix Verifications - By looking at crash-stats
- [ON TRACK] Final sign-off sent and updates qualified on release channel
Automated Testing
Platform | Build #1 Results (2013-02-01) |
Windows XP | PASS - 2013-02-01 |
Windows Vista | PASS - 2013-02-01 |
Windows 7 32-bit | PASS - 2013-02-01 |
Windows 7 64-bit | PASS - 2013-02-01 |
Windows 8 32-bit | PASS - 2013-02-01 |
Windows 8 64-bit | PASS - 2013-02-01 |
Mac OS X 10.7 | PASS - 2013-02-01 |
Mac OS X 10.8 | PASS - 2013-02-01 |
Ubuntu 12.04 32-bit | PASS - 2013-02-01 |
Ubuntu 12.04 64-bit | PASS - 2013-02-01 |
Manual Testing
Run the Desktop Firefox Regression Tests in MozTrap and report the results below. Do some exploratory testing around the testcases to try to exercise some edge-cases.
Platform | Tester | Result |
Windows 7 | Manuela | PASS - 2013-02-04 |
Windows 8 | Simona | PASS - 2013-02-04 |
Mac OSX 10.7 | Paul | PASS - 2013-02-04 |
Mac OSX 10.8 | Virgil | PASS - 2013-02-04 |
Ubuntu 12.04 | Bogdan | PASS - 2013-02-04 |
Bug Fix Verifications
Drivers:
- bug 806820 Crash in SuppressDeletedPropertyHelper (fixed by bug 831626)
- We don't have good steps to reproduce either bug but Facebook is highly correlated
Ride-alongs:
Exploratory testing around Facebook - etherpad
Update Testing
- Partial: TBD
- Complete: TBD
Platform | Betatest (report) | Releasetest ([ report]) | Release ([ report]) |
Windows XP | PASS - 2013-02-05 | RESULT - YYYY-MM-DD | RESULT - YYYY-MM-DD |
Windows Vista | PASS - 2013-02-05 | RESULT - YYYY-MM-DD | RESULT - YYYY-MM-DD |
Windows 7 32-bit | PASS - 2013-02-05 | RESULT - YYYY-MM-DD | RESULT - YYYY-MM-DD |
Windows 7 64-bit | PASS - 2013-02-05 | RESULT - YYYY-MM-DD | RESULT - YYYY-MM-DD |
Windows 8 32-bit | PASS - 2013-02-05 | RESULT - YYYY-MM-DD | RESULT - YYYY-MM-DD |
Windows 8 64-bit | PASS - 2013-02-05 | RESULT - YYYY-MM-DD | RESULT - YYYY-MM-DD |
Mac OS X 10.6 | PASS - 2013-02-05 | RESULT - YYYY-MM-DD | RESULT - YYYY-MM-DD |
Mac OS X 10.7 | PASS - 2013-02-05 | RESULT - YYYY-MM-DD | RESULT - YYYY-MM-DD |
Mac OS X 10.8 | PASS - 2013-02-05 | RESULT - YYYY-MM-DD | RESULT - YYYY-MM-DD |
Ubuntu 12.04 32-bit | PASS - 2013-02-05 | RESULT - YYYY-MM-DD | RESULT - YYYY-MM-DD |
Ubuntu 12.04 64-bit | PASS - 2013-02-05 | RESULT - YYYY-MM-DD | RESULT - YYYY-MM-DD |
- Issues
Social API
18.0b4 Multi-window Changes
RE: bug 811247 In general the idea is to do some A-B testing, comparing 18.0b3 to 18.0b4.
Methodology
- Start Firefox 18.0b3 with a new profile
- Activate the Social API and note your current memory usage in about:memory
- Run through the list of Things to Check below, make note of the panel loading times
- Do some exploratory testing related to the toolbar button panels and multiple windows
- Make note of the current memory usage in about:memory again
- Start Firefox 18.0b4 with the same profile
- Make note of the current memory usage in about:memory again
- Run through the list of Things to Check below, make note of the panel loading times
- Do some exploratory testing related to the toolbar button panels and multiple windows
- Make note of the current memory usage in about:memory again
- Clear the cache in 18.0b4 and minimize memory in about:memory
- Make note of the current memory usage in about:memory again
- Run through the list of Things to Check below, make note of the panel loading times
- Do some exploratory testing related to the toolbar button panels and multiple windows
- Make note of the current memory usage in about:memory again
What We're Looking For
- Loading time regressions (reproduce in 18.0b4 but not 18.0b3)
- UI/UX regressions (reproduce in 18.0b4 but not 18.0b3)
- Stability regressions (reproduce in 18.0b4 but not 18.0b3)
- Memory regressions (reproduce in 18.0b4 but not 18.0b3)
Things to Check
- Clicking Friends toolbar button displays the Friends panel
- Clicking Messages toolbar button displays the Messages panel
- Clicking Notifications toolbar button displays the Notifications panel
- Verify the UI of each panel is loaded properly
- Verify you can scroll the panel if it's loaded with content
- Verify you can select a panel item
- Verify you can click the "Show All" link in each panel and the correct page loads
- Verify panel behaviour switching back and forth between multple windows
- Verify panel behaviour switching back and forth between multiple monitors
Platform Coverage (in priority order):
- Windows: at least one of Win7, Win8, WinXP, Vista
- Mac: at least one of OSX 10.8, 10.7, or 10.6
- Linux: any recent version of Ubuntu
Results:
- Ioana (Windows 7 32bit)
- Load times:
- 18.0b3: less than 1 second for each panel.
- 18.0b4: less than 1 second for each panel.
- 18.0b4 (no-cache): less than 1 second for each panel.
- Memory (start | end):
- 18.0b3: (202MB | 412MB)
- 18.0b4: (198MB | 349MB)
- 18.0b4 (no-cache): (111MB | 337MB) (111MB after cleaning the cache, 190MB after logging back in, 337MB after the exploratory)
- Issues Found:
- Opening chat windows from the messages notifications list displays inconsistent behavior (821284)
- Load times:
- Virgil (Mac OS X 10.8)
- Load times:
- 18.0b3: less than 1 second for each panel.
- 18.0b4: less than 1 second for each panel.
- 18.0b4 (no-cache): less than 1 second for each panel.
- Memory (start | end):
- 18.0b3: (232.55MB | 270.55MB)
- 18.0b4: (114.09MB | 135.61MB)
- 18.0b4 (no-cache): (81.68MB | 126.36MB)
- Load times:
- Simona (Windows 8 64bit)
- Load times:
- 18.0b3: less than 1 second for each panel.
- 18.0b4: less than 1 second for each panel.
- 18.0b4 (no-cache): less than 1 second for each panel.
- Memory (start | end):
- 18.0b3: (214MB | 463MB)
- 18.0b4: (112MB | 320MB)
- 18.0b4 (no-cache): (181MB | 400MB) (181MB after cleaning the cache, 219MB after logging back in, 400MB after the exploratory)
- Issues Found:
- Not all the notifications are displayed in the Social API sidebar (821316)
- The panels are empty if the logging in is not kept and Firefox is closed (821323)
- Load times:
- Manuela (Ubuntu 12.04)
- Load times:
- 18.0b3: less than 1 second for each panel.
- 18.0b4: less than 1 second for each panel.
- 18.0b4 (no-cache): less than 1 second for each panel.
- Memory (start | end):
- 18.0b3: (137MB | 145MB)
- 18.0b4: (129MB | 134MB)
- 18.0b4 (107MB | 124MB)
- Issues Found:
- Can't close Settings Menu when clicking outside the panel (821313)
- Load times:
Bugs
Triage
- QAWANTED: triage these bugs and test to fulfill the QA request
- UNCONFIRMED: triage these bugs for potential new regressions
- VERIFYME: verify the fix and do some exploratory testing around the issue
- FIXED: triage these bugs and test to ensure the fix is effective
New Issues
Bold any potential blockers. After every cycle make sure to call these out.
Social API
- bug 821313 - Can't close Settings Menu when clicking outside the panel
- bug 821316 - Not all the notifications are displayed in the Social API sidebar
- bug 821323 - The panels are empty if the logging in is not kept and Firefox is closed
- bug 821284 - Opening chat windows from the messages notifications list displays inconsistent behavior
- bug 821642 - Scroll in Facebook notification panel makes background page to scroll as well
Menus
- bug 814404 - [Mac OS] Missing menu items from Tools menu when focus is set to another Firefox window
Show PDF inline
- bug 814686 - Extra space above the PDF documents
- bug 814707 - Page doesn't fit the window correctly when using "Page fit" zoom option
Graphics
- bug 815118 - Black pixels on Firefox button when hovering over tabs in taskbar
- bug 815602 - Invalid drawable error in console after starting Firefox
- bug 827364 - Fonts are not properly displayed
- bug 827389 - Text from search box does not maintain the font and proportion of other text from the same page
Theme
- bug 821240 - When Background Theme applied, Feedback button is improperly colored
- bug 823560 - Collision between HTML5 videos and personas
Plugins
- bug 821247 - Plugin Check displays that Java 7 Update 9 is outdated even if is up to date
- bug 821304 - [Mac OS] Shockwave games no longer work in FF 18
- bug 823533 - [Mac OS] Java doesn't work in 32-bit mode
- bug 825172 - In full screen - video image freezes and the sound keeps on going if About Adobe Flash Player is selected
- bug 825188 - Sound still plays after closing the video
Shell Integration
- bug 823517 - Internet connection breaks after a crash in Firefox
Session Restore
- bug 827374 - Menu Bar is not restored after Firefox crashes
Known Issues
Bold any potential blockers.
Themes
- bug 616671 - Graphical glitch at the bottom of tabs when using darker Personas with tabs on top
HTML5 Video
- bug 806095 - [10.8] Using JavaScript fullscreen API (mozRequestFullScreen) leaves part of the upper UI visible.
- bug 708557 - Html5 youtube context menu not persistent when right clicking
- bug 747909 - In Full Screen mode - copy actions are not allowed on HTML5 videos
Session Store
- bug 791195 - Session is not restored after Firefox crash
WebGL
- bug 787056 - MapsGL maps.google.com renders blurry on machines Chrome doesn't allow MapsGL enabling on