Releases/Firefox 12/Test Plan
« Firefox 11 | Firefox 12 | Firefox 13 »
Contents
Firefox 12 Test Plan
Summary
Milestone | Firefox 12 |
Status | Currently in Beta -- Moves to Release on April 24, 2012 |
Lead | Juan Becerra (irc: juanb) |
2nd | Anthony Hughes (irc: ashughes) |
The following is the test plan for Firefox 12 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 12. After the release this document will be used as an archive of what was done to validate this release.
If you have some free time, please pick a task below and get in touch with one of the leads.
Details
The following is a description of the tasks which need to be completed throughout this release.
Task | Owner | Status |
Automation Triage | Softvision | |
Bug Triage | Softvision | |
Feature Sign-offs | Softvision | |
Full Functional Tests (Aurora) | iBeta | |
Website Compatibility (Aurora) | uTest | |
Milestones | ||
Aurora Merge | ashughes | |
Nightly Merge | ashughes |
Automation
BFTs
Task | Triage the Mozmill dashboard for test failures and regressions |
Lead | Vlad Maniac (Softvision) |
Peer | Alex Lakatos (Softvision) |
Status | NEW (13), ASSIGNED (15), FIXED (3), REGRESSIONS (0) |
Instructions | |
| |
Bugs Found | |
|
Endurance
Task | Triage the Mozmill Endurance dashboard for test failures and performance regressions |
Lead | Mihaela Velimiroviciu (Softvision) |
Peer | |
Status | Spreadsheet (Nightly) |
Instructions | |
| |
Bugs Found | |
|
Bug Triage
Lead | Vlad Ghetiu (Softvision) |
Peers | Virgil Dicu, Ioana Budnar, Mihaela Velimiroviciu, Simona Badau, Paul Silaghi (Softvision) |
Queries | QA+, FIXED, UNCONFIRMED |
Bugs Found | |
|
Features
See the Feature Release Tracking wiki for a general overview
- [DONE]
- [ON TRACK]
- Silent Update: Removal of OS security dialog for Windows (Simona Badau)
- Media Controls Refresh (Mihaela Velimiroviciu)
- New Tab Page (Virgil Dicu)- Retargeted for Firefox 13, but available on Firefox 12 pref'ed off.
- Console Queued Message Display (Vlad Maniac)- Retargeted for Firefox 13 release, but available in Firefox 12.
- [AT RISK]
- [MISSED]
- Inline URL autocomplete (Vlad Ghetiu) - Retargeted for Firefox 13
- Firefox Home Tab (Vlad Ghetiu)- Retargeted for Firefox 13
- Silent Update: Updated workflow for users with incompatible add-ons (Ioana Budnar)- Retargeted for Firefox 13
- Web Apps Integration to Desktop (Virgil Dicu)- Retargeted for Firefox 13
- Panel-based Download Manager (Simona Badau)- Retargeted for Firefox 13
- Silent Update: Background updates (Vlad Ghetiu)- Retargeted for Firefox 13
Full Functional Tests
The following tracks testing done by iBeta for full functional test coverage. This task is performed once during Aurora only.
- Lead: Juan Becerra
- [ link to test suite]
- [ link to results of testing]
- [ link to bugs filed by iBeta]
Website Compatibility
The following tracks testing done by uTest for website compatibility. This task is performed after every merge.
- Lead: Juan Becerra
- Link to test suite
- Link to report for Nightly Web Compatibility
- Link to report for Aurora Web Compatibility
- Link to report for Beta Web Compatibility
- Link to bugs filed
Nightly Merge
Verify that the builds generated from the merge to Nightly have updated properly.
- Nightly
- [ON TRACK] [ashughes] Windows XP:
- [ON TRACK] [ashughes] Windows 7:
- [ON TRACK] [ashughes] Mac OS X 10.6:
- [ON TRACK] [ashughes] Ubuntu 11.04:
- Issues Found
Aurora Merge
The following are the detailed results of the automated BFT test runs.
Platform | Result: report |
Windows 2000 | PASS - 2012-02-02 |
Windows XP | PASS - 2012-02-02 |
Windows Vista | PASS - 2012-02-02 |
Windows 7 32-bit | PASS - 2012-02-02 |
Windows 7 64-bit | PASS - 2012-02-02 |
Mac OS X 10.6 | PASS - 2012-02-02 |
Mac OS X 10.7 | PASS - 2012-02-02 |
Ubuntu 11.04 32-bit | PASS - 2012-02-02 |
Ubuntu 11.04 64-bit | PASS - 2012-02-02 |
- Issues Found
Updates
The following are details of update verifications for Firefox 12 merge to Aurora
- Auroratest
- [ON TRACK] [ashughes] Windows XP:
- [ON TRACK] [ashughes] Windows 7:
- [ON TRACK] [ashughes] Mac OS X 10.6:
- [ON TRACK] [ashughes] Ubuntu 11.04:
- Aurora
- [ON TRACK] [ashughes] Windows XP:
- [ON TRACK] [ashughes] Windows 7:
- [ON TRACK] [ashughes] Mac OS X 10.6:
- [ON TRACK] [ashughes] Ubuntu 11.04:
- Issues Found
Manual Spot Checks and Feature Sign-Off
Are all features implemented at this point working?
Manual spot Checks [spreadsheet]
- Platforms: Windows XP, Windows 7 x64, Ubuntu 11.10, Mac 10.6
- Found known issues:
- New issues:
- Status:
Beta 1
Builds: HTTPS
Litmus: testrun, report
- Checklist
- [DONE] Functional automated tests, synced to Litmus
- [DONE] Manual tests
- [ON TRACK] Bug fix verifications (10%)
- [ON TRACK] Check with Support for top issues from previous Beta/Release
- [DONE] Check with Marcia for top-crashers spiking
- [ON TRACK] Update tests on releasetest
- [ON TRACK] Get final sign-off from Bob Moss
- [ON TRACK] Update tests on beta
Automated Tests
The following are the detailed results of the automated BFT test runs.
Platform | Assignee | Result: report |
Windows 2000 | ashughes | PASS - 2012-03-14 |
Windows XP | ashughes | PASS - 2012-03-14 |
Windows Vista | ashughes | PASS - 2012-03-14 |
Windows 7 32-bit | ashughes | PASS - 2012-03-14 |
Windows 7 64-bit | ashughes | PASS - 2012-03-14 |
Mac OS X 10.6 | ashughes | PASS - 2012-03-14 |
Mac OS X 10.7 | ashughes | PASS - 2012-03-14 |
Ubuntu 11.10 32-bit | ashughes | PASS - 2012-03-14 |
Ubuntu 11.10 64-bit | ashughes | PASS - 2012-03-14 |
- Test Failures
- testTabbedBrowsing/testBackgroundTabScrolling - Right scroll arrow has been highlighted in testScrollBackgroundTabIntoView
Manual Tests
The following are the detailed results of the manual testing.
- Smoketests
Run the manual smoketests and report results below.
Platform | Assignee | Result: report |
Windows 2000 | Simona | PASS - 2012-03-15 |
Windows XP | Ioana | PASS - 2012-03-15 |
Windows Vista | Simona | PASS - 2012-03-15 |
Windows 7 32-bit | Mihaela | PASS - 2012-03-15 |
Windows 7 64-bit | Paul | PASS - 2012-03-15 |
Mac OS X 10.6.8 | VladG | PASS - 2012-03-15 |
Mac OS X 10.7.2 | VladG | PASS - 2012-03-15 |
Ubuntu 11.10 32-bit | Ioana | PASS - 2012-03-15 |
Ubuntu 11.10 64-bit | Virgil | PASS - 2012-03-15 |
- Focus Testing
The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc
Platform | Assignee | Result: report |
Windows 7 32-bit | Ioana, Mihaela | PASS - 2012-03-16 |
Mac OS X 10.6.8 | VladG | PASS - 2012-03-15 |
Ubuntu 11.10 64-bit | Virgil | PASS - 2012-03-15 |
Windows XP | Simona, Paul | PASS - 2012-03-15 |
- [DONE] Rich Profile Updates Etherpad
- Feature Sign-off Testing
The following are the results of feature testing for those features which are shipping. Use the Firefox 12.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.
Feature | Owner | Win32 | Mac | Linux |
Media Controls Refresh | Mihaela Velimiroviciu | PASS - 2012-03-15 | PASS - 2012-03-15 | PASS - 2012-03-15 |
Silent Update: Removal of OS security dialog for Windows | Simona Badau | PASS WITH KNOWN ISSUES - 2012-03-16 bug 735177 | NA - 2012-03-15 | NA - 2012-03-15 |
Update Tests
The following are details of update verifications
Platform | Releasetest: report | Beta: report |
Windows 2000 | PASS - 2012-03-15 | PASS - 2012-03-16 |
Windows XP | PASS - 2012-03-15 | PASS - 2012-03-16 |
Windows Vista | PASS - 2012-03-15 | PASS - 2012-03-16 |
Windows 7 32-bit | PASS - 2012-03-15 | PASS - 2012-03-16 |
Windos 7 64-bit | PASS - 2012-03-15 | PASS - 2012-03-16 |
Mac OS X 10.6 | PASS - 2012-03-15 | PASS - 2012-03-16 |
Mac OS 10.7 | PASS - 2012-03-15 | PASS - 2012-03-16 |
Ubuntu 11.10 32-bit | PASS - 2012-03-15 | PASS - 2012-03-16 |
Ubuntu 11.10 64-bit | PASS - 2012-03-15 | PASS - 2012-03-16 |
- Issues
Bug Fix Verifications
The following are the detailed results of verifications of bug fixes landed in Beta.
- Any fix you very should have the status-firefox12 flag set to verified
- If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
- Queries
- QA+ - used by testers to triage and verify fixes
- UNTRIAGED: X, VERIFIED: X
- Softvision verifications etherpad
- QA+ SG - used by the Security team to triage and verify security fixes
- UNTRIAGED: X, VERIFIED: X
- status-firefox12:fixed - used by QA lead to triage for potential QA+ bugs
- QA?: X, QA+: X, QA-: X
Issues Found
Bug | Status |
bug 736021 - The "list all tabs" button is missing in Firefox 12 beta 1 | NEW |
bug 734837 - Broken history when navigating to and from cnn.com | NEW |
bug 735243 - On HTML5 videos Esc key exits from both full screen and Highlighter | NEW |
bug 725614 - Shortcut keys fail for pages containing Java/Shockwave/PDF | NEW |
bug 729872 - Window deactivate while in HTML5 fullscreen causes text fields to not show cursor or selection | Fixed on FF13 |
Beta 2
Builds: HTTPS
Litmus: testrun, report
- Checklist
- [DONE] Functional automated tests, synced to Litmus
- [DONE] Manual tests
- [DONE] Bug fix verifications (27%)
- [DONE] Check with Support for top issues from previous Beta/Release
- [DONE] Check with Marcia for top-crashers spiking
- [DONE] Update tests on releasetest
- [ON TRACK] Get final sign-off from Bob Moss
- [ON TRACK] Update tests on beta
Automated Tests
The following are the detailed results of the automated BFT test runs.
Platform | Assignee | Result: report |
Windows 2000 | juanb | PASS - 2012-03-20 |
Windows XP | juanb | PASS - 2012-03-20 |
Windows Vista | juanb | PASS - 2012-03-20 |
Windows 7 32-bit | juanb | PASS - 2012-03-20 |
Windows 7 64-bit | juanb | PASS - 2012-03-20 |
Mac OS X 10.6 | juanb | PASS - 2012-03-20 |
Mac OS X 10.7 | juanb | PASS - 2012-03-20 |
Ubuntu 11.10 32-bit | juanb | PASS - 2012-03-20 |
Ubuntu 11.10 64-bit | juanb | PASS - 2012-03-20 |
- Test Failures
- Mac OS X 10.6/10.7 - /testSecurity/testIdentityPopupOpenClose.js - Identity popup has been closed
- Failure to do the test, not a functional problem - Manual verification showed it to be working
Manual Tests
The following are the detailed results of the manual testing.
- Smoketests
Run the manual smoketests and report results below.
Platform | Assignee | Result: report |
Windows 2000 | Simona | PASS - 2012-03-21 |
Windows XP | Paul | PASS - 2012-03-22 |
Windows Vista | Simona | PASS - 2012-03-21 |
Windows 7 32-bit | Mihaela | PASS - 2012-03-22 |
Windows 7 64-bit | Paul | PASS - 2012-03-21 |
Mac OS X 10.6.8 | VladG | PASS - 2012-03-22 |
Mac OS X 10.7.3 | VladG | PASS - 2012-03-22 |
Ubuntu 11.10 32-bit | Ioana | PASS - 2012-03-21 |
Ubuntu 11.10 64-bit | Ioana | PASS - 2012-03-21 |
- Focus Testing
The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc
Platform | Assignee | Result: report |
Windows 7 32-bit | Mihaela | PASS - 2012-03-22 |
Mac OS X 10.6.8 | VladG | PASS - 2012-03-22 |
Ubuntu 11.10 32-bit | Ioana | PASS - 2012-03-21 |
Windows XP | Paul | PASS - 2012-03-22 |
- [DONE] Rich Profile Updates Etherpad
- Feature Sign-off Testing
The following are the results of feature testing for those features which are shipping. Use the Firefox 12.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.
Feature | Owner | Win32 | Mac | Linux |
Media Controls Refresh | Mihaela Velimiroviciu | PASS - 2012-03-22 | PASS - 2012-03-22 | PASS - 2012-03-22 |
Silent Update: Removal of OS security dialog for Windows | Simona Badau | PASS - 2012-03-22 | N/A - 2012-03-22 | N/A - 2012-03-22 |
Update Tests
The following are details of update verifications
Platform | Releasetest: report | Beta: report |
Windows 2000 | PASS - 2012-03-20 | PASS - 2012-03-22 |
Windows XP | PASS - 2012-03-20 | PASS - 2012-03-22 |
Windows Vista | PASS - 2012-03-20 | PASS - 2012-03-22 |
Windows 7 32-bit | PASS - 2012-03-20 | PASS - 2012-03-22 |
Windos 7 64-bit | PASS - 2012-03-20 | PASS - 2012-03-22 |
Mac OS X 10.6 | PASS - 2012-03-20 | PASS - 2012-03-22 |
Mac OS 10.7 | PASS - 2012-03-20 | PASS - 2012-03-22 |
Ubuntu 11.10 32-bit | PASS - 2012-03-20 | PASS - 2012-03-22 |
Ubuntu 11.10 64-bit | PASS - 2012-03-20 | PASS - 2012-03-22 |
- Issues
Bug Fix Verifications
The following are the detailed results of verifications of bug fixes landed in Beta.
- Any fix you very should have the status-firefox12 flag set to verified
- If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
- Queries
- QA+ - used by testers to triage and verify fixes
- UNTRIAGED: X, VERIFIED: X
- Softvision verifications etherpad
- QA+ SG - used by the Security team to triage and verify security fixes
- UNTRIAGED: X, VERIFIED: X
- status-firefox12:fixed - used by QA lead to triage for potential QA+ bugs
- QA?: X, QA+: X, QA-: X
QAWANTED
Bugs: query
Bug | Status |
bug 714320 Firefox Crash @ nsStyleContext::AddChild with AMD Radeon HD 6xxx series | Engineering investigating (2012-03-26) |
bug 738661 crash in nmsvc from CovenantEyes | Marcia investigating (2012-03-26) |
bug 738392 Plugin doesn't render inside a CSS transform on Mac | Not reproducible (2012-03-26) |
bug 725503 Crash in radhslib.dll from Naomi Internet Filter | What more can QA do? (2012-03-26) |
bug 722538 block AMD Radeon HD 6290/6300/6310/6320 for DIRECT2D | What more can QA do? (2012-03-26) |
bug 720987 Focus rings keep growing when repeatedly tabbing through elements | Unclear what's needed (2012-03-26) |
Issues Found
Bug | Status |
bug 736021 - The "list all tabs" button is missing in Firefox 12 beta | NEW |
bug 734837 - Broken history when navigating to and from cnn.com | NEW |
bug 735243 - On HTML5 videos Esc key exits from both full screen and Highlighter | NEW |
bug 725614 - Shortcut keys fail for pages containing Java/Shockwave/PDF | NEW |
bug 729872 - Window deactivate while in HTML5 fullscreen causes text fields to not show cursor or selection | Fixed on FF13 |
bug 725296 - Browser is unresponsive while Silverlight content is loading | NEW |
bug 731981 - Missing steps for missing Shockwave plugin notification | NEW |
bug 709029 - The RealPlayer plugin crashes when trying to play a RealPlayer video | NEW |
Feedback
If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.
Builds can be found here: HTTPS
You can post your feedback below or in this etherpad.
Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.
Beta 3
Builds: HTTPS
Litmus: testrun, report
- Checklist
- [DONE] Functional automated tests, synced to Litmus
- [DONE] Manual tests
- [DONE] Bug fix verifications
- [DONE] Check with Marcia for top-crashers spiking
- [DONE] Update tests on releasetest
- [DONE] Get final sign-off from Bob Moss
- [DONE] Update tests on beta
Automated Tests
The following are the detailed results of the automated BFT test runs.
Platform | Assignee | Result: report |
Windows 2000 | jsmith | PASS - 2012-03-28 |
Windows XP | jsmith | PASS - 2012-03-28 |
Windows Vista | jsmith | PASS - 2012-03-28 |
Windows 7 32-bit | jsmith | PASS - 2012-03-28 |
Windows 7 64-bit | jsmith | PASS - 2012-03-28 |
Mac OS X 10.6 | jsmith | PASS - 2012-03-28 |
Mac OS X 10.7 | jsmith | PASS - 2012-03-28 |
Ubuntu 11.10 32-bit | jsmith | PASS - 2012-03-28 |
Ubuntu 11.10 64-bit | jsmith | PASS - 2012-03-28 |
- Test Failures
- /testAwesomeBar/testSuggestHistoryBookmarks.js - controller.assertJS: Failed for 'subject.enteredTitle == subjective.underlinedTitle'
Manual Tests
The following are the detailed results of the manual testing.
- Smoketests
Run the manual smoketests and report results below.
Platform | Assignee | Result: report |
Windows 2000 | Simona | PASS - 2012-03-29 |
Windows XP | Ioana | PASS - 2012-03-29 |
Windows Vista | Simona | PASS - 2012-03-29 |
Windows 7 32-bit | Paul | PASS - 2012-03-29 |
Windows 7 64-bit | Paul | PASS - 2012-03-29 |
Mac OS X 10.6.8 | VladG | PASS - 2012-03-29 |
Mac OS X 10.7.3 | VladG | PASS - 2012-03-29 |
Ubuntu 11.10 32-bit | Virgil | PASS - 2012-03-29 |
Ubuntu 11.10 64-bit | Ioana | PASS - 2012-03-29 |
- Focus Testing
The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc
Platform | Assignee | Result: report |
Windows 7 32-bit | Simona | PASS - 2012-03-29 |
Mac OS X 10.6.8 | VladG | PASS - 2012-03-29 |
Ubuntu 11.10 32-bit | Virgil | PASS - 2012-03-29 |
Windows XP | Ioana | PASS - 2012-03-29 |
- [DONE] Rich Profile Updates Etherpad
- Feature Sign-off Testing
The following are the results of feature testing for those features which are shipping. Use the Firefox 12.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.
Feature | Owner | Win32 | Mac | Linux |
Media Controls Refresh | Mihaela Velimiroviciu | PASS - 2012-03-29 | PASS - 2012-03-30 | PASS - 2012-03-30 |
Silent Update: Removal of OS security dialog for Windows | Simona Badau | PASS - 2012-03-30 | NA - 2012-03-29 | NA - 2012-03-29 |
Update Tests
The following are details of update verifications
Platform | Releasetest: report | Beta: report |
Windows 2000 | PASS - 2012-03-29 | PASS - 2012-03-30 |
Windows XP | PASS - 2012-03-29 | PASS - 2012-03-30 |
Windows Vista | PASS - 2012-03-29 | PASS - 2012-03-30 |
Windows 7 32-bit | PASS - 2012-03-29 | PASS - 2012-03-30 |
Windos 7 64-bit | PASS - 2012-03-29 | PASS - 2012-03-30 |
Mac OS X 10.6 | PASS - 2012-03-29 | PASS - 2012-03-30 |
Mac OS 10.7 | PASS - 2012-03-29 | PASS - 2012-03-30 |
Ubuntu 11.10 32-bit | PASS - 2012-03-29 | PASS - 2012-03-30 |
Ubuntu 11.10 64-bit | PASS - 2012-03-29 | PASS - 2012-03-30 |
- Issues
Bug Fix Verifications
The following are the detailed results of verifications of bug fixes landed in Beta.
- Any fix you very should have the status-firefox12 flag set to verified
- If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
- Queries
- Functional fixes to verify: QA+ (4 / 57)
- Security fixes to verify: QA+ SG (0 / 19)
- Softvision verifications etherpad
- Fixes landed for Firefox 12: status-firefox12:fixed
QAWANTED
Bugs: query
- Need Follow-up
ID | Summary | Priority | Status |
---|---|---|---|
714320 | Firefox Crash @ nsStyleContext::AddChild with AMD Radeon HD 6xxx series | -- | RESOLVED |
720987 | Focus rings keep growing when repeatedly tabbing through elements | -- | RESOLVED |
738392 | Plugin doesn't render inside a CSS transform on Mac | -- | VERIFIED |
3 Total; 0 Open (0%); 2 Resolved (66.67%); 1 Verified (33.33%);
- QA Investigating
ID | Summary | Priority | Status |
---|---|---|---|
722538 | block AMD Radeon HD 6290/6300/6310/6320 for DIRECT2D | -- | RESOLVED |
738661 | crash in nmsvc from CovenantEyes | P3 | RESOLVED |
2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);
Issues Found
10 Total; 0 Open (0%); 9 Resolved (90%); 1 Verified (10%);
Feedback
If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.
Builds can be found here: HTTPS
You can post your feedback below or in this etherpad.
Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.
Beta 4
Builds: HTTPS
Litmus: testrun, report
- Checklist
- [DONE] Functional automated tests, synced to Litmus
- [DONE] Manual tests
- [ON TRACK] Bug fix verifications
- [DONE] Check with Marcia for top-crashers spiking
- [DONE] Update tests on releasetest
- [DONE] Get final sign-off from Bob Moss
- {{ |}} Update tests on beta
Automated Tests
The following are the detailed results of the automated BFT test runs.
Platform | Assignee | Result: report |
Windows 2000 | jsmith | PASS - 2012-04-04 |
Windows XP | jsmith | PASS - 2012-04-04 |
Windows Vista | jsmith | PASS - 2012-04-04 |
Windows 7 32-bit | jsmith | PASS - 2012-04-04 |
Windows 7 64-bit | jsmith | PASS - 2012-04-04 |
Mac OS X 10.6 | jsmith | PASS - 2012-04-04 |
Mac OS X 10.7 | jsmith | PASS - 2012-04-04 |
Ubuntu 11.10 32-bit | jsmith | PASS - 2012-04-04 |
Ubuntu 11.10 64-bit | jsmith | PASS - 2012-04-04 |
- Test Failures
- testSuggestHistoryBookmarks - controller.assertJS: Failed for 'subject.enteredTitle == subject.underlinedTitle'
Manual Tests
The following are the detailed results of the manual testing.
- Smoketests
Run the manual smoketests and report results below.
Platform | Assignee | Result: report |
Windows 2000 | Simona | PASS - 2012-04-04 |
Windows XP | Ioana | PASS - 2012-04-04 |
Windows Vista | Simona | PASS - 2012-04-04 |
Windows 7 32-bit | Mihaela | PASS - 2012-04-04 |
Windows 7 64-bit | Paul | PASS - 2012-04-04 |
Mac OS X 10.6.8 | VladG | PASS - 2012-04-04 |
Mac OS X 10.7.3 | VladG | PASS - 2012-04-04 |
Ubuntu 11.04 32-bit | Mihaela | PASS - 2012-04-04 |
Ubuntu 11.10 64-bit | Ioana | PASS - 2012-04-04 |
- Regression Testing
The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc
Platform | Assignee | Result: report |
Windows 7 32-bit | Paul | PASS - 2012-04-04 |
Mac OS X 10.6.8 | Vlad | PASS - 2012-04-04 |
Ubuntu 11.10 32-bit | Virgil | PASS - 2012-04-04 |
Windows XP | Ioana | PASS - 2012-04-04 |
- Focus Testing
Test the following across as many platforms as possible (at least one Windows, one Mac, and one Linux)
- [DONE] Bookmarks - verify full functional bookmarks functionality before and after an update (things like import, backup, restore, rearranging, adding, deleting, folders, feeds, toolbars, etc) - Etherpad
- Feature Sign-off Testing
The following are the results of feature testing for those features which are shipping. Use the Firefox 12.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.
Feature | Owner | Win32 | Mac | Linux |
Media Controls Refresh | Mihaela Velimiroviciu | PASS - 2012-04-04 | PASS - 2012-04-05 | PASS - 2012-04-04 |
Silent Update: UAC Service | Simona Badau | PASS - 2012-04-05 | N/A | N/A |
Update Tests
The following are details of update verifications
Platform | Releasetest: report | Beta: report |
Windows 2000 | PASS - 2012-04-04 | PASS - 2012-04-05 |
Windows XP | PASS - 2012-04-04 | PASS - 2012-04-05 |
Windows Vista | PASS - 2012-04-04 | PASS - 2012-04-05 |
Windows 7 32-bit | PASS - 2012-04-04 | PASS - 2012-04-05 |
Windows 7 64-bit | PASS - 2012-04-04 | PASS - 2012-04-05 |
Mac OS X 10.6 | PASS - 2012-04-04 | PASS - 2012-04-05 |
Mac OS 10.7 | PASS - 2012-04-04 | PASS - 2012-04-05 |
Ubuntu 11.10 32-bit | PASS - 2012-04-04 | PASS - 2012-04-05 |
Ubuntu 11.10 64-bit | PASS - 2012-04-04 | PASS - 2012-04-05 |
- Issues
Bug Fix Verifications
The following are the detailed results of verifications of bug fixes landed in Beta.
- Any fix you very should have the status-firefox12 flag set to verified
- If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
- Queries
- Functional fixes to verify: QA+ (11 / 38)
- Security fixes to verify: QA+ SG (2 / 20)
- Softvision verifications etherpad
- Fixes landed for Firefox 12: status-firefox12:fixed
QAWANTED
Bugs: query
- Need Follow-up
ID | Summary | Priority | Status |
---|---|---|---|
741179 | crash in nsDiskCacheBlockFile::Write | -- | RESOLVED |
1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);
- QA Investigating
ID | Summary | Priority | Status |
---|---|---|---|
722538 | block AMD Radeon HD 6290/6300/6310/6320 for DIRECT2D | -- | RESOLVED |
738661 | crash in nmsvc from CovenantEyes | P3 | RESOLVED |
742723 | Empty bookmarks toolbar on startup due to All-in-one sidebar add-on | -- | RESOLVED |
3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);
Known Issues
11 Total; 0 Open (0%); 10 Resolved (90.91%); 1 Verified (9.09%);
Issues Found
ID | Summary | Priority | Status |
---|---|---|---|
742363 | Crash when restarting Firefox after add-on installation and pdf document opened with Adobe Reader 9.4 | -- | RESOLVED |
742745 | The service is not uninstalled with the last build if the installation directory is deleted | -- | RESOLVED |
2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);
Feedback
If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.
Builds can be found here: HTTPS
You can post your feedback below or in this etherpad.
Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.
Beta 5
Builds: HTTPS
Litmus: testrun, report
- Checklist
- [ON TRACK] Functional automated tests, synced to Litmus
- [DONE] Manual tests
- [ON TRACK] Bug fix verifications (10%)
- [ON TRACK] Check with Support for top issues from previous Beta/Release
- [ON TRACK] Check with Marcia for top-crashers spiking
- [ON TRACK] Update tests on releasetest
- [ON TRACK] Get final sign-off from Bob Moss
- [ON TRACK] Update tests on beta
Automated Tests
The following are the detailed results of the automated BFT test runs.
Platform | Assignee | Result: report |
Windows 2000 | ashughes | PASS - 2012-04-11 |
Windows XP | ashughes | PASS - 2012-04-11 |
Windows Vista | ashughes | PASS - 2012-04-11 |
Windows 7 32-bit | ashughes | PASS - 2012-04-11 |
Windows 7 64-bit | ashughes | PASS - 2012-04-11 |
Mac OS X 10.6 | ashughes | PASS - 2012-04-11 |
Mac OS X 10.7 | ashughes | PASS - 2012-04-11 |
Ubuntu 11.10 32-bit | ashughes | PASS - 2012-04-11 |
Ubuntu 11.10 64-bit | ashughes | PASS - 2012-04-11 |
- Test Failures
Manual Tests
The following are the detailed results of the manual testing.
- Smoketests
Run the manual smoketests and report results below.
Platform | Assignee | Result |
Windows 2000 | Simona | PASS - 2012-04-12 |
Windows XP | Paul | PASS - 2012-04-12 |
Windows Vista | Simona | PASS - 2012-04-12 |
Windows 7 32-bit | Mihaela | PASS - 2012-04-12 |
Windows 7 64-bit | Paul | PASS - 2012-04-12 |
Mac OS X 10.6.8 | VladG | PASS - 2012-04-12 |
Mac OS X 10.7.2 | VladG | PASS - 2012-04-12 |
Ubuntu 11.10 32-bit | Virgil | PASS - 2012-04-12 |
Ubuntu 11.10 64-bit | Virgil | PASS - 2012-04-12 |
Results in Litmus
- Focus Testing
The following are the detailed results of the focused feature testing. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc
Platform | Assignee | Result: report |
Windows 7 32-bit | Mihaela | PASS - 2012-04-12 |
Mac OS X 10.6.8 | Vlad | PASS - 2012-04-12 |
Ubuntu 11.10 32-bit | Virgil | PASS - 2012-04-12 |
Windows XP | Paul | PASS - 2012-04-12 |
- [DONE] Rich Profile Updates Etherpad
- Feature Sign-off Testing
The following are the results of feature testing for those features which are shipping. Use the Firefox 12.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.
Feature | Owner | Win32 | Mac | Linux |
Media Controls Refresh | Mihaela Velimiroviciu | PASS - 2012-04-13 | PASS - 2012-04-13 | PASS - 2012-04-13 |
Silent Update: UAC Service | Simona Badau | PASS - 2012-04-13 | N/A | N/A |
Update Tests
The following are details of update verifications
Platform | Releasetest: report | Beta: [ report] |
Windows 2000 | PASS - 2012-04-12 | RESULTS - 2012-04-13 |
Windows XP | PASS - 2012-04-12 | RESULTS - 2012-04-13 |
Windows Vista | PASS - 2012-04-12 | RESULTS - 2012-04-13 |
Windows 7 32-bit | PASS - 2012-04-12 | RESULTS - 2012-04-13 |
Windows 7 64-bit | PASS - 2012-04-12 | RESULTS - 2012-04-13 |
Mac OS X 10.6 | PASS - 2012-04-12 | RESULTS - 2012-04-13 |
Mac OS 10.7 | PASS - 2012-04-12 | RESULTS - 2012-04-13 |
Ubuntu 11.10 32-bit | PASS - 2012-04-12 | RESULTS - 2012-04-13 |
Ubuntu 11.10 64-bit | PASS - 2012-04-12 | RESULTS - 2012-04-13 |
- Issues
Bug Fix Verifications
The following are the detailed results of verifications of bug fixes landed in Beta.
- Any fix you very should have the status-firefox12 flag set to verified
- If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
- Queries
- QA+ - used by testers to triage and verify fixes
- UNTRIAGED: X, VERIFIED: X
- Softvision verifications etherpad
- QA+ SG - used by the Security team to triage and verify security fixes
- UNTRIAGED: X, VERIFIED: X
- status-firefox12:fixed - used by QA lead to triage for potential QA+ bugs
- QA?: X, QA+: X, QA-: X
QAWANTED
Bugs: query
- Need Follow-up
ID | Summary | Priority | Status |
---|---|---|---|
741179 | crash in nsDiskCacheBlockFile::Write | -- | RESOLVED |
1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);
- QA Investigating
ID | Summary | Priority | Status |
---|---|---|---|
722538 | block AMD Radeon HD 6290/6300/6310/6320 for DIRECT2D | -- | RESOLVED |
738661 | crash in nmsvc from CovenantEyes | P3 | RESOLVED |
742723 | Empty bookmarks toolbar on startup due to All-in-one sidebar add-on | -- | RESOLVED |
3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);
Issues Found
Bug | Status |
Known Issues
3 Total; 0 Open (0%); 2 Resolved (66.67%); 1 Verified (33.33%);
Feedback
If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.
Builds can be found here: HTTPS
You can post your feedback below or in this etherpad.
Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.
Beta 6
Builds: HTTPS
Litmus: testrun, report
- Checklist
- [DONE] Functional automated tests, synced to Litmus
- [DONE] Manual tests
- [ON TRACK] Bug fix verifications (10%)
- [DONE] Check with Support for top issues from previous Beta/Release
- [DONE] Check with Marcia for top-crashers spiking
- [DONE] Update tests on releasetest
- [DONE] Get final sign-off from Bob Moss
- [DONE] Update tests on beta
Automated Tests
The following are the detailed results of the automated BFT test runs.
Platform | Assignee | Result: report |
Windows 2000 | ashughes | PASS - 2012-04-18 |
Windows XP | ashughes | PASS - 2012-04-18 |
Windows Vista | ashughes | PASS - 2012-04-18 |
Windows 7 32-bit | ashughes | PASS - 2012-04-18 |
Windows 7 64-bit | ashughes | PASS - 2012-04-18 |
Mac OS X 10.6 | ashughes | PASS - 2012-04-18 |
Mac OS X 10.7 | ashughes | PASS - 2012-04-18 |
Ubuntu 11.10 32-bit | ashughes | PASS - 2012-04-18 |
Ubuntu 11.10 64-bit | ashughes | PASS - 2012-04-18 |
- Test Failures
- Ubuntu 11.1 32-bit /testAwesomeBar/testSuggestHistoryBookmarks - controller.assertJS: Failed for subject.enteredTitle == subject.underlinedTitle'
- OS X 10.7 64-bit: /testManagerKeyboardShortcut.js - Selected category has been loaded
Manual Tests
The following are the detailed results of the manual testing.
- Smoketests
Run the manual smoketests and report results below.
Platform | Assignee | Result |
Windows 2000 | Simona | PASS - 2012-04-18 |
Windows XP | Ioana | PASS - 2012-04-18 |
Windows Vista | Simona | PASS - 2012-04-18 |
Windows 7 32-bit | Mihaela | PASS - 2012-04-18 |
Windows 7 64-bit | Mihaela | PASS - 2012-04-18 |
Mac OS X 10.6.8 | VladG | PASS - 2012-04-18 |
Mac OS X 10.7.3 | VladG | PASS - 2012-04-18 |
Ubuntu 11.10 32-bit | Virgil | PASS - 2012-04-18 |
Ubuntu 11.10 64-bit | Virgil | PASS - 2012-04-18 |
Results in Litmus
- Focus Testing
The following are the detailed results of the focused feature testing. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc
Platform | Assignee | Result: report |
Windows 7 32-bit | Mihaela | PASS - 2012-04-18 |
Mac OS X 10.6.8 | Vlad | PASS - 2012-04-18 |
Ubuntu 11.10 32-bit | Virgil | PASS - 2012-04-18 |
Windows XP | Simona | PASS - 2012-04-18 |
- [DONE] Rich Profile Updates Etherpad
- Feature Sign-off Testing
The following are the results of feature testing for those features which are shipping. Use the Firefox 12.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.
Feature | Owner | Win32 | Mac | Linux |
Media Controls Refresh | Mihaela Velimiroviciu | PASS - 2012-04-19 | PASS - 2012-04-19 | PASS - 2012-04-19 |
Silent Update: UAC Service | Simona Badau | PASS - 2012-04-19 | N/A | N/A |
Update Tests
The following are details of update verifications
Platform | Releasetest: report | Beta: report |
Windows 2000 | PASS - 2012-04-18 | PASS - 2012-04-19 |
Windows XP | PASS - 2012-04-18 | PASS - 2012-04-19 |
Windows Vista | PASS - 2012-04-18 | PASS - 2012-04-19 |
Windows 7 32-bit | PASS - 2012-04-18 | PASS - 2012-04-19 |
Windows 7 64-bit | PASS - 2012-04-18 | PASS - 2012-04-19 |
Mac OS X 10.6 | PASS - 2012-04-18 | PASS - 2012-04-19 |
Mac OS 10.7 | PASS - 2012-04-18 | PASS - 2012-04-19 |
Ubuntu 11.10 32-bit | PASS - 2012-04-18 | PASS - 2012-04-19 |
Ubuntu 11.10 64-bit | PASS - 2012-04-18 | PASS - 2012-04-19 |
- Issues
Bug Fix Verifications
The following are the detailed results of verifications of bug fixes landed in Beta.
- Any fix you very should have the status-firefox12 flag set to verified
- If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
- Queries
- QA+ - used by testers to triage and verify fixes
- UNTRIAGED: X, VERIFIED: X
- Softvision verifications etherpad
- QA+ SG - used by the Security team to triage and verify security fixes
- UNTRIAGED: X, VERIFIED: X
- status-firefox12:fixed - used by QA lead to triage for potential QA+ bugs
- QA?: X, QA+: X, QA-: X
QAWANTED
Bugs: query
- QA Investigating
ID | Summary | Priority | Status |
---|---|---|---|
722538 | block AMD Radeon HD 6290/6300/6310/6320 for DIRECT2D | -- | RESOLVED |
738661 | crash in nmsvc from CovenantEyes | P3 | RESOLVED |
742723 | Empty bookmarks toolbar on startup due to All-in-one sidebar add-on | -- | RESOLVED |
3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);
Issues Found
Bug | Status |
Known Issues
3 Total; 0 Open (0%); 2 Resolved (66.67%); 1 Verified (33.33%);
Feedback
If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.
Builds can be found here: HTTPS
You can post your feedback below or in this etherpad.
Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.
Release Candidate Build 1
Builds: HTTPS
Litmus: testrun, report
Mozmill: [1], updates
Build ID: 20120420145725
- Checklist
- [DONE] Functional automation
- [DONE] Sync automation to Litmus
- [DONE] Manual tests
- {{ |}} Bug fix verifications
- {{ |}} Check with Support for top issues from previous Beta/Release
- {{ |}} Check with Marcia for top-crashers spiking
- [DONE] Update tests on releasetest
- {{ |}} Get final sign-off from Bob Moss
- {{ |}} Update tests on beta
Automated Tests
The following are the detailed results of the automated BFT test runs.
Platform | Assignee | Result: report |
Windows 2000 | ashughes | PASS - 2012-04-21 |
Windows XP | ashughes | PASS - 2012-04-21 |
Windows Vista | ashughes | PASS - 2012-04-21 |
Windows 7 32-bit | ashughes | PASS - 2012-04-21 |
Windows 7 64-bit | ashughes | PASS - 2012-04-21 |
Mac OS X 10.6 | ashughes | PASS - 2012-04-21 |
Mac OS X 10.7 | ashughes | PASS - 2012-04-21 |
Ubuntu 11.10 32-bit | ashughes | PASS - 2012-04-21 |
Ubuntu 11.10 64-bit | ashughes | PASS - 2012-04-21 |
- Test Failures
- ALL search tests regressed: bug 747639
Manual Tests
The following are the detailed results of the manual testing.
- Smoketests
Run the manual smoketests and report results below.
Platform | Assignee | Result |
Windows 2000 | Simona | PASS - 2012-04-23 |
Windows XP | Ioana | PASS - 2012-04-23 |
Windows Vista | Simona | PASS - 2012-04-23 |
Windows 7 32-bit | Mihaela | PASS - 2012-04-23 |
Windows 7 64-bit | Teodosia | PASS - 2012-04-23 |
Mac OS X 10.6.8 | VladG | PASS - 2012-04-23 |
Mac OS X 10.7.3 | Alex | PASS - 2012-04-23 |
Ubuntu 11.10 32-bit | Remus | PASS - 2012-04-23 |
Ubuntu 11.10 64-bit | Ioana | PASS - 2012-04-23 |
Results in Litmus
- Focus Testing
The following are the detailed results of the focused feature testing. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc
Platform | Assignee | Result: report |
Windows 7 32-bit | Mihaela | PASS - 2012-04-23 |
Mac OS X 10.6.8 | VladG | PASS - 2012-04-23 |
Ubuntu 11.10 32-bit | Virgil | PASS - 2012-04-23 |
Windows XP | Ioana | PASS - 2012-04-23 |
- Feature Sign-off Testing
The following are the results of feature testing for those features which are shipping. Use the Firefox 12.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.
Feature | Owner | Win32 | Mac | Linux |
Media Controls Refresh | Mihaela Velimiroviciu | PASS - 2012-04-23 | PASS - 2012-04-23 | PASS - 2012-04-23 |
Silent Update: UAC Service | Simona Badau | PASS - 2012-04-23 | N/A | N/A |
Update Tests
The following are details of update verifications
Platform | Releasetest: report | Beta: report |
Windows 2000 | PASS - 2012-04-22 | PASS - 2012-04-24 |
Windows XP | PASS - 2012-04-22 | PASS - 2012-04-24 |
Windows Vista | PASS - 2012-04-22 | PASS - 2012-04-24 |
Windows 7 32-bit | PASS - 2012-04-22 | PASS - 2012-04-24 |
Windows 7 64-bit | PASS - 2012-04-22 | PASS - 2012-04-24 |
Mac OS X 10.6 | PASS - 2012-04-22 | PASS - 2012-04-24 |
Mac OS 10.7 | PASS - 2012-04-22 | PASS - 2012-04-24 |
Ubuntu 11.10 32-bit | PASS - 2012-04-22 | PASS - 2012-04-24 |
Ubuntu 11.10 64-bit | PASS - 2012-04-22 | PASS - 2012-04-24 |
- Additional Spotchecks
- No What's New page on update - PASS (2012-04-24)
- Issues
3.6.28 Major Update
Platform | Releasetest: report | Release: report |
Windows 2000 | PASS - 2012-04-22 | PASS - 2012-04-24 |
Windows XP | PASS - 2012-04-22 | PASS - 2012-04-24 |
Windows Vista | PASS - 2012-04-22 | PASS - 2012-04-24 |
Windows 7 32-bit | PASS - 2012-04-22 | PASS - 2012-04-24 |
Windows 7 64-bit | PASS - 2012-04-22 | PASS - 2012-04-24 |
Mac OS X 10.6 | PASS - 2012-04-22 | PASS - 2012-04-24 |
Mac OS 10.7 | PASS - 2012-04-22 | PASS - 2012-04-24 |
Ubuntu 11.10 32-bit | PASS - 2012-04-22 | PASS - 2012-04-24 |
Bug Fix Verifications
The following are the detailed results of verifications of bug fixes landed in Beta.
- Any fix you very should have the status-firefox12 flag set to verified
- If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
- Queries
- QA+ - used by testers to triage and verify fixes
- UNTRIAGED: X, VERIFIED: X
- Softvision verifications etherpad
- QA+ SG - used by the Security team to triage and verify security fixes
- UNTRIAGED: X, VERIFIED: X
- status-firefox12:fixed - used by QA lead to triage for potential QA+ bugs
- QA?: X, QA+: X, QA-: X
QAWANTED
Bugs: query
- QA Investigating
ID | Summary | Priority | Status |
---|---|---|---|
722538 | block AMD Radeon HD 6290/6300/6310/6320 for DIRECT2D | -- | RESOLVED |
738661 | crash in nmsvc from CovenantEyes | P3 | RESOLVED |
742723 | Empty bookmarks toolbar on startup due to All-in-one sidebar add-on | -- | RESOLVED |
3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);
Issues Found
ID | Summary | Priority | Status |
---|---|---|---|
747858 | A horizontal line is displayed on the Play button from HTML5 videos | -- | RESOLVED |
747946 | [linux] Slow rendering in Firefox with crazybugs.ivank.net web gl game | -- | RESOLVED |
2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);
Known Issues
5 Total; 0 Open (0%); 4 Resolved (80%); 1 Verified (20%);
Feedback
If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.
Builds can be found here: HTTPS
You can post your feedback below or in this etherpad.
Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.