Releases/Firefox 3.6RC1/Test Plan:FFTs
From MozillaWiki
< Releases
Contents
Firefox 3.6 RC1 FFTs
We are going to run a subset of the FFTs on a combination of Windows XP and Windows 7. We'll prioritize Windows, for now. Get your 3.6 RC1 builds here.
FFT Results (full run)
We'll divide the full functional tests in several choice sections so multiple people can join in and help. Please alternate sections between operating system versions, for example run some on XP and some on Windows 7, some on Mac OS 10.5 and some on 10.6.
- Litmus FFT Testrun
- Date: 01/07 - 01/?
- Build Versions:
- Windows:
- Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100105 Firefox/3.6 (.NET CLR 3.5.30729)
- Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2) Gecko/20100105 Firefox/3.6 (.NET CLR 3.5.30729)
- Mac:
- Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100105 Firefox/3.6
- Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2) Gecko/20100105 Firefox/3.6
- Linux:
- Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2) Gecko/20100105 Firefox/3.6
Assignments
- 1) Section: Installation --> General:
- Windows: retornam Pass
- Mac OS: Tracy - Pass - 10.6
- Linux: Dietrich - Fail
- migration bugs: bug 539133, bug 539139
- find bar sound: bug 539388
- weird geolocation test failure
- Notes:
- 2) Section: Awesome Bar:
- Windows: retornam Pass
- Mac OS: abillings Pass - Aravind Fail
- Context Menu: Undo - bug 540590
- Notes: Test Case 8480 and Test Case 8543 fail.
- Linux: tester ddahl - Aravind Pass
- Notes: Run on Windows 7, found an issue with https://litmus.mozilla.org/show_test.cgi?id=8950 [screencast] http://screencast.com/t/YjBhZjIxYmU
- 3) Section: Tabbed Browsing:
- Windows: abillings - Pass
- Mac OS: Aravind - Pass
- Linux: Aravind - Pass
- Notes:
- 4) Section: Search --> Find in Page:
- Windows: retornam Pass
- Mac OS: Tracy - Pass - 10.4
- Linux: juanb - Ubuntu - Pass
- Notes: Follow pattern above, alternating between XP and Windows 7
- 5) Section: Bookmarks --> History:
- Windows: retornam - PASS
- Mac OS: Aravind - Fail
- Regression Bug : bug 440093
- New Bugs : bug 540595, bug 540593
- Linux: beltzner - Fail
- Notes: Linux
- bug 540474 - drag and drop from Bookmarks Menu to desktop is fragile; not a regression
- bug 540493 - drag and drop doesn't work for History items
- 6) Section: Add-ons Manager --> Lightweight Themes
- Windows: abillings - Pass
- Mac OS: Tracy - Pass - 10.4 - Aravind - Pass
- Linux: beltzner - Fail
- Notes: Windows
- Run on XP
- Notes: Linux
- bug 539862 filed for failure w/drag&drop of theme
- 7) Section: Downloading:
- Windows: gmontagu - Run on XP.PASS: 69; NOT RUN: 25; UNCLEAR: 19; FAIL 1
- Mac OS: shorlander - Fail - 10.6 - Aravind Pass
- Linux: Aravind - Pass
- Notes:
- 8) Section: Printing:
- Windows: tchung - FAIL
- Mac OS: tchung - PASS
- Linux: juanb - Pass (basic printing)
- Notes:
- Windows printing is a mess.
- Filed bug 539339 - Printing websites with multiple divs cause ill printing issues
- Filed bug 539322 - Printing formfields on Windows does not show text in buttons nor dropdown arrows
- Filed bug 539365 - Lightweight themes text-shadow exists on print-preview screen
- 9) Section: Password Manager --> Private Browsing:
- Windows: juanb, XP(vm) - PASS
- Mac OS: Tracy - Pass - 10.6 - Aravind - Pass
- Linux: juanb - Ubuntu 9.10 - PASS
- Notes: XP
- 10) Section: Cookies, Plug Ins, Layout, Content Handling:
- Windows: marcia - Win7 - Pass, individual failures noted below
- Mac OS: Tracy - Fail - 10.6
- Linux: juanb, beltzner - Fail
- Notes: Linux
- bug 495609 and bug 492200 resulted in expected APNG errors
- bug 539866 - drag and drop XPI isn't compatible with Firefox 3.6
- bug 539871 - Yahoo! Mail mailto: handler doesn't accept body: and subject: values
- In general, the plugins tests for Linux need some simplification. They are very time consuming to setup. Several tests need revision or deletion altogether as they don't apply to Linux or they just plain fail.
- Notes: Linux
- Notes: Mac 10.6:
- bug 539763 - Remnant across all tabs from Windows Media Player Test
- 11) Section: Security, Session Store:
- Windows: marcia - Security Pass
- Windows: rstrong - Session Store Fail
- Notes: Windows 7, 92% completed - didn't test forms due to issues that I passed to juanb.
- Mac OS: Tracy, Aravind Fail
- Linux: zpao, juanb - PASS (mostly) - Aravind In Progress
- Notes: Windows 7
- 12) Section: Online/Offline Events, Technical Tools:
- Windows: gmontagu Pass
- Windows 7: marcia - Ran Technical Tools - Pass
- Mac OS: mak - PASS(but missing Online/Offline ones from 6 on(campd.org is timing out)) - 10.6
- Linux: juanb - Ubuntu 9.10(vm) - Pass
- Notes: I edited several test cases in the offline section; and I found a bug with clearing the cache which is present in 3.5.7, and which had already been filed.
- Notes: Run on XP. Online/Offline Events: Test cases 2 to 9: BROKEN. Technical Tools: no issues.
- 13) Section: Multitouch (not part of FFT):
- Windows: juanb - Windows 7 - PASS
- Mac OS: mak - pass - 10.6 (https://litmus.mozilla.org/show_test.cgi?id=9243)
- Linux: n/a
- Notes: This is a separate test group, and can only be used on machines with a touch screen. Please ask Marcia or juanb for details.
- 14) Uninstall:
- Windows:retornam pass
- Mac OS: mak - Pass - 10.6
- Linux : juanb - Ubuntu 9.10 - PASS
Mozmill Tests
All the tests run with Mozmill have been synced with Litmus.
Litmus Test Case Failures on Windows 7
- Layout
- https://litmus.mozilla.org/single_result.cgi?id=228900 - Fails on Windows Vista and Mac. mw22 indicates Bug 463465 was not checked into 1.9.2. So this bug should probably be nominated for 1.9.2.1
- Content Handling
- Security
- https://litmus.mozilla.org/single_result.cgi?id=230801
- https://litmus.mozilla.org/single_result.cgi?id=230798
- https://litmus.mozilla.org/single_result.cgi?id=230798
- https://litmus.mozilla.org/single_result.cgi?id=230796
- https://litmus.mozilla.org/single_result.cgi?id=230794
- https://litmus.mozilla.org/single_result.cgi?id=230785
- https://litmus.mozilla.org/single_result.cgi?id=232421
Crashes during FFT testing
- Windows 7
- http://crash-stats.mozilla.com/report/index/bp-291dd72c-fa33-4b8d-bd1a-839d92100111
- Bug 535873
- Now we have a set of STR to reproduce this crasher.
- Plugin is only compatible with Firefox 2.0. Filed Bug 539282 to blocklist it and remove it from PFS.
- http://crash-stats.mozilla.com/report/index/bp-291dd72c-fa33-4b8d-bd1a-839d92100111
Extension Related Issues?
- Installed Skype 4.0 which includes Skype for Firefox Browser Highlighter. Extension version is 3.3.0.3971. Experienced some freezes after this installation.
- Most likely Bug 530995 Skype addon causes perf hit
Let's Not Forget
Please make sure to check the features you signed up to test, because some of these features' tests may not be in the FFT, such as multitouch. For reference, check the areas that went in 3.6 and make sure you run the tests you created for your features:
3.6 Fx Features
- Component folder lockdown: PASS (Henrik)