MozillaQualityAssurance:Home Page:Firefox 3.0 TestPlan:Fx3 QA Daily Smoketest
From MozillaWiki
Contents
- 1 Tiger Team Testing Squad
- 2 Tiger Testing Procedure
- 3 Plans to include Leopard Testing and Debug Builds
- 4 Schedule
- 5 Major Bugs Discovered during smoketesting
- 5.1 Week of September 15
- 5.2 Week of July 14, 2008
- 5.3 Week of June 23, 2008
- 5.4 Week of April 28, 2008
- 5.5 Week of April 21, 2008
- 5.6 Week of April 7, 2008
- 5.7 Week of March 31, 2008
- 5.8 Week of February 25, 2008
- 5.9 Week of February 18, 2008
- 5.10 Week of January 25, 2008
- 5.11 Week of January 14, 2008
- 5.12 Week of January 7, 2008
- 5.13 Week of January 2, 2008
- 5.14 Week of December 3, 2007
- 5.15 Week of October 22, 2007
- 5.16 Week of October 15, 2007
- 5.17 Week of October 8, 2007
- 5.18 Week of October 1, 2007
- 5.19 Week of September 24, 2007
- 5.20 Week of September 17, 2007
- 5.21 Week of September 3, 2007
- 5.22 Week of August 20, 2007
- 5.23 Week of August 13, 2007
- 5.24 Week of August 6, 2007
- 5.25 Week of July 30, 2007
- 5.26 Week of July 23, 2007
- 5.27 Week of July 16, 2007
- 5.28 Week of July 9, 2007
- 5.29 Week of July 2, 2007
Tiger Team Testing Squad
al, marcia, tchung, tracy, stephend
Platform Teams for Daily Smoketesting of Firefox Trunk Builds
- Windows:
- Vista - Marcia/Tony/Hasham
- XP - Al/Tracy/Stephen
- Mac:
- PPC - Marcia/Stephen/Tracy
- Intel - Al/Marcia/
- Linux
- Various Flavors - Tony//Tracy
Platforms are covered double so that we have :
- a.) the chance for double testing on different machines
- b.) 4 eyes looking at a test and not only 2
- c.) in case of vacation / illness/ other urgent tasks there is always a backup
- d.) one team member can run the smoketest, the other focus testing
- e). in times of a low checkin rate, one team member can regenerate or working *for a week on other goals.
As Example the XP Platform Team would be also flexible that one member can concentrate on other tasks or focus testing and the 3rd one can pause for a week.
Key:
- ST = Smoketest
- The tester will run the Litmus smoketest.
- Foc = Focused Testing
- The tester will run focused testing around any checkins that have occurred. This means reviewing Burning Edge or bonsai.mozilla.org to see what has been checked in. Don't spend more than 2-3 hours on this.
- Test Results can be found in Litmus by using Smoketest Results. That is a load in sidebar bookmarkable page.
Tiger Testing Procedure
- Refer to the schedule and perform smoketest or focused testing as noted for the platform(s) you're assigned to that week.
- Helpful links:
- Check what has landed on Mozilla Central by going here: Mozilla Central. Use the ellipis to expand the query if you want to see more of the checkins. If the bug has been fixed on the trunk, please mark it as "Verified Fixed" during your focused testing.
- Use Litmus for smoketests reporting.
- File bugs in bugzilla.
- http://www.squarefree.com/burningedge/ - Good to check this to see what has been checked in if you're assigned focused testing.
- Elevate Smoketest blocker bugs with the sheriff of the day.
- Smoketest blocker is any bug that completely breaks some critical area(s) of the browser; breakage that can not be worked around in anyway.
- Refer to the Sheriff Schedule to see who is on duty. Then notify them of the blocker bug, via irc or email, so they can do their Sheriff stuff.
- Remain accessible to help with QA of the issue.
Plans to include Leopard Testing and Debug Builds
- marcia will formulate a plan going forward to incorporate Leopard trunk testing into the Tiger Team testing schedule.
- Initial plan will be to build in a repertoire of testing Leopard on PPC and Intel Macs once every week.
- tomcat will formulate a plan going forward to incorporate debug build testing into the Tiger Team testing schedule.
Schedule
Sept 15-19
Monday | Tuesday | Wednesday | Thursday | Friday | |
---|---|---|---|---|---|
Win XP | Tomcat ST | ||||
Vista | stephend - Foc | tony - Foc | - | - | |
Intel Mac | - | - | al - ST | - | |
PPC Mac | marcia - Foc | - | marcia - Tiger ST | - | - |
Linux | tomcat - ST | tracy - Foc | - |
Sept 22-26
Monday | Tuesday | Wednesday | Thursday | Friday | |
---|---|---|---|---|---|
Win XP | tracy ST | ||||
Vista | tony - Foc | tomcat - Foc | - | tomcat - ST | |
Intel Mac | - | - | owner - ST | - | |
PPC Mac | marcia - Foc | - | marcia - Leopard Debug ST | - | al - Foc |
Linux | al - ST | stephen - Foc | - |
Major Bugs Discovered during smoketesting
- Please paste the major bug numbers in here so we can track metrics as we go along. This will be much easier than going back and having to dredge through Bugzilla later on. If you find a bug that has been reported previously or you end up confirming it, include that as well. If you are covering focused testing, it would be great to get some information about what you focused on testing by day.
- Please put latest finding on top of this weekly list
Week of September 15
- Bugs filed by marcia:
Week of July 14, 2008
Week of June 23, 2008
Filed by Al:
Week of April 28, 2008
Filed by Al:
Week of April 21, 2008
- Filed by marcia;
- Bug 430704
- Bug 430575
- Bug 430493
- Bug 430482
Week of April 7, 2008
- Filed by marcia;
Week of March 31, 2008
- Filed by marcia;
Week of February 25, 2008
- Filed by marcia;
Week of February 18, 2008
- Filed by marcia:
Week of January 25, 2008
- Filed by marcia:
Week of January 14, 2008
- Filed by marcia:
Week of January 7, 2008
- Filed by marcia:
Week of January 2, 2008
- Bug 410536 - marcia
- Bug 410528 - marcia
Week of December 3, 2007
- Bug 407200 - marcia
- Bug 407192 - marcia
- Bug 407180 - marcia
Week of October 22, 2007
- Bug 400865 - marcia
- Bug 401321 - marcia
Week of October 15, 2007
- Bug 400079 - del.icio.us rss feed does not populate live bookmark]
Week of October 8, 2007
Week of October 1, 2007
- [1] View -> Toolbars -> customize result in JS nsIAnnotationService.addObserver Errors [tomcat]
- [2] Testcase from Bug 398157 freeze Intel Machines
- [3] Crash on Load [@ js_Interpret] [tomcat]
- [4] Opening/closing toolbar customize breaks the location bar [marcia] - Blocker
- [5] "Don't Quit" button not respected when Command Q is invoked during download [marcia]
Week of September 24, 2007
- Bug 397688 - Unable to create an account on surveymonkey.com]
- Bug 397296 - Firefox builds need to use the current NSS CVS tag]
Week of September 17, 2007
- bug 396811 - My.Netscape.com search textfield overlays navigation area
- bug 396821 - Weird wrapping behavior for nested floats
- bug 397225 - RSS feed subscription is broken
- bug 397296 - Firefox builds need to use the current NSS CVS tag
Week of September 3, 2007
- bug 395428 Can't tab through mail fields in Yahoo Mail Beta (mac-marcia)
- bug 395445 Tag and Star icon both show for Location bar entries (marc-marcia)
- bug 389744 Crash [@ nsPropertyTable::PropertyList::Equals] when zooming in a dd with overflow:scroll; position:fixed (mac - marcia). This bug was already on file but I will track the checkin of the patch to make sure it is fixed.
- bug 395098 WMP plugin does not show controls on trunk (mac-marcia)
Week of August 20, 2007
TBD
Week of August 13, 2007
n/a - Mofo all-hands
Week of August 6, 2007
- bug 385925 "Open" link in download manager broken (linux - tracy)
- bug 390711 autocomplete menus have no border (linux - tracy)
- bug 391380 moz-icon: opens Launch Applicaion (linux - tracy)
- bug 391388 modal dialog don't have focus on open (linux - tracy)
Week of July 30, 2007
- 2007-08-03-08 (Intel Mac, marcia) Focused testing after josh's checkin Bug 354768 I hit some rough seas with lots of crashes, Bug 390707 was on file already and may cover my crashes. In general today's build is quite crashy. I crashed frequently in form fields and also hit the text zoom bug which crashes many sites when you increase the font size.
Week of July 23, 2007
- 2007-07-24 (Intel Mac, marcia) Bug 389423
- 2007-07-27- 04 (Intel Mac, marcia) Bug 389856
Week of July 16, 2007
- 2007-07-017-04 Mac (marcia) bug 388522 Found this regression while doing focused testing of drag and drop. I noticed there was a landing affecting d&d. Found regression window and cc'd devs.
Week of July 9, 2007
- 2007-07-10-08 (Intel Mac, marcia) bug 387573
- 2007-07-10-08 - (WinXP - ctalbert) bug 355538 Did not file, but found while running smoketest
- 2007-07-10-08 - (WinXP - ctalbert) bug 387568 Found and filed
- 2007-07-11-05 - (WinXP - ctalbert) bug 334305 Existing bug, found during test
- 2007-07-11-05 - (WinXP - ctalbert) bug 351049 Existing bug, found during test
- 2007-07-11-05 - (WinXP - ctalbert) bug 351049 Existing bug, found during test
- 2007-08-12-05 - (WinXP - ctalbert) bug 387568 Existing regression, encountered during test
- 2007-08-13-04 - (Intel Mac - marcia) bug 388082 Existing regression, encountered during test
- 2007-07-09-08 ALL (Tomcat) bug 387446 Worked with devs on this Bug
Week of July 2, 2007
- 2007-07-03-04 (Windows - tracy) Bug 386739
- 2007-07-03-04 (Windows - tomcat)Bug 386656
- 2007-07-05-04 (Mac - marcia) Bug 387002
- 2007-07-06-04 (Mac - marcia) Bug 387084 I did not file the bug, but I discovered it during testing.
- 2007-07-06-04 Bug 387188