Calendar:QA Home
Welcome to the Calendar QA Community
This is our central page for the calendar QA community. This will provide a place for people to help organize plan and execute testing on the Sunbird and Lightning codebases. Anything you have that will benefit our QA effort is welcome here.
Who We Are
We are a worldwide network of individuals, striving to create a viable, open source, standards based calendar solution. We work closely with the rest of the Mozilla_QA_Community and with the calendar developers. For questions related to any of the material here, please try asking in irc://irc.mozilla.org#calendar or in irc://irc.mozilla.org#calendar-qa.
How to Get Involved
There are various ways you can get involved in helping the Calendar QA team, even with no previous experience!
Filing Bugs
Use Sunbird and/or Lightning. Find bugs. Write good bug reports.
We test the Sunbird and Lightning calendar projects. Sunbird and Lightning share much of the same code, and it is necessary to understand the high-level component architecture in order accurately report problems.
- Download a nightly build (for Sunbird) (for Lightning) and check to see that the bug has not already been fixed.
- Get a Bugzilla Account if you don't already have one - it's simple and it's free
- Next, go to the New Bug page on Bugzilla, and it can walk you through the rest.
- BE SURE TO SEARCH FOR EXISTING DEFECTS.
The 'qawanted' keyword
Calendar developers will place the 'qawanted' keyword on bugs where they need help from a QA person. Generally, the bug should include a comment about what is needed from QA. If it doesn't and you're unsure what needs to be done, please add a comment saying so.
List of current 'qawanted' bugs
Blocking Nominations
If a bug you've filed (or even just seen) seems to qualify as a bug that should be solved for the next release of Sunbird and/or Lightning, set the wanted0.8 flag to "?". If some bug should definitely block the release set the blocking0.8 flag to "?". For more information see Calendar:For_Everyone:Blocking_Flags
Verifying bugs
Bugs get resolved all the time. When this happens, it's helpful for QA to come along and ensure that the resolution was correct. For instance, if a patch was checked in, the bug will be marked FIXED. QA can then test the next nightly build to see if the bug has indeed been fixed. If you find a bug that has been fixed, then use Bugzilla to "Mark the Bug as Verified".
See QA Links for Bugzilla queries to recently resolved bugs and some useful resources (external calendars and extensions).
Confirming Unconfirmed Bugs
Many bugs are reported to Bugzilla in the Unconfirmed state. These bugs need QA folks to examine them and help determine (1) whether a bug really exists (2) what information a developer will need to fix the bug (3) where the bug should best be filed. See also the Mozilla guidelines for triaging.
See QA Links for Bugzilla queries to unconfirmed bugs.
Finding a Regression Range on a Bug
Very often something will break in a newer build that worked in a previous version. When this happens, it is always useful to determine the regression range for the bug. This means that you check the nightly builds, and try to find the two successive builds where the bug works in one build, and fails in the next one. From there, we can easily determine what changed in the source code using the bonsai tool.
An example using a binary search: Let's say that test X worked in version 0.3 and did not work in the current nightlies. You find out that 0.3 was released on October 11, 2006. So, you start looking at builds to find out when test X stopped working.
- Check 0.3 just to be sure that it did in fact work
- Check a nightly build created on a date halfway between now and October 11, 2006.
- If test X works on this build, then check it with a nightly that is halfway between this date and the current date.
- If test X fails on this build, then check it with a nightly that is halfway between this date and the date of your last known working build (in this case, October 11, 2006)
Keep halfing like this until you find the date where in the nightly before this date test X worked, and in the nightly built on this date, test X fails.
Here are the locations of nightlies (for lightning - for sunbird, replace "lightning" with "sunbird in the URL")
Testdays and other events
We are currently working on formalizing the calendar QA effort. There is much to be done, if you want to help make Sunbird and Lightning the best calendaring applications out there, you've come to the right place. We have a weekly calendar QA chat where you can connect with the other people doing calendar QA.
Visit our blog to see what is going on.
Feature Specifications
Do you think you've found a bug, but you're not sure what the application is supposed to do? Check here. The calendar developers keep these pages updated as they add and change features in Sunbird and Lightning.
Test Cases
- Feel free to add TestCases. We are regulary moving these cases into Litmus.
- Have a look if any bug that was recently resolved as Fixed|W4M|Duplicate need to be supported by Litmus, if so add flag [litmus testcase wanted] to whiteboard and add details about scenario if needed. Here is list of bugs that are currently supported by Litmus.
- We have created test cases that were stored in Litmus. Use nightly build of Sunbird or Lightning to run those tests.
Litmus Test Runs
Test Automation
We are currently experimenting with several automated test strategies. Automated testing will provide Regression and Unit testing. No automated User Interface (UI) testing is planned at this time because the UI for both Sunbird and Lightning still continues to change fairly often. That said, if you're burning to create/configure an automated UI testing scheme for the calendar applications, we'd love to hear from you.
Most of the discussion of these types of testing is taking place on [1] and for messages that are more specific to calendar QA please post on [2] or visit us at irc://irc.mozilla.org#calendar.
