Thunderbird:Testing/bugzilla

Tracking defects and making sure they are know, analyzed, is the best way to make a product better.

Introduction

The testing effort can not cover all the combinations of servers, protocols, use case and third party installs. At the time of this writing there are 5800+ bugs alone for Thunderbird that need attention (NEW+UNCO). We have different needs They are explained below and sorted by how much time you'll need to invest yourself.

To help in this area, you need a bugzilla account and some time.

Running queries on Thursdays

Every Thursday we have virtual meetups where we work on bugs present in bugzilla. We try to make sure that :

• The bug is still valid with current supported versions
• Try to make the bug move forward :
  • Can we reproduce the bug ?
  • Can we figure out specifics that trigger the bug ?
  • Is that bug a potential duplicate from another one ?
  • Is it a regression , when did it regress ?
  • Did we ask question in the past that never got answered ?

Useful bugzilla queries

Here are a few link to find bugs to work on :

• trunk/Thunderbird 3 UNCOnfirmed, no activity in last 7 days
• UNCOnfirmed, recent bugs with no responses

This activity lets us work with the already filed bugs and make sure we didn't miss anything worth missing. The idea behind the virtual meeting is that we'll make sure to try to be available for you if you have questions and don't what to do on a given bug.

Following a component

We've noticed that when a bug is taken into account very shortly after it's creation, bug openers were following the bug more closely. We would like to be more responsive - as in answer initial bug reports faster. Following a component is juts following a subset of bugs in bugzilla. And interacting with the bug reporters.

setup

You'll need a bugzilla account of course. In the email settings just select to follow one of these QA contacts or these fake email addresses. You can also do it just using bugzilla's Preferences

Which component to follow

We think it's important that helping us should be fun. The first component that you should follow, should be something you care a lot about in Thunderbird. This will make you more confident on how to use Bugzilla and how to deal with following a component. Once you feel comfortable with that, ask us and we'll tell you which one need more attentions then others.

How do easily deal with new bugs

When I deal with a new bug and depending on the bug , there are Three easy question one can ask :

1. Anything in Tools -> Error console when your issue happens
2. Does it also happens in -safe-mode (see http://support.mozillamessaging.com/en-US/kb/Safe+Mode)
3. Are these local or imap folders

With that you are gathering important information.

The first question let you rule out or not if the bug is related to an add-on. If it works in -safe-mode then an extension or theme is responsible for the bug. If it is the bug can be closed as INVALID - as extensions should be fixed by the extension authors.

The second question might help diagnose what the issue is - but rarely helps :(

The last question is a good starting point to try to reproduce - as once you've reproduce you can confirm the existence of the bug. If you can't reproduce say , so in a bug comment. You might also want to ask for precisions (ie on windows is there an anti-virus program running etc ...)

Dealing with regressions

For some bugs the reporters will tell you that something used to work and doesn't work anymore. These are called regressions. (But first, make sure that it's really only the Thunderbird build that's different between "works" and "doesn't work", and the reporter didn't change 15 other things between testing the new and old Thunderbird build, to avoid confusion). If it's a real regression, flag these bugs with the "regression" keyword, and then find a regression range. This can be easily found using our regression instructions.

Docs

In order to classify bugs we use the standard bugzilla keywords (like mlk, regression).

We also use a number of free formed strings in the Whiteboard area. We usually use square brackets around those free strings. These strings give us a lot of freedom and are especially useful when developers are looking at bugzilla list

• closeme yyyy-mm-dd When a bug stals - because the inital reporter doesn't reply - we use this flag and at the same time ask the reporter to answer our questions. We usually give the reporter a good 3 to 5 weeks to answer. At some point we'll just close those bugs as incomplete.
• dupme : is used on a bug that we think is a duplicate and we have been unable to find the duplicate in question. Sometime you can use dupme? to say that you are not so sure that the bug is a duplicate.
• has protocol logs - logs of some sort are attached to the bug and need to be read by a developer so he can rule them in or out.