Thunderbird/Release Driving/Test Firedrill/Post Mortem
From MozillaWiki
< Thunderbird | Release Driving | Test Firedrill
Contents
Post Mortem for the test firedrill: Thunderbird/Release_Driving/Test_Firedrill
Done via email/irc.
- We're looking for:
- How well did new processes introduced go?
- Any problem/pain points?
- Can we reduce time spent or speed up processes?
Schedule of Events
This is how we did on timescales (times are UTC):
Time | Time Offset | Description |
---|---|---|
13:46 | N/A | Original bug nominated (Standard8 was afk!) |
15:45 | 0 | Made final decision on a good bug and sent out go for firedrill |
17:07 | 1h 22m | Bug fixed and landed. Go for build once tree had cycled green. |
18:13 | 2h 28m | Tagging started |
22:35 | 6h 50m | All unsigned builds completed |
23:18 | 7h 33m | Signing started |
01:19 | 9h 34m | Win32 signing complete |
Due to gozer needing to be afk for 3h, this is where firedrill ended.
Estimate for remainder of signing and update generation: 2h.
We'd have then been looking at approx 3h for final release (1-2 hours mirror uptake, 1h pushing website and announcements).
So probably could do a firedrill in 12h.
Leading up to the release (patch landing, code freeze etc)
- Picked the wrong bug for a drill! (no testcase/STR). So found a simpler one.
Build
- Too many emails to tb-drivers, some of the needed detail got lost in the noise.
- Need steps to do items (build guide) and someone else in case gozer is afk or otherwise (2nd build engineer already looked for).
- Bouncer entries
- Could be an issue that if just TB is doing a firedrill at a weekend then may be difficult for getting bouncer entries added.
- Suggestion is that we submit new entries for the next release so that we have them there if we need them in a firedrill situation.
- Its possible, but we may need consider our own bouncer or access to MoCo's. Think this touches on existing bugs.
- May also be able to file blocker bug on MoCo IT.
- Standard8/gozer to consider.
QA
- Did we do smoketests or just verify the bug? Does it matter if we're using the same revisions?
- No smoke test - I just made sure the bug was fixed and I could send emails, and That address book was unaffected (call that a ab smoketest - but that was done ad hoc).
- I would have done more testing if more than one bug would have been involved in the firedrill (and that would have been a complete smoke test)
- Do we have a guide/plan so that other people can do the job if Ludovic isn't about?
- NO but I can take an action item on writing what I did (send an email so I don't loose that fact)
Security
N/A
Website (Release Notes, What's New, L10n issues etc)
- We didn't actually do this for real. However, based on 3.0.2, we could easily get pages updated in an hour or two whilst build is going on. We're getting well-versed in how to do this now.
- thunderbirdDetails.class.php could be the only sticking point as it needs review from MoCo webdevs.
- Worst case we could stop importing the product-details section and include it direct in our site. Then update the core version later.
- Alternately we could just land with review-pending and stick it in a bug.
Actual Release (Pushing out, updates, etc)
- We didn't do this.
- See the 3.0.2 post mortem page for a release we did the day after.
Anything else
- Standard8 feels we need to get backups for everyone (who are familiar with the process) so that we can do hand-offs in the event of firedrill so that other folks can handle the process as well.