« previous week | index | next week »
< most recent | upcoming >

Release Duty

• FF 46 cycle: kmoir

Misc

Shipped

Firefox 38.7.1esr

• http://hg.mozilla.org/releases/mozilla-esr38/rev/543165802342
• build 1 failed to run checkconfig because of unrelated buildbot-config repo health.

Firefox 45.0.1

• http://hg.mozilla.org/releases/mozilla-release/rev/7e7b02185615
• build 1 intermittent antivirus failures bug 1248299 Frequent antivirus job failures with IncompleteRead after several retries. Had to rebuild ~5 times for anti-virus failures.
• final_verification failed because updates were disabled when they ran, reran this job when they were enabled again

Fennec 45.0.1

• https://hg.mozilla.org/releases/mozilla-release/rev/910c9ac881ee
• same as 44.0.1, a sub-optimal set of partials were used, including 39.0 and 41.0.1. Neither of those shipped win64 but the uptake check expects them to be there, so drop little text files at these locations to fix, and then retriggered the uptake check builder which had timed out:

pub/firefox/releases/45.0.1/update/win64/zh-TW/firefox-41.0.2-45.0.1.partial.mar
pub/firefox/releases/45.0.1/update/win64/zh-TW/firefox-39.0-45.0.1.partial.mar

• intermittent antivirus failures bug 1248299 Frequent antivirus job failures with IncompleteRead after several retries. Had to rebuild ~5 times for anti-virus failures.
• final_verification failed because updates were disabled when they ran, reran this job when they were enabled again

Firefox 45.0.1esr (nthomas/kmoir)

• http://hg.mozilla.org/releases/mozilla-esr45/rev/9b294782389a
• http://hg.mozilla.org/releases/mozilla-esr45/rev/cab241354c10

build1

• nthomas checked all 13 of the update verify failures for 45.0.1esr, and they all come from a Balrog config issue, and in turn to the overlap of ESR38 and ESR45. The failing updates start from ESR24/31/38, Balrog serves 38.7.1esr on esr-localtest, which applies OK but then we do a diff against 45.0.1esr. We'd probably have to use a separate channel to make this work.
• we'll need a build2 because we're not fixing the update-settings.ini file when updating to 45.0.1esr, and get this diff for updated vs installer:

diff -r source/firefox/update-settings.ini target/firefox/update-settings.ini
5c5
< ACCEPTED_MAR_CHANNEL_IDS=firefox-mozilla-release
---
> ACCEPTED_MAR_CHANNEL_IDS=firefox-mozilla-esr

Handled in bug 1255245 ESR45 generates mar files with the wrong channel ID

• intermittent antivirus failures bug 1248299 Frequent antivirus job failures with IncompleteRead after several retries. Jordan and nthomas fixed the failures
• display version shows up at 45.0 because we're not bumping version_display.txt, bug 1257230

build2

• nthomas cancelled builds, source, and tag_l10n after reading the graph for esr45 repo incorrectly, and thinking the fix for update packaging had been missed. Jobs were restarted, no resulting scheduling problems
• 13 update verify jobs failed, 4 each on mac/linux32/linux64 and 1 windows. Same issue as build1 where we're testing ESR24/31/38 but offering 38.7.1esr instead of 45.0.1esr. Going to ignore that for now. The 45.0esr --> 45.0.1esr updates are working
• manually added win64 bouncer locations for bug 1254784
• final_verification failed because updates were disabled when they ran, reran this job when they were enabled again
• I forgot to update the balrog rule to set updates at 100% again, thanks rail for reminding me!

Firefox 46.0b2 (rail/jlund)

• http://hg.mozilla.org/releases/mozilla-beta/rev/2a62a6f3237ecda90639ec803e6a89d27fe448e5
• build 1 - None of the BBB based update verify jobs reported as completed back to Taskcluster. rail used `tctalker report_completed` to mark them green. Possible reason: "update verify" matches '^.*date.*$' used in BBB to split dev/prod tasks. As a result those tasks are processed by dev BBB, which was in the middle of cred replacement.
• build2 - Bouncer submitter task was "cancelled"
• 3 en-US balrog submitter tasks failed: "Gateway Time-out"
• final verify failed a couple of times until we fixed the cdntest channel (it was en-US only)

Fennec 46.0b2 (kmoir)

• http://hg.mozilla.org/releases/mozilla-beta/rev/2a62a6f3237e
• http://hg.mozilla.org/releases/mozilla-beta/rev/5904e3eb711d
• build 1 - repacks all failed due to bug 1256765 fennec 46.0b2 build1 repacks are failing. nalexander wrote patch to remove need for bouncer-unsigned-unaligned.apk
• build 2 - very low uptake over the weekend for the new release - relman has followed up with Google.

Firefox 46.0b4 (rail/mtabara)

• http://hg.mozilla.org/releases/mozilla-beta/rev/ccd48b57944a0429fc8dc7d602baafb7d04b4d76
• http://hg.mozilla.org/releases/mozilla-beta/rev/2f6f69a19150
• build1 - bug 1258644 Incorrect version displayed for 46.0b4 builds
• build2: needed it as release build promotion is now setup to handle skipping beta
  • intermittent issues
    • bug 1258644 Incorrect version displayed for 46.0b4 builds

Fennec 46.0b4 (kmoir)

• http://hg.mozilla.org/releases/mozilla-beta/rev/ccd48b57944a

Ongoing

Thunderbird 45.0b3 (nthomas/kmoir)

• http://hg.mozilla.org/releases/comm-esr45/rev/bab23fd1d89b
• didn't receive mail notifications for repack failures, nthomas sent email about this
• '[release] Thunderbird 45.0b3 build1: failed at repack_9/10 on win32' at Tue Mar 22 19:34:16 UTC 2016 (failed twice, ran on third retry)
• '[release] Thunderbird 45.0b3 build1: failed at repack_6/10 on win32' at Tue Mar 22 19:48:39 UTC 2016
• '[release] Thunderbird 45.0b3 build1: failed at repack_10/10 on win32' at Tue Mar 22 19:58:01 UTC 2016
• '[release] Thunderbird 45.0b3 build1: failed at repack_10/10 on win32' at Tue Mar 22 20:44:00 UTC 2016

Thunderbird 38.7.1

• http://hg.mozilla.org/releases/comm-esr38/rev/537d63e899c7

Roundtable

Action items