Releases/Firefox 21.0/BuildNotes
Contents
Summary
- Who: nthomas
- Tracking: bug 854372
- Bugs encountered:
- bug 869401 - disk full on signing3
Build 1
Checklist
-
Ship the l10n milestone DESKTOP MOBILE RELEASE BETA -
Submit to Ship It DESKTOP MOBILE RELEASE BETA ESR - --> ABANDONED
Notes
- merge from mozilla-beta, l10n milestone creation, and ship it submission done by bajaj. Review done by nthomas.
- xulrunner checksums and push_to_mirrors jobs hit hg server errors when cloning the tools repo. The automation retried them successfully
Recover from full signing server disk
(bhearsum)
signing3's disk filled up (bug 869401) and failed the win32 build and many linux repacks. After the server was fixed, I did the following to recover:
- Rebuilt the win32 build (failed to upload symbols, bug 869522, second rebuild was green)
- Used the standalone builder to run the failed repacks for Linux (br bs ca cs csb et eu fa ff mai mk ml mr nb-NO si sk sl vi zh-CN zu) and Linux64 (bn-BD bn-IN bs ca cs es-ES es-MX et eu fa ff hr hu hy-AM id is it lt lv mai mk ml nb-NO rm ro ru si sk sl)
- Once the standalone builders completed, ran "repack complete" builders with force build to get the automation unstuck.
Build 2 (Desktop)
This is a straight rebuild of the same revision as build1 for the AMD crasher problem. We may not ship it, and builds are hidden from the general public to avoid confusion & diluting feedback on build1.
Checklist
-
Ship the l10n milestone DESKTOP MOBILE RELEASE BETA -
Submit to Ship It DESKTOP MOBILE RELEASE BETA ESR - --> ABANDONED
Notes
- ship it submission done by bajaj
- review done by nthomas: moved from same revision as build1 to using the same relbranch (GECKO210_2013050704_RELBRANCH)
- xulrunner push_to_mirrors failed because build1 had already been pushed, this is no problem
Things to undo
Due to the 'run build2 while build1 not actually dead' situation there are some things we had to undo:
- repush 21.0build1 snippets after build2 are generated (QA had some issues with this until we repushed build1)
If we end up shipping build1:
- undo release config change, move tags and ensure reconfig before running any builders on build1
- move tags on tools repo back
- move FIREFOX_21_0_RELEASE tag back on to revision 3402fc52312c
Build 2 (Android) - SHIPPED THIS ONE
Code change resolving regression from bug 853522
Checklist
-
Ship the l10n milestone DESKTOP MOBILE RELEASE BETA -
Submit to Ship It DESKTOP MOBILE RELEASE BETA ESR - Publish Fennec MOBILE RELEASE BETA
- Push Fennec to mirrors MOBILE RELEASE BETA
Notes
- bajaj submitted, nthomas review, no issues
Publish Fennec
Published to Yandex + Google Play.
Build 3 (Desktop) SHIPPED THIS ONE
This is a new build to address several bugs.
Checklist
-
Ship the l10n milestone DESKTOP MOBILE RELEASE BETA -
Submit to Ship It DESKTOP MOBILE RELEASE BETA ESR -
Push to mirrors DESKTOP RELEASE -
Apply Throttling DESKTOP RELEASE - Verify throttling DESKTOP RELEASE
- Run pushsnip DESKTOP RELEASE BETA ESR
- Update bouncer links DESKTOP BETA RELEASE
- Update XULRunner wiki pages DESKTOP RELEASE
- Post-release tasks DESKTOP RELEASE BETA ESR
- Get the Browser Choice website updated DESKTOP RELEASE
Notes
- ship it submission done by bajaj
- review done by hwine: unchecked "l10n dashboard check"
- requires undo of directory changes to partner repacks:
- xulrunner push_to_mirrors failed because build1 had already been pushed
- removed /pub/mozilla.org/xulrunner/releases/21.0 as xrbld@stage, then rebuilt
- QA automation expects there to be N directories if you're up to buildN, but we have hidden build2. Created empty dir at .../21.0-candidates/build2-for-qa-automation/ to work around this.
- separate out 21.0 build1 and build2 --> build3 paths so we can push that early
# aus3-staging cd /opt/aus2/snippets/staging/ rsync -a --include='Firefox' --include='Firefox/21.0' --include='Firefox/21.0/**' --exclude='*' \ ../pushed/Firefox-21.0-build3-test/ Firefox-21.0-build3-early-update/ cd Firefox-21.0-build3-early-update # get rid of unwanted releasetest find . -maxdepth 6 -type d -name 'releasetest' -exec rm -rf {} \; # move betatest to release find . -mindepth 5 -maxdepth 5 -type d -exec mv {}/betatest {}/release \; # swap from stage.m.o to ftp.m.o for f in `find . -type f`; do sed -i -e 's/stage\.mozilla\.org/ftp.mozilla.org/' $f done
- push 21.0 build1 and build2 --> build3 updates
# ffxbld@aus3-staging pushsnip Firefox-21.0-build3-early-update
Reset automation to build3
To ship build3 after the automation ran for build4 we need to reset the automation.
- undo build4's release config change in buildbot-config, move FIREFOX_21_0_RELEASE tag back to FIREFOX_21_0_BUILD3 (pushlog)
- in tools backout out build4 changes, move FIREFOX_21_0_RELEASE tag back to FIREFOX_21_0_BUILD3, and FIREFOX_21_0_RELEASE_RUNTIME tag back FIREFOX_21_0_BUILD3_RUNTIME (pushlog)
- move FIREFOX_21_0_RELEASE tag back to FIREFOX_21_0_BUILD3 on mozilla-release, plus buildbotcustom, partner-repacks (mozharness would be a noop)
- reconfig build masters and schedulers
Also repushed the 21.0build3 test snippets
# aus3-staging cd /opt/aus2/snippets/pushed mv Firefox-21.0-build3-test ../staging/Firefox-21.0-build3-test-repush pushsnip Firefox-21.0-build3-test-repush
and obsolete the production snippets we're not going to use
mkdir /opt/aus2/snippets/obsolete mv /opt/aus2/snippets/staging/Firefox-21.0-build{1,2,4} /opt/aus2/snippets/obsolete/
Later fix releasetest which was going old --> 21.0 build3 --> 21.0 build4 (bug 871851). build3 is 20130511120803:
$ cd /opt/aus2/incoming/3/Firefox/21.0 # verify everything in */20130511120803 is betatest or releasetest $ find */20130511120803 -type f | wc -l 890 $ find */20130511120803 -type f | egrep '/(releasetest|betatest)/' | wc -l 890 $ rm -rf */20130511120803
Apply/Verify Throttling
Throttling was requested and applied in bug 870883. Watched https://aus3.mozilla.org/update/1/Firefox/10.0.2/20120215223356/WINNT_x86-msvc/en-US/release/update.xml before and after the push to verify it.
Push snippets
As ffxbld@aus3-staging:
cd /opt/aus2/snippets/staging ~/bin/pushsnip Firefox-21.0-build3
Build 4 (Desktop) NOT SHIPPED
This is build 3 rebuilt for AMD crash issue.
Checklist
-
Ship the l10n milestone DESKTOP MOBILE RELEASE BETA -
Submit to Ship It DESKTOP MOBILE RELEASE BETA ESR - We decided to ship build3, abandon build4 unless there's a crash spike
Notes
- ship it submission done by bajaj
- reviewed by callek
- xulrunner push_to_mirrors failed because build3 had already been pushed, this is no problem
-
repush 21.0build3 snippets after build4 are generatedQA completed build3 testing before build4 got this far