Releases/Firefox 15.0/BuildNotes
From MozillaWiki
< Releases
Contents
- 1 Notes About Releasing
- 2 Bugs hit
- 3 Build Engineers
- 4 Signed-off Revision(s)
- 5 L10N changesets
- 6 Tags
- 7 Build data
- 8 Notes
- 8.1 Build 1
- 8.1.1 Diff mozconfigs
- 8.1.2 Set reserved slaves
- 8.1.3 Preparing to start Automation
- 8.1.4 Bouncer Submitter
- 8.1.5 Update Verify
- 8.1.6 Android Signing
- 8.1.7 Reset reserved slaves
- 8.1.8 Push to internal mirrors
- 8.1.9 Backupsnip
- 8.1.10 Release Day
- 8.1 Build 1
Notes About Releasing
Please update the Notes Template and the Release:Primer for future releases (bug fixes, changes to automation) as needed
Bugs hit
Enter any bugs pre-existing or newly discovered and filed during the release:
- bug ??? - bouncer_submitter failed because Firefox 15.0 already existed as a product
- bug 785661 Update verify failed for TB/Firefox 15 versions < 12.0
Build Engineers
Callek and coop - Tracking bug: bug 772038
Signed-off Revision(s)
Build 1: 28af55043104
L10N changesets
Instructions on how to get them
- Firefox: https://l10n-stage-sj.mozilla.org/shipping/l10n-changesets?ms=fx15
- Fennec: https://l10n-stage-sj.mozilla.org/shipping/json-changesets?ms=fennec15&platforms=android&multi_android-multilocale_repo=releases%2Fmozilla-beta&multi_android-multilocale_rev=default&multi_android-multilocale_path=mobile%2Fandroid%2Flocales%2Fmaemo-locales
Tags
Manually tag the automation code, then record the generated tags below. (details)
| Build # | Branch, Tags | Changeset |
| 1 | MOBILE150_201208????_RELBRANCH, FENNEC_15_0_RELEASE, FENNEC_15_0_BUILD1 | d470654392d9 |
| GECKO150_201208????_RELBRANCH, FIREFOX_15_0_BUILD1 FIREFOX_15_0_RELEASE | 450143d2d810 |
Build data
Firefox
| Build # | Type | Build ID | Build machine | Time to build |
| 1 | Linux | 20120824154833 | linux-ix-slave20 | 1 hrs, 11 mins, 57 secs |
| Linux64 | linux64-ix-slave06 | 1 hrs, 12 mins, 22 secs | ||
| Mac | bld-lion-r5-080 | 1 hrs, 26 mins, 26 secs | ||
| Windows | w64-ix-slave21 | 3 hrs, 27 mins, 26 secs |
Fennec
| Build # | Type | Build ID | Build machine | Time to build |
| 1 | android | 20120824153430 | linux-ix-slave06 | 55 mins, 7 secs |
Notes
Build 1
Diff mozconfigs
rail helpfully ran the diff. He says to ignore 'l10n_tree':
--- beta 2012-08-24 13:06:50.791772258 -0400
+++ release 2012-08-24 13:06:11.671771671 -0400
@@ -24,7 +24,7 @@
'disable_tinderbox_mail': True,
'download_base_url': 'http://ftp.mozilla.org/pub/mozilla.org/firefox',
'enUS_binaryURL': 'http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-release',
- 'enable_blocklist_update': True,
+ 'enable_blocklist_update': False,
'enable_l10n': False,
'enable_l10n_onchange': True,
'enable_mac_a11y': True,
@@ -45,7 +45,6 @@
'hgurl': 'http://hg.mozilla.org/',
'l10nDatedDirs': True,
'l10nNightlyUpdate': False,
- 'l10nUploadPath': '/home/ftp/pub/mozilla.org/firefox/nightly/latest-mozilla-release-l10n/',
'l10n_modules': ['browser',
'dom',
'extensions/reporter',
@@ -58,7 +57,7 @@
'l10n_platforms': ['linux', 'linux64', 'win32', 'macosx64'],
'l10n_repo_path': 'releases/l10n/mozilla-release',
'l10n_tinderbox_tree': 'Mozilla-l10n',
- 'l10n_tree': 'fxrelease',
+ 'l10n_tree': 'fxrel',
'localesURL': 'http://hg.mozilla.org//build/buildbot-configs/raw-file/production/mozilla/l10n/all-locales.mozilla-release',
'lock_platforms': True,
'mobile_download_base_url': 'http://ftp.mozilla.org/pub/mozilla.org/mobile',
@@ -82,7 +81,6 @@
'download_symbols': False,
'enable_ccache': True,
'enable_codesighs': False,
- 'enable_dep': True,
'enable_opt_unittests': False,
'enable_shared_checkouts': True,
'enable_xulrunner': False,
@@ -203,129 +201,6 @@
5)],
'update_platform': 'Android_arm-eabi-gcc3',
'upload_symbols': True},
- 'android-armv6': {'android_signing': True,
- 'app_name': 'browser',
- 'base_name': 'Android Armv6 mozilla-release',
- 'build_space': 6,
- 'builds_before_reboot': 1,
- 'enable_ccache': True,
- 'enable_codesighs': False,
- 'enable_opt_unittests': False,
- 'enable_shared_checkouts': True,
- 'enable_xulrunner': False,
- 'env': {'CCACHE_COMPRESS': '1',
- 'CCACHE_DIR': '/builds/ccache',
- 'CCACHE_UMASK': '002',
- 'DISPLAY': ':2',
- 'HG_SHARE_BASE_DIR': '/builds/hg-shared',
- 'JAVA_HOME': '/tools/jdk6',
- 'LC_ALL': 'C',
- 'MOZ_CRASHREPORTER_NO_REPORT': '1',
- 'MOZ_OBJDIR': 'obj-firefox',
- 'MOZ_SYMBOLS_EXTRA_BUILDID': 'android-armv6-mozilla-release',
- 'PATH': '/tools/jdk6/bin:/opt/local/bin:/tools/python/bin:/tools/buildbot/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/',
- 'POST_SYMBOL_UPLOAD_CMD': '/usr/local/bin/post-symbol-upload.py',
- 'PYTHON26': '/tools/python-2.6.5/bin/python',
- 'SYMBOL_SERVER_HOST': 'symbols1.dmz.phx1.mozilla.com',
- 'SYMBOL_SERVER_PATH': '/mnt/netapp/breakpad/symbols_mob/',
- 'SYMBOL_SERVER_SSH_KEY': '/home/cltbld/.ssh/ffxbld_dsa',
- 'SYMBOL_SERVER_USER': 'ffxbld',
- 'TINDERBOX_OUTPUT': '1'},
- 'is_mobile_l10n': False,
- 'mobile_dir': 'mobile/android',
- 'mozconfig': 'android-armv6/mozilla-release/nightly',
- 'packageTests': True,
- 'platform_objdir': 'obj-firefox',
- 'post_upload_include_platform': True,
- 'product_name': 'firefox',
- 'slaves': ['bld-centos5-32-vmw-001',
- 'bld-centos5-32-vmw-002',
- 'bld-centos5-32-vmw-003',
- 'bld-centos5-32-vmw-004',
- 'bld-centos5-32-vmw-005',
- 'bld-centos5-32-vmw-006',
- 'bld-centos5-32-vmw-007',
- 'bld-centos5-32-vmw-008',
- 'bld-centos5-32-vmw-009',
- 'bld-centos5-32-vmw-010',
- 'bld-centos5-32-vmw-011',
- 'bld-centos5-32-vmw-012',
- 'bld-centos5-32-vmw-013',
- 'bld-centos5-32-vmw-014',
- 'bld-centos5-32-vmw-015',
- 'bld-centos5-32-vmw-016',
- 'bld-centos5-32-vmw-017',
- 'bld-centos5-32-vmw-018',
- 'bld-centos5-32-vmw-019',
- 'bld-centos5-32-vmw-020',
- 'bld-centos5-32-vmw-021',
- 'bld-centos5-32-vmw-022',
- 'mv-moz2-linux-ix-slave02',
- 'mv-moz2-linux-ix-slave03',
- 'mv-moz2-linux-ix-slave04',
- 'mv-moz2-linux-ix-slave05',
- 'mv-moz2-linux-ix-slave06',
- 'mv-moz2-linux-ix-slave07',
- 'mv-moz2-linux-ix-slave08',
- 'mv-moz2-linux-ix-slave09',
- 'mv-moz2-linux-ix-slave10',
- 'mv-moz2-linux-ix-slave11',
- 'mv-moz2-linux-ix-slave12',
- 'mv-moz2-linux-ix-slave13',
- 'mv-moz2-linux-ix-slave14',
- 'mv-moz2-linux-ix-slave15',
- 'mv-moz2-linux-ix-slave16',
- 'mv-moz2-linux-ix-slave17',
- 'mv-moz2-linux-ix-slave18',
- 'mv-moz2-linux-ix-slave19',
- 'mv-moz2-linux-ix-slave20',
- 'mv-moz2-linux-ix-slave21',
- 'linux-ix-slave01',
- 'linux-ix-slave02',
- 'linux-ix-slave06',
- 'linux-ix-slave12',
- 'linux-ix-slave13',
- 'linux-ix-slave14',
- 'linux-ix-slave15',
- 'linux-ix-slave16',
- 'linux-ix-slave17',
- 'linux-ix-slave18',
- 'linux-ix-slave19',
- 'linux-ix-slave20',
- 'linux-ix-slave21',
- 'linux-ix-slave22',
- 'linux-ix-slave23',
- 'linux-ix-slave24',
- 'linux-ix-slave25',
- 'linux-ix-slave26',
- 'linux-ix-slave27',
- 'linux-ix-slave28',
- 'linux-ix-slave29',
- 'linux-ix-slave30',
- 'linux-ix-slave31',
- 'linux-ix-slave32',
- 'linux-ix-slave33',
- 'linux-ix-slave34',
- 'linux-ix-slave35',
- 'linux-ix-slave36',
- 'linux-ix-slave37',
- 'linux-ix-slave38',
- 'linux-ix-slave39',
- 'linux-ix-slave40',
- 'linux-ix-slave41',
- 'linux-ix-slave42'],
- 'src_mozconfig': 'mobile/android/config/mozconfigs/android-armv6/nightly',
- 'stage_platform': 'android-armv6',
- 'stage_product': 'mobile',
- 'talos_masters': [('buildbot-master36.build.mozilla.org:9301',
- True,
- 5)],
- 'tooltool_manifest_src': 'mobile/android/config/tooltool-manifests/android-armv6/releng.manifest',
- 'unittest_masters': [('buildbot-master36.build.mozilla.org:9301',
- True,
- 5)],
- 'update_platform': 'Android_arm-eabi-gcc3-armv6',
- 'upload_symbols': True},
'android-debug': {'android_signing': True,
'app_name': 'browser',
'base_name': 'Android Debug mozilla-release',
@@ -337,7 +212,6 @@
'download_symbols': False,
'enable_ccache': True,
'enable_codesighs': False,
- 'enable_dep': True,
'enable_leaktests': False,
'enable_opt_unittests': False,
'enable_shared_checkouts': True,
@@ -462,7 +336,6 @@
'create_partial': False,
'download_symbols': False,
'enable_codesighs': False,
- 'enable_dep': True,
'enable_opt_unittests': False,
'enable_shared_checkouts': True,
'enable_xulrunner': False,
@@ -1409,7 +1282,6 @@
'talos_masters': [('buildbot-master36.build.mozilla.org:9301', True, 5)],
'tinderbox_tree': 'Mozilla-Release',
'tooltool_url_list': ['http://runtime-binaries.pvt.build.mozilla.org/tooltool'],
- 'unittest_build_space': 6,
'unittest_masters': [('buildbot-master36.build.mozilla.org:9301', True, 5)],
'unittest_suites': [('mochitests',
{'chunkByDir': 4,
Set reserved slaves
- set reserved_slaves to 8:
#cltbld@bm12 cd /builds/buildbot/build1 echo 8 > master/reserved_slaves
Preparing to start Automation
- set a clobber for any master, release-mozilla-release, any builder as a speed up
- Locked for 6 mw32-ix-slaves to bm12 and rebooted them. Locked 8 bld-r5-lion slaves to bm12.
- Shipped the l10n milestones
- Clicked "Ship" for Fennec
- Clicked "Add"
- repo: releases/mozilla-release
- branch: default
- path: mobile/android/locales/maemo-locales
- Clicked "Ship it"
- Clicked "Add"
- Clicked "Ship" for Firefox
- Clicked "Ship it"
- Clicked "Ship" for Fennec
- Landed buildbot-configs patch w/ updated l10n-changesets:
wget -O- --no-check-certificate "https://bugzilla.mozilla.org/attachment.cgi?id=654915" | patch -p1 sed -i -e "s/TBD/'???'/" mozilla/release-firefox-mozilla-release.py mozilla/release-fennec-mozilla-release.py wget -O mozilla/l10n-changesets_mobile-release.json "https://l10n-stage-sj.mozilla.org/shipping/json-changesets?ms=fennec15&platforms=android&multi_android-multilocale_repo=releases%2Fmozilla-beta&multi_android-multilocale_rev=default&multi_android-multilocale_path=mobile%2Fandroid%2Flocales%2Fmaemo-locales" wget -O mozilla/l10n-changesets_mozilla-release "https://l10n-stage-sj.mozilla.org/shipping/l10n-changesets?ms=fx15" # Remove the locales we aren't shipping from l10n changesets (bug 784642) sed -i -e '/^ach.*$/d' -e '/^mn.*$/d' -e '/^sw.*$/d' mozilla/l10n-changesets_mozilla-release hg commit -m "Bug 772038 - tracking bug for build and release of Firefox/Fennec 15.0 - r=bhearsum" hg up -r production -C hg transplant default hg tag -r production {FIREFOX,FENNEC}_15_0_{RELEASE,BUILD1} hg push ssh://hg.mozilla.org/build/buildbot-configs
- Tagged buildbotcustom:
hg tag -r production-0.8 {FIREFOX,FENNEC}_15_0_{RELEASE,BUILD1}
hg push ssh://hg.mozilla.org/build/buildbotcustom
- Tagged tools:
hg tag -r default {FIREFOX,FENNEC}_15_0_{RELEASE,BUILD1}
hg push ssh://hg.mozilla.org/build/tools
- Updated and reconfiged buildbot-master12:
cd /builds/buildbot/build1 source bin/activate make update reconfig
- ERROR didn't fix quoting in csets as instructed via review
- Fixed and commited + retagged in buildbot-configs
- re-ran update/reconf on bm12
- Ran release sanity in dry run mode:
cd /builds/buildbot/build1/master source ../bin/activate PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u Callek \ -V 15.0 --branch mozilla-release --build-number 1 \ --release-config release-firefox-mozilla-release.py \ --release-config release-fennec-mozilla-release.py --products firefox,fennec \ --dryrun --l10n-dashboard-version=15 localhost:9001
- Complained about ach, mn, and sw being missing, because the l10n dashboard is wrong
- Ran release sanity with --bypass-l10n-check to get the release going:
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u Callek \ -V 15.0 --branch mozilla-release --build-number 1 \ --release-config release-firefox-mozilla-release.py \ --release-config release-fennec-mozilla-release.py --products firefox,fennec \ --bypass-l10n-check --l10n-dashboard-version=15 localhost:9001
Bouncer Submitter
Builder failed because Firefox 15.0 already existed as a product in bouncer. (bug ???) Not sure how this happened...an errant staging run perhaps?
Callek removed the existing product from bouncer and re-ran the builder successfully.
Update Verify
The *last* chunk (6/6) update verify for each platform failed. Callek filed: bug 785661 - Update verify failed for TB/Firefox 15 versions < 12.0
- Nick did a temporary fix by pulling out all releases < 10 from the verify configs, and we retriggered all update verifies, which then succeeded.
Android Signing
Signing was delayed, presumably because Callek was working from the notes for 14.0.1 and we didn't sign the build #1 Android builds due to a respin.
# cltsign@signing1 cd ~/signing-work/mozharness hg pull -u && hg up -C -r default python2.6 ~/signing-work/mozharness/scripts/sign_android.py --config-file signing/android_mozilla-release.py --platform android
Reset reserved slaves
Coop initially set reserved_slaves to 2 on Saturday in case we needed to re-run #Update_Verify, and then set it to 0 on Monday morning.
# buildbot-master12 cd /builds/buildbot/build1/master/ echo 0 > reserved_slaves
Push to internal mirrors
Forced push_to_mirrors builder after verifying the the rsync excludes had landed in svn -> https://bugzilla.mozilla.org/attachment.cgi?id=655378
Adjusted index files:
# ffxbld@stage version=15.0 cd /pub/mozilla.org/firefox/releases/$version wget --no-check-certificate -O index.html https://bugzilla.mozilla.org/attachment.cgi?id=631778 sed -i -e "s/13/$version/g" index.html for dir in `find . -mindepth 1 -type d `; do cp -pv index.html $dir/; done
- CDN is also our internal mirror now, so the files went live to releases.m.o as well, but were missing win32
- It appears the CDN poll interval just hit while we were in the middle of pushing, about 15 minutes later we saw the index.html cover page and the win32/ directory existed.
- Ben filed bug 785965 to update docs.
Backupsnip
Ran backupsnip
# As ffxbld or tbirdbld@aus3-staging cd /opt/aus2/snippets/staging ~/bin/backupsnip Firefox-15.0-build1
Release Day
Push to Release Channel (for beta releases and release releases)
Pushed snippets:
# ffxbld@aus3-staging cd /opt/aus2/snippets/staging ~/bin/pushsnip Firefox-15.0-build1
Confirmed partials for 12.0, 13.0.1 and 14.0.1:
- 12.0: https://aus3.mozilla.org/update/1/Firefox/12.0/20120420145725/WINNT_x86-msvc/en-US/release/update.xml?force=1
- 13.0.1: https://aus3.mozilla.org/update/1/Firefox/13.0.1/20120614114901/WINNT_x86-msvc/en-US/release/update.xml?force=1
- 14.0.1: https://aus3.mozilla.org/update/1/Firefox/14.0.1/20120713134347/WINNT_x86-msvc/en-US/release/update.xml?force=1
Publish Fennec to the Play Store (for beta releases and release releases)
Push the files (for beta releases and release releases)
- Updated http://hg.mozilla.org/build/braindump/raw-file/tip/releases-related/push_fennec.sh to point at 15.0build1
- Ran the following on upload1:
curl -O http://hg.mozilla.org/build/braindump/raw-file/tip/releases-related/push_fennec.sh bash push_fennec.sh
Play Store (for beta releases and release releases)
https://wiki.mozilla.org/Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Play_Store
- Downloaded multi locale APK to my laptop.
- Went to android market
- Clicked on "Firefox"
- Clicked "APK Files"
- Clicked "Upload APK", uploaded 15.0 multi APK.
- Clicked "Save"
- Clicked "Deactivate" on 14.0.1
- Clicked "Deactivate" on 14.0.2
- Clicked "Activate" on 15.0
- Clicked "Save"
Publishing happened at 7:12am PT. Confirmed that the caches had cleared at 8:15am PT.
Update browser choice website
Filed bug 786270
Update XULRunner wiki page
Updated https://developer.mozilla.org/en/XULRunner and https://developer.mozilla.org/en/Gecko_SDK.
Update symlinks
As ffxbld@stage.mozilla.org:
cd /pub/mozilla.org/firefox/releases rm latest && ln -s 15.0 latest
Remove index.html files
As ffxbld@stage.mozilla.org:
find /pub/mozilla.org/firefox/releases/15.0 -name index.html -exec rm {} \;
DONE
