Releases/Firefox 3.5/BuildNotes
Until we're about to ship this page will serve as a spot to put down a rough checklist of what needs to be done to ship.
Contents
- 1 Release Engineers
- 2 Tracking Bug
- 3 Notes
- 3.1 Generate major update
- 3.2 Stage and Rename files
- 3.3 Content verification
- 3.4 Ownernship/permission verification
- 3.5 Add index files
- 3.6 Tag and Source package regeneration
- 3.7 Bouncer Setup
- 3.8 Push to Mirrors
- 3.9 Update rsync modules
- 3.10 Trim mk builds out
- 3.11 Push Major Update
- 3.12 Symlink update
- 3.13 Remove index.html files
Release Engineers
bhearsum,nthomas
Tracking Bug
Notes
Generate major update
Updated the patcher and update verify configs.
# ssh cltbld@prometheus-vm.build.mozilla.org mkdir -p /builds/3.0.11-3.5rc3build2-major-update-test/snippets cd /builds/3.0.11-3.5rc3build2-major-update-test/snippets cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R8 -d patcher mozilla/tools/patcher cd patcher cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R8 -d MozBuild mozilla/tools/release/MozBuild export CVSROOT=:ext:ffxbld@cvs.mozilla.org:/cvsroot # build tools ./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R8 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log # download mars ./patcher2.pl --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log # FIXME - patcher needs to see that the MARs that it thinks are partials # are there or else it will not attempt to generate patchinfo cd temp/firefox ln -s 3.5rc3 3.0.11-3.5rc3 cd ../.. # Create partial patches and snippets ./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log
Separate out beta snippets
Skipped over this because beta and release will be pushed together, done like this.
Quick verify
Check that releasetest = beta = release.
cd temp/firefox/3.0.11-3.5rc3 # releasetest == beta find aus2.test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/aus2\.test/aus2/; system("diff -r -u $_ $a");' find aus2 -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/aus2/aus2.test/; system("diff -r -u $_ $a");'
# beta == release find aus2 -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta/release/; system("diff -r -u $_ $a");' find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; system("diff -r -u $_ $a");'
Push to AUS
cd /builds/3.0.11-3.5rc3build2-major-update-test/snippets/patcher/temp/firefox/3.0.11-3.5rc3 rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090625-Firefox-3.0.11-3.5rc3build2-MU-test rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090625-Firefox-3.0.11-3.5rc3build2-MU
Enable test snippets
# cltbld@aus2-staging ~/bin/backupsnip 20090625-Firefox-3.0.11-3.5rc3build2-MU-test ~/bin/pushsnip 20090625-Firefox-3.0.11-3.5rc3build2-MU-test
Update verify
Ran update verify on fx-{linux,mac,win32}-1.9-slave2 as follows:
mkdir -p /builds/verify/firefox-3011-35rc3build2-major-test/ cd /builds/verify/firefox-3011-35rc3build2-major-test/ hg clone http://hg.mozilla.org/build/tools # get patch to trawl "Only in" directories cd tools/release/common # on mac & linux curl -sL https://bugzilla.mozilla.org/attachment.cgi?id=367544 | patch -p3 # on win32 wget --no-check-certificate -O patch https://bugzilla.mozilla.org/attachment.cgi?id=367544 patch -p3 < patch # everyone ... cd ../updates platform=linux # or mac or win32 ./verify.sh -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log
Results Everything the same as the latest test run with one addition:
- mn removed amazondotcom.xml, answers.xml, eBay.xml and yahoo.xml bug 477989
Use empty snippets for mn and tr snippets
bug 500642 was filed after the MU update generation was done. We can't just regenerate the snippets because test snippets were already pushed, so we'll modify the MU snippets to serve nulls to these locales:
# cltbld @ aus2-staging cd ~/ # test snippets rsync -av /opt/aus2/snippets/staging/20090625-Firefox-3.0.11-3.5rc3build2-MU-test/ 20090625-Firefox-3.0.11-3.5rc3build2-MU-test/ cd 20090625-Firefox-3.0.11-3.5rc3build2-MU-test for i in `find . -name mn -or -name tr`; do for file in `find $i -type f`; do rm $file && touch $file; done; done # verification - should have no output cd .. diff --brief -Nar /opt/aus2/snippets/staging/20090625-Firefox-3.0.11-3.5rc3build2-MU-test/ 20090625-Firefox-3.0.11-3.5rc3build2-MU-test/ | grep -v '/\(tr\|mn\)/' # push them back to snippets/staging rsync -av 20090625-Firefox-3.0.11-3.5rc3build2-MU-test/ /opt/aus2/snippets/staging/20090625-Firefox-3.0.11-3.5rc3build2-MU-test-no-mn-or-tr/ # real snippets cd ~/ rsync -av /opt/aus2/snippets/staging/20090625-Firefox-3.0.11-3.5rc3build2-MU/ 20090625-Firefox-3.0.11-3.5rc3build2-MU/ cd 20090625-Firefox-3.0.11-3.5rc3build2-MU for i in `find . -name mn -or -name tr`; do for file in `find $i -type f`; do rm $file && touch $file; done; done # verification - should have no output cd .. diff --brief -Nar /opt/aus2/snippets/staging/20090625-Firefox-3.0.11-3.5rc3build2-MU/ 20090625-Firefox-3.0.11-3.5rc3build2-MU/ | grep -v '/\(tr\|mn\)/' # push them back to snippets/staging rsync -av 20090625-Firefox-3.0.11-3.5rc3build2-MU/ /opt/aus2/snippets/staging/20090625-Firefox-3.0.11-3.5rc3build2-MU-no-mn-or-tr/
Re-verify that releasetest = beta = release
cd /opt/aus2/snippets/staging # releasetest == beta find 20090625-Firefox-3.0.11-3.5rc3build2-MU-test-no-mn-or-tr -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/20090625-Firefox-3.0.11-3.5rc3build2-MU-test-no-mn-or-tr/20090625-Firefox-3\.0\.11-3\.5rc3build2-MU-no-mn-or-tr/; system("diff -r -u $_ $a");' find 20090625-Firefox-3.0.11-3.5rc3build2-MU-no-mn-or-tr -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/20090625-Firefox-3\.0\.11-3\.5rc3build2-MU-no-mn-or-tr/20090625-Firefox-3.0.11-3.5rc3build2-MU-test-no-mn-or-tr/; system("diff -r -u $_ $a");'
# beta == release find 20090625-Firefox-3.0.11-3.5rc3build2-MU-no-mn-or-tr -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta/release/; system("diff -r -u $_ $a");' find 20090625-Firefox-3.0.11-3.5rc3build2-MU-no-mn-or-tr -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; system("diff -r -u $_ $a");'
Push updated test snippets
# cltbld@aus2-staging ~/bin/backupsnip 20090625-Firefox-3.0.11-3.5rc3build2-MU-test-no-mn-or-tr ~/bin/pushsnip 20090625-Firefox-3.0.11-3.5rc3build2-MU-test-no-mn-or-tr
Stage and Rename files
We'll have to do something similar to Firefox 3.0 with at least the following caveats:
- Don't trim out contrib builds if they exist
- Work is done as ffxbld
# ffxbld@stage mkdir firefox-3.5 cd firefox-3.5 rsync -av --exclude='*partial.mar' --exclude=source --exclude=*SUMS /pub/mozilla.org/firefox/releases/3.5rc3/ . find contrib linux-i686 update -type f | perl -nle '$nf = $_; if ( $nf =~ s/rc3// ) { system("mv","-v",$_,$nf)}' find mac win32 -type f | perl -nle '$nf = $_; if ( $nf =~ s/\ RC\ 3// ) { system("mv","-v",$_,$nf)};' wget -O../firefox-3.5-shipped-locales http://hg.mozilla.org/releases/mozilla-1.9.1/raw-file/a625a31a0ad1/browser/locales/shipped-locales ~cltbld/bin/verify-locales.pl -m ~/firefox-3.5-shipped-locales -l bz2 ~cltbld/bin/checksum-files . chown -R ffxbld:firefox linux-i686 mac update win32 chmod 644 *SUMS
Content verification
find /pub/mozilla.org/firefox/releases/3.5rc3/ -type f | perl -nle '$a = substr($_,41); $a =~ s/rc3//; $a =~ s/\ RC\ 3//; system("diff","-q",$_,$a);' # Output: diff: contrib/solaris_tarball/firefox-3.5b99-3.5.en-US.opensolaris-i386.partial.mar: No such file or directory diff: source/firefox-3.5-source.tar.bz2: No such file or directory diff: source/firefox-3.5-source.tar.bz2.asc: No such file or directory diff: source/firefox-3.5.bundle: No such file or directory diff: source/firefox-3.5.bundle.asc: No such file or directory diff: update/linux-i686/af/firefox-3.5rc2-3.5.partial.mar: No such file or directory diff: update/linux-i686/ar/firefox-3.5rc2-3.5.partial.mar: No such file or directory ... diff: update/win32/zh-CN/firefox-3.5rc2-3.5.partial.mar: No such file or directory diff: update/win32/zh-TW/firefox-3.5rc2-3.5.partial.mar: No such file or directory Files /pub/mozilla.org/firefox/releases/3.5rc3/MD5SUMS and MD5SUMS differ Files /pub/mozilla.org/firefox/releases/3.5rc3/SHA1SUMS and SHA1SUMS differ
Everything above is expected: We don't want the partial, we regenerated *SUMS because of filename changes, and we haven't yet regenerated source packages
Ownernship/permission verification
find . ! -user ffxbld find . ! -group firefox find . -type f ! -perm 644 find . -type d -mindepth 1 ! -perm 755 ! -name 'contrib*' find . -type d -maxdepth 1 ! -perm 2775 -name 'contrib*'
contrib/solaris_* dirs are 2755, which is OK.
Add index files
for dir in `find . -type d -mindepth 1`; do cp -p index.html $dir/; done
Tag and Source package regeneration
Because we have so many l10n repositories to deal with it will be best to do this with the automation. We'll need to patch both release_config.py. [This patch] was used to do so.
We have to call it build2 due to the way relbranchoverride works (bug 500471 bug 500473). You don't need to touch the old* vars since Tag and Source don't use them, but you can if you want. The Source packages will be pushed to 3.5-candidates/build2. Those files should be quickly moved to ~ffxbld and disposed of or moved.
After landing the patch, manually kicked off the tag factory. Once that completed, kicked off source. Once that completed, ran the following to grab and then wipe out the files:
# ffxbld @ stage rsync -av /home/ftp/pub/firefox/nightly/3.5-candidates/ 3.5-candidates/ && rm -rf /home/ftp/pub/firefox/nightly/3.5-candidates
Then, to stage:
cd ~/firefox-3.5 rsync -av ~/3.5-candidates/build2/source/ source/
Then, pull the source/ dir in on the signing machine and generate a sig for it:
mkdir -p ~/signing-work/firefox-3.5 cd ~/signing-work/firefox-3.5 rsync -av -e 'ssh -i /home/cltsign/.ssh/ffxbld_dsa' ffxbld@stage.mozilla.org:~/firefox-3.5/source/ stage-unsigned/ rsync -av stage-unsigned/ stage-signed/ cd stage-signed/ sign-files . cd .. # should only push the asc here rsync -nav -e 'ssh -i /home/cltsign/.ssh/ffxbld_dsa' stage-signed/ ffxbld@stage.mozilla.org:~/firefox-3.5/source/ rsync -av -e 'ssh -i /home/cltsign/.ssh/ffxbld_dsa' stage-signed/ ffxbld@stage.mozilla.org:~/firefox-3.5/source/
Then back on stage, regenerate SUMS files:
cd ~/firefox-3.5 rm *SUMS ~cltbld/bin/checksum-files .
Bouncer Setup
Done
Push to Mirrors
- push the covering index.html in the top level (Monday 2240 EDT/1940 PDT)
# ffxbld@stage rsync -av /home/ffxbld/firefox-3.5/index.html /pub/mozilla.org/firefox/releases/3.5/ # 0045 EDT/2145 PDT rsync -av --exclude 'contrib*' --exclude '*.exe' --exclude '*.dmg' \ --exclude '*.bz2' --exclude '*.asc' --exclude '*.mar' --exclude '*.xpi' \ --exclude '*.bundle' \ /home/ffxbld/firefox-3.5/ /pub/mozilla.org/firefox/releases/3.5/
- push the stage-merged directory to the releases area (Tuesday 0100 EDT/Monday 2200 PDT)
rsync -av /home/ffxbld/firefox-3.5/ /pub/mozilla.org/firefox/releases/3.5/
Update rsync modules
As cltbld, replaced two references to 3.5rc3 with 3.5 in /pub/mozilla.org/zz/rsyncd-mozilla-current.exclude
Trim mk builds out
mk builds had a horrible bug that prevents updating so we decided not to ship it. bug 501344. First, trim the builds out of the releases dir
# ffxbld @ stage # start in the staging dir cd ~/firefox-3.5 find . -type d -iname mk -exec rm -r {} \; # update SUMS files sed -i -e '/^.*mk.*$/d' SHA1SUMS sed -i -e '/^.*mk.*$/d' MD5SUMS # then rsync over to releases rsync -av --delete ./ /pub/mozilla.org/firefox/releases/3.5/
Now, trim out the MU snippets
# cltbld @ aus2-staging # get rid of the live test snippets first cd /opt/aus2/incoming/3/Firefox/3.0.11 find . -maxdepth 3 -type d -wholename "*200906021*/mk*" -exec rm -r {} \; # and now get rid of the beta+release channel snippets in the staging dir rsync -av 20090625-Firefox-3.0.11-3.5rc3build2-MU-no-mn-or-tr/ 20090625-Firefox-3.0.11-3.5rc3build2-MU-no-mn-or-tr-or-mk/ cd 20090625-Firefox-3.0.11-3.5rc3build2-MU-no-mn-or-tr-or-mk/Firefox/3.0.11 find . -maxdepth 3 -type d -wholename "*200906021*/mk*" -exec rm -r {} \;
Push Major Update
Prep
time ~/bin/backupsnip 20090625-Firefox-3.0.11-3.5rc3build2-MU-no-mn-or-tr real 50m34.052s user 0m48.859s sys 2m17.473s
Had to re-run backupsnip because it was invalidated after we trimmed 3.0.11 mk snippets:
time ~/bin/backupsnip 20090625-Firefox-3.0.11-3.5rc3build2-MU-no-mn-or-tr-or-mk real 52m38.078s user 0m47.961s sys 1m35.855s
~/bin/pushsnip 20090625-Firefox-3.0.11-3.5rc3build2-MU-no-mn-or-tr-or-mk
Symlink update
Update the 'latest' symlink.
cd /pub/mozilla.org/firefox/releases rm latest-3.5 && ln -s 3.5 latest-3.5 rm latest && ln -s latest-3.5 latest
Remove index.html files
Once we are fully announced we can remove the index.html files:
cd /pub/mozilla.org/firefox/releases/3.5/ find . -name index.html | less find . -name index.html | xargs rm -v