Firefox 3.0b3:BuildNotes
From MozillaWiki
Contents
Build Engineers
rhelmer
Bonsai queries
Was anything checked in after the "door closed"?
Last checkins:
- RC1 2008-02-04 17:20 PST
- RC2 - same pulldate as RC1, plus new mozconfig
- RC3 - 2008-02-06 03:30 PST (l10n only)
Bugs
Filed version bump bug in bug 409880
Tags
Module | Tag | Pull date |
cvsroot/mozilla | GECKO19b3_20080205_RELBRANCH | HEAD @ 2008-02-04 17:20 PST |
FIREFOX_3_0b3_RC1 | HEAD @ 2008-02-04 17:20 PST | |
FIREFOX_3_0b3_RC2 | HEAD @ 2008-02-04 17:20 PST | |
FIREFOX_3_0b3_RC3 | HEAD @ 2008-02-04 17:20 PST | |
FIREFOX_3_0b3_RELEASE | HEAD @ 2008-02-04 17:20 PST | |
l10n/l10n | GECKO19b3_20080205_RELBRANCH | HEAD @ 2008-02-04 17:20 PST |
FIREFOX_3_0b3_RC1 | HEAD @ 2008-02-04 17:20 PST | |
FIREFOX_3_0b3_RC2 | HEAD @ 2008-02-04 17:20 PST | |
FIREFOX_3_0b3_RC3 | HEAD @ 2008-02-06 03:30 PST | |
FIREFOX_3_0b3_RELEASE | HEAD @ 2008-02-06 03:30 PST |
Build data
Type | Build ID | SHA1 | Push date | Build machine |
[Windows installer] | 2008020514 | e4b3e4e8d43367f6f8552778f591f75af5af74f3 | 2008-02-05-14 | en-US:fx-win32-1.9-slave2 l10n:fx-win32-1.9-slave2 |
[Mac compressed] | 2008020511 | 9b9d1827796fcdf2a08ec13db50d24ac3e1dd422 | 2008-02-05-11 | en-US:fx-mac-1.9-slave2 l10n:fx-mac-1.9-slave2 |
[Linux compressed] | 2008020513 | fdabc2c88cecc0207d225c5fbbbb04c3446aa2c9 | 2008-02-05-13 | en-US:fx-linux-1.9-slave2 l10n:fx-linux-1.9-slave2 |
Notes
RC1
Bootstrap tag: RELEASE_AUTOMATION_M7_1
Setup before starting:
- used existing version bump bug to attached diff for bootstrap.cfg and master.cfg
- restarted master
- compare nightly and release mozconfig/tinder-config.pl files, merge any desired changes
- On fx-mac-1.9-slave2, fx-win32-1.9-slave2, fx-linux-1.9-slave2
- update tinderbox to RELEASE_AUTOMATION_M7_1
- FIXME update should be automatic - bug 397554
- remove the contents of /builds/verify/*
- FIXME should be automatically backed up or removed - bug 413178
- restart buildbot
- update tinderbox to RELEASE_AUTOMATION_M7_1
- On production-1.9-master
- remove the contents of /builds/updates/
- remove private and public areas
- rm -rf /data/cltbld/firefox-*
- FIXME should not be needed - bug 415970
- rm -rf /builds/tags/* - did not do this, plenty of space
- FIXME should not be needed - bug 415970
- in /home/ftp/pub/firefox/nightly/, remove all except the 3.x-candidates directory for the previous release
- rm -rf /home/ftp/pub/firefox/nightly/2007*
- FIXME should not be needed - bug 415970
- rm -rf /data/cltbld/firefox-*
- kick off buildbot (run as cltbld):
buildbot sendchange --username=rhelmer --master=localhost:9989 -m "Firefox 3.0 Beta 3 RC1" release
Prestage
- had to mkdir /home/cltbld/trunk-automation/prestage/build, not created automatically
- FIXME buildbot bug?
Tag
- noticed that linux VMs/slaves are running one hour ahead of Mac, Win32 slaves. This doesnt make sense because we thought they were all ntp-enabled.
Source
- had to chmod g-s /home/ftp/ again, looks like it's coming from prestage step
- rhelmer found we're setting it right here! http://mxr.mozilla.org/mozilla/source/tools/buildbot-configs/automation/production-1.9/master.cfg#102
- FIXED in bug 409880
- rhelmer found we're setting it right here! http://mxr.mozilla.org/mozilla/source/tools/buildbot-configs/automation/production-1.9/master.cfg#102
Build
- win32 failed unit tests
- Config::General was not installed
- installed it
- /builds/logs did not exist
- created it
- this was an oversight when MozillaBuild was upgraded
- Config::General was not installed
- mac failed repack
- af was not tagged, so had to checkout a tagged locale so CVS would cache it
- FIXME should just build all locales bug 415895
- af was not tagged, so had to checkout a tagged locale so CVS would cache it
- linux failed build
- AliveTest could not run, X server was refusing connections
- ran "xhost +"
- FIXME slaves should come up usable on boot - bug 417887
- ran "xhost +"
- AliveTest could not run, X server was refusing connections
Sign
- Build declared rc1 broken because of missed updates to mozconfig files as part of bug #407794. Manually bringing release and nightly mozconfig files back in sync, and starting rc2.
RC2
Bootstrap tag: RELEASE_AUTOMATION_M7_1
Setup before starting:
- used existing version bump bug to attached diff for bootstrap.cfg and master.cfg
- restarted master
- compare nightly and release mozconfig/tinder-config.pl files, merge any desired changes
- On fx-mac-1.9-slave2, fx-win32-1.9-slave2, fx-linux-1.9-slave2
- FIXME should be automatically backed up or removed - bug 413178
- restart buildbot
- noticed that linux VMs/slaves are running one hour ahead of Mac, Win32 slaves. This doesnt make sense because we thought they were all ntp-enabled.
- fixed manually and synced with hwclock
- kick off buildbot (run as cltbld):
buildbot sendchange --username=rhelmer --master=localhost:9989 -m "Firefox 3.0 Beta 3 RC2" release
Prestage
Tag
Source
Build
- set year incorrectly doing manual fix for Linux VMs
- fixed and restarted Linux slave and master
- master restart forced Win32 build restart
- Linux clobber not working; kernel update not reflected in bootstrap.cfg
- clobbered manually and restarted
- Mac and Win32 failed to upload breakpad symbols bug 415913
- fixed by rerunning "make uploadsymbols" with key set correctly
- FIXME this is due to the path to the key not set correctly in bootstrap.cfg - bug 417889
Sign
RC3
Bootstrap tag: RELEASE_AUTOMATION_M7_1
Calling this RC3, but really it's RC2 with extra locales and win32 re-signed.
Setup before starting:
- used existing version bump bug to attached diff for bootstrap.cfg and master.cfg
- also l10n_release mozconfigs
- comment out build and 3.0b3 clean steps
- restarted master
- landed new shipped-locales on relbranch
- forced mozconfigs to only build locales we want
- on master, copy rc2 en-US to rc3
mv rc2 rc3 cd rc3 rm -rf *win32* *partial.mar *source* *.log
- On fx-mac-1.9-slave2, fx-win32-1.9-slave2, fx-linux-1.9-slave2
- remove the contents of /builds/verify/*
- FIXME should be automatically backed up or removed - bug 413178
- restart buildbot
- kick off buildbot (run as cltbld):
buildbot sendchange --username=rhelmer --master=localhost:9989 -m "Firefox 3.0 Beta 3 RC2" release
Prestage
Tag
- manually pulled in locales:
- es-ES ja ja-JP-mac pa-IN
- used timestamp 2008-02-06 03:24 and merged to GECKO19b3_20080205_RELBRANCH
cvs co l10n cvs up -D 2008-02-06 03:30 PST es-ES ja ja-JP-mac cvs tag -b GECKO19b3_20080205_RELBRANCH es-ES ja ja-JP-mac
Source
Build
Sign
L10nVerify
Updates to Test Channel
- had to add DisableCompleteJump to rc list and re-push AUS config
- verify beta snippets versus betatest snippets
- only URL (download vs. stage) and appv ("3 Beta 3" vs. "3 Beta 3rc3")
- FIXME automate this in bug 409449
cd /opt/aus2/snippets/staging/20080207-Firefox-3.0b3/ find -type d -iregex '.*beta.*' | perl -nle '$a = $_; $a =~ s/beta/betatest/; system("diff -r -u $_ ../20080207-Firefox-3.0b3-test/$a");' 2>&1 | tee /tmp/beta.log
- verify beta snippets versus releasetest snippets
- no changes expected
- FIXME automate this in bug 409449
cd /opt/aus2/snippets/staging/20080207-Firefox-3.0b3/ find -type d -iregex '.*beta.*' | perl -nle '$a = $_; $a =~ s/beta/releasetest/; system("diff -r -u $_ ../20080207-Firefox-3.0b3-test/$a");'
- verify failed, PatcherConfig made beta like betatest not releasetest. Posted patch in bug 409880, recreated/repushed snippets
- ensure that new test snippets are the same as old
- no changes expected
diff -r 20080207-Firefox-3.0b3-test 20080208-Firefox-3.0b3-test
- verify beta snippets versus betatest snippets
- only URL (download vs. stage) and appv ("3 Beta 3" vs. "3 Beta 3rc3")
- FIXME automate this in bug 409449
cd /opt/aus2/snippets/staging/20080208-Firefox-3.0b3/ find -type d -iregex '.*beta.*' | perl -nle '$a = $_; $a =~ s/beta/betatest/; system("diff -r -u $_ ../20080208-Firefox-3.0b3-test/$a");' 2>&1 | tee /tmp/beta.log
- verify beta snippets versus releasetest snippets
- no changes expected
- FIXME automate this in bug 409449
cd /opt/aus2/snippets/staging/20080208-Firefox-3.0b3/ find -type d -iregex '.*beta.*' | perl -nle '$a = $_; $a =~ s/beta/releasetest/; system("diff -r -u $_ ../20080208-Firefox-3.0b3-test/$a");'
update Verify
- results uploaded to https://build.mozilla.org/logs/Firefox_3.0b3_RC3/
- FIXME need to manually remove bump, updateverify config bump only works for rc1 bug 415261
Stage
Sign Installers
Done manually.
- pull stage:/data/cltbld/firefox-3.0b2/batch1/stage-signed/
using these installer-signing-instructions here
- push signed bits back to same location on stage
- complete stage-merged:
# on stage cd /data/cltbld/firefox-3.0b3/ rsync -av batch1/mar/ stage-merged/ rsync -av batch1/stage-signed/ stage-merged/
- Create MD5 and SHA1 checksum files
# on stage cd /data/cltbld/firefox-3.0b3/stage-merged/ ~/bin/checksum-files .
- Fix permissions & ownership (on the two SUM files, and the detached sigs)
chown -R cltbld:firefox . chmod 644 *SUMS
Update Bouncer
- Manually add entries to download.mozilla.org
Final Verification
- manually verified files under /data/cltbld/firefox-3.0b3/stage-merged/ looked ok.
- verified that filename problems from 3.b2 have been fixed. bug 409394
Push to mirrors
- push the stage-merged directory to the releases area:
# on stage rsync -av /data/cltbld/firefox-3.0b3/stage-merged/ /home/ftp/pub/firefox/releases/3.0b3/
Updates to Beta Channel
- on Mac build machines, run quick updateverify test for releastest channel
cd /builds/verify/firefox-3.0b3/updates cat moz19-firefox-*.cfg > releasetest.cfg sed -i.bak 's/betatest/releasetest/' releasetest.cfg ./verify.sh -t releasetest.cfg 2>&1 | tee releasetest.log # upload result to build.m.o/logs
Should be all HTTP/200 OK results e.g.:
grep 'HTTP' releasetest.log | grep -v Found | grep -v 200
- Ensure that releasetest channel contents match beta channel contents
cd /opt/aus2/snippets/staging/20080208-Firefox-3.0b3 find -type d -iregex '.*beta.*' | perl -nle '$a = $_; $a =~ s/beta/releasetest/; system("diff -r -u $_ ../20080208-Firefox-3.0b3-test/$a");'
No difs found - which is correct.
- Once QA and Website gives formal "go"
# put snippets on beta $ sudo su - cltbld $ cd /opt/aus2/snippets/staging $ ~/bin/pushsnip 20080208-Firefox-3.0b3
- on Mac build machines, run quick updateverify test for beta channel
cd /builds/verify/firefox-3.0b3/updates cat moz19-firefox-*.cfg > beta.cfg sed -i.bak 's/betatest/beta/' beta.cfg ./verify.sh -t beta.cfg 2>&1 | tee beta.log # upload result to build.m.o/logs
Should be all HTTP/200 OK results e.g.:
grep 'HTTP' releasetest.log | grep -v Found | grep -v 200