Releases/Firefox 3.6.4/BuildNotes-early-builds

From MozillaWiki
Jump to: navigation, search

Contents

Notes

Build 1

  • Clobbered 'mozilla-1.9.2' on pm01 with clobberer.
  • Axel landed l10n-changesets update
  • Landed release config update - which updates the 1.9.2 config for 3.6.4, the 1.9.1 config for the major update bits, and symlinks to make both of those configs active
  • Reconfiged pm01, pm02
    • Didn't succeed, had to restart
  • Started the automation with:
buildbot sendchange --username=bhearsum --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.4 build1" Go

Tag

No problems

Source

No problems

Build/Repack

  • Linux: no problems
  • Mac: no problems
  • Win32: no problems

Windows and Mac l10n schedulers didn't fire. Kicked them with:

hg clone http://hg.mozilla.org/build/tools
cd tools/buildbot-helpers
# After Mac en-US finished
python force_release_l10n.py -m http://production-master.build.mozilla.org:8010 -t FIREFOX_3_6_4_RELEASE -v -b releases/mozilla-1.9.2 -p macosx -n bhearsum
# After Windows en-US finished
python force_release_l10n.py -m http://production-master.build.mozilla.org:8010 -t FIREFOX_3_6_4_RELEASE -v -b releases/mozilla-1.9.2 -p win32 -n bhearsum

Removed en-US.xpi from the candidates directory.

Unit Tests

  • Linux:
    • First run: mochitest-browser-chrome hung - bug 542928, had to restart it
    • Second run: PASS, full green
  • Win32: PASS, full green
  • Mac: PASS, full green

Partner Builds

Pinged Kev to generate them (because of bug 558800 and bug 554321

Signing

Signing Instructions
No problems

l10n verification

halted, because of ta-LK problems

Updates

  • Updates blew out, failing to download the win32 ta-LK build.
    • After reviewing logs, discovered that force_release_l10n.py timed out when forcing it, and I never noticed the error.

Re-run is discussed further down.

win32 ta-LK build

Repack
  • Forced by filling out the win32_repack builder with the following values:
    • Branch: ta-LK
    • Properties:
      • en_revision: FIREFOX_3_6_4_RELEASE
      • l10n_revision: FIREFOX_3_6_4_RELEASE
      • locale: ta-LK
Signing

On cm-keymaster01:

PRODUCT=firefox
VERSION=3.6.4
BUILD=1
TAG=FIREFOX_3_6_4_RELEASE
REPO='releases/mozilla-1.9.2'
EMAIL=bhearsum@mozilla.com
KEYDIR=d:/2009-keys

cd ~/signing-work/${PRODUCT}-${VERSION}
rsync -av --exclude=*SUMS.asc --include=*.asc --include=en-US --include=unsigned --include=win32 --include=ta-LK --include=xpi --include=ta-LK.xpi --include=*.zip --include=*.exe --include=*.mar --include=*SUMS --include=update --exclude=* -e 'ssh -i /home/cltsign/.ssh/ffxbld_dsa' ffxbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build${BUILD}/ unsigned-win32-ta-LK-build${BUILD}/
rsync -av --exclude=*unsigned* unsigned-win32-ta-LK-build${BUILD}/ signed-win32-ta-LK-build${BUILD}/
# signcode signing
python sign-release.py --keydir ${KEYDIR} --product ${PRODUCT} -o signed-win32-ta-LK-build${BUILD}/ --first-locale en-US -p unsigned-win32-ta-LK-build${BUILD}/
rsync -av ./unsigned-win32-ta-LK-build${BUILD}/unsigned/win32/xpi ./signed-win32-ta-LK-build${BUILD}/win32

Paranoid verification:

mkdir paranoid-verify
cd paranoid-verify
mkdir original-en-US now-en-US ta-LK
cd original-en-US
7z x ../../signed-build1/win32/en-US/Firefox\ Setup\ 3.6.4.exe
cd ../now-en-US
7z x ../../signed-win32-ta-LK-build1/win32/en-US/Firefox\ Setup\ 3.6.4.exe
cd ../ta-LK
7z x ../../signed-win32-ta-LK-build1/win32/ta-LK/Firefox\ Setup\ 3.6.4.exe
# check the sums of firefox.exe -- they should all match
cd ..
md5sum original-en-US/nonlocalized/firefox.exe now-en-US/nonlocalized/firefox.exe ta-LK/nonlocalized/firefox.exe

Output:

923fb88cf87496df00cce934d2abf14f *original-en-US/nonlocalized/firefox.exe
923fb88cf87496df00cce934d2abf14f *now-en-US/nonlocalized/firefox.exe
923fb88cf87496df00cce934d2abf14f *ta-LK/nonlocalized/firefox.exe

So, ta-LK signing is OK.
Also checked that win32 installer is unchanged:

md5sum signed-build1/win32/en-US/Firefox\ Setup\ 3.6.4.exe signed-win32-ta-LK-build1/win32/en-US/Firefox\ Setup\ 3.6.4.exe
bdb1de22618824fafbbcbca9509c830a *signed-build1/win32/en-US/Firefox Setup 3.6.4.exe
bdb1de22618824fafbbcbca9509c830a *signed-win32-ta-LK-build1/win32/en-US/Firefox Setup 3.6.4.exe

Next, the SUMS files were manually edited to include the sums of the signed ta-LK files. Then, the SUMS files and ta-LK files were signed with new detached sigs as follows:

cd signed-win32-ta-LK-build${BUILD}/
../signing/sign-files .
cd ..
python ./verify-gpg.py --verbose --key-file=key-checkout/PUBLIC-KEY signed-win32-ta-LK-build1

Upload:

rsync -av -e 'ssh -i /home/cltsign/.ssh/ffxbld_dsa' signed-win32-ta-LK-build${BUILD}/ ffxbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build${BUILD}/

l10n verification (with ta-LK)

  • Set 'l10n_verification' on pm01 for mozilla-1.9.2 to clobber.
  • Started with "Force Build"
  • No problems

A few differences in crash reporter and other-OOPP looking strings.

Updates (with ta-LK)

  • Set 'updates' on pm01 for mozilla-1.9.2 to clobber.

Updates for this release are quite involved because we're doing two special things:

  • The usual 3.6* -> 3.6.4 beta and betatest snippets are being shipped first as an advertised major update, and then as a regular, background, minor update later.
  • We need to update 3.6.3plugin1 users to 3.6.4 with a regular, background, minor update.

We also have the usual major update refresh. This section tracks creation of all of the updates.

3.6* -> 3.6.4
  • Started 'updates' builder with "Force Build"
  • 'diff patcher config' step turned red because this run of updates didn't change anything - that's fine.
  • No problems
Switch 3.6* -> 3.6.4 beta/betatest snippets to major

Per release-drivers request, 3.6.3 -> 3.6.4 beta/betatest snippets are to be served as Advertised Major Updates for now:

# cltbld@aus2-staging
cd /opt/aus2/snippets/staging
# These ones will be used later in the release
rsync -av --exclude=*releasetest* --include=* 20100414-Firefox-3.6.4-test/ 20100414-Firefox-3.6.4-betatest-minor/
rsync -av --exclude=*releasetest* --include=* 20100414-Firefox-3.6.4-test/ 20100414-Firefox-3.6.4-betatest-major/
for i in `find 20100414-Firefox-3.6.4-betatest-major/ -type f`; do echo "updateType=major" >> $i; done
# Now, the beta channel ones
rsync -av 20100414-Firefox-3.6.4-beta/ 20100414-Firefox-3.6.4-beta-major/
for i in `find 20100414-Firefox-3.6.4-beta-major/ -type f`; do echo "updateType=major" >> $i; done
mv 20100414-Firefox-3.6.4-beta 20100414-Firefox-3.6.4-beta-minor

~/bin/backupsnip 20100414-Firefox-3.6.4-betatest-major
~/bin/pushsnip 20100414-Firefox-3.6.4-betatest-major
Update Verify

Run with the 'update_verify' builders on pm01.

  • Linux: PASS
  • Win32: PASS
  • Mac: PASS
3.5.9 -> 3.6.4 Major Update

Manually forced 'major_update' builder on pm02.

Major Update Verify
  • Results:
    • All platforms
      • No update found for 'mn' on any platform. This is expected, as we don't have mn in 3.6.2.
      • 'sl' on all platforms:
iskalnik-gov-si.xml
najdi-si.xml
yahoo.xml
    • Linux: PASS
    • Windows: PASS
      • All the normal problems (chk files), nothing else of note
    • Mac: PASS
3.6.3plugin1 -> 3.6.4 updates

Per release-drivers, these are to be minor, background updates for the beta channel.

Generation

Done on moz2-linux-slave02, in the existing 'updates' directory:

# cltbld@moz2-linux-slave02
cd /builds/slave/updates/build
wget https://bug558769.bugzilla.mozilla.org/attachment.cgi?id=439064
perl patcher2.pl --download --app=firefox --brand=Firefox --config=moz192-from-lorentz.patcher2.cfg
cd temp/firefox
ln -s 3.6.4 3.6.3plugin1-3.6.4
cd ../../
perl patcher2.pl --create-patches --partial-patchlist-file=patchlist.cfg --app=firefox --brand=Firefox --config=moz192-from-lorentz.patcher2.cfg
cd temp/firefox/3.6.3plugin1-3.6.4
rsync -av -e 'ssh -oIdentityFile=~/.ssh/cltbld_dsa' aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100414-Firefox-3.6.4-from-3.6.3plugin1-test
rsync -av -e 'ssh -oIdentityFile=~/.ssh/cltbld_dsa' aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100414-Firefox-3.6.4-from-3.6.3plugin1-beta

And then on aus2-staging, as cltbld:

cd /opt/aus2/snippets/staging
# backupsnip does nothing, becauseu 3.6.3plugin1 has never been updated to anything
~/bin/backupsnip 20100414-Firefox-3.6.4-from-3.6.3plugin1-test
~/bin/pushsnip 20100414-Firefox-3.6.4-from-3.6.3plugin1-test
Update Verify

On fx-{linux,mac,win32}-1.9-slave2, as cltbld:

hg clone http://hg.mozilla.org/build/tools tools
cd tools/release/updates
# linux
wget -Overify.cfg https://bug558769.bugzilla.mozilla.org/attachment.cgi?id=439051
# mac
wget --no-check-certificate -Overify.cfg https://bug558769.bugzilla.mozilla.org/attachment.cgi?id=439052
# win32
wget --no-check-certificate -Overify.cfg https://bug558769.bugzilla.mozilla.org/attachment.cgi?id=439053
bash verify.sh -c verify.cfg | tee verify.log

Results:

  • All Platforms
    • channel-prefs.js shows difference in update channel
      • Expected and OK because 3.6.3plugin1 was on 'beta' and 3.6.4 is on 'release', and MARs do not change the update channel
  • Win32: PASS
    • The usual chk file differences
  • Linux: PASS
  • Mac: PASS

Socorro product data updates

Added 3.6.4 and 3.6.5pre with the crash-stats admin page.

Push 3.6* -> 3.6.4 Advertised Update and 3.6.3plugin1 -> 3.6.4 minor updates to beta

# cltbld@aus2-staging
~/bin/backupsnip 20100414-Firefox-3.6.4-beta-major
~/bin/backupsnip 20100414-Firefox-3.6.4-from-3.6.3plugin1-beta
~/bin/pushsnip 20100414-Firefox-3.6.4-beta-major
~/bin/pushsnip 20100414-Firefox-3.6.4-from-3.6.3plugin1-beta

...and then we discovered that the releasenotes hadn't been pushed yet. Turned off updates with:

chmod 700 /opt/aus2/incoming/3

And then it was discovered that the snippets were using the wrong detailsURL. Fix is in the next section

Change detailsURL in 3.6* -> 3.6.4 advertised update snippets

# cltbld@aus2-staging
rsync -av 20100414-Firefox-3.6.4-beta-major/ 20100414-Firefox-3.6.4-beta-major-url-fixed/
cd 20100414-Firefox-3.6.4-beta-major-url-fixed/
find . -type f -exec sed -i -e s,/releasenotes/,/details/index.html, {} \;
cd ..
# verification - confirms that only the detailsUrl has changed
diff -Naur 20100414-Firefox-3.6.4-beta-major/ 20100414-Firefox-3.6.4-beta-major-url-fixed/ | grep -v '[+-][+-]' | grep '^[+-]' | grep -v detailsUrl
# betatest
rsync -av 20100414-Firefox-3.6.4-betatest-major/ 20100414-Firefox-3.6.4-betatest-major-url-fixed/
cd 20100414-Firefox-3.6.4-betatest-major-url-fixed/
find . -type f -exec sed -i -e s,/releasenotes/,/details/index.html, {} \;
cd ..
# verification - confirms that only the detailsUrl has changed
diff -Naur 20100414-Firefox-3.6.4-betatest-major/ 20100414-Firefox-3.6.4-betatest-major-url-fixed/ | grep -v '[+-][+-]' | grep '^[+-]' | grep -v detailsUrl

Snippets were then reverted to their previous state:

mkdir ~/3.6.4-bs
cd /opt/aus2/incoming/3/Firefox
mv 3.6.3 ~/3.6.4-bs
cd ..
tar -zvxf /opt/aus2/snippet/backup/20100416-1-pre-20100414-Firefox-3.6.4-beta-major.tar.gz

And then I pushed the new betatest snippets:

cd /opt/aus2/snippets/staging
~/bin/pushsnip 20100414-Firefox-3.6.4-betatest-major-url-fixed

And then updates were re-enabled:

chmod 775 /opt/aus2/incoming/3/

Change detailsURL in 3.6* -> 3.6.4 advertised update snippets (again)

# cltbld@aus2-staging
rsync -av 20100414-Firefox-3.6.4-beta-major/ 20100414-Firefox-3.6.4-beta-major-url-fixed-try2/
cd 20100414-Firefox-3.6.4-beta-major-url-fixed-try2/
find . -type f -exec sed -i -e s,/releasenotes/,/details/index.html, {} \;
cd ..
# verification - confirms that only the detailsUrl has changed
diff -Naur 20100414-Firefox-3.6.4-beta-major/ 20100414-Firefox-3.6.4-beta-major-url-fixed-try2/ | grep -v '[+-][+-]' | grep '^[+-]' | grep -v detailsUrl
# betatest
rsync -av 20100414-Firefox-3.6.4-betatest-major/ 20100416-Firefox-3.6.4-betatest-major-url-fixed-try2/
cd 20100416-Firefox-3.6.4-betatest-major-url-fixed-try2/
find . -type f -exec sed -i -e s,/releasenotes/,/details/index.html, {} \;
cd ..
# verification - confirms that only the detailsUrl has changed
diff -Naur 20100414-Firefox-3.6.4-betatest-major/ 20100416-Firefox-3.6.4-betatest-major-url-fixed-try2/ | grep -v '[+-][+-]' | grep '^[+-]' | grep -v detailsUrl

And then I pushed the new betatest snippets:

cd /opt/aus2/snippets/staging
~/bin/pushsnip 20100416-Firefox-3.6.4-betatest-major-url-fixed-try2

And then, after QA tested betatest, pushed the beta snippets:

~/bin/pushsnip 20100414-Firefox-3.6.4-beta-major-url-fixed-try2

Create build1 bouncer entries

Created Firefox-3.6.4build1 bouncer entries for linux, osx, and win platforms. These were pointed directly at the candidates directory, which Bouncer sees on dm-download02.mozilla.org

respin declared

Build 2

  • Marked pm01, mozilla-1.9.2, all builders for clobber with clobberer
  • Landed release config update
  • Tagged buildbot-configs, buildbotcustom, and tools with FIREFOX_3_6_4_BUILD2 and moved FIREFOX_3_6_4_RELEASE tag (to tip of default)
  • Reconfig pm01
    • didn't complete, master was very confused about state of mw32-ix-slave09/13; ended up closing tree and waiting for m-c jobs to finish, then restarting buildbot master
  • started the automation with
buildbot sendchange --username=nthomas --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.4 build2" Go

Tag

No problems.

Source

No problems.

Build/Repack

Got slow slaves on all three platforms, need bug 559205. Removed en-US.xpi from build2 directory on ftp server.

No problems.

Unit Tests

One failure in win32 mochitest-a11y tests:

... 
569 INFO Running chrome://mochikit/content/a11y/accessible/test_bug420863.html...
570 INFO TEST-PASS | chrome://mochikit/content/a11y/accessible/test_bug420863.html | td1: shouldn't have actions
571 INFO TEST-PASS | chrome://mochikit/content/a11y/accessible/test_bug420863.html | td2: should have one action
572 INFO TEST-PASS | chrome://mochikit/content/a11y/accessible/test_bug420863.html | td2: should have 'click' action
573 INFO TEST-PASS | chrome://mochikit/content/a11y/accessible/test_bug420863.html | td2: 'click' action hasn't been performed
574 INFO TEST-PASS | chrome://mochikit/content/a11y/accessible/test_bug420863.html | td3: should have one action
575 INFO TEST-PASS | chrome://mochikit/content/a11y/accessible/test_bug420863.html | td3: should have 'click' action
576 ERROR TEST-UNEXPECTED-FAIL | chrome://mochikit/content/a11y/accessible/test_bug420863.html | td3: 'click' action hasn't been performed
577 INFO TEST-KNOWN-FAIL | chrome://mochikit/content/a11y/accessible/test_bug420863.html | gAcc.numActions should not throw after click handler was removed!
579 INFO Running chrome://mochikit/content/a11y/accessible/test_cssattrs.html...
...

which looks like bug 515133. That landed in m-c after 1.9.2 had branched. Green on second run, so this is a known intermittent fault.

Partner Builds

Signing

Signing Instructions
No problems. Signature verification succeeded.

l10n verification

Some string changes in crash reporter dialogs for some locales. PASS.

Updates

3.6* -> 3.6.4
Generation

Failed with the following error:

diffing: chrome/th.jar
Traceback (most recent call last):
  File "/builds/slave/updates/build/mozilla/tools/update-packaging/make_incremental_updates.py", line 453, in <module>
    main(sys.argv[1:])
  File "/builds/slave/updates/build/mozilla/tools/update-packaging/make_incremental_updates.py", line 450, in main
    create_partial_patches(patches)
  File "/builds/slave/updates/build/mozilla/tools/update-packaging/make_incremental_updates.py", line 397, in create_partial_patches
    partial_filename = create_partial_patch(work_dir_from, work_dir_to, patch_filename, shas, PatchInfo(work_dir, ['channel-prefs.js','update.manifest','removed-files'],['/readme.txt']),forced_updates)
  File "/builds/slave/updates/build/mozilla/tools/update-packaging/make_incremental_updates.py", line 283, in create_partial_patch
    create_partial_patch_for_file(from_marfile_entry, to_marfile_entry, shas, patch_info)
  File "/builds/slave/updates/build/mozilla/tools/update-packaging/make_incremental_updates.py", line 189, in create_partial_patch_for_file
    exec_shell_cmd("mbsdiff "+from_marfile_entry.abs_path+" "+to_marfile_entry.abs_path+" "+patch_file_abs_path)
  File "/builds/slave/updates/build/mozilla/tools/update-packaging/make_incremental_updates.py", line 134, in exec_shell_cmd
    raise Exception, "cmd failed "+cmd
Exception: cmd failed mbsdiff /builds/slave/updates/build/tmppdlH79-fastmode/217/from/chrome/th.jar /builds/slave/updates/build/tmppdlH79-fastmode/217/to/chrome/th.jar /builds/slave/updates/build/tmppdlH79-fastmode/217/chrome/th.jar.patch

Couldn't reproduce the failed command on the build machine or locally -- could just be a fluke occurrence. Kicked a new updates run to try that idea out.

Failed again on a different locale, running 'mar': Extracting /builds/moz2_slave/updates/build/temp/firefox/3.6.3/ftp/firefox-3.6.3.te.win32.complete.mar to /builds/moz2_slave/updates/build/tmprFJHvz-fastmode/216/from Extracting /builds/moz2_slave/updates/build/temp/firefox/3.6.4/ftp/firefox-3.6.4.te.win32.complete.mar to /builds/moz2_slave/updates/build/tmprFJHvz-fastmode/216/to

Traceback (most recent call last):
 File "/builds/moz2_slave/updates/build/mozilla/tools/update-packaging/make_incremental_updates.py", line 453, in <module>
   main(sys.argv[1:])
 File "/builds/moz2_slave/updates/build/mozilla/tools/update-packaging/make_incremental_updates.py", line 450, in main
   create_partial_patches(patches)
 File "/builds/moz2_slave/updates/build/mozilla/tools/update-packaging/make_incremental_updates.py", line 389, in create_partial_patches
   extract_mar(to_filename, work_dir_to)
 File "/builds/moz2_slave/updates/build/mozilla/tools/update-packaging/make_incremental_updates.py", line 167, in extract_mar
   exec_shell_cmd("mar -x "+filename)    
 File "/builds/moz2_slave/updates/build/mozilla/tools/update-packaging/make_incremental_updates.py", line 134, in exec_shell_cmd
   raise Exception, "cmd failed "+cmd
Exception: cmd failed mar -x /builds/moz2_slave/updates/build/temp/firefox/3.6.4/ftp/firefox-3.6.4.te.win32.complete.mar

Kicked it through manually with the following on moz2-linux-slave16:

cd /builds/slave/updates/build
perl patcher2.pl --create-patches --partial-patchlist-file=patchlist.cfg --app=firefox --brand=Firefox --config=patcher-configs/moz192-branch-patcher2.cfg
find build/temp/firefox/3.6.3-3.6.4/ftp/firefox/nightly/3.6.4-candidates/build2 -type f -exec chmod 644 {} \;
find build/temp/firefox/3.6.3-3.6.4/ftp/firefox/nightly/3.6.4-candidates/build2 -type d -exec chmod 755 {} \;
rsync -av -e 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa' --exclude=*complete.mar update ffxbld@stage-old.mozilla.org:/pub/mozilla.org/firefox/nightly/3.6.4-candidates/build2/
rsync -av -e 'ssh -oIdentityFile=~/.ssh/cltbld_dsa' aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100503-Firefox-3.6.4
rsync -av -e 'ssh -oIdentityFile=~/.ssh/cltbld_dsa' aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100503-Firefox-3.6.4-test
rsync -av -e 'ssh -oIdentityFile=~/.ssh/cltbld_dsa' aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100503-Firefox-3.6.4-beta

And then on aus2-staging:

# cltbld @ aus2-staging
cd /opt/aus2/snippets/staging
~/bin/backupsnip 20100503-Firefox-3.6.4-test
~/bin/pushsnip 20100503-Firefox-3.6.4-test

Filed bug 563347 on the issue.

respin declared

Build 3

  • Marked pm01, mozilla-1.9.2, all builders for clobber with clobberer
  • Landed release config update
  • Tagged buildbot-configs, buildbotcustom, and tools with FIREFOX_3_6_4_BUILD3 and moved FIREFOX_3_6_4_RELEASE tag (to tip of default)
  • Reconfig pm01
  • started the automation with
buildbot sendchange --username=bhearsum --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.4 build3" Go

Tag

Failed out the first time because clobberer didn't work properly. Catlee discovered that tag and other builders don't set a 'branch' when reporting to clobberer, and hence, don't get clobbered if a 'branch' is specified when requesting it. To fix, requested a clobber on pm01 for all builders without a branch set, and restarted the automation.
No problems on the second try.

Source

No problems

Build/Repack

No problems anywhere. Removed en-US.xpi from stage.

Unit Tests

  • Linux: PASS
  • Windows: PASS
  • Mac: Conditional PASS
    • test_reactivate.html timed out. Known orange bug 528703

Partner Builds

Signing

Signing Instructions
No problems. Signature verification succeeded.

l10n verification

Some string changes in crash reporter dialogs for some locales. PASS.

Updates

3.6* -> 3.6.4
Generation

No problems.

Update Verify
  • Linux: PASS
  • Mac: Conditional PASS
    • Got one fail in the log, which I couldn't reproduce manually:
Using  https://aus2.mozilla.org/update/1/Firefox/3.6b2/20091108163852/Darwin_Universal-gcc3/en-US/betatest/update.xml?force=1
FAIL: no complete update found for https://aus2.mozilla.org/update/1/Firefox/3.6b2/20091108163852/Darwin_Universal-gcc3/en-US/betatest/update.xml?force=1
FAIL: download_mars returned non-zero exit code: 1
  • Windows: PASS
3.6.4build1 -> 3.6.4build3 beta channel
Generation

As cltbld@aus2-staging

cd /opt/aus2/snippets/staging
for i in beta test; do
  mkdir -p 20100503-Firefox-3.6.4-$i-build1-to-build3/Firefox
  rsync -av --exclude 'partial.txt' 20100503-Firefox-3.6.4-$i/Firefox/3.6.3/ 20100503-Firefox-3.6.4-$i-build1-to-build3/Firefox/3.6.4/
  mv 20100503-Firefox-3.6.4-$i-build1-to-build3/Firefox/3.6.4/Linux_x86-gcc3/20100401074458 20100503-Firefox-3.6.4-$i-build1-to-build3/Firefox/3.6.4/Linux_x86-gcc3/20100413154310
  mv 20100503-Firefox-3.6.4-$i-build1-to-build3/Firefox/3.6.4/Darwin_Universal-gcc3/20100401064631 20100503-Firefox-3.6.4-$i-build1-to-build3/Firefox/3.6.4/Darwin_Universal-gcc3/20100413152922
  mv 20100503-Firefox-3.6.4-$i-build1-to-build3/Firefox/3.6.4/WINNT_x86-msvc/20100401080539 20100503-Firefox-3.6.4-$i-build1-to-build3/Firefox/3.6.4/WINNT_x86-msvc/20100413172113
done
# backupsnip is a no-op
~/bin/backupsnip 20100503-Firefox-3.6.4-test-build1-to-build3
~/bin/pushsnip 20100503-Firefox-3.6.4-test-build1-to-build3
Generation (redux)

QA found that not having any <partial> blocks in the build1 -> build3 updates causes the software update dialog to completely hang if the update doesn't work. To fix, we need to add partial.txt files which point to the complete MARs.

cd /opt/aus2/snippets/staging
for i in `find 20100503-Firefox-3.6.4-test-build1-to-build3/ -type f`; do dir=`dirname $i` && cp $i $dir/partial.txt && sed -i -e 's/type=complete/type=partial/' $dir/partial.txt; done
for i in `find 20100503-Firefox-3.6.4-beta-build1-to-build3/ -type f`; do dir=`dirname $i` && cp $i $dir/partial.txt && sed -i -e 's/type=complete/type=partial/' $dir/partial.txt; done

And then they need pushing:

~/bin/backupsnip 20100503-Firefox-3.6.4-test-build1-to-build3
~/bin/pushsnip 20100503-Firefox-3.6.4-test-build1-to-build3
Update Verify

Manual checks by QA and bhearsum.

3.6.4build1 -> 3.6.4build3 release channel
Generation

We need to base these off of the beta channel ones because the release channel points to bouncer. As cltbld@aus2-staging

cd /opt/aus2/snippets/staging
mkdir -p 20100505-Firefox-3.6.4-build1-to-build3/Firefox
rsync -av --exclude 'partial.txt' 20100503-Firefox-3.6.4-beta/Firefox/3.6.3/ 20100505-Firefox-3.6.4-build1-to-build3/Firefox/3.6.4/
mv 20100505-Firefox-3.6.4-build1-to-build3/Firefox/3.6.4/Linux_x86-gcc3/20100401074458 20100505-Firefox-3.6.4-build1-to-build3/Firefox/3.6.4/Linux_x86-gcc3/20100413154310
mv 20100505-Firefox-3.6.4-build1-to-build3/Firefox/3.6.4/Darwin_Universal-gcc3/20100401064631 20100505-Firefox-3.6.4-build1-to-build3/Firefox/3.6.4/Darwin_Universal-gcc3/20100413152922
mv 20100505-Firefox-3.6.4-build1-to-build3/Firefox/3.6.4/WINNT_x86-msvc/20100401080539 20100505-Firefox-3.6.4-build1-to-build3/Firefox/3.6.4/WINNT_x86-msvc/20100413172113
for i in `find 20100505-Firefox-3.6.4-build1-to-build3/ -type f`; do dir=`dirname $i` && cp $i $dir/partial.txt && sed -i -e 's/type=complete/type=partial/' $dir/partial.txt; done
find 20100505-Firefox-3.6.4-build1-to-build3/ -type d -mindepth 5 -maxdepth 5 -exec mv {}/beta {}/release \;
Verification

Verify that the build1-build3 release snippets match the beta ones (which were already verified):

find 20100505-Firefox-3.6.4-build1-to-build3 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; $a =~ s/20100505-Firefox-3.6.4-build1-to-build3/20100503-Firefox-3.6.4-beta-build1-to-build3/; system("diff -r -u $_ $a");'
Push

Didn't ask for any QA on these because they're exactly the same as the beta snippets. Manually tested with a 3.6.4build1 build after pushing, and it updated to build3.

# cltbld@aus2-staging
~/bin/backupsnip 20100505-Firefox-3.6.4-build1-to-build3
~/bin/pushsnip 20100505-Firefox-3.6.4-build1-to-build3
3.6* -> 3.6.4 advertised update

it turns out that we actually don't want to do this for build3, so these snippets are never used.

Generation

Directory naming changed from 'major' to 'advertised' to obey the Update Terms page.

# cltbld@aus2-staging
cd /opt/aus2/snippets/staging
# These ones will be used later in the release
rsync -av --exclude=*releasetest* --include=* 20100503-Firefox-3.6.4-test/ 20100505-Firefox-3.6.4-betatest-advertised/
for i in `find 20100505-Firefox-3.6.4-betatest-advertised/ -type f`; do echo "updateType=major" >> $i; done
# Now, the beta channel ones
rsync -av 20100503-Firefox-3.6.4-beta/ 20100505-Firefox-3.6.4-beta-advertised/
for i in `find 20100505-Firefox-3.6.4-beta-advertised/ -type f`; do echo "updateType=major" >> $i; done

Quick verification,

# should only output a bunch of 'updateType=major'
diff -x releasetest -Naur 20100503-Firefox-3.6.4-test/ 20100505-Firefox-3.6.4-betatest-advertised/ | grep '^[+-]' | grep -v '\-\-\-' | grep -v '+++'
diff -Naur 20100503-Firefox-3.6.4-beta/ 20100505-Firefox-3.6.4-beta-advertised/ | grep '^[+-]' | grep -v '\-\-\-' | grep -v '+++'

Push the test snippets:

~/bin/backupsnip 20100505-Firefox-3.6.4-betatest-advertised
~/bin/pushsnip 20100505-Firefox-3.6.4-betatest-advertised
Update Verify
  • Linux: PASS
  • Mac: PASS
  • Windows: PASS

Push 3.6.4build1 -> 3.6.4build3 beta updates

~/bin/backupsnip 20100503-Firefox-3.6.4-beta-build1-to-build3
~/bin/pushsnip   20100503-Firefox-3.6.4-beta-build1-to-build3

Repush 3.6* -> 3.6.4 automatic updates to test channels

Mistakenly, we pushed 3.6* -> 3.6.4 advertised updates to the test channels. To correct this, the original test channel snippets were re-pushed:

~/bin/backupsnip 20100503-Firefox-3.6.4-test
~/bin/pushsnip 20100503-Firefox-3.6.4-test

respin declared

Build 4

  • Marked pm01, any release, all builders for clobber with clobberer
  • Landed release config update
  • Reconfig'ed pm
  • Tagged buildbot-configs, buildbotcustom, and tools with FIREFOX_3_6_4_RELEASE, FIREFOX_3_6_4_BUILD4
  • Kicked off tagging:
buildbot sendchange --username=bhearsum --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.4 build4" Go

Tag

No problems

Source

No problems with 'source' or 'xulrunner_source' builders

Build/Repack

  • Firefox:
    • Linux
      • build failed out with the following hg error when updating:
abort: data/.hgtags.i@b99fc90449e3: no match found!
  • These are usually one-off things; set a clobber for linux_build on pm and forced a new build
      • Forced l10n repacks with:
python force_release_l10n.py -m http://production-master.build.mozilla.org:8010 -t FIREFOX_3_6_4_RELEASE -v -b releases/mozilla-1.9.2 -p linux -n bhearsum
      • No other problems
    • Mac:
      • No en-US problems
      • pt-BR and fa failed out when cloning build/tools
        • Forced them with the web interface:
          • branch: $locale, en_revision: FIREFOX_3_6_4_RELEASE, l10n_revision: FIREFOX_3_6_4_RELEASE, locale: $locale
    • Win32: No problems
  • XULRunner
    • Linux: No problems
    • Mac: No problems
    • Win32: No problems

Checked that all locales were present for each platform (should have 74):

# ffxbld@stage
cd /home/ftp/pub/firefox/nightly/3.6.4-candidates/build4/linux-i686
[ffxbld@surf linux-i686]$ ls | grep -v xpi | wc -l
74
[ffxbld@surf linux-i686]$ cd ../mac
[ffxbld@surf mac]$ ls | grep -v xpi | wc -l
74
[ffxbld@surf mac]$ cd ../unsigned/win32
[ffxbld@surf win32]$ ls | grep -v xpi | wc -l
74

Also removed en-US.xpi from the root of the build4 dir.

Unit Tests

  • Linux: PASS
  • Mac: Conditional PASS:
    • Known random orange in test_reactivate.html bug 528703
    • Known random orange in test_browserGlue_distribution.js bug 523936
  • Win32: PASS

Partner Builds

Signing

Signing Instructions
No problems. Verify-signatures succeeded.

l10n verification

Some changes to crash reporter strings. PASS.

Updates

3.6* -> 3.6.4
Generation

No problems

Update Verify
  • Linux: PASS
  • Mac: PASS
  • Win32: PASS
3.6.4build3 -> 3.6.4build4 beta and release channels
Generation

As cltbld@aus2-staging

cd /opt/aus2/snippets/staging
for i in beta test; do
  mkdir -p 20100514-Firefox-3.6.4-$i-build3-to-build4/Firefox
  rsync -av --exclude 'partial.txt' 20100513-Firefox-3.6.4-$i/Firefox/3.6.3/ 20100514-Firefox-3.6.4-$i-build3-to-build4/Firefox/3.6.4/
  mv 20100514-Firefox-3.6.4-$i-build3-to-build4/Firefox/3.6.4/Linux_x86-gcc3/20100401074458 20100514-Firefox-3.6.4-$i-build3-to-build4/Firefox/3.6.4/Linux_x86-gcc3/20100503113541
  mv 20100514-Firefox-3.6.4-$i-build3-to-build4/Firefox/3.6.4/Darwin_Universal-gcc3/20100401064631 20100514-Firefox-3.6.4-$i-build3-to-build4/Firefox/3.6.4/Darwin_Universal-gcc3/20100503113315
  mv 20100514-Firefox-3.6.4-$i-build3-to-build4/Firefox/3.6.4/WINNT_x86-msvc/20100401080539 20100514-Firefox-3.6.4-$i-build3-to-build4/Firefox/3.6.4/WINNT_x86-msvc/20100503122926
done
# Create release channel snippets
mkdir -p 20100514-Firefox-3.6.4-build3-to-build4/Firefox
rsync -av --exclude 'partial.txt' 20100513-Firefox-3.6.4-beta/Firefox/3.6.3/ 20100514-Firefox-3.6.4-build3-to-build4/Firefox/3.6.4/
mv 20100514-Firefox-3.6.4-build3-to-build4/Firefox/3.6.4/Linux_x86-gcc3/20100401074458 20100514-Firefox-3.6.4-build3-to-build4/Firefox/3.6.4/Linux_x86-gcc3/20100503113541
mv 20100514-Firefox-3.6.4-build3-to-build4/Firefox/3.6.4/Darwin_Universal-gcc3/20100401064631 20100514-Firefox-3.6.4-build3-to-build4/Firefox/3.6.4/Darwin_Universal-gcc3/20100503113315
mv 20100514-Firefox-3.6.4-build3-to-build4/Firefox/3.6.4/WINNT_x86-msvc/20100401080539 20100514-Firefox-3.6.4-build3-to-build4/Firefox/3.6.4/WINNT_x86-msvc/20100503122926
find 20100514-Firefox-3.6.4-build3-to-build4/ -type d -mindepth 5 -maxdepth 5 -exec mv {}/beta {}/release \;
# Add fake partial snippets
for i in `find 20100514-Firefox-3.6.4-test-build3-to-build4/ -type f`; do dir=`dirname $i` && cp $i $dir/partial.txt && sed -i -e 's/type=complete/type=partial/' $dir/partial.txt; done
for i in `find 20100514-Firefox-3.6.4-beta-build3-to-build4/ -type f`; do dir=`dirname $i` && cp $i $dir/partial.txt && sed -i -e 's/type=complete/type=partial/' $dir/partial.txt; done
for i in `find 20100514-Firefox-3.6.4-build3-to-build4/ -type f`; do dir=`dirname $i` && cp $i $dir/partial.txt && sed -i -e 's/type=complete/type=partial/' $dir/partial.txt; done
# Push test snippets
~/bin/backupsnip 20100514-Firefox-3.6.4-test-build3-to-build4
~/bin/pushsnip 20100514-Firefox-3.6.4-test-build3-to-build4
Update Verify

Manual verifications by QA.

Push 3.6.4build3 -> 3.6.4build4 updates

~/bin/backupsnip 20100514-Firefox-3.6.4-beta-build3-to-build4
~/bin/backupsnip 20100514-Firefox-3.6.4-build3-to-build4
cd /opt/aus2/snippets/staging
~/bin/pushsnip 20100514-Firefox-3.6.4-beta-build3-to-build4
real    0m11.200s
user    0m0.012s
sys     0m0.492s
cd /opt/aus2/snippets/staging
time ~/bin/pushsnip 20100514-Firefox-3.6.4-build3-to-build4
real    0m14.793s
user    0m0.022s
sys     0m0.420s

Push 3.6* -> 3.6.4build4 beta channel updates

As cltbld@aus2-staging

cd /opt/aus2/snippets/staging
~/bin/backupsnip 20100513-Firefox-3.6.4-beta
~/bin/pushsnip 20100513-Firefox-3.6.4-beta

Build 5

  • Marked pm01, any release, all builders for clobber with clobberer
  • Landed release config and l10n-changesets updates
  • Tagged buildbot-configs, buildbotcustom, and tools with FIREFOX_3_6_4_BUILD5 and moved FIREFOX_3_6_4_RELEASE tag (to tip of default)
  • Reconfig pm01
  • started the automation with
buildbot sendchange --username=nthomas --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.4 build5" Go

Tag

No problems.

Source

No problems with 'source' or 'xulrunner_source' builders.

Build/Repack

  • Firefox
    • Linux: no problems
    • Mac: no problems
    • Windows: no problems
  • XULRunner
    • Linux: no problems
    • Mac: no problems
    • Windows: no problems apart from bug 565882 putting the files in unsigned/

Removed en-US.xpi from the root of the build5 dir on ftp.

Unit Tests

  • Linux: PASS
  • Mac: PASS
  • Win32: PASS

Signing

Signing Instructions
No problems. Verify-signatures succeeded.

Partner Builds

No problems in buildbot run but not signed yet.

l10n verification

Some changes to crash reporter strings. Looks OK.

Updates

3.6* -> 3.6.4
Generation

First run bombed out when stage failed to serve a file during the download step. Set a clobber for pm/mozilla-1.9.2/updates and forced another run. Was red because there was no diff in the patcher config but otherwise succeeded (this is not a real error as the config was bumped the first time round).

Update Verify

Forced, results:

  • Linux: PASS
  • Mac: PASS
  • Win32: PASS
3.6.4buildN -> 3.6.4build5 beta and release channels
Generation

Rewrote this a bit to make it easier to point all the previous builds at the latest.

As cltbld@aus2-staging

cd /opt/aus2/snippets/staging
for i in beta test; do
  mkdir -p 20100524-Firefox-3.6.4-$i-buildN-to-build5/Firefox
  # build4 --> build5
  rsync -a --exclude 'partial.txt' 20100523-Firefox-3.6.4-$i/Firefox/3.6.3/ 20100524-Firefox-3.6.4-$i-buildN-to-build5/Firefox/3.6.4/
  mv 20100524-Firefox-3.6.4-$i-buildN-to-build5/Firefox/3.6.4/Linux_x86-gcc3/{20100401074458,20100513140540}
  mv 20100524-Firefox-3.6.4-$i-buildN-to-build5/Firefox/3.6.4/Darwin_Universal-gcc3/{20100401064631,20100513134853}
  mv 20100524-Firefox-3.6.4-$i-buildN-to-build5/Firefox/3.6.4/WINNT_x86-msvc/{20100401080539,20100513144105}
  for j in `find 20100524-Firefox-3.6.4-$i-buildN-to-build5/ -type f`; do dir=`dirname $j` && cp $j $dir/partial.txt && sed -i -e 's/type=complete/type=partial/' $dir/partial.txt; done
  # build3 --> build5
  rsync -a 20100524-Firefox-3.6.4-$i-buildN-to-build5/Firefox/3.6.4/Linux_x86-gcc3/{20100513140540,20100503113541}/
  rsync -a 20100524-Firefox-3.6.4-$i-buildN-to-build5/Firefox/3.6.4/Darwin_Universal-gcc3/{20100513134853,20100503113315}/
  rsync -a 20100524-Firefox-3.6.4-$i-buildN-to-build5/Firefox/3.6.4/WINNT_x86-msvc/{20100513144105,20100503122926}/
  # build2 --> build5
  rsync -a 20100524-Firefox-3.6.4-$i-buildN-to-build5/Firefox/3.6.4/Linux_x86-gcc3/{20100513140540,20100502202326}/
  rsync -a 20100524-Firefox-3.6.4-$i-buildN-to-build5/Firefox/3.6.4/Darwin_Universal-gcc3/{20100513134853,20100502202401}/
  rsync -a 20100524-Firefox-3.6.4-$i-buildN-to-build5/Firefox/3.6.4/WINNT_x86-msvc/{20100513144105,20100502221517}/
  # build1 --> build5
  rsync -a 20100524-Firefox-3.6.4-$i-buildN-to-build5/Firefox/3.6.4/Linux_x86-gcc3/{20100513140540,20100413154310}/
  rsync -a 20100524-Firefox-3.6.4-$i-buildN-to-build5/Firefox/3.6.4/Darwin_Universal-gcc3/{20100513134853,20100413152922}/
  rsync -a 20100524-Firefox-3.6.4-$i-buildN-to-build5/Firefox/3.6.4/WINNT_x86-msvc/{20100513144105,20100413172113}/
done

# release snippets, copied from beta to get ftp.m.o instead of bouncer
rsync -a 20100524-Firefox-3.6.4{-beta,}-buildN-to-build5/
find 20100524-Firefox-3.6.4-buildN-to-build5/ -mindepth 5 -maxdepth 5 -type d -exec mv {}/beta {}/release \;
# Push test snippets
~/bin/backupsnip 20100524-Firefox-3.6.4-test-buildN-to-build5
~/bin/pushsnip   20100524-Firefox-3.6.4-test-buildN-to-build5
Update Verify

Manual verifications by QA.

Push 3.6.4buildN -> 3.6.4build5 updates

cltbld@aus2-staging;

~/bin/backupsnip 20100524-Firefox-3.6.4-beta-buildN-to-build5
~/bin/pushsnip 20100524-Firefox-3.6.4-beta-buildN-to-build5
~/bin/pushsnip 20100524-Firefox-3.6.4-buildN-to-build5

Push 3.6* -> 3.6.4build5 beta channel updates

cltbld@aus2-staging:

~/bin/backupsnip 20100523-Firefox-3.6.4-beta
~/bin/pushsnip 20100523-Firefox-3.6.4-beta

Add 3.6.4build5 Bouncer Entries

Added through bounceradmin.m.c

Build 6

  • Marked pm01, any release, all builders for clobber with clobberer
  • Landed release config update
  • Tagged buildbot-configs, buildbotcustom, and tools with FIREFOX_3_6_4_BUILD6 and moved FIREFOX_3_6_4_RELEASE tag (to tip of default)
  • Reconfig pm01
  • started the automation with
buildbot sendchange --username=bhearsum --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.4 build6" Go

Tag

No problems

Source

No problems with source or xulrunner_source

Build/Repack

  • Linux: No en-US problems
  • Mac: No en-US problems
  • Win32: No en-US problems

Network issues killed many repacks across all platforms. Ran the following to ensure all repacks were present:

# ffxbld@stage
cd /home/ftp/pub/firefox/3.6.4-candidates/build6/linux-i686
ls | grep -v xpi | wc -l
cd ../mac
ls | grep -v xpi | wc -l
cd ../unsigned/win32
ls | grep -v xpi | wc -l

Each should return '74'.

Removed en-US.xpi from the candidates directory.

Unit Tests

  • Linux: PASS
  • Win32: PASS
  • Mac: Two known-intermittent oranges on mochitest, conditional PASS.

Signing

Signing Instructions
No problems. Signature verification succeeded.

Partner Builds

l10n verification

Some crashreporter string changes; automatic log parsing failed due to a reconfig while the step was running (FIXME). Log looked fine, PASS.

Updates

3.6* -> 3.6.4
Generation
/usr/sbin/squidclient -m purge http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/3.6.3-candidates/build1/update/mac/lv/firefox-3.6.3.complete.mar
  • Failed out a fifth time with a different bad mar
    • Purged all 3.6.4-candidates and 3.6.3-candidates MARs from the cache and tried again
for i in `grep "3.6.4-candidates/build6" access.log | grep mar | sed -e 's/.*http/http/' | sed -e 's/\.mar.*/.mar/'`; do /usr/sbin/squidclient -m purge $i; done
  • 6th time go through generation and uploading MARs, failed to upload snippets because of an out-of-disk-space problem on aus2-staging.
    • Cleared some space by moving /opt/aus2/snippets/backup/200[89]* to ~cltbld/old-backups
    • Kicked through with:
# cltbld @ mv-moz2-linux-ix-slave09
cd /builds/slave/updates/build/temp/firefox/3.6.3-3.6.4
rsync -av -e 'ssh -oIdentityFile=~/.ssh/cltbld_dsa' aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100527-Firefox-3.6.4-test
rsync -av -e 'ssh -oIdentityFile=~/.ssh/cltbld_dsa' aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100527-Firefox-3.6.4-beta

And on aus2-staging as cltbld:

cd /opt/aus2/snippets/staging
~/bin/backupsnip 20100527-Firefox-3.6.4-test
~/bin/pushsnip 20100527-Firefox-3.6.4-test
Update Verify

Kicked with 'Force Build'.

# cltbld@aus2-staging
/usr/sbin/squidclient -m purge "http://stage-old.mozilla.org/pub/mozilla.org//firefox/releases/3.6.3/win32/bg/Firefox%20Setup%203.6.3.exe"

Re-ran update verify for win32, and it was a PASS this time.

3.6.4buildN -> 3.6.4build6 beta and release channels
Generation

This is very similar to what we did for build5, but this time we need to point all of these people to bouncer.

As cltbld@aus2-staging

cd /opt/aus2/snippets/staging
for i in beta test; do
  channel=$i
  if  $i == test ; then
    channel=betatest
  fi
  export $channel
  mkdir -p 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox
  # build5 --> build6
  rsync -a --exclude 'partial.txt' 20100527-Firefox-3.6.4/Firefox/3.6.3/ 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/
  mv 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/Linux_x86-gcc3/{20100401074458,20100523181754}
  mv 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/Darwin_Universal-gcc3/{20100401064631,20100523180910}
  mv 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/WINNT_x86-msvc/{20100401080539,20100523185824}
  # Change the bouncer links
  find 20100528-Firefox-3.6.4-$i-buildN-to-build6/ -type f -exec sed -i -e 's/3.6.4-complete/3.6.4build6-complete/' {} \;
  # Create "partials"
  for j in `find 20100528-Firefox-3.6.4-$i-buildN-to-build6/ -type f`; do dir=`dirname $j` && cp $j $dir/partial.txt && sed -i -e 's/type=complete/type=partial/' $dir/partial.txt; done
  # Change the update channel
  for j in `find 20100528-Firefox-3.6.4-$i-buildN-to-build6/ -mindepth 6 -maxdepth 6 -type d`; do newdir=`echo $j | sed -e "s/release$/$channel/"` && mv $j $newdir; done
  # build4 --> build6
  rsync -a 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/Linux_x86-gcc3/{20100523181754,20100513140540}/
  rsync -a 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/Darwin_Universal-gcc3/{20100523180910,20100513134853}/
  rsync -a 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/WINNT_x86-msvc/{20100523185824,20100513144105}/
  # build3 --> build6
  rsync -a 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/Linux_x86-gcc3/{20100523181754,20100503113541}/
  rsync -a 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/Darwin_Universal-gcc3/{20100523180910,20100503113315}/
  rsync -a 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/WINNT_x86-msvc/{20100523185824,20100503122926}/
  # build2 --> build6
  rsync -a 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/Linux_x86-gcc3/{20100523181754,20100502202326}/
  rsync -a 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/Darwin_Universal-gcc3/{20100523180910,20100502202401}/
  rsync -a 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/WINNT_x86-msvc/{20100523185824,20100502221517}/
  # build1 --> build6
  rsync -a 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/Linux_x86-gcc3/{20100523181754,20100413154310}/
  rsync -a 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/Darwin_Universal-gcc3/{20100523180910,20100413152922}/
  rsync -a 20100528-Firefox-3.6.4-$i-buildN-to-build6/Firefox/3.6.4/WINNT_x86-msvc/{20100523185824,20100413172113}/
done

# release snippets, copied from beta
rsync -a 20100528-Firefox-3.6.4{-beta,}-buildN-to-build6/
find 20100528-Firefox-3.6.4-buildN-to-build6/ -mindepth 5 -maxdepth 5 -type d -exec mv {}/beta {}/release \;
# Push test snippets
~/bin/backupsnip 20100528-Firefox-3.6.4-test-buildN-to-build6
~/bin/pushsnip   20100528-Firefox-3.6.4-test-buildN-to-build6
Update Verify

Manual checks by QA and bhearsum.

Push to mirrors

Because this isn't a final release we don't want to push the files anywhere, so instead, we do some editing of the exclude file. Patched /pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude as follows (as cltbld @ stage):

--- /pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude	2010-05-19 04:27:54.013758000 -0700
+++ rsyncd-mozilla-releases.exclude	2010-05-28 06:26:03.000000000 -0700
@@ -294,6 +294,10 @@
 + firefox
 + firefox/releases
 + firefox/releases/**
++ firefox/nightly
++ firefox/nightly/3.6.4-candidates/
++ firefox/nightly/3.6.4-candidates/build6
++ firefox/nightly/3.6.4-candidates/build6/**
 + mobile
 + mobile/releases
 + mobile/releases/**

Add 3.6.4build6 Bouncer Entries

Added 3.6.4build6 and 3.6.4build6-complete entries. The -complete ones are new in this build, and needed because the snippets now point to the mirrors.

Push 3.6.4buildN -> 3.6.4build6 updates

# Only need to run this once, because it backs up all channels for any version it finds
~/bin/backupsnip 20100528-Firefox-3.6.4-beta-buildN-to-build6
~/bin/pushsnip 20100528-Firefox-3.6.4-beta-buildN-to-build6
~/bin/pushsnip 20100528-Firefox-3.6.4-buildN-to-build6

Push 3.6* -> 3.6.4build6 beta channel updates

~/bin/backupsnip 20100527-Firefox-3.6.4-beta
~/bin/pushsnip 20100527-Firefox-3.6.4-beta

Update 3.6.4build6 release users to beta

bug 570797 covers this - the motivation is to catch early adopter types who downloaded a 3.6.4 installer, where the release channel isn't a good description of the risk they're willing to accept.

Mar Generation

We replace the existing channel-prefs.js with

// Firefox 3.6.4build6 user converted from release to beta (https://bugzilla.mozilla.org/show_bug.cgi?id=570797)
pref("app.update.channel", "beta");

We need two mar files

mkdir -p target/defaults/pref/
cp channel-prefs.js target/defaults/pref/

rsync -a target/ target-mac/
mkdir -p target-mac/Contents/MacOS
mv target-mac/defaults target-mac/Contents/MacOS/

Set up the scripts by copying tools/packaging/{make_full_update.sh,common.sh} from a mozilla-central checkout. Patch common.sh so that channel-prefs.js isn't excluded from the mar. Ensure you have a copy of the mar utility on your path.

Create the mars and get useful info

./make_full_update.sh update-channel-beta.mar target/
./make_full_update.sh update-channel-beta-mac.mar target-mac/

Publish both of those to build6/update/beta.

Snippet Generation

Done as cltbld on aus2-staging:

cd ~
wget -Ocreate-build6-release-to-beta-snippets.sh https://bugzilla.mozilla.org/attachment.cgi?id=450117
chmod +x *.sh
cd /opt/aus2/snippets/staging
~/create-build6-release-to-beta-snippets.sh

Test snippets were pushed right after generation:

~/bin/backupsnip 20100609-Firefox-3.6.4-release-to-beta-test
~/bin/pushsnip 20100609-Firefox-3.6.4-release-to-beta-test
Update Verify

Done on fx-linux-1.9-slave2, fx-mac-1.9-slave2, and win32-slave45. Configs are in bug 570797

hg clone http://hg.mozilla.org/build/tools
cd tools/release/updates
bash verify.sh -c verify.cfg > verify.log
  • Linux
    • 'be' failed first time because of NFS cache issues. Passed on second run.
    • PASS
  • Mac
    • PASS
  • Win32
    • 'sl' failed first time because of NFS cache issues. Passed on second run.
    • PASS
Push to release channel

As cltbld on aus2-staging:

~/bin/backupsnip 20100609-Firefox-3.6.4-release-to-beta
~/bin/pushsnip 20100609-Firefox-3.6.4-release-to-beta

Push release channel updates

not done yet Before release channel updates to 3.6.4build6 are pushed we need to disable the 3.6.4build6 release -> beta switch. First push the test null snippets:

~/bin/backupsnip 20100609-Firefox-3.6.4-release-to-beta-test-null
~/bin/pushsnip 20100609-Firefox-3.6.4-release-to-beta-test-null

Then do some spotchecks, then push the real null snippets:

~/bin/backupsnip 20100609-Firefox-3.6.4-release-to-beta-test
~/bin/pushsnip 20100609-Firefox-3.6.4-release-to-beta-test