Template:BuildNotes

From MozillaWiki
Jump to: navigation, search


Build Engineers

{{{UserName}}} - Tracking bug: bug {{{TrackingBug}}}

Notes About Releasing

Please update the Notes Template and the Release:Primer for future releases (bug fixes, changes to automation) as needed

Bugs hit

Enter any bugs pre-existing or newly discovered and filed during the release:

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


{{{BugsHitComments}}}

Signed-off Revision(s)

{{{MozillaRevUrl}}}
{{{CommRevUrl}}}

L10N changesets

Instructions on how to get them

  • {{{Product}}}: {{{L10nChangesetsUrl}}}
  • Fennec: {{{L10nJSONUrl}}}

Tags

Manually tag the automation code, then record the generated tags below. (details)

Build # Branch, Tags Changeset
MOBILE80_#_RELBRANCH, FENNEC_VERSION_RELEASE, FENNEC_VERSION_BUILD1 #
GECKO80_#_RELBRANCH, FIREFOX_VERSION_BUILD1 FIREFOX_VERSION_RELEASE #

Build data

{{{Product}}}

Type Build ID Build machine Time to build
{{{BuildID1Desc}}} {{{BuildID1}}} {{{BuildID1Machine}}} {{{BuildID1Time}}}
{{{BuildID2Desc}}} {{{BuildID2}}} {{{BuildID2Machine}}} {{{BuildID2Time}}}
{{{BuildID3Desc}}} {{{BuildID3}}} {{{BuildID3Machine}}} {{{BuildID3Time}}}
{{{BuildID4Desc}}} {{{BuildID4}}} {{{BuildID4Machine}}} {{{BuildID4Time}}}
{{{BuildID5Desc}}} {{{BuildID5}}} {{{BuildID5Machine}}} {{{BuildID5Time}}}


Preparation

  • Did you ship the L10n milestone?
  • Is this is a *.0 beta -> release? If so, you may also need to update the whitelist of nightly vs release mozconfig differences for release sanity.

Set reserved slaves

You need ~5 each of linux, linux64, win32 (w32), and macosx64 (moz2-darwin10). Doing this 2-3 hours in advance should speed things up. Just set the desired number of reserved slaves in /builds/buildbot/build1/master/reserved_slaves.

Preparing to start Automation

detailed instructions

  • Go and make sure that there are L10n milestones for your release. Otherwise create them
  • Set clobbers for the appropriate masters. Doing this 24-48 hours in advance should speedup the build.
  • For build1 only, create a symlink for your $version (eg 11.0b4, 10.0.2esr, 10.0.2, 3.6.27) (until bug 725839 is resolved)
# ffxbld@stage, or tbirdbld and use thunderbird in path
cd /pub/mozilla.org/firefox/nightly/
mkdir ../candidates/$version-candidates
ln -s ../candidates/$version-candidates $version-candidates
  • Update l10n changesets for desktop, mobile
  • Land automation configs (are you or buildduty going to reconfig? great, merge. otherwise double-land only your configs)
  • Tag buildbot-configs, buildbotcustom, & tools with build & release tags. (details)
    • Sample command for Firefox 10.0 beta 5 build 1:
hg tag -f FIREFOX_10_0b5_RELEASE FIREFOX_10_0b5_BUILD1
hg tag -f FENNEC_10_0b5_RELEASE FENNEC_10_0b5_BUILD1
# bash shortcut: hg tag -f {FIREFOX,FENNEC}_10_0b5_{RELEASE,BUILD1}
  • update and reconfigure the master

XXX: add notes on how to update the master 

cd /builds/buildbot/build1/master
source ../bin/activate
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u armenzg -V 13.0b1 --branch mozilla-beta --build-number 1 \
    --release-config release-firefox-mozilla-beta.py --release-config release-fennec-mozilla-beta.py --products firefox,fennec  \
    --dryrun localhost:9001
  • start automation ← monitor progress on buildbot (e.g. beta)
    • the same command as above but without the "--dryrun"
  • reconfigure other masters (or ask the buildduty person to do this) if you have done a merge (and not double-land) to production

E-mail Metrics

Use the address "metrics-alerts < AT > mozilla < PERIOD > org". Note for first-time-releasers: your email will get held for moderator approval - that is expected. If it happens more than once, get help on #metrics.

For 3.6.25 you would send 

Firefox,firefox,firefox,3.6.25,3.6
Firefox,firefox,firefox,3.6.26pre,3.6

For non-3.6 releases, we need to include Fennec in the email if we build Fennec.

For 9.0b2 it would be 

Firefox,firefox,firefox,9.0b2,9.0
Fennec,firefox-mobile,fennec,9.0b2,9.0

And for 9.0 final it's 

Firefox,firefox,firefox,9.0,9.0
Fennec,firefox-mobile,fennec,9.0,9.0

ESRs look like this: 

Firefox,firefox,firefox,10.0.3,10.0
Firefox,firefox,firefox,10.0.3esr,10.0

E-mail mirrors@mozilla.org

Not necessary for rapid release betas or ESR builds, as the files only go as far as the 'internal mirrors'.

They want to know approximately what time push to external mirrors will occur. Ideally this is a 24hr notice to them, with the estimated time. Subject: {{{Product}}} {VERSION1} [and {VERSION2}] coming to mirrors on {DATE} Body: We're planning on pushing our Firefox {VERSION1} [and {VERSION2}] release(s) to mirrors 24 hours before {DATE}.

  1. If this is a chemspill, please mention that in the email as it will mean pinging
  2. for CDN to be enabled when the push to internal mirrors happens

If you believe these releases or these dates will cause any problems, please notify release at mozilla.com. If you have any problems with mirror status/weights/etc, please notify mirror-submissions at mozilla.org.

Edit rsync exclude files

After getting svn setup (auth req'd) edit files/rsync/rsyncd-mozilla-releases.exclude so that it excludes the current release, get review and deploy. This prevents external mirrors from picking up the new release prematurely. Since Thunderbird doesn't inject index.html files into the file hierarchy, adding "- thunderbird/releases/13.0/**" (adjust for version) to the top of the file is all that's needed.

For rapid release betas and ESR builds, this is not necessary as there's already an exclude that matches them.

Tag

Bouncer Submitter

Source

Build

Firefox

Firefox repacks

Fennec

Fennec repacks

Unittests / Talos

details

  1. Look to see that they ran
  2. Document any oranges (per platform) for unittests - if possible, try to confirm it's known/random
  3. Make sure there's no red/failures that we need to have a dev look at

XULRunner

Source

Build

Push to Mirrors

Partner Repack

Firefox

Fennec

Android Signing

Instructions in Signing: Android Signing

Now run on signing1.

L10N verify

Checksums

Updates

Update verify

Major Update

Major update generation is not kicked off automatically. You should only create them once the final builds for the "latest" build are ready. For instance, the 3.6.26 builds got created a week before the 10.0 release but the "final" 10.0 builds were not created after few days.

Follow instructions from the Major update documentation.

Major Update verify

Reset reserved slaves

Needs to be reset to 0.

Check permissions / AV scan

It is supposed to be triggered automatically after updates are done. File a bug if it doesn't. details

Push to internal mirrors

This is done by automation for rapid release betas. Note: if you have to reconfig the release buildbot master in during the release, the uptake monitoring will fail (bug 629648). That means you'll need to manually send the "ready for releasetest" emails when you believe things are "good enough". Refer to a prior release for email details.

For other releases, this should be done manually at this point, after check permissions / AV scan have completed.

Mirrors Policy

Final verification

Record receipt of "completed_final_verification" emails (1 per platform)

Going to Beta

NOTE: Skip to "Before Release Day" for beta releases

Run backupsnip

NOTE: Remember to do this at least 2 hours ahead of the expected "go to beta" email. details

Remember to ssh to aus3-staging using your short LDAP name and then run sudo su - ffxbld once on. If you intend to use screen, initiate your screen session while you are still yourself.

Push to beta

Wait for "go" from release managers
For Firefox:

  • Publish (and record run time here): details
  • on completion, email r-d that VERSION updates are live on the beta channel

For Android:

  • publish on Google Play (nee Android Marketplace), using instructions below
  • on completions, email r-d that VERSION is live on Google Play

Wait for confirmation from QA that each of the above releases is okay, then close the tracking ticket.

Before Release Day

Assuming this is not a chemspill.

Run backupsnip for release snippets

Do this the night before release day as it can take up to two hours.

Push index file to mirrors

Push to external mirrors

If this is a chemspill you will want to ping justdave (or other sysadmin) in #release-drivers and ask them to enable CDN.

  • Remove previously added rsync exclusion from stage.mozilla.org:/pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude
  • Replace old version with the current one in stage.mozilla.org:/pub/mozilla.org/zz/rsyncd-mozilla-current.exclude

See details

Release Day

Publish Fennec to the Play Store (for beta releases and release releases)

Push the files (for beta releases and release releases)
Play Store (for beta releases and release releases)

IMPORTANT - Make sure to follow instructions!

  • download both multi apks. These have the same name so save them to separate directories.
  • visit https://market.android.com/publish
  • choose "Firefox Beta" or "Firefox"
  • select the "APK Files" tab and choose "Upload APK"
    • choose the android multi apk that you downloaded and hit "upload"
      • NOTE: the progress bar does not seem to show progress (at least on Google Chrome)
    • when it finishes uploading, verify VersionCode is the build date (almost anyway, the market reports an hour later than our buildID)
    • hit "Save"
  • If you're also uploading XUL Fennec, repeat the same steps for the android-xul multilocale apk
  • activate new android apk (You should see an Error message since both apk's are active)
  • deactivate old android apk (You should not see the Error message anymore)
  • If you're also activating XUL Fennec, repeat those steps for the android-xul multilocale apk.
  • hit "Save"
  • If this is a beta 1, go to "Product Details" tab -> Recent Changes, change the url to http://www.mozilla.com/en-US/mobile/{VERSION}beta/releasenotes/
    • and hit "Save"

Update XULRunner wiki page

For major releases or chemspills, update the links on:

Also for older releases:

Check Throttling

See http://people.mozilla.com/~nthomas/update-watch/release/ for example AUS links

Some links to check:

Push to Release Channel (for beta releases and release releases)

Once there is enough uptake and we get "go" from release driver.

Update symlinks

Once we're signed off on the release channel.

Remove index.html files

If you created them earlier to hide the release.

Update browser choice website

Retrieved from "https://wiki.mozilla.org/index.php?title=Template:BuildNotes&oldid=440518"