BuildbotTalos

	THIS PAGE IS PROPOSED FOR DELETION	2018-08-06
	Buildbot is being retired in bug 1488913 and all relevant Talos content has been moved to TestEngineering/Performance/Talos.

Talos Buildbot

Installation

Please see the buildbot reference installation docs.

Notes and differences:

• Python24 is setup on the machines already along with the win32 extensions (possibly using ActiveState installer?)
• Zope and Twisted are installed through the Twisted installer at http://tmrc.mit.edu/mirror/twisted/Twisted/2.4/Twisted_NoDocs-2.4.0.win32-py2.4.exe
• There is no ctlbld user on the slave machines.
• This setup is not currently using Botrunner

Slave setup can be found on ReferencePlatforms/Test/WinXP.

Operation

The talos buildbot masters currently live on qm-buildbot01. There are 3 masters setup under the /build directory: perfmaster, baseliner and branchliner. The only one of these three that runs regularly is "perfmaster" and it's configured to run the windows branch and trunk hourly performance runs. It is publishing a waterfall to http://qm-buildbot01:2004 (note that you will need access - vpn or otherwise - to the MoCo Colo facility).

The other two masters, baseliner (http://qm-buildbot01:2003) and branchliner (http://qm-buildbot01:2002) are for running baselines on trunk and branch. These are brought up as needed. Configuration of the two baseliners are done through their respective directory's master.cfg file. The area of interest is the 'sources' entry in BuildbotConfig. The ManifestDirectoryStream should be created with a url parameter pointing to the download location, (e.g., http://archive.mozilla.org/pub/firefox/nightly/), a manifestURL entry, (e.g., http://people.mozilla.org/~rcampbell/manifests/trunk-mod7-2) and a dateRange of builds. A branch identifier may also be included.

Because the ManifestDirectoryStream will just read through a file each time a poll event is fired, a pollInterval should be set to a value greater than the length of a talos run. Typical values range from 25 - 125 minutes depending on the pageset and build used.

Note that the results from the baseliners should only be run once. Multiple runs of the same build will corrupt the graphserver.

Slave machines are allocated according to Buildbot/Talos/Machines.

Future Development

To bring up additional ports, care should be taken not to disrupt the operation of slaves that are currently "getting it done". I would suggest a phased solution, creating a new master for additional platforms with the intention of folding additional masters back into the main "perfmaster" configuration when stabilized. There are enough variables in these setups that it can take a few tries to get everything right in Talos and on the buildbot end.

Current Issues

• ManifestDirectoryStreamer needs parameters for non-windows platforms
• Startup automation
  • Depending on task, may be undesireable to automatically startup particular masters and/or slaves
• PerfConfigurator.py checked in with Talos code
• Buildbot master should probably be moved to a VM for easier backup and restoration/migration - qm-rhel02?
• buildbot should clean up after faulty runs better
  • better dependent build steps