QA/Automation/Projects/TPS
Overview
Our goal is to setup a CI system for TPS, which allows to run tests for each daily build of Firefox across all platforms.
At the current when developers push a change to mozilla central it will ran only a testrun.
We download the build and run the tests, it rans two tests one in desktop mode and one for mobile.
We want to cover all branches and all platforms(OSX Windows and Linux) the current CI is only a python script which listens for the pulse and runs a testrun, we want to use Jenkins so we would have an interface for management and queuing jobs
| Name: | TPS CI |
| Leads: | Henrik Skupin, Cosmin Malutan |
| Contributors: | n/a |
| Tracking Bug: | https://bugzilla.mozilla.org/show_bug.cgi?id=1013981 |
| Repository: | https://github.com/jonallengriffin/coversheet/ |
| Etherpad: | https://etherpad.mozilla.org/automation-tps |
Roadmap
| Date | Version | Details |
| June 30th, 2014 | - | CI system which can run tests for all versions of Firefox on all supported platforms |
List of Tasks
In the following table you can find all the tasks, which have to be done before we can sign-off from the next goal on the road map. The entries are sorted by action to be taken, so we have to take them mostly from top to down. Some parallelism should be possible.
| Task | Owner | Status |
| Request virtual machines for Windows (8.1), Linux (Ubuntu 14.04), and OSX (10.9) | Henrik | In progress |
| Update INSTALL.sh to work on all platforms, and can handle packaged tests 1009004 | Cosmin | Done |
| Create Python script to download build and packaged tests, and for running the tests | Cosmin | Done |
| Update Coversheet to use Jenkins similar to Mozmill CI | Cosmin | Done |
| Puppetize machines | Not started |
Documentation
As of now the TPS CI is run via a Python script on Ubuntu only. Also it only runs TPS tests for each check-ins on mozilla-central. Currently each test run takes about 25 minutes.
With our quarterly goal we want to increase the test coverage to support the following:
- Coverage of all desktop platforms: OS X (10.9), Linux (Ubuntu 14.04), and Windows (8.1)
- Coverage of all supported branches: mozilla-central, mozilla-aurora, mozilla-beta, mozilla-release, and mozilla-esr31 (later)
- If necessary system should be expandable to also cover landings on mozilla-inbound
- Jenkins as continuous integration system installed on the master node
- Mozilla Pulse for build broadcasts
Every time when a new build gets pushed to the FTP server, we will receive a message via Mozilla Pulse. Given the configuration only en-US builds will be obeyed for now, but we should be able to extend if requested later. The build information together with the locations for the build binary and the packaged tests will be passed to Jenkins, where a TPS job will be started. Jenkins itself will find a free slave and automatically starts a testrun on that box. Each box might be able to run several testruns in parallel.
