July 15th/16th, 2015

Attendees

• TW/EU session: No session
• US/EU: PBylenga, Oliverthor, mwargers, Pallavi, njpark, jlorenzo

Previous Action Items

• [CARRY OVER][atsai] Find external resources to port Gip to Gij

Status Highlights / Agenda

• Investigation for making the acceptance tests locally run on Aries is done.
  • See dependencies of https://bugzilla.mozilla.org/show_bug.cgi?id=1181535 for Aries specific issues
  • More details at: https://docs.google.com/spreadsheets/d/1gja3wnHS-5pif3kTGThWER0ZFgg5vFFt_TpXSM5Ehu4/edit#gid=452596172

• Docker Image

https://registry.hub.docker.com/u/silne30/fxos-docker

• Let's finish up the Q3 Automation goals

1. Support aries in automation (2 months) => Johan

• 95% of existing tests are able to be run locally
• We have a range of devices set up in Mountain and running the tests on daily basis

2. Switch automation to taskcluster (1 month) => Johan

• Every download job in Jenkins will take the lastest builds from TaskCluster

3. Triage the tests by confidence (3 months) => Martijn

• Keep maintaining the 2 suites between stable and not stable
• Figure out intermittent-cy criteria thanks to weekly triages, and the different type of fixes we can address
• Create Jenkins jobs for the stable and unstable runs

4. Test alerting (email escalation) => John D. (1 Month)

• Turn off the email alterting on b2g-inbound (nobody takes a look at it)
• Super sanity

5. Port half of the dogfood suite to automation (https://moztrap.mozilla.org/manage/cases/?filter-suite=846&pagesize=100&sortfield=created_ofn&filter-productversion=230&sortdirection=desc&pagenumber=1 ) (3 months) => Team

• • Add 1 new test/week/person

6. Put a strategy to deal with intermittent failures (3 months) =>Team

• • Verify which strategy is the best: 1 day/week to fix intermittent failures or 1 person/week
  • Implement weekly triage them

7. Create a base image (virtualization) for our gaia-test environments => John D. (1 Month)

• • Docker image hosted on docker-hub
  • Linux-based image that clones the gaia-repo + sets up the environment

8. [POC] Run a subset of reftest on device locally (2 months) => Martijn

• • Pick a subset of reftests for graphic testing
  • The results (failures) of the test run will be displayed on the output of the command line
  • Make them run locally

Extra:

• Automate RTL => NoJun
  • Leverage Engineering people to port Gip tests to Gij
  • Ask to EM to put some effort into migrating tests from Gip to Gij => John D., Johan
  • If they buy in, check if half a day per week of porting Gip tests to Gij help to reduce the gap.

Defered for a following quarter:

• Look at the existing disabled Gij tests and try to get them re-enabled?
• Memory regression tool with gaia test (need to check if raptor already does it)
• Locale-agnostic acceptance tests - Bug 1184600 - Make Gaia UI tests locale agnostic
• Automatically trigger an adhoc job once somebody changes the tests in tests/python/gaia-ui-tests

Week - Decisions & Action Items

• jlorenzo: Action plan for aries migration
• jlorenzo: Ask stas, tchevalier or gandalf about throwing errors in logcat when a string is truncated (ellipsis).