Mobile/Testing/05 25 16

Status reports

A Team

• android-4-3-armv7-api15 Total failure rate: [2.29%]
  • last week 2.77%
  • total failure excluding retries 1.54%

Data fetched for: 2016-05-18 14:35 — 2016-05-24 12:21

• TC Android builds (and some local build variants) broken - bug 1275016
• gbrown on pto May 30 until London

Autophone

• Nexus 6P devices for developers
  • gwright mentioned he doesn't have a Nexus 6P to use. I imagine neither does esawin or others. Can mobile get them phones?
• bug 1275074 Autophone - try is busted
  • Appears to be fixed by bug 1275047 Try build txt file contains mozilla-central revision instead of try revision
• bug 1268571 Autophone - support debug builds
  • ready to land. will support opt perf test and opt, debug unit tests for now
• bug 1275272 Autophone - nexus-6p-8 usb disconnected 2016-05-24
  • Repeated issues with Nexus 6P running these tests. Disabled rca on nexus-6p-8 with the hope the pattern I saw in the logs implicates it in horking the device.
• bug 1126448 Autophone - improve phonedash graph UI for try builds
  • Testing a patch which will treat try / author builds as separate repos. Will require database change.
• bug 1275344 - Autophone - download builds from the TaskCluster index
  • May need to deal sooner than I thought.

Round Table

• [dminor] Moving WebRTC testing to autophone
  • We consistently hit problems with tests that fail on the android emulator but work elsewhere, most likely because the emulators run so slowly that the timing is completely different than on other platforms. This could be due to how our tests are written.
  • Latest is Bug 1271585, fails on emulators, passes on autophone, locally running emulator (e.g. https://treeherder.mozilla.org/#/jobs?repo=try&revision=b78238802c09&selectedJob=21448168)
  • We're getting to a point where we need to decide whether we want to either move our tests to autophone or invest a good chunk of time in investigating exactly what is going on with the emulators and/or trying to make them faster.
  • Our timeframe to move to autophone would be by end of Q4.
  • I'd like to set up a meeting with jmaher, bc and some WebRTC folks to discuss this further, but any feedback you can give me right now would be helpful :)