Mobile/Testing/09 23 15

From MozillaWiki
Jump to: navigation, search

Previous Action Items

Status reports

Dev team

Rel Eng

  • Landed changes to enable additional Android chunking for debug tests on cedar this morning, waiting for reconfig to finish and then I can run tests. {{bug|1201236}
  • Added 4 new AWS Linux masters so we can handle new load of these tests when they are enabled on other branches bug 1205409
  • After masters are fully deployed I'll increase the tst-emulator64 pool again so we have enough capacity for these new tests bug 1204756

A Team

Autophone

  • tl;dr
    • Backlog too high.
    • Try now runs on all devices
    • Try chooser updated to only allow selection of autophone-smoketest,autophone-s1s2,autophone-webapp,autophone-mochitest-dom-browser-element,autophone-mochitest-dom-media,autophone-mochitest-skia,autophone-mochitest-toolkit-widgets. Full set of tests can be selected via manual try commit.
    • Unit tests now running using adb instead of SUTAgent; no longer bound to specific devices.
  • bug 1204287 - Autophone - 2015-09-17 deployment
    • bug 1197374 Autophone - create separate logs for devices and tests.
    • bug 1204071 Autophone - runtestsremote.py must explicitly set environment variables via the command line
    • bug 1204288 Autophone - do not short circuit test downloads based on existence of tests directory
    • bug 1205382 Autophone - PhoneWorkerSubProcess:stop - kill process if terminate does not succeed
    • bug 1205383 Autophone - catch device exceptions in PhoneWorkerSubProcess:ping
    • bug 1205384 Autophone - suppress BeautifulSoup4 parser warning
    • bug 1205386 Autophone - frequently reap child processes
    • bug 1205388 Autophone - better handle logcat collection
    • bug 1205389 Autophone - Update preferences for new SafeBrowsing prefs structure added in Bug 1107372
    • bug 1205393 Autophone - wait between polls and collect logcat when polling unit test process
    • bug 1205394 Autophone - Set MOZ_CRASHREPORTER_NO_REPORT in environment to suppress crashreporter ui
    • bug 1205396 Autophone - limit revision to 12 characters in job url email
    • bug 1205660 Autophone - disable global autophone-tests selector on try
  • On Deck
    • bug 1205864 Autophone - phonetest.py:Logcat collects duplicate messages
    • bug 1206954 Autophone - better handle failures to submit results to PhoneDash
    • Setting up Nexus 6/9, getting ready to ship to Mountain View.

Perfherder

  • Working on supporting autophone data, see bug 1175295 and round-table

Round Table

  • How to handle autophone data in perfherder: leads to a question of what autophone should be doing.

Action Items

  • kmoir to determine the number of pandas currently used to run talos tests.
    • We have 380 pandas in production. We run about 400 ~talos jobs a day on them out of a total of ~2300 jobs a day. In other words 17% job current panda jobs are on talos. So in theory if we just run talos jobs this would require 65 pandas to maintain our current wait times.
  • wlach to meet with jmaher/bc concerning perfherder and autophone requirements.