Mobile/Testing/06 25 14
From MozillaWiki
Contents
Previous Action Items
- :snorp to send :bc info for his custom autophone needs
- someone(!) should file bugs to capture ideas for additional autophone features discussed today: make it easier to run locally, autophone via try, etc.
- bug 1030244 - Make Autophone more developer friendly.
Status reports
Dev team
Rel Eng
- We have a lot of dead Tegras (~100). Callek is wondering if it is worth doing another sutagent deploy given their EOL. The problems they are exhibiting is that they aren't allowing a connection on the SUTAgent ports.
- reconfig to enable 2.3 tests on armv6 and ride the trains failed last night despite passing in staging and pre-production tests. Investigated and it requires a rolling restart of certain test masters to implement in production.
- we may start using a more powerful AWS instance type (c3.large) for selected tests. This will allow some b2g tests currently running on ix move to AWS. Not sure if this will help us move more 2.3 tests to AWS.
Relops
A Team
- android2.2 Total failure exclude retry rate: [3.26%]
- last week 7.66%
- android2.3 Total failure exclude retry rate: [0.22%]
- last week 0.25%
- android4.0 Total failure exclude retry rate: [17.06%]
- last week 15.19
- total failures excluding retries 2.33%
Android 2.3 emulator
- Needed to disable more Robocop tests for aws. Tests disabled on 2.3:
testAboutHomeVisibility (aws) 946656 testAboutPage bug 975187 -> bug 1025968 testAddSearchEngine bug 979552 testBookmarkFolder bug 979552 testBookmarksPanel bug 979615 testClearPrivateData bug 948591 testDoorHanger (aws) bug 1028929 testImportFromAndroid bug 979552 -> bug 1025968 testInputConnection bug 1025968 testMasterPassword bug 979603 testNewTab (aws) bug 995696 testOverscroll (aws) bug 836818 testSelectionHandler bug 980074 -> bug 1025968 testSessionOOMRestore (aws) bug 979600 testSessionOOMSave (aws) bug 945395 testSettingsMenuItems bug 979552 -> bug 1025968 testSystemPages bug 979603 -> bug 1025968 testTitleBar bug 979552
- A few random mochitest timeouts noted on aws -- watching frequency.
- media mochitests?
Autophone
- Are we ready to call Autophone and Eideticker our official perf frameworks?
- [bc] Now that the wifi situation in the QA lab has settled down, I need to investigate getting the remote tests running again.
- Ran tests for 2014-06-21 through part of the day 2014-06-22 with and without ondemand linking and forwarded the results to snorp. See ondemand testing.
- Retested webappstartup.
- snorp noticed a marked decrease in times on 2014-06-20. I had deployed an update to Autophone on that day which may have been responsible for the change. I retested the time range with the production devices posting to phonedash-dev and found the previous results prior to the 20th were too high in general. This may have been due to an error during the initial deployment of the webappstart up. I cleared the results and have submitted jobs to backfill to the 18th. The decrease in start and stop time is still apparent for the nexus s devices, but no longer for the remainder of devices.
- nexus-5-kot49h-2 and nexus-7-jdq39-1 both went offline probably due to battery drain due to the load caused by the backfilling. Will have back up later today.
- Waiting final reviews for bug 1012711 for adb.py for mozbase and finalizing patch for bug 1007220 for the final webappstartup deployment.
Eideticker
- We may actually set up the Galaxy Nexus in Mountain View after all; we got HDMI working with it and the network seems to be better than before
Round Table
Action Items
- :kmoir will ping blassey when armv6 reconfig done