Mobile/Testing/04 30 14
From MozillaWiki
Contents
Previous Action Items
- gbrown to file a bug for the Android 2.3 R5 crashes - bug 1000538
- kmoir will check into ix capacity for Android 2.3 - yes, I think we have sufficient capacity
Status reports
Dev team
Rel Eng
IT
A Team
- Android 2.2 failure rate: [3.06%]
- last week 4.06%
- Android 2.3 failure rate: [0.95%]
- last week 3.52%
- Android 4.0 failure rate: [17.98%]
- last week 17.37%
- excluding retries 2.46%
Android 2.3 emulator
- R5 crashes - bug 1000538 - may have self-healed
Autophone
- Work progressing on bug 990601 to use adb instead of the SUTAgent.
- Issues identified
- Previously with the SUTAgent, I used the logcat entry for SUTAgent's command to start fennec to identify the start time. Without the SUTAgent, there is no clear indicator in the log which I can use to detect the start time. I am using the first mention of Gecko or fennec in logcat after starting fennec to determine the start time. If you have a better approach, I would very much appreciate hearing about it.
- Some issues with devicemanagerADB hung commands. Am working on a short term replacement for autophone before incorporating the changes into mozbase's devicemanagerADB.
- We require the device to boot to an unlocked screen in order to run tests. On Nexus 4 and later I can set the device to not lock on boot but for older devices such as Nexus One and Nexus 2, I need to use something to automatically unlock the device.
- Test results
- Have moved the -2 nexus devices from production testing to development testing.
- Ran a test run with the SUTAgent and the Watcher installed on all phones and another with only the Watcher installed on Nexus One and Nexus S phones
- Most recent test results for the nexus devices using adb can be found at for April 27: http://phonedash-dev.allizom.org/#/org.mozilla.fennec/throbberstart/local-blank/norejected/2014-04-27/2014-04-27/notcached/noerrorbars/standarderror
- The presence of the SUTAgent does have an affect on the measurements though it is not consistent across devices.
- Next steps
- Will finish the adb command replacements today.
- Will submit patch for comment and review by tomorrow.
- Would like to move all phones to the new adb based code for testing.
- Need to finalize code in preparation for shipping the devices to Mountain View for installation in the QA lab during the week of May 12.
- Issues identified
Eideticker
- Dashboard still down, save a brief testrun on Friday. :( Ports issue resolved (bug 988606) but devices keep on falling off network. Netops investigating in bug 1003507