Mobile/Testing/02 26 14
From MozillaWiki
Contents
Previous Action Items
- Callek to followup on ant install - bug 971841
- wlach will send email about Datazilla, notifications, etc (for improvements to find gradual, long-term regressions)
- Talos tests for GeckoView?
Status reports
Dev team
Rel Eng
IT
A Team
- tegra failure rate: [2.01%]
- last week 2.08
- panda failure rate: [15.86%]
- last week 15.03
- excluding retries 0.86%
Slow Reftests bug 967913
- Not successful with SPS
- Difficulties connecting / retrieving profile on emulators on ec2
- just too slow to deal with remote debugging connection
- Able to get profile from local instance of emulator
- Fennec would crash after 15 - 20 seconds with profiler connected
- Profile I retrieved did not seem useful
- Difficulties connecting / retrieving profile on emulators on ec2
- When running locally, emulator can complete 32% of chunk 1 vs. 4% on ec2
- I've created a c3.large instance to see if aws instance type has substantial impact on running time
Android 2.3 emulator
- See https://tbpl.mozilla.org/?tree=Ash&jobname=Android.*2.3.*&rev=50c948d1c8c8
- Reftests too slow: bug 967913
- Many robocop tests disabled: bug 975155, bug 975187, ...
- mochitest-gl very broken: bug 975487
- Need to report more bugs and land several test-disabling patches on m-c
- Otherwise, seems reliable...almost ready for trunk!
Autophone
- 2014-02-24 PM - Landed bug 968920 - use standard error percentage as a discriminator.
- Currently 'rejects' tests which have standard error as percentage of mean over 10%. If a measurement is 'rejected', it is retried once. You can toggle the display of 'rejected' measurements via a checkbox. This is intended to reduce noise from transient phone conditions. Currently, 20_64_32_4f_e9_4f_samsung-gs2 is not producing reliable remote test results and is falling behind.
- Working on bug 969518 - attempt to use reverse tethering so we can use ethernet over usb instead of wifi.
- Possible approaches are using ppp over adb and RNDIS over usb. First investigating ppp over adb.
Eideticker
Round Table
Action Items
- Callek to followup on ant install - bug 971841 - delayed
- wlach will send email about Datazilla, notifications, etc (for improvements to find gradual, long-term regressions)
- Talos tests for GeckoView -- gbrown will try to find a "how to write a talos test" primer for blassey
- gbrown to assign robocop on 2.3 bugs to snorp