B2G/QA/Meetings/Automation/2015-01-29

Overview

This is a weekly meetup to discuss all things regarding B2G automation and Quality. Please use wiki style.

• Meeting Name: QA Automation Roundtable
• Meeting Time: Thursday @ 9:30am - 10am PST
• Vidyo Room: QA Vidyo Room
• Video Link: https://v.mozilla.com/flex.html?roomdirect.html&key=nL4PESn1IaVh
• Old link is gone, should we use this one?
  • https://v.mozilla.com/flex.html?roomdirect.html&key=0HHQN7SSNIkAxuJq04fq7uZk6J8
• Dial in: 1-800-707-25331
• Meeting Notes: https://wiki.mozilla.org/B2G/QA/Meetings
• Previous Meeting:https://wiki.mozilla.org/B2G/QA/Meetings/Automation/2015-01-22

Q1 2015 Goals

2.2 - (OS QA (in Taiwan)

• • Support the stability of 2.2 branch with focused test runs, qawanted, bug triage, and daily smoke tests (manual and automation)
  • Support new feature testing
  • Manage the release out the door until CC date, and hand off to Devices QA team into Q2.

3.0 - (OS QA (in US/EU)

• • Automation goals:
    • Re-evaluate the smoke tests on v3 that will be needed for automation in javascript
    • Focus on getting existing Gij tests running against the lab of devices
    • Creating a set of best practices on creating JS gaia-UI automation tests
    • Focus on getting smoke tests running against the lab of devices (could be a combination of Gij, python, or JS tests).
  • Support the stability of master with focused test runs, qawanted, bug triage, and daily smoke tests (manual and automation)
  • Support lightweight dogfooding program off of nightly
  • Develop feature testplans when features are concrete

Previous Action Items

[Tony] request v18D-1 with adb enabled access from T2M

• • Bebe has tested at desk, looks good, but https://bugzilla.mozilla.org/show_bug.cgi?id=1122119 might be relevant

[Tony] MarionetteJS bugs from James

• • Geo to follow up. (Bhavana followed up, Geo and Bhavana to meet w/ James)
• [jlorenzo] Bug 1122119 - Leave a phone shallow flashed running on the clock the entire night and see if it crashes.
  • Done
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1025265 would allow us to get some logs and diagnose what's going on on the kernel side
• [Peter] Draft a proposal for Developer Status and present it to QA Owners initially
  • Spreadsheet proposed to reference all apps/areas and where they are in the dev cycle
    • Add column if it's stable to automate
• [Bebe] When should we update to v18D-1
  • Geo to follow up with releng to know what we're bundling -- are we frankenbuilding?
    • Verified that 18D was being used as the bundle base. Phones have been moved to that base.

Status Highlights

• [Bebe] Changes in the way we run the daily mozilla-central build - We are running build from TinderBox now instead of Nightly as we used to.
  • Geo to follow up with Dave on gaia-ui-automation
• [Bebe] Bug 1127304 - Flame device fails to start after flashing latest build M-C and M-I
  • All the tests are failing and the device is not starting.
  • Black screen
• [Geo] Lab status update
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1126262 -- Fixed, but was a problem (AT&T sims)
• [Bebe] Updated the build time out for the UI builds
• https://github.com/Mozilla-TWQA/B2G-flash-tool/blob/master/check_versions.py
  • the shell script will show you the version : i.e. : Bootloader L1TC000118D0
    • we can check for this before running the automation or use this to verify the builds.