Details

January 29, 2013 @ 11:30am PDT / 2:30pm EDT

dial-in:

• vidyo: Jgriffin's vidyo room

Backchannel #ateam on irc.mozilla.org

Take meeting notes here: https://etherpad.mozilla.org/great-automation-refactor Then copy and paste them below afterwards

Meeting Notes

The Great Automation Refactor

• 2013 Q1 Goal to move mochitests atop mozbase (see https://wiki.mozilla.org/Auto-tools/Goals/2013Q1#General_Automation)
• TRACKING BUG: https://bugzilla.mozilla.org/show_bug.cgi?id=775756
• See also: https://wiki.mozilla.org/Auto-tools/Projects/MozBase/Roadmap

Overall Strategy

• Groundwork: do the work needed to allow mochitest etc to consume mozbase packages in the tree
  • Figure out strategy for testing in production
  • How to deal with running tests in tests.zip?
    • mozharness, but when will this be ready?
• Housekeeping: scripts to mirror, versionbump mozbase
• Iteration -> replace one piece at a time and test
  • no need to worry about "flicking the on switch" and breaking everything
• Lowest hanging fruit first (?), e.g mozcrash
  • Replace the parts that are easiest first making it simpler to visualize the remaining pieces
• Generalize architecture: aim for the future
  • Goals for reftests/jsreftests/crashtests, xpcshell etc. likely targeted later in the year
  • Should still keep these harnesses in mind when porting mochitests -> generalize now to make these efforts easier in the future
• Documentation as we go (mozbase docs, test harness docs, https://developer.mozilla.org/en-US/docs/Python , etc)
  • This will be the best time to get documentation updated for a while
  • As pieces land, is there anything about it that could/should be documented? Update MDN, wiki or mozbase docs as appropriate, Python docs in devmo (https://developer.mozilla.org/en-US/docs/Python)
• B2G might be a good place to start as it is at the bottom of the inheritance tree
• triage: do we want any specific practices or owner(s) for this?

General

• Import mozbase modules within tests.zip -> bug #?
• Will each runner become its own package?
  • What about things that we'll still need in common, like the ProxyAutoConfig profile stuff in automation.py.in?
• How can we modify pieces in the dependency chain (e.g., automation.py.in) without breaking other things?
• What to do with automation.py.in's Process abstraction (mozprocess vs mozrunner), and what do to about the remote cases

Areas of Work

• Setup: making virtualenv and m-c environment usable to mozbase
  • get everything to run out of the m-c virtualenv
  • remove duplicate code: https://bugzilla.mozilla.org/show_bug.cgi?id=707976
  • kill pythonpath.py : https://bugzilla.mozilla.org/show_bug.cgi?id=774911
• Refactor: move our harnesses atop mozbase
  • porting our harnesses
  • packaging our harnesses
  • internal mozbase fixes
  • unfulfillment functionality from m-c should be migrated to mozbase
• Support: make our mozbase workflow as automated and accurate as possible
  • mozbase on `make check`
  • scripts: setup_development, versionbump, generate_diff, test.py
  • what depends on mozbase? https://bugzilla.mozilla.org/show_bug.cgi?id=734235
  • documentation; e.g. https://bugzilla.mozilla.org/show_bug.cgi?id=835553

Testing

• We can possibly use ash or cedar
• Need to identify mozharness/buildbot work
  • Need mobile mochitest on mozharness?
• Buildbot only tests one code path, how will we test the others?

Goals Review

• Do we have the right set of goals for this quarter?
• ted: refactor mozprocess
• will: add remote API for mozrunner
• jeff: refactor automation.py.in and time permitting all of mochitest (and support work)
• ahal: b2g mochitest on mozbase
  • Current goals suggest we are not waiting for tests to roll out on mozharness first

Mochitest

• Completely remove dependency on *automation.py files

CURRENT STATUS

• TBD

Reftest

• TBD

Xpcshell

• TBD

Misc

Somewhat relevant links:

• https://developer.mozilla.org/en-US/docs/Mochitest
• https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/B2G_Mochitests
• https://github.com/mozilla/mozbase
• https://wiki.mozilla.org/Auto-tools/Projects/Mozbase
• http://mozbase.readthedocs.org/en/latest/
• http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/
• http://mxr.mozilla.org/mozilla-central/source/layout/tools/reftest/
• http://mxr.mozilla.org/mozilla-central/source/testing/xpcshell/