Auto-tools/Projects/Mozbase

Overview

Mozbase is a set of easy-to-use Python packages forming a supplemental standard library for Mozilla. It provides consistency and reduces redundancy in automation and other system-level software. All of Mozilla's test harnesses use mozbase to some degree, including Talos, mochitest, reftest, Autophone, and Eideticker.

This is the project page for mozbase development, outlining current development practices, methodologies, and protocol. Other resources related to mozbase:

Repository: https://hg.mozilla.org/mozilla-central under testing/mozbase

Documentation: https://firefox-source-docs.mozilla.org/mozbase/

Bugs:

Please file bugs against https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=Mozbase

The roadmap of getting mozilla-central python to use mozbase is detailed at https://wiki.mozilla.org/Auto-tools/Projects/Mozbase/Roadmap . The effort to this effect is detailed at https://wiki.mozilla.org/Auto-tools/Projects/Mozbase/Automation-Refactor

Mozbase requires python 2.7 or greater

Development Practices

If you're developing mozbase code, you're in luck! You've found the right place to read about our development practices. Please do :)

Getting Help with Mozbase

Mozbase is developed by the Automation and Tools team. It is under the governorship of Will Lachance (:wlach) and Andrew Halberstadt (:ahal). Please feel free to stop by #ateam on IRC with any questions and we will be happy to help you! Or if something is wrong, please file a bug and we'll look at it as soon as possible.

Installing Mozbase for Development

Initial Setup

Install python-pip.
Install virtualenv and virtualenvwrapper:

   $ sudo pip install virtualenv
   $ sudo pip install virtualenvwrapper
   $ echo "source `which virtualenvwrapper.sh`" >> ~/.bashrc
   $ source ~/.bashrc

Grab a copy of mozilla-central.
Create a new virtualenv and install mozbase into it:

   $ cd mozilla-central/testing/mozbase
   $ mkvirtualenv mozbase
   $ python setup_development.py

Run the mozbase tests as a sanity check:

   $ python test.py

Working on Mozbase and Contributing Patches

Working on mozbase is the same as working on any other part of the mozilla-central source tree. See the following links for tips on getting set up with Mercurial and how to submit a patch:

Design Principles

if we're extending the standard library, we should extend via inheritance
- we are free to extend and add new methods, but we should support the existing methods when possible

Adding a New Mozbase Package

A new mozbase package should start with version '0.0'. It should be bumped to a real version when released to pypi

Scripts

In addition to Mozbase's python packages, there are several scripts written to aid Mozbase developers in pushing the workflow along:

setup_development.py : sets up mozbase packages for in development mode; see #Initial_Setup
test.py : test runner for Mozbase unittests; see #Tests

More about these scripts can be found from running `$SCRIPT --help` or by consulting the documentation and source.

Tests

Mozbase packages are accompanied by tests in order to ensure and illustrate proper functionality.

each mozbase package should have tests in a 'tests' subdirectory of its root
- Example: http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/mozprofile/tests/
these tests should be contained in a manifest; the master test manifest, is executed via test.py

Running the tests

You will need make and gcc in order to run the tests. If you're running windows, this is available in MozillaBuild.

Then, from the root of mozilla-central, run:

   $ ./mach python-test testing/mozbase

This will run all tests from every mozbase module. You can run individual tests by passing in that directory specifically, e.g:

   $ ./mach python-test testing/mozbase/mozprocess

Mozbase Testing Framework

TODO: document how testing works in mozbase (test.py, etc)

Mozbase Testing Strategy

TODO

Tests with Python Dependencies

Tests for a mozbase package may require dependencies beyond which those listed in the package's 'setup.py' 'install_requires'. Tests may utilize other mozbase packages even if they are not required by the package under test.

Third-party packages -- those not in python's stdlib -- should be introduced only if they provide considerable value to testing (e.g. ability to test what is otherwise not possible, considerable savings in code complexity/manhours, etc.). In general, adding a test-only dependency should be taken (nearly) as seriously as adding a mozbase package dependency via install_requires. Required packages should, if at all possible, be pure python and of high quality. Preferably, test-required packages should be ad hoc standards if possible; for instance, if the package is slated to be included in the stdlib or is in a future python version, it is a good candidate.

If a third-party package is added for testing, it should be added as part of extra_packages in setup_development.py (Ref: http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/setup_development.py ), to the python/ directory and to build/virtualenv/packages.txt (Ref: [1] ) in mozilla-central, please file a bug to get the dependency added).

Third-party Packages Used in Mozbase Testing:

mock

In general, additional requirements add overhead to code complexity, new contributor learning curve, maintenance cost, and portability, so the value added should justify the costs. Candidate packages should be discussed with Mozbase stakeholders preferably in a bug so that a record is kept.

We're always looking for ways to make this easier; please weigh in on bug 885342 with your ideas!

Continuous Integration

We currently run tests automatically on every mozilla-central commit. They are run as part of make check in the build job, but bug 1003417 tracks pulling them out into a separate job.

Versioning

NOTE: you need to have a ~/.pypirc file filled out in order for python packaging scripts to upload packages to the python package index. The right hand side below of username/password are your account credentials for pypi.python.org:

 [server-login]
 username:username
 password:password

The major version should be bumped when the API changes in a non-backwards-compatible way or for other large conceptual changes. Otherwise, the minor version may be bumped. The version doesn't have to be bumped if there is no new release planned for PyPI.

When you bump a version of any mozbase package, it should be released to pypi:

   python setup.py sdist upload

Please be careful when doing this that there are no unwanted files in the package directory. If there are, they will be uploaded to pypi.

Note, if you are uploading a new package, it needs to be registered first:

   python setup.py sdist register

PyPI

Releases of mozbase packages are uploaded to the Python Package Index (pypi) so that they may be easily consumed by setuptools and other installers.

Mozbase use in Mozilla-Central

The mozbase packages are installed into the mozilla-central virtualenv instance. The mozbase packages.txt file is included from the master packages.txt file and includes all mozbase packages (really, directories) in mozilla-central via .pth files.

The various mozbase packages are used in mozilla-central python code; see https://developer.mozilla.org/en-US/docs/Python#Python_in_Mozilla-Central . Mozbase is used extensively throughout mozilla-central, including test harnesses, tools and the build system itself.

Adding new Packages

If you are adding a new package to mozbase, there are unfortunately several files you'll need to update in addition to your added files. Those are:

Uploading to Puppetagain

Mozharness uses packages from http://puppetagain.pub.build.mozilla.org/data/python/packages/ in production. If a test harness invoke by mozharness requires updated mozbase packages.

In order to mirror, file a bug with a URL of the package(s) of interest.

Licensing

Mozbase should be appropriately licensed with MPL2. See https://www.mozilla.org/MPL/headers/ to find a copy.

Documentation

full docs are found in the top-level docs directory and hosted at http://mozbase.readthedocs.org. We're always looking for help writing docs.
this page is the mozbase project page that gives information for core mozbase developers and about the project itself. Documentation may move to and from this page and the various READMEs and full docs.

Directory Structure and Imports

The mozbase repository contains several packages. Each package should have a directory structure like:

   ${PACKAGE}
   ${PACKAGE}/setup.py
   ${PACKAGE}/README.md
   ${PACKAGE}/${PACKAGE}
   ${PACKAGE}/${PACKAGE}/__init__.py

Even if the package consists of a single module, this structure should be used (vs. py_modules) and '__init__.py' set to

   from ${MODULE} import *

(with a sensible '__all__' defined in the module).

This structure serves several purposes:

this allows packages to be installed with the 'setup.py' script
if packages cannot be installed (for deployment reasons), altering 'sys.path' (or '$PYTHONPATH') can be used to utilized
with single-module packages, you get a sensibly named file that is imported in '__init__.py' and can be mirrored easily and it conforms to how all of the other packages work

Integration Notes

Notes on integration with existing test harnesses.

Mozbase and Mozharness

Mozbase and mozharness overlap intent with respect to making testing software and tools reusable and easy to write and extend.

mozharness docs: http://escapewindow.com/mozharness/

Production use of mozharness will use mozbase packages from an internal pypi: http://puppetagain.pub.build.mozilla.org/data/python/packages/ (from https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/Python )