Introduction

This document describes a simple build and packaging system that allows Cuddlefish-based SecurableModules to be developed, tested, and reused in a variety of contexts.

For the time being, this system and its related command-line tool will be referred to as cfx.

Philosophy

cfx is largely influenced by paradigms present in the Python and Narwhal open-source communities. Among these are:

• Test-Driven Development. cfx attempts to make writing tests as easy as possible, taking its inspiration from testing tools like py.test and nose. It also aims to make test execution as fast as possible, so developers aren't wasting precious time waiting for their suites to complete.

• Code Reuse. The packaging systems of Python and Narwhal make it easy to share and reuse other people's code. The Mozilla platform lacks such a system, which has historically made reuse difficult: as a result, shared code tends to be in the form of "snippets" or single self-contained files. cfx attempts to resolve this problem, and in doing so inherits as much as it can from the Narwhal package format to maximize compatibility and reuse of CommonJS-compliant JavaScript code.

• Developer Ergonomics. cfx uses Cuddlefish's introspection capabilities to provide full stack tracebacks when errors occur; it also provides facilities for making it easy to spot memory leaks, and generally tries to make writing production-quality code as hassle-free as possible.

• No Dependency Hell For Users. the XPIs generated by cfx are fully self-sufficient and their only requirement is the Mozilla platform. This means that all their package-level dependencies are automatically rolled into the XPI and stored such that multiple versions of them can exist side-by-side with other cfx-generated XPIs.

Specification

Packages

A package is simply a directory that contains a JSON file called package.json.

package.json may contain the following keys:

• name - the name of the package. The package system will only load one package with a given name. This name cannot contain spaces. The name defaults to the name of the parent directory. If the package is ever built as an XPI and the fullName key is not present, this is used as the extension's em:name element in its install.rdf.

• fullName - the full name of the package. It can contain spaces. If the package is ever built as an XPI, this is used as the extension's em:name element in its install.rdf.

• description - a String describing the package. If the package is ever built as an XPI, this is used as the extension's em:description element in its install.rdf.

• author - the original author of the package. The author may be a String including an optional URL in parentheses and optional email address in angle brackets. If the package is ever built as an XPI, this is used as the extension's em:creator element in its install.rdf.

• contributors - may be an Array of additional author Strings.

• url - the URL of the package's website.

• license - the name of the license as a String, with an optional URL in parentheses.

• id - a globally unique identifier for the package, which is usually either a String in the form of a GUID or an email address. If the package is ever built as an XPI, this is used as the extension's em:id element in its install.rdf.

• version - a String representing the version of the package. If the package is ever built as an XPI, this is used as the extension's em:version element in its install.rdf.

• dependencies - a String or Array of Strings representing package names that this package requires in order to function properly.

• lib - a String or Array of Strings representing top-level module directories provided in this package. Defaults to "lib".

• tests - a String or Array of Strings representing top-level module directories containing test suites for this package. Defaults to "tests".

• packages - a String or Array of Strings representing paths to directories containing additional packages, defaults to "packages".

• main - a String representing the name of a program module that is located in one of the top-level module directories specified by lib.

Modules

All modules in cfx packages are CommonJS modules, and also support the global objects provided by Cuddlefish.

Test Modules

A test module is any module located in one of the top-level module directories specified by the "tests" key in package.json whose filename starts with the prefix test-. All its exported functions are assumed to be test cases or a suite of test cases; each is called with a single argument, test, which is a Test Runner Object.

Program Modules

A program module is a module named by the "main" key in package.json and located in one of the top-level directories specified by the "lib" key. It is expected to export a function called main() which contains a program body.

Usage

Prerequisites

The current reference implementation of cfx is available via Mercurial:

 http://hg.mozilla.org/users/avarma_mozilla.com/jep-28/

Using this implementation requires Python 2.5 or greater. On Windows, it also requires the Win32 Extensions for Python.

Environments

Unlike many package management systems, there isn't a system-wide location for a cfx installation. Instead, developers can have as many installations of cfx as they want, each configured separately from one another. Each installation is called an environment.

To set up a new environment, simply create a new clone of the cfx repository. To activate the environment under a unix-based operating system, launch a bash shell, enter the root directory of the repository checkout, and run:

 source bin/activate

Alternatively, if the OS is Windows-based, launch cmd.exe and run:

 bin\activate

This will activate the checkout's environment such that a number of cfx-specific command-line tools are globally available from any directory.

At any time, deactivate can be entered to deactivate the environment and return the shell to its original state.

Package Management

Packages are simply subdirectories of the packages directory, which is located in the root of the cfx repository.

At the moment, package management functionality is outside the scope of cfx. This means that it is currently the developer's responsibility to manually manage the installation, removal, and updating of packages on their own.

Command-Line Tools

cfx

The cfx command-line tool, made globally available upon environment activation, offers a variety of mechanisms to aid in the development, testing, and deployment of packages.

All cfx commands perform some action based, by default, on the current directory, which is expected to be the root of a package and hence contain a package.json file.

Executing cfx --help from the command line yields the following output:

Usage: cfx [options] [command]

Package-Specific Commands:
  xpcom      - build xpcom component
  xpi        - generate an xpi
  test       - run tests
  run        - run program

Global Commands:
  testall    - test all packages
  update     - update all packages

Options:
  -h, --help            show this help message and exit
  -p PKGDIR, --pkgdir=PKGDIR
                        package dir containing package.json; default is
                        current directory
  -t TEMPLATEDIR, --templatedir=TEMPLATEDIR
                        XULRunner app/ext. template
  -g CONFIG, --use-config=CONFIG
                        use named config from local.json
  -v, --verbose         enable lots of output

  XPCOM Compilation Options:
    -s MOZ_SRCDIR, --srcdir=MOZ_SRCDIR
                        Mozilla source dir
    -o MOZ_OBJDIR, --objdir=MOZ_OBJDIR
                        Mozilla objdir

  Application Options:
    -a APP, --app=APP   app to run: xulrunner (default), firefox, or
                        thunderbird
    -b BINARY, --binary=BINARY
                        path to app binary

  Testing Options:
    -d, --dep-tests     include tests for all deps
    -c COMPONENTS, --components=COMPONENTS
                        extra XPCOM component dir(s), comma-separated
    -n, --no-quit       don't quit after running tests
    -x ITERATIONS, --times=ITERATIONS
                        number of times to run tests

mozrunner

cfx includes Mikeal Rogers' mozrunner command-line tool, which in turn uses his mozrunner Python package.

Executing mozrunner --help yields the following output:

Usage: mozrunner [options]

Options:
  -h, --help            show this help message and exit
  -n, --no-new-profile  Do not create new profile.
  -b BINARY, --binary=BINARY
                        Binary path.
  -w PLUGINS, --plugins=PLUGINS
                        Plugin paths to install.
  -p PROFILE, --profile=PROFILE
                        Profile path.
  -d DEFAULT_PROFILE, --default-profile=DEFAULT_PROFILE
                        Default profile path.

Examples

A Simple Package

Here's a package that makes hard math easy.

package.json:

{
  "name": "hardmath",
  "description": "Ridiculously complex math routines."
}

lib/hardmath.js:

exports.add = function add(a, b) {
  return a + b;
};

tests/test-hardmath.js:

exports.testAdditionWorks = function(test) {
  test.assertEqual(require("hardmath").add(1,1), 2,
                   "hard addition should work.");
};

Running Tests

Executing cfx test from the root of the hardmath package yields:

.
Malloc bytes allocated (in use by application): 6548640
Malloc bytes mapped (not necessarily committed): 14262272
Malloc bytes committed (r/w) in default zone: 6556144
Malloc bytes allocated (in use) in default zone: 13213696
Tracked memory objects in testing sandbox: 2

1 of 1 tests passed.
OK
Total time: 1.109022 seconds
Program terminated successfully.

If we run cfx test --verbose, the single dot in the above output is replaced with more useful feedback:

info: executing 'test-hardmath.testAdditionWorks'
info: pass: hard addition should work.

Undocumented Features

A number of features of cfx's reference implementation remain to be documented. Among them are:

• It's possible to build and load XPCOM binary components that are specific both to the host operating system and the host Gecko platform version. An example package can be found here.

• Packages can have Python plugins associated with them which modify the build process in some way. The cuddlefish package currently has one that automatically downloads the CommonJS SecurableModule interoperability tests if necessary, for example. The code can be found here.

• Modules in cfx-built packages actually have another global injected into them called packaging, which gives them access to package metadata and allows them to create sandboxed module loaders whose lifetime isn't tied to the lifetime of the containing XULRunner application. Example code can be found here.

• Packages can also have a data top-level directory that contains arbitrary static content such as HTML, CSS, images, and so forth. This can be accessed via packaging.getURLForData(). An example package can be found here.

• When compiling XPCOM components or otherwise reusing lots of command-line options, it may be helpful to create a local.json file in the root of your checkout and then use the --use-config option to specify a configuration in it. Here's an example local.json one might use to make it easier to build binary components for different versions of the Gecko platform:

{
  "configs": {
    "ff36": [
      "--srcdir", "~/Documents/firefox-3.6-stuff/mozilla",
      "--objdir", "~/Documents/firefox-3.6-stuff/basic-firefox",
      "--binary", "/Applications/Firefox.app/Contents/MacOS/firefox-bin"
    ],
    "ff35": [
      "--srcdir", "~/Documents/firefox-3.5-stuff/mozilla",
      "--objdir", "~/Documents/firefox-3.5-stuff/basic-firefox",
      "--binary", "~/Documents/firefox-3.5-stuff/basic-firefox/dist/bin/firefox-bin"
    ]
  }
}

• It's possible to use the --templatedir option to cfx to use a cfx configuration from an existing extension.
  • Copy the file python-lib/cuddlefish/app-extension/components/harness.js to your extension's components directory.
  • Access a module for your package via code like this, where moduleName is the name of the module and id is the value of "id" from your package's package.json file:

var module = Cc['@mozilla.org/harness-service;1?id=' + id]
             .getService().wrappedJSObject.loader
             .require(moduleName);