AMO:Developers/JavaScriptTesting

UPDATE: This was a brainstorming page that led to the creation of JS TestNet

The Zamboni Django app has quite a bit of JavaScript now for features on the site. We currently don't have any automated tests to run so here are some ideas about how we can make a test suite.

Why?

• Tests help to refactor existing code
• Tests make it easier to upgrade libraries like jQuery or external plugins
• It's easier to work on another developer's features without fear when there are tests
• A good testing environment helps to simulate errors and timeouts that can be hard or impossible to test manually

What Do We Want?

• Quick tests, easy to run during development
• A test environment as close as possible to production, which is mainly the Firefox web browser
  • Really? What does that mean in practice (os/browser matrix)? A simpler goal is unit testing (pure JS environment) and integration testing (Fx or other).
  • If a pure JS environment was "pretty close" to production then that would be fine. A problematic scenario would be where something worked in the tests but not in production (the browser has its quirks!) --Kumar303 08:45, 16 November 2010 (PST)
• A test suite that can run reliably in CI and deliver meaningful results
• the ability to use a DOM since most features involve attaching behavior to the DOM
• We want to create small integration tests that verify one or more widgets, not large functional tests that focus on website behavior (QA writes tests for that)
• User interface tests that do not rely on a server. That is, no actual Ajax requests just mock requests (if needed).

Test Runners

QUnit

• Pros
  • CommonJS Unit Test compliant
  • popular framework, well supported
  • very simple and easy to write tests with
  • Runs primarily in a web browser like production
  • Can provide visual feedback if necessary (useful for development)
  • uses a real DOM
  • since tests are written in HTML, Zamboni template logic can be reused in some cases
  • has several adapters for the command line
    • QUnit in node.js
    • QUnit with Rhino + env.js (example in NoseJS and this fork)
    • Qunit for JsTestDriver (Cannot do async testing)
• Cons
  • Might be tricky to get working for CI. We could load a single webpage in a web browser VM though and use builtin hooks to get test results
  • All user events need to be simulated by triggering the event or otherwise.
  • does not fit seamlessly into the current test suite (but maybe with NoseJS?)
• Proof of Concept
  • You can clone this branch https://github.com/kumar303/zamboni/tree/new-addon-validator-609355 and open http://127.0.0.1:8000/en-US/firefox/qunit/
    • At the moment you'll need to pip install https://github.com/kumar303/django-qunit/

doctest.js

• Pros
  • Tests are in a more readable format
  • Doubles as documentation
  • some builtin utilities for async testing (like wait())
  • runs in a browser, like production
• Cons
  • Might be tricky to integrate into CI

nodeunit

• Pros
  • can take advantage of the node.js world (like require statements!)
  • nice command line output
  • easy for CI
• Cons
  • Needs a mock dom (like jsdom?)
  • might take some fiddling to get things working outside of the Zamboni template stack (loading JS libraries, etc)
  • will it be similar enough to production? (i.e. Firefox)
• Notes
  • see also node-qunit (above)

Mock Objects

To simulate errors and not depend on a web server, it makes sense to mock out Ajax requests.

• The mockjax jQuery plugin works well for this