« Back to Update:Remora

Test Framework

We use the SimpleTest php testing framework and some helper files for unit tests dug up from the grave of the internets.

Test Structure

Test locations

All test related things can be found in app/tests. These are the items you'll need to deal with in this directory:

• controllers - controller tests are placed in this directory
• controllers/components - component tests are placed in this directory
• models - model tests are placed in this directory
• views - view tests are placed in this directory
• groups.php - test groups are modified here
• services - Tests for external services that are not cake-dependent will be placed here (pfs, update, blocklisting)
• global.test.php - global test file included with all tests

Test Data

Test data is at app/tests/data/remora-test-data.sql. This is data only, you need to run app/config/sql/remora.sql first to get the schema setup. Currently, MicroFarmer(id 7) is the most data-complete addon.

Taking a Dump

Use phpmyadmin to create a test dump, and select all tables except for addontypes and cake_sessions. Uncheck "Structure" and check "Disable foreign key checks". Select all tables except addontypes and cake_sessions. Finally, check "Save as file" and click Go(bottom right corner). You will need to remove certain rows from the dump that are also in remora.sql, but the best way to find those is to try to run the SQL and correct the duplicate key errors.

Test Naming

All tests should be named according to their real names and locations but with the extension .test.php. This is important for the SVN links to work properly. For example:

• app/controllers/addons_controller.php -> app/tests/controllers/addons_controller.test.php
• app/controllers/components/amo.php -> app/tests/controllers/components/amo.test.php
• app/models/addon.php -> app/tests/models/addon.test.php
• app/views/addons/view.thtml -> app/tests/views/addons/view.test.php

Web Interface

The web interface for tests can be found at the root level. So, if your site is http://addons.mozilla.org, the tests will be found at http://addons.mozilla.org/tests. The main test page has two columns, one with single test cases, and one with group tests.

Clicking on a single case runs that test. Clicking on a group test runs all cases in that group. Clicking on run all runs all tests.

Groups

To create a new group or add cases to a particular group, just open up groups.php and modify the arrays there. Please make sure your group works after editing this file, as it is easy to add an extra comma and break all group tests.

Note that you can use * as a recursive listing of all tests in that directory. For example, putting controllers/* will include all tests in the controllers directory, as well as all tests in the controllers/components directory.

Writing Tests

Model Tests

Coming soon!

Controller Tests

Below is a sample controller test. The testLoad() method loads the controller, mocks all the models referenced by $uses, and mocks all components.

To call a controller action, callControllerAction should be used. This manipulates the output buffer so that nothing will be output by the views, among other things.

<?php
class DevelopersControllerTest extends UnitTestCase {

    /**
    * Setup the Developers Controller
    */
    function testLoad() {
        $this->helper = new UnitTestHelper();
        $this->controller = $this->helper->getController('Developers', $this);
        //$this->helper->mockModels($this->controller, $this);
        $this->helper->mockComponents($this->controller, $this);
    }

    /**
    * Tests something
    */
    function testSomething() {
        $this->assertTrue($this->helper->callControllerAction($this->controller, 'add', $this));

    }
}
?>

Component Tests

Below is a sample test that only tests a single component.

<?php
class AmoTest extends UnitTestCase {
    
    //Setup the Amo Component
    function setUp() {
        loadComponent('Amo');
        $this->Amo =& new AmoComponent();
    }

    //Make sure the addontypes is an array
    function testAddonTypes() {
        $this->assertIsA($this->Amo->addonTypes, 'array');
    }
}
?>

View Tests

Below is a sample test of a view.

class AddonTest extends WebTestHelper {

    function AddonTest() {
        $this->WebTestCase("Views->Addons->Display Tests");
    }

    function setUp() {
        $identifier = 7;//$_GET['id'];
        $model =& new Addon();
        $model->id = $identifier;
        $this->data = $model->find("Addon.id=$identifier", null , null , 2);
        $this->getAction("/addons/display/" . $identifier);
    }

    function testRemoraPage() {
        //just checks if the page works or not
        $this->assertWantedPattern('/Mozilla Add-ons/i', "pattern detected");
    }

Nagios

Nagios is a tool used by IT to periodically ping certain pages to check for responsiveness. It is the first line of automated testing that we have to ensure our site is up and operational. Below are definitions of Nagios tests that are implemented (and a list of ones that should be but aren't yet).

Tests currently running

Tests we need to set up

Other Tests

In general we'd want to use the same framework for external tests. External tools should go in app/services and have corresponding tests in app/tests/services.

Why do we write tests?

We write tests because they let us go faster, even early in development, by revealing downstream effects of changes earlier.

Some good reads:

• Extreme Programming
• Test-driven Development
• Design-by-Contract

We write tests because they improve the quality of our code, by pushing us to factor better, reduce dependencies on other state, and keep control flow simpler.

We write tests because they document code better than comments, and because seeing a green bar makes us feel tingly inside, like when Sally held our hand on the school bus in grade 4.

We write tests the best we can, knowing that we won't be perfect, and adding more tests as we encounter them. We know that writing tests as we write code will encourage us to make smaller changes and focus on the specific problem at hand. We are learning to trust and fear the feeling in our stomach that says that a change is risky because there isn't a good test for some of the things it affects.

We think morgamic is pretty cute. But we don't have a test for it, so we're not really confident.