User:MarkFinkle/RunningTests

From MozillaWiki
Jump to: navigation, search

Being able to quickly run tests is an important tool for developers. Here is some information about running different tests with Fennec.

Browser Chrome

Browser chrome tests are important for testing the UI itself. Make sure you --enable-tests in your MOZCONFIG. You do not need a debug build, but you do need to enable tests.

cd objdir/mobile/_tests/testing/mochitest
python runtests.py --browser-chrome --appname=../../../dist/bin/fennec --test-path=mobile
  • --appname is needed because the test framework always assumes it's running Firefox by default.
  • --xre-path is needed so the test framework can find the runtime libraries.
  • --xre-path=../../../dist/bin/xulrunner is needed if you are running a XULRunner enabled build.
  • --setenv=XPCOM_DEBUG_BREAK=warn might be needed if the build is throwing a lot of assertions. By default, runtests.py will set this to "stack" and spew a lot to the console.

This example is for running tests from your build.

Reftests

Reftests are important for testing the web content rendering. There are ways to run reftests from a desktop build, but it's more important for Fennec to run reftests on the device.

Our build process will package the reftests (and others) into a handy archive. Look for the xulrunner-*.multi.linux-gnueabi-arm.tests.tar.bz2 files in the nightly or trunk FTP folders.

1. Make a /opt/unittests folder on your N900.

2. Copy the archive to the /opt/unittests folder on your N900. You might want to bunzip2 the archive before copying to the device, since the N900 has "tar" but not "bunzip2" by default.

3. Unarchive the file and remove it. You should have "bin", "certs", "mochitest", "reftest", "xpcshell" folders.

4. You should have a version of fennec installed somewhere as well.

5. You can run the reftests using:

python reftest/runreftest.py --appname=/full/path/to/fennec reftest/tests/layout/reftests/reftest.list

Crashtests

Same setup as reftests, but run the tests using:

python reftest/runreftest.py --appname=/full/patch/to/fennec reftest/tests/testing/crashtest/crashtests.list

Talos Performance Tests

Same setup as reftests, but run the tests using:

python talos/run_tests.py --screen --debug --noisy mobile.config

mobile.config is a configuration file that setups up the appname (fennec) and the tests you want to run. Talos comes with a sample config you can use to make a mobile specific file. Make sure you use the mobile profile_path (mobile_profile)

Tp Tests

To run Tp tests on N900 or Android (using Zippity) you currently need a webserver. I use nginx, as does tinderbox. Configure nginx. For me, the file is /etc/nginx/nginx.conf and I need to use sudo to edit it.

For the N900, youare installing directly to the N()) and the %USER% is "user", %IPADDR% is "localhost" and %ROOT% is wherever you installed the talos Tp set. I put mine at "/opt/talos".

For Android and Zippity, you are installing to a Linux desktop machine and %USER% is your username, %IPADDR% is the Linux box IP, and %ROOT% is somewhere in your home directory - where you copied the Tp set.

worker_processes 1;
user %USER% users;

events {
    worker_connections 1024;
}

http {
    include /etc/nginx/mime.types;
    default_type text/plain;

    sendfile on;
    keepalive_timeout 65;

    server {
        listen 80;
        server_name %IPADDR%;

        location / {
            root %ROOT%;
            index index.html index.htm;
        }
    }
}

I typically need to start or restart nginx:

Start – "sudo /etc/init.d/nginx start"
Stop – "sudo /etc/init.d/nginx stop"
Restart – "sudo /etc/init.d/nginx restart"