Sisyphus/Installation

This document outlines the steps necessary for setting up Sisyphus to run the JavaScript tests.

Note that Sisyphus is currently implemented as a number of bash shell scripts along with some Perl and Python script. On Windows, this requires that you install and use cygwin.

Set up Windows

If you are not using Windows, skip to Install Sisyphus .

NOTE: these steps require Administrator privileges. Replace occurences of <youruserid> with either youruserid for a QA Farm machine or your local user id if you are installing on your local workstation.

NOTE: If you are running Vista, first turn off UAC

If you wish to use RDP to connect to the computer remotely, then right click "My Computer" (xp), System Properties, Remote, Allow users to connect remotely to this computer.

Windows - Install Cygwin

• Download setup.exe from http://cygwin.com/setup.exe
• Perform default cygwin install

Once you have performed the default Cygwin install, you should install the following packages:

Additional Packages

• dash
• binutils
• bzip2
• coreutils
• curl
• cvs
• cygrunsrv
• diffutils
• file
• fileutils
• findutils
• gawk
• gcc-core
• gcc-g++
• git
• gnupg
• grep
• gzip
• make
• mercurial
• mktemp
• openssh
• perl
• perl-XML-Simple
• python
• rsync
• sed
• shutdown
• subversion
• time
• unzip
• vim
• wget
• zip
• readline
• libncursesw-devel

To do this from the command line, use:

./setup -P dash,binutils,bzip2,coreutils,curl,cvs,cygrunsrv,diffutils,\
                file,fileutils,findutils,gawk,gcc-core,gcc-g++,git,gnupg,\
                grep,gzip,make,mercurial,mktemp,openssh,perl,perl-XML-Simple,\
                python,rsync,sed,shutdown,subversion,time,unzip,vim,wget,\
                zip,readline,libncursesw-devel

Windows - Start Cygwin

Start Cygwin by clicking on the Cygwin icon on the desk top. This will create /home/youruserid/ in the Cygwin file system.

Windows - Set up Cygwin Services

If you do not need to use the ssh server to access the machine remotely, or will not be running browser-based tests, then you can skip to Windows - Regional and Language Settings.

Run cygserver-config to set up as a service and choose Run as a service: yes

Start the service (only necesary the first time)

cygrunsrv -S cygserver

Windows - Install sshd as a Service (Optional)

This step is only necessary if remote access to the machine via ssh is required.

See http://ist.uwaterloo.ca/~kscully/CygwinSSHD_W2K3.html for details.

ssh-host-config -y

This will set privilege separation, create the required non-privileged account sshd, add the user sshd to the /etc/passwd and /etc/group files and add sshd as a service. See /usr/share/doc/openssh/README.privsep for more information.

Note: If you ever recreate the /etc/passwd file via mkpasswd -l > /etc/passwd you will need to change the default shell for sshd to be /bin/false.

Add sshd port 22 to the Windows firewall exceptions. Next, start the sshd service

cygrunsrv -S sshd

Windows - Install Apache as a Service (Optional)

NOTE: If you do not need to serve web content from the same machine where you have installed Sisyphus, then you do not need to install the Apache web server and can skip to Windows - Regional and Language Settings.

In order to run the the Cygwin Apache server, you will need to add a new user www for the web server to run under. Create User www in group Users, using a password, uncheck user must change password at next logon and check user cannot change password and password never expires. You should add www prior to installing Cygwin as the user www will automatically be added to Cygwin's /etc/passwd and /etc/group files. If you add a new Windows user later, you should update these files via:

mkgroup -l > /etc/group
mkpasswd -l > /etc/passwd

Next install the Apache web server as a Windows service.

cygrunsrv -I httpd2 -d "CYGWIN Apache 2.0" -p /usr/sbin/httpd2 -y cygserver -y dnscache -e "CYGWIN=server" -a "-D NO_DETACH" -u www -w <em><wwwpassword></em> -o

Goto Control Panel, Administrative tools, Services and double click CYGWIN Apache, Click Log On and set the password. Once it is applied www will have the logon as service right.

Start the web server (only needed first time as it will start automatically on subsequent reboots

cygrunsrv -S httpd2

Install Sisyphus

The default configuration places the Firefox build environment into /work/mozilla/builds/ and Sisyphus into <code>/work/mozilla/builds/hg.mozilla.org/.

Sisyphus Directories

• bin
• data
• plugins
  • darwin
    • components
    • plugins
  • linux
    • components
    • plugins
  • nt
    • components
    • plugins
• prefs
• results
• tests
  • mozilla.org
    • top-sites
• xpi
  • all
  • darwin
  • linux
  • nt

The environment variable TEST_DIR must be defined to point to the directory where Sisyphus was checked out. For the standard installation it should be /work/mozilla/mozilla.com/test.mozilla.com/www.

Add the following to your ~/.bashrc:

export TEST_DIR=/work/mozilla/builds/hg.mozilla.org/sisyphus

If you install Sisyphus to a different location, adjust TEST_DIR and the following steps as appropriate.

Create Directories

By default, everything in Sisyphus is installed under the work subdirectory of the root directory. To start, create a directory work as a subdirectory of your root directory and make yourself the owner.

mkdir /work  # Use sudo on Linux/Mac OS X
chown -R <youruserid>:<youruserid> /work

Windows: To keep the paths in Cygwin/Windows the same as used in Linux and Mac OS X, you need to change the cygwin mount point and create a soft link from /work to c:\work. Note that mozilla-build's Msys uses a different drive specifier /c for drive C: than Cygwin. For mozilla-build compatibility, we need to change the cygdrive prefix to / and create a soft link /c/work->/work

sed -i.bak 's|none /cygdrive cygdrive binary,posix=0,user 0 0|none / cygdrive binary,posix=1,user 0 0|' /etc/fstab
cd /
ln -s /c/work /work

Clone Sisyphus

The basic Sisyphus framework is available through mercurial.

mkdir -p /work/mozilla/builds/hg.mozilla.org # On Linux and Mac, use sudo
cd /work/mozilla/builds/hg.mozilla.org
hg clone http://hg.mozilla.org/automation/sisyphus
mkdir siyphus/results # directory where all test results are stored.
mkdir -p xpi/{darwin,linux,nt,all} # directories containing Firefox extensions to be installed
mkdir -p plugins/{darwin,linux,nt}/{components,plugins} # directories containing plugins to be installed
mkdir -p profiles/

• prefs - contains the default user pref files used to set Firefox and Thunderbird preferences for test runs.
• bin - contains the scripts and auxillary files used to implement the Sisyphus framework.
• data - contains configuration files used to set environment variables for given test configurations.
• plugins - Sisyphus installs plugins by copying "pre-installed" images from the plugins directory. To set up the plugins for a particular, platform, first install the plugin globally into an existing Firefox or Thunderbird installation. Then copy the components and plugins sub-directories of the Firefox or Thunderbird installation into the appropriate platform subdirectory, e.g. plugins/{darwin,nt,linux}.
• talkback - contains talkback.ini prototypes for each platform used for automatic talkback incident submission for Firefox 2 and earlier. XXX Need to update these for new platform naming convention introduced in bug 422942
• tests - contains tests organized by organization.
• xpi - contains extension xpi installers which will be globally installed into Firefox or Thunderbird prior to testing. If the extension is not platform specified, it should be placed in xpi/all. Otherwise, the extension should be placed in the appropriate platform specified subdirectory, e.g. xpi/{darwin,linux,nt}.

mkdir /work/mozilla/builds/{beta,aurora,nightly}

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug]
"Auto"="1"
"UserDebuggerHotKey"=dword:00000000
"Debugger"=""

Note: You can keep the other debugger registry entries around by changing their names. When I don't want the Visual Studio debugger, I change its name to <code>vdebugger, and when I want the debugger invoked, I change the "empty" debugger's name from debugger to bdebugger and vdebugger to debugger.

Windows - Turn off Windows Error Reporter

Note: See WER Settings

Under either

• HKEY_CURRENT_USER\Software\Microsoft\Windows\Windows Error Reporting
• HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Windows Error Reporting

Set Disable and DisableUI to 1 to prevent crashes from invoking the Windows Error Reporting dialog.

Mercurial

The environment variable TEST_MOZILLA_HG controls which repository Sisyphus uses. If TEST_MOZILLA_HG is not set, it will default to http://hg.mozilla.org/mozilla-central/.

The environment variable TEST_MOZILLA_HG_REV controls which branch of the repository Sisyphus pulls. If TEST_MOZILLA_HG_REV is not set, it will default to tip for the "trunk" of the repository.

When Sisyphus checks out source trees from hg.mozilla.org, it will first create a local clone of the requested repository, then clone that respository for each source tree created from that repository. The environment variable TEST_MOZILLA_HG_LOCAL controls where the local clones of the remote mercurial repositories are stored. If TEST_MOZILLA_HG_LOCAL is not set, it will default to $BUILDDIR/hg.mozilla.org/repositoryname.

For example, in /work/mozilla/builds/

• hg.mozilla.org
  • mozilla-beta
  • mozilla-aurora
  • mozilla-central

Configure Apache (Optional)

NOTE: If you are not going to run the browser-based tests, then you do not need to set up the Apache web server and are finished with the set up of Sisyphus.

The environment variable TEST_HTTP controls which web server is assumed to host the tests. If TEST_HTTP is not set, it will default to http://test.mozilla.com/.

Configure /etc/hosts

The name test.mozilla.com is resolved to the local machine using the /etc/hosts. In Windows and Linux, you can add the address of test.mozilla.com to your /etc/hosts file. On Mac OS X, you must use either NetInfo on Mac OS X desktops or Server Admin on Max OS X servers. For more information about configuring virtual hosts on Mac, see Enabling Virtual Hosts on MacOS X.

127.0.0.1 test.mozilla.com

In order to activate the virtual web servers in Apache, You will need to add the configuration directives to your Apache configuration:

• 0-name-virtual-host.conf - turn on virtual hosts, required.

  NameVirtualHost *:80

• 1-test.mozilla.com.conf - create a test.mozilla.com virtual host, required.

  <VirtualHost *:80>
  ServerName test.mozilla.com
  DocumentRoot "/work/mozilla/mozilla.com/test.mozilla.com/www"
  AddHandler cgi-script .cgi
  <Directory "/work/mozilla/mozilla.com/test.mozilla.com/www">
      AllowOverride All
      Options Indexes FollowSymLinks ExecCGI
      Deny from all
      Allow from 127.0.0.1
      Satisfy all
    </Directory>
  </VirtualHost>
  

If you have customized the location of TEST_DIR, adjust the paths appropriately.

The Apache configuration can be found in /etc/httpd/ in Linux and Mac and in /etc/apache2/ in Windows (Cygwin). If your installation has the conf.d directory, you can simply copy the necessary configuration files there and comment out any directives in the /etc/httpd/conf.d/welcome.conf file. If your Apache installation does not have the conf.d directory, you can append the contents of the configuration files to the end of the httpd.conf file in your Apache configuration directory. Note that 0-name-virtual-host.conf must appear before the other configuration directives.

Once you have updated the configuration files, you must restart Apache for them to take effect. On Linux and Mac, do sudo /usr/sbin/apachectl restart while on Windows (Cygwin), do /usr/sbin/apachectl2 -k restart.

NOTE: Make sure that Apache is set to start at boot.

Spider

If you are running the browser based tests, then you need to build the Spider extension.

cd $TEST_DIR/spider
make
# spider.xpi will be placed in $TEST_DIR/xpi/all

Back to Sisyphus