Background

What is Funsize?

Funsize (erstwhile Senbonzakura) is a Release Engineering project to improve the automation for the Update generation process for it's products. Read more about the internals on the project page for Funsize.

Etymology

The original working title for this project was 'Senbonzakura', which is roughly Japanese for 'Thousand Cherry Trees'. As you can imagine pronouncing and spelling 'Senbonzakura' is hard, thus began the search for a new name. As you will soon come to see, Funsize deals with a file format called MAR (Mozilla ARchive). What do you have when you have multiple MAR files? MARS! Funsize's job is to partly reduce the size of updates (Which are basically MAR files). I thought naming it 'mars-mini' would be clever. But we have a lot of people who also know the same delicious choclate by the name 'mars-funsize' Thus we decided to christen it 'Funsize'!

How Do I Start Hacking?

Congratulations and thank you for considering contributing to this project! To start hacking on this project there are a couple of things you need to do.

Prerequisites

You'll need a few things installed on your machine before you can start making changes to code.

1. Python (2.7+)
2. Git
3. virtualenv
4. RabbitMQ
5. SQLite (This may change to MySQL in the future)

Other things that you might require but are typically installed in your operating system are:

• Bourne Again SHell (BASH) -- your typical terminal in almost all *nix distributions will drop you into a Bash prompt.
• Curl -- the command line tool.
• Text Editor -- vi(m), Emacs, Sublime Text, Atom, whatever else you like/prefer.

If you also wish to hack on the deployment bits, you will also need to install docker. For how to install docker, see the Installation section for your relevant OS at https://docs.docker.com/

If you already have these installed or know of a better way to install these than what is going to be described below, feel free to use your method of choice.

Common ways to install things on your machine:

For OSX:

brew install python git rabbitmq # installs pip automatically
pip install virtualenv

For Fedora:

sudo yum install python python-virtualenv rabbitmq-server sqlite

For Ubuntu:

Caveat: The following are a best guess based on http://packages.ubuntu.com, but have not been tested.

sudo apt-get install python python-virtualenv rabbitmq-server sqlite

For Windows:

Windows is not supported currently (and probably will never be).

Getting the code

The code resides in two locations at the moment: - https://github.com/ffledgling/Senbonzakura/ - https://github.com/mozilla/build-funsize

Although Senbonzakura might be a couple of commits ahead at any point in time but build-funsize is the canonical repository and when in doubt, that's the one that should always be used. Unless ofcourse you know better.

TL;DR

virtualenv Funsize; cd $_; . bin/activate; git clone https://github.com/mozilla/build-funsize src;

Detailed Explanation

Typically Python applications are developed in isolated working environments called virtual environments or venvs . We will be setting one up for the development of our project as well.

The steps are as follows:

1. Create a virtual environment for our application.
   virtualenv Funsize
2. Enter the directory and activate the environment, this means we source another file into our shell env. . is a short-cut for the source command in bash.
   cd Funsize; . bin/activate
3. Clone the repository from Funsize
   git clone https://github.com/mozilla/build-funsize src

Running the application

After you've got the application. Let's make sure everything works by generating a proper partial MAR file.

TL;DR

• ./startup.sh
• tail -f /usr/local/var/log/celeryworker.log (In a new tab/terminal/pane)
• tests/curl-test.sh trigger-release
• tests/curl-test.sh get-release

Detailed Explanation

There's a few steps you need to take if you want to make sure everything's in shape. The application itself consists of 3 major 'running' components that need to be started separately, these are:
1. Flask (This is the frontend and web-server) 2. Celery (These are the workers that do the heavy lifting in the background) 3. RabbitMQ (This is the queue that allows the frontend and the backend to talk to each other)

There is a script in the root directory of the repository called startup.sh that starts up all the required services in one go. Open up the script and take a look at what's inside. If you understand what's happening, great!, if not don't worry we'll explain it soon enough. You can go run the script like so:

./startup.sh

If you don't want to use the script and run all the services in different terminals (A terminal Multiplexer is mighty handy in these situations), you can do that as well. We'll assume you're in the root of the repository.

• To run the Flask frontend: python senbonzakura/frontend/api.py
• To run RabbitMQ do the following: rabbitmq-server

This will start the AMQP Server (RabbitMQ is just an implementation of an AMPQ server in Earlang) You do not need to configure the RabbitMQ with any special credentials, the application uses the default credentials by ... default. TODO: Add way to configure both RabbitMQ creds and make the correct changes in the config of the application to keep everything still running correctly.

• To run the celery background workers you can do the following: celery worker -l DEBUG -A senbonzakura.backend.tasks This starts the celery with the tasks file listing all the functions that the workers can run. These workers are run in interactive mode with the log output level of DEBUG.

If everything went fine so far, ideally, the application should be up and running correctly, but we're going to send an API request to the flask server just to make sure.
To do so you can use your choice of REST API testing tool, you can find plenty of plugins for Firefox that will do the job. My tool of choice is curl and that is what I'll be using for the rest of this document. It's also what the application's integration tests rely on for the moment.

You can send an API request to the flask webserver (running on localhost:5000 by default) like so:

curl -X GET localhost:5000/partial/doesnotexist123

This should return an 'invalid' request response.

To make sure we got everything though, lets try generating a partial MAR. To do this we must make an HTTP POST request to http://localhost:5000/parital

Running the tests

TL;DR

Detailed Explanation

TODO: - Running unit-tests. - Running end-to-end generation test. - Applying the update to firefox after generation.

Typical Workflow

• important links to peruse before you start hacking on code:
  • Releng Best Practices
  • Mozilla Development Guide
  • Development Introduction.
    You can skip the step where you build Firefox, Firefox OS, Thunderbird etc. (Step-1)
  • MXR Code search
    Use this to search the entire Mozilla codebase!

TODO: - Correctly setting up your git.conf (username etc. etc.) - Git branches, new branch for new feature/bug# generate patch from master, rebaseing, merging, link to a good intro to git.

Before you do anything else, create a new branch for the feature/bug you're working on before touching any other code.

Finding an existing Bug to work on

If you're looking for an existing bug to work on you can find one in any of the following places: - Bugs Ahoy! - Ask in #releng on irc://irc.mozilla.org - Browse Bugzilla, or use this readymade query

Once you've found a bug you like, leave a comment on the bug expressing your interest. If the bug has a mentor mentioned you should log into IRC and contact the mentor. Even if the bug is not mentored, you're encouraged to drop by #releng and ask if someone can show you the ropes.

Note: If you're new and/or inexperienced look for a good first bug or a mentored bug. If you find none, ask in IRC and someone will be more than happy to give you something to work on. (Ask and Ye Shall Receive!)

Filling a Bug

If you've found a bug, please file a one and alert the people in #releng! All bugs for Mozilla related projects are filed in Bugzilla All bugs to be filed under:

Product: Release Engineering
Component: Tools

Important things to mention while filing a bug:

• What platform are you on?
• What steps did you take before you noticed the bug?
• Can you reproduce it? If so, mention the steps to reproduce the bug.
• Any other details that you think might be useful to someone who's trying to debug the problem!

Note: You will need a valid Bugzilla account to file a bug.

Submitting a Patch

A brief summary of how to submit a patch once you're done making changes follows. Please note this process is fairly consistent across all of Mozilla

• Create patch using git format-patch
• Attach a patch to the relevant bug on Bugzilla (if there isn't one, file one!).
• Leave a comment asking for feedback (set the f? flag) and enter the mentor's name/irc nick/email if there is no mentor for you bug, simply ask in IRC if someone would be willing to look at your patch and give you feedback. More often than not you'll find multiple people willing to do so.
• Improve/Update your patch based on the feedback and attach the new patch to the bug, asking for a review(set the r? flag for review).
  Note: Don't forget to mark the old attachment as obsolete
• What else should go in here?

TODO: Expand this to be more detailed.