CI Automation

From MozillaWiki
Jump to: navigation, search

Welcome to the CI Automation team's landing page!

Who we are

We are a team of 9 engineers, 2 managers, 16 sheriffs (24x7 shift work in teams), and always a few interns, volunteers, and contractors.

Our team is split into 3 categories:

  • Integration: integration of tests, platforms, configs into our CI system
  • Reporting: dashboards and tools to view and work with data and the CI system
  • Sheriffing: code sheriffing (not perf sheriffing) for mozilla-central based trees

Goal

The CI Automation and Reporting teams are responsible for ensuring that we schedule appropriate tests, those tests produce useful results, we can view the results of the tests and collect data to make decisions, and finally that we are experts with the tools and provide sheriffing features for Firefox.

Mission

To make the scheduling and reporting of our test results more accurate and easier to understand, leading to a yes/no answer from a try push.

Vision

Our CI system schedules tests as efficiently as possible, highlights regressions as quick as possible, and our tools to monitor and manage what we schedule, what is run, and what was run is easy to use.

How we work

We don't write automated tests for Firefox features, but we love to work with teams that have unique scenarios for automation or just questions. We work closely with product development to ensure that our tests are run in the right environments and configurations, that developers know how to test their code and understand their results, and when code lands the sheriffs actively monitor the trees.

Requests are done via Bugzilla or IRC chat (#cia, #treeherder, #sheriffs). We prefer to have 2-3 weeks heads up on projects and more if possible. Our work is 20% reactive where we help put out fires or treat a new issue as a P1, and 80% proactive where we plan work and get it done.

Integration:

Routine work we do

  • new platforms
    • windows10/aarch64
  • new configs
    • fission
  • intermittents
  • test efficiencies

Other work we do

Contact Us

We hang out on irc.mozilla.org in the #cia channel. Feel free to drop by and say hello, we're a friendly bunch. Here are some related channels that might also be of interest:

  • #ci: General channel for CI-related discussion
  • #cia: Our team channel
  • #developers: Channel where developers all across Mozilla hang out
  • #introduction: Channel where newcomers can ask questions around contributing, processes, or anything

Note on IRC communication

While IRC may appear to be a real-time communication channel, it is also used for asynchronous communication. Many people connect over persistent servers and will appear online even when away from their computers. Or sometimes they are simply busy at the moment and will respond when they have time. It's also easy for a question to get missed in a busy channel. So don't be discouraged if you don't get a response right away and don't be afraid to ask again at a later time.

Contributing

There are so many different aspects to contributing that implementing the patch is often the easy part! This is a high-level overview of the contribution process, with links to resources that go into greater detail for each step.

Finding a Bug

The best way to find a bug is to ask us on IRC! Many of us have tasks set aside just for this purpose. But Mozilla also has several tools to help you find good bugs to work on:

Generally all issue tracking happens in Bugzilla, so this is one of the first accounts you'll need to make.

Finding the Source Code

Once you've been assigned a bug by someone (you won't have permission to do this yourself initially), you'll need to grab the source code. Most of Mozilla's code lives in a monorepo called "mozilla-central" (often abbreviated "central" or "m-c"). This is a Mercurial repository that lives on hg.mozilla.org.

You can use either Mercurial or Git to interact with the repository. Though Git support requires that you install a special library called git-cinnabar. While we do try to support git with cinnabar as much as possible, there might be workflows that don't work as nicely as their Mercurial counterparts.

Here are some resources to help you get set up with the source code:

Building Firefox

After cloning the source code, one of the most common first tasks is building Firefox. While the work our team tends to do rarely requires a build, it's still a good step to help familiarize yourself with the repository and some of the tools we use.

Follow these instructions to set up your dependencies and perform a build (note: if you've already cloned, you can run ./mach bootstrap instead of downloading bootstrap.py). Once your build is finished, use ./mach run to spin up your very own Firefox!

When going through this process, you'll notice that you'll interact a lot with mach. Mach is a command-line dispatcher that we use to provide a central entry point to a vast array of tools. You can look through ./mach help to get a sense of all the different tasks that can be performed in the repository.


Bonus Challenge

Find a string in Firefox (e.g. in one of the menus or preference dialogs) and make sure that it's fairly unique. Then try and use searchfox to find where that string is defined. Change the value to something else and re-run a build. Then use ./mach run to see if your change is there!

Fixing the Bug

Now that you have the source and have familiarized yourself with the repo a bit, it's time to fix the bug! You can use any platform or editor that you want. If you get stuck, don't be shy to ask for clarification. IRC, the bug or e-mail are all appropriate places to ask.

Once you have your fix commit it with a message that follows Mozilla's guidelines. Be sure to test your patch (ask your mentor if instructions for testing weren't included in the bug). You should also run ./mach lint --outgoing to make sure you didn't cause any lint errors. If the patch is complicated enough, your mentor might also want to do a try push (you won't have permission to do this on your own, so they'll do it on your behalf).

The Review Cycle

After you've committed and tested your change, it's time to get it reviewed! Mozilla uses a tool called phabricator to perform reviews. You'll need to make an account for this as well, following the documentation here.

Once you're set up use arc diff (or better yet moz-phab submit) to push your changes to Mozilla's phabricator instance. If all goes well the reviewer (usually your mentor) will Accept your changes and land your patch. You can remind them to land it on your behalf if they forget.

But usually, the reviewer will Request Changes and leave comments telling you what to do. Don't feel bad if this happens, it's perfectly normal! In fact even the most experienced developers need to re-submit their changes more often than not (and sometimes even multiple times in the same bug). If this happens to you, the most common way to deal with it is to use history rewriting. History rewriting is a term used in the version control world which simply means modifying existing commits instead of creating new ones. This helps keep your repo clean from so called "fix-up commits".

If using Mercurial, some history rewriting capabilities include:

  • hg commit --amend
  • hg histedit
  • The evolve extension

See Mercurial For Mozillians for more information.

If using Git, some ways to rewrite history include:

  • git commit --amend
  • git rebase --interactive

Once you have the requested changes amended to the proper commits, you can re-submit them to phabricator and the process will repeat itself.

What to Expect After Landing

Congratulations, your patch just landed! But what does that mean?

Mozilla has a multi-tiered branching system (in reality each branch is a separate repository, but let's gloss over that fact for now). As we mentioned earlier, all mainline development happens on mozilla-central. Since we are all human, every so often we accidentally land something that causes things to break (e.g we might cause a test to fail). Now imagine that developers were landing their patches directly on mozilla-central and occasionally causing things to break. If you were developing your own patch and notice a test failure it would be really hard for you to determine whether your patch was to blame, or if you just happened to be based on someone else's change that caused it to break. At a scale of hundreds of developers it's not hard to imagine how this would get out of hand very quickly!

Instead Mozilla uses something called an "integration branch". These are branches where it's *ok* if things get broken since no one bases their local work on top of them. If your change breaks something, you'll get "backed out" (i.e the exact reverse of your commit will be committed). In this scenario, someone will comment on the bug to notify you of the backout and it will be your (and your mentor's) responsibility to figure out what failed and how to fix. Again, don't feel bad if this happens, it's a normal part of our workflow.

If all the builds and tests pass on the integration branch, your patch will eventually be merged into mozilla-central (these merges typically happen twice a day). At this point you're in the clear! The next time someone pulls in the latest changes from central, your changes will be included. Congratulations!

You can see all the build and test results using a service called treeherder. On the left you'll see the push information (who pushed and which commits were in the push). On the right you'll see the results. Each symbol represents a "task" which runs builds or tests. If something fails, the symbol will turn orange (or red). It's normal for every push to have a few "intermittent failures" on it. If your patch landed via phabricator, you'll be able to see it on the autoland branch of treeherder.

Projects

This is an incomplete list of historical, current and future projects.

Windows 10 ARM64

  • Project page: link
  • Contact: gbrown@mozilla.com, egao@mozilla.com

Glossary

Mozillians use a dizzying amount of terminology that can make any new contributor's head spin. You'll often be able to find what you're looking for in the Glossary. If you come across something not mentioned there, feel free to ask us in IRC (and remind us to add it to the list).

Retrieved from "https://wiki.mozilla.org/index.php?title=CI_Automation&oldid=1219108"