Performing autonomous network requests from any population of Firefox users needs special care. Simply put, making requests to central services from Firefox is inherently a DDoS waiting to happen. This page has a set of design guidelines, and suggestions for how to proceed in a safe fashion. You don’t want to join the list of people who’ve caused an outage of essential services!

This guidance is relevant whether you are adding code into mozilla-central, developing a Go Faster system addon, a Web Extension, or something else entirely.

Design Guidelines

Talk to the operators of any machines that will service the request

Don’t expect the release or approval process to identify the stakeholders for you. You need to reach out and get buy-in from the operators of any URLs you fetch. If the services aren’t run by Mozilla, you should talk to legal as well as getting buy-in.

If you don’t know how to find the stakeholders, start by talking to MOC (Mozilla Operations Center), either in IRC or by email.

Use the minimal network request possible

Unless you actually need to issue an HTTP request, don’t. If you do need to send HTTP, consider using HEAD instead of GET or POST.

Make sure that the URL being fetched is static - e.g., that it can be served efficiently.

Talk to the operator about the effects - both good and bad - of caching, depending on what you’re trying to accomplish.

Identify yourself

If you’re speaking HTTP, put the tracking bug number you’re using into the User Agent string. Everyone should be on the same page because you’re coordinating, but if something unexpected happens, it’s extremely helpful to have a way to figure out context for a flood of requests.

Use a sampled rollout

If this is a system add-on, use mechanisms to affect subsets of the Firefox users at a time, and ramp it up. Go Faster contains mechanisms for this, and certainly there are plenty of ways to accomplish the goal.

Communicate the rollout schedule to the operators of each service, so they know when to expect traffic.

Code in a back-off

Every HTTP request to a central service from Firefox needs to have logic to recognize the symptoms of a remote service in trouble and back off appropriately. For example, Firefox Sync interprets HTTP 503 as "don't hit the server for N minutes." There is also a mechanism in Firefox Sync where the server can instruct the client via an HTTP response header to not talk to the server for N seconds. This allows inadvertent DDoS to be mitigated by fast responding with a "go away" error response from the server.

Server operators should understand the back-off code that is implemented, so they are prepared to save themselves should the deployment go awry.

Getting Reviews

You absolutely should get a code review from someone with experience either causing or fixing outages before you proceed. Reach out to MOC for reviewer suggestions (or, if that fails, to one of the authors of this wiki).

Notes

This page was created as part of Bug 1353532: "Publish guidance for system addons that perform network requests or network investigations"