Current Manual Testing

code

https://github.com/jonallengriffin/dirtyharry

results

Top 500 addons

Linux:

sorted average Talos Ts

Talos Ts

Automated Testing

Limitations

• no plans to allow addons to 'call home' - we will still be working in the talos testing environment where we are proxied to localhost, so there will be no live web interaction
• no current plans to interact with the addon (no clicks, no visiting specific pages)
  • this sort of perf test would have to be designed/built per-addon to get the most bang for the buck

Plans

• integrate into buildbot and have run on production machines
  • how frequently?
  • which tests?
    • for now, we'll limit to clean ts starts with just the addon installed
  • where would the list of addons be maintained?
  • where do we download the addons from?
• results reported as .csv files
  • where should these be sent?
  • do we want the data on the graph server?
  • how do we compare the results?

Information for Add-on Authors

• How to test for performance differences yourselves: Measuring Startup.
• Authors of add-ons with the greatest performance impact are being contacted about this, almost always with suggestions on where to improve.
• How to Improve Extension Startup Performance.