Sfink/Projects

Add-on Performance Cost

Goal: Detect performance costs of individual add-ons

Vague plan: Find entry point/s where add-ons create their JS, and remember which scripts belong to which addons. On script exit, associate the elapsed time to the nearest containing add-on in the JS stack. Report time due to add-ons. Have a separate report for startup time. For non-startup time, report percentage of non-idle time in each startup.

Handwaving: Maintain a separate "add-on stack" that only gets updated when an add-on different from the topmost add-on gets pushed onto the stack, on exit blame elapsed time minus time blamed on inner add-ons to the outer add-on. Challenge is doing all of this cheaply. I wonder if all these crossings are guaranteed to be cross-compartment, so I could ride on

Issues: This blame mechanism is incomplete. It either doesn't account for extra rendering time due to an add-on, or possibly overcounts rendering time on non-addon elements that were triggered by add-on code.

Do something semi-reasonable when accounting for multiple threads.

Chrome/add-on Performance Analysis

Goal: Figure out where a specific add-on is spending its time (to either optimize the add-on or the engine)

Bug 558200 addresses this to some degree. (It uses the FunctionTimer to get insight into what "expensive" activities were triggered by an add-on.)

Overall Browser Performance Benchmarking

Goal: How fast is this browser? Specifically: how long does it make a user wait?

Vague plan: Maintain a global notion of whether the user is waiting or not. Infer as much of this as possible from the event queue, and perhaps the stream of user actions.

Handwaving: It would be nice to have videos of browser sessions manually annotated with "user is waiting" intervals.

Subgoal: This can be applied to components too, for example garbage/cycle collection -- we have heuristics for deciding when to trigger a collection, trying to make it happen when it's not going to make the user wait. But it's a tricky tradeoff: if you only collect when more memory is needed, you're pretty much guaranteed to make the user wait for every collection. But if you aggressively collect whenever the browser is idle (or even more frequently), you'll end up making the user wait for collections to finish every time s/he does anything, instead of just occasionally (not to mention the amount of wasted work scanning through the same live objects over and over.) And it's important to know when the user is actually waiting; optimally you always collect just before the user does something that requires more memory. Or something. (Even when assuming full knowledge of the future, figuring out the optimal behavior is hard.) Which just means that it is important to have a realistic metric of GC cost, one that doesn't just look at the total time spent in GC.

Realtime Performance Benchmarking

Goal: Make Firefox do well in situations where jitter is important

Discussion: Now that Firefox has WebGL, video, and canvas, and is getting used for gaming, jitter and predictable timing are getting more important. For some types of games, minor variations in callback frequency are highly visible in the form of stuttering movement (or even worse, audio dropouts) and can even make the platform unusable. Game engines care a lot about this. I don't know, but my guess is that we haven't done a lot yet to measure and tune for jitter.

Vague plan: too vague to put down.

In the Field Performance Profiling

Goal: When a user notices a performance problem, they can figure out what's causing it and perhaps help make it better in the next version

This could be split into a whole bunch of different subgoals, depending on (1) whether this is a regular release user (maybe with, maybe without admin rights), a TestPilot user, someone who can download and rerun a debug build, or whatever; (2) whether the mechanism needs to be retroactive (ie always-on flight recorder stuff) or if the user can re-run the scenario at least once after turning on instrumentation; and (3) how the information is presented and who converts raw data into meaningful information -- is this integrated into the browser? Is the data even usable by the user, or is it in some opaque log file that can be attached to a bug or sent to someone for analysis?

Motivation: there seem to be many horrendously bad perf-related episodes out there that we don't see in our controlled environments. I want to be able to get useful information in a way that people might actually follow through with, rather than just whining about it. And when my friends & family complain to me about things, I want to have some hope of actually telling them what's going on.

Vague plan: Install a tool with the browser that the user can run to capture traces (eg ETW trace logs, but possibly others as well) and allows the user to upload them. Require re-running with tracing active for now, and don't worry about making nice in-browser UIs or anything. Make sure we can consume (analyze) users' traces on our machines (raw trace files sometimes don't contain the necessary metadata.) Provide a tool that can display the collected information, preferably with ways to see browser-specific events and to narrow the view to the critical portions. Enable the browser to emit browser-specific events (so it can tell you "collecting garbage" rather than "unknown internal browser stuff".)

Vaguer plans: Come up with an alternative tool to xperf that is cross-platform, and probably hosted within the browser. Accept ETW traces as well as other forms of profile-y input. Make it extensible so it can accept custom browser-specific events. Mimic SpeedTracer to some extent. Think of it as the unholy love child of Shark and SpeedTracer.

Issues:

• ETW is only for Windows
• Working on Windows makes me beat my head against a wall until bloody
• I only have one head

Web Developer Performance Tools

Goal: Allow web page/app developers to clearly see and understand the performance characteristics of their apps. This includes networking, layout, rendering, memory usage, etc.

Vague plan: This is probably best done by augmenting Firebug, web console, etc. with whatever is needed. SpeedTracer probably has the right set of taillights to chase right now. The Shark/SpeedTracer-like thing mentioned above could also be relevant here, but it would be a bit more internally focused.

Performance Benchmarking Suite

Goal: Get and track solid numbers representing real-world browser usage, slanted towards sites/apps where performance matters.

Vague plan: Construct canned test scenarios closely matching real-world usage scenarios, by creating tools to record and replay complex applications. See https://wiki.mozilla.org/User:Sfink/Working_On#Performance_Analysis for more detail. Set those scenarios up as automated tests (probably running on nightlies, not on commits.) Extract and display useful perf data mostly reusing existing infrastructure (graph server etc.)

Javascript Debugger

Goal: make the existing JS debuggers work well.

Note: JSDv2 is out of scope for this. Here, I'm only talking about the JSDv1 clients. Which turns out to be fairly constraining, because JSDv1 has a lot of problems even before the introduction of JM and compartments, and the best way to fix many of them would be API changes. But there's not much point in doing that with JSDv2 on the horizon (and the minor fact that FF4.0's feature set is supposed to be locked down right now.)

Vague plans: Single stepping is mostly fixed after being clobbered by JM. Break-on-next is in progress (as in, possibly fixed, but I don't understand what it is well enough to test it, so the Firebug folks are looking at it.) I may take a stab at some small API changes that would fix the larger nuisances.