Electrolysis/Firefox/20101117

Brainstorm - Tools for Measuring Responsiveness

Alex's Wish List

I met with Alex Limi to brainstorm a list of all user actions he wishes he could measure - ignoring priority.

General Scenarios:

• Opening new window/closing a window
• Opening a new tab
• Trying to do anything with the browser when there is a long latency, how busy the browser is...waiting for something.
• Awesome bar results - how fast it returns
• Auto-complete in the URL bar (inline auto-complete)
• Closing tabs
• Closing down the browser
• Going into and out of panorama (with large number of tabs)
• Time from starting the browser to typing in a url
• Resizing windows
• Switching between tabs
• Switching between tabs - where one tab is showing an image
• Enabling and disabling toolbars/sidebars
• Bookmarks/history - deleting, creating, moving in the "places database"
• File dialog actions
• Print dialog actions
• Going in and out of full screen mode
• Opening the download window
• Opening any in content pages - add-ons manager
• Switching between sections in the ad ons manager
• Opening the prefs window
• Individual panels within prefs - applications
• Start/stop private browser
• Clearing history (1 week, 1 day)
• How long from when you signal you want to quit the browser to when the window disappears.

Specifically on Windows (some things render non natively on Windows):

• Opening/closing menu

Meeting Discussion

• We met to discuss the development of a tool to better measure application responsiveness.
• We don't currently have something.
• We will need something for Electrolysis that helps us determine how we are doing against our targets and helps pin point areas in which we will regress or have problems.
• We would need to define our baseline machine - what's the minimum machine would should be using.
• Some questions came up around garbage collection.
  • We could run tests without garbage collection.
  • We could keep garbage collection but be notified when it starts and stops.
• In discussion with the Tools team it looks like there are some basics in place already that we can build on.
  • We already measure the time it takes to do things - do that in Talos now.
  • What is perhaps more difficult is what happens to these actions over time, after some period of usage.
  • We agreed that some visual on top of the data would be easier for developers and others to interpret.
  • The take-away is that we can probably develop something iteratively, starting with a short list of actions we want to track.

Next Actions

• Sheila and Chris will pick out a list of 6 or so key actions that we would like to measure.
  • 1. Startup
    • Time between startup and visible window
    • Time between visible window appearing and the browser is completely responsive (type in a url)
  • 2. Switching tabs
  • 3. Opening a new window
  • 4. Quit the browser
  • 5. Awesome bar search results (how fast it returns).
  • 6. Right click menu.
  • 7. Responsiveness in foreground tab during page load.
• Bob's team will group Alex's wish list into things that are easy/hard to measure.