User:Bhashem/InstrumentationThoughts
Contents
Problem Statement
The Firefox team needs more data about how users actually use Firefox in the field.
Proposal
Although, we can obtain this data through detailed user studies. A more scalable solution is to technically instrument the browser to collect anonymous usability information and submit to a central site. A web application would aggregate this information into actionable summaries.
Detail
There are several types of Firefox user data that would be useful to help guide product decisions and provide visibility about how users customize Firefox. There are the following "per user" data sets:
- Static metrics [S] (e.g. current number of bookmarks)
- Dynamic metrics [D] (e.g. average number of search done per session)
- Trend data of static and dynamic metrics [T] (e.g. average number of open tabs across sessions).
In addition, aggregated metrics from all user data collected can be calculated.
- Average static metrics [aS]
- Average dynamic metrics [aD]
- Average trend metrics [aT]
Other stats include low/high values across all users.
Profile Sets
A profile set is a particular set of values for various static metrics. You can think of values that are "burned in" as part of the Mozilla browser source code as a set "Source". When Firefox branding is applied, it creates set "Firefox". When each localization is created, it creates set "Firefox en-US" and "Firefox de". When a Google partner build is created, it creates set "Firefox zh-TW Google". All these sets are "starting values" (or defaults) for end users. Users in different countries and who have obtained Firefox from different sources will have different starting profile sets.
As Firefox is customized by end users, this creates unique profile sets. We can compare the values in these customized sets versus what was shipped as the starting value for that particular browser. This tells us how much customization has been done by the end user.
User customized values are uniquely identified in the preferences database so, it should be easier to call out customizations (e.g. if a user changes their home page, it's noted). What this doesn't tell you is if the new value is different than the as-shipped value. It only tells you that it's been customized. The point of the above is simply to outline that there is no single, authoritative set of values that can be considered the Firefox default set for all Firefox releases. (Note: there may be a subset of preferences for which this is true, e.g. Javascript being enabled, etc...)
Subject Areas
Operating & Install Environment
- Operating System Version
- CPU (type/speed)
- Memory (installed)
- Disk (size/available)
- OS Locale
- Fx Install Dir
- Fx Profiles Dir
- Number of Profiles (in ProfileManager vs. on disk)
- Original version installed
- Update Log (may include trouble/errors in updating)
- Current Version
- Build ID
- User Agent String
Sessions
- Number of sessions per day
- Mem (available)
Search
- Number of plugins installed
- Number of searches per session
- Number of times people switch per session
- Number of times
Bookmarks
Extensions
Plugins
Password Mgmt
Prefs
- Changes from the default
Home Page
- Changed from default?
about:config
- Values not set to defaults?
References
- MozMetrics Project from Seneca
- Opera and and independent entity has setup some interesting stats and drawn some good conclusions about Opera usage.
- Instrumented Gimp
Firefox 2 metrics work:
