User:Polvi/InstrumentationThoughts

From MozillaWiki
Jump to: navigation, search

Synopsis

Understanding how people use Firefox will help everyone see how people use the internet.

Ground Rules

  1. All users must maintain anonymity
  2. Results must be open
  3. Participation must be opt-in

Abridged Plan

A super-high-level order of operations...

  1. Determine how we will use the results
  2. Determine initial set of metrics to be collected
  3. Determine how we will sample and collect said metrics
  4. Determine how we will analyze the the data
  5. Implement client and server software
  6. Implement analysis framework

Implementation

Overview

Extension

A group of students from Seneca are hacking on an extension to start reporting feature use data.

Backend

Mozilla will maintain infrastructure where data is collected and stored.

API

Data in the backend should be accessible via an Mozilla supplied API. This will help us distribute the data, along with do our own analysis.

Analysis

Will work with Kovash to make sure our data collection syncs up with sane analysis.

The Data

Determining what to collect

In order to determine what to collect we will collaborate with the various groups involved with the Mozilla project.

Front-end developers

  • How many tabs are used at one time?
  • How many bookmarks?

Back-end

  • How many plugins are loaded?
  • How many images are on a given web page?

Marketing

QA

IT

Products

Extension Distribution

Simple download

The simplest way to distribute the extension is to offer it on addons and see who takes it.

  • Pros
    • Clearly opt-in
    • Collects data on savvy users
  • Cons
    • Limited to users that know about addons

Installer

Offer an opt-in check box during the install process.

  • Pros
    • Allows users to "help make Firefox better"
  • Cons
    • Not really our style, a bit too MS like

Distribute on public workstations

Pre-install extension on systems in very public places, like libraries and internet cafes.

  • Pros
    • Very anonymous sampling of users
    • Opens data to potentially non-savvy users.
  • Cons
    • Users may not know they are being monitored (red flag! red flag!)

Beta users

Distribute extension with beta builds of software.

  • Pros
    • Distributed directly to people who care about the product
    • Automatic wide user base
  • Cons
    • If done incorrectly, could hurt our reputation in the testers community

Other Considerations

  • mozmetrics guys have until December to make something happen
  • Songbird does metric collection in the client, is looking to do collection using AUS2 -- this is very complimentary
  • Browser as a butler -- expose helpful metrics to the user... in the same fashion that a mail client will often save contacts as you email them.
    • How many people are browsing the same page
    • shared tabs
  • Flock wants in