Status

Content Preferences, a prototype extension implementing some of the proposed functionality, has been developed and is being distributed on AMO. The prototype implements the persistent datastore, service, controller, sidebar, and handlers plus sidebar widgets for the text zoom, page style, and character encoding settings.

Two functional requirements have been added to the Firefox 3 product requirements document: a framework for site-specific preferences comprising the datastore, service, and controller (bug 378547) and persistent text zoom (bug 378549).

A meeting to review the API, extensibility, and security of the feature is scheduled for Thursday, May 3 at 11:00AM PDT (6:00PM UTC) at 1-800-707-2533, pin 369, conf 224 (intl: 1-650-215-1282, ext 91, conf 224).

Overview

Motivation

The purpose of this feature is to enable users to set preferences for browser and content settings (f.e. text zoom, page style, and character encoding) on a site-specific basis instead of just a tab- or page-specific basis, and to persist those preferences across page visits and browsing sessions, in order to improve the usefulness of those settings.

For this feature, the words setting and preference are used as follows:

setting 

a runtime configuration option that affects the browser's or a page's appearance or behavior

preference

a user-defined value for a setting

Use Cases

Text Zoom

Every day, Joe visits a web site whose text size is too small for him to read it easily. After going to the site, Joe uses the View > Text Size > Increase command to increase the browser's text zoom value until the text is a reasonable size.

Joe has to do this each time he goes to the site, and when he leaves the site, he has to use the View > Text Size > Decrease command to return the text zoom to its normal value for the other sites he browses.

If the text zoom setting was site-specific, Joe would only have to change it once for the site whose size is too small, and he would never have to reset the setting to its normal value after leaving the site.

Note: John Lilly and Dan Veditz have described personal use cases which aren't satisfied by a site-specific text zoom.

John sometimes increases the text zoom for the browser in order to browse the web (including potentially multiple sites) while sitting farther away from his monitor. Afterwards, he decreases the text zoom back to its normal value.

Dan changes the text zoom each time he switches between two displays (I think his laptop's built-in display and an external monitor).

Although site-specific text zoom doesn't work well in these cases, tab-specific text zoom, which is what we currently have, probably doesn't work well either, because it makes them set (and later reset) the text zoom setting for every tab.

Character Encoding

-> Verify that the character encoding functionality currently in Firefox is accurately described in this section.

Jane lives in a country whose primary language uses a non-Western encoding. She frequently visits web sites whose character encoding is specified incorrectly (or not at all) by the server or page and is not correctly auto-detected by Firefox.

Every time she visits such a site, she has to set the correct encoding for it using the View > Character Encoding menu, but because Firefox only stores the character encoding for pages that she bookmarks, and then only for the specific bookmarked page (not for all pages on that site), Jane finds that she repeatedly has to set the encoding.

If the character encoding setting was site-specific (instead of or in addition to the bookmark-based page-specific mechanism), then Jane would no longer have to set the character encoding each time she visited a site for which it was set incorrectly. Instead, she could set it once and have Firefox reapply that encoding each subsequent time she visited the site.

Requirements

Note: these are only suggested requirements for Firefox 3. Actual requirements for the release are in the Firefox3/Product Requirements Document.

Service

• P1 FR a service that stores and permits chrome code to get, set, remove, and observe changes to preferences for arbitrary settings on a site-specific basis
• P1 FR support for defining sites by host name
• P1 FR support for defining sites by ETLD + 1
• P3 FR a hidden preference that allows users to switch between definitions of a site
• P3 FR ability for extensions to define and register additional site definitions

Controller

• P2 FR a controller for each browser window that simplifies the process of listening for location changes and retrieving preferences for specific sites by defining an API that pref handlers can use for the purpose

Sidebar

• P? FR a sidebar or other UI area that displays and allows users to set and unset the consolidated group of site-specific preferences
• P? FR support for setting the default value for each site-specific setting

Text Zoom Setting

• P1 FR the value of the text zoom setting, which is set by commands in the View > Text Zoom menu or via the scrollwheel, is automatically recorded and restored on a site-specific basis
• P1 FR a mechanism for changing the default value of the setting

Page Style Setting

• P3 FR the value of the page style setting, which is set by commands in the View > Page Style menu, is automatically recorded and restored on a site-specific basis
• P3 FR a mechanism for setting the default value of the setting to either "page default" or "no style"

Character Encoding Setting

• P2 FR the value of the character encoding setting, which is set by commands in the View > Character Encoding menu, is automatically recorded and restored on a site-specific basis

Other Settings?

(are there other settings important and common enough to add to the core?)

• Language preferences, as implemented in Dictionary Switcher extension, for example.

Schedule

• 2006 December 12: Content Preferences (cpref) extension, version 0.1 (announcement) - first prototype implementation, with support for site-specific text zoom
• December 13: cpref, version 0.1.1 - minor bug fix release
• 2007 January 26: cpref, version 0.1.2 - minor bug fix release
• February 13: cpref, version 0.2 (announcement) - second prototype implementation, with API updates and support for scrollwheel-based text zoom changes
• February 14: API/architecture discussion started - blog post mda.firefox article
• March XX: API/security review
• March XX April 24: cpref version 0.3 - third prototype implementation, with page style and character encoding handlers, undo buttons, and datastore/API updates
• May 1: cpref version 0.3.1 - minor bug fix release
• May 3: API, extensibility, security review
• March/April May: land service on trunk
• April May: cpref version 0.4 - fourth prototype implementation?
• April/May: land additional elements of feature (controller, handlers for core prefs, sidebar or other UI) on trunk
• May: release stable implementation of elements not destined for the core as extensions

Also see history of extension on AMO.

Repository

The code for the feature currently lives in the extension. The service, controller, and text zoom handler should land in the Mozilla CVS repository. The sidebar and other handlers should either land in the Mozilla repository or a Mozdev project repository depending on whether it ends up as a core component or remains as an extension.

Design & Implementation

Site-specific preferences in Firefox 3 comprises:

1. a datastore that persists prefs across sessions;
2. an application-wide service with an API for accessing prefs;
3. a grouper that categorizes URIs by site;
4. a controller for each browser window with an API for observing pref-related events;
5. a handler for the text zoom setting.

To this core functionality, the Content Preferences extension adds:

1. handlers for the page style and character encoding settings;
2. a sidebar for viewing and editing site-specific prefs.

Datastore

The datastore is a SQLite database named content-prefs.sqlite that lives in the profile directory. It gets created automatically if it doesn't already exist when the service gets instantiated. Like Places, we record the database schema version in the user_version pragma so existing databases can be updated if the schema is modified in future versions of Firefox.

The schema, as specified by a JavaScript data structure in the service's code, is as follows:

 _dbSchema: {
   groupers:   "id           INTEGER PRIMARY KEY, \
                name         TEXT NOT NULL",

   groups:     "id           INTEGER PRIMARY KEY, \
                name         TEXT NOT NULL, \
                grouperID    INTEGER NOT NULL REFERENCES groupers(id)",

   settings:   "id           INTEGER PRIMARY KEY, \
                name         TEXT NOT NULL",

   prefs:      "id           INTEGER PRIMARY KEY, \
                groupID      INTEGER REFERENCES groups(id), \
                settingID    INTEGER NOT NULL REFERENCES settings(id), \
                value        BLOB"
 },

Some notes:

1. The schema uses arbitrary integers to identify all records, even though all tables have another unique column (or, for the prefs table, a unique combination of columns), primarily because in my experience this approach makes future schema changes easier.

   Integer IDs might also provide minor space savings and general performance improvements while increasing the complexity of client code and hurting performance for some transactions, but I expect these positive and negative effects to be minor.

2. The schema tracks groupers (components that determine the sites to which URIs belong) as a way of namespacing sites created by different groupers.

   An earlier version of the schema didn't track groupers, and I think we should revert to that version, making groupers responsible for namespacing sites in cases where collisions are possible, because I haven't found a good use case for this functionality, while it adds significant additional complexity.

3. Pref values are stored in a column with a BLOB declared type because that type has a NONE type affinity, which means SQLite will do no type conversion on values inserted into the column, maximizing round-trip integrity of data passed from consumers through the service to the datastore and back.

   The actual extent of this integrity across the types supported by nsIVariant is yet to be determined and should be investigated via unit tests.

4. The prefs.groupID column is nullable to enable the service to store global preferences (i.e. prefs that apply to all sites/the entire web). We could make consumers store such prefs using the existing nsIPrefService, but that would add significant complexity to consumers, which would have to support two distinct and incompatible interfaces (since nsIPrefService doesn't use nsIVariant).

   Supporting global prefs in the site-specific pref service, on the other hand, is straightforward and adds relatively little complexity to its code.

Service

The service is implemented as a JavaScript XPCOM component with the contract ID @mozilla.org/content-pref/service;1 that implements the nsIContentPrefService interface.

Should that be nsISitePrefService?

Core Methods

The service's core API comprises the following four methods for accessing prefs:

 nsIVariant    getPref(in nsIURI uri, in AString name);
 void          setPref(in nsIURI uri, in AString name, in nsIVariant value);
 boolean       hasPref(in nsIURI uri, in AString name);
 void       removePref(in nsIURI uri, in AString name);

All four methods take as input a URI belonging to the site in question and a name for the preference. The service is responsible for extracting the site from the URI. It does so via the effective top-level domain (ETLD) plus one grouper, which defines a site as the ETLD plus one hostname segment in a URI (f.e. mozilla.org, bbc.co.uk).

Should the service use the entire host grouper, which defines a site as the entire hostname in a URI (f.e. bugzilla.mozilla.org, www.bbc.co.uk)?

Should consumers be responsible for extracting sites from URIs so they can set preferences using custom groupers? Or should the service expose an API for registering custom groupers for specific settings/URIs?

The name of the preference is an arbitrary string. Preference namespacing is the responsibility of the consumer, as it is with application-wide preferences accessed via nsIPrefBranch.

The setPref method also takes as input an nsIVariant representing the value of the preference, while the getPref method returns an nsIVariant. Consumers can set the value of a preference to null (nsIVariant::VTYPE_EMPTY), and the service distinguishes preferences set to null from those that have not been set at all by representing the latter with an undefined value (nsIVariant::VTYPE_VOID).

Consumers can pass a null URI to access a global preference.

Getting All Prefs

The getPrefs method allows consumers to retrieve all preferences for a given site:

 nsIPropertyBag getPrefs(in nsIURI uri);

The method returns an instance of @mozilla.org/hash-property-bag;1, which implements both the nsIPropertyBag and the nsIPropertyBag2 interfaces. Bag keys are preference names; bag values are their values.

Observers

The service also implements methods for adding and removing observers:

 void    addObserver(in AString name, in nsIObserver observer, in boolean holdWeak);
 void removeObserver(in AString name, in nsIObserver observer);

Unlike nsIPrefBranch2, which permits a consumer to watch an entire preference branch, nsIContentPrefService takes only an exact preference name.

Should it support preference branches?

When a pref changes, the service notifies registered observers via the content-pref-changed topic.

Is that the right name for this topic?

The service passes the new value of the pref in the notification's aSubject parameter, which is an nsIPropertyBag containing the following properties:

• group: the site to which the preference belongs;
• name: the name of the preference;
• oldValue: the old value of the preference (can be undefined);
• newValue: the new value of the preference (can be undefined).

Should the service use a custom interface (like nsILiveTitleNotificationSubject) instead of a property bag?

Should the service use the notification's aData parameter for anything?

Should we have separate content-pref-set and content-pref-removed notifications?

Groupers

Groupers are components that categorize URIs into sites. They implement a simple interface, nsIContentURIGrouper:

   readonly attribute AString name;
   AString group(in nsIURI uri);

The name attribute is an arbitrary identifier that the database uses to associate a site with the grouper responsible for extracting it from a URI. It is unnecessary if we decide not to store groupers in the database.

Should we use the contract ID as the arbitrary identifier for the grouper, and how do we retrieve that from an instance of the component?

The group method extracts and returns the site to which the given site belongs.

Groupers are independent of the service to permit extensibility (overriding of the default grouper, registration of alternative groupers for specific URIs/prefs, etc.), although there is little extensibility in the current architecture, and it's not entirely clear what kinds of extensibility will prove useful.

Should the default grouper categorize URIs by entire hostname?

The service implements an attribute representing the grouper it uses to extract a site from a URI:

 attribute nsIContentURIGrouper grouper;

The attribute is read/write so that extensions can change the way URIs get grouped by setting this attribute.

Wouldn't it make more sense for this to be read-only and for the service to use the grouper specified by an application preference?

Controller

The controller is a JavaScript object assigned to the ContentPrefController top-level variable in the chrome window context of browser windows.

Shouldn't this be called something like a "helper" and the "handler" be called a "controller" given the handler's controllerish role as mediator between the datastore "model" and the setting "view"?

It implements the following API for pref handlers to register themselves for notification of pref-related events:

 void addObserver(in AString name, in nsIObserver observer);
 void removeObserver(in AString name, in nsIObserver observer);

Should the controller be able to hold weak references to observers?

The controller registers itself as a web progress listener and DOMContentLoaded event listener for the tab browser widget and then notifies its observers when the tabbrowser's location changes (nsIWebProgressListener::onLocationChange) or a DOMContentLoaded event fires for a browser in the widget.

For location changes, the controller calls nsIObserver::observe with:

• aSubject: the URI of the new location;
• aTopic: "content-pref-location-changed";
• aData: the value of the site-specific preference for the new location.

For DOMContentLoaded events, the controller calls nsIObserver::observe with:

• aSubject: the document object whose content was loaded;
• aTopic: "content-pref-dom-content-loaded";
• aData: the value of the site-specific preference for the document.

Note: this doesn't actually conform to the nsIObserver interface, in which aData is a wstring, so it will only work with pref handlers implemented in JavaScript and registered without going through XPCOM.

Should we define an nsIObserver2 interface in which aData is an nsIVariant?

Are these the right topic names for these notifications?

Should aSubject be the event object rather than the document object for DOMContentLoaded notifications?

Will any pref handlers want to know about pageshow/hide?

Text Zoom Handler

The text zoom handler is a JavaScript object assigned to the TextZoomHandler top-level variable in the chrome window context of browser windows. It registers itself with the controller as a "text.zoom" preference handler and takes over responsibility for handling the View > Text Size commands (both the menu items and their keyboard shortcuts) as well as zoom-triggering scrollwheel events.

Should the name of this setting be text.zoom, browser.text.zoom, or something else?

When the user changes the text zoom for the current page, the handler applies the change to the text zoom setting and then sets the preference for the site to the new value of the setting.

Should we provide an affordance for setting a global text zoom preference?

If we provide an affordance for setting a global text zoom preference, and a site has a site-specific preference, should we apply that preference in addition to the global preference or instead of it? For example, if the global preference is 120% and the site-specific preference for a given site is 130%, should we set the text zoom for the site to 130% or 156% (100 * 1.2 * 1.3)?

When a user browses to a different page (i.e. the location of the tab browser widget changes) the handler applies the site-specific preference, if any, provided to it by the controller. Otherwise, it applies the global preference, if any. Otherwise it applies the default value (100%).

Cross-Cutting Concerns

API Changes

This feature changes no existing APIs, but it does add several APIs. First, it adds a application-wide "service" API for getting and setting site-specific preferences. Second, it adds a window-specific "controller" API for registering as a pref handler and observing pref-related events. Third, it adds a "sidebar" API for adding controls to a sidebar presenting a consolidated view of preferences for the current site.

Extensibility

Extensibility of this feature is a top priority, since numerous and popular extensions provide users with ways to alter browser settings affecting site appearance and behavior, and the ability to set preferences for those settings on a site-specific basis is a common function of (or feature request for) those extensions.

Examples of extensions that permit site-specific control over browser settings include Flashblock and NoSquint. An experimental version of the HashColouredTabs+ extension supports site-specific functionality via the version 0.2 of the Content Preferences prototype extension.

This feature should be extensible in the following ways:

1. it should be possible for extensions to get and set site-specific preferences via the same APIs that core code uses;
2. it should be possible for extensions to overlay themselves onto the site preferences sidebar so that their controls for setting preferences are visible alongside (or perhaps below) core controls and are as usable as the core controls;
3. extensions should be able to register themselves as pref handlers in the same way that core code registers themselves as pref handlers and receive the same notifications that core code receives.

Should it be possible for extensions to override core pref handlers so that they can implement enhanced/modified functionality to that provided by the core handlers?

Should it be possible for extensions to provide additional ways of defining sites so that users who want a different way of looking at sites can get it and to promote innovation in site definition? F.e. one could say that all Mozilla domains (mozilla.org, mozilla.com) are a single site because they all relate to the Mozilla project, or one could define a site as every domain that uses an SSL certificate issued to the same organization, or every site that uses a certain stylesheet (for browser settings like text zoom and page style that affect style), or perhaps finer-grained controls that define different paths on a certain host as separate sites.

Customization

This feature adds customization controls to a sidebar, which has potential impact on many preferences and customization controls, particularly the Preferences > Content pane and items in the View menu like Text Zoom, Page Style, and Character Encoding.

The potential impact includes:

• items in the View menu might start to affect browser settings on a site-specific basis rather than a tab- or page-specific basis;
• the Content pane might provide an option for viewing and editing preferences for sites;
• other site-specific lists in the Preferences dialog like the list of cookies and extension whitelists might be combined with UI for viewing preferences for sites generally;
• items in the View menu and Content prefpane might be removed or modified in favor of UI in the consolidated preference view.

Performance

There are two notable potential consequences for a performance regression from this functionality.

First, as the user goes from site to site, the controller will get preferences for each site from the service, which queries the datastore for them. There is some cost to doing this, but it's not yet clear what that is. A test for this would be to populate a database with some site-specific preferences and then measure the amount of time it takes for the controller to retrieve a set of preferences for a site.

Second, because settings may be changed on a site-specific basis, switching from one site to another may now prompt costly changes to browser settings.

The effect of this is mitigated by the fact that such changes will only take place when the user has explicitly set preferences for a site, so users who don't use the feature won't be affected, and users who do use the feature probably are willing to experience the regression in exchange for the functionality provided.

But we should keep an eye on that latter assertion, since users may assume that site-specific preferences (at least those built into the core product) do not cause performance regressions and thus not understand why certain sites load more slowly than others. At a minimum, we must make it obvious and easy to undo a site-specific preference. And if settings built into the core product turn out to be costly, we should reconsider making them site-specific or provide additional feedback to the user about the cost (f.e. by providing progress or some other indication that the preference was applied) to inform the user about the cost so she can make an informed decision whether or not to accept it.

Reliability/Stability

In general, this feature should not impact reliability or stability, although it could expose or amplify the effect of bugs in core browser functionality (f.e. the code that adjusts text zoom) to the extent that it increases the frequency by which that functionality is invoked.

Security

The service and the controller are only available to chrome code, so they should have no security impact.

Site-specific preferences can be loosely grouped into "browser" and content categories. Browser prefs are those that affect the browser object (browser, docShell, presShell, etc.) embedded into each tab in Firefox, f.e. the text zoom preference. Content prefs are those that affect the document object for a given web page which is loaded into a browser object, f.e. the page style preference.

Existing prefs of either type should have no additional security impact, although they could expose security holes in core browser functionality to the extent that they increase the frequency by which that functionality is invoked.

New content prefs could have additional security impact, although no such prefs are being proposed yet.

To the extent that this functionality makes it easier to write extensions that add content prefs, however, it may cause the number of such extensions, and thus the danger of security holes in them, to increase.

Privacy

In general, site-specific preferences are not expected to be particularly high value or personal, but they can provide some information about users should be considered private. Preferences are stored persistently in a file in the user's profile, so they are subject to the same security measures we use to protect other personal data in the profile directory. Preference data is transiently stored in the memory of chrome objects to which content does not have access and in chrome controls to which content does not have access.

Global Audience

Since this feature introduces new UI, care should be taken to ensure that it remains accessible in the same way that the current non-site-specific UI is accessible.

L10n requirements apply as they do to other chrome.

Web Compatibility

Users might find that some of their preferences are incompatible with web sites, so it should be easy to remove a preference that results in an incompatibility.

To the extent this feature and its popularity causes web sites to become less insistent on appearing and behaving in a particular way, this feature should have a positive impact on web compatibility.

Other

Discussion & Implications

Caveats / What We've Tried Before

References