<<Back to Features

Timezone Backend Design (DRAFT) v3

Discussion in mozilla.dev.apps.calendar

"To understand timezones, it really, really helps to be drunk." -- Stephen Mook

So, here we are for round three. We are thinking to use an extension to manage the timezone database since the general database server (as discussed below) is not looking to happen any time soon. So, in order to do this in an extension, we need to stub in an interface that the extension can then implement. This interface will be the calITimezoneResolver. Providers that use server stored timezones which are different from the Olsen timezone database (I'm looking at you, Outlook), can implement this interface also and provide their own timezone definitions.

It's a rather simple interface:

calITimezoneResolver:

AddTimezone(in string Tzid, 
            in calIIcalComponent timezone data to add,
            in string latitude,
            in string longitude,
            in string stdOffset,
            out TimezoneEntry of timezone added/found)

• Attempts to find this Tzid in the timezone database and if it is found, returns successfully and returns the timezone found
• If the Tzid is not found in the database, then this adds the timezone to the database, and returns the timezone added

GetTimezone(in string Tzid,
            out TimezoneEntry of timezone found/null if not found)

• Finds the timezone with the matching tzid in the database

GetTimezoneList(out An nsIUTF8StringEnumerator of all Tzid's in the database) 

• Returns a string enumerator of all known tzid's in the database

GetVersion (out string version)

• Returns the version of the underlying timezone datastore

Putting the timezone database into the extension changes the nature of upgrading the timezone information from a code update into more of a build/release update. As long as the underlying database structure does not change, there should be no need to update the associated code in the extension. But, I do think it makes sense to also package the database, the TimezoneStorageService (the C++ wrapper for the database) and the calITimezoneResolver implementation together in the extension. So, that amount of code will be updated each time the timezone database is updated by the extension update mechanism.

TODO: What happens if a provider implements a timezone definition for one that already exists in the extension? In general, the extension will probably be more up to date, so it might make sense to have an "isBuiltIn" method that will return true ONLY if the calITimezoneResolver is the extension. This way calICSService can give precedence to the Olson definitions from the extension over those of the providers.

NOTE: The version of the timezone information coming from the extension will be formatted as follows: <DATESTAMP_OF_OLSON_DATA>.<DB_SCHEMA_VERSION>.<EXTENSION_VERSION>. Of course, providers that implement the calITimezoneResolver interface are free to use whatever versioning system they want. This is not meant to be consumed by software, but by a person when diagnosing support issues. Usually the auto-upgrade offered by AMO should prevent someone from ever needing to parse this string

Timezone Backend Design (DRAFT) v2

Discussion in mozilla.dev.apps.calendar

In general, when we read in calendars, we'll get VTIMEZONEs that fall into two categories: ones that are builtin to our implementation, and foreign timezones. Authoritatively, we'll store these in single files, each containing a single VTIMEZONE, but they'll be cached in and used from SQLite databases (we may choose not to ship the directory tree version of the builtin timezones, since that just adds unnecessary download size).

When we read in calendars (or are given new VCOMPONENTS like VEVENTS or VTODOS),there will be a number of different types of TZIDs that we'll find. Below, I will use "Foreign" timezone to refer to a zone that was not created by the current or previous version of the currently running Sunbird/Lightning. For example, if a user on 0.3.1 receives an event from 0.5, that would be handled as a "Foreign" timezone. However, if that user receives an event with a 0.3 timezone definition, that would NOT be a "Foreign timezone". Each of these timezone types will be dealt with according to the list below:

Current Mozilla Timezones

• Starts with /mozilla.org/current_dtstamp/
  • The current internal database version
  • Nothing to do here
  • Raw path: installdir/timezones/current_dtstamp/Continent/City.ics
  • DB path: installdir/timezones/current_dtstamp/builtin-zones.sqlite3

Older Mozilla Timezones

• Starts with /mozilla.org/older_dtstamp/
  • An older internal version
  • Change reference to current version
  • Raw path, DB path: (none, see current version)

Newer Mozilla Timezones

• Starts with /mozilla.org/newer_dtstamp/
  • treat as any other globally unique zone (see below)

Foreign Zones -- Timezones from other applications (including newer Mozilla zones)

• Other
  • DB path: profiledir/timezones/imported-zones.sqlite3
  • Assign a UUID for tracking
    • Claims to be globally unique
    • Not in "foreign timezone database"? Add it.
    • Already in database? Compare...
      • ==, nothing to do
      •  !=, reject (in the future, we could rename)
    • Raw path: profiledir/timezones/uuid/

Other Issues

Foreign Zone Lifetime

Until there is a reliable "standard" copy of timezone definitions available from a server side timezone database, we will need to maintain non-Sunbird/Lightning timezones. Once there is a standard copy of timezone definitions that we can depend on to be current, we can consider merging "foreign" timezones into the standard copy, and we may not need to keep copies of those zones. In the meantime, I propose that we use a refcounting scheme in order to manage the lifetime of the foreign timezone definitions.

• Use refcounts to manage the lifetime of a all zones (including the built-in ones, this is useful for updating our definitions).
  • Once the number of events referencing that zone goes to zero, we remove that zone from the database
  • Should these refcounts be maintained in the flat file copy of the timezone? I'm not sure, it depends on the function of the flat files (see TODO's below)
  • Each event that references this foreign zone will add one to the zone's refcount. When the event is deleted (an entire event -- not an instance of an appointment is deleted) then the refcount will be decremented by 1.

Displaying Foreign Timezones

• Users will NOT be able to create events using foreign zones.
• Users will be able to edit a VEVENT or VTODO that contains a Foreign timezone reference without disturbing the timezone reference (unless the user edits the timezone specification, of course).
• Users will be able to view a foreign timezone in our timezone picker.
  • We will do this by displaying the TZID in text
  • If we have a graphical picker, we will highlight the proper zone offset, so that the user can see geographically where the zone lies.
  • (Enhancement) If needed, we may be able to map the zone using the TZID to our set of known zones so that a proper city location can be displayed to the user. In most cases we can do this because by convention TZID's will usually contain enough descriptive text for us to establish a linkage with our own city list from our internal zones. Of course, should this not be possible for the given TZID, then we will have to forgo this level of mapping and will have to rely instead on showing the user the TZID and the UTC offset.

Updating timezone Definitions

• Updating Built-in Timezones
• All timezones in database will have a new TZID field. This will be blank until an update occurs. When the update occurs, the field will be populated with a key referencing the new TZID, and the subsystem will use this new key to map the old timezone definition to the updated timezone defnition.
• When updating, foreign timezones will be left alone. They will not be updated or destroyed. (When there is a standard timezone database service, we will revisit this.)
• When a VEVENT or VTODO performs a lookup on an old timezone definition, the newer definition will be returned, and the refcounts will be handled as follows:
  • • Old timezone defintion refcount-- (if this is 0 we will remove the old definition)
    • New timezone definition refcount++

Timezone Mappings between Builtin zones and Foreign Zones

• While there is no standard for timezone mappings, there are conventions
• Other implementations that use the Olson database should be fairly trivial to map, we simply compare the offsets then the continent, then the city.
• The other primary convention we will find will be the Windows timezone naming system, which can be mapped to Olson.
• In order to map between the flat file store and the database store for the foreign timezones, it would be fairly simple to have a path descriptor and/or filename (since we know it will live in the Profile Dir) in the database. Then the timezone service can simply query the database to find the file information and use that. TODO: Should the database be able to operate using only a flat-file system?

TODO

• What problems does the flat file system solve? What does it gain over the database, or is it simply a handy backup?
• Do we need a user accessible backup utility, perhaps something that will dump all the timezone definitions (builtin and foreign) to a set of flat files?
• How much user control do users exercise over the timezone database? Is there/should there be an "Update Timezone Definitions" button in prefs somewhere?
• Is there a cost to using a different database for foreign zones vs. builtin zones? Would it make comparisons simpler/more performant to use the same sqlite database for both sets of timezones? I tend to think we should use the same database for both, is there some valid reason to not do this?

References:

• bug 314339 – need to handle non-native timezone bits
• bug 363191 - Handle Timezones more efficiently (Timezone Database)