Calendar:Cache

Calendar Caching

The main goal of the cache is to speed up queries. All the roundtrips needed for caldav or groupdav can make the usage of calendar be very slow.

Additionally, it should be possible to access and modify calendar data while offline. If the user can modify data while offline, there should be a log of those changes that can be applied to the remote (authorative) store when coming back online.

Different, but related, problem is the problem of using one single ics file on a (webdav) server simultaniously from multiple clients. When trying to upload the ics file it might turn out that the external file has been modified. In that case, the changes made after the last succesful upload need to be re-applied to the new file, and uploaded again. Again, a changelog. If the cache can take care of this problem too, that would be a nice way to share code.

Filling the cache will depend on the backend-storage. For caldav, the initial filling is problematic (as in: I don't know how to do that in a smart way yet). For ics, it's simply done by downloading the whole file and parsing it.

Updating the cache can be done by asking the caldav server for changed items. At the same time, the changes made by the user that are not saved yet, need to be send to the server. Conflicts between those two changesets need to be resolved.

Shared ics uses almost the same scheme. The provider needs to figure out the difference between the previous file downloaded and the file as it sits on the server. Those differences form a changeset. This changeset is used to update the cache and the display. The changes made to the cache ar applied to the remote file, and uploaded. If the upload fails (maybe because the user is offline, or the remote file changed in the meantime) the changelogs are not deleted, but stored to be used in the next try.

This changelog thing sounds a lot like syncing. That's what is really happening.

In short, the backend needs a way to get the changed items since last upload. The cache alse needs to keep a changelog. Then those are used to sync the two calendars.

Interface Changes

The calendar needs to expose a few flags indicating what it can do:

• get a changeset
• do 'complex' queries

It needs some methods:

• Get changelist
• Update store and return if it succeeded. (No external changes since last time chagelist was retrieved)

Getting the Changelist

easy for caldav, hard for ics.

Ics can parse the remote file, and while doing that check the last-changed time of the events with a stored database of (id, lastchange). Using this change, it can build a changelist and immidiatly remove the non-changed events from memory.

Implementation

The cached data is stored in a storage calendar. The cache engine is a wrapper around the storage.

Two models are possible:

• The storage represents the last known state of the autorative calendar, and changes made by the user are queued
• The storage represents what the user sees, and changes are directly propagated.

The second model makes querying much easier, so that is choosen to be really used. In this model the cache needs to keep a list of items as they weere before the user changed them, so as they are in the autorative store. This is needed for conflict detection and resolution.

More technical details

The cache is stored as a storage calendar. storage calendar needs an option to return a different type and uri. (the uri should not point to the cache file, but to the real file)

The auth calendar (ics) can return a list of changes since the last flush. The calendar can flush the current data while checking if the file didn't change since last getUpdates.

And observer watches for changes to the cache calendar, and builds a changelog. On a timer, this changelog is checked with the changelog from the auth calendar for conflicts. Then the two changelogs are applied to the 'other' calendar.

This observer also fills the cache calendar on initial setup. In the future a better way for caldav can be implemented, where the storage calendar has added knowledge about the auth calendar, and can query it.

Notes from Toronto meeting discussion

caching calendar/offline calendar/device sync

history: mvl/dmose came up with basic design for using storage calendar to hold offline calendars

already implemented by wcap. could be genericized so it could be used for all calendars

change to ui -> ui writes to local storage calendar and writes changelog -> changelog used to push changes to non-local providers (ics, wcap, etc.)

in the case of connectivity loss, changelog can be played back when connectivity returns. (implies sync mechanism) who wins in the case of conflicts?

could create an ics from the cache so we can use 3rd party sync stuff

- scaling issues? - caldir helps for fs sharing

1. calICaching 2. syncing from offline changes 3. ics backend? and device sync storing 4. file locking

How SimDesk dealt with conflicts Type Server Local Action

1     Update  Update   merge dialog
2     Update  Delete   server wins (cause it got updated)
3     Delete  Update   dialog
4     Delete  Delete
5     Insert  Insert