User:NThomas:Mar Generation

From MozillaWiki
Jump to: navigation, search

The aim of this page is to document changes in format of the mar container, and the files within it.

Big picture

A 'mar file' is an update to a Mozilla application, an archive of files to add/patch/replace as well as a 'manifest' of instructions to be performed. It is delivered by a query to an update server, which is out of scope for this page, we only want to talk about how the file is structured and created.

MAR archive

Relevant directories: modules/libmar

Software_Update:MAR is the main documentation for the MAR format. To date the only major revision is ...

Signing

From Version: Firefox/Gecko 12.0; bug 481815.

A maintenance service was added on Windows, which can install updates without user intervention (no password or UAC prompts). It landed in mozilla-central during the Firefox 12.0 cycle (see comment 481), and rode the trains to release. Initially it was for administrator accounts, but bug 711475 extended it to limited users accounts in Firefox 26.0.

To secure the update process a digital signature is included in the mar file, as well as a product information block. The latter labels the mar with a

  • ProductVersion - the application of the bits contained by mar file
  • ChannelID - the combination of the product name and code branch, eg firefox-mozilla-central, firefox-mozilla-beta.

These are set when the mar utility creates a mar file. Default values are set at compile time, but can be overridden by -V and -H arguments when mar is called. A signature can be added to an existing mar using the signmar utility; Release Engineering set up automated signing in bug 509158.

When it's time to apply an update, the updater checks the signature of the mar file using a builtin certificate, and uses the product information block to ensure that the version doesn't decrease, and that ChannelID is allowed in <install_dir>/update-settings.ini. More information about the process flow is available at Windows_Service_Silent_Update.

Older copies of the updater (ie before v12.0) ignore the signature and product information block, so we can serve 'modern' mar files to old applications.

Signing of all platforms started with bug 974570 for Firefox 30.0, in preparation for verifying mar files on all platforms bug 973933.

MAR contents

Relevant directories

The mar contains a set of files belonging to the application being updated, and one (or more) manifests which describe how to apply them. When stored in the mar, the files are all bzip2 compressed.

Application files

There are two types of files - patches created by mbsdiff, and full copies. The patches encode a CRC to ensure the local file is as expected.

Update manifests

The manifest supports various actions, eg ADD (aka overwrite), PATCH, REMOVE. The updater processes the manifest in three passes - PREPARE, EXECUTE, and FINISH.

update.manifest

From Version: Gecko 1.8.0, Firefox/Thunderbird 1.5
Last Version: Gecko/Firefox 29.0, Thunderbird 24.x.y (End of 24 ESR); bug 896224|, bug 896223

This initial format dates back to the introduction of the current update system at Firefox 1.5 (aka Mozilla 1.8.0 branch). It supports the following actions:

ADD <file>                                     # replace <file>
ADD-IF <testfile> <file>                       # replace <file> if <testfile> exists
PATCH <patchfile> <filetopatch>                # apply binary diff <patchfile> to <filetopatch>
PATCH-IF <testfile> <patchfile> <filetopatch>  # apply <patchfile> to <filetopatch> if <testfile> exists
REMOVE <file>                                  # delete <file>

Support for v1 was removed in Gecko/Firefox 30.0 by bug 896224 (updater) and bug 896223 (mar generation).

updatev2.manifest

From Version: Gecko/Firefox/Thunderbird 5.0; bug 386760
Modified Version: Gecko/Firefox/Thunderbird 12.0; bug 660038 Removal of ADD-CC and REMOVE-CC;

A new manifest, updatev2.manifest, was introduced to add support for directory removal and channel changing. The new actions were

RMDIR <deaddir>/                               # remove an empty directory <deaddir>
RMRFDIR <deaddir>/                             # recursively empty and remove <deaddir>
ADD-CC <file>                                  # replace <file> if the channel is changing functionality removed by bug 660038
TYPE <type>                                    # 'partial' or 'complete', must be first line of manifest

The updater looked first for updatev2.manifest, and if not present looked for update.manifest.

In addition, a precomplete file was added, which describes how to uninstall the current application using a subset of instructions:

REMOVE <file>                                  # delete <file>
REMOVE-CC <file>                               # remove <file> if channel is changing functionality removed by bug 660038
RMDIR <deaddir>/                               # remove an empty directory <deaddir>

The precomplete is used when a complete update is to be applied, removing the existing install before laying down the new set of files. In a partial update, deprecated files are deleted by REMOVE isntructions, and precomplete gets a PATCH. This change meant we no longer needed to maintain a list of files to remove on update (ie with browser/installer/removed-files.in), which greatly simplified update testing for rapid releases.

bug 386760 was also where the mar generation changes happened, with channel change removals in {{bug|660038}.

updatev3.manifest

From Version: Gecko/Firefox 29.0, Thunderbird 31.0; bug 759469

This new manifest adds support for a new instruction

ADD-IF-NOT <testfile> <newfile>                # add <newfile> if <testfile> exists

The updater tries to read updatev3.manifest, and falls back to updatev2.manifest if needed. Support for update.manifest was removed at a very similar time (see above).

This change was primarily in support of bug 908134 to offer give release builds to users on the beta channel. We needed to exclude update-settings.ini from mar files, otherwise beta users would be querying on the beta channel but only able to install mar files with the release channel-id. ADD-IF-NOT lets us replace the file if it had gone missing on the user side, although not properly in the beta user case.

bug 900251 handled the mar generation changes.