User:P.A./Download architecture improvements

Overview

This page describes a proposed incremental redesign of the internal architecture that handles downloads in Firefox and other Mozilla products.

The current architecture is the result of the combination of different networking and front-end components, integrated at different times, and unfortunatley suffers from the lack of an encompassing structure that allows downloads to be handled consistently and more intuitively from calling code. There is also a lot of code duplication.

The new design will aggregate all the download operations under the same umbrella, making the execution flow straightforward, and delegating work to existing components where appropriate. The covered aspects include:

• Choose the target file based on appropriate defaults and limits
• Handle authentication prompts
• Execute the download by copying data
• Choose the action to execute based on file type
• Execute actions on the downloaded file
• Add the download to browsing history
• Control the download and view progress

This will make it possible to:

• Perform asynchronous and off-main-thread file handling in all code paths
• Solve some long-standing issues, like distinguishing failing downloads from canceled ones in all cases, or avoiding that retrying a complete web page download saves the HTML only.
• Enable more of the proposed download user experience improvements.

Initially, many aspects will be covered by the existing components, plugged into the new infrastructure. Incrementally, more aspects can be rewritten natively in the new infrastructure.

Existing components and modules

This is a summary of the current interfaces, folders, or modules implementing some of the logic related to downloads:

• Choose target: contentAreaUtils.js, DownloadLastDir.jsm
• Prompts: nsIAuthPrompt, contentAreaUtils.js
• Execute download: uriloader/exthandler, nsIWebBrowserPersist, nsDownloadManager.cpp, any nsICancelable
• Choose action: uriloader/exthandler
• Execute action: uriloader/exthandler, browser/components/downloads, toolkit/mozapps/downloads
• Add to history: nsIDownloadHistory
• Control and progress: nsITransfer, nsIDownloadManager, nsIDownload

Initial implementation

The goal for the first implementation is to remove downloads.sqlite in Firefox for Desktop.

1. Get JSON serialization in place, working as a Downloads Panel back-end
2. Implement nsIDownloadManager / nsIDownload compatibility (conversion of download states, synchronous control functions, ...)
3. Implement an import path for downloads.sqlite (no need to export, since downloads are per session, and on downgrade the download history will not be lost (it's stored in the Places database), only paused downloads will be lost)

Complete design

Data layer

Main module

• One JavaScript module to import: Downloads.jsm
• No localizable strings
  • Make sure that specific error codes are available when needed, like disk full or source network error, but don't propagate arbitrary messages from the download implementation.
    • This helps in removing the current modal message boxes.
• One singleton named like the module: Downloads
  • Works well with lazy module loading
• Just loading the module will not initialize the downloads system
  • Functions that don't require full initialization can be made available
• All the functions of "Downloads" that give access to a "Download" object or to a list of downloads are asynchronous
  • This allows asynchronous initialization if requested
  • Example:

 // Generic, empty Download object creation
 return Downloads.createNew().then(function (aDownload) {
   aDownload.source = ...;
   aDownload.target = ...;
   aDownload.saver = ...;
   return aDownload.start();
 });

Download object

• Encapsulates all the operations needed for one download
• Can be used in a transient way, for background downloads
• Can be linked to a list of downloads that can be controlled globally
• Doesn't directly implement the download mechanism

• Status properties are read-only and synchronous
• One "view" can subscribe to asynchronous update notifications
• All the action methods are asynchronous
• Has multiple state-related properties, based on caller needs
  • Example: Separate "isBlocked" and "blockReason"

• Has a "source" property, that is an object that:
  • Is self-contained (no reference to the outer download object)
  • Has a source "uri"
  • May have a "document" (for saving complete pages, or reading metadata)
  • Has properties like:
    • canSerialize / canPersistAcrossSessions
      • False for documents, URLs with POST data, ...

• Has a "target" property, that is an object that:
  • Is self-contained (no reference to the outer download object)
  • Has a "file" or "URL" property for the actual target
  • Has properties like:
    • isTemporary (will be deleted after executing the action)
    • canSuspendHalfway / canResumeInSession
      • Potentially false for network targets, safe output streams, ...

• Has a "saver" property, that implements the download mechanism
  • This determines the download "class". Prototype-based inheritance could be used, but is more difficult to implement with lazy creation of the Download object.
  • Can access the "source" and "target" objects
  • Can call back into the download object to report progress and errors
  • Has properties like:
    • canSuspendHalfway / canResumeInSession
    • canSerialize / canPersistAcrossSessions
      • For example, saving a complete web page can't be resumed in another session
  • The saver is not directly linked to a window
    • Figure out how nsIAuthPrompt is currently handled when initiating window is closed

• The download object has properties and methods like:
  • canResume (= target.canResumeInSession && saver.canResumeInSession)
  • open()
  • cancel()
  • showContainingFolder()
  • finish()
    • Example:

 return aDownload.finish().then(function() aDownload.open());

• The download object handles adding to history at the right time
• Can be serialized (source, target, and saver-specific properties)
  • NOTE: Deserialization should know about the available saver types before it starts

DownloadList object

• Private and public downloads can be in different lists
• Methods to get enumerable download lists are asynchronous
  • Full asynchronous enumeration like OS.File may not be needed here
• Handles operations like "clean up"
• Can serialize/deserialize children

DownloadStore object

• Handles serialization with off-main-thread file access
• Writes the list of public downloads periodically to disk
• Initializes the list of public downloads by reading from disk

User interface helper layer

• Does not implement the front-end interface
• May contain standard user interface strings or error messages
• Provides a module with functions like:
  • chooseTarget()
  • saveWebPage()
    • Asks for the desired save type if needed (determines which saver to use)