Platform/Features/Camera API - Phase 2 (getUserMedia)

Status

Camera API Phase 2 - based on getUserMedia
Stage	Design
Status	In progress
Release target	Firefox 15 - Mobile only, still image support only
Health	OK
Status note	Planning will need to be somewhat aggressive to make Firefox 15 lift off (June 5), but it's very doable. The piece that is at greatest risk is the UX/UI pieces.

Team

Product manager	Maire Reavy
Directly Responsible Individual	Maire Reavy
Lead engineer	Anant Narayanan
Security lead	Lucas Adamski (currently) Curtis Koenig (soon)
Privacy lead	Lucas Adamski
Localization lead	`
Accessibility lead	TBD
QA lead	Jason Smith
UX lead	TBD
Product marketing lead	`
Operations lead	`
Additional members	`

Open issues/risks

• The getUserMedia spec is still being defined. We will prefix the first release.
• Privacy UI/UX for this feature is a challenge since this involves camera and microphone data. Security will be difficult due to the wish for applications to be able to provide useful interfaces, in conflict with the need to guarantee users are safe with regards to access to their camera and microphones. This design is being discussed with the security team.

Stage 1: Definition

1. Feature overview

Ability to take a picture from a computer's attached camera and use it in a web application. This will eventually (after the first release) include live camera preview and recording/sending a video (with audio). First release will support Mobile (Android and B2G) only.

2. Users & use cases

The first release will include the same use cases as Camera API - Phase 1 (<input type="picture">) Initially, this will provide a programmatic complement to the declarative <input> API (Phase 1) and gives us early developer feedback on the API:

• Take a picture on flickr, or upload from your gallery.
• Take a profile photo on facebook, or upload from your gallery.
• Instagram for the web.

In future releases (schedule is TBD but after first release):

• App allows user to take a picture and record an audio clip
• App allows user to record a video with audio to send to someone else
• App allows user to record an audio clip to send to someone else
• App allows users to record video from multiple webcams
• App allows foreground photo sharing with realtime preview and special effects. Needs live video stream and the ability to manipulate the stream on the fly.
• App allows video monitoring such as a baby monitor or security camera that can run for extended periods of time
• App allows the user to start a podcast, open other tabs/apps while the recording continues (to look up and comment on information, etc) and then comes back to the tab/original app to finish the podcast. Note: the user may continue to record while opening or switching to other tabs/apps

3. Dependencies

Android and Desktop both need:

• UX/UI design work which is being determined now

Desktop support also needs

• part of WebRTC to land

4. Requirements

For first release: The ability to take a still image shot (without preview or video).

Non-goals

The first release does NOT include

• the ability to capture a video and send that video somewhere
• the ability to preview the image in a canvas or modify it before uploading it to a server won't be possible with this implementation.

The first release will be Mobile only (not desktop).

Stage 2: Design

5. Functional specification

Proposal to W3C Media Task Force

6. User experience design

Being designed now.

TBDs as part of UX design:

• Is there a visible indicator that the camera and/or microphone is active (because this is currently mandated by the getUserMedia spec)? Is this indicator visible even if the browser window is partially or completed obscured?
• What gets shown when recording audio only?
• What are appropriate limitations to fingerprinting?

Note: Video stream may need to be accessible while focus is given to another app

Stage 3: Planning

7. Implementation plan

1st release plan for Mobile:

• Finish security discussion for Camera API UI/UX by April 17th (target date).
• Pull together all use cases and have a UX/UI meeting to nail down what the experience for Camera API will be. Target UX/UI meeting for April 23/24.
• Target date to review mock ups: May 1 so we can finalize them by May 4.
• Target date to start implementation of mock ups: May 7.
• Goal: To provide mock ups to developers by May 7 so they have approximately 4 weeks to implement, ask questions and land their stuff. Firefox 15 goes to Aurora on June 5.

Schedule for getUserMedia/Camera API support on desktop:

• The big pieces are
  • landing the parts of WebRTC needed for the backend support (camera drivers, etc)
  • landing Rob O'Callahan's Media Stream patches
  • getting resources for desktop frontend to implement the mock ups (which may prove to be pretty simple if the mock ups are simple).
• Looking like the MediaStream patches will land in Firefox 14.
• Randell, Anant, and Maire are currently looking at how quickly we can land the software pieces of WebRTC needed to support getUserMedia on desktop (intermingled with the WebRTC project work).
• UX/UI desktop mock ups should be completed when mobile mock ups are done. Then we just need developers to implement them on desktop.

8. Reviews

Security review

`

Privacy review

`

Localization review

`

Accessibility

`

Quality Assurance review

`

Operations review

`

Stage 4: Development

9. Implementation

Bug 738528 shows how the getUserMedia spec was implemented.

Stage 5: Release

10. Landing criteria

`

Feature details

Priority	P1
Rank	999
Theme / Goal	`
Roadmap	Firefox Mobile
Secondary roadmap	`
Feature list	Mobile
Project	Apps
Engineering team	Mobile platform

Team status notes

	status	notes
Products	`	`
Engineering	`	`
Security	sec-review-needed	bug 749221
Privacy	`	`
Localization	`	`
Accessibility	`	`
Quality assurance	`	`
User experience	`	`
Product marketing	`	`
Operations	`	`