Firefox/Channels/Postmortem/70

Notes for Firefox 70 postmortem

The meeting notes were lost in a disappearing Etherpad but here's some notes from the release owner's draft and follow up on action items.

• Attendees: Liz, others from relman

Stats

• Bugs fixed in 70: 5406
• uplifts during beta: 345
• skyline related uplifts: 64 https://mzl.la/2ptFZZi
• uplifts during RC week: https://mzl.la/2VMrNGz

What went well

• Skyline launched, and was fairly smooth.
  • Tags, burndown lists worked well
  • EPMs and planning were super helpful
• We (Dan) did sec review on more features in more depth

What didn't go so well

• Brief outage when we launched protection report

• Minor problems with build/infra, ship-it, in beta 1 -4

• Process for the WNP specs and testing still not clear

• last minute/RC week stuff
  • Oct 7 beta 13 build. Many last minute requests.
  • Branding left off Fennec and ESR. even tho we signed off that it was there and it was in the release notes up until Thursday RC week. https://bugzilla.mozilla.org/show_bug.cgi?id=1589422
  • VPN, lockwise last minute changes
  • Catalina macOS broke download addons
  • Pushing new OpenH264 had issues
  •  issue between balrog & product-details (we pushed the RC and signed off on the rules, but are still serving beta 14)
  • DoH study failing QA then launching mid week then failing in multiple ways on Friday
  • The holdback study had a lot of churn/issues

Some action items

• adding some CVE linter checks for the date and version numbers
• Following up to find triage owners or check on triage for components where bugs are going untriaged (Firefox::Untriaged, Firefox::General, Core::General)
• (marcia) After we released there was some confusion about our uptake in regard to the Delivery Dashboard - https://mozilla.github.io/delivery-dashboard/#pollbot/firefox/70.0
• https://metrics.mozilla.com/~sguha/mz/missioncontrol/ex1/mc2/archive/2019-10-16/release.html wasn't up to date, and would have been useful to look at since we are trying to use the new dashboard for our releasesd
• ACTION: Liz suggested a slide or two to knowledge share at the CF what dashboards we use to look at the release uptake.
• ** Protection Report requests that failed: Unique IPs + UAs (best I can do for unique users) requesting /user/breach-stats during that time period that got 500s: 1160 (1715 total).
• Failed logins during that period ( people who weren't able to login with FxA to Monitor)
• Unique: 4232
• Total: 7276
• * ACTION - liz will talk with habib or ops team again