Firefox/Channels/Postmortem/29

« previous release | index | next release »

Talking Points

• the numbers on the webwewant site — why so high? how can we get a breakdown of what they were based on?
• Web We Want site data:
  • manual downloads (we were counting on restarted downloads? every request on download page?) -- querying zeus logs from d.m.o -- were they all unique? segmenting?
  • auto update pings
  • people who share the site

Timeline

29.0

• Pushed to mirrors on the day before release so we could ship at 6am PST Tuesday April 29th
  • Caused some confusion, didn't get the message out to enough functional areas to prepare for this
  • People happened to manually trigger and see new version
  • There were 404 in product for early updaters
  • Should consider having something other than 404 in the future that is less confusing/scary to users
• Went smoothly, we hit the target ship time within ~10 mins
• Last minute fix for bug 988156 (fullscreen netflix) in 29
• Took an unnecessary uplift for bug 991767 (too late for much testing) and resulted in bug 1003707 (pdf printing blank page)
  • Need to be more averse to last minute stability fixes that aren't verified on a previous beta
  • Look at crash fix bugs closer in the end days for areas that were impacted by the fix
  • In this case, no one used the STR which would have highlighted the issue -- on 30 we only watched for reduced crash data

29.0.1 Desktop

• respin Desktop for bug 1003707 (pdf.js) - non urgent dot release
  • all desktop fixes: bug 1003707,bug 1001167,bug 907373,bug 1005958
• 29.0.1 release with background updates enabled 100% on Friday May 9th

29.0.1 Mobile

• respin for bug 1005074

   bug 1005074 Re-enable 'Send Tab to Devices' on Beta and Release with a string swap 

   bug 987867 JB & KK crash in java.util.ConcurrentModificationException: at java.util.LinkedList$LinkIterator.next(LinkedList.java)   

   bug 1003897  Character Encoding selection broken in Fennec 29 

• Shipped to Google Play on Friday May 9th

Final Beta being RC

• Worked well for getting stability feedback

Marketing

• April 30th saw the largest spike of manual downloads since Firefox 4 days.
• The download/updates/share counter was removed from the webwewant.mozilla.org on May 12th after it passed 500,000,000.
• Firstrun and whatsnew tours are looking great from a data perspective and got lots more data after 29.0.1 was released on Friday.
• Mozilla.org is seeing +95,000 active users on the site every minute and it is mostly /whatsnew/ and /firstrun/ tours from users upgrading to 29.0 and 29.0.1.
• Funnelcake test data is looking good and we are seeing a positive ADI impact for users who saw a firstrun tour.
  • noscript installed will not trigger tours.
    • Was this tested?
  • ESR whatsnew will need to trigger tours in Fx 31
    • Will mention this to habber/jbertsch

Other

• Stability: We went out well, but the scrambling to get things fixed on beta took us right to the last minute. Hope that doesn't happen too often.

User Advocacy

• Can we make sure to QA things like high contrast mode in windows in the future?
  • let's also bring in a11y team, UX team so it's early in the design of new UI
• Also, figuring out what would be worth it as a "driver" for a patch. Broken sessions = dataloss but that wasn't a "driver". Unreadable tabs is also a bigger problem and we probably should have waved more flags. Maybe tighter tracking of what we're agreeing is /not/ getting fixed?
• We should have caught a lot more during the beta period but Eichgate happened. So, we should be aware that the lack of a full beta leads to a lot of added risk.

Notes

• qa testing pdf print - can we get a bug on this (if not one already?)
  • it's in the test plan, we could have (and in future) should
• why did we uplift a patch that didn’t have crashes on 29 (pdf)?
  • multiple factors - different methods for verifying (based on lack of crashes vs. STR)
  • based on how inflated the numbers turned out to be, release drivers should never rely on data from marketing tools for our decision making on update enabling/disabling
• what do people think about always pushing to mirrors on Monday?
  • manual checkers will always get update