Firefox 36 Post mortem

• MSE
  • Lot of patches landed at the same time.
  • Impact on other features
  • pressure to push features quickly in beta. poor planning impacts quality of the release. developing features on beta doesn't have good results
  • uplifting eme on mac, feature not as isolated as it may have appeared, breaks stuff
  • Seems like requiring development on Beta is a result of poor planning - need to follow up with bizdev about better communication of requirements and timelines

• MSE / Flash / D2D 1.1
  • Sometime hard to keep track of the bugs because of various changes (backouts, multiple patches )
    • even if we limited parallel changes
  • Also hard to see separate crash volume effects because of all those and shutdownhangs
  • changes that are thought to be isolated to trivial can often cause regressions

• Need to get the gfx team off of fixing on Beta +1
  • is part of why they do this, that someone thinks there isn't any value to dev/testing on nightly/aurora because of limited or different user base? (even if that were true for some parameters it isn't true of all)

• UITour testing bug
  • Push on the release-cdntest sooner?

• No test on Windows XP SP2 ?
  • We should follow up with SV
    • [Florin Mezei] Testing covered Windows XP (both x86 & x64) for almost all Firefox 36 Beta builds, starting with Beta 1 (build2), and up to Beta 10. Windows XP was probably covered more than in any previous release, mainly because of the request to verify fix for https://bugzilla.mozilla.org/show_bug.cgi?id=1119535. The Windows XP machines used have the latest OS updates installed (SP2).

• • crash report does not work in XP? (server side issue)
    • Still broken, some versions of XP need SSLv3 and SHA1-signed certs and don't support newer (and actually secure) protocols/certs, see bug 1138794

• -remote
• [Florin Mezei] Implementation of features that can break compatibility for some users (e.g. removal of the "-remote" option)
  • Would it be possible to get an analysis of the potential impact prior to implementation, and have a backup plan in case it generates too much negative feedback (e.g. implement behind a pref)?
  • This gets to how do we deprecate features? I said i would write up a draft of how we'd like to deprecate features (but i haven't yet) will put it on the wiki https://wiki.mozilla.org/Release_Management/Deprecating_features in a similar way that we have https://wiki.mozilla.org/Release_Management/Uplift_rules - lizzard
  • Didn't have any alternative for this feature
  • warn (in the ui), intent to deprecate announcement, throw a proper warning message, have a plan to back out if needed once the change hits release, put telemetry on the feature to see how broadly it is used, give a path for users of the deprecated feature to change to (document it on mdn for example)
  • Somehow people in this bug just ignored all feedback?

• Device (roku, chromecast, etc) discovery causing firewall requests on OSX and Windows Vista+
  • Can we identify network issues in self help and assist people in resolving this issue
  • security, performance concerns

• deprecation of accepting _ and other characters in certificates & hostname
  • This seems similar to the -remote discussion about deprecating features. We've seen a number of issues related to security hardening recently. We should be careful and give lots of advance warning for changes.
  • Bug 1136616 (reported only a week ago)
  • sec team should work in coordination with ie and chrome teams so that we all remove features in a similar timeframe - so that the solution if your site isn't working isn't just to use another browser

• DNS usage of ANY (nobody is using it) bug 1093983

- general user feedback

• Snippet for Hello was a bit too distracting.

   -To be fair, we always receive a significant amount of feedback with every animated snippet or even static ones that are not the Firefox logo

• MSE back and forth was pretty rough for beta users (and honestly for us)
  • Turning features on and off in aurora still preferable in some ways
  • we had also the Youtube change for 24h
• We should make sure to respect people's preferences w.r.t. removing toolbar items
• Issues with TLS (something?, we saw stuff in SUMO but not input)
  • https://blog.mozilla.org/security/2015/01/28/phase-2-phasing-out-certificates-with-1024-bit-rsa-keys/

• Better info on what buildid is what would be helpful for input team. if they could query it (or the input site could get info on which buildid = which beta, that would be useful.

• Focus on MSE might have impacted quality in other areas, possibly also quality reorg

• Can we build something that looks for a drop in ADI (or whatever metric) that is concentrated on particular platform/os? Then if winxp sp2 drops 100K users or some deviation from the normal curve we know something is wrong