Firefox/Channels/Meetings/2017-09-12

Channel Meeting Details

• Tuesdays and Thursdays at 10am Pacific Time
• irc.mozilla.org #planning for backchannel

Video/Teleconference Details - NEW

• 650-903-0800 or 650-215-1282 x92 Conf# 99951 (US/INTL)
• 1-800-707-2533 (pin 369) Conf# 99951 (US)
• Vidyo Room: Release Coordination
• Vidyo Guest URL

Attendees

lizzard, RyanVM, julien, Sylvestre, nicole, philipp, dveditz

Previous Actions

Schedule Update

• 56.0b11 mobile, desktop released today

• 56.0b12 build will be early (Wed. evening PST)
• Beta 12 release can be Friday, or Thursday if QE is finished.

   ** We can be flexible, but earlier is better

• release calendar updated
  • (syl) Merci!
  • https://docs.google.com/document/d/1jeypuqBqEyIh-4qxXT0UnE2aVjetN7uVD8W7L7TbWKg/edit

release blocking issues for 56

2 top crashes from Win64 migration

• Kaspersky https://bugzilla.mozilla.org/show_bug.cgi?id=1268470 (patch landed to blocklist for beta 11)
• Bug 1347867 - Crash in CrashReporter::OOPInit

Fennec locale issues, some affecting partner builds

• Bug 1389709 - Locales are wrongly mapped in Fennec, causing an incorrect list of search engines to be displayed - Still no assignee. But we have a patch, testing underway. Sadly this missed beta 11.

Graphics layers shutdown crash, very high volume, many user complaints

• Bug 1389021 - shutdownhangs in mozilla::layers::CompositorThreadHolder::Shutdown
• Patch with possible fix likely will be ready for beta 12

Audio/Video playback crash

• Bug 1389470 - Crash in alloc::oom::oom Patched, didn't work, reopened. New patch landing now

• Possible beta 12 fennec tomorrow.

Add-ons

Shield studies

• Quick review of shield studies or experiments.
• Is anything waiting for signoff or testing?
• Shield studies in flight: https://gregglind.github.io/x-shield-inflight-dashboard

Stability

Nightly

awsy rate: 3.68 (browser 1.36, content 2.32) telemetry (m+c-s) from sunday: 12.07 (week before: 7.61, this time last cycle: 8.63)

[marcia]

• Rate is in high threshold but seems to have fallen over the last few days
• Top non browser crash https://bugzilla.mozilla.org/show_bug.cgi?id=1397214 is pretty high volume
• Top crash https://bugzilla.mozilla.org/show_bug.cgi?id=1396368 was just fixed
• Crash triage crashes: https://wiki.mozilla.org/NightlyCrashTriage/2017Q3

Beta

awsy rate: 1.16 (browser 0.89, content 0.27) telemetry (m+c-s) from sunday: 5.85 (week before: 5.25, this time last cycle: 4.73)

[philipp]

• win64 startup crashes are less common in b10 than in b9, but that probably only means affected users are stuck/leaving
• Kaspersky crash https://bugzilla.mozilla.org/show_bug.cgi?id=1268470
  • we could work around with dll blocklisting in b11 & kaspersky has a fix too
• Quickheal crashes https://bugzilla.mozilla.org/show_bug.cgi?id=1347867 & https://bugzilla.mozilla.org/show_bug.cgi?id=1397750
  • afaik we only have heard from quickheal that they'll look into the matter; our normal DLL blocklisting doesn't work -> we would need someone to look into a more targeted solution (similar to 1380335)
• Lenovo OneKeyTheater crash https://bugzilla.mozilla.org/show_bug.cgi?id=1369361
  • product is outdated and most likely won't receive fixes anymore; our normal DLL blocklisting doesn't work -> we would need someone to look into a more targeted solution (similar to 1380335)

56.0b10 top issues:

• Gfx shutdownhangs with mozilla::layers::CompositorThreadHolder::Shutdown: https://bugzilla.mozilla.org/show_bug.cgi?id=1389021 - 16% of browser crashes
  • potential patch landed on nightly today but comments in the bug aren't overly confident - we'll see in a day
• Startup crash in mozilla::CacheDataAppendElement: https://bugzilla.mozilla.org/show_bug.cgi?id=1276488 - 3.4% of browser crashes
  • diagnostic patch didn't produce much actionable (greprefs.js file can't be read); only affects two dozen installations
• Crash in rust mp4 parser alloc::oom::oom | alloc::raw_vec::RawVec<T>::double<T>: https://bugzilla.mozilla.org/show_bug.cgi?id=1389470 - 1.5% of browser crashes
  • bug has a patch; will need to get it landed & into beta
• Crash in DLL modules relating to RelevantKnowledge (adware/malware): https://bugzilla.mozilla.org/show_bug.cgi?id=1394550 - 1.2% of browser crashes
  • regressing in volume in 56; our normal DLL blocklisting doesn't work -> we would need someone to look into a more targeted solution (similar to 1380335)

• list of crash fixes going into b11: https://mzl.la/2eSOioE

Release

awsy rate: 0.78 (browser 0.56, content 0.22) telemetry (m+c-s) from sunday: 3.12 (week before: 3.15, this time last cycle: 2.30)

[marcia]

• 55.0.3 rate looks good.
• #14 top crash

shutdownhang | NtWaitForKeyedEvent | RtlSleepConditionVariableSRW | mozilla::TimeStamp::Now | GetTickCount64 | SleepConditionVariableSRW | mozilla::detail::ConditionVariableImpl::wait | mozilla::CondVar::Wait | nsEventQueue::GetEvent | nsThread::nsChai... needs a bug. Comments mention freezing and crashing. Might be an antivirus issue and need to look at the correlations.

Mobile

awsy rate: release 0.66 - beta 0.85 - nightly 2.52

[marcia]

• Nightly
  • New Nightly crash https://bugzilla.mozilla.org/show_bug.cgi?id=1399020
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1399148 is #7 in top nightly crashes
• Beta
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1340959 still the top overall browser crash
  • #2 overall browser crash is https://bugzilla.mozilla.org/show_bug.cgi?id=1394788, also fairly high on release. Not clear if https://bugzilla.mozilla.org/show_bug.cgi?id=1394428 will address these crashes or not
  • https://bugzilla.mozilla.org/show_bug.cgi?id=760394 continues to be in top part of beta
• Release
  • Legacy bug https://bugzilla.mozilla.org/show_bug.cgi?id=760394 continues to be very high, over 10626 crashes in a seven day span
    • Turns out we are processing 100% of crashes on Mobile so this should not be an issue

Performance (Thu)

Aurora / Dev Edition

Beta

Release

QE

Nightly

Aurora / Dev Edition

Beta

Action items

Mobile

Performance

RelEng

Marketing/Communications

User Advocacy

Roundtable

• We have slack (!) channel to discuss about 57 triage: https://mozilla.slack.com/messages/C719PRABY/
• e10s-multi to 100% for 55? only users with webextensions based add-ons get e10s-multi. In 57 only webextensions will load - so add-ons won't be a blocker for users getting e10s-multi by 57.
• war room for 56? did that work for 55?
  • It was ... not useful ? :) I used the slack channel to give updates, I think some people appreciated the updates, almost nobody showed up in the vidyo as I prepared everything Paris morning time
    • which slack channel?
• Nightly renamed to Firefox Nightly
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1378834

Special Topics

Aurora/Beta Feature Review

Post-Mortem (Tues 2wks from GA Release)

Sign Off (Thurs prior to GA Release)

Actions