Summary

There is a large discrepancy between the platform/hardware/driver configurations we see on Release versus what we see on our test branches. Additionally, the Graphics team and our testing community does not and can not provide sufficient test coverage to fill this gap. As a result we've historically been notified far too late about critical graphics issues triggering multiple chemspill releases and a loss of users.

In an effort to improve pre-release visibility of graphics regressions we have partnered with Betabreakers, a company that maintains a lab specifically designed to test various modern and legacy configurations of Windows versions, graphics hardware, and graphics drivers. We try to coordinate a graphics sanity testrun with them once per mozilla-aurora cycle, as detailed below.

More information about our relationship with Betabreakers can be found on mana.mozilla.org.

Test Runs

• Firefox 50: Hardware Accelerated Video
• [DONE] Firefox 48: Windows 10 Anniversary Update
• [DONE] Firefox 48: WARP disabled further
• [DONE] Firefox 46: HTML5 video with hardware acceleration disabled
• [DONE] Firefox 45: WinXP D3D9 E10S compatibility
• [DONE] Firefox 44: Country-specific top sites
• [DONE] Firefox 43: Typical sanity check plus bug cleanup
• [DONE] Firefox 42: Windows 10 AMD/NVidia stability
• [DONE] Firefox 41: WARP disabled on Windows 7 and earlier
• [DONE] Firefox 40: WebGL with e10s
• [DONE] Firefox 39: beta sanity check
• [DONE] Firefox 38: MSE stress test

Bugs

99 Total; 5 Open (5.05%); 90 Resolved (90.91%); 4 Verified (4.04%);