User:Rcampbell/Toronto Summit Notes
From MozillaWiki
These are robcee's bulletized notes from the Toronto QA Development Summit on January 25th-26th, 2007. They are rough and there are large pieces missing. Feel free to fill in the gaps.
Contents
automated testing
Buildbot functions
- pull nightly
- initiate test runs
- build given patch
- Ben Hearsum doing work with Build team on this
- select platforms
- reporting handled by performance testing framework
- litmus reporting
- clean up build directory/create cleaner mirror
- get rid of kruft
- drive performance testing
Reporting Tools
Survey of reporting tools
- litmus
- buildbot
- tinderbox
Report aggregation
- RSS
- Atom
- Microformats
- Security
- passworded version
- irc
- report drill-down
- history
- test details
- logs
- store links in database for detailed analysis
Rollups
- graphserver
- buildbot?
History
- store some set period
- retain important historical milestones / benchmarks
JS Tests
- specialized builds
- buildbot vs tinderbox canned builds
- requires js shell, xpcshell, can't use 'off-the-shelf' versions.
Metrics
- bug density
- per component
- map bugzilla modules
- checkins churns
- lines of code
- performance thresholds
- links on wiki
- top-crashers