Performance/Status Meetings/2007-August-01
From MozillaWiki
Contents
Participants
Agenda
Infrastructure
- Generate reliable, relevant performance data (already underway as talos). Talos status update? http://tinderbox.mozilla.org/MozillaTest/
- running against hourly builds on trunk, see MozillaTest
- lots of crashes with historical runs, wondering if its doable, but good progress when using smaller Tp set
- http://graphs.mozilla.org/graph.html#spst=range&spss=1144483776&spse=1178280000&spstart=1144411200&spend=1178280000&bpst=Cursor&bpstart=1144483776&bpend=1178280000&m1tid=7313&m1bl=0&m1avg=0
- smaller sets going well, april06 - may07 done so far.
- AI:alice/robcee why april06 start date? version manifest change.
- AT:alice/robcee will continue back to branch date of 12aug2005.
- Discuss talos cpu usage metric
- cpu usage numbers... discussion on how cpu usage being calculated.
- going to ignore those numbers for now, no humans looking at them.
- qm-pmac04 setup as a buildbot slave as per the mac test ref platform, but not yet hooked up to the master.
- use --enable-tests or else copy files in after the build completes
- decided to copy files after build, AI:alice to write script for this.
- AI:joduinn meet with robcee, rhelmer, zach, alice to figure out buildbot slave-in-talos issues. No update. Will try today 8/1.
- zach: Page-load-server fixes in place, running on new page load server, in parallel for a while.
- Areas where help is needed
- expand the scope of performance testing beyond Ts/Tp/TXUL/TDHMTL
- reduce noise in tests
- reduce to ~1% (suggested by bz, not started)
- some hardware for large-sample-set-Tp-100times-per-page
- havent had time for buildbot integration, not needed.
- improve performance reporting and analyses:
- Better reports for sheriffs to easily spot perf regressions
- Tracking down specific performance issues
- stats change to track AUS usage by osversion. bug 382044. AI:justin
- Priorities for infra:
- Generate historical baselines
- General profile data regularly on builds
- Getting the perf numbers more stable
- Developing the graph server to display time spent in each module
- AI:rob arnold proxy-server work available last week bug 389703
- New ideas
- Question: How are we tracking perf bugs, specifically, and are we doing this the same way we are triaging security bugs? Can we do it the same way if not? (AI:damon)
Gecko: Perf discussion
- Perfathon update
- vlad has data, wants to provide Mac data, but is blocking on cairo (because otherwise the topmost stuff by far is random cairo rendering that's fixed)
- pageload test rewrite (bug 387110)
- converting talos to new pageloader
- porting other tests -- vlad has patch for self-timed tests
- already converted old DHTML perf tests
- how do we collect pages?
- TODO: add --enable-nightly-tests
- (makefile variable, not C++ variable)
- Todo: verification of test results
- Should have been verified -- looks fine
- But the talos pageset needs more work -- rob arnold looking into proxy solution
- Plan: we will do a) new machines; b) new pageloader; c) old (300-page alexa) pageset
- probe update
- module owners should let stan know if they have any situations that they'd like to analyze, so that he has more information about how to structure the probe work.
- 25jul: no feedback from module owners yet
- module owners should let stan know if they have any situations that they'd like to analyze, so that he has more information about how to structure the probe work.
Previous weeks
- jrpof tinderbox
- synthetic tests (TODO)
- put specific tests into talos framework
- poach mochitest performance tests
- individual reftest-like items
- need to get a bunch of people looking at profiles (TODO)
- take up some of these meetings to sit down and look at profiles
- timer-based profiling is better (vtune/jprof/oprofile/etc., not quantify)
- TODO: vlad to generate profile for next week's meeting
- running without Fx chrome
- new pageloader stuff can do this
- examining default theme for performance issues
- Tp vs. Tp2
- new pageloader stuff avoids this problem
- Tp2 was flawed in some ways -- one big one is that it loaded pages in an iframe, so initial paint delay was never coming into play, and we were potentially reflowing more thannecessary