ParticipationMetrics/Community Dashboard/Codereviewdbspec
Code Review Dashboard Specifications
Overview
The goal of this dashboard is to give community members and module owners a better understanding of the choke points in the developer/contributor process. We've highlighted the issue of submitting patches as this is a known sore point, but the dashboard should probably be open to any such choke points that can be measured.
Please see the discussion page for questions and clarifications on the spec described below.
Specs
Dashboards for four types of users:
Project Overview
Personas: This is for a fan of firefox or an executive wanting to see what is going on at the project level.
features: Average code review weight time at the project level
Module Owner
Personas:
Contributor
Personas:
Code Reviewer
Personas:
Ideally the dashboard would be able to answer the following questions:
- Tell users the averages wait times for code review in several ways:
- average time all bugzilla bugs spend in any given "state" in bugzilla (submitted, confirmed, assigned, review, etc...) It might be good to maybe compute that average by removing that outlier 1 or 2 percentile bugs, but I'm not totally sure of that).
- average time a module's bugs spend in any given "state" in bugzilla (submitted, confirmed, assigned, review, etc...) (recognize this may not be possible
- average time a specific users bugs spend in any given "state" in bugzilla (submitted, confirmed, assigned, review, etc...)
- a view that looks at a users patches and tells us how long each one has been in review.
- Create a view that shows code-reviewers performance.
- Allow module owners to identify who turns patches around quickly. This of course should also show quantity - something like the first dashboard but rather than number of patches, it shows, the number of code reviews completed by month, number of code reviews requested by month (e.g. what % of reviews are they completing of those requested of them).
- At the module level and the project level, one should be able to see the average wait times over a given period of time, that way we can see if wait times for code review are getting shorter or longer.
The above features feels like the core requirements. I hope that other users might add features they would like to see as well.
Feedback
- Taras (monitor review lag)
- Treat r-/r+ as "review accomplished".
- Another action is removing r?(to indicate reviewer isn't interested in reviewing this), or reassigning r? to another person(these should probably count as "review accomplished" too)
- Might be more useful to measure review response time within the last 6months or base
- Alternatively response time could be measured on last x patches where x would be 10-100
- replace green/red dot with a single indicator showing "this reviewer is having trouble keeping"(either due to patch overload or just reviewing slowly)
