Contribute/Workshops/Metrics for growth/Documentation

From MozillaWiki
Jump to: navigation, search

Anyone at Mozilla - module owner, newcomer, veteran, staff, volunteer should feel welcome to initiate or run the Metrics for Growth Workshop.

How to Use this Manual

The purpose of this wiki page is to enable interested parties in leading - or simply drawing the lessons of - the Metrics for Growth Workshop. As such this page is a sort of loose manual or guide. Please read the entire page and familiarize yourself with the lessons and goals. The better prepared you are, the more fun and rewarding the workshop will be, both for yourself, and for others participating in it.

That said, here are some high level guidelines for using this manual:

  • While a lot of thought has gone into creating the syllabus and lessons, everything here is hackable, you need to run a workshop that speaks to your needs.
  • Reach out to others who have run this - at the very least that will include David Eaves
  • With luck others will have added content, examples to reference, or new lessons, to this page, please make use of them and add your own!
  • Most importantly, have fun and be open to the unexpected.

Purpose & Goals

What are the Goal of this Workshop?

The Identifying Contributors workshop is a sequel to the Designing For Participation and the Identifying Contributors workshops. The Metrics for Growth workshop was created to help Mozilla community members think about how measure - and this improve - the size, nature, contribution rate and overall happiness of their pool of contributors. As such the workshop has these goals:

  • impart on participants some best practices around open source community metrics
  • foster a conversation where best practices and ideas can be surfaced and critically assessed, particularly around how suggestion may or may not be relevant to areas the participants work on
  • provide participants with action items they can take back to where they contribute to Mozilla that will help improve how they measure community participation
  • share differing experiences, perceptions, challenges and opportunities around effective metrics as understood by managers, employees, staff, volunteers, newcomers and veteran contributors.

Who Should Participate?

Almost anyone at Mozilla should feel like this is open to them. "Roles" that might want to participate or lead this workshop include:

  • module owners
  • newcomers
  • veterans
  • staff
  • volunteers
  • managers

Some of the more common questions participants often have that make them a good fit for the course include:

  • We are starting a new product/activity/function and want to think about how we can encourage community participation
  • I'd like to understand if our community is growing/shrinking
  • I'd like to try to experiment more around community participation and want to know what the impact will be
  • I need to expand the scope of our work without increasing the financial resources I have
  • I’m a manager and I'd like some tools for thinking about how to evaluate how effectively those who work for me create space and opportunity for community participation.
  • I've just been asked to expand our community participation, what do I do?
  • I'm a long time contributor with lots of thoughts how we could manage community more effecitvely
  • I need to expand the scope of our work without increasing our groups financial resources

Slide Deck

The slides for this presentation can be downloaded here in

Assumptions & Background Material

Some assumptions about both participants and facilitators:

Logistics

A few things to note before sending out invites:

  • this course works best with something between 3-25 participants. Anything more than that and it gets unwieldy, especially if you are not used to facilitating/teaching
  • please ensure that participants have access to the internet so they complete the workshop, or print them out in advance

Basic Tips on Facilitation

Link?

Syllabus breakdown

The course is broken down into four parts:

  • introduction
  • review of existing dashboards
  • exploring conversion points
  • application to personal work situations
  • wrap up

Introduction

This short part of the workshop is to enable people to quickly get to know one another and align around the shared goals of the workshop.

Slide 1: introductions

  • this should ideally take no more than 5 minutes
  • Introductions are important, it helps foster a sense of community and a sense of shared space where people feel it is safe to share their stories, questions or ideas.
  • Make introductions brief - we only have one hour
  • Ask each participant to share their name, their role/title in the Mozilla community and 3 words about open source projects, this always generates great feedback
  • Model good behaviour by going first (and adhering to whatever guidelines you set out)

Slide 2: Context on Social Capital

  • 2 minutes
  • goal: why this workshop matters
  • talking points:
    • For normal companies financial capital is the lifeblood, they live and breath on cash-flow
      • They measure where their capital flows
      • How much there is
      • Is it ebbing or flowing
    • But as an open source community our social capital is our lifeblood, the social capital (while important in a normal company) is even more important for an open source community. Buying community is probably difficult, possibly impossible.
    • So if a normal company measures its financial capital, because how much it has matters so much, why don’t we measure our social capital as closely?

Slide 3: Goals Slide

  • this slide should take no more than 5 minutes (including Q&A)
  • goal: get everyone aligned around common objectives for the workshop
    • Can we measure the size and effectiveness of our community
    • Are our strategies for growing/sustaining community working
    • Can we experiment more (and increase the rate of learning) which means, can we measure impact?
    • Cane we develop structured ways for thinking about how we measure community
  • see if there are other goals that people have
  • good communication about the goals BEFORE the workshop is key to ensure that this slide does not turn into a large debate

Review Existing Dashboards

Time: The goal of this exercise is to get participants aware of:

  • What is already being done
  • The potential of metrics and dashboard
  • Ease them into this topic

Slide 4

‘’step 1’’ Time: about 3 minutes.

‘’Step 2’’ Time 5-7 minutes

  • Debrief
  • Sample prompting questions
    • What are decisions are these dashboards trying to enable
    • Who are they enabling and educating
    • What is useful and not useful
    • How might a community manager use such a dashboard
    • How might a volunteer use these dashboards.
  • Slide 5 and 6 here to serve as a way to help talk about lessons and to pull up if/when someone is talking about them.

Some best practices/actions

  • If you are looking at number of contributors you can see the impact of experiments, did it create or diminish contributors if you alter tools, change recruitment language, etc…
  • You can begin to see who is contributing more or less, and use that to intervene and replicate conditions that is foster the former and alter those that are fostering the latter
  • Story: During the 2010 Mozilla summit it was a struggle to assess who to invite since there was so little data and thus few metrics for figuring out who contributors were.

Key Lesson

  • Dashboards and metrics are key to improving
  • Only by measuring can we create a learning culture that allows us to know what is working

Slide 7

Time: 4-7 minutes (depending on discussion time) Purpose: Show some best practices/examples from other open source communities

Tracking Conversion Points

Slide 8

Lecture slide

  • key points
    • In order to create a dashboard it is often helpful to think of:
      • What are the key pieces of information a manager and/or contributor would like to know to make decisions
      • What are the key steps that a contributor makes as they complete a task in your part of Mozilla

Slide 9 &10

At Mozilla some of the conversion points for several core activities have already been mapped.

  • What are the activities someone must do to volunteer
  • Can we measure when they have reached these points
  • How many people make it past each step (what is the rate of attrition?)

Slide 11

  • Spend 2-3 minutes looking over some of the conversion points outlined on the mozilla wiki.
  • Spend 3-4 minutes and jot down the 2-10 steps that a volunteer needs to do to contribute to your part of the mozilla project (this does not need to be perfect)

Slide 12

  • Debrief
    • Any discoveries?
    • Do you feel like you have conversion points

Some additional thoughts:

  • Are their conversion points that are common across all of Mozilla (such as create a Mozilla login)
  • It is not always a strict progression (creating a bug is not necessary to do before triaging a bug)
  • But even if not linear, but the more activities one has done the better, and their are certain activities are more valuable to the project

Slide 13

‘’Step 1’’ Activity

  • Take 4-5 minutes
  • Ask participants to look at the conversion points they have brainstormed and:
    • assess what activities/conversion points could be tracked by a system (e.g. Bugzilla)
      • write these examples write below the activity/congestion point you brainstromed
    • assess what are the most important activities to their part of the project that you should measure?

‘’Step 2’’ Debrief

  • Have group share out their brainstorms

Some key lessons/thoughts:

  • Definite risks of ignoring people who do things that are not tracked (such as evangelists, or people who install FF on lots of friends and family member computers)
  • Data collection cannot become a burden that deters contribution
  • Reputation systems also happen to create good data about contributions
  • You don’t need to have complicated systems to do this. SUMO used a simple google spreadsheet to track weekly stats to give them longitudinal data
  • Creating “beta” dashboards can be a good starting point

Slide 14

  • 5 stage process agile process for using open source community data
    • Most of what we are talking about in this workshop is stage “1” (e.g. where are we at/what should we measure?)
    • You then gather data measure where you are at
    • Generate insights from this data (are we where we think we are)
    • Make decisions on new actions
    • Assess impact of new actions
    • Being anew
  • Discuss how this process might work on our part of mozilla?
    • What actions would you like to take

Slide 14

  • Discussion how data can help drive more human contact
    • How can data drive you have to have better conversations with community members
    • How can dashboards help contributors flag issues to module owners or managers

Wrap up

  • time: about 2-5 minutes
  • Purpose: to create a sense of shared value (or determine if the workshop did not create value for participants
  • Process:
    • Ask participants to share best thing they learned or heard
    • Ask participants to share one action they intend to take as a result of the workshop
    • Ask each participant to name one thing they're taking away from the workshop that was useful, and one thing they want to learn more about.
  • Logistical questions to consider
    • If there's a follow-up class, share details
    • Invite feedback via email
Retrieved from "https://wiki.mozilla.org/index.php?title=Contribute/Workshops/Metrics_for_growth/Documentation&oldid=1019120"