Breakpad/Status Meetings/2017-03-01

« previous meeting — index – next week » ^create?

Meeting Info

Breakpad status meetings occur on Wed at 10:00am Pacific Time.

Conference numbers:

   Vidyo: Stability 
   650-903-0800 x92 conf 98200#
   800-707-2533 (pin 369) conf 98200# 

IRC backchannel: #breakpad
Mountain View: Dancing Baby (3rd floor)

Operations Updates

• Socorro has been stable
  • AWS outage was US-East, we're US-West, unaffected
• Mostly working antennae
  • datadog antennae
  • load testing
  • deployment pipeline should be done
• admin node updating has a PR waiting
• ES status
  • no hosted ES setup

Project Updates

• Plan for making admin nodes auto-rebuild on deployments
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1341755#c4
  • tl;dr peterbe to brush up that 1.5yo PR, then miles to add some features to let long jobs finish
• amiyaguchi (and mreid) are hacking on why the Spark ingestion job stopped working on Feb 22
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1343018#c3
  • Should we inform the stability list??
  • filing follow up bugs for monitoring

Deployment Triage

• https://whatsdeployed.io/s-7M7

PR Triage

• http://prs.mozilla.io/mozilla/socorro

Major Projects

Splitting out collector (Antenna)

• (miles, willkg) watched load tests happening via datadog; wrote up a handful of bugs and fixed some stuff: health stats reporting, better startup error capture, better logging, retry code
• (miles, willkg, lonnen) figured out which s3 bucket to use
• (willkg) working on migration process
• (rpapa) failure in the load test harness and we're working to verify that the broker is working as expected.
• (rpapa) Since Miles and will are working on DataDog, rpapa will notify on every run
• (rpapa) expect a spreadsheet with results

Deprecation rampage

• current status
• bunch of small endpoint removals
• correlations is the next thing to be removed
• leaving behind the database stuff for now
• on track as a Q1 goal
• crontabber jobs we expect to delete: https://docs.google.com/spreadsheets/d/1qowM5Qy5DKX-vU_ezk9PlXOgo5rRsOTMeRwi3L-aHe8/edit#gid=0

Processor rewrite

• we will need to meet in person to discuss some of the larger architectural stuff
• planning on doing it at the all hands, unless that falls through or we think we need to do it sooner

Upgrading elasticsearch

• (Adrian) done with code updates!
• (Adrian) need to test locally with fake data
• (Adrian) need to import SuperSearch Fields data locally and export an ES5-compatible mapping to use during reindex

Other Business

Travel, etc

• (Adrian) next week is my political event
  • I'll be working but probably not at full capability
  • might take unexpected time off if needed
• (peterbe) out half-day on Friday 3 March
• (lonnen) out Friday
• (mattyb) out all next week, rpapa covering

Links