QA/StaffMeetings/QA-Staff-01-25-2012
« previous mtg | QA Staff Home | next mtg »
Contents
QA Staff Meeting Notes January 25, 2012
Meeting Details
# Every other Weds @ 1:30p Pacific Time - Mozilla HQ, Warp Core # # Vidyo: https://v.mozilla.com/flex.html?roomdirect.html&key=UK1zyrd7Vhym # Phone: +1 650 903 0800 or 650-215-1282 x92 Conf# 95312 (US/INTL) # +1 800 707 2533 (pin 369) Conf# 95312 (US Toll Free) # +1 416 848 3114 x92 Conf# 95312 (Canada) # IRC: irc://irc.mozilla.org:6697/#qa
QA Org Items
- New Hires & Interviews
- item
- Upcoming Events
- item
- PTOs
Discussion Items
- Usefulness of qa-contribute mailing list -- too much spam recently; what is it used for again? (Aaron)
- Test Day 1/20 Reflection (Jason)
- (DavidB) Check out powerball and try a simple addon to earn points for bugs you file and comment on.
- Follow up on team discussion that stephend started last evening
- Virtual Machine image maintenance transitioning from Al to Geo. Want to try setting update schedule at every six months. Is this OK?
- Has QMO handoff happened?
QA Team Updates
Desktop Firefox (Juan)
- Fx.Next
- Firefox 10 sign off meeting is today at 2:30 PM in WarpCore
- If you know of any blocking issues please raise them to Juan, Marcia, or Anthony
- Firefox 10 sign off meeting is today at 2:30 PM in WarpCore
- Silent Update
- Default Compatible Add-ons:
- Huge thank you to every who helped with Default Compatible Add-ons testing; shipping for add-ons back to Firefox 4
- Add-on Hotfix
- Hotfix add-on is live on AMO for beta users, will enable us to simplify the chemspill process for *some* chemspills
- Default Compatible Add-ons:
- Crash Investigation
- https://wiki.mozilla.org/Platform/2012-01-24#Stability_Report
- Firefox 9.0.1 has a number of issues.
- Maintenance and Security (ashughes)
- Firefox 3.6.26
- Testing is underway, targeting push to Beta tomorrow, release on Tuesday
- Firefox 3.6.26
Browser Technologies (Tony)
- Issues
- Tracy taking Services QA Community stewardship.
- Fennec Native is at risk for 1/31 beta date
- List of things we need done by beta and release
- Fennec (kbrosnan/aaronmt)
- Sync Client (tracy)
- Android Native Sync code drop .3 is nearly ready. It should go to m-c today or tomorrow.
- Add-on sync is bubbling up the channels (Fx 11) gps has added a series of test to TPS for this feature.
- Switching from Mozmill to Sikuli for UI automation of sync
- Test pilot (tracy)
- nothing to report
- Sync Server (jbonacci, jrgm)
- The backlog of deployments is being addressed now that some of the pressure is off OPs for BrowserID work
- QA work will begin very soon on handling server-side test requirements for Sync+BrowserID
- BrowserID (jbonacci, jrgm)
- Trains continue to roll through
- Train 17: Bug 713961 - QA and deploy BrowserID train-2011.12.28 to production
- Train 18: Bug 715728 - QA and deploy BrowserID train-2012.01.05 to production
- Train 19: Bug 719243 - QA and deploy BrowserID train-2012.01.18 to production
- Big additions in the current and next train:
- Support for primaries
- Support for localization
- UI changes
- BrowserID team is actively pursuing leads to act as primaries (email domains, etc).
- QA work continues towards client-side automation
- QA is also working with Dev on front-end and back-end unit tests
- Other upcoming activities include build-out and deployment of PHX colo
- Trains continue to roll through
- Jetpack(?)
- Pancake (nhirata)
- Infrastructure changes, UX changes still occurring with nothing to test on the front end.
- Dria (Deb Richardson) is now officially PM
- Talked to Browser ID/WebQA team to figure out how to best tackle the issue of testing Browser ID and having infrastructure to have selenium tests for Pancake to run daily.
- webapis (jhammink)
- presently main focus with these is now with marionette on B2G (P2), mainly
- SMS
- Telephony
- Insofar as they tie into work with B2G
- presently main focus with these is now with marionette on B2G (P2), mainly
- b2G (jhammink)
- now building, test driving marionette against b2g on both devices and emulator; have gaia (frontend) working with marionette, python, and JS
- coming soon - blog post coming for all of the above
- Currently manually testing Telephony, SMS, GAIA UI, very hard!
- Test plan for MWC congress demo (last week Feb)
- test plan will cover mainly functionality (use-cases), (but also reliability, performance, security tbd)
- attending B2G workweek in feb
- Two new interns to onboard
- wanna use Vlad Maniak as a resource - and he has expressed interest!
- now building, test driving marionette against b2g on both devices and emulator; have gaia (frontend) working with marionette, python, and JS
WebQA (Stephen)
- Affiliates
- have a new contributor, Blayne Wilson.
- setting up rshetty as one of the primary qa contacts on the project
- AMO
- Engagement Projects
- De Todos Para Todos: Phase II release went out, added Portuguese site
- Mozilla Careers is in progress- should release this week
- Input
- Small regression caught by automation.
- Mozilla.com
- Shipped 1.2 and 1.3 bugs
- MDN
- Working on 2.1
- Mozillians
- 1.3 milestone still in active development
- on-boarding a new contributor, Austin Davis
- Socorro
- Code freeze today, shipping 2.4.1 milestone next Wednesday 2/1/2012.
- SUMO
Community (Marcia)
- QA Community Team meeting today
- Community Action Plan
- Discussion to develop an action plan complete with long and short-term goals; 'please participate in the discussion
- Upcoming Testdays/BugDays/Meetups
- Firefox 10 Testday (juanb)
- Video Work
- QMO.next (Al):
'
QA Automation Service (Henrik, David)
- General
- Selenium 2.17 has been released
- Mozmill
- Mozmill 1.5.8 release has been targeted for today (including support for running tests in parallel)
Project updates
- CI system for on-demand Firefox builds (Geo)
- Window systems turned out to be challenging re: setup
- Currently finalizing job setup to get prototype fully rolled
- CI system for daily and l10n Firefox Builds (Henrik)
- System runs stable within the last week - no fall-outs from Pulse
- Due to lack of documentation for Pulse some issues with queues had to be fixed
- Automated testing for add-ons default to compatible
- Currently a WIP patch is available for testing
- Waiting for the final list of add-ons to test including platforms and the MPT machines
- MemChaser extension
- Need to finish up the logging feature before we can go public on AMO
- Working with the JS team on getting an API for garbage collection implemented
- Collecting ideas for a zombie compartment detector
- Feedback from various groups is absolutely positive
- Doing preliminary testing on 0.8 (pure-Django) drop
- Feature list for 0.8 up-to-date in Tracker
- Everything above the ".8 Feature Complete" bar is expected to be in 0.8 release
- On-track for 0.8 complete on 2/3
- Q1/2012 Goals created.
Apps (David)
- Test Day on 1/20 Results (All)
- We built a well documented software project, with github pages, pydocs ..etc.
- ~7 contributors, ran into issues getting code up and running
- Gained insight on future direction of apps extension automation framework
- Began shift to java + selenium + sikuli + jbehave.
- Mochi Tests
- Met with Clint, got details on mochi tests, how to write / where to check them in ..etc.
- Began experimentation process
- Mobile (Aaron)
- Reviewing sites on native fennec, for possible Android soup from Webkit -> gecko move.
- Dashboard (jsmith)
- In progress of getting a QA Bugzilla Dashboard for Web Apps through Metrics team
- https://metrics.mozilla.com/pentaho/content/pentaho-cdf-dd/Render?solution=metrics&path=bugzilla/solr/QA&file=QA.wcdf
- Goals
- Seed new test framework
- Prepare for MWC.
Notes, Takeaways and Action Items
Today
- Add notes here
Ideas around structure for QA - no names attach - general buckets - general work needing to be done - not cast in stone - subject for talks in/out of - quickly go through it
high points:
Desktop - Jan 21st. - 2:30 warpcore: sign off on 10 - chase.com not working ex. try to catch it all - general web usage testing - if there is any bugs, is it blocking? - plans have been discussed - mitigating plans - so far, no wiki for it - Anthony on 3.6 - in the wiki
Mobile - risk for 1/31 - maybe adjusting the release date for Beta - Moving parts - reiterate try to use mobile and get performance in it, responsiveness, pulling up tabs, etc. - Services side - new work for back end - browserid: next 2 trains are big - b2g - goal for world congress - ux demo - how to flash gecko/gia on a device
WebQA - Affliates - new contributer - rajiv is doing onboarding - Mozillians - contributer Austin
Community - planning community goals - Matt approached bmoss for a headset for contributer - if you want to send a thank you/or tool for a contributer, we will find a way to help - community giving program - funding; something small; non specific, just for contributing/in the community; if we get into specifics then it lends to mercenaries. - studies on compensation. We don't want mercenaries we want community
Automation services - speed improvements due to support for running in parallel - memchaser
Case Conductor - 0.8 release - 1.0 release for the quarter - ~ 3/1 0.9 - middle of march stabilization - end of march 1.0 - small projects into case conductor
Note: - rules of snakes - don't send Bob in a knife to a gunfight - when you mean a ton of time... - how much are we going to save ahead of time - give data for the time, for the bug - if it doesnt make sense to them, you're not going to persuade them - give the next level of information
Raymond - took over QMO
Org: META groups: - Community (Marcia) - triage (bugs) : - help people log bugs - pick up the phone; help understand issues to get a solid bug - QMO
- Automation - tools : some set that make the right set of tools happen; QA/community can help implement, but no necessarily set it up (permenant) - tests : need people to actually write the tests in automated tools (nonpermenant/pull to various groups/everyone eventually get there)
- Web - Correctness
App Store (UI) M. O. Mozillians Socorro
- Server - Performance testing/Load testing/Failover testing/Crypto/Data Store
Sync Browser ID -> Services App Store Socorro
- Client (Native; Mac, Win, Android)
Mobile, Desktop Apps B2G (might be different)
-> we stand behind it because we said it... => shell -> of passing it to waverly. -> at some point we have to say no. what willing to stop?
--> What's the problem that we're trying to solve? --> Learn to say "No"
- Automation Team versus QA? Probably no difference.
[Clear understanding what the load is / What's going to break?]
[Amount of work is limited -> get the automation; maintenance work, = > do it incrementally; maintenance]
[Testing/Security testing is in all three buckets]
[Security Team] [Taros : Snappy, Telemetry] [try to have a standardization - one off ]
[strattling buckets? Community and Web/Server/Client? ]
[ Be able to cross train and do what needs to be done; specialty seems broken, expertise should be there though]
[ Accountability for the tools? - make sure that the group is happy; tool is useful? - check back with the customer - break down of communication/management]
[ Do it through influence/not authority ]
[ Skill sets are different for Web, Server, Client, ex selenium for web, but not client; server doesn't necessary have a ui, JSON thingy, ]
Next Step is to figure out details
Last Meeting
- Action Items