QA/Test Automation/2010-08-25
From MozillaWiki
< QA | Test Automation
previous meeting | Meetings | next meeting »
Contents
Attendees
- Al, Geo, Matt, Anthony, Aaron, Henrik, Adrian
Last weeks action Items
- [DONE] Henrik: Send testday invitation to mozmill-dev and planet
- [DONE] Henrik: We need a bug for Mozmill 2.0 to allow usage of different web servers (bug 590432)
- [DONE] Henrik: Create outline proposal for
teamrecruiting video - [MISSED] Matt: Check what limits test harnesses on buildbot to use other local webservers
- [DONE] Matt: Discuss with Marcia the length of each video for teams
- [DONE] Geo: We need feedback from Clint about the scope and feature set for Mozmill 2.0
- [MISSED] All: Check the proposed docs for the shared module refactoring from Geo
Projects
- Mozmill (Henrik)
- Mozmill 1.5 has been released on Monday
- Thanks everyone for your help in filing bugs and verifying fixes!
- Mozmill 1.5.1: Not targeted yet but will contain bug fixes and the events patch
- Mozmill 2.0: Add your proposals
- We should wait at least a week until we upgrade Mozmill on our qa-set and qa-horus boxes
- Post-Mortem for Mozmill 1.5
- Scheduled maintenance releases with clear set of fixes and features (1 per month?)
- Setup a list of prioritized feature requests for 2.0
- Test creation / shared modules (Henrik)
- All l10n backports have been landed for older branches (1.9.2 and 1.9.1)
- Local Test Data (Anthony)
- 20% complete
- No progress due to heavy release work (4.0b4, 3.6.9, 3.5.12)
- Targeting Sept 1st completion
- Broken Tests
- Firefox 4.0 (Geo)
- Broken tests are still broken, re: missed goal in personal section below.
- Have started test runs with Mozmill 1.5 final, will be generating new bugs/fix list from those runs.
- Firefox 3.5/3.6 (Anthony)
- No update
- Update Tests (Henrik)
- SoftwareUpdateAPI has been successfully backported to 1.9.2 bug 567258
- Releng is interested to run Mozmill nightly updates bug 588398
- MozMill Crowd extension (Henrik)
- no update
- Add-ons Testing (Henrik)
- no update
- Automation (Henrik)
- Automation scripts allow a custom repository url now bug 571978
- Support software updates for Thunderbird bug 580611
- Add support to automation scripts to modify the report data bug 586336
- Renamed BFT testrun to General testrun bug 586513
- Mozmill calls sys.exit() - restart-test run doesn't get executed bug 589003
- Other fixes:
- Global timeout of Mozmill killed software update tests bug 586652
- Use canonical path to test modules bug 586792 (needs review)
- Dashboard (Henrik)
- No update - I can hopefully start this week
Risky Goals
Personal
- Henrik
- Last two weeks
- 1 week vacation
- [ON TRACK] Create shared module for add-ons manager and finalize list of tests
- [DONE] Update all automation scripts to be compatible with Mozmill 1.5
- [DROPPED] Update the software update script to send full reports to brasstacks (next quarter)
- [MISSED] Finalize specs for Web dashboard
- [MISSED] Continue work on the crowd extension
- Next two weeks
- Update AddonsAPI and tests to work with Firefox 4.0
- Create list of Mozmill tests to implement
- Create spec for the dashboard (General test-run pages)
- Anthony
- Last two weeks
- [MISSED] Complete conversion of 31 tests to use local-data
- Next two weeks
- Complete conversion of 31 tests to use local-data
- Al
- Last two weeks
- Nothing, focused on release testing work.
- Next two weeks
- Get script for Bug 553618 re-fixed and submitted.
- Figure out what we can do with microsummary restrictions.
- Geo
- Last two weeks:
- Continuing with Shared API Refactor project
- [MISSED] Fixes for bug 573582, bug 573584, bug 575250
- Reviewing test results for any additional breakage
- Help with 1.4.2 beta release as needed
- Next two weeks:
- Generating functioning prototypes for refactored UI modules
- Collecting team feedback re: prototypes
- [CARRY OVER] Fixes for bug 573582, bug 573584, bug 575250
- Aaron
- Last two weeks:
- Help with the 1.4.2 release
- Continue conversion of tests to use local-data
- Continue investigating Linux VM failures (qa-mozmill/horus)
- Next two weeks:
- [CARRY OVER] of conversion of 31 tests to use local-data
- Refactoring tasks in tests for 1.5
- Adrian
- Last two weeks:
- [DONE] L10n support in existing tests backport to 1.9.1
- [ON TRACK] prototyped idea of CommonA11yL10nAPI and L10nAPI
- Next two weeks:
- Documentation + UML Diagrams for CommonA11yL10nAPI and L10nAPI
- implementation of the CommonA11yL10nAPI and L10nAPI
- Targeting the end of September for all the L10n work completion
Contributor Status
- Name
Roundtable
- General
- QA Automation video
- 1st level: Global overview to recruite new community members (Work Week)
- How can we knock someone up to help out in our team?
- What is Mozmill
- Why is UI automation important (i.e. time reduction, different locales)
- In which areas you can help
- Test execution and result reporting (Crowd extension)
- Test and shared module creation
- Test failure analysis (dashboard) and fixing broken tests
- Improving automation scripts and the Crowd extension
- Suggestions
- High level what we do and how you get involved
- Matrix of people in the room without automation vs. with automation
- Sience Fair like introduction video of QA members (eventually Work Week)
- Detailed QA Automation videos with members
- 2nd level: Howto for major automation goals by team members (All Hands?)
- In depth presentation of main projects with screencasts
- 2nd level: Howto for major automation goals by team members (All Hands?)
- QA Automation presentation (Henrik)
- Who has time to help and put the presentation together?
- Henrik has existing presentations from a couple of talks as template
- Release testing automation tasks (Matt)
- yesterday it became evident that we need to redefine automation tasks and responsibilities for release testing
- Improve communication with Auto-Tools team (Matt)
- Issues
Action Items
- Geo: Schedule meeting for debugging/execution labs for automation
- Geo: Send out list of broken tests w/ Fx4 and MM1.5
- [DONE] Henrik: Update QA lab machines and databases to use Mozmill 1.5 (Sep. 1st)
- [ON TRACK] Henrik: Send out email for more details to introduction (recruiting) video
- [DONE] Henrik: Send out email to start the work for the automation presentation
- [DONE] Henrik: Schedule meeting for release tasks and machine reservation
- [DROPPED] Henrik: Setup a list of prioritized feature requests for 2.0
- Will work together with the A-team during our work week
- Matt: Check what limits test harnesses on buildbot to use other local webservers (connect with John O'Duin)
- [DONE] Matt: Request a post mortem for Mozmill 1.5
- All: Check the proposed docs for the shared module refactoring from Geo