2005Offsite/LitmusFutures

From MozillaWiki
Jump to: navigation, search

Topics for Discussion

  • Vision of Litmus
  • Current State
  • Future milestones
  • Open discussion of useful features
  • How developers can help

Session Notes

  • admin interface:
    • test events/runs built from testgroups
      • ability to select specific subgroups
        • must firm up granularity with which we want to be able to build test runs;
      • ability to tag runs with random tags (e.g. rc2)
    • test case versioning:
      • tagging
      • localization (l10n)
  • test runs:
    • mandatory comments for FAILing results
    • report opt/cflags, obtainable from about:buildconfig (investigate the build ID scraping code that Zach already has in place)
  • l10n
    • localize litmus itself
      • issues with test cases/results submitted in languages other than english is that the core MozQA staff cannot help parse results, however if we enlist localizers/locale coordinators and give them some ownership of locale-specific results, we may actually encourage more l10n testing (which is a good thing).
  • search:
    • allow for personal saved searches (a la Bugzilla)
      • allow admins to publish personal saved searches to everyone/subsets
    • allow searching by user type
      • requires auth
      • allows for quality of results reporting
  • auth:
    • we have to allow for anonymous testing, i.e. we cannot require a Bugzilla account for Litmus use. The perception is that this is too high a barrier to entry for casual testers. Suggestions to overcome this:
      • straight anonymous testing
      • password-based logins with no relation to Bugzilla (con: another account for users to remember/forget)
      • redirect users through Bugzilla on result submission
      • ability to associate results with a Bugzilla/Litmus account after the fact.
        • track some form of data about the user in the db (temp username, cookie) and backfill that once the user signs up with an account.
      • NOTE: these are not necessarily competing paradigms; we might adopt more than one
  • general:
    • do we need RSS?
      • perhaps only for automated test results
  • QMO
    • return to the idea of QMO -> similar to MDC in that QMO is the clearinghouse for testing-related information:
      • QA blog posts
      • testday info
      • intro to testing/Bugzilla info
      • Litmus tutorial/instructions
      • highlighted testing results
      • etc.
    • speak to dria about lessons learned from MDC
  • short term:
    • in prep for Thunderbirrd 1.5 testing, add l10n property to test results (default to en-US). Update search tools to allowing limiting by locale.