SoftwareTesting

From MozillaWiki
Jump to: navigation, search

In The Ongoing Revolution in Software Testing (pdf), Cem Kaner defines testing as

[...] a technical investigation of a product, done to expose quality-related information.

This page is a starting point to find, learn about, and track technical investigations of the Mozilla codebase.

Please note: this catalog is not limited to investigations done by members of the Mozilla qa team and community.

Projects

Testing in General

We need your help! There is a small but growing set of software tools for testing Firefox functionality. Please see MDC:Mozilla_automated_testing for details on what we have currently.

Black-Box Testing

Software Update Testing

Project Ideas

  • Defining a common invocation mechanism for tests, with example code
  • Defining a common reporting format for tests
    • perl has TAP, xUnit has textrunner (. vs F)
  • fixture-based frameworks for test development
    • hosting of Fit/Fitnesse to run tests against a browser instance
    • tinderbox-based tests

Areas/concepts to investigate

Wishlist for xpcshell test harness

  • Ability to mark tests as XFAIL when they are known to fail at the moment
  • non-fatal do_check_* (as new functions) to be able to see all the test failures, not just the first one

Tools

People

Ideas to Collect

(stuff that needs to be organised)

  • SoftwareTesting:Tools:jsUnit
  • SoftwareTesting:Tools:SimpleXulTestHarness
  • example extension test using foxunit
    • davel thinks xul/js-run-as-chrome is probably better
  • jsunit in chrome proof-of-concept - file and Makefile.in in the same directory
  • dbaron's layout reftest stuff
    • uses copy of domi code to handle command-line args in both firefox and seamonkey
  • continuous integration/test issues
    • bug 351968 filed to track ENABLE_TEST issues
    • build team has decided to avoid spending time getting new things to work with tinderbox
      • we need a tinderbox replacement, so this would be wasted time
    • if a new test can be run in a manner very similar to an existing tinderbox feature, then it probably will be added
  • aus2 tests using pyFit/fitnesse
    • check-in pending from morgamic