QA/Execution/Web Testing/Project Checklist

From MozillaWiki
Jump to: navigation, search
Purpose of this document

This is meant to be a quite-literal checklist for determing "go/no-go" status for your website. Where it doesn't apply, feel free to use it rather as a discussion point, but be sure all points are raised and noted.

Other Wiki Resources

Browser compatibility/coding standards

  • Get the list of supported browsers somewhere in writing (srsly)
  • All supported browsers look "pretty good" and are 100% functional
    • If not, exceptions are noted and called out prior to shipping
  • Team has gone through https://wiki.mozilla.org/WebDev:FrontendCodeStandards

Video Support

  • What are the supported video formats?
    • Appropriate fallback in place and working?
      • No Flash
      • Firefox 3.6+
      • IE 7/8/9
      • Opera (latest)
      • Chrome (latest)
      • Safari (latest)
      • Latest Firefox beta

General

  • If this is an outsourced project, a test plan and the test cases that the consultants used to validate their work should be included in the hand-off process.
  • In URL paths, / and "" (without the trailing slash) both resolve to the same URI/resource
  • HTML validates: http://html5.validator.nu/ or http://validator.w3.org/
    • Exceptions can be made for social-network/sharing sites, that may have malformed tags or markup
  • Is the robots.txt file configured correctly to prevent bots from spidering to nonsensical pages? (example)

RSS Feeds

  • Validate: http://feedvalidator.org/
  • Links all work
  • l10n: other locales look fine (special chars, Unicode, etc.)
  • Correct timestamps

Logged-in vs. logged-out functionality

  • Ask/test what the difference between the logged-in vs. logged-out functionality is
    • It's probably quite a bit more than you expected

Social media

  • Facebook
  • Twitter
    • Enable "HTTPS Only | Always use HTTPS" (https://twitter.com/settings/account)
    • Length of pre-populated tweets in en-US and other locales (l10n)
    • Unescaped vs. escaped characters (string literals)
    • Unicode chars
  • ShareThis, etc.
  • QR Codes
    • Try different readers (AT&T Scanner, QuickMark, QR Droid, etc.)

Performance/load testing

  • Doesn't tank on YSlow! (Strive for an A with ruleset v2)
    • CDN is hooked up, if needed (static images, CSS, JS, videos) -- see "Site Config" section, too, below
  • Load-tested, if needed? Make sure to bring this up

Accessibility

  • Is the site generally accessible?

JavaScript-disabled

  • Does the site support JavaScript disabled? If so, where?
    • What's the user messaging?

Security

Project/release-cycle

  1. Has a Webdev bug, filed from https://intranet.mozilla.org/webtools/authenticate/login
  2. Has a test plan (which can augment this, but lists specific bugs, milestones, project owners, etc.)
    1. Reviewed by team
  3. Has clear bug-filing/tracking, either in a new Bugzilla component, existing (with whiteboard, etc.), or in Basecamp
    1. If in Bugzilla, has clear milestones
  4. Listed on https://wiki.mozilla.org/QA/Execution/Web_Testing#Current_Projects
  5. Regular status meetings
  6. IRC channel?
  7. If needed, a tracking bug with dependencies set for other groups' work (IT, Webdev, Marketing, etc.)
  8. Make sure to schedule a brief meeting where all parties (Marketing/Webdev/QA) are present and discuss the open issues, and any last-minute (but necessary) changes
  9. Ask for PRDs, user-flow diagrams, testing notes
  10. Are the third-party developers cc:d on all relevant bugs? If possible (i.e. not "mozilla-confidential" flagged), have them follow the Bugzilla component (and make sure they have accounts).
  11. Has a push bug, and a release checklist, even if you don't think they need it (they do)

Optimal Bugzilla Workflow

Each project can be different, but please use this flow if starting a new project:

  • A bug is filed in the correct product/component
  • Bug is triaged for severity, priority and assigned to a target milestone
  • Bug is fixed in work based on next target milestone release
    • Dev puts bug# in git comment
    • Dev puts commit url in bugzilla comment
  • Dev marks bug RESOLVED:FIXED
  • Tester verifies bug on stage, bug marked RESOLVED:VERIFIED
  • Code is deployed to production
  • Tester verifies bug
  • Tester verifies deployment/push/server-ops bug

Devs may fix bugs before triage, they should update Target Milestone to the next release.

Whiteboard may be used for various reasons:

  • [good first bug] - Communicates a contributor opportunity

Site Config

  • Make sure you get Django-traceback emails, on both prod and trunk/staging
  • Any vanity domains? Registered/set up, and working?
  • Has a favicon? That works in IE, too?
  • reCaptcha APIs on staging?
  • Google Maps API key on staging?
  • Metrics code is correct/working for the site (typically Webtrends)
    • Ask Blake Cutler or Laura Forrest (preferably Laura)
  • Any rewrite/redirect rules are in place and working?
  • Is the site planned to be hooked up to a CDN? Is it set up for testing on staging, and without any cert-signing problems?

Tools

  • You're using as many of the tools as appropriate
    • Suggested:
      • PowerFuzzer
      • NetSparker
      • Xenu
      • XSS Me

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Execution/Web_Testing/Project_Checklist&oldid=400915"