L10n Talk:Testing
From MozillaWiki
... or
why we shouldn't test search
- Some search engines are coming with revenue generating keys. Pounding those entry points with bots might violate the terms of service, in fact, I'd be surprised if it wouldn't.
- We're testing a volatile implementation (in CVS) against a more volatile specification (wiki). The assumption that someone breaking one wouldn't fix the test by breaking the other isn't justified to me.
- The complexity of search makes a exhaustive testing framework exhaustive.
- The win for that cost is low.
Alternative proposal
How about making the current process
- review
- freeze
more complete and add a well-defined litmus test in between. We'd carry over that test for new plugins only, checking for things like:
- comes up
- seems to search for the words in question (try "mozilla", "firefox", "test")
- doesn't query a different server than the result is on, server is obviously belonging to the provider
- (if applicable) search suggest provides results, check for some inflammatory terms. Record response time and warn if it's near or above 500ms.