QA/Fennec1.0/ReleaseTest/1.0Maemo RC1

From MozillaWiki
< QA
Jump to: navigation, search

Fennec 1.0 Maemo RC1 Test Results

The covered testing for this release will focus on bfts, l10n, top plugins, top financial sites, updates, and spotchecks.

Project contacts

PM: Stuart Parmenter (stuart)
Build: John O'duinn (oduinn)
QA: Aakash Desai (aakashd)


Volunteer Testers

  • N810 - Armen Zambrano <armenzg@mozilla.com> - I will be on holidays so I am not sure which day/time I will be able to help out
  • N900, N810 - Stas Malolepszy <stas@mozilla.com>
  • N900 - Asa Dotzler <asa@mozilla.com> - I'm available Monday to help out.
  • N900 - Joe Drew <joe@drew.ca>
  • N900, N810 - Vlad <vladimir@mozilla.com>
  • N900, bsmedberg <benjamin@smedbergs.us> - I'm on holidays a lot, but send email and I may be able to help out

Schedule

Code Freeze
-- 12/28
Builds hand to QA
-- 12/30
QA test signoff
-- 12/31
  • See Test Coverage below

Build Location and IDs

Download:

Builds:

Test Coverage

  • Areas Covered:
    • Smoketests - PASS
    • BFTs - PASS
    • Bug Fixes - PASS
    • Website Spotchecks
    • L10n Spotchecks - PASS

Test Results

Types of Tests n900 n810
Smoketests tchung - PASS jmaher - PASS
BFTs aakashd/marcia - PASS/PASS jmaher - PASS
L10n Build n900 n810
Auto-detect abillings - PASS jmaher - PASS
cs abillings - PASS jmaher - PASS
da abillings - PASS jmaher - PASS
es-ES abillings - PASS jmaher - PASS
fi abillings - PASS jmaher - PASS
fr abillings - PASS jmaher - PASS
de abillings - PASS jmaher - PASS
it abillings - PASS jmaher - PASS
nb-NO abillings - PASS jmaher - PASS
nl abillings - PASS jmaher - PASS
pl abillings - PASS jmaher - PASS
pt-PT abillings - PASS jmaher - PASS
ru abillings - PASS jmaher - PASS

BFT Testing

BFTs are a more comprehensive set of litmus tests that cover the build testing after smoketests result. Tests are available on litmus under the Fennec 1.0 test suite.

FFTs are a full comprehensive set of litmus tests that extend coverage in areas that BFTs dont often get to. These are run at least once during the major milestone timeline.

If you are running a BFT, Please record your detailed results in the section below.

Fixed Bugs Verifications

The following link constain a list of bugs that are fixed for beta 5 and need verifications:

https://wiki.mozilla.org/Mobile/Fennec/Fennec1.0/Maemo-RC

L10n Spotchecks

Run the Fennec 1.0 Localizer String Guide over each locale and device that is associated with the release. Verify each test case results in a UI element that has strings localized to the correct language.

Detailed Results

Smoketests Results

n900 [tchung]

  • Date: 12/30/2009
  • Build: Mozilla/5.0 (X11; U; Linux armv71; en-US; rv:1.9.2b6pre) Gecko/20091230 Firefox/3.6b6pre Fennec/1.0b6pre
  • Results: PASS
  • Notes:
    • See litmus for results, buildID "20091230"
    • Failed litmus testcase 7245 Need a clean profile to re-verify.
      • update joel confirmed the testcase is wrong and needs an update.
    • Failed litmus testcase 7908 Need another n900 to verify.
      • update joel confirms youtube video playback for him. could be my profile.
    • Filed BLOCKER bug 537341 - Change fennec 1.0 rc to firefox branding
    • Filed bug 537336 - Fennec uninstaller doesnt uninstall when app is still open

n810 [jmaher]

  • Build: Mozilla/5.0 (X11; U; Linux armv6l; en-US; rv:1.9.2b6pre) Gecko/20091230 Firefox/3.6b6pre Fennec/1.0b6
  • Results: PASS
  • Notes:
** See notes from BFT

BFT Results

n900 [aakashd] Installation - Language Selector

  • Build: Mozilla/5.0 (X11; U; Linux armv7l; Nokia N900; en-US; rv:1.9.2b3pre) Gecko/20091228 Namoroka/3.6b6pre Fennec/1.0b6
  • Results: PASS
  • Notes:
    • Lots of slowness using the awesomebar and panning pagse with flash enabled after browsing for about 10-15 minutes

n900 [marcia] Layout - Screen Navigation

  • Litmus results under Build ID: 20091229
  • Build: 20091228010629
    • Did Layout
  • Build: 1.9.2b6pre-20091230135310 (Release Candidate)
  • Results: PASS Please review Litmus results for failures.
  • Notes:

n900 [tchung]

  • Covering: general, content handling, addons manager, downloading, password manager, menu bar, installation, Search, tile caching, layout
  • Build: Mozilla/5.0 (X11; U; Linux armv71; en-US; rv:1.9.2b6pre) Gecko/20091229 Firefox 3.6b6pre Fennec/1.0b6pre
  • Results: PASS
  • Notes:
    • Filed bug 537109 - Flash video content on dailymotion.com wont play (
    • Filed bug 537127 - Flash ads can overlay the security warning notifications
    • Filed bug 537183 - About Us page is unbookmarked, but still starred
    • Filed bug 537185 - Fennec favicons are pixelated
    • I crashed disabling plugins in the preferences. Need to find a repro.
    • Searching for addons returns nothing. even the ones that are listed as recommended
    • restarting the added addon takes about 30 seconds to restart. (bug 534982
    • For some reason, my "allow cookies" keeps disabling by itself. Investigating the cause.
      • update turns out i was double tapping the options button, which directly overlays the cookie toggle switch. We should probably redesign this as it can be a common issue

n810 [jmaher] Installation - Language Selector

  • Build: Mozilla/5.0 (X11; U; Linux armv7l; Nokia N900; en-US; rv:1.9.2b3pre) Gecko/20091229 Namoroka/3.6b6pre Fennec/1.0b6
  • Results: PASS
  • Notes:
 * bug 537095 - n810: form manager does not launch when browsing to gmail
 * bug 537099 - n810: browser crashes while loading yahoo.com if plugins are enabled (flash)
 * bug 537138 - n810: geolocation pages do not display any data

n810 [jmaher] Layout - Screen Navigation

  • Build: Mozilla/5.0 (X11; U; Linux armv7l; Nokia N900; en-US; rv:1.9.2b3pre) Gecko/20091229 Namoroka/3.6b6pre Fennec/1.0b6
  • Results: PASS
  • Notes:

Websites Testing

A list of tough websites with the n810 and no-swap: http://bit.ly/5ZDSRF

Please build up a list of hard to use (ui interaction, or too heavyweight and slow) so we can add to future testplans.

n900 [timr]

  • Build:
  • Results: result
  • Notes:
    • Testing sites from troublesome sites for the n810 list, along with Alexa top 50, and a bunch of login sites (bank, memberships, etc)
    • Tested troublesome sites:
      • boingboing.net (including video)
      • fan590.com - slow loading and scrolling
      • csnation.net - (including video) - loads super slow but has lots of graphics
      • espn.com - "Herbstreit On Leach" video takes _forever_ to load. Like 8 minutes. Then very stuttery and only bottom half of video is showing.
      • jquerystyle.com - ok
      • maxim.com - very slow site
      • Fandango - trouble playing videos - "This video player has encountered a problem please refresh the page". Refresh doesn't help. Fandango videos work on microbee. Filed bug 537397.
    • Top Alexa sites
      1. google.com - ok
      2. facebook.com - ok
      3. yahoo.com - very slow site, particularly videos. 1 minute vid took 20 minutes+ to play. Lost sound after the first 5 minutes. Browser hung when trying to get back to URL bar and type in next top site. Came back after about 15 mins. This happened when watching the Young Raccoons Saved from Pepsi Machine video.
      4. youtube.com - crashed about about 50sec loading first page. Worked great the secod time. I think the browser was still hosed from yahoo.com. Yeah. youtube seems fine. One of the better responding sites for fennec.
      5. live.com - good
      6. wikipedia.org - English site ok. pictures are good. Japanese site - no pictures!?
      7. blogger.com - good, responsive.
      8. baidu.com - simple site. Like google.com. responsive.
      9. msn.com - good. responsive.
      10. yahoo.co.jp - good.
    • login sites
      • Bank - one bank login, acct review, on-line billpay was successful. One time I miss-typed a password I got a crash. Not reproducible.
      • MLB.com - ok. _Very_ slow to update.
      • Apple Developer Connection member site
        • Scrolling Terms and Conditions box doesn't work. Not sure how I am going to repro this as it only comes up if you haven't reviewed this in the last 2 years.
        • Otherwise ok
      • Amazon.com member site - ok

L10n Testing

n900 [abillings]

  • Build: Mozilla/5.0 (X11; U; Linux armv7l; Nokia N900; en-US; rv:1.9.2b3pre) Gecko/20091229 Namoroka/3.6b6pre Fennec/1.0b6
  • Results: PASS
  • Notes:

n810 [jmaher]

  • Build: Mozilla/5.0 (X11; U; Linux armv7l; Nokia N900; en-US; rv:1.9.2b3pre) Gecko/20091229 Namoroka/3.6b6pre Fennec/1.0b6
  • Results: PASS
  • Notes:
* bug 537270 - n810 (de): browsing to ww.mozilla.org, yields a possible xml error page 
* bug 537292 -  n810 (fi): amazon.com is en-us version, not fi

Desktop Testing

Mac OSX [tchung]

  • Build: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2b6pre) Gecko/20091231 Namoroka/3.6b6pre Fennec/1.0b6pre
  • Results: PASS
  • Notes:
    • Flash audio streams normal, but video is super laggy. (still images, instead of video).
      • update this is bug 331462. flash doesnt render on canvas.
    • Fennec crashes after playing video for awhile, and then toggling the plugins option.
    • Tested websites, bookmarks, addons, enable/disable prefs, downloads, layout
  • Build: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2b6pre) Gecko/20091231 Namoroka/3.6b6pre Fennec/1.0rc1
  • Results: PASS
  • Notes:
    • Ran the same tests as above

Linux [jmaher]

  • Build: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2b6pre) Gecko/20091231 Namoroka/3.6b6pre Fennec/1.0b6pre
  • Results: PASS
  • Notes:


Windows (xp) [jmaher]

  • Build: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2b6pre) Gecko/20091231 Namoroka/3.6b6pre Fennec/1.0b6pre
  • Results: PASS
  • Notes:
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Fennec1.0/ReleaseTest/1.0Maemo_RC1&oldid=194415"