Fennec 4.0 RC 1 Test Results

The covered testing for this release will focus on smoketests, verifications and updates. For a list of new features coming in for Fennec 4.0, take a look at the Mobile Planning Page and for test coverage, look at our test plan.

Project contacts

PM: Stuart Parmenter (stuart)
Build: John O'duinn (oduinn)
QA: Aakash Desai (aakashd)

Schedule

Code Freeze: -- 3/16
Builds hand to QA: -- 3/17
QA test signoff: --

Build Location and IDs

Go to the candidates folder in our ftp folders
Go to the "4.0rc-candidates" folder
Download the candidate:
- For the n900, install the application catalog and update to the candidate build via your device App Manager.
- For Android, download and install the actual build over any other instance

Test Coverage

Smoketests

Lite verification tests (or smoketests) will be first run when the builds are ready. Tests are available on litmus under the Fennec 2.0 Test Suite. In any case of failure in any Smoketest, blockers need to be raised ASAP before proceeding to the BFTs.

Please record your detailed results in the "Detailed Test Results" section below.

BFTs for 2.0 Features

BFTs are a more comprehensive set of litmus tests that cover functional testing of features that have landed on Fennec 2.0. Tests are available on litmus under the Fennec 2.0 Test Suite.

Please record your detailed results in the "Detailed Test Results" section below.

OS Integration

The Operating System Integration testsuite is a set of tests that target how Fennec works with the platform (i.e. Android, Maemo, etc.) and features of the device itself (i.e. Orientation, hard keyboard, soft keyboard, etc.). Tests are available on litmus under the Fennec 2.0 Test Suite under the "OS Integration Tests" testgroup.

Please record your detailed results in the "Detailed Test Results" section below.

Fixed Bugs Verifications

The following link contains a list of bugs that are fixed for RC and need verifications:

2.0+ Fennec blockers at a resolved state, but un-verified

We are holding a Fennec RC Bug Bash Session in Zombocom on Thursday, 3/17, at 1:30PM-3:00PM. In the meeting, all bugs in the query above will be verified until we get to a 0 state or, at least, all bugs that can be verified.

Extras Build 2 required the verification of bugs:

bug 638164 - Checkerboarding after zooming in, visiting other page, then going back in history
bug 642291 - crash [@ nsBufferedInputStream::Write] demos.mozilla.org motovational poster

Build 3 required the verification of bugs:

bug 642205 - Do not build layers for asynchronous scrolling if element shouldn't be scrollable
bug 642755 - DTD comment compression breaks localizations, at least Japanese
bug 642718 - Upgrading from beta to RC build leaves feedback add-on enabled

L10n Checks

We don't have coverage of localized builds until the candidate builds are available, so we'll need to run through the tests within the Fennec 2.0 String Guide on the following locales:

German
Finnish
French

Top Add-ons Testing

To verify the entire browsing experience, QA will need to spotcheck for the following use cases across each add-on in the recommended add-ons list:

Installation through the Add-ons Manager or AMO
Spotcheck to make sure the UI is not entirely broken
Uninstall the add-on through the Add-ons Manager

Website & Device Stability Checks

The covered testing for this release candidate will include that of a test plan for website compatibility. Please read and use the website compatibility test plan, available here as a companion to a form that we would like you to complete, which is available here. Results are viewable here

Issues Found

bug 642839 - Font rendering issue on my.senecacollege.ca (Blackboard CMS)

Updates

Test the update path for the released builds only, not the nightlies. (eg. 1.1->4.0RC (maemo only) and 4.0b5->4.0)

Fennec 4.0 RC can be downloaded from the Mozilla Beta Catalog or the Android Marketplace.

Updates tests to perform for each build

Test Results

Types of Tests	Maemo	Android
Smoketests	Ioana - PASS	Anna - PASS
BFTs	Ioana - PASS	Anna - PASS
OS Integration	mw22 - PASS	nhirata - Done/PASS
L10n	ashah - Pass	nhirata - Done/PASS
Top Add-ons	aakashd - PASS	ashah - PASS
Fixed Bug Verifications	All	All
Website and Device Stability Checks	--	AaronMT - PASS
Updates	aakashd - PASS	--

Detailed Test Results

Smoketests Results

HTC Desire [Anna]

Build: Mozilla /5.0 (Android;Linux armv7l;rv:2.0b13pre) Gecko/20110317 Firefox/4.0b13pre Fennec /4.0
Results: PASS

Notes:
- No unknown issues were found
- Known bugs:
  - bug 628269 : Pages are zoomed in at reload
  - bug 639504 : Unable to zoom in/out a webpage on header/footer when a double tap gesture is made

n900 [Ioana]

Both RC 1 and RC 2
Build: Mozilla /5.0 (Maemo; Linux armv7l;rv:2.1) Gecko/20110317 Firefox/4.0b13pre Fennec /4.0
Results: PASS

Notes:
- No major issue found
- Known bugs:
  - bug 611294 - After sharing a page using Facebook, the browser returns to the shared page.
  - bug 639504 : Unable to zoom in/out a webpage on header/footer when a double tap gesture is made

BFT Results

HTC Desire [Anna]

Build: Mozilla /5.0 (Android;Linux armv7l;rv:2.1) Gecko/20110317 Firefox/4.0b13pre Fennec /4.0
Results: PASS

Notes: RC2
- Options (Preferences):
  - There is no feedback pannel in RC1 Build 2. Could this be related with bug 642718 ?
- General
  - When google mobile version webpage is loaded there arem't any suggestions offered. (Google suggestions are offered only in the classic version)
  - known bugs:
    - bug 628269: Pages are zoomed in at reload
    - bug 629645: Strange behavior when using geolocation pages.
    - bug 639829: Page position is not persistent on reload
- Downloading
  - known bugs:
    - bug 619054: Inconsistency at auto-renaming
- Tabbed Browsing
  - known bugs:
    - bug 639794: Opening a webpage from an external app should hide sidebars and panels
- Find in Page
  - reported bugs:
    - bug 642788: [Regression] Find in Page does not scroll down when hitting the Down Arrow from the findbar
- Security
  - known bugs:
    - bug 341604: (framesandbox) Implement HTML5 sandbox attribute for IFRAMEs
    - bug 443564: Implement window.toStaticHTML
    - bug 446344: Implement Origin header CSRF mitigation
    - bug 471020: Add X-Content-Type-Options: nosniff support to Firefox
    - bug 528661: Heuristics to block reflected XSS (like in IE8)
    - bug 607613: HttpOnly cookies flushed by non-HttpOnly cookies (useful for session fixation)
- Context Menu
  - known bugs:
    - bug 641845: Long tap on a video which has a "poster" attribute will show an image context menu instead of a video context menu
- System pages
  - known bugs:
    - bug 641836: [Regression]The search input should get the focus after loading about:config

n900 [Ioana Chiorean]

Build: Mozilla /5.0 (Maemo; Linux armv7l;rv:2.1) Gecko/20110317 Firefox/4.0b13pre Fennec /4.0
Results: IN PROGRESS

Notes: RC 2
- Options (Preferences):
  - There is no feedback panel in RC1 Build 2. bug 642718
- Downloading
  - known bugs:
    - bug 619054 - Inconsistency at auto-renaming
- Tabbed Browsing
  - known bugs:
    - bug 639794 - Opening a webpage from an external app should hide sidebars and panels
- Layout
  - known bugs:
    - bug 580920 - Acid 2 test fails in Fennec - reload /zoom
    - bug 614045 - Drop down menu items on http://www.bestbuymobile.com/ do not work - http://www.w3.org/MarkUp/Test/HTML401/current/tests/sec17_6_1-BF-01.html
- Security
  - known bugs:
    - bug 341604 - (framesandbox) Implement HTML5 sandbox attribute for IFRAMEs
    - bug 443564 - Implement window.toStaticHTML
    - bug 446344 - Implement Origin header CSRF mitigation
    - bug 471020 - Add X-Content-Type-Options: nosniff support to Firefox
    - bug 528661 - Heuristics to block reflected XSS (like in IE8)
    - bug 607613 - HttpOnly cookies flushed by non-HttpOnly cookies (useful for session fixation)
- Context Menu
  - known bugs:
    - bug 641845 - Long tap on a video which has a "poster" attribute will show an image context menu instead of a video context menu
- System pages
  - known bugs:
    - bug 641836 - [Regression]The search input should get the focus after loading about:config
- Screen Navigation
  - known bugs:
    - bug 639504 - Unable to zoom in/out a webpage on header/footer when a double tap gesture is made

- Found these bugs, while testing RC1: (from mw22)
- bug 642642 Exception... "'Find cluster denied: illegal-method' when calling method: [nsIObserver::observe] in resource://services-sync/ext/Observers.js:122
- bug 642635 Adding/removing a bookmark isn't reflected in the awesomebar results popup
- bug 642571 UI bustage after a while on Maemo (no steps to reproduce yet)

OS Integration Results

Nexus One [AaronMT]

Build:
- Mozilla/5.0 (Android; Linux armv7l; rv:2.1) Gecko/20110317 Firefox/4.0b13pre Fennec/4.0
Results: PASS

Notes:

Droid 2 [nhirata]

Build : RC1 Build 2:
- Mozilla/5.0 (Android; Linux armv71; rv:2.1) Gecko20110317 Firefox/4.0b13pre Fennec/4.0
Build : RC1 Build 3:
- Mozilla/5.0 (Android; Linux armv71; rv:2.1) Gecko20110318 Firefox/4.0b13pre Fennec/4.0
Device: Droid 2
OS: Android 2.2

Results: PASS
Notes: The following are reported bugs that are related to but not directly tested with the test cases:
- bug 642959 SCR: Please add the filename in the system download notification
- bug 642953 Arrows from the form assistant will move to the right after bringing up the tab panel
- bug 642943 Checkerboarding appears briefly when doing two finger gestures on a ftp site
- bug 642932 Occasionally, kxtv web page might not load
- bug 642919 Search box is located in the wrong location for cbsnews.com
- bug 642889 Form helper does not work well on Sync.in nor iEtherpad.com
- bug 642854 Webpage (www.mozilla.com) is not completely filled after showing the control panel sidebar in portrait mode
- Note: update is not working even though Beta 5 is suppose to update to RC 1. Is it only during the market release where it would update?

n900 [aakashd] [RC1 build 2 - mw22]

Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:2.1) Gecko/20110317 Firefox/4.0b13pre Fennec/4.0
Results: PASS

Notes:
- Still see bug 624691 - [Maemo-only] 'Open' button is disabled for downloaded files with .deb, .install and .pdf extension

RC1 build 2:

Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:2.1) Gecko/20110317 Firefox/4.0b13pre Fennec/4.0

Results: PASS

Notes:
- Major issue: bug 642571 - UI bustage after a while on Maemo (no steps to reproduce yet). This happens very occassionally, I've seen it 3 times now.
- bug 642750 - [Maemo] Tapping second time on url bar in landscape mode does not bring up virtual keyboard
- bug 642771 - [Maemo] Dismissing the virtual keyboard causes the top bar with "All pages", "Bookmarks" and "History" to disappear, part 2
- I'm also seeing bug 624691 - [Maemo-only] 'Open' button is disabled for downloaded files with .deb, .install and .pdf extension

L10n Results

Motorola Droid 2 [nhirata]

Build:
- Mozilla/5.0 (Android; Linux armv71; rv:2.1) Gecko20110318 Firefox/4.0b13pre Fennec/4.0
- OS: Android 2.2
Results: PASS

Notes:
- bug 643049 The word "Loading" is not translated into Finnish
- bug 643044 Going to about:home after having a fennec notification open will cause a white space at the bottom.
- bug 643033 Splash screen is still in english when the device OS is set to Spanish and the Language for Fennec is set to Autodetect

n900 [ashah]

Build: Mozilla/5.0 (Maemo; Linux arm7vl; rv:2.1) Gecko/20110318 Firefox/4.0b13pre Fennec/4.0
Results: Pass

Notes:
- bug 643370 - The addon description is not translated in l10n
- bug 643442 - The word 'Desktop' is not translated into german
- bug 643445 - The words 'Interger', 'Boolean' and 'String' are not translated into german
- bug 643446 - Buttons not aligned on the about:home page of french locale

Top Add-ons Results

Nexus One [ashah]

Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:2.1) Gecko/20110317 Firefox/4.0b13pre Fennec/4.0
Results: Pass

Notes: RC1 build 2
- Found bug - bug 642878 After removing the persona addon, the installed persona does not go away from the addons manager

n900 [aakashd]

Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:2.1) Gecko/20110317 Firefox/4.0b13pre Fennec/4.0
Results: PASS

Notes:
- Installed and Uninstalled the following addons: Phony, Readability, Personas, Bigger Text, Easy Reading, Quit Firefox for Mobile, Reading List, URL Fixer, Home Links, HomeSkin
- Found an error exception with HomeSkin, notified fabrice

Updates Results

n900 [aakashd]

Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:2.1) Gecko/20110318 Firefox/4.0b13pre Fennec/4.0 ID:20110318105707
Results: PASS

Notes:
- Saw bug 629303 - Profile gets corrupted after upgrading from 1.1 to beta4

QA/Fennec/Milestones/2.0/ReleaseTest/RC1

Contents

Fennec 4.0 RC 1 Test Results

Project contacts

Schedule

Build Location and IDs

Test Coverage

Smoketests

BFTs for 2.0 Features

OS Integration

Fixed Bugs Verifications

L10n Checks

Top Add-ons Testing

Website & Device Stability Checks

Updates

Test Results

Detailed Test Results

Smoketests Results

HTC Desire [Anna]

n900 [Ioana]

BFT Results

HTC Desire [Anna]

n900 [Ioana Chiorean]

OS Integration Results

Nexus One [AaronMT]

Droid 2 [nhirata]

n900 [aakashd] [RC1 build 2 - mw22]

L10n Results

Motorola Droid 2 [nhirata]

n900 [ashah]

Top Add-ons Results

Nexus One [ashah]

n900 [aakashd]

Updates Results

n900 [aakashd]

Navigation menu

Search