QA/Fennec/Milestones/2.0/ReleaseTest/Beta5

From MozillaWiki
< QA‎ | Fennec‎ | Milestones‎ | 2.0
Jump to: navigation, search

Fennec 4.0 Beta 5 Test Results

The covered testing for this release will focus on smoketests, verifications and updates. For a list of new features coming in for Fennec 4.0, take a look at the Mobile Planning Page and for test coverage, look at our test plan.

Project contacts

PM: Stuart Parmenter (stuart)
Build: John O'duinn (oduinn)
QA: Aakash Desai (aakashd)

Schedule

Code Freeze
-- 2/16
Builds hand to QA
-- 2/17
QA test signoff
-- 2/18

Build Location and IDs

  1. Go to the candidates folder in our ftp folders
  2. Go to the "4.0b5-candidates" folder
  3. Download the candidate:
    • For the n900, install the application catalog and update to the candidate build via your device App Manager.
    • For Android, download and install the actual build over any other instance

Test Coverage

Smoketests

Lite verification tests (or smoketests) will be first run when the builds are ready. Tests are available on litmus under the Fennec 2.0 Test Suite. In any case of failure in any Smoketest, blockers need to be raised ASAP before proceeding to the BFTs.

Please record your detailed results in the "Detailed Test Results" section below.

BFTs for 2.0 Features

BFTs are a more comprehensive set of litmus tests that cover functional testing of features that have landed on Fennec 2.0. Tests are available on litmus under the Fennec 2.0 Test Suite.

Please record your detailed results in the "Detailed Test Results" section below.

Here are the BFT's that need to be run:

  • Panning & Zooming (Screen Navigation)
  • Form Assistant
  • Sync

OS Integration

The Operating System Integration testsuite is a set of tests that target how Fennec works with the platform (i.e. Android, Maemo, etc.) and features of the device itself (i.e. Orientation, hard keyboard, soft keyboard, etc.). Tests are available on litmus under the Fennec 2.0 Test Suite under the "OS Integration Tests" testgroup.

Please record your detailed results in the "Detailed Test Results" section below.

Fixed Bugs Verifications

The following link contains a list of bugs that are fixed for beta 5 and need verifications:

L10n Checks

We don't have coverage of localized builds until the candidate builds are available, so we'll need to run through the tests within the Fennec 2.0 String Guide on the following locales:

  • German
  • Finnish
  • French

Updates

Test the update path for the released builds only, not the nightlies. (eg. 1.1->4.0b5 (maemo only) and 4.0b4->4.0b5)

Fennec 2.0 Beta 5 can be downloaded from the Mozilla Beta Catalog or the Android Marketplace.

Updates tests to perform for each build

Test Results

Types of Tests Maemo Android
Smoketests
  • Ioana - Done
  • kbrosnan - Done
BFTs
  • ashah - Done
  • kbrosnan - Done
OS Integration
  • aakashd - Done
  • nhirata - Done
L10n
  • Ioana - Done
  • nhirata - Done
Updates
  • Ioana- Done
  • AaronMT - Done

Detailed Test Results

Smoketests Results

G2 [kbrosnan]

  • Build: Mozilla/5.0 (Android; Linux armv7l; rv:2.0b12pre) Gecko/20110217 Firefox/4.0b12pre Fennec/4.0b5 ID:20110217140304
  • Results: PASS
  • Notes:
    • Reported bugs:
      • bug 635131 - Preview of closed tab not updated when undoing a closed tab w/ keyboard
      • bug 635133 - Can only restore one tab via the undo close tab

n900 [Ioana]

[build 2 and 3]

  • Build 2: Mozilla/5.0 (Maemo;Linux armv7l;rv:2.0b12pre)Gecko/20110216 Firefox/4.0b12pre Fennec /4.0b5pre
  • Build 3: Mozilla/5.0 (Maemo;Linux armv7l;rv:2.0b12pre)Gecko/20110217 Firefox/4.0b12pre Fennec /4.0b5pre
  • Results: Pass
  • Notes:
    • Known bugs:
      • bug 611294 - After sharing a page using Facebook, the browser returns to the shared page
    • New issue:
      • bug 634883 - Inconsistency at Share Link for 'About:' / System pages
    • Unreproducible issue:
      • The Return Arrow from preference sign remains over the closing X of the application

BFT Results

G2 [kbrosnan]

  • Build: Mozilla/5.0 (Android; Linux armv7l; rv:2.0b12pre) Gecko/20110217 Firefox/4.0b12pre Fennec/4.0b5 ID:20110217140304
  • Results: Pass
  • Notes:
    • known bugs:
      • bug 610532 - Truncated URLs in awesomescreen due to tags
    • reported bugs:
      • bug 635289 - menu click on link should open in a new tab
      • bug 635358 - edit site popup preferences not showing in site options after clear private data
      • bug 635375 - Preferences > add-ons > get add-ons 'go to page' disabled
      • bug 635513 - apple.com header not rendered correctly
      • bug 635522 - [rtl] tab close buttons in the wrong spot

n900 [ashah]

  • Build: Mozilla/5.0 (Maemo; Linux; armv7l; rv:2.0b12pre) Gecko/20110217 Firefox/4.0b12pre Fennec 4.0b5
  • Results: done
  • Notes:
  • [ashah on build 3]
    • No new bugs found.
    • I experienced same things as Ioana said below.
  • [Ioana on build 2]
    • Form assistant
      • There is no more search in form fill - talked to Vivien and I understood is intended.
      • Known bugs:
        • bug 606466- Unnecessary ">" button in form autocomplete suggestions list
    • Sync
      • New Bug:
        • bug 634904 - "Wrong sync key" error message when trying to connect to a sync account using a 26 digit Sync key on Firefox 3.6.12
    • Panning&Zooming
      • Browser checkerboards everytime one pans up/down, zooms out. This gives a poor user experience.
      • Known bugs - checkerboarding problems still present:
        • bug 607777 - Make a good metric for checkerboarding
        • bug 629336 - Bad checkerboarding in Advanced search form on bugzilla.mozilla.org

OS Integration Results

Motorola Droid 2 [nhirata]

  • Build: Beta 5 candidate 3

Mozilla/5.0 (Android; Linux armv71; rv2.0b12pre) Gecko/20110217 Firefox/4.0b12pre Fennec/4.0b5 Device: Droid 2 OS: Android 2.2

  • Results: Pass with concerns : bug 617297, bug 634555, bug 635297
  • Notes:
    • Bugs encountered:
      • bug 611730 - tap + long tap ime over context menu
      • bug 635297 - No update notification from Beta 4 to Beta 5 on Android
      • bug 635292 - Error: strings is null in CrashSubmit.jsm Line:180 when crashing with bug 617297
      • bug 617297 - crash after trying to download an item by just clicking on it
      • bug 634555 - sometimes wrong order when typing on hkb ; occurs on google login text field
    • Other notable unreproducible issues
      • keyboard size with undo landscape to portrait - can't repro
      • closing 3 popups will cause the side panel to open?
      • sometimes adding an addon will not restart the app.

n900 [aakashd]

  • Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:2.0b12pre) Gecko/20110217 Firefox/4.0b12pre Fennec/4.0b5 ID:20110217130947
  • Results: PASS
  • Notes:
    • found bug 624691 - 'Open' button is disabled for downloaded files with .deb, .install and .pdf extension
    • found bug 535793 - Connect in gtk port on n900 works, but you get a page load error directly after connection setup and need to try again

L10n Results

Motorola Droid 2 [nhirata]

  • Build: Beta5 Candidate 3

Mozilla/5.0 (Android; Linux armv71; rv2.0b12pre) Gecko/20110217 Firefox/4.0b12pre Fennec/4.0b5 Device: Droid 2 OS: Android 2.2

  • Results: Fail - blocker 635361
  • Notes:
    • Bugs Encountered
      • bug 635342 - The word "Loading" on startup is not translated into any languages
      • bug 635347 - about:fennec has english in it at the license info section
      • bug 635349 - "Edit Bookmark" is still in English
      • bug 635353 - Get help is in English on the about:home page
      • bug 635361 - Uncaught exception when trying to download a file with the browser set to a non English language (edit) [cannot download]
      • bug 635366 - integer, boolean, string are not translated in the about:config page
    • Bugs not reproduced
      • change of language restart does not restart the app
      • occassionally the context will not resize correctly

n900 [Ioana]

[ build 2]

  • Build: Mozilla/5.0 (Maemo;Linux armv7l;rv:2.0b12pre)Gecko/20110216 Firefox/4.0b12pre Fennec /4.0b5pre
  • Results: -
  • Notes:
    • No single locale builds: bug 630475 - Maemo single locale deb packaging broken

[build 3]

  • Build: Mozilla/5.0 (Maemo;Linux armv7l;rv:2.0b12pre)Gecko/20110217 Firefox/4.0b12pre Fennec /4.0b5pre
  • Results: -
  • Notes:
    • No single locale builds: bug 630475 - Maemo single locale deb packaging broken
    • Localization on multi language build:
      • French :
        • Get help in About:home is not translated.
      • Finnish :
        • Get help in About:home is not translated.

Updates Results

Nexus One [AaronMT]

  • Build: Mozilla/5.0 (Android; Linux armv7l; rv:2.0b12pre) Gecko/20110217 Firefox/4.0b12pre Fennec/4.0b5
  • Results: Pass
  • Notes:
    • None

n900 [Ioana]

  • Build: Mozilla/5.0 (Maemo;Linux armv7l;rv:2.0b12pre)Gecko/20110217

Firefox/4.0b12pre Fennec /4.0b5pre

  • Results: FailPass
  • Notes:
    • bug 635238 - Updated app from 4.0b4 to 4.0b5 fails to open

n900 [mw22] (on 2011-02-23)

  • Build: Mozilla/5.0 (Maemo;Linux armv7l;rv:2.0b12pre)Gecko/20110217

Firefox/4.0b12pre Fennec /4.0b5pre

  • Results: Pass
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Fennec/Milestones/2.0/ReleaseTest/Beta5&oldid=289340"