B2G/QA/Meetings/2015-10-07

From MozillaWiki
< B2G‎ | QA‎ | Meetings
Jump to: navigation, search

Overview

Announcements:

  • List your ideas for Topics for Moz Orlando work week : qa@mozlando

https://etherpad.mozilla.org/mozorlando-qa-topics

    • same idea as what we did for work week for Whistler

Off plan

Gerry - Oct. 7~23, Oct. 30~Nov. 6 (Euro Star) Paul - Oct. 14-19 Al - Oct. 16-21 TPE office - Oct 9

2.2r

  • Contact Window: Shing Lyu
    • Status: Shing and Al starts to draft SOW for outsource QA

// Following items are for read-only

2.5 Feature

  • Bugzilla lite: Cynthia/Gerry
    • Marigold will run the regression test next week
    • Bug list: http://mzl.la/1SmOoEQ (50 bugs) - none of them are blockers
      • Done
  • Telemetry: Iris/Gerry
    • Test plan draft: https://goo.gl/6ANJuO, modified test plan based on developer's review comments
    • Exploratory test for Add-on metrics/Developer HUD/API
    • Bug list: https://goo.gl/GXRnIH
    • Verified bugs:
      • bug 1171685 - FTU Ping hash, code landed and have verified
      • bug 1181295 - FTE UX changes for privacy, code not landed, have played local patches
      • bug 1208205 - Settings changes for privacy, code not landed, have played local patches
      • bug 1177099 - compression, code not landed, have played local patches
  • Pin the Web: Alison
    • Keep exploratory test for new landing feature
    • Bug list: http://mzl.la/1hVD8yw (19 bugs, include 7 blockers)
  • Private Browsing: Alison
    • Only exist one minor feature need to be landed: remove button, other depends bugs on meta bug all were landed.
    • Execute the full run test for this feature
    • Bug list: http://mzl.la/1JRJ0iJ (4 bugs)


   *** Waiting for review: https://bugzilla.mozilla.org/show_bug.cgi?id=1207097

   *** Upload site: https://marketplace-dev.allizom.org/content/addon/submit/
   *** Upload is successfully
   *** Cannot find uploaded add-on from staging server


  • Performance: Mike
    • App/Device startup time
   *** Serious regression from Sep. 15th: https://bugzilla.mozilla.org/show_bug.cgi?id=1204837
       **** RESOLVED FIXED
   *** Music NGA causes performance regression: https://bugzilla.mozilla.org/show_bug.cgi?id=1210440
   *** Raptor unable to upload to database from Taipei for 3 days, wait for Eli’s investigation
    • Media performance issue is fixed but verification failed
   *** https://bugzilla.mozilla.org/show_bug.cgi?id=1159343

   *** Test NSec packages will generate from current gaia apps
    • Move landing to S9


_* NGA: Gerry/Johan

    • Calendar - Wait for review+ for below 2 bugs
      • [jlorenzo] It was initially planned for the end of September, but the major stakeholder left last week. Gareth Aye was supposed to finish the PR (by adding tests), but he got sidetracked by MarionetteJS. It wasn’t in the go/no-go meeting last week. I think it’s safe to not commit on this feature by the end of November.
    • [jlorenzo] The other apps are working on increasing the performance, now that the gecko regression is gone


Riding the train features:

  • Data Sync: Gerry/Iris [read-only for v2.5]
    • Marigold will arrange resources to do exploratory testing next week. (9/29)
  • RTL: William
    • Phoxygen has delivered the first 2.5 RTL patch (System)
   - https://bugzilla.mozilla.org/show_bug.cgi?id=1208170
    • 63% 2.5 RTL test cases have been updated.
    • Phoxygen delivered three packages. 1) System 2) FTU 3) Home screen


  • updates: FOTA : nhirata
    • Aries FOTA
      • IMEI whitelisting part has been done, but there needs to be some changes done to balrog data
      • we also need to figure out the location of the update.zip file
      • side loading FOTA has been tested; the update.zip file has been released.
      • build making works
      • need to test out mar file
    • Flame FOTA
      • Flame FOTA build will be pushed later this evening and could potentially be tested on nightly-test channel
      • otherwise it appears that we may need to do a separate testing of the gecko/gaia FOTA
      • need to test out mar file.
  • Vaani : nhirata - still haven't tested yet due to Dogfood issues with Aries
    • dropped for 2.5; work still being done, QA still needed. Will be an ongoing tweak of fuzzy logic
    • please use the MozSpeech app in dogfood-latest build
    • some bugs occurred and regressions on how it works; filed a bug, need to follow up with information in it.
  • Others
    • PTS testing: Teri
      • V2.5 FL run on Aries DONE and all bugs are resolved
      • [BlueDroid][PTS][Meta] Bluetooth PTS test (v2.5) https://bugzil.la/1206635
    • View Source: No-Jun
      • All major features are verified, there is only one bug that's being checked right now, but it's not a blocker (Bug 1204011)
      • Test suite is in moztrap


2.5 Spark/Aries-KK Smoketests

  • Tester: QAnalysts (KevinT)
    • Health Status: Aries: Yellow, 1 blocker

New: None

Existing: - https://bugzil.la/1211976 - Unable to connect to a secure WiFi network in Settings, unselectable 'OK' button.

  • This has been resolved as fixed. Should be fixed on a later build.

--> also https://bugzilla.mozilla.org/show_bug.cgi?id=1211883 

 Severe Issues:

- https://bugzil.la/1183033 [B2G] Keyboard doesn't have click sound - https://bugzil.la/1203603 [Cost Control][Aries] Data Limit and Usage Graph not displaying accurate results 

 ** This issue only occurs after OTA from RC4 and the user has turned on wi-fi and data in the RC4 FTU

- https://bugzil.la/1204617 [Aries][Bluetooth] does not sends files from Mobile device to Computer. 

 ** This issue only occurs after OTA from RC4.

- https://bugzil.la/1196177 Launch Camera from Gallery, but Camera can not work if you repeatly tap Home key to exit while Camera is launching. 

- https://bugzil.la/1210190 - Alarm time created is displayed as a different time

2.5 Flame Smoketests

  • Tester: QAnalysts (KevinT)
    • Health Status: Aries: Red, 2 blockers

New: None

Existing: - https://bugzil.la/1211976 Unable to connect to a secure WiFi network in Settings, unselectable 'OK' button 

 *Fails [6075]
 ** This issue is resolved as fixed, we will verify this issue in the next build.

- https://bugzil.la/1199863 - [System][OTA] OTA fails during uncompression, appears to run out of space in /system folder 

* Fails [2313]

Severe Issues: - https://bugzil.la/1183033 [B2G] Keyboard doesn't have click sound - https://bugzil.la/1196177 Launch Camera from Gallery, but Camera can not work if you repeatedly tap Home key to exit while Camera is launching. - https://bugzil.la/1210190 Alarm time created is displayed as a different time.

2.5 Aries FOTA Results

  • Health Status: Aries: Green 0 blockers

Tested FOTA today and the report was sent out. 69/69 tests passed

  • Sanity, Smoke and the Dogfood suite was ran. Added back the 14 OTA cases to bring the Dogfood suite up to 17 cases. All data from the 8/12 build was retained after sideloading the update except for user set volume levels which was a known issue on the 8/12 build.

2.5 Exploratory

=== 2.5 fullrun === (Update: 10/07)

  • The third 2.5 full run will be kick-off on Oct. 8th

=== 2.5 smoketest (Marigold) === (Update: 10/07)

  • QAnalyst is helping with it because Marigold is OOO.


2.5 MTBF

flame-kk 512MB avg. running hours: 34.26 Blocker: 

- https://bugzilla.mozilla.org/show_bug.cgi?id=1206634
- https://bugzilla.mozilla.org/show_bug.cgi?id=1198950

mtbf bugs in Q3 https://bugzilla.mozilla.org/show_bug.cgi?id=1182954

    • RA exit criteria: 100 hrs
    • CC exit criteria: 200 hrs

Previous action items

(Update: Oct. 7th)

  • [In progress] marcia: follow up with pallavi regarding community budget.
    • Some discussion of this issue will happen at the Work Week
  • [In progress][whsu] William to confirm ETA for RTL test case ready
 [William 10/07] It will be 10/16 or earlier. 
 GIP for OGA music still works; follow up for NGA music with GIP in Q4
  • [Jamie_/Marcia] to host a meeting about community engagement

== Agenda == (update October 7))

  • FxOS QA Team Work Week - March of Next year [marcia]
    • pallavi and I talked about having a half yearly synch up before the next Work Week in June. Thoughts?

- Strings for clock have been completely reworked (+ changes in dates.properties) - New System Update notification with link to release notes - changes in accessibility.properties can be ignored - brandShortName has been changed into Foxfood, and we cannot update the entity name, so until localizers actually update the string, the old value will be used. - Firefox Sync strings will likely be covered by automation, but needs to be confirmed as it seems not everything is covered

  • [Jamie_] Test day info, also a few questions.
  • Scope of this meeting: Is it supposed to be public or private?
    • If private: Remove this event from the public QA calendar, and remove the notes the public wiki
    • If public: Use a public etherpad for the meeting notes

Community Updates [marcia-read only]

[Jamie_]

  • will be starting on revamp of firefox OS QA pages soon, have to talk to some people about it.
  • Setting up a QA test day for firefox os (working with Noah and Mermi)
  First one to be held October 16, would like to know best day to help with 2.5 release
  also a suggestion list of features or stuff most relevant to test with 2.5 for the aim.
  add to this document please https://public.etherpad-mozilla.org/p/requested-areas&testcases
  email : jamie@mozillausa.org

https://public.etherpad-mozilla.org/p/firefox-os-testday

This Week - Decisions & Action Items

[marcia] Action item to follow up with Michael Treese in Orlando [team] Finalize sessions/Items for Agenda for Orlando [Al/Pallavi] To follow up the QA ww between Moz ww


Testing information

Retrieved from "https://wiki.mozilla.org/index.php?title=B2G/QA/Meetings/2015-10-07&oldid=1100705"