B2G/QA/Meetings/2015-09-23

From MozillaWiki
< B2G‎ | QA‎ | Meetings
Jump to: navigation, search

// For 2015/09/23

Overview


Off plan

  • Johan: Sep 14-27
  • Cynthia - Sep 24
  • Marigold - Sep. 26-27 (Mid-autumn festival), Oct. 1-7 (National Day)
  • TPE Office - Sep 28 (Mid-autumn festival), Oct. 9-11 (National Day)
  • Delphine Sep 07-28 (3 weeks summer vacation \o/ ): flod is covering up for me for RTL/l10n testing, Marcia/William have knowledge on test cases w/Marigold.
  • Gerry - Oct. 7~23
  • Paul - Oct. 14-19
  • Al - Oct. 16-21
  • marcia - Traveling September 28 and October 2

2.2r

2.5 Feature

  • Telemetry: Gerry
  • Private browsing - Tracking Protection: Alison
    • Risk: blocked by Pin the Web, pin site card needs to be reworked for the control panel
    • Wait for Wilfred to decide if we can ship it (ai: Candice to check with Wilfred by the end of this week)
  • Replaceable Homescreen: Teri
  • Low Device Storage: William (Descope)
    • UX spec 3.0 link update
   - https://mozilla.box.com/s/bfrwwxdaz1o646h8tmmnlq61lv92sy3f
    • VSD spec
   - https://mozilla.box.com/s/zc2he933m7oluvvq5rwu8h97clhi0g2e
    • QA status
   - Test plan creation -> Draft ( https://goo.gl/zVfJpJ ). Not yet completed. In progress.
   - Test case creation -> Not yet completed. In progress
    • De-scope this for v2.5 (confirmed by Candice) on Sep. 23rd
  • New security model: Mike
    • Event - Workweek in MV on Sep. 28th
    • Feature ready on Oct. 2nd

Riding the train features:

  • RTL: William
    • RTL UX spec timeline: v2.2 spec -> v2.0m spec -> v2.5 spec
    • Rob has provided the UX spec last Saturday (9/19)
  - https://mozilla.app.box.com/s/y60s1ngyza9jcltmpaqwfoye4f311n5r
    • Marigold engineers has created 54 RTL test cases by referring v2.0m RTL spec (9/22).
  - https://goo.gl/yl6q4I
    • What's next?
  - William will deliver documents to help Marigold engineers revise RTL test cases by referring v2.5 RTL spec.
      • William to confirm ETA
  • updates: FOTA : nhirata
    • see https://bugzilla.mozilla.org/show_bug.cgi?id=1204658#c1 3 weeks out.
    • still needs work to be done; At risk though it's not a release item. Chances are this is dropped as an item.
    • rollback feature : need to follow up with EPM/PM; confirmed dropped : column J of exit criteria to be updated.
    • aries and flame are two separate FOTA parts
      • working on getting a FOTA fix for aries out first,
      • then working on getting flame to FOTA gecko/gaia only.
  • Vaani : nhirata
    • new build out, you have to build and install or OTA using dogfood-latest
      • contacts grammar to only include favorited contacts.
      • Latest build includes:
        • Open <App>
        • Dial <Number>
        • Call <Favorite Contact>
    • Spanish and French are still in plan.
    • Haven't had time to test, will start testing tomorrow or friday
    • MozSpeech (voice sampler) should be in dogfood-latest as well.
  • Others
    • PTS testing: Teri
    • NFC test: Alison
      • Arrange NFC regression test on both Flame and Aries
      • NFC is broken if using OTA to update the device. Full flash is fine.
    • View Source: No-Jun
      • All major features are verified, there is only one bug that's being checked right now, but it's not a blocker (Bug 1204011)
      • Test suite is in moztrap
    • Taskcluster: changes in task cluster today where the images have moved to docker hub from quay.io and the image should be 0.0.20 / 0.99.20; having said that I'm having build issues with 0.99.20 image. Asking Wcosta to back out of this change

2.5 Spark/Aries-KK Smoketests

  • Tester: QAnalysts (KevinT)
    • Health Status: Aries: Red 3 blockers

Existing: - https://bugzil.la/1207048 [Browser] A video can't be played, even it has been loaded completely.

  • Fails[6073]

- regression windows found, a test build with bug 236989 backed out is being made - https://bugzil.la/1206028 [Aries KK][Hackerplace] Hackerplace app is unuseable.

  • Fails[16424]
    • Resolved fixed, It will be verified in the next build

- https://bugzil.la/1206272 can not go into settings -> home screens because panels/homescreens/wallpaper isn't loaded

  • Fails [13006]
    • Resolved fixed, It will be verified in the next build

Severe Issues: - https://bugzil.la/1183033 [B2G] Keyboard doesn't have click sound - https://bugzil.la/1203603 [Cost Control][Aries] Data Limit and Usage Graph not displaying accurate results

  • This issue only occurs after OTA from RC4 and the user has turned on wi-fi and data in the RC4 FTU

- https://bugzil.la/1204622 crash in strlen | __vfprintf

  • Reproduce-able crash after completing FTU when going quickly, doesn't affect testing.

- https://bugzil.la/1204617 [Aries][Bluetooth] does not sends files from Mobile device to Computer.

  • This issue only occurs after OTA from RC4.

2.5 Aries Dogfood

  • Health Status: Aries: Red 3 blockers

Existing: - https://bugzil.la/1207048 [Browser] A video can't be played, even it has been loaded completely.

  • Fails[6073]
    • Will be pushing out the FOTA regardless if this issue is resolved or not. It will be fixed through an OTA update

- https://bugzil.la/1206028 [Aries KK][Hackerplace] Hackerplace app is unuseable.

  • Fails[16424]
    • Resolved fixed, will be verified in the next FOTA build

- https://bugzil.la/1206272 can not go into settings -> home screens because panels/homescreens/wallpaper isn't loaded

  • Fails [13006]
    • Resolved fixed, will be verified in the next FOTA build

Please note that the builds kept failing today due to errors so no FOTA build was tested. Naoki reached out to Wander to revert some patches and go back to a state where this was not occurring.

2.5 Exploratory

  • Tester: QAnalysts (KevinT)

2.5 fullrun

(Update: 9/24) @ 2.5 fullrun-2 

   - 8/26~9/17 (3 weeks)
   - Test case: https://moztrap.mozilla.org/runtests/environment/7813/
   * test results for the latest Firefox OS 2.5-2 Full Test Run on Aries-KK (Full Flash) are:
     - 4664 passed, 109 failed, 10 blocked, 537 invalid / 6504 cases total.
     - Coverage is at 95.73%
     - 42 new bugs are submitted during the 2nd fullrun.  https://goo.gl/thM3k5
     - 10 bugs are new blockers (v2.5+), https://goo.gl/8537B7
   * The third 2.5 full run will be kick-off on Oct. 8th

2.5 smoketest (Marigold)

(Update: 9/24)

  • Aries-KK Master Smoke Test Results
 ● Existing:
   1. [Browser] A video can't be played, even it has been loaded completely.
      - https://bugzil.la/1207048
   2. Panels/homescreens/wallpaper isn't loaded
      - https://bugzil.la/1206272
   3. [Aries KK][Hackerplace] Hackerplace app is unuseable.
      - https://bugzil.la/1206028
      
 ●  Skipped Test Case Summary
    a. Aries doesn't support DSDS, TC 10739 is skipped.
    b. FM Radio doesn't work on Sony Xperia Z3C. TC 2313 is skipped.
    c. The OTA update notification can’t be received after you tap Settings
  • Flame-KK Master Smoke Test Results
 ● Existing:
   1. [OTA] Latest Flame build is a day behind
       - https://bugzil.la/1188992
   2. [Browser] A video can't be played, even it has been loaded completely.
      - https://bugzil.la/1207048
   3. Panels/homescreens/wallpaper isn't loaded
      - https://bugzil.la/1206272
      
 ● Skipped Test Case Summary:
   * TC 16424 is skipped because SPARK related apps do not exist on Flame v2.5 build.
  • Severe Issue:
  - https://bugzil.la/1183033 [B2G] Keyboard doesn't have click sound
  - https://bugzil.la/1203603 Data Limit and Usage Graph not displaying accurate results
  - https://bugzil.la/1204622 crash in strlen | __vfprintf
  - https://bugzil.la/1204617 Bluetooth cannot send files from Mobile device to Computer.

2.5 MTBF

flame-kk 512MB avg. running hours: 20.23 Blocker: 

- https://bugzilla.mozilla.org/show_bug.cgi?id=1206634
- https://bugzilla.mozilla.org/show_bug.cgi?id=1198950

mtbf bugs in Q3 https://bugzilla.mozilla.org/show_bug.cgi?id=1182954

    • RA exit criteria: 100 hrs
    • CC exit criteria: 200 hrs

Previous action items

(Update: Sep. 16th)

  • [DEFERED] delphine to try to figure out mid cycle l10n run; sept 7th -- [UPDATE] this needs to be later, test case coverage isn't done yet. ETA: the week of Sep. 21st
  • [DEFERRED] nhirata to sync up with delphine about Vaani
  • [PENDING] jlorenzo: follow up Comms issue with devs
 ** The tests are still valid to me. I haven't looked into the details of each, but we should have about the same amount of bug in SMS than in Contacts. I'll follow up with the SMS dev in Paris, once he's back from vacation (it should be today)
  • [In progress] marcia: follow up with pallavi regarding community budget.
    • Some discussion of this issue will happen at the Work Week
  • [Remove it after meeting][William] have Marigold check regression issues for Calendar
  • [Not done yet][atsai] send out e-mail about test strategy of "riding on train" feature to EPM and QA.

Agenda

(update Sep. 23th)

  • Marigold help on l10n test case coverage: how many people, who, when will start?
  >>> [William 9/17] Lancy, Zouyi, and Shally are helping on it.

FYI, the newcomer QA Page: https://quality.mozilla.org/teams/firefox-os-qa/ Daily Bulletin board (Feel free to post news here): https://wiki.mozilla.org/B2G/QA/Daily_Bulletin

  • 2.5 feature list : couple of features will land in the last sprint?
    • keep reporting risks and test when things land; we're already at risk.
  • [njpark] Updated Music test cases in moztrap (minor updates). Put tags on test cases that are automated in gaia-integration test, but those tests should still be run, since gaia-integration tests only run on B2G sim. The search and playlist feature should be complete by end of this week.
  • [njpark] it should be noted that once NGA apps replace the OGA apps (Music, SMS, Contacts), our python tests will not run, since they have a very different architecture, and conversion will not be trivial according to the devs. Our best bet is to make sure marionette-js on device is completed on schedule. Until then, we might need to do some extra manual testing on those apps.
  • [Jamie_] could we set up a list of things that is updated of things that need to be done or tested for contributors (b2g daily bulletin is there but that only tells you so much)
    • Jamie suggested having a meeting about working on writing One and Done tasks that can eventually be used for a Test Day.
  • [nhirata] need to update the wiki meeting with the notes. should we assign someone and a backup person?
  • [Kevin] [Location] - Do we want this in our titles or not? I vote yes personally. Makes it easier for querying and for triaging purposes.

Community Updates

[marcia-read only]

This Week - Decisions & Action Items

  • [whsu] William to confirm ETA for RTL test case ready
  • [Candice] To de-scope Private browsing or not will be decided after the meeting between Candice and Wilfred
  • Data storage features for v2.5 will be de-scoped for v2.5 (confirmed by Candice) on Sep. 23rd
  • Go/No Go meeting will be held next week on Sep. 30th
  • [Gerry/No-jun/Paul] to discuss about effects OGA to NGA with GIJ.
  • [Jamie_/Marcia] to host a meeting about community engagement
  • [Al] be the backup for putting meeting notes to Wiki; nhirata to be primary on putting meetings notes to wiki.


Testing information

  • View source is in: Dev option and enable view source, hold the power button and the menu will appear there.
  • Vaani voice sampler is in the dogfood; dogfood being blocked.
  • FOTA white listing for IMEI not implemented yet
  • Marketplace go to : Marketplace-dev.allozim.org, search for :debug and use addons
Retrieved from "https://wiki.mozilla.org/index.php?title=B2G/QA/Meetings/2015-09-23&oldid=1100712"