QA/Suggested Tiles/Test Plan

From MozillaWiki
< QA
Jump to: navigation, search

Suggested Tiles

Ownership

Developer contact: Ed Lee, :Mardak on IRC

QA Lead: Cornel Ionce, :cornel_ionce on IRC

Status

Overview

[LANDED] Nightly 41: landed

[LANDED] Aurora 40: uplifted

[LANDED] Beta 39: partially uplifted

[BACKED OUT] Release 38: backed-out

[BACKED OUT] Release 38.0.5: backed-out

[LANDED] Beta 40: uplifted

Summary

Suggested Tiles is a new feature on the new tab page where users who visit certain sites see additional content. There's new onboarding experiences for new as well as existing users that explains the feature and how to turn it off. There's various "business logic" that a server-provided json file can trigger to have suggestions be shown or not shown.

Context

There are 17 bugs to uplift from Aurora40 to Beta39:

Onboarding:

  • 1138818 - New tab user onboarding for sponsored suggested tiles
  • 1165430 - Switch unlocalizable "Example Title" to "mozilla.org"
  • 1165525 - Onboarding messaging poorly words how to turn off suggestions by implying blank is the only way

UI changes:

  • 1158853 - Replace history tile titles with base domains and include titles as part of tiles
  • 1158859 - Update newtab cog menu styling
  • 1138817 - Change [SPONSORED] tags to [SUGGESTED] and add [SUGGESTED] tags for suggested tiles
  • 1165386 - Always show SPONSORED for any type of sponsored tiles and SUGGESTED for other suggested
  • 1167805 - Not enough padding for page title causing pin icon to overlap

Business logic:

  • 1156549 - Allow ramp up time for campaigns with strict start/stop times
  • 1159571 - Use server provided frequency caps for daily and lifetime totals
  • 1139496 - Allow server provided explanation / ad group name to be displayed on Suggested Tiles
  • 1159884 - Implement inadjacency with a hardcoded list of hashed sites
  • 1145428 - Suggested Tiles pins (becomes a user history tile after end time)

Misc fixes:

  • 1045766 - Add telemetry of users who can see new tab Tile suggestions
  • 1160369 - Allow for sponsored suggested tiles
  • 1166552 - browser_newtab_reportLinkAction.js fails on timeout
  • 1167243 - Intermittent browser_newtab_enhanced.js | Suggested for {'Web Education'|'Technology'} enthusiasts

References

  • The UX planned for Suggested Tiles is available here.

Bug work

Tracking bugs

meta: Bug 1140185 - Suggested Tiles: v1.0 Sponsored Release (FX 39)

meta: Bug 1120311 - Suggested tiles: Affiliate Release (FX 38)

meta: Bug 1148859 - Uplift to 38 Suggested tiles: Affiliate Release

Back-out bug: Bug 1156921 - Backout Suggested Tiles (bug 1120311) from 38.0

Bug triage +/- for verification

[QE-VERIFY+] Bug 1105360 - Only enhance tiles that are explicitly enhanceable
[QE-VERIFY+] Bug 1126186 - Allow users to turn off recommendations
[QE-VERIFY+] Bug 1126188 - Show explanation text for a suggested tile with appropriate styling
[QE-VERIFY+] Bug 1140496 - Only show a suggested tile url for some number of times or until clicked
[QE-VERIFY+] Bug 1143797 - Allow clicking on suggested explanation text to see overlay explaining the suggested tile
[QE-VERIFY+] Bug 1146146 - Maximize the number of rows of tiles by reducing the suggested explanation maximum line count to 2 instead of 3
[QE-VERIFY+] Bug 1146249 - Tiles on the newtab page don't wrap properly
[QE-VERIFY-] Bug 1126182 - Extract related tiles data from links json and store for later selection
[QE-VERIFY-] Bug 1126183 - Provide a way to check if a site is part of the top 100 PlacesProvider links
[QE-VERIFY-] Bug 1126184 - Select a single related tile to show as the first unpinned tile
[QE-VERIFY-] Bug 1136208 - Rename various references of related in Firefox
[QE-VERIFY-] Bug 1143745 - Update the way Firefox reads directoryLinks.json sent from the server for the tiles v3 endpoint
[QE-VERIFY-] Bug 1149021 - Suggested tile with just an image shows a thumbnail instead
[QE-VERIFY-] Bug 1149680 - Include Firefox channel in tiles fetch
[QE-VERIFY-] Bug 1149682 - Filter suggested tiles to only allow non-sponsored content
[QE-VERIFY-] Bug 1152145 - Filter for specific suggested tiles adgroups/buckets/frecent_sites lists with display name for 38

Sign off

Criteria

  • All the blocker, critical, major bugs have been fixed.

Results

Feature was successfully signed-off. Latest testing was performed on Firefox 40.0beta9 (2015-07-31).

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Suggested_Tiles/Test_Plan&oldid=1087648"