B2G/QA/Automation/Style Guide/Best Practices

The Page Object Pattern

Tests should implement the Page Object pattern. Here's a graphical representation:

+------+        +------------+        +------------+
| Test | -----> | Page class | -----> | Marionette |
+------+        +------------+        +------------+

Each class has given responsibilities:

• Marionette: Communicates with the browser to simulate user's actions and what he sees.
• Page Class: Represents a page. It interacts with Marionette to expose the necessary information a test needs. It also shows the operations a test can do on that page.
• Test Class: Creates the input data, passes it to the page class and assert the output is correct. No assertion should be done in the Page Class.

In more details

The arrows have their importance in the graphic.

A test uses an instance of the Page class. For example:

def test_something():
    my_page = PageClass()
    my_page.do_something()
    self.assertTrue(my_page.is_something_done)

The page class implements the way (the taps and waits) something is done, thanks to the Marionette APIs.

class PageClass(Base):

    def do_something():
        button = self.marionette.find_element(By.ID, 'something-button')
        button.tap()
        Wait(self.marionette).until(expected.element_not_displayed(button))

    def is_something_done():
        something_done_label = self.marionette.find_element(By.ID, 'something-label')
        return something_done_label.is_displayed()

There is no arrow that directly link Test and Marionette. This means a test doesn't use marionette directly, it's always done by the page class.

Finally, the page class shouldn't be aware of the tests using it. Test data should be provided in the functions. This is why the arrows are 1-way-only.

PageRegions

In some circumstances, for example where a header or a list element is common across the app, we will use a page region. The page region is a child class of the base Base object, which is inherited by all page objects. This means that the navigation can be reached from any page object and herein lies the DRY!

+------+        +------------+        +------------+
| Test | -----> | Page class | --┬--> | Marionette |
+------+        +------------+   |    +------------+
                      |          |
                      |          |
                      ⌄          |
                +------------+   |
                | PageRegion | --┘
                +------------+

A brief example:

class MyAppBase():

    @property
    def header(self):
        return MyApp.HeaderRegion(self.marionette)

    class HeaderRegion(PageRegion):

        _login_link = (By.ID, "home")

        def tap_login(self):
            self.marionette.find_element(*self._login_link).tap()

Referring to this page region with a property makes it very readable and concise from within the test. Clicking login during a test would be performed like this:

my_page.header.tap_login()

Another example where this might be used is on a search results page, the page region being the search results element.

One possible issue of using PageRegion object is that changing frames after instantiating PageRegion object might make the root_element stale. This will cause an error when trying methods that needs access to self.root_element. The workaround in such case would be to refresh the root_element again, by locating it again with the locator.

app.py and regions/helper.py

PageRegions can be defined outside of main class. In gaiatest/apps folder, each subfolder contains the helper python method files. The rule of thumb is as follows:

• app.py: Contains the class for the main screen when the app is instantiated. There should be a method that instantiates the subpage objects.
• regions/*.py: Contains the classes for the each subpage of the app.

Widgets

In Firefox OS automated tests, widgets are PageRegions that represent elements (like header, switches). This helps to standardize components of web pages and to make sure we wait on every necessary event when we perform an action.

Every widget that starts by "Gaia" represents a custom component created for gaia. Here are below some examples.

GaiaHeader

GaiaHeader class is mainly used to tap the back button on the gaia-header of the app page. This helps to standardize the method to return to the previous page on most of the app page.

  # ID of the gaia header tag
  _header_locator = (By.ID, 'header')

  # specify the locator, and in this case, going back actually exits the app
  # if it is not a fullscreen app (status bar is visible), then statusbar=True
  def tap_back_button(self):
        GaiaHeader(self.marionette, self._header_locator).go_back(exit_app=True, app=self)

GaiaBinaryControl

GaiaBinaryControl represents buttons that have 2 states: enabled or disabled. It can be, for instance, a gaia-switch or a gaia-checkbox.
In some rare cases where the switch UI is using the older <switch> tag, one has to use the HtmlBinaryControl class, but most of the switches were recently converted to gaia-switch.

    _gps_switch_locator = (By.CSS_SELECTOR, 'gaia-switch[name="geolocation.enabled"]')

    def enable_gps(self):
        self._gps_checkbox.enable()

    def disable_gps(self):
        self._gps_checkbox.disable()

    @property
    def _gps_checkbox(self):
        return GaiaBinaryControl(self.marionette, self._gps_switch_locator)

Going further with the Page Object pattern

One issue encountered with the page object pattern is: sometimes, functions in the page class can be full of similar code due to Waits.

To solve this problem, here's an specialization of the Page Object pattern

+------+        +-----+        +------+        +-----------+        +------------+
| Test | *   1  | App | 1    * | View | 1    1 | Accessors | 1    1 | Marionette |
|      | -----> |     | -----> |      | --┬--> |           | -----> |            |
+------+        +-----+        +------+ 1 |    +-----------+        +------------+
                                   ^ *    |
                                   └------┘

A page gets split into 3 classes: app, views and accessors. The test class from the previous model is a bit changed. The marionette client remains identical.

Accessors

They are meant to manipulate the Marionette Client, in order to bubble up a ready-to-be-used HtmlElement. The most basic case is where you call marionette.find_element(). Nevertheless, we regularly need the element to be displayed. Moreover, sometimes, elements are moving (like a left-to-right transition). element.is_displayed() returns true once a single pixel of it is displayed. This can lead to race conditions

In short, this class has to make sure the element is:

• present in the DOM
• visible
• not moving.

For instance, in the SMS app, we can return the HtmlElement of a button this way:

class InboxAccessors:
    @property
    def root(self):
        panel = Wait(self.marionette).until(expected.element_present(self._main_locator))
        Wait(self.marionette).until(expected.element_displayed(panel))
        Wait(self.marionette).until(lambda m: panel.rect['x'] == 0)
        return panel

or in JavaScript:

InboxAccessors.prototype = {
  get root() {
    var panel = this.client.helper.waitForElement(SELECTORS.main);
    this.client.waitFor(function() {
      return panel.rect().x === 0;
    });
    return panel;
  }
}

View

General definition

A view represents what a user is able to do with the UI he/she's shown. It uses the elements given by the accessors to perform users' interactions. Once the action is done, it calls the right accessors to make sure the reaction of the UI is over. If needed the view is in charge of initializing the next view.

Taking the example started above: In the SMS app, first panel, the user wants to create a new message. After the button tapped, he will be presented the New message view. Translated into code, this gives:

class InboxView:
    def create_new_message(self):
        self.accessors.new_message_button.tap()
        return NewMessageView(self.marionette)

or in JavaScript:

InboxView.prototype.createNewMessage = function() {
  this.accessors.newMessageButton.tap();
  // The constructor waits until the New message panel is ready to be used
  return new NewMessageView(this.client);
}

Naming

The methods of the views are named after the user's intent. In other words, they represent what the user wants to do, and what is the outcome of it. For instance:

// bad
inboxView.tapNewMessage();

// better
inboxView.createNewMessage();

There are several drawbacks in calling a method like tapNewMessage():

• If the method returns the next view, then it's doing more than just tapping on the button. This can be confusing.
• If the method just taps on the button, like advertised, why not calling directly this.accessors.newMessageButton.tap()?
• Integration tests are usually verifying that a functionality is working. If a functionality is now triggered by another mean, but the user flow stays the same; the test should remain unchanged. Then, the way a user interacts with the UI is a detail in regards of the test itself; and views implement this detail.

Switching frames

Like said above, views are in charge of initializing the next view. Sometimes, the next view in not in a reachable part of the current DOM. In this case, views have to ask to switch to this other part of the DOM, so the test is ready to use the next view.

This "other part of the DOM", is implemented by iFrames, in Firefox OS. For example, the SMS app and the Dialer app are in 2 different frames, you switch from one to another like here:

ConversationView.prototype.callContact = function() {
  this.accessors.callButton.tap();

  var dialer = this.client.loader.getAppClass('dialer')
  dialer.switchTo(); // Implementation details described in the next part
  return dialer;
}

Special cases

Views usually implement the happy path of a feature. In the example above, we consider that no error will be displayed to the user. If one of your integration tests breaks the happy path defined, the best way to implement it is to use accessors directly in the test.

App class

This class is the first entrance of a test. When started, this class is in charge of making sure:

• The app is launched
• We're in the right app frame
• The first view is initialized

Then, if we left the app frame, this class will remain responsible for knowing how to get back. For instance:

Dialer.URL = 'app://communications.gaiamobile.org';

Dialer.prototype.switchTo = function() {
  // Switch to the system app, where all the apps iFrames live.
  this.client.switchToFrame();
  // switchToApp already waits for the app to be displayed
  this.client.apps.switchToApp(Dialer.URL, 'dialer');
}

// Then dialer.switchTo() will be called by views, like the example shown above.

Test class

The test is responsible for:

• Instantiating the data and pass it along views (like in the Page Object pattern).
• Describing the user's intents to achieve a goal.
• Verifying the goal is achieved, thanks to assertions.

For instance, in JavaScript:

var messagesApp;
var inboxView;

setup(function() {
  messagesApp = Messages.launch();
  inboxView = new InboxView(client);
});

test('user can send a message', function() {
  var text = 'Test';
  var composerView = inboxView.createNewMessage();
  composerView.typeMessage(text);
  var conversationView = composerView.send();
  assert.strictEqual(conversationView.lastMessage, text);
})

TL;DR

All this above can be summarized by:

• Tests launch apps.
• Apps instantiate the first views.
• Tests list users' intents exposed by views.
• Accessors bubble up HtmlElements, which are in a ready state.
• Views interact with these elements, and finally retrieve the text or the attributes and give it to the test.
• Tests verify if the output is correct.

Equivalence table

Use External Parameters

Using test variable file when running gaiatest can avoid defining variables inside the script. The testvars template is located here.
Make sure to fill in the appropriate section if you're planning to use it, and supply the name and location of the .json file as the parameter to the gaiatest command with --testvars= option.

If you want to access the varable value defined in the .json file, you can do as the following example:

test_phone_number = self.testvars['remote_phone_number']

If you need to access the sub-variable, consider below example as well:

self.testvars['plivo']['auth_id'],
self.testvars['plivo']['auth_token'],
self.testvars['plivo']['phone_number']

Make sure that you are not including your testvars.json file in your PR request, as the testvars file for jenkins is managed separately.

Use sleep() calls only if you have no other choice

There are several ways to wait for something to happen:

# Good
## Wait for an element to be displayed
Wait(self.marionette).until(expected.element_displayed(By, locator))

## Wait for an element to disappear
Wait(self.marionette).until(expected.element_not_displayed(By, locator))

## Wait for a transition to be over
Wait(self.marionette).until(lambda m: element.rect['y'] == expected_y_position)

# Bad
import time
time.sleep(seconds)

sleep() calls should be used only when there is no other way to delay the action of Marionette. Using sleep() instead of Wait() is bad for following reasons:

• sleep() does not care about the UI changes in app. If you're using sleep() to just 'wait enough', you'll run into problems when the app behavior changes and requires more/less time to wait.
• sleep() does not care about the phone performance. If the speed of the execution changes because of the changes in memory allocation or running on a newer/older devices, it will still wait for specified time.

If have to use the sleep() call, make sure to put in the comment explaining why other methods won't work.

Limit the Use of Conditionals

• Methods should not contain logic that depends on properties of the page. The logic and expectations should be within the test, and adding this to the page object could guard your tests against genuine failures.

# Good
def click_login(self)
    self.selenium.find_element(*self._login_locator).click()

# Bad
def click_login(self)
    if not self.is_user_logged_in:
        self.selenium.find_element(*self._login_locator).click()
    else:
        pass

Assertions

• Like said in the Page Object pattern part: Tests should handle the asserts -- not the page objects.
• Tests should use unittest's assert methods.

# Good
a = some_function()
self.assertEqual(a, 'expected result')

# Bad
a = some_function()
assert 'expected result' == a

Assertion Messages

Put a custom error message in Assert() call only when it provides more information than the one given by default. `self.asserEqual()` by itself is a nice assertion method: if it fails, it'll show you the expected value and the actual.

For example,

self.assertEqual(displayed_phone_number, expected_phone_number)

 AssertionError: u'+11234567890' != u'1234567890'
 Stacktrace:

But if you provide an error message as follows,

self.assertEqual(displayed_phone_number, expected_phone_number, msg='Phone numbers are not the same')

 AssertionError: Phone numbers are not the same
 Stacktrace:

In other words, by adding more context, you might actually remove some useful debug data.

Check what an end-user would

• Check what an end-user would check (i.e., presence of dialogs, texts, icons). In end-to-end testing, there is little need to check the value of internal state inaccessible to the user. Verifying internal state should held in integration tests.

Clean Up Afterwards

If your script have changed the data settings or other settings that are not reverted by resetting B2G, it is recommended to revert your setting on the teardown() method. tearDown() method gets invoked at the end of the script execution, regardless of the test result.

    def tearDown(self):
        self.marionette.switch_to_frame()

        # turn off the cell data that was enabled during the test
        self.data_layer.disable_cell_data()

        # don't forget call the super method as well
        GaiaTestCase.tearDown(self)

Update Manifest File

After the script is done, make sure the corresponding manifest.ini file is updated with the right flags.

• If the test should not run on a particular device, use skip-if (and provide explanation)

[test_browser_bookmark.py]
# Bug 1178859 - test_browser_bookmark.py: "IOError: Connection to Marionette server is lost."
skip-if = device == "flame"

• If the test should be failed under particular device, use fail-if (and provide explanation)

fail-if = device == "desktop"

Here you can find an explanation of the available keywords