Overview

Now Xcode 8 is out it was time to migrate some UI testing code over to Swift 3 as well ensure all the tests continued to work in iOS 10. Doing so uncovered some interesting changes in behaviour that weren’t necessarily advertised in this year’s WWDC (or perhaps I just missed them).

API wise, there hasn’t been any new additions or deprecations, on the surface it appears unchanged. The behaviour however, has changed slightly. Here are my findings so far:

1) Hidden elements no longer exist

In the past (and by past I mean Xcode 7 & iOS 9 days) all elements in the view hierarchy were available to query. This included hidden and non hidden elements. They all “exist” and the only distinction between them was whether they were “hittable”.

for example take this simple story board, with three buttons.

  • Visible
  • Hidden
    • hidden = true
  • Alpha 0
    • alpha = 0

Sample Storyboard

Running the following test passes when running on iOS 9

    func testVisibility() {

        let app = XCUIApplication()

        XCTAssertTrue(app.buttons["Visible"].exists)
        XCTAssertTrue(app.buttons["Hidden"].exists)
        XCTAssertTrue(app.buttons["Alpha 0"].exists)
    }

Sample Storyboard

In iOS 10 however, hidden elements no longer exist, they are completely omitted from the accessibility tree exposed to the test. Oddly enough, the same occurs for elements with an alpha or frame of zero.

Sample Storyboard

2) Elements beneath modals are no longer hittable

Arguably this used to be a bug in the previous iteration of the API. Whenever a modal element appears on screen (a popover or action sheet on iPad for example) visible elements beneath the modal were still hittable.

Suppose our app now presents an action sheet on tapping the “Visible” button, the following test passes when running on iOS 9:

    func testActionSheets() {
        let app = XCUIApplication()
        let button = app.buttons["Visible"]

        button.tap()
        XCTAssertTrue(button.isHittable)
    }

Sample Storyboard

In iOS 10 however, the test above will fail due to the button not being hittable.

Sample Storyboard

This brings on an interesting question, how does one dismiss an action sheet or popover on iPad ??

It’s handy to record tests or inspect the debug information of XCUIApplication. You’ll find the overlay background is an accessibility element of its own with an identifier of “PopoverDismissRegion”.

As such dismissing popovers can be achieved by simply tapping that element:

let popoverDismissRegion = app.otherElements["PopoverDismissRegion"]
popoverDismissRegion.tap()

If you find you need to do this often, adding this to an extension may be useful:

extension XCUIApplication {
    func dismissPopover() {
       otherElements["PopoverDismissRegion"].tap()
    }
}

3) Landscape coordinates are inverted

This is probably a regression in iOS 10, XCUICoordinates don’t seem to respect orientation changes any more.

for example, suppose our app now draws touches and we have the following test code:

   func testCoordinates() {
        let app = XCUIApplication()
        let v = app.otherElements["view.container"]

        XCUIDevice.shared().orientation = .landscapeLeft

        let c = v.coordinate(withNormalizedOffset: CGVector(dx: 0, dy: 0.5))
        let d = c.withOffset(CGVector(dx: 100, dy: 0))
        c.press(forDuration: 0.5, thenDragTo: d)
    }

in iOS 9, regardless of orientation a horizontal line is drawn as you’d expect from the centre left edge of the screen.

Sample Storyboard Sample Storyboard

on iOS 10 however, when in landscape the coordinates are inverted.

Sample Storyboard

The current workaround is to ensure portrait orientation when performing the test or to manually account for orientation changes when dealing with coordinates.

4) Tests are slower!

This was one of the saddest discoveries which I hope will be improved. Tests running on iOS 10 appear to be a little slower than those run on iOS 9.

Let’s take a test that simply taps a button a 100 times:

    func testSpeed() {
        let app = XCUIApplication()
        let button = app.buttons["Visible"]

        for _ in 0..<100 {
            button.tap()
        }
    }

When running in iOS 10 Simulator:

  • iPad Air 2: 177 Seconds
  • iPhone 6s: 141 Seconds

When running in iOS 9 Simulator:

  • iPad Air 2: 152 Seconds
  • iPhone 6s: 113 Seconds

These figures will vary based on the machine the tests are run on, however iOS 10 results are consistently slower than iOS 9. What’s odd here is, you’d think the changes made for item 1 above would mean an improved performance due to having less elements to process now hidden ones are taken out of the equation.

While a small difference, it does add up over a large number of tests.

Related