Testing for iOS applications

Testing ensures the application is error-free while it still meets all the product functionalities and requirements. Testing helps verify that new code changes do not cause negative effects or unexpected behaviors in the application. Furthermore, it can check whether the defects are resolved or not.

Having a standard structure for the test suite is necessary as it will help keep the testing code consistent, maintainable and changeable corresponding to the project’s specifications.

Test Types

In an iOS Project, the application includes the three main test categories.

Non-UI Tests

These tests correspond to what are commonly named unit tests and integration tests.

Unit testing is a level of software testing where individual units/components of the software are tested. The purpose is to validate that each software unit performs as designed. Each unit test should assert the expected behavior of a single path through a method or function. To cover multiple scenarios, write one test for each case.

Integration testing is a level of software testing where individual units/components are combined and tested as a group.

The purpose of this level of testing is to expose faults in the interaction between integrated units.

Because integration tests use the same APIs and follow the same testing pattern, integration tests look very similar to unit tests. The difference between them is about the scale described in the following table:

As both integration tests and unit tests use the same APIs, the code of these non-UI tests is contained under the same folder Tests.

UI Tests

UI testing is a level of software testing where the UI is tested for acceptability. The purpose of this test is to evaluate UI compliance with the business requirements for common use cases.

The code of UI tests is contained under the folder UI Tests.

Testing Methodology

Directory Structure

When setting up a project, Xcode provides an option to include tests. If the option to include tests is checked, Xcode automatically generates 2 folders:<ProjectName>Tests and <ProjectName>UITests.

It is recommended to remove <ProjectName> prefix from the default folder name. After removing the prefix, there are two folders to contains testing code:

• The code for unit tests and integration tests must be located in the Tests folder.
• The code for UI tests must be located in the UI Tests folder.

Test Class Naming

Add suffix Spec to the original Class name.

Example: Test class for MainActivity:

MainActivityTest
MainActivity_Spec

MainActivitySpec

UI Tests

Create UI tests to verify that critical paths in the user journey can be completed in the application. This ensures bugs, which can break the behavior of UI controls, cannot be introduced. UI tests replicating real user interactions provide confidence that the application can be used for its intended task.

Test Case Prioritization

When it comes to UI testing, there can be a huge number of scenarios and combinations in which the test cases may run. Technically, it is possible to test all of them. However, as it is not desirable nor pragmatic due to the time-consuming nature of both writing and executing the tests. Hence, the best practice is to have test case prioritization.

Test case prioritization is a technique of prioritizing and scheduling test cases. This technique helps to cover the test cases (from higher to lower priority) while minimizing time, cost, and effort during the software testing phase.

The aspects that impact the test prioritization are the following:

• Business impact
• Critical features
• Frequently used functionalities
• Complex implementation
• Buggy areas of the software

Use the following criteria to decide which test cases have higher priority.

• The main features of the application, also known as the key features of a business, should be tested carefully during the development process. Avoiding critical issues on such features is the most emphasized task of UI testing. Some common key features:
  • Log in
  • Payment
  • Other business flows (such as booking flow, retail flow, etc.)
• Besides the main features, there are some utility features that the user will use regularly, such as changing settings for the themes or localization.
• There are some test cases that are difficult as well as time-consuming to perform. Set a low priority for these test cases. Since UI testing is done locally or on the CI machine, it is advisable to manage the duration of the test.

Despite the prioritization of test cases, there might still be some test cases that are not eligible for writing UI Test.

• The test cases of which the requirements are frequently changing
• The test cases which are recently designed and not executed manually at least once
• The test cases which are run only on the ad-hoc environment for testing purpose

After prioritizing the test cases, categorize them into test suites.

Categorize Test Suite And Define Test Case

Once the test requirements are identified and the test cases are prioritized, it is time to design the test cases based on the application’s requirement and categorize them into test suites. Make sure to define at least one test case for each identified requirement.

At the beginning, the test case can be a simple draft or with very few details. However, as the project expands and more requirements are appended, the test suites and test cases need to be scaled up or down. Such activities should be continuously performed throughout the product’s life cycle.

A test suite is a collection of test cases. In a test suite, the test cases are organized in a logical order.

A test case is a set of pre-conditions, procedures (inputs and actions), expected results and post-conditions. It is used to determine whether the application under testing meets the requirements.

For instance, User A is a member user whilst User B is an anonymous user. The member user will get 10% off any product, but the anonymous user will not.

- Test suite A:
  - Test case A1: Verify the member type after logging in with account A
  - Test case A2: Verify the price of the product displaying 10% off when logging in with account A
  - Test case A3: Verify user can add a discounted product
  - Test case A4: Verify the total price when checking out
- Test suite B:
  - Test case B1: Log in with account B
  - Test case B2: Verify the price of the product displaying the normal price when logging in with account B
  - Test case B3: Verify user can add a (no discount) product
  - Test case B4: Verify the total price when checking out

Bear in mind that each test case in a test suite is dependent on the success of the previous test case. Hence, if User A does not add any product, then the user cannot check out.

According to the example above, some test cases reuse a sequence of test steps, but with adjusted input data. So while writing UI test scripts, it is recommended to define test flows and take advantage of them as much as possible.

Test data in UI testing can be manually prepared or automatically generated from data generation tools. Avoid using privacy-sensitive data for testing according to the data protection regulations (For example: General Data Protection Regulation).

Define identifiers for UI components

During the implementation of UI testing, it is important to define identifiers for UI components so that the machine can distinguish between different elements. Use a 2-level or 3-level domain naming scheme (useCase.description or feature.useCase.description). Use camel case instead of capitals or snake case to avoid ambiguity.

Although some general identifiers can be reused over the application (such as an identifier for back buttons, navigation bar, etc.), it is recommended to set distinctive identifiers inside each screen, which means there should not be two components having the same identifier within one single screen.

For example:

main.tabbar.newFeed
main.tabbar.settings
authorization.signIn.usernameTextField
authorization.signIn.passwordTextField
authorization.signIn.signInButton

When defining a test flow, focus on providing an interface with input data. It is also suggested to verify whether the UI element is available for interaction before interacting with it:

Example 1

A general test flow such as tapping a back button on a screen:

// Testing using XCUIApplication
// GeneralUIFlows.swift
extension XCUIApplication {

    func tapBackButton(onScreen screenIdentifier: String) {
        // Because there may be back buttons on different screens.
        // Hence, it is recommended to verify the screen in which the back button is.
        let view = views[screenIdentifier].firstMatch
        guard view.exists else {
            Nimble.fail("View is hidden. \n+ Identifier: \(screenIdentifier)")
            return
        }
        buttons[General.backButton.identifier].firstMatch.tap()
    }
}

// Testing using KIF
// GeneralUIFlows.swift
extension KIFTestActor {

    func tapBackButton(onScreen screenIdentifier: String) {
        // Because there may be back buttons on different screens.
        // Hence, it is recommended to verify the screen in which the back button is.
        do {
            try tester().tryFindingView(withAccessibilityLabel: screenIdentifier)
            tester().waitAndTapView(with: General.backButton.identifier)
        } catch {
            Nimble.fail("View is hidden. \n+ Identifier: \(screenIdentifier)")
        }
    }
}

Example 2

A sample test flow in the List Repo screen:

// Testing using XCUIApplication
// ListRepoUIFlows.swift
extension XCUIApplication {

    func tapRepoCell(at index: Int) {
        let table = tables[ListRepos.tableView.identifier].firstMatch
        let cell = table.cells.element(boundBy: index)
        expect(table.exists && cell.exists && cell.isHittable)
            .toEventually(
                beTrue(),
                timeout: .long,
                description: "The cell at index \(index) of the table view \(identifier) is not available."
            )
        cell.tap()
    }
}

// Testing using KIF
// ListRepoUIFlows.swift
extension KIFTestActor {

    func tapRepoCell(at index: Int) {
        let indexPath = IndexPath(row: index, section: 0)
        tester().tapCell(at: indexPath, inTableView: ListRepos.tableView.identifier)
    }
}

Example 3

Make sure to cover the high prioritized test cases as much as possible when preparing test data and implementing test cases.

A simple test case in the List Repo screen:

// Testing using XCUIApplication
// ListRepoUISpecs.swift
final class ListRepoUISpecs: QuickSpec {

    override func spec() {
        var application: XCUIApplication!
        describe("list repos") {
            beforeEach {
                self.setUpSpec()
                application = setUpTestEnvironment()
                application.launch()
            }

            context("a repo is added to bookmark") {

                beforeEach {
                    application.tapRepoCell(at: 3)
                    application.tapBookmarkButton()
                    application.tapBackButton(onScreen: DetailRepo.view.identifier)
                }

                it("navigate back to the list repos screen and verify the repo item is bookmarked") {
                    let view = application.views[ListRepos.view.identifier]
                    expect(view.exists).toEventually(beTrue(),timeout: .medium)

                    let repoCellItem = application.getRepoCellItem(at: 3)
                    expect(repoCellItem.repoFullName).toEventuallyNot(beEmpty(), timeout: .medium)
                    expect(repoCellItem.isBookmarked).toEventually(beTrue(),timeout: .medium)
                }
            }

            afterEach {
                application.terminate()
            }
        }
    }
}

// Testing using KIF
// ListReposSpecs.swift
final class ListReposUITests: QuickSpec {

    override func spec() {
        let tester = self.tester()
        describe("list repos") {
            beforeEach {
                setUpTestEnvironment()
                self.setUpSpec()
            }

            context("a repo is added to bookmark") {

                beforeEach {
                    tester.tapRepoCell(at: 3)
                    tester.tapBookmarkButton()
                    tester.tapBackButton(onScreen: DetailRepo.view.identifier)
                }

                it("navigate back to the list repos screen and verify the repo item is bookmarked") {
                    let view = tester.waitForView(withAccessibilityLabel: ListRepos.view.identifier)
                        expect(view).notTo(beNil())

                    let repoCellItem = tester.getRepoCellItem(at: 3)
                    expect(repoCellItem?.repoFullName) != ""
                    expect(repoCellItem?.isBookmarked) == true
                }
            }

            afterEach {
                tester.resetToRootScreen()
            }
        }
    }
}

Tools - Libraries

There are some third-party libraries that facilitate writing tests: