Testing for Android applications

Testing is an essential part of the software development lifecycle, Android development is no different. With automated or manual testing being applied to the testing phase, it can help detect unexpected behaviors or application failures in development or different environments. Furthermore, the high percentage of test coverage in the projects also improves maintainability and enhances applications before releasing publicly.

Testing Principles

Test Coverage

Following levels of the Testing Pyramid, which is also documented on the official Android developer site, the application should include the three categories of tests in the following proportions:

• Unit Tests: 70%
• Integration Tests: 20%
• UI Tests: 10%

Test Types

Local Unit Tests

These tests focus on method invocation assertions and can run on local JVM or simulated devices (such as Robolectric).

Located at module-name/src/test/java/.

• Pros:
  • Minimize execution time.
  • Can mock the Android framework dependencies (Mockito, PowerMock, etc.).
  • Can be used to validate the individual project’s components.
• Cons:
  • Can not detect unexpected errors between components or modules.

Instrumented Tests

These tests focus on user interaction assertions and can run on the following types of devices:

• Real devices (Physical device)
• Virtual devices (Android emulator)

Instrumented Tests for Android can be classified into:

• Unit Tests - which requires Android system to execute test cases
• UI Tests (Espresso ☕)

Located at module-name/src/androidTest/java/.

• Pros:
  • Can take advantage of the Android framework APIs and supporting APIs, such as AndroidX Test, and provide more fidelity than local unit tests.
  • Can be used for Integration Tests and UI Tests.
• Cons:
  • Requires testing devices.
  • Project requires time to be built and executed on devices, much more slowly than unit tests.

Testing Strategy

Based on the current development projects, which are applying MVVM architecture, Unit Test should cover the following components/packages:

Repository

It’s easier to start by writing test cases on this layer. OkHttp provides MockWebServer to test requests and responses from specific endpoints.

• Expected test cases:
  • Remote (API request/response):
    • Verify parsing API responses to data models.
    • Verify sending requests with the expected params and HTTP methods (POST/GET/DELETE…).
    • Verify network error handling.
  • Local Storage (Database, SharedPreferences):
    • Verify storing and retrieving data correctly with CRUD actions.
    • Verify migrating database (e.g., update database version)
• Test method:
  • Unit Test.
  • Instrumented Test.

UseCase

UseCase is the contact point between Repository and ViewModel, so the best approach is to test from the bottom to the top layer.

• Expected test cases:
  • Verify the UseCase’s logic with expected inputs and outputs.
  • Verify error handling.
• Test method:
  • Unit Test.

ViewModel

ViewModel will handle most logic, so it’s important to have test cases for this layer.

• Expected test cases:
  • Verify the functionalities inside ViewModel with LiveData or Rx.
  • Verify loading behaviors.
  • Verify retrieving and mapping data returns from UseCases.
  • Verify navigation events.
  • Verify error handling.
• Test method:
  • Unit Test.

UI (Activity, Fragment, View)

The main point of the test case for the UI layer is to verify the features are working as expected with the user’s interaction.

• Expected test cases:
  • Verify user’s interaction.
  • Verify displaying data.
  • Verify loading behaviors.
  • Verify navigation events.
  • Verify error handling.
• Test method:
  • Unit Test.
  • Instrumented Test.

Service/Manager

The team applies testing to the Service part to ensure that it doesn’t have unexpected behaviors.

• Expected test cases:
  • Verify the execution, service bind, and unbind state.
  • Verify doing background jobs are working with logic handling.
• Test method:
  • Unit Test.
  • Instrumented Test.

Tools - Libraries

To write practical unit tests with shorter, meaningful test cases and faster execution time, third-party libraries are great options, along with the Android-supported built-in testing frameworks. They also help make the test cases more accessible and build much faster.

• Test Runner:
  • AndroidX Test includes a set of JUnit rules to be used with the AndroidJUnitRunner.
  • Robolectric is a framework that brings fast and reliable unit tests to Android.
• Mocking/Stubbing:
  • Mockito is a mocking framework for unit tests.
  • Mockito-Kotlin is a small library that provides helper functions to work with Mockito in Kotlin.
  • PowerMock is a framework that extends the capabilities of other mock libraries such as Mockito with the support of mocking static methods, constructors, final classes and methods, private methods, removal of static initializers, and more.

  PowerMock also provides Test Runner to execute the test cases with mocked Android framework stuff.

  • MockK is a mocking framework built for Kotlin that supports executing regular unit tests and Android Instrumented Tests via Subclassing or Inlining.
• Assertion:
  • Kluent is a “Fluent Assertions” library explicitly written for Kotlin.
  • Kotest is a flexible and comprehensive testing tool for Kotlin with multi-platform support.

Conventions

Test Directory Structure

When creating a new project, Android Studio creates the test source sets for Local Unit Tests and Instrumented Tests. Their test classes will be located in the directories (test or androidTest), depending on the type of test.

Test Class Naming

Add the suffix Test to the original Class name.

Example: Test class for MainActivity

MainActivityTest

Mock Object Naming

Add prefix mock to the original Object name.

Example:

private val mockObject = mock<Object>()

Test Case Naming

Include the test condition and expected behavior with the <When> _ <Expect> _ convention.

Example:

`When age is less than 18, it emits isAdult as false`

Example

@RunWith(TestRunner::class)
class MainActivityTest {

    // Test Rules
    @Rule
    val rule: TestRule = TestRule()

    // Mock objects
    private val mockObject = mock<Object>()

    @Before
    fun setup() {
        // Do something
    }

    @After
    fun after() {
        // Do something
    }

    @Test
    fun `When age is less than 18, it emits isAdult as false`() {
        // Do something
    }

    ...
}

Test Case Design Methodology

• A test case will contain multiple test functions that have a specific description, and one assertion for each observer used per test function.
  • Pros:
    • The test function’s description expresses clearly and exactly what happens in that test function.
    • No potential duplicated observers assertions in multiple test cases.
  • Cons:
    • Cannot exactly show the control flow of all the logic that happens in a given test.
    • More test functions are needed per test case.

Example

@Test
fun `When calling getData and all use cases respond positive results, it emits the success data value correspondingly`() {
    val dataObserver = viewModel.output.data.test()

    viewModel.input.getData()

    dataObserver
        .assertValueCount(1)
        .assertValue(Unit)
}

@Test
fun `When calling getData regardless of success or failure, it emits loading states as true then false`() {
    val loadingObserver = viewModel.output.isLoading.test()

    viewModel.input.getData()

    loadingObserver
        .assertValueCount(2)
        .assertValues(true, false)
}

@Test
fun `When calling getData responds positive result, it emits no error`() {
    val errorObserver = viewModel.output.error.test()

    viewModel.input.getData()

    errorObserver
        .assertNoValues()
}