Chapters
Chapters

Chapters

Hide chapters

Android Test-Driven Development by Tutorials

Second Edition · Android 11 · Kotlin 1.5 · Android Studio 4.2.1

Before You Begin

Section 0: 5 chapters
Show chapters Hide chapters

Section I: Introduction to Test-Driven Development

Section 1: 3 chapters
Show chapters Hide chapters

Section II: Testing on a New Project

Section 2: 8 chapters
Show chapters Hide chapters

Section III: TDD on Legacy Projects

Section 3: 8 chapters
Show chapters Hide chapters

Appendix

Section 4: 1 chapter
Show chapters Hide chapters

4. The Testing Pyramid
Written by Fernando Sproviero

Traditionally, software testing was done manually. It consisted of deploying the application to a test environment similar to the real environment but with fake (test) data. The Quality Assurance (QA) members would perform black-box testing on it — testing the application without knowing the internals — and raise bug tickets. Then, the developers went back and fixed the bugs.

Even nowadays, without any kind of automation, this is happening on the Android ecosystem. Testing usually consists of compiling the release candidate application, installing it on a physical device or emulator, and passing it to the QA team. The QA members would then follow a test plan, executing their use cases manually to find bugs.

You’ll find that automating these repetitive use cases is the way to go. For each use case, you can write one or more automated tests. However, first, you need to understand that there are different kind of tests and how to classify them.

Tests are typically broken into three different kinds:

This is the testing pyramid, a concept originally explained by Mike Cohn in his book Succeeding with Agile. The testing pyramid gives you a way to group different types of tests and gives an understanding of how many tests you should consider on each layer.

You should have lots of small unit tests, some integration and fewer UI tests.

You’ll now go through each of the layers.

Unit tests

Unit tests are the quickest, easiest to write and cheapest to run. They generally test one outcome of one method at a time. They are independent of the Android framework.

The System Under Test (SUT) is one class and you focus only on it. All dependencies are considered to be working correctly — and ideally have their own unit tests — so they are mocked or stubbed. This way, you have complete control of how the dependencies behave during the test.

These tests are the fastest and least expensive tests you can write because they don’t require a device or emulator to run. They are also called small tests. To give an example of an unit test, consider a game app.

The Game class is one of the main classes.

A common use case is to increment the score using a function like incrementScore(). Whenever the score increments and exceeds the highscore, it should also increment the highscore. A simple and incomplete definition of the Game class can look like this:

class Game() {

  var score = 0
    private set

  var highScore = 0
    private set

  fun incrementScore() {
    // Increment score and highscore when needed
  }
}

Therefore, a test that checks this could be as follows:

fun shouldIncrementHighScore_whenIncrementingScore() {
  val game = Game()  

  game.incrementScore()  

  if (game.highScore == 1) {
    print("Success")
  } else {
    throw AssertionError("Score and HighScore don't match")
  }
}

If you run this test, you’ll see the test doesn’t pass. We now have our failing (red) test. You can then fix this to get our passing (green) test by writing the actual method for the Game class:

fun incrementScore() {
  score++
  if (score > highScore) {
    highScore++
  }
}

Some common libraries for unit testing are JUnit and Mockito. You’ll explore both of these in later chapters.

Google, in its testing fundamentals documentation, also suggests Robolectric for local unit tests. Robolectric simulates the Android runtime, it allows you to test code that depends on the framework without using a device or emulator. This means that these tests run fast because they run using just the regular JVM of your computer, just like any other test that uses JUnit and Mockito. However, some may consider Robolectric as an integration testing tool, because it helps you test integrating with the Android framework.

Integration tests

Integration tests move beyond isolated elements and begin testing how things work together. You write these type of tests when you need to check how your code interacts with other parts of the Android framework or external libraries. Usually, you’ll need to integrate with a database, filesystem, network calls, device sensors, etc. These tests may or may not require a device or emulator to run; they are a bit slower than unit tests. They are also called medium tests.

For a simple example of an integration test, think about a Repository class that depends on a JSON parser class that reads from a file. The repository asks the parser to retrieve the data. Then the repository transforms the data to your domain model. You could create a test that given a JSON file verifies that the repository correctly returns the domain data. You would be testing the integration between the repository and the JSON parser.

Note: If you mock the JSON parser and verify only the transformation to your domain model you would be creating a unit test. You should create unit tests for both the repository and also the JSON parser to ensure they work as expected in isolation. Then, you can create integration tests to verify they work together correctly.

Another example could be found in a retail app. You could ensure that the LoginActivity is launched whenever the user wants to add a favorite but hasn’t signed into the app yet.

The test could look like this:

fun shouldLaunchLogin_whenAddingFavorite() {
  // 1
  val user: User = null
  val detailActivity = createProductDetailActivity(user)
  detailActivity.findViewById(R.id.addFavorite).performClick()

  // 2
  val expectedIntent = Intent(detailActivity,
  	LoginActivity::class.java);

  // 3
  val actualIntent = getNextStartedActivity()
  if (expectedIntent == actualIntent) {
    print("Success")
  } else {
    throw AssertionError("LoginActivity wasn't launched")
  }
}

Here’s what this test does:

  1. Creates a new “details” activity, finds the favorites button and clicks on it.
  2. Creates the expected result: an intent to navigate to the login screen.
  3. Checks if the activity that launched is the same as the one expected to be launched.

Although this test deals with activities, you’ll see that it doesn’t require a screen for them to be rendered.

Another example of integration tests: In a social app, you could check if the list of friends is retrieved correctly from a REST API. Because this automated test will run frequently, you shouldn’t use the real production API. Usually, it is replaced with a local or fake testing server. This is to avoid using server quota and because the tests shouldn’t alter any production values.

This will also ensure that the tests are repeatable.

You can still use JUnit, Mockito and Robolectric in integration tests to verify state and behavior of a class and its dependencies as you’ll see in later chapters.

Google, in its testing fundamentals documentation, also suggests Espresso for medium tests. For example, to perform validation and stubbing of intents or to perform actions on view objects. However, some may consider these kind of tests as UI tests because you would be interacting with UI elements.

UI tests

Finally, every Android app has a User Interface (UI) with related testing. The tests on this layer check if the UI of your application works correctly. They usually test if the data is shown correctly to the user, if the UI reacts correctly when the user inputs something, etc. They are also called large tests.

These tests emulate the user behavior and assert UI results. These are the slowest and most expensive tests you can write if you run them on a device or emulator. While these are helpful for testing your interactions on screen, you should limit your UI tests. As in the pyramid diagram, you should perform your tests with unit and integration tests as much as you can.

For a test example of this layer, think of an app with a login screen. You’d like to check that, after logging in, the app shows a TextView welcoming the user.

So, the UI test could look like this:

fun shouldWelcomeUser_whenLogin() {
  onView(withId(R.id.username)).perform(typeText("Fernando"))
  onView(withId(R.id.password)).perform(typeText("password"))
  onView(withId(R.id.login_button)).perform(click())
  onView(withText("Hello Fernando!"))
    .check(matches(isDisplayed()))
}

On Android, a tool suggested by Google for UI testing is Espresso. You’ll write these kinds of test in later chapters. You could also use Robolectric (since version 4.0) for UI tests to run them without an emulator or a device. However, there are times that you need to run them on an emulator or device. These are called Android instrumentation tests.

There’s another tool called UI Automator. Google recommends it only when you have to do cross-app functional UI testing across system and installed apps.

Distributing the tests

A typical rule of thumb is to have the following ratio among the categories:

  • UI Tests: 10%

  • Integration Tests: 20%

  • Unit Tests: 70%

Google, in its testing fundamentals documentation, suggests these percentages. This doesn’t have to be absolutely exact, but it’s important to retain the pyramid shape. Remember that tests in the lower layers are easier to maintain and run faster. So you should avoid the following anti-patterns:

  • Ice cream cone or Inverted pyramid: The team is relying on lots of UI tests, having less integration tests and yet fewer unit tests. This type is commonly found in organizations that don’t have a testing culture, or are not encouraging developers to create tests. Usually, a QA team is responsible for testing and, in many cases, they don’t even have access to the code repository. In this case, if developers don’t create unit and integration tests, no one will. As a result, the QA team will try to compensate by creating tests of the upper layers, forming this anti-pattern.

  • Hourglass: You start writing lots of unit tests, you don’t care that much about integration tests and you write many UI tests.

Bear in mind that not following the correct pyramid shape could affect productivity. This is because the test suite will run slower, thus, taking a longer time to provide feedback to the developers.

Key points

  • Testing is commonly organized into the testing pyramid.
  • There are three kinds of tests in this pyramid: unit, integration and UI tests. These are also called small, medium and large tests, respectively.
  • On Android, you can also distinguish between local tests, which run on the JVM and instrumentation tests, which require a device or emulator. Local tests run faster than instrumented tests.
  • You’ll write tests of different granularity for different purposes.
  • The further down you get, the more focused and the more tests you need to write
  • Be mindful of how expensive the test is to perform.

Where to go from here?

In the following chapters, you’ll start doing TDD by writing each kind of test with the appropriate tools and libraries.

If you want to go deeper on this subject, check the following:

3. What Is TDD?
5. Unit Tests
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2024 Kodeco Inc.