What’s New in Testing With Xcode 12

Adding Unit Tests

You wrote a couple of UI tests. Now it’s time to add a couple of unit tests.

Go to ScattergoriesTests.swift and add these two tests:

func testCategories() throws {
  let gameData = GameData()
  XCTAssertEqual(
    gameData.categories.count, 
    numOfCategories, 
    "The game must have \(numOfCategories) categories")
}

func testPlayerData() throws {
  let gameData = GameData()

  // TODO: Implement logic to test player words 
  //   (depending on actual game implementation)
  XCTFail("Player testing logic not implemented yet")
}

Add this to the import section at the top of the file:

@testable import Scattergories

Also, be sure to add the numOfCategories constant as the first line of the ScattergoriesTests class implementation:

let numOfCategories: Int = 14

These are two simple tests to verify your basic game logic is working as expected. testCategories() verifies your game data has exactly 14 categories. This is set as a constant in case the number of categories changes in the future. In this case, you don’t want to go over all of your tests, but rather change a constant to affect all of them.

The second test focuses on the game logic. Even though you haven’t implemented the logic yet, you want to have a test, or at least a test stub, ready for it so:

• You’ll remember this needs to be properly implemented and tested, and,
• Decrease your chances of matching the tests to the code you’ve already written, but rather to the expected behavior.

Run your newly added unit tests by clicking Play for the test suite:

You’ll see the first test passes. The second one fails, which makes sense because you haven’t implemented the game logic yet, and there’s an explicit XCTFail call there.

On one hand, you want to adhere to test-driven development, or TDD, principles and write your tests early in the process. On the other hand, you don’t want to have red tests you can’t address yet. This can create blindness and prevent you from noticing real red tests.

So what can you do? XCTSkip to the rescue.

XCTSkip

Apple introduced XCTSkip in Xcode 11.4. It provides a new possible test result. Now, instead of pass or fail, tests can also have the result skip.

An annotation in the source code shows you where and why the test was skipped. The result also appears as an indication next to the test, like pass and fail results. You can even filter the report to show only the skipped tests, or passed or failed ones.

Using XCTSkip resolves the conflict of whether to implement a test, and then possibly forget about it, or have a failed test as part of the test run, which may desensitize you to red tests and prevent you from noticing real reds.

XCTSkip lets you enjoy both worlds. While you can still see the test didn’t pass, it won’t be red and therefore won’t distract from anything that may need your immediate attention. You can also document why the test isn’t running using an optional message.

You’ll usually use XCTSkip for skipping tests that aren’t relevant for the platform or OS the test is currently running on, or for cases when you haven’t fully implemented the functionality yet. Sometimes, you may have a test that started failing but you can’t fix for a while. Skipping such tests prevents you from having to keep triaging this failure over and over, while at the same time ensuring you won’t lose track of the test by disabling it.

To use XCTSkip on the test for the not-yet-implemented game logic, add the following line to testPlayerData(), right above the // TODO: line:

try XCTSkipUnless(
  !gameData.playerWords.isEmpty,
  "The player must have more than 0 words")

Now, once your player has more than 0 words, which means you got to a point where your game logic is at least partially implemented, the test will run. Until then, the test will skip.

Run your unit tests and you’ll see the test is indeed skipped, preventing you from having to deal with a red test you can’t do anything about yet.

Testing Dos and Don’ts

Here are some useful tips that can make your testing experience with Xcode better and more efficient:

• Each test should have one specific goal in mind. The test name should reflect this goal for easy triaging after a failed or successful run.
• Command-U runs all of the tests on your current test plan. Control-Option-Command-G repeats the previous test run.
• Use enums for all string values because UI elements labels change often. Doing so will make it much easier to fix tests to match changed UI.
• Factor common code into helper methods. This is especially true given Xcode 12’s enhanced error handling features, such as including call stacks for each test failure. These features will help you focus on fine-tuning your test logic.
• Always throw from shared code rather than asserting. Shared code is run from multiple tests and, in some cases, you may be purposefully testing negative cases.
• Don’t add file paths to an assert comment. Instead, add any information needed to triage or debug the issue as an attachment. It’ll be much easier to triage the failure later.
• Unwrap optionals. When there’s a failure to unwrap optionals in a continuous integration bundle, you get a message saying test crashed with signal ill.
• Parallel distributed testing isn’t the same as parallel destination testing, which simultaneously runs your tests on more OS versions and devices and has been available since 2018. Use both methods wisely.

Where to Go From Here?

Use the Download Materials button at the top or bottom of this tutorial to download the final Scattergories app.

Now that you’ve had a chance to experiment with some of Xcode 12’s new testing enhancements, you can use this new found knowledge to add tests to improve your own app. Take a look at some WWDC’20 sessions, discussing these capabilities:

• Triage test failures with XCTIssue
• Get your tests results faster
• Write tests to fail
• XCTSkip your tests

And, of course, there is also great testing-related content right here on raywenderlich.com. Specifically:

• iOS Test-Driven Development by Tutorials
• Testing in iOS video course
• What’s new in Xcode 11: Testing This screencast is useful if you want to learn more about test plans and how to use them.

I encourage you to use Xcode 12’s new testing enhancements to find new and creative ways to get your app to fail, crash, and burn so you can fix everything before it reaches real users. If you have any questions or comments, please join the forum below.