Behavior-Driven Testing Tutorial for iOS with Quick & Nimble

Step in and start using Quick and Nimble!

Writing beautiful, performant applications is one thing, but writing good tests that verify your application’s expected behavior is a much harder task. In this tutorial, you’ll examine one of the available approaches for testing applications, called behavior-driven testing, using two extremely popular frameworks named Quick and Nimble.

You’ll learn about behavior-driven testing: what it is, why it’s an extremely powerful concept, and how easy it is to write maintainable and readable tests using Quick and Nimble.

You’ll be writing tests for an amazingly simple and fun game called AppTacToe in which you play a game of Tic Tac Toe vs. the computer, portraying the iOS character playing against the evil Android player!

Getting Started

The best way to get started testing is to work on a real app, which in your case will be the beautiful AppTacToe game introduced earlier.

Use the Download Materials button at the top or bottom of this tutorial to download the starter project, which already has Quick and Nimble bundled, and open AppTacToe.xcworkspace.

Open Main.storyboard and examine the basic structure of the app. It contains two screens: the Board itself, where the game is played, and the Game Over screen, responsible for displaying the game’s result.

Build and run the app, and play a quick game or two to get the hang of it.

You’ll also see some useful logs printed to your console, portraying the game play and printing out the resulting board as the game ends.

Most of the app’s logic is contained in one of two files:

• Components/Board.swift: This file provides the logical implementation of a Tic Tac Toe game. It has no UI associated with it.
• ViewControllers/BoardViewController.swift: This is the main game screen. It uses the aforementioned Board class to play the game itself, and is solely responsible for drawing the state of the game on screen and handling user interaction with the game.

What you really want to test in this case is the the logic of the game, so you’ll be writing tests for the Board class.

What is Behavior-Driven Testing?

An application is comprised of many pieces of code. In traditional unit tests, you test the ins-and-outs of every one of these pieces. You provide some inputs to some piece of code, and assert that it returns the result you expect.

A downside of this approach is that it emphasizes testing the inner workings of your applications. This means you spend more time testing implementation details then actual business logic, which is the real meat of your product!

Wouldn’t it be nice if you could simply verify your application behaves as expected, regardless of how it was implemented?

Enter behavior-driven testing!

Behavior Driven Tests vs. Unit Tests

In behavior-driven testing (or BDT), your tests are based on user stories that describe some specific expected behavior of your application. Instead of testing implementation details, you’re actually testing what matters: does your app deliver your user stories correctly?

This approach makes tests extremely readable and maintainable, and helps describe the behavior of logical portions in your application to other developers who might be lucky enough to go through your beautiful code one day.

Some examples of user stories you might end up writing as part of the AppTacToe game might be:

• Playing a single move should switch to other player.
• Playing two moves should switch back to the first player.
• Playing a winning move should switch to a Won state.
• Playing a move leaving no remaining moves should switch to a Draw (Tie) state.

Quick and Nimble’s role in Behavior-Driven Testing

Tests written in a behavior-driven way are based on user stories, which are regular sentences, written in plain English. This makes them much easier to understand when compared to the usual unit tests you’re accustomed to writing.

Quick and Nimble provide an extremely powerful syntax that let you write tests that read exactly like regular sentences, allowing you to easily and swiftly describe the behavior you wish to verify. Beneath the surface, they work exactly like regular XCTestCase(s).

Quick itself provides most of the basic syntax and abilities related to writing tests in a behavior-driven way, while Nimble is its companion framework. It provides additional expressive matching and assertion abilities via Matchers, which you’ll learn about a bit later in this tutorial.

The Anatomy of a Quick Test

Break up one of the user stories into three clauses based on GWT – Given (the action/behavior you’re describing), When (the context of that action/behavior) and Then (what you expect to happen):

• Given: User is playing.
• When: It is a single move.
• Then: The game should switch to other player.

In Quick, you use three functions as the counterparts of each: describe, context and it.

Anatomy of a Quick test

Your First Test

In Quick, test suites are named Specs, and every test suite you create starts off with a class inheriting from QuickSpec in the same way you inherit from XCTestCase in non-Quick tests. The test suite includes a main method, spec(), that contains the entirety of your test cases.

The starter project already contains an empty test suite. Open AppTacToeTests/BoardSpec.swift and take a look at the BoardSpec test spec, inheriting from QuickSpec and containing a single method, spec(), in which you’ll be writing test cases and expectations.

Start by adding the following code inside spec():

var board: Board! // 1

beforeEach { // 2
  board = Board()
}

This code performs two actions:

1. Defines a global board to be used across test cases.
2. Resets that board to a new instance of Board before every test using Quick’s beforeEach closure.

With some basic boilerplate out of the way, you can start writing your very first test!

For the purposes of this app, the game will always start with Cross (e.g. X), and the opponent will be Nought (e.g. O).

Let’s start with the first user story mentioned above: playing a single move should switch to nought.

Add the following code immediately after the end of the beforeEach closure:

describe("playing") { // 1
  context("a single move") { // 2
    it("should switch to nought") { // 3
      try! board.playRandom() // 4
      expect(board.state).to(equal(.playing(.nought))) // 5
    }
  }
}

Here’s what this does, line by line:

1. describe() is used to define what action or behavior you’ll be testing.
2. context() is used to define the specific context of the action you’ll be testing.
3. it() is used to define the specific expected result for the test.
4. You play a random move on the Board class using playRandom().
5. You assert the board’s state has been changed to .playing(.nought). This step uses the equal() matcher from Nimble, which is one of many available functions you can use to assert a matching of specific conditions on an expected value.

Run your tests by either navigating in the menu bar to Product ▸ Test or by using the Command-U shortcut.

You’ll see your very first test pass. Awesome!

Your Test navigator will look like this:

You can already notice a few interesting points by going through this code. First of all, it is extremely readable. Going through the lines of code, any person could relatively easily read it as a plain English sentence:

“Playing a single move should switch to nought. Play a random move and expect the board’s state to equal nought playing.”

You were also just introduced to your first usage of Nimble Matchers. Nimble uses these Matchers to let you express the expected outcome of your test in a very fluent, sentence-like way. equal() is just one of the matcher functions available in Nimble and, as you’ll see shortly, you can even create your own custom ones.

Your Second Test

The second user story — “playing two moves should switch back to cross” — sounds fairly close to the first.

Add the following code right after the end of your previous context(), inside the closing curly brace of describe():

context("two moves") { // 1
  it("should switch back to cross") {
    try! board.playRandom() // 2
    try! board.playRandom()
    expect(board.state) == .playing(.cross) // 3
  }
}

This test is similar to the last one, different in the fact you’re playing two moves instead of one.

Here’s what it does:

1. You define a new context() to establish the “two moves” context. You can have as many context()s and describe()s as you want, and they can even be contained within each other. Since you’re still testing gameplay, you added a context inside describe("playing").
2. You play two consecutive random moves.
3. You assert the board’s state is now .playing(.cross). Notice that this time you used the regular equality operator ==, instead of the .to(equal()) syntax you used earlier. Nimble’s equal() matcher provides its own operator overloads that let you choose your own flavor/preference.

Arrange, Act & Assert

The tests you’ve just written have been relatively simple and straightforward. You perform a single call on an empty Board, and assert the expected result. Usually, though, most scenarios are more complex, thus requiring a bit of extra work.

The next two user stories are more complex:

• Playing a winning move should switch to the won state.
• Playing a move leaving no remaining moves should switch to the draw state.

In both of these user stories, you need to play some moves on the board to get it into a state where you can test your assertion.

These kind of tests are usually divided into three steps: Arrange, Act and Assert.

Before you plan your tests, you must understand how the Tic Tac Toe board is implemented under the hood.

The board is modeled as an Array consisting of 9 cells, addressed using indices 0 through 8.

On each turn, a player plays a single move. To write a test for the winning user story, you’ll need to play both moves to bring the board to a state where the next move would be a winning move.

Now that you understand how the board works, it’s time to write the winning test.

Add the following code below your previous “two moves” context():

context("a winning move") {
  it("should switch to won state") {
    // Arrange
    try! board.play(at: 0)
    try! board.play(at: 1)
    try! board.play(at: 3)
    try! board.play(at: 2)

    // Act
    try! board.play(at: 6)

    // Assert
    expect(board.state) == .won(.cross)
  }
}

Here’s what this does:

• Arrange: You arrange the board to bring it to a state where the next move would be a winning move. You do this by playing the moves of both players at their turn; starting with X at 0, O at 1, X at 3 and finally O at 2.
• Act: You play Cross (X) at position 6. In the board’s current state, playing at position 6 should cause a winning state.
• Assert: You assert the board’s state to be equal to won by cross (e.g. .won(.cross))

Run the test suite again by going to Product ▸ Test or using the Command + U shortcut.

Something is wrong; you played all of the right moves, but the test failed unexpectedly.

Add the following code immediately below the expect() code line to see what went wrong:

print(board)

By printing the board immediately after the assertion you will get better clarity of the situation:

As you can see, the board should be in a winning state, but the test is still failing. Seems like you found a bug.

Switch to the Project navigator and open Board.swift. Go to the isGameWon computed property on line 120.

The code in this section tests for all possible winning positions across rows, columns and diagonals. But looking at the columns section, the code seems to only have 2 columns tested, and is actually missing one of the winning options. Whoops!

Add the following line of code immediately below the // Columns comment:

[0, 3, 6],

Run your test suite again and bask in the glory of three green checkmarks!

This kind of scenario would be much harder to detect with regular unit tests. Since you’re using behavior-driven testing, you actually tested a specific use case of the app and detected a fault. Fixing the underlying implementation fixed the tested behavior, resolving the issue your user story was experiencing.

A Short Exercise

Time for a challenge. You have one last user story you haven’t tested yet: “Playing a move leaving no remaining moves should switch to draw state”

Using the previous examples, write a test to verify the board correctly detects a Draw state.

Tap the button below to see the full solution.

context("a move leaving no remaining moves") {
  it("should switch to draw state") {
    // Arrange
    try! board.play(at: 0)
    try! board.play(at: 2)
    try! board.play(at: 1)
    try! board.play(at: 3)
    try! board.play(at: 4)
    try! board.play(at: 8)
    try! board.play(at: 6)
    try! board.play(at: 7)

    // Act
    try! board.play(at: 5)

    // Assert
    expect(board.state) == Board.State.draw
  }
}

Happy Path Isn’t The Only Path

All of the tests you’ve written up until now have one thing in common: they describe the correct behavior of your app while following the happy path. You verified that when the player plays the correct moves, the board behaves correctly. But what about the not-so-happy path?

When writing tests, you mustn’t forget the concept of expected errors. You, as a developer, should have the ability to confirm your board behaves correctly even when your player doesn’t (e.g. makes an illegal move).

Consider the two final user stories for this tutorial:

• Playing a move that was already played should throw an error.
• Playing a move when the game is already won should throw an error.

Nimble provides a handy matcher named throwError() you can use to verify this behavior.

Start with verifying that an already played move can’t be played again.

Add the following code right below the last context() you’ve added, while still inside the describe("playing") block:

context("a move that was already played") {
  it("should throw an error") {
    try! board.play(at: 0) // 1

    // 2
    expect { try board.play(at: 0) }
      .to(throwError(Board.PlayError.alreadyPlayed))
  }
}

Here’s what this does:

1. You play a move at position 0.
2. You play a move at the same position, and expect it to throw Board.PlayerError.alreadyPlayed. When asserting error throwing, expect takes a closure, in which you can run the code that causes the error to be thrown.

As you have come to expect from Quick tests, the assertion reads much like an English sentence: expect playing the board to throw error – already played.

Run the test suite again by going to Product ▸ Test or using the Command + U shortcut.

The last user story you’re going to cover today will be: Playing a move when the game is already won should throw an error.

This test should feel relatively similar to the previous Arrange, Act and Assert tests: you’ll need to bring the board to a winning state, and then try to play another move while the board has already been won.

Add the following code right below the last context() you added for the previous test:

context("a move while the game was already won") {
  it("should throw an error") {
    // Arrange
    try! board.play(at: 0)
    try! board.play(at: 1)
    try! board.play(at: 3)
    try! board.play(at: 2)
    try! board.play(at: 6)

    // Act & Assert
    expect { try board.play(at: 7) }
      .to(throwError(Board.PlayError.noGame))
  }
}

Building on the knowledge you’ve gained in this tutorial, you should be feeling right at home with this test!

You Arrange the board by playing five moves that cause the board to be in a winning state (e.g. .won(.cross)). You then Act & Assert by trying to play a move while the board was already in a winning state, and expect Board.PlayError.noGame to be thrown.

Run your test suite one more time, and give yourself a pat on the back for all those great tests!

Custom Matchers

While writing your tests in this tutorial, you’ve already used several matchers built into Nimble: equal() (and its == operator overload), and .throwError().

Sometimes, you want to create your very own matchers, either to encapsulate some complex form of matching, or to increase the readability of some of your existing tests.

Consider how you might improve the readability of the “playing a winning move should switch to won state” user story mentioned earlier:

expect(board.state) == .won(.cross)

Reword this statement as an English sentence: expect board to be won by cross. Then the test can look like this:

expect(board).to(beWon(by: .cross))

Matchers in Nimble are nothing more than simple functions that return Predicate<T>, where the generic T is the type you compare against. In your case, T will be of type Board.

In Project navigator, right click the AppTacToeTests folder and select New File. Select Swift File and click Next. Name your file Board+Nimble.swift. Confirm that you correctly set the file as a member of your AppTacToeTests target:

Replace the default import Foundation with the following three imports:

import Quick
import Nimble
@testable import AppTacToe

This simply imports Quick and Nimble, and also imports your main target so you can use Board within your matcher.

As mentioned earlier, a Matcher is a simple function returning a Predicate of type Board.

Add the following base of your matcher below your imports:

func beWon(by: Board.Mark) -> Predicate<Board> {
  return Predicate { expression in
    // Error! ...your custom predicate implementation goes here
  }
}

This code defines the beWon(by:) matcher that returns a Predicate<Board>, so it correctly matches against Board.

Inside of your function, you return a new Predicate instance, passing it a closure with a single argument — expression — which is the value or expression you match against. The closure must return a PredicateResult.

At this point you’ll see a build error, since you haven’t yet returned a result. You’ll fix that next.

To generate a PredicateResult, you must consider the following cases:

How the beWon(by:) matcher works

Add the following code inside of your Predicate‘s closure, replacing the comment, // Error!:

// 1
guard let board = try expression.evaluate() else {
  return PredicateResult(status: .fail,
                         message: .fail("failed evaluating expression"))
}

// 2
guard board.state == .won(by) else {
  return PredicateResult(status: .fail,
                         message: .expectedCustomValueTo("be Won by \(by)", "\(board.state)"))
}

// 3
return PredicateResult(status: .matches,
                       message: .expectedTo("expectation fulfilled"))

This predicate implementation might seem confusing at first, but it is quite simple if you take it step-by-step:

1. You try to evaluate the expression passed to expect(). In this case, the expression is the board itself. If the evaluation fails, you return a failing PredicateResult with a proper message.
2. You confirm the board’s state is equal to .won(by), where by is the argument passed to the Matcher function. If the state doesn’t match, you return a failing PredicateResult with an .expectedCustomValueTo message.
3. Finally, if everything looks good and verified, you return a successful PredicateResult.

That’s it! Open BoardSpec.swift and replace the following line:

expect(board.state) == .won(.cross)

With your new matcher:

expect(board).to(beWon(by: .cross))

Run your tests one final time by navigating to Product ▸ Test or using the Command-U shortcut. You should see all of your tests still pass, but this time with your brand new custom Matcher!

Where To Go From Here?

You can download the completed version of the project using the Download Materials button at the top or bottom of this tutorial.

You now have the knowledge you need to start applying behavior-driven testing to your app.

You’ve learned all about testing user stories, instead of testing implementation details, and how Quick helps achieve just that. You’ve also learned about Nimble matchers, and even written your very own matcher. Very exciting!

To get started with Quick and Nimble in your own project, start off by following the Installation guide and choose the installation method that works for your project.

When you have everything set up and you want to learn more about Quick, the best place to continue your reading is Quick’s Official Documentation. You might also want to read Nimble’s Readme to discover the vast number of available matchers and abilities it provides.

In the meantime, if you have any questions or comments about this tutorial or behavior-driven tests in general, please join the forum discussion below!

The post Behavior-Driven Testing Tutorial for iOS with Quick & Nimble appeared first on Ray Wenderlich.