Unit Testing on macOS: Part 2/2

In Part 1 of this Unit testing tutorial, you learned how to use Test Driven Development to test new code and how to add unit tests to existing code. In this part, you’ll learn how to test the user interface, how to test networking code and learn about a couple of Xcode tools to help your testing.

If you haven’t completed Part 1, or want a fresh start, download the completed project from Part 1 project here. This project uses Swift 3 and requires, at a minimum, Xcode 8 beta 6. Open it in Xcode and press Command-U to run all the tests to confirm that everything is working as expected.

Testing the Interface

As you saw in part 1, Xcode includes the ability to run UITests. While these can be useful, it is much faster to test views and view controllers programmatically. Instead of having Xcode run the app and send fake clicks to interface objects, get your tests to create a new instance of your view or view controller and work with that directly. You can get and set properties, call methods — including IBAction methods — and test the results much more quickly.

Select the High RollerTests group in the File Navigator and use File\New\File… to create a new macOS\Unit Test Case Class named ViewControllerTests. Delete all the code and add the following import to the top of the file:

@testable import High_Roller

Insert the following code into the ViewControllerTests class:

// 1
var vc: ViewController!
 
override func setUp() {
  super.setUp()
 
  // 2
  let storyboard = NSStoryboard(name: "Main", bundle: nil)
  vc = storyboard.instantiateController(withIdentifier: "ViewController") as! ViewController
 
  // 3
  _ = vc.view
}

So what’s going on here?

1. The entire class will have access to a ViewController property names vc. It’s OK to make this non-optional because if it crashes, that’s still a useful test result.
2. This view controller is instantiated from the storyboard setup().
3. To trigger the view lifecycle, get the view controller’s view property. You don’t need to store it; the act of getting it makes the view controller create it correctly.

Instantiating a view controller this way will only work if the view controller has a storyboard ID. Open Main.storyboard, select ViewController and show the Identity Inspector on the right. Set the Storyboard ID to ViewController.

The first test will confirm that the ViewController was created properly. Go back to ViewControllerTests.swift and add the following test function:

func testViewControllerIsCreated() {
  XCTAssertNotNil(vc)
}

Run the tests; if this test fails or crashes, go back to Main.storyboard and check that you set the storyboard ID correctly.

Build and run the app to have a look at the interface. All the controls are functional, so click the Roll button to roll the dice. Change the settings and roll again; notice that the number of dice can be set using a test field or a stepper and that you set the number of sides for the dice using a popup.

Before testing that the controls operate as expected, you first need to confirm the interface elements start off with the expected values.

Add this test to ViewControllerTests:

func testControlsHaveDefaultData() {
  XCTAssertEqual(vc.numberOfDiceTextField.stringValue, String(2))
  XCTAssertEqual(vc.numberOfDiceStepper.integerValue, 2)
  XCTAssertEqual(vc.numberOfSidesPopup.titleOfSelectedItem, String(6))
}

Run the tests to make sure the initial setup is valid.

Once that is confirmed, you can test what happens when you change the parameters through the interface. When you change the number in the text field, the value of the stepper should change to match and vice versa.

If you were using the app and clicked on the up or down arrows to change the stepper, the IBAction method numberOfDiceStepperChanged(_:) would be called automatically. Similarly, if you edited the text field, numberOfDiceTextFieldChanged(_:) would be called. When testing, you have to call the IBAction methods manually.

Insert the following two tests into ViewControllerTests:

func testChangingTextFieldChangesStepper() {
  vc.numberOfDiceTextField.stringValue = String(4)
  vc.numberOfDiceTextFieldChanged(vc.numberOfDiceTextField)
 
  XCTAssertEqual(vc.numberOfDiceTextField.stringValue, String(4))
  XCTAssertEqual(vc.numberOfDiceStepper.integerValue, 4)
}
 
func testChangingStepperChangesTextField() {
  vc.numberOfDiceStepper.integerValue = 10
  vc.numberOfDiceStepperChanged(vc.numberOfDiceStepper)
 
  XCTAssertEqual(vc.numberOfDiceTextField.stringValue, String(10))
  XCTAssertEqual(vc.numberOfDiceStepper.integerValue, 10)
}

Run the tests to see the result. You should also test the view controller’s variables and confirm that they are being changed as expected by events from the interface elements.

The view controller has a Roll object which has its own properties. Add the following test to check that the Roll object exists and has the expected default properties:

func testViewControllerHasRollObject() {
  XCTAssertNotNil(vc.roll)
}
 
func testRollHasDefaultSettings() {
  XCTAssertEqual(vc.roll.numberOfSides, 6)
  XCTAssertEqual(vc.roll.dice.count, 2)
}

Next, you need to confirm that changing a setting using one of the interface elements actually changes the setting in the Roll object. Add the following tests:

    func testChangingNumberOfDiceInTextFieldChangesRoll() {
      vc.numberOfDiceTextField.stringValue = String(4)
      vc.numberOfDiceTextFieldChanged(vc.numberOfDiceTextField)
 
      XCTAssertEqual(vc.roll.dice.count, 4)
    }
 
    func testChangingNumberOfDiceInStepperChangesRoll() {
      vc.numberOfDiceStepper.integerValue = 10
      vc.numberOfDiceStepperChanged(vc.numberOfDiceStepper)
 
      XCTAssertEqual(vc.roll.dice.count, 10)
    }
 
    func testChangingNumberOfSidesPopupChangesRoll() {
      vc.numberOfSidesPopup.selectItem(withTitle: "20")
      vc.numberOfSidesPopupChanged(vc.numberOfSidesPopup)
 
      XCTAssertEqual(vc.roll.numberOfSides, 20)
    }

These three tests operate the text field, the stepper and the popup. After each change, they check that the roll property has changed to match.

Open ViewController.swift in the assistant editor and look at rollButtonClicked(_:). It does three things:

1. Makes sure that any ongoing edit in the number of dice text field is processed.
2. Tells the Roll struct to roll all the dice.
3. Displays the results.

You have already written tests to confirm that rollAll() works as expected, but displayDiceFromRoll(diceRolls:numberOfSides:) needs to be tested as part of the interface tests. The display methods are all in ViewControllerDisplay.swift, which is a separate file containing an extension to ViewController. This is just an organizational split to keep ViewController.swift smaller and to keep the display functions all collected in one place.

Look in ViewControllerDisplay.swift and you’ll see a bunch of private functions and one public function: displayDiceFromRoll(diceRolls:numberOfSides:), This clears the display, fills in the textual information and then populates a stack view with a series of sub-views, one for each die.

As with all testing, it is important to start in the right place. The first test to write is one that checks that the results text view and stack view are empty at the start.

Go to ViewControllerTests.swift and add this test:

func testDisplayIsBlankAtStart() {
  XCTAssertEqual(vc.resultsTextView.string, "")
  XCTAssertEqual(vc.resultsStackView.views.count, 0)
}

Run this test to confirm that the display starts off as expected.

Next, add the test below to check if data appears after the Roll button is clicked:

func testDisplayIsFilledInAfterRoll() {
  vc.rollButtonClicked(vc.rollButton)
 
  XCTAssertNotEqual(vc.resultsTextView.string, "")
  XCTAssertEqual(vc.resultsStackView.views.count, 2)
}

Since the default setting for the number of dice is 2, it’s safe to check that the stack view has two views. But if you don’t know what the settings might be, you can’t test to see whether the data displayed is correct.

Look back at rollButtonClicked(_:) in ViewController.swift. See how it rolls the dice and then displays the result? What if you called displayDiceFromRoll(diceRolls:numberOfSides:) directly with known data? That would allow exact checking of the display.

Add the following test to ViewControllerTests.swift:

func testTextResultDisplayIsCorrect() {
  let testRolls = [1, 2, 3, 4, 5, 6]
  vc.displayDiceFromRoll(diceRolls: testRolls)
 
  var expectedText = "Total rolled: 21\n"
  expectedText += "Dice rolled: 1, 2, 3, 4, 5, 6 (6 x 6 sided dice)\n"
  expectedText += "You rolled: 1 x 1s,  1 x 2s,  1 x 3s,  1 x 4s,  1 x 5s,  1 x 6s"
 
  XCTAssertEqual(vc.resultsTextView.string, expectedText)
}

Run it to confirm that the test result is as expected for a roll with 6 six-sided dice showing one of each possible sides.

The stack view shows the results in a more graphical way using dice emojis if possible. Insert this test into ViewControllerTests.swift:

func testGraphicalResultDisplayIsCorrect() {
  let testRolls = [1, 2, 3, 4, 5, 6]
  vc.displayDiceFromRoll(diceRolls: testRolls)
 
  let diceEmojis = ["\u{2680}", "\u{2681}", "\u{2682}", "\u{2683}", "\u{2684}", "\u{2685}" ]
 
  XCTAssertEqual(vc.resultsStackView.views.count, 6)
 
  for (index, diceView) in vc.resultsStackView.views.enumerated() {
    guard let diceView = diceView as? NSTextField else {
      XCTFail("View \(index) is not NSTextField")
      return
    }
    let diceViewContent = diceView.stringValue
    XCTAssertEqual(diceViewContent, diceEmojis[index], "View \(index) is not showing the correct emoji.")
  }
}

Run the tests again to check that the interface is acting as you expect. These last two tests demonstrate a very useful technique for testing, by supplying known data to a method and checking the result.

If you’re feeling keen, it looks like there is some re-factoring that could be done here! :]

UI Testing

Time to move on to the UITests. Close the assistant editor and open High_RollerUITests.swift. The default code is very similar to the testing code you’ve been using so far, with just a couple of extra lines in setup().

One interesting thing about UITests is their ability to record interface interactions. Remove the comments from inside testExample(), place the cursor on a blank line inside the function and click the red dot at the bottom left of the edit pane to start recording:

When the app starts, follow this sequence of interactions, pausing after each step to let Xcode write at least one new line:

1. Click the up arrow on the “Number of dice” stepper.
2. Click the up arrow on the “Number of dice” stepper again.
3. Double-click inside the “Number of dice” text field.
4. Type 6 and press Tab.
5. Open the “Number of sides” popup and select 12.
6. Click the “Roll” button.

Click the record button again to stop recording.

Xcode will have filled in the function with details of your actions, but you will see one odd thing where you selected 12 from the popup. Xcode can’t quite decide what option to use, so it shows you a number of possibilities. In a complex interface, this might be important to distinguish between different controls, but in this case the first option is sufficient for your needs.

Click on the down arrow beside menuItems[“12”] to see the popup. Choosing the one to use is easy enough, but convincing Xcode of your choice is not so straightforward.

Select the first option in the list which will dismiss the popup. Then click on the item which will still have a pale blue background. When it’s selected, the background will have a slightly darker shade of blue; you can then press Return to accept this choice, which will leave your code looking like this:

  func testExample() {
 
    let highRollerWindow = XCUIApplication().windows["High Roller"]
    let incrementArrow = highRollerWindow.steppers.children(matching: .incrementArrow).element
    incrementArrow.click()
    incrementArrow.click()
 
    let textField = highRollerWindow.children(matching: .textField).element
    textField.doubleClick()
    textField.typeText("6\t")
    highRollerWindow.children(matching: .popUpButton).element.click()
    highRollerWindow.menuItems["12"].click()
    highRollerWindow.buttons["Roll"].click()
 
  }

The main use for recording is to show the syntax for accessing the interface elements. The unexpected thing is that you aren’t getting NSButton or NSTextField references; you’re getting XCUIElements instead. This gives you the ability to send messages and test a limited number of properties. value is an Any that will usually hold the most important content of the XCUIElement.

Using the information in the recording to work out how to access the elements, this test function checks to see that editing the number of dice using the stepper also changes the text field:

func testIncreasingNumberOfDice() {
  let highRollerWindow = XCUIApplication().windows["High Roller"]
 
  let incrementArrow = highRollerWindow.steppers.children(matching: .incrementArrow).element
  incrementArrow.click()
  incrementArrow.click()
 
  let textField = highRollerWindow.children(matching: .textField).element
  let textFieldValue = textField.value as? String
 
  XCTAssertEqual(textFieldValue, "4")
}

Save the file and run the test by clicking in the little diamond in the margin beside it. The app will run, the mouse pointer will be moved to the stepper’s up arrow and it will click twice. This is fun, like having a robot operate your app!

It’s a lot slower than the previous tests, though, so UITesting is not for everyday use.

Network and Asynchronous Tests

So far, everyone is happy. Family games night is going ahead, your role-playing friends have the option to roll all their weird dice, your tests prove that everything is working correctly…. but there is always someone who causes trouble:

“I still don’t trust your app to roll the dice. I found a web page that generates dice rolls using atmospheric noise. I want your app to use that instead.”

Sigh. Head to Random.org to see how this works. If the URL contains a num parameter, the page shows the results of rolling that many 6-sided dice. Inspecting the source code for the page, it looks like this is the relevant section:

<p>You rolled 2 dice:</p>
<p>
<img src="dice6.png" alt="6" />
<img src="dice1.png" alt="1" />
</p>

So you could parse the data returned and use that data for the roll. Check out WebSource.swift and you’ll see this is exactly what it does. But how do you test this?

The first thing is to make a WebSourceTests.swift test file. Select the High RollerTests group in the File Navigator and use File\New\File… to make a new macOS\Unit Test Case Class and name it WebSourceTests.

Delete the contents of the class and add the following import statement:

@testable import High_Roller

Open WebSource.swift in the assistant editor.

Look at findRollOnline(numberOfDice:completion:) in WebSource.swift. This function creates a URLRequest and a URLSession and then combines them into a URLSessionDataTask which tries to download the web page for the selected number of dice.

If data arrives, it parses the result and calls the completion handler with the dice results or an empty array.

As a first attempt at testing, try adding the following to WebSourceTests.swift:

func testDownloadingOnlineRollPage() {
  let webSource = WebSource()
  webSource.findRollOnline(numberOfDice: 2) { (result) in
    XCTAssertEqual(result.count, 2)
  }
}

When you run this test, it passes suspiciously fast. Click in the margin to add a breakpoint to the XCTAssertEqual() line.

Run the test again, and your breakpoint never gets triggered. The test is completing without waiting for the results to come back. This is a bit of a trap, as you could have erroneously assumed that the test passed. Never worry, XCTests has the solution to this: expectations!

Replace the previous test with this one:

func testDownloadingPageUsingExpectation() {
  // 1
  let expect = expectation(description: "waitForWebSource")
  var diceRollsReceived = 0
 
  let webSource = WebSource()
  webSource.findRollOnline(numberOfDice: 2) { (result) in
    diceRollsReceived = result.count
    // 2
    expect.fulfill()
  }
 
  // 3
  waitForExpectations(timeout: 10, handler: nil)
  XCTAssertEqual(diceRollsReceived, 2)
}

There are several new things to look at here:

1. Create an XCTestExpectation with a human-readable description.
2. When the closure is called after the data has been returned, fulfill this expectation by indicating whatever it’s been waiting for has now happened.
3. Set up a timeout for the test function to wait until the expectation has been fulfilled. In this case, if the web page hasn’t returned the data within 10 seconds, the expectation will timeout.

This time, put a breakpoint on the XCTAssertEqual() line, and it should trigger and the test will pass for real. If you want to see what happens when an expectation times out, set the timeout to something really small (0.1 works for me) and run the test again.

Now you know how to test asynchronously, which is really useful for network access and long background tasks. But what if you want to test your network code and you don’t have access to the internet, or the site is down, or you just want your tests to run faster?

In this case, you can use a testing technique called mocking to simulate your network call.

Mocking

In the real code, URLSession was used to start a URLSessionDataTask which returned the response. Since you don’t want to access the internet, you can test that the URLRequest is configured correctly, that the URLSessionDataTask is created and that the URLSessionDataTask is started.

You’re going to create mock versions of the classes involved: MockURLSession and MockURLSessionDataTask, which you can use instead of the real classes.

At the bottom of the WebSourcesTests.swift file, outside the WebSourceTests class, add the following two new classes:

class MockURLSession: URLSession {
  var url: URL?
  var dataTask = MockURLSessionTask()
 
  override func dataTask(with request: URLRequest, completionHandler: @escaping (Data?, URLResponse?, Error?) -> Void) -> MockURLSessionTask {
      self.url = request.url
      return dataTask
  }
}
 
class MockURLSessionTask: URLSessionDataTask {
  var resumeGotCalled = false
 
  override func resume() {
    resumeGotCalled = true
  }
}

MockURLSession sub-classes URLSession, supplying an alternative version of dataTask(with:completionHandler:) that stores the URL from the supplied URLRequest and returns a MockURLSessionTask instead of a URLSessionDataTask.

MockURLSessionTask sub-classes URLSessionDataTask and when resume() is called, does not go online but instead sets a flag to show that this has happened.

Add the following to the WebSourceTests class and run the new test:

func testUsingMockURLSession() {
  // 1
  let address = "https://www.random.org/dice/?num=2"
  guard let url = URL(string: address) else {
    XCTFail()
    return
  }
  let request = URLRequest(url: url)
 
  // 2
  let mockSession = MockURLSession()
  XCTAssertFalse(mockSession.dataTask.resumeGotCalled)
  XCTAssertNil(mockSession.url)
 
  // 3
  let task = mockSession.dataTask(with: request) { (data, response, error) in }
  task.resume()
 
  // 4
  XCTAssertTrue(mockSession.dataTask.resumeGotCalled)
  XCTAssertEqual(mockSession.url, url)
}

What’s going on in this test?

1. Construct the URLRequest as before.
2. Create a MockURLSession and confirm the initial properties.
3. Create the MockURLSessionTask and call resume().
4. Test that the properties have changed as expected.

This test checks the first part of the process: the URLRequest, the URLSession and the URLSessionDataTask, and it tests that the data task is started. What is missing is any test for parsing the returned data.

There are two test cases you need to cover here: if the data returns matches the expected format, and if it does not.

Add these two tests to WebSourcesTests.swift and run them:

func testParsingGoodData() {
  let webSource = WebSource()
  let goodDataString = "<p>You rolled 2 dice:</p>\n<p>\n<img src=\"dice6.png\" alt=\"6\" />\n<img src=\"dice1.png\" alt=\"1\" />\n</p>"
  guard let goodData = goodDataString.data(using: .utf8) else {
    XCTFail()
    return
  }
 
  let diceArray = webSource.parseIncomingData(data: goodData)
  XCTAssertEqual(diceArray, [6, 1])
}
 
func testParsingBadData() {
  let webSource = WebSource()
  let badDataString = "This string is not the expected result"
  guard let badData = badDataString.data(using: .utf8) else {
    XCTFail()
    return
  }
 
  let diceArray = webSource.parseIncomingData(data: badData)
  XCTAssertEqual(diceArray, [])
}

Here you have used expectations to test the network connection, mocking to simulate the networking to allow tests independent of the network and a third-party web site, and finally supplied data to test the data parsing, again independently.

Performance Testing

Xcode also offers performance testing to check how fast your code executes. In Roll.swift, totalForDice() uses flatMap and reduce to calculate the total for the dice, allowing for the fact that value is an optional. But is this the fastest approach?

To test performance, select the High RollerTests group in the File Navigator and use File\New\File… to create a new macOS\Unit Test Case Class named PerformanceTests.

Delete the contents of the class and — you guessed it — add the following import as you’ve done before:

@testable import High_Roller

Insert this test function:

  func testPerformanceTotalForDice_FlatMap_Reduce() {
    // 1
    var roll = Roll()
    roll.changeNumberOfDice(newDiceCount: 20)
    roll.rollAll()
 
    // 2
    self.measure {
      // 3
      _ = roll.totalForDice()
    }
  }

The sections of this function are as follows:

1. Set up a Roll with 20 Dice.
2. self.measure defines the timing block.
3. This is the code being measured.

Run the test and you will see a result like this:

As well as getting the green checkmark symbol, you will see a speed indicator which in my test shows “Time: 0.000 sec (98% STDEV)”. The standard deviation (STDEV) will indicate if there are any significant changes from the previous results. In this case, there is only one result — zero — so STDEV is meaningless. Also meaningless is a result of 0.000 seconds, so the test needs to be longer. The easiest way to do this is to add a loop that repeats the measure block enough times to get an actual time.

Replace the test with the following:

  func testPerformanceTotalForDice_FlatMap_Reduce() {
    var roll = Roll()
    roll.changeNumberOfDice(newDiceCount: 20)
    roll.rollAll()
 
    self.measure {
      for _ in 0 ..< 10_000 {
        _ = roll.totalForDice()
      }
    }
  }

Run the test again; the result you get will depend on your processor, but I get about 0.2 seconds. Adjust the loop counter from 10_000 until you get around 0.2.

Here are three other possible ways of adding up the total of the dice. Open Roll.swift in the assistant editor and add them as follows:

  func totalForDice2() -> Int {
    let total = dice
      .filter { $0.value != nil }
      .reduce(0) { $0 + $1.value! }
    return total
  }
 
  func totalForDice3() -> Int {
    let total = dice
      .reduce(0) { $0 + ($1.value ?? 0) }
    return total
  }
 
  func totalForDice4() -> Int {
    var total = 0
    for d in dice {
      if let dieValue = d.value {
        total += dieValue
      }
    }
    return total
  }

And here are the matching performance tests which you should add to PerformanceTests.swift:

  func testPerformanceTotalForDice2_Filter_Reduce() {
    var roll = Roll()
    roll.changeNumberOfDice(newDiceCount: 20)
    roll.rollAll()
 
    self.measure {
      for _ in 0 ..< 10_000 {
        _ = roll.totalForDice2()
      }
    }
  }
 
  func testPerformanceTotalForDice3_Reduce() {
    var roll = Roll()
    roll.changeNumberOfDice(newDiceCount: 20)
    roll.rollAll()
 
    self.measure {
      for _ in 0 ..< 10_000 {
        _ = roll.totalForDice3()
      }
    }
  }
 
  func testPerformanceTotalForDice4_Old_Style() {
    var roll = Roll()
    roll.changeNumberOfDice(newDiceCount: 20)
    roll.rollAll()
 
    self.measure {
      for _ in 0 ..< 10_000 {
        _ = roll.totalForDice4()
      }
    }
  }

Run these tests and work out which option is the fastest. Did you guess which one would win? I didn’t!

Code Coverage

The final Xcode test tool to discuss is code coverage, which is the measure of how much of your code is covered during a series of tests. It’s turned off by default. To turn it on, select Edit Scheme… in the schemes popup at the top of the window. Select Test in the column on the left and then check Gather coverage data.

Close that window and press Command-U to re-run all the tests. Once the tests are complete, go to the Report Navigator and select the latest entry.

You’ll see the test report showing a series of green checkmarks, plus some timings for the performance tests. If you don’t see this, make sure both All toggles are selected at the top left.

Click on Coverage at the top of this display and mouse over the top of the blue bars to see that your tests cover nearly 80% of your code. Amazing work! :]

The two model objects (Dice and Roll) are very well covered. If you are only going to add some tests, the model is the best place to start.

There is another good, fast way to improve code coverage: delete code that isn’t being used. Look at the coverage for AppDelegate.swift, it’s at 50%.

Go to the AppDelegate.swift file. On the gutter on the right-hand side, mouse up and down and you’ll see it shows green for methods called during the tests, and red for methods that are not called.

In this case, applicationWillTerminate(_:) is not used at all; it’s dramatically decreasing the code coverage for this file. Since the app is not using this function, delete it. Now run all the tests again and AppDelegate.swift has jumped to 100% coverage.

This may seem to be cheating the system, but it is actually good practice to remove any dead code that is cluttering up your app. Xcode tries to be helpful when you make a new file and supplies lots of boilerplate code by default, but if you don’t need any of this, delete it.

Now for the warning about code coverage: it is a tool, not a goal! Some developers and employers treat it as a goal and insist on a certain percentage. But it is possible to get a good percentage without testing meaningfully. Tests have to be well thought out and not just added for the sake of increasing your code coverage.

Tests may call numerous functions in your code without actually checking the result. While a high code coverage number is probably better than a low one, it doesn’t say anything about the quality of the tests.

Where to Go From Here?

You can download the final version of the sample project here.

Apple has a set of docs about Testing with Xcode with links to relevant WWDC videos.

NSHipster has a useful summary of the various assertions and what you really need to know to write tests.

For information about Test Driven Development, check out Uncle Bob’s excellent site.

Interested in learning more about UITests? Check out the Joe Masilotti’s excellent cheat sheet.

I hope you enjoyed this Unit testing tutorial; if you have any questions or comments, please join the forum discussion below!

The post Unit Testing on macOS: Part 2/2 appeared first on Ray Wenderlich.