Swift Asynchronous Unit Testing with Busy Assertion Pattern

All of us have written asynchronous Swift code and know how hard it is to get it right. Testing such code is no easier. To make it possible, we must adopt special patterns. While we’ve already touched on the subject in Unit Testing Asynchronous Code in Swift, in this article lets add one more technique to your Swift async testing toolbox.

Problem Statement

Since Swift unit tests are synchronous, they finish earlier than the asynchronous code, and give false result. In order to test asynchronous function calls and operations, we must write more complex test cases, so that they do not end prematurely.

The conventional way of doing this is by means of XCTestExpectation, which is a part of XCTest framework. However, there is a problem with this approach: it is cumbersome to write and read. It requires at least 4 steps: create expectation, fulfill it, wait for it, assert on the result. What if I tell you that this can be done in just 1 line of code? The solution is Busy Assertion pattern.

Busy Assertion Pattern

Inspired by busy-waiting, which I applied during my C programming days, and asynchronous expectations from Nimble, I’ve came up with an idea of Busy Assertion.

Quick and Nimble are popular Swift unit testing frameworks.

Busy Assertion makes expectations on values that are updated asynchronously. The idea is to re-evaluate a value until the condition holds or the timeout is reached. If the condition appears to be true, the expectation passes. If after being constantly checked, say for a second, the condition does not hold, the expectation fails.

Let’s see how it applies in practice.

Waiting for Value to Update

One common pattern of busy asserting is waiting for a value to update asynchronously.

func testWithoutBusyAssertion() {
    var oneTwo: [Int] = []
    DispatchQueue.global().asyncAfter(deadline: .now() + 0.1) {
        oneTwo.append(1)
        oneTwo.append(2)
    }
    XCTAssertEqual(oneTwo, [1, 2])
}

This test fails, since the array is empty at the time it is evaluated. XCTest will print:

XCTAssertEqual failed: ("[]") is not equal to ("[1, 2]")

Let’s busy assert on the oneTwo array:

func testUsingBusyAssertion() {
    var oneTwo: [Int] = []
    DispatchQueue.global().async {
        oneTwo.append(1)
        oneTwo.append(2)
    }
    expectToEventually(oneTwo == [1, 2])
}

The method expectToEventually re-evaluates oneTwo until it becomes equal to [1, 2] or the time runs out (whatever happens first). This makes the test pass.

Chaining Asynchronous Operations

Busy Assertion can be used to chain asynchronous operations.

func testOneTwoChained() {
    var oneTwo: [Int] = []
    
    DispatchQueue.global().asyncAfter(deadline: .now() + 0.1) {
        oneTwo.append(1)
    }
    expectToEventually(oneTwo == [1])

    DispatchQueue.global().asyncAfter(deadline: .now() + 0.1) {
        oneTwo.append(2)
    }
    expectToEventually(oneTwo == [1, 2])
}

Until the condition in expectToEventually fulfills or the timeout reached, it prevents the rest of the test from running. This allows to chain asynchronous operations and verify them one-by-one.

Implementing Busy Assertion

extension XCTest {
    func expectToEventually(_ test: @autoclosure () -> Bool, timeout: TimeInterval = 1.0, message: String = "") {
        let runLoop = RunLoop.current
        let timeoutDate = Date(timeIntervalSinceNow: timeout)
        repeat {
            // 1
            if test() {
                return
            }
            // 2
            runLoop.run(until: Date(timeIntervalSinceNow: 0.01))
        } while Date().compare(timeoutDate) == .orderedAscending // 3
        // 4
        XCTFail(message)
    }
}

1. Exit once the condition becomes true.
2. Wait for 0.01 seconds, while the run loop will be serving other events. After that the run loop will execute one more iteration of the cycle.
3. Stop the repeat-while loop once we reach the timeout.
4. The test is considered failed if we reach the timeout.

Inverted Expectations

Another common task is to verify something not to happen. For this purpose let’s use another method expect(_:for:message:). It verifies that a condition fulfills during given time period. If the condition is evaluated to be false, the test considered failed. Otherwise it passes.

Let’s see how it applies in practice. This time we test Scheduler that runs a closure every fixed period of time. Our test verifies that the closure is not called immediately on start.

class Scheduler: NSObject {
    // Init and deinit methods, which are not relevant to our example
    
    func start() {
        timer?.invalidate()
        timer = Timer.scheduledTimer(timeInterval: interval, target: self, selector: #selector(ping), userInfo: nil, repeats: true)
        timer?.fire()
    }
    
    func stop() {
        timer?.invalidate()
    }
    
    @objc private func ping() {
        onFire()
    }
}

Once you inspect the start() method, you’ll see that it does not meet our assumption. This is exactly what we want to catch in our test.

class SchedulerTests: XCTestCase {

    func testDoesNotFireImmediately() {
        // 1
        var didFire = false

        // 2
        let sut = Scheduler(interval: 100) {
            didFire = true
        }
        
        sut.start()
        
        // 3
        expect(didFire == false, for: 1.0)
    }
}

1. didFire represents an assumption that we want to test.
2. Once scheduler fires the callback, we set didFire to true. Set the interval to 100 seconds to emphasize that this is not what we are expecting.
3. After calling star(), we expect didFire to remain false for the whole second.

And the test indeed fails, since Scheduler fires the timer in the start() method.

Implementing Inverted Busy Assertion

extension XCTest {
    func expect(_ test: @autoclosure () -> Bool, for duration: TimeInterval, message: String = "") {
        let runLoop = RunLoop.current
        let timeoutDate = Date(timeIntervalSinceNow: duration)
        repeat {
            if !test() {
                XCTFail(message)
                return
            }
            runLoop.run(until: Date(timeIntervalSinceNow: 0.01))
        } while Date().compare(timeoutDate) == .orderedAscending
    }
}

The primary that has changed, compared to expectToEventually method, is how we verify the test closure.

Source Code

Here is a sample project with all the source code for this article with both busy assertion methods and several test examples. The code is slightly modified code to remove duplication. Shortly I am going to distribute it as a library.

Further reading

If you enjoyed this article, you might find interesting my other posts on the subject:

• Real-World Unit Testing in Swift
• Unit Testing Asynchronous Code in Swift
• iOS Unit Testing Best Practices