Swift Mutation Testing
Traditional test coverage tells you which lines ran — not whether the tests would actually catch a bug. Mutation testing introduces small, controlled changes to your code and re-runs your test suite. Every change that goes undetected exposes a missing assertion or a blind spot. The result is a mutation score that reflects how much your tests actually verify.
brew tap ericodx/homebrew-tools brew install swift-mutation-testing
Example mutation
A small change to the source. If your tests still pass against the mutant, you have a gap — the test you thought protected this line doesn’t actually distinguish the original behavior from the change.
func isAdult(age: Int) -> Bool {
return age >= 18
}
func test_isAdult() {
XCTAssertTrue(isAdult(age: 20))
XCTAssertFalse(isAdult(age: 10))
} func isAdult(age: Int) -> Bool {
return age > 18 // mutated: >= → >
}
func test_isAdult() {
XCTAssertTrue(isAdult(age: 20)) // still passes
XCTAssertFalse(isAdult(age: 10)) // still passes
}
A mutation test works by deliberately corrupting your code in small, controlled ways and
then asking whether your test suite notices. The change above — >= becoming > — is a single mutation. swift-mutation-testing
applies many of these across your code, runs your tests against each variant, and records
the outcome. A mutation that passes the test suite is said to have survived;
one that fails is killed.
What survival reveals is uncomfortable but useful: a test that ran, a line that was covered, an assertion that didn’t actually distinguish the original behavior from the corrupted one. Coverage tells you which lines your tests touched. Mutation testing tells you whether those tests would notice if the lines were wrong. The percentage of mutations killed — the mutation score — is a more honest read on test effectiveness than coverage alone.