Swift Mutation Testing
Coverage tells you which lines your tests ran. It can’t tell you whether a single assertion would notice if those lines were wrong. swift-mutation-testing plants small, deliberate bugs in your Swift code and runs your suite against each one. Every bug that slips through points at a test that isn’t checking what you think it is — and the mutation score tells you how much your tests actually verify.
brew tap ericodx/homebrew-tools brew install swift-mutation-testing
100% coverage, one missing test
Both tests run every line of isAdult. Change >= to > and they still pass — nothing checks the boundary
at 18. A coverage report calls this function done. A mutation report calls it a survivor,
with the file, the line and the exact change.
func isAdult(age: Int) -> Bool {
return age >= 18
}
func test_isAdult() {
XCTAssertTrue(isAdult(age: 20))
XCTAssertFalse(isAdult(age: 10))
} func isAdult(age: Int) -> Bool {
return age > 18 // mutated: >= → >
}
func test_isAdult() {
XCTAssertTrue(isAdult(age: 20)) // still passes
XCTAssertFalse(isAdult(age: 10)) // still passes
}
The fix is one assertion: XCTAssertTrue(isAdult(age: 18)). Finding it by hand, across a codebase, is the hard part. That’s the part the tool does.
Measured on real code
Every operator earns its place from a campaign on open source Swift packages: kill rates, compile failures, and hand-reviewed survivors. The everyday run uses only the operators whose survivors are almost always real gaps. Here is the last campaign, every operator on.
| Package | Mutants | Score | Wall time |
|---|---|---|---|
| swift-algorithms | 1,112 | 87.4% | 16 min |
| swift-argument-parser | 822 | 65.3% | 18 min |
| swift-log | 209 | 77.8% | 1 min |
| swift-cpd | 987 | 92.2% | 15 min |
| swift-mutation-testing | 1,275 | 80.9% | 146 min |
October 2026, Apple M4 Max, all seven operators. The last package is the tool itself; on the default operators its own suite scores 99.8%. Raw data and the steps to reproduce it are in Operators.
How it works
The naive approach — one build per mutant — turns a 20-second build and 200 mutants into an hour before a single test runs. The pipeline is designed around never doing that.
- 01 Discover SwiftSyntax parses every source file and finds each place an operator applies. Test code, package manifests and #if branches the build leaves out are skipped before they can inflate the count.
- 02 Schematize Every mutant is written into the source at once, behind a runtime switch. The project builds a single time; each test run flips the switch to one mutant.
- 03 Run Mutants run in parallel. Packages run the compiled test bundles directly, Xcode runs get one cloned simulator per worker. Each run stops at the first failing test.
- 04 Verify Each mutant records whether its code actually executed. A passing suite that never reached it is no coverage, not a survivor — and a kill that never reached it is flagged.
- 05 Report Text in the terminal, plus HTML, Stryker-compatible JSON, SonarQube, SARIF and Markdown — each one a flag away.
A score you can defend
A mutation score is only worth something if every verdict in it is real. A flaky test that fails at random “kills” mutants it never looked at, and a high number built on that is worse than no number at all. So the tool checks its own work.
Built for CI
You don’t need a perfect score to adopt it. Accept the survivors you have today in a
baseline, and let the quality gate fail the build on the first new one — exit code
2, after every report is written.
# Once: accept the survivors you have today, and commit the file swift-mutation-testing --write-baseline .swift-mutation-testing-baseline.json # In CI: fail on the first survivor that is not in the baseline swift-mutation-testing --quiet \ --baseline .swift-mutation-testing-baseline.json \ --max-new-survivors 0 \ --sarif-output mutation-report.sarif \ --markdown-output mutation-summary.md
Quality gate: FAILED
✓ score 90.4% ≥ 85.0%
✓ score drop 0.7 pts ≤ 2.0 pts
✗ 2 new undetected mutants (max 0)
Sources/Parser.swift:88 RelationalOperatorReplacement < → <=
Sources/Cache.swift:12 RemoveSideEffects remove store()
ℹ 3 mutants detected now that were undetected in the baseline Two commands to a score
init writes a small .swift-mutation-testing.yml with what it found. For a Swift package there’s nothing to configure at all.
# Detects project type, scheme, destination and test targets swift-mutation-testing init # Run it swift-mutation-testing
✓ Discovery: 147 mutants (143 schematizable, 4 incompatible) in 2.3s Building for testing... ✓ Built in 18.4s ✓ 3 simulators ready Testing mutants... ✓ 1/147 LogicalOperatorReplacement Validator.swift:18 ✗ 2/147 NegateConditional Validator.swift:34 ✓ 3/147 SwapTernary FeatureFlags.swift:9 Undetected mutants: Sources/Validator.swift:34:5 NegateConditional survived Overall mutation score: 85.3%
Works with coding agents
The repository is also a Claude Code plugin. Its skill teaches the agent when to run mutation testing, how to read the report, and how to write the test that kills a surviving mutant. Ask “how good are the tests in this package?” and it takes it from there.
claude plugin marketplace add ericodx/swift-mutation-testing claude plugin install swift-mutation-testing@swift-mutation-testing
Also included
From the blog
Notes from building it — the bugs, the measurements, and the design decisions.
Documentation
Free and open source
MIT-licensed, no account, no telemetry. It runs on your machine and your CI runners, and your code never leaves them. Issues, ideas and pull requests are welcome.