Why we removed tests from a codebase built with AI tools
TL;DR
For most of my career I thought chasing a high test coverage number was a waste of time. Lots of tests checked trivial things and broke whenever something nearby changed.
AI tools changed my mind. They can write code that looks right and is wrong, and an automated test is a cheap way to catch that.
With AI tools, tests got cheap to write and fix. Every failing test also makes the tool stop and check whether its change is right.
Cheap tests pile up. Taproot I/O went from its first test in February 2026 to about 1,700 test files by November, with a lot of the same setup copied over and over.
So we started deleting tests that couldn't catch a real problem. One cleanup removed about 2,000 lines of tests, and the share of code covered by tests went up slightly.
The rule now: a test has to fail when something real breaks. Coverage is a minimum to protect, not a score to chase. We're due for another cleanup.
Why I changed my mind about coverage
Before I was using AI tools, I always viewed code coverage as a false metric. Tests often tested trivial things, and then they had to be updated when something else changed and made that trivial thing slightly different. Tests were important for critical business logic. High code coverage wasn't.
That changed when I started building with AI tools. An AI tool can hallucinate: it writes code that looks plausible, compiles, and does something nobody asked for. When the thing writing the code can make things up, coverage tells me how much of that code something is actually checking. Code coverage became important to me.
The cost of writing and maintaining tests went down too, up to a point. AI tools write tests quickly and fix them quickly. And each test is another point where the tool has to ask whether the change it just made is correct.
As the scope and breadth of the tests grew, it became obvious that they weren't DRY (don't repeat yourself), and they were turning into a slow point in development. I needed to refactor them to make them faster and to cut the repeated boilerplate.
It's important to routinely evaluate a codebase, and its tests, for clarity, conciseness and general hygiene. Ours is due for another pass.
How it played out in Taproot
When coverage started to matter
The first test in Taproot arrived in late February 2026, in the same change that set the codebase up for Claude Code. We didn't measure coverage until June. The first numbers were 80.6% of lines in the API and 63.9% across the whole repository, with the static site generator lowest at 42%.
We set a minimum for each service a couple of points below what we measured, and CI fails if coverage drops under it. The minimums are meant as a regression alarm, not a target. AGENTS.md puts it this way: "Coverage is a floor, not a goal." The floors are allowed to move up but never down, and AI tools are told never to lower one to make a build pass.
Coverage kept climbing after that. By November, the API was around 88% of lines, up from 80.6% in June.
What a test catches that review misses
The clearest example came from our component library. AI tools kept inventing design token names that sounded right and didn't exist, like a muted text color variable that was never defined. The browser ignores a CSS variable it doesn't recognize, so nothing errors and the style just doesn't apply. In April 2026 we added a contract test that compares every token used in the code against the list of tokens that actually exist. A made-up name now fails the build instead of shipping.
Tests also need protecting from the tools. In August, a review caught an AI tool that had weakened an assertion in one test while fixing a flaky, unrelated one. The test still passed. It just checked less than before. Claude Code and OpenAI Codex review each other's work, and test changes get the same scrutiny as product code.
What we removed
In June we audited about 2,800 test cases and estimated that 15 to 20% of them added little. We captured a coverage baseline first, so we could prove the cleanup didn't lower coverage instead of just claiming it. Then we removed four kinds of tests:
Thirty smoke tests for web components, whose whole job was checking that a component registered itself or had a shadow root. Lit guarantees that, so these tests could only fail if the framework broke.
Five snapshot assertions and a 1,780-line snapshot file. A snapshot fails on any change to the output, intended or not, so the usual fix is to regenerate it without looking closely.
Dozens of tests that opened a source file and searched it for a string instead of running the code. They break on harmless refactors and pass on broken behavior. We replaced them with tests that actually exercise the editor and components. One assumption going in was that our rich text editor couldn't run in the test environment. It could.
Five tests in the generator that checked fallback behavior the server renderer already owns and tests.
The first pass removed 42 test cases and about 2,000 lines of test code, most of it the snapshot file. The follow-up on source-reading tests removed 65 more and added 12 behavioral ones. Coverage didn't drop. It went up slightly, because several of the deleted tests never ran any product code at all.
Deleting tests also exposed something. Seven page tests only worked because a smoke test happened to load the component they depended on. Once the smoke tests were gone, those tests had to import what they actually used.
Keeping them from coming back
A written rule wasn't enough to keep these from creeping back in, so a script, check-test-antipatterns.mjs, fails CI if it finds a component smoke test, a snapshot assertion, or a test that reads source code. Each one has an opt-out comment for the rare case where it's justified, so an exception is visible in review. There's more on that approach in how we enforce coding standards.
Where the slowness came from
By September there were over a thousand test files, so we ran a second audit focused on repetition. A scan for identical 20-line blocks across 1,068 test files found copied setup everywhere. The audit treated those matches as leads to review, not as proof that the tests were redundant.
The biggest single waste was in the database tests. A group of 19 tests each regenerated the full set of migration scripts, and nine more did it again elsewhere. Generating them once per database context took that group from 6.8 seconds to 3.2, and the whole API suite from about 41 seconds to 37. Shared helpers replaced four identical 47-line seeding methods and eight copies of the same file walker. That pass removed 588 net lines and 23 test cases.
The audit's rule for those helpers was to share setup but never hide assertions or build a general purpose testing language on top of the tests. A helper that hides what a test checks makes it harder to review, for people and AI tools alike.
It also measured where the time actually goes in CI, and the answer wasn't unit tests. In the sampled run, the .NET build alone took about three minutes. The audit says it plainly: removing unit tests won't erase that build cost. Cutting tests made local runs and maintenance cheaper. It didn't make CI dramatically faster.
What we keep on purpose
Tests for business rules, permissions, and anything security related stay, even when they're slow or look repetitive. The September audit kept a 10-second network deadline test and an eight-second retry test because shortening the waits risks flaky results. It also kept the tests that prove retired fields and old routes are rejected. The write pipeline is a good example of where tests earn their keep: every stage of every write gets tested once, in one place.
The rule in our instructions is short: "A test earns its place by failing when behavior or a contract breaks."
Coverage holding steady doesn't prove the remaining tests protect as much as the ones we deleted. Coverage measures what ran, not what was checked. That's why each removal was reviewed by category, not by the coverage number.
What's next?
We're due for another hygiene pass. The September audit left a list of candidates, including one test file that's grown to almost 7,000 lines, and there's an open task to remove tests that only document history instead of guarding against a real failure. Its guidance: "A tombstone documents history; a guard prevents a real failure mode. When in doubt, keep."
What I'd tell someone starting out
Measure coverage early, and set the minimum a little below where you are. Treat it as an alarm, and raise it as you improve.
Delete tests that can't fail. A test that only proves the framework works is noise, so block the patterns you've already removed.
Before you trust a guard, make it fail once on purpose.
Review test changes as carefully as code changes, especially when an AI tool touched a test while fixing something else.
Share setup between tests, but keep each test's assertions in plain sight.
Schedule cleanups. AI tools make adding tests nearly free, so the pile grows faster than you'd expect.