Test Coverage: What Auditors Actually Look For
The number everyone reports and nobody should trust
“What’s your test coverage?” is the first question boards, acquirers, and new CTOs ask about quality, and the answer (40%, 70%, 90%) tells an auditor almost nothing. Here’s why, in one legal-but-damning example:
// This test achieves 100% line coverage of calculateInvoiceTotal
test('calculates invoice total', () => {
calculateInvoiceTotal(invoice);
expect(true).toBe(true);
});Every line executed, nothing verified. Coverage measures what code ran during tests, not what behavior was checked. A suite full of weak assertions, or tests that mock so much they only test the mocks, produces beautiful coverage numbers and catches nothing. Meanwhile we’ve audited codebases at 35% coverage where the 35% was exactly the right 35% (dense, adversarial tests around money and permissions) and rated them safer than 85% codebases whose coverage was auto-generated CRUD assertions.
So an audit mostly ignores the headline number. Here’s what it looks at instead.
Question 1: Is the dangerous code the tested code?
Coverage is only meaningful weighted by consequence. The auditor builds a short list of the code where a bug costs real money or trust (billing and anything touching currency, permission and tenant-isolation logic, data migrations and transformations, external-payment and webhook handling, anything cryptographic) and checks coverage there, function by function.
The most common finding in this category: inverse correlation. The simple CRUD endpoints are tested to death (they’re easy to test, and easy targets for hitting a coverage gate), while the gnarly pricing engine (14 branches of discounts, proration, and tax) has one happy-path test. Coverage gates cause this: when the metric is the target, engineers rationally test the cheap code. It’s a textbook case of what an auditor means by “good” being different from what the dashboard rewards.
Question 2: Do the tests test behavior or implementation?
Two suites can cover the same lines and have opposite value. Behavior tests pin down what the system promises: “a declined card leaves the order unpaid and emails the customer.” Implementation tests pin down how the code currently happens to work: “the service calls repository.save() exactly twice.”
The second kind is worse than useless: it fails on every refactor while passing on actual bugs, training the team to see red tests as noise. The audit tell is mock density: when a test mocks five collaborators and then asserts those mocks were called, it’s a mirror, not a test. Related tells we read for:
- Assertion quality. Do tests check outcomes (returned values, database state, emitted events) or just “didn’t throw”?
- Sad paths. Count tests for failures: declined payments, timeouts from external APIs, malformed input, concurrent updates. Production incidents live on sad paths; suites that only encode happy paths verify the demo, not the product.
- Boundary values. Zero items, negative amounts, empty strings, the 10,000-row upload. Off-by-one and edge-case bugs are the bulk of real defects.
- Tests as documentation. Can a new engineer read the test file for the pricing module and learn the pricing rules? The best suites double as the spec.
Question 3: Can the suite be trusted?
A test suite is a safety net only if the team actually stands on it. Three operational checks:
Does it run, and does red block? The suite runs in CI on every PR, and a failing test blocks merge. We’ve seen suites, good ones, that had been skipped in CI for months via a continue-on-error someone added during an incident and never removed. A test suite that doesn’t gate anything is a museum.
Is it deterministic? Flaky tests (the ones that fail 5% of the time due to timing, shared state, or network calls) are corrosive out of proportion to their number, because they teach engineers to hit re-run without reading the failure. The audit checks for retry wrappers, sleep() calls, and tests hitting real external services, all classic flakiness sources.
Is it fast enough to run? A 45-minute suite doesn’t get run locally, so feedback arrives post-push at the earliest and post-deploy at the worst. Speed is a quality attribute of tests, not a luxury.
Question 4: What do the gaps say?
Untested code isn’t uniformly risky, and the audit maps the gaps to a reasonable testing shape rather than demanding uniformity. Missing unit tests on glue code: fine. Missing any integration test that exercises the real database layer: a finding. ORM misuse, transaction bugs, and N+1 catastrophes only show up when real queries run (query-shape problems are their own audit chapter; see performance bottlenecks a code audit reveals). Missing even one end-to-end test of the signup-to-paid path: a finding, because that path is the company.
One more gap with a modern twist: AI-generated code tends to arrive with AI-generated tests, which look thorough and assert little: the expect(result).toBeDefined() school of verification. If assistants write a lot of your code, the test suite deserves specific skepticism; the broader pattern is covered in hidden debt in AI-generated code.
What “good” looks like, concretely
Not a percentage. A shape:
- Dense, adversarial unit tests on the algorithmic core: money, permissions, parsing, state machines
- Integration tests against a real (containerized) database for the data layer
- A handful of end-to-end tests for the flows the business dies without
- Sad paths and boundaries represented everywhere consequences are high
- Zero tolerance for flakes; suite fast enough that engineers run it before pushing
- CI green means deployable, and the team believes it
Teams that have this at 50% line coverage are in excellent shape. Teams with a 90% badge and none of it are carrying risk their dashboard actively conceals.
Get a free code audit
Reading a test suite for what it would actually catch is slow, judgment-heavy work, which is why it’s part of the manual audit we offer rather than a script. The audit is genuinely free: a Webisoft engineer reviews your repository, including the suite, and sends a written report on where your real coverage is thin, which tests are decorative, and what to write first. If your coverage number and your incident count are telling different stories, get a free code audit and find out which one is lying.