Coverage measures lines executed, not risk retired

Coverage is the most quoted number in engineering and one of the least useful, and the reason is a category error.

It measures which lines of code ran during your test suite. It does not measure whether the risk in those lines was retired. Those are different things, and the gap between them is where production incidents live.

You can hit ninety percent coverage on a payments flow and still have no test asserting that a failed charge does not decrement inventory. Every line ran. The invariant that matters was never checked.

Here is the version that is worse. Coverage is a target people optimise for. When it becomes a goal, engineers write tests that execute lines rather than tests that catch failures, because the first is faster and the dashboard cannot tell the difference. The number goes up while the codebase gets no safer, and now you have a false signal that people are making decisions with.

The framing that replaces it: a test suite is a risk portfolio.

Some parts of your system fail expensively. Payments, auth, data loss, anything that touches money or identity or a binary you cannot take back. Those deserve deep coverage and explicit invariant tests.

Some parts fail cheaply. A misaligned label, a spacing regression, an animation that stutters. Those deserve almost nothing, and testing them thoroughly is how you build a suite so slow that people stop running it.

The question is not what is our coverage. It is what would hurt most if it broke, and what proves it will not.

In an interview, this is a level marker. Quoting a percentage is a mid-level answer. Naming what you deliberately do not test, and why, is a senior one. Explaining how you would make that decision repeatable across a team, so the next engineer does not have to relitigate it, is a staff one.

Nobody was ever paged because coverage fell to seventy-one percent. People get paged because something nobody tested did something nobody expected.

The Senior SDET Interview Handbook at salari.dev.

# The Brief, archive issues 16 to 20

Career and level. Undated.