This week I worked with a team to get code coverage over 80% (a corporate...
This week I worked with a team to get code coverage over 80% (a corporate minimum).
The problem with this effort: Code coverage can be gamed.
Sure, low code coverage means there's a lot of untested code.
But, high code coverage doesn't mean the code is well tested.
Code coverage helps find untested code. But using code coverage as a *target* is a problem because of Goodhart's Law:
“When a measure becomes a target, it ceases to be a good measure.”
Ironically, code coverage targets incentivize adding low quality tests to hit the number.
Low quality tests are arguably worse than no tests. Why? Because they make it look like the code is well tested. This creates a false sense of confidence. And lousy tests that report 100% coverage mask opportunities to improve.
The real solution: Thoughtful, human review.