Track test coverage or not?


In my opinion tracking of coverage, percent gives a false impression of "well tested" code.
Test coverage is defined as a percentage of lines executed during the test.
If test coverage becomes the main metric of testing it requires a lot of work.
When high coverage is achieved, it gives the false impression of a well-tested code. See my artificial example on the image.
Even "being well tested" is not a goal of software engineering.
The tests are there to give developers and stakeholders confidence in the stability of software in real circumstances. And therefore focus during testing should be focused on the coverage of typical situations, and dangerous situations.

The coverage calculation can still be valuable in such cases when developers understand that the project is not tested well.  In this case, coverage shows which module should be tested first. And looking at coverage precisely developer can see that branch, which handles an important use case is not covered.

Another important use case is the unit-testing of dynamic languages (like Python). Correctness of the code there can be often verified only by executing it. In this case unit tests will play the role of "compiler". They will ensure that all variables are defined before usage, and invalid class members are not used.

The test coverage is a great tool to find problems in the code, but shouldn't be abused.

Comments

Popular posts from this blog

Левиафан

Key West, FL

How prototype Arduino-based fan controller and break 3 boards