Track test coverage or not?


In my opinion tracking of coverage, percent gives a false impression of "well tested" code.
Test coverage is defined as a percentage of lines executed during the test.
If test coverage becomes the main metric of testing it requires a lot of work.
When high coverage is achieved, it gives the false impression of a well-tested code. See my artificial example on the image.
Even "being well tested" is not a goal of software engineering.
The tests are there to give developers and stakeholders confidence in the stability of software in real circumstances. And therefore focus during testing should be focused on the coverage of typical situations, and dangerous situations.

The coverage calculation can still be valuable in such cases when developers understand that the project is not tested well.  In this case, coverage shows which module should be tested first. And looking at coverage precisely developer can see that branch, which handles an important use case is not covered.

Another important use case is the unit-testing of dynamic languages (like Python). Correctness of the code there can be often verified only by executing it. In this case unit tests will play the role of "compiler". They will ensure that all variables are defined before usage, and invalid class members are not used.

The test coverage is a great tool to find problems in the code, but shouldn't be abused.

Comments

Post a Comment

Popular posts from this blog

Compilation from macOS(Intel) to linux x86_64 finaly works. Cross compilation of the rust code with bazel. Part 2.

This is the continuation of cross-compilation  topic that I've started in previous part What? Do I really need to configure C++ compiler to compile rust? Cross compilation of the rust code with bazel. Part 1. Since we need to add C++ cross-compilation toolchain, let's add the pure C++ project. This will help us to test the C++ toolchain separately I've added a typical C++ hello world impliemntaion and a BUILD file for it: cc_binary ( name = "hello-world" , srcs = [ "main.cpp" ], ) And our goal is to make it compilable for linux by command: bazel build //cpp/hello_world:hello-world --platforms=//:x86_64-linux --incompatible_enable_cc_toolchain_resolution If we just try it, we get and error,  (23:01:17) ERROR: /private/var/tmp/_bazel_evgenypetrov/e83d964deb47c21842622d1b7ffa55fc/external/bazel_tools/tools/cpp/BUILD:58:19: in cc_toolchain_alias rule @baze l_tools//tools/cpp:current_cc_toolchain: Unable to find a CC toolchain using toolchain resolution. ...
Read more

Cross compilation for ARM v7. Cross compilation of the rust code with bazel. Part 3.

This is the third part of the series "Cross compilation of rust with bazel". Previous topics: What? Do I really need to configure C++ compiler to compile rust? Cross compilation of the rust code with bazel. Part 1. Compilation from macOS(Intel) to linux x86_64 finaly works. Cross compilation of the rust code with bazel. Part 2.   Next challenge is to cross-compile for armv7. This part we did together with my colleague Dima and we did it on linux. First thing, that we need is to add the target platform into our root BUILD.bazel platform ( name = "armv7-linux-musleabihf" , constraint_values = [ "@platforms//cpu:armv7" , "@platforms//os:linux" , ], ) And rust toolchain for the new target in the WORKSPACE file: rust_register_toolchains ( edition = "2021" , extra_target_triples = [ "x86_64-unknown-linux-musl" , "armv7-unknown-linux-musleabihf" , # NEW ], ) And as we remember from the part 1 ,...
Read more

Scaling number of supported platforms. Cross compilation of the rust code with bazel. Part 5

Image
  This is the 5th part of Basel cross-compilation series. Previous posts: What? Do I really need to configure C++ compiler to compile rust? Compilation from macOS(Intel) to linux x86_64 finaly works.  Cross compilation for ARM v7.  Fetching toolchains at build time.   All the build files in the previous steps looked pretty much the same, the only differences were: URLs exec architecture target architecture some compiler flags (like paths) Of cause we don't want to support a separate build file for every host and target combination. The way around it maybe to generate it from the template. The easiest way would be to use string formatting int the *.bzl file. So we create `toolchains/cpp_toolchains.bzl`, and store the BUILD file content in the variable there: BUILD_FILE_TEMPLATE = """ load("@bazel_tools//tools/cpp:unix_cc_toolchain_config.bzl", "cc_toolchain_config") load...
Read more