Gcovr Cookbook

How to collect coverage for C extensions in Python

Collecting code coverage data on the C code that makes up a Python extension module is not quite as straightforward as with a regular C program.

As with a normal C project, we have to compile our code with coverage instrumentation. Here, we export CFLAGS="--coverage" and then run python3 setup.py build_ext.

Unfortunately, build_ext can rebuild a source file even if the current object file is up to date. If multiple extension modules share the same source code file, gcov will get confused by the different timestamps and report inaccurate coverage. It is nontrivial to adapt the build_ext process to avoid this.

Instead, we can use the ccache utility to make the compilation lazy (works best on Unix systems). Before we invoke the build_ext step, we first export CC="ccache gcc". Ccache works well but isn’t absolutely perfect, see the ccache manual for caveats.

A shell session might look like this:

# Set required env vars
export CFLAGS="--coverage"
export CC="ccache gcc"

# clear out build files so we get a fresh compile
rm -rf build/temp.*  # contains old .gcda, .gcno files
rm -rf build/lib.*

# rebuild extensions
python3 setup.py build_ext --inplace  # possibly --force

# run test command i.e. pytest

# run gcovr
rm -rf coverage; mkdir coverage
gcovr --filter src/ --print-summary --html-details -o coverage/index.html

Out-of-Source Builds with CMake

Tools such as cmake encourage the use of out-of-source builds, where the code is compiled in a directory other than the one which contains the sources. This is an extra complication for gcov. In order to pass the correct compiler and linker flags, the following commands need to be in CMakeLists.txt:

# This flags are used if cmake is called with -DCMAKE_BUILD_TYPE=PROFILE
set(CMAKE_C_FLAGS_PROFILE --coverage)
set(CMAKE_CXX_FLAGS_PROFILE --coverage)

add_executable(program example.cpp)

The --coverage compiler flag is an alternative to fprofile-arcs -ftest-coverage for recent version of gcc. In versions 3.13 and later of cmake, the target_link_libraries command can be removed and add_link_options("--coverage") added after the add_compile_options command.

We then follow a normal cmake build process:

cd $BLD_DIR
cmake -DCMAKE_BUILD_TYPE=PROFILE $SRC_DIR
make VERBOSE=1

and run the program:

cd $BLD_DIR
./program

However, invocation of gcovr itself has to change. The assorted .gcno and .gcda files will appear under the CMakeFiles directory in BLD_DIR, rather than next to the sources. Since gcovr requires both, the command we need to run is:

cd $BLD_DIR
gcovr -r $SRC_DIR .