During the software development process, software testing is a critical component. Code coverage is a widely-used metric for measuring the completeness of software testing. In this tutorial on code coverage, we’ll define code coverage and explore tools for generating code coverage reports. Our aim is to help software testers more easily understand how to systematically complete reliable testing tasks. Read on to learn more about achieving comprehensive code coverage and enhancing your software testing process.
Table of Contents
What is Code Coverage?
Code coverage is a metric used in software development projects to measure the completeness of software testing. During testing, some parts of the program may not be executed, such as exception handling or unused functionality, resulting in the testing program only covering a portion of the code. For example, if there are 1000 lines of code and only 500 are executed during testing, the code coverage for the test would be 50%.
Types of Code Coverage
Code coverage is often analyzed further by statement coverage, branch coverage, and function coverage.
Statement Coverage
Statement coverage is calculated by counting the number of executed instructions in a program, excluding variable declarations or header file references. For example, if a program has 1000 lines of code and 200 are variable declarations, the remaining 800 lines of code are used to calculate how many lines of code are executed by the testing program. Statement coverage is the most widely-used type of coverage calculation.
Branch Coverage
A program typically has many branches, such as an if/else statement that creates a branch. Branch coverage calculates the total number of branches in a program and how many of those branches are executed by the testing program. This is also a widely-used type of coverage calculation.
Function Coverage
A large software project contains many functions, and function coverage is a metric that measures how many of those functions are executed during testing. For example, if a program has 100 functions and 50 of them are executed during testing, the function coverage for the test would be 50%. Function coverage provides less detailed information about testing and is often considered too coarse for practical use.
How to Generate Code Coverage Information?
There are many tools available for collecting and generating code coverage information. In this article, I will introduce gcovr, a tool I frequently use in C++ software development to manage and generate code coverage reports using GNU gcov. The process for using this tool consists of three steps:
- Compile the program with code coverage information
- Execute the compiled program
- Use gcovr to generate a code coverage report
1. Compile the program with code coverage information
To compile the program for analysis with code coverage, use Compiler Flags such as –coverage or -fprofile-arcs -ftest-coverage when compiling with GNU gcc or clang. The compiler will then insert code that allows for code coverage analysis into the executable file. For example, the following command can be used for compilation:
$ g++ -fprofile-arcs -ftest-coverage foo.cpp -o foo2. Execute the compiled program
In the example above, a executable file called “foo” is created, which can be executed with the following command:
$ ./fooThe program will run as originally designed, usually including all unit tests for software testing.
3. Use gcovr to generate a code coverage report
Here is a tutorial for using gcovr, a tool for generating code coverage reports. For detailed documentation, please refer to its website.
Install gcovr
gcovr can be installed using pip, the Python package manager. The latest stable release can be installed with the following command:
$ pip install gcovrThe latest development version can also be installed with the following command:
$ pip install git+https://github.com/gcovr/gcovr.gitConfigure the filter
When a project is too large, a filter is usually configured to select the code files to be observed. Therefore, a cfg file is written to describe which files to observe. For example, the following filter file describes all program files in the “example/src/” directory that I want to observe. This step can also be skipped.
filter = example/src/* json = yesRun gcovr
Use the following command to convert the configured filter file to a JSON file:
$ gcovr --config filter.cfg > filter.jsonUse the following command to generate an HTML format code coverage report:
$ gcovr --add-tracefile filter.json --html-details coverage.htmlThis will produce a code coverage report as shown below, including statement coverage and branch coverage. With the “–html-details” parameter, each file can be clicked on to see which lines of code were executed and which were not, making it a highly useful tool.
Summary
This article introduced code coverage and tools for generating code coverage analysis reports. The higher the code coverage of a project, the more complete the software testing. In general, code coverage is analyzed alongside unit tests to understand how much code is covered by the basic unit tests. Ideally, the code coverage for the entire project should be above 75%, with each individual code file also ideally above 75%. Achieving a code coverage of 90% or higher is considered excellent testing. We hope that everyone can have a complete and systematic process for software testing, and successfully develop software of a certain quality.
- [Further Reading] Comprehensive MPI Tutorial (1): Mastering the Basics of MPI in 5 Minutes
- [Further Reading] From Python beginner to intermediate developer: Master these 7 basic syntaxes
- [Further Reading] What is ChatGPT? Discover the World of Artificial Intelligence in Just 6 Minutes with ChatGPT
- [Further Reading] The application of ChatGPT: Improve Programming Efficiency by 50% with ChatGPT
