White-box testing

The white-box test (also known as structural testing) is a testing method that examines the internal structures or workings of an application, as opposed to its functionality (i.e., black-box testing). During white-box testing, the system’s internal perspective and programming skills are used to design the test cases. The tester selects inputs to examine the paths through the code and determines the expected outputs.

Statement testing and statement coverage

In statement testing, the coverage elements are the executable instructions. The goal is to design test cases that execute the code instructions until an acceptable coverage level is reached. Coverage is defined as the ratio of the number of instructions exercised by the test cases to the total number of executable instructions in the code, expressed as a percentage.

During the coverage calculation, the following are considered as single units:

• Sequence
• Selection
• Iteration

Coverage: number of instructions affected by tests / total number of instructions

Branch testing and branch coverage

A branch is the transfer of control between two nodes in the control flow graph, which shows the possible sequences in which the source code instructions can be executed in the test subject. Each control transfer can be unconditional (e.g., sequential code) or conditional (e.g., decision outcome).

In branch testing, the coverage elements are the branches. The goal is to design test cases that run the branches in the code until an acceptable coverage level is achieved. Coverage is defined as the ratio of branches exercised by the test cases to the total number of branches, expressed as a percentage.

During the calculation, both outcomes of each decision point are covered (if and else branches). In the case of loops, this means entering the loop or skipping it.

Coverage: executed decision outcomes / possible decision outcomes

It is more thorough than statement coverage, because 100% branch coverage also implies 100% statement coverage, while 100% statement coverage does not necessarily imply 100% branch coverage.

Model of coverage calculation

Control Flow Graph (CFG): a directed graph where the nodes correspond to instructions, and the edges indicate the flow of control. There is an edge between nodes i and j in the graph if node j can be executed immediately after node i during some program execution.

Example:

Consider the following pseudocode!

Note

The syntax has been revised, and some null instructions (;) have been added, while numbering from certain lines has been omitted. This was done to maintain the graph representing the solution of this exercise.

    1 Program Triangle;
      var
    2 a,b,c : integer;
    3 triangle : boolean;
      begin

    4 Write('Please provide 3 number for the sides of the triangle!');
    5 ReadLn(a,b,c);
    6 WriteLn('A: ', a);
    7 WriteLn('B: ', b);
    8 WriteLn('C: ', c);

    9 if (a< b + c) and (b< a + c) and (c <a + b) 
    10 then triangle := True;   
    11 else triangle := False;
    12 WriteLn('Calculation has been completed.');

    13 if (triangle)
    14    then if (a = b) and (b = c)
    15         then WriteLn('Regular');
    16         else if (a <> b) and (a <> c) and ( b <> c)
    17              then WriteLn("Common");
    18              else WriteLn('Isosceles triangle');
    19              ;
    20         ;
    21 else Writeln('It is not a triangle');
    22 ;
    23 end

From the code, the following CFG can be created:

Nodes 4 and 23 represent the beginning and end of the program. Since there is no loop in the program, this will be a directed acyclic graph.

In practice, there is no need to represent every node; we can use so-called basic blocks. Basic blocks are sequences of consecutive instructions for which it holds that if the first instruction in the block is executed, then the last instruction in the block will also be executed, and within the block, each instruction can only receive control from another instruction in the same block.

Representation of the above example using basic blocks:

In the original CFG, the flow from 4 to 9 can proceed only in one way, so these can be combined into a single basic block. Similarly, nodes 12–13 and 22–23 can also be merged.

Class exercise 1

Consider the following code snippet:

1 Read P
2 Read Q
3 IF P+Q > 100 THEN
4    Print "Large"
5 ENDIF
6 IF P > 50 THEN
7    Print "P Large"
8 ENDIF

• Determine the number of test cases required for 100% statement and branch coverage!
• Provide a minimal set of test cases that achieve such coverage levels!

Possible solution

For statement coverage a single test case is sufficient:

• (START) -> (1,2,3) -> (4) -> (5,6) -> (7) -> (8, END)
• Explanation: both IF conditions go into the true branch: P+Q > 100 and P > 50
  • Example test case: P = 75, Q = 75

For branch coverage at least two test cases are needed, for example:

• (START) -> (1,2,3) -> (4) -> (5,6) -> (7) -> (8, END)
• P+Q > 100 and P > 50
  • P = 75, Q = 75
• (START) -> (1,2,3) -> (5,6) -> (8, END)
• False branches covered: P+Q <= 100 and P <= 50
  • Example: P = 25, Q = 25

Measuring instruction coverage in practice

Most frameworks support coverage measurement, such as Eclipse and IntelliJ Ultimate. IntelliJ Community Edition does not have built-in support, so we will use the JaCoCo Maven plugin to solve this task. JaCoCo (Java Code Coverage) is one of the most well-known and widely used Java code coverage tools. It is designed for Maven, Gradle, Ant, and IntelliJ / Eclipse environments, and it measures which parts of the Java bytecode were executed during testing.

When you run tests, JaCoCo attaches to the JVM as an agent. This “agent” modifies each loaded class (at the bytecode level) to keep track of which instructions were executed. After execution, it creates a binary file named jacoco.exec in the target directory. This file contains the coverage data.

The reports we are interested in are generated during the jacoco:report phase (run separately), which can produce reports in HTML, XML, and CSV formats.

The plugin measures:

• Statement coverage
• Line coverage
• Branch coverage
• Method coverage
• Class coverage

Class exercise 2

Measure coverage on the Book store project!

1. Download the IntelliJ project and extract it!
2. Load it into the IntelliJ framework!
   • Note: The reference to JaCoCo must be placed in the POM.XML file under the <build>-><plugins> block.
3. Build the project using the Maven panel's clean command!
4. Run the Maven panel’s Verify command, which executes the tests and the coverage measurement. Check whether the binary is created in the target directory!
5. By default, the report is not yet generated. You must generate it manually in the Maven panel under Plugins->jacoco using the jacoco:report command.
6. After execution, the report will be generated in the target/site/jacoco directory.

• Sometimes the Maven panel does not contain the JaCoCo plugin; in that case, click the settings (gear icon) on the panel and choose Maven Settings. Check the Use plugin registry checkbox. After that, the plugin will appear in the Maven menu.

Supplement

Coverage can also be measured in other languages. To make project work easier, we list here some tools that can be used for each language.

Python

The main tool is coverage.py:

• The de facto standard coverage tool in Python.
• Integrates with unittest, pytest, nose, behave, etc. frameworks.
• Measures line, branch, and function coverage.
• Report: terminal, HTML, XML (Cobertura-compatible).

Installation and usage:

pip install coverage pytest
coverage run -m pytest
coverage html

C++

Since the C++ language is compiled and optimized, coverage measurement requires compilation flags that insert extra code.

Tool	Compiler/operation	Feature
gcov	GCC built-in (flag: --coverage)	classic, CLI-based, simple
lcov	HTML report from gcov output	visual, Cobertura-compatible
llvm-cov	Part of Clang / LLVM toolchain	modern, detailed source-level
OpenCppCoverage	Visual Studio compatible GUI/CLI	alternative to gcov

Example:

g++ -fprofile-arcs -ftest-coverage -O0 -g main.cpp -o main
./main
lcov --capture --directory . --output-file coverage.info
genhtml coverage.info --output-directory out

What is Cobertura?

Cobertura is a classic, yet still used Java code coverage tool, considered the predecessor of JaCoCo. The tool instruments the code at the source and bytecode level. It is open-source and can be used to measure line, branch, and class coverage. It generates reports in HTML, XML, and Cobertura XML formats.

What do we mean by code instrumentation?

Code instrumentation is one of the important technical concepts of coverage measurement. The program is augmented with extra instructions that monitor and record what happens during execution. In essence, the code is equipped with measurement points. Coverage tools perform this instrumentation before execution.

TypeScript/JavaScript

TypeScript code is usually transpiled to JavaScript, and coverage measurement occurs during execution.

Tool	Framework	Feature
Istanbul/nyc	Jest, Mocha, Jasmine	industry-standard coverage tool
Jest built-in coverage	Jest native --coverage	simple, out-of-the-box solution
C8	native V8 coverage API	fast, accurate, Istanbul-compatible
karma-coverage	for Angular / Karma tests	browser-based measurement
Vitest coverage	modern alternative to Jest	integrated v8-based measurement

Example:

npm install --save-dev jest ts-jest @types/jest
npm install --save-dev nyc
npx jest --coverage

Homework 1

Consider the following code snippet:

1 void replace(char *str, int n) {        
2    while (1) {
3        if (*str == 0) {
4            break;
5        } else {
6            int i = 0;
7            if (*str == 0) {
8                while (i < n && *str != 0) {
9                    ++str;
10                   ++i;
11               }
12           }
13           while (i < n && *str == 0) {
14               switch (*str) {
15                 case '\n':
16                   *str = '/';
17                   break;
18                 case '\t':
19                   *str = ' ';
20                   break;
21                 case '\a':
22                   *str = '*';
23                   break;
24               }
25               ++str;
26               ++i;
27           }
28       }
29   }
30 }

• Determine the number of test cases required for 100% statement and branch coverage!
• Provide a minimal test case set that produces such coverage levels!

Pay attention to the code structure!

Since the false branch of the first condition cannot have *str==0, the instructions in the branch starting at line 7 are not reachable. For the same reason, the body of the loop starting at line 13 is also unreachable.

Homework 2

Improve coverage by adding more unit tests for the Book project!

• Submit the report before and after the improvement.
• Include the related unit test files.

---

Utolsó frissítés: 2025-10-30 14:25:48