C++ Unit Tests With Boost.Test, Part 2

In Part 1 of this series, I discussed how to setup a basic C++ unit test project that executed the unit tests when you built your code. In this part, I will discuss the assertion framework provided by Boost.Test in the context of a simple programming exercise that demonstrates test-driven development.

Assertion Macros in Boost.Test

In Part 1, I had a simple test that exercised the test framework by simply asserting true or false with the BOOST_REQUIRE macro. Ultimately all checks can be broken down into simple true or false assertions, but Boost.Test provides other assertion macros that provide better diagnostics when the tests fail. This is important because good unit tests become a sort of automatic bug detector as you accumulate them. When a unit test fails unexpectedly, it should give you enough information that you immediately know the source of the problem.

The following table shows some of the most commonly used assertion macros in Boost.Test. For a complete list, see the Boost.Test documentation. In Boost.Test, assertions can be issued at one of three levels: warnings, checks and requirements. The three levels are summarized in the following table:

Level	Meaning
WARN	A warning message is issued and execution of the test case continues. The test case does not fail.
CHECK	An error message is issued and execution of the test case continues. The test case fails.
REQUIRE	A fatal error message is issued and execution of the test case terminates. The test case fails.

Normally you would use the require variation of the assertion macros. Once one assertion fails, it usually does not make sense to continue with further checks. It is good test design to have a test fail as soon as possible. By keeping each test focused on a specific outcome, generally all we need to know is the name of the failed test in order to diagnose the problem.

Macro	Meaning
BOOST_REQUIRE(expr)	Requires that expr evaluates to true.
BOOST_REQUIRE_GE(left, right)	Requires that left >= right.
BOOST_REQUIRE_GT(left, right)	Requires that left > right.
BOOST_REQUIRE_LE(left, right)	Requires that left <= right.
BOOST_REQUIRE_LT(left, right)	Requires that left < right.
BOOST_REQUIRE_NE(left, right)	Requires that left != right.
BOOST_REQUIRE_EQUAL(expected, actual)	Requires that expected == actual.
BOOST_REQUIRE_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end)	Compares two collections for equality using a pair of iterators for each collection.

Another common task in unit tests is to make assertions about the exceptions thrown by production code. The BOOST_REQUIRE_NO_THROW(expr) macro requires that expr does not throw any exceptions. The BOOST_REQUIRE_THROW(expr, exception) macro requires that expr throws an exception of type exception (or an exception derived from exception).

There are additional macros for performing floating-point comparison within a tolerance. The Boost.Test documentation provides a complete reference to the assertion macros provided. A common practice when writing unit tests is to build domain specific assertions as functions or methods with an intention revealing name. The assertion building blocks provided by the unit test framework are used to construct these domain assertions.

The Prime Factors Kata

The example I’ll work through in this post is Bob Martin’s Prime Factors Kata. This is a short programming exercise exploring an algorithm for computing the prime factors of an integer. In test-driven development with tools like JUnit and NUnit, you have a graphical test runner that shows a large green bar when the tests pass and a large red bar when the tests fail. With Boost.Test we don’t have red or green bars, but we will still refer being “red” as having a failing unit test and being “green” as having all unit tests passing.

The Requirements

Write a class named “PrimeFactors” that has one static method: Generate. The generate method takes an integer and returns a std::list<int>. That list contains the prime factors in numerical order.

Begin

1. Start by taking the project and solution we created in Part 1.
2. Create a new C++ static library project named PrimeFactors. This will contain the production code we write in response to tests.
3. Delete the Test1.cpp sample test we created in Part 1.
4. Build the solution to obtain a failure result from the Post-Build build event because there are no tests.

The First Failing Test: Going Red

Now we are ready to write our first failing test. Add a new source file to the Test project called TestPrimeFactors.cpp and enter this code:

#include "stdafx.h"
#include "PrimeFactors.h"

BOOST_AUTO_TEST_CASE(OneHasNoFactors)
{
    std::list<int> result = PrimeFactors::Generate(1);
    BOOST_REQUIRE_EQUAL(0, result.size());
}

As the name of the test case indicates, this test checks that calling Generate with an argument of one produces an empty list because one has no prime factors. Creating good names for test cases is just as important as creating good names for variables and classes. At a later time if I see that the test case OneHasNoFactors is failing, that immediately gives me an idea where the problem may lie in the code.

Now we add just enough production code to make the test compile. Create a new header file in the PrimeFactors project called PrimeFactors.h and enter the following code:

#if !defined(PRIME_FACTORS_H)
#define PRIME_FACTORS_H

#include <list>

class PrimeFactors
{
public:
    static std::list<int> Generate(int n);
};

#endif

Now we will stub out the implementation just enough to let the code compile. Create a new source file in the PrimeFactors project called PrimeFactors.cpp and enter the following code:

#include "stdafx.h"

std::list<int> PrimeFactors::Generate(int n)
{
    throw "Not Implemented";
}

Open stdafx.h and an include directive to include PrimeFactors.h so its declarations will be visible by PrimeFactors.cpp.

We want the test code to be dependent upon the production code. From the Project menu select Project Dependencies… to bring up the project dependencies dialog. In the dialog, select the Test project in the combobox and then check the PrimeFactors project in the listbox and click OK. This will automatically link the test project against the PrimeFactors static library.

Finally, we need to adjust the include search order in the test project so that it can see the headers in the production code. In the Solution Explorer window, select Properties from the context menu on the Test project. Select All Configurations in the Configurations combobox in the upper left. Then select the Configuration Properties / C/C++ / General item in the treeview. On the right enter “$(SolutionDir)/PrimeFactors” into the Additional Include Directories property. Click OK.

You should now be able to compile and build the entire solution containing the production code project and the test project. The Post-Build build event should run the unit tests, which consists of a single failing test. The text in the Output window should look similar to this:

1>------ Build started: Project: PrimeFactors, Configuration: Debug Win32 ------
1>Compiling...
1>stdafx.cpp
1>Compiling...
1>PrimeFactors.cpp
1>Creating library...
1>Build log was saved at "file://c:\tmp\Code\Boost.Test\PrimeFactors\Debug\BuildLog.htm"
1>PrimeFactors - 0 error(s), 0 warning(s)
2>------ Build started: Project: Test, Configuration: Debug Win32 ------
2>Compiling...
2>stdafx.cpp
2>Compiling...
2>TestPrimeFactors.cpp
2>Compiling...
2>Test.cpp
2>Compiling manifest to resources...
2>Microsoft (R) Windows (R) Resource Compiler Version 6.1.6723.1
2>Copyright (C) Microsoft Corporation.  All rights reserved.
2>Linking...
2>LINK : C:\tmp\Code\Boost.Test\Debug\Test.exe not found or not built by the last incremental link; performing full link
2>Embedding manifest...
2>Microsoft (R) Windows (R) Resource Compiler Version 6.1.6723.1
2>Copyright (C) Microsoft Corporation.  All rights reserved.
2>Running unit tests...
2>Running 1 test case...
2>unknown location(0): fatal error in "OneHasNoFactors": C string: Not Implemented
2>*** 1 failure detected in test suite "Master Test Suite"
2>Project : error PRJ0019: A tool returned an error code from "Running unit tests..."
2>Build log was saved at "file://c:\tmp\Code\Boost.Test\Test\Debug\BuildLog.htm"
2>Test - 2 error(s), 0 warning(s)
========== Build: 1 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

Pass The First Test: Going Green

Now we write production code in response to our failing test. Change the Generate method to the following code:

std::list<int> PrimeFactors::Generate(int n)
{
    return std::list<int>();
}

Build the solution to compile the changes and run the unit tests:

1>------ Build started: Project: PrimeFactors, Configuration: Debug Win32 ------
1>Compiling...
1>PrimeFactors.cpp
1>Creating library...
1>Build log was saved at "file://c:\tmp\Code\Boost.Test\PrimeFactors\Debug\BuildLog.htm"
1>PrimeFactors - 0 error(s), 0 warning(s)
2>------ Build started: Project: Test, Configuration: Debug Win32 ------
2>Linking...
2>Embedding manifest...
2>Running unit tests...
2>Running 1 test case...
2>*** No errors detected
2>Build log was saved at "file://c:\tmp\Code\Boost.Test\Test\Debug\BuildLog.htm"
2>Test - 0 error(s), 0 warning(s)
========== Build: 2 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========

Now that might seem like quite a bit of work just to write a function that returns an empty list, so let’s consider what we’ve done. A bunch of the work described above is one-time “setup” work for creating a production code project and its associated unit test project. That work won’t be repeated every time we add functionality, it will only be repeated when we add a new project. Also, we’ve done more than just write a function that returns an empty list of integers. We’ve locked down one part of the requirements in a regression test that covers the prime factors of one.

Covering functionality one part at a time with unit tests allows us to lock down positive progress. We always proceed from the basis of working code and each time we have another passing test, we have more working code. Notice also that we spent very little time with a failing test. In test-driven development you want to spend very little time “red” with a failing test. You want to develop a rhythm of writing a failing test to go “red” and then quickly writing just enough production code to make that test pass and go “green”.

If you are used to writing the production code first, it will be very tempting to write more code than is necessary to pass the test. For instance, in our existing example above you might be tempted to test the input argument n against one before returning an empty list. However, in a strict sense this would be “over engineering” the production code with respect to the tests we have written. Currently we have a single test case and the production code satisfies that single test case without checking the input argument. However, we’re not done writing test cases by a long shot! As we write more test cases, we will cover more of the functionality implied by the requirements and our production code will evolve in response to the test cases.

In Part 3, we will continue with more test cases for the Prime Factors Kata and elaborate the production code.

[Updated to favor BOOST_REQUIRE over BOOST_CHECK.]