C++ Unit Tests With Boost.Test, Part 5

In Part 4 of this series, we completed some UI functionality using a test-driven development style. In this final part of the series, I will cover the facilities in Boost.Test for sharing common setup and teardown actions between tests and organizing tests into suites.

SetUp and TearDown

Unit tests have the following structure:

• SetUp for the test.
• Execute the system under test.
• TearDown for the test.

In the tests we wrote for PrimeFactors::Generate, there was no SetUp or TearDown phase to the tests because we were testing a static method. No objects needed to be created and no collaborators needed to be configured. Because there was no SetUp, there was also no corresponding TearDown. This is not a typical situation, but it simplified the demonstration of test-driven development.

When a test requires SetUp and TearDown phases, we can simply insert this code into the test case itself. However, doing so can result in repeated code. You may have noticed this repetition in the tests we wrote for PrimeFactorsMediator. We could eliminate this duplication by extracting a function from our test cases as we did in part 3 of this series with the custom assertion function AssertValueHasPrimeFactors. However, when there is a signficant amount of state to be managed, this can lead to long parameter lists or global variables in our test code. Given that we are programming in C++, isn’t there a way to encapsulate this state into an object and have it managed automatically? Boost.Test provides a way to do exactly that.

If you dig into the BOOST_AUTO_TEST_CASE macro, you’ll see that it expands into a call to another macro called BOOST_FIXTURE_TEST_CASE with the supplied test name as the first argument and BOOST_AUTO_TEST_CASE_FIXTURE as the second argument.

Test Fixtures

So what’s a test fixture? It is the class that holds the code needed to perform repeated SetUp and TearDown operations for each test case. In Boost.Test, the SetUp code is implemented in the constructor for the fixture class and the TearDown code is implemented in the destructor for the fixture class. The fixture class can also hold custom assertion methods that operate on the state held within the fixture.

When you use BOOST_FIXTURE_TEST_CASE it creates a new class that derives from your test fixture. The new class has all the registration boiler plate logic for the test framework and the code you write after the invocation of the macro becomes the body of the test method on the derived class. Because your test case is in a class derived from your test fixture, instantiating the test case invokes the constructor for the fixture which performs the SetUp for the test. When the test case exits (either normally or abnormally), the test case class is destroyed and the destructor for the fixture is invoked which performs the TearDown for the test.

To motivate our discussion of SetUp and TearDown, let’s revisit the test code we wrote for PrimeFactorsMediator. Each of those tests had duplicated setup code: a fake dialog was created and a mediator was created with a pointer to the fake dialog. Let’s refactor that SetUp code into a Boost.Test fixture.

In TestPrimeFactorsMediator.cpp add the following code just after the definition of FakePrimeFactorsDialog:

struct TestPrimeFactorsMediator
{
    TestPrimeFactorsMediator()
        : m_dialog(),
        m_mediator(&m_dialog)
    {
    }

    FakePrimeFactorsDialog m_dialog;
    PrimeFactorsMediator m_mediator;
};

Now we can refactor our test cases from BOOST_AUTO_TEST_CASE to BOOST_FIXTURE_TEST_CASE. Here is what the test cases look like as fixture tests:

BOOST_FIXTURE_TEST_CASE(OKButtonInitiallyDisabled,
    TestPrimeFactorsMediator)
{
    m_dialog.AssertOKButtonEnabled(false);
}

BOOST_FIXTURE_TEST_CASE(OKButtonEnabledWithValidInteger,
    TestPrimeFactorsMediator)
{
    m_dialog.ValueTextFakeResult = "123";
    m_mediator.ValueTextChanged();
    BOOST_REQUIRE(m_dialog.ValueTextCalled);
    m_dialog.AssertOKButtonEnabled(true);
}

BOOST_FIXTURE_TEST_CASE(OKButtonDisabledWithEmptyText,
    TestPrimeFactorsMediator)
{
    m_mediator.ValueTextChanged();
    BOOST_REQUIRE(m_dialog.ValueTextCalled);
    m_dialog.AssertOKButtonEnabled(false);
}

BOOST_FIXTURE_TEST_CASE(OKDisabledWithInvalidIntegerText,
    TestPrimeFactorsMediator)
{
    m_dialog.ValueTextFakeResult = "Invalid";
    m_mediator.ValueTextChanged();
    BOOST_REQUIRE(m_dialog.ValueTextCalled);
    m_dialog.AssertOKButtonEnabled(false);
}

Now that we have a common base class for our test cases, we could move the custom assertion to the common base class instead of having to invoke that custom assertion on the dialog member. However, I don’t think this makes the code that much more clearer than it is. Having the assertion on the dialog may be more useful if we wish to use the assertion in other test cases or fixtures that do not share the TestPrimeFactorsMediator base class.

Test Suites

As you can see, it doesn’t take much functionality to start racking up test cases. For the simple functionality of the prime factors generator and a little UI behavior, we ended up with 11 test cases. As you create more and more test cases, you might consider organizing them into test suites. A test suite is just a container of test cases.

The easiest way to create test suites is with the BOOST_AUTO_TEST_SUITE(test_suite_name) and BOOST_AUTO_TEST_SUITE_END() macros. All test cases defined between this pair of macros will be members of the named suite.

There is always at least one test suite, the Master Test Suite, to which all tests belong if they are not part of a named test suite. Test suites are arranged in a hierarchy, with the Master Test Suite at the root of the hierarchy. The following shows the output of the test runner when logging is enabled and the PrimeFactors and PrimeFactorsMediator tests have been arranged into their own suites:

C:\tmp\Code\Boost.Test\Debug>test --log_level=test_suite
Running 11 test cases...
Entering test suite "Master Test Suite"
Entering test suite "PrimeFactorsMediatorSuite"
Entering test case "OKButtonInitiallyDisabled"
Leaving test case "OKButtonInitiallyDisabled"
Entering test case "OKButtonEnabledWithValidInteger"
Leaving test case "OKButtonEnabledWithValidInteger"
Entering test case "OKButtonDisabledWithEmptyText"
Leaving test case "OKButtonDisabledWithEmptyText"
Entering test case "OKDisabledWithInvalidIntegerText"
Leaving test case "OKDisabledWithInvalidIntegerText"
Leaving test suite "PrimeFactorsMediatorSuite"
Entering test suite "PrimeFactorsSuite"
Entering test case "OneHasNoFactors"
Leaving test case "OneHasNoFactors"
Entering test case "TwoHasOneFactor"
Leaving test case "TwoHasOneFactor"
Entering test case "ThreeHasOneFactor"
Leaving test case "ThreeHasOneFactor"
Entering test case "FourHasTwoFactors"
Leaving test case "FourHasTwoFactors"
Entering test case "SixHasTwoFactors"
Leaving test case "SixHasTwoFactors"
Entering test case "EightHasThreeFactors"
Leaving test case "EightHasThreeFactors"
Entering test case "NineHasTwoFactors"
Leaving test case "NineHasTwoFactors"
Leaving test suite "PrimeFactorsSuite"
Leaving test suite "Master Test Suite"

*** No errors detected

Once you accumulate a large number of unit tests, you may not want to execute all of the tests every time you build. (It is wise to run all unit tests before you commit changes to the version control system, however.) You can control which tests are executed by supplying command-line parameters to the test executable.

A complete reference of command-line parameters that control the execution of tests is provided in the Boost.Test documentation.

This concludes this series on C++ Unit Testing With Boost.Test. You can download a ZIP archive containing the final version of the source code for this series for use with Visual Studio .NET 2008 and Visual C++ Express 2008.

[Updated to favor BOOST_REQUIRE over BOOST_CHECK.]