BOOST_TEST: universal and general purpose assertions

The Unit Test Framework provides an almost unique interface to a great range of test-case scenarios, through the BOOST_TEST macro. The general form of BOOST_TEST is the following:

BOOST_TEST(statement);
BOOST_TEST_<level>(statement, optional_modifiers)

An example of use might be the following:

Example: BOOST_TEST overview

Code
#define BOOST_TEST_MODULE boost_test_macro_overview #include <boost/test/included/unit_test.hpp> BOOST_AUTO_TEST_CASE( test_macro_overview ) { namespace tt = boost::test_tools; int a = 1; int b = 2; BOOST_TEST(a != b - 1); BOOST_TEST(a + 1 < b); BOOST_TEST(b -1 > a, a << " < " << b - 1 << " does not hold"); BOOST_TEST(a == b, tt::bitwise()); BOOST_TEST(a + 0.1 == b - 0.8, tt::tolerance(0.01)); }

Code

#define BOOST_TEST_MODULE boost_test_macro_overview
#include <boost/test/included/unit_test.hpp>

BOOST_AUTO_TEST_CASE( test_macro_overview )
{
  namespace tt = boost::test_tools;
  int a = 1;
  int b = 2;
  BOOST_TEST(a != b - 1);
  BOOST_TEST(a + 1 < b);
  BOOST_TEST(b -1 > a, a << " < " << b - 1 << " does not hold");
  BOOST_TEST(a == b, tt::bitwise());
  BOOST_TEST(a + 0.1 == b - 0.8, tt::tolerance(0.01));
}

Output
> ./boost_test_macro_overview --log_level=all Running 1 test case... Entering test module "boost_test_macro_overview" test.cpp:12: Entering test case "test_macro_overview" test.cpp:17: error: in "test_macro_overview": check a != b - 1 has failed [1 == 1] test.cpp:18: error: in "test_macro_overview": check a + 1 < b has failed [1 + 1 >= 2] test.cpp:19: error: in "test_macro_overview": 1 < 1 does not hold test.cpp:20: error: in "test_macro_overview": check a == b has failed [1 != 2]. Bitwise comparison failed Mismatch at position 0 Mismatch at position 1 test.cpp:21: error: in "test_macro_overview": check a + 0.1 == b - 0.8 has failed [1 + 0.10000000000000001 != 1.2]. Relative difference exceeds tolerance [0.0909091 > 0.01] test.cpp:12: Leaving test case "test_macro_overview"; testing time: 380us Leaving test module "boost_test_macro_overview"; testing time: 459us *** 5 failures are detected in the test module "boost_test_macro_overview"

Output

> ./boost_test_macro_overview --log_level=all
Running 1 test case...
Entering test module "boost_test_macro_overview"
test.cpp:12: Entering test case "test_macro_overview"
test.cpp:17: error: in "test_macro_overview": check a != b - 1 has failed [1 == 1]
test.cpp:18: error: in "test_macro_overview": check a + 1 < b has failed [1 + 1 >= 2]
test.cpp:19: error: in "test_macro_overview": 1 < 1 does not hold
test.cpp:20: error: in "test_macro_overview": check a == b has failed [1 != 2]. Bitwise comparison failed
Mismatch at position 0
Mismatch at position 1
test.cpp:21: error: in "test_macro_overview": check a + 0.1 == b - 0.8 has failed [1 + 0.10000000000000001 != 1.2]. Relative difference exceeds tolerance [0.0909091 > 0.01]
test.cpp:12: Leaving test case "test_macro_overview"; testing time: 380us
Leaving test module "boost_test_macro_overview"; testing time: 459us

*** 5 failures are detected in the test module "boost_test_macro_overview"

The major features of this tool are:

a great flexibility for statement which may be almost anything: full expression composed by several operations are supported and handled,
an extended reporting capability in case of failure: not only BOOST_TEST reports the location of the failure and a copy of statement itself, but also the values of the operands that permits a rapid identification of the issues related to the failed assertion,
the possibility to control better the behavior or the reports of the checks, in particular:
- floating point comparison: the tolerance may be provided, either using the BOOST_TEST directly with optional_modifiers, or with decorators (see here for more details),
- container/collection comparisons: different operations for comparison are provided out of the box for comparing collection of elements (default, per-element, lexicographic), with extended diagnostic on failures (covered in this section),
- string comparison: C-strings operands are automatically detected and the comparisons are performed as if std::string objects were used,
- optional failure message,
- bitwise comparison, providing extended diagnostic in case of failure

	Warning
	To get all the functionalities of `BOOST_TEST` family of assertions, a C++11 capable compiler is required, especially supporting the `auto` and `decltype` keywords and the variadic macros. The documentation focuses on these set of compilers. For compilers not supporting all the features of `BOOST_TEST`, the macro `BOOST_TEST_MACRO_LIMITED_SUPPORT`.

Complex statements

BOOST_TEST provides an enhanced reporting capability: additional details of the failing operands and operations are provided in the log, as shown on the example below:

Example: BOOST_TEST enhanced reporting

Code
#define BOOST_TEST_MODULE boost_test_macro3 #include <boost/test/included/unit_test.hpp> BOOST_AUTO_TEST_CASE( test_op_reportings ) { int a = 13, b = 12; BOOST_TEST(a == b); BOOST_TEST(a < b); BOOST_TEST(a - 1 < b); BOOST_TEST(b > a - 1); }

Code

#define BOOST_TEST_MODULE boost_test_macro3
#include <boost/test/included/unit_test.hpp>

BOOST_AUTO_TEST_CASE( test_op_reportings )
{
  int a = 13, b = 12;
  BOOST_TEST(a == b);
  BOOST_TEST(a < b);
  BOOST_TEST(a - 1 < b);
  BOOST_TEST(b > a - 1);
}

Output
> ./boost_test_macro3 --log_level=all Running 1 test case... Entering test module "boost_test_macro3" test.cpp(12): Entering test case "test_op_reportings" test.cpp(15): error: in "test_op_reportings": check a == b has failed [13 != 12] test.cpp(16): error: in "test_op_reportings": check a < b has failed [13 >= 12] test.cpp(17): error: in "test_op_reportings": check a - 1 < b has failed [13 - 1 >= 12] test.cpp(18): error: in "test_op_reportings": check b > a - 1 has failed [12 <= 12] test.cpp(12): Leaving test case "test_op_reportings"; testing time: 484us Leaving test module "boost_test_macro3"; testing time: 588us *** 2 failures are detected in the test module "boost_test_macro3"

Output

> ./boost_test_macro3 --log_level=all
Running 1 test case...
Entering test module "boost_test_macro3"
test.cpp(12): Entering test case "test_op_reportings"
test.cpp(15): error: in "test_op_reportings": check a == b has failed [13 != 12]
test.cpp(16): error: in "test_op_reportings": check a < b has failed [13 >= 12]
test.cpp(17): error: in "test_op_reportings": check a - 1 < b has failed [13 - 1 >= 12]
test.cpp(18): error: in "test_op_reportings": check b > a - 1 has failed [12 <= 12]
test.cpp(12): Leaving test case "test_op_reportings"; testing time: 484us
Leaving test module "boost_test_macro3"; testing time: 588us

*** 2 failures are detected in the test module "boost_test_macro3"

BOOST_TEST parses the statement and constructs an expression out of it. statement may be a complex expressions containing almost any of the overloadable operators in C++:

Class of operation	operators
binary comparisons	`==`, `!=`, `<`, `>`, `<=`, `>=`
arithmetic compositions	`+`, `-`, `*`, `/`, `%`
bitwise compositions	`\|`, `&`, `^`, `<<`, `>>`
assignments	`=`, `+=`, `-=`, `*=`, `/=`, `%=`, `<<=`, `>>=`, `&=`, `^=`, `\|=`

statement is evaluated and cast to bool, as if it would appear as argument to an if statement: this is the result of the assertion

This tool is provided in three variants corresponding to the corresponding severity levels. These three levels of assertions are reported into the test log and output, as described in details in the section. The granularity of the report depends on the current log level and report level.

Limitations & workaround

There are a few constructions that are however unsupported, but adding an extra bracket usually solves that:

statements containing ternary conditions: those statement should be surrounded by parenthesis as they cannot be overloaded
statements containing commas: those statements will be intercepted by the preprocessor
compound statements containing any logical composition ||, &&. Those are disabled intentionally and should be surrounded by parenthesis
```
BOOST_TEST((true || false));
```
The full details are given in this section.

Example: BOOST_TEST limitation and workaround

Code
#define BOOST_TEST_MODULE boost_test_macro_workaround #include <boost/test/included/unit_test.hpp> #include <sstream> BOOST_AUTO_TEST_CASE( test_logical_not_allowed ) { // Boost Unit Test Framework prevents compilation of // BOOST_TEST(true && true); BOOST_TEST((true && true)); // with extra brackets, it works as expected } BOOST_AUTO_TEST_CASE( test_ternary ) { int a = 1; int b = 2; // Boost Unit Test Framework prevents compilation of // BOOST_TEST(a == b ? true : false); BOOST_TEST((a + 1 == b ? true : false)); // again works as expected with extra brackets }

Code

#define BOOST_TEST_MODULE boost_test_macro_workaround
#include <boost/test/included/unit_test.hpp>
#include <sstream>

BOOST_AUTO_TEST_CASE( test_logical_not_allowed )
{
  // Boost Unit Test Framework prevents compilation of
  // BOOST_TEST(true && true);
  BOOST_TEST((true && true)); // with extra brackets, it works as expected
}

BOOST_AUTO_TEST_CASE( test_ternary )
{
  int a = 1;
  int b = 2;
  // Boost Unit Test Framework prevents compilation of
  // BOOST_TEST(a == b ? true : false);
  BOOST_TEST((a + 1 == b ? true : false)); // again works as expected with extra brackets
}

Output
> ./boost_test_macro_workaround --log_level=all Running 2 test cases... Entering test module "boost_test_macro_workaround" test.cpp:13: Entering test case "test_logical_not_allowed" test.cpp:17: info: check (true && true) has passed test.cpp:13: Leaving test case "test_logical_not_allowed"; testing time: 183us test.cpp:20: Entering test case "test_ternary" test.cpp:26: info: check (a + 1 == b ? true : false) has passed test.cpp:20: Leaving test case "test_ternary"; testing time: 123us Leaving test module "boost_test_macro_workaround"; testing time: 547us *** No errors detected

Output

> ./boost_test_macro_workaround --log_level=all
Running 2 test cases...
Entering test module "boost_test_macro_workaround"
test.cpp:13: Entering test case "test_logical_not_allowed"
test.cpp:17: info: check (true && true) has passed
test.cpp:13: Leaving test case "test_logical_not_allowed"; testing time: 183us
test.cpp:20: Entering test case "test_ternary"
test.cpp:26: info: check (a + 1 == b ? true : false) has passed
test.cpp:20: Leaving test case "test_ternary"; testing time: 123us
Leaving test module "boost_test_macro_workaround"; testing time: 547us

*** No errors detected

Boost C++ Libraries