CMake and testing compilation failures

Creating header-only libraries with CMake is easy, one has to declare only INTERFACE parts of the library:

# Nothing to build, so only declare name
add_library(lib_name INTERFACE)

# List all include directories with library headers
target_include_directories(lib_name INTERFACE include)

# Declare dependencies - other header only libraries
target_link_libraries(int_sum INTERFACE Boost::Boost)

Of course, CMakeLists.txt for a library will have install and test steps. Testing compilable code can be done with help of many frameworks (Boost::Test, Google Test, lest or Catch), registered by add_test command and executed by ctest tool.

When testing, for each branch in code, one should consider testing it. With header-only libraries, there is a high probability, that there are compile-time conditional constructs (templates, macros). But there are currently no tools to test compile failures. CMake has a try_compile command, but it works during build generation and not during the build itself. So it will not rerun when a developer changes test or library code and executes build and tests. That is why, I have created AddCompileFailureTest - a simple CMake module, which enables testing compilation failures. Using it is straightforward. First, it has to be loaded in the main CMakeLists.txt:

include(AddCompileFailureTest)

Each expected compilation failure should be placed in proper source file within main function. Let us consider a simple sum function:

template<typename T>
inline T sum(const T a, const T b) {
    return a + b;
}

It is obvious, that it will not compile for types, which do not have operator+ defined. The test case is:

#include <sum.h>

class empty {};

int main() {
    empty a;
    empty b;
    empty c =  hlib::sum(a, b);
}

Executing the test is simple and done from tests' CMakeLists.txt:

add_compile_failure_test(
    NAME FailWithoutPlus
    SOURCE ${CMAKE_CURRENT_SOURCE_DIR}/failing_withous_plus.cpp
    INCLUDE_DIRECTORIES ${HLIB_INCLUDES}
)

HLIB_INCLUDES is a variable, which contains include path for a library with the sum function. Presented example is available on GitHub.

Function add_compile_failure_test works by creating a directory in the build tree and creating simple CMake project in it. This project contains a file with test case and CMakeLists.txt, which uses the try_compile command for checking for compilation errors in the test file. If the file can be compiled without errors, a build generation of the test project fails. The function add_compile_failure_test adds to ALL target a target for copying test file to build directory and test, which tries to generate build for the test project. The test fails, when build generation fails, ie. when test file can be compiled without errors.