Thursday, 1 February 2018

CMake and testing compilation failures

Creating header-only libraries with CMake is easy, one has to declare only INTERFACE parts of the library:
# Nothing to build, so only declare name
add_library(lib_name INTERFACE)

# List all include directories with library headers
target_include_directories(lib_name INTERFACE include)

# Declare dependencies - other header only libraries
target_link_libraries(int_sum INTERFACE Boost::Boost)

Of course, CMakeLists.txt for a library will have install and test steps. Testing compilable code can be done with help of many frameworks (Boost::Test, Google Test, lest or Catch), registered by add_test command and executed by ctest tool.

When testing, for each branch in code, one should consider testing it. With header-only libraries, there is a high probability, that there are compile-time conditional constructs (templates, macros). But there are currently no tools to test compile failures. CMake has a try_compile command, but it works during build generation and not during the build itself. So it will not rerun when a developer changes test or library code and executes build and tests. That is why, I have created AddCompileFailureTest - a simple CMake module, which enables testing compilation failures. Using it is straightforward. First, it has to be loaded in the main CMakeLists.txt:
include(AddCompileFailureTest)
Each expected compilation failure should be placed in proper source file within main function. Let us consider a simple sum function:
template<typename T>
inline T sum(const T a, const T b) {
    return a + b;
}
It is obvious, that it will not compile for types, which do not have operator+ defined. The test case is:

#include <sum.h>

class empty {};

int main() {
    empty a;
    empty b;
    empty c =  hlib::sum(a, b);
}

Executing the test is simple and done from tests' CMakeLists.txt:
add_compile_failure_test(
    NAME FailWithoutPlus
    SOURCE ${CMAKE_CURRENT_SOURCE_DIR}/failing_withous_plus.cpp
    INCLUDE_DIRECTORIES ${HLIB_INCLUDES}
)

HLIB_INCLUDES is a variable, which contains include path for a library with the sum function. Presented example is available on GitHub.

Function add_compile_failure_test works by creating a directory in the build tree and creating simple CMake project in it. This project contains a file with test case and CMakeLists.txt, which uses the try_compile command for checking for compilation errors in the test file. If the file can be compiled without errors, a build generation of the test project fails. The function add_compile_failure_test adds to ALL target a target for copying test file to build directory and test, which tries to generate build for the test project. The test fails, when build generation fails, ie. when test file can be compiled without errors.


Friday, 19 January 2018

Modern CMake 101: Introduction


This post is part of a modern CMake tutorial: table of contents.

Modern CMake 101: Introduction

CMake is a cross-platform build system generator, tool to test and package applications. Over the years, it gained attention and now it is widespread. It can even be used in Visual Studio, from 2017 version up, out of the box. Over the years, CMake language has grown and philosophy of writing good build script has changed. There are many tutorials for writing CMake build scripts, but even the official one is outdated and doesn't show modern practices. This post summarizes key facts about creating modern CMake scripts.

The first thing to do is to use recent enough version of CMake, ie. at least 3.0. Without it, one cannot follow best practices. See cmake_minimum_required.

The second thing is to forget about global variable, global compiler flags and thinking about directories. Modern CMake is all about modules and treating targets as objects. Targets have:
  1. named constructors:
  2. properties
  3. member functions
Because targets should be viewed as objects, one should carefully consider which parts of target create its interface and which are internal and used only to build a given target. There are three kinds of target properties:
  1. INTERFACE - properties creating an interface of target and used to build other targets depending on considered one.
  2. PRIVATE - private properties of the target, needed only to build target and should not be propagated to other targets.
  3. PUBLIC - properties which are both: INTERFACE and PRIVATE.

The third thing to keep in mind, when using a modern build tool, is to think about target dependencies and not about compile flags and directories. For expressing, that one target depends on the other, use target_link_libraries. As with all target data, one should carefully consider whether targets dependency should be in interface or not. To properly use targets from a different project, one should always export library interface with help of the install command and its exports mode.

More info can be found: here, here, here and here.

Thursday, 18 January 2018

Windows 10 Pro

I have bought my laptop with Windows 8 Home upgraded to Pro edition, but I haven't got upgrade key from a shop (or it was hidden in the receipt and I forgot where the receipt was). Years have passed, Windows was upgraded to Windows 10 and then Visual Studio stopped working. I have fought bravely, to avoid Windows reinstallation, but MS help said that it was the only way to fix Visual Studio. Reluctantly, I have reinstalled Windows with the help of Media Creation Tool. Then, my Windows was reverted to Home edition and lost the ability to execute virtual machines! Few days with Google lead me to the page with a solution, and I have learned that:
If you performed a clean install using the Media Creation Tool it installs the specific edition based on the OEM Marker in the BIOS. So if the computer originally came preinstalled with Windows 7 Starter, Home Basic, Home Premium or Windows 8/8.1 Core/Single Language Windows 10 Home will be installed.

Workaround:
Upgrade to Windows 10 Pro using the following default product key:

VK7JG-NPHTM-C97JM-9MPGT-3V66T

So, if You bought PC with Home edition of Windows and have upgraded it to Pro version after Windows reinstallation, You should use key VK7JG-NPHTM-C97JM-9MPGT-3V66T!

Saturday, 2 April 2016

Vulkan, first steps 04: pipeline layout and uniform buffer creation

List of all Vulkan tutorial posts is here.

In versions 1 and 2 of the OpenGL GPU pipeline was the fixed one. You have to set matrices, lights' parameters, bind textures and send some vertices to GPU. From the third version onward, the fixed pipeline was deprecated and focus was placed on a programmable one. There were no more matrices and light parameters. All data was passed by uniform variables and buffers to shaders - programs which run on GPU. In Vulkan, GPU pipeline is also programmable. But, before one can send anything to shaders, she or he has to define which resources will be available to which stage of the pipeline. In terms of nVidia's Open-Source Vulkan C++ API, that definition is called vk::PipelineLayout and is created in two steps. First, an object of class vk::DescriptorSetLayout is created and contains information about resources bindings. Resources are the buffers, textures and so on. After creating a descriptor object, Vulkan can create pipeline layout on GPU by creating vk::PipelineLayout object. Example code, where there is only one resource (uniform buffer), is shown below.
#include <iostream>
#include <vector>
#include <algorithm>

#include <madvk/instance.h>
#include <madvk/device.h>


using namespace std;
using namespace mad::vk;

int main(int argc, char* argv[]) {
    try {
        auto instance = Instance("vulkan playground 01");
        cout << "Vulkan instance created" << endl;
        auto device = instance.createDevice(::vk::QueueFlagBits::eGraphics);
        cout << "Vulkan device created" << endl;

        // Only one binding - uniform buffer for projection matrix
        auto layout_binding = ::vk::DescriptorSetLayoutBinding()
            .descriptorType(::vk::DescriptorType::eUniformBuffer)
            .descriptorCount(1)
            .stageFlags(::vk::ShaderStageFlagBits::eVertex);
        auto descriptor_layout = ::vk::DescriptorSetLayoutCreateInfo()
            .bindingCount(1)
            .pBindings(&layout_binding);
        auto layout = device.createDescriptorSetLayout(descriptor_layout, nullptr);
        cout << "Vulkan layout created" << endl;

        auto pipeline_layout = ::vk::PipelineLayoutCreateInfo().pSetLayouts(&layout);
        auto pipeline = device.createPipelineLayout(pipeline_layout, nullptr);
        cout << "Vulkan pipeline created" << endl;

        device.destroyPipelineLayout(pipeline, nullptr);
        cout << "Vulkan pipeline destroyed" << endl;
        device.destroyDescriptorSetLayout(layout, nullptr);
        cout << "Vulkan layout destroyed" << endl;
    }
    catch(const exception& err) {
        cerr << "[ERROR] " << err.what() << endl;
        return 1;
    }
}


To fill buffer data, I need the model-view-projection (MVP) matrix (in OpenGL terms). To calculate one, I use OpenGL Mathematics (GLM) library. Using it is even easier than writing matrices by hand on a piece of paper! Snippet for creating MVP matrix is presented below.
// Calculate matrix (like in OpenGL)
const auto projection = ::glm::perspective(glm::radians(45.0f), 1.0f, 0.1f, 100.0f);
const auto look_at = ::glm::lookAt(
    ::glm::vec3(0, 3, 10), // Camera is at (0,3,10), in World Space
    ::glm::vec3(0, 0, 0),  // and looks at the origin
    ::glm::vec3(0, -1, 0)  // Head is up (set to 0,-1,0 to look upside-down)
);
const auto model = ::glm::mat4(1.0f);
const auto matrix = projection * look_at * model;
cout << "Matrix calculated" << endl;

Creating and filling buffer requires a few steps. First, the Vulkan buffer object (vk::Buffer) has to be created. Then, GPU memory has to be allocated. Handle to GPU memory will be stored in an object of type vk::DeviceMemory. After GPU memory has been allocated, data can be copied to it just like it has been copied between two RAM locations. But allocating GPU memory in Vulkan requires two more steps. First, one has to find an index of memory type, which is visible from CPU and can be mapped. Without that, we could not copy data to it. The second step is to let Vulkan calculate GPU's memory requirements for the created buffer. Full source code is presented below.
#include <iostream>
#include <vector>
#include <algorithm>

#define GLM_FORCE_RADIANS
#include <glm/glm.hpp>
#include <glm/gtc/matrix_transform.hpp>

#include <madvk/instance.h>
#include <madvk/device.h>


using namespace std;
using namespace mad::vk;

int main(int argc, char* argv[]) {
    try {
        auto instance = Instance("vulkan playground 01");
        cout << "Vulkan instance created" << endl;
        auto device = instance.createDevice(::vk::QueueFlagBits::eGraphics);
        cout << "Vulkan device created" << endl;

        // Only one binding - uniform buffer for projection matrix
        const auto layout_binding = ::vk::DescriptorSetLayoutBinding()
            .descriptorType(::vk::DescriptorType::eUniformBuffer)
            .descriptorCount(1)
            .stageFlags(::vk::ShaderStageFlagBits::eVertex);
        const auto descriptor_layout = ::vk::DescriptorSetLayoutCreateInfo()
            .bindingCount(1)
            .pBindings(&layout_binding);
        auto layout = make_raii(
            device.createDescriptorSetLayout(descriptor_layout, nullptr),
            [&device](auto& o) {device.destroyDescriptorSetLayout(o, nullptr); });
        cout << "Vulkan layout created" << endl;

        const auto pipeline_layout = ::vk::PipelineLayoutCreateInfo()
            .pSetLayouts(&layout);
        auto pipeline = make_raii(
            device.createPipelineLayout(pipeline_layout, nullptr),
            [&device](auto& o) {device.destroyPipelineLayout(o, nullptr); });
        cout << "Vulkan pipeline created" << endl;

        // Calculate matrix (like in OpenGL)
        const auto projection = ::glm::perspective(glm::radians(45.0f), 1.0f, 0.1f, 100.0f);
        const auto look_at = ::glm::lookAt(
            ::glm::vec3(0, 3, 10), // Camera is at (0,3,10), in World Space
            ::glm::vec3(0, 0, 0),  // and looks at the origin
            ::glm::vec3(0, -1, 0)  // Head is up (set to 0,-1,0 to look upside-down)
        );
        const auto model = ::glm::mat4(1.0f);
        const auto matrix = projection * look_at * model;
        cout << "Matrix calculated" << endl;

        // Create uniform buffer object for matrix
        const auto buffer_info = ::vk::BufferCreateInfo()
            .usage(::vk::BufferUsageFlagBits::eUniformBuffer)
            .sharingMode(::vk::SharingMode::eExclusive)
            .size(sizeof(matrix));
        auto buff = make_raii(
            device.createBuffer(buffer_info, nullptr),
            [&device](auto& o) { device.destroyBuffer(o, nullptr); });
        cout << "Buffer created" << endl;

        // Find memory block index, which is visible from CPU and can be mapped
        const auto memory_properties = device.getPhysicalDevice().getMemoryProperties();
        const auto mem_types_begin = memory_properties.memoryTypes();
        const auto mem_types_end = mem_types_begin + memory_properties.memoryTypeCount();
        const uint32_t mem_idx = static_cast<uint32_t>(find_if(
            mem_types_begin, mem_types_end,
            [](const auto& mt) {
            return mt.propertyFlags() & ::vk::MemoryPropertyFlagBits::eHostVisible;
        }) - mem_types_begin);
        if(mem_idx >= memory_properties.memoryTypeCount())
            throw runtime_error("Unable to find memory block visible from CPU");

        // Obtain buffer memory requirements on GPU
        const auto memory_requirements = device.getBufferMemoryRequirements(buff);
        cout << "Buffer memory requirements obtained" << endl;

        // Allocate memory on GPU, which can be mapped to RAM.
        const auto memory_info = ::vk::MemoryAllocateInfo()
            .allocationSize(memory_requirements.size())
            .memoryTypeIndex(mem_idx);
        auto memory = make_raii(
            device.allocateMemory(memory_info, nullptr),
            [&device](auto& o) { device.freeMemory(o, nullptr); });
        cout << "Memory on GPU allocated" << endl;

        // Copy data from CPU to GPU
        auto data = reinterpret_cast<uint8_t*>(
            device.mapMemory(memory, 0, memory_requirements.size(), ::vk::MemoryMapFlags()));
        auto begin = reinterpret_cast<const uint8_t*>(&matrix);
        copy(begin, begin + sizeof(matrix), data);
        device.unmapMemory(memory);
        cout << "Data copied" << endl;

        // Bind buffer to pipeline
        device.bindBufferMemory(buff, memory, 0);
        cout << "Buffer bound" << endl;
    }
    catch(const exception& err) {
        cerr << "[ERROR] " << err.what() << endl;
        return 1;
    }
}

One more note. As a lazy person, I have created template function mad::vk::make_raii, which creates RAII wrapper around an object passed as a first parameter. The second parameter of this function is a functor, which will be called in RAII wrapper destructor. In examples, I have used lambdas.

Thursday, 31 March 2016

Vulkan, first steps 03: devices

This post is the third in series of presenting API samples from Lunar's Vulkan SDK recoded to use nVidia's Open-Source Vulkan C++ API. In the first post Vulkan Instance was created. The second post presented Visual Studio project properties, and had little in common with Vulkan but helped organize samples solution. Now it is time to get back to Vulkan!

List of all Vulkan tutorial posts is here.

Second thing to do, in order to use Vulkan, is to create device. Before that, one can enumerate all Vulkan devices, which are present in system running application. The code is shown below.
#include <iostream>
#include <iomanip>
#include <vector>
#include <string>

#include <madvk/instance.h>


using namespace std;
using namespace mad::vk;

int main(int argc, char* argv[]) {
    try {
        auto instance = Instance("vulkan playground 01");
        cout << "Vulkan instance created" << endl;

        cout << "Vulkan devices:" << endl;
        std::vector<::vk::PhysicalDevice> devices;
        instance.enumeratePhysicalDevices(devices);
        for(const auto& dev : devices) {
            const auto& props = dev.getProperties();
            cout
                << std::showbase << std::internal << std::setfill('0')
                << std::hex
                << "\tID: " << props.deviceID()
                << std::noshowbase << std::right << std::setfill(' ')
                << std::dec
                << "\tname: " << props.deviceName()
                << "\ttype: " << to_string(props.deviceType())
                << "\tdriver version: " << props.driverVersion()
                << "\tapi version: "
                << ((props.apiVersion() >> 22) & 0xfff) << '.' // Major.
                << ((props.apiVersion() >> 12) & 0x3ff) << '.' // Minor.
                << (props.apiVersion() & 0xfff)                // Patch.
                << endl;
        }
    }
    catch(const std::system_error& err) {
        cerr << "[ERROR] " << err.what() << endl;
        return 1;
    }
}

There is difference in creating Vulkan instance between this post and the first one. From this post onward, I will use my RAII wrapper around vk::Instance object and not the bare object itself, to make sure that memory will be freed correctly.

Creating display device object requires two steps:
  1. Enumerating all devices and finding one, which is capable of graphical output.
  2. Creating device object with proper parameters and proper queues.
The first step is almost done in the example above and only minor corrections have to be made. The second one is similar to creating Vulkan instance, but requires vk::PhysicalDevice object from step one. Whole code is presented below. In later examples, vk::Device will be hidden in RAII wrapper.
#include <iostream>
#include <vector>
#include <algorithm>

#include <madvk/instance.h>


using namespace std;
using namespace mad::vk;

int main(int argc, char* argv[]) {
    try {
        auto instance = Instance("vulkan playground 01");
        cout << "Vulkan instance created" << endl;

        // Get all Vulkan devices
        std::vector<::vk::PhysicalDevice> devices;
        instance.enumeratePhysicalDevices(devices);

        // Find Vulkan GPU
        const ::vk::PhysicalDevice* gpu = nullptr;
        for(const auto& dev : devices) {
            const auto& qprops = dev.getQueueFamilyProperties();
            const auto it = find_if(
                qprops.cbegin(), qprops.cend(),
                [](auto& qpr) { return qpr.queueFlags() & ::vk::QueueFlagBits::eGraphics; });
            if(it != qprops.cend()) {
                gpu = &dev;
                break;
            }
        }
        if(!gpu)
            throw runtime_error("Unable to find GPU");

        // Create device object
        float priorities[] = {1.0f};
        auto queue_info = ::vk::DeviceQueueCreateInfo().queueCount(1).pQueuePriorities(priorities);
        auto device_info = ::vk::DeviceCreateInfo().queueCreateInfoCount(1).pQueueCreateInfos(&queue_info);
        auto device = gpu->createDevice(device_info, nullptr);
        cout << "Vulkan device created" << endl;

        device.destroy(nullptr);
        cout << "Vulkan device destroyed" << endl;
    }
    catch(const exception& err) {
        cerr << "[ERROR] " << err.what() << endl;
        return 1;
    }
}

If one would use Vulkan device for doing GPU calculations, then she or he should use vk::QueueFlagBits::eCompute flag in searching for proper device.

Vulkan, first steps 02: Visual Studio Project Properties

I have decided to create a Visual Studio solution with API samples from Lunar's Vulkan SDK recoded to use nVidia's Open-Source Vulkan C++ API. But for each project in the solution, I have to add: paths for Vulkan SDK, tools and libraries from it. Doing it by hand would be tedious. It is a place, where project properties come in handy! The solution presented below is inspired by "Sharing project properties in Visual C++" post.

List of all Vulkan tutorial posts is here.

In solution's directory, I have created vulkan.props file intended for storing all changes in project properties corresponding to using Vulkan SDK. Setting include directory and the additional link-time library was easy because these are shared among all configurations. But libraries directory is different for Win32 and x64 builds. There I had to use a choose construct with proper condition. The whole file is presented below.
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <ImportGroup Label="PropertySheets" />
  
  <Choose>
    <When Condition="'$(Platform)'!='x64'">
      <PropertyGroup Label="UserMacros">
        <VULKAN_BIN_DIR>$(VULKAN_SDK)\Bin32</VULKAN_BIN_DIR>
      </PropertyGroup>
    </When>
    <Otherwise>
      <PropertyGroup Label="UserMacros">
        <VULKAN_BIN_DIR>$(VULKAN_SDK)\Bin</VULKAN_BIN_DIR>
      </PropertyGroup>
    </Otherwise>
  </Choose>

  <PropertyGroup>
    <IncludePath>$(VULKAN_SDK)\Include;$(IncludePath)</IncludePath>
    <LibraryPath>$(VULKAN_BIN_DIR);$(LibraryPath)</LibraryPath>
  </PropertyGroup>
  <ItemDefinitionGroup>
    <Link>
      <AdditionalDependencies>vulkan-1.lib;%(AdditionalDependencies)</AdditionalDependencies>
    </Link>
  </ItemDefinitionGroup>
  <ItemGroup>
    <BuildMacro Include="VULKAN_BIN_DIR">
      <Value>$(VULKAN_BIN_DIR)</Value>
      <EnvironmentVariable>true</EnvironmentVariable>
    </BuildMacro>
  </ItemGroup>
</Project>


The $(VULKAN_SDK) is a system variable created by Vulkan SDK installer. Now, to reference Vulkan SDK I have only to add vulkan.props property file to each project once. Thanks to the choose construct my property file is working in all project configurations.

Wednesday, 30 March 2016

Vulkan, first steps 01

When I was at university, I was interested in games and algorithms of computer graphics. I have taken lectures on it and OpenGL. My work does not require graphics programming, and skills gained during the lectures are used mainly in photography. When The Khronos Group Inc. released Vulkan, I have decided to try it and refresh my graphics skills. Because I am c++ programmer, I have decided to use nVidia's Open-Source Vulkan C++ API instead of original c API. This post is an introduction to Open-Source Vulkan C++ API. List of all Vulkan tutorial posts is here.


The first thing, which is necessary to use Vulkan, is to create its instance. The instance is a representation of Vulkan itself. It is the main object, which allows us to interact with Vulkan and do something useful. The minimal example of creating an instance is shown below. There is only one custom param: app title Vulkan APP in line 13th. Most of the time, the default parameters from Open-Source Vulkan C++ API are used. There is one exception, which is the API version (set in line 14th). The default value is 0, which on my machine resulted in an error of missing compatible device. After setting the API version to VK_MAKE_VERSION(1, 0, 0), Vulkan instance is successfully created and destroyed.
#include <iostream>
#include <string>

// Use more c++ friendly version of Vulkan cpp
#define VKCPP_ENHANCED_MODE
#include <vulkan/vk_cpp.h>

using namespace std;

int main(int argc, char* argv[]) {
 try {
  auto app_info = vk::ApplicationInfo()
   .pApplicationName("Vulkan APP")
   .apiVersion(VK_MAKE_VERSION(1, 0, 0));
  auto instance_info = vk::InstanceCreateInfo().pApplicationInfo(&app_info);

  auto instance = vk::createInstance(instance_info, nullptr);
  cout << "Vulkan instance created" << endl;

  instance.destroy(nullptr);
  cout << "Vulkan instance destroyed" << endl;
 }
 catch(const std::system_error& err) {
  cerr << "[ERROR] " << err.what() << endl;
  return 1;
 }
}

Unfortunately, vk::Instance is not an RAII wrapper, and the instance has to be freed manually. After deciding on memory allocation policy (nullptr parameters for vk::createInstance and vk::Instance::destroy), one should create RAII wrapper around vk::Instance.