Build Systems

RavEngine Development Blog

Home Games Software In Your Browser Animation Tutorials Miscellaneous

5: Build Systems

Last Updated: 10/3/2020

Note: This is NOT engine documentation. I write these articles as I learn. As such, information present here may be incorrect or out-of-date. I'm leaving these pages online for historical purposes only.

In the past, I manually created IDE projects for each platform that I supported. Typically, I supported macOS and Windows, with some Linux support. This practice was fine for small projects, like my Minecraft Sounds Extractor, but with a project the size of a game engine, it could not work. In general, this practice has a few main issues:

  1. IDE configuration must be manually repeated for every supported platform
  2. It forces the client programmers to use a specific toolchain (Xcode on Mac, Visual Studio on Windows) rather than use what they are comfortable with
  3. Each build system has a different way of doing things, all of which must be learned
  4. Client programmers must deal with all of the aforementioned problems

Since I am writing my engine in C++, I have access to the wealth of open source C++ libraries that have been developed over the lifetime of C++. Indeed, if you can think it, there is a C++ library for it. Mostly. There may be a library, but not the way that you want. For each task I evaluated multiple libraries and weighed their ease of use against the time cost for implementing a custom solution.

Ultimately I decided to provide a unified interface which may or may not call into a library underneath. Ideally, a client programmer should not need to know or care that a library is in use, and this also leaves me free to switch out underlying libraries as long as I keep the API's behavior the same.

Scratching the surface of the libraries I researched for this project:

As I was researching libraries to use for my project, I noticed that the vast majority used as system named CMake to generate an IDE project of your choice. I didn't know what it was, and used it to generate the projects so that I could link them all together manually. However, I found that IDEs weren't designed to do this and so I ran into many strange errors when trying to compile my engine, many of which only happened sometimes and were not repeatable.

I decided that I would learn how to use CMake, and make it the main build system of my engine. This was probably the best decision I have made in the making of this project. Yes, CMake is not perfect and can be obtuse at times, but all of the aforementioned main issues disappeared. In addition, I can automate aspects of the build process of my engine in ways that would be difficult to do in a cross-platform way.

Automating shader compilation

CMake allows the creation of custom targets. With these targets, one can execute command line calls. I leverage this to compile shaders and pack resources all completely automatically. To compile shaders, first the client programmer must inform the engine about their shader files, through a separate cmake file: CMakeLists.txt. 

# debug wireframes shader
declare_shader("debug" "${CMAKE_CURRENT_LIST_DIR}/vs_debug.glsl" "${CMAKE_CURRENT_LIST_DIR}/fs_debug.glsl" "${CMAKE_CURRENT_LIST_DIR}/debug_varying.def.hlsl")

# deferred geometry material
declare_shader("pbrmaterial" "${CMAKE_CURRENT_LIST_DIR}/vs_deferred_geo.glsl" "${CMAKE_CURRENT_LIST_DIR}/fs_deferred_geo.glsl" "${CMAKE_CURRENT_LIST_DIR}/deferred_varying.def.hlsl")

declare_shader("deferred_blit" "${CMAKE_CURRENT_LIST_DIR}/deferred_blit.vsh" "${CMAKE_CURRENT_LIST_DIR}/deferred_blit.fsh" "${CMAKE_CURRENT_LIST_DIR}/blit_varying.def.hlsl")

# lighting shaders
declare_shader("pointlightvolume" "${CMAKE_CURRENT_LIST_DIR}/pointlight.vsh" "${CMAKE_CURRENT_LIST_DIR}/pointlight.fsh" "${CMAKE_CURRENT_LIST_DIR}/pointlight_varying.def.hlsl")
Afterwards, the client programmer simply tells the engine about this cmake file: 
file(GLOB meshes "${sample_dir}/meshes/*.obj")
file(GLOB textures "${sample_dir}/textures/*")
file(GLOB shaders "${sample_dir}/shaders/*.cmake")

pack_resources(TARGET "${APPNAME}" 
    MESHES ${meshes}
    SHADERS ${shaders}
    TEXTURES ${textures}
)
CMake then knows to execute my shader compiler on the files in that directory as a pre-build action. Only files that have changed are evaluated each build. Of course, I did not write a shader compiler. Instead, I call into the BGFX shader compiler and pass it the correct arguments, based on the current target platform. This process is hidden from the client programmer, making the engine easier to use. To the client programmer, there is only one compilation step.

Packing resources

To consolidate resources for easier loading within the engine across platforms, RavEngine packs all game assets into a single .zip file, which it then mounts like a disk at runtime. It builds this asset package automatically in a similar method to the shader compilation step. The pack_resources command instructs the engine which files to pack and what types they are. CMake executes a second tool I wrote which takes all the assets and creates a compressed Zip file. On Mac, this zip file is placed inside the app bundle, and on Linux, I plan to include it inside the .appimage.

If you want to learn CMake quickly and easily, I wrote a tutorial, which you can read here: CMake Made Easy

Next up: Events, Templates and Interfaces