Article: Cross-platform library integration in Unreal Engine 4

A proposed workflow for simplifying the integration of third-party libraries into projects built with Epic Games’ Unreal Engine 4.

Tags: C++, Conan, DevOps, OpenCV, Unreal Engine

Posted: 12 April 2018

This content is no longer up-to-date.

The original text of this article is preserved here for historical reasons and reflects the state of the Unreal Engine at the time that it was written in April 2018. The current version of this information can be found here and reflects the state of the Unreal Engine today.

Contents

Rationale

The ability to integrate third-party libraries into Unreal Engine 4 projects and plugins allows developers to augment the existing capabilities of the Engine with the functionality of any arbitrary codebase. A number of resources are available online that provide guidance on how to do so (see 123 for examples from the Epic Wiki alone.) However, most of these guides utilise simple manual addition of build rules to a project’s .Build.cs file (the set of rules that UnrealBuildTool uses to build a UE4 project or plugin.) This simplistic approach quickly becomes verbose when integrating multiple libraries and their dependencies, particularly when supporting versions for multiple operating systems.

In addition to the poor scalability inherent in the approach described by most existing materials, such guides rarely address the more complex issues that can be encountered when integrating third-party libraries. Nuances related to compiler toolchain invocations, linker behaviour, and Application Binary Interfaces (ABIs) can result in obstacles that require significant developer effort to diagnose and address. A manual workflow requiring manual solutions to these issues is entirely insufficient. There is a clear need for a workflow that automates as much of the library integration process as possible. Such a workflow should integrate smoothly with UnrealBuildTool and scale to arbitrary numbers of dependencies, whilst simultaneously abstracting away the solutions to the key issues that developers may face during the integration of third-party libraries.

Key issues

Symbol interposition

Symbol interposition is a linker feature that allows executables to handle the existence of multiple instances of a given symbol within a process’s address space. When attempting to resolve a reference to a symbol for which multiple instances are available, a linker’s default behaviour will typically be to select the first instance that it finds. Under macOS and Linux, the dlsym() function supports the RTLD_DEFAULT and RTLD_NEXT flags, which provide functionality to control this symbol resolution behaviour. 4 Equivalent functionality is not provided by the Windows API because DLLs must explicitly export public symbols and applications must either explicitly link against the corresponding import libraries or else retrieve symbols programmatically.

Unreal Engine 4 bundles a number of prebuilt third-party libraries in the Engine/Source/ThirdParty subdirectory of the Engine’s source tree. The prebuilt binaries for the majority of these dependencies are static libraries. As a consequence of symbol interposition, linking against additional third-party libraries which declare symbols with the same name as those that exist within a UE4-bundled library can cause unintended collisions. As an example, consider a scenario in which we are linking against a static build of the OpenCV computer vision library. If OpenCV has been built against a different version of libpng than the version which is bundled with UE4, then we will encounter a symbol collision when attempting to read a PNG image using the OpenCV API. The linker will select the symbols from the UE4 version of libpng instead of those from the OpenCV version, and the library version mismatch will result in an error.

Symbol interposition issues can occur under the following circumstances:

The most effective way to prevent symbol interposition issues is to ensure that all third-party libraries are built against the UE4-bundled versions of their own dependencies (where applicable.) Although this does require building each library from source, the use of a package management system and associated repository can restrict the occurrence of such builds to once per combination of operating system and Engine version.

STL and libc++ issues under Linux

The default implementation of the C++ standard library that ships with most Linux distributions is libstdc++, the GNU C++ Library. libstdc++ is licensed under the GNU General Public License, a strong copyleft license. To allow non-GPL licensed software to link against libstdc++, its license includes the GCC Runtime Library Exception, which provides additional clauses stating that any independent code that links against the runtime library is exempt from the terms of the GPL, so long as a set of eligibility criteria are met. However, the extent to which it is permissible to distribute libstdc++ itself (including its accompanying header files) alongside non-GPL software appears to be a matter of contention.

The Unreal Engine does not utilise libstdc++. Linux builds of the Engine and all of its bundled third-party libraries are built against libc++, the implementation of the C++ standard library from the LLVM project. libc++ is dual licensed under the MIT license and the BSD-like UIUC License, which are permissive, non-copyleft licenses. libc++ and its headers are bundled with the Engine in the ThirdParty directory of the Engine’s source tree. This choice of runtime library is ostensibly due to legal concerns regarding the ability to distribute libstdc++ with the Engine 56.

libc++ is not ABI compatible with libstdc++, except for a small handful of low-level features. As a consequence, any code that has been compiled with libc++ and utilises features of the C++ standard library (such as the container classes from the STL) cannot interoperate with other code that uses these features and has been compiled against libstdc++ (and vice versa.) The practical upshot of this incompatibility is that any third-party libraries which have been built against libstdc++ can only communicate with UE4 C++ code through the use of pure C interfaces. In order to achieve full C++ interoperability, it is necesary to compile third-party libraries against libc++ (preferably the version of libc++ that is bundled with the Engine, for maximum compatibility.) This is effectively an extension of the solution to symbol interposition (building libraries from source against the UE4-bundled versions of their dependencies), whereby we treat libc++ as a dependency of all libraries when building under Linux.

Workflow overview

To address the issues described in the sections above, I propose a workflow that automates the integration of third-party libraries with UE4 through the use of the Conan package management system. A high-level overview of the proposed workflow is depicted in Figure 1:

Figure 1: High-level overview of the proposed workflow
Figure 1: High-level overview of the proposed workflow

The workflow allows an Unreal Engine 4 project or plugin to seamlessly consume prebuilt third-party libraries, which have in turn been compiled against dependencies bundled in the Engine/Source/ThirdParty subdirectory of the Engine’s source tree (including libc++ under Linux.) To facilitate this workflow, I have implemented several key components:

The Conan package manager was selected for the implementation of the proposed workflow for three reasons:

Step 1: Conan wrappers for UE4-bundled third-party libraries

Note: the full code from this section is available in the conan-ue4cli GitHub repository.

The first step to integrating UE4-bundled libraries into any external system is to retrieve the relevant build flags from UnrealBuildTool. To abstract away the details of locating and invoking UnrealBuildTool, I created the ue4cli command-line tool. ue4cli handles the detection of installed Engine versions and automatically locates the relevant batch files and shell scripts that provide access to the internals of the UE4 build system. The command-line interface of ue4cli makes it simple to query the list of UE4-bundled third-party libraries and retrieve the necessary build flags for consumption by any external build system. Additional flags specific to the CMake build system are also supported.

Consuming the output of ue4cli from within Conan is extremely straightforward. The conan-ue4cli GitHub repository contains a script to generate wrapper packages for UE4-bundled libraries. The script performs the following steps:

Each generated wrapper package contains only the name of the library. All other information is retrieved dynamically when the package is resolved during the conan install process of a consuming conanfile. For example, the generated package for zlib looks like this:

from conans import ConanFile

class zlibConan(ConanFile):
    name = "zlib"
    version = "ue4"
    settings = "os", "compiler", "build_type", "arch"
    requires = (
        "ue4lib/0.0.1@adamrehn/generated",
        "libcxx/ue4@adamrehn/generated"
    )
    
    def package_info(self):
        from ue4lib import UE4Lib
        details = UE4Lib("zlib")
        self.cpp_info.includedirs = details.includedirs()
        self.cpp_info.libdirs = details.libdirs()
        self.cpp_info.libs = details.libs()
        self.cpp_info.defines = details.defines()
        self.cpp_info.cppflags = details.cxxflags()
        self.cpp_info.sharedlinkflags = details.ldflags()
        self.cpp_info.exelinkflags = details.ldflags()

Listing 1: The generated conanfile.py for the zlib wrapper package.

The package_info() method will be called when Conan consumes the package. The generated code simply invokes the UE4Lib class from the base ue4lib package, which in turn queries ue4cli for the relevant information. Under Windows and macOS, this simple mechanism is all that is necessary in order to consume UE4-bundled libraries. Under Linux, however, it is also necessary to build against the UE4-bundled libc++, which requires special treatment.

In theory, all that is necessary in order to build against the UE4-bundled libc++ and its accompanying headers is to specify the correct compiler and linker flags. These flags are listed most clearly in the Linux build script for OpenEXR (login required) from the Engine source tree, and are also hardcoded in ue4cli (the string $3RDPARTY refers to the Engine/Source/ThirdParty subdirectory of the Engine’s source tree):

CXXFLAGS="-nostdinc++ -I$3RDPARTY/Linux/LibCxx/include -I$3RDPARTY/Linux/LibCxx/include/c++/v1"

LDFLAGS="-nodefaultlibs $3RDPARTY/Linux/LibCxx/lib/Linux/x86_64-unknown-linux-gnu/libc++.a $3RDPARTY/Linux/LibCxx/lib/Linux/x86_64-unknown-linux-gnu/libc++abi.a -lm -lc -lgcc_s -lgcc"

Listing 2: The flags required to build against the UE4-bundled libc++ and its accompanying headers.

Unfortunately, the order in which the linker flags appear in a compilation command is crucial to their functioning. If the linker flags appear after the input files being compiled or linked, then all references to symbols in libc++.a and libc++abi.a will be resolved correctly. However, if the linker flags appear before the input files, the link operation will fail due to unresolved symbols. This ordering requirement is problematic when working with build systems such as autotools or CMake that generate compiler commands without providing any control over linker flag ordering. CMake in particular is notorious for placing linker flags before input files in its generated compiler commands.

To prevent meddling by the build system, the libcxx package does not directly specify the compiler and linker flags for libc++. Instead, it provides two wrapper scripts, clang.py and clang++.py, that act as an intermediary between the build system and the real compiler. The build system is pointed to these scripts using the CC and CXX environment variables, and the scripts then inject the required flags before invoking the real compiler and returning its output to the build system. The relevant code from the libcxx conanfile.py (full source here) is as follows:

def package_info(self):
	from ue4lib import UE4Lib
	if self.settings.os == "Linux":
		
		# Gather our custom compiler and linker flags for building against UE4 libc++
		libcxx = UE4Lib("libc++")
		compilerFlags = libcxx.combined_compiler_flags()
		linkerFlags = libcxx.combined_linker_flags()
		
		# Inject our custom clang wrapper into the relevant build-related environment variables
		# (This allows the use of UE4 libc++ to be transparent when building consumer packages)
		self.env_info.REAL_CC = os.environ["CC"] if "CC" in os.environ else "cc"
		self.env_info.REAL_CXX = os.environ["CXX"] if "CXX" in os.environ else "c++"
		self.env_info.CLANG_INTERPOSE_CXXFLAGS = compilerFlags
		self.env_info.CLANG_INTERPOSE_LDFLAGS = linkerFlags
		self.env_info.CLANG_INTERPOSE_CC = os.path.join(self.package_folder, "bin/clang.py")
		self.env_info.CLANG_INTERPOSE_CXX = os.path.join(self.package_folder, "bin/clang++.py")
		self.env_info.CC = self.env_info.CLANG_INTERPOSE_CC
		self.env_info.CXX = self.env_info.CLANG_INTERPOSE_CXX
		self.env_info.LDFLAGS = "---link"

Listing 3: The key lines from the conanfile.py for the libcxx wrapper package.

The linker flag ---link is specified so that clang.py and clang++.py know when to inject the libc++ linker flags in addition to the necessary compiler flags (injecting the linker flags indiscriminately can cause issues with some of the autotools and CMake compiler detection routines, so it is best to only include them when an actual link operation is taking place.) The relevant code common to clang.py and clang++.py that receives this information is shown below (full source here):

def interpose(cxx):
	
	# Filter out any `-stdlib=<LIB>` flags from the supplied command-line arguments
	args = list([arg for arg in sys.argv[1:] if arg.startswith("-stdlib=") == False])
	
	# Prepend our custom compiler flags
	args = shlex.split(os.environ["CLANG_INTERPOSE_CXXFLAGS"]) + args
	
	# If this is a link invocation, append our custom linker flags
	if '---link' in args:
		args.remove('---link')
		args.extend(shlex.split(os.environ["CLANG_INTERPOSE_LDFLAGS"]))
	
	# Forward all arguments to the real clang executable
	realClang = tools.which(os.environ["REAL_CXX"] if cxx == True else os.environ["REAL_CC"])
	sys.exit(subprocess.call([realClang] + args))

Listing 4: The code used by clang.py and clang++.py to inject build flags.

The use of these wrapper scripts makes the presence of the UE4-bundled libc++ entirely transparent to any build system that is used to build packages which depend on UE4-bundled libraries. This is particularly important due to the build system-agnostic nature of Conan, since any arbitrary build system could conceivably be used to consume the UE4 wrapper packages. The platform check for Linux which enables this behaviour also allows the libcxx package to be used harmlessly under Windows and macOS, as it is a no-op on these platforms.

Step 2: Building libraries against UE4 versions of dependencies

Note: the full code from this section is available in the ue4-opencv-demo GitHub repository.

The wrapper packages for UE4-bundled third-party libraries are of little use in isolation - after all, UE4 projects and plugins can already link against the bundled libraries simply by specifying them as dependencies in the relevant .Build.cs file. The power of the workflow lies in the ability to build external third-party libraries against these UE4-bundled dependencies and then link against the custom-built versions of these external libraries.

Consuming the generated wrapper packages in the Conan recipe for another package is extremely straightforward, and requires minimal boilerplate code. Following on from the example of the OpenCV computer vision library disussed in the section Symbol interposition, we can create a custom build of OpenCV using the UE4-bundled zlib and libpng like so (full source here):

from conans import ConanFile, CMake, tools

class OpenCVUE4Conan(ConanFile):
    name = "opencv-ue4"
    version = "3.3.0"
    url = "https://github.com/adamrehn/ue4-opencv-demo"
    description = "OpenCV custom build for UE4"
    settings = "os", "compiler", "build_type", "arch",
    generators = "cmake",
    requires = (
        "libcxx/ue4@adamrehn/generated",
        "zlib/ue4@adamrehn/generated",
        "UElibPNG/ue4@adamrehn/generated"
    )
    
    def cmake_flags(self):
        flags = [
            "-DWITH_PNG=ON",
            "-DBUILD_ZLIB=OFF", # Don't use bundled zlib, since we use the version from UE4
            "-DBUILD_PNG=OFF"   # Don't use bundled libpng, since we use the version from UE4
        ]
        
        # Append the flags to ensure OpenCV's FindXXX modules use our UE4-specific dependencies
        zlib = self.deps_cpp_info["zlib"]
        libpng = self.deps_cpp_info["UElibPNG"]
        flags.append("-DPNG_PNG_INCLUDE_DIR=" + libpng.includedirs[0])
        flags.append("-DZLIB_INCLUDE_DIR=" + zlib.includedirs[0])
        flags.append("-DZLIB_LIBRARY=" + zlib.libs[0])
        flags.append("-DPNG_LIBRARY=" + libpng.libs[0])
        return flags
    
    def source(self):
        self.run("git clone --depth=1 https://github.com/opencv/opencv.git -b {}".format(self.version))
    
    def build(self):
        
        # Under Linux, restore CC and CXX if the current Conan profile has overridden them
        from libcxx import LibCxx
        LibCxx.set_vars(self)
        
        # Build OpenCV
        cmake = CMake(self)
        cmake.configure(source_folder="opencv", args=self.cmake_flags())
        cmake.build()
        cmake.install()
    
    def package_info(self):
        self.cpp_info.libs = tools.collect_libs(self)

Listing 5: An example conanfile.py for building OpenCV against the UE4-bundled zlib and libpng.

The deps_cpp_info ConanFile attribute allows us to access the build flags from our dependency packages. In the case of the CMake build process for OpenCV, we need to pass the relevant information to the FindZLIB and FindPNG modules to ensure the UE4-bundled versions of zlib and libpng are found. In general, it should be sufficient to simply add the relevant include directories to the CMAKE_INCLUDE_PATH variable and the library directories to the CMAKE_LIBRARY_PATH variable to allow all FindXXX modules to see the UE4-bundled libraries. However, the FindPNG module appears to be somewhat finnicky about using these paths in certain versions of CMake, and must be forced to use the correct paths by setting the variables specific to that module.

As can be seen in the build() method, only two lines of boilerplate code are required to enable the use of libc++ under Linux. The libcxx package actually sets the CC and CXX environment variables in its package_info() method, and ideally there should be no further work required to retain these changes. Unfortunately, if the CC or CXX environment variables are specified in the Conan profile then these values will override any changes specified by dependency packages. The environment variables could also conceivably be modified by other dependency packages that are loaded after libcxx. The LibCxx.set_vars() function re-applies the changes from the libcxx package to ensure they survive any external interference. If you know for certain that neither the Conan profile you are using or any of your recipe’s other dependency packages modify the values of the CC and CXX environment variables then you can omit the two boilerplate lines.

The one piece of information not contained in the example Conan recipe is the version of UE4 against which the package was built. There are a number of potential candidates for specifying this information:

Storing the Engine version string in the package channel offers the most advantages. The build.py script from the ue4-opencv-demo repository does exactly this:

#!/usr/bin/env python3
import subprocess

# Query ue4cli for the UE4 version string
proc = subprocess.Popen(["ue4", "version", "short"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
(stdout, stderr) = proc.communicate(input)
if proc.returncode != 0:
    raise Exception("failed to retrieve UE4 version string")

# Build the Conan package, using the Engine version as the channel name
subprocess.call(["conan", "create", ".", "adamrehn/{}".format(stdout.strip()), "--profile", "ue4"])

Listing 6: The Python build script for the custom OpenCV package, which uses the UE4 version string as the package name.

Note that the ue4cli invocation is ue4 version short, which returns only the major and minor version numbers (e.g. “4.19”) and not the patch number (e.g. “4.19.1”). This is to accommodate the re-use of the built package when the Engine receives a hotfix, as such updates rarely modify the versions of the bundled third-party libraries.

Step 3: Consuming Conan packages in an Unreal project or plugin

Note: the full code from this section is available in the ue4-opencv-demo GitHub repository.

One of the strengths of the Conan package manager is that it is build system-agnostic. New build systems can simply be added as new generators, and the built-in json generator produces JSON files containing all of the relevant build flags for consumption by build systems for which explicit support does not yet exist. Integrating Conan into UnrealBuildTool involves two simple steps:

Following on from our OpenCV example in the previous section, the contents of our conanfile.txt would be as follows:

[requires]
opencv-ue4/3.3.0@adamrehn/4.19

[generators]
json

Listing 7: The contents of the conanfile.txt for our OpenCV example.

Consuming the JSON output presents a slight problem due to the manner in which UnrealBuildTool processes .Build.cs files. Each file is compiled into a .NET assembly and then dynamically loaded and invoked during the build process. As of Unreal Engine 4.19, the only assembly references that are included by default are System and the UnrealBuildTool assembly itself 7. Fortunately, UnrealBuildTool includes JSON parsing functionality in the Tools.DotNETCommon namespace. This makes it straightforward to simply leverage the UBT-provided functionality (although this is an implementation detail that could potentially change in future Engine versions.) As such, the .Build.cs file for our OpenCV example looks like so:

using System.IO;
using UnrealBuildTool;
using System.Diagnostics;

//For Tools.DotNETCommon.JsonObject and Tools.DotNETCommon.FileReference
using Tools.DotNETCommon;

public class OpenCVDemo : ModuleRules
{
	private string ModuleRoot {
		get { return Path.GetFullPath(Path.Combine(ModuleDirectory, "../..")); }
	}
	
	private bool IsWindows(ReadOnlyTargetRules target) {
		return (target.Platform == UnrealTargetPlatform.Win32 || Target.Platform == UnrealTargetPlatform.Win64);
	}
	
	private void ProcessDependencies(string depsJson, ReadOnlyTargetRules target)
	{
		//We need to ensure libraries end with ".lib" under Windows
		string libSuffix = ((this.IsWindows(target)) ? ".lib" : "");
		
		//Attempt to parse the JSON file
		JsonObject deps = JsonObject.Read(new FileReference(depsJson));
		
		//Process the list of dependencies
		foreach (JsonObject dep in deps.GetObjectArrayField("dependencies"))
		{
			//Add the header and library paths for the dependency package
			PublicIncludePaths.AddRange(dep.GetStringArrayField("include_paths"));
			PublicLibraryPaths.AddRange(dep.GetStringArrayField("lib_paths"));
			
			//Add the preprocessor definitions from the dependency package
			PublicDefinitions.AddRange(dep.GetStringArrayField("defines"));
			
			//Link against the libraries from the package
			string[] libs = dep.GetStringArrayField("libs");
			foreach (string lib in libs)
			{
				string libFull = lib + ((libSuffix.Length == 0 || lib.EndsWith(libSuffix)) ? "" : libSuffix);
				PublicAdditionalLibraries.Add(libFull);
			}
		}
	}
	
	public OpenCVDemo(ReadOnlyTargetRules Target) : base(Target)
	{
		//Link against our engine dependencies
		PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs;
		PublicDependencyModuleNames.AddRange(new string[] { "Core", "CoreUObject", "Engine", "InputCore" });
		
		//Install third-party dependencies using conan
		Process.Start(new ProcessStartInfo
		{
			FileName = "conan",
			Arguments = "install . --profile ue4",
			WorkingDirectory = ModuleRoot
		})
		.WaitForExit();
		
		//Link against our conan-installed dependencies
		this.ProcessDependencies(Path.Combine(ModuleRoot, "conanbuildinfo.json"), Target);
    }
}

Listing 8: The contents of the .Build.cs file for our OpenCV example.

When the project is built, the code from the .Build.cs file will automatically invoke Conan and pass the generated build flags to UnrealBuildTool. So long as the specified packages are available in a repository that Conan is configured to use, the required libraries will be downloaded and linked against automatically. UnrealBuildTool will also propagate the include paths for the dependency packages when generating project files, so code completion in all IDEs supported by UE4 will also function correctly when including headers from the linked libraries.

Note that no runtime dependency information is specified (via PublicDelayLoadDLLs or similar properties.) This is because OpenCV was built as a static library, and so introduces no additional runtime dependencies. If you are working with shared libraries, it will be necessary to modify the ProcessDependencies() method to identify shared libraries and process them accordingly. However, since the Conan-based workflow makes it simple to build and consume static libraries, there is little reason to build shared libraries unless required to do so by the license terms of a dependency (e.g. the “Combined Works” section of the GNU Lesser General Public License.)

The code in the .Build.cs file completes the information loop from UnrealBuildTool to Conan and back. This provides seamless integration between Conan and the UE4 build process across all platforms, significantly reducing the developer effort required to integrate third-party libraries. With the underlying details abstracted away, developers are free to focus on creating applications and games that combine the functionality of external libraries with the power of Unreal Engine 4.

Future directions

The proposed workflow is just the first step in enhancing the development pipeline for UE4 projects and plugins that integrate third-party libraries. This workflow serves as the foundation for future work to enable a Continuous Integration (CI) pipeline for projects utilising external libraries. Going forward, there are a number of possibilities to be examined and addressed:

I look forward to exploring all of these possibilities as well as those that have yet to present themselves, and to seeing what can be created by developers who are empowered by these new tools and workflows.

References

  1. Linking Static Libraries Using The Build System - Epic Wiki ↩

  2. Integrating OpenCV Into Unreal Engine 4 - Epic Wiki ↩

  3. Integrating a third party library in the engine - Epic Wiki ↩

  4. dlsym POSIX manual page ↩

  5. Linker error with C++ std lib on Linux - UE4 AnswerHub ↩

  6. Compile Unreal with gcc and system STL for linux - UE4 AnswerHub ↩ ↩2

  7. UnrealBuildTool source code - DynamicCompilation.cs (login required) ↩