Article: Preview the future of the ue4-docker project

Significant changes are planned for ue4-docker, but one of its most important new features is already available for use today.

Tags: Docker, Unreal Engine

The ue4-docker project is the pioneering source of Dockerfiles for both Windows and Linux container images that encapsulate the Unreal Engine. Since its creation in 2018, the project has grown to encompass a wide variety of patches and tweaks to improve the Unreal Engine build experience inside containers, in addition to a suite of testing and diagnostic tools to identify issues in host system build environments. As a result of this organic growth, the codebase has begun pushing up against the limitations of its fundamental design, and a complete redesign is planned for the future.

Although a lack of feature parity between the available tooling for Windows containers and their more mature Linux counterparts is currently blocking implementation of the new design, one of the the most important features of the planned redesign has recently been added to the current ue4-docker codebase in a preliminary form. Although the new functionality is rather limited when used with Windows containers, it provides several extremely powerful capabilities for Linux containers that developers can benefit from today.

TL;DR Version:

Exporting generated Dockerfiles with custom template options (and optionally combining them into a single Dockerfile) is available in ue4-docker version 0.0.80 and newer, see the relevant section of the documentation for usage details. It works best with Linux containers.


Limitations of the current design

In its current form, ue4-docker is a monolithic tool that stores Dockerfiles inside its internal codebase and is responsible for all aspects of building container images from those Dockerfiles. This design was chosen for security purposes, since the manner in which Docker builds container images makes it inherently difficult to leverage sensitive data (such as authentication credentials) without risking its exposure. As discussed in the article Build secrets and SSH forwarding in Docker 18.09 by Tõnis Tiigi, sensitive data stored in files or environment variables during the build process of a container image will become embedded in the metadata of that image, and even if a multi-stage build is used to isolate the relevant build steps the sensitive data will still remain in the local build cache of the Docker daemon on the host filesystem.

The official solution to this build-time security dilemma is to use the special BuildKit features discussed in the article linked above, which allow secrets to be securely mounted during the build process without leaking into the resulting container image. However, the BuildKit build backend was designed for Linux containers and does not currently support Windows containers, which precludes the use of its features by ue4-docker when protecting user-supplied credentials during the build process. In order to keep credentials secure when building both Windows and Linux containers, ue4-docker implements a HTTP endpoint (called the “credential endpoint”) that is queried for credentials on-demand during the build process, as illustrated in the diagram below:

The current architecture used by ue4-docker to prevent user credentials from leaking into built container images.
The current architecture used by ue4-docker to prevent user credentials from leaking into built container images.

To prevent other local processes from accessing the credential endpoint and stealing credentials, a security token is randomly generated and injected into the container. This token is inevitably leaked in the metadata of the built images, but the one-time nature of its use renders this merely an annoyance rather than a security vulnerability. This convoluted process is currently the simplest mechanism that I am aware of which facilitates the secure use of sensitive data across both Windows and Linux containers. It is the single most brittle component of the build process and a major source of issues for developers working with ue4-docker (second only to the myriad problems that have plagued Windows containers over the past few years, particularly those that prevent the creation of large filesystem layers.)

The inclusion of the credential endpoint tightly couples the build process to ue4-docker itself, preventing the use of its Dockerfiles in environments where ue4-docker is not present or cannot run. This monolithic nature imposes a number of frustrating limitations:

These limitations have motivated the planned redesign of ue4-docker, which is discussed in the section below.

Benefits of the planned redesign

The planned redesign reframes ue4-docker as primarily taking the role of a plugin-based Dockerfile generation tool rather than a build tool. In this new design, ue4-docker provides developers with functionality to generate and customise Dockerfiles, which are then exported to the filesystem so that users can perform any desired modifications and commit the results to a version control system. These Dockerfiles can then be built independently of ue4-docker itself, in any desired build environment. This provides a wide array of benefits:

Unfortunately, it will only be feasible to implement the new design once container build tools support secure build-time secret injection for Windows containers as well as Linux containers. In particular, support from BuildKit is critical for maintaining compatibility with Docker. As a result, the redesign is currently blocked until Windows container support is added to BuildKit. However, a subset of its functionality is available today, as discussed in the section below.

Using the new features today

At the time of writing, a large amount of development work is still required to add Windows container support to BuildKit and achieve feature parity with Linux containers. As such, implementation of the ue4-docker redesign is likely to be postponed for quite some time. To help bridge the gap in the interim, I added basic support for template-driven Dockerfile generation to ue4-docker version 0.0.79 and subsequently added support for combining generated Dockerfiles into a single output file in ue4-docker version 0.0.80. These new features do not address all of the limitations of the current codebase, but serve as a starting point to begin an incremental transition towards the new design.

The simplest change is the introduction of a flag called -layout, which allows the user to specify a directory to which the Dockerfiles from the ue4-docker codebase should be copied without being built. (The name of this flag is borrowed from the Microsoft Visual Studio installer, where it provides roughly analogous functionality, and was chosen to avoid confusion with the ue4-docker export command. The name generate was also considered, but it lacked the satisfying irony of borrowing terminology from a component used exclusively in Windows container images to denote a feature that is primarily of benefit to Linux containers.) In isolation, this flag would be of little to no use, since the Dockerfiles are designed to be built by ue4-docker and by default will not function correctly in an external build environment. The key to making this export functionality useful is the introduction of a templating system that permits the introduction of alternate behaviour for use outside of ue4-docker itself.

Dockerfiles in the ue4-docker codebase are now pre-processed using the Jinja templating system prior to being exported or built, which facilitates the introduction of conditional blocks which are not possible when using the regular Dockerfile syntax. These conditional blocks are now used to gate patches and tweaks behind options that the user can specify with a flag called --opt, allowing developers to strip out code which is not necessary for newer versions of the Unreal Engine without impacting the compatibility of ue4-docker with older Engine versions. More importantly, conditional blocks are now used to provide alternative implementations of the logic for retrieving the Unreal Engine source code, and users have the ability to select which implementation will be used. The default implementation still relies on the ue4-docker credential endpoint, but the following alternative implementations can be used in external build environments:

In addition to the ability to generate customised Dockerfiles that can be built in environments where ue4-docker itself is not present, ue4-docker version 0.0.80 introduced a flag called --combine which causes the generated Dockerfiles to be combined into a single output Dockerfile. In this mode, the contents of each original Dockerfile are transformed into stages within a large multi-stage build. This is useful for scenarios where developers are only interested in the final image in a sequence (typically either ue4-minimal or ue4-full) and are not interested in retaining the intermediate ue4-build-prerequisites and ue4-source images. The generated Dockerfile will be prefixed with a comment indicating which images were included and which Jinja template options were used to generate them, which is otherwise stored in a separate JSON file when not combining generated Dockerfiles.

To demonstrate these new features, the section below walks through an example of generating a combined Dockerfile for Linux container images and building it with BuildKit. You can also find detailed usage information in the relevant section of the documentation.

Example: building a Linux container image with BuildKit

First, use the ue4-docker build command to generate a combined Dockerfile for the ue4-build-prerequisites, ue4-source and ue4-minimal images:

# Generate a combined Dockerfile for ue4-build-prerequisites, ue4-source and ue4-minimal with BuildKit build secrets support
# (Note that the Unreal Engine version specified here is actually unused, since the version will be specified at build time)
ue4-docker build 4.26.1 -layout "/path/to/outdir" --combine --no-engine --no-full --opt credential-mode=secrets

This will create a Dockerfile in the directory /path/to/outdir/combined, accompanied by the shell scripts and Python scripts used by the various patches and tweaks implemented by ue4-docker. The BuildKit build secret implementation expects to find the user’s git username and password in text files named username.txt and password.txt, which will be mounted as secrets during the build process. Note that if you are cloning the Unreal Engine source code from GitHub then you will need to use a personal access token rather than a password, but this is not the case if you are cloning from a self-hosted git repository (e.g. an internal mirror operated by your organisation.)

Once the git credentials have been placed in text files alongside the Dockerfile, you will need to build the container image using BuildKit. If you have the latest version of Docker installed then the simplest way to do this is to use the docker buildx build command:

docker buildx build \
	-t "adamrehn/ue4-minimal:4.26.1-opengl" \
	--build-arg "BASEIMAGE=nvidia/opengl:1.0-glvnd-devel-ubuntu18.04"
	--build-arg "GIT_REPO="
	--build-arg "GIT_BRANCH=4.26.1-release" \
	--build-arg "BUILD_DDC=true" \
	--build-arg "EXCLUDE_DEBUG=0" \
	--build-arg "EXCLUDE_TEMPLATES=0" \
	--secret id=username,src=username.txt \
	--secret id=password,src=password.txt \
	--progress=plain \

You need to specify values for all of the Docker build arguments that ue4-docker would ordinarily provide, including:

BuildKit will then build the image as one large multi-stage build, mounting the credential text files as secrets during the step that clones the Unreal Engine source code. Once the build is complete, the finished image will be tagged as adamrehn/ue4-minimal:4.26.1-opengl (or whatever tag value you specify when running the command) and the local build cache will contain the filesystem layers of each intermediate step for reuse by subsequent builds. Neither the finished image or the build cache will contain any trace of the sensitive git credentials.

It is worth noting that the default Docker configuration for BuildKit at the time of writing truncates log output which exceeds 1MiB in size, so it may be necessary to modify the configuration files for the Docker daemon to increase this limit in order to view the full log output during the build process. The exact steps required to perform this configuration will vary across different Linux distributions.

When you’re ready to further explore the new ue4-docker features, check out the relevant section of the documentation for full usage details.