Article: Pixel Streaming in Linux containers

Linux support for the Unreal Engine’s Pixel Streaming system is now available to all Engine licensees.

Tags: C++, Docker, Unreal Engine

Note: this article will continue to be updated as our implementation of Linux-enabled Pixel Streaming expands to encompass more versions of the Unreal Engine and continues to progress towards inclusion in the upstream Unreal Engine source code.

Contents

Background

Epic Games describes the Unreal Engine’s Pixel Streaming system as the “ideal solution for distributing real-time, interactive content to multiple types of devices”. 1 However, a significant limitation of the official Pixel Streaming implementation is that it only supports rendering on Windows host systems and relies on APIs that are not supported by Microsoft’s experimental implementation of GPU acceleration inside Windows Server containers, precluding its use in a container-based environment. The inability to deploy Pixel Streaming applications inside GPU-accelerated containers increases both the complexity and cost of running these applications at scale in the cloud.

In November 2019, myself and my colleague Aidan Possemiers began work on implementing Linux support for the Pixel Streaming system in Unreal Engine 4.23 and released the video shown below to demonstrate our progress. I’m happy to report that our implementation is now available for use by all Unreal Engine licensees, and will also be available out-of-the-box in the cloud-hosted version of the upcoming Admiral CI/CD system for developers who wish to utilise CI/CD pipelines for their Pixel Streaming applications without the need for building the relevant container images themselves.

The sections below demonstrate how to get started with Pixel Streaming in Linux containers today.

Where to get the code and file issues

The code for Linux-enabled Pixel Streaming is available in my fork of the UnrealEngine GitHub repository (GitHub login required), which contains separate branches for each supported version of the Unreal Engine:

A separate public repository has been created to provide the issue tracker for the Linux Pixel Streaming code: https://github.com/adamrehn/pixel-streaming-linux. If you encounter any problems, please file your issues against this repository.

Building container images

You can build container images with Linux support for Pixel Streaming using ue4-docker, in the same manner that you would create images for any custom version of the Engine:

ue4-docker build \
  custom:4.23.1-pixelstreaming \                        # Tag the image as adamrehn/ue4-full:4.23.1-pixelstreaming
  -repo=https://github.com/adamrehn/UnrealEngine.git \  # Use Adam's fork of the Unreal Engine
  -branch=4.23.1-pixelstreaming                         # Use the branch for the Engine version we are targeting

You can then use a Docker multi-stage build to build and package your Pixel Streaming application inside the created container image, copying the packaged files into a new container image based on the lightweight ue4-runtime base image at the end of the process. (See the relevant section of the Unreal Containers community hub documentation for more details on this process.)

An example Dockerfile might look something like this:

# Perform the build in an Unreal Engine container image that includes the Engine Tools and Linux support for Pixel Streaming
FROM adamrehn/ue4-full:4.23.1-pixelstreaming AS builder

# Clone or copy the source code for your Unreal project here, e.g.:
RUN git clone --progress --depth 1 https://github.com/user/project.git /tmp/project
# or:
COPY --chown=ue4:ue4 . /tmp/project

# Build and package our Unreal project
# (We're using ue4cli for brevity here, but we could just as easily be calling RunUAT directly)
WORKDIR /tmp/project
RUN ue4 package

# Copy the packaged files into a container image that doesn't include any Unreal Engine components
# (Note that the relevant Pixel Streaming files will be copied too, since they're automatically staged with the packaged files)
FROM adamrehn/ue4-runtime:latest
COPY --from=builder --chown=ue4:ue4 /tmp/project/dist/LinuxNoEditor /home/ue4/project

# Enable the NVIDIA driver capabilities required by the NVENC API
ENV NVIDIA_DRIVER_CAPABILITIES ${NVIDIA_DRIVER_CAPABILITIES},video

# Create a symbolic link to the path where libnvidia-encode.so.1 will be mounted, since UE4 seems to ignore LD_LIBRARY_PATH
# (Replace "ProjectName" with the name of your actual Unreal project)
ln -s /usr/lib/x86_64-linux-gnu/libnvidia-encode.so.1 /home/ue4/project/ProjectName/Binaries/Linux/libnvidia-encode.so.1

To run the container image produced by this Dockerfile your host system will need an NVIDIA GPU, the latest NVIDIA binary drivers, and the NVIDIA Container Toolkit (formerly known as NVIDIA Docker.) For more information about running Linux containers with GPU acceleration, see the NVIDIA Container Toolkit page of Unreal Containers community hub documentation.

How you run the packaged Pixel Streaming application inside the container will vary depending on the version of the Unreal Engine being used. For Unreal Engine 4.23, you will need to run the packaged project itself, the WebRTCProxy program (which was subsequently merged into the Pixel Streaming plugin itself in Unreal Engine 4.24) and a signalling server. For more information, see the official Pixel Streaming documentation.

References