Article: Pixel Streaming in Linux containers

Tags: C++, Docker, Unreal Engine

Posted: 12 February 2020

Contents

• Background
• Where to get the code and file issues
• Building container images
• References

Background

Epic Games describes the Unreal Engine’s Pixel Streaming system as the “ideal solution for distributing real-time, interactive content to multiple types of devices”. ¹ However, a significant limitation of the official Pixel Streaming reference implementation is that it only supports rendering on Windows host systems and relies on APIs that are not supported by Microsoft’s experimental implementation of GPU acceleration inside Windows Server containers, precluding its use in a container-based environment. The inability to deploy Pixel Streaming applications inside GPU-accelerated containers increases both the complexity and cost of running these applications at scale in the cloud.

In November 2019, myself and my colleague Aidan Possemiers began work on implementing Linux support for the Pixel Streaming system in Unreal Engine 4.23 and released the video shown below to demonstrate our progress. I’m happy to report that our implementation (dubbed “Pixel Streaming for Linux”) is now available for use by all Unreal Engine licensees, and will also be available for use in the upcoming Admiral CI/CD system for developers who wish to utilise automated CI/CD pipelines for their Pixel Streaming applications.

The sections below demonstrate how to get started with Pixel Streaming for Linux in GPU-accelerated Linux containers today.

Where to get the code and file issues

The source code of Pixel Streaming for Linux is available in different locations based on the supported version of the Unreal Engine:

• Unreal Engine 4.23: the code for this version can be found in the branch 4.23.1-pixelstreaming of my fork of the UnrealEngine GitHub repository (GitHub login required). This version supports OpenGL only, not Vulkan.

• Unreal Engine 4.24: Epic Games made a number of significant architectural changes in the way Pixel Streaming is implemented in 4.24, so our implementation of Linux support for 4.23 could not be simply ported over without major alterations. Although we began early work on a 4.24 port, we ultimately scrapped this version in favour of focussing all of our attention on 4.25 instead, which is the first version of the Unreal Engine to support offscreen rendering with Vulkan under Linux.

• Unreal Engine 4.25: the code for this version can be found in the branch 4.25-pixelstreaming of Aidan’s fork of the UnrealEngine GitHub repository (GitHub login required). This version supports Vulkan only, not OpenGL. Note that NVIDIA CUDA is required in order to build and run this version, since CUDA is used to implement the interop between Vulkan and the NVENC video encoder.

Note that Unreal Engine 4.25 is the first Engine version to support offscreen rendering with Vulkan under Linux, and is also the version that deprecated OpenGL, so each version of Pixel Streaming for Linux is limited to only the rendering backend supported by the corresponding Engine version.

A separate public repository has been created to provide the issue tracker for Pixel Streaming for Linux: https://github.com/adamrehn/pixel-streaming-linux. If you encounter any problems, please file your issues against this repository.

Building container images

You can build container images with Pixel Streaming for Linux support using ue4-docker, in the same manner that you would create images for any custom version of the Engine.

To build container images for the Unreal Engine 4.23 version of Pixel Streaming for Linux:

ue4-docker build \
  custom:4.23.1-pixelstreaming \                        # Tag the image as adamrehn/ue4-full:4.23.1-pixelstreaming
  -repo=https://github.com/adamrehn/UnrealEngine.git \  # Use Adam's fork of the Unreal Engine
  -branch=4.23.1-pixelstreaming \                       # Use the branch for the Engine version we are targeting
  --no-engine                                           # Don't build the ue4-engine image, just source, minimal and full

To build container images for the Unreal Engine 4.25 version of Pixel Streaming for Linux:

ue4-docker build \
  custom:4.25-pixelstreaming \                                 # Tag the image as adamrehn/ue4-full:4.25-pixelstreaming
  -repo=https://github.com/ImmortalEmperor/UnrealEngine.git \  # Use Aidan's fork of the Unreal Engine
  -branch=4.25-pixelstreaming \                                # Use the branch for the Engine version we are targeting
  --cuda=10.2 \                                                # Use a base image with NVIDIA CUDA 10.2
  --no-engine                                                  # Don't build the ue4-engine image, just source, minimal and full

You can then use a Docker multi-stage build to build and package your Pixel Streaming application inside the created container image, copying the packaged files into a new container image based on the lightweight ue4-runtime base image at the end of the process. (See the relevant section of the Unreal Containers community hub documentation for more details on this process.)

An example Dockerfile might look something like this:

# Perform the build in an Unreal Engine container image that includes the Engine Tools and Pixel Streaming for Linux
FROM adamrehn/ue4-full:4.25-pixelstreaming AS builder

# Clone or copy the source code for your Unreal project here, e.g.:
RUN git clone --progress --depth 1 https://github.com/user/project.git /tmp/project
# or:
COPY --chown=ue4:ue4 . /tmp/project

# Build and package our Unreal project
# (We're using ue4cli for brevity here, but we could just as easily be calling RunUAT directly)
WORKDIR /tmp/project
RUN ue4 package

# Copy the packaged files into a container image that includes CUDA but doesn't include any Unreal Engine components
FROM adamrehn/ue4-runtime:18.04-cudagl10.2
COPY --from=builder --chown=ue4:ue4 /tmp/project/dist/LinuxNoEditor /home/ue4/project

# Enable the NVIDIA driver capabilities required by the NVENC video encoding API
ENV NVIDIA_DRIVER_CAPABILITIES ${NVIDIA_DRIVER_CAPABILITIES},video

# Create a symbolic link to the path where libnvidia-encode.so.1 will be mounted, since UE4 seems to ignore LD_LIBRARY_PATH
# (Replace "ProjectName" with the name of your actual Unreal project)
RUN ln -s /usr/lib/x86_64-linux-gnu/libnvidia-encode.so /home/ue4/project/ProjectName/Binaries/Linux/libnvidia-encode.so

To run the container image produced by this Dockerfile your host system will need an NVIDIA GPU, the latest NVIDIA binary drivers, and the NVIDIA Container Toolkit (formerly known as NVIDIA Docker.) For more information about running Linux containers with GPU acceleration, see the NVIDIA Container Toolkit page of Unreal Containers community hub documentation.

How you run the packaged Pixel Streaming application inside the container will vary depending on the version of the Unreal Engine being used. For Unreal Engine 4.23, you will need to run the packaged project itself, the WebRTCProxy program (which was subsequently merged into the Pixel Streaming plugin itself in Unreal Engine 4.24) and a signalling server. For Unreal Engine 4.25, you will just need to run the packaged project and the signalling server. For more information, see the official Pixel Streaming documentation.

References