Tags: C++, DevOps, Docker, Living Document, Unreal Engine

Posted: 15 December 2019

Contents

• Revision history
• Vision
  • Overview
  • Industries and use cases
    • Game development
    • Research and simulation
    • Enterprise
  • Key implementation elements
• Implementation status
  • Container support
  • Interoperability with third-party frameworks and protocols
  • Partner and community engagement
• Getting involved

Revision history

This is a living document that will continue to be updated over time as further progress is made toward achieving the vision of a cloud-native Unreal Engine. Noteworthy revisions to the document are listed below in reverse-chronological order:

• 12th February 2020: updated the Pixel Streaming section to reflect the public release of my implementation of Linux support for Pixel Streaming.

• 15th December 2019: the initial public version of this document was created and added to the author’s website. The public version updates and expands upon much of the content and replaces the recommendations addressed to Epic Games with information addressed instead to the broader Unreal Engine community.

• 17th September 2019: the initial private version of this document was created. The document originally took the form of a report addressed to Epic Games, prepared for Epic engineers who had contacted the author and expressed an interest in Unreal Engine containers. The report was designed to provide a concise overview of the work that I had undertaken during the two years prior and present the rationale underpinning that work. The report concluded with a list of recommended actions for Epic Games to undertake that would accelerate the implementation of cloud-native functionality in the Unreal Engine.

Vision

Overview

By expanding beyond its original game development focus to embrace new industries and use cases, the Unreal Engine has firmly established itself as a powerful and innovative middleware platform with the potential to completely transform content-oriented workflows. The open source nature of the Engine also facilitates significantly increased development velocity and community engagement when compared to competing technologies. As a result of this foundational work, the Unreal Engine is uniquely positioned to benefit from the adoption of modern cloud-oriented technologies and become the world’s first cloud-native game engine.

Cloud-native technologies such as containers and microservices are designed to support the development of modern server-based applications that need to operate at extreme scale whilst retaining the agility to deploy frequent updates. (For more details, see the formal definition from the Cloud Native Computing Foundation). The past decade has seen rapidly accelerating adoption of these technologies and their associated development practices across a wide variety of industries and within organisations of all sizes. Compatibility with cloud-native technologies will allow the Unreal Engine to strengthen its position within a number of key industries that it has already embraced and expand its reach to encompass many additional industries, facilitating an unprecedented breadth of adoption and collaboration.

Industries and use cases

The adoption of cloud-native technologies will strengthen the applicability and value proposition of the Unreal Engine as a middleware platform in each of the industries listed below. Relevant use cases are highlighted for each industry. Note that for reasons of brevity this list is non-exhaustive and focuses only on a selection of key industries and use cases.

Game development

• Operating dedicated servers: it is already public knowledge that Epic Games utilises the Kubernetes container orchestration framework to operate the server infrastructure for Fortnite. Improved workflow support for container-based packaging and deployment will allow Engine Licensees to benefit from the same agility and scalability when deploying dedicated servers for other games developed in the Unreal Engine.

• Continuous Integration and Continuous Delivery: CI/CD practices are fast becoming the de facto standard for developing modern software systems, and the benefits provided by these practices have been one of the driving factors behind the widespread adoption of cloud-native technologies. There is increasing interest in CI/CD amongst the game development community, ranging from small indie developers to large studios producing AAA titles such as Sea of Thieves. A combination of improved container support and scriptable automation infrastructure will allow Engine Licensees to leverage standardised tools and workflows for CI/CD and benefit from best practices established by the wider software industry.

• Cloud streaming: although publicly available information suggests that upcoming streaming platforms such as Google Stadia and Microsoft xCloud do not currently utilise cloud-native technologies such as containers as part of their platform implementations, it is very likely that containers will be adopted at some point in the future if these platforms prove to be successful. Google heavily promotes the use of their Kubernetes container orchestration framework to customers of their public cloud offerings, and Microsoft has recently implemented experimental support for GPU acceleration in Windows Server containers. Improved container support in the Unreal Engine will simplify workflows for Engine Licensees developing games for potential future container-based versions of these or other cloud streaming platforms.

• Distributed simulation: platforms such as SpatialOS and Cloudgine (which was acquired by Epic Games in early 2018) have demonstrated the power that large-scale distributed physics simulations can bring to games and other interactive applications. Improved workflow support for integrating third-party libraries will make it significantly easier to develop similar integrations in the future, providing opportunities to further enhance the Engine with distributed processing models built on cloud-native technologies.

Research and simulation

• AI and machine learning: generation of training data for machine learning models is an extremely popular use of the Unreal Engine within both academic and enterprise contexts. GPU accelerated containers are becoming widely used for packaging and deploying machine learning models in cloud environments and receive a great deal of attention from both public cloud providers and from GPU vendors such as NVIDIA. Improved container support in the Unreal Engine will allow simulations to be packaged and deployed alongside the machine learning models that interact with them, using the same familiar technologies and deployment pipeline. These workflow improvements will facilitate greater adoption of the Unreal Engine amongst machine learning practitioners in both academia and industry.

Enterprise

• Enterprise application streaming: outside the scope of cloud streaming for games (covered in the Game development subsubsection), there exist a variety of enterprise applications that stream rendered images from the Unreal Engine to web browsers and low-powered client devices. This use case is targeted by the Engine’s official Pixel Streaming system, which cannot currently be run inside container-based environments. Although publicly available information suggests that most existing streaming solutions in this area do not currently support containers, both Epic Games themselves and a number of enterprise organisations have contacted the author of this document and expressed interest in implementing container support.

Key implementation elements

The realisation of a cloud-native Unreal Engine requires the implementation of three key elements:

• Container support: container technology is core to cloud-native methodologies and workflows. Containers provide immutable, software-defined environments for building or running applications, as well as standardised mechanisms for packaging and distributing binary images. Containers are commonly used for performing reproducible builds as part of CI/CD pipelines and for deploying applications into production environments where they can then be orchestrated, scaled, and monitored. Existing applications must often be modified in order to ensure full compatibility with container-based environments, and the Unreal Engine is no exception. The nature of the Engine as a powerful multimedia tool also presents unique challenges that must be addressed when implementing and maintaining container compatibility. In addition to overcoming these technical issues, prebuilt container images for each new Engine version need to be made readily available to all Engine Licensees, along with a means of building the images from source.

• Interoperability with third-party frameworks and protocols: cloud-native applications communicate primarily through network protocols that are implemented by additional middleware frameworks. The ability to easily integrate such frameworks into the Unreal Engine on a project-by-project basis is necessary in order to facilitate the development of Unreal-powered applications that operate within modern cloud-native ecosystems. Since these protocols and frameworks are continually evolving, the focus must be on providing a flexible integration workflow for Engine Licensees that is capable of accommodating arbitrary frameworks whose versions may change frequently. Comprehensive interoperability with standardised streaming protocols such as WebRTC builds upon generalised framework support to facilitate interactive streaming applications that operate independently of vendor-supplied cloud streaming platforms. This latter capability is particularly important for enterprise customers whose applications fall outside the scope of game-oriented streaming platforms such as Google Stadia or Microsoft xCloud.

• Partner and community engagement: Epic Games is well-known for its excellence in engaging in fruitful collaboration with external partners and the broader Unreal Engine user community. For a cloud-native Unreal Engine to realise its full potential it is necessary to engage with key partners and the community to both educate users about the benefits and opportunities afforded by cloud-native workflows and to ensure that the implementation of cloud-native Engine support meets the real-world needs of developers. It is only through collaboration and discussion that these new technologies can be adopted and capitalised on, and that innovative new opportunities can be identified and explored.

Implementation status

Since late 2017, the author of this document has worked to establish the foundations of a cloud-native Unreal Engine and pave the way for future development. Several key successes during this time have demonstrated the feasibility of implementing this vision from both a technical perspective and with respect to community engagement. The subsections below list actionable items for of each of the three implementation elements outlined in the Key implementation elements section, discussing the current implementation status of each item. The possible status values are as follows:

• Planned: work on the actionable item has not yet begun, but is planned for the near future.

• In Progress: work on the actionable item has begun and is currently in its early stages.

• Ongoing: foundational work on the actionable item has been completed and work has progressed to advanced stages.

• Completed: work on the actionable item has been completed.

 

Container support

• Actionable: Implement container images for build use

  Status: Complete

  Project: ue4-docker

  Created in early 2018, the ue4-docker project provides fully featured Dockerfiles for producing both Linux and Windows container images that encapsulate the Unreal Engine build tools. The images produced by this project are suitable for building and packaging Unreal Engine projects and plugins, and can also be used to run the Unreal Editor in an interactive sandbox under Linux host systems with NVIDIA GPUs. The source code and accompanying documentation for this project demonstrate solutions to the many technical issues that must be addressed in order to make the Unreal Engine compatible with container-based environments, and act as an exemplar for other developers who wish to write their own Dockerfiles.

   

• Actionable: Implement container images for runtime use

  Status: Complete

  Project: ue4-runtime

  Created shortly after the first version of ue4-docker, the ue4-runtime repository complements the former project by providing container base images that are suitable for encapsulating packaged Unreal projects and deploying them to production environments. Image variants are provided for a number of common configurations, and all images are designed to be extended by other developers for further customisation.

   

• Actionable: Implement enhanced automation support

  Status: Complete

  Project: ue4cli

  Conceived in late 2017 and made public in early 2018, the ue4cli project provides a convenient and platform-agnostic command-line interface to access the existing automation functionality provided by the Unreal Engine. This interface complements the Gauntlet automation system and simplifies developer workflows when scripting build and test phases in CI/CD pipelines. Additionally, the ue4-ci-helpers Python package builds upon ue4cli to provide reusable functionality for developers who wish to implement their build and packaging workflows using the Python scripting language.

   

• Actionable: Enable distribution of prebuilt container images for build use

  Status: Planned

  A key step in facilitating the adoption of cloud-native workflows amongst the Unreal Engine developer community is to provide a ready source of prebuilt container images encapsulating the Engine Tools. However, due to specific wording in the Unreal Engine EULA, the only distribution mechanism that Engine Licensees can currently use to share container images encapsulating the Unreal Engine build tools with other Licensees in a compliant manner is GitHub Packages (and even then, only via forks of the official Unreal Engine GitHub repository.) Unfortunately, the storage and bandwidth-based pricing of GitHub Packages makes it prohibitively expensive for use with large images containing the multi-gigabyte Engine build tools.

  The author of this document is currently in the process of establishing communication with the Epic Games legal team in order to discuss options for enabling alternative distribution mechanisms, such as:

  • Creation of a container registry run by Epic itself that integrates with Epic Games Account credentials for user authentication; or

  • Amendment of the Unreal Engine EULA terms to permit the distribution of container images encapsulating the Engine Tools via third-party container registries that integrate with Epic Games Account credentials or GitHub credentials for user authentication.

   

Interoperability with third-party frameworks and protocols

• Actionable: Implement a framework and workflow for integrating third-party libraries

  Status: Ongoing

  Project: conan-ue4cli

  Created in early 2018, the conan-ue4cli project is a plugin for ue4cli that implements a complete framework for integrating third-party libraries into the Unreal Engine on a project-by-project basis. Due to several technical challenges arising from the underlying architecture of the Engine, many common third-party libraries must be built from source using a custom configuration to ensure their compatibility with the Engine’s own dependency libraries. The conan-ue4cli framework abstracts these details to the greatest extent possible and provides a complete developer workflow for writing library build scripts and consuming the built libraries from Unreal projects and plugins. The existing Conan package management system is leveraged to provide a mechanism for packaging and distributing built libraries to other Engine Licensees, facilitating collaboration and code reuse amongst the Unreal Engine user community.

  Refinement and documentation of the conan-ue4cli workflow is ongoing. The author of this document is also in the process of establishing communication with the Epic Games legal team in order to discuss the conditions under which packages containing the Engine’s dependency libraries can be distributed to other Engine Licensees, since the ability to use prebuilt packages would greatly simplify the setup required for developers to start utilising the workflow.

   

• Actionable: Implement exemplar build scripts for popular cloud-native frameworks

  Status: Ongoing

  Project: ue4-conan-recipes

  Created in late 2018, the ue4-conan-recipes repository provides functioning build scripts for a variety of popular third-party libraries. Included in this repository are build scripts for Google’s extremely popular gRPC framework and its dependencies, allowing developers to easily integrate one of the most common cloud-native communication frameworks into their Unreal Engine projects. This facilitates rapid development of applications that interoperate with other cloud-native systems or even fully fledged microservices powered by the Unreal Engine.

  Implementation of build scripts for additional libraries and frameworks remains an ongoing process.

   

• Actionable: Implement support for using standardised streaming technologies inside containers

  Status: In progress

  Several months prior to the public release of the official Pixel Streaming system in late 2018, the author of this document created an Unreal plugin called UE4Capture to provide a cross-platform implementation of audio and framebuffer capture, designed for use with standardised streaming protocols such as WebRTC. Although the UE4Capture plugin is significantly less sophisticated than Pixel Streaming, it was (until February 2020) the only publicly-available streaming implementation for the Unreal Engine that functioned inside GPU accelerated containers. The fact that this proof-of-concept prototype is already in use by production applications developed by enterprise customers demonstrates a clear demand for container-based streaming functionality and motivates the need to enable container support for the official Pixel Streaming system.

  The official Pixel Streaming implementation currently only supports Windows and relies on APIs that are not supported by Microsoft’s experimental implementation of GPU acceleration inside Windows Server containers, precluding its use in a container-based environment. In order to function inside GPU accelerated containers, it is necessary to implement Linux support for Pixel Streaming, which will allow it to function inside Linux containers running on host systems with NVIDIA GPUs.

  The author of this document and his colleagues at TensorWorks began the implementation of experimental Linux support for the Pixel Streaming system in November 2019, and publicly released our code for Unreal Engine 4.23 in February 2020. You can find the details of this public release (and all subsequent releases) here: Pixel Streaming in Linux containers

   

Partner and community engagement

• Actionable: Establish a platform for educating the Unreal Engine community about cloud-native capabilities

  Status: Ongoing

  Initiative: Unreal Containers Community Hub

  Created in early 2019, the Unreal Containers community hub aims to educate the Unreal Engine user community about the benefits of cloud-native workflows by showcasing projects that use Unreal Engine containers and providing comprehensive documentation and training resources for using the infrastructure that is currently implemented and available. The community hub received an Epic MegaGrant in July 2019 to help fund the development and promotion of further training resources. To date the funds have been used to facilitate discussions with a number of Unreal Engine users that are informing the direction of future development of training resources, and to prepare for the production of video resources to accompany the existing written documentation and code examples.

  Development and maintenance of success stories, documentation and training resources remains an ongoing process.

   

• Actionable: Engage in ongoing community consultation with game developers

  Status: In Progress

  Note: extensive consultation with key partners and the broader Unreal Engine developer community is essential in order to gather information that will inform the design and implementation of future cloud-native support in the Engine. Consultation with a broad range of user demographics will result in a more holistic understanding of the cloud-native needs of Unreal Engine users. The demographic addressed by this actionable item is only one of many groups whose needs must be addressed.

  A number of game developers have engaged with the existing infrastructure and training resources, through both GitHub issues on the ue4-docker and ue4cli repositories and through direct email communication with the author of this document. These conversations have revealed that multiple developers are testing or actively using the available infrastructure, including developers from sizeable organisations such as Wargaming. In addition to liaising with individuals who are interested in using Unreal Engine containers for CI/CD purposes, the author has provided information and advice to the developers of the Gamebuildr CI service, which aims to provide the first hosted cloud CI service for Unreal Engine games. These discussions have highlighted the demand for container-based CI/CD amongst the Unreal user community, particularly with respect to the packaging and deployment of dedicated servers.

  Consultation with this community demographic has only just begun to scratch the surface and the author of this document encourages users to get in touch via email.

   

• Actionable: Engage in ongoing community consultation with enterprise users

  Status: In Progress

  Note: extensive consultation with key partners and the broader Unreal Engine developer community is essential in order to gather information that will inform the design and implementation of future cloud-native support in the Engine. Consultation with a broad range of user demographics will result in a more holistic understanding of the cloud-native needs of Unreal Engine users. The demographic addressed by this actionable item is only one of many groups whose needs must be addressed.

  A number of enterprise organisations have directly contacted the author of this document via email requesting information about the use of containers to deploy cloud applications powered by the Unreal Engine, and many more have starred or forked the ue4-docker GitHub repository, including the developers of Calgary Scientific’s PureWeb streaming platform. Several organisations have engaged the author and his colleagues at TensorWorks to provide consulting services and assist in the development and deployment of container-based Unreal Engine applications. At the time of writing, a number of these applications are nearing release.

  Consultation with this community demographic has only just begun to scratch the surface and the author of this document encourages users to get in touch via email.

   

• Actionable: Engage in ongoing community consultation with academia

  Status: In Progress

  Note: extensive consultation with key partners and the broader Unreal Engine developer community is essential in order to gather information that will inform the design and implementation of future cloud-native support in the Engine. Consultation with a broad range of user demographics will result in a more holistic understanding of the cloud-native needs of Unreal Engine users. The demographic addressed by this actionable item is only one of many groups whose needs must be addressed.

  Unsurprisingly, the author of this document has been contacted by fellow academics who are interested in using containers to deploy Unreal Engine applications for use in scientific research. Most of these academics have discovered the author after engaging with the UE4Research project, which aims to advance the use of the Unreal Engine in a scientific context by providing documentation and highlighting existing academic publications that use the Engine. The largest academic project currently exploring the use of Unreal Engine containers is being conducted by the University of California San Diego and involves the deployment of packaged Unreal projects to the university’s SunCAVE system using the Kubernetes container orchestration framework.

  Consultation with this community demographic has only just begun to scratch the surface and the author of this document encourages users to get in touch via email.

   

Getting involved

If you are interested in contributing to the realisation of a cloud-native Unreal Engine or would simply like to utilise cloud-native technologies such as containers in your own workflows, feel free to promote this document and its associated projects and initiatives via your own networks and/or contact the author via email at adam@adamrehn.com. Feedback from the Unreal Engine user demographics listed in the Partner and community engagement section is particularly valuable in shaping the direction of future developments to ensure they meet the needs of the entire Unreal Engine community.

For those readers who are just starting out on their cloud-native Unreal Engine journey, be sure to explore the documentation and training resources provided by the Unreal Containers community hub and follow the Unreal Containers Twitter feed for news about new and updated resources.