Deploying source-built engine (through P4)

So this question as been asked in various pieces, but nothing has ever fully brought everything together. Basically, we are evaluating UE4 for future projects and are trying to work out the logisitics of working with it in a team environment. While many of the pieces are clear, some of them seem suboptimal, so we think it’d be great if Epic could give some feedback relating to their own internal flow for this.

Basically, our goal would be to have a build machine build the engine, check in the binaries, then everyone else can operate as though it is a binary release of the engine. If and when we make engine changes, the build machine would be what makes sure we are up to date. Projects would be a similar situation: the game binaries would be built (for artists, designers) by the build machine and no rebuild would be needed.

Right now, the latter works, but the former does not. As soon as a programmer were to try and build the project, it triggers a rebuild of the engine, even though the exact binaries used to make it in the first place are the ones checked in. How can we synchronize this? How can we make it recognize the binaries as the correct version?

Since this is a lengthy process, I expect Epic does not need to do this internally, so I’d like to know how exactly it needs to be setup to get the optimal workflow (especially given we are also using P4). If you can detail exactly what your source control and build looks like on your own internal projects, we can perhaps figure out how best to adapt to that for our own (and I figure there may be others who have similar questions).

Sorry for the delay. This is an excellent question though. Here is a simple overview of how our process works.

Our build machine compiles and checks in all binaries for “Development Editor” configuration into Perforce. Our engine QA team frequently (multiple times a day) syncs these binaries and quickly runs through a special suite of tests to make sure the tools work well and do not corrupt content or crash. If all goes well, we create a “label” in Perforce for the binaries and code that passed the test. This is what we call a “build promotion”. Our artists and level designers use a special tool to sync to the “latest promoted build” in any branch they want. It’s important to note that only the engine binaries, engine code and game code are part of the label. Artists and designers always sync their content files to the very latest revision to avoid contention between each other over files. Finally, if a programmer needs to checkin any content files, they are required to also use the latest promoted build to do that. This is to make sure that no content “from the future” is ever submitted, as a promoted editor build might not be able to read those files.

We also have a traditional “CIS” system that continuously syncs code from Perforce for most projects, tries to compile everything, and sends out warning emails to potential culprits to ask them to fix the issues as soon as possible.

This is the simple overview, our build system is actually very complicated because of the large number of combinations of products, sample games, plugins and engine versions we need to support.

–Mike