Creating a custom engine build for internal projects

Development Programming & Scripting
1

Crossposting from AVRDevelopers (Google VR support mailing list):

We are trying to move the studio over to using a source build of Unreal, but are running into a dll issue. Each time the project is opened, users are getting this message.

The following DLLs are missing or built with a different version of the engine:

UE4Editor-Myproject.dll

The file is there, so we have been going under the assumption that the engine version is mismatched. There is a thread online that did not solve the issue.

There are a number of people that do not do source builds, or have visual studio installd. How can we prevent this error from coming up each time a code change is submitted? It would be best if users did not have to try to “rebuild them now” each time we change something.

One more piece of useful information

We currently do not have our game under the engine folder. our folder structure is

/SomeFolder/
/SomeFolder/Engine
/SomeFolder/Game

Hey Terry,

There are a few metadata files generated during the build process that need to be checked into source control with your EXE and DLLs. In particular, the Unreal build system generates a UE4Editor.target file and various UE4Editor.modules files that contain a “BuildId” field. In order for UE4Editor to launch, the “BuildId” GUID needs to be consistent across all of these files to ensure the EXE and DLLs are binary compatible. That’s how I understand the system to work, anyway.

I hope this helps!

Josh May

To clarify the current situation/problem a little more:

Distributing Terry’s builds works fine (after some fiddling)
He checks in the Engine and Project binaries.
Everyone on the floor syncs, and continues working fine.

Another programmer (me) syncs and starts working on a c++ Project feature.
The dependency graph (missing intermediate files, svn modified times, etc) forces a recompile of the entire engine.
This generates a new GUID for the engine on my machine.
I need to check in my Project feature and related assets.

The problem is that if I only check in the Project specific files, the dll is associated to a different GUID than the checked in Engine GUID.

If the project dll changes force a checkin of the Engine files:

  • wastes a lot of syncing time for everyone on the floor
  • programmers will have to recompile the engine even more often
  • kinda defeats the purpose of dlls

Does the project dll actually store the engine version internally, or just use the .target / .modules file? Could I just not check in those support files, and the new dll will think it is matching the engine?

We tried this locally (copied my generated .dll and .pdb), and it worked on one machine, and not on another programmer’s. Investigating further, the guid locations are inconsistent on which values match, even between the engine module and target files mentioned above, as well as other values for the gameEditor module/target files. So at this point, I basically have little idea what is actually going on.

Thoughts?

In the grand scheme of things, is there a way to avoid

  1. Programmers needing to check in the engine when only making changes to the game
  2. Users needing to sync the game and engine, when only game changes have been made by a programmer
2

Yes! Let me explain why this elaborate system exists for context first though. :slight_smile:

Unreal’s tagged property serialization is tolerant to properties being added and removed, so you can load content from pretty much any version of the editor unless someone changes a custom UObject::Serialize function. Developers are adding and removing properties all the time, so we always want to prevent artists loading content that someone created with a newer version of the editor, and losing properties when it’s saved out because their version of the editor didn’t load it. We normally do that by embedding the build CL into the compiled binaries, and it gets saved out into any assets created with those binaries.

Separately to that, some engine modules are discovered and loaded dynamically; commonly modules which add support for different target platforms that might be under NDA, or which may be licensed separately to the rest of the engine. Because these modules can be added or removed at any time, or moved from one location to another, we want the engine to be able to detect whether they are valid and up to date before trying to load them. We normally use the same embedded CL number to match up binaries from the same build of the engine.

This system doesn’t work for people building locally because we don’t know what CL number they are on. They may have a bunch of binaries that they compiled locally, mixed in with a bunch of other binaries that they’ve synced from source control. To identify a valid working set of DLLs for those users, we have a randomly generated “build id” which gets saved into a .modules file in each output directory, and we match up the build id with the one in the executable directory before loading anything.

There are some optimizations to try to locally recycle a build id when you’re only touching game code to support a feature where you compile the engine binaries separately to the game code. To use that feature, you first build the game with the -precompile command line option to UBT, then with the -useprecompiled option every time after that.

Anyway, the problem you’re seeing is that you’re using the local iteration workflow - using the “build id” rather than CL number - but trying to check everything in. It’s not meant to maintain consistency between machines in that way; it is deliberately random for each user to avoid collisions. Your easiest workaround is to check in binaries which have a hard coded CL number instead, which will remove the need for it to use the build id entirely. In Engine/Source/Runtime/Launch/Resources/Version.h, set ENGINE_IS_LICENSEE_VERSION to 1, and BUILT_FROM_CHANGELIST to any non-zero number you like.

I would encourage you to actually submit correctly versioned binaries to avoid accidental data loss due to the reasons I mentioned at the top though.