Creating a shared Derived Data Cache

We’ve started running into issues with the large DDC folders on our teams computers, and when I was looking through the documentation I noticed it’s possible to create a shared DDC. We already have a large, fast server for our studio, so it seems like the ideal solution.

Unfortunately, I can’t seem to get it to work. I’ve added the over-write code mentioned on this wiki page to the DefaultEngine.ini file in our project, replacing \mystudio.net\DDC with our own path, but it doesn’t seem to do anything. In the startup logs when I open the editor it says its using the local ddc path.

Anyone have any luck getting this working? Or have any ideas how to do it?

Hey Ineni.Realtime-

Could you upload your BaseEngine.ini file for us to look at. If you’d like to do so privately you could put the file inside a dropbox and then send a PM on the forums with the shared link to the folder.

Cheers

Hi Ineni.Realtime,

We have not heard back from you in a few days, so we are marking this post as Resolved for tracking purposes. If you are still experiencing the issue you reported, please respond to this message with additional information and we will offer further assistance.

Thank you.

Hello,
I encounter exactly the same issue as mentionned: I follow your documentation to create a shared DDC and modify my DefaultEngine.ini file in my project, but nothing hapen. Have an idea to solve it ?

Hey jpichard-

Would it be possible for you to upload your BaseEngine.ini file for us? Let us know if there are any problems in doing so.

Cheers

Hey ,

Please find our C:\Program Files\Epic Games\4.7\Engine\Config\BaseEngine.ini. Thank you for your help

Jérômelink text

Are you able to remove the quotation marks from the Filename=… lines? The %GAMEDIR% returns the base directory for the game and the quotation marks are causing it to see the line as a specific path to the Boot.ddc file.

I’ve removed all quotation marks in every BaseEngine.ini found in C:\Program Files\Epic Games\4.7\Engine\ directory, but my shared directories are still empty.
This is details of our DefaultEngine.ini, in section [DerivedDataBackendGraph]:


Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=19, FoldersToClean=-1, Path=\server\DDC, EnvPathOverride=UE-SharedDataCachePath)
AltShared=(Type=FileSystem, ReadOnly=true, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=19, FoldersToClean=-1, Path=\server\DDC2, EnvPathOverride=UE-SharedDataCachePath2)

But %GAMEDIR% is not used in “shared” neither “AltShared”. What is the link between %GAMEDIR% and shared DDC ?

Your defaultengine.ini would be more interesting tbh. The one you actually set your DDC hierarchy in.

I’d also try settign the shared path to some local folder, C:\SharedDDCTest\ and see if it actually saves any files in there. That would tell us if the issue is with backend config or network access.

%GAMEDIR% is replaced with game directory when inis are parsed, has nothing to do with DDC unless you use it for your DDC paths.

Is there anything in the logs that says your \server\DDC path can’t be accessed?

Hey Robert,

Thank you for your answer. Nothing in the log about accessibility of my server.

So I put C:\SharedData\ in my game DefaultEngine.ini: Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=19, FoldersToClean=-1, Path=C:\SharedData, EnvPathOverride=UE-SharedDataCachePath)
AltShared=(Type=FileSystem, ReadOnly=true, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=19, FoldersToClean=-1, Path=C:\SharedData2, EnvPathOverride=UE-SharedDataCachePath2)

And there is nothing in this directories when my project is loaded.

Nothing appears in the logs about shared DDC.Should we enable any network option somewhere ?

Ok, it sounds like it’s not even being used then. Where in your ini are you adding it? I mean, what section (the name in square brackets)? It should be something like:

[DerivedDataBackendGraph]
AltShared=(Type=FileSystem, ReadOnly=true, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=23, FoldersToClean=10, MaxFileChecksPerSec=1, Path=C:\SharedCachePath, EnvPathOverride=UE-SharedDataCachePath2)

Does that not work?

I meant ‘Shared=’, not AltShared, but both should work.

this is my full DefaultEngine.ini file:
[URL]
[/Script/HardwareTargeting.HardwareTargetingSettings]

[/Script/Engine.UserInterfaceSettings]
RenderFocusRule=NavigationOnly

[/Script/Engine.RendererSettings]

[InstalledDerivedDataBackendGraph]
Boot=(Type=Boot, Filename=%GAMEDIR%DerivedDataCache/Boot.ddc, MaxCacheSize=512)
Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=34, FoldersToClean=-1, Path=%GAMEDIR%DerivedDataCache)

[DerivedDataBackendGraph]
MinimumDaysToKeepFile=7
Root=(Type=KeyLength, Length=120, Inner=AsyncPut)
AsyncPut=(Type=AsyncPut, Inner=Hierarchy)
Hierarchy=(Type=Hierarchical, Inner=Boot, Inner=Pak, Inner=EnginePak, Inner=Local, Inner=Shared)
Boot=(Type=Boot, Filename=%GAMEDIR%DerivedDataCache/Boot.ddc, MaxCacheSize=256)
Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=17, FoldersToClean=-1, Path=…/…/…/Engine/DerivedDataCache)
Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=19, FoldersToClean=-1, MaxFileChecksPerSec=1, Path=C:\SharedCachePath, EnvPathOverride=UE-SharedDataCachePath)
AltShared=(Type=FileSystem, ReadOnly=true, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=23, FoldersToClean=10, MaxFileChecksPerSec=1, Path=C:\SharedCachePath2, EnvPathOverride=UE-SharedDataCachePath2)
Pak=(Type=ReadPak, Filename=%GAMEDIR%DerivedDataCache/DDC.ddp)
EnginePak=(Type=ReadPak, Filename=…/…/…/Engine/DerivedDataCache/DDC.ddp)

I added your parameter “MaxFileChecksPerSec=1” which didn’t exist in my DefaultEgine.ini, but I have always nothing in my directories: C:\SharedCachePath and C:\SharedCachePath2. Is it normal I haven’t any error messages is the log ? Thanks

I just managed to reproduce it. This could be something in ini parsing that’s not overriding the Shared key with the new value from DefaultEngine.ini. I’ll investigate and let you know when it’s fixed.

Ok, so in my case it turned out to be working fine. I got confused because I had the UE-SharedDataCachePath environment variable set to my shared cache path and changing the path in ini didn’t help much as the env var always overrides the path from ini.

Maybe it’s the same issue in your case?

Try running the editor with -logcmds=“LogDerivedDataCache Verbose” to output more info from DDC to the log file.

Hello Robert,

Could you please give me the exact name of the environment variable ?

For information, we have compiled Unreal Engine 4.8 from branch 4.8 and our Shared DerivedDataCache on network is working with this version. But with our project, we can’t use this version and we need to solve the problem with your standalone version 4.8.1 provided by Epic Launcher.

please find in attachment my log filelink text. Thank you for your help

It’s the UE-SharedDataCachePath (they’re specified in BaseEngine.ini with EnvPathOverride= param).

It has also been pointed out that you’re using the installed DDC backend hierarchy (which is used by the installed version of the engine): InstalledDerivedDataBackendGraph so you should be modifying that section in your game’s ini.

You can also override the backed hierarchy from the commandline with -ddc=BackendName

I have the exact same problem, mind sharing how you manage to resolve at the end?