Derived / Local Data Cache errors and Net Serialization: Error: Archive is corrupted

Development Platform & Builds
, , , , ,
1

I can’t Launch my games. I’m using UE 4.13, from C++ source, Development mode, windows x64. When I try to Launch for Windows, I get the following set of problems.

Frst, I get some sort of data cache errors:

[2016.12.07-16.43.14:022][  0]LogTargetPlatformManager:Display: Building Assets For Windows
[2016.12.07-16.43.14:183][  0]LogDerivedDataCache:Display: Max Cache Size: 512 MB
[2016.12.07-16.43.14:185][  0]LogDerivedDataCache:Display: Starting with empty Boot cache
[2016.12.07-16.43.14:185][  0]LogDerivedDataCache: FDerivedDataBackendGraph:  Pak pak cache file ../../../../UE4ShaderPluginDemo/DerivedDataCache/DDC.ddp not found, will not use a pak cache.
[2016.12.07-16.43.14:188][  0]LogDerivedDataCache: Unable to find inner node Pak for hierarchical cache Hierarchy.
[2016.12.07-16.43.14:189][  0]LogDerivedDataCache: FDerivedDataBackendGraph:  EnginePak pak cache file ../../../Engine/DerivedDataCache/DDC.ddp not found, will not use a pak cache.
[2016.12.07-16.43.14:190][  0]LogDerivedDataCache: Unable to find inner node EnginePak for hierarchical cache Hierarchy.
[2016.12.07-16.43.14:203][  0]LogDerivedDataCache: Using Local data cache path ../../../Engine/DerivedDataCache: Writable
[2016.12.07-16.43.14:203][  0]LogDerivedDataCache: Shared data cache path not found in *engine.ini, will not use an Shared cache.
[2016.12.07-16.43.14:204][  0]LogDerivedDataCache: Unable to find inner node Shared for hierarchical cache Hierarchy.
[2016.12.07-16.43.14:206][  0]LogShaders: Verifying Global Shaders for PCD3D_SM5
[2016.12.07-16.43.14:207][  0]LogShaders:Warning: 	Empty global shader map, recompiling all global shaders
[2016.12.07-16.43.15:924][  0]LogInit: Selected Device Profile: [Windows]
[2016.12.07-16.43.15:931][  0]LogContentStreaming: Texture pool size is 0.00 MB
[2016.12.07-16.43.16:016][  0]LogMaterial: Missing cached shader map for material WorldGridMaterial, compiling. Is special engine material.
[2016.12.07-16.43.16:021][  0]LogShaders:Warning: 	Compiling WorldGridMaterial: Platform=PCD3D_SM5, Usage=LightingModel=MSM_DefaultLit, BlendMode=BLEND_Opaque, SpecialEngine=1, TwoSided=0, TSNormal=1, Masked=0, Distorted=0, WritesEveryPixel=1, ModifiesMeshPosition=0, Usage={}
[2016.12.07-16.43.16:750][  0]LogShaders:Warning: 		529 Shaders among 16 VertexFactories

I tried deleting the DerivedDataCache folder in my project as well as in the engine source, and it gets recreated but I still get the above errors.

Then it continues compiling for launching for one hour and 30 minutes (on a 8 thread i7 with 16gb ram), but ends with a loop of "Archive is corruptedLogNetSerialization:Error: Archive is corrupted" lines.

  • Why does it take 1h30min to publish a build of a very barebones game?

  • Why do I see things like Oculus and Mobile and HTML5 stuff being printed into a Windows Only Executable compilation log?

  • How do I debug what’s wrong with corruptedLogNetSerialization?

I’ve attached a full log of what happens: log 1. Interesting parts to ctrl+f are:

  • "will not use a pak cache" and
  • "will not use an Shared cache" at the beginning, and
  • [2016.12.07-18.48.36:009][622]LogPlayLevel: CommandUtils.Run: Run: Took 5338.8630499s to run UnrealBuildTool.exe, ExitCode=0 and
  • [2016.12.07-18.57.01:006][212]LogPlayLevel: ShaderPluginDemo: [2016.12.07-18.56.27:209][ 0]LogNetSerialization:Error: Archive is corruptedLogNetSerialization:Error: Archive is corrupted at the end of the file.

PS: I tried Launching a standard FPS template and that worked successfully (after 1 hour of “launching”). But I still got the "will not use a pak cache" errors. log 2.

[EDIT]

I tried downloading a new version of UE from source (ue 4.14), compiled it, ran great, made a new project, added all my stuff back in, compiled, ran fine. Then I closed UE4 and re-opened the project, and I am getting the Unable to find Local Data Cache and Derived Data Cache (and crashes, out of memory).

Is somehow my code or a blueprint causing this?! How can you debug this!?

[EDIT 2]

So it seems that the BaseEngine.ini and Engine.ini have correct paths set for the DerivedDataCache/DDC.ddp for both the engine and the GAMEDIR versions. ( [DerivedDataBackendGraph] and [InstalledDerivedDataBackendGraph]).

Even so, my engine fails to create DDC.ddp when opening my project. I succeed when I open the engine itself. But not trying to open my project.

I tried recompiling the engine, clearing the project intermediate folders, rebuilding project, running UE4 with the following command [game path] -run=DerivedDataCache -fill -DDC=CreatePak -log -ddc=noshared. Every time I get a

LogDerivedDataCache:Display: Pak cache opened for reading ../../../Engine/DerivedDataCache/DDC.ddp - good

and then a

FDerivedDataBackendGraph: Pak cache file [path]+DerivedDataCache/DDC.ddp not found, will not use a pak cache

And eventually it tries to do everything in ram and it runs out and UE loading crashes.

2

Ok I have no clue why these things are happening, and am overall unimpressed about how hard it is to deal with UE’s c++ shenanigans.

But the problem magically went away after I:

  • became an expert on many ini files

  • deleted project Binaries

  • deleted project Derived Data Cache

  • deleted project Intermediate

  • deleted project Saved

  • deleted project .sdf file

  • Then regenerated visual studio project files

  • Opened the sln project in Visual Studio and did a Build

^ this got rid of the Net Serialization: Error, and data cache errors for my project.

Then I:

  • disabled tens of plugins UE4 loads with by default that I’ll never use (which slow down loading and sometimes log warnings or errors)

  • deleted Engine’s Derived Data Cache

  • deleted engine’s Intermediate folder and recompiled

^ this got rid of a “missing global shader” publishing error with regards to my project’s shader plugin, and got rid of the data cache crash/errors for the engine.

There really should be a sandbox that prevents the engine from crashing if “something’s wrong” with your code.

1 Like