Linux 4.9 Bad hlslcc header error

[2015.08.20-10.29.02:868][ 0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found
[2015.08.20-10.29.02:868][ 0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found! Missing ‘#’!

Gets stuck at 35% and just spamms that error over and over again in red (launching via gdb or console)

Can confirm. I’ve checked out the source clean multiple times while trying various workarounds from other threads here (which just ended in other problems for me), and 4.9 always produces the error mentioned above. Previous versions, compiled on the same Ubuntu 15.10 machine some months back, ran without encountering this error.

[2015.08.20-15.21.41:827][  0]LogTextLocalizationManager: No specific translations for ('en-US') exist, so ('en') translations will be used.
[2015.08.20-15.21.41:890][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found
    ...                                                                                  
ShaderCompileWorker: ../../src/hlslcc_lib/symbol_table.cpp:189: void _mesa_symbol_table_pop_scope(struct _mesa_symbol_table *): Assertion `hdr->symbols == sym' failed.                   
[2015.08.20-15.21.42:719][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found                                  
    ...
ShaderCompileWorker: ../../src/hlslcc_lib/symbol_table.cpp:189: void _mesa_symbol_table_pop_scope(struct _mesa_symbol_table *): Assertion `hdr->symbols == sym' failed.
*** Error in `./UE4Editor': free(): invalid pointer: 0x00007f8e740e1010 ***

It appears that the master/4.10 branch exhibits the same issue. I compiled it all clean, and ended up with similar errors:

[2015.08.26-18.33.35:139][  0]LogInit:  - Physical RAM available (not considering process quota): 16 GB (15939 MB, 16322260 KB, 16713994240 bytes)
[2015.08.26-18.33.35:145][  0]LogTextLocalizationManager: No specific translations for ('en-US') exist, so ('en') translations will be used.
[2015.08.26-18.33.35:398][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found
[2015.08.26-18.33.35:398][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found! Missing '#'!
[2015.08.26-18.33.35:910][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found
[2015.08.26-18.33.35:911][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found! Missing '#'!
[2015.08.26-18.33.36:049][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found
[2015.08.26-18.33.36:049][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found! Missing '#'!
[2015.08.26-18.33.36:187][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found
[2015.08.26-18.33.36:187][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found! Missing '#'!
[2015.08.26-18.33.36:198][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found
[2015.08.26-18.33.36:198][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found! Missing '#'!
Signal 11 caught.
EngineCrashHandler: Signal=11
Starting ../../../engine/binaries/linux/crashreportclient
Aborted (core dumped)

[2015.08.26-18.33.37:818][  0]LogGenericPlatformMisc: FPlatformMisc::RequestExit(0)
[2015.08.26-18.33.37:828][  0]LogExit: Preparing to exit.
[2015.08.26-18.33.37:831][  0]LogExit: Exiting.

I had that problem. I rebuilt the shader compiler to get rid of the “Error: Bad hlslcc header found! Missing ‘#’!” issues

I’ve rebuilt the entire project clean (even from fresh checkouts) many times and still end up with this hlslcc error in both the 4.9 and master/4.10 branches.

I’ve tried going into Engine/Source/ThirdParty/hlslcc/hlslcc/projects/Linux and running make (successfully), then going back to Engine and making ShaderCompileWorker and UE4Editor, but I still get the same hlslcc errors when running the editor afterwards.

I saw one of the other threads about copying a .a file around, but that just led to other errors the last time I tried it… and eventually to me doing a full clean checkout again to get rid of them and back to the original problem.

ive noticed when building my 4.8 project now, that its using headers from the 4.9 folder…

Is it possible the files from the different engine versions are being mixed together to cause these issues?

I have UnrealEngine/4.8/ and UnrealEngine/4.9/ with the different versions, and now neither version seems to work :frowning:

Tried deleting the 4.8 folder with no luck… Are there any other temp files from building hidden around that I can manually remove?

Tried again on 4.9 release. Same error!

Tried this as a “fix”:

But no go!

Anyone find a way to fix this yet??

Might help to post the build command im using:

make SlateViewer CrashReportClient UnrealPak UnrealLightmass ShaderCompileWorker UE4Editor

I don’t believe it’s due to different versions mixing. I have two completely separate git clones - one of which has never been set to any branch other than 4.9, and have the same symptoms.

Now might be a good idea to try and figure out what we have in common.

ok, ive tried

  • removing 4.8
  • cleaning
  • using clang 3.6 only
  • recompile hlslcc (resulting in a segfault error instead)
  • tried various machines

Only thing I can think of is that im running ubuntu 15.10 beta

Yeah - I’ve been suspecting the same thing (I’m using 15.10 beta / clang 3.6.2 also). I was successfully compiling it in 15.10 a couple of months back - but I’m sure plenty of packages have changed since.

When I get a chance, I’m going to install 15.04 in a VM, map a shared folder, and try to compile from the exact same source.

I’ve confirmed it’s definitely a toolchain issue with the newer version of something in 15.10.

I just built it from a 15.04 guest VM (w/ clang 3.6.0) using a shared folder pointed to exact same the source that’s been failing with the hlslcc errors, and the resulting build then runs fine on the 15.10 host system.

I am having the same problem in Fedora 23, with clang 3.7.0, trying to start both UEEditor 4.10 and 4.11.
Did you determine if it is a Clang issue, or something else in the toolchain? Were you able to work around the issue?

I didn’t troubleshoot it further, as I have only played with UE4E once or twice out of curiousity. My workaround is to just compile it in a VM (and accept the huge time penalty that entails), then run it from my host system.

My recent experience is, almost word for word, just as 3vi1’s experience. I’m using Solydxk EE (k version), which for those unaware, is like Ubuntu/Mint. The EE version of Solydxk follows debian testing and has the issues above. The official stable build (non-EE) does work. Like 3vi1, I installed the stable version in VirtualBox to do the compile, though I had to run the Setup.sh and GenerateProjectFiles.sh scripts from the host, then compile in the guest. After compilation the editor starts and runs on host fine (so far).

Since I started looking for more info on this, I’ve noticed that the issues was noticed back around July. Since this seems to be affecting the newer toolchains coming down the pipe, I’m hopeful that the team is working on a fix (but Epic’s attitude towards linux so far does leave me concerned).

I think I may have a fix for this issue.

There is very likely a bug inside Clang that is causing some of these builds to fail. I can build two copies of the same version of the game with the same compiler, on the same OS, with the same nvidia driver versions, same RAM count and CPU generation, and one build will consistently see the error while the other will consistently be fine. I can even transfer the compiled binaries between the machines and the one compiled on the good machine will run fine on the bad machine. The bad machine stared exhibiting the issue after a big system update (apt-get dist-upgrade).

I’ve added a workaround in this commit, in case anyone wanted to give it a shot:
https://github.com/ExpiredPopsicle/UnrealEngine/commit/ec9c9ffb4e83ae15db9700f22bf2110bbfb99464

This commit is based of Dmitry Rekman’s 4.9-rcl Linux branch ( https://github.com/RCL/UnrealEngine ), so if you’re already running something from there, it should go in cleanly.

After cloning, cherry-picking, or applying it as a patch, you will need to rebuild the HLSLCC library. ( “BuildThirdParty.sh -b HLSLCC” in the Engine/Build/Batchfiles/Linux directory.) When that’s done, you should be able to just build UE4Editor and ShaderCompileWorker.

I haven’t yet made a pull request from it because I haven’t tested it on anything but a Debian Testing machine, and the fact that it’s a weird workaround to a compile bug may or may not sit well with the developers at Epic. But one way or another, I believe I have a reasonable understanding of the issue causing this now.

Let me know if this works for any of you.

Oh wow, I was just going down this debugging path this afternoon. You have beat me too it. I just applied your patch and I am recompiling now. I will let you know the outcome.

I am on the promoted branch of unreal engine, which is currently between 4.10 and 4.11, the patch didn’t go in cleanly so I applied the changes manually.

My machine is running Fedora 23 Beta, with Clang 3.7.0 so if it works I hope it can be included as a pull request to Epic.

Edit: Ok. I got it working :slight_smile: The patch as is didn’t work outright, I had to go through and also replace all references to std::string and std::stringstream in hlslcc_lib/IRDump.cpp and hlslcc_lib/IRDump.h. Those files may not be a problem in v4.9. I recompiled again and it worked for me.

Edit2: The discussion of the Clang bug you referenced includes this link: 21001 – clang++ doesn't export operator delete with -flto to a llvm bug report.
Unfortunately they narrowed that down to a problem in the linker, which is not an LLVM/Clang component.
The binutils bugreport is here: 17458 – Symbol that normally exported from a binary reported as PREVAILING_DEF_IRONLY to a plugin

Awesome! Glad it worked for you! (With some tweaking.) I guess I missed the std::string use in IRDump.*.