Search in
Sort by:

Question Status:

Search help

  • Simple searches use one or more words. Separate the words with spaces (cat dog) to search cat,dog or both. Separate the words with plus signs (cat +dog) to search for items that may contain cat but must contain dog.
  • You can further refine your search on the search results page, where you can search by keywords, author, topic. These can be combined with each other. Examples
    • cat dog --matches anything with cat,dog or both
    • cat +dog --searches for cat +dog where dog is a mandatory term
    • cat -dog -- searches for cat excluding any result containing dog
    • [cats] —will restrict your search to results with topic named "cats"
    • [cats] [dogs] —will restrict your search to results with both topics, "cats", and "dogs"

Linux 4.9 Bad hlslcc header error

[2015.08.20-10.29.02:868][ 0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found [2015.08.20-10.29.02:868][ 0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found! Missing '#'!

Gets stuck at 35% and just spamms that error over and over again in red (launching via gdb or console)

Product Version: UE 4.9
more ▼

asked Aug 20 '15 at 10:31 AM in Linux

avatar image

61 3 7 14

avatar image 3vi1 Aug 20 '15 at 03:25 PM

Can confirm. I've checked out the source clean multiple times while trying various workarounds from other threads here (which just ended in other problems for me), and 4.9 always produces the error mentioned above. Previous versions, compiled on the same Ubuntu 15.10 machine some months back, ran without encountering this error.

 [2015.08.20-15.21.41:827][  0]LogTextLocalizationManager: No specific translations for ('en-US') exist, so ('en') translations will be used.
 [2015.08.20-15.21.41:890][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found
 ShaderCompileWorker: ../../src/hlslcc_lib/symbol_table.cpp:189: void _mesa_symbol_table_pop_scope(struct _mesa_symbol_table *): Assertion `hdr->symbols == sym' failed.                   
 [2015.08.20-15.21.42:719][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found                                  
 ShaderCompileWorker: ../../src/hlslcc_lib/symbol_table.cpp:189: void _mesa_symbol_table_pop_scope(struct _mesa_symbol_table *): Assertion `hdr->symbols == sym' failed.
 *** Error in `./UE4Editor': free(): invalid pointer: 0x00007f8e740e1010 ***
avatar image 3vi1 Aug 26 '15 at 06:41 PM

It appears that the master/4.10 branch exhibits the same issue. I compiled it all clean, and ended up with similar errors:

 [2015.08.26-18.33.35:139][  0]LogInit:  - Physical RAM available (not considering process quota): 16 GB (15939 MB, 16322260 KB, 16713994240 bytes)
 [2015.08.26-18.33.35:145][  0]LogTextLocalizationManager: No specific translations for ('en-US') exist, so ('en') translations will be used.
 [2015.08.26-18.33.35:398][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found
 [2015.08.26-18.33.35:398][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found! Missing '#'!
 [2015.08.26-18.33.35:910][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found
 [2015.08.26-18.33.35:911][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found! Missing '#'!
 [2015.08.26-18.33.36:049][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found
 [2015.08.26-18.33.36:049][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found! Missing '#'!
 [2015.08.26-18.33.36:187][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found
 [2015.08.26-18.33.36:187][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found! Missing '#'!
 [2015.08.26-18.33.36:198][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found
 [2015.08.26-18.33.36:198][  0]LogOpenGLShaderCompiler:Error: Bad hlslcc header found! Missing '#'!
 Signal 11 caught.
 EngineCrashHandler: Signal=11
 Starting ../../../engine/binaries/linux/crashreportclient
 Aborted (core dumped)
 [2015.08.26-18.33.37:818][  0]LogGenericPlatformMisc: FPlatformMisc::RequestExit(0)
 [2015.08.26-18.33.37:828][  0]LogExit: Preparing to exit.
 [2015.08.26-18.33.37:831][  0]LogExit: Exiting.
avatar image kayosiii Aug 27 '15 at 03:34 AM

I had that problem. I rebuilt the shader compiler to get rid of the "Error: Bad hlslcc header found! Missing '#'!" issues

avatar image Gibbz Aug 28 '15 at 05:48 AM

ive noticed when building my 4.8 project now, that its using headers from the 4.9 folder...

Is it possible the files from the different engine versions are being mixed together to cause these issues?

I have UnrealEngine/4.8/ and UnrealEngine/4.9/ with the different versions, and now neither version seems to work :(

avatar image 3vi1 Sep 01 '15 at 04:30 PM

I don't believe it's due to different versions mixing. I have two completely separate git clones - one of which has never been set to any branch other than 4.9, and have the same symptoms.

avatar image Gibbz Aug 29 '15 at 02:05 AM

Tried deleting the 4.8 folder with no luck... Are there any other temp files from building hidden around that I can manually remove?

avatar image Gibbz Sep 01 '15 at 07:55 AM

Tried again on 4.9 release. Same error!

Tried this as a "fix": https://answers.unrealengine.com/questions/258959/ue4editor-linux-crash.html

But no go!

Anyone find a way to fix this yet??

avatar image Gibbz Sep 01 '15 at 07:56 AM

Might help to post the build command im using:

make SlateViewer CrashReportClient UnrealPak UnrealLightmass ShaderCompileWorker UE4Editor

avatar image kayosiii Sep 01 '15 at 05:48 PM

Now might be a good idea to try and figure out what we have in common.

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

3 answers: sort voted first

I think I may have a fix for this issue.

There is very likely a bug inside Clang that is causing some of these builds to fail. I can build two copies of the same version of the game with the same compiler, on the same OS, with the same nvidia driver versions, same RAM count and CPU generation, and one build will consistently see the error while the other will consistently be fine. I can even transfer the compiled binaries between the machines and the one compiled on the good machine will run fine on the bad machine. The bad machine stared exhibiting the issue after a big system update (apt-get dist-upgrade).

I've added a workaround in this commit, in case anyone wanted to give it a shot: https://github.com/ExpiredPopsicle/UnrealEngine/commit/ec9c9ffb4e83ae15db9700f22bf2110bbfb99464

This commit is based of Dmitry Rekman's 4.9-rcl Linux branch ( https://github.com/RCL/UnrealEngine ), so if you're already running something from there, it should go in cleanly.

After cloning, cherry-picking, or applying it as a patch, you will need to rebuild the HLSLCC library. ( "BuildThirdParty.sh -b HLSLCC" in the Engine/Build/Batchfiles/Linux directory.) When that's done, you should be able to just build UE4Editor and ShaderCompileWorker.

I haven't yet made a pull request from it because I haven't tested it on anything but a Debian Testing machine, and the fact that it's a weird workaround to a compile bug may or may not sit well with the developers at Epic. But one way or another, I believe I have a reasonable understanding of the issue causing this now.

Let me know if this works for any of you.

more ▼

answered Oct 04 '15 at 03:45 AM

avatar image

11 1 3

avatar image ashleysommer Oct 04 '15 at 08:16 AM

Oh wow, I was just going down this debugging path this afternoon. You have beat me too it. I just applied your patch and I am recompiling now. I will let you know the outcome.

I am on the promoted branch of unreal engine, which is currently between 4.10 and 4.11, the patch didn't go in cleanly so I applied the changes manually.

My machine is running Fedora 23 Beta, with Clang 3.7.0 so if it works I hope it can be included as a pull request to Epic.

Edit: Ok. I got it working :) The patch as is didn't work outright, I had to go through and also replace all references to std::string and std::stringstream in hlslcc_lib/IRDump.cpp and hlslcc_lib/IRDump.h. Those files may not be a problem in v4.9. I recompiled again and it worked for me.

Edit2: The discussion of the Clang bug you referenced includes this link: https://llvm.org/bugs/show_bug.cgi?id=21001 to a llvm bug report. Unfortunately they narrowed that down to a problem in the linker, which is not an LLVM/Clang component. The binutils bugreport is here: https://sourceware.org/bugzilla/show_bug.cgi?id=17458

avatar image ExpiredPopsicle Oct 05 '15 at 04:32 PM

Awesome! Glad it worked for you! (With some tweaking.) I guess I missed the std::string use in IRDump.*.

avatar image Kronykus Oct 11 '15 at 04:56 AM

Workded for me just now. Thanks.

avatar image ExpiredPopsicle Oct 12 '15 at 03:20 AM

Cool! More confirmations that it's working are making me want to put it together as PR already.

avatar image ashleysommer Oct 17 '15 at 03:41 AM

I just did a git pull today from the promoted branch, and I forgot that I'd applied this patch. The pull overwrote the changes, so I had to apply the patch again. It applied cleanly this time, so I did not need to edit any files. I noticed you have a second commit in your branch now with some additional fixes. I applied that one too.

You should definitely create a pull request for this change.

avatar image ExpiredPopsicle Oct 17 '15 at 04:21 AM

I'm working on it right now. Hope I'll have one up by the end of the weekend. I just want to be really thorough, so I'm testing the Windows build after the changes, too. I might even make the functional parts of the change into a Linux-specific modification with some #ifdefs, just to be sure.

avatar image ExpiredPopsicle Oct 18 '15 at 12:28 AM
avatar image Kronykus Oct 24 '15 at 05:41 PM

didn't work for me with the new 4.10-preview2 release branch. Instead, I end up with the issue reported in this thread

avatar image Kronykus Nov 04 '15 at 09:49 PM

With the new 4.10.0-preview-4, I got it to work using patches I made based off of your progress. I'm attaching a link to my Google Drive where I made a folder with the patches and the resulting files fully patched for comparison to the original. There's a ReadMe file that details my experience and what steps I took to get it to work. The patch files are the same as the ones I used to compile the official 4.9.2 release.

avatar image RCL STAFF Nov 05 '15 at 02:08 AM

Would you mind making a PR out of that?

avatar image ExpiredPopsicle Nov 05 '15 at 05:22 AM

There are a few issues with the branch in the state I have it right now. Have a look at the changes here: https://github.com/EpicGames/UnrealEngine/compare/master...ExpiredPopsicle:FinalClangFix5

I think @Kronykus 's version is based on an older version of my branch, from back when I made the first PR.

It's a pretty ugly fix. It relies on C's malloc() and free() to bypass all the issues with operator new/delete. It also lacks and conversion code between std::string and the new custom string type that uses the custom allocator that uses malloc/free. So parts of the Metal shader system break when compiled against it (but only on Linux because the whole change is reduced to a typedef to std::string on non-Linux builds). The whole thing seems very hacky and inappropriate, so I closed the PR myself once I learned about the Metal shader issues.

If you really want me to make another PR for it, let me know.

avatar image ExpiredPopsicle Nov 05 '15 at 01:50 PM

The branch I made my PR from has a few issues (and that's still what @Kronykus has, it seems). It breaks CrossCompilerTool because of code related to Metal and, depending on what version in the commit history of that branch using, breaks Windows or Linux builds. I fixed the Windows breakage, but I don't think @Kronykus 's version has those changes.

I could add more string conversion junk to the code that touches HLSLCC to convert from the new string type, and that would probably make the issue go away, but it would make the code ugly and slightly less maintainable.

I'm also not pleased that the code I wrote simply resorts to using C's malloc() and free() as a way to bypass all the C++ operator new/delete overloading issue entirely. It's hacky and gross.

At the moment I'm sitting on a mostly-functional branch based off the master branch that I haven't made another PR for just because it's so damn ugly. Have a look here https://github.com/ExpiredPopsicle/UnrealEngine/tree/FinalClangFix5 and tell me if you still want me to make another PR for this.

avatar image Kronykus Nov 05 '15 at 04:50 PM

You are correct. I'm using an older version. The last time I attempted the v5 fix was with preview 2, and I reported the issue I had there. It could have been a problem with preview 2 though, I can't say for sure. I don't generally have time to dig too deeply into it, so I've kept these patches and continued to use them until you had something new to try. My main concern was / is, can I get the editor to run, and is it usable? I wasn't overly concerned with bugs it might cause or platform checks, as the editor is still a little buggy and I'm only on Linux anyway. I just wanted to provide feedback and show what I was using. I'll try the v5 fix again, knowing that what I have now works, but it may take a day or two.

avatar image skadge Oct 15 '15 at 11:31 AM

Excellent! I've been able to apply successfully your patch on top of the branch 4.9, it compiles and solves the issue for me on Ubuntu 15.10. Worth a PR, I suppose!

avatar image ashleysommer Oct 17 '15 at 03:48 AM

Just a reminder, the release branch was promoted to version 4.10-preview1 and it is the one you should be using, unless you have a specific reason to use 4.9 :)

avatar image ndeakin Feb 07 '16 at 07:35 PM

I had to install UE 4.9.2 this past week on a fresh Arch Linux install, and encountered the hlslcc header issue. The issue appeared when I was using the "clang" package (clang 3.7) or "clang35" package (clang 3.5.2).

Using your patch (as described at https://github.com/EpicGames/UnrealEngine/pull/1674/files) fixed the problem with the clang35 package (didnt try with the clang package). Thanks so much! Was struggling with this for a while.

I did not have this issue when building around 5 months ago, with either clang or clang35 equivalent. I think this clang bug (assuming that is the root cause) may have been exposed by a change to hlslcc in that time? That, or I was using some intermediate version of clang (3.6?) that did not have the bug, but seems unlikely.

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

Ubuntu 15 - I did get this error using the release branch (4.10 at the time of writing). However, after checking out the '4.10' branch, which now has 4.10.1 (not in release branch yet) and doing a rebuild with clang-3.6 the problem is gone. I noticed that Setup.sh did rebuild hlslcc for me as well.

more ▼

answered Dec 02 '15 at 08:42 PM

avatar image

41 3 5

avatar image SalahAdDin Dec 04 '15 at 01:24 AM

I have the same error with clang 3.6 and release version, i didn't try with branch version but i'll try now, My trace back is:

alt text

My traceback it this:

avatar image SalahAdDin Dec 04 '15 at 01:25 AM
unreal log 2.txt (126.7 kB)
avatar image gesuwall Dec 13 '15 at 04:10 PM

I pulled branch 4.10 this morning and now UE4Edito builds successfully on Arch Linux using clang35 from official repositories. Just had to run Setup.sh again,accept to overwrite some hlslcc files, and then built the whole thing again to be safe.

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

I've rebuilt the entire project clean (even from fresh checkouts) many times and still end up with this hlslcc error in both the 4.9 and master/4.10 branches.

I've tried going into Engine/Source/ThirdParty/hlslcc/hlslcc/projects/Linux and running make (successfully), then going back to Engine and making ShaderCompileWorker and UE4Editor, but I still get the same hlslcc errors when running the editor afterwards.

I saw one of the other threads about copying a .a file around, but that just led to other errors the last time I tried it... and eventually to me doing a full clean checkout again to get rid of them and back to the original problem.

more ▼

answered Aug 27 '15 at 10:47 AM

avatar image

39 3 6

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

ok, ive tried - removing 4.8 - cleaning - using clang 3.6 only - recompile hlslcc (resulting in a segfault error instead) - tried various machines

Only thing I can think of is that im running ubuntu 15.10 beta

more ▼

answered Sep 02 '15 at 12:28 AM

avatar image

61 3 7 14

avatar image 3vi1 Sep 02 '15 at 01:14 AM

Yeah - I've been suspecting the same thing (I'm using 15.10 beta / clang 3.6.2 also). I was successfully compiling it in 15.10 a couple of months back - but I'm sure plenty of packages have changed since.

When I get a chance, I'm going to install 15.04 in a VM, map a shared folder, and try to compile from the exact same source.

avatar image 3vi1 Sep 02 '15 at 06:36 PM

I've confirmed it's definitely a toolchain issue with the newer version of something in 15.10.

I just built it from a 15.04 guest VM (w/ clang 3.6.0) using a shared folder pointed to exact same the source that's been failing with the hlslcc errors, and the resulting build then runs fine on the 15.10 host system.

avatar image ashleysommer Sep 29 '15 at 09:04 AM

I am having the same problem in Fedora 23, with clang 3.7.0, trying to start both UEEditor 4.10 and 4.11. Did you determine if it is a Clang issue, or something else in the toolchain? Were you able to work around the issue?

avatar image 3vi1 Sep 29 '15 at 12:56 PM

I didn't troubleshoot it further, as I have only played with UE4E once or twice out of curiousity. My workaround is to just compile it in a VM (and accept the huge time penalty that entails), then run it from my host system.

avatar image Kronykus Oct 01 '15 at 07:42 PM

My recent experience is, almost word for word, just as 3vi1's experience. I'm using Solydxk EE (k version), which for those unaware, is like Ubuntu/Mint. The EE version of Solydxk follows debian testing and has the issues above. The official stable build (non-EE) does work. Like 3vi1, I installed the stable version in VirtualBox to do the compile, though I had to run the Setup.sh and GenerateProjectFiles.sh scripts from the host, then compile in the guest. After compilation the editor starts and runs on host fine (so far).

Since I started looking for more info on this, I've noticed that the issues was noticed back around July. Since this seems to be affecting the newer toolchains coming down the pipe, I'm hopeful that the team is working on a fix (but Epic's attitude towards linux so far does leave me concerned).

(comments are locked)
10|2000 characters needed characters left
Viewable by all users
Your answer
toggle preview:

Up to 5 attachments (including images) can be used with a maximum of 5.2 MB each and 5.2 MB total.

Follow this question

Once you sign in you will be able to subscribe for any updates here

Answers to this question