ShaderCompileWorker eats memory like crazy

During initial editor startup when it compiles lots of shaders, I observe ShaderCompileWorker processes steadily growing up to 2GB+ RAM each while they compile shaders (after what I have to shutdown editor due to insufficient RAM). On Windows, ShaderCompileWorker are almost always under 100MB. So, is it a leak that should be fixed or there is a really strong reason why OpenGL shader compilation should take 20x RAM more than DirectX?

I thought that we had these leaks sorted out way back when…

There were couple of major leaks in hlslcc that we patched, but come to think of it after submitting so many PRs I don’t remember if that specific one was ever incorporated or not.

If you have the time and inclination, please send a PR with a fix and I’ll pest0r RCL to add it in. SCW should definitely not eat 2GB+ of RAM.

I thought that we had these leaks sorted out way back when…

I don’t believe others who play with UE on Linux don’t notice 3-5 (depending on number of cores) processes each eating 2GB+ RAM.

Whatever, I’l try to valgrind this beast.

We ran SCW through valgrind and found couple of major memory leaks in hlslcc. They were patched in my 4.4-linux branch, and iirc I did submit PR upstream to Epic for those.

I’m not sure what happened afterwards, as Epic made further changes to SCW and more integration of hlslcc into the engine occurred as well.

I forgot to include a link to my branch, so here is the commit for the hlslcc leaks identified back in 4.4 days https://github.com/3dluvr/UnrealEngine/commit/f9911a3f713409add857ea3869d100ff7cf11ea3

Thanks, I’ll work on pushing this upstream.