Lightmaps badly broken after lighting build

I’ve used Unreal for a while now but recently started to look at learning all the ins and outs of high quality rendering. Recently, I’ve started to get this random issue where the lighting build totally breaks and refuses to update any further. Here’s where it looks like:

I’m sure most people are used to having to move the camera or doing something similar to get the viewport to update when you start the engine or after you’ve built lighting, it seems to me like my editor is refusing to do that anymore. As an example, here’s a couple of shots that show how the packaged version looks at different stages of starting up:

[Stage 1][2]

[Stage 2][3]

[Stage 3][4]

When the issue occurs, it seems to basically stick to that first image and not update. The weird part is, it’s project specific. So far, I’ve managed to get around it by reverting to old backups but I’d really like to know what’s causing it.

Here’s a few important points:

  • The meshes aren’t made for games, some of them are quite terrible in terms of geometry but they look fine most of the time
  • I’m using distributed rendering to build lighting
  • I’ve tweaked the BaseLightmass.ini at times but kept a backup of the default version which I’m using at the moment
  • All my scalability options are set to Epic, using shader model 5, material quality high

Things I’ve tried:

  • Restarted editor, PC, swarm agent, coordinator and all agents
  • Removed all files from User/AppData/Local/UnrealEngine
  • Cleaned and validated the local Swarm agent
  • Built lighting both with and without distributed rendering
  • Built in a new level in the same project with the same meshes, issue is the same
  • Forced no precomputed lighting and built to remove lightmaps, rebuilt and still broken
  • Built on all lighting quality levels, just get slightly different versions of the same thing

Any help or suggestions would be amazing!

I think I am having the same issue as you, the lighting builds perfectly several times and then all the sudden breaks. Lights and shadows no longer have any detail and things look awful. This is the second project I have had this happen to and can’t figure out why this is happening. I have done everything you listed above including migrating levels over to a new project (this worked once, but not after the second build and has not worked since), deleting the intermediate folder (after making a copy of course) and even tried building in the new 4.13 version. Nothing seems to work. It doesn’t happen with every project though, and it seems totally random - fine one build, broken forever the next.

In other words, I am adding my vote for this being an issue.

Hey Imarie, just curious what similarities we have with our projects; maybe we can narrow down the issue. Are you doing any of these things?

  • Using Oculus

  • Pushing the lightmaps to high resolutions (some of mine are at 2048)

  • Building lighting over a few PCs using Swarm

  • Really pushing the light mass with high settings

Hi Capt!

Not using oculus, so that isn’t the issue. These seemed to break earlier on before changing settings so those were still at default. We are using swarm over multiple pcs though and do have some lightmaps that go up to 4096 in size, so those are similarities. I also have complex geometry in the scene as you do, although this wasn’t happening until recently so that and the higher lightmap size were never a problem before. The only more recent addition is distributed rendering, even though turning that off doesn’t seem to fix things once they are broken.

meant to post that as a comment, oops.

Hm that’s interesting. It triggered earlier while I was playing with materials. The editor struggled to redo the shaders when I kept updating materials and then lighting broke. I haven’t built lighting all day so it could just be an issue with Unreal struggling to handle shaders when there is a lot of complex geometry, big lightmaps, lots of reflections etc. Do you have a lot of materials in your scenes? I have hundreds.

I also found a much better way to fix the issue: create a new project and migrate your map into the new project. Everything required for the map should come across so you just have to fix up the project settings (including player controllers etc.). I’ve successfully built lighting in a previously broken map this way. This makes me think that there should be a solution somewhere in the project files, like deleting a corrupt file.

I have gotten that to work once, but it doesn’t work every time. I am still trying to figure out why it worked once and not since, maybe because everything was in one level the first time and sub levels after. Not sure. I do have a ton of materials in this scene as you do, but I have had this happen in a much less complex scene also.

It would be nice to know a real solution to this, or at least what could be causing it so it could be avoided. I am going to try migrating everything over from a single level and see if that works better. I wonder if I deleted the intermediate folder before I migrated the level over the one time too. There was something I did that made it work, just wish I remembered what it was. :slight_smile:

Yeah it’s definitely a very tricky one to fix :confused: I’ll keep tackling it as it comes up and see if I can narrow down the issue. Hopefully someone who knows a bit more about the inner workings of Unreal rendering will see this sometime soon!

Bringing over my suggestion from another post you pinged me on:

Looks like low quality lightmaps streaming in, and not to be confused with low lightmap resolution. You can test this by disabling texture streaming in the Project Settings > Rendering.

And you’re follow up response:

That’s great information. I’ve narrowed it down to being caused whenever I pushed lightmaps from 1024 to 2048 resolution, is there any reason doing that would break the texture streaming? Turning off streaming altogether does seem to solve the issue, but that’s probably not a good solution in the long run right? Thanks for looking into it, I understand if you don’t have more time for it at the moment.

I pushed lightmaps from 1024 to 2048 resolution, is there any reason doing that would break the texture streaming?

Yes. If you have a lot of textures that are required to be streamed in and especially large lightmap resolutions with these it’s much more texture memory being used and may need more time to stream in or may not have a large enough streaming texture pool to draw from. Default is ~1gb for all textures.

Turning off streaming altogether does seem to solve the issue, but that’s probably not a good solution in the long run right?

For most, no this is definitely not a good solution! Texture streaming makes use of Mips to lower the resolution of the texture are further distances. When this is disabled the full resolution of the texture is continually loaded into memory rather than being lowered to make things more optimized. The suggestion was just a quick way to identify if that was in fact the issue you were running into.

If you’re goal here is a small scene to render and you have the hardware to handle a the large texture memory consumption then by all means do what works best for you. If you’re deploying this to lower-end hardware of need it to be deployed to different users or clients to run then optimization is best!

Continued…

Some things you can do to curb this:

  • Increase your texture streaming pool by adding adjusting the value in the ini file. You can find more information here: Texture Streaming in Unreal Engine | Unreal Engine 5.1 Documentation
  • Have a loading screen or give more time for things to load in and see if that resolves the issue.
  • You can try adjusting the LightmapStreamingFactor console command. Default uses 0.000, where lower values make streaming more aggressive. Try a higher value to see if that helps curb the streaming.
  • Lastly, manually setting the lightmaps to never stream (No Mips) by opening the textures manually after a light build in the World Settings. However, this method would require this to happen every time after a light build since the previous data is wiped and you could have tons of textures here to do this for! If that’s the route you’re willing to take it would be better to programmatically adjust this through source code so that you don’t have to worry about it. As a reminder, having no mips/never streaming for any textures can be costly for texture memory.

You are my hero! Turning off texture streaming fixed the problem. Quick question though - will leaving it off use more GPU or RAM?

@Melissa Marcy

It absolutely will! It will use more VRAM on your GPU and if it’s using too much may likely crash the video driver if you are not careful. My suggestion above was more or less to quickly assess the issue and then following suggestions are good to help you resolve or workaround your streaming texture issue.

Thank you for the quick response!

Quick question from our IT guy: does increasing internal computer RAM help at all, or is it solely video card memory?

Trust me, I am going to optimize this anyway. He is just curious so we know going forward for future machines. :slight_smile:

This is great Tim, thanks for finding the time to go into depth with this!

I always push the texture streaming pool up to 4096 and above but it has never fixed this issue after it happened, which is why I never looked much into texture streaming. Disabling the texture streaming entirely seems to make things look normal again but like you mentioned crashes occur regularly due to low memory. As soon as I reduce all 2048 lightmaps and rebuild it fixes it though.

To get around the issue we’ve broken up all our walls etc. into small meshes and given them lightmaps below 2048. Instead of one 2048 wall we now have more than 10 wall meshes each at 512-1024. I would think this would cause the same (if not more) strain on memory, but the issue has never occurred and we get nice shadows. Are 2048 lightmaps really that much more of a strain? It seems like the engine struggles (and usually fails) to process maps of this size, but lots of maps of a smaller size are fine. Either way, it’s strange things get sort of stuck in this ugly mode even after increasing texture streaming pool size.

We’re currently testing this on a PC with 64GB of memory running a GTX 980 (4GB of memory) if that helps at all.

Also, I can’t seem to find the LightmapStreamingFactor console command referenced anywhere, I just typed it into the console with ‘1’ at the end and that didn’t seem to do anything.