[4.9.1] Creating hundreds of level assets leads to 'Out of video memory' crash

I have attached a repro project, where I spawn hundreds of levels one after another from C++. This was a bug encountered in an actual project, where architectural data was used to split actors amongst hundreds of levels based on certain criteria (we’re dealing with a large site/building).

Please check the LevelCreationBugPlugin for the relevant code (levels are created in LevelCreator.cpp).

To actually cause the crash, click the ‘Create Levels’ button in the top editor toolbar. Note: the levels being added are completely empty.

Specs:

  • Windows 8.1
  • i7 4790k
  • GTX 970
  • 16gb RAM

Edit:

I have converted to using a static int32 for the level counter / naming so I can try it over and over to see if it’s more an issue with making hundreds at once, or hundreds in the one session.

void FLevelCreationBugPlugin::LevelCreationButtonClicked()
{
	// make lots of levels
	static int32 i = 0;
	int32 limit = i + 100;
	for (; i < limit; ++i)
	{
		m_levelCreator->CreateLevel("Level" + FString::FromInt(i));
	}
}

Clicking the button three times with this new code leads to the error halfway through the third attempt - for me it’s around ~260-270 levels created in total.

If you save the level assets and then restart the editor (making sure to have a differently-named main/master level loaded, so the new levels can still be named from index 0 onwards but are put into a different subfolder in order to not conflict with levels created in the previous session), it still occurs at roughly the same spot. The problem is creating hundreds of levels in the one editor session (tried both Development Editor and DebugGame Editor).

Even if this takes forever to fix, a way to spawn hundreds of levels in one editor session would be amazing. Any workaround, no matter how gross, would be appreciated!

EDIT#2:

The callstack when the out of memory occurs suggests that the problem is inside of UWorld::Rename’s call to GetLightMapsAndShadowMaps, even though it’s being called on a world containing only the default setup and is only returning an empty TArray by reference. This is all triggered from my call to EditorLevelUtils::CreateNewLevel. I’ll keep digging.

EDIT#3:

I’ve changed the naming of levels so that it uses the ‘Now’ time (I know it’s culling the int64 down to an int32, but it’s all good).

void FLevelCreationBugPlugin::LevelCreationButtonClicked()
{
	m_levelCreator->CleanupPreviousLevels();

	// make lots of levels
	static int32 i = 0;
	int32 limit = i + 100;
	for (; i < limit; ++i)
	{
		m_levelCreator->CreateLevel("Level" + FString::FromInt(FDateTime::Now().GetTicks()));
	}
}

I’ve used this to test if it was a problem with having x number of levels in the one world, since when I was testing creating levels in different sessions, they were levels added to a new world each time. Using this new code, create a new, default world, and save it so it can be opened next time. Click the ‘Create Levels’ button, and then save the content browser. Close the editor, then reopen it and load in the world you created earlier. Repeat this step forever (I got to 400 levels before I moved on). It is not linked to the number of levels assigned to a world, or the loading/storage of those levels in memory (since all of a world’s levels are loaded in when that world is opened in the editor).

EDIT#4:

Simply re-opening the master level/world after spawning 100 levels and saving the content browser allows for the creation of another 100. As long as the original level/world is re-opened each time, no errors occur. This would be a terrible work-around, but can I force reload of the world from C++?

Relevant Files:

Repro project

DxDiag

I downloaded and ran the project, made it to 399 before I got the same error.

Hey -

The crash is caused by the editor attempting to run setup on each of the levels at once. When that many levels are calling the same setup code repeatedly it can potentially cause memory errors. It would be best to limit the number of levels created at once in order to limit the number of calls to the setup code.

Cheers

Hi ,

My issue with this answer is that I have made clear that the problem is not with making too many levels at once, but rather too many in one ‘session’ (since reloading the map and/or restarting the editor resets this limitation). There is some kind of leak, or memory otherwise sticking around when it’s not needed, whenever a new level is created in the editor.

This ‘setup code’ should not have an effect on memory minutes later - it seems that something isn’t being cleaned up on level creation :slight_smile:

The number of levels that I can create is also unpredictable - yesterday it was up to 250 at a time, but today even 35 levels in one map-editing session is causing an ‘Out of video memory’ error.

Hey -

I’ve entered a bug report for the Out of Memory crash when creating levels (UE-21790). For now the best solution would be to limit the number of levels being created at once.

Cheers