[Editor] Editor crash and memory leak during importing huge landscape

Development World Creation
, , , ,
1

Here is my project: DropMeFiles – free one-click file sharing service (minimal project to reproduce this problem)

Try to:

  • Open Import_Tiles_4k project in archive

  • Open map Main (if not opened by default)

  • Check what world configured as “Enable World Composition”

  • At Menu “Levels” call “Import Tiled Landscape”

  • Set included material in poroject (“Landscape_Automaterial_Inst”) and select all heightmaps on folder “Content/Source” such as on Screenshot_1 and check memory ~1.2Gb

  • Press Import

  • After importing ~1/4 chunks. Was allocated ~4.4Gb (Screenshot_2)

  • After importing ~2/4 chunks. Was allocated ~13Gb (Screenshot_3)

  • After importing ~3/4 chunks. Was allocated ~24.5Gb (Screenshot_4)

  • On Landscape_x64_y49 - crash and allocate more than have phys memory (32Gb total)

    [2017.07.10-20.43.08:059][531]LogSavePackage: Moving ‘D:/Projects/Import_Tiles_4k/Saved/Landscape_x63_y480E9CADD64C36E9D8A96124B5A04B62A9.tmp’ to ‘D:/Projects/Import_Tiles_4k/Content/Landscape/Landscape_x63_y48.umap’
    [2017.07.10-20.43.08:061][531]LogSavePackage:Display: Finished SavePackage D:/Projects/Import_Tiles_4k/Content/Landscape/Landscape_x63_y48.umap
    [2017.07.10-20.43.12:468][531]Cmd: OBJ SAVEPACKAGE PACKAGE="/Game/Landscape/Landscape_x63_y49" FILE=“D:/Projects/Import_Tiles_4k/Content/Landscape/Landscape_x63_y49.umap” SILENT=true AUTOSAVING=false KEEPDIRTY=false
    [2017.07.10-20.43.16:992][531]LogSavePackage: Save=11.12ms
    [2017.07.10-20.43.16:992][531]LogSavePackage: Moving ‘D:/Projects/Import_Tiles_4k/Saved/Landscape_x63_y4927ABADA5420AC5C1BBCF0B8A571F9CBD.tmp’ to ‘D:/Projects/Import_Tiles_4k/Content/Landscape/Landscape_x63_y49.umap’
    [2017.07.10-20.43.16:992][531]LogSavePackage:Display: Finished SavePackage D:/Projects/Import_Tiles_4k/Content/Landscape/Landscape_x63_y49.umap
    [2017.07.10-20.43.18:602][531]LogWindows: Windows GetLastError: Операция успешно завершена. (0)
    [2017.07.10-20.43.18:791][531]LogCrashTracker:

    [2017.07.10-20.43.18:791][531]LogWindows:Error: === Critical error: ===
    [2017.07.10-20.43.18:798][531]LogWindows:Error:
    [2017.07.10-20.43.18:798][531]LogWindows:Error: Assertion failed: NumElements + 1 <= MaxElements [File:C:\Unreal Engine\Engine\Source\Runtime\CoreUObject\Public\UObject/UObjectArray.h] [Line: 245]
    [2017.07.10-20.43.18:799][531]LogWindows:Error: Maximum number of UObjects (8388607) exceeded, make sure you update MaxObjectsInGame/MaxObjectsInEditor in project settings.
    [2017.07.10-20.43.18:800][531]LogWindows:Error:
    [2017.07.10-20.43.18:801][531]LogWindows:Error:
    [2017.07.10-20.43.18:801][531]LogWindows:Error:
    [2017.07.10-20.43.19:875][531]LogExit: Executing StaticShutdownAfterError
    [2017.07.10-20.43.20:658][531]LogWindows: FPlatformMisc::RequestExit(1)
    [2017.07.10-20.43.20:683][531]Log file closed, 07/10/17 23:43:20

P.S. full log file in attach.

2

I’m tryed enable memory profile at chunk import cycle and get Screenshot_1

Then I’m insert my function which print the number of objects of all classes Screenshot_2

  • 100 chunks - 3417 instances of WorldTileDetails
  • 500 chunks - 81017 instances of WorldTileDetails

Deeper mining: SWorldHierarchy → STreeView contains all root (Persistent) WorldTileDetails from import cycles which recreate each cycle in SparseItemInfos which not clearing; Screeshot_3 & Screeshot_4

Then I found function RefreshView() at SWorldHierarchy.cpp which recreate LevelModel hierarchy and add call clear treeview as:

/** Regenerates current items */
void RefreshView()
{
	TreeWidget->ClearExpandedItems();

	// Sync items expansion state
	struct FExpander : public FLevelModelVisitor 
	{
		TSharedPtr<SLevelsTreeWidget> TreeWidget;
		virtual void Visit(FLevelModel& Item) override
		{
			TreeWidget->SetItemExpansion(Item.AsShared(), Item.GetLevelExpansionFlag());
		};
	} Expander;
	Expander.TreeWidget = TreeWidget;
	
	//Apply expansion
	WorldModel->IterateHierarchy(Expander);
}

After this I take - Screenshot_5 (compare original and with my fix import) on half chunks and fully imported whithout crashes and huge memory leaks Screenshot_6

P.S. seems like my other import problem (link to thread) is depend on this. After fix - seems like worked for now and not stucking

3

Fixed at 4.17.0 and latest

1457×660 183 KB

by fully rewritten this UI mechanism