Scene Outliner and TSharedRef crash

Development Programming & Scripting
1

Hi,

We’re currently on 4.6 with some fixes from 4.7/4.8 integrated. This crash has been happening at seemingly random moments and without a clear repro case. Some reports state that it happened when the user was pressing Undo, others when placing a mesh in the scene.

The output in the log-file with the stacktrace looks like this:

[2015.08.05-13.59.11:075][850]LogEditorTransaction: Undo Move Actors
[2015.08.05-13.59.11:114][850]LogVillainNavCache: Villain cache invalidated
[2015.08.05-13.59.11:166][850]LogCrashTracker: 
<snip>
[2015.08.05-13.59.37:355][850]LogWindows: === Critical error: ===
Fatal error!

Unhandled Exception: EXCEPTION_ACCESS_VIOLATION 0x5380ec00

UnknownModule!UnknownFunction (0x000000355380ec00) + 0 bytes [UnknownFile:0]
UE4Editor-SceneOutliner.dll!SceneOutliner::SSceneOutliner::GetParentsExpansionState() (0x00007ffe1f48f368) + 162 bytes [d:\tc\757805abc141f763\yankee\engine\source\editor\sceneoutliner\private\ssceneoutliner.cpp:1167]
UE4Editor-SceneOutliner.dll!SceneOutliner::SSceneOutliner::Populate() (0x00007ffe1f4b711f) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\editor\sceneoutliner\private\ssceneoutliner.cpp:721]
UE4Editor-SceneOutliner.dll!SceneOutliner::SSceneOutliner::Tick() (0x00007ffe1f4cba18) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\editor\sceneoutliner\private\ssceneoutliner.cpp:3107]
UE4Editor-SlateCore.dll!SWidget::TickWidgetsRecursively() (0x00007ffe339ece83) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slatecore\private\widgets\swidget.cpp:334]
UE4Editor-SlateCore.dll!SWidget::TickWidgetsRecursively() (0x00007ffe339ecf62) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slatecore\private\widgets\swidget.cpp:338]
UE4Editor-SlateCore.dll!SWidget::TickWidgetsRecursively() (0x00007ffe339ecf62) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slatecore\private\widgets\swidget.cpp:338]
UE4Editor-SlateCore.dll!SWidget::TickWidgetsRecursively() (0x00007ffe339ecf62) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slatecore\private\widgets\swidget.cpp:338]
UE4Editor-SlateCore.dll!SWidget::TickWidgetsRecursively() (0x00007ffe339ecf62) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slatecore\private\widgets\swidget.cpp:338]
UE4Editor-SlateCore.dll!SWidget::TickWidgetsRecursively() (0x00007ffe339ecf62) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slatecore\private\widgets\swidget.cpp:338]
UE4Editor-SlateCore.dll!SWidget::TickWidgetsRecursively() (0x00007ffe339ecf62) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slatecore\private\widgets\swidget.cpp:338]
UE4Editor-SlateCore.dll!SWidget::TickWidgetsRecursively() (0x00007ffe339ecf62) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slatecore\private\widgets\swidget.cpp:338]
UE4Editor-SlateCore.dll!SWidget::TickWidgetsRecursively() (0x00007ffe339ecf62) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slatecore\private\widgets\swidget.cpp:338]
UE4Editor-SlateCore.dll!SWidget::TickWidgetsRecursively() (0x00007ffe339ecf62) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slatecore\private\widgets\swidget.cpp:338]
UE4Editor-SlateCore.dll!SWidget::TickWidgetsRecursively() (0x00007ffe339ecf62) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slatecore\private\widgets\swidget.cpp:338]
UE4Editor-SlateCore.dll!SWidget::TickWidgetsRecursively() (0x00007ffe339ecf62) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slatecore\private\widgets\swidget.cpp:338]
UE4Editor-SlateCore.dll!SWidget::TickWidgetsRecursively() (0x00007ffe339ecf62) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slatecore\private\widgets\swidget.cpp:338]
UE4Editor-Slate.dll!FSlateApplication::TickWindowAndChildren() (0x00007ffe33c91532) + 72 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slate\private\framework\application\slateapplication.cpp:893]
UE4Editor-Slate.dll!FSlateApplication::TickWindowAndChildren() (0x00007ffe33c91655) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slate\private\framework\application\slateapplication.cpp:898]
UE4Editor-Slate.dll!FSlateApplication::Tick() (0x00007ffe33c90dad) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\slate\private\framework\application\slateapplication.cpp:1297]
UE4Editor.exe!FEngineLoop::Tick() (0x00007ff7130c10c1) + 15 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\launch\private\launchengineloop.cpp:3025]
UE4Editor.exe!GuardedMain() (0x00007ff7130b42cc) + 0 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\launch\private\launch.cpp:154]
UE4Editor.exe!GuardedMainWrapper() (0x00007ff7130b433a) + 5 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\launch\private\windows\launchwindows.cpp:126]
UE4Editor.exe!WinMain() (0x00007ff7130c291a) + 17 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\launch\private\windows\launchwindows.cpp:214]
UE4Editor.exe!__tmainCRTStartup() (0x00007ff7130c3a49) + 21 bytes [f:\dd\vctools\crt\crtw32\dllstuff\crtexe.c:618]
KERNEL32.DLL!UnknownFunction (0x00007ffe5f5613d2) + 0 bytes [UnknownFile:0]
ntdll.dll!UnknownFunction (0x00007ffe60905444) + 0 bytes [UnknownFile:0]
ntdll.dll!UnknownFunction (0x00007ffe60905444) + 0 bytes [UnknownFile:0]

The other crash logs look slightly different:

[2015.08.06-11.19.22:512][234]LogWindows: === Critical error: ===
Fatal error!

Unhandled Exception: EXCEPTION_ACCESS_VIOLATION reading address 0xffffffff

UE4Editor-UnrealEd.dll!SharedPointerInternals::DestroyObject<SAssetThumbnail>() (0x00007fffa52c682d) + 13 bytes [d:\tc\757805abc141f763\yankee\engine\source\runtime\core\public\templates\sharedpointerinternals.h:283]
UE4Editor-SceneOutliner.dll!SceneOutliner::SSceneOutliner::GetParentsExpansionState() (0x00007fff9367f368) + 162 bytes [d:\tc\757805abc141f763\yankee\engine\source\editor\sceneoutliner\private\ssceneoutliner.cpp:1167]

and:

[2015.07.31-15.53.03:252][574]LogWindows: === Critical error: ===
Fatal error!

Unhandled Exception: EXCEPTION_ACCESS_VIOLATION reading address 0xffffffff

UE4Editor-SceneOutliner.dll!SceneOutliner::SSceneOutliner::GetParentsExpansionState() (0x00007ff84f0df1c5) + 159 bytes [d:\tc\757805abc141f763\yankee\engine\source\editor\sceneoutliner\private\ssceneoutliner.cpp:1167]

All of them revolve around the SSceneOutliner.cpp file in the SSceneOutliner::GetParentsExpansionState function.
The exact line is:

if (Item.Key.IsValid() && !OutlinerTreeView->IsItemExpanded(Item.Value))

Note, the Item.Key.IsValid() was moved to the front here in a previous attempt to fix it, but it seems to me that the Key.Value here is the problem.

In one of the log-files the last line is at sharedpointerinternals.h:283, which matches SharedPointerInternals::DestroyObject call. Should this code have additional checks to safeguard the usage of the TSharedRef ? The documentation states “TSharedRef is a non-nullable, non-intrusive reference-counted authoritative object reference.” Could it be that it’s not thread safe? Or is the issue something completely different here?

2

Hi artofcode,

Would it be possible to find out what fixes from 4.7 and 4.8 you have merged into your 4.6 Engine? I’d like to try to get as close to the Engine you are using as possible when trying to repro this issue.

3

Just had a look and the number of fixes we ported over are a lot, too many to list here I think. There was no clear list available, but a search through perforce shows that it’s a regular activity.

Even with all changes, I doubt there is a way to reproduce it. There seems to be an average of 2 or 3 crashes per day (in quite a large user base) involving the same stack-trace, depending on the values that it tries to call, so the last line might be different, depending on a successful call or not (more often than not, the call fails).

I’ve been digging a bit deeper, looking at the disassembly and the line where it crashes is the “call qword ptr [rdi+10h]”. Looking at the memory while breaking just before and stepping through in the assembly view shows me that [rdi+10h] (in a normal run where it didn’t crash) points towards the “SharedPointerInternals::DestroyObject” function. (SharedPointerInternals.h).

When looking at a dump file, this value is what I believe corrupted, or it might not have been included in the dump file (it’s a minidump file). Another thing to note is that the value is in a completely different place: 0x0000005CCE4E0650h in the crashed dump, compared to 00007ffe22ca7f40h.

So it seems to me that the SharedPointer is corrupted? Thing is, I have no clue on how this got corrupted in the first place.

4

Just wanted to provide a quick update for you on this. Apparently a few minutes after I left the office on Friday, one of our Engine developers whom I had asked to take a look at this issue got back to me and said that this looks somewhat similar to an issue that had appeared occasionally in 4.6. He said that he believes this issue was corrected as part of some major changes to the Scene Outliner in 4.7 or 4.8, and that he is planning to take a closer look at this today to see if he can put together a spot fix for your 4.6 Engine.

5

Hi , any more news on this?

6

Hi artofcode,

Sorry for the delay on this. I checked in with the developer who was looking into this, and he let me know that what is most likely occurring here is that a TBitArray in the SceneOutliner may occasionally contain an invalid state, which causes the Engine to trip up. A fairly simple fix for this was implemented here, and we don’t anticipate that you will have much trouble merging it into your 4.6 Engine. Please let us know if you see this crash again after merging in this fix.

7

I’ll let you know, thanks!

8

This seems to have done the trick, the Crash Reporter doesn’t show any crashes with the matching stack trace since implementing it.