Hello
I am facing a serious issue. Unreal engine crashes at a certain function most of the times. Here is the log.
LoginId:3015281a4af7986995ad0b93bac9a80e
EpicAccountId:806d36664f024562b2ddef02bfdb062d
Access violation - code c0000005 (first/second chance not available)
UE4Editor_Core!rml::internal::ExtMemoryPool::initTLS()
UE4Editor_Core!scalable_realloc()
UE4Editor_Core!FMallocTBB::Realloc() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\private\hal\malloctbb.cpp:88]
UE4Editor_ChessGame!TArray<wchar_t,FDefaultAllocator>::ResizeGrow() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\public\containers\array.h:2249]
UE4Editor_ChessGame!ChessEngineHolder::OnProcessOutput() [c:\dosyalar\ue4_projects\chessgame_release\source\chessgame\chessengineholder.cpp:198]
UE4Editor_ChessGame!TBaseRawMethodDelegateInstance<0,ChessEngineHolder,void __cdecl(FString)>::ExecuteIfSafe() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\public\delegates\delegateinstancesimpl.h:646]
UE4Editor_Core!TBaseDelegate<void,FString>::ExecuteIfBound() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\public\delegates\delegatesignatureimpl.inl:624]
UE4Editor_Core!FInteractiveProcess::ProcessOutput() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\private\misc\interactiveprocess.cpp:147]
UE4Editor_Core!FInteractiveProcess::Run() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\private\misc\interactiveprocess.cpp:208]
UE4Editor_Core!FRunnableThreadWin::Run() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\private\windows\windowsrunnablethread.cpp:76]
Another log here
LoginId:3015281a4af7986995ad0b93bac9a80e
EpicAccountId:806d36664f024562b2ddef02bfdb062d
Access violation - code c0000005 (first/second chance not available)
UE4Editor_Core!rml::internal::ExtMemoryPool::initTLS()
UE4Editor_Core!scalable_realloc()
UE4Editor_Core!FMallocTBB::Realloc() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\private\hal\malloctbb.cpp:88]
UE4Editor_Core!TArray<wchar_t,FDefaultAllocator>::ResizeForCopy() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\public\containers\array.h:2281]
UE4Editor_Core!FString::FString() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\public\containers\unrealstring.h:1700]
UE4Editor_ChessGame!ChessEngineHolder::OnProcessOutput() [c:\dosyalar\ue4_projects\chessgame_release\source\chessgame\chessengineholder.cpp:183]
UE4Editor_ChessGame!TBaseRawMethodDelegateInstance<0,ChessEngineHolder,void __cdecl(FString)>::ExecuteIfSafe() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\public\delegates\delegateinstancesimpl.h:646]
UE4Editor_Core!TBaseDelegate<void,FString>::ExecuteIfBound() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\public\delegates\delegatesignatureimpl.inl:624]
UE4Editor_Core!FInteractiveProcess::ProcessOutput() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\private\misc\interactiveprocess.cpp:147]
UE4Editor_Core!FInteractiveProcess::Run() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\private\misc\interactiveprocess.cpp:208]
UE4Editor_Core!FRunnableThreadWin::Run() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\private\windows\windowsrunnablethread.cpp:76]
The function at chessengineholder.cpp:178-211. This function is being called by another thread and if currmove or bestmove is found we send it to game thread. From the logs it seems this function is the cause?
void ChessEngineHolder::OnProcessOutput(FString Message)
{
if (Message.Find("currmove ") >= 0 || Message.Find("bestmove ") >= 0)
{
int32 tmp = WhichChessEngineThisIs;
FFunctionGraphTask::CreateAndDispatchWhenReady([tmp, Message]() {
// code here runs on game thread
UChessEngineCommunicator::OnChessEngineOutput(tmp, Message);
}, TStatId(), NULL, ENamedThreads::GameThread);
}
if (CEProcess.IsValid() && CEProcess->IsRunning())
{
int32 From = Message.Find("uciok");
if (From >= 0)
{
SendCommandToCE("isready", false);
return;
}
From = Message.Find("readyok");
if (From >= 0)
{
bIsReady = true;
// Send yet not sent commands to chess engine and empty the array
for (const FString &command : YetNotSentCommands)
{
SendCommandToCE(command);
}
YetNotSentCommands.Empty();
return;
}
}
}
I am not sure why this is happening but it seems to me i am doing something wrong while using multi-threading, lambdas, fstring all together.
Can somebody at least point me some direction. I am really at a loss and i have some doubts if passing variables as params from a thread to another is dangerous.
Thanks in advance.
One thing i realized is, the crash happens when ResizeAllocation is called. It is being called maybe a thousand times but only one causes crash. Also for some reason debugger doesn’t exactly pinpoint which line inside ResizeAllocation causes crash.
For example at one crash: engine\source\runtime\core\public\containers\array.h:2249
AllocatorInstance.ResizeAllocation(OldNum, ArrayMax, sizeof(ElementType));
At another crash: engine\source\runtime\core\public\containers\array.h:2281
AllocatorInstance.ResizeAllocation(0, NewMax, sizeof(ElementType));
It looks like it is crashing when constructing a string inside your ChessEngineHolder::OnProcessOutput method. Can you check what line in the code sample is
chessengineholder.cpp:183
(or could you add line numbers for the code? 
just the line number for the first line might be enough).
One initial workaround you may be able to use is FName instead of FString (the crash is in memory allocation during string construction, FName will not allocate every time).
I have updated the code and because of it the new crash line is 190.
Here is the lines from 178-213:
void ChessEngineHolder::OnProcessOutput(FString Message)
{
// @note No return after confirming the message because we want the message to be logged
// by game thread at the end of this function
if (CEProcess.IsValid() && CEProcess->IsRunning())
{
int32 From = Message.Find("uciok");
if (From >= 0)
{
SendCommandToCE("isready", false);
}
From = Message.Find("readyok"); ///< THIS IS WHERE IT SAYS CRASH HAPPENS: LINE 190 IS THIS LINE
if (From >= 0)
{
bIsReady = true;
// Send yet not sent commands to chess engine and empty the array
for (const FString &command : YetNotSentCommands)
{
SendCommandToCE(command);
}
YetNotSentCommands.Empty();
}
}
if (Message.Find("currmove ") >= 0 || Message.Find("bestmove ") >= 0)
{
int32 tmp = WhichChessEngineThisIs;
FString * theMessage = new FString(Message); ///< Will be deleted right after execution of below task
FFunctionGraphTask::CreateAndDispatchWhenReady([tmp, theMessage]() { ///< CRASH SOMETIMES HAPPENS HERE!
// code here runs on game thread
UChessEngineCommunicator::OnChessEngineOutput(tmp, *theMessage);
delete theMessage; ///< delete the pointer the moment its lifetime is over
}, TStatId(), NULL, ENamedThreads::GameThread);
}
}
And here is the crash:
...
UE4Editor_ChessGame!TArray<wchar_t,FDefaultAllocator>::ResizeGrow() [c:\users\ceset\desktop\ue_4.15_git\engine\source\runtime\core\public\containers\array.h:2249]
UE4Editor_ChessGame!ChessEngineHolder::OnProcessOutput() [c:\dosyalar\ue4_projects\chessgame_release\source\chessgame\chessengineholder.cpp:190]
...
This ResizeAllocation really has some problems. This time it crashed at render thread, which happens extremely rare. But the crash line isn’t surprising since the other crash was also happening because ResizeAllocation func.
Here is an ss of that crash:
It’s unlikely that this is the allocaor’s fault. This kind of apparent randomness makes me think of memory being written where it shouldn’t.
To discard possible causes, if you don’t CreateAndDispatch your Lambda, does it crash?
If i don’t create the lambda the game won’t continue which will make it impossible to test the problem. But i think you are probably correct regarding the create and dispatch. I just need to figure out the correct way of doing this.
EDIT: Sorry, i was half sleepy. I corrected my sentences
I made some changes.
Before i made changes, crash would happen at the middle of the game randomly. Good news is, now it happens when i unload the current level and load another level.
What happens when level changed is that the existing ChessEngineHolder will be deleted immediately(Since it is a TSharedPtr). After that the new level will try to create a new one. Crash immediately happens when the new level is opened.
In this version of the function the crash happens at 3rd line. The line where i create the pointer.
void ChessEngineHolder::OnProcessOutput(const FString& Message)
{
FString * theMessage = new FString(Message); ///< Will be deleted right at the end of this function
...
int32 WhichCE = WhichChessEngineThisIs;
FFunctionGraphTask::CreateAndDispatchWhenReady([WhichCE, theMessage]() {
// code here runs on game thread
UE_LOG(LogChessEngines, Log, TEXT("ChessEngine%d: %s"), WhichCE, **theMessage);
if (theMessage->Find("currmove ") >= 0 || theMessage->Find("bestmove ") >= 0)
{
UChessEngineCommunicator::OnChessEngineOutput(WhichCE, *theMessage);
}
delete theMessage; ///< delete the pointer since its lifetime is over
}, TStatId(), NULL, ENamedThreads::GameThread);
}
It seems i fixed the crash issue. Though i am not sure the exact cause of the problem, i can no longer reproduce it. Two changes i made:
-
Used TSharedPtr instead of raw pointer, because in case the life of the deleter ends the pointer will get to exist whch will cause problems later on.
-
Made a small change to how i destroyed the FInteractiveProcess. I used to call theProcess->Stop(), which it already calls itself when destructor is called. What happens if the two are called together? Though no function call should work from outside at the moment of destruction (it is not an UObject), i had to be sure. Might be causing some unwanted things. Therefore i removed it.
Now the code looks like this:
void ChessEngineHolder::OnProcessOutput(const FString& Message)
{
TSharedPtr<FString, ESPMode::ThreadSafe> theMessage(new FString(Message)); ///< Thread safe shared pointer, how lovely!
...
...
int32 WhichCE = WhichChessEngineThisIs; ///< So we don't have this passed as argument!
FFunctionGraphTask::CreateAndDispatchWhenReady([WhichCE, theMessage]() {
// code here runs on game thread
UE_LOG(LogChessEngines, Log, TEXT("ChessEngine%d: %s"), WhichCE, **theMessage);
if (theMessage->Find("currmove ") >= 0 || theMessage->Find("bestmove ") >= 0)
{
UChessEngineCommunicator::OnChessEngineOutput(WhichCE, *theMessage);
}
}, TStatId(), NULL, ENamedThreads::GameThread);
}
