Editor freezing while parsing large text file

Hey everyone,

I’m trying to parse the english dictionary into the Editor and its causing the editor to freeze for hours at a time. I understand that over 355k words (and however many letters that is) is a lot, but is it really enough to freeze the engine? I could be wrong, but I figured all of the physics rendering and things the engine can do would be more intensive.

I guess my question is, is there a fix for this? Could it be due to low CPU specs? I have a Quad CoreI7 and a nVidia GTX780M and I’ve never really had issues compiling lighting data or anything like that. So I was thinking it could be a memory allocation issue.

Would there be a way to dictate how the engine handles tasks like this or would it be easier to just parse each file separately to decrease the file size?

It should also be known that I’m using 's victory plugin for the Load File Into String Array node. There are no errors while loading small files and I haven’t tested the file threshold.

Regards,
Entropikz

How big is the file you are parsing?

The thing is that parsing can be resource intensive. Maybe the problem is that you are doing this in the main thread and this causes the freezing. You should try multi-threading if you wanna parse that big file at run-time.

Check these tutorials on multi-threading:

https://wiki.unrealengine.com/Multi-Threading:_How_to_Create_Threads_in_UE4

https://wiki.unrealengine.com/Multi-Threading:_Task_Graph_System

Hey sorry for the late response!

The file is 2 MB so it’s relatively big for a text file, but I didn’t think that would be too much. I didn’t think about setting up multi-threading though. I’m definitely going to try this out right now and let you know what happens.

Thank you so much!

I’ve been trying to get the project working, but I’m having errors getting the code to compile. I keep getting Error MSB3073 “” exit code -1. I can’t seem to find a way around it either.

I also noticed that this solution works with Prime Number generation. How would I get this to work with my parse function? I’m not great when it comes to C++…

Hey Entropikz,

To start, when you say, “…physics rendering and things the engine…” - these are not related to the performance requirements for input / output for files. While modern game engines handling of physics, rendering, etc is exceptional, they are their own thing.

With that said, I am not using 's plugin so I cannot speak for it as for performance but I have created my own Actor class that loads a text file. The one I am testing is 2.56MB (or 2.6 million characters) and loads exceptionally fast; no noticeable lag.

[.h]

#pragma once

#include "GameFramework/Actor.h"
#include "TextLoader.generated.h"

UCLASS()
class AH509306_API ATextLoader : public AActor
{
	GENERATED_BODY()
	
public:	
	ATextLoader();
    
    UFUNCTION( BlueprintCallable, Category = "Test" )
    void LoadTextFile( );
};

[.cpp]

#include "AH509306.h"
#include "Runtime/Core/Public/Misc/CoreMisc.h"
#include "TextLoader.h"


ATextLoader::ATextLoader()
{
	PrimaryActorTick.bCanEverTick = false;

}

void ATextLoader::LoadTextFile( )
{
    FString Result;
    FString FileName = FString( "C:/Users/kyle.langley/Documents/Unreal Projects/AH509306/Content/wlist_match3.txt" );
    const TCHAR* FileNamePtr = *FileName;
    bool bLoaded = FFileHelper::LoadFileToString( Result, FileNamePtr, 0 );
    if( bLoaded )
    {
        UE_LOG( LogTemp, Warning, TEXT("Loaded file!") );
    }
    else
    {
        UE_LOG( LogTemp, Error, TEXT("Failed to load file!") );
    }
}

And the result:

Thank you for submitting a bug report, however at this time we believe that the issue you are describing is not actually a bug with the Unreal Engine, and so we are not able to take any further action on this. If you still believe this may be a bug, please provide steps for us to reproduce the issue, and we will continue our investigation.

Hey,

You’re absolutely right, this isn’t a bug. I figured out why I’m having issues. I’m trying to have everything handled on the game thread and its causing my UI to freeze. Thank you so much for showing me your code as well!

If you don’t mind I’d like to try it out and see how it works!

Can you explain to me more about what’s happening? Is there a error message or something telling you about the .dll?

Thanks.

I know we just closed the ticket, but I’m trying to compile my project and visual studio and it seems like VI is deleting a .dll file for my project, but then it can’t recreate it. Is this a permissions issue?

Yeah sorry about that. And sorry for the delay. I was trying to recreate the error, but I’m running some other processes and I don’t want to interfere with them.

I get error M(SB3073 or something like that) and then it gives me a path with -waitmutex- exited with exit code -1 (sometimes it’s code 3 or 5)

and

ERROR: UBT ERROR: Failed to produce item: C:\Users\georg\Documents\Unreal Projects\Prosim\Binaries\Win64\UE4Editor-Prosim.dll

Here’s the exact error

C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.MakeFile.Targets(46,5): error MSB3073: The command ““C:\Program Files (x86)\Epic Games\4.13\Engine\Build\BatchFiles\Rebuild.bat” ProLangBPPCEditor Win64 Development “C:\Users\georg\Documents\Unreal Projects\ProLangBPPC\ProLangBPPC.uproject” -waitmutex” exited with code -1

Code 5 is usually because you are doing something with a macro that isn’t correct, such as

UPROPERTY( Server, Reliable, WithValidation )

This isn’t valid because UPROPERTY( ) doesn’t take in UFUNCTION( ) parameters.

It could also be something like using a type that you no longer are using in your project but the generated headers still have them compiled, so it looks like they are valid types but really aren’t, causing a compiling issue.

I seemed to have fix the MSB3073 error, but I’m still getting the UBT Error. Do you know what causes this?

I was able to figure it out. You were right! I had a function that was being called that I removed. Sorry to waste your time! Thank you for the help!