After hours on stack overflow I think I’ve finally found a solution to my issues parsing the text file. I just need help getting the function to work in UE4. I have the .cpp, but I need help creating the header and stuff.
Here’s the code:
const char* map_file(const char* fname, size_t& length);
int main()
{
size_t length;
auto f = map_file("english3.txt", length);
auto l = f + length;
uintmax_t m_numLines = 0;
while (f && f != l)
if ((f = static_cast<const char*>(memchr(f, '\n', l - f))))
m_numLines++, f++;
std::cout << "m_numLines = " << m_numLines << "\n";
}
void handle_error(const char* msg) {
perror(msg);
exit(255);
}
const char* map_file(const char* fname, size_t& length)
{
int fd = open(fname, O_RDONLY);
if (fd == -1)
handle_error("open");
// obtain file size
struct stat sb;
if (fstat(fd, &sb) == -1)
handle_error("fstat");
length = sb.st_size;
const char* addr = static_cast<const char*>(mmap(NULL, length, PROT_READ, MAP_PRIVATE, fd, 0u));
if (addr == MAP_FAILED)
handle_error("mmap");
// TODO close fd at some point in time, call munmap(...)
return addr;
}
}
I could also use some help finding a better way to define the text file to parse. I feel like the way UE4 packages things might mess with how the function works.
Thank you so much to everyone who’s helped me get this far. I won’t forget your contributions!
What does your question has to do with blueprints? Would you mind updating the title and tags to not mislead people?
Regarding the code, don’t. You’ll get yourself into lots of trouble if you put it in UE4 context (compilation, runtime and cook-time). UE4 has code for everything you’re trying to do in that code snippet. Do yourself a favor and dig through UE4 code and some samples on the internet and familiarize yourself with the related code. You can start by looking at FFileHelper.
I forgot to add that I wanted to call this C++ to interact with blueprints. I need to call a pure function and have a string array output node, much like 's LoadStringArrayToFile function which is why I added the blueprints tag.
I didn’t really ask my question properly…its been a long day so I apologize…
I basically want to create a function nearly identical to 's, but something that can parse through whitespace more efficiently.
I spent a lot of time looking through CoreMisc.h to see if there was anything I could use to make the parsing go faster, but I couldn’t seem to find anything that worked.
My main problem is it takes around a day to parse a 2MB text file using the functions in CoreMisc.h.
Using the code I have takes about 60 seconds…
Would you know of a way to utilize CoreMisc.h in a way that would parse the files faster?
Thanks for taking the time to answer…I’ve been driving myself into the ground trying to figure this out. xD
My main problem is it takes around a day to parse a 2MB text file using the functions in CoreMisc.h.
I find it hard (impossible!) to believe. Would you mind sharing a piece of code that was resulting in such a horrible performance? You can compile and cook all of our internal projects in a day, UE4 is pretty efficient at low level operations
I agree! I couldn’t really believe it either because I’ve never really had issues with any lighting computations or anything like that, and if I did it could definitely be narrowed down to hardware limitations rather than the engine itself.
Here it is though:
.h
/** Loads a text file from hard disk and parses it into a String array, where each entry in the string array is 1 line from the text file. Option to exclude lines that are only whitespace characters or '\n'. Returns the size of the final String Array that was created. Returns false if the file could be loaded from hard disk. */
UFUNCTION(BlueprintPure, Category = "VictoryBPLibrary")
static bool LoadStringArrayFromFile(TArray<FString>& StringArray, int32& ArraySize, FString FullFilePath = "Enter Full File Path", bool ExcludeEmptyLines = false);
I ran out of room on the last comment so here are some details you might need.
The code provided is from 's Victory BP Library so I can’t really provide too many details on what he was trying to accomplish with it, but I thought it looked fine. I scrapped my last attempt at trying to parse the file because I wanted to start fresh or I could include that too.
There was also some code provided by Langley, but I haven’t really been able to get that to do what I want either I need it to return a string array like 's.
.h
#pragma once
#include "GameFramework/Actor.h"
#include "TextLoader.generated.h"
UCLASS()
class AH509306_API ATextLoader : public AActor
{
GENERATED_BODY()
public:
ATextLoader();
UFUNCTION( BlueprintCallable, Category = "Test" )
void LoadTextFile( );
};
I tried to make this function into a static bool, but I couldn’t get it functioning properly.
I don’t know if it matters but I have a Quad Core Core i7 with 3.0 GHZ and 32 GB of RAM with a nVIDIA GTX 780M. So my build is a bit old, but I don’t think it’s THAT old. What do you think?
I’ve been seeing a lot of interesting things in CoreMisc.h, but I don’t really have the knowledge to effectively pull snippets out and get them to work correctly in Visual Studio.
If there’s any more information I can provide let me know! =]
So I saw that your function was a void function, but I needed it to return the string value so I saw that was using a static bool.
I figured since your code parsed faster than 's did, I could mess around with it a bit to get it to work as a static bool and return a string array value. I also wanted it to be a pure function instead of a callable function.
I don’t have the code anymore because I’m starting to think it’s a hardware issue. Even with bad code (whether it’s in my blueprints or in my C++) and even if 's node isn’t designed to work the way I want it to; I can’t see how it would take over a day to parse a 2MB text file.
My hardware is getting old anyway so I might just upgrade and see if that fixes my issues.
That works like a charm! 2 million characters in around a minute and a half!
My only question now would be how to expose the FString as an array instead of the normal output pins.
If that’s not possible I can mess around with some other stuff in BP to see if I could get something to work otherwise, but I basically want to run a for each loop off of the loaded file array to do some other stuff.
I want to separate each word and run a for loop off of each word. I’m essentially trying to make a random word generator based off of the English dictionary.
I tried using the explode string and parse into array nodes and delimiting off of the space to run the for loop, but it’s taking all 200k words and looping it 200k times. It’s extremely fast (thanks to your code) but it is a lot of data that doesn’t need to be there.