Reading text files stops working when we package

We recently implemented a system to load some data from custom text files – see the thread

Unfortunately, it turns out there’s a problem with this approach. When you package up your project to run the game as a standalone executable, it saves everything into a big .pak file, and this seems to stop it from working.

Dear Epic: How to deal with this?

Should we be NOT be packaging them into the .pak file, and is there a way to specifically omit specific files, or files by type, during the packing process?

Alternatively, is there a way to modify this so that FFileHelper::LoadFileToString() works properly on files inside .pak files?

We use IFileManager to load non-asset Slate brushes and fonts from disk, even during a pak’d build, so this should work.

I’m wondering if they’re not being packaged at all; things not referenced by an asset typically aren’t, as we have some special code in place to explicitly package Content/Slate. This is inside the CreateStagingManifest function in CopyBuildToStagingDirectory.Automation.cs.

Where are you files going on disk? I assume somewhere inside FPaths::GameContentDir?

Yes, I’m sure that’s the problem; they aren’t getting packaged.

My files are special *.UIX files in the root of the Content folder.

Is there an easy way to tell the engine to package all the *.UIX files in that folder?

Unfortunately, having investigated this with MattK, it seems there’s not currently a way to do this.

We were hoping you might be able to use the “Directories to Always Cook” feature in the editor to force your files to always be added, however that won’t work as cooking expects .uasset files, and ignores anything else.

What we’d like to see added is a way to specify a folder that should always be staged (either into the pak file for things we have IO control over, or as individual files for things we don’t - like things using 3rd party IO that we can’t hook in to).

I’m currently sorting out the details for this between the editor and core teams, so I’ll keep you posted with updates.

My horrible suggestion for now would be to put your .UIX files into their own folder (you’ll likely have to do this regardless of what solution we ultimately come up with to avoid having to scan the whole Content directory), and then hack the CopyBuildToStagingDirectory.Automation.cs file to always stage that folder (like it does with the Slate folders).

It might look like this:

SC.StageFiles(StagedFileType.UFS, CombinePaths(SC.ProjectRoot, "Content/UIX"), "*", true, null, CombinePaths(SC.RelativeProjectRootForStage, "Content/UIX"), true, !Params.Pak);

Thanks, Jamie. Yep, it’s an ugly workaround, but if you guys can work on a fix for an upcoming update I’m more than happy to do this for now!

Hey – I have an idea.

Why don’t you guys add a folder called “CustomPackagedContent” or something like that inside the Content folder, which would be empty by default.

Then, tweak the engine so anything in that folder (or any subfolders) gets packaged.

That way, users could add custom packaged content just by adding it inside that folder.

That would work, however this problem has been a long time coming.

A small update for you: We should now be able to load INI files into UAT (which is what handles the staging), so all being well, tomorrow I’ll be taking a look at getting this set-up so that you can add new directories to stage via the game settings (DefaultGame.ini) for your project (this file is editable in the editor UI).

Awesome!!!

Okay, it’s done (and in master), but it won’t be in an official release build until 4.3.

If you’re handling your own builds, the following changes should hopefully apply and work correctly over the top of a 4.2 checkout:

You’ll now see some advanced options under packaging, and these will let you stage additional files into the package.

Fantastic! You guys kick ***!