Additional Asset Directories not Copied to Packaged Build

My Info:

  • Windows 8.1 64-bit
  • Visual Studio 2013 Update 4
  • Unreal Engine 4.8 (binary from launcher)
  • Project was created in 4.7 and updated to 4.8
  • [DefaultGame.ini][1]

I am not able to get the Additional Asset Directories to Cook feature working in my project. Here is how I am validating this result.

Add an asset directory that I want to cook.

47117-1.jpg

This asset directory contains textures that I reference in C++:

They are initialized in my constructor via a static like so (all works fine in PIE):

struct FConstructorStatics
{
	ConstructorHelpers::FObjectFinderOptional<UTexture2D> T_Face_Button_Left;
	ConstructorHelpers::FObjectFinderOptional<UTexture2D> T_Face_Button_Right;
	ConstructorHelpers::FObjectFinderOptional<UTexture2D> T_Face_Button_Top;
	ConstructorHelpers::FObjectFinderOptional<UTexture2D> T_Face_Button_Bottom;
	ConstructorHelpers::FObjectFinderOptional<UTexture2D> T_Left_Shoulder;
	ConstructorHelpers::FObjectFinderOptional<UTexture2D> T_Left_Trigger;
	ConstructorHelpers::FObjectFinderOptional<UTexture2D> T_Right_Shoulder;
	ConstructorHelpers::FObjectFinderOptional<UTexture2D> T_Right_Trigger;
	ConstructorHelpers::FObjectFinderOptional<UTexture2D> T_DPAD;
	ConstructorHelpers::FObjectFinderOptional<UTexture2D> T_Left_Analog;
	ConstructorHelpers::FObjectFinderOptional<UTexture2D> T_Right_Analog;
	ConstructorHelpers::FObjectFinderOptional<UTexture2D> T_Start;
	ConstructorHelpers::FObjectFinderOptional<UTexture2D> T_Special;

#if IsPS4
	FConstructorStatics()
		: T_Face_Button_Left(TEXT("/Game/UI/GamepadIcons/PS4/PS4_Square")),
		T_Face_Button_Top(TEXT("/Game/UI/GamepadIcons/PS4/PS4_Triangle")),
		T_Face_Button_Right(TEXT("/Game/UI/GamepadIcons/PS4/PS4_Circle")),
		T_Face_Button_Bottom(TEXT("/Game/UI/GamepadIcons/PS4/PS4_Cross")),
		T_Left_Shoulder(TEXT("/Game/UI/GamepadIcons/PS4/PS4_L1")),
		T_Left_Trigger(TEXT("/Game/UI/GamepadIcons/PS4/PS4_L2")),
		T_Right_Shoulder(TEXT("/Game/UI/GamepadIcons/PS4/PS4_R1")),
		T_Right_Trigger(TEXT("/Game/UI/GamepadIcons/PS4/PS4_R2")),
		T_DPAD(TEXT("/Game/UI/GamepadIcons/PS4/PS4_Dpad")),
		T_Left_Analog(TEXT("/Game/UI/GamepadIcons/PS4/PS4_Left_Stick")),
		T_Right_Analog(TEXT("/Game/UI/GamepadIcons/PS4/PS4_Right_Stick")),
		T_Start(TEXT("/Game/UI/GamepadIcons/PS4/PS4_Options")),
		T_Special(TEXT("/Game/UI/GamepadIcons/PS4/PS4_Share"))
	{
	}
#elif IsXboxOne
	FConstructorStatics()
		: T_Face_Button_Left(TEXT("/Game/UI/GamepadIcons/XboxOne/XboxOne_X")),
		T_Face_Button_Top(TEXT("/Game/UI/GamepadIcons/XboxOne/XboxOne_Y")),
		T_Face_Button_Right(TEXT("/Game/UI/GamepadIcons/XboxOne/XboxOne_B")),
		T_Face_Button_Bottom(TEXT("/Game/UI/GamepadIcons/XboxOne/XboxOne_A")),
		T_Left_Shoulder(TEXT("/Game/UI/GamepadIcons/XboxOne/XboxOne_LB")),
		T_Left_Trigger(TEXT("/Game/UI/GamepadIcons/XboxOne/XboxOne_LT")),
		T_Right_Shoulder(TEXT("/Game/UI/GamepadIcons/XboxOne/XboxOne_RB")),
		T_Right_Trigger(TEXT("/Game/UI/GamepadIcons/XboxOne/XboxOne_RT")),
		T_DPAD(TEXT("/Game/UI/GamepadIcons/XboxOne/XboxOne_Dpad")),
		T_Left_Analog(TEXT("/Game/UI/GamepadIcons/XboxOne/XboxOne_Left_Stick")),
		T_Right_Analog(TEXT("/Game/UI/GamepadIcons/XboxOne/XboxOne_Right_Stick")),
		T_Start(TEXT("/Game/UI/GamepadIcons/XboxOne/XboxOne_Menu")),
		T_Special(TEXT("/Game/UI/GamepadIcons/XboxOne/XboxOne_Windows"))
	{
	}
#endif

};

After packaging the build, I expect this folder to exist in the final package folder, it does not.

47119-3.jpg

Am I misinterpreting the asset directory parameter? How do I get these additional assets into my packaged build?

Things Iā€™ve tried:

  1. Creating a User Widget that uses these textures as images. I then create a reference to that User Widget in my main HUD but again there is no change in packaging.
  2. Copying the .uasset files manually to the packaged folder, this results in a [crash in GarbageCollection.cpp][5]. Iā€™m unsure if the assets are problematic or something else.
  3. Creating a .uasset file in the GamepadIcons folder, thinking that perhaps an empty folder was preventing the inner XboxOne folder from being scanned.
  4. Using the Cook Everything in the project content folder option
  5. Every variant of 32/64-bit development/shipping/DebugGame
  6. Packaging in editor
  7. Packaging via UnrealFrontend

Here is the [log output from packaging in editor][6]
None of these attempts resulted in any change in output. Please tell me what Iā€™m doing wrong here, Iā€™m sure itā€™s something fundamental or alternatively something is fundamentally broken.

1 Like

Iā€™ve added an additional firehose worth of extra details. This issue is killing my studio.

I donā€™t think these are related. My issue has to do with content not being copied.

I am not sure this is related. I just tried 4.8 today. I cannot compile a default scene in 4.8 without fatal errors. When I do a build it is blowing away a folder it ā€œappearsā€ to be looking for. Here is the link to my thread.

Another note. After packaging completes, if I look in Saved\Cooked\WindowsNoEditor, I see the missing content there. So it seems that the packaging step is where things are failing.

I canā€™t seem to get traction on this bug. Here is the smallest C++ project I could make that replicates the issue.

Manually adding missing content via copy/paste works for windows but not for mobile.

Iā€™m with you, packaging is broken in UE4. Itā€™s fine for anything thatā€™s referenced; anything else isnā€™t included.

You can see that simply by looking at the command line the editor passes to UAT (it starts with BuildCookRun) when you start packaging your project. It wonā€™t pass along any extra directories specified in the .ini file.

The only workaround I have is to have a build script that removes content I donā€™t want packaged and then to specify that I want it to package all content (which uses the argument -CookAll). Iā€™ve looked through the parameters supported by UAT and I canā€™t see one that would cook and/or package other content.

Please upvote the question if you can. I think packaging C++ projects falls into the < 1% of users bin, so Iā€™m guessing this isnā€™t even on the radar. Even a recommended workaround of how to make sure a texture asset is referenced would be ok with me.

Your workaround seems doable, but very heavy-handed and difficult to maintain.

Oh, itā€™s ridiculous to do it that way, I agree. In our case, though, it takes our packaged builds from 2.75GB to 1.3GB - so itā€™s worth maintaining for now. But I wouldnā€™t have to do that if I could specify the maps to be packaged properly using the editor.

The wiki (A new, community-hosted Unreal Engine Wiki - Announcements - Unreal Engine Forums) suggests that I can add maps under the [AllMaps] section of DefaultEditor.ini. But then why does the editor add MapsToCook under [/Script/UnrealEd.ProjectPackagingSettings]? It seems like all of the array options under the Packaging UI in the editor are just thrown away when packaging - only the binary on/off options are used.

I actually donā€™t believe this problem is specific to C++ projects, itā€™s specific to any project that doesnā€™t want to package everythingā€¦ or one that wants to package things that arenā€™t directly referenced by maps!

I guess I could look into fixing it myself since Iā€™ve already done a lot of investigating - the problem is finding time with deadlines looming! Iā€™m sure Epic has the same issue too - itā€™s just frustrating because in all other ways the packaging process is very good!

yep I agree, all of the array options seem to be no-ops. Iā€™m not sure how to create unreferenced content using only blueprints, but if there were an easy repro it might help the cause.

I know what you mean about fixing it yourself. Itā€™s on my todo list to look at the packaging code, if you ever want to collaborate on this, maybe we can share notes! I havenā€™t yet upgraded to 4.9 by the way, so unsure of changes there.

Iā€™m on 4.9 - it still doesnā€™t work.

As for working on it together - I can tell you that we have to start with Engine\Source\Programs\AutomationTool\Scripts\BuildCookRun.Automation.cs - all the magic happens inside DoBuildCookRun(). Iā€™ll definitely get around to going further down the rabbit hole, I just donā€™t have any time this week. :frowning:

Actually, I went further down the rabbit hole. It looks like the problem might be inside Engine\Source\Editor\UnrealEd\Private\Commandlets\CookCommandlet.cpp. In 4.9 at least, the line in question starts at 1146.

Iā€™ll let you know what I find out!

Iā€™m going to pull down master tonight and dig around.

I donā€™t have time to continue looking today but what Iā€™ve found is that where the settings are being ignored starts in SetupParams() in BuildCookRun.Automation.cs. It creates a new ProjectParams object in there - thatā€™s the object that needs to contain all of the correct data.

In theory passing the ā€œcookdirā€ parameter with directories delimited by the ā€˜+ā€™ character should solve your problem! So far that seems to be specified in CookCommandlet.cpp and CommandletUtils.cs - so either theyā€™re not working properly or it needs to be specified somewhere else too!

Hopefully that gives you something to start working with!

I spent a good 2 hours familiarizing myself with the code. Iā€™m currently investigating the CreateStagingManifest() function in CopyBuildToStagingDirectory.Automation.cs and determining why it isnā€™t honoring the UFS and Non-UFS parameters in the packaging settings.

edit: Top editing because I found something useful.

Someone came across this. Looks like unreal doesnā€™t like to package directories with ā€œxboxā€ or ā€œps4ā€ in their names. Maybe try changing the directory name and see if it packages?

end edit:

I believe all the asset directories that get cooked get compiled into an intermediate file format that lives under Your_Cooked_Project_Path\Content\Paks as .pak files and the directory structure is not preserved. If you want the directory structure and all files copied across it should go under ā€œAdditional Non-Asset Directories to Copyā€ or Cook depending on what you want.

The reason it probably doesnā€™t work in the cooked build is probably because your paths in your constructor arenā€™t asset references, which should look like this: ā€œTexture2Dā€™/Game/Textures/Light01.Light01ā€™ā€

I think this is more that you took a really roundabout way of loading your assets that happened to work in the editor by some random quirk of the FObjectFinder. You can copy and paste the actual asset paths by right clicking the asset in the content browser and ā€œCopy Referenceā€ then paste that in for your strings. Try that and see what happens.

edit: Hrm. I think the first bit I said is because we have ā€œCreate compressed cooked packagesā€ ticked. hrmrmrm. Iā€™ll poke around more, but weā€™ve had no issues using ā€œAdditional Asset Directories to Cookā€.

Well, I finally found the issue. It wasnā€™t a bug, just an unexpected feature of the staging system.

In CopyBuildToStagingDirectory.Automation.cs, a function is called to copy UFS and NonUFS folders to the staging folder (SC.StageFiles(...)):

The final argument passed to the function is a bool called bStripFilesForOtherPlatforms that is determined using:

!Params.UsePak(SC.StageTargetPlatform)

So basically if you arenā€™t using a pakfile, bStripFilesForOtherPlatforms will be true and youā€™re in danger of getting the same behavior I describe.

public bool UsePak(Platform PlatformToCheck)
{
    return Pak || PlatformToCheck.RequiresPak(this) == Platform.PakType.Always;
}

When this boolean is set, thereā€™s some code in DeploymentContext.cs::StageFiles that looks at the full path of the content you are including. If this content contains a folder whose name matches any platforms that you are not building for then the content gets ignored - as in not copied.

So for instance trying to include content named /Game/Content/UI/XboxOne/button.uasset when building for Win32 will get ignored because XboxOne is a platform name. If the folder name was XboxOne1 or MyXboxOne it would have been ok since the match string is ā€œ/Platform/ā€.

I looked through the docs and couldnā€™t find any mention of platform specific folder names as part of the packaging process! Kind of a huge deal here I think.

For future generations, strategically use platform names in your build folders:
Win32
Win64
WinRT
WinRT_ARM
UWP
Mac
XboxOne
PS4
IOS
Android
HTML5
Linux
AllDesktop

In short, renaming my folder from XboxOne to Xbone fixed the problem.

Iā€™ve been having this same issue for the past two months. Thank you SO MUCH for getting to the bottom of it and documenting it in this thread.

Thanks for this answer, it helped me as well!
For anyone interested, this bug (anything which acts in an unexpected way, and is undocumented, is a bug in my eyes), still exists in 4.11.2