Shipping Size is double the project size!

I was about to post this under “Packaging and Deployment” but I am pretty sure this has to be a bug of some sort. A Shipped blueprint project whose project folder amounts to 5gb is being shipped to 10gb (10gb pak file) I am not even sure how is this even possible. I would assume the engine optimizes the assets upon cooking, for example if I import a texture file at 40964096 but the “maximum in-game” is set do 20482048 I would assume the engine downsamples the texture to 2048*2048 upon shipping to save space. I would assume the engine maybe applies some sort of compression or strips some unnecessary information from the assets as well. I would like clarification on either this happens or not. (Edit: Clarification no longer needed, we’ve realized this happens, thanks Epic Devs)

But regardless of how optimized the shipping process is I don’t see how it can be double of what the assets are, to note this is a blueprint project (so no code) with just static meshes and their materials…and how is this possible that the engine is shipping the project to twice the size of the project folder? Even if the engine packs everything into the .pak file without any regard for optimization it would still never be more than 5gb for this project, so why is this happening? is the engine somehow duplicating every asset inside the .pak file?

Things can get out of hand pretty quickly this way, I don’t want to be distributing a 90gb game to my clients, is there something I can do or get clarification on this issue?

This is a x64 windows shipping build.

Hey ,

What kind of game is it and what kind of assets you working with?

It’s a simple project with only static meshes and materiais, there is no blueprint coding or c++ or anything else, just a map with a bunch of static meshes. (quite a few in fact). This is not the main game project but I would like to migrate this assets to the main project but I don’t want the shipping size to grow this rapidly.

Could you provide a screenshot? Hard to imagine what you’re going through based on what you’re describing. What’s the poly count of your meshes, how many you using what format are your textures in etc?

It’s all the meshes from “Crazy Insane Patio Set” from the marketplace.

40964096 but the “maximum in-game” is
set do 2048
2048 I would assume the
engine downsamples the texture to
2048*2048 upon shipping to save space.

I don’t think so. You have to resize the texture by yourself.

Since your scene is static… How big is the lightmap resolution of the meshes? How big is your Light data in general. Have you build light before shipping?
Have you triple checked your packaging settings?

Hey thanks for answering, when I mean “maximum in-game” I don’t mean I only use 2048*2048 in-game, I mean I set the maximum texture size in the texture properties in-editor, there is no way this setting can be overriden at runtime so I would assume the engine could/should make use of it, specially cause it even reduces the “resource size” label in editor. I might make some tests later on to verify this. Nontheless even without downsampling I don’t see how the .pak file can have twice the size of my project folder.

In answer to your other question, I’m not using static lightning at all, it’s all fully dynamic so I have no lightmaps.

Yeah I checked the packaging settings…one thing that solves it is to enable .pak compression, reducing the .pak file size from 10gb to 2.5gb…but then again so does compressing the shipped build in a .rar/. file :stuck_out_tongue: The disavantage of using .pak compression is well…the .pak is now compressed. The fact a . file is able to compress texture information to less than half I think proves the engine in packing redudant data into the .pak file.

Hi Again, I’m not talking about MaxLODSize, it says here:

Under LOD Bias:
The number of miplevels to drop before uploading the Texture. This value can be negative to compensate for the LODGroup’s LOD bias for a particular Texture. The LODBias, the LODGroup’s LOD bias and the NumCinematicMipLevels are all added together to form a final LOD bias value for the Texture. Mip-levels that cannot be reached in-game because of its LOD bias are removed from the cooked data on disk by Unreal Frontend to save disk space.

I guess this confirms the engine does indeed scrape off unnecessary data.

As for compression…my problem is not that its not compressing enough, my problem is that the .pak file is duplicating the size of the avaliable uncompressed content :confused:

It would increase distributable
package size with no rendering quality
benefit if a game shipped with 2048
textures assigned to a TextureGroup
with a MaxLODSize of 1024

i cannot say anything about the .pak compression, but it wont be the best. will be a little better, because .pak was build to grab data fast in the first place, is slower, but better compression.

However… it’s very extreme in your case, so maybe. yeah it’s a bug.

Oh yeah. I created a blank project with two used 2048x2048 Textures, and used LOD-Bias 0 and 4, and packaged two projects.

The ressource size difference shown in the Texture-Editor is for both textures together -10mb, however, the packaged project (no other settings changed) gets away with only -2mb.

LOD Bias 0 : 192MB
LOD Bias 4: 190MB (expected ~182 =( )

So you are right, it will have an impact to package size. Thanks for that!

No problem :slight_smile: One application of this for us game devs is that we can import oversized textures and reduce the lodbias, and as the hardware evolves overtime we can re-ship with a new lod bias to keep up with the “current gen” without much work :slight_smile:

Hello DavidNSilva,

When you mention that the size is being doubled, what exact files/folders (please include file paths, as there can be certain folders with the same names in a project directory) are you comparing? Have you tried reproducing this in a fresh project using a few of these assets? If not, please do.

I’m aware this doesn’t justify the size being increased, but I would concur with TankbusterGames’ comment about the 40964096 texture not becoming smaller based off a setting that downsamples the texture, as that 40964096 texture will still be packed as is and the downsampling would come later.

Can you also include the log file for your packaging process? The information here may be able to help with looking into what is going on here.

Hi Matthew, What we were talking about was the LOD Bias and it does indeed reduce the .pak file size, it says so in the documentation also TankbusterGames made an experiment with it.

I’ll try to clarifify what I mean by “Duplicating”: My project folder has 5gb (Checked in windows by going to properties…what I mean is…it’s not an estimate and I am not talking about the “Content” folder only, I mean the entireproject folder that contains the .uproject file). I ship the project, the resulting build has a .pak file of 10gb. It would be very difficult for me to try with a few assets only as the difference might not be as noticable.

I was speaking about TankbusterGames’ first comment that quoted your mention of the 4096x4096 textures, I was not referencing the LOD bias conversation.

Trying to reproduce this in a fresh project with the assets that are already giving you this issue would be much easier than me trying to reproduce it with completely different assets. Please take the time to do so.

Another thing that may help diagnose the reason for this increase in size is to try packaging without the “Use pak file” option selected. This will just give you all of the assets uncompressed. This could let you see what assets are being duplicated or added that are increasing the size, if that is what is happening.

Oh sorry about that, I thought you were talking about the LOD bias.
The assets that I am using are the “Crazy Insane Patio Set” from the marketplace. I actually have tried what you said before, packaging without the “Use pak file”, I got folders and folders full of .uassets files but I am not really sure were to go from there, there are really a lot of assets in there.
I am on a fresh project…heres what I did:

I downloaded the “Crazy Insane Patio Set” as mentioned and added the content from the launcher to an empty 4.15 blueprint-only project.
I ticked “force no-precomputed lightning” in the provided map/level to make sure lightmaps were not in play.
I also disabled “Allow static lightning” in project setting to improve shader performance.
I deleted all SM_PatioChair assets (if you try to ship any of them it will fail cooking, something about convex collision and physX).
I checked the project folder size (about 3.5gb now without the chairs and stuff)
I shipped everything else (So mostly tables). the .pak file is now 9gb.

I know I said 5gb-10gb, but that was before I took all lightmaps.

Is this of use to you? You want me to try to ship with how many assets? if I use only 2 or 3 assets how do I know if the .pak file is acceptable or not?

Oh that’s what you meant by another project. I don’t have this problem with any of my other projects, only with this one specifically, but something must be wrong right? the engine shouldn’t be doing this under any circunstance, I think it might be related to physx collision cooking cause that used to spam the log until I deleted all chairs. Do you want me to try with the Infinity Blade packs regardless? here’s the screenshot

One of the main reasons I want you to try this with another project is because we need to determine if is possibly something specific to this marketplace pack. To give an example of why this seems suspect: The content folder for ShooterGame (one of our samples from the Learn tab) is 1.02 GB while the pak file, which also includes light maps, is only 977 MB. This also means it could be caused by some setting that you have enabled. Due to that, could you provide a screenshot of your packaging settings?

While this documentation is focused on APK sizes and packaging for Android, a good amount of this information can still be useful for packaging for other platforms. See if any of the tips here can help out with the size.

Could you test with other assets even if it’s one of our free asset packs, such as the Infinity Blade packs on the Marketplace?

I’m going to continue looking into this and come back with a response but one thing I wanted to clear up first is that, just because this is happening doesn’t mean that it’s a bug with the engine. The fact that it is only happening in this project actually makes it less likely to be a bug and a settings or content issue instead.

Well I hardly changed any setting as you can see, I think it’s a content related also, like something in the content is triggering this abnormal behavior, I find it very strange the .pak file occupies twice as much the .uassets it should have inside. I’ll be happy to help in any way I can. Thank you for looking into this issue. I’d also like to add that enabling the “create compressed cooked packages” “solves” the issue…but that’s not very ideal because it compresses the content that shouldn’t need any compression.

I did that once, the files did not appear to be literally duplicated but there are a lot of them. I can do that again just need to know how do I know the original size of an asset so I can compare them? as for “create compressed cooked packages” I assume it should affect texture streaming performance etc do you think the difference would be negligible? The engine is so fast I don’t notice any loss on my side, but client’s specs might be lower than the machines I have available for testing.