How to reduce HTML5 package size?

trutty · December 8, 2014, 7:39pm

I am using UE 4.6 and developing a HTML5 game. I want the final package size to be as small as possible but right now, it seams that a lot of unnecessary stuff gets packed into it.

I work with Blueprints. The size of the content folder, including all assets, is 18.5 MB and the resulting HTML5 package is about 74 MB

I researched some possibilities to minimise the size. Here is what I did:

project has no starter content
used assets do not reference to assets in the engine folder (except for MatineeCam) (checked with the Reference Viewer on my map)
packaging settings are set to “Full Rebuild”, “Full Distribution” and “Shipping”
excluded " /Engine/MobileResources" from content directories
disabled Slate
disabled plugins in the editor and checked in the uproject file:

“Plugins”: [
{
“Name”: “AndroidDeviceProfileSelector”,
“Enabled”: false
},
{
“Name”: “AppleMoviePlayer”,
“Enabled”: false
},
{
“Name”: “AvfMedia”,
“Enabled”: false
},
{
“Name”: “BlankPlugin”,
“Enabled”: false
},
{
“Name”: “CableComponent”,
“Enabled”: false
},
{
“Name”: “CharacterAI”,
“Enabled”: false
},
{
“Name”: “CustomMeshComponent”,
“Enabled”: false
},
{
“Name”: “EpicSurvey”,
“Enabled”: false
},
{
“Name”: “ExampleDeviceProfileSelector”,
“Enabled”: false
},
{
“Name”: “IOSDeviceProfileSelector”,
“Enabled”: false
},
{
“Name”: “MacGraphicsSwitching”,
“Enabled”: false
},
{
“Name”: “MediaPlayerEditor”,
“Enabled”: false
},
{
“Name”: “MessagingDebugger”,
“Enabled”: false
},
{
“Name”: “OculusRift”,
“Enabled”: false
},
{
“Name”: “PerforceSourceControl”,
“Enabled”: false
},
{
“Name”: “ScriptGeneratorPlugin”,
“Enabled”: false
},
{
“Name”: “ScriptPlugin”,
“Enabled”: false
},
{
“Name”: “SlateRemote”,
“Enabled”: false
},
{
“Name”: “SpeedTreeImporter”,
“Enabled”: false
},
{
“Name”: “UdpMessaging”,
“Enabled”: false
},
{
“Name”: “UObjectPlugin”,
“Enabled”: false
},
{
“Name”: “WmfMedia”,
“Enabled”: false
},
{
“Name”: “WindowsMoviePlayer”,
“Enabled”: false
},
{
“Name”: “XCodeSourceCodeAccess”,
“Enabled”: false
},
{
“Name”: “SubversionSourceControl”,
“Enabled”: false
}
modified .target.cs

public override void SetupGlobalEnvironment(
TargetInfo Target,
ref LinkEnvironmentConfiguration OutLinkEnvironmentConfiguration,
ref CPPEnvironmentConfiguration OutCPPEnvironmentConfiguration
)
{
UEBuildConfiguration.bCompileFreeType = false;
UEBuildConfiguration.bCompilePhysXVehicle = false;
UEBuildConfiguration.bCompileSpeedTree = false;
UEBuildConfiguration.bCompileRecast = false;
UEBuildConfiguration.bCompileAPEX = false;
UEBuildConfiguration.bCompileLeanAndMeanUE = true;
UEBuildConfiguration.bCompileICU = true;
UEBuildConfiguration.bCompilePhysX = false;
UEBuildConfiguration.bBuildEditor = false;
UEBuildConfiguration.bBuildWithEditorOnlyData = false;
UEBuildConfiguration.bBuildDeveloperTools = false;
UEBuildConfiguration.bCompileBox2D = false;
UEBuildConfiguration.bCompileSimplygon = false;
UEBuildConfiguration.bCompileSteamOSS = false;
UEBuildConfiguration.bCompileMcpOSS = false;
UEBuildConfiguration.bWithServerCode = false;
UEBuildConfiguration.bCompileWithPluginSupport = false;
// this seams to not affect the package size; only works for C++ code?
UEBuildConfiguration.bCompileForSize = true;
}

The thing is, that even though I disabled all plugins etc. the resulting .data.js file still references these assets, which I am definitively not using. Even the standard “Entry.umap” is listed but this map doesn’t even exist. Here is a small excerpt of these unwanted files:

The TappyChicken demos’ .data.js loads way less assets than mine.
What can I do to get rid of unnecessary stuff? Maybe I am just missing a detail. Any comment will help! Thanks

Update

I am now using UE 4.6.1, started a completely fresh project and implemented everything in C++. Unfortunately this changed nothing with the package size. I did a full memreport ([Debugging and Optimizing Memory][4]) but had no luck in optimizing. I really hope this will get better.

anonymous_user_83949151 · December 9, 2014, 10:22am

I have the same problem. You have more options to packaging project with UnrealFrontend, but I have not been able to improve the result.
I continue investigating for this problem, if i have got lucky, I’ll publish my results in this post.

piXelicidio · December 18, 2014, 10:19pm

I think I’m collecting (bookmarking) this kind of package-size questions that are never answered. Is very sad. The only thing that could take me away from making mobile games with UE4 is the ridiculous big file size.

Moss · January 14, 2015, 5:26pm

To shrink the size of a HTML5 deployment I recommend you to pre-gzip the content, I was able to get the final size to about 15 MB for the engine and the game for an HTML project (just select the big blue button https://ikea.pr/puertorico/desktop/es_pr/catalogos/categoria/escenario/detalle/apartamento-de-verano_PR).

Once you have gzipped your content (apache example: Serving pre-compressed files using Apache) you have two options, always serve gzip or serving both and if the client is able to read gzip content serving the gzipped version. Deciding if the client can handle gzip is easy, just create a .js file, gzip it to .js.gzip (gzip with the -9 option which indicates best compression rates) and try to load it. If you add a variable in that file like var enable_gzip = true; and your client can load that file the enable_gzip will be defined, if not enable_gzip will be undefined and you can change the extension you request from the server.

From you file sizes the engine should go under 10 MB and the game around 6 MB.

trutty · January 14, 2015, 6:25pm

Thanks for your reply and the examples, Moss. I looked at your example and it turns out that I downloaded around 37 MB (CloudStore.data.jgz alone is already 27.5 MB large).
I already transfer my data compressed (forgot to mention that; currently using deflate but I am going to check out gzip), however I forgot to include the *.mem file in the compression filter which now saved me another 6.5 MB.

Afaik transferring the data compressed is only good for the downloading purpose but after all the users browser still has to load the full uncompressed files in the memory and needs to deal with them - my actual question is how to reduce the package size in the first place because there really is a lot of unnecessary stuff in the package.

Thanks anyway. This is helpful for others, too.

piXelicidio · January 14, 2015, 7:01pm

That’s the same problem I have with android non-source project. Is not about game content compression, is the whole engine is included in the package and the smallest .apk size I could achieve is 36MB with only 2MB of my actual game content.
If Epic wants to continuew advertising that UE4 produce games for android and HTML5 they need to do something to reduce engine package size. Also this solution has to be easy for us, because to reduce my package size from 54MB to 36MB i had to do a lot of tricks, some of them a bit dirty.

Moss · January 15, 2015, 8:24am

HTML5 is still experimental so it will get better for sure

Moss · January 15, 2015, 8:30am

Without pre-compression the browser was not able to even download the build and crashed with Out of Memory errors.

I guess the only way to get rid of all those plugins and additional is really using a source release and not compile the plugins. If you disable the plugins for the project from the editor are they getting included too?

Moss · January 15, 2015, 9:04am

You can either delete them from the Engine/Plugins folder (might be tricky). Or set them to be ‘Editor’ or ‘Developer’ plugins instead of ‘Runtime’ plugins in the .uplugin files.

I have not tried it but that would be the starting point.

trutty · January 15, 2015, 9:01am

I guess the only way to get rid of all those plugins and additional is really using a source release and not compile the plugins.

Do you know how I can achieve that?

If you disable the plugins for the project from the editor are they getting included too?

Yes, I uncheck the plugins in the Plugins Browser (Window → Plugins) and check if they are disabled in my uproject file, too. Still they get included.

anonymous_user_83949151 · January 21, 2015, 12:13pm

Very good post!! We reduced our project 40 MB with this compresion.
You must to active de mod_deflate module in Apache and add .htaccess with this code on the hosting site:

#LoadModule deflate_module modules/mod_deflate.so

<IfModule mod_deflate.c>

    # Activamos la compresión
    SetOutputFilter DEFLATE

    # Indicamos los tipos de contenido a comprimir
    AddOutputFilterByType DEFLATE text/plain
    AddOutputFilterByType DEFLATE text/html
    AddOutputFilterByType DEFLATE text/xml
    AddOutputFilterByType DEFLATE text/gml
    AddOutputFilterByType DEFLATE text/css
    AddOutputFilterByType DEFLATE application/json
    AddOutputFilterByType DEFLATE application/xml
    AddOutputFilterByType DEFLATE application/xhtml+xml
    AddOutputFilterByType DEFLATE application/rss+xml
    AddOutputFilterByType DEFLATE application/javascript
    AddOutputFilterByType DEFLATE application/x-javascript
    AddOutputFilterByType DEFLATE data


    # Indicamos las extensiones de los ficheros a comprimir
    <files *.html>
        SetOutputFilter DEFLATE
    </files>
    <files *.xml>
        SetOutputFilter DEFLATE
    </files>
    <files *.gml>
        SetOutputFilter DEFLATE
    </files>
    <files *.json>
        SetOutputFilter DEFLATE
    </files>
    <files *.js>
        SetOutputFilter DEFLATE
    </files>
    <files *.css>
        SetOutputFilter DEFLATE
    </files>
    <files *.data>
        SetOutputFilter DEFLATE
    </files>


</IfModule>

But we need more. Do you know if we can reduce the resolution, qualities or engine escalability settings to package in HTML5?
Do you know how can we compress de .DATA file?
Thanks!!

Moss · January 21, 2015, 12:31pm

You can add a HTML5 engine ini file and lower the TEXTURE group sizes for all. Just go into base engine ini file and you copy the TEXTUREGROUP settings over to your HTML5 engine ini file, then just lower the max lod size, I use 1024 or 512 and it’s still quite nice. You can also change the filters and tweak them a bit.

anonymous_user_2f922134 · April 23, 2015, 7:57pm

Hello there,

after looking into this for my own HTML5 project, I noticed that by default, the plugin configuration in the .uproject file is not allowed to disable default plugins. To fix this, edit Engine/Source/Programs/UnrealBuildTool/System/UProjectInfo.cs in GetEnabledPlugins and replace
List EnabledPlugins = new List(DefaultEnabledPlugins);
with
List EnabledPlugins = new List();
( for me, it’s line 313 )

Then recompile while specifying the uproject file, that is using
Engine/Build/BatchFiles/Mac/Build.sh UE4Game HTML5 Shipping /path/to/your.uproject
instead of
Engine/Build/BatchFiles/Mac/Build.sh UE4Game HTML5 Shipping

And voila, only the modules you enabled are compiled and linked into your HTML5 runtime

piXelicidio · April 23, 2015, 10:12pm

Hi, you think I can apply your solution to an android project?
Do i need to know anything about c++?

piXelicidio · April 24, 2015, 5:50pm

C# or C++? I can understand simple C++ code but I don’t know anything about visual studio. Do I need to install Visual Studio? and re-compile something or It will work with my blueprint only project? Thanks for your patience

jwatte · April 24, 2015, 5:48pm

This solution works for any platform. However, you need to know a small amount of C# to actually make that code edit, and re-run the build files.

Moss · May 2, 2015, 6:32pm

You will have to compile the engine from source. Here you have a complete tutorial on how to compile the engine from sources.

6r0m · December 7, 2016, 1:56pm

could you please update you answer for current engine? I can’t find this strokes.

Luiz_Sotero · August 3, 2017, 7:54pm

I’m having the same problem here, but the UProjectInfo file does not have the same code from this answer. Can anyone update the answer to reflect new changes?

SRombauts · November 14, 2017, 9:55am

Just a quick note since I was looking for MaxLODSize, it now has it’s default values (4096) in Engine/Config/BaseDeviceProfiles.ini

[/Script/Engine.TextureLODSettings] @TextureLODGroups=Group TextureLODGroups=(Group=TEXTUREGROUP_World,MinLODSize=1,MaxLODSize=4096,LODBias=0,MinMagFilter=aniso,MipFilter=point,MipGenSettings=TMGS_SimpleAverage)