There are two different major operations being performed while the text “Compiling” is spinning on the screen: JS page compilation, and UE4 engine startup.
The JS page compilation is the process where the JS code containing the engine and the game is added to the web page DOM, and the browser parses and compiles the JavaScript code that was added. In Firefox (and also Edge), you can see how much time this took by opening the page console (Menu → Developer → Web Console), and looking for the line
"Successfully compiled asm.js code (total compilation time 6805ms; stored in cache)"
at the top of the log. In Chrome, there is unfortunately no such information posted by the browser. In Firefox and Edge, this asm.js compilation step is done only once, and subsequent page loads will load the UE4 engine already compiled from browser’s cache. So the 6805ms reported above occurs only on the first “cold” load time.
The second operation is that after the JS compilation, the UE4 engine starts up and loads all assets and shaders and the scene. This process starts up when the text “Running…” appears on the console window shown on the page. The time spent in this step is not logged, so you’ll need to manually time it e.g. by instrumenting the HTML file.
Investigating which one of the two operations is the slow one can give a clue how to mitigate.
If the slowdown looks like it is the JS page compilation, it might be useful to profile the first time load vs subsequent loads. The difference in asm.js compile times in Firefox stable vs Nightly might be due to this bug: 1246132 - asm.js compilation takes a very long time on AdVenture Capitalist Facebook asm.js game. The fix for that went in to Firefox 47 release channel, which is due to be out on 2016-06-07 (Firefox Release Calendar - MozillaWiki). Another difference might be if you were testing on 32-bit stable Firefox vs 64-bit Nightly Firefox?
Tried on 32-bit stable Firefox 46.0.1 vs 64-bit Firefox Nightly 49.0a1 (2016-05-15), where I do get 7 seconds as asm.js compile times on both browsers versions. UE4 engine startup takes about 5 seconds on both, which gives a 12 second first run cold load time, and about 5-6 seconds as the warm load time. However this is a very fast computer with 16 logical cores.
If the bulk of the time is consumed by the UE4 engine startup, it is likely to do with the number of assets in the package and the amount of startup code. One random guess is that it might be related to shader compilation, which is an operation that has been identified to be slow on some Windows systems. To verify this, one can use the browser performance profiler to test where the load time is spent. OS X is known to have more efficient shader compilation times, so also testing the startup times on an equal performance OS X system can be insightful. If the startup is being slowed down by shader compilation, there is this bug entry 918941 - (webgl-shader-cache) cache results of shader compilation which discusses mitigating the shader compilation performance issue for warm loads in Firefox. Not sure how this relates to Chrome, although Firefox and Chrome reuse the same architecture for shader compilation on Windows (browser → ANGLE → Microsoft HLSL compiler).
In Firefox, WebAssembly will greatly improve JS compilation times for cold loads, although for warm loads, there is very little change. Warm loads with asm.js in Firefox should be very fast already, and on second load you should see something like
"Successfully compiled asm.js code (loaded from cache in 643ms)"
In Chrome unfortunately load times are pretty bad, and it doesn’t implement any kind of caching of compiled code that I’m aware. With WebAssembly they will hopefully get closer to Firefox and Edge performance.
