[HTTP] Why is HTTP GET so slow, UE4 is performing 15x slower than Python at downloading a file
Dear Epic and Community,
Does anyone know a way to speed up the use of HTTP GET requests on windows machine?
I can download 1.67 GB in about 1.5 min using chrome or my ftp server, but using HTTP GET request from within UE4 takes at least 30 minutes.
Why is it literally about 20x slower?
For comparison I downloaded the 1.67 gb file in 2 min and 20 seconds using python, which is about the same as Chrome and my FTP server
I really wish UE4 could download things faster for me than python can :)
As more and more UE4 projects are maturing and needing fast download speeds for DLC and paks and all sorts of things, I really think fast download speeds should be a high priority :)
My research so far:
I feel like libcurl is capped somehow in a way it shouldnt be.
I tried turning lib curl off but then the download does not occur at all. (I tried both config and commandline methods)
I tried accessing the public static FCurlHttpManager ptr and including the private .h file, but the compiler was not finding it even with HTTP module included
Even if I did succeed in that I wouldnt really know what to do to make UE4 download things faster.
Is there anyway to make HTTP GETs go faster on a windows machine?
Relevant code can be found in AsyncTaskDownloadImage.cpp, this is what I am doing, but for a 1.67 gb file.
I wrote my own download progress bar using
So I know the download is working, it just takes a really really long time compared to Chrome or my FTP server.
Thank you for any insights in this matter!
A Neutral Test ~ 1GB Speed Test
If you want to test you can test with a 1 GB download here:
I endured it, in UE4 it took:
17 minutes and 43 seconds
Exact same url, using python:
1 minute and 19 seconds
Clearly libcurl is throttled somewhere, and cannot use the full power of my internet connection, in a way that Chrome, my ftp server, and python are not limited.
Can you please address this so I can use UE4 instead of python to download DLC?
PS: Here is the python script I used:
This is already marked as resolved question, but I want to share I discovered. I found that HTTP request's fetch rate is controlled by
You can control this value from the Engine configuration file like this:
answered Nov 20 '17 at 09:33 AM
Along the lines of nullbus's answer, you can increase the buffer size to the libcurl maximum allowed value. For a large download over a very strong internet connection, this dropped my response times from ~8.3 seconds to ~1.3 seconds.
Follow this question
Once you sign in you will be able to subscribe for any updates here