Dedicated Server Performance

Development Programming & Scripting
1

We’re currently developing a client-server game using Unreal Engine 4.19.2.

At the moment we are trying to optimize the server side of the online game. Our goal is to use the computing power of the hardware server as efficiently as possible.

For testing, we use a server based on the Intel Xeon E3-1220 processor with 3 GHz clock speed, and Ubuntu Server 18 LTS, running under Windows 10 hypervisor as OS. The hypervisor allocates the guest Ubuntu 30 GB RAM and maximum priority to all 4 CPU cores.

We configured the engine for our game so that the Dedicated Server should tick 10 times per second (NetServerMaxTickRate=10), that is, the process of the Dedicated server has 100 ms to perform all its calculations in time.

According to the data obtained from profiling by standard methods, it can be seen that GameThread uses about 6 ms to perform all calculations; and the rest of the time the thread sleeps, which can be seen from the values of statistics variables from profiling: FEngineLoop_UpdateTimeAndHandleMaxTickRate = ~ 94 ms.

At the moment, we managed to run 45 Dedicated Servers under load and at the same time play on client PCs without any lags. In this case, the server OS shows 100% CPU usage, but there is still enough RAM to run the OS, and to run about 10 servers of the game simultaneously.

The question arises: why are the processes of the 45 servers so CPU intensive? Considering that the effective time of server calculations does not exceed 10 ms (if you put together the execution time of all threads of the Dedicated Server), then theoretically, you can run less than 100 servers, but certainly more than 50.

The most possible cause of this problem, as we think, is the “struggle” of the Dedicated Server for OS resources, namely, when in a certain CPU cycle more game servers “wake up” than the hardware can handle simultaneously.

Could you please help us solve this problem, if you have any ideas. We assume that you probably faced a similar problem of distribution of the load on the server when you deployed the infrastructure for the game Fortnite, or other games. Or, perhaps someone from the Unreal Engine 4 community may have solved this problem?

2

not sur i got my math correct, but here what i see :

6ms/100ms per server => 60ms/sec/per server

1000ms/60ms = 16.6 meaning you can run 16 server on that hardware

thing is you got 4 cores and UE probably don’t use the 4 cores at 100% in those 6ms, so seems fair to say you could run 16.6 *4 / 1.3 ( the server use the equivalent of 1.3 core at 100% durring the 6ms ) => == 44.5 server

3

UE4 is multi threaded, so in those 6ms it will use more than a core because of that, so you won’t get the 66 servers. ( i don’t have the values for you server, you have to profile the cpu there).

you can always lunch on server and try a “stat DumpFrame -ms=0.1” to have a look what’s taking time on you server. ( not sure if this console command work on dedicated server ) but what you want is make those 6ms go down to 5ms or less( to run 60 servers or more )

Also, are those 6ms for a server under load ? or it’s empty servers ? Because you would want to also profile server under load ( both could be important )

But yeah running dedicated servers is expensive and i thinks if you manage to get 40 server on a machine it doesn’t seems bad ( i have no idea how heavy your game is supposed to be ) but definitely profiling is the way to go to have an idea of what could be improved

4

If we assume that the UE uses a processor with 100 efficiency, then based on your calculations, you can run 4 (cores) * 1000 (ms) / 60 (10 ticks * 6 ms) = 66.7 servers. It is not clear to us why the UE uses a CPU with such low efficiency and is it possible to increase it.