I want to take out the server logic to a custom separate server, but nowhere can I find an api in which the server and the client communicate
If you wanted to write your own server, you would also need to write your own API to interact with that server. I very strongly would recommend against this, and if you wanted to go this route I wouldn’t recommend using UE4. One of the many benefits of UE4 is its networking capabilities, and I imagine you will run into many problems trying to separate out the networking stuff. If you want to look more into the source code of UE4’s networking module, you can find it here https://github.com/EpicGames/UnrealEngine/tree/release/Engine/Source/Runtime/Networking
-Testy