I have compiled dedicated server using 4.19 source and I would like to know if I have to download 4.20 source now to compile dedicated server.
Yes, but note that any UE4 build with build in server support turned on can work as dedicated server
You only need this build if want to distribute dedicated server without client
Can you elaborate on your answer? Is it possible to build a headless server executable from the editor without downloading the source code and building yourself? If so, what are the steps?
If you want light weight headless server then you need to build from source.
You can run casual server with client with:
YourProject.exe YourMap?Listen
And It will work for all UE4 Build