Beacon Timeout Too Short and Uses Same Port as Server

Development Platform & Builds
, , , , ,
1

Using Unreal Engine 4.7.2, I attempt to connect to a beacon on a remote server using Steam sockets (Steam SDK v1.30). remote server is behind a firewall and therefore it seems like it’s using UPnP or punch-through or something, not sure if that is making difference in this case. initial connection timeout as set by engine is 5 seconds. Apparently, this is too short. If I double it to 10 seconds it is able to connect and communicate just fine.

Futhermore, it seems that when client connects to server’s beacon port (15000) it also initiates connection from a local port of 15000. This causes a problem because when connection times out, I host my own session and beacon. After beginning hosting beacon, it receives late UDP messages from remote server in response to my initial connection attempt and these cause an assert in engine, as it results in two beacon hosts communicating with each other.

Not sure what right fix is here. I’ve increased initial connection timeout, but I imagine these beacon client connections should probably be originating from a different local port to avoid this second issue.

2

Sorry for delay here.

beacon code does have that 5 sec delay, but I never intended that to be fixed. You are free to adjust that as you please and it is exposed in ini system as you’ve found out.

From Valve’s site, I would suggest 20 seconds at least
Error handling

SendP2PPacket() will try for up to 20 seconds to establish a connection to  remote user and deliver  packet. Your first sends may be delayed by that long. If there is an error establishing  connection, a callback will be posted:

As for second problem, which assert are you hitting? Is it one in NotifyControlMessage()?

I’m not sure if larger delay would allow for proper clean up and guarantee no stray packet from potential host. That being said, I’m looking through Steam P2P docs because I swear that, unlike udp/tcp, SteamChannel is reciprocal. Valve’s single page on p2p doesn’t even mention channel. header says this

nChannel is a routing number you can use to help route message to different systems - you'll have to call ReadP2PPacket() with  same channel number in order to retrieve  data on  other end using different channels to talk to  same user will still use  same underlying p2p connection, saving on resources

I don’t think you can set client to “any port” and send to proper host port. If SteamChannels don’t match I think communication would fail. It’s been a little while, but I wonder if something like asymmetry would have prevent issue. Maybe some kind of “client offset” would allow this (send to host port XXX, return data on client port XXX + constant).

underlying issue is that beacons are attempting to use same code as regular networking, so socket isn’t aware of nature of data to reject something like stray packets.

It may also be an issue with DeadConnections array I have internally to Steam socket cleanup. I move old p2p connections to this array for a time period to prevent stray data and reconnections when I know I want socket dead. But it is only 3 seconds. You may want to investigate that at time of stray packet. You would find that Steam socket subsystem is getting a AcceptP2PConnection call after you’ve disconnected. I would expect it to be rejected.

If you don’t mind doing a little more digging and report back I can try to help further.

3

assert it hits is in UChannel::ReceivedNextBunch:

check( !HandleBunch->bReliable );

It receives a CHTYPE_Control message but OpenedLocally and OpenAcked are both false. It never enters AcceptP2PConnection.

edit: timeout does not seem to be editable via ini because it’s #defined in OnlineBeacon.h and set explicitly in AOnlineBeaconClient::InitClient. I had to override InitClient in my AOnlineBeaconClient subclass. Am I missing something?

Side Note: I am able to repro this behavior on a local LAN by setting initial connect timeout to 1 second.

4

Hi BrainDx,

Just wanted to let you know that we haven’t forgotten about you, and I’ve let Josh know that you have some additional questions. We’ll get back to you as soon as we can. Thanks for your patience!

5

timeout values are configurable in ini

[/Script/OnlineSubsystemUtils.OnlineBeacon]
BeaconConnectionInitialTimeout=5.0
BeaconConnectionTimeout=45.0

I’ll have to ask one of our network guys why this particular message isn’t getting shunted sooner. UNetConnection should have been cleaned up from your previous attempt to disconnect and then separated from UNetDriver so any future communication would have to start handshaking again and fail with packets of this nature. Can you see NetConnection::Close or ::Cleanup get called in between when you decide to abandon connection and time you make your host? There may need to be a more deliberate check on your part before allowing once client to become a host.

When you increase timeouts does crash go away?

Did you follow DeadConnections code? When you shutdown connection, this active connection should be moved to DeadConnections via P2PTouch and P2PRemove in Steam/Engine code.

6

As of 4.7.3, it is not configurable this way, although I see this has been fixed in master branch.

I’ve looked into this more and I believe issue is that dead connections array does not close p2p connection for 3 seconds. As a result, what happens it that there is no additional call to AcceptP2PConnection because connection is still active. In fact, even if dead connections timeout were 0 bug still occurs because in this case you don’t get a call to Tick (which is where connections in DeadConnections get closed) between call to P2PRemove and stray packet’s RecvFrom.

Maybe FSocketSubsystemSteam::UnregisterConnection would be better off closing connection instantly than doing it via dead connections array?

If dead connections array is necessary for connection to get flushed before closing, then maybe right solution is for Steam socket’s RecvFrom to handle discarding packets from IDs in DeadConnections array? P2PTouch already checks DeadConnections array and is called from RecvFrom. It could return a bool indicating whether or not packet is to be discarded.

7

I’ve made this change and pushed it to a branch in my local fork here: https://github.com/braindx/UnrealEngine/tree/steam-sockets-dead-connections-fix

Thoughts?

8

So I admit its been a little while, but I remember DeadConnections was required because remote side was chatty during shutdown and I needed a way to ignore that traffic and not initiate another AcceptP2PConnection. 3 seconds I believe was also necessary to make sure that any traffic flush would occur rather than disconnecting directly. 3 seconds might be too much, but up until now I hadn’t seen any issues with it.

I do like your idea for RecvFrom. I don’t see any harm in discarding packets for connections that are meant to be closed. Can you work with that in place for a while and report back if you notice any issues with it?

If that turns out to be a good fix, we can turn it into a pull request.

9

This seems to be working well. I’ve created a pull request:

https://github.com/EpicGames/UnrealEngine/pull/1005

10

Turns out this pull request has some issues after all. Ultimately, I couldn’t see how to easily solve all of issues with communicating over multiple ports, so I’ve had to refactor a bunch of SocketSubsystemSteam to clean it up. new pull request is here:

https://github.com/EpicGames/UnrealEngine/pull/1075

11

I realize this is nearly two years old, but did anything come of it? We plan on using online beacons with steam subsystem and hope that this isn’t still a possible edge case. I do see that PR was never merged though.

12

Our game is releasing soon and we are most definitely using beacons over Steam sockets.

To clarify my earlier post, there are a few snags you may run into due to fact that Unreal treats Steam channels as ports when they’re not exactly 1:1 in functionality. For example, typically sockets will use an arbitrary (different) client port to connect to a specific server port. With Steam sockets, however, client must be using same channel as server in order to communicate. As a result, there are a few hiccups:

  1. When closing and reopening connections on same channel you may receive some old data on socket. This can cause a variety of problems.
  2. If attempting to communicate via several beacons simultaneously on same channel you will run into issues where one beacon client will receive a packet meant for another beacon client and discard it.
  3. Steam connection lifetimes can be managed a bit more efficiently to mitigate some of these issues. My pull request is bulk of that work.

So, in short, it mostly works but, depending on your use case, be prepared to deal with these edge cases appropriately.

1 Like