In networked game, AttachActorToActor for other player not working?

So i have a very basic test setup with a FPS camera and a Sphere actor with physics and collision on it.

Locally i can pick up the ball and attach it to my hand socket, then drop the ball and add an impulse to throw it. Works great.

The issue comes in a multiplayer environment. The ball is set to replicate, and to replicate movement. Both the throw and pickup events are set to replicate.

When in a multiplayer setup we both see the ball get kicked around, but if the local player picks up the ball the remote player see’s the ball snap to the local players hand then fall back down to the ground from gravity rather than staying attached to the local players character. The local player see’s the ball attached to their hand as it should be. So things for the ball are now out of sync.

When the local player then throws the ball, the “fake” ball on the ground the remote player see’s the ball immediately jump to the local players hand and be thrown as if it was there the whole time.

I’m at a bit of a loss as for whats happening, any insight would be much appreciated.