Character movement networking - physics interaction

hello,
I have Problem in making a physics-based multiplayer ballgame.
Here´s a video of it:

The Ball reacts differently on contacts with the client as with the server.
I´m pretty sure, this is because of the server-side movement correction, or the network smooth update.
I mean:
The server corrects the position of the client
- the client gets moved or teleported by the server really fast
- the client hits the ball with a speed, higher than the average walking speed
- the ball gets pushed away really fast

I´ve made a lot of trial and error, but can´t avoid the ball from being pushed away too fast by the clients.

Any suggestions how I can solve this, or even minimize the effect?