Physics constrain tracking problem on moving object

I’m having problems on physics constrains not holding their components in place on a moving actor.
In this case it’s a weapon with a rotating blade on the end of the handle.
The weapon is attached to vr hands, and whenever i move my weapon around the blade seems to drift behind the handle instead of being fixed to it.
Almost as if it is tracking the handles position with latency.
I tried setting the projection limits really low but no success.

Any Suggestions?