Physics Pitch Bug

So I know that there is this weird pitch bug when you set your actors position.
I tried to avoid that by changing my 6DOF flight system to be physics based and let the physics system do the rotation for me. But wow, the bug still appears.
Is there no way to fix that?

Wait. So with AddActorLocalRotation it works, but the SpringArm still bugs.

And with AddTorque the rotation is limited at 90 degress and my object won’t rotate further.