PhysicsHandle force

Hello, in my simple game you can move objects around with PhysicsHandle. The problem is PhysicsHandle is applying too big force to holded object, and can for example hit other object and make it fly through whole map. It happens especially when you’re rotating character - and holded object is rotating with you. I tried changing Damping and Speed, but doing it can’t fix the problem. Lowering InterpolationSpeed helps a bit, but, as the name says, it makes my object to move slower and in result my characters need then to move slower too. Lowering holded object’s mass would probably fix the problem, it would be harder to move another objects with holded object but sadly you can’t change mass of object in UE.
Any ideas how to fix it?