Collision breaks down

I have an actor which I would like to apply force to move it. Both when I used addForce and addImpulse for one frame the object tends to pass trough walls sometimes. My guess was the object was too fast so the next iteration of the engine was at the other side of the world, but it also passed on low velocities and didn’t in higher velocities. What is happening? Is there any way of fixing it without lowering the force too much?