Confusion on physics thrusters and collision mesh

I have placed a physics thruster on the side of a spaceship model. It is perfectly placed at 0,0,0, along with the center of mass of the object. When I activate it for a short period of time, the ship will start moving sideways as expected. However, there is a small rotational drift. Every second it adds 0.01 degree on the Y and Z rotational axis. If I go into the asset view and tick “use complex collision as simple” there is no rotational drifting. The spaceship will move perfectly only along its X-axis and rotational values will not change.

Why is this happening? How do I make the “project default” option (use simple and complex) behave in this way as well?