Faking Point Gravity while maintaining player control

So I’m trying to fake point gravity by using a Get Direction Vector and an Add Force (see ‘gravity hack’ below). This works; it attracts the player towards my gravity ‘point’, but it also completely eliminates my ability to control the player character (I have to turn on Physics Simulation on the MyCharacter blueprint).

Anyone have any idea how I can use my gravity hack and still maintain player control? Is this even possible?