External Tracking System to Player with Oculus too

We have an external tracking system (OptiTrak overhead cameras) and an Oculus. We want to use the position from the external tracking system, but have the oculus control rotation. How can we do this?

We also need to be able to test in WASD PC mode if the tracker is not present.

I have tried turning off physics and such in the Player controller but the player ends up bobbing up and down rapidly.

We are setting the positon successfully because X and Y track correctly.