Motion tracker causes my midpoint to shift

Hi everyone,
I have been developing a simulation using the Oculus Rift but without the motion tracker. I recently started using it but its causing some trouble.
As an interface, im using the middle of the screen to work as a “mouse pointer” (drawing a vector) and is being shown by a sphere that is shown on the surface it hits.
Without the positional tracking, the sphere always remains on the center of the screen regardless of the distance between the camera and the object its hitting. With the positional tracking, if I turn my head to the right, the center sphere starts shifting to the left as if its being left behind or something.

Is there an explanation why is this happening and how I could fix this?

Thanks in advanced,
Seichi