DeprojectScreenPositionToWorld not working properly with HMD

DeprojectScreenPositionToWorld (DSPTW) does not appear to be working properly with head mounted displays.

The returning WorldLocation appears to be off by arbitrary xyz(0.25, 3.0, 0.0) compared to the actual position return by HMDDevice->GetCurrentOrientationAndPosition + camera->GetComponentLocation + DSPTW’s WorldDirection (intersected with the near plane). Additionally, for visual confirmation if I use LineTraceSingle (with EDrawDebugTrace::Type::ForOneFrame) given the returning DSPTW’s WorldLocation and a projected WorldDirection, the debug trace line renders a single line always coming from the right side of the screen. If I use my computed position then the debug trace line properly (given the stereoscopic views) appears as two line coming from both directions.

ProjectWorldLocationToScreen appears to be similarly not working properly.

I’m using UE 4.7.2.

Please advise.

Hi ,

Ultimately it comes down to the following…

Attempt to use DeprojectScreenPositionToWorld and then LineTraceSingle from the data it returns. You will see that the position is not aligned with the actual HMD camera.

I’ve figured out how to get the correct location (as mentioned above), but because ProjectWorldLocationToScreen is calculated equally incorrect, I’m not able to use this correction. Because the two are equally off, I’m currently just allowing the discrepancy.

Cheers,

Hey -

I’m having some trouble following exactly what it is you’re describing. It sounds like you’re trying to measure distance from the camera in VR mode. Could you list the exact steps and code you used to test this? You may want to check out this post by on how to get a more “centered” ray cast as well.

Cheers

To make sure I’m getting the same results I’d like to confirm that I’m using the same setup you are. Are you calling the function from a character class? Could you post the code class where you’re making the function call?

void ANavigationController::MouseTrace(const FVector2D & mouse, float endOffset, FHitResult & hit)
{
FVector start;
FVector end;
FVector direction;
DeprojectScreenPositionToWorld(mouse.X, mouse.Y, start, direction);
end = start + (direction * endOffset);

	TArray<AActor *> arrayActors;
	UKismetSystemLibrary::LineTraceSingle_NEW(this, start, end, UEngineTypes::ConvertToTraceType(ECollisionChannel::ECC_Visibility), false, arrayActors, EDrawDebugTrace::Type::ForOneFrame, hit, false);
}

Hey -

What type of class was this function used in? Was it Character or Player Controller or something else? Is adding this function to the class the only thing needed to reproduce the bug? Where is this function being called from? Would it be possible to include the entire code class that uses this function? If not can you give additional information so that I can add this to a project to test.

Cheers

Hi JasonKn,

We have not heard back from you in a few days, so we are marking this post as Resolved for tracking purposes. If you are still experiencing the issue you reported, please respond to this message with additional information and we will offer further assistance.

Thank you.

Hi ,

Oops, I thought I had replied earlier - sorry.

ANavigationController is derived from PlayerController. Adding this function is the only thing required to reproduce. The function was being called on the tick.

I do have additional information. It turns out that the DeprojectScreenToWorld is based on where the camera is located and not where the player is viewing from given where the HMD is relative to the (DK2) positional tracker. This was why the position was arbitrary. I work a little off to the left and below where the positional tracker is facing. If align myself directly in front of the positional tracker and at it same height then the red tracer comes out of the view as expected.

I consider this to be a bug in how DeprojectScreenToWorld and ProjectWorldLocationToScreen deals with HMD based hardware. Can this be fixed?

Thanks,

Hey -

I’m still trying to reproduce what you’re seeing and am having trouble using the function you provided. I was able to fix a compile issue by adding an include for the KismetSystemLibrary but I’m not sure where the FVector2D, FHitResult, and float parameters being passed in from?

Hi ,

The call should look like…

float locationX;  
float locationY;  
GetMousePosition(locationX, locationY);  
FVector2D mousePos(locationX, locationY);  
FHitResult hit;
MouseTrace(mousePos, 10000.0f, hit);

All of this code is essentially the same as a call to APlayerController::GetHitResultUnderCursor which I would expect would demonstrate the same offset from the HMD view.

Cheers,

Hey -

I was able to create and use the function that called DeprojectScreenPositionToWorld() and saw the trace line appearing off of the right shoulder rather than from the center of the screen in a FPS template project and has been bugged (UE-15043). I do not currently have a workaround for this behavior.

Cheers

@, I think I’ve tracked down the issue, I made a reply on the 4.8 preview thread(Unreal Engine 4.8 Preview - Announcements - Epic Developer Community Forums ). Mind checking it out and adding your thoughts?

It seems like putting in an optional parameter for which eye to deproject based on in DeprojectScreenPositionToWorld would fix the problem, but I’m unsure if that’s an acceptable fix/how it’s normally handled.

Thanks!

Having a simular problem in my project (line-trace from VR camera center to object). The position of the VR-camera is IHMO not the left or right eye. Shouldn’t it should be exactly between and a little bit back?

@, I think that’s actually a separate issue. The VR game template (here: VR Game Template - XR Development - Epic Developer Community Forums) has an example of what you need to do to get the unified camera position for line tracing in VR.