Deproject SceneCaptureComponent2D

I currently have SceneCaptureComponent2D rendering to a texture that I’m using in a UMG widget. I want to deproject mouse clicks on that UMG widget to the 3D space in front of the SceneCaptureComponent2D. I’ve dug around the source to see if there’s anything that I can use, but I couldn’t quite find anything useful. And, unfortunately, I’m still trying to wrap my head around the math needed to make it work.

Any suggestions?

Thanks,

Answering my own question. This is what I did. (It involves editing UE4 source.)

First, I added two variables to USceneCaptureComponent.

public:
	FIntRect ViewRect;
	FMatrix InvViewProjectionMatrix;

Then I populate those two variables at the end of FScene::CreateSceneRenderer (in SceneCaptureRendering.cpp)

// Copy the data needed to do deprojection.
	SceneCaptureComponent->ViewRect = View->ViewRect;
	SceneCaptureComponent->InvViewProjectionMatrix = View->InvViewProjectionMatrix;

	return FSceneRenderer::CreateSceneRenderer(&ViewFamily, NULL);

With that in place, I can use the above and take the screen position of the click event (or whatever) and call FSceneView::DeprojectScreenToWorld.

bool ADeprojectTest::Deproject(const FVector2D& ScreenPosition, FVector& Origin, FVector& Direction)
{
	if (SceneCaptureComponent)
	{
		FSceneView::DeprojectScreenToWorld(ScreenPosition, SceneCaptureComponent->ViewRect, SceneCaptureComponent->InvViewProjectionMatrix, Origin, Direction);
		return true;
	}
	return false;
}

That’s it. It should return an origin and direction normal for your tracing needs.

It seems that InvViewProjectionMatrix is not in FSceneView anymore. Have you found a solution for Unreal 4.15 ?

I got it from View->ViewMatrices->GetInvViewProjectionMatrix(), but it seems broken.