How to change the projection matrix at the scene hit proxies calculation

I managed to change the projection-matrix in the way Slawek.Pudlis suggested here: https://answers.unrealengine.com/questions/65003/howto-modify-the-projection-matrix.html

But now I’ve got the problem, that the “onClick”-event for any Actor in the Scene no longer works properly.
I think I also have to to change the projection-matrix of the view, where the HitProxies are calculated/rendered…

Anybody got an idea, where i can start this task?

Did you ever solve this issue? :slight_smile:
I’m facing the same issue…

I got this working, but had to change engine code…
Not sure if this is the finest way, but i got no problems so far…

So “onClick”-events use the function “DeprojectScreenToWorld” from
\Engine\Source\Runtime\Engine\Private\GameplayStatics.cpp

In the function they use the LocalPlayer:
ULocalPlayer* const LP = Player ? Player->GetLocalPlayer() : nullptr;

I added two variables to the parent LocalPlayer class:

Engine\Source\Runtime\Engine\Classes\Engine\LocalPlayer.h

    UPROPERTY(VisibleAnywhere, BlueprintReadWrite)
        float LPScale;

    UPROPERTY(VisibleAnywhere, BlueprintReadWrite)
        FVector2D LPShift;

Like Slawek.Pudlis suggested I got my derived class MyLocalPlayer, where I edit the ProjectionMatrix in the CalcSceneView function. But there I also set the variables LPScale and LPShift of the parent LocalPlayer class:

        FScaleMatrix scaleM(FVector(scale, scale, 1.0f));
        FTranslationMatrix shiftM(FVector(shift.X, shift.Y, 0.0f));
        LPScale = scale;
        LPShift = shift;
        UpdateProjectionMatrix(View, scaleM * shiftM); 

After that I got the edited projection matrix and the localplayer has the scale and shift values. Now I got to change the DeprojectScreenToWorld function:

bool UGameplayStatics::DeprojectScreenToWorld(APlayerController const* Player, const FVector2D& ScreenPosition, FVector& WorldPosition, FVector& WorldDirection)
{
	ULocalPlayer* const LP = Player ? Player->GetLocalPlayer() : nullptr;
	if (LP && LP->ViewportClient)
	{
		// get the projection data
		FSceneViewProjectionData ProjectionData;
		if (LP->GetProjectionData(LP->ViewportClient->Viewport, eSSP_FULL, /*out*/ ProjectionData))
		{		
            //-------Shift-and-Scale-Edit-------	
			FScaleMatrix scaleM(FVector(LP->LPScale, LP->LPScale, 1.0f));
			FTranslationMatrix shiftM(FVector(LP->LPShift.X, LP->LPShift.Y, 0.0f));
			FMatrix UpdatedMatrix = ProjectionData.ProjectionMatrix * scaleM * shiftM;
			UpdatedMatrix = FTranslationMatrix(-ProjectionData.ViewOrigin) * ProjectionData.ViewRotationMatrix * UpdatedMatrix;
            //----------------------------------
            
			FMatrix const InvViewProjMatrix = UpdatedMatrix.InverseFast();
			FSceneView::DeprojectScreenToWorld(ScreenPosition, ProjectionData.GetConstrainedViewRect(), InvViewProjMatrix, /*out*/ WorldPosition, /*out*/ WorldDirection);
			return true;
		}
	}

	// something went wrong, zero things and return false
	WorldPosition = FVector::ZeroVector;
	WorldDirection = FVector::ZeroVector;
	return false;
}

Because this is working for me, I will mark this as the answer, but if you find a better way, feel free to correct me :slight_smile:

Thx for the reply. I’m trying something similar right now. I’ll respond in time :slight_smile:

Thanks SebaSopp for the hint!
I implemented a similar solution without engine modification.

If you create a custom LocalPlayer class and calculate the projection matrix there, you can use this matrix for a new static member (and blueprint node) doing the raycast.
The implementation of the new static member is then similar to LineTraceByChannel and uses a modified DeprojectScreenToWorld method as indicated by SebaSopp.

bool UOffAxisLocalPlayer::OffAxisDeprojectScreenToWorld(APlayerController const* Player, const FVector2D& ScreenPosition, FVector& WorldPosition, FVector& WorldDirection)
{
ULocalPlayer* const LP = Player ? Player->GetLocalPlayer() : nullptr;

if (LP && LP->ViewportClient)
{
	// get the projection data
	FSceneViewProjectionData ProjectionData;
	
	if (LP->GetProjectionData(LP->ViewportClient->Viewport, eSSP_FULL, /*out*/ ProjectionData))
	{
		ProjectionData.ProjectionMatrix = s_ProjectionMatrix;
		s_ProjectionMatrix = FTranslationMatrix(-ProjectionData.ViewOrigin) * ProjectionData.ViewRotationMatrix * s_ProjectionMatrix;
		FMatrix const InvViewProjMatrix = s_ProjectionMatrix.InverseFast();

		FSceneView::DeprojectScreenToWorld(ScreenPosition, ProjectionData.GetConstrainedViewRect(), InvViewProjMatrix, /*out*/ WorldPosition, /*out*/ WorldDirection);
		return true;
	}
}

// something went wrong, zero things and return false
WorldPosition = FVector::ZeroVector;
WorldDirection = FVector::ZeroVector;
return false;
}
bool UOffAxisLocalPlayer::OffAxisDeprojectScreenToWorld(APlayerController const* Player, FVector& WorldPosition, FVector& WorldDirection)
{
float x, y;
Player->GetMousePosition(x, y);
return OffAxisDeprojectScreenToWorld(Player, FVector2D(x,y), WorldPosition, WorldDirection);
}
bool UOffAxisLocalPlayer::OffAxisLineTraceByChannel(
		UObject* WorldContextObject, 
		/*out*/ struct FHitResult& OutHit, 
		FVector _eyeRelativePosition,
		bool bDrawDebugLine,
		FColor _color,
		bool bPersistentLines /*= false*/,
		float _lifeTime /*= 10.f*/,
		uint8 _depthPriority /*= 0*/,
		float _thickness /*= 1.f*/,
		float _LengthOfRay /*= 1000.f*/)
{
//transform eyeRelativePosition to UE4 coordinates
FVector _eyeRelativePositioninUE4Coord = FVector(_eyeRelativePosition.Z, _eyeRelativePosition.X, _eyeRelativePosition.Y);

//get end position for ray trace
FVector WorldPosition, WorldDirection;
OffAxisDeprojectScreenToWorld(UGameplayStatics::GetPlayerController(WorldContextObject, 0), WorldPosition, WorldDirection);
FVector end = WorldPosition + _LengthOfRay * WorldDirection;

if (bDrawDebugLine)
	DrawDebugLine(WorldContextObject->GetWorld(), _eyeRelativePositioninUE4Coord, end, _color, bPersistentLines, _lifeTime, _depthPriority, _thickness);

if (s_ShowDebugMessages)
{

	UE_LOG(OffAxisLog, Log, TEXT("Start: %s"), *_eyeRelativePosition.ToString());
	UE_LOG(OffAxisLog, Log, TEXT("End  : %s"), *end.ToString());
	
	GEngine->AddOnScreenDebugMessage(300, 10, FColor::Cyan, FString::Printf(TEXT("Start: %s"), *_eyeRelativePosition.ToString()));
	GEngine->AddOnScreenDebugMessage(310, 10, FColor::Cyan, FString::Printf(TEXT("End: %s"), *end.ToString()));
}

//do raytrace
return WorldContextObject->GetWorld()->LineTraceSingleByChannel(OutHit, _eyeRelativePositioninUE4Coord, end, ECollisionChannel::ECC_Visibility);

}

Not the prettiest code but it’s working nicely :slight_smile:

PS: If someone knows a good page/tutorial/method on how to format code properly, hit me up! Thanks!