Hello. I am using UE4 for robot simulation and would like to employ several SceneCapture components as virtual cameras located on the robot. To do this, I have created a custom class, UCamera, which inherits from USceneComponent. When UCamera::BeginPlay is called, I create a internal instance of USceneCaptureComponent2D as follows:
bool UCamera::CreateSceneCapturer()
{
sceneCapturer_ = NewObject<USceneCaptureComponent2D>();
if (!sceneCapturer_)
{
UE_LOG(LogTemp, Warning, TEXT("Error creating scene capturer"));
return false;
}
sceneCapturer_->AttachTo(this);
sceneCapturer_->FOVAngle = 90;
sceneCapturer_->bCaptureEveryFrame = true;
sceneCapturer_->TextureTarget = renderTarget_;
sceneCapturer_->DepthBuffer = &depthBuffer_;
sceneCapturer_->CaptureSource = ESceneCaptureSource::SCS_FinalColorLDR;
sceneCapturer_->PostProcessSettings.bOverride_AntiAliasingMethod = true;
sceneCapturer_->PostProcessSettings.AntiAliasingMethod =
EAntiAliasingMethod::AAM_TemporalAA;
sceneCapturer_->RegisterComponentWithWorld(this->GetWorld());
return true;
}
I then create an instance of UCamera in Unreal Editor and attach directly to the FirstPersonCamera in the FirstPersonShooter demo project. The component tree of the FirstPersonCharacter is as follows:
Every call to UCamera::TickComponent I call UpdateContent on my internal instance of USceneCaptureComponent2D and send there results of ReadPixels to an external program for visualization. This is my current result (my custom camera on top, the game’s main camera on bottom):
As you can see, my biggest issue is that my custom camera is not in sync with the other camera and character mesh. I’m not sure if my camera is ahead in time (captures before other objects have moved) or is ahead in position (capturing from the wrong location). I have tried playing with Tick Groups with no success. I have also seen another post that suggested using a SpringArm to make the SceneCapture camera lag behind. This gets closer to the results I want, but is not precise enough. To truly use this in a perception pipeline, the cameras need to be fixed to the robot and cannot be sloshing around.
Any ideas on what I could be doing wrong?
Many thanks!
EDIT: I should also mention that I have tried creating the same setup completely from the UE editor (without writing any C++ code) and I am getting the same results.