Game FPS locked to video stream FPS

Hello, I am working in UE4 to implement a Mixed Reality Lab setup and I have come a long way toward making it happen. I started by implementing this openCV integration for video capture from a webcam and have since included a chroma key and a video output from scene capture component. Everything is working amazingly, but I still have a couple of issues left before I can call it done.

The first and most important issue is with the games’s FPS. The game FPS is locking to the incoming video FPS. I started testing for this system using Xplay virtual camera which allows me to set the outgoing FPS (how I determined the issue at hand), but in the actual MRL I will not have such a function. I need to be able to hook up a camera that records at 25/30 FPS and feed it’s output into UE without going below 90FPS for VR headsets.

My Tick Override function:

void AWebcamReader::Tick(float DeltaTime)
{
	Super::Tick(DeltaTime);

	RefreshTimer += DeltaTime;
	if (isStreamOpen && RefreshTimer >= 1.0f / RefreshRate)
	{
		RefreshTimer -= 1.0f / RefreshRate;
		UpdateFrame();
		UpdateTexture();
		OnNextVideoFrame();
	}
}

I am hoping for an easy fix for this, but there is not much documentation for UE4 when it comes to incoming video feeds.

If anyone is interested in the second issue, it is that the incoming video will only record at 640x480. I believe this to be an OpenCV issue however, but any insight is welcome.

To “unlock the fpses” you have to retrieve and decode data from the camera in a seperate thread as this is very time consuming operation. As for the second issue, for me it worked that I added these lines just before opening the stream: stream.set(CV_CAP_PROP_FRAME_WIDTH, DesiredSize.X); stream.set(CV_CAP_PROP_FRAME_HEIGHT, DesiredSize.Y);

and set the DesiredSize to be ABOVE the highest supported by your camera :o My webcam supports FullHD so I set the DesiredSize to 2000x2000 and finally the code captures the frames in… 1920x1080 (I have followed this tips: Python OpenCV access webcam maximum resolution - Stack Overflow).