[ARkit] No more camera feedback on Metal Desktop Forward Renderer

Hi everyone!

I’m experimenting with ARKit on an iPad and everything works fine when I use the Metal Mobile Renderer. However, When i activate the Metal Desktop-Forward Renderer my camera feedback turns black. I can still spawn my assets and everything else works, but no more camera feedback.

I hope you guys know a way around this issue,

Thanks!

I’d be interested as well.

I found this but… Metal Camera Tutorial Part 1: Getting raw camera data - Alex Staravoitau’s Blog

Bump. Same here .4.18.2 binary release

The solution is to place a cube with an arkit camera material as a background, so that it feeds with the capture.

To make the material - search for and duplicate the included ARKitCameraMaterial and modify it as in the picture. Just break and add ScreenPosition.

Sorry that you are experiencing difficulty with this. I want to confirm that the camera pass-through rendering is indeed having issues with Metal Desktop-Forward Renderer. We are working on a fix for 4.19.

However, the camera data itself is there and you should be able to utilize it inside of a material. Have you tried the work-around proposed by a_bolog? it seems like it would work.

Did you place a standard unreal cube? I had to create one in a 3D DCC app and flip the normals, otherwise it was still black? Also, the material preview is green all the way in the editor for me, not pink like yours. What Engine Version are you using? Overall, it works with Desktop Forward Renderer enabled and 4.18.2. Thanks for the workaround!

It seems in 4.19 that using this material does not project the same scale/pos/resolution as a blank background. Meaning the passthrough material doesn’t match the pure camera passthrough of a blank background. Therefore anything seen through the material isn’t scaled correctly and doesn’t match tracked objects.