When I use the template AR scene in 4.19, blank space (black in editor) is considered passthrough for the camera. when I create a material to act as passthrough, the scaling is different, and that causes anything seen through the material to not track properly, while the blank world does.
Why is scaling different between the two? Am I missing something critical? What is the scaling offset? It looks like it might be non-uniform as well.
Here’s an image of my mtl.
I’m guessing the issue is that the material is drawing to a differently scaled “viewport” than the default blank scene. How to I get them to match?
Nothing? You should see the fix I had to do. Nah, you don’t want to. It’s crazy;)
I’m in the same situation. The camera image appears stretched on the object and not screen aligned.
EDIT: The comment by haowang1013 here [ARKit] Dynamic Object Shadows in 4.18? - XR Development - Unreal Engine Forums might have the answer. This doesn’t work anymore on 4.18 and above.
Hello,
I think I might have found the solution. This at least works on landscape-left and portrait. You have to crop the UVs to feed into the ARKit Passthrough Camera node in the same way that UE4 does it for the background (I copied the logic from AppleARKitVideoOverlay.cpp).
Then for landscape-left:
Or for portrait:
This is not a finished material and just intended as an explanation. It should be optimized by calculating the offset from a BluePrint at BeginPlay and pass it to the material as a parameter. I’ll post the final version once I get it done.
Thanks for this! I ended up solving it a less elegant way. I made a little UI that would allow me to tap small intervals of mult and offset on the different axis of screenPosition. The UI also had a visible and invisible toggle for the camera material object. So I would tap different offsets and toggle visibility until they matched. Then I stored the values and plugged them into the screenPosition and got something working. What a pain;)
I tried this for portrait and it was unsuccessful.
I cannot verify landscape yet.
When you are able to test portrait, can you post the changes to make it work?
I’ll definitely do that. I will more likely get it done on Monday. I suspect it’s just a matter of clipping the V axis for portrait instead of the U. If in the make float node you swap the X-Y and Z-A input pairs, it might just work.
I tried flipping those like you suggested and instead of flipping the passthrough camera material 90 degrees for portrait, it flips it 180 degree for the other landscape orientation.
Just updated my answer with the changes needed for portrait.
Hi,
Thanks for the material! It works great on iPhone’s… But when we try it on iPad Pro 12.9’’ it gives wrong scale on Landscape and Portrait orientations… Even default material works same on landscape… We tried native & custom resolutions, but no luck…
Does anybody have solution for iPad Pro 12.9’’ ?
Hello,
Things are changing for ARKit since I posted this. If you are using 4.20 and iOS 12, now you don’t have to do any more UV cropping. The passthrough camera texture doesn’t need anything connected to the UV input as long as you are in landscape-left orientation and it just works. For portrait you just have to invert the V coordinate and swap UV (U becomes V). This has been tested on an iPad Pro 10’’, but haven’t tested in a different size. I’ll try to post a node screenshot soon.
I altered my ARKit Material in this way, but I still notice the same issue I’ve been having in 4.20 that I also had with your original material where the ar material is narrow in portrait mode making things look slightly distorted. I’m curious if you know a solution for this?
Altered material
Objects through AR material look more narrow
I’ve never used the rotator but instead remapping the UV by swapping the coordinates. Just recently I also found that for certain aspect ratios the straight material with UV swapping is not working. for example with the new iPhone XS. The correct way of doing it would be implementing the same UV cropping that is done in the engine for the backdrop. I’ll be looking into this for our project soon and I’ll share any results. The file to look into is AppleARKitVideoOverlay.cpp around line 126 a quad is UV cropped considering orientation.
Wups, I put the wrong material screenshot up so I edited my original comment. I’ll look more into that AppleARKitVideoOverlay.cpp file and see what I can come up with, thanks.
Did you find a solution to the problem? This turns out to be pretty frustrating to quite a few people out there.
I haven’t found a solution but working on it. I’ll share it once it’s done. On the other hand, I heard from an Epic developer that for 4.21 there will be a material node that does the UV cropping properly.
Tested 4.21, did not notice any new nodes that could solve this issue. Standard ARKitPassthroughCamera produces still the same stretched images.
When I asked Unreal about this they informed me its on their road map and probably would be in 4.22.
Well, that’s bad news. I was hoping they would fix it for 4.21. It’s a big problem for our upcoming release…
