I create the following test project with one map “TestMap”. I enable depth stencil pass “enabled with stencil” in project settings. I create one large cube, custom depth pass enabled, customdepth 201. I create an infinite extent postprocess volume. I assign it a custom material DebugMaterial, which looks at SceneTexture:CustomStencil and displays green if R is 0 and DebugFloat4Values on the CustomStencil value otherwise. Here is the project:
[link text][1]
I test this on two machines.
Machine 1:
Laptop, Windows 10 version 1709 with Unreal Editor 4.18.2. Laptop with “display scaling” enabled. Lists its video card[s] as “Intel(R) HD Graphics 630” and “NVIDIA GeForce GTX 1060”.
Result:
Looks good.
Machine 2:
Mac OS X 10.12.6 (16G29) with Unreal Version: 4.18.2-3794801+++UE4+Release-4.18 . Retina macbook. Lists its video card as “Intel Iris Graphics 6100 1536 MB”. I believe the editor runs with Metal because I’ve seen it in stack traces and when I do standalone mode I see SF_METAL_SM5 in the window title.
Result:
I get only green [value 0].
Another weird thing: If I switch into Buffer Visualizer->Custom Depth mode, then switch back to “Lit”, I get a freeze frame of the cube with the debug values printed over it as expected, except that (1) the debug numbers are the nonsensical value “600”, and (2) it does not “animate”, if I move the camera it remains frozen. I don’t know what this means.
Additional notes:
I have filed a separate bug about “buffer visualization->custom stencil” not working as expected, again on 4.18.2 mac. "Buffer Visualization" > "Custom Stencil" does not animate properly, is not readable [on mac] [mostly fixed in High Sierra] - Rendering - Epic Developer Community Forums
Expected behavior:
Custom stencils should work on OS X Metal. This older answer CustomStencil not working on Mac - Rendering - Epic Developer Community Forums implies it worked in 4.11.
If for some reason custom stencils don’t work on Mac, or don’t work with certain video cards or something, this should be clearly documented, and the editor should display whatever helpful warnings are needed when running in Metal mode instead of just emitting garbage values.