We are trying to use a media texture MP4 file to drive a media texture with animated world displacement.
The way we used to do it in 4.16 is have a split texture, with RGB on one side and HSV color gradient in another, this way we could split the depth into 360 steps of hue, and use the V channel for a 2bit mask.
In the latest preview (4) the RGBtoHSV node errors out with “(Node If) If input A must be of type float.” and won’t compile.
Even if we drop the RGBtoHSV and just grab a single color channel and pipe that directly into world displacement. Nothing happens, the tessellation is active in wire view but no displacement occurs.
Does that mean the change to having to use sampler type: External kills being able to use mediatextures for anything other than emissive and base color now?
The problem is that the sampler’s white pin outputs a float4, but the RgbToHsv function expects a float3. One simple way to fix this is to insert a MakeFloat3 node and wire it up to the RGB components of the sampler.
Also you are plugging it into the emissive, we are using the output of the RGBtoHSV in world displacement.
I just made a clean 4.18 project and set up this material. It works fine plugged into base color or emissive, but gives the error on world displacement
This should be fixed in CL# 3733463. The problem was that the media texture only supports per-pixel attributes, but not does not support per-vertex attributes, which is a requirement for WorldPositionOffset to work.
Our external texture implementation did not return a correct error code, so the shader compiler spit out a completely misleading error message instead. The fix adds appropriate error messages to external textures. Please note that this will not make WorldPositionOffset work with media textures, but only that an appropriate error message will be printed by the shader compiler.
6-DOF 360 video is starting to become viable with cameras like the Kandao Obsidian outputting 360 depth maps. Being able to drive vertex shaders with h264/h265 video makes it viable. Hopefully when oculus go/gearvr finally gets on the vulkan train it can work on mobile as well.
