Differences between render targets on PC and iOS Metal?

Hi,

I have a render target on PC that I’ve been drawing values outside the range 0.0 - 1.0. I’m using the UCanvasRenderTarget2D and drawing lines of various colours like so:

FVector2D CanvasSpaceStart = (RippleSpaceStart + 1.f) * 0.5f * Width;
FVector2D CanvasSpaceEnd = (RippleSpaceEnd + 1.f) * 0.5f * Width;

FCanvasLineItem LineItem(CanvasSpaceStart, CanvasSpaceEnd);

LineItem.SetColor(DisturbanceLineList[i].Intensity * IntensityScale);

Canvas->DrawItem(LineItem);

The IntensityScale can get quite high such that I’m drawing lines with values has high as 16.0.

This all works fine on PC. Now that I’m trying to get this to work on mobile I’m finding that everything breaks if I write values greater than 1.0.

What is the difference between render targets on iOS Metal compared to PC. I’ve verified that both platforms are creating the render target with a pixel format of PF_FloatRBGA which seems to translate to DXGI_FORMAT_R16G16B16A16_FLOAT on PC and MTLPixelFormatRGBA16Float on iOS Metal. They should be the same, right?

Regards,

Ryan.

It would be easier to understand if the problem could be seen. Capture an Xcode trace of iOS Metal to look at whether this is really a rendering issue into the float-16 target or a difference in the way the value is resolved into the final RGBA8 output. I suspect it is the latter from the information you’ve provided.

On PC, you may also be rendering with the full deferred desktop renderer and on mobile it would use the forward renderer. They use different tonemappers. And even with fp16 rendertargets, I think there may still be limits to HDR and tonemapping on mobile that needs to be set up properly, or they may get clamped to 1…

What is your intention for the >1.0 lines? In one way or another, they need to be brought down to <=1.0 for the final screen. Do you wish them to be tonemapped and bloomed out on the final screen? Or do you use these during an intermediate offscreen step for your own private purposes?

The render targets are used during an intermediate offscreen step. I’m generating a height map that I then use in another material. I’ve attached some screen shots so you can see the render targets as they look on PC. The first is the render targets and the second is the result when applied to the ocean surface normal. Is there something I can do to stop the values being clamped? I’ve never done an Xcode trace of iOS Metal so it’ll take a bit to figure that out.

Were you able to launch it in Xcode and do a capture in the graphics debugger?

I think I’ve managed to capture it. The file is too big to attach so you can download it here:

Let me know if it doesn’t work. It was captured with Xcode 7.3.1 (7D1014) from an iPad Pro 12.9 with iOS 9.3.2 (13F69).

Hi, thanks for the capture. I can open it fine, though I can unfortunately not see the shader code… Xcode says “could not find the library source. Make sure debugging information is enabled for library compilation under target build settings”. Though it may simply not work in UE4 or in Xcode anyway, not sure…

In any case, I could verify that you’re indeed using fp16 rendertarget and texture for your lines. It looks like your line drawing is in draw event 58, does that seem correct?

Since I couldn’t read the shader code, I’m wondering if you are somehow using GammaMain in SimpleElementPixelShader.usf, which would clamp your color to [0,1]. Could you check that your “Gamma” value is 1.0 in FBatchedElements::PrepareShaders() and that the if-statement on line 650 is TRUE? So that it uses FSimpleElementPS and not any FSimpleElementGammaPS pixelshader?

Also, did you use any intensity > 1.0 in this particular capture?

I think I’ve found the issue.

We were having a problem with a couple of our materials that turned out to be caused by half floats being used in the pixel shaders for mobile. I fixed that by allowing FORCE_FLOATS to be set per material so two of our materials don’t get half floats anymore.

The problem seems to be that FORCE_FLOATS doesn’t apply to the output type of the pixel shader. So on PC the defered renderer material output is a float but the iOS Metal material output is this:

struct FPSOut
{
	half4 FragColor0 [[ color(0) ]];
};

Even though I’ve set FORCE_FLOATS.

Does the PS output have to be a half4 on mobile and if not what would be the best way to try and change it. I went digging in the code and couldn’t find where the output type actually gets defined.

But you want to write out fp16 values (half4), right?

But if you did want to use FORCE_FLOATS, I would’ve expected any occurrance of “half4” to be replaced by “float4” during pre-processing of the shader source (e.g. “#define half4 float4” in Common.usf).

How did you set FORCE_FLOATS?

Though if you did want to use fp32 output, you might have to modify our cross-compiler:

MetalUtils.cpp:

static FSystemValue MobilePixelSystemValueTable[] =
{

{“SV_Target0”, glsl_type::half4_type, “FragColor0”, ir_var_out, “[[ color(0) ]]”},

Compared to the desktop version:

static FSystemValue DesktopPixelSystemValueTable[] =
{

{“SV_Target0”, glsl_type::vec4_type, “FragColor0”, ir_var_out, “[[ color(0) ]]”},

After this change, you should make sure you recompile all Metal shaders, e.g. by changing the GUID in MetalCommon.usf.

Oh duh, you’re right. Obviously writing to a FP16 render target with half4 is correct.

So I think I’ve found the actual problem. Every texture sample is getting squared for some reason.

So here is the Metal shader code for Mac (SF_METAL_SM4):

	v13.x = (((v12.x*Material.Material_ScalarExpressions[0].y)+(-sqrt(ps1.sample(s0, (v10+Material.Material_VectorExpressions[11].xy)).x)))*Material.Material_ScalarExpressions[0].z);

It just samples the texture and then uses the value. Now here is the texture sample code for the same section of the material but for iOS (SF_METAL):

	float4 v26;
	v26.xyzw = ps1.sample(s0, (v11+Material.Material_VectorExpressions[11].xy));
	float4 v27;
	v27.xyzw = v26;
	if (Frame.Frame_MobilePreviewMode>0.50000000)
	{
		v27.xyz = pow(v26.xyz,float3(0.45454544,0.45454544,0.45454544));
	}
	v27.xyz = (v27.xyz*v27.xyz);

Note the last line where is just seems to square the value. Putting square roots after every texture sample in my material fixes the problem.

Looking into it further it seems that ProcessMaterialColorTextureLookup does the squaring because of “sRGB read approximation” but only for ES2 and ES3_1 profiles (I believe iOS Metal is considered ES3_1 feature level?) I’ve fixed it by changing my material to use the LinearColor sampling type (instead of just Color) which just returns the texture value.

Thanks for your help Smedis. I’ve learned a lot about how materials get turned into shaders and debugging those shaders! :smiley: