4.20.2 Stencils & CustomDepth not working

Development Rendering
, , , , , ,
1

4.20.2 stencils and custom depth does not work if called with c++ or blueprints. Only works if directly set in the editor.

2

I think i have found out the problem in Engine Version 4.20.2. While Enabling RenderCustomDepth in the Editor enables the flag “bRenderCustomDepth” in the primitive component, and it works just fine. Now if we use these functions

  void UPrimitiveComponent::SetRenderCustomDepth(bool bValue)
  void UPrimitiveComponent::SetCustomDepthStencilValue(int32 Value)

The functions call a game thread function and enqueues Render thread function to set the variable “bRenderCustomDepth” in Render thread (IDK why). The functions used for this are

  void FPrimitiveSceneProxy::SetCustomDepthEnabled_GameThread(const bool bInRenderCustomDepth)
  void FPrimitiveSceneProxy::SetCustomDepthEnabled_RenderThread(const bool bInRenderCustomDepth)

This for some reason does not work. But if i set the bRenderCustomDepth Directly as in previous engine version like this

  void UPrimitiveComponent::SetRenderCustomDepth(bool bValue)
  {
      if (bRenderCustomDepth != bValue)
      {
          bRenderCustomDepth = bValue;
          MarkRenderStateDirty();
      }
  }
  
  void UPrimitiveComponent::SetCustomDepthStencilValue(int32 Value)
  {
      // Clamping to currently usable stencil range (as specified in property UI and tooltips)
      int32 ClampedValue = FMath::Clamp(Value, 0, 255);
  
      if (CustomDepthStencilValue != ClampedValue)
      {
          CustomDepthStencilValue = ClampedValue;
          MarkRenderStateDirty();
      }
  }

This works and stencils now work properly. The only place this flag “bRenderCustomDepth” is used is in “SceneVisibility.cpp” in line 1831 and 1994. I am not sure if these places are in a function called by the render thread. Would like to know from a UE4 staff about this. But for now setting it from the game thread works fine without crashes in the editor. (Will check built version later).