Create binary segmentation images based on object flags

My goal is to use images generated with unreal engine as input and ground truth data to image processing algorithms. I have actors moving around in the scene and means to export the captured images. Now I want to create segmentation images in which the moving actors are represented by white pixels while everything else is black (like in the following examples).

Unfortunately I don’t really have an idea where to start (e.g. if it is a custom shader or rather done in post processing).

Just using a white material which does not respond to lighting for the actors and a black one for everything else does not seem feasible. I’d like to be able to assign a flag to an actor which causes it to be displayed as white. Ideally the render mode should be switchable so that I can recrod a scene once in its regular look and once in segmentation mode.

We already have a custom SceneCapture to simulate the effects of a real video camera - maybe it can be extended to record in segmentation mode. I am grateful for any suggestion how these goals can be achieved.

Thank you.

What you probably need to do is create a post-process material with ‘SceneTexture’ node which you should set to CustomStencil in the dropdown in its properties. You should also enable Render Custom Depth on all static meshes in the scene and assign to each of them a different Custom Stencil (or assign them based on object type or whatever you want the objects to be segmented to).

Then what you do is create apply this post process material to a post process volume. As it’s quite hard to change post process materials in post process volumes from blueprints, as far as I know, it might be easier to just create a separate post process volume and switch between them. I’m posting a few screen shots below:

This is the node-graph of segmentation post-process material. The starting node on the left is Scene Texture with CustomStencil.

This is where I find all the static meshes in the scene and assign a random custom stencil between 10-255. I kept the stencils below 10 reserved for the landscape and the sky.

213567-screenshot-2017-09-15-091433.png

The properties of a static mesh in the rendering part. This is where you enable Custom depth rendering and where you can assign a custom stencil.

This is the final look. I’ve manually set the landscape’s custom stencil to 1, which is rendered green (using bitmask node) and the sky is set to custom stencil 0 (you can’t change that actually), which is rendered black.

I hope this helps.

And don’t forget to enable post processing rendering with stencil as described here:

Hi~
Is it still work under 4.18 version? Could you please offer more details about how to setup the post-process material? Thanks.

I wouldn’t know about the 4.18 version. I am still working on 4.16.2 for this project.

In regards the post-process material. Have you checked the screenshot above?

Thank you for the reply~ I had and I got this now. I can’t see any segmentation thing.

I still have two problems now. The first one is the segmentation area has different RGB data. For instance, I may get (192,33,43) and (193,33,45) under the same mesh. So it’s hard to use it.
The second one is I can’t initialize the custom depth stencil because of error below.

It seems you are trying to convert actor references to static meshes. You have to select static mesh actor in the dropdown of Get All Actors of Class node.

Thank you mitjaprelovsek!
Here are some sources from the documentation with some additional information on the topic:
Post Process Materials Demo
Post Process Materials Overview