Visualizing CT data (signed 16bit integer)

Development Asset Creation
, , ,
1

I’d like to visualize DICOM CT data on Unreal Engine 4.

In most cases, CT data is expressed as a sequence of 16bit signed integer. And to visualize the data, we have to set Window Center, and Window Width.

So I’d like to use 16bit signed integer array for UTexture2D and map the color at runtime through the dynamic material.

Unreal Engine 4 seems to be able to handle the 16bit signed integer at least on the code level, by setting PF_R16_SINT.

Here is the code:

  UTexture2D* dicom_texture;
  
  TArray<short> signed_pixels = ReadSampleData();

  int32 width = 512;
  int32 height = 512;
  dicom_texture = UTexture2D::CreateTransient(width, height, PF_R16_SINT); // In D3D11, PF_R16_SINT = DXGI_FORMAT_R16_SINT: A single-component, 16-bit unsigned-integer format that supports 16 bits for the red channel.
  dicom_texture->CompressionSettings = TextureCompressionSettings::TC_Grayscale;
  dicom_texture->Filter = TextureFilter::TF_Bilinear;
  dicom_texture->SRGB = false;

  void* locked_bulk_data = dicom_texture->PlatformData->Mips[0].BulkData.Lock(LOCK_READ_WRITE);
  FMemory::Memcpy(locked_bulk_data, signed_pixels.GetData(), sizeof(short) * signed_pixels.Num());
  dicom_texture->PlatformData->Mips[0].BulkData.Unlock();

  dicom_texture->UpdateResource();

And here is the material I made to handle Window Center and Window Width.

1472×549 181 KB

However the result is the following.

463×502 36.9 KB

The image should be like this (opend in another software (ImageJ)).

514×514 82.5 KB

I wonder if PF_R16_SINT is controllable via a material blueprints. I’ve tried changing the texture compression settings from TC_Grayscale to TC_Displacementmap but the result was the same.

Do I have to write some very complicating shader and the associated C++ code (ENQUEUE_UNIQUE_RENDER_COMMAND_… or something like that) ?

I’ve uploaded the entire project to the following GitHub.

Thanks.

2

PF_R16_SINT does not seem to be supported when the UTexture2D with PF_R16_SINT is exposed to Material Blueprints. So I tried with PF_G16 and it seems to work well.

However this is not the best solution because PF_G16 = DXGI_FORMAT_R16_UNORM according to the source code, and the DXGI_FORMAT_R16_UNORM is “A single-component, 16-bit unsigned-normalized-integer format that supports 16 bits for the red channel” as described here ([DXGI_FORMAT (dxgiformat.h) - Win32 apps | Microsoft Docs][1]).

So I had to shift each value from short (-32768 to 32767) to unsigned short (0 ～ 65535) by adding 32768 to store each value to DXGI_FORMAT_R16_UNORM, and shift back in Material Blueprints.

There does exists DXGI_FORMAT_R16_SNORM which supports 16-bit signed-normalized-integer but Unreal Engine does not seem to support this pixel format.

In addition, I had to convert the automatically converted (calculated) sRGB value to linear RGB value using the formula described here ([A close look at the sRGB formula][2]), because the visualization of medical images does not use sRGB color space in general.

I looked for the option for not applying sRGB color to the final color output, but I could not find the option unfortunately. Is there anyone who knows the option for it?

And I would like to know if there are better solutions.

Anyway, here is the material I modified,

1571×697 185 KB

and the visualized CT image on Unreal Engine 4.

641×607 156 KB

The image is exactly the same as the image visualized by ImageJ!

3

If you create a texture as R16/32_UINT or R16/32_SINT, you can read from it using a custom node.
Put this code in a custom node and return it - will get converted to a float and be useable.

int LabelValue = asint(VolumeTexture.Load(int4(CurPos, 0)));
 return LabelValue;

Of course, you can make it into a one-liner. For 2D textures, it will be

int LabelValue = asint(Texture.Load(int3(CurPos, 0)));

And for unsigned, use

asuint();

I tested it with 16 and 32 bit signed and unsigned and it worked with all of them.
Hope this helps somebody, I wasted way too much bloody time to figure this out :smiley:

Bear in mind that there is no interpolation on integer textures (that’s why you have to use Load instead of Sample), so results might be uglier than you expect :slight_smile: