Is there any way to access the raw data of a UMediaTexture?

Development Cinematics & Media
, , ,
1

Dear all,

I am trying to access the raw data (RGBA) of a UMediaTexture in order to modify its Alpha component using my own chromakey calulations. I set it up as a bluenote receiving as Input a VideoTexture and returns as output the modified videotexture. The VideoTexture itself is of class UMediaTexture.

There are examples using UTexture2D reading the bulk data but I fail to go from UMediaTexture to UTexture2D. UMediaTexture and UTexture2D have a common parent UTexture, so they are both siblings(?) so casting is not the way. :slight_smile:

I assume that the FTextureReference TextureReference ist the correct starting point ? But it seems, that this is a dead end too.

Any hint for me pointing me into the right direction to read the raw data from a UMediaTexture?

Kind regards
Andreas

2

The data of the media texture resides on the GPU. To modify/access it on the CPU, you’d have to lock it, which transfers it into CPU memory, then process it, then unlock it, which transfers it back to the GPU. This is very slow and not recommended.

Instead, I recommend that you use a Material to manipulate the texture in a shader. You can access the texture with a Texture Sample node. You’ll have the entire Material system available to process the pixels in any way you want. Most importantly, the shader will be executed on the GPU, so no memory needs to be copied, and it will also process much faster.

3

Thanks… I will dig myself into materials and shaders :slight_smile:

4

This wont work at all, since TextureParameter2D you set in Material will still have the UMediaTexture as its texture configured when you access it at run time using GetTextureParameterValue on dynamic material instance. So you cannot access the raw data as you could easily do with UTexture2D reference pointer. Unfortunately unlike UTexture2D, UMediaTexture is not equipped with functionality to give you access. You will have to go through GPU route until and unless UMediaTexture gets inherited from UTexture2D.

5

In case anyone need to get raw data from UMediaTexture, here is the example:

void ReadColorFromTexture(UMediaTexture* Texture, TArray<FColor> &OutColors)
{
    FMediaTextureResource* texResource = static_cast<FMediaTextureResource*>(Texture->Resource);
    texResource->ReadPixels(OutColors);
}
1 Like