x

Search in
Sort by:

Question Status:

Search help

  • Simple searches use one or more words. Separate the words with spaces (cat dog) to search cat,dog or both. Separate the words with plus signs (cat +dog) to search for items that may contain cat but must contain dog.
  • You can further refine your search on the search results page, where you can search by keywords, author, topic. These can be combined with each other. Examples
    • cat dog --matches anything with cat,dog or both
    • cat +dog --searches for cat +dog where dog is a mandatory term
    • cat -dog -- searches for cat excluding any result containing dog
    • [cats] —will restrict your search to results with topic named "cats"
    • [cats] [dogs] —will restrict your search to results with both topics, "cats", and "dogs"

Accessing camera images

Hello everybody,

Does anyone know a way to programmatically access the images rendered by a camera ?

I do not want to change the player's viewport or influence the environment in any kind.

The only thing I need is an object (camera) to place somewhere in the game which continously produces images from it's point of view, that I can get access to in c++

I really hope you'll be able to help me somehow.

Product Version: UE 4.10
Tags:
more ▼

asked Sep 12 '16 at 12:54 PM in C++ Programming

avatar image

TrackLine
3 1 2 5

(comments are locked)
10|2000 characters needed characters left

3 answers: sort voted first

Second attempt - I wrote this code especially for you, so don't tell me I'm lazy pasting all code here :)

Header file:

 #pragma once
 
 #include "GameFramework/Actor.h"
 #include "MyActor.generated.h"
 
 UCLASS()
 class MYPROJECT_API AMyActor : public AActor
 {
     GENERATED_BODY()
     
 public:    
     // Sets default values for this actor's properties
     AMyActor();
 
     // Called when the game starts or when spawned
     virtual void BeginPlay() override;
     
     // Called every frame
     virtual void Tick( float DeltaSeconds ) override;
 
     USceneCaptureComponent2D *Camera;
     UTextureRenderTarget2D *RenderTarget;
     UTexture2D *Texture2D;
     
     
 };

.cpp file:

 #include "MyProject.h"
 #include "Runtime/Engine/Classes/Engine/TextureRenderTarget2D.h"
 #include "MyActor.h"
 
 
 // Sets default values
 AMyActor::AMyActor()
 {
     PrimaryActorTick.bCanEverTick = true;
     RootComponent = CreateDefaultSubobject<USphereComponent>(TEXT("Root"));
     Camera = CreateDefaultSubobject<USceneCaptureComponent2D>(TEXT("Camera"));
     ConstructorHelpers::FObjectFinder<UTextureRenderTarget2D> RenderTargetAsset(TEXT("/Game/MyLittleRenderTarget"));
     //here you need to have prepared MyLittleRenderTarget asset, type RenderTarget2D. You can have one for many actors, it is duplicated. What is not resolved by me: i don't know if it is stable solution or it will make crash after many calls
 
     RenderTarget = DuplicateObject(RenderTargetAsset.Object, NULL);
     RenderTarget->InitAutoFormat(1024, 1024);
     Camera->TextureTarget = RenderTarget;
 }
 
 void AMyActor::BeginPlay()
 {
     Super::BeginPlay();
     Camera->TextureTarget = RenderTarget; 
 
     int X = RenderTarget->GetSurfaceHeight();
     int Y = RenderTarget->GetSurfaceWidth();
     GLog->Logf(TEXT("Size: %d %d"), X, Y);
     Texture2D = RenderTarget->ConstructTexture2D(this, FString("Tex2D"), EObjectFlags::RF_NoFlags);
     //FTextureRenderTargetResource *Resource = RenderTarget->GetRenderTargetResource();
     int xx = Texture2D->GetSizeX();
     int yy = Texture2D->GetSizeY();
     GLog->Logf(TEXT("Texture size: %d %d"), xx, yy);
 
     FTexturePlatformData *Data = Texture2D->PlatformData;
     EPixelFormat Format = Data->PixelFormat;
     GLog->Logf(TEXT("Pixel format: %d"), (uint8)(Format));
     //format of pixel is PFloatRGBA
 
     int size = Data->Mips[0].BulkData.GetElementSize();
     int N = Data->Mips[0].BulkData.GetElementCount();
     GLog->Logf(TEXT("Number of elements: %d, size of one element: %d"), N, size);
     //i've got 1 byte size of element
 
     const void *Table = Data->Mips[0].BulkData.LockReadOnly();
     Data->Mips[0].BulkData.Unlock();
 
     const uint16 *Tab2 = StaticCast<const uint16*>(Table);
     //ok, quick calculation. I get 8*1024*1024 bytes from 1024*1024 pixels, so one pixel got 8 bytes of data. Format is RGBA, so you can figure it yourself :)
     for(int i=0;i<xx;i++)
         for (int j = 0; j < yy; j++) {
             int k = 4*(i*yy + j);
             int R = Tab2[k];
             int G = Tab2[k + 1];
             int B = Tab2[k + 2];
             int A = Tab2[k + 3];
         }
 
 }

I guess that's what you wanted, you get info about raw RGBA data. Probably it should work with access every frame - if not, you should probably look for function like RenderTarget->UpdateResourceImmediate();

more ▼

answered Sep 14 '16 at 08:00 AM

avatar image

mortmaire
132 12 10 22

avatar image TrackLine Sep 14 '16 at 12:47 PM

Thank you very much !

I really appreciate your help and effort.

This finally lead me to a satisfying solution.

avatar image mortmaire Sep 14 '16 at 04:07 PM

I'm glad I could help. When I started (~2 months ago) I spend really long time trying to understand those things and I didn't had any example codes. So let's make it simpler for people after me :D

avatar image nlholdem Dec 13 '17 at 10:36 PM

Hello, sorry to resurrect an old question, but can you give some guidance on how you created your rendertarget asset? I presume you don't just mean adding a var to the Level Blueprint, you mean an actual asset? How is this done? Thanks!

avatar image nlholdem Dec 13 '17 at 10:56 PM

Ach forget that - you mean a literal texture of type RenderTarget. Thanks for your great answer!

(comments are locked)
10|2000 characters needed characters left

Another way is to use a SceneCapture2D, the actor class which is surprisingly different from common Camera actor class (they does not inherit from each other). Here is some code from my project, hope it'll help.

 PortalCamera = CreateDefaultSubobject<USceneCaptureComponent2D>(TEXT("Camera"));
 //(...)
 ConstructorHelpers::FObjectFinder<UTextureRenderTarget2D> RenderTarget(TEXT("/Game/PortalTexture"));
 PortalView = DuplicateObject(RenderTarget.Object, NULL);
 PortalView->InitAutoFormat(1024, 1024);

 UTexture *AuxiliaryTexture = static_cast<UTexture*>(PortalView);
 Texture = static_cast<UTexture2D*>(AuxiliaryTexture);

in BeginPlay():

 PortalView->UpdateResourceImmediate();
 UMaterialInstanceDynamic* RV_MatInst = UMaterialInstanceDynamic::Create(Material, NULL);
 RV_MatInst->SetTextureParameterValue(FName("T2DParam"), Texture);
 PortalMesh->SetMaterial(0, RV_MatInst);

 //very important!
 PortalCamera->TextureTarget = PortalView;
 if (SecondPortal) {
     PortalCamera->TextureTarget = SecondPortal->GetPortalView();
 }

here are types of objects in header:

 USceneCaptureComponent2D *PortalCamera;
 UStaticMeshComponent *PortalMesh;
 UTextureRenderTarget2D *PortalView;
 UTexture2D *Texture;
 UMaterial *Material;
 UTextureRenderTarget2D *Render;

A little explanation. Here is created a USceneCaptureComponent2D, the camera attached to the portal, which gets us the view. View is send to copied Texture object, and, after start of the game, is attached to copy of material with input for texture. From the level of editor you need to create SceneCapture2D camera, create the Render Target object, set output of SceneCapture to this RenderTarget, and use this specific RenderTarget as a feed for Material. Because I spend some time trying to connect these things in script, I attach it, hoping that it can help :)

more ▼

answered Sep 13 '16 at 02:50 PM

avatar image

mortmaire
132 12 10 22

avatar image mortmaire Sep 14 '16 at 06:33 AM

Ok, soon I'll upload a full solution :)

avatar image RDaneelOlivaw Dec 01 '17 at 04:36 PM

Hi, I'm using your code but for some reason Its generating blanc images or very strange imagesalt text. Which is the configuration needed for the RenderTarget? I've tried RenderTarget formats: RF RGBA16f and RF RGBA8. First one generated empty images and the second random images always the same. Any idea?

alt text

cam4.png (1.0 MB)
cam2.png (1.3 MB)
(comments are locked)
10|2000 characters needed characters left

If I understand the question you would like to spawn a camera at a specific location to capture an image?

If this is so, you could try spawning an actor of class _camera in the desired location and reference the camera you spawned to get the image you want.

Hope this leads to a solution

more ▼

answered Sep 12 '16 at 02:17 PM

avatar image

LiquidJoy
23 9 15 17

(comments are locked)
10|2000 characters needed characters left
Your answer
toggle preview:

Up to 5 attachments (including images) can be used with a maximum of 5.2 MB each and 5.2 MB total.

Follow this question

Once you sign in you will be able to subscribe for any updates here

Answers to this question