Slowly Read A RenderTarget

Overview

Epic has done a good job of giving UTextureRenderTarget2D the capacity of being used to read texture data out of an arbitrary viewport in the world seen through a USceneCaptureComponent2D. The biggest problem being that doing so is certainly going to have a severe cost to performance, so much so that users could see >200ms hitches in gameplay. This is a short tutorial outlining a simple UObject wrapper I wrote to slowly scan the surface of a render target from the render thread and return the resulting image on the main thread. The purpose here is to get all of the texture data from the render target in a fairly short period of time without overloading the render thread, and keeping hitches to a minimum.

It is assumed if you are reading this you know how to set up a render target and capture component and just want to reduce the lag spike getting a texture out of them creates.

How do we do this

In order to get a texture without interrupting the game we need to...

1. Create a reader object on the game thread that will handle watching and reporting on the read status of the render target
2. Use the reader object figure out how many parts of the texture need reading.
3. Read each segment over a period of time that the render thread can handle. Polling for when to read the next segment from the main thread.
4. Report the data we found from the render thread so that game classes can use it.

Reading from a Render Target

FTextureRenderTargetResource::ReadPixels() will take a few arguments, one of which is a rectangle that you can form to the image it is reading off of and use to scan over the image over a period of time, which is much less taxing on the render thread. It is very important that the calling code creating and using this object restrain the capture component from writing to the render target during this time, as it will be over possibly many frames. This can make a very artistic looking result, but is likely not the desired behavior.

Calling Code

if (!RenderTargetReader)
{
        // You could stop your capture now, refreshing if need be
    RenderTargetReader = URenderTargetReader::Create(this, RenderTarget);
    RenderTargetReader->OnPixelsReady.BindLambda([this](TArray Pixels, int32 Width, int32 Height)
    {
        //@TODO - enjoy your pixels
                // You could start your capture again now, we have all the data we need
        RenderTargetReader = nullptr;
    });
    RenderTargetReader->StartRead();
}

.h

DECLARE_DELEGATE_ThreeParams(FOnTargetPixelsRead, TArray, int32, int32);

UCLASS()
class URenderTargetReader : public UObject
{
    GENERATED_BODY()
public: 
    virtual UWorld* GetWorld() const override { return GetOuter()->GetWorld(); }
    static URenderTargetReader* Create(UObject* WorldContext, UTextureRenderTarget2D* Target);
    void StartRead();
    FOnTargetPixelsRead OnPixelsReady;

protected:
    URenderTargetReader();
        void ReadNextPixels();
    void SwapInPixels();
    void CheckFence();
    void StopReading();
    bool IsFinishedReading();

        FRenderCommandFence* RenderFence;
    FRenderTarget* RenderResource;
    FTimerHandle PollingHandle;
    
    TArray ReadingPixels;
    TArray ResultPixels;
    
    int32 SeekY;
    int32 SizeY;
    FIntRect ReadingRectangle;
};

.cpp

You will note that in the SwapInPixels() function We are changing R for B from the source channels, the reason for this is that we are using the data to save .jpg files for the user locally, and this may not be needed for all implementations.

URenderTargetReader::URenderTargetReader()
{
    SeekY = 0;
    SizeY = 8;
}

URenderTargetReader* URenderTargetReader::Create(UObject * WorldContext, UTextureRenderTarget2D * Target)
{
    if (URenderTargetReader* reader = NewObject(WorldContext, "RenderTargetReader"))
    {
        reader->RenderFence = new FRenderCommandFence();
                reader->RenderResource = RenderTarget->GameThread_GetRenderTargetResource();
        reader->ResultPixels.Init(FColor::Magenta, reader->RenderResource->GetSizeXY().X * reader->RenderResource->GetSizeXY().Y);
                
        return reader;
    }
    return nullptr;
}

void URenderTargetReader::StartRead()
{   
    if (!GetWorld() 

 PollingHandle.IsValid())
    {
        UE_LOG(LogTemp, Log, TEXT("Unable to start read for render target, world is nullptr or polling handle is in use"));
        return;
    }   
    GetWorld()->GetTimerManager().SetTimer(PollingHandle, this, &URenderTargetReader::CheckFence, 0.01f, true);
        ReadNextPixels();
}

void URenderTargetReader::CheckFence()
{
    if (!RenderResource)
    {
        StopReading();
        return;
    }
    if (RenderFence->IsFenceComplete())
    {
        SwapInPixels();
        if (IsFinishedReading())
        {
            OnPixelsReady.ExecuteIfBound(ResultPixels, RenderTarget->SizeX, RenderTarget->SizeY);
            StopReading();
        }
        else
        {
            ReadNextPixels();
        }
    }
}

void URenderTargetReader::ReadNextPixels()
{   
    ReadingPixels.Reset();
    
    int32 y = SeekY;
    int32 bottom = SeekY = SeekY + SizeY;
    if (RenderResource->GetSizeXY().Y GetSizeXY().Y;
    }

    struct FReadSurfaceContext
    {
        FRenderTarget* SrcRenderTarget;
        TArray* OutData;
        FIntRect Rect;
        FReadSurfaceDataFlags Flags;
    };
    ReadingRectangle = FIntRect(0, y, RenderResource->GetSizeXY().X, bottom);
    FReadSurfaceContext ReadSurfaceContext =
    {
        RenderResource,
        &ReadingPixels,
        ReadingRectangle,
        FReadSurfaceDataFlags(RCM_UNorm, CubeFace_MAX)
    };

    ENQUEUE_UNIQUE_RENDER_COMMAND_ONEPARAMETER(
        ReadSurfaceCommand,
        FReadSurfaceContext, Context, ReadSurfaceContext,
        {
            RHICmdList.ReadSurfaceData(
                Context.SrcRenderTarget->GetRenderTargetTexture(),
                Context.Rect,
                *Context.OutData,
                Context.Flags
            );
        }
    );
    RenderFence->BeginFence();
}

void URenderTargetReader::SwapInPixels()
{
    for (int32 y = 0; y < ReadingRectangle.Height(); y++)
    {
        for (int32 x = 0; x < ReadingRectangle.Width(); x++)
        {
            int32 targetX = ReadingRectangle.Min.X + x;
            int32 targetY = ReadingRectangle.Min.Y + y;
            const uint32 targetindex = targetY * RenderResource->GetSizeXY().X + targetX;
            uint32 readingindex = y * ReadingRectangle.Width() + x;
            ResultPixels[targetindex].R = ReadingPixels[readingindex].B;
            ResultPixels[targetindex].B = ReadingPixels[readingindex].R;
            ResultPixels[targetindex].G = ReadingPixels[readingindex].G;
            ResultPixels[targetindex].A = ReadingPixels[readingindex].A;
        }
    }
}

bool URenderTargetReader::IsFinishedReading()
{
    return RenderTarget->SizeY < SeekY + SizeY;
}

void URenderTargetReader::StopReading()
{
    if (GetWorld())
    {
        GetWorld()->GetTimerManager().ClearTimer(PollingHandle);
    }
    ConditionalBeginDestroy();
}

Enjoy Your Pixels

Doing things with the pixels is somewhat out of the scope of this document, but you could easily imagine using this to quietly save an image the user requested from a point of view outside of their HUD. You could slowly refresh a birds-eye view of your map, or you could even use the pixels for more data driven gameplay, the choice is yours!

PS:

This is my first full article on here and I welcome any edits people may have for me, as I do not have a lot of time to write or maintain documents such as this. Also, SUPER big thanks to for his article:

It was the starting point for writing this article and a huge help. I certainly hope someone makes something great with this. :)