New RHI backend Architecture

[Pikachuxxxx]

Currently we use virtual functions to abstract over different APIs, but the new idea is that we use function pointers and drop the number of class files into just a few. We keep the Renderer module to a .dll and enable MT over the engine, and hot load it.

lowlevel:
The new API will be split into these main categories:
Device (resource allocation functions), Context, Memory, Utility, ShaderReflection, DescriptorHandler, GraphicsStructs(Impl),
IRenderContextPolicy (flow + util)/RenderUtilities/RenderContext

THIS DESIGN IF NEEDED MORE CONTROL ONLY-->UE6.

IRenderContextPolicy (readymade util functions + to create a specified structure, ex. num of ring buffers, how to sync diff threads/queues, manage res/mem budgets) with ISubTypePolicy (ex. QueueSyncAsyncCompute: public ISubTypePolicy) as further abstract able customization points.

These will be their files, we use as much type safety as possible and manage their CPU memory into simple pools, keep all the high-level and low-level data as structs. Multi-threaded will be handled inside RenderContext.cpp. High-Level will have this file, that describes how the queue sync, frame sync etc. works. Diana Renderer will supply the meshes to this pipeline, helps with drawing by binding necessary per-mesh resources (UBOs, Updating them, Transforms calculation, ShadersParams etc.)

RenderContextImple ex has, ring buffer for main grapihsc/compute commands--> multi-threaded safe functions, Asynchronous compute on different CPU thread + their GPU sync logic, how/where memo comes from, memory budget checks etc.

Diana converts Scene Graph to RHI commands.

FrameGraph runs what then???-->Diana::drawScene(SceneHandle); SceneHandleCache, just for draw calls generation, pass specific stuff RHI commands are inside the FG, like SSAO resources, FXAA info if we want scene and meshes and materials to be resolved use drawSceen for more customization use Diana::sceneDrawOverrides(!SceneOverrides) ex. disableMaterialLoad/Binding for DepthPass

Final Design:
WorldRenderer Calls -> FG --> (Diana + RHI) --> FrameSync/QueueManagement/MT inside RenderContext (FIXED DESIGN) --> bunch of command buffers are submitted all at once by RHI/Diana (FIXED DESIGN) +++ Delay GPU sync options management strategy per platform --> VK_RenderContext.cpp --> other same level, Make them only one way visible top to bottom API.

highlevel:
Diana, GraphicsStructs, RHI

Proto1:
port all but we don't have to refactor RenderContext/Diana yet just write-RHI in C API and call those functions instead and re-write DescriptorSetHelper in C-style. or keep it as but remove poly morphism. Fast refactor prototype. for low-level RHI implementation. For low-level RenderContext it has non-virtual functions that's it. we won't add any new fancy features like MT it's a simple ring buffer submitting n-per frame on a single thread. just as it is now.

Remove repetition shit in FG API, like RenderingInfo, can be automatically deduced, keep the Executre function more simple and less extensible?? + FreePassAPI template overload!!!

[Pikachuxxxx]

New API Design

C-Style Function Dispatch Table

// Functions pointers for API specific Implementation
typedef void (*RHI_InitAPIFunc)(void);
typedef void (*RHI_AcquireImageFunc)(RZSemaphore* signalSemaphore);
typedef void (*RHI_BeginFunc)(RZDrawCommandBufferHandle cmdBuffer);
typedef void (*RHI_SubmitFunc)(RZDrawCommandBufferHandle cmdBuffer);
typedef void (*RHI_SubmitWorkFunc)(std::vector<RZSemaphore*> waitSemaphores, std::vector<RZSemaphore*> signalSemaphores);
typedef void (*RHI_PresentFunc)(RZSemaphore* waitSemaphore);
typedef void (*RHI_BindPipelineFunc)(RZPipelineHandle pipeline, RZDrawCommandBufferHandle cmdBuffer);
typedef void (*RHI_BindDescriptorSetFunc)(RZPipelineHandle pipeline, RZDrawCommandBufferHandle cmdBuffer, const RZDescriptorSet* descriptorSet, u32 setIdx);

// Function table struct for RHI API
typedef struct RHI_API {
    RHI_InitAPIFunc               InitAPI;
    RHI_AcquireImageFunc          AcquireImage;
    RHI_BeginFunc                 Begin;
    RHI_SubmitFunc                Submit;
    RHI_SubmitWorkFunc            SubmitWork;
    RHI_PresentFunc               Present;
    RHI_BindPipelineFunc          BindPipeline;
    RHI_BindDescriptorSetFunc     BindDescriptorSet;
} RHI_API;

// Global function table
static RHI_API g_RHI_API;

// API functions
void RHI_Create(u32 width, u32 height);
void RHI_Release(void);
void RHI_InitAPI(void);
void RHI_AcquireImage(RZSemaphore* signalSemaphore);
void RHI_Begin(RZDrawCommandBufferHandle cmdBuffer);
void RHI_Submit(RZDrawCommandBufferHandle cmdBuffer);
void RHI_SubmitWork(std::vector<RZSemaphore*> waitSemaphores, std::vector<RZSemaphore*> signalSemaphores);
void RHI_Present(RZSemaphore* waitSemaphore);
void RHI_BindPipeline(RZPipelineHandle pipeline, RZDrawCommandBufferHandle cmdBuffer);

Backend Implementation and Wrapper

void Vulkan_InitAPI() {
   g_Context.Init();
   g_Device.Init();
}

void Vulkan_AcquireImage(RZSemaphore* signalSemaphore) {
    vkAcquireNextImageKHR(...);
}

Selecting API func pointers (dynamic assignment)

void RHI_Create(u32 width, u32 height) {
    g_RHI_API.InitAPI = Vulkan_InitAPI;
    g_RHI_API.AcquireImage = Vulkan_AcquireImage;
    g_RHI_API.Begin = Vulkan_Begin;
    g_RHI_API.Submit = Vulkan_Submit;
    g_RHI_API.Present = Vulkan_Present;

    g_RHI_API.InitAPI(); // Initialize the backend
    printf("RHI created with width=%u, height=%u.\n", width, height);
}

High-Level RHI API function wrappers

void RHI_InitAPI(void) {
    g_RHI_API.InitAPI();
}

void RHI_AcquireImage(RZSemaphore* signalSemaphore) {
    g_RHI_API.AcquireImage(signalSemaphore);
}

void RHI_Begin(RZDrawCommandBufferHandle cmdBuffer) {
    g_RHI_API.Begin(cmdBuffer);
}

void RHI_Submit(RZDrawCommandBufferHandle cmdBuffer) {
    g_RHI_API.Submit(cmdBuffer);
}