Problem: GltfLoad.cpp hides data about nodes transformation.

[vrazumov]

I'm trying to use RaZ in a simulator project.

I need the ability to transform Meshes imported from a Gltf file in Runtime.

For example:

We have a model of a clock:

node1 {transform, mesh} - watch case
node2 {transform, mesh} - hour hand
node3 {transform, mesh} - minutes hand
node4 {transform, mesh} - seconds hand

After exporting the model, you need to be able to:

model.getNodeTransform(node2.name).setRotation(hours_value)
model.getNodeTransform(node2.name).setRotation(minutes_value)
model.getNodeTransform(node2.name).setRotation(seconds_value)

Possible Solution.

In RaZ:

Add two modules: GltfSceneFormat (from GltfLoad.cpp) SceneRenderer (new component).

// GltfSceneFormat.cpp

void loadVertices(...)
{
...
//if (transform.has_value()) {
// for (Vertex& vert : submesh.getVertices()) {
// vert.position = transform->getPosition() + transform->getRotation() * (vert.position * transform->getScale());
// vert.normal = (transform->getRotation() * vert.normal).normalize();
// vert.tangent = (transform->getRotation() * vert.tangent).normalize();
// }
//}
}

std::vector<std::optional> loadTransforms(
SceneRenderer & sceneRenderer,
const std::vectorfastgltf::Node& nodes,
std::size_t meshCount)
{
std::vector<std::optional> transforms;
transforms.resize(meshCount);

for (const fastgltf::Node& node : nodes)
{
Transform transform = loadTransform(node);
sceneRenderer.m_transforms.emplace_back(transform);
//computeNodeTransform(node, std::nullopt, nodes, transforms);
}

return transforms;
}

void loadNodes(SceneRenderer & sceneRenderer, const std::vectorfastgltf::Node& nodes)
{
unsigned long transformIndex = 0;

for (auto & node_ : nodes)
{     
    SceneRenderer::Node node;
    node.name = node_.name.c_str();
    node.transformIndex = transformIndex;
    transformIndex++;
    if (node_.meshIndex.has_value())
        node.meshIndex = node_.meshIndex.value();
    for (auto child : node_.children)
    {
        node.children.emplace_back(child);
    }
    sceneRenderer.m_nodes.emplace_back(node);
}

}

std::pair<Mesh, MeshRenderer> loadMeshes(...)
{
...
Mesh loadedMesh;
MeshRenderer loadedMeshRenderer;
for (const fastgltf::Primitive& primitive : meshes[meshIndex].primitives) {
...
Submesh& submesh = loadedMesh.addSubmesh();
SubmeshRenderer& submeshRenderer = loadedMeshRenderer.addSubmeshRenderer();
...
loadedMeshRenderer.getMaterials().emplace_back(std::move(Material(MaterialType::COOK_TORRANCE)));
...
}
...

}

SceneRenderer load(const FilePath& filePath) {
...
const std::vector<std::optional> transforms = loadTransforms(sceneRenderer, asset->nodes, asset->meshes.size());

loadNodes(sceneRenderer, asset->nodes);

for (auto & node : asset->nodes)
{
if (node.meshIndex.has_value())
{
auto [mesh, meshRenderer] = loadMeshes(asset->meshes, node.meshIndex.value(), asset->buffers, asset->bufferViews, asset->accessors, transforms);
sceneRenderer.m_meshRenderers.emplace_back(std::move(meshRenderer));
}
}

const std::vector<std::optional> images = loadImages(asset->images, asset->buffers, asset->bufferViews, parentPath);

for (auto & meshRenderer : sceneRenderer.m_meshRenderers)
loadMaterials(asset->materials, asset->textures, images, meshRenderer);
...
return sceneRenderer;
}

And:
// RendererSystem.cpp
void RenderSystem::linkEntity(const EntityPtr& entity) {
...
}

void RenderSystem::initialize() {
// add GltfRenderer
registerComponents<Camera, Light, MeshRenderer, SceneRenderer>();
...
// add GltfRenderer
if (entity->hasComponent())
{
auto & gltfrenderer = entity->getComponent();
//for (auto & meshrenderer : gltfrenderer.getMeshRenderers())
for (auto & meshrenderer : gltfrenderer.m_meshRenderers)
updateMaterials(meshrenderer);
}

}

// RenderGph.cpp
void RenderGraph::execute(RenderSystem& renderSystem) {
...

// add GltfRenderer
}
else if (entity->hasComponent())
{
if (!entity->isEnabled() || !entity->hasComponent())
continue;

const auto& gltfRenderer = entity->getComponent<SceneRenderer>();

if (!gltfRenderer.isEnabled())
  continue;
auto m_ = entity->getComponent<Transform>().computeTransformMatrix();
renderSystem.m_modelUbo.sendData(entity->getComponent<Transform>().computeTransformMatrix(), 0);
auto m = entity->getComponent<Transform>().computeTransformMatrix();
gltfRenderer.draw(renderSystem.m_modelUbo, m);

}

In App:

Raz::Entity& clock = world.addEntityWithComponentRaz::Transform();
auto & clockRenderer = clock.addComponentRaz::SceneRenderer(Raz::GltfSceneFormat::load(RAZ_ROOT "assets/watch_case.gltf"));
auto hours = clockRenderer.getTransformNode("hours_hend");
auto minutes = clockRenderer.getTransformNode("hours_hend");
auto seconds = clockRenderer.getTransformNode("hours_hend");

app.run([&] (float deltaTime) {
// get current hours_value, minutes_value, seconds_value
hours.value()->setRotation(hours_value);
minutes.value()->setRotation(minutes_value);
seconds.value()->setRotation(seconds_value);
});

Example implementation in attached files:

This really works.

You are free to use this code as you wish.

Vladimir Razumov.

updatedfiles.zip

[Razakhel]

There are two separate issues here:

1. The glTF loader merges all meshes from the file

Indeed, at the moment I always assume that a glTF file references a single mesh as a whole. As it is a scene format this is obviously wrong, and I considered adding several functions allowing to load every single feature independently. For example:

// Will load all meshes, light sources, ... and return all of them as entities
//  and related components in a dedicated world
Raz::World loadScene(...);

// Will load all meshes as a node hierarchy
// The return type will probably be either Raz::Graph<std::pair<Raz::Mesh, Raz::MeshRenderer>>,
//  Raz::Graph<Raz::Entity>, or any other means of recovering the hierarchy
??? loadMeshes(...);

std::vector<Raz::Light> loadLights(...); // Or most likely Raz::Graph<Raz::Light> too

// Etc.

I don't believe I have to keep the vertices as they are and return their transform though; transforming them in place (and thus making the loaded meshes implicitly have an identity transform) should still be viable, although a bit more costly at loading time. This will have to be decided for good later.

2. There can currently be no hierarchy between objects

This is a feature that has been missing since the engine started, and I still don't really have an ideal solution, although I haven't really tried anything yet. I considered:

• Making an entity hierarchy, which I guess would be functionally ideal; while writing this I actually can't think of any major drawback right now, even performance-wise, so I think this is what I'll try first;
• Making a transform hierarchy, which would be easy to implement as it is localized to transforms only. This is also precisely why I don't like it though, as other components may have uses for parenting too (for example the velocity, which I believe could technically also be applied down the hierarchy; but I guess this is taking it a bit too far for now);
• Add a component dedicated to this, which would allow for maximal flexibility but would most likely take a toll on performance, notably due to data locality.

In any case, applying hierarchical transforms each frame would also have an impact on performance, which I'll have to be wary of. I can think of a few solutions right now that I'll have to try out.

---

Thank you for your code too! I see what you're doing, but I would rather avoid having very specific structures just for this in the engine. If your solution works for you, you can keep using it for now; I don't know when I will manage doing something that works well. Please let me know if this is something a bit critical for your workflow, and/or if you have any other issue with the engine!

As a side note, you seem to have gotten started with RaZ really fast. If you have time, and of course if you want to (no worries if not), don't hesitate to give your feedback regarding what you found either easy or hard with the API, the existing example code, the documentation/wiki (which is admittedly quite sparse), or anything else you can think of!