Welcome to dmt-meshes Discussions! Please share your findings

[mosesalan]

Share your findings! so we can all discover!

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[AugmentedRealityCat]

First, thanks a lot for making this !
Here is a little video I made from my first "good enough" model after many experiments to better understand how this was all working.

DMT-mesh_dino_tyraV11.mp4

The model itself was synthesized using DMT-Meshes from the following parameters:
text prompt: Tyrannosaurus Rex
Image to PC model: 1B model
PC to mesh method: OpenAI
Steps: 10 000
Mesh rendering frequency: 500
Step Preview: None
Mesh learning rate: 0.0005
Regulization weight: 0.30
Gridres: 128
Multires: 4
Run Mode: Text to Mesh

And here are pictures of the raw poly mesh, and of the textured version after using the wonderful Dream Texture add-on from the equally wonderful @carson-katri

[AugmentedRealityCat]

Follow-up: I discovered I had been missing a key piece of information. Basically, I produced like a hundred models and ranked them and grouped them according to different parameters I was setting, because I wanted to learn what each parameter meant and how to use them. And my conclusion was that it did not make any sense. And in a sense I was right: all those parameters I was playing with, all but the resolution parameter in fact, are reserved exclusively for use with the DMTET method, and I never used that method in my tests, I only used OpenAI ! So all those parameters I was playing with methodically were doing nothing at all, and all I was observing was the natural variation from the text2pointcloud part, where I suppose there is a random seed or something similar varying the results even if you reuse the exact same text as a prompt.

Just in case you would miss that very important piece of information, and other important instructions, here is the link:
https://github.com/Firework-Games-AI-Division/dmt-meshes/blob/0.0.1/docs/MESH_GENERATION.md#advanced

And here is a copy of the info I had been missing (I thought the readme was all there was to read !)

Advanced

Image to point cloud model - Pre-trained openAI model to generate mesh from an image.

Point cloud to mesh model - You can use either point cloud to mesh implementation from openAI or DMTet.

Steps, mesh rendering, mesh learning rate, regularization, gridres, multires - parameters for point cloud to mesh. Except gridres, other parameters are only for DMTet. We will revamp the UI in the next release. For more information about the parameters, please visit here.

NOTE: DMTet mode only supports gridres=128

Hopefully this will be useful for someone !