Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How do I properly modify the fusion options given scenes with dramatically different depth? #59

Open
yahskapar opened this issue Oct 2, 2023 · 0 comments

Comments

@yahskapar
Copy link

The TSDF fusion volume in the TANDEM pipeline is initialized as follows here:

DrFusionOptions dr_options{
.voxel_size = 0.01,
.num_buckets = 1000000,
.bucket_size = 10,
.num_blocks = 1000000,
.block_size = 8,
.max_sdf_weight = 64,
.truncation_distance = 0.04,
.max_sensor_depth = 10,
.min_sensor_depth = 0.1,
.num_render_streams = num_render_streams,
.fx = Hcalib.fxl(),
.fy = Hcalib.fyl(),
.cx = Hcalib.cxl(),
.cy = Hcalib.cyl(),
.height = dr_height,
.width = dr_width,
};

If I want to run TANDEM on a scene with dramatically different scale than what TANDEM appears to be designed for, say a range of 0 to 0.1 meters, how would I modify the above options? Obviously the max_sensor_depth and min_sensor_depth would change, but it's not clear to me how other parameters would change despite my read of the paper's supplementary materials here in section B. There, it is noted:

The grid uses a voxel size of 0.01 which roughly corresponds to 2.5cm depending on the exact scale of a particular scene and run. We use a truncation distance for the TSDF of 0.1 which roughly corresponds to 25cm. [...] However, when considering scenes of different extent, the voxel size and truncation distance have to be set accordingly, which is also necessary when running classical TSDF fusion.

How are the aforementioned values (e.g., 0.01 for voxel size) derived within the context of the TANDEM paper itself? I was thinking it might have to relate to the global depth scale, but that that can't be based on the global depth scale values in the TANDEM format datasets (Replica and EuRoC) linked to in this repo. Even digging into the ReFusion code here, it's not clear to me how the depth min and max relates to the voxel size.

Any pointers in the right direction to understand this more clearly and figure out how to modify the values would be appreciated! I've tried a few different configurations of values and unfortunately have not been able produce a mesh at all (this is despite the data in question being formatted correctly as required by TANDEM, and already used for training and evaluation in the CVA_MVSNet portion of the pipeline).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant