Bug fix

[fan-ziqi]

Description

• add save_path to data_collect.py
• fix readme script error
• updated the import path for MatterportSemanticCostMapping from nav_collectors.configs to omni.viplanner.config
• in headless mode, we cannot visualize the graph and omni.debug.draw is not available

Type of change

• Bug fix (non-breaking change which fixes an issue)

Screenshots

Please attach before and after screenshots of the change if applicable.

Checklist

• I have run the pre-commit checks with ./formatter.sh
• I have made corresponding changes to the documentation
• My changes generate no new warnings

[fan-ziqi]

Using headless mode can significantly improve the speed of collect data. Headless can be used normally in the matterport environment, but it cannot be directly used in carla and warehouse environment. The script must be run with non-headless mode in carla and warehouse environment once, and then stop the script and then can using headless mode. The problem is that hits is empty in following code:

viplanner/omniverse/extension/omni.viplanner/omni/viplanner/collectors/terrain_analysis.py

Lines 883 to 910 in 7d481c1

	def _raycast_usd_stage(
	self,
	ray_starts: torch.Tensor,
	ray_directions: torch.Tensor,
	max_dist: float = 1e6,
	return_distance: bool = False,
	return_normal: bool = False,
	return_class: bool = False,
	) -> tuple[torch.Tensor, torch.Tensor | None, torch.Tensor | None, list | None]:
	"""
	Perform raycasting over the entire loaded stage.
	Interface is the same as the normal raycast_mesh function without the option to provide specific meshes.
	"""
	hits = [
	get_physx_scene_query_interface().raycast_closest(carb.Float3(ray_single), carb.Float3(ray_dir), max_dist)
	for ray_single, ray_dir in zip(ray_starts.cpu().numpy(), ray_directions.cpu().numpy())
	]
	# get all hit idx
	hit_idx = [idx for idx, single_hit in enumerate(hits) if single_hit["hit"]]
	# hit positions
	hit_positions = torch.zeros_like(ray_starts).fill_(float("inf"))
	hit_positions[hit_idx] = torch.tensor([single_hit["position"] for single_hit in hits if single_hit["hit"]]).to(
	ray_starts.device
	)

error:

/home/ubuntu/anaconda3/envs/viplanner/lib/python3.10/site-packages/torch/functional.py:513: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3609.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
Traceback (most recent call last):
  File "/home/ubuntu/workspaces/viplanner_ws/viplanner/omniverse/standalone/data_collect.py", line 136, in <module>
    main()
  File "/home/ubuntu/workspaces/viplanner_ws/viplanner/omniverse/standalone/data_collect.py", line 118, in main
    samples = explorer.sample_viewpoints(args_cli.num_samples)
  File "/home/ubuntu/workspaces/viplanner_ws/viplanner/omniverse/extension/omni.viplanner/omni/viplanner/collectors/viewpoint_sampling.py", line 66, in sample_viewpoints
    self.terrain_analyser.analyse()
  File "/home/ubuntu/workspaces/viplanner_ws/viplanner/omniverse/extension/omni.viplanner/omni/viplanner/collectors/terrain_analysis.py", line 74, in analyse
    self.construct_height_map()
  File "/home/ubuntu/workspaces/viplanner_ws/viplanner/omniverse/extension/omni.viplanner/omni/viplanner/collectors/terrain_analysis.py", line 770, in construct_height_map
    hit_point = self._raycast_usd_stage(
  File "/home/ubuntu/workspaces/viplanner_ws/viplanner/omniverse/extension/omni.viplanner/omni/viplanner/collectors/terrain_analysis.py", line 908, in _raycast_usd_stage
    hit_positions[hit_idx] = torch.tensor([single_hit["position"] for single_hit in hits if single_hit["hit"]]).to(
RuntimeError: shape mismatch: value tensor of shape [0] cannot be broadcast to indexing result of shape [0, 3]

I don't know how to fix this. After fixing this issue, the data_collect.py script should run in headless mode to accelerate collection.

[pascal-roth]

Thanks a lot for the fixes!!!

[pascal-roth]

Using headless mode can significantly improve the speed of collect data. Headless can be used normally in the matterport environment, but it cannot be directly used in carla and warehouse environment. The script must be run with non-headless mode in carla and warehouse environment once, and then stop the script and then can using headless mode. The problem is that hits is empty in following code:

viplanner/omniverse/extension/omni.viplanner/omni/viplanner/collectors/terrain_analysis.py

Lines 883 to 910 in 7d481c1

	def _raycast_usd_stage(
	self,
	ray_starts: torch.Tensor,
	ray_directions: torch.Tensor,
	max_dist: float = 1e6,
	return_distance: bool = False,
	return_normal: bool = False,
	return_class: bool = False,
	) -> tuple[torch.Tensor, torch.Tensor | None, torch.Tensor | None, list | None]:
	"""
	Perform raycasting over the entire loaded stage.
	Interface is the same as the normal raycast_mesh function without the option to provide specific meshes.
	"""
	hits = [
	get_physx_scene_query_interface().raycast_closest(carb.Float3(ray_single), carb.Float3(ray_dir), max_dist)
	for ray_single, ray_dir in zip(ray_starts.cpu().numpy(), ray_directions.cpu().numpy())
	]
	# get all hit idx
	hit_idx = [idx for idx, single_hit in enumerate(hits) if single_hit["hit"]]
	# hit positions
	hit_positions = torch.zeros_like(ray_starts).fill_(float("inf"))
	hit_positions[hit_idx] = torch.tensor([single_hit["position"] for single_hit in hits if single_hit["hit"]]).to(
	ray_starts.device
	)

error:

/home/ubuntu/anaconda3/envs/viplanner/lib/python3.10/site-packages/torch/functional.py:513: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3609.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
Traceback (most recent call last):
  File "/home/ubuntu/workspaces/viplanner_ws/viplanner/omniverse/standalone/data_collect.py", line 136, in <module>
    main()
  File "/home/ubuntu/workspaces/viplanner_ws/viplanner/omniverse/standalone/data_collect.py", line 118, in main
    samples = explorer.sample_viewpoints(args_cli.num_samples)
  File "/home/ubuntu/workspaces/viplanner_ws/viplanner/omniverse/extension/omni.viplanner/omni/viplanner/collectors/viewpoint_sampling.py", line 66, in sample_viewpoints
    self.terrain_analyser.analyse()
  File "/home/ubuntu/workspaces/viplanner_ws/viplanner/omniverse/extension/omni.viplanner/omni/viplanner/collectors/terrain_analysis.py", line 74, in analyse
    self.construct_height_map()
  File "/home/ubuntu/workspaces/viplanner_ws/viplanner/omniverse/extension/omni.viplanner/omni/viplanner/collectors/terrain_analysis.py", line 770, in construct_height_map
    hit_point = self._raycast_usd_stage(
  File "/home/ubuntu/workspaces/viplanner_ws/viplanner/omniverse/extension/omni.viplanner/omni/viplanner/collectors/terrain_analysis.py", line 908, in _raycast_usd_stage
    hit_positions[hit_idx] = torch.tensor([single_hit["position"] for single_hit in hits if single_hit["hit"]]).to(
RuntimeError: shape mismatch: value tensor of shape [0] cannot be broadcast to indexing result of shape [0, 3]

I don't know how to fix this. After fixing this issue, the data_collect.py script should run in headless mode to accelerate collection.

Good catch, I am not entirely sure. Moving it now to an issue to keep track of it.