view_group keyerror

[ghoshaw]

Hi, thanks for your sharing!
I run my own detector result by step 2 and 3 as in ReadMe, But it says:

Setting OMP and MKL num threads to 1.
pybullet build time: Sep 20 2020 23:49:22
0:00:00.002000 - --------------------------------------------------------------------------------
0:00:00.002170 - scenario: example
0:00:00.002226 - sv_score_th: 0.3
0:00:00.002262 - n_symmetries_rot: 64
0:00:00.002293 - ransac_n_iter: 2000
0:00:00.002325 - ransac_dist_threshold: 0.02
0:00:00.002354 - ba_n_iter: 10
0:00:00.002384 - nms_th: 0.04
0:00:00.002414 - no_visualization: False
0:00:00.002442 - --------------------------------------------------------------------------------
0:00:03.464109 - Loaded 708 candidates in 708 views.
0:00:03.480213 - Loaded cameras intrinsics.
0:00:03.737966 - Loaded 1 3D object models.
0:00:03.738085 - Running stage 2 and 3 of CosyPose...
0:00:04.871425 - Num candidates: 708
0:00:04.871581 - Num views: 708
0:00:04.874115 - Estimating camera poses using RANSAC.
0:00:05.532733 - Matched candidates: 0
0:00:05.532855 - RANSAC time_models: 0:00:00.649863
0:00:05.532897 - RANSAC time_score: 0:00:00.000510
0:00:05.532930 - RANSAC time_misc: 0:00:00.009830
Traceback (most recent call last):
File "/opt/conda/envs/cosypose/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2891, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 101, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1675, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1683, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'view_group'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/conda/envs/cosypose/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/envs/cosypose/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/XX/cosypose/cosypose/scripts/run_custom_scenario.py", line 184, in
main()
File "/home/XX/cosypose/cosypose/scripts/run_custom_scenario.py", line 153, in main
for view_group in np.unique(objects.infos['view_group']):
File "/opt/conda/envs/cosypose/lib/python3.7/site-packages/pandas/core/frame.py", line 2902, in getitem
indexer = self.columns.get_loc(key)
File "/opt/conda/envs/cosypose/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2893, in get_loc
raise KeyError(key) from err
KeyError: 'view_group'

I put the detect results and model in local_data in bop format, and run "run_custom_scenario.py". And what the view_group is?

And another questions is I only have one object in each frame, and the frame is taken from a video, Does this method still work in this situation?

[ylabbe]

Hi, thanks for your interest in the work !

If you have only one object in each view (and do not know camera poses), the multi-view method won't bring any improvements over single-view since it needs multiple objects to estimate relative camera poses. This seems to be your case with 708 views and 708 candidates.

If you give pose predictions in images that cannot be placed with respect to each others, the output cameras will have different view_group because they cannot be placed in the same scene. I think in your case there is a bug because there are are no cameras that can be placed with respect to another one (it requires to have at least 2 inlier candidates between a pair of views).

Could you share your custom scenario so that I can reproduce the bug ? I will add some informations to mention the method is not suited for such scenario.

[ghoshaw]

Hi, thanks for your answer, and the link of my data is https://drive.google.com/file/d/1ySdj8PB7pbliee7XEaGGJpCfHs4b6IhN/view?usp=sharing

[mauku]

Hi, i am facing the same issue , reported by @ghoshaw. I am also trying to run the stage 2 and 3 of the multi view pose estimation. Gettting the following output:

`[INFO] [1617110034.166546]: Loaded 11 candidates in 5 views.
[INFO] [1617110034.168708]: Loaded cameras intrinsics.
[INFO] [1617110038.442613]: Loaded 9 3D object models.
[INFO] [1617110038.443591]: Running stage 2 and 3 of CosyPose...
Matching outputs: {'filtered_candidates': PandasTensorCollection(
poses: torch.Size([0, 4, 4]) torch.float32 cuda:0,

infos:

Empty DataFrame
Columns: [view_id, scene_id, score, label, group_id, cand_id, obj_id]
Index: []
), 'scene_infos': Empty DataFrame
Columns: [obj_id, score, label, n_cand]
Index: [], 'pairs_TC1C2': PandasTensorCollection(
TC1C2: torch.Size([0, 4, 4]) torch.float32 cuda:0,

infos:

Empty DataFrame
Columns: [view1, view2]
Index: []
), 'time_models': datetime.timedelta(microseconds=216865), 'time_score': datetime.timedelta(microseconds=24715), 'time_misc': datetime.timedelta(microseconds=105096)}
predictions {'cand_inputs': PandasTensorCollection(
poses: torch.Size([11, 4, 4]) torch.float32 cuda:0,

infos:
view_id  scene_id     score       label  group_id  cand_id

0 1 1 0.985435 obj_000001 0 0
1 1 1 0.890519 obj_000004 0 1
2 2 1 0.999385 obj_000004 0 2
3 2 1 0.946256 obj_000006 0 3
4 3 1 0.999469 obj_000004 0 4
5 3 1 0.998749 obj_000006 0 5
6 3 1 0.993534 obj_000001 0 6
7 4 1 0.998625 obj_000004 0 7
8 4 1 0.893412 obj_000001 0 8
9 5 1 0.984479 obj_000004 0 9
10 5 1 0.967003 obj_000001 0 10
), 'cand_matched': PandasTensorCollection(
poses: torch.Size([0, 4, 4]) torch.float32 cuda:0,

infos:

Empty DataFrame
Columns: [view_id, scene_id, score, label, group_id, cand_id, obj_id]
Index: []
), 'scene/objects': PandasTensorCollection(

infos:

Empty DataFrame
Columns: []
Index: []
), 'scene/cameras': PandasTensorCollection(

infos:

Empty DataFrame
Columns: []
Index: []
), 'ba_output': PandasTensorCollection(

infos:

Empty DataFrame
Columns: []
Index: []
), 'ba_input': PandasTensorCollection(

infos:

Empty DataFrame
Columns: []
Index: []
), 'ba_output+all_cand': PandasTensorCollection(
poses: torch.Size([11, 4, 4]) torch.float32 cuda:0,

infos:
view_id  scene_id     score       label  group_id

0 1 1 0.985435 obj_000001 0
1 1 1 0.890519 obj_000004 0
2 2 1 0.999385 obj_000004 0
3 2 1 0.946256 obj_000006 0
4 3 1 0.999469 obj_000004 0
5 3 1 0.998749 obj_000006 0
6 3 1 0.993534 obj_000001 0
7 4 1 0.998625 obj_000004 0
8 4 1 0.893412 obj_000001 0
9 5 1 0.984479 obj_000004 0
10 5 1 0.967003 obj_000001 0
)}
Objects info : Empty DataFrame
Columns: []
Index: []
[ERROR] [1617110039.066448]: Error processing request: 'view_group'
['Traceback (most recent call last):\n', ' File "/home/markus/anaconda3/envs/cosypose/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2646, in get_loc\n return self._engine.get_loc(key)\n', ' File "pandas/_libs/index.pyx", line 111, in pandas._libs.index.IndexEngine.get_loc\n', ' File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc\n', ' File "pandas/_libs/hashtable_class_helper.pxi", line 1619, in pandas._libs.hashtable.PyObjectHashTable.get_item\n', ' File "pandas/_libs/hashtable_class_helper.pxi", line 1627, in pandas._libs.hashtable.PyObjectHashTable.get_item\n', "KeyError: 'view_group'\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 632, in _handle_request\n response = convert_return_to_response(self.handler(request), self.response_class)\n', ' File "/home/markus/Masterarbeit/practical/object_detection_ws/src/pose_core/scripts/scripts_client_pose_estimation.py", line 254, in multi_view_pose_cb\n for view_group in np.unique(objects.infos['view_group']):\n', ' File "/home/markus/anaconda3/envs/cosypose/lib/python3.7/site-packages/pandas/core/frame.py", line 2800, in getitem\n indexer = self.columns.get_loc(key)\n', ' File "/home/markus/anaconda3/envs/cosypose/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2648, in get_loc\n return self._engine.get_loc(self._maybe_cast_indexer(key))\n', ' File "pandas/_libs/index.pyx", line 111, in pandas._libs.index.IndexEngine.get_loc\n', ' File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc\n', ' File "pandas/_libs/hashtable_class_helper.pxi", line 1619, in pandas._libs.hashtable.PyObjectHashTable.get_item\n', ' File "pandas/_libs/hashtable_class_helper.pxi", line 1627, in pandas._libs.hashtable.PyObjectHashTable.get_item\n', "KeyError: 'view_group'\n"]`

Are there any updates about the bug ? Thanks in advance