Monocular Localization against map created with RGB-D with very poor performance

[jsreveal]

Hi guys,

I have created with success a map with Intel RealSense D455. I am trying now to do localization in that map using a different camera, monocular RGB camera, however most of the frames are not correctly localized even though I am l looking into the same place as when creating the map.

Is this use case possible to work well at all?

What settings could be changed to try to improve localization?

[matlabbe]

I think the big difference is the "different camera". Is the field of view similar to D455? Is the resolution similar?

You can compare by feeding the same RGB topic used for mapping from the D455 (simulating a monocular camera without the depth).

If FOV and resolution are very different, you may want to use SIFT features for localization (Kp/DetectorStrategy=1).

[borongyuan]

I have tested using OAK-D W to build the map in the first session and OAK-D S2 in the second session, and localization can be triggered normally. So the camera FOV actually has little impact. You may need further examination.