IOS - Issue with Pose Extraction and Verification in SQLite Database

[dscho15]

Hi there,

I'm working on a hobby project which involves extracting poses, images, depth maps, and camera calibration data from your application's SQLite3 database. So far, I've been able to do the following:

• Extracted calibration data from the "Data" table, which seems to give values that make sense (fx, fy, cx, cy...)
• Retrieved images and depth maps from the "Data" table
• Extracted poses and opt-indices from the "Admin" table
• Decoded poses using zlib, resulting in N x 3 x 4 pose matrices

However, I've encountered an unexpected discrepancy. When comparing the first two poses using the transformation np.linalg.inv(poses[2]) @ poses[1], the results do not align with Open3D's local adjustment algorithm's (ICP) output, even when using the same initial pose calculation. It is like a couple of centimeters off... Also just to check if I was unfortunate, I double-checked on a couple of different video's (1-3 seconds each).

So, I believe, there could be an error in my pose decoding process.

def extract_opt_poses(self):
      opt_poses = zlib.decompress(opt_poses goes here) # opt_poses is a byte_string
      opt_poses = decode_array(opt_poses, np.float32, shape=(-1, 3, 4)) 
      zeros = np.zeros((len(opt_poses), 1, 4))
      zeros[:, 0, 3] = 1
      return np.hstack([opt_poses, zeros])

def decode_array(x: str, dtype: np.dtype = np.float32, shape: tuple = None) -> np.ndarray:
      array = np.frombuffer(bytearray(x), dtype=dtype)
      return array.reshape(shape) if shape else array

I would appreciate any insights you can provide to help understand what could be wrong (:

Also the code base is available at https://github.com/dscho15/rtabmap-splatting

[dscho15]

Update

It is not the poses. RTAB-Map seems to agree on the poses. So I guess it is either the depth-maps which are decoded wrongly, or the calibration data.