Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Positional Tracking with IMU Fusion supported? #13

Open
limshoonkit opened this issue Nov 16, 2024 · 0 comments
Open

Positional Tracking with IMU Fusion supported? #13

limshoonkit opened this issue Nov 16, 2024 · 0 comments

Comments

@limshoonkit
Copy link

limshoonkit commented Nov 16, 2024

Environment

ZED SDK = 4.2.1
Isaac Sim = 4.2
zed-issac-sim = main branch
zed-ros2-wrapper = main branch

Error
Positional Tracking GEN2 with IMU fusion enabled requires high frequency sensors data (available with Streaming version 2, with ZED SDK >=4.1). IMU Fusion will be disabled, performance will be degraded.

ros2 launch zed_wrapper zed_camera.launch.py camera_model:=zedx sim_mode:=true use_sim_time:=true
[INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2024-11-16-21-29-26-932694-ubuntu-Alienware-m15-R7-10917
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [robot_state_publisher-1]: process started with pid [10919]
[INFO] [zed_wrapper-2]: process started with pid [10921]
[robot_state_publisher-1] [INFO] [1731763767.128107097] [zed.zed_state_publisher]: got segment zed_camera_center
[robot_state_publisher-1] [INFO] [1731763767.128182676] [zed.zed_state_publisher]: got segment zed_camera_link
[robot_state_publisher-1] [INFO] [1731763767.128187519] [zed.zed_state_publisher]: got segment zed_left_camera_frame
[robot_state_publisher-1] [INFO] [1731763767.128190064] [zed.zed_state_publisher]: got segment zed_left_camera_optical_frame
[robot_state_publisher-1] [INFO] [1731763767.128192385] [zed.zed_state_publisher]: got segment zed_right_camera_frame
[robot_state_publisher-1] [INFO] [1731763767.128194685] [zed.zed_state_publisher]: got segment zed_right_camera_optical_frame
[zed_wrapper-2] [INFO] [1731763767.252857799] [zed.zed_node]: ********************************
[zed_wrapper-2] [INFO] [1731763767.252983123] [zed.zed_node]:       ZED Camera Component 
[zed_wrapper-2] [INFO] [1731763767.252995408] [zed.zed_node]: ********************************
[zed_wrapper-2] [INFO] [1731763767.253004474] [zed.zed_node]:  * namespace: /zed
[zed_wrapper-2] [INFO] [1731763767.253013134] [zed.zed_node]:  * node name: zed_node
[zed_wrapper-2] [INFO] [1731763767.253019851] [zed.zed_node]: ********************************
[zed_wrapper-2] [INFO] [1731763767.303703134] [zed.zed_node]: *** DEBUG parameters ***
[zed_wrapper-2] [INFO] [1731763767.303856670] [zed.zed_node]:  * SDK Verbose: 1
[zed_wrapper-2] [INFO] [1731763767.303915014] [zed.zed_node]:  * Debug Common: FALSE
[zed_wrapper-2] [INFO] [1731763767.303951837] [zed.zed_node]:  * Debug Simulation: FALSE
[zed_wrapper-2] [INFO] [1731763767.303986020] [zed.zed_node]:  * Debug Video/Depth: FALSE
[zed_wrapper-2] [INFO] [1731763767.304027805] [zed.zed_node]:  * Debug Control settings: FALSE
[zed_wrapper-2] [INFO] [1731763767.304064084] [zed.zed_node]:  * Debug Point Cloud: FALSE
[zed_wrapper-2] [INFO] [1731763767.304097998] [zed.zed_node]:  * Debug GNSS: FALSE
[zed_wrapper-2] [INFO] [1731763767.304131574] [zed.zed_node]:  * Debug Positional Tracking: FALSE
[zed_wrapper-2] [INFO] [1731763767.304169743] [zed.zed_node]:  * Debug sensors: FALSE
[zed_wrapper-2] [INFO] [1731763767.304205958] [zed.zed_node]:  * Debug Mapping: FALSE
[zed_wrapper-2] [INFO] [1731763767.304238131] [zed.zed_node]:  * Debug Object Detection: FALSE
[zed_wrapper-2] [INFO] [1731763767.304270351] [zed.zed_node]:  * Debug Body Tracking: FALSE
[zed_wrapper-2] [INFO] [1731763767.304302366] [zed.zed_node]:  * Debug Streaming: FALSE
[zed_wrapper-2] [INFO] [1731763767.304340508] [zed.zed_node]:  * Debug ROI: FALSE
[zed_wrapper-2] [INFO] [1731763767.304372762] [zed.zed_node]:  * Debug Advanced: FALSE
[zed_wrapper-2] [INFO] [1731763767.304419317] [zed.zed_node]:  *** SIMULATION MODE ACTIVE ***
[zed_wrapper-2] [INFO] [1731763767.304462546] [zed.zed_node]:  * Sim. server address: 127.0.0.1
[zed_wrapper-2] [INFO] [1731763767.304504044] [zed.zed_node]:  * Sim. server port: 30000
[zed_wrapper-2] [INFO] [1731763767.304520227] [zed.zed_node]:  * Use Sim Time: TRUE
[zed_wrapper-2] [INFO] [1731763767.304535084] [zed.zed_node]: *** GENERAL parameters ***
[zed_wrapper-2] [INFO] [1731763767.304659826] [zed.zed_node]:  + Simulating a ZED X camera model.
[zed_wrapper-2] [INFO] [1731763767.304675744] [zed.zed_node]:  * Camera model: zedx - ZED X
[zed_wrapper-2] [INFO] [1731763767.304714798] [zed.zed_node]:  * Camera name: zed
[zed_wrapper-2] [INFO] [1731763767.304754665] [zed.zed_node]:  * Camera SN: 0
[zed_wrapper-2] [INFO] [1731763767.304789793] [zed.zed_node]:  * Camera timeout [sec]: 5
[zed_wrapper-2] [INFO] [1731763767.304828632] [zed.zed_node]:  * Camera reconnection temptatives: 5
[zed_wrapper-2] [INFO] [1731763767.304838722] [zed.zed_node]: * [Simulation mode] Camera framerate forced to 60 Hz
[zed_wrapper-2] [INFO] [1731763767.304871490] [zed.zed_node]:  * GPU ID: -1
[zed_wrapper-2] [INFO] [1731763767.304880927] [zed.zed_node]: * [Simulation mode] Camera resolution forced to 'HD1080'
[zed_wrapper-2] [INFO] [1731763767.304920464] [zed.zed_node]:  * Publishing resolution: CUSTOM
[zed_wrapper-2] [INFO] [1731763767.304969177] [zed.zed_node]:  * Publishing downscale factor: 2
[zed_wrapper-2] [INFO] [1731763767.305004815] [zed.zed_node]:  * OpenCV custom calibration: 
[zed_wrapper-2] [INFO] [1731763767.305039201] [zed.zed_node]:  * Camera self calibration: TRUE
[zed_wrapper-2] [INFO] [1731763767.305076458] [zed.zed_node]:  * Camera flip: FALSE
[zed_wrapper-2] [INFO] [1731763767.305085761] [zed.zed_node]: * [Simulation mode] Publish framerate forced to 60 Hz
[zed_wrapper-2] [INFO] [1731763767.305094775] [zed.zed_node]: *** VIDEO parameters ***
[zed_wrapper-2] [INFO] [1731763767.305128679] [zed.zed_node]:  * [DYN] Saturation: 4
[zed_wrapper-2] [INFO] [1731763767.305162088] [zed.zed_node]:  * [DYN] Sharpness: 4
[zed_wrapper-2] [INFO] [1731763767.305195740] [zed.zed_node]:  * [DYN] Gamma: 8
[zed_wrapper-2] [INFO] [1731763767.305236193] [zed.zed_node]:  * [DYN] Auto Exposure/Gain: TRUE
[zed_wrapper-2] [INFO] [1731763767.305273654] [zed.zed_node]:  * [DYN] Exposure: 80
[zed_wrapper-2] [INFO] [1731763767.305307224] [zed.zed_node]:  * [DYN] Gain: 80
[zed_wrapper-2] [INFO] [1731763767.305339312] [zed.zed_node]:  * [DYN] Auto White Balance: TRUE
[zed_wrapper-2] [INFO] [1731763767.305372259] [zed.zed_node]:  * [DYN] White Balance Temperature: 42
[zed_wrapper-2] [INFO] [1731763767.305410924] [zed.zed_node]:  * [DYN] ZED X Exposure time: 16000
[zed_wrapper-2] [INFO] [1731763767.305452465] [zed.zed_node]:  * [DYN] ZED X Auto Exp. time range min: 28
[zed_wrapper-2] [INFO] [1731763767.305487059] [zed.zed_node]:  * [DYN] ZED X Auto Exp. time range max: 30000
[zed_wrapper-2] [INFO] [1731763767.305520307] [zed.zed_node]:  * [DYN] ZED X Exposure comp.: 50
[zed_wrapper-2] [INFO] [1731763767.305589680] [zed.zed_node]:  * [DYN] ZED X Analog Gain: 1255
[zed_wrapper-2] [INFO] [1731763767.305625777] [zed.zed_node]:  * [DYN] ZED X Auto Analog Gain range min: 1000
[zed_wrapper-2] [INFO] [1731763767.305665908] [zed.zed_node]:  * [DYN] ZED X Auto Analog Gain range max: 16000
[zed_wrapper-2] [INFO] [1731763767.305700296] [zed.zed_node]:  * [DYN] ZED X Digital Gain: 1
[zed_wrapper-2] [INFO] [1731763767.305737769] [zed.zed_node]:  * [DYN] ZED X Auto Digital Gain range min: 1
[zed_wrapper-2] [INFO] [1731763767.305771842] [zed.zed_node]:  * [DYN] ZED X Auto Digital Gain range max: 256
[zed_wrapper-2] [INFO] [1731763767.305805283] [zed.zed_node]:  * [DYN] ZED X Auto Digital Gain range max: 50
[zed_wrapper-2] [INFO] [1731763767.305819848] [zed.zed_node]: *** DEPTH parameters ***
[zed_wrapper-2] [INFO] [1731763767.306047259] [zed.zed_node]:  * Depth mode: NEURAL PLUS [5]
[zed_wrapper-2] [INFO] [1731763767.306092260] [zed.zed_node]:  * Min depth [m]: 0.3
[zed_wrapper-2] [INFO] [1731763767.306129047] [zed.zed_node]:  * Max depth [m]: 10
[zed_wrapper-2] [INFO] [1731763767.306162923] [zed.zed_node]:  * Depth Stabilization: 1
[zed_wrapper-2] [INFO] [1731763767.306195623] [zed.zed_node]:  * OpenNI mode (16bit point cloud): FALSE
[zed_wrapper-2] [INFO] [1731763767.306236807] [zed.zed_node]:  * [DYN] Point cloud rate [Hz]: 10
[zed_wrapper-2] [INFO] [1731763767.306275629] [zed.zed_node]:  * [DYN] Depth Confidence: 50
[zed_wrapper-2] [INFO] [1731763767.306310298] [zed.zed_node]:  * [DYN] Depth Texture Confidence: 100
[zed_wrapper-2] [INFO] [1731763767.306342980] [zed.zed_node]:  * [DYN] Remove saturated areas: TRUE
[zed_wrapper-2] [INFO] [1731763767.306354490] [zed.zed_node]: *** GNSS FUSION parameters ***
[zed_wrapper-2] [INFO] [1731763767.306388118] [zed.zed_node]:  * GNSS fusion enabled: FALSE
[zed_wrapper-2] [INFO] [1731763767.306398840] [zed.zed_node]: *** POSITIONAL TRACKING parameters ***
[zed_wrapper-2] [INFO] [1731763767.306435900] [zed.zed_node]:  * Positional tracking enabled: TRUE
[zed_wrapper-2] [INFO] [1731763767.306475185] [zed.zed_node]:  * Positional tracking mode: GEN 2
[zed_wrapper-2] [INFO] [1731763767.306510126] [zed.zed_node]:  * Map frame id: map
[zed_wrapper-2] [INFO] [1731763767.306554174] [zed.zed_node]:  * Odometry frame id: odom
[zed_wrapper-2] [INFO] [1731763767.306598332] [zed.zed_node]:  * Broadcast Odometry TF: TRUE
[zed_wrapper-2] [INFO] [1731763767.306632911] [zed.zed_node]:  * Broadcast Pose TF: TRUE
[zed_wrapper-2] [INFO] [1731763767.306667869] [zed.zed_node]:  * [DYN] Depth minimum range: 0
[zed_wrapper-2] [INFO] [1731763767.306703550] [zed.zed_node]:  * [DYN] TF timestamp offset: 0
[zed_wrapper-2] [INFO] [1731763767.306741471] [zed.zed_node]:  * [DYN] Path publishing rate: 2
[zed_wrapper-2] [INFO] [1731763767.306776020] [zed.zed_node]:  * Path history lenght: -1
[zed_wrapper-2] [INFO] [1731763767.306814878] [zed.zed_node]:  * Initial pose: [0,0,0,0,0,0,]
[zed_wrapper-2] [INFO] [1731763767.306851969] [zed.zed_node]:  * Area Memory: TRUE
[zed_wrapper-2] [INFO] [1731763767.306886679] [zed.zed_node]:  * Area Memory DB: 
[zed_wrapper-2] [INFO] [1731763767.306926998] [zed.zed_node]:  * Camera is static: FALSE
[zed_wrapper-2] [INFO] [1731763767.306961224] [zed.zed_node]:  * Gravity as origin [not for ZED]: TRUE
[zed_wrapper-2] [INFO] [1731763767.306995316] [zed.zed_node]:  * IMU Fusion [not for ZED]: TRUE
[zed_wrapper-2] [INFO] [1731763767.307028486] [zed.zed_node]:  * Floor Alignment: FALSE
[zed_wrapper-2] [INFO] [1731763767.307065743] [zed.zed_node]:  * Reset Odometry with Loop Closure: TRUE
[zed_wrapper-2] [INFO] [1731763767.307099916] [zed.zed_node]:  * 2D mode: FALSE
[zed_wrapper-2] [INFO] [1731763767.307110654] [zed.zed_node]: *** Region of Interest parameters ***
[zed_wrapper-2] [INFO] [1731763767.307142021] [zed.zed_node]:  * Automatic ROI generation: FALSE
[zed_wrapper-2] [INFO] [1731763767.307324747] [zed.zed_node]:  * Manual ROI polygon: []
[zed_wrapper-2] [INFO] [1731763767.307350437] [zed.zed_node]: *** SENSORS STACK parameters ***
[zed_wrapper-2] [INFO] [1731763767.307405363] [zed.zed_node]:  * Broadcast IMU TF [not for ZED]: TRUE
[zed_wrapper-2] [INFO] [1731763767.307423795] [zed.zed_node]:  * Sensors Camera Sync: FALSE
[zed_wrapper-2] [INFO] [1731763767.307438946] [zed.zed_node]:  * Sensors publishing rate: 200 Hz
[zed_wrapper-2] [INFO] [1731763767.307443705] [zed.zed_node]: *** Spatial Mapping parameters ***
[zed_wrapper-2] [INFO] [1731763767.307458967] [zed.zed_node]:  * Spatial Mapping Enabled: FALSE
[zed_wrapper-2] [INFO] [1731763767.307477455] [zed.zed_node]:  * Spatial Mapping resolution [m]: 0.05
[zed_wrapper-2] [INFO] [1731763767.307489422] [zed.zed_node]:  * 3D Max Mapping range [m]: 5
[zed_wrapper-2] [INFO] [1731763767.307500890] [zed.zed_node]:  * Map publishing rate [Hz]: 1
[zed_wrapper-2] [INFO] [1731763767.307514586] [zed.zed_node]:  * Clicked point topic: /clicked_point
[zed_wrapper-2] [INFO] [1731763767.307544497] [zed.zed_node]:  * Plane Det. Max Dist. Thresh.: 0.15
[zed_wrapper-2] [INFO] [1731763767.307556212] [zed.zed_node]:  * Plane Det. Normals Sim. Thresh.: 15
[zed_wrapper-2] [INFO] [1731763767.307560357] [zed.zed_node]: *** Object Det. parameters ***
[zed_wrapper-2] [INFO] [1731763767.307573166] [zed.zed_node]:  * Object Det. enabled: FALSE
[zed_wrapper-2] [INFO] [1731763767.307608658] [zed.zed_node]:  * Object Det. model: MULTI CLASS BOX FAST
[zed_wrapper-2] [INFO] [1731763767.307621968] [zed.zed_node]:  * Object Det. allow reduced precision: TRUE
[zed_wrapper-2] [INFO] [1731763767.307635035] [zed.zed_node]:  * Object Det. maximum range [m]: 20
[zed_wrapper-2] [INFO] [1731763767.307646469] [zed.zed_node]:  * Object Det. min. confidence: 75
[zed_wrapper-2] [INFO] [1731763767.307659884] [zed.zed_node]:  * Object Det. prediction timeout [sec]: 0.5
[zed_wrapper-2] [INFO] [1731763767.307670339] [zed.zed_node]:  * Object Det. tracking: TRUE
[zed_wrapper-2] [INFO] [1731763767.307692278] [zed.zed_node]:  * Object Filtering mode: 1 - NMS3D
[zed_wrapper-2] [INFO] [1731763767.307720360] [zed.zed_node]:  * MultiClassBox people: TRUE
[zed_wrapper-2] [INFO] [1731763767.307735603] [zed.zed_node]:  * MultiClassBox vehicles: TRUE
[zed_wrapper-2] [INFO] [1731763767.307751424] [zed.zed_node]:  * MultiClassBox bags: TRUE
[zed_wrapper-2] [INFO] [1731763767.307765231] [zed.zed_node]:  * MultiClassBox animals: TRUE
[zed_wrapper-2] [INFO] [1731763767.307779113] [zed.zed_node]:  * MultiClassBox electronics: TRUE
[zed_wrapper-2] [INFO] [1731763767.307792857] [zed.zed_node]:  * MultiClassBox fruits and vegetables: TRUE
[zed_wrapper-2] [INFO] [1731763767.307809502] [zed.zed_node]:  * MultiClassBox sport-related objects: TRUE
[zed_wrapper-2] [INFO] [1731763767.307813021] [zed.zed_node]: *** Body Track. parameters ***
[zed_wrapper-2] [INFO] [1731763767.307824922] [zed.zed_node]:  * Body Track. enabled: FALSE
[zed_wrapper-2] [INFO] [1731763767.307842472] [zed.zed_node]:  * Body Track. model: HUMAN BODY MEDIUM
[zed_wrapper-2] [INFO] [1731763767.307858538] [zed.zed_node]:  * Body Track. format: BODY 38
[zed_wrapper-2] [INFO] [1731763767.307870840] [zed.zed_node]:  * Body Track. allow reduced precision: TRUE
[zed_wrapper-2] [INFO] [1731763767.307883802] [zed.zed_node]:  * Body Track. maximum range [m]: 20
[zed_wrapper-2] [INFO] [1731763767.307897648] [zed.zed_node]:  * Body Track. KP selection: FULL
[zed_wrapper-2] [INFO] [1731763767.307917026] [zed.zed_node]:  * Body fitting: FALSE
[zed_wrapper-2] [INFO] [1731763767.307931527] [zed.zed_node]:  * Body joints tracking: TRUE
[zed_wrapper-2] [INFO] [1731763767.307946050] [zed.zed_node]:  * Body Track. prediction timeout [sec]: 0.5
[zed_wrapper-2] [INFO] [1731763767.307958711] [zed.zed_node]:  * Body Track. confidence thresh.: 50
[zed_wrapper-2] [INFO] [1731763767.307972784] [zed.zed_node]:  * Body Track. min. KP thresh.: 5
[zed_wrapper-2] [INFO] [1731763767.307976712] [zed.zed_node]: *** Streaming Server parameters ***
[zed_wrapper-2] [INFO] [1731763767.307990692] [zed.zed_node]:  * Streaming Server enabled: FALSE
[zed_wrapper-2] [INFO] [1731763767.308002782] [zed.zed_node]:  * Stream codec: H264
[zed_wrapper-2] [INFO] [1731763767.308014963] [zed.zed_node]:  * Stream port:30000
[zed_wrapper-2] [INFO] [1731763767.308033058] [zed.zed_node]:  * Stream bitrate: 12500
[zed_wrapper-2] [INFO] [1731763767.308048885] [zed.zed_node]:  * Stream GOP size: -1
[zed_wrapper-2] [INFO] [1731763767.308060895] [zed.zed_node]:  * Stream Chunk size: 16084
[zed_wrapper-2] [INFO] [1731763767.308073043] [zed.zed_node]:  * Adaptive bitrate: FALSE
[zed_wrapper-2] [INFO] [1731763767.308086307] [zed.zed_node]:  * Target frame rate:0
[zed_wrapper-2] [INFO] [1731763767.308093333] [zed.zed_node]: *** Advanced parameters ***
[zed_wrapper-2] [INFO] [1731763767.308109651] [zed.zed_node]:  * Thread sched. policy: SCHED_BATCH
[zed_wrapper-2] [INFO] [1731763767.308128734] [zed.zed_node]: *** SERVICES ***
[zed_wrapper-2] [INFO] [1731763767.308987706] [zed.zed_node]:  * '/zed/zed_node/reset_odometry'
[zed_wrapper-2] [INFO] [1731763767.309195357] [zed.zed_node]:  * '/zed/zed_node/reset_pos_tracking'
[zed_wrapper-2] [INFO] [1731763767.309966937] [zed.zed_node]:  * '/zed/zed_node/set_pose'
[zed_wrapper-2] [INFO] [1731763767.310213971] [zed.zed_node]:  * '/zed/zed_node/enable_obj_det'
[zed_wrapper-2] [INFO] [1731763767.310417803] [zed.zed_node]:  * '/zed/zed_node/enable_body_trk'
[zed_wrapper-2] [INFO] [1731763767.310600353] [zed.zed_node]:  * '/zed/zed_node/enable_mapping'
[zed_wrapper-2] [INFO] [1731763767.310775418] [zed.zed_node]:  * '/zed/zed_node/enable_streaming'
[zed_wrapper-2] [INFO] [1731763767.311002686] [zed.zed_node]:  * '/zed/zed_node/start_svo_rec'
[zed_wrapper-2] [INFO] [1731763767.311189794] [zed.zed_node]:  * '/zed/zed_node/stop_svo_rec'
[zed_wrapper-2] [INFO] [1731763767.311381971] [zed.zed_node]:  * '/zed/zed_node/set_roi'
[zed_wrapper-2] [INFO] [1731763767.311519277] [zed.zed_node]:  * '/zed/zed_node/reset_roi'
[zed_wrapper-2] [INFO] [1731763767.312834528] [zed.zed_node]: ***** STARTING CAMERA *****
[zed_wrapper-2] [INFO] [1731763767.444195508] [zed.zed_node]: ZED SDK Version: 4.2.1 - Build 94676_d35891e9
[zed_wrapper-2] [2024-11-16 13:29:27 UTC][ZED][INFO] Logging level INFO
[zed_wrapper-2] [INFO] [1731763767.447628819] [zed.zed_node]: *** CONNECTING TO THE SIMULATION SERVER [127.0.0.1:30000] ***
[zed_wrapper-2] [INFO] [1731763767.447653070] [zed.zed_node]: Disable async recovery for GMSL2 cameras
[zed_wrapper-2] [2024-11-16 13:29:27 UTC][ZED][INFO] Logging level INFO
[zed_wrapper-2] [Streaming] Warning : receiving port 30000 is not available (already used)... switching to port 30002. Retrying...
[zed_wrapper-2] [Streaming] No backward compatibility required.
[zed_wrapper-2] [Streaming] Warning: Corrupted frame chunk received (recv: 196 / expect 4112)  from ip : 127.0.0.1 at ts 1731763760051(ms)
[zed_wrapper-2] [Streaming] Metadata timeout. the size is equal to 196 instead of 21960. Skipping.
[zed_wrapper-2] [2024-11-16 13:29:28 UTC][ZED][INFO] [Init]  Depth mode: NEURAL PLUS
[zed_wrapper-2] [2024-11-16 13:29:28 UTC][ZED][INFO] [Init]  Serial Number: S/N 40976320
[zed_wrapper-2] [2024-11-16 13:29:28 UTC][ZED][WARNING] Simulated ZED camera used.
[zed_wrapper-2] [2024-11-16 13:29:28 UTC][ZED][WARNING] [Init]  Invalid calibration file. On Linux, call export LC_ALL="C" and restart your application.
[zed_wrapper-2] [2024-11-16 13:29:28 UTC][ZED][WARNING] Self Calibration Disabled.
[zed_wrapper-2] [2024-11-16 13:29:28 UTC][ZED][WARNING] [Init]  Requested depth_minimum_distance: 0.3METER is too close, clamped to 0.4METER.
[zed_wrapper-2] [WARN] [1731763769.064100075] [zed.zed_node]: !!! `general.grab_frame_rate` value is not valid: '60'. Automatically replaced with '30'. Please fix the parameter !!!
[zed_wrapper-2] [INFO] [1731763769.064156708] [zed.zed_node]: ZED SDK running on GPU #0
[zed_wrapper-2] [INFO] [1731763769.064170940] [zed.zed_node]:  * Camera Model  -> ZED X
[zed_wrapper-2] [INFO] [1731763769.064197763] [zed.zed_node]:  * Serial Number -> 40976320
[zed_wrapper-2] [INFO] [1731763769.064209021] [zed.zed_node]:  * Focal Lenght -> 4.4496 mm
[zed_wrapper-2] [INFO] [1731763769.064225256] [zed.zed_node]:  * Input   -> STREAM
[zed_wrapper-2] [INFO] [1731763769.064237827] [zed.zed_node]:  * Camera FW Version  -> 1523
[zed_wrapper-2] [INFO] [1731763769.064256845] [zed.zed_node]:  * Sensors FW Version -> 0
[zed_wrapper-2] [INFO] [1731763769.064268692] [zed.zed_node]:  * Camera grab frame size -> 1280x720
[zed_wrapper-2] [INFO] [1731763769.064280604] [zed.zed_node]:  * Publishing frame size  -> 640x360
[zed_wrapper-2] [INFO] [1731763769.064302003] [zed.zed_node]: *** TF FRAMES ***
[zed_wrapper-2] [INFO] [1731763769.064311726] [zed.zed_node]:  * Map                    -> map
[zed_wrapper-2] [INFO] [1731763769.064320130] [zed.zed_node]:  * Odometry               -> odom
[zed_wrapper-2] [INFO] [1731763769.064328557] [zed.zed_node]:  * Base                   -> zed_camera_link
[zed_wrapper-2] [INFO] [1731763769.064336599] [zed.zed_node]:  * Camera                 -> zed_camera_center
[zed_wrapper-2] [INFO] [1731763769.064344984] [zed.zed_node]:  * Left                   -> zed_left_camera_frame
[zed_wrapper-2] [INFO] [1731763769.064358153] [zed.zed_node]:  * Left Optical           -> zed_left_camera_optical_frame
[zed_wrapper-2] [INFO] [1731763769.064366857] [zed.zed_node]:  * RGB                    -> zed_left_camera_frame
[zed_wrapper-2] [INFO] [1731763769.064375257] [zed.zed_node]:  * RGB Optical            -> zed_left_camera_optical_frame
[zed_wrapper-2] [INFO] [1731763769.064383397] [zed.zed_node]:  * Right                  -> zed_right_camera_frame
[zed_wrapper-2] [INFO] [1731763769.064391697] [zed.zed_node]:  * Right Optical          -> zed_right_camera_optical_frame
[zed_wrapper-2] [INFO] [1731763769.064400123] [zed.zed_node]:  * Depth                  -> zed_left_camera_frame
[zed_wrapper-2] [INFO] [1731763769.064408646] [zed.zed_node]:  * Depth Optical          -> zed_left_camera_optical_frame
[zed_wrapper-2] [INFO] [1731763769.064417092] [zed.zed_node]:  * Point Cloud            -> zed_left_camera_optical_frame
[zed_wrapper-2] [INFO] [1731763769.064425744] [zed.zed_node]:  * Disparity              -> zed_left_camera_frame
[zed_wrapper-2] [INFO] [1731763769.064434232] [zed.zed_node]:  * Disparity Optical      -> zed_left_camera_optical_frame
[zed_wrapper-2] [INFO] [1731763769.064442343] [zed.zed_node]:  * Confidence             -> zed_left_camera_frame
[zed_wrapper-2] [INFO] [1731763769.064450464] [zed.zed_node]:  * Confidence Optical     -> zed_left_camera_optical_frame
[zed_wrapper-2] [INFO] [1731763769.064458857] [zed.zed_node]:  * IMU                    -> zed_imu_link
[zed_wrapper-2] [INFO] [1731763769.064491157] [zed.zed_node]: *** PUBLISHED TOPICS ***
[zed_wrapper-2] [INFO] [1731763769.344165710] [FFMPEGEncoder]: enc: libx264 prof:  preset: 
[zed_wrapper-2] [INFO] [1731763769.344191919] [FFMPEGEncoder]: qmax: 0 bitrate: 4194304 gop: 10
[zed_wrapper-2] [INFO] [1731763769.353973610] [zed.zed_node]: Advertised on topic: /zed/zed_node/rgb/image_rect_color
[zed_wrapper-2] [INFO] [1731763769.354003321] [zed.zed_node]: Advertised on topic: /zed/zed_node/rgb/camera_info
[zed_wrapper-2] [INFO] [1731763769.354563971] [FFMPEGEncoder]: enc: libx264 prof:  preset: 
[zed_wrapper-2] [INFO] [1731763769.354576773] [FFMPEGEncoder]: qmax: 0 bitrate: 4194304 gop: 10
[zed_wrapper-2] [INFO] [1731763769.355203338] [zed.zed_node]: Advertised on topic: /zed/zed_node/rgb_gray/image_rect_gray
[zed_wrapper-2] [INFO] [1731763769.355222219] [zed.zed_node]: Advertised on topic: /zed/zed_node/rgb_gray/camera_info
[zed_wrapper-2] [INFO] [1731763769.355689004] [FFMPEGEncoder]: enc: libx264 prof:  preset: 
[zed_wrapper-2] [INFO] [1731763769.355699193] [FFMPEGEncoder]: qmax: 0 bitrate: 4194304 gop: 10
[zed_wrapper-2] [INFO] [1731763769.356210057] [zed.zed_node]: Advertised on topic: /zed/zed_node/rgb_raw/image_raw_color
[zed_wrapper-2] [INFO] [1731763769.356223899] [zed.zed_node]: Advertised on topic: /zed/zed_node/rgb_raw/camera_info
[zed_wrapper-2] [INFO] [1731763769.356692657] [FFMPEGEncoder]: enc: libx264 prof:  preset: 
[zed_wrapper-2] [INFO] [1731763769.356702657] [FFMPEGEncoder]: qmax: 0 bitrate: 4194304 gop: 10
[zed_wrapper-2] [INFO] [1731763769.357190496] [zed.zed_node]: Advertised on topic: /zed/zed_node/rgb_raw_gray/image_raw_gray
[zed_wrapper-2] [INFO] [1731763769.357212798] [zed.zed_node]: Advertised on topic: /zed/zed_node/rgb_raw_gray/camera_info
[zed_wrapper-2] [INFO] [1731763769.357693345] [FFMPEGEncoder]: enc: libx264 prof:  preset: 
[zed_wrapper-2] [INFO] [1731763769.357703277] [FFMPEGEncoder]: qmax: 0 bitrate: 4194304 gop: 10
[zed_wrapper-2] [INFO] [1731763769.358200031] [zed.zed_node]: Advertised on topic: /zed/zed_node/left/image_rect_color
[zed_wrapper-2] [INFO] [1731763769.358214024] [zed.zed_node]: Advertised on topic: /zed/zed_node/left/camera_info
[zed_wrapper-2] [INFO] [1731763769.358683763] [FFMPEGEncoder]: enc: libx264 prof:  preset: 
[zed_wrapper-2] [INFO] [1731763769.358695083] [FFMPEGEncoder]: qmax: 0 bitrate: 4194304 gop: 10
[zed_wrapper-2] [INFO] [1731763769.359234638] [zed.zed_node]: Advertised on topic: /zed/zed_node/left_gray/image_rect_gray
[zed_wrapper-2] [INFO] [1731763769.359248867] [zed.zed_node]: Advertised on topic: /zed/zed_node/left_gray/camera_info
[zed_wrapper-2] [INFO] [1731763769.359700487] [FFMPEGEncoder]: enc: libx264 prof:  preset: 
[zed_wrapper-2] [INFO] [1731763769.359709781] [FFMPEGEncoder]: qmax: 0 bitrate: 4194304 gop: 10
[zed_wrapper-2] [INFO] [1731763769.360240636] [zed.zed_node]: Advertised on topic: /zed/zed_node/left_raw/image_raw_color
[zed_wrapper-2] [INFO] [1731763769.360254013] [zed.zed_node]: Advertised on topic: /zed/zed_node/left_raw/camera_info
[zed_wrapper-2] [INFO] [1731763769.360716294] [FFMPEGEncoder]: enc: libx264 prof:  preset: 
[zed_wrapper-2] [INFO] [1731763769.360726098] [FFMPEGEncoder]: qmax: 0 bitrate: 4194304 gop: 10
[zed_wrapper-2] [INFO] [1731763769.361252473] [zed.zed_node]: Advertised on topic: /zed/zed_node/left_raw_gray/image_raw_gray
[zed_wrapper-2] [INFO] [1731763769.361265939] [zed.zed_node]: Advertised on topic: /zed/zed_node/left_raw_gray/camera_info
[zed_wrapper-2] [INFO] [1731763769.361729878] [FFMPEGEncoder]: enc: libx264 prof:  preset: 
[zed_wrapper-2] [INFO] [1731763769.361740635] [FFMPEGEncoder]: qmax: 0 bitrate: 4194304 gop: 10
[zed_wrapper-2] [INFO] [1731763769.362340525] [zed.zed_node]: Advertised on topic: /zed/zed_node/right/image_rect_color
[zed_wrapper-2] [INFO] [1731763769.362359651] [zed.zed_node]: Advertised on topic: /zed/zed_node/right/camera_info
[zed_wrapper-2] [INFO] [1731763769.362851371] [FFMPEGEncoder]: enc: libx264 prof:  preset: 
[zed_wrapper-2] [INFO] [1731763769.362861603] [FFMPEGEncoder]: qmax: 0 bitrate: 4194304 gop: 10
[zed_wrapper-2] [INFO] [1731763769.363448065] [zed.zed_node]: Advertised on topic: /zed/zed_node/right_gray/image_rect_gray
[zed_wrapper-2] [INFO] [1731763769.363462700] [zed.zed_node]: Advertised on topic: /zed/zed_node/right_gray/camera_info
[zed_wrapper-2] [INFO] [1731763769.363947935] [FFMPEGEncoder]: enc: libx264 prof:  preset: 
[zed_wrapper-2] [INFO] [1731763769.363958251] [FFMPEGEncoder]: qmax: 0 bitrate: 4194304 gop: 10
[zed_wrapper-2] [INFO] [1731763769.364510009] [zed.zed_node]: Advertised on topic: /zed/zed_node/right_raw/image_raw_color
[zed_wrapper-2] [INFO] [1731763769.364545274] [zed.zed_node]: Advertised on topic: /zed/zed_node/right_raw/camera_info
[zed_wrapper-2] [INFO] [1731763769.364979249] [FFMPEGEncoder]: enc: libx264 prof:  preset: 
[zed_wrapper-2] [INFO] [1731763769.364987746] [FFMPEGEncoder]: qmax: 0 bitrate: 4194304 gop: 10
[zed_wrapper-2] [INFO] [1731763769.365499248] [zed.zed_node]: Advertised on topic: /zed/zed_node/right_raw_gray/image_raw_gray
[zed_wrapper-2] [INFO] [1731763769.365511492] [zed.zed_node]: Advertised on topic: /zed/zed_node/right_raw_gray/camera_info
[zed_wrapper-2] [INFO] [1731763769.365986537] [FFMPEGEncoder]: enc: libx264 prof:  preset: 
[zed_wrapper-2] [INFO] [1731763769.365995687] [FFMPEGEncoder]: qmax: 0 bitrate: 4194304 gop: 10
[zed_wrapper-2] [INFO] [1731763769.366534017] [zed.zed_node]: Advertised on topic: /zed/zed_node/depth/depth_registered
[zed_wrapper-2] [INFO] [1731763769.366547745] [zed.zed_node]: Advertised on topic: /zed/zed_node/depth/camera_info
[zed_wrapper-2] [INFO] [1731763769.367558558] [zed.zed_node]: Advertised on topic: /zed/zed_node/depth/depth_info
[zed_wrapper-2] [INFO] [1731763769.368066041] [FFMPEGEncoder]: enc: libx264 prof:  preset: 
[zed_wrapper-2] [INFO] [1731763769.368076931] [FFMPEGEncoder]: qmax: 0 bitrate: 4194304 gop: 10
[zed_wrapper-2] [INFO] [1731763769.368490050] [zed.zed_node]: Advertised on topic: /zed/zed_node/stereo/image_rect_color
[zed_wrapper-2] [INFO] [1731763769.368943732] [FFMPEGEncoder]: enc: libx264 prof:  preset: 
[zed_wrapper-2] [INFO] [1731763769.368968917] [FFMPEGEncoder]: qmax: 0 bitrate: 4194304 gop: 10
[zed_wrapper-2] [INFO] [1731763769.369389580] [zed.zed_node]: Advertised on topic: /zed/zed_node/stereo_raw/image_raw_color
[zed_wrapper-2] [INFO] [1731763769.369535006] [zed.zed_node]: Advertised on topic: /zed/zed_node/confidence/confidence_map
[zed_wrapper-2] [INFO] [1731763769.370861874] [zed.zed_node]: Advertised on topic: /zed/zed_node/disparity/disparity_image
[zed_wrapper-2] [INFO] [1731763769.389644530] [zed.zed_node]: Advertised on topic: /zed/zed_node/point_cloud/cloud_registered
[zed_wrapper-2] [INFO] [1731763769.390288060] [zed.zed_node]: Advertised on topic: /zed/zed_node/pose
[zed_wrapper-2] [INFO] [1731763769.390609627] [zed.zed_node]: Advertised on topic: /zed/zed_node/pose/status
[zed_wrapper-2] [INFO] [1731763769.391035910] [zed.zed_node]: Advertised on topic: /zed/zed_node/pose_with_covariance
[zed_wrapper-2] [INFO] [1731763769.392538836] [zed.zed_node]: Advertised on topic: /zed/zed_node/odom
[zed_wrapper-2] [INFO] [1731763769.392979969] [zed.zed_node]: Advertised on topic: /zed/zed_node/path_map
[zed_wrapper-2] [INFO] [1731763769.393286501] [zed.zed_node]: Advertised on topic: /zed/zed_node/path_odom
[zed_wrapper-2] [INFO] [1731763769.395935017] [zed.zed_node]: Advertised on topic: /zed/plane_marker
[zed_wrapper-2] [INFO] [1731763769.396751693] [zed.zed_node]: Advertised on topic: /zed/plane
[zed_wrapper-2] [INFO] [1731763769.397479070] [zed.zed_node]: Advertised on topic: /zed/zed_node/imu/data
[zed_wrapper-2] [INFO] [1731763769.397881555] [zed.zed_node]: Advertised on topic: /zed/zed_node/imu/data_raw
[zed_wrapper-2] [INFO] [1731763769.398287619] [zed.zed_node]: Advertised on topic: /zed/zed_node/temperature/imu
[zed_wrapper-2] [INFO] [1731763769.398622152] [zed.zed_node]: Advertised on topic: /zed/zed_node/left_cam_imu_transform
[zed_wrapper-2] [INFO] [1731763769.398646019] [zed.zed_node]: Camera-IMU Translation: 
[zed_wrapper-2]  0 0 0
[zed_wrapper-2] [INFO] [1731763769.398848149] [zed.zed_node]: Camera-IMU Rotation:
[zed_wrapper-2] 7A5FEDFFA030
[zed_wrapper-2] 1.000000 0.000000 0.000000
[zed_wrapper-2] 0.000000 1.000000 0.000000
[zed_wrapper-2] 0.000000 0.000000 1.000000
[zed_wrapper-2] 
[zed_wrapper-2] [INFO] [1731763769.398856973] [zed.zed_node]: *** Subscribers ***
[zed_wrapper-2] [INFO] [1731763769.399339818] [zed.zed_node]:  * Plane detection: '/clicked_point'
[zed_wrapper-2] [INFO] [1731763769.400128739] [zed.zed_node]:  * Sim Clock: '/clock'
[zed_wrapper-2] [WARN] [1731763769.400316291] [zed.zed_node]: Waiting for a valid simulation time on the '/clock' topic...
[zed_wrapper-2] [INFO] [1731763769.430281704] [zed.zed_node]: *** Starting Positional Tracking ***
[zed_wrapper-2] [INFO] [1731763769.430327801] [zed.zed_node]:  * Waiting for valid static transformations...
[zed_wrapper-2] [INFO] [1731763769.430415297] [zed.zed_node]:  Static transform ref. CMOS Sensor to Base [zed_left_camera_frame -> zed_camera_link]
[zed_wrapper-2] [INFO] [1731763769.430426739] [zed.zed_node]:   * Translation: {0.010,-0.060,-0.016}
[zed_wrapper-2] [INFO] [1731763769.430437811] [zed.zed_node]:   * Rotation: {0.000,-0.000,0.000}
[zed_wrapper-2] [INFO] [1731763769.430455165] [zed.zed_node]:  Static transform ref. CMOS Sensor to Camera Center [zed_left_camera_frame -> zed_camera_center]
[zed_wrapper-2] [INFO] [1731763769.430462354] [zed.zed_node]:   * Translation: {0.010,-0.060,0.000}
[zed_wrapper-2] [INFO] [1731763769.430470586] [zed.zed_node]:   * Rotation: {0.000,-0.000,0.000}
[zed_wrapper-2] [INFO] [1731763769.430487477] [zed.zed_node]:  Static transform Camera Center to Base [zed_camera_center -> zed_camera_link]
[zed_wrapper-2] [INFO] [1731763769.430494291] [zed.zed_node]:   * Translation: {0.000,0.000,-0.016}
[zed_wrapper-2] [INFO] [1731763769.430502403] [zed.zed_node]:   * Rotation: {0.000,-0.000,0.000}
[zed_wrapper-2] [INFO] [1731763769.431590974] [zed.zed_node]: Initial ZED left camera pose (ZED pos. tracking): 
[zed_wrapper-2] [INFO] [1731763769.431615807] [zed.zed_node]:  * T: [-0.01,0.06,0.016]
[zed_wrapper-2] [INFO] [1731763769.431640504] [zed.zed_node]:  * Q: [0,0,0,1]
[zed_wrapper-2] [2024-11-16 13:29:29 UTC][ZED][WARNING] Positional Tracking GEN2 with IMU fusion enabled requires high frequency sensors data (available with Streaming version 2, with ZED SDK >=4.1). IMU Fusion will be disabled, performance will be degraded.
[zed_wrapper-2] [2024-11-16 13:29:30 UTC][ZED][WARNING] Factory calibration bias is not set, disabling safety guard for IMU initialization
[zed_wrapper-2] [WARN] [1731763770.613665097] [zed.zed_node]: Pos. Tracking not started: SENSORS DATA REQUIRED FOR THIS MODULE
[zed_wrapper-2] [INFO] [1731763770.701501670] [zed.zed_node]: *** Starting Positional Tracking ***
[zed_wrapper-2] [INFO] [1731763770.701543223] [zed.zed_node]:  * Waiting for valid static transformations...
[zed_wrapper-2] [INFO] [1731763770.702634312] [zed.zed_node]: Initial ZED left camera pose (ZED pos. tracking): 
[zed_wrapper-2] [INFO] [1731763770.702660612] [zed.zed_node]:  * T: [-0.01,0.06,0.016]
[zed_wrapper-2] [INFO] [1731763770.702688187] [zed.zed_node]:  * Q: [0,0,0,1]
[zed_wrapper-2] [2024-11-16 13:29:30 UTC][ZED][WARNING] Positional Tracking GEN2 with IMU fusion enabled requires high frequency sensors data (available with Streaming version 2, with ZED SDK >=4.1). IMU Fusion will be disabled, performance will be degraded.
[zed_wrapper-2] [2024-11-16 13:29:32 UTC][ZED][WARNING] Factory calibration bias is not set, disabling safety guard for IMU initialization
[zed_wrapper-2] [WARN] [1731763772.204579440] [zed.zed_node]: Pos. Tracking not started: SENSORS DATA REQUIRED FOR THIS MODULE
[zed_wrapper-2] [INFO] [1731763772.207150229] [zed.zed_node]: *** Starting Positional Tracking ***
[zed_wrapper-2] [INFO] [1731763772.207180375] [zed.zed_node]:  * Waiting for valid static transformations...
[zed_wrapper-2] [INFO] [1731763772.208253846] [zed.zed_node]: Initial ZED left camera pose (ZED pos. tracking): 
[zed_wrapper-2] [INFO] [1731763772.208268321] [zed.zed_node]:  * T: [-0.01,0.06,0.016]
[zed_wrapper-2] [INFO] [1731763772.208284494] [zed.zed_node]:  * Q: [0,0,0,1]
[zed_wrapper-2] [2024-11-16 13:29:32 UTC][ZED][WARNING] Positional Tracking GEN2 with IMU fusion enabled requires high frequency sensors data (available with Streaming version 2, with ZED SDK >=4.1). IMU Fusion will be disabled, performance will be degraded.
[zed_wrapper-2] [2024-11-16 13:29:33 UTC][ZED][WARNING] Factory calibration bias is not set, disabling safety guard for IMU initialization
[zed_wrapper-2] [WARN] [1731763773.723875694] [zed.zed_node]: Pos. Tracking not started: SENSORS DATA REQUIRED FOR THIS MODULE
[zed_wrapper-2] [FATAL] [1731763773.723931371] [zed.zed_node]: It's not possible to enable the required Positional Tracking module.
[ERROR] [zed_wrapper-2]: process has died [pid 10921, exit code -11, cmd '/home/ubuntu/Desktop/uosm-cirg/sl_ros2/install/zed_wrapper/lib/zed_wrapper/zed_wrapper --ros-args -r __node:=zed_node -r __ns:=/zed --params-file /home/ubuntu/Desktop/uosm-cirg/sl_ros2/install/zed_wrapper/share/zed_wrapper/config/common.yaml --params-file /home/ubuntu/Desktop/uosm-cirg/sl_ros2/install/zed_wrapper/share/zed_wrapper/config/zedx.yaml --params-file /home/ubuntu/Desktop/uosm-cirg/sl_ros2/install/zed_wrapper/share/zed_wrapper/config/ffmpeg.yaml --params-file /tmp/launch_params_mjb5lcko'].

What i have tried

  1. Setting physics time_step to 120fps
  2. Set camera fps to VGA resolution and frame rate to 120fps
  3. Set zed-ros2-wrapper config to publish imu at camera rate and sync image
    Screenshot from 2024-11-16 21-36-33
    Screenshot from 2024-11-16 21-38-47
    Screenshot from 2024-11-16 21-40-50

According to following line, the imu data and synthetic images from isaac sim are streaming at the same rate.
Any solution to this or the position tracking with imu is not possible as of now in simulation?
Is it possible to stream the imu at higher rate compared to the image?

res = db.internal_state.pyzed_streamer.stream(

Changing the zed-ros2-wrapper common.yaml file with either of the following settings will allow the pos_tracking module to work:
pos_tracking_mode: "GEN_1"
imu_fusion: false

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant