Progress-bar for ros2 bag play

[nicolaloi]

Closes #1762: Adds a progress bar for ros2 bag play, showing the bag time and duration, similar to what is seen in ROS1.

In ROS1, the progress bar was easy to handle because the rosbag play command didn't print additional output during playback. However, in ROS2, many logs can be printed during playback (e.g., keyboard control logs).

The progress bar is printed and continuously updated on the last line of the playback output. I have tested this initial implementation with keyboard controls (play, pause, play next, set rate), Ctrl+C, throwing an exception, and the flags --log-level debug, --delay X, and --start-offset X.

To try it, use the new --progress-bar flag when playing a bag.

[nicolaloi]

video-example.mp4

(at 2.6-2.7 seconds of bag duration, the progress bar is slightly updating while in paused state because I was clicking the left arrow for the play next keyboard control)

Since ros2 bag play generates more logs than its ROS1 counterpart, the key aspect of this PR is to print a string shorter than any potential log message not coming from rosbag2_player. For example, with the current draft of the progress bar, this prints nicely:

[INFO] [1729028986.845909253] [rosbag2_player]: Pausing play.
 Bag Time 1718553782.525239  Duration 1.272325/6.974378 [P]

while this longer version of the progress bar doesn't work properly without extra line clearing (in this case, it's easy to clear the line since the logs come from rosbag2_player., but for "external" logs, this becomes more difficult):

[INFO] [1729028986.845909253] [rosbag2_player]: Pausing play.us:Running]
 Bag Time 1718553782.525239  Duration 1.272325/6.974378 [status:Paused]

@MichaelOrlov looking forward to your feedback, for now I have made a test MVP code to check the feasibility of the feature.

[MichaelOrlov]

@nicolaloi Thank you for your contribution.
I have a few concerns about the design and implementation.
First, we need to limit the frequency with which we will print in std::cout. The best way to do this is to create a local variable, aka size_t progress_bar_update_freq_ (in Hz).
e.g. 1 Hz = once per/sec, 2 Hz = twice per second, 3 = three times per second etc..
Then, check the delay between the last call in the print_status(..). If the delay is more than the threshold, print; otherwise, skip.
We also need to move the check for the if (disable_progress_bar_) inside print_status(..) to make the logic cleaner and in one place.
Calling double current_time_secs = RCUTILS_NS_TO_S(static_cast<double>(clock_->now())); each time in the print_status(..) is very inefficient and expensive since we are locking mutex inside and transorming current time to the ROS time. We actually know the current ROS time at which we are playing. This is a timestamp from the message. I would suggest passing it to the print function as a parameter.
i.e., add tiemstamp parameter to the print_status(..) function and use message_ptr->recv_timestamp as a source of timestamp.

I found the implementation of the print_status(..) function a bit messy.
That is a quintessence of the modern C++17 with templates and old style plain C code.
Need to avoid using C code. Please use C++ alternatives. I am not sure if we really need to flush buffer each time. Flush is an expensive operation since this is a blocking system call. Need to avoid it if possible.

IMO templating print_status(..) function is overengineering and make code less readable and less understandable. The cleaner solution would be if using enum class values for different modes. In this case all extra wrappers like check_and_print_status() will not be needed. We will be able to call print_status(PrintStatus::Paused) directly from where we needed it.

[nicolaloi]

@MichaelOrlov thanks for the comments.

• limit printing frequency, mutex and flushing: yes I already had a local change to allow the user to choose the update frequency from the CLI (default 10-20 Hz) to reduce the mutex locking and output stream flushing. But indeed using the timestamp of the last published message is better than calling clock_->now().
  -- The rationale of clock_->now() was to be able to have a consistent time flow if you want to print the progress bar at a fixed rate but messages are not close in time. For example, if the next message is 3 seconds in the future, for the next 3 seconds the progress bar time will appear frozen since it is printing the same timestamp. But yes in practice is a niche case.
  -- Regarding the flush, updating the progress bar without flushing will not update the printed time (or anyway it will get flushed automatically after some update calls, but there can be artifacts if other external ros logs are being printed). So instead of calling the progress bar without flushing, I would just reduce the calls to the progress bar printing with the already mentioned user-defined frequency update. I would say that 10-20 Hz can be a good default value? Or even lower to be conservative?
• add timestamp parameter to the print_status: for this I would save it as a class attribute and use it within print_status instead of adding it as an input, since in many instances where print_status is being called there is no direct easy access to the last message.
• print_status implementation: ok I will remove the template and use enums.

Should the progress bar be disabled by default, or enabled by default with a default low update frequency?

[nicolaloi]

I have made changes as discussed.

Linked modifications:

I have modified the play_next function to accept a boolean input argument 'is_burst_mode' (default to false); so when play_next is called in burst mode, the print_status is not called at all, to avoid any delay (but now the progress bar will freeze during burst).

print_status will update the progress bar at the defined update frequency if the status is RUNNING; for the other states (or if the new boolean argument 'force_update' is true) it will update it disregarding the update frequency. This is to take care of the istances where a new log message from the rosbag player is being printed (like 'rate changed to 1.1', 'player paused', warnings..., anyway these are not high-frequency logging) to make the progress bar visible again. Otherwise, it will temporarily or permanently (permanently if the player gets paused) disappear since it is overwritten in the terminal by the previous log message.

[MichaelOrlov]

@nicolaloi I have not had a chance yet to look deeply into the code after your latest changes. However, I have a few comments.

1. I am ok with a flush; I anticipated that it is likely some constraints as you described.
2. Making the progress bar enabled by default is a good idea. However, the frequency should be of integer type (size_t) in C++, and 10 Hz IMO is to fast. I would put 3-5 Hz at max by default.
3. We don't need flag play_options.disable_progress_bar at all we can rely on the play_options.progress_bar_update rate. i.e. if it is 0 = disabled.

[nicolaloi]

Ok, I'll set the default rate to 3 Hz.

For 2nd and 3rd points, there is no longer the --disable_progress_bar option, but a --progress_bar option that saves the frequency rate as a double, and if the value is ==0 the progress bar is disabled, and if <0 it is updated for every message. So the user has the extra option to enable it for each message if they need to, without having to set a large enough frequency rate. If it's not okay I'll change to size_t as you suggested.

[nicolaloi]

I have realized the newly added variables progress_bar_last_time_printed_ and recv_timestamp_last_published_ in commit 43fa0f8 could have synchronization issues since the function print_progress_bar_status can be called contemporaneously from a service callback (like /play, /pause, /play_next), I need to take a better look at them.

[nicolaloi]

@MichaelOrlov So, to avoid synchronization issues without using mutexes, etc to not increase overhead, the variable progress_bar_last_time_printed_, which tracks the last update time of the progress bar, is now accessed exclusively within the specialized function update_progress_bar_after_message_published. This function is called in code sections that are already protected by an existing mutex (skip_message_in_main_play_loop_mutex_). The progress bar updates at the specified rate only during the "standard" and recurrent event of a message being published.
For all other "extraordinary" and less common events occurring in separate threads (user triggers pause, resume, change rate, play next, etc, via keyboard controls or services) the function update_progress_bar is used. To avoid synchro issues this function does not access progress_bar_last_time_printed_. It will always print the progress bar so it appears again immediately after the new log messages; however, it does not update progress_bar_last_time_printed_. If I can use a mutex or an atomic int64, I can update progress_bar_last_time_printed_ for these "extraordinary" events as well (i.e. ensuring the progress bar timing accounts for these extraordinary updates), but I don't think is needed given they are not common.

Regarding how to print the current timestamp: when a message is published, the update_progress_bar_after_message_published function is invoked with the message's timestamp as input as suggested. However, for the "extraordinary" events (user triggers pause, resume, change rate, play next, etc, via keyboard controls or services) where the timestamp is not directly available, the saved timestamp of the latest published message (recv_timestamp_last_published_) is not used anymore, since it could have synchro issues, but clock_->now() is used. This locks the inner mutex of clock_, but these events are very rare (or may not occur at all) compared to message publishing.

Extras:

• fixed printed timestamp when extarordinary events occur while in delayed status (used an atomic bool, but it is accessed only for these rare extraordinary events), related to Rosbag2 play --clock option ignores delay #1858 (clock_ is not paused when in delayed status);
• added burst status [B], so in case of a large number of messages being burst, the progress bar will not be frozen at [P]aused status. It will be frozen at [B]urst status, with the [B] conveying the information that messages are currently being burst (progress bar timestamp is frozen because normal continuous updates are blocked during burst to avoid delays);
• reduced the fixed precision of the displayed bag duration, from microseconds to milliseconds (Duration 1.862/74.434), to reduce the overall length of the progress bar string. Anyway, the displayed bag time (Time 1718553783.196209) has still microsecond precision.