You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Here is the native plan for a join. The join metrics of build_time and join_time get reported in the Spark UI but we do not report the metrics for fetching the input batches from the JVM or for unpacking dictionaries and performing deep copies where needed.
For this example it means we are reporting a time of ~410ms when the actual time is closer to ~600ms, and this is just for one partition.
Describe the bug
Here is the native plan for a join. The join metrics of
build_time
andjoin_time
get reported in the Spark UI but we do not report the metrics for fetching the input batches from the JVM or for unpacking dictionaries and performing deep copies where needed.For this example it means we are reporting a time of ~410ms when the actual time is closer to ~600ms, and this is just for one partition.
Steps to reproduce
No response
Expected behavior
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: