Skip to content

[PySpark] Improve validation performance by enabling cache()/unpersist() toggles #2206

[PySpark] Improve validation performance by enabling cache()/unpersist() toggles

[PySpark] Improve validation performance by enabling cache()/unpersist() toggles #2206

Job Run time
1m 26s
1m 27s
1m 26s
7m 4s
1m 42s
7m 7s
7m 6s
5m 37s
5m 35s
5m 33s
5m 20s
59s
7m 6s
7m 5s
6m 1s
6m 14s
7m 5s
7m 7s
7m 6s
7m 1s
1s
1s
1s
1s
1s
1s
1s
1s
7m 7s
7m 10s
7m 4s
7m 4s
7m 12s
1s
1s
1s
1s
1s
1s
1s
1s
1s
1s
1s
1s
30s
1s
1s
1s
1s
1s
1s
2h 21m 40s