You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#197 added an option to force columns to use the Large variants of String and Binary to handle columns with data larger than 2GB. We should take this one step further to automatically reduce the batch size of records to fit within int32 offsets if the column data is too large to fit within a single String/Binary column (data is more than 2GB).
Component(s)
Parquet
The text was updated successfully, but these errors were encountered:
Describe the enhancement requested
#197 added an option to force columns to use the Large variants of String and Binary to handle columns with data larger than 2GB. We should take this one step further to automatically reduce the batch size of records to fit within int32 offsets if the column data is too large to fit within a single String/Binary column (data is more than 2GB).
Component(s)
Parquet
The text was updated successfully, but these errors were encountered: