Complex fails for 24 bits per value

[joobog]

Complex algorithm fails for some data at 24 bits per value:

+ wgrib2 --version
3.4.0
+ wgrib2 kwbc_simple.grib -set_grib_type c3 -set_grib_max_bits 24 -grib_out kwbc_complex.grib
*** FATAL ERROR: add_many_bitstream: n_bits = (26) ***

kwbc_simple.grib.zip

[webisu]

The 24-bit scaling is applied to the range of values. For complex-1, the actual values are stored. For complex-2, the deltas are stored. The 24-bit scaling works for both these versions of complex packing. For complex-3, the deltas of the delta are stored. In your example, the deltas of the delta need 26 bits to store. The code that determines the scaling needs to be fixed to prevent the number of bits being too large. For fields like precip, complex-1 compresses better than complex-3. For long term archives, it is worth the CPU to find the best compression method on a field by field . We routinely use 4 core solutions to create an "optimal" compressed grib files.

…

On Mon, Nov 18, 2024 at 1:46 PM joobog ***@***.***> wrote: Complex algorithm fails for some data at 24 bits per value: + wgrib2 --version 3.4.0 + wgrib2 kwbc_simple.grib -set_grib_type c3 -set_grib_max_bits 24 -grib_out kwbc_complex.grib *** FATAL ERROR: add_many_bitstream: n_bits = (26) *** kwbc_simple.grib.zip <https://github.com/user-attachments/files/17801060/kwbc_simple.grib.zip> — Reply to this email directly, view it on GitHub <#292>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEIB7ZWNKJSM26SD7YLAZ5L2BHVTLAVCNFSM6AAAAABR7WJMN2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGY3DQNRRHEZTGOI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[edwardhartnett]

@webisu could you submit a PR with that information added to the documentation?

In general, all such questions should result in documentation update, not just answering the question. For every person who asks a question, there are 10 times as many who will wonder the same thing but won't ask. So let's put out answers out for everyone.

[joobog]

@webisu, thanks for the detailed explanation. I hope this problem can be fixed soon.
@edwardhartnett, I agree, the comment from @webisu should be made public. The information is very useful.