arki-query --sort option doesn't sort correctly by timerange

[brancomat]

Trying to sort by timerange results in forecast at +0 as the last grib (it should be the first one).
Attaching a test grib (the first query can use the same grib as a source)
qualcosa.tar.gz

arki-query --sort timerange --data "Reftime:=today 0:00 ; product: GRIB1,98,128,167 ; level: GRIB1,1" http://arkiope.metarpa:8090/dataset/ifs_ita010 > qualcosa.grib

$ grib_ls qualcosa.grib 
qualcosa.grib
edition      centre       typeOfLevel  level        dataDate     stepRange    dataType     shortName    packingType  gridType     
1            ecmf         surface      0            20211228     3            fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     6            fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     9            fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     12           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     15           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     18           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     21           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     24           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     27           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     30           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     33           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     36           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     39           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     42           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     45           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     48           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     51           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     54           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     57           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     60           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     63           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     66           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     69           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     72           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     75           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     78           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     81           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     84           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     87           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     90           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     96           fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     102          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     108          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     114          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     120          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     126          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     132          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     138          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     144          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     150          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     156          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     162          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     168          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     174          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     180          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     186          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     192          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     198          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     204          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     210          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     216          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     222          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     228          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     234          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     240          fc           2t           grid_simple  regular_ll  
1            ecmf         surface      0            20211228     0            fc           2t           grid_simple  regular_ll  
56 of 56 messages in qualcosa.grib

56 of 56 total messages in 1 files

[brancomat]

Ok, it's actually a bit more complicated than that:
The forecast at +0 it's an initialized analysis and its timerange is coded differently from the rest (timeRangeIndicator 1, see: https://apps.ecmwf.int/codes/grib/format/grib1/ctable/5/).

In general:

• Uninitialized analysis is coded with timeRangeIndicator = 0, stepRange = 0, marsType = 2 and for arkimet GRIB1(000, 000h)
• Initialized analysis is coded with timeRangeIndicator = 1, stepRange = 0, marsType = 9 and for arkimet GRIB1(001)

And this is probably the reason it ends up in the last grib.
Maybe it's technically correct (the best kind of correct), still it would be great to have a way to sort the output in a chronological way.

[spanezz]

Ok, at the moment sorting of timeranges is mostly lexicographical sorting of the timerange type and components. From this ticket I understand it's not that useful, and that we could define a better sorting. Let's then figure out how to define a better sorting for timeranges.

There are currently 4 types of timeranges:

• GRIB1 (type, unit, p1, p2), with unit, p1, or p2 existing or not depending on type
• GRIB2 (type, unit, p1, p2), with unit, p1, or p2 existing or not depending on type
• Timedef (length, length_unit) or (length, length_unit, stat_type, stat_length, stat_length_unit)
• BUFR (value, unit)

Some, but not all of them, can be converted to Timedef.

Comparison needs to be defined for any combination of all of them.

A possible trategy could be to define a comparison function between timedef, then compare first everything that can map to timedef, and everything that cannot map to timedef can be sorted later lexicographically.

Would that help? If so, how do we sort Timedefs?

[brancomat]

A possible strategy could be to define a comparison function between timedef, then compare first everything that can map to timedef, and everything that cannot map to timedef can be sorted later lexicographically.

This sounds good, I don't know if it's feasible but to keep backward compatibility it could be added a syntax to the --sort timerange option to switch sorting by timedef instead of the current sorting.

Another partially related aspect is that at the moment there's no way to see the Timedef of a set of data in the summary. While somewhat redundant (it duplicates the Timerange info) this could be useful, especially when/if the Timedef will be included in sorting.
Some possible alternatives:

• showing it alongside the current timerange as a sort of translation
• adding a switch to the arki-query to express the timerange as Timedef