Add missing codestream marker segments

[bitsgalore]

Dev Effort

TBC

Description

Currently jpylyzer doesn’t support the full set of marker segments that can occur in a codestream. Missing ones should ideally be included at some point (although some of them are pretty rare).

A good first step would be to create empty validator functions for all missing marker segments, which is quick and easy to implement. This would allow jpylyzer to report at least the presence of these markers in a file (without providing any details about its fields). This is already very useful information by itself, as some decoders appear to choke on some of the more obscure marker segments, so knowing they're there can already be helpful.

[bitsgalore]

Minimal support for missing marker segments added in 1.7.0. Full support to follow later ..

[bitsgalore]

Started with COC marker here. Turns out the SPCOC parameter follows the same format as SPCOD in the COD marker. So first refactor code for parsing in COD, then re-use in COC.

Similarly the QCC marker reuses things from QCD.

[bitsgalore]

For additional test images see JP2s in Tika Regression corpus:

http://162.242.228.174/share/jp2.tgz

Update - this yielded images with qcc, tlm and crg markers!

The openjpeg-data repo contains some more images with coc markers.

[bitsgalore]

UPDATE: current jpylyzer-2-dev branch now includes validation of COC, QCC, POC, RGN and CRG markers segments, which means that all delimiting (section A.4), fixed information (section A.5), functional (section A.6) and informational marker segments (section A.9) are now covered. Not included (yet) are:

• Pointer marker segments (section A.7): TLM, PLM, PLT, PPM, PPT
• In-bit-stream marker segments (section A.8): SOP, EPH. These are packet-level error resilience marker segments which should probably remain out of scope because including them would involve parsing the bit stream, which would probably slow down jpylyzer quite a lot and also result in overly verbose output.

Note that jpylyzer-test-files now has at least 1 test image for all of the above marker segments.

[VirtualTim]

PR for plt segment here: #170, which will further this issue a bit.

[bitsgalore]

Further update: @boxerab provided some useful pointers (sample files + code) for TLM parsing here:

Here is an example of an image with a corrupt TLM marker:

https://l3harrisgeospatial-webcontent.s3.amazonaws.com/MM_Samples/Pleiades_ORTHO_UTM_BUNDLE.zip

also, here is some code that parses a TLM marker

[JBoisseau-ByDeluxe]

Hi @bitsgalore
I'm interested in parsing TLM.

I did a first version of it, seems to work fine

` def validate_tlm(self):
"""Tile-part lengths, main header (TLM) marker segment (ISO/IEC 15444-1 Section A.7.1).
"""

    # Length of TLM marker
    ltlm = bc.bytesToUShortInt(self.boxContents[0:2])
    self.addCharacteristic("ltlm", ltlm)

    # TLM marker segment index
    ztlm = bc.bytesToUnsignedChar(self.boxContents[2:3])
    self.addCharacteristic("ztlm", ztlm)

    # Size of Ttlm and Ptlm
    stlm = bc.bytesToUnsignedChar(self.boxContents[3:4])
    st = (stlm >> 4) & 3
    self.addCharacteristic("st", st)
    self.testFor("tlmStIsValid", (st in [0,1,2]))

    sp = (stlm >> 6) & 1
    self.addCharacteristic("sp", sp)
    self.testFor("tlmSpIsValid", (sp in [0,1]))

    ttlmLength = st
    ptlmLength = (sp + 1) *2

    tilePartsCount = (ltlm - 4) / ( ttlmLength + ptlmLength )
    tlmIsValid = tilePartsCount.is_integer()
    self.testFor("tlmIsValid", tlmIsValid)

    if tlmIsValid:
        offset = 4
        # iterate each tilepart Length
        for _ in range(int(tilePartsCount)):
            ttlm = bc.bytesToInteger(self.boxContents[offset:offset+ttlmLength])
            self.addCharacteristic("ttlm", ttlm)
            offset = offset+ttlmLength

            ptlm = bc.bytesToInteger(self.boxContents[offset:offset+ptlmLength])
            self.addCharacteristic("ptlm", ptlm)
            offset = offset+ptlmLength

`

It produces those characteristics:
"tlm": { "ltlm": 22, "ztlm": 0, "st": 2, "sp": 1, "ttlm": [ 0, 0, 0 ], "ptlm": [ 428108, 143974, 77610 ] },

If you're fine with the code, would you mind I contribute ?