ggshield secret scan docker fails with recent version of Docker

[agateau-gg]

Environment

• ggshield version: 1.14.3
• Operating system (Linux, macOS, Windows): macOS
• Operating system version: -
• Python version: -

Describe the bug

ggshield secret scan docker fails with an error message like this:

$ ggshield secret scan docker --verbose gitguardian/ggshield
Saving docker image... OK

ERROR: invalid header.

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/ggshield/1.14.2/libexec/lib/python3.10/site-packages/ggshield/cmd/secret/scan/docker.py", line 54, in docker_name_cmd
    scan = docker_scan_archive(
  File "/opt/homebrew/Cellar/ggshield/1.14.2/libexec/lib/python3.10/site-packages/ggshield/scan/docker.py", line 257, in docker_scan_archive
    files = get_files_from_docker_archive(archive)
  File "/opt/homebrew/Cellar/ggshield/1.14.2/libexec/lib/python3.10/site-packages/ggshield/scan/docker.py", line 62, in get_files_from_docker_archive
    return Files(list(chain((config_file_to_scan,), layer_files_to_scan)))
  File "/opt/homebrew/Cellar/ggshield/1.14.2/libexec/lib/python3.10/site-packages/ggshield/scan/docker.py", line 136, in _get_layers_files
    yield from _get_layer_files(archive, layer_info)
  File "/opt/homebrew/Cellar/ggshield/1.14.2/libexec/lib/python3.10/site-packages/ggshield/scan/docker.py", line 157, in _get_layer_files
    layer_archive = tarfile.TarFile(
  File "/opt/homebrew/Cellar/[email protected]/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/tarfile.py", line 1542, in __init__
    self.firstmember = self.next()
  File "/opt/homebrew/Cellar/[email protected]/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/tarfile.py", line 2362, in next
    raise ReadError(str(e)) from None
tarfile.ReadError: invalid header

Steps to reproduce:

1. Make sure you have a recent enough version of Docker. This version is known to cause the crash:

$ docker version
Client:
 Cloud integration: v1.0.29
 Version:           20.10.22
 API version:       1.41
 Go version:        go1.18.9
 Git commit:        3a2c30b
 Built:             Thu Dec 15 22:28:41 2022
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      trueServer: Docker Desktop 4.16.2 (95914)
 Engine:
  Version:          22.06.0-beta.0-917-gf888bd4531.m
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.18.4
  Git commit:       f888bd4531
  Built:            Wed Jan 18 09:46:59 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.14
  GitCommit:        9ba4b250366a5ddde94bb7c9d1def331423aa323
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

2. Run command ggshield secret scan docker <anyimage>

Actual result:

It crashes.

Expected result:

Scans work as expected.

Explanation

It seems Docker changed the file format used to save images. It used to use its own format, documented here: https://github.com/moby/moby/blob/master/image/spec/v1.md. This is the format we currently supports. Nowadays it uses the OCI file format, documented here: https://github.com/opencontainers/image-spec/blob/main/spec.md.

TODO

• Add support for the new format
• The old format must continue to be supported: the Docker version shipped in many Linux distributions (in particular the current Ubuntu LTS: 22.04) still use this format