Delete only handles docker v2 image manifests

[sudo-bmitch]

If I push a multi-platform image, or an OCI image, they aren't being cleaned after the timeout. I believe it's related to

ttl.sh/hooks/src/commands/reap.ts

Line 52 in f42ce4d

"Accept": "application/vnd.docker.distribution.manifest.v2+json",

which only has one accept header. In my own projects, I accept:

• application/vnd.docker.distribution.manifest.v1+json
• application/vnd.docker.distribution.manifest.v1+prettyjws
• application/vnd.docker.distribution.manifest.v2+json
• application/vnd.docker.distribution.manifest.list.v2+json
• application/vnd.oci.image.manifest.v1+json
• application/vnd.oci.image.index.v1+json

The registry itself will automatically resolve a multi-platform image to linux/amd64 by default if you don't request the manifest list, so you end up with the following (5 minutes had passed):

$ docker pull --platform linux/amd64 ttl.sh/library/busybox:5m
5m: Pulling from library/busybox
manifest for ttl.sh/library/busybox:5m not found: manifest unknown: manifest unknown

$ docker pull --platform linux/arm/v5 ttl.sh/library/busybox:5m
5m: Pulling from library/busybox
3a7a3789f986: Pull complete 
Digest: sha256:139abcf41943b8bcd4bc5c42ee71ddc9402c7ad69ad9e177b0a9bc4541f14924
Status: Downloaded newer image for ttl.sh/library/busybox:5m
ttl.sh/library/busybox:5m

[marccampbell]

Ah yes. @sudo-bmitch This is a good find. We should definitely be accepting and cleaning OCI images (and multi-arch). It would be relatively easy to just update the list of Accept headers to the list you provided, I'm wondering what else might be out there too.

[sudo-bmitch]

There are some proposals to add more (a new artifact-spec and a manifest for a single descriptor come to mind) but I haven't seen anything else accepted by OCI yet. There could always by other projects out there, not sure how much validation the registry image does on that media type and if it allows unknown manifests.

Here's the OCI list: https://github.com/opencontainers/image-spec/blob/main/media-types.md

[marccampbell]

Thanks. This still seems pretty doable, just iterating over this list. I'm happy to give it a try if nobody else does, and add some tests to validate that it works.

[sudo-bmitch]

I'm wondering what else might be out there too.

If my reading of the registry code is correct, the provided list covers them all and anything else will trigger a marshalling error. It looks like schema v1 was also disabled in your registry config, which removes the need for those older docker media types.

https://github.com/distribution/distribution/blob/main/registry/handlers/manifests.go

[sudo-bmitch]

Thanks. This still seems pretty doable, just iterating over this list. I'm happy to give it a try if nobody else does, and add some tests to validate that it works.

I'm not much of a typescript coder, so I'll leave the fix up to those that know it. You should be able to pass all of the accept entries in one http request and the server will include a content-type header that lets you know which one it found.

[linhklattsolo]

Hi, I need to check the image status which Manifest version is it supporting please execute this command and send its result here.

Command: docker manifest inspect ttl.sh/library/busybox:5m

[sudo-bmitch]

Hi, I need to check the image status which Manifest version is it supporting please execute this command and send its result here.

Command: docker manifest inspect ttl.sh/library/busybox:5m

I'm guessing the manifest inspect is trying to resolve the platform. The image itself is a multi-platform image where only the linux/amd64 platform manifest was deleted.

$ docker manifest inspect ttl.sh/library/busybox:5m
no such manifest: ttl.sh/library/busybox:5m

$ regctl manifest get --list ttl.sh/library/busybox:5m
Name:        ttl.sh/library/busybox:5m
MediaType:   application/vnd.docker.distribution.manifest.list.v2+json
Digest:      sha256:139abcf41943b8bcd4bc5c42ee71ddc9402c7ad69ad9e177b0a9bc4541f14924
             
Manifests:   
             
  Name:      ttl.sh/library/busybox@sha256:6066ca124f8c2686b7ae71aa1d6583b28c6dc3df3bdc386f2c89b92162c597d9
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/amd64
             
  Name:      ttl.sh/library/busybox@sha256:399e1e4a0d587717dc9e3a85150cec8498cb6dc73dcb7eddb94959fedb331104
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm/v5
             
  Name:      ttl.sh/library/busybox@sha256:4ecc3dc2e06a24df931cb719c3784611d15721c3cb64ab069141071b73f6598b
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm/v6
             
  Name:      ttl.sh/library/busybox@sha256:53c212bcc0501f011c232df0fb6c837651d0b2f3257b6478a50c0e006b0dabc5
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm/v7
             
  Name:      ttl.sh/library/busybox@sha256:ce53e9b0310447d0e851ff0d2c9b90f358dbffe719a723147e84b93a4799396c
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm64/v8
             
  Name:      ttl.sh/library/busybox@sha256:0bec409945b4e48517645d4f1bc44a965dc09a0c9647be4494f82ce0f1306c27
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/386
             
  Name:      ttl.sh/library/busybox@sha256:77df281071dd7e01972ec2c4a33c1a6c00d13b24238375fd6622fce97f622fa2
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/mips64le
             
  Name:      ttl.sh/library/busybox@sha256:b70f0f45692830c2990b42f770aa29488c20ac41f1c3dcaa242920b73cb1399b
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/ppc64le
             
  Name:      ttl.sh/library/busybox@sha256:06b206c1f1a38094697c7e8bf868f9d326e56a256bc516dbb8ff0ee9c1178999
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/riscv64
             
  Name:      ttl.sh/library/busybox@sha256:86824a27910bd2a8c6a8478fe99206e6cf4bcada7cb8435c0060cbe885559e53
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/s390x

[linhklattsolo]

I have some questions.
1: Docker image name ttl.sh/library/busybox:5m where i can found this image Dockerfile?

Because what i have get is this repo https://github.com/miguelvizdev/ttl.sh inside this code base I haven't found any Git CI action related to this image name.

[sudo-bmitch]

The busybox was just an example image to show the issue. It was copied from Docker Hub (using regctl image copy to preserve the manifest list).

[linhklattsolo]

If our issue is related to this repo - https://github.com/miguelvizdev/ttl.sh, I have seen there is three Dockerfiles one is for Dockerfile.hooks Dockerfile.reap which is using by Github CI and inside these we are using Node 10 version base image which is very old version of Node which is not supporting latest configs.

Question is can we change this Node version to latest if our code is not break or any developer can help which version is supporting our code base.

If it will not effect our code base we can change version and test it.