Paketo integration doesn't work with Docker's new overlayfs storage driver #49251
Description
Docker Engine 29 switched the default storage backend from overlay2 to overlayfs, see this documentation.
GitHub Actions switched to Docker 29 for ubuntu-latest some time ago. This broke something in Paketo (see #49214), but this has been fixed now.
Our Paketo integration is now in a half-broken state: stock-standard bootBuildImage works, but this doesn't:
bootBuildImage {
runImage = "paketobuildpacks/ubuntu-noble-run:latest"
}
This results in this error:
[creator] Added 5/5 app layer(s)
[creator] Adding layer 'buildpacksio/lifecycle:launcher'
[creator] Adding layer 'buildpacksio/lifecycle:config'
[creator] Adding layer 'buildpacksio/lifecycle:process-types'
[creator] Adding label 'io.buildpacks.lifecycle.metadata'
[creator] Adding label 'io.buildpacks.build.metadata'
[creator] Adding label 'io.buildpacks.project.metadata'
[creator] Adding label 'org.opencontainers.image.title'
[creator] Adding label 'org.opencontainers.image.version'
[creator] Adding label 'org.springframework.boot.version'
[creator] Setting default process type 'web'
[creator] Saving docker.io/library/demo:0.0.1-SNAPSHOT...
[creator] ERROR: failed to export: saving image: failed to fetch base layers: open /tmp/imgutil.local.image.82301454/blobs/sha256/23a8c5303fec5323ef417f9fd39ccaec49bb046a46081aa4670bc05a25e2ae83: no such file or directory
which is the same problem as in our system test. All branches are affected.
Upgrading to the latest Docker version (v29.2.1 at the time of writing) doesn't help.
Switching to overlay2 (the old storage driver) or disabling the snapshots outlined in this comment solves the problem.
My Fedora and my ubuntu-24.04 VM use overlay2 (the old storage driver) by default, maybe because they have upgraded Docker from 28 to 29.
To switch to the new storage driver, create a file named /etc/docker/daemon.json and put this in:
{
"features": {
"containerd-snapshotter": true
}
}docker info then shows:
Storage Driver: overlayfs
driver-type: io.containerd.snapshotter.v1
instead of
Storage Driver: overlay2
But even with those options in place, I can't get the tests to fail on my local machine.