Buildkit: "docker build --cache-from" doesn't use cache from local tagged images (multi-stage builds) #39003
Description
I'm using docker build with --cache-from on a multi-stage build to allow caching in a gitlab-ci (docker in docker) environment. Notably, this means that the images named in the --cache-from option may not exist in the registry when docker build is run, as it might the first build on a new branch or repo fork.
This worked in the classic docker builder (cache could load local tagged images not pushed to a registry), but doesn't work with buildkit.
Steps to reproduce the issue:
- Create a simple Dockerfile, e.g.
FROM alpine:3.9 AS stage1
RUN touch /file1
FROM stage1 AS stage2
RUN touch /file2- Ensure no image exists matching the --cache-from
docker rm local.registry/docker-test/stage1:latest local.registry/docker-test/stage2:latest- Run the docker build command with --cache-from and --tag
docker build \
--cache-from local.registry/docker-test/stage1:latest \
--target stage1 \
--tag local.registry/docker-test/stage1:latest \
-f Dockerfile .
docker build \
--cache-from local.registry/docker-test/stage1:latest \
--cache-from local.registry/docker-test/stage2:latest \
--target stage2 \
--tag local.registry/docker-test/stage2:latest \
-f Dockerfile .Describe the results you received:
The first stage is not used as a cache when building the second stage:
$ DOCKER_BUILDKIT=1 docker build --cache-from local.registry/docker-test/stage1:latest --target stage1 --tag local.registry/docker-test/stage1:latest -f Dockerfile .
[+] Building 1.2s (7/7) FINISHED
=> [internal] load build definition from Dockerfile 0.2s
=> => transferring dockerfile: 185B 0.0s
=> [internal] load .dockerignore 0.2s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/alpine:3.9 0.0s
=> ERROR importing cache manifest from local.registry/docker-test/stage1:latest 0.0s
=> [stage1 1/2] FROM docker.io/library/alpine:3.9 0.0s
=> => resolve docker.io/library/alpine:3.9 0.0s
=> [stage1 2/2] RUN touch /file1 0.8s
=> exporting to image 0.1s
=> => exporting layers 0.1s
=> => writing image sha256:674f0a872a085bfbc17e5e8b094d103d9ce23d6d04c8ce3ba4eb980bb3a9846e 0.0s
=> => naming to local.registry/docker-test/stage1:latest 0.0s
------
> importing cache manifest from local.registry/docker-test/stage1:latest:
------
$ DOCKER_BUILDKIT=1 docker build --cache-from local.registry/docker-test/stage1:latest --cache-from local.registry/docker-test/stage2:latest --target stage2 --tag local.registry/docker-test/stage2:latest -f Dockerfile .
[+] Building 1.9s (9/9) FINISHED
=> [internal] load build definition from Dockerfile 0.2s
=> => transferring dockerfile: 96B 0.0s
=> [internal] load .dockerignore 0.2s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/alpine:3.9 0.0s
=> ERROR importing cache manifest from local.registry/docker-test/stage1:latest 0.0s
=> ERROR importing cache manifest from local.registry/docker-test/stage2:latest 0.0s
=> [stage1 1/2] FROM docker.io/library/alpine:3.9 0.0s
=> => resolve docker.io/library/alpine:3.9 0.0s
=> [stage1 2/2] RUN touch /file1 0.7s
=> [stage2 1/1] RUN touch /file2 0.8s
=> exporting to image 0.1s
=> => exporting layers 0.1s
=> => writing image sha256:f6526297588e4c4365b520dbd5d382e22a20bc8cb56fef1f6e992e137ba6a15b 0.0s
=> => naming to local.registry/docker-test/stage2:latest 0.0s
------
> importing cache manifest from local.registry/docker-test/stage1:latest:
------
------
> importing cache manifest from local.registry/docker-test/stage2:latest:
------
Note how [stage1 2/2] is not cached when building the stage2 image.
If you re-run both steps again, still nothing is cached.
Describe the results you expected:
The first stage is used as a cache when building the second stage:
$ docker build --cache-from local.registry/docker-test/stage1:latest --target stage1 --tag local.registry/docker-test/stage1:latest -f Dockerfile .
Sending build context to Docker daemon 2.048kB
Step 1/2 : FROM alpine:3.9 as stage1
---> 5cb3aa00f899
Step 2/2 : RUN touch /file1
---> Running in 60e6093cc777
Removing intermediate container 60e6093cc777
---> cb8d39fdacdc
Successfully built cb8d39fdacdc
Successfully tagged local.registry/docker-test/stage1:latest
$ docker build --cache-from local.registry/docker-test/stage1:latest --cache-from local.registry/docker-test/stage2:latest --target stage2 --tag local.registry/docker-test/stage2:latest -f Dockerfile .
Sending build context to Docker daemon 2.048kB
Step 1/4 : FROM alpine:3.9 as stage1
---> 5cb3aa00f899
Step 2/4 : RUN touch /file1
---> Using cache
---> cb8d39fdacdc
Step 3/4 : FROM stage1 as stage2
---> cb8d39fdacdc
Step 4/4 : RUN touch /file2
---> Running in 8179df339c2e
Removing intermediate container 8179df339c2e
---> c1ab7c10a04b
Successfully built c1ab7c10a04b
Successfully tagged local.registry/docker-test/stage2:latest
Note how "Step 2/4" in the second command is cached.
Also if you re-run both steps again, everything is cached.
Additional information you deem important (e.g. issue happens only occasionally):
Output of docker version:
Client:
Version: 18.09.4
API version: 1.39
Go version: go1.10.8
Git commit: d14af54
Built: Wed Mar 27 18:36:04 2019
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 18.09.4
API version: 1.39 (minimum version 1.12)
Go version: go1.10.8
Git commit: d14af54
Built: Wed Mar 27 18:04:46 2019
OS/Arch: linux/amd64
Experimental: false
Output of docker info:
Containers: 52
Running: 0
Paused: 0
Stopped: 52
Images: 132
Server Version: 18.09.4
Storage Driver: overlay2
Backing Filesystem: btrfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84
runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 5.0.4-200.fc29.x86_64
Operating System: Fedora 29 (Workstation Edition)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.56GiB
Name: rocky
ID: EZ5T:7HUA:5BNM:CT23:O7MU:HHXB:AMME:37UC:Y3P3:TR3U:X6GN:GBTX
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine
Additional environment details (AWS, VirtualBox, physical, etc.):