8000 Buildkit-optimized dockerfile + buildx by cpuguy83 · Pull Request #39340 · moby/moby · GitHub
[go: up one dir, main page]

Skip to content

Conversation

cpuguy83
Copy link
Member
@cpuguy83 cpuguy83 commented Jun 7, 2019
Since the dockerfile now requires buildkit, let's just use buildx can
bootstrap itself into even an old version of Docker which does not
support buildkit.

This also decouples the Dockerfile/build from the version of Docker
which is installed.

One major downside:

If buildx needs to setup a container driver (ie, docker does not support
buildkit), the `make shell` target (and others which call
`DOCKER_RUN_DOCKER`) must export the image from buildkit and into
docker. This added an extra 70s to a full build for me (agan only for targets
which call `DOCKER_RUN_DOCKER`) and 40s on a rebuild (with no changes).

@cpuguy83
Copy link
Member Author
cpuguy83 commented Jun 7, 2019

docker-bake.hcl file included b/c I was just messing around with it.
It may be nice to use it in the Makefile just to hold defaults values or something... not sure.

@cpuguy83
Copy link
Member Author
cpuguy83 commented Jun 7, 2019

I expect CI to fail on this because it's using docker build directly without buildkit enabled.

Makefile Outdated
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is the epitome of windows-last development

@cpuguy83
Copy link
Member Author
cpuguy83 commented Jun 7, 2019

One thing to note, now make binary (and other such things) are handled in a Dockerfile target. This means they can take advantage of the docker build cache AND they are built with a buildkit cache mount for the go build cache, so even in the event of rebuild it should still benefit from package level caching.

Dockerfile Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add stages with precompiled test suites?

FROM dev as test-dynbinary
RUN --mount=target=--mount=type=cache,target=/root/.cache/go
F440
-build \
 build-integration-test-binary dynbinary
ENV DOCKER_INTEGRATION_TESTS_VERIFIED=1
ENV KEEPBUNDLE=1

(ref: https://github.com/AkihiroSuda/kube-moby-integration/blob/master/Dockerfile)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have this, however it's a bit tricky because the tests expecting a certain work dir.
Some more polishing to do.

Dockerfile Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably we should add back comment lines with updated contents

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure most of them are useful since you can see that by looking at the build targets.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Realizing I can't use the sha here since this dockerfile is used for multiple platforms.

@cpuguy83
Copy link
Member Author

@andrewhsu power and z still need updates to support buildkit.

@cpuguy83 cpuguy83 force-pushed the buildkit_dockerfile branch 2 times, most recently from b4e816c to 4d4ac70 Compare August 16, 2019 23:15
@cpuguy83 cpuguy83 marked this pull request as ready for review August 16, 2019 23:15
@cpuguy83 cpuguy83 force-pushed the buildkit_dockerfile branch 2 times, most recently from 7ec5b43 to f1aacc5 Compare August 16, 2019 23:49
@cpuguy83
Copy link
Member Author

@thaJeztah Ok so I've made buildx opt-in here and updated the Jenkinsfile so power/z can build using buildx.

@cpuguy83 cpuguy83 force-pushed the buildkit_dockerfile branch from f1aacc5 to 91f1945 Compare August 17, 2019 00:21
@cpuguy83 cpuguy83 force-pushed the buildkit_dockerfile branch 3 times, most recently from 177b1c2 to c095fc5 Compare September 4, 2019 23:26
@cpuguy83 cpuguy83 requested a review from AkihiroSuda September 4, 2019 23:26
@cpuguy83
Copy link
Member Author

Ok, think I fixed it, we'll see... magic GOOS and GOARCH detection was setting a default value for cases like go is not installed on the system, which were incorrect. Instead this just doesn't set a value in that case.

@cpuguy83
Copy link
Member Author

Yep, that was it, those arches work now.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Since the dockerfile now requires buildkit, let's just use buildx can
bootstrap itself into even an old version of Docker which does not
support buildkit.

This also decouples the Dockerfile/build from the version of Docker
which is installed.

One major downside:

If buildx needs to setup a container driver (ie, docker does not support
buildkit), the `make shell` target (and others which call
`DOCKER_RUN_DOCKER`) must export the image from buildkit and into
docker. This added an extra 70s to a full build for me (agan only for targets
which call `DOCKER_RUN_DOCKER`) and 40s on a rebuild (with no changes).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Z and Power does not currently work with buildkit so use buildx instead.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
@cpuguy83 cpuguy83 force-pushed the buildkit_dockerfile branch from cfa0433 to c04ea11 Compare October 3, 2019 21:07
@thaJeztah
Copy link
Member

Oy I don't know what happened to the indentation, it was perfectly lined up before... 🤔

So, the problem is that the first line of each command is basically indented with spaces;

RUN<space>whatever is in your RUN

tab-width is configurable, so I think we should use spaces for indentation in Dockerfiles if we want things to line up.

I've been playing a bit with possible changes we could make (including the indentation), and pushed it to a branch; master...thaJeztah:buildkit_dockerfile_formatting

Copy link
Member
@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the indentation (not a show-stopper, but if we want to reduce churn in this file, and if we want to change it, would be good to have it in this PR).

So, LGTM, but we can discuss that change (and perhaps there's changes in the branch I mentioned above that you think would be useful)

@cpuguy83
Copy link
Member Author
cpuguy83 commented Oct 5, 2019

So indentation depends on the editor.
If I open in vscode it's ok and vim it's messed up (or vice-versa?)... but that's what happened I "fixed" it in one and it came out weird in the other.

So yeah, maybe spaces will fair better.

@cpuguy83
Copy link
Member Author
cpuguy83 commented Oct 5, 2019

But let's get this in first 😅😇

@thaJeztah
Copy link
Member

But let's get this in first 😅😇

SGTM; I can open a WIP pr for my other changes for easier discussion on those; do you want @AkihiroSuda to have another look for the changes you made since his LGTM? If you think it's ok without, I can merge (or feel free to press the button yourself)

@AkihiroSuda AkihiroSuda merged commit ed2f50f into moby:master Oct 5, 2019
@AkihiroSuda
Copy link
Member

still LGTM, merged 🙂

@thaJeztah
Copy link
Member

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
< 3F26 input type="hidden" name="_method" value="put" autocomplete="off" />
Development

Successfully merging this pull request may close these issues.

5 participants
0