8000 Split docker images by shyouhei · Pull Request #1 · ruby/ruby-ci-image · GitHub
[go: up one dir, main page]
Skip to content

Split docker images #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 11, 2021
Merged

Split docker images #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8cd71f0
split docker images
shyouhei May 7, 2021
38904bb
don't push for pull requests
shyouhei May 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 56 additions & 7 deletions .github/workflows/publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,65 @@ on:

jobs:
push:
name: Publish
strategy:
fail-fast: false
matrix:
entry:
- { os: 'focal', baseruby: '2.7', tag: 'gcc-11', extras: 'g++-11' }
- { os: 'focal', baseruby: '2.7', tag: 'gcc-10', extras: 'g++-10' }
- { os: 'focal', baseruby: '2.7', tag: 'gcc-9', extras: 'g++-9' }
- { os: 'focal', baseruby: '2.7', tag: 'gcc-8', extras: 'g++-8' }
- { os: 'focal', baseruby: '2.7', tag: 'gcc-7', extras: 'g++-7' }
- { os: 'bionic', baseruby: '2.5', tag: 'gcc-6', extras: 'g++-6' }
- { os: 'bionic', baseruby: '2.5', tag: 'gcc-5', extras: 'g++-5' }
- { os: 'bionic', baseruby: '2.5', tag: 'gcc-4.8', extras: 'g++-4.8' }

- { os: 'focal', baseruby: '2.7', tag: 'clang-13', extras: 'llvm-13' }
- { os: 'focal', baseruby: '2.7', tag: 'clang-12', extras: 'llvm-12' }
- { os: 'focal', baseruby: '2.7', tag: 'clang-11', extras: 'llvm-11' }
- { os: 'focal', baseruby: '2.7', tag: 'clang-10', extras: 'llvm-10' }
- { os: 'focal', baseruby: '2.7', tag: 'clang-9', extras: 'llvm-9' }
- { os: 'bionic', baseruby: '2.5', tag: 'clang-8', extras: 'llvm-8' }
- { os: 'bionic', baseruby: '2.5', tag: 'clang-7', extras: 'llvm-7' }
- { os: 'bionic', baseruby: '2.5', tag: 'clang-6.0', extras: 'llvm-6.0' }
- { os: 'bionic', baseruby: '2.5', tag: 'clang-5.0', extras: 'llvm-5.0' }
- { os: 'bionic', baseruby: '2.5', tag: 'clang-4.0', extras: 'llvm-4.0' }
- { os: 'bionic', baseruby: '2.5', tag: 'clang-3.9', extras: 'llvm-3.9' }

- { os: 'focal', baseruby: '2.7', tag: 'mingw-w64' }
- { os: 'focal', baseruby: '2.7', tag: 'crossbuild-essential-arm64' }
- { os: 'focal', baseruby: '2.7', tag: 'crossbuild-essential-ppc64el' }
- { os: 'focal', baseruby: '2.7', tag: 'crossbuild-essential-s390x' }

name: Publish ${{ matrix.entry.tag }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: docker/build-push-action@v1
- uses: actions/cache@v2
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-
- uses: docker/setup-buildx-action@v1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is great to see that we start to use the docker/setup-buildx-action. At the moment, the published platform is only amd64 right? I could not find other platform's info for example on this page.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No other packages at this moment but I'm open to add one if necessary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. I might suggest to add other platforms arm64, ppc64le, s390x with a PR in near future. These platforms could be used in Travis.

- uses: docker/login-action@v1
with:
username: ${{ github.actor }}
password: ${{ secrets.ACCESS_TOKEN }}
registry: ghcr.io
repository: ${{ github.repository }}
tag_with_ref: true
env:
DOCKER_BUILDKIT: 1
- uses: docker/build-push-action@v2
with:
build-args: |
version=${{ matrix.entry.os }}
baseruby=${{ matrix.entry.baseruby }}
packages=${{ matrix.entry.tag }} ${{ matrix.entry.extras }}
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache-new
push: ${{ github.event_name != 'pull_request' }}
tags: |
ghcr.io/${{ github.repository }}:${{ matrix.entry.tag }}

# https://github.com/docker/build-push-action/issues/252
# https://github.com/moby/buildkit/issues/1896
- run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
59 changes: 22 additions & 37 deletions 10000 Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
FROM ubuntu:bionic as assets
MAINTAINER shyouhei@ruby-lang.org
ARG os=ubuntu
ARG version=
ARG variant=
ARG baseruby=
ARG packages=

FROM ${os}:${version}${variant} as assets
ARG os
ARG version
ARG variant

RUN apt-get update
RUN apt-get install -y wget gnupg2 ca-certificates
RUN wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key \
Expand All @@ -12,44 +21,20 @@ RUN grep '^deb ' /etc/apt/sources.list \
| tee /etc/apt/sources.list.d/deb-src.list
ADD assets/99apt.conf /etc/apt/apt.conf.d/
ADD assets/99dpkg.cfg /etc/dpkg/dpkg.cfg.d/
ADD assets/98gcc.list /etc/apt/sources.list.d/
ADD assets/99llvm.list /etc/apt/sources.list.d/
ADD assets/99${version}.list /etc/apt/sources.list.d/

FROM ${os}:${version}${variant} as compilers
ARG baseruby
ARG packages

FROM ubuntu:bionic
MAINTAINER shyouhei@ruby-lang.org
LABEL maintainer=shyouhei@ruby-lang.org
ENV DEBIAN_FRONTEND=noninteractive
COPY --from=assets /etc/ssl /etc/ssl
COPY --from=assets /etc/apt /etc/apt
COPY --from=assets /etc/dpkg /etc/dpkg

RUN set -ex \
&& apt-get update \
&& apt-get install \
clang-3.9 llvm-3.9 \
clang-4.0 llvm-4.0 \
clang-5.0 llvm-5.0 \
clang-6.0 llvm-6.0 \
clang-7 llvm-7 \
clang-8 llvm-8 \
clang-9 llvm-9 \
clang-10 llvm-10 \
clang-11 llvm-11 \
clang-12 llvm-12 \
clang-13 llvm-13 \
gcc-4.8 g++-4.8 \
gcc-5 g++-5 \
gcc-6 g++-6 \
gcc-7 g++-7 \
gcc-8 g++-8 \
gcc-9 g++-9 \
gcc-10 g++-10 \
mingw-w64 \
crossbuild-essential-arm64 \
crossbuild-essential-ppc64el \
crossbuild-essential-s390x \
libjemalloc-dev \
openssl \
ruby \
tzdata \
valgrind \
&& apt-get build-dep ruby2.5
RUN set -ex \
&& apt-get update \
&& apt-get install ${packages} \
libjemalloc-dev openssl ruby tzdata valgrind \
&& apt-get build-dep ruby${baseruby}
5 changes: 0 additions & 5 deletions assets/98gcc.list
View file
Open in desktop

This file was deleted.

11 changes: 11 additions & 0 deletions assets/99bionic.list
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
deb http://ppa.launchpad.net/ubuntu-toolchain-r/ppa/ubuntu bionic main
deb http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu bionic main
deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-6.0 main
deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-7 main
deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-8 main

deb-src http://ppa.launchpad.net/ubuntu-toolchain-r/ppa/ubuntu bionic main
deb-src http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu bionic main
deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic-6.0 main
deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic-7 main
deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic-8 main
15 changes: 15 additions & 0 deletions assets/99focal.list
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
deb http://ppa.launchpad.net/ubuntu-toolchain-r/ppa/ubuntu focal main
deb http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu focal main
Copy link
Member
@junaruga junaruga May 10, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the ubuntu-toolchain-r repository is used to download gcc-11 on focal, right? I think that's a good idea!

I wondered why ruby/ruby's .github/workflows/compilers.yml did not use it before while we had used the repository in .travis.yml in the past.

deb http://apt.llvm.org/focal/ llvm-toolchain-focal-9 main
deb http://apt.llvm.org/focal/ llvm-toolchain-focal-10 main
deb http://apt.llvm.org/focal/ llvm-toolchain-focal-11 main
deb http://apt.llvm.org/focal/ llvm-toolchain-focal-12 main
deb http://apt.llvm.org/focal/ llvm-toolchain-focal main

deb-src http://ppa.launchpad.net/ubuntu-toolchain-r/ppa/ubuntu focal main
deb-src http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu focal main
deb-src http://apt.llvm.org/focal/ llvm-toolchain-focal-9 main
deb-src http://apt.llvm.org/focal/ llvm-toolchain-focal-10 main
deb-src http://apt.llvm.org/focal/ llvm-toolchain-focal-11 main
deb-src http://apt.llvm.org/focal/ llvm-toolchain-focal-12 main
deb-src http://apt.llvm.org/focal/ llvm-toolchain-focal main
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose the deb-src repositories are not necessary only to download the deb binary packages, right? But seeing for example the LLVM's deb packages pages where the deb-src repositories are documented, and I see the benefits of using deb-src repository is we can have a choice to build the packages by ourselves in the containers. So, I agree it's better to add the deb-src repositories here. 👍🏼

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I found the deb-src repositories were used in the assets/99llvm.list. Okay.

24 changes: 0 additions & 24 deletions assets/99llvm.list
View file
Open in desktop

This file was deleted.

0