8000 bpo-43811: Test multiple OpenSSL versions on GHA by tiran · Pull Request #25360 · python/cpython · GitHub
[go: up one dir, main page]
Skip to content

bpo-43811: Test multiple OpenSSL versions on GHA #25360

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 13, 2021
Merged

bpo-43811: Test multiple OpenSSL versions on GHA #25360

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
771ad62
bpo-43811: Test multiple OpenSSL versions on GHA
tiran Apr 12, 2021
3669c9f
Only run ssl tests when OpenSSL related files are changed
tiran Apr 12, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 63 additions & 2 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,15 @@ jobs:
runs-on: ubuntu-latest
outputs:
run_tests: ${{ steps.check.outputs.run_tests }}
run_ssl_tests: ${{ steps.check.outputs.run_ssl_tests }}
steps:
- uses: actions/checkout@v2
- name: Check for source changes
id: check
run: |
if [ -z "$GITHUB_BASE_REF" ]; then
echo '::set-output name=run_tests::true'
echo '::set-output name=run_ssl_tests::true'
else
git fetch origin $GITHUB_BASE_REF --depth=1
# git diff "origin/$GITHUB_BASE_REF..." (3 dots) may be more
Expand All @@ -46,6 +48,7 @@ jobs:
#
# https://github.com/python/core-workflow/issues/373
git diff --name-only origin/$GITHUB_BASE_REF.. | grep -qvE '(\.rst$|^Doc|^Misc)' && echo '::set-output name=run_tests::true' || true
git diff --name-only origin/$GITHUB_BASE_REF.. | grep -qE '(ssl|hashlib|hmac|^.github)' && echo '::set-output name=run_ssl_tests::true' || true
fi

check_generated_files:
Expand Down Expand Up @@ -138,6 +141,11 @@ jobs:
run: echo "::add-matcher::.github/problem-matchers/gcc.json"
- name: Install Dependencies
run: sudo ./.github/workflows/posix-deps-apt.sh
- name: Configure OpenSSL env vars
run: |
echo "MULTISSL_DIR=${GITHUB_WORKSPACE}/multissl" >> $GITHUB_ENV
echo "OPENSSL_DIR=${GITHUB_WORKSPACE}/multissl/openssl/${OPENSSL_VER}" >> $GITHUB_ENV
echo "LD_LIBRARY_PATH=${GITHUB_WORKSPACE}/multissl/openssl/${OPENSSL_VER}/lib" >> $GITHUB_ENV
- name: 'Restore OpenSSL build'
id: cache-openssl
uses: actions/cache@v2.1.4
Expand All @@ -146,12 +154,65 @@ jobs:
key: ${{ runner.os }}-multissl-openssl-${{ env.OPENSSL_VER }}
- name: Install OpenSSL
if: steps.cache-openssl.outputs.cache-hit != 'true'
run: python3 Tools/ssl/multissltests.py --steps=library --base-directory $PWD/multissl --openssl $OPENSSL_VER --system Linux
run: python3 Tools/ssl/multissltests.py --steps=library --base-directory $MULTISSL_DIR --openssl $OPENSSL_VER --system Linux
- name: Add ccache to PATH
run: |
echo "PATH=/usr/lib/ccache:$PATH" >> $GITHUB_ENV
- name: Configure ccache action
uses: hendrikmuhs/ccache-action@v1
- name: Configure CPython
run: ./configure --with-pydebug --with-openssl=$PWD/multissl/openssl/$OPENSSL_VER
run: ./configure --with-pydebug --with-openssl=$OPENSSL_DIR
- name: Build CPython
run: make -j4
- name: Display build info
run: make pythoninfo
- name: Tests
run: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu"

build_ubuntu_ssltests:
name: 'Ubuntu SSL tests with OpenSSL ${{ matrix.openssl_ver }}'
runs-on: ubuntu-20.04
needs: check_source
if: needs.check_source.outputs.run_tests == 'true' && needs.check_source.outputs.run_ssl_tests == 'true'
strategy:
fail-fast: false
matrix:
openssl_ver: [1.0.2u, 1.1.0l, 1.1.1k, 3.0.0-alpha14]
Copy link
Member
@vstinner vstinner Apr 12, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we have an accepted PEP which requires OpenSSL 1.1.1 or newer?

Each additional CI increases the error rate since we still have unstable tests in asyncio and multiprocessing tests. I would prefer to not add too many CIs. At least, don't make them mandatory (allow to merge a PR even if they fail).

Rather than pre-commit CI jobs run on every since PR and at every PR update, can't we use a post-commit CI, like a buildbot worker? Issues in pre-commit are way more annoying.

cc @pablogsal @zware

Copy link
Member Author
@tiran tiran Apr 12, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we have an accept PEP which requires OpenSSL 1.1.1 or newer?

Yes, but the PEP applies to 3.10 only. You want OpenSSL 3.0.0 compatibility in 3.8 and 3.9, don't you?

Rather than pre-commit CI jobs run on every since PR and at every PR update, can't we use a post-commit CI, like a buildbot worker? Issues in pre-commit are way more annoying.

I need an easy and reliable way to ensure that Python works with all supported OpenSSL versions. Post-commit CI is too late.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can add OpenSSL < 1.1.1 in the CI jobs when you backport this PR to 3.9 and older. But this PR targets master, so I expect n 8000 ot see OpenSSL < 1.1.1 in this branch.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Master still supports OpenSSL 1.1.0 and 1.0.2. I have not finished and pushed the PEP 644 PR. I deliberately postponed PEP 644 changes until OpenSSL 3.0.0 support is done.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each additional CI increases the error rate since we still have unstable tests in asyncio and multiprocessing tests. I would prefer to not add too many CIs. At least, don't make them mandatory (allow to merge a PR even if they fail).

The additional SSL tests are now only executed when any OpenSSL or GHA related file is modified. The SSL tests only execute a couple of selected tests. Multiprocessing tests are skipped. The CI steps are not mandatory.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Master still supports OpenSSL 1.1.0 and 1.0.2. I have not finished and pushed the PEP 644 PR. I deliberately postponed PEP 644 changes until OpenSSL 3.0.0 support is done."

Ok.

env:
OPENSSL_VER: ${{ matrix.openssl_ver }}
MULTISSL_DIR: ${{ github.workspace }}/multissl
OPENSSL_DIR: ${{ github.workspace }}/multissl/openssl/${{ matrix.openssl_ver }}
LD_LIBRARY_PATH: ${{ github.workspace }}/multissl/openssl/${{ matrix.openssl_ver }}/lib
steps:
- uses: actions/checkout@v2
- name: Register gcc problem matcher
run: echo "::add-matcher::.github/problem-matchers/gcc.json"
- name: Install Dependencies
run: sudo ./.github/workflows/posix-deps-apt.sh
- name: Configure OpenSSL env vars
run: |
echo "MULTISSL_DIR=${GITHUB_WORKSPACE}/multissl" >> $GITHUB_ENV
echo "OPENSSL_DIR=${GITHUB_WORKSPACE}/multissl/openssl/${OPENSSL_VER}" >> $GITHUB_ENV
echo "LD_LIBRARY_PATH=${GITHUB_WORKSPACE}/multissl/openssl/${OPENSSL_VER}/lib" >> $GITHUB_ENV
- name: 'Restore OpenSSL build'
id: cache-openssl
uses: actions/cache@v2.1.4
with:
path: ./multissl/openssl/${{ env.OPENSSL_VER }}
key: ${{ runner.os }}-multissl-openssl-${{ env.OPENSSL_VER }}
- name: Install OpenSSL
if: steps.cache-openssl.outputs.cache-hit != 'true'
run: python3 Tools/ssl/multissltests.py --steps=library --base-directory $MULTISSL_DIR --openssl $OPENSSL_VER --system Linux
- name: Add ccache to PATH
run: |
echo "PATH=/usr/lib/ccache:$PATH" >> $GITHUB_ENV
- name: Configure ccache action
uses: hendrikmuhs/ccache-action@v1
- name: Configure CPython
run: ./configure --with-pydebug --with-openssl=$OPENSSL_DIR
- name: Build CPython
run: make -j4
- name: Display build info
run: make pythoninfo
- name: SSL tests
run: ./python Lib/test/ssltests.py
1 change: 1 addition & 0 deletions .github/workflows/posix-deps-apt.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ apt-get update

apt-get -yq install \
build-essential \
ccache \
gdb \
lcov \
libbz2-dev \
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Tests/2021-04-12-11-14-28.bpo-43811.vGNbnD.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Tests multiple OpenSSL versions on GitHub Actions. Use ccache to speed up
testing.
0