8000 gh-135401: Test AWS-LC as a cryptography library in CI (GH-135402) · python/cpython@db47f4d · GitHub
[go: up one dir, main page]
Skip to content

Commit db47f4d

Browse files
WillChilds-KleinAA-Turnerpicnixzhugovkzware
authored
gh-135401: Test AWS-LC as a cryptography library in CI (GH-135402)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Zachary Ware <zach@python.org>
1 parent 7f1e66a commit db47f4d

File tree

6 files changed

+147
-33
lines changed

6 files changed

+147
-33
lines changed

‎.github/workflows/build.yml

Lines changed: 82 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -260,7 +260,7 @@ jobs:
260260
free-threading: ${{ matrix.free-threading }}
261261
os: ${{ matrix.os }}
262262

263-
build-ubuntu-ssltests:
263+
build-ubuntu-ssltests-openssl:
264264
name: 'Ubuntu SSL tests with OpenSSL'
265265
runs-on: ${{ matrix.os }}
266266
timeout-minutes: 60
@@ -322,6 +322,81 @@ jobs:
322322
- name: SSL tests
323323
run: ./python Lib/test/ssltests.py
324324

325+
build-ubuntu-ssltests-awslc:
326+
name: 'Ubuntu SSL tests with AWS-LC'
327+
runs-on: ${{ matrix.os }}
328+
timeout-minutes: 60
329+
needs: build-context
330+
if: needs.build-context.outputs.run-tests == 'true'
331+
strategy:
332+
fail-fast: false
333+
matrix:
334+
os: [ubuntu-24.04]
335+
awslc_ver: [1.55.0]
336+
env:
337+
AWSLC_VER: ${{ matrix.awslc_ver}}
338+
MULTISSL_DIR: ${{ github.workspace }}/multissl
339+
OPENSSL_DIR: ${{ github.workspace }}/multissl/aws-lc/${{ matrix.awslc_ver }}
340+
LD_LIBRARY_PATH: ${{ github.workspace }}/multissl/aws-lc/${{ matrix.awslc_ver }}/lib
341+
steps:
342+
- uses: actions/checkout@v4
343+
with:
344+
persist-credentials: false
345+
- name: Runner image version
346+
run: echo "IMAGE_OS_VERSION=${ImageOS}-${ImageVersion}" >> "$GITHUB_ENV"
347+
- name: Restore config.cache
348+
uses: actions/cache@v4
349+
with:
350+
path: config.cache
351+
key: ${{ github.job }}-${{ env.IMAGE_OS_VERSION }}-${{ needs.build-context.outputs.config-hash }}
352+
- name: Register gcc problem matcher
353+
run: echo "::add-matcher::.github/problem-matchers/gcc.json"
354+
- name: Install dependencies
355+
run: sudo ./.github/workflows/posix-deps-apt.sh
356+
- name: Configure SSL lib env vars
357+
run: |
358+
echo "MULTISSL_DIR=${GITHUB_WORKSPACE}/multissl" >> "$GITHUB_ENV"
359+
echo "OPENSSL_DIR=${GITHUB_WORKSPACE}/multissl/aws-lc/${AWSLC_VER}" >> "$GITHUB_ENV"
360+
echo "LD_LIBRARY_PATH=${GITHUB_WORKSPACE}/multissl/aws-lc/${AWSLC_VER}/lib" >> "$GITHUB_ENV"
361+
- name: 'Restore AWS-LC build'
362+
id: cache-aws-lc
363+
uses: actions/cache@v4
364+
with:
365+
path: ./multissl/aws-lc/${{ matrix.awslc_ver }}
366+
key: ${{ matrix.os }}-multissl-aws-lc-${{ matrix.awslc_ver }}
367+
- name: Install AWS-LC
368+
if: steps.cache-aws-lc.outputs.cache-hit != 'true'
369+
run: |
370+
python3 Tools/ssl/multissltests.py \
371+
--steps=library \
372+
--base-directory "$MULTISSL_DIR" \
373+
--awslc ${{ matrix.awslc_ver }} \
374+
--system Linux
375+
- name: Add ccache to PATH
376+
run: |
377+
echo "PATH=/usr/lib/ccache:$PATH" >> "$GITHUB_ENV"
378+
- name: Configure ccache action
379+
uses: hendrikmuhs/ccache-action@v1.2
380+
with:
381+
save: false
382+
- name: Configure CPython
383+
run: |
384+
./configure CFLAGS="-fdiagnostics-format=json" \
385+
--config-cache \
386+
--enable-slower-safety \
387+
--with-pydebug \
388+
--with-openssl="$OPENSSL_DIR" \
389+
--with-builtin-hashlib-hashes=blake2 \
390+
--with-ssl-default-suites=openssl
391+
- name: Build CPython
392+
run: make -j
393+
- name: Display build info
394+
run: make pythoninfo
395+
- name: Verify python is linked to AWS-LC
396+
run: ./python -c 'import ssl; print(ssl.OPENSSL_VERSION)' | grep AWS-LC
397+
- name: SSL tests
398+
run: ./python Lib/test/ssltests.py
399+
325400
build-wasi:
326401
name: 'WASI'
327402
needs: build-context
@@ -628,7 +703,8 @@ jobs:
628703
- build-windows-msi
629704
- build-macos
630705
- build-ubuntu
631-
- build-ubuntu-ssltests
706+
- build-ubuntu-ssltests-awslc
707+
- build-ubuntu-ssltests-openssl
632708
- build-wasi
633709
- test-hypothesis
634710
- build-asan
@@ -643,7 +719,8 @@ jobs:
643719
with:
644720
allowed-failures: >-
645721
build-windows-msi,
646-
build-ubuntu-ssltests,
722+
build-ubuntu-ssltests-awslc,
723+
build-ubunt 10000 u-ssltests-openssl,
647724
test-hypothesis,
648725
cifuzz,
649726
allowed-skips: >-
@@ -661,7 +738,8 @@ jobs:
661738
check-generated-files,
662739
build-macos,
663740
build-ubuntu,
664-
build-ubuntu-ssltests,
741+
build-ubuntu-ssltests-awslc,
742+
build-ubuntu-ssltests-openssl,
665743
build-wasi,
666744
test-hypothesis,
667745
build-asan,

‎.github/workflows/posix-deps-apt.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ apt-get -yq install \
55
build-essential \
66
pkg-config \
77
ccache \
8+
cmake \
89
gdb \
910
lcov \
1011
libb2-dev \

‎Misc/NEWS.d/next/Tests/2025-06-11-16-52-49.gh-issue-135401.ccMXmL.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Add a new GitHub CI job to test the :mod:`ssl` module with `AWS-LC <https://github.com/aws/aws-lc>`_ as the backing cryptography and TLS library.

‎Tools/ssl/multissltests.py

Lines changed: 61 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
11
#!./python
2-
"""Run Python tests against multiple installations of OpenSSL and LibreSSL
2+
"""Run Python tests against multiple installations of cryptography libraries
33
44
The script
55
6-
(1) downloads OpenSSL / LibreSSL tar bundle
6+
(1) downloads the tar bundle
77
(2) extracts it to ./src
8-
(3) compiles OpenSSL / LibreSSL
9-
(4) installs OpenSSL / LibreSSL into ../multissl/$LIB/$VERSION/
8+
(3) compiles the relevant library
9+
(4) installs that library into ../multissl/$LIB/$VERSION/
1010
(5) forces a recompilation of Python modules using the
1111
header and library files from ../multissl/$LIB/$VERSION/
1212
(6) runs Python's test suite
@@ -61,6 +61,10 @@
6161
LIBRESSL_RECENT_VERSIONS = [
6262
]
6363

64+
AWSLC_RECENT_VERSIONS = [
65+
"1.55.0",
66+
]
67+
6468
# store files in ../multissl
6569
HERE = os.path.dirname(os.path.abspath(__file__))
6670
PYTHONROOT = os.path.abspath(os.path.join(HERE, '..', '..'))
@@ -70,9 +74,9 @@
7074
parser = argparse.ArgumentParser(
7175
prog='multissl',
7276
description=(
73-
"Run CPython tests with multiple OpenSSL and LibreSSL "
77+
"Run CPython tests with multiple cryptography libraries"
7478
"versions."
75-
)
79+
),
7680
)
7781
parser.add_argument(
7882
'--debug',
@@ -102,6 +106,14 @@
102106
"OpenSSL and LibreSSL versions are given."
103107
).format(LIBRESSL_RECENT_VERSIONS, LIBRESSL_OLD_VERSIONS)
104108
)
109+
parser.add_argument(
110+
'--awslc',
111+
nargs='+',
112+
default=(),
113+
help=(
114+
"AWS-LC versions, defaults to '{}' if no crypto library versions are given."
115+
).format(AWSLC_RECENT_VERSIONS)
116+
)
105117
parser.add_argument(
106118
'--tests',
107119
nargs='*',
@@ -111,7 +123,7 @@
111123
parser.add_argument(
112124
'--base-directory',
113125
default=MULTISSL_DIR,
114-
help="Base directory for OpenSSL / LibreSSL sources and builds."
126+
help="Base directory for crypto library sources and builds."
115127
)
116128
parser.add_argument(
117129
'--no-network',
@@ -124,8 +136,8 @@
124136
choices=['library', 'modules', 'tests'],
125137
default='tests',
126138
help=(
127-
"Which steps to perform. 'library' downloads and compiles OpenSSL "
128-
"or LibreSSL. 'module' also compiles Python modules. 'tests' builds "
139+
"Which steps to perform. 'library' downloads and compiles a crypto"
140+
"library. 'module' also compiles Python modules. 'tests' builds "
129141
"all and runs the test suite."
130142
)
131143
)
@@ -453,6 +465,34 @@ class BuildLibreSSL(AbstractBuilder):
453465
build_template = "libressl-{}"
454466

455467

468+
class BuildAWSLC(AbstractBuilder):
469+
library = "AWS-LC"
470+
url_templates = (
471+
"https://github.com/aws/aws-lc/archive/refs/tags/v{v}.tar.gz",
472+
)
473+
src_template = "aws-lc-{}.tar.gz"
474+
build_template = "aws-lc-{}"
475+
476+
def _build_src(self, config_args=()):
477+
cwd = self.build_dir
478+
log.info("Running build in {}".format(cwd))
479+
env = os.environ.copy()
480+
env["LD_RUN_PATH"] = self.lib_dir # set rpath
481+
if self.system:
482+
env['SYSTEM'] = self.system
483+
cmd = [
484+
"cmake",
485+
"-DCMAKE_BUILD_TYPE=RelWithDebInfo",
486+
"-DCMAKE_PREFIX_PATH={}".format(self.install_dir),
487+
"-DCMAKE_INSTALL_PREFIX={}".format(self.install_dir),
488+
"-DBUILD_SHARED_LIBS=ON",
489+
"-DBUILD_TESTING=OFF",
490+
"-DFIPS=OFF",
491+
]
492+
self._subprocess_call(cmd, cwd=cwd, env=env)
493+
self._subprocess_call(["make", "-j{}".format(self.jobs)], cwd=cwd, env=env)
494+
495+
456496
def configure_make():
457497
if not os.path.isfile('Makefile'):
458498
log.info('Running ./configure')
@@ -467,9 +507,10 @@ def configure_make():
467507

468508
def main():
469509
args = parser.parse_args()
470-
if not args.openssl and not args.libressl:
510+
if not args.openssl and not args.libressl and not args.awslc:
471511
args.openssl = list(OPENSSL_RECENT_VERSIONS)
472512
args.libressl = list(LIBRESSL_RECENT_VERSIONS)
513+
args.awslc = list(AWSLC_RECENT_VERSIONS)
473514
if not args.disable_ancient:
474515
args.openssl.extend(OPENSSL_OLD_VERSIONS)
475516
args.libressl.extend(LIBRESSL_OLD_VERSIONS)
@@ -496,22 +537,15 @@ def main():
496537

497538
# download and register builder
498539
builds = []
499-
500-
for version in args.openssl:
501-
build = BuildOpenSSL(
502-
version,
503-
args
504-
)
505-
build.install()
506-
builds.append(build)
507-
508-
for version in args.libressl:
509-
build = BuildLibreSSL(
510-
version,
511-
args
512-
)
513-
build.install()
514-
builds.append(build)
540+
for build_class, versions in [
541+
(BuildOpenSSL, args.openssl),
542+
(BuildLibreSSL, args.libressl),
543+
(BuildAWSLC, args.awslc),
544+
]:
545+
for version in versions:
546+
build = build_class(version, args)
547+
build.install()
548+
builds.append(build)
515549

516550
if args.steps in {'modules', 'tests'}:
517551
for build in builds:
@@ -539,7 +573,7 @@ def main():
539573
else:
540574
print('Executed all SSL tests.')
541575

542-
print('OpenSSL / LibreSSL versions:')
576+
print('OpenSSL / LibreSSL / AWS-LC versions:')
543577
for build in builds:
544578
print(" * {0.library} {0.version}".format(build))
545579

‎configure

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎configure.ac

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7529,8 +7529,8 @@ WITH_SAVE_ENV([
75297529
], [
75307530
OBJ_nid2sn(NID_md5);
75317531
OBJ_nid2sn(NID_sha1);
7532+
OBJ_nid2sn(NID_sha512);
75327533
OBJ_nid2sn(NID_sha3_512);
7533-
OBJ_nid2sn(NID_blake2b512);
75347534
EVP_PBE_scrypt(NULL, 0, NULL, 0, 2, 8, 1, 0, NULL, 0);
75357535
])], [ac_cv_working_openssl_hashlib=yes], [ac_cv_working_openssl_hashlib=no])
75367536
])

0 commit comments

Comments
 (0)
0