ssl.create_default_context(): add VERIFY_X509_STRICT and VERIFY_X509_PARTIAL_CHAIN to the default verify_flags
#107361
Comments
|
@python/crypto-team |
See python#107361: this adds `VERIFY_X509_STRICT` to make the default SSL context perform stricter (per RFC 5280) validation, as well as `VERIFY_X509_PARTIAL_CHAIN` to enforce more standards-compliant path-building behavior. As part of this changeset, I had to tweak `make_ssl_certs.py` slightly to emit 5280-conforming CA certs. This changeset includes the regenerated certificates after that change.
|
Opened #112389 for this. |
This adds `VERIFY_X509_STRICT` to make the default SSL context perform stricter (per RFC 5280) validation, as well as `VERIFY_X509_PARTIAL_CHAIN` to enforce more standards-compliant path-building behavior. As part of this changeset, I had to tweak `make_ssl_certs.py` slightly to emit 5280-conforming CA certs. This changeset includes the regenerated certificates after that change. Signed-off-by: William Woodruff <william@yossarian.net> Co-authored-by: Victor Stinner <vstinner@python.org>
|
Thanks for taking the time to do this! |
This adds `VERIFY_X509_STRICT` to make the default SSL context perform stricter (per RFC 5280) validation, as well as `VERIFY_X509_PARTIAL_CHAIN` to enforce more standards-compliant path-building behavior. As part of this changeset, I had to tweak `make_ssl_certs.py` slightly to emit 5280-conforming CA certs. This changeset includes the regenerated certificates after that change. Signed-off-by: William Woodruff <william@yossarian.net> Co-authored-by: Victor Stinner <vstinner@python.org>
This adds `VERIFY_X509_STRICT` to make the default SSL context perform stricter (per RFC 5280) validation, as well as `VERIFY_X509_PARTIAL_CHAIN` to enforce more standards-compliant path-building behavior. As part of this changeset, I had to tweak `make_ssl_certs.py` slightly to emit 5280-conforming CA certs. This changeset includes the regenerated certificates after that change. Signed-off-by: William Woodruff <william@yossarian.net> Co-authored-by: Victor Stinner <vstinner@python.org>
…_X509_STRICT In python 3.13, this flag was added to the default created ctx. This seems reasonable for production validation, but the unittest code generates a certificate that fails this. Inside test code, it seems fine to relax the constraints again. Our goal is to test the server itself, anyway, not the ssl module. See also: python/cpython#107361
…_X509_STRICT In python 3.13, this flag was added to the default created ctx. This seems reasonable for production validation, but the unittest code generates a certificate that fails this. Inside test code, it seems fine to relax the constraints again. Our goal is to test the server itself, anyway, not the ssl module. See also: python/cpython#107361
The "ssl.VERIFY_X509_STRICT" flag for openssl verification has been enabled by default in python 3.13+ (it was disabled before that). see python/cpython#107361 and https://discuss.python.org/t/ssl-changing-the-default-sslcontext-verify-flags/30230/16 We explicitly disable it for self-signed certs, thereby restoring the pre-3.13 defaults, as it seems to break lots of servers. For example, using python 3.13 (or setting `sslc.verify_flags |= ssl.VERIFY_X509_STRICT`), - I can connect to `btc.electroncash.dk:60002:s` - but not to `electrum.emzy.de:50002:s` despite both using self-signed certs. We should investigate more why exactly "strict" verification fails for some self-signed certs and not for others, and make sure that at least newly generated certs adhere to the stricter requirements (e.g. update guide in e-x?).
The "ssl.VERIFY_X509_STRICT" flag for openssl verification has been enabled by default in python 3.13+ (it was disabled before that). see python/cpython#107361 and https://discuss.python.org/t/ssl-changing-the-default-sslcontext-verify-flags/30230/16 We explicitly disable it for self-signed certs, thereby restoring the pre-3.13 defaults, as it seems to break lots of servers. For example, using python 3.13 (or setting `sslc.verify_flags |= ssl.VERIFY_X509_STRICT`), - I can connect to `btc.electroncash.dk:60002:s` - but not to `electrum.emzy.de:50002:s` despite both using self-signed certs. We should investigate more why exactly "strict" verification fails for some self-signed certs and not for others, and make sure that at least newly generated certs adhere to the stricter requirements (e.g. update guide in e-x?).
The new flag enforced in Python 3.13 with python/cpython#107361 doesn't work with the semi broken Freebox self signed certificates. It should fix home-assistant/core#132333 Fixes hacf-fr#734
CA certs need the key usage extension as a result of python/cpython#107361
Python 3.13 has started requiring the authorityKeyIdentifier f 8000 ield as per python/cpython#107361 . After iterating a bit, it appears that we only need to pass "-addext keyUsage=keyCertSign" to openssl during CA certificate creation and the server certificate will have the proper field. It's also possible to use something like `trustme` to generate the certificates but that would have been a much larger change and more work (maybe leading to most of the script being dropped however).
Python 3.13 has started requiring the authorityKeyIdentifier field as per python/cpython#107361 . After iterating a bit, it appears that we only need to pass "-addext keyUsage=keyCertSign" to openssl during CA certificate creation and the server certificate will have the proper field. It's also possible to use something like `trustme` to generate the certificates but that would have been a much larger change and more work (maybe leading to most of the script being dropped however).
Python 3.13 has started requiring the authorityKeyIdentifier field as per python/cpython#107361 . After iterating a bit, it appears that we only need to pass "-addext keyUsage=keyCertSign" to openssl during CA certificate creation and the server certificate will have the proper field. It's also possible to use something like `trustme` to generate the certificates but that would have been a much larger change and more work (maybe leading to most of the script being dropped however). Forwarded: breezy-team#143 Bug-Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1092458
Python 3.13 has started requiring the authorityKeyIdentifier field as per python/cpython#107361 . After iterating a bit, it appears that we only need to pass "-addext keyUsage=keyCertSign" to openssl during CA certificate creation and the server certificate will have the proper field. It's also possible to use something like `trustme` to generate the certificates but that would have been a much larger change and more work (maybe leading to most of the script being dropped however). Forwarded: breezy-team#143 Bug-Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1092458
Python 3.13 has started requiring the authorityKeyIdentifier field as per python/cpython#107361 . After iterating a bit, it appears that we only need to pass "-addext keyUsage=keyCertSign" to openssl during CA certificate creation and the server certificate will have the proper field. It's also possible to use something like `trustme` to generate the certificates but that would have been a much larger change and more work (maybe leading to most of the script being dropped however). Forwarded: breezy-team#143 Bug-Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1092458
Python 3.13 has started requiring the authorityKeyIdentifier field as per python/cpython#107361 . After iterating a bit, it appears that we only need to pass "-addext keyUsage=keyCertSign" to openssl during CA certificate creation and the server certificate will have the proper field. It's also possible to use something like `trustme` to generate the certificates but that would have been a much larger change and more work (maybe leading to most of the script being dropped however). Forwarded: breezy-team#143 Bug-Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1092458
Python 3.13 has started requiring the authorityKeyIdentifier field as per python/cpython#107361 . After iterating a bit, it appears that we only need to pass "-addext keyUsage=keyCertSign" to openssl during CA certificate creation and the server certificate will have the proper field. It's also possible to use something like `trustme` to generate the certificates but that would have been a much larger change and more work (maybe leading to most of the script being dropped however). Forwarded: breezy-team#143 Bug-Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1092458
|
Hello, the No browsers seem to require this either, so openssl seems stricter here. I'm not trying to say this is a wrong change or that you should not do it, just trying to provide some feedback, not sure if you get much of it after changes like these. Also, looking at the discussion, there were some examples mentioned of other projects making SSL stricter, and introducing env variables to disable this stricter mode. AFAIK golang project also does this, and I'd like to propose you to adopt the same approach with future changes like this - setting env variable is significantly easier than hunting down every place where ssl context is created, and setting some flag to it. |
This is possible: the AKI is tied to whatever the SSL inspection proxy's root cert is. For context, I believe another large SSL proxy vendor has already fixed this in their products based on the Python change here. I'll ping some people and see if they're willing to share more information about that.
There's some nuance here: modern browsers are aware of inspection proxies, and apply looser validation rules to them because they're persistently misconfigured (e.g. violating RFC 5280 in this case). Chrome and Firefox should both refuse a similarly mis-issued certificate on the Web PKI, although this is difficult to provide a public example of since Web PKI CAs will also refuse to issue an invalid certificate here.
This is ultimately the maintainers' call, but I would be a -1 on it as an outsider: IMO environment variables are too easy to "set and forget," leaving deployments in a permanently degraded security posture. This is the point I made in this DPO comment. (I believe the official advice with Python and the |
|
Thank you for this information, it is very hard to come by, so this is much appreciated!
Thank you, we'll try to talk to our vendor - if you'd be able to get any more information, I'd be very grateful.
I understand your reasoning, and I'd love this to have the wanted effect, but the reality is the SSL proxy will not be fixed anytime soon even if it is entirely at fault, so I now have two options - either not use Python 3.13, or disable the extra validation. And without env var, the "easy to set and forget" thing is now everywhere an ssl context is created instead of, say, one base Docker image. |
Uh oh!
There was an error while loading. Please reload this page.
Feature or enhancement
My proposal is to add two new flags to the
SSLContext.verify_flagscreated withinssl.create_default_context():VERIFY_X509_STRICT: This will enable stricter RFC 5280 compliance checks within OpenSSL's X.509 path validation implementation. This will have no effect on the overwhelming majority of users (since the overwhelming majority of TLS connections are likely going through the Web PKI, which is already much stricter than RFC 5280).VERIFY_X509_PARTIAL_CHAIN: This will allow OpenSSL's path validation to terminate at the first certificate listed in the user's root of trust, even if that certificate is not self-signed. Despite the confusing name and the fact that this isn't the default, this is the correct behavior per the various X.509 RFCs (3280, 5280): a trust anchor is defined to be an a priori trust relationship with a subject and its public key, regardless of whether the anchor comes in certificate form or is signed by another member of the trusted store. This should have no breaking effect on any users, but may cause some validations to produce shorter chains than the currentSSLContextenables.My proposal is consistent with the stability policy for
create_default_context, which says that CPython may introduce changes to the default context without a prior deprecation period:As with previous changes to the default context (such as the removal of 3DES support with 3.6), these proposed changes will not prevent people from constructing their own
SSLContextwithout these new flags. In other words: users who do experience breakage or other behavioral changes will have a well-trodden alternative available to them, one that is documented as a matter of policy.Pitch
To summarize from DPO:
VERIFY_X509_STRICTis a net security win: it reduces the amount of flexibility in the X.509 certificates that thesslmodule accepts by default, which means less attacker controlled flexibility. It also makes CPython itself more compatible with the X.509 profile defined in RFC 5280, meaning that end-users can more confidently expect interoperation with PKIs that use the 5280 profile.VERIFY_X509_PARTIAL_CHAINmakessslcomply more closely with other path validation/building implementations (like Go's), and eliminates an error mode that's only possible because of OpenSSL's non-standard default behavior (a chain is built to the root of trust successfully, but can still fail because OpenSSL can't find a self-signed root also within the root of trust). It's also consistent with what curl and other downstream consumers of OpenSSL do.Previous discussion
See DPO discussion here: https://discuss.python.org/t/ssl-changing-the-default-sslcontext-verify-flags/30230
Linked PRs
The text was updated successfully, but these errors were encountered: