8000 Use TUF to download key/cert material (#351) · tetsuo-cpp/sigstore-python@5a68d1e · GitHub
[go: up one dir, main page]
Skip to content

Commit 5a68d1e

Browse files
jkujoshuaglwxjdsrwoodruffwtetsuo-cpp
authored
Use TUF to download key/cert material (sigstore#351)
* tuf: Add initial TUF trust root updater TrustUpdater can be used to fetch specific trust roots: * Currently supports fetching * ctfe keys and * the rekor key * Caches target files in ~/.cache/sigstore-python/ * Stores metadata in ~/.local/share/sigstore-python/ * Expects to either * find the metadata _for the given URL_ in metadata store * (for prod and stage only) find the boostrap root.json in sigstore/_store The "API" that TrustUpdater provides is not meant to be final: it is the minimal one that should fulfill current needs. Nothing uses the TrustUpdater yet, but it's testable: >>> from sigstore._tuf import TrustUpdater, DEFAULT_TUF_URL >>> updater = TrustUpdater(DEFAULT_TUF_URL) >>> rekor_key_bytes = updater.get_rekor_key() Co-authored-by: Joshua Lock <jlock@vmware.com> Co-authored-by: wxjdsr <wxjdsr@126.com> Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * Rekor: Refactor CTKeyring, Use TUF in prod/staging CTKeyring: * Take bytes as constructore input: this makes it easier to feed things from either CLI arguments or the TUF trust updater. * Remove tests that no longer make sense. The prod/staging contant should still be tested but TUF is now used in the same flows: Unsure how to best test this. Use TUF to find the CTFE and rekor key when using "production" or "staging". Note that "staging" is currently untested: I am not sure even the URL makes sense. Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * cli: Use TUF for rekor/ctfe keys if not in args Use TUF to get CTFE/Rekor keys in the non-staging, non-production flow. As before, the assumption is that user wants production keys in this case. Refactor TrustUpdater so that it does not do network traffic if nothing is requested. Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * Fix linter issues in TUF related code Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * tuf: Fetch Fulcio certificates with TUF * Assume that the active fulcio certs in the repository form a certificate chain that cryptography can ingest * Refactor RekorClient construction so that we avoid constructing multiple TrustTupdaters Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * cli: Use production rekor key by default If this is not production or staging but rekor key is not given, use production: this is what original (non-tuf) code was doing as well. Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * tuf: Enable staging support https://tuf-root-staging.storage.googleapis.com/ does work as staging repository. There is still a bug somewhere as staging verify currently fails. Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * _store: Add missing staging root.json This is needed to bootstrap the TUF metadata with --staging. Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * verifier: blacken Signed-off-by: William Woodruff <william@trailofbits.com> * pyproject, sigstore/tuf: use appdirs for local state Signed-off-by: William Woodruff <william@trailofbits.com> * verifier: unused import Signed-off-by: William Woodruff <william@trailofbits.com> * _internal/tuf: disambiguate caches correctly Signed-off-by: William Woodruff <william@trailofbits.com> * sign, verify, internal: refactor rekor client handling Signed-off-by: William Woodruff <william@trailofbits.com> * test/verify: fix TestVerificationMaterials test Signed-off-by: William Woodruff <william@trailofbits.com> * Refactor RekorClient construction once more * Bring back RekorClient.production() and RekorClient.staging(): these are simple but make the calling code slightly clearer maybe * Add a TrustUpdater argument to those methods: if you use production/staging, you need a TrustUpdater * The TrustUpdater can not be constructed inside RekorClient as other components may need it as well. It's not perfectly elegant for the caller but it's not horrible either: updater = TrustUpdater.staging() client = RekorClient.staging(updater) This design means TrustUpdater does not know anything about the sigstore mechanisms: it just discovers and downloads files. Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * internal: Improve tuf docstrings Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * internal: Refactor tuf No functional change, just refactor the target discovery into a single method. Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * tests: Remove test for _store The test doesn't make a lot of sense now that the keys are not being read from the _store. Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * _store: Remove all certificates and keys These are now made available via the _internal.tuf module. There is still a case to be made for embedding keys and certs in the wheel (to optimize the first run experience, and the experience for those who might not persist their caches, e.g. CI systems). But: * Testing this without embedded keys first likely makes sense: We get more experience and feedback on the trust update system * There should be some automated system that updates the embedded keys. Otherwise obsolete keys will be embedded and no-one notices. Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * tests: Add mock TUF fetcher for staging This allows running (otherwise offline) staging tests without network access. * Add a fixture that mocks tuf.ngclient fetcher: it returns files from test assets * Mark the relevant tests with mock_staging_tuf fixture * Mark test_verifier_production() as "online": there is no way to test production tuf repository offline as it expires every two weeks Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * tests: Don't require network in parametrized setup Signer.production() and Signer.staging() now require a network connection for TUF initialization: they can't be used in parametrized test setup as that happens even if the test is marked online. Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * cli: Silence python-tuf logging a little Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * tests: Add TrustUpdater test This test asserts that we make the network requests that we expect: * Uses mock staging TUF repository * Uses empty HOME dir to ensure known starting point for caches * tests both cold and hot caches Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * tests: Add basic test for TrustUpdater Make sure the rekor key content is correct * use empty home dir * use mock TUF staging repository Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * _utils: lintage Signed-off-by: William Woodruff <william@trailofbits.com> * test/unit: put TUF assets under assets dir Signed-off-by: William Woodruff <william@trailofbits.com> * tests/unit: re-parametrize Signed-off-by: William Woodruff <william@trailofbits.com> * _store, _utils: remove obsolete comment, re-add helper Signed-off-by: William Woodruff <william@trailofbits.com> * test/unit: re-add store tests Signed-off-by: William Woodruff <william@trailofbits.com> * tuf: re-use our read_embedded helper Signed-off-by: William Woodruff <william@trailofbits.com> * README: update `--help` texts Signed-off-by: William Woodruff <william@trailofbits.com> * gitignore, test: allow staging-tuf assets Annoying. Signed-off-by: William Woodruff <william@trailofbits.com> * tuf: Switch to using f-strings for logging Signed-off-by: Alex Cameron <asc@tetsuo.sh> * test: document TUF staging mock better Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * _internal/rekor: Mention updater arg in docsstrings Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * _internal/tuf: Reword a TODO into a NOTE This is a potential improvement, not a necessary one. Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * _internal/tuf: Add nosec for mypy-related assert Also tweak one annotation (remove unneeded quotes) Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> * _internal/tuf: replace nosec with type ignore Signed-off-by: William Woodruff <william@trailofbits.com> Signed-off-by: Jussi Kukkonen <jkukkonen@google.com> Signed-off-by: William Woodruff <william@trailofbits.com> Signed-off-by: Alex Cameron <asc@tetsuo.sh> Co-authored-by: Joshua Lock <jlock@vmware.com> Co-authored-by: wxjdsr <wxjdsr@126.com> Co-authored-by: William Woodruff <william@trailofbits.com> Co-authored-by: Alex Cameron <asc@tetsuo.sh>
1 parent 020280c commit 5a68d1e

34 files changed

+881
-236
lines changed

‎.gitignore

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,5 @@ build
2121
!sigstore/_store/*.crt
2222
!sigstore/_store/*.pem
2323
!sigstore/_store/*.pub
24-
!test/unit/assets/*.txt
25-
!test/unit/assets/*.crt
26-
!test/unit/assets/*.sig
27-
!test/unit/assets/*.rekor
24+
!test/unit/assets/*
25+
!test/unit/assets/staging-tuf/*

‎README.md

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -138,12 +138,11 @@ Sigstore instance options:
138138
(default: https://rekor.sigstore.dev)
139139
--rekor-root-pubkey FILE
140140
A PEM-encoded root public key for Rekor itself
141-
(conflicts with --staging) (default: rekor.pub
142-
(embedded))
141+
(conflicts with --staging) (default: None)
143142
--fulcio-url URL The Fulcio instance to use (conflicts with --staging)
144143
(default: https://fulcio.sigstore.dev)
145144
--ctfe FILE A PEM-encoded public key for the CT log (conflicts
146-
with --staging) (default: ctfe.pub (embedded))
145+
with --staging) (default: None)
147146
```
148147
<!-- @end-sigstore-sign-help@ -->
149148
@@ -198,8 +197,7 @@ Sigstore instance options:
198197
(default: https://rekor.sigstore.dev)
199198
--rekor-root-pubkey FILE
200199
A PEM-encoded root public key for Rekor itself
201-
(conflicts with --staging) (default: rekor.pub
202-
(embedded))
200+
(conflicts with --staging) (default: None)
203201
```
204202
<!-- @end-sigstore-verify-help@ -->
205203

‎pyproject.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,13 +26,15 @@ classifiers = [
2626
"Topic :: Security :: Cryptography",
2727
]
2828
dependencies = [
29+
"appdirs ~= 1.4",
2930
"cryptography >= 38",
3031
"importlib_resources ~= 5.7; python_version < '3.11'",
3132
"pydantic",
3233
"pyjwt >= 2.1",
3334
"pyOpenSSL >= 22.0.0",
3435
"requests",
3536
"securesystemslib",
37+
"tuf >= 2.0.0",
3638
]
3739
requires-python = ">=3.7"
3840

‎sigstore/_cli.py

Lines changed: 29 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -40,13 +40,9 @@
4040
RekorClient,
4141
RekorEntry,
4242
)
43+
from sigstore._internal.tuf import TrustUpdater
4344
from sigstore._sign import Signer
44-
from sigstore._utils import (
45-
SplitCertificateChainError,
46-
load_pem_public_key,
47-
read_embedded,
48-
split_certificate_chain,
49-
)
45+
from sigstore._utils import SplitCertificateChainError, split_certificate_chain
5046
from sigstore._verify import (
5147
CertificateVerificationFailure,
5248
RekorEntryMissing,
@@ -57,23 +53,12 @@
5753
)
5854

5955
logger = logging.getLogger(__name__)
60-
logging.basicConfig(level=os.environ.get("SIGSTORE_LOGLEVEL", "INFO").upper())
61-
62-
63-
class _Embedded:
64-
"""
65-
A repr-wrapper for reading embedded resources, needed to help `argparse`
66-
render defaults correctly.
67-
"""
68-
69-
def __init__(self, name: str) -> None:
70-
self._name = name
71-
72-
def read(self) -> bytes:
73-
return read_embedded(self._name)
56+
level = os.environ.get("SIGSTORE_LOGLEVEL", "INFO").upper()
57+
logging.basicConfig(level=level)
7458

75-
def __repr__(self) -> str:
76-
return f"{self._name} (embedded)"
59+
# workaround to make tuf less verbose https://github.com/theupdateframework/python-tuf/pull/2243
60+
if level == "INFO":
61+
logging.getLogger("tuf").setLevel("WARNING")
7762

7863

7964
def _boolify_env(envvar: str) -> bool:
@@ -116,7 +101,7 @@ def _add_shared_instance_options(group: argparse._ArgumentGroup) -> None:
116101
metavar="FILE",
117102
type=argparse.FileType("rb"),
118103
help="A PEM-encoded root public key for Rekor itself (conflicts with --staging)",
119-
default=os.getenv("SIGSTORE_REKOR_ROOT_PUBK 6377 EY", _Embedded("rekor.pub")),
104+
default=os.getenv("SIGSTORE_REKOR_ROOT_PUBKEY"),
120105
)
121106

122107

@@ -238,7 +223,7 @@ def _parser() -> argparse.ArgumentParser:
238223
metavar="FILE",
239224
type=argparse.FileType("rb"),
240225
help="A PEM-encoded public key for the CT log (conflicts with --staging)",
241-
default=os.getenv("SIGSTORE_CTFE", _Embedded("ctfe.pub")),
226+
default=os.getenv("SIGSTORE_CTFE"),
242227
)
243228

244229
sign.add_argument(
@@ -427,12 +412,21 @@ def _sign(args: argparse.Namespace) -> None:
427412
elif args.fulcio_url == DEFAULT_FULCIO_URL and args.rekor_url == DEFAULT_REKOR_URL:
428413
signer = Signer.production()
429< A851 /code>414
else:
430-
ct_keyring = CTKeyring([load_pem_public_key(args.ctfe_pem.read())])
415+
# Assume "production" keys if none are given as arguments
416+
updater = TrustUpdater.production()
417+
if args.ctfe_pem is not None:
418+
ctfe_keys = [args.ctfe_pem.read()]
419+
else:
420+
ctfe_keys = updater.get_ctfe_keys()
421+
if args.rekor_root_pubkey is not None:
422+
rekor_key = args.rekor_root_pubkey.read()
423+
else:
424+
rekor_key = updater.get_rekor_key()
425+
426+
ct_keyring = CTKeyring(ctfe_keys)
431427
signer = Signer(
432428
fulcio=FulcioClient(args.fulcio_url),
433-
rekor=RekorClient(
434-
args.rekor_url, args.rekor_root_pubkey.read(), ct_keyring
435-
),
429+
rekor=RekorClient(args.rekor_url, rekor_key, ct_keyring),
436430
)
437431

438432
# The order of precedence is as follows:
@@ -560,10 +554,16 @@ def _verify(args: argparse.Namespace) -> None:
560554
except SplitCertificateChainError as error:
561555
args._parser.error(f"Failed to parse certificate chain: {error}")
562556

557+
if args.rekor_root_pubkey is not None:
558+
rekor_key = args.rekor_root_pubkey.read()
559+
else:
560+
updater = TrustUpdater.production()
561+
rekor_key = updater.get_rekor_key()
562+
563563
verifier = Verifier(
564564
rekor=RekorClient(
565565
url=args.rekor_url,
566-
pubkey=args.rekor_root_pubkey.read(),
566+
pubkey=rekor_key,
567567
# We don't use the CT keyring in verification so we can supply an empty keyring
568568
ct_keyring=CTKeyring(),
569569
),

‎sigstore/_internal/ctfe.py

Lines changed: 4 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -25,12 +25,7 @@
2525
from cryptography.hazmat.primitives import hashes
2626
from cryptography.hazmat.primitives.asymmetric import ec, rsa
2727

28-
from sigstore._utils import (
29-
PublicKey,
30-
key_id,
31-
load_pem_public_key,
32-
read_embedded,
33-
)
28+
from sigstore._utils import key_id, load_pem_public_key
3429

3530

3631
class CTKeyringError(Exception):
@@ -58,48 +53,16 @@ class CTKeyring:
5853
This structure exists to facilitate key rotation in a CT log.
5954
"""
6055

61-
def __init__(self, keys: List[PublicKey] = []):
56+
def __init__(self, keys: List[bytes] = []):
6257
"""
6358
Create a new `CTKeyring`, with `keys` as the initial set of signing
6459
keys.
6560
"""
6661
self._keyring = {}
67-
for key in keys:
62+
for key_bytes in keys:
63+
key = load_pem_public_key(key_bytes)
6864
self._keyring[key_id(key)] = key
6965

70-
@classmethod
71-
def staging(cls) -> CTKeyring:
72-
"""
73-
Returns a `CTKeyring` instance capable of verifying SCTs from
74-
Sigstore's staging deployment.
75-
"""
76-
keyring = cls()
77-
keyring._add_resource("ctfe.staging.pub")
78-
keyring._add_resource("ctfe_2022.staging.pub")
79-
keyring._add_resource("ctfe_2022.2.staging.pub")
80-
81-
return keyring
82-
83-
@classmethod
84-
def production(cls) -> CTKeyring:
85-
"""
86-
Returns a `CTKeyring` instance capable of verifying SCTs from
87-
Sigstore's production deployment.
88-
"""
89-
keyring = cls()
90-
keyring._add_resource("ctfe.pub")
91-
keyring._add_resource("ctfe_2022.pub")
92-
93-
return keyring
94-
95-
def _add_resource(self, name: str) -> None:
96-
"""
97-
Adds a key to the current keyring, as identified by its
98-
resource name under `sigstore._store`.
99-
"""
100-
key_pem = read_embedded(name)
101-
self.add(key_pem)
102-
10366
def add(self, key_pem: bytes) -> None:
10467
"""
10568
Adds a PEM-encoded key to the current keyring.

‎sigstore/_internal/rekor/client.py

Lines changed: 16 additions & 13 deletions
Original file 10000 line numberDiff line numberDiff line change
@@ -33,19 +33,14 @@
3333
from securesystemslib.formats import encode_canonical
3434

3535
from sigstore._internal.ctfe import CTKeyring
36-
from sigstore._utils import base64_encode_pem_cert, read_embedded
36+
from sigstore._internal.tuf import TrustUpdater
37+
from sigstore._utils import base64_encode_pem_cert
3738

3839
logger = logging.getLogger(__name__)
3940

4041
DEFAULT_REKOR_URL = "https://rekor.sigstore.dev"
4142
STAGING_REKOR_URL = "https://rekor.sigstage.dev"
4243

43-
_DEFAULT_REKOR_ROOT_PUBKEY = read_embedded("rekor.pub")
44-
_STAGING_REKOR_ROOT_PUBKEY = read_embedded("rekor.staging.pub")
45-
46-
_DEFAULT_REKOR_CTFE_PUBKEY = read_embedded("ctfe.pub")
47-
_STAGING_REKOR_CTFE_PUBKEY = read_embedded("ctfe.staging.pub")
48-
4944

5045
class RekorBundle(BaseModel):
5146
"""
@@ -466,20 +461,28 @@ def __del__(self) -> None:
466461
self.session.close()
467462

468463
@classmethod
469-
def production(cls) -> RekorClient:
464+
def production(cls, updater: TrustUpdater) -> RekorClient:
470465
"""
471466
Returns a `RekorClient` populated with the default Rekor production instance.
467+
468+
updater must be a `TrustUpdater` for the production TUF repository.
472469
"""
473-
return cls(
474-
DEFAULT_REKOR_URL, _DEFAULT_REKOR_ROOT_PUBKEY, CTKeyring.production()
475-
)
470+
rekor_key = updater.get_rekor_key()
471+
ctfe_keys = updater.get_ctfe_keys()
472+
473+
return cls(DEFAULT_REKOR_URL, rekor_key, CTKeyring(ctfe_keys))
476474

477475
@classmethod
478-
def staging(cls) -> RekorClient:
476+
def staging(cls, updater: TrustUpdater) -> RekorClient:
479477
"""
480478
Returns a `RekorClient` populated with the default Rekor staging instance.
479+
480+
updater must be a `TrustUpdater` for the staging TUF repository.
481481
"""
482-
return cls(STAGING_REKOR_URL, _STAGING_REKOR_ROOT_PUBKEY, CTKeyring.staging())
482+
rekor_key = updater.get_rekor_key()
483+
ctfe_keys = updater.get_ctfe_keys()
484+
485+
return cls(STAGING_REKOR_URL, rekor_key, CTKeyring(ctfe_keys))
483486

484487
@property
485488
def log(self) -> RekorLog:

0 commit comments

Comments
 (0)
0