Refactor client trust/trust root management (#1010)

* sigstore: refactor trust state management

Signed-off-by: William Woodruff <william@trailofbits.com>

* test: rename

Signed-off-by: William Woodruff <william@trailofbits.com>

* docstring

Signed-off-by: William Woodruff <william@trailofbits.com>

* sigstore, test: propagate rename

Signed-off-by: William Woodruff <william@trailofbits.com>

* _internal: hackety hack

Signed-off-by: William Woodruff <william@trailofbits.com>

* sigstore: refactor purpose handling

Signed-off-by: William Woodruff <william@trailofbits.com>

* trust: docstring

Signed-off-by: William Woodruff <william@trailofbits.com>

* fixup trust tests

Signed-off-by: William Woodruff <william@trailofbits.com>

* test_sign: fixup

Signed-off-by: William Woodruff <william@trailofbits.com>

* hook up client trust config

Signed-off-by: William Woodruff <william@trailofbits.com>

* README: update `--help`

Signed-off-by: William Woodruff <william@trailofbits.com>

* README: document BYO PKI

Signed-off-by: William Woodruff <william@trailofbits.com>

* sigstore: enforce client trust config media type

Signed-off-by: William Woodruff <william@trailofbits.com>

* fix type

Signed-off-by: William Woodruff <william@trailofbits.com>

* enforce media types, unit tests

Signed-off-by: William Woodruff <william@trailofbits.com>

* test: more trust tests

Signed-off-by: William Woodruff <william@trailofbits.com>

* CHANGELOG: record changes

Signed-off-by: William Woodruff <william@trailofbits.com>

* README: fix `--help`

Signed-off-by: William Woodruff <william@trailofbits.com>

---------

Signed-off-by: William Woodruff <william@trailofbits.com>

Author: woodruffw

CHANGELOG.md

@@ -28,6 +28,10 @@ All versions prior to 0.9.0 are untracked.
   for representing in-toto statements and DSSE envelopes
   ([#930](https://github.com/sigstore/sigstore-python/pull/930))
 
+* CLI: The `--trust-config` flag has been added as a global option,
+  enabling consistent "BYO PKI" uses of `sigstore` with a single flag
+  ([#1010](https://github.com/sigstore/sigstore-python/pull/1010))
+
 * CLI: The `sigstore verify` subcommands can now verify bundles containing
   DSSE entries, such as those produced by
   [GitHub Artifact Attestations](https://docs.github.com/en/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds)
@@ -49,6 +53,11 @@ All versions prior to 0.9.0 are untracked.
   The public verification and policy APIs now raise
   `sigstore.errors.VerificationError` on failure.
 
+* **BREAKING CLI CHANGE**: The `--rekor-url` and `--fulcio-url`
+  flags have been entirely removed. To configure a custom PKI, use
+  `--trust-config`
+  ([#1010](https://github.com/sigstore/sigstore-python/pull/1010))
+
 ### Changed
 
 * **BREAKING API CHANGE**: `Verifier.verify(...)`  now takes a `bytes | Hashed`

README.md

@@ -24,6 +24,7 @@ else!
   * [Verifying](#verifying)
     * [Generic identities](#generic-identities)
     * [Signatures from GitHub Actions](#signatures-from-github-actions)
+  * [Advanced usage](#advanced-usage)
 * [Example uses](#example-uses)
   * [Signing with ambient credentials](#signing-with-ambient-credentials)
   * [Signing with an email identity](#signing-with-an-email-identity)
@@ -96,29 +97,26 @@ Top-level:
 
 <!-- @begin-sigstore-help@ -->
 ```
-usage: sigstore [-h] [-v] [-V] [--staging] [--rekor-url URL] COMMAND ...
+usage: sigstore [-h] [-v] [-V] [--staging | --trust-config FILE] COMMAND ...
 
 a tool for signing and verifying Python package distributions
 
 positional arguments:
-  COMMAND             the operation to perform
-    sign              sign one or more inputs
-    verify            verify one or more inputs
+  COMMAND              the operation to perform
+    sign               sign one or more inputs
+    verify             verify one or more inputs
     get-identity-token
-                      retrieve and return a Sigstore-compatible OpenID Connect
-                      token
+                       retrieve and return a Sigstore-compatible OpenID
+                       Connect token
 
 optional arguments:
-  -h, --help          show this help message and exit
-  -v, --verbose       run with additional debug logging; supply multiple times
-                      to increase verbosity (default: 0)
-  -V, --version       show program's version number and exit
-
-Sigstore instance options:
-  --staging           Use sigstore's staging instances, instead of the default
-                      production instances (default: False)
-  --rekor-url URL     The Rekor instance to use (conflicts with --staging)
-                      (default: https://rekor.sigstore.dev)
+  -h, --help           show this help message and exit
+  -v, --verbose        run with additional debug logging; supply multiple
+                       times to increase verbosity (default: 0)
+  -V, --version        show program's version number and exit
+  --staging            Use sigstore's staging instances, instead of the
+                       default production instances (default: False)
+  --trust-config FILE  The client trust configuration to use (default: None)
 ```
 <!-- @end-sigstore-help@ -->
 
@@ -132,8 +130,7 @@ usage: sigstore sign [-h] [-v] [--identity-token TOKEN] [--oidc-client-id ID]
                      [--oidc-disable-ambient-providers] [--oidc-issuer URL]
                      [--oauth-force-oob] [--no-default-files]
                      [--signature FILE] [--certificate FILE] [--bundle FILE]
-                     [--output-directory DIR] [--overwrite] [--staging]
-                     [--rekor-url URL] [--fulcio-url URL]
+                     [--output-directory DIR] [--overwrite]
                      FILE [FILE ...]
 
 positional arguments:
@@ -178,18 +175,6 @@ Output options:
                         (default: None)
   --overwrite           Overwrite preexisting signature and certificate
                         outputs, if present (default: False)
-
-Sigstore instance options:
-  --staging             Use sigstore's staging instances, instead of the
-                        default production instances. This option will be
-                        deprecated in favor of the global `--staging` option
-                        in a future release. (default: False)
-  --rekor-url URL       The Rekor instance to use (conflicts with --staging).
-                        This option will be deprecated in favor of the global
-                        `--rekor-url` option in a future release. (default:
-                        None)
-  --fulcio-url URL      The Fulcio instance to use (conflicts with --staging)
-                        (default: https://fulcio.sigstore.dev)
 ```
 <!-- @end-sigstore-sign-help@ -->
 
@@ -207,7 +192,7 @@ to by a particular OIDC provider (like `https://github.com/login/oauth`).
 usage: sigstore verify identity [-h] [-v] [--certificate FILE]
                                 [--signature FILE] [--bundle FILE] [--offline]
                                 --cert-identity IDENTITY --cert-oidc-issuer
-                                URL [--staging] [--rekor-url URL]
+                                URL
                                 FILE [FILE ...]
 
 optional arguments:
@@ -234,16 +219,6 @@ Verification options:
   --cert-oidc-issuer URL
                         The OIDC issuer URL to check for in the certificate's
                         OIDC issuer extension (default: None)
-
-Sigstore instance options:
-  --staging             Use sigstore's staging instances, instead of the
-                        default production instances. This option will be
-                        deprecated in favor of the global `--staging` option
-                        in a future release. (default: False)
-  --rekor-url URL       The Rekor instance to use (conflicts with --staging).
-                        This option will be deprecated in favor of the global
-                        `--rekor-url` option in a future release. (default:
-                        None)
 ```
 <!-- @end-sigstore-verify-identity-help@ -->
 
@@ -260,7 +235,7 @@ usage: sigstore verify github [-h] [-v] [--certificate FILE]
                               [--signature FILE] [--bundle FILE] [--offline]
                               [--cert-identity IDENTITY] [--trigger EVENT]
                               [--sha SHA] [--name NAME] [--repository REPO]
-                              [--ref REF] [--staging] [--rekor-url URL]
+                              [--ref REF]
                               FILE [FILE ...]
 
 optional arguments:
@@ -294,19 +269,32 @@ Verification options:
                         under (default: None)
   --ref REF             The `git` ref that the workflow was invoked with
                         (default: None)
-
-Sigstore instance options:
-  --staging             Use sigstore's staging instances, instead of the
-                        default production instances. This option will be
-                        deprecated in favor of the global `--staging` option
-                        in a future release. (default: False)
-  --rekor-url URL       The Rekor instance to use (conflicts with --staging).
-                        This option will be deprecated in favor of the global
-                        `--rekor-url` option in a future release. (default:
-                        None)
 ```
 <!-- @end-sigstore-verify-github-help@ -->
 
+## Advanced usage
+
+### Configuring a custom root of trust ("BYO PKI")
+
+Apart from the default and "staging" Sigstore instances, `sigstore` also
+supports "BYO PKI" setups, where a user maintains their own Sigstore
+instance services.
+
+These are supported via the `--trust-config` flag, which accepts a
+JSON-formatted file conforming to the `ClientTrustConfig` message
+in the [Sigstore protobuf specs](https://github.com/sigstore/protobuf-specs).
+This file configures the entire Sigstore instance state, *including* the URIs
+used to access the CA and artifact transparency services as well as the
+cryptographic root of trust itself.
+
+To use a custom client config, prepend `--trust-config` to any `sigstore`
+command:
+
+```console
+$ sigstore --trust-config custom.trustconfig.json sign foo.txt
+$ sigstore --trust-config custom.trustconfig.json verify identity foo.txt ...
+```
+
 ## Example uses
 
 `sigstore` supports a wide variety of workflows and usages. Some common ones are

sigstore/_cli.py

@@ -27,24 +27,15 @@
 from rich.logging import RichHandler
 
 from sigstore import __version__, dsse
-from sigstore._internal.fulcio.client import (
-    DEFAULT_FULCIO_URL,
-    ExpiredCertificate,
-    FulcioClient,
-)
+from sigstore._internal.fulcio.client import ExpiredCertificate
 from sigstore._internal.rekor import _hashedrekord_from_parts
-from sigstore._internal.rekor.client import (
-    DEFAULT_REKOR_URL,
-    RekorClient,
-)
-from sigstore._internal.trustroot import KeyringPurpose, TrustedRoot
+from sigstore._internal.trust import ClientTrustConfig
 from sigstore._utils import sha256_digest
 from sigstore.errors import Error, VerificationError
 from sigstore.hashes import Hashed
 from sigstore.models import Bundle
 from sigstore.oidc import (
     DEFAULT_OAUTH_ISSUER_URL,
-    STAGING_OAUTH_ISSUER_URL,
     ExpiredIdentity,
     IdentityToken,
     Issuer,
@@ -95,35 +86,6 @@ def _boolify_env(envvar: str) -> bool:
         raise ValueError(f"can't coerce '{val}' to a boolean")
 
 
-def _add_shared_instance_options(group: argparse._ArgumentGroup) -> None:
-    """
-    Common Sigstore instance options, shared between all `sigstore` subcommands.
-    """
-    group.add_argument(
-        "--staging",
-        dest="__deprecated_staging",
-        action="store_true",
-        default=False,
-        help=(
-            "Use sigstore's staging instances, instead of the default production instances. "
-            "This option will be deprecated in favor of the global `--staging` option "
-            "in a future release."
-        ),
-    )
-    group.add_argument(
-        "--rekor-url",
-        dest="__deprecated_rekor_url",
-        metavar="URL",
-        type=str,
-        default=None,
-        help=(
-            "The Rekor instance to use (conflicts with --staging). "
-            "This option will be deprecated in favor of the global `--rekor-url` option "
-            "in a future release."
-        ),
-    )
-
-
 def _add_shared_verify_input_options(group: argparse._ArgumentGroup) -> None:
     """
     Common input options, shared between all `sigstore verify` subcommands.
@@ -230,21 +192,19 @@ def _parser() -> argparse.ArgumentParser:
         "-V", "--version", action="version", version=f"sigstore {__version__}"
     )
 
-    global_instance_options = parser.add_argument_group("Sigstore instance options")
+    global_instance_options = parser.add_mutually_exclusive_group()
     global_instance_options.add_argument(
         "--staging",
         action="store_true",
         default=_boolify_env("SIGSTORE_STAGING"),
         help="Use sigstore's staging instances, instead of the default production instances",
     )
     global_instance_options.add_argument(
-        "--rekor-url",
-        metavar="URL",
-        type=str,
-        default=os.getenv("SIGSTORE_REKOR_URL", DEFAULT_REKOR_URL),
-        help="The Rekor instance to use (conflicts with --staging)",
+        "--trust-config",
+        metavar="FILE",
+        type=Path,
+        help="The client trust configuration to use",
     )
-
     subcommands = parser.add_subparsers(
         required=True,
         dest="subcommand",
@@ -324,16 +284,6 @@ def _parser() -> argparse.ArgumentParser:
         help="Overwrite preexisting signature and certificate outputs, if present",
     )
 
-    instance_options = sign.add_argument_group("Sigstore instance options")
-    _add_shared_instance_options(instance_options)
-    instance_options.add_argument(
-        "--fulcio-url",
-        metavar="URL",
-        type=str,
-        default=os.getenv("SIGSTORE_FULCIO_URL", DEFAULT_FULCIO_URL),
-        help="The Fulcio instance to use (conflicts with --staging)",
-    )
-
     sign.add_argument(
         "files",
         metavar="FILE",
@@ -385,9 +335,6 @@ def _parser() -> argparse.ArgumentParser:
         required=True,
     )
 
-    instance_options = verify_identity.add_argument_group("Sigstore instance options")
-    _add_shared_instance_options(instance_options)
-
     # `sigstore verify github`
     verify_github = verify_subcommand.add_parser(
         "github",
@@ -449,9 +396,6 @@ def _parser() -> argparse.ArgumentParser:
         help="The `git` ref that the workflow was invoked with",
     )
 
-    instance_options = verify_github.add_argument_group("Sigstore instance options")
-    _add_shared_instance_options(instance_options)
-
     # `sigstore get-identity-token`
     get_identity_token = subcommands.add_parser(
         "get-identity-token",
@@ -476,22 +420,6 @@ def main() -> None:
 
     _logger.debug(f"parsed arguments {args}")
 
-    # A few instance flags (like `--staging` and `--rekor-url`) are supported at both the
-    # top-level `sigstore` level and the subcommand level (e.g. `sigstore verify --staging`),
-    # but the former is preferred.
-    if getattr(args, "__deprecated_staging", False):
-        _logger.warning(
-            "`--staging` should be used as a global option, rather than a subcommand option. "
-            "Passing `--staging` as a subcommand option will be deprecated in a future release."
-        )
-        args.staging = args.__deprecated_staging
-    if getattr(args, "__deprecated_rekor_url", None):
-        _logger.warning(
-            "`--rekor-url` should be used as a global option, rather than a subcommand option. "
-            "Passing `--rekor-url` as a subcommand option will be deprecated in a future release."
-        )
-        args.rekor_url = args.__deprecated_rekor_url
-
     # Stuff the parser back into our namespace, so that we can use it for
     # error handling later.
     args._parser = parser
@@ -594,18 +522,14 @@ def _sign(args: argparse.Namespace) -> None:
     if args.staging:
         _logger.debug("sign: staging instances requested")
         signing_ctx = SigningContext.staging()
-        args.oidc_issuer = STAGING_OAUTH_ISSUER_URL
-    elif args.fulcio_url == DEFAULT_FULCIO_URL and args.rekor_url == DEFAULT_REKOR_URL:
-        signing_ctx = SigningContext.production()
+    elif args.trust_config:
+        trust_config = ClientTrustConfig.from_json(args.trust_config.read_text())
+        signing_ctx = SigningContext._from_trust_config(trust_config)
     else:
-        # Assume "production" trust root if no keys are given as arguments
-        trusted_root = TrustedRoot.production(purpose=KeyringPurpose.SIGN)
-
-        signing_ctx = SigningContext(
-            fulcio=FulcioClient(args.fulcio_url),
-            rekor=RekorClient(args.rekor_url),
-            trusted_root=trusted_root,
-        )
+        # If the user didn't request the staging instance or pass in an
+        # explicit client trust config, we're using the public good (i.e.
+        # production) instance.
+        signing_ctx = SigningContext.production()
 
     # The order of precedence for identities is as follows:
     #
@@ -745,8 +669,8 @@ def _collect_verification_state(
                 missing.append(str(cert))
             input_map[file] = {"cert": cert, "sig": sig}
         else:
-            # If a user hasn't explicitly supplied `--signature`, `--certificate` or
-            # `--rekor-bundle`, we expect a bundle either supplied via `--bundle` or with the
+            # If a user hasn't explicitly supplied `--signature` or `--certificate`,
+            # we expect a bundle either supplied via `--bundle` or with the
             # default `{input}.sigstore(.json)?` name.
             if not bundle.is_file():
                 missing.append(str(bundle))
@@ -761,16 +685,11 @@ def _collect_verification_state(
     if args.staging:
         _logger.debug("verify: staging instances requested")
         verifier = Verifier.staging()
-    elif args.rekor_url == DEFAULT_REKOR_URL:
-        verifier = Verifier.production()
+    elif args.trust_config:
+        trust_config = ClientTrustConfig.from_json(args.trust_config.read_text())
+        verifier = Verifier._from_trust_config(trust_config)
     else:
-        trusted_root = TrustedRoot.production(purpose=KeyringPurpose.VERIFY)
-        verifier = Verifier(
-            rekor=RekorClient(
-                url=args.rekor_url,
-            ),
-            trusted_root=trusted_root,
-        )
+        verifier = Verifier.production()
 
     all_materials = []
     for file, inputs in input_map.items():

sigstore/_internal/rekor/checkpoint.py

@@ -27,7 +27,7 @@
 
 from pydantic import BaseModel, Field, StrictStr
 
-from sigstore._internal.trustroot import RekorKeyring
+from sigstore._internal.trust import RekorKeyring
 from sigstore._utils import KeyID
 from sigstore.errors import VerificationError