[META] OIDC federation roadmap for Warehouse

[woodruffw]

This is meta-issue, tracking different strands of development/design for Warehouse's use of OIDC.

Goals

There are two high-level goals for Warehouse's OIDC work:

1. Enable "OIDC macaroon minting" for a handful of CI providers, enabling users to configure trusted repositories via OIDC instead of manually provisioning API tokens
2. Add support for an OIDC IdP to PyPI, allowing it to become its own identity provider for ecosystems like Sigstore

Task 2 is tracked separately under #12466. No longer planned.

Task 1: OIDC macaroon minting

This task has two subtasks: core OIDC support work, and individual support for each CI provider's OIDC IdP.

Core OIDC support work

The core work required for OIDC JWT consumption is tracked under https://github.com/pypi/warehouse/projects/4, and is almost entirely complete (but currently disabled via a feature flag). Some of the key work included additional models and services for OIDC state management, as well as a general refactor of Warehouse's AuthN/AuthZ layers to use Pyramid 2.0-style APIs.

Core refs:

• Interfaces and services for JWK management #10628
• Limetime management for OIDC JWKS #10620
• Models, routes and views for creating OIDC publishers #10753
• Update Macaroon claims/constraints for OIDC token minting #10792
• Add ExpiryCaveat #11122
• OIDC: More claims for GitHub's provider #11239
• Refactor: Migrate to 2.0-style security policies #11218

GitHub OIDC macaroon minting

This is also tracked under https://github.com/pypi/warehouse/projects/4.

Core refs:

• OIDC macaroon minting for GitHub #11272

Once #11272 is merged, this will be functionally complete (but again, disabled via a feature flag).

One sidecar feature is OIDC tokens from reusable workflows. This is tracked in #11096.

Adjacent tasks

Separately from these two tasks, there are also separate issues that track usability/UX changes that the OIDC work would benefit from.

Core refs:

• "De novo" project creation via API tokens/OIDC #11296
• Formally deprecate journal events? #11918
• Meta: Warehouse's handling and validation of distribution filenames #12316

---

CircleCI trusted publishing

This is still in the planning phase.

Google trusted publishing

This is tracked under #13551.

GitLab trusted publishing

This is tracked under #13575.

[woodruffw]

cc @di for visibility.

[di]

Some follow-on things we'll need to do once this is merged/enabled:

• Update https://github.com/marketplace/actions/pypi-publish and related docs to support this
• Update https://github.com/pypa/twine and related docs to support this
• Update https://github.com/pypa/packaging.python.org/ with a brief guide on using this

[woodruffw]

Other refs:

• Closed beta for OIDC publishing #12965 and warehouse: OIDC beta flag/controls #13073 for enabling an initial OIDC beta period
• [admin] OIDC details for projects and users #13060 for admin app/view support for OIDC publishers

[woodruffw]

Just as a reminder to myself: we probably also want to vary the OIDC audience by PyPI's FQDN, e.g. instead of pypi everywhere it should be pypi.org for the main PyPI instance and test.pypi.org for TestPyPI.

This should be easy to do, since we can grab the expected FQDN from warehouse.domain.

[di]

Just as a reminder to myself: we probably also want to vary the OIDC audience by PyPI's FQDN, e.g. instead of pypi everywhere it should be pypi.org for the main PyPI instance and test.pypi.org for TestPyPI.

We chatted about this and decided to stick with pypi/testpypi as audiences for PyPI and TestPyPI respectively because this removes the assumption of "one index per domain", which is currently true but might not be true in the future.

[woodruffw]

See #13148 for another follow-up bugfix.

[di]

For anyone following along, you can sign up for the closed beta for this feature here: #12965 (comment).

[di]

Some additional prior art to consider here as we roll this out: pub.dev also supports OIDC-based publishing:

• https://medium.com/dartlang/screenshots-and-automated-publishing-for-pub-dev-9bceb19edf79
• https://dart.dev/tools/pub/automated-publishing

[woodruffw]

Hmm, their model is interesting: rather than checking the workflow's filename, they only allow tag patterns and accept any workflow so long as it was triggered by a tag event (under the theory that only sufficiently privileged users can cut tags?).

If I'm right, that's a little weaker than our model (since any repo maintainer can cut tags by default, and anybody with commit access can modify a workflow). But the fact that they support tag patterns is a good sign that we should additionally support them 🙂

[di]

Looks like GitLab now supports a customizable aud claim: https://docs.gitlab.com/ee/ci/secrets/id_token_authentication.html

[woodruffw]

For tracking: #13551 tracks support for Google Cloud as an OIDC provider, similar to GitHub.

[woodruffw]

Opened #13575 as well, for GitLab support.

[GergelyKalmar]

Tag patterns would be a great addition to support. We typically kick off a publishing workflow on releases (which create new tags), so the branch-based environment protections are pretty useless. Using protected tags in those cases instead seems to be a better and more secure way to actually control access to PyPI publishing workflows.

[GergelyKalmar]

Actually in this case it's not even possible to sufficiently secure the PyPI publishing workflows currently. Any deployment branch rule will block the workflow from running when checked out at a tag, unless the branch rule matches the tag itself (e.g. v* seems to work for a tag like v1.7.0). However, this also allows committers to kick off a workflow from any branch that starts with a v, so that's a problem.

[woodruffw]

Tag patterns would be a great addition to support.

Agreed! This was identified during the beta period as a useful addition, and it's on my feature backlog.

We typically kick off a publishing workflow on releases (which create new tags), so the branch-based environment protections are pretty useless. Using protected tags in those cases instead seems to be a better and more secure way to actually control access to PyPI publishing workflows.

To be clear: it's useless for your case.

That's fine and valid, but there are multiple ways to secure a workflow on GitHub; we started with environment names (and corresponding environment restrictions, like required reviewers) because they impose fewer idiomatic constraints on PyPI users: not everybody releases from a tag (much less a GitHub release), so tag patterns would have been confusing to users who e.g. run a locked-down release workflow manually from a commit once it's ready for release.

The current plan is to support tag patterns as an additional optional constraint, similar to how we added support for environments.

Actually in this case it's not even possible to sufficiently secure the PyPI publishing workflows currently.

Can you say more about your expectations here?

This is why we encourage users to set up required reviewers in their environment, if their threat model is an "untrusted committer" one: regardless of tag or other event source, a required reviewer setup forces a trusted user to manually approve or reject the publishing workflow below it actually runs.

You can see an example of that in action here: https://github.com/pypa/pip-audit/actions/runs/4884995570

[GergelyKalmar]

Of course, I meant it's not particularly useful in those cases where GitHub releases are used to trigger publishing. I don't doubt there's plenty of other equally viable approaches!

As far as I understood the idea behind environments was to restrict who is able to publish to PyPI, i.e. it's possible to set up restrictions where users with commit rights are unable to obtain an OIDC token to upload an unapproved package. This is because OIDC tokens can be restricted to specific workflows and environments, and environments can add further restrictions, e.g. restricting runs to only the main branch (where one can set up branch protection rules to require reviews for example).

When releasing via GitHub releases, this approach doesn't work because the deployment does not happen from a single branch, it happens from tags (hence the deployment branch rules are inapplicable). With the matching trick we can create a deployment branch rule to restrict runs to the v* "branches" (which are actually tags in this case), however, then anyone with a commit right can create a new branch called v-hack, obtain an OIDC token and push whatever they want to PyPI.

With protected tags we could ensure that only maintainers are able to publish a release. Of course, having a manual approval still works, but we often would prefer to use user roles to restrict who is able to release/publish (instead of adding the friction of manual workflow approvals).

[woodruffw]

As far as I understood the idea behind environments was to restrict who is able to publish to PyPI, i.e. it's possible to set up restrictions where users with commit rights are unable to obtain an OIDC token to upload an unapproved package. This is because OIDC tokens can be restricted to specific workflows and environments, and environments can add further restrictions, e.g. restricting runs to only the main branch (where one can set up branch protection rules to require reviews for example).

Yep, that's the idea 🙂 -- an environment can include a specific list of people (or teams) that are allowed to run it, which means that someone has to manually approve the workflow (on GitHub's side) before an OIDC token is issued. For example, this is what that looks like on the pip-audit repository:

With protected tags we could ensure that only maintainers are able to publish a release. Of course, having a manual approval still works, but we often would prefer to use user roles to restrict who is able to release/publish (instead of adding the friction of manual workflow approvals).

Making sure I understand: the use case you'd like solved here is removing the manual approval step, right? And tag protections rather than environment protections enable that?

If so, that's perfectly reasonable, and something we want to support!

I can't promise an exact timeline for it, but it was raised during beta testing as well (we added environment support in response, as a starting point), but it's good to know that PyPI users want it and will benefit from it 🙂

[GergelyKalmar]

Yes, you understand it correctly, the idea is to remove the need for a manual approve step but still have tight control over who is allowed to publish.

It's more of a feeling but I don't like the idea of manual workflow approvals that much because it is a small out-of-context action (as opposed to doing a review of a PR and approving/declining it right away as part of that process). To me it inherently feels more vulnerable to fatigue attacks (e.g. someone triggering multiple valid approval requests in a relatively short amount of time in order to let a malicious execution slip through). A smooth release-based workflow seems more natural to me (maintainer approves a PR, merges it and creates a new release which automatically runs all necessary workflows in the background without having to think about them at all), but I also admit it's probably just a mostly irrational personal preference.

[di]

CircleCI now supports a customizable aud claim: https://circleci.com/docs/api/v2/index.html#tag/OIDC-Token-Management

[di]

I think I'm going to close this: with #13883 we've removed the feature flag for this, and we're tracking additional providers in #13551, #13575, and #13888.

Thanks everyone for your help rolling this out! 🎉

[GergelyKalmar]

Is there a separate issue tracking the tag protection support feature that we discussed earlier?

[woodruffw]

Is there a separate issue tracking the tag protection support feature that we discussed earlier?

Thanks for reminding me -- I'll check for one, but if there isn't then I'll file one.

[woodruffw]

Thanks again for reminding me @GergelyKalmar; I've opened #13911 for tracking.