Support additional OIDC configuration with shoot-oidc-service extension

[pbochynski]

Description
Enable the option to trust the additional identity provider compliant with OpenID. The provider can be registered in the Kyma cluster and kubernetes API server will authenticate tokens that match the provider issuer.
The complete solution should allow to establish the trust during the provisioning so the cluster can be accessed from fully automated processes (without user presence). To accomplish that the following changes are required:

• enable shoot-oidc-service extension for shoot clusters
• add possibility to pass the OpenIDConnect resource in the shoot cluster as Kyma instance parameter (KEB)
• grant permissions to accounts authenticated by new provider by some role and role binding (or use the existing feature of extending administrator list)

Acceptance Criteria

• Ensure Isolation between user facing and operator-facing OIDC configs. User should not be able to override Opereator facing OIDC
  • Configure Operator facing OIDC
  • Extend Kyma Provisioning Parameters to allow multiple OIDC definitions kyma-environment-broker#423

Reasons
Many users request a possibility to deploy software to freshly creaated Kyma clusters in automated way. Changing the default IDP for the cluster is the only solution available for now, but then IDP has to support both human users and service accounts what is usually challenging. With additional OIDC provider it can be used only for system to system authorization and will be much easier to set up.

Links

• Provide tooling for automated Kyma lifecycle and subscription management #18198
• https://github.com/gardener/oidc-webhook-authenticator/blob/master/README.md
• https://blogs.sap.com/2022/09/23/using-github-actions-openid-connect-in-kubernetes/
• https://github.com/gardener/gardener-extension-shoot-oidc-service/blob/master/docs/usage/openidconnects.md

[maximilianbraun]

Would love to test it at earliest convinience.

[kyma-bot]

This issue or PR has been automatically marked as stale due to the lack of recent activity.
Thank you for your contributions.

This bot triages issues and PRs according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 7d of inactivity since lifecycle/stale was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Close this issue or PR with /close

If you think that I work incorrectly, kindly raise an issue with the problem.

/lifecycle stale

[kwiatekus]

The goal would be to allow kyma users to deploy apps (and run tests ) on the provisioned kyma runtimes in the CI jobs .

Execution can be divided into several stages:

• [POC] Authenticate towards gardener's shoot cluster using shoot-oidc-service extension and GH token  #18519
• Handle shoot-oidc-service extension when provisioning kyma [KIM/feature] infrastructure-manager#381 - Q2 2024 (see comment)
• Extend Kyma Provisioning Parameters to allow multiple OIDC definitions kyma-environment-broker#423

[kwiatekus]

OIDC configuration will be included in the gardener provisioning in q2
kyma-project/infrastructure-manager#381

[github-actions]

This issue has been automatically marked as stale due to the lack of recent activity. It will soon be closed if no further activity occurs.
Thank you for your contributions.

[kwiatekus]

Kyma Infrastructure Manager (KIM) should apply Operator-Facing OIDC separately from User-facing OIDC(s)

In order to allow this we need to adjust the interfaces:

1. User -> Kyma Environment Broker (KEB)

• User should be able to define multiple OIDCs
• User should be able to define requiedClaims (key-value pairs)

2. KEB -> KIM

• If user provides OIDC(s) KEB populates them as additionalOidcConfigs
• If user provides no OIDC config KIM defaults the additionalOidcConfigs[0] with the default User-facing OIDC config
• KIM defaults oidcConfig with Operator-facing OIDC

3. KIM -> Gardener

• KIM enables shoot-oidc-service extension in shoot CR
• KIM defines Operator-facing OIDC explicitely in shoot CR

4. KIM -> Kyma

• KIM reconciles authentication.gardener.cloud/v1alpha1/OpenIDConnect objects in kyma based on additionalOidcConfigs
• KIM reconciles the clusterrolebindings based on administrators it got from KEB
• KIM ensures clusterrole binding for operators

to avoid unwanted impersonations, we should:

• ensure prefixes in Operator-facing OIDC (for username and groups)
• document recommendation about prefixes in user facing documentation

[Disper]

Regarding migration from existing shoots to Runtime CRs.

• oidcConfig <- Populate withoidcConfig
• additionalOidcConfigs <- Populate as the first entry withoidcConfig

[burkardendres]

Would be great to have this available soon, as a dependency we want to use requires that and also to bring our kya into production it would be kind of a prerequisite.

[tobiscr]

To unblock customers who are waiting for the enablement of the OIDC extension in GArdener, we updated the Provisioner last week to activate this extension per default for all new created clusters. So, any new created SKR cluster will have the OIDC extension now enabled and customers can configure their own OIDC provider by creating the corresponding CR in their SKR clusters.

It is planned to start the replacement of the Provisioner with KIM (Kyma Infrasturcture Manager) by end of this month. KIM will also per default enable the OIDC extension for all managed clusters.

[s-u-b-h-a-k-a-r]

it would be very helpful to have this available soon, as a dependency we want to use requires that and also to bring our kyma into production it would be kind of a prerequisite and this will help us to manage BTP Kyma Workload via Cloud Orchestrator/Crossplane without the need for manual task

[vrenjith]

It will be super helpful if we have this capability so that we can bootstrap ArgoCD in the provisioned Kyma cluster so that ArgoCD to take over the rest of the deployments that are coming in from the service pipelines.

[ptesny]

It will be super helpful if we have this capability so that we can bootstrap ArgoCD in the provisioned Kyma cluster so that ArgoCD to take over the rest of the deployments that are coming in from the service pipelines.

That's sth which is already done via a terraform based automation module: https://github.com/quovadis-btp/btp-automation/tree/main/btp-context/runtime-context
At the end of kyma env provisioning an existing ArgoCD instance is being bootstrapped with the newly created kyma cluster. I also create a dedicated ArgoCD project on-the-fly

I'm trying hard to finalize a public root module which will then allow you to use the above child module. Will keep you posted. Thx

#18666

[abhijith-darshan]

This is great news that shoot-oidc-service extension is available in all newly created kyma clusters. What is the timeline to roll this out to already existing SAP Kyma clusters?

[tobiscr]

Sorry for the late reply @abhijith-darshan. We were facing and fixing several unexpected challenges in the new Infrastructure Manager during the last weeks which were again causing delay in our rollout.

Unfortunaltey, we were not able to migrate all productive clusters to the new Kyma Infrastructure Manager yet which is a pre-requisite for releasing the OIDC feature. We are currently finalising the testing phase of KIM and plan to migrate all productive clusters beginning of next year.

Our SRE team will, after KIM was released, migrate all existing clusters to support the new OIDC configuration. @ebensom can provide more details when this migration will happen.