KCP Sharding

[janmedrek]

Description

The KCP sharding concept should be introduced to allow us to scale KCP clusters.

The whole concept revolves around providing multiple K8S clusters, this way the Kube API server will not get stuffed with all the necessary resources to serve SKR lifecycle functionalities.

The following questions would need to be answered:

• Should KEB be the singleton component responsible for the round-robin dispatching over multiple KCP clusters?
• Which operators should be "sharded"? Naturally, KLM should pop up on that list, what about KIM/other KCP components?
• Should we set up separate Gardener projects for specific KCP shards?

Acceptance Criteria (TBD)

• KCP sharding introduced

Context

Problem
With the increasing number of resources that need to be reconciled, problems with performance occur. Some sort of horizontal scaling needs to be introduced as operators serving such resources cannot be scaled indefinitely.