Dex : Unable to connect to the server: failed to refresh token: oauth2: cannot fetch token: 400 Bad Request Response: {"error":"invalid_request","error_description":"Refresh token is invalid or has already been claimed by another client."}

[newbasks]

I am looking to separate Dex token authentication from a group of k8s clusters, the clusters have OIDC client and Dex runs on a one of the clusters.
Currently testing this with only 2 clusters, one has dex running and controls authentication for both clusters. The authentication works well for the initial few hours, however, it starts failing for either one of them and I get below error later while using kubectl

Error:

kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.6", GitCommit:"9f8ebd171479bec0ada837d7ee641dec2f8c6dd1", GitTreeState:"clean", BuildDate:"2018-03-21T15:21:50Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"windows/amd64"}
Unable to connect to the server: failed to refresh token: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"invalid_request","error_description":"Refresh token is invalid or has already been claimed by another client."}

Dex version :: v2.10.0

time="2019-03-06T10:35:43Z" level=info msg="performing ldap search ou=People,dc=abc,dc=com sub (uid=abc)" connector=LDAP
time="2019-03-06T10:35:43Z" level=info msg="username "abc" mapped to entry uid=abc,ou=people,dc=abc,dc=com" connector=LDAP
time="2019-03-06T10:35:43Z" level=info msg="login successful: connector "ldap", username="abc", email="[email protected]", groups=[]"
time="2019-03-06T10:36:05Z" level=error msg="failed to get refresh token: not found"
time="2019-03-06T12:12:40Z" level=info msg="keys expired, rotating"
time="2019-03-06T12:12:42Z" level=info msg="keys rotated, next rotation: 2019-03-06 18:12:42.033490942 +0000 UTC"
time="2019-03-06T14:03:17Z" level=error msg="failed to get refresh token: not found"
time="2019-03-06T14:06:12Z" level=error msg="failed to get refresh token: not found"

Currently, OIDC client information has remained the same on both clusters and Dex config has the OIDC client reference accordingly. I am wondering if the client secret is creating an issue or if you could please suggest the area this has to be looked into, Many thanks

[ericchiang]

Dupe of #981.

A signal client/user combination can only have one refresh token out for a single dex instance. Your different clusters need to use different client IDs.

[newbasks]

Thanks, Eric, could you tell the maximum number of oidc client entries that can be used for a single dex instance. I am centralizing/separating dex authentication and need it to support <30 clusters. The OIDC client is Gangway but for uniqueness, the client ID and secret are different for every entry in dex config.

staticClients:
- id: K8s-Authentication
  redirectURIs:
  - 'https://login-app.ex1.com/ui'        
  name: 'Dev Cluster1’
  secret: ZXhhbXBsZS1hcHAtc2VjcmV0
- id: k8s-Auth-Dev2
  redirectURIs:
  - 'https://login-app.ex2.com/ui'
  name: 'Dev Cluster2’
  secret: ZXhhbXBsZS1hcHAtc2VjcmV022
  #trustedPeers:
  # - K8s-Authentication
  - id: k8s-Auth-Dev3
  redirectURIs:
  -  'https://login-app.ex3.com/ui'
   name: 'Dev Cluster3’
  secret: ZXhhbXBsZS1hcHAtc2VjcmV033
  #trustedPeers:
  #- K8s-Authentication

On Dev2 cluster the kube-apiserver yaml

• --oidc-issuer-url='https://dex-app.ex1.com
• --oidc-client-id= k8s-Auth-Dev2
• --oidc-username-claim=email
• --oidc-groups-claim=groups
• --oidc-ca-file= /etc/pki/ca.crt

On Dev3 cluster the kube-apiserver yaml

• --oidc-issuer-url='https://dex-app.ex1.com
• --oidc-client-id= k8s-Auth-Dev3
• --oidc-username-claim=email
• --oidc-groups-claim=groups
• --oidc-ca-file= /etc/pki/ca.crt

Are there extra or anything on the same config needs to be changed