Second mirror registry with rewrite configuration not working properly #11191
Comments
|
You're using |
|
I'm trying to let the pod pull image from the mirror registry I look at the example here: https://docs.k3s.io/installation/private-registry. And also try to both configuration in rewrite part and But I got the same error which seems rewrite part does not effect it. |
|
Can you confirm that you're not using a custom Also, verify the contents of You might also check the containerd logs to see if it contains any interesting errors regarding the pull. |
The The message also suggests that there is an extra hyphen coming from somewhere... the scope is |
|
I did not use containerd.toml.tmpl, and rewrite part is in the host |
Yes, I still get the same error after fixing the regex. The interesting thing is the first registry mirror working well. Not sure if the issue is because of registry name 'cp.icr.io' includes 'cp' which part of regex? |
|
It occurs to me - you've got registries.yaml on BOTH the nodes, right? That is node-specific configuration; it is not global cluster config. You need to configure that on the agent AND the server individually. Assuming you've don that, You might try doing the following on whatever node the pod is being pulled from: That'll give you more info in the containerd.log |
|
Yes, I put registries.yaml in both server and agent nodes. |
|
Just on the off chance the replacement is doing something weird, you might also try anchoring it? rewrite:
"^cp/se-data-center-edge/(.+)$": "se-next-gen-docker-local/$1" |
|
I have similar problem. I cannot access Docker Hub, so I have placed the images on my own registry. /etc/rancher/k3s/registries.yaml mirrors:
"docker.io":
endpoint:
- https://swr.cn-east-3.myhuaweicloud.com
rewrite:
"(.*)": "hmirror/$1"
configs:
swr.cn-east-3.myhuaweicloud.com:
auth:
username: xx
password: yyInstall k3s on a single node curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_SKIP_SELINUX_RPM=true K3S_KUBECONFIG_MODE="644" INSTALL_K3S_MIRROR=cn K3S_TOKEN=SECRET INSTALL_K3S_VERSION="v1.29.4+k3s1" sh -I can see images have pulled normally from my registry when k3s install. [root@ecs-free-0001 tmp]# crictl images
IMAGE TAG IMAGE ID SIZE
docker.io/rancher/klipper-helm v0.8.3-build20240228 0929b4140ada6 91.2MB
docker.io/rancher/klipper-lb v0.4.7 edc812b8e25d0 4.78MB
docker.io/rancher/local-path-provisioner v0.0.26 c54dcef6214cb 17.2MB
docker.io/rancher/mirrored-coredns-coredns 1.10.1 ead0a4a53df89 16.2MB
docker.io/rancher/mirrored-library-traefik 2.10.7 ee69e8120b64a 43.2MB
docker.io/rancher/mirrored-metrics-server v0.7.0 b9a5a1927366a 19.3MB
docker.io/rancher/mirrored-pause 3.6 6270bb605e12e 298kB[root@ecs-free-0001 tmp]# kubectl get po -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system local-path-provisioner-6c86858495-m7p9f 1/1 Running 0 15m
kube-system svclb-traefik-839f5d4c-rkz2c 2/2 Running 0 12m
kube-system helm-install-traefik-crd-tssm4 0/1 Completed 0 15m
kube-system helm-install-traefik-frdwz 0/1 Completed 1 15m
kube-system coredns-6799fbcd5-9z2gm 1/1 Running 0 15m
kube-system traefik-7d5f6474df-kfzgh 1/1 Running 0 12m
kube-system metrics-server-54fd9b65b-fd5nn 1/1 Running 0 15m
when I pull another one image with original url from my registry , it's OK. [root@ecs-free-0001 ~]# crictl pull swr.cn-east-3.myhuaweicloud.com/hmirror/rabbitmqoperator/cluster-operator:2.8.0
Image is up to date for sha256:c0a9306b27689ddde5429e1333bac7b5ca9dc49cf005918a49518fbebbfd9d8b
[root@ecs-free-0001 ~]# crictl images | grep cluster-operator
swr.cn-east-3.myhuaweicloud.com/hmirror/rabbitmqoperator/cluster-operator 2.8.0 c0a9306b27689 26MB
[root@ecs-free-0001 ~]#but I can't pull it with rewrite. I don't know why. [root@ecs-free-0001 tmp]# crictl pull rabbitmqoperator/cluster-operator:2.8.0
E1101 17:31:20.620215 16360 remote_image.go:180] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"docker.io/rabbitmqoperator/cluster-operator:2.8.0\": failed to resolve reference \"docker.io/rabbitmqoperator/cluster-operator:2.8.0\": failed to authorize: failed to fetch oauth token: unexpected status from GET request to https://swr.cn-east-3.myhuaweicloud.com/swr/auth/v2/registry/auth/?scope=repository%3Ahmirror%2Frabbitmqoperator%2Fcluster-operator%3A&scope=repository%3Arabbitmqoperator%2Fcluster-operator%3Apull&service=dockyard: 404 Not Found" image="rabbitmqoperator/cluster-operator:2.8.0"
FATA[0000] pulling image: failed to pull and unpack image "docker.io/rabbitmqoperator/cluster-operator:2.8.0": failed to resolve reference "docker.io/rabbitmqoperator/cluster-operator:2.8.0": failed to authorize: failed to fetch oauth token: unexpected status from GET request to https://swr.cn-east-3.myhuaweicloud.com/swr/auth/v2/registry/auth/?scope=repository%3Ahmirror%2Frabbitmqoperator%2Fcluster-operator%3A&scope=repository%3Arabbitmqoperator%2Fcluster-operator%3Apull&service=dockyard: 404 Not FoundDid I make a mistake in my configuration somewhere? But why is it able to normally pull the rancher images during the k3s installation? |
|
@codering this looks like an issue with the registry you're using as a mirror. I don't know why it would be returning a 404 when you're authenticating to use it as a mirror. I do see that the auth request has the scope set twice, once for the original image repo, and once with the rewritten name. I don't think I've seen this before, but I suspect this is confusing the auth service. You might turn on containerd debug and trace the requests to see where this is coming from. |
|
@brandond But I don't understand why image pull inside the pod still failed. Do you have any other suggestions? Thank you. |
|
One of these has a double hyphen and the other does not. I point this out earlier. Is this intentional? |
@brandond Set containerd debug level to debug. Here are the detail logs. Any ideas on the issue? crictl pull docker.io/rabbitmqoperator/cluster-operator:2.7.0crictl pull swr.cn-east-3.myhuaweicloud.com/hmirror/rabbitmqoperator/cluster-operator:2.7.0 |
It looks like containerd is failing to modify the repository name in the second scope. I'm not sure why that'd be, but I do note that you're using containerd |
|
@brandond same error in newer version (v1.29.10+k3s1) crictl pull docker.io/rabbitmqoperator/cluster-operator:2.7.0 |
|
I'm not sure why you have two scopes there, but I can see if I can reproduce this internally and figure out if this is a containerd bug, or what. |
brandond
added a commit
to brandond/containerd
that referenced
this issue
Nov 6, 2024
Support CRI configuration to allow for request-time rewrite rules
applicable only to the repository portion of resource paths when pulling
images. Because the rewrites are applied at request time, images
themselves will not be "rewritten" -- images as stored by CRI (and the
underlying containerd facility) will continue to present as normal.
As an example, if you use the following config for your containerd:
```toml
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io/v2"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io".rewrite]
"^library/(.*)" = "my-org/$1"
```
And then subsequently invoke `crictl pull alpine:3.13` it will pull
content from `docker.io/my-org/alpine:3.13` but still show up as
`docker.io/library/alpine:3.13` in the `crictl images` listing.
This commit has been reworked from the original implementation. Rewites
are now done when resolving instead of when building the request, so
that auth token scopes stored in the context properly reflect the
rewritten repository path. For the original implementation, see
06c4ea9.
Ref: k3s-io/k3s#11191 (comment)
Signed-off-by: Jacob Blain Christen <[email protected]>
Co-authored-by: Brad Davidson <[email protected]>
Signed-off-by: Brad Davidson <[email protected]>
|
The issue here is that there are two scopes in the auth request. One of them comes from the Unauthorized response:
The other is generated internally by containerd based on the repository of the image being pulled, but unfortunately this is not properly rewritten. I suspect that most auth providers just ignore the unknown claim for the unmodified registry scope and use the one that they do recognize, or perhaps only evaluate the first scope, but this one apparently returns a 404 for the bearer token request because it does not recognize all of the requested scopes. To be fair, this is probably safer behavior.
I believe I have addressed this in brandond/containerd@c18a421 Note for QA: This may be difficult to reproduce, as it requires specific behavior from the registry auth provider. It appears that only artifactory and huawei cloud are affected? |
muicoder
pushed a commit
to k3s-vip/containerd
that referenced
this issue
Nov 6, 2024
Support CRI configuration to allow for request-time rewrite rules
applicable only to the repository portion of resource paths when pulling
images. Because the rewrites are applied at request time, images
themselves will not be "rewritten" -- images as stored by CRI (and the
underlying containerd facility) will continue to present as normal.
As an example, if you use the following config for your containerd:
```toml
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io/v2"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io".rewrite]
"^library/(.*)" = "my-org/$1"
```
And then subsequently invoke `crictl pull alpine:3.13` it will pull
content from `docker.io/my-org/alpine:3.13` but still show up as
`docker.io/library/alpine:3.13` in the `crictl images` listing.
This commit has been reworked from the original implementation. Rewites
are now done when resolving instead of when building the request, so
that auth token scopes stored in the context properly reflect the
rewritten repository path. For the original implementation, see
06c4ea9.
Ref: k3s-io/k3s#11191 (comment)
Signed-off-by: Jacob Blain Christen <[email protected]>
Co-authored-by: Brad Davidson <[email protected]>
Signed-off-by: Brad Davidson <[email protected]>
muicoder
pushed a commit
to k3s-vip/containerd
that referenced
this issue
Nov 6, 2024
Support CRI configuration to allow for request-time rewrite rules
applicable only to the repository portion of resource paths when pulling
images. Because the rewrites are applied at request time, images
themselves will not be "rewritten" -- images as stored by CRI (and the
underlying containerd facility) will continue to present as normal.
As an example, if you use the following config for your containerd:
```toml
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io/v2"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io".rewrite]
"^library/(.*)" = "my-org/$1"
```
And then subsequently invoke `crictl pull alpine:3.13` it will pull
content from `docker.io/my-org/alpine:3.13` but still show up as
`docker.io/library/alpine:3.13` in the `crictl images` listing.
This commit has been reworked from the original implementation. Rewites
are now done when resolving instead of when building the request, so
that auth token scopes stored in the context properly reflect the
rewritten repository path. For the original implementation, see
06c4ea9.
Ref: k3s-io/k3s#11191 (comment)
Signed-off-by: Jacob Blain Christen <[email protected]>
Co-authored-by: Brad Davidson <[email protected]>
Signed-off-by: Brad Davidson <[email protected]>
This was referenced Nov 6, 2024
This was referenced Nov 6, 2024
brandond
added a commit
to brandond/containerd
that referenced
this issue
Nov 7, 2024
Support CRI configuration to allow for request-time rewrite rules
applicable only to the repository portion of resource paths when pulling
images. Because the rewrites are applied at request time, images
themselves will not be "rewritten" -- images as stored by CRI (and the
underlying containerd facility) will continue to present as normal.
As an example, if you use the following config for your containerd:
```toml
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io/v2"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io".rewrite]
"^library/(.*)" = "my-org/$1"
```
And then subsequently invoke `crictl pull alpine:3.13` it will pull
content from `docker.io/my-org/alpine:3.13` but still show up as
`docker.io/library/alpine:3.13` in the `crictl images` listing.
This commit has been reworked from the original implementation. Rewites
are now done when resolving instead of when building the request, so
that auth token scopes stored in the context properly reflect the
rewritten repository path. For the original implementation, see
06c4ea9.
Ref: k3s-io/k3s#11191 (comment)
Signed-off-by: Jacob Blain Christen <[email protected]>
Co-authored-by: Brad Davidson <[email protected]>
Signed-off-by: Brad Davidson <[email protected]>
muicoder
pushed a commit
to k3s-vip/containerd
that referenced
this issue
Nov 8, 2024
Support CRI configuration to allow for request-time rewrite rules
applicable only to the repository portion of resource paths when pulling
images. Because the rewrites are applied at request time, images
themselves will not be "rewritten" -- images as stored by CRI (and the
underlying containerd facility) will continue to present as normal.
As an example, if you use the following config for your containerd:
```toml
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io/v2"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io".rewrite]
"^library/(.*)" = "my-org/$1"
```
And then subsequently invoke `crictl pull alpine:3.13` it will pull
content from `docker.io/my-org/alpine:3.13` but still show up as
`docker.io/library/alpine:3.13` in the `crictl images` listing.
This commit has been reworked from the original implementation. Rewites
are now done when resolving instead of when building the request, so
that auth token scopes stored in the context properly reflect the
rewritten repository path. For the original implementation, see
06c4ea9.
Ref: k3s-io/k3s#11191 (comment)
Signed-off-by: Jacob Blain Christen <[email protected]>
Co-authored-by: Brad Davidson <[email protected]>
Signed-off-by: Brad Davidson <[email protected]>
muicoder
pushed a commit
to k3s-vip/containerd
that referenced
this issue
Nov 8, 2024
Support CRI configuration to allow for request-time rewrite rules
applicable only to the repository portion of resource paths when pulling
images. Because the rewrites are applied at request time, images
themselves will not be "rewritten" -- images as stored by CRI (and the
underlying containerd facility) will continue to present as normal.
As an example, if you use the following config for your containerd:
```toml
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io/v2"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io".rewrite]
"^library/(.*)" = "my-org/$1"
```
And then subsequently invoke `crictl pull alpine:3.13` it will pull
content from `docker.io/my-org/alpine:3.13` but still show up as
`docker.io/library/alpine:3.13` in the `crictl images` listing.
This commit has been reworked from the original implementation. Rewites
are now done when resolving instead of when building the request, so
that auth token scopes stored in the context properly reflect the
rewritten repository path. For the original implementation, see
06c4ea9.
Ref: k3s-io/k3s#11191 (comment)
Signed-off-by: Jacob Blain Christen <[email protected]>
Co-authored-by: Brad Davidson <[email protected]>
Signed-off-by: Brad Davidson <[email protected]>
muicoder
pushed a commit
to k3s-vip/containerd
that referenced
this issue
Nov 11, 2024
Support CRI configuration to allow for request-time rewrite rules
applicable only to the repository portion of resource paths when pulling
images. Because the rewrites are applied at request time, images
themselves will not be "rewritten" -- images as stored by CRI (and the
underlying containerd facility) will continue to present as normal.
As an example, if you use the following config for your containerd:
```toml
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io/v2"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io".rewrite]
"^library/(.*)" = "my-org/$1"
```
And then subsequently invoke `crictl pull alpine:3.13` it will pull
content from `docker.io/my-org/alpine:3.13` but still show up as
`docker.io/library/alpine:3.13` in the `crictl images` listing.
This commit has been reworked from the original implementation. Rewites
are now done when resolving instead of when building the request, so
that auth token scopes stored in the context properly reflect the
rewritten repository path. For the original implementation, see
06c4ea9.
Ref: k3s-io/k3s#11191 (comment)
Signed-off-by: Jacob Blain Christen <[email protected]>
Co-authored-by: Brad Davidson <[email protected]>
Signed-off-by: Brad Davidson <[email protected]>
muicoder
pushed a commit
to k3s-vip/containerd
that referenced
this issue
Nov 12, 2024
Support CRI configuration to allow for request-time rewrite rules
applicable only to the repository portion of resource paths when pulling
images. Because the rewrites are applied at request time, images
themselves will not be "rewritten" -- images as stored by CRI (and the
underlying containerd facility) will continue to present as normal.
As an example, if you use the following config for your containerd:
```toml
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io/v2"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io".rewrite]
"^library/(.*)" = "my-org/$1"
```
And then subsequently invoke `crictl pull alpine:3.13` it will pull
content from `docker.io/my-org/alpine:3.13` but still show up as
`docker.io/library/alpine:3.13` in the `crictl images` listing.
This commit has been reworked from the original implementation. Rewites
are now done when resolving instead of when building the request, so
that auth token scopes stored in the context properly reflect the
rewritten repository path. For the original implementation, see
06c4ea9.
Ref: k3s-io/k3s#11191 (comment)
Signed-off-by: Jacob Blain Christen <[email protected]>
Co-authored-by: Brad Davidson <[email protected]>
Signed-off-by: Brad Davidson <[email protected]>
muicoder
pushed a commit
to k3s-vip/containerd
that referenced
this issue
Nov 15, 2024
Support CRI configuration to allow for request-time rewrite rules
applicable only to the repository portion of resource paths when pulling
images. Because the rewrites are applied at request time, images
themselves will not be "rewritten" -- images as stored by CRI (and the
underlying containerd facility) will continue to present as normal.
As an example, if you use the following config for your containerd:
```toml
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io/v2"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io".rewrite]
"^library/(.*)" = "my-org/$1"
```
And then subsequently invoke `crictl pull alpine:3.13` it will pull
content from `docker.io/my-org/alpine:3.13` but still show up as
`docker.io/library/alpine:3.13` in the `crictl images` listing.
This commit has been reworked from the original implementation. Rewites
are now done when resolving instead of when building the request, so
that auth token scopes stored in the context properly reflect the
rewritten repository path. For the original implementation, see
06c4ea9.
Ref: k3s-io/k3s#11191 (comment)
Signed-off-by: Jacob Blain Christen <[email protected]>
Co-authored-by: Brad Davidson <[email protected]>
Signed-off-by: Brad Davidson <[email protected]>
muicoder
pushed a commit
to k3s-vip/containerd
that referenced
this issue
Nov 20, 2024
Support CRI configuration to allow for request-time rewrite rules
applicable only to the repository portion of resource paths when pulling
images. Because the rewrites are applied at request time, images
themselves will not be "rewritten" -- images as stored by CRI (and the
underlying containerd facility) will continue to present as normal.
As an example, if you use the following config for your containerd:
```toml
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io/v2"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io".rewrite]
"^library/(.*)" = "my-org/$1"
```
And then subsequently invoke `crictl pull alpine:3.13` it will pull
content from `docker.io/my-org/alpine:3.13` but still show up as
`docker.io/library/alpine:3.13` in the `crictl images` listing.
This commit has been reworked from the original implementation. Rewites
are now done when resolving instead of when building the request, so
that auth token scopes stored in the context properly reflect the
rewritten repository path. For the original implementation, see
06c4ea9.
Ref: k3s-io/k3s#11191 (comment)
Signed-off-by: Jacob Blain Christen <[email protected]>
Co-authored-by: Brad Davidson <[email protected]>
Signed-off-by: Brad Davidson <[email protected]>
muicoder
pushed a commit
to k3s-vip/containerd
that referenced
this issue
Nov 21, 2024
Support CRI configuration to allow for request-time rewrite rules
applicable only to the repository portion of resource paths when pulling
images. Because the rewrites are applied at request time, images
themselves will not be "rewritten" -- images as stored by CRI (and the
underlying containerd facility) will continue to present as normal.
As an example, if you use the following config for your containerd:
```toml
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io/v2"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io".rewrite]
"^library/(.*)" = "my-org/$1"
```
And then subsequently invoke `crictl pull alpine:3.13` it will pull
content from `docker.io/my-org/alpine:3.13` but still show up as
`docker.io/library/alpine:3.13` in the `crictl images` listing.
This commit has been reworked from the original implementation. Rewites
are now done when resolving instead of when building the request, so
that auth token scopes stored in the context properly reflect the
rewritten repository path. For the original implementation, see
06c4ea9.
Ref: k3s-io/k3s#11191 (comment)
Signed-off-by: Jacob Blain Christen <[email protected]>
Co-authored-by: Brad Davidson <[email protected]>
Signed-off-by: Brad Davidson <[email protected]>
muicoder
pushed a commit
to k3s-vip/containerd
that referenced
this issue
Nov 22, 2024
Support CRI configuration to allow for request-time rewrite rules
applicable only to the repository portion of resource paths when pulling
images. Because the rewrites are applied at request time, images
themselves will not be "rewritten" -- images as stored by CRI (and the
underlying containerd facility) will continue to present as normal.
As an example, if you use the following config for your containerd:
```toml
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io/v2"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io".rewrite]
"^library/(.*)" = "my-org/$1"
```
And then subsequently invoke `crictl pull alpine:3.13` it will pull
content from `docker.io/my-org/alpine:3.13` but still show up as
`docker.io/library/alpine:3.13` in the `crictl images` listing.
This commit has been reworked from the original implementation. Rewites
are now done when resolving instead of when building the request, so
that auth token scopes stored in the context properly reflect the
rewritten repository path. For the original implementation, see
06c4ea9.
Ref: k3s-io/k3s#11191 (comment)
Signed-off-by: Jacob Blain Christen <[email protected]>
Co-authored-by: Brad Davidson <[email protected]>
Signed-off-by: Brad Davidson <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Environmental Info:
K3s Version:
k3s version v1.30.5+k3s1
go version go1.22.6
Node(s) CPU architecture, OS, and Version:
Linux 5.14.0-284.30.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Aug 25 09:13:12 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux
Cluster Configuration:
1 server, 1 agent
Describe the bug:
I set up two mirror registries with rewrite configuration in registries.yaml in k3s(both server and agent)
The first mirror registry configuration works well. I can start a pod that needs to pull image from
icr.io/cpopen/edge-operator-catalog@sha256:4f9725b23c8560eae25be0a9fac01c74c9d4a9fee8200e31aad9842f7c338433, but actually pull image from mirror registry:https://docker-na-public.artifactory.test.com/se-next-gen-docker-local/edge-operator-catalog@sha256:4f9725b23c8560eae25be0a9fac01c74c9d4a9fee8200e31aad9842f7c338433successfullyHowever, the second mirror registry configuration does not work properly. Another pod which needs to pull image from cp.icr.io/cp/se-data-center-edge/mini-test@sha256:c718d3f996061aef92966a2171713af1cfdbac93cbea7a753107e3d5430c3687 can not pull image from mirror registry
https://docker-na-public.artifactory.test.com/se-next-gen-docker-local/mini-test@sha256:c718d3f996061aef92966a2171713af1cfdbac93cbea7a753107e3d5430c3687.
The error shows
The thing is I could manually pull that image from mirror registry
Is that rewrite configuration wrong for the second mirror registry?
Steps To Reproduce:
Expected behavior:
Actual behavior:
Additional context / logs:
The text was updated successfully, but these errors were encountered: