Quick note on adding mirror registry to Aliyun Kubernetes Container Services (ACK)

Trying to “fight?” the Great Firewall… (Generated by AI)

Background

Given lately the docker pulling image from docker.io seems to be blocked by the GFW, so from mainland China, to pull docker images would need to use mirror registries.

For my work I have a cluster in Aliyun within mainland China region, thus I too experienced the image pulling issue.

Environment

Note that my Kubernetes cluster is still at version 1.2x, with containerd version also being 1.2x, and because I do not have direct SSH to cluster machine, so I have to rely on the schedule job schell, so all file update would be using command line (tool like sed helps)

Solution

The direction is find replacement of source of docker image, but I do not want to change my Kubernetes config files as I would use the same setup in world outside of mainland China, I do not want to maintain 2 sets of config.

And the Aliyun Kubernetes cluster is running contianerd, so my solution is to add mirror for the containerd regarding docker.io.

According to https://github.com/containerd/containerd/blob/main/docs/hosts.md, if we are to mirror docker.io, we can create a file at /etc/containerd/certs.d/docker.io/hosts.toml and expected to be like follow:

server = "https://registry-1.docker.io"    # Exclude this to not use upstream

[host."https://public-mirror.example.com"]
  capabilities = ["pull"]                  # Requires less trust, won't resolve tag to digest from this host
[host."https://docker-mirror.internal"]
  capabilities = ["pull", "resolve"]

There is a catch, in which, when I check the /etc/containerd/config.toml, the initial observation for config path:

[plugins.”io.containerd.grpc.v1.cri”.registry]
 config_path = “”

While after creation of the /etc/containerd/certs.d/docker.io/hosts.toml file, view the /etc/containerd/config.toml again I see the config become:

[plugins."io.containerd.grpc.v1.cri".registry]
   config_path = "/etc/containerd/cert.d" 

Note that the cert.d is missing the “s”.

Finally I come up with a script to replace that line:

sudo sed -i 's/config_path = .*/config_path = "\/etc\/containerd\/certs.d"/' /etc/containerd/config.toml

Finally, a script that can repeatedly run by changing the MIRRORS array, the mirrors, please expect them to be failed at any time.

# Reference: https://github.com/containerd/containerd/blob/main/docs/hosts.md

# Check /etc/containerd/config.toml about the registry config path, 
#   it could be wrongly be:
#     /etc/containerd/cert.d instead of /etc/containerd/certs.d
#   if so, run the following:
# sudo sed -i 's/config_path = .*/config_path = "\/etc\/containerd\/certs.d"/' /etc/containerd/config.toml
# cat /etc/containerd/config.toml

MIRRORS=("https://docker-0.unsee.tech" "https://docker.1panel.live" "https://registry.dockermirror.com" "https://docker.imgdb.de" "https://hub.iyuu.cn" "https://docker.m.daocloud.io") # Replace with your mirror URLs
sudo mkdir -p /etc/containerd/certs.d/docker.io
{ printf 'server = "https://registry-1.docker.io"\n'; for mirror in "${MIRRORS[@]}"; do printf '\n[host."%s"]\n  capabilities = ["pull", "resolve"]\n' "$mirror"; done; } | sudo tee /etc/containerd/certs.d/docker.io/hosts.toml > /dev/null

cat /etc/containerd/certs.d/docker.io/hosts.toml
sudo systemctl restart containerd

Conclusion

Fighting (not actually fighting…mainly getting around…and getting defeated) the GFW is exhausting and … I have had enough of it.

In future, if I need an official image and mirrors are no longer available, I just put it on my AWS private repo and pull from it, very not productive…

👋 If you find this helpful, please click the clap 👏 button below a few times to show your support for the author 👇

🚀Join FAUN Developer Community & Get Similar Stories in your Inbox Each Week