Kubernetes upgrade from an old version — 1.29 to 1.35.

I have on old cluster on my laptop and while testing one POC on my cluster, I faced an issue as I was using the old version 1.29.

I am posting the installation process here, with all the challenges and solutions. It may not follow a conventional blog language as I am just pasting what I am doing.

It may help the beginers who play with their homesetup.

If you check the available versions directly, you wont see the latest versions. Now the latest version is 1.35 and my machine is showing a different stotry

syam@master:~$ sudo apt-cache madison kubeadm kubeadm | 1.29.3–1.1 | https://pkgs.k8s.io/core:/stable:/v1.29/deb Packages kubeadm | 1.29.2–1.1 | https://pkgs.k8s.io/core:/stable:/v1.29/deb Packages kubeadm | 1.29.1–1.1 | https://pkgs.k8s.io/core:/stable:/v1.29/deb Packages kubeadm | 1.29.0–1.1 | https://pkgs.k8s.io/core:/stable:/v1.29/deb Packages

Check your /etc/apt/sources.list.d/kubernetes.list

syam@master:~$ cat /etc/apt/sources.list.d/kubernetes.list deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /

It is pointing to 1.29, you need to change it to the next version.

I am not changing it to the latest 1.35, as kubernetes does not support long minor version upgrades.

Ref : https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/

I have changed the repo to 1.30

syam@master:~$ cat /etc/apt/sources.list.d/kubernetes.list deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /

Now I am getting the upgrade paths.

syam@master:~$ apt-cache madison kubeadm kubeadm | 1.30.14–1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.13–1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.12–1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.11–1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.10–1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.9–1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.8–1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.7–1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.6–1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.5–1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.4–1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.3–1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.2–1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.1–1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Packages kubeadm | 1.30.0–1.1 | https://pkgs.k8s.io/core:/stable:/v1.30/deb Package

Let me choose the latest from the above

syam@master:~$ sudo apt install -y kubeadm=1.30.14–1.1 Reading package lists… Done Building dependency tree… Done Reading state information… Done The following additional packages will be installed: cri-tools The following held packages will be changed: kubeadm The following packages will be upgraded: cri-tools kubeadm 2 upgraded, 0 newly installed, 0 to remove and 134 not upgraded. E: Held packages were changed and -y was used without — allow-change-held-packages. syam@master:~$ sudo apt install -y kubeadm=1.30.14–1.1 — allow-change-held-packages Reading package lists… Done Building dependency tree… Done Reading state information… Done The following additional packages will be installed: cri-tools The following held packages will be changed: kubeadm The following packages will be upgraded: cri-tools kubeadm 2 upgraded, 0 newly installed, 0 to remove and 134 not upgraded. Need to get 31.8 MB of archives. After this operation, 5,395 kB of additional disk space will be used. Get:1 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb cri-tools 1.30.1–1.1 [21.3 MB] Get:2 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.30/deb kubeadm 1.30.14–1.1 [10.5 MB] Fetched 31.8 MB in 9s (3,715 kB/s) (Reading database … 110904 files and directories currently installed.) Preparing to unpack …/cri-tools_1.30.1–1.1_amd64.deb … Unpacking cri-tools (1.30.1–1.1) over (1.29.0–1.1) … Preparing to unpack …/kubeadm_1.30.14–1.1_amd64.deb … Unpacking kubeadm (1.30.14–1.1) over (1.29.3–1.1) … Setting up cri-tools (1.30.1–1.1) … Setting up kubeadm (1.30.14–1.1) … Scanning processes… Scanning linux images…

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.

Now the kubeadm is upgraded to 1.30

syam@master:~$ kubeadm version kubeadm version: &version.Info{Major:”1", Minor:”30", GitVersion:”v1.30.14", GitCommit:”9e18483918821121abdf9aa82bc14d66df5d68cd”, GitTreeState:”clean”, BuildDate:”2025–06–17T18:34:53Z”, GoVersion:”go1.23.10", Compiler:”gc”, Platform:”linux/amd64"}

Lets plan the upgrade

syam@master:~$ sudo kubeadm upgrade plan [preflight] Running pre-flight checks. [upgrade/config] Reading configuration from the cluster… [upgrade/config] FYI: You can look at this config file with ‘kubectl -n kube-system get cm kubeadm-config -o yaml’ [upgrade] Running cluster health checks [upgrade] Fetching available versions to upgrade to [upgrade/versions] Cluster version: 1.29.3 [upgrade/versions] kubeadm version: v1.30.14 I0201 06:24:46.094288 26755 version.go:256] remote version is much newer: v1.35.0; falling back to: stable-1.30 [upgrade/versions] Target version: v1.30.14 [upgrade/versions] Latest version in the v1.29 series: v1.29.15

Components that must be upgraded manually after you have upgraded the control plane with ‘kubeadm upgrade apply’: COMPONENT NODE CURRENT TARGET kubelet master v1.29.3 v1.29.15 kubelet worker1 v1.29.3 v1.29.15 kubelet worker2 v1.29.3 v1.29.15

Upgrade to the latest version in the v1.29 series:

COMPONENT NODE CURRENT TARGET kube-apiserver master v1.29.3 v1.29.15 kube-controller-manager master v1.29.3 v1.29.15 kube-scheduler master v1.29.3 v1.29.15 kube-proxy 1.29.3 v1.29.15 CoreDNS v1.11.1 v1.11.3 etcd master 3.5.12–0 3.5.15–0

You can now apply the upgrade by executing the following command:

kubeadm upgrade apply v1.29.15

Components that must be upgraded manually after you have upgraded the control plane with ‘kubeadm upgrade apply’: COMPONENT NODE CURRENT TARGET kubelet master v1.29.3 v1.30.14 kubelet worker1 v1.29.3 v1.30.14 kubelet worker2 v1.29.3 v1.30.14

Upgrade to the latest stable version:

COMPONENT NODE CURRENT TARGET kube-apiserver master v1.29.3 v1.30.14 kube-controller-manager master v1.29.3 v1.30.14 kube-scheduler master v1.29.3 v1.30.14 kube-proxy 1.29.3 v1.30.14 CoreDNS v1.11.1 v1.11.3 etcd master 3.5.12–0 3.5.15–0

You can now apply the upgrade by executing the following command:

kubeadm upgrade apply v1.30.14

The table below shows the current state of component configs as understood by this version of kubeadm. Configs that have a “yes” mark in the “MANUAL UPGRADE REQUIRED” column require manual config upgrade or resetting to kubeadm defaults before a successful upgrade can be performed. The version to manually upgrade to is denoted in the “PREFERRED VERSION” column.

API GROUP CURRENT VERSION PREFERRED VERSION MANUAL UPGRADE REQUIRED kubeproxy.config.k8s.io v1alpha1 v1alpha1 no kubelet.config.k8s.io v1beta1 v1beta1 no

Now we can see two upgrade paths, one is patch upgrade and another one is minor upgrade. In production level clusters, we need to perform the patch first,but here I am proceeding with the minor directly.

I have upgraded the kubeadm

sudo kubeadm upgrade apply v1.30.14

[upgrade/successful] SUCCESS! Your cluster was upgraded to “v1.30.14”. Enjoy!

[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven’t already done so.

This upgraded the API server version not the kubelet version.

The API server version is upgraded, lets upgrade the other resources.

sudo apt install -y kubelet=1.30.14-* — allow-change-held-packages

sudo apt install -y kubectl=1.30.14-* — allow-change-held-packages

Now the master is upgraded to 1.30

syam@master:~$ kubectl describe pod kube-apiserver-master -n kube-system | grep “Image:” Image: registry.k8s.io/kube-apiserver:v1.30.14 syam@master:~$

No we need to perform the same activity in all the clusters.

I have to start my second worker to drain the worker1 before upgrade.

Now the worker 2 is ready.

If you get key errors, you can update the new key.

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg — dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg File ‘/etc/apt/keyrings/kubernetes-apt-keyring.gpg’ exists. Overwrite? (y/N) yes

The steps on workers are simple,

1. Cordon and drain the node
2. update the key and update the version on the file /etc/apt/sources.list.d/kubernetes.list
3. apt update
4. upgrade kubectl

sudo apt install -y kubelet=1.30.14-* — allow-change-held-packages

5. Uncorden and make the node ready.

Now the cluster is upgraded to V1.30.

Then, I have created one shell script to repleat the process and upgraded the cluster to latest version.

I’ve faced kubelet down error when upgraded to 1.35, and it is sorted now after removing the flag — pod-infra-container-image=registry.k8s.io/pause:3.9" from the below env conf.

syam@worker1:~$ cat /var/lib/kubelet/kubeadm-flags.env KUBELET_KUBEADM_ARGS=” — container-runtime-endpoint=unix:///var/run/containerd/containerd.sock — pod-infra-container-image=registry.k8s.io/pause:3.9"

Ref : https://github.com/kubernetes/kubeadm/issues/3279

The cluster is stable now with latest version 1.35.