Migrate kubernetes kubeadm clusters from docker to cri-o runtime

As you may known, kubernetes is deprecating docker as container runtime, the deprecation will be effective for 1.23 in late 2021.

I maintain some production clusters (single & multi nodes) running on debian buster and managed with kubeadm. Here’s how I migrated them from docker to cri-o.

Disclaimer: test this on a staging cluster, learn cri-o basics and backup before doing this on production!

My clusters were running on kubernetes 1.19.7, my plan was to migrate on cri-o and then upgrade kubernetes to 1.20.2, so I can test everything, including kubeadm upgrades, works. And actually kubeadm will require additional work, see bellow.

Migrate to cri-o

First uninstall docker:

$ systemctl stop kubelet
$ docker rm -f $(docker ps -aq)
$ apt-get purge docker-ce
$ apt-get autoremove --purge
# let's save some diskspace
$ rm -rf /var/lib/docker*

Then install cri-o as described in the documentation, the cri-o version must match the kubernetes version (see “1.19” occurrence in repository url).

$ cat << EOF > /etc/apt/sources.list.d/crio.list
deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/Debian_10/ /
deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.19/Debian_10/ /
EOF
$ wget -O - https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:1.19/Debian_10/Release.key | apt-key --keyring /etc/apt/trusted.gpg.d/crio.gpg add -
$ wget -O - https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/Debian_10/Release.key | apt-key --keyring /etc/apt/trusted.gpg.d/crio.gpg add -

$ apt-get update
$ apt-get install cri-o cri-o-runc
$ cat << EOF > /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

$ sysctl --system
$ systemctl enable crio
$ systemctl start crio

At this point you should be able to run crictl info and crictl ps with success.

Then edit /etc/default/kubelet to inform we are going to use cri-o:

KUBELET_EXTRA_ARGS=--container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --runtime-request-timeout=10m --cgroup-driver="systemd"

Then restart kubelet:

$ systemctl start kubelet

Repeat this operation on all nodes and you are done.

Upgrade to 1.20

We have to inform kubeadm we now use cri-o, otherwise kubeadm refuse to work, so on the first control-plane node:

$ apt-get install kubeadm=1.20.2-00
$ cat << EOF > kubeadm-config.yml
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
nodeRegistration:
  criSocket: /var/run/crio/crio.sock
EOF
$ kubeadm upgrade apply v1.20.2 --config kubeadm-config.yml

Also upgrade cri-o to 1.20 and upgrade your kubelet:

$ cat << EOF
deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/Debian_10/ /
deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.20/Debian_10/ /
EOF
$ apt-get update
$ apt-get install kubelet=1.20.2-00 cri-o
$ systemctl restart kubelet

On other nodes upgrade as usual kubeadm upgrade node and don’t forget to upgrade cri-o as well.