Building a Raspberry Pi Kubernetes Cluster on Ubuntu 20.04 using kubeadm and Containerd as the runtime.

Reading Time: 7 minutes

I am currently studying for the Certified Kubernetes Administrator exam (CKA) and as a hands on learning I wanted to get down and dirty with Kubernetes. In my day job I work a lot with Azure Kubernetes Service (AKS) and I would say I am very comfortable with AKS and managing Kubernetes objects on an AKS cluster. But with AKS and other managed Kubernetes offerings the control plane is handle by the provider, so you don’t get to see it, or configure it. That’s a really good thing for businesses, but not for people who want to study.

I was going to use an old HPE Microserver I have, but the noise was just too much. I was then going to use VM’s in Azure, but I was running out of credits and wanted something a bit more hands on. So, I decided to try and build a Raspberry Pi cluster. I wanted to use Ubuntu as the operating system as that is what I know and what is used in AKS, although an older version in the cloud. They have just started with 18.04. Ubuntu 20.04 is the latest LTS version, so I decided to go with that.

For the runtime, recently Kubernetes made the decision to use containerd as the container runtime. So, I thought let’s do that to. If I chose to use docker then it would have been a lot easier. In fact, if I had chosen to use say k3s then again it would have been a lot easier, but I would not have a pure Kubernetes cluster and no etcd datastore.

Well, that’s the background to this blog post, enough with all the words let’s get on with the build.

Getting started

First things first, you need the hardware, below is what i am using.

Hardware

• 4 x raspberry Pi 4 Model B 4GB ram (You can get away with 3) https://thepihut.com/products/raspberry-pi-4-model-b?variant=20064052740158
• 1 x case. I used this one. https://www.amazon.co.uk/gp/product/B07F6Y1MJ6/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1
• 4 x 128 GB micro SD card. One for each Pi. https://www.amazon.co.uk/gp/product/B08GYKNCCP/ref=ppx_yo_dt_b_asin_title_o00_s01?ie=UTF8&psc=1
• 1 x Anker multi-port power adapter. https://www.amazon.co.uk/gp/product/B00PK1IIJY/ref=ppx_yo_dt_b_asin_title_o00_s01?ie=UTF8&psc=1
• 4 x 0.3m short usb c cables. One for each Pi. (3pack https://www.amazon.co.uk/gp/product/B07W12JK3J/ref=ppx_yo_dt_b_asin_title_o00_s02?ie=UTF8&psc=1)
• 4 x 0.3m short ethernet cable. One for each Pi. (5 pack https://www.amazon.co.uk/gp/product/B01J8MDV5G/ref=ppx_yo_dt_b_asin_title_o00_s02?ie=UTF8&psc=1)
• 1 x switch 5 port or more. (I already had one, but you can use https://www.amazon.co.uk/TP-Link-TL-SF1005D-5-Port-Desktop-Ethernet/dp/B0766D8HZ3/ref=sr_1_3?dchild=1&keywords=5+port+desktop+switch&qid=1614965097&sr=8-3)
• 1 x network cable to connect to your Lan

Software

• Windows subsystem for Linux (wsl2)
• Linux distro setup in wsl2. I prefer ubuntu or pengwin
• Nmap or another tool to search for pi’s IP (sudo apt-get install nmap)
• Raspberry Pi imager (https://www.raspberrypi.org/software/)

Awesome, so your laptop and kit are ready to go. I won’t go through building the case, but make sure you have that done and all Pi’s are connected to the network.

Installing the OS

Pop an SD card into your computer, you may need a memory card reader for this.

Open the raspberry pi imager and click Choose OS

Then choose Ubuntu LTS 64 bit. It is under other general-purpose OS

Then choose your SD card. Please make sure it’s the right one as you will be wiping it and installing an OS over it)

Then click write.

Once written pop it in a Pi that you want to be the master. I have chosen the top of my rack. And power on the Pi.

After a while, the Pi will bot and get an IP address from your DHCP server. (home router). Open windows terminal and go to your Linux distro. Next type, changing the IP range to match your home network.

Find one called ubuntu.local that’s the IP you will use to ssh into the Pi.

In the same window type the following to ssh into the Pi. The username will be ubuntu by default and the default password is ubuntu. Type yes when promoted.

Password ubuntu

Change password and then reconnect using the same ssh command, you can press the up arrow.

Setting up the OS

First, you need to update the OS. Use the two commands below to do that. If you get a lock out/time out when running the second command, just reboot and try it again.

Now it is time to change the host Name. As you are setting up the control plane first, I am going to call it k8s-control01. For the worker nodes I would change the host name to something like k8s-worker01.

Then you need to edit the hosts file also.

Add 127.0.0.1 and hostname.

Save the changes and exit nano. Next perform a reboot for the changes to take.

Set IP

Ssh back into the node after a short while. You can use the up arrow to get to the previous command.

Now it’s time to set a static IP address. You could do this via your router/dhcp server, but for this guide, I am going to set it via the Pi.

Copy the below into the file and save and close nano.

Now use the next command to actually set the IP address.

In this file copy the below text but change the details to match your IP range.

Apply the changes by using the next command.

Now you need to edit the cloud-init file and delete all text apart from the comments.

Install containerd for the container runtime.

This is nice and simple to install the package just use the following command.

Now you just need to do a little bit of configuration.

Install Kubernetes

You need to add the kubernetes repo and GPG key. To do that just use the following.

Add GPG key

Add the Kubernetes repository

Next, you need to install the kubeadm, kublet and kubectl packages. Use the following to do that.

Now it is time to configure the memory cgroup. without this you will not be able to complete the setup. Just use the following command.

You will also need to disable swap. Again, this is nice and easy when you know the command.

Now it is time to give the Pi another reboot.

A bit more configuration

Ssh back into your Pi. It will be on the new IP address you set up earlier in this guide. You will need to type yes again, as it is a new IP address.

Now copy and paste the following.

Give containerd one last restart to make sure it has picked up the changes.

Start Kubernetes (Master only)

First, you need to pull the container images needed for Kubernetes. You will be using kubeadm for this.

Now it is time to create the cluster. Kubeadm will do all the work for us. All you need to do is specify the pod network cidr.

Once it has been setup you will get an output like below. Keep it safe.

You will see from the output that you need to run some commands. Go ahead and run them.

The output also says you need to deploy a pod network to the cluster. I am using Flannel. You can instal it with the following.

After a short while check everything is up and running by using the following.

You are now ready to join the worker nodes.

Worker Nodes

Now repeat this guide till you get to the Start Kubernetes (Master only) section. Once you have gotten there come back here.

Adding worker node to the Cluster

So, when you created the control node you were given a command to use to join the workers to the cluster. You just now need to run that, but make sure you put sudo in front.

Now ssh back to the Control node and run kubectl get nodes -o wide to see the new worker node.

Continue to add the rest of your worker nodes to the cluster. You now have your very own Kubernetes cluster running on Raspberry Pi’s! How cool is that.

Thanks for reading. Let me know if you have any questions or comments and why not let me know what you are planning to do with your cluster.