For as long as I remember AKS has only officially supported two CNI’s, Kubenet and Azure CNI. That is until 2nd April 2022 when they announced the ability to create an AKS cluster with no CNI. This means you can deploy any CNI you would like. In this blog post I am going to show you how to create an AKS cluster with no CNI and then deploy cilium.
Deploy the cluster
As with anything in preview for AKS you need to make sure you are using the az AKS preview cli extension. To do this use the following commands.
| # Install the aks-preview extension | |
| az extension add --name aks-preview |
Update the extension to make sure you have the latest version installed
| # Update the extension to make sure you have the latest version installed | |
| az extension update --name aks-preview |
Now you have the latest AKS preview extension it’s time to move on to creating the resource group. You can use the following command for that.
| # Create resource group | |
| az group create -l westeurope -n rg-aks-byo-cni |
To create the cluster, we will use the az aks create command, just like you would normally. For the --network-plugin parameter we will supply none.
| # Deploy cluster | |
| az aks create -l westeurope -g rg-aks-byo-cni -n aks-byo-cni --generate-ssh-keys --network-plugin none |
The cluster is now built. Let’s look at the state of the nodes. First you need to connect to your new cluster. Use the following command to do that.
| az aks get-credentials -g rg-aks-byo-cni -n aks-byo-cni |
Now, you can use kubectl get nodes for this. If you need to install kubectl you can use az aks install-cli.
| kubectl get nodes |
You will see they are showing as NotReady This is to be expected as there is no CNI to allow network communication. You can confirm this by using the following command.
| kubectl get node -o custom-columns='NAME:.metadata.name,STATUS:.status.conditions[?(@.type=="Ready")].message' |
You will notice the status message of the nodes says container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized. It’s now time to install a CNI.
Deploy a CNI
For this we will use cilium.
Before you install cilium to your cluster you need to install the cilium cli tools to your local machine. To do that use the following command.
| # Install cilium cli | |
| curl -L --remote-name-all https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz{,.sha256sum} | |
| sha256sum --check cilium-linux-amd64.tar.gz.sha256sum | |
| sudo tar xzvfC cilium-linux-amd64.tar.gz /usr/local/bin | |
| rm cilium-linux-amd64.tar.gz{,.sha256sum} |
Now that the cilium cli has been installed locally, you can use the following command to install cilium CNI into your AKS cluster.
| # Install CNI | |
| cilium install --azure-resource-group rg-aks-byo-cni --datapath-mode vxlan --ipam=cluster-pool --config cluster-pool-ipv4-cidr=10.240.0.0/16 --config cluster-pool-ipv4-mask-size=24 |
Once that finishes the installation you can confirm the status by using the following command.
| cilium status --wait |
To confirm everything is working you can run the cilium connectivity test. This can take some time, but worth doing.
You have now created a new AKS cluster and brought your own CNI, in this case cilium. The steps above are from my own testing and not following any official guid from cilium. Hopefully, they will release their docs soon and just remember this feature is still in preview at the time of writing this blog post, so stuff can change.
I hope you found this article helpful. Reach out to me and let me know what CNI you will be using.
2 Comments
Chris · July 16, 2022 at 8:22 pm
Thank`s for the article, nice read! What kind of terminal do you use, at least it looks good? 😉
Pixel Robots. · August 26, 2022 at 4:22 pm
I am using Windows terminal. From the Microsoft store.