Reading Time: 3 minutes
Share:
Twitter
LinkedIn
Facebook
Reddit
Whatsapp
Follow by Email

For as long as I remember AKS has only officially supported two CNI’s, Kubenet and Azure CNI. That is until 2nd April 2022 when they announced the ability to create an AKS cluster with no CNI. This means you can deploy any CNI you would like. In this blog post I am going to show you how to create an AKS cluster with no CNI and then deploy cilium.

Information

Whilst AKS now officially supports Bring Your Own CNI you must remember that Microsoft will support your AKS cluster but not the CNI. You will need to support that yourself or get support from the CNI vendor.

Deploy the cluster

As with anything in preview for AKS you need to make sure you are using the az AKS preview cli extension. To do this use the following commands.

# Install the aks-preview extension
az extension add --name aks-preview
view raw aks no cni hosted with ❤ by GitHub


Update the extension to make sure you have the latest version installed

# Update the extension to make sure you have the latest version installed
az extension update --name aks-preview
view raw aks no cni hosted with ❤ by GitHub

Now you have the latest AKS preview extension it’s time to move on to creating the resource group. You can use the following command for that.

# Create resource group
az group create -l westeurope -n rg-aks-byo-cni
view raw aks no cni hosted with ❤ by GitHub

To create the cluster, we will use the az aks create command, just like you would normally. For the --network-plugin parameter we will supply none.

# Deploy cluster
az aks create -l westeurope -g rg-aks-byo-cni -n aks-byo-cni --generate-ssh-keys --network-plugin none
view raw aks no cni hosted with ❤ by GitHub

The cluster is now built. Let’s look at the state of the nodes. First you need to connect to your new cluster. Use the following command to do that.

az aks get-credentials -g rg-aks-byo-cni -n aks-byo-cni
view raw aks no cni hosted with ❤ by GitHub

Now, you can use kubectl get nodes for this. If you need to install kubectl you can use az aks install-cli.

kubectl get nodes
view raw aks no cni hosted with ❤ by GitHub

You will see they are showing as NotReady This is to be expected as there is no CNI to allow network communication. You can confirm this by using the following command.

kubectl get node -o custom-columns='NAME:.metadata.name,STATUS:.status.conditions[?(@.type=="Ready")].message'
view raw aks no cni hosted with ❤ by GitHub

You will notice the status message of the nodes says container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized. It’s now time to install a CNI.

Deploy a CNI

For this we will use cilium.

Before you install cilium to your cluster you need to install the cilium cli tools to your local machine. To do that use the following command.

# Install cilium cli
curl -L --remote-name-all https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz{,.sha256sum}
sha256sum --check cilium-linux-amd64.tar.gz.sha256sum
sudo tar xzvfC cilium-linux-amd64.tar.gz /usr/local/bin
rm cilium-linux-amd64.tar.gz{,.sha256sum}
view raw aks no cni hosted with ❤ by GitHub

Now that the cilium cli has been installed locally, you can use the following command to install cilium CNI into your AKS cluster.

# Install CNI
cilium install --azure-resource-group rg-aks-byo-cni --datapath-mode vxlan --ipam=cluster-pool --config cluster-pool-ipv4-cidr=10.240.0.0/16 --config cluster-pool-ipv4-mask-size=24
view raw aks no cni hosted with ❤ by GitHub

Once that finishes the installation you can confirm the status by using the following command.

cilium status --wait
view raw aks no cni hosted with ❤ by GitHub

To confirm everything is working you can run the cilium connectivity test. This can take some time, but worth doing.

You have now created a new AKS cluster and brought your own CNI, in this case cilium. The steps above are from my own testing and not following any official guid from cilium. Hopefully, they will release their docs soon and just remember this feature is still in preview at the time of writing this blog post, so stuff can change.

I hope you found this article helpful. Reach out to me and let me know what CNI you will be using.

Share:
Twitter
LinkedIn
Facebook
Reddit
Whatsapp
Follow by Email

Pixel Robots.

I’m Richard Hooper aka Pixel Robots. I started this blog in 2016 for a couple reasons. The first reason was basically just a place for me to store my step by step guides, troubleshooting guides and just plain ideas about being a sysadmin. The second reason was to share what I have learned and found out with other people like me. Hopefully, you can find something useful on the site.

2 Comments

Chris · July 16, 2022 at 8:22 pm

Thank`s for the article, nice read! What kind of terminal do you use, at least it looks good? 😉

    Pixel Robots. · August 26, 2022 at 4:22 pm

    I am using Windows terminal. From the Microsoft store.

Leave a Reply

Avatar placeholder

Your email address will not be published. Required fields are marked *