Ensuring High Priority for Azure Kubernetes Service (AKS) Daemon Sets: A Guide to Smooth Cluster Startup

Reading Time: 2 minutes

Introduction

One crucial aspect of Azure Kubernetes Service (AKS) is the ability to ensure that critical components, such as Daemon Sets, start reliably when the cluster starts in the morning after being off over night to save money. In this blog post, we will explore how you can guarantee that your Daemon Sets have priority on a node, ensuring they always start smoothly, even after a night of downtime.

Understanding Daemon Sets in AKS

Before we delve into prioritizing Daemon Sets, let’s quickly recap what they are and why they are important in an AKS cluster. Daemon Sets are a type of workload in Kubernetes that ensure a specific pod is deployed and running on each node within a cluster. They are ideal for running cluster-level tasks, logging agents, monitoring tools, or any other application that should be deployed on every node.

The Challenge: Reliable Startup After Downtime

When an AKS cluster experiences downtime, such as during planned maintenance or unexpected failures, it’s crucial to have mechanisms in place to ensure the smooth startup of Daemon Sets upon cluster recovery. By default, Kubernetes attempts to schedule Daemon Sets on all available nodes, but this does not guarantee priority or immediate startup, especially in scenarios with limited resources or nodes with higher priority workloads.

Solution: Assigning Priority to Daemon Sets

To ensure that your Daemon Sets receive the highest priority during cluster startup, you can take advantage of Kubernetes’ scheduling features. Specifically, we’ll focus on the Pod Priority feature.

Pod Priority is a Kubernetes feature that assigns priority levels to pods. Pods with higher priority are scheduled before lower priority pods. By assigning a higher priority to your Daemon Sets, you can ensure they are scheduled first when the cluster starts up, even if resources are limited.

Configuration Steps

Let’s walk through the steps required to configure priority for Daemon Sets in AKS:

Step 1: Modify the Daemon Set YAML

In your Daemon Set YAML file, add the following spec section to define the priorityClassName:

Step 2: Create the Priority Class

Create a Priority Class resource with a higher priority value. This can be achieved by applying the following YAML:

Step 3: Apply the Changes

Apply the modified Daemon Set YAML and the Priority Class YAML using the kubectl apply command. This will update the Daemon Set and create the high-priority class.

Conclusion

Ensuring the smooth startup of Daemon Sets in AKS is crucial for maintaining the stability and availability of your applications. By configuring priority using the Pod Priority feature, you can guarantee that your Daemon Sets have the highest precedence during cluster recovery after a period of downtime. This approach will help you eliminate potential delays or issues associated with the startup of critical components and allow your applications to resume normal operation seamlessly.

Remember, while prioritizing Daemon Sets is essential, it’s also vital to carefully manage your resources and node capacity to ensure optimal performance and avoid resource constraints