FQDN Filtering in AKS: Enhancing Security with Advanced Container Networking Services (ACNS)

Reading Time: 5 minutes

In today’s cloud-native world, securing your Kubernetes clusters is more critical than ever. Traditional IP-based security measures often fall short in dynamic environments where IP addresses change frequently. This is where Fully Qualified Domain Name (FQDN) filtering comes into play, offering a more efficient and scalable way to manage network policies. In this blog post, we’ll explore how Azure Kubernetes Service (AKS) leverages FQDN filtering through Advanced Container Networking Services (ACNS) to enhance your cluster’s security.

What is FQDN Filtering?

FQDN filtering allows you to define network policies based on domain names rather than IP addresses. This approach simplifies policy management by eliminating the need to update policies whenever IP addresses change, a common occurrence in Kubernetes clusters where pods are ephemeral.

Why FQDN Filtering Matters

• Scalable Policy Management: Manage network policies more efficiently without constantly updating IP addresses.
• Enhanced Security Compliance: Implement zero-trust security models by restricting traffic to trusted domains.
• Resilient Policy Enforcement: Ensure continuous DNS resolution and policy enforcement even during component upgrades or failures.

Advanced Container Networking Services (ACNS)

ACNS is an add-on for AKS that provides advanced networking features, including observability and enhanced security measures like FQDN filtering. If you’re interested in learning more about ACNS’s observability features, check out my previous post: Advanced Network Observability: Supercharging Container Network Observability in Azure Kubernetes Service (AKS).

How Does FQDN Filtering Work in AKS?

When FQDN filtering is enabled, DNS requests from pods are intercepted and evaluated against defined network policies. Here’s a high-level overview:

• DNS Request Redirection: The Cilium Agent marks DNS request packets from pods enforcing FQDN policies, redirecting them to the ACNS DNS Proxy.
• Policy Evaluation: The DNS Proxy checks if the DNS request matches the allowed domains specified in the network policy.
• DNS Resolution and Policy Update: If permitted, the DNS Proxy forwards the request to the DNS server. Upon receiving the response, it updates the Cilium Agent with the FQDN-to-IP mappings.
• Network Policy Enforcement: The Cilium Agent updates the network policy, ensuring that pods can communicate only with the specified domains.

Setting Up FQDN Filtering in AKS

Before we dive into commands, let’s ensure you meet the prerequisites.

Prerequisites

• Azure CLI: Ensure you have Azure CLI version 2.56.0 or later installed.
• AKS Cluster with Cilium: FQDN filtering requires AKS clusters using Azure CNI powered by Cilium and Kubernetes version 1.29 or greater.

Enable the AdvancedNetworkingPreview Feature Flag

First, you’ll need to install or update the aks-preview Azure CLI extension. Run the following commands to install the required extensions.

Next, register the feature flag for Advanced Networking Preview using the command below:

After a few moments, check the registration status to ensure the feature is ready:

Once the registration is complete, refresh the resource provider with the following command:

Create an AKS Cluster with ACNS and FQDN Filtering

Now that you’ve enabled the necessary features, you can create a new AKS cluster with ACNS enabled (both Observability and FQDN filtering). Use the following command to create the cluster, just dont forget a resource group and change the cluster name and resource group to match your needs:

This will deploy a new AKS cluster with all ACNS features, including FQDN filtering and network observability.

Create an AKS Cluster with FQDN Filtering (Without ACNS Observability)

If you’re only interested in enabling FQDN filtering without activating the broader ACNS features, such as network observability, you can use the following command. The --enable-fqdn-policy flag focuses solely on domain-based filtering, reducing overhead and cost by avoiding ACNS observability.

This command creates a new AKS cluster with FQDN filtering enabled, allowing you to manage network policies based on domain names. It’s useful if you want simplified policy management but don’t require ACNS observability.

Enabling FQDN Filtering on an Existing Cluster

If you have an existing AKS cluster, you can still enable FQDN filtering and observability by updating the cluster. Ensure your cluster uses the Cilium data plane, then run the following command:

This command will update your cluster to support ACNS, including FQDN filtering.

Enabling FQDN Filtering on an Existing Cluster (Without ACNS Observability)

If you already have an existing AKS cluster and wish to add FQDN filtering without enabling all ACNS features, you can update your cluster using the following command. This is ideal if you’re looking for domain-based security without additional monitoring capabilities.

This command updates your current AKS cluster to enable FQDN filtering. With this, you can enforce network policies that are easier to manage while keeping your cluster lightweight by avoiding the full ACNS suite.

Implementing FQDN-Based Network Policies

Once you’ve enabled FQDN filtering, you can start creating network policies that use domain names to manage pod communication. Here’s an example of a CiliumNetworkPolicy that allows egress traffic only to the domain pixelrobots.co.uk:

In this policy:

• All traffic is blocked except for DNS traffic to the Kubernetes DNS server and traffic to pixelrobots.co.uk.
• Pods with the label app=my-app in the namespace my-namespace are allowed to communicate with the domain pixelrobots.co.uk.

Note: Make sure you allow traffic to kube-dns, as it is essential for resolving FQDNs in the cluster.

Benefits of FQDN Filtering

By implementing FQDN filtering in your AKS clusters, you gain several key advantages:

• Simplified Management: There’s no need to constantly track and update changing IP addresses of external services.
• Improved Security: FQDN filtering allows you to enforce a zero-trust security model by controlling which domain names pods can communicate with.
• High Availability: The ACNS DNS Proxy ensures that DNS resolution continues even during upgrades or failures of the Cilium agent, keeping your security policies intact.

Considerations

• Cost: Keep in mind that ACNS incurs additional costs, approximately $18 per node per month ($0.025 per node per hour).
• Preview Feature: As of this writing, ACNS is in preview. Be cautious when deploying this feature in production environments.
• Cilium Data Plane: FQDN filtering requires the Cilium network data plane, which may require recreating existing clusters if they aren’t using Cilium.

Conclusion

FQDN filtering in AKS offers a powerful tool for managing network security in a dynamic, cloud-native environment. By moving from IP-based policies to domain-based policies, you can simplify your security operations, enhance compliance with zero-trust models, and maintain policy enforcement even during system upgrades.

If you’re already using AKS and looking for ways to improve security, enabling Advanced Container Networking Services with FQDN filtering is a great place to start. Just be sure to test it thoroughly in a non-production environment before rolling it out cluster-wide.

References

• Microsoft Docs: FQDN Filtering for Enhanced Security with ACNS
• Advanced Network Observability in AKS

---

Thank you for reading! Feel free to leave a comment or reach out if you have any questions or insights to share.