Microsoft has just shipped the Inspektor Gadget cluster extension for Azure Kubernetes Service (AKS), and it is worth a look if you have ever been stuck trying to figure out why a pod is making strange DNS calls, why a process is misbehaving, or where that mysterious TCP connection is going. It is currently in preview, but the extension model makes it far more usable than the previous manual Helm-based install.
What is Inspektor Gadget?
Inspektor Gadget is an open source, CNCF sandbox project that uses eBPF (extended Berkeley Packet Filter) to observe what is happening inside your Kubernetes workloads at the kernel level, without modifying your applications or restarting anything. You run purpose-built tools called gadgets, each targeting a specific type of system activity. For example, DNS queries, TCP connections, file opens, process execution, capability checks. The kernel captures the raw events, and Inspektor Gadget enriches them with Kubernetes metadata so you see a pod name and namespace rather than a process ID and an IP address.
Why this matters
Inspektor Gadget has been around in the Kubernetes community for a while, but wiring it up to your cluster took effort. You had to manage the Helm chart yourself, handle upgrades manually, and pull images from wherever the project hosted them.
The new AKS extension changes that. You install it through the Azure resource management plane using az k8s-extension, and from that point Azure handles the lifecycle. Images are pulled from Microsoft Artifact Registry (MCR), upgrades are managed automatically (minor version by default), and the extension fits into the same ARM-driven workflow as other cluster extensions. That is a meaningful shift if you manage multiple clusters and care about consistency.
Underneath, Inspektor Gadget runs as a DaemonSet in the gadget namespace, one pod per schedulable node. Each pod loads eBPF programs into the kernel and correlates the raw kernel events with Kubernetes metadata, so instead of seeing a process ID and an IP address, you see the pod name, namespace, and node. That context is the thing that makes eBPF-based observability actually useful day-to-day.
How to install and start running gadgets
You need the k8s-extension Azure CLI extension and an existing AKS cluster with kubectl access. If you previously deployed Inspektor Gadget manually, run kubectl gadget undeploy or helm uninstall to clean it up first.
Install the extension with a single command:
|
1 2 3 4 5 6 7 |
az k8s-extension create \ --cluster-type managedClusters \ --cluster-name myAKSCluster \ --resource-group myResourceGroup \ --name inspektor-gadget \ --extension-type microsoft.inspektorgadget \ --release-train preview |
Once that completes, verify the DaemonSet is up and running:
|
1 2 3 4 5 6 7 8 |
az k8s-extension show \ --cluster-type managedClusters \ --cluster-name myAKSCluster \ --resource-group myResourceGroup \ --name inspektor-gadget kubectl get ds gadget -n gadget kubectl get pods -n gadget -l k8s-app=gadget |
All pods should show Running and Ready. If they do not, check the daemon logs for anything referencing skip, unsupported, or error. A mismatch between the hook mode and the node’s container runtime is the most common cause on non-default configurations.
Installing the kubectl plugin
To actually run gadgets, you need the kubectl gadget plugin. Install it via Krew:
|
1 2 |
kubectl krew install gadget kubectl gadget version |
If you are in Azure Cloud Shell, the plugin is already there.
Running your first gadgets
Once the plugin is installed, you run gadgets against a namespace, a pod, or the whole cluster. The pattern is consistent across all gadget types, which makes it easy to pick up.
Trace DNS queries from the default namespace for 30 seconds, including which pod originated each query:
|
1 2 3 4 |
kubectl gadget run trace_dns \ --namespace default \ --timeout 30 \ --fields k8s.node,k8s.podName,id,qr,name,rcode,nameserver |
An rcode of NoError means the lookup succeeded. Anything else and you have a name-resolution problem, and you know exactly which pod triggered it.
Trace outbound TCP connections across the whole cluster:
|
1 2 3 |
kubectl gadget run trace_tcp \ --all-namespaces \ --timeout 30 |
Trace process execution inside pods, which is useful when something is spawning child processes you did not expect:
|
1 2 3 4 |
kubectl gadget run trace_exec \ --namespace default \ --timeout 30 \ --fields k8s.node,k8s.podName,comm,pid,args |
I ran the DNS trace against a cluster where an application had intermittent name-resolution failures that were difficult to reproduce. Being able to see the exact rcode per pod in real time, with zero instrumentation changes, was genuinely quicker than the usual approach of adding logging, redeploying, and waiting for the issue to surface again.
One thing worth noting is that configuration changes to the extension are not picked up live. After any az k8s-extension update, you need to restart the DaemonSet manually:
|
1 2 |
kubectl rollout restart daemonset/gadget -n gadget kubectl rollout status daemonset/gadget -n gadget --timeout=120s |
If an update fails and a broken setting persists, run az k8s-extension update again and explicitly revert the problematic setting. The update command is additive, so failed settings do not get automatically rolled back.
The hard Kubernetes problems are usually in the plumbing
The hours-long debugging sessions on Kubernetes are rarely caused by something obvious in your application logs. They are caused by a DNS lookup that fails one in fifty times, a sidecar quietly dropping connections, a process that spawns something it should not. Logs do not show you that. Metrics do not show you that. And up until now, getting that level of visibility meant instrumenting the app, redeploying it, and hoping the issue reproduced.
Inspektor Gadget solves a different problem to your APM (Application Performance Monitoring) tool or your log aggregator. It sits at the kernel and watches what is actually happening, not what your application decided to record. The AKS extension just removes the last remaining excuse not to have it available.
Install it on a non-production cluster now. Run trace_dns for thirty seconds and see what comes back. You will almost certainly see something you did not expect. The full gadget catalog is at learn.microsoft.com/azure/aks/inspektor-gadget-catalog.
0 Comments