Azure proxy agent blocks k8s workload

[ljqqqqq]

We noticed that the latest Canonical Ubuntu images (22.04 & 24.04, x86_64 & arm64, version 24.04.202512100) now ship with azure-proxy-agent installed and enabled by default. In our Kubernetes environments, this appears to interfere with normal pod networking, specifically affecting CoreDNS and eBPF-based CNI (Cilium).

It prevents coredns pod from accessing IMDS endpoint so the pods fail to become Ready

We also observed that azure-proxy-agent directly attaches an eBPF program to the root cgroup (/sys/fs/cgroup). This conflicts with Cilium, which expects to manage cgroup eBPF hooks itself.

This is reproducible on a minimal Kubernetes cluster with no additional components installed.

We understand that this behavior may be by design for IMDS protection. However, the current default behavior appears intrusive for Kubernetes workloads. From the documentation, it seems that azure-proxy-agent is part of the implementation of MSP (Metadata Security Protocol). Questions:

1. Is azure-proxy-agent expected to be active even when the MSP extension is not installed?
   In our testing, not installing the extension does not stop the agent from intercepting traffic.
2. When the MSP extension is installed and IMDS mode is set to disabled, what is the expected behavior?
   On an x86 Linux VM, even after disabling IMDS via the extension and restarting core-dns by rollout restart, CoreDNS traffic is still blocked.
3. Is there a supported way to exempt Kubernetes / CNI-managed pods (e.g. CoreDNS, Cilium) from IMDS interception?
   We are aware of allowlist-based configurations, but they appear too fine-grained for typical customer Kubernetes workloads.

[ZhidongPeng]

1. when azure-proxy-agent baked into the OS/base image, this service is active and running.
2. when MSP IMDS set to disabled, GPA /w eBPF should NOT intercepting the IMDS traffic. It hit our timing bug when redirect eBPF module finished later, the IMDS mode will not remove the intercepting from eBPF map. I am having the fix now.
3. Canonical aware this impact to k8s too, we decided to schedule the existing images containing the agent for deprecation for quick mitigation. We will re-incorporate the agent back into our images once the fix has been incorporated into the azure-proxy-agent package.

[ZhidongPeng]

Here is the fix: #298

[bareckidarek]

Hi. We noticed a similar problem with the version 24.04.202512100.
@ZhidongPeng, thanks for the fix, but I suppose it will not be enough when we want to enable MSP for our VMs

Let me explain our case

We need to connect with IMDS from a Docker container running on the machine hosted on Azure.

The proxy works well on the host:

• proxy listening on 127:0:0:1:3080
• eBPF program, attaches to cgroup/connect4, watches the destination address, and replaces it, based on eBPF map policy_map (in our case 169.254.169.254:80 -> 127.0.0.1:3080)
• finally, the request reaches the proxy, and the proxy does the job

It doesn't work for Docker containers with network=bridge style, because there is no listening proxy on the "loopback" network interface inside the container.

# docker run -it alpine/curl curl -H "Metadata: true" "http://169.254.169.254/metadata/instance?api-version=2021-02-01" -v
*   Trying 169.254.169.254:80...
* connect to 169.254.169.254 port 80 from 127.0.0.1 port 35308 failed: Connection refused
* Failed to connect to 169.254.169.254 port 80 after 0 ms: Could not connect to server
* closing connection #0
curl: (7) Failed to connect to 169.254.169.254 port 80 after 0 ms: Could not connect to server

We could use network=host, but it is not an option, ofc.

Any ideas on how to work with it in the future?
Thanks, Darek.

[ZhidongPeng]

Here is the fix: #298

extend the fix with PR (#301) to only enable the host endpoints intercepting when provision finished.

[ZhidongPeng]

Hi. We noticed a similar problem with the version 24.04.202512100. @ZhidongPeng, thanks for the fix, but I suppose it will not be enough when we want to enable MSP for our VMs

Let me explain our case

We need to connect with IMDS from a Docker container running on the machine hosted on Azure.

The proxy works well on the host:

• proxy listening on 127:0:0:1:3080
• eBPF program, attaches to cgroup/connect4, watches the destination address, and replaces it, based on eBPF map policy_map (in our case 169.254.169.254:80 -> 127.0.0.1:3080)
• finally, the request reaches the proxy, and the proxy does the job

It doesn't work for Docker containers with network=bridge style, because there is no listening proxy on the "loopback" network interface inside the container.

# docker run -it alpine/curl curl -H "Metadata: true" "http://169.254.169.254/metadata/instance?api-version=2021-02-01" -v
*   Trying 169.254.169.254:80...
* connect to 169.254.169.254 port 80 from 127.0.0.1 port 35308 failed: Connection refused
* Failed to connect to 169.254.169.254 port 80 after 0 ms: Could not connect to server
* closing connection #0
curl: (7) Failed to connect to 169.254.169.254 port 80 after 0 ms: Could not connect to server

We could use network=host, but it is not an option, ofc.

Any ideas on how to work with it in the future? Thanks, Darek.

You are correct, MSP feature does not support container-based scenarios yet on purpose, nor nested VM either.

[bareckidarek]

Hi @ZhidongPeng. Thanks for the confirmation.

Regarding your answer, it would be helpful to have that information as a warning on the site about MSP (https://learn.microsoft.com/en-us/azure/virtual-machines/metadata-security-protocol/overview) or, even better, in an area where we can enable it. 90% of our VMs or VMSSs, hosted on Azure Cloud, are established to work in container-based scenarios, and it is a common case, I think.

At this moment, even when I want to create a new VM on Azure, there is no option to select MSP as written in the article https://learn.microsoft.com/en-us/azure/virtual-machines/metadata-security-protocol/other-examples/portal#deploy-a-vm-with-msp. Is this fully disabled now?