Buildkit on Windows Server 2019 Unable to Reuse Namespaces / Endpoints #5668
Comments
|
I did test buildkit v0.19.0-rc2 to make sure it wasn't already fixed and the same error occurs. |
|
ACK, will take a look and get back. |
|
I am facing the same issue on Server 2019, first RUN instruction succeeds, the second one fails always with: The requested operation for attach namespace failed.: unknown |
|
HI @nerddtvg -- Can I get this from you please, I'd like to confirm something about the Get-NetAdapter # list all of them, I'd like to see which one the HnsNetwork is connected to
$nat = Get-HnsNetwork | where {$_.Name -eq "nat" }And also, what do you get when you run this ctr run --rm --cni mcr.microsoft.com/windows/nanoserver:ltsc2019 cni-test curl -I example.comI'm working to solve this CNI setup issue once and for all. Seems like we have a couple of setup scripts around that are not all coherent... |
|
@profnandaa Below are the requested responses. I had to change the test image because I already had this one pulled. I am able to access the network fine, it's the second RUN command in a build that fails. |
|
@profnandaa - I also sent you a message on the Docker Community Slack that might help. |
|
From your ctr run --cni run, seems to me there no issue with your CNI settings. Fixed my CNI setup issues, I can repro this now. Let work on this on Monday. will respond to you on Slack and we can post back here the key findings for others. My dockerfile: FROM mcr.microsoft.com/windows/nanoserver:ltsc2019
RUN curl.exe -I example.com
RUN curl.exe -I bing.com
RUN curl.exe example.comBuild log: |
|
Spam |
|
cross-linking with an old issue here - containerd/containerd#5729 |
|
Another one - microsoft/hcsshim#1822 |
There is an issue with `HnsEndpoints` on WS2019, after first time use, they can't be re-attached. See moby#5668. HCS CreateComputeSystem fails with: ``` The requested operation for attach namespace failed. ``` As a work-around, have an option for PoolSize=-1 to turn off the namespace pooling functionality and have each namespace only be used once and discarded. To enable this, you run buildkit with the additional flag: `--containerd-cni-pool-size=-1`. Fixes moby#5668 Signed-off-by: Anthony Nandaa <[email protected]>
When running on Server 2019, buildkit is unable to reuse a namespace for the second RUN command in a Dockerfile (or subsequent runs). The namespace state is
1and endpoint state is4(Detatched).An identical buildkit, containerd, cni configuration in Server 2022 works without issue.
I have tried the Windows CNI plugin versions 0.3.0 and 0.3.1.
This is a similar error to: #4960 However, the subnet and NAT configurations are correct and match the existing
natHnsNetwork.Sample Dockerfile:
buildkitd logs:
Windows Hyper-V-Compute event log:
HNS Output:
If I remove the HnsEndpoint (
Get-HnsEndpoint | Where-Object {$_.State -eq 4} | Remove-HnsEndpoint), the container will run one step and fail on the next RUN trying to re-use the endpoint. The container won't be able to connect out to the Internet though, ping and DNS lookups fail.Since the CNI pool size cannot be less than one (setting
0provisions a single namespace) and the cache time cannot be adjusted (const 5 minutes), I can find no workaround to prevent the namespace reuse on 2019. If the namespace was cleaned up each time, this would work.I'm unsure the root cause of the bug, if it is a 2019 HCS issue or not.
The text was updated successfully, but these errors were encountered: