Insulate container operations from Docker Desktop under WSL - #66
Insulate container operations from Docker Desktop under WSL#66keithharvey wants to merge 3 commits into
Conversation
Docker Desktop leaks client-side state into a WSL distro's home and PATH, and BAR-Devtools only ever hardened the daemon side. Two failures follow. `compose build teiserver` dies before it pulls anything: Docker Desktop writes `"credsStore": "desktop.exe"` into ~/.docker/config.json, and the Go docker-compose provider podman delegates to reads that file regardless of which daemon it's talking to. The helper is a Windows binary that isn't on PATH. Point DOCKER_CONFIG at our own runtime dir instead. `engine::build` silently leaves podman: docker-build-v2/build.sh resolves its own runtime and prefers docker whenever one is on PATH, so the build lands in the docker-desktop distro's VM with the source tree, ccache, and build output all bind-mounted across a distro boundary. Pin CONTAINER_RUNTIME=podman to match every other container operation here. Doctor grows a check for both signals, since anyone invoking docker-compose or build.sh directly still hits them.
`just lua::check` was the one recipe still shelling out to `docker compose`. On a podman-only host it fails with command-not-found; under WSL with Docker Desktop it silently routes into the Docker Desktop VM.
Match the DEVTOOLS_DISTROBOX pattern -- default in code, .env or the environment overrides. The hard assignment blocked anyone whose compose pulls from a private registry from keeping their own docker login. The default stays in code rather than moving to .env: `set dotenv-load` reads .env when just starts, so a key written at step 0 isn't in the environment for step 4 of that same run. It would be unset for the first setup::init and correct on every run after.
094a352 to
a1d0559
Compare
| : "${DEVTOOLS_SYNC_DISTROBOX:=bar-sync}" | ||
| export DEVTOOLS_SYNC_DISTROBOX | ||
|
|
||
| # Our own Docker client config; Docker Desktop's ~/.docker/config.json names a |
There was a problem hiding this comment.
Debated putting this in .env, but that file holds user selections and this is derived rather than chosen, so it doesn't really fit.
There's also a timing problem with .env: set dotenv-load reads it when just starts, so a key written at step 0 isn't in the environment for step 4 of that same run. It'd be unset for the first setup::init and correct on every run after — a rough way to fail, since it'd only ever bite a new contributor.
That said, the : below does allow users to override it in their own .env file and it will be honored.
This does add a minor risk if someone has DOCKER_CONFIG on their shell, but that's kind of expected/I'm fine with it — Docker Desktop itself doesn't set that var, it writes ~/.docker/config.json.
There was a problem hiding this comment.
Seems fine given the use case, I have never heard of someone setting $DOCKER_CONFIG on their environment. Possibly it would be good to emit an info message of $DOCKER_CONFIG envvar set so someone with eagle-eyes can spot it in the log.
|
The LLM-ese in the PR description around how the proposed fix solves the problem is a bit abstract and I'm struggling to grok it. But I'm getting there slowly... |
NortySpock
left a comment
There was a problem hiding this comment.
Left a question about fixing build.sh directly.
I guess I have a larger set of questions around 'Are we supporting podman-first, or docker-first? Which of the two is expected first, and which is the fallback? Do we need to make "podman vs docker" some sort of environment variable configured by the user and flex between the two?'
| ;; | ||
| esac | ||
| bash "$build_script" {{args}} | ||
| # build.sh picks docker over podman when both are on PATH -- under Docker |
There was a problem hiding this comment.
From this comment.... should you just fix build.sh to pick podman first and fallback to docker if podman is unavailable?
(I am aware from other discussion in this PR that podman still uses docker runtime in some cases)
| : "${DEVTOOLS_SYNC_DISTROBOX:=bar-sync}" | ||
| export DEVTOOLS_SYNC_DISTROBOX | ||
|
|
||
| # Our own Docker client config; Docker Desktop's ~/.docker/config.json names a |
There was a problem hiding this comment.
Seems fine given the use case, I have never heard of someone setting $DOCKER_CONFIG on their environment. Possibly it would be good to emit an info message of $DOCKER_CONFIG envvar set so someone with eagle-eyes can spot it in the log.
Two separate
setup::initfailures reported on Ubuntu 24 under WSL2 with Docker Desktop installed on the Windows side. Same root cause: Docker Desktop leaks client-side state into the distro's home and PATH, and we only ever hardened the daemon side.1.
compose build teiserverfails before pulling anythingThe daemon is podman, but the client isn't.
podman composedelegates to the Godocker-composebinary we install at/usr/local/bin/docker-compose(setup.sh:250-261), and Docker clients read~/.docker/config.jsonfor registry auth no matter which daemon they're pointed at. Docker Desktop writes"credsStore": "desktop.exe"there. That helper is a Windows binary, reachable only through WSL PATH interop with Docker Desktop integration enabled for the distro.Fixed by pointing
DOCKER_CONFIGat.devtools/dockerso we carry our own client config. Every image the stack uses is public, so no credential helper is needed.2.
engine::buildsilently leaves podmandocker-build-v2/build.shresolves its own runtime and prefers docker whenever one is on PATH:Nothing in this repo set
CONTAINER_RUNTIME, so this is the one operation that jumps out of podman into the Docker Desktop VM. The source tree, ccache, and build output are then all bind-mounted across a distro boundary, which turns an engine build into an all-day affair.Fixed by pinning
CONTAINER_RUNTIME=podmanto match every other container operation here.3.
just lua::checkwas still ondocker composejust/lua.just:8definedCOMPOSE := "docker compose -f ..."while every other module usespodman compose. On a podman-only host that recipe fails with command-not-found; under WSL with Docker Desktop it silently routes into the Docker Desktop VM. Switched topodman compose.4. Doctor check
Both pins only cover our own recipes. Anyone invoking
docker-composeordocker-build-v2/build.shdirectly still hits these, sodoctornow names the condition. Detection reads thedockersymlink target rather than runningdocker version, so a wedged Docker Desktop can't hang a read-only diagnostic.Verification
The credsStore failure is purely client-side, so it reproduces on Linux with a synthetic
HOME:just doctorpasses clean, and the new check is correctly silent off-WSL.The engine-build half is reasoned from the resolver source rather than reproduced — I don't have a Docker Desktop box. Worth a second pair of eyes from someone who does.
Not fixed here
That resolver has a real bug:
docker version's stdout is piped intogrep, so when the daemon is unreachable the pipeline still exits non-zero,!flips it true, and docker gets selected anyway — podman is never tried. Our pin sidesteps it, but it'll keep biting anyone building Recoil directly. Belongs upstream in RecoilEngine.LLM Disclosure
Opus 5