ci: cross-compile Go builds instead of QEMU emulation#382
Conversation
Add --platform=$BUILDPLATFORM to the builder FROM in all three
Dockerfiles and drop the -a flag from go build. This pins the
builder stage to the host arch so Go cross-compiles to $TARGETARCH
instead of running under QEMU.
Effect on the release pipeline (kagenti-operator image, latest run):
the linux/arm64 builder step takes 22m15s under emulation vs ~155s
amd64 native — a ~9x slowdown. Cross-compiling is bit-for-bit
identical (CGO_ENABLED=0, no per-arch dependencies) and the existing
GOARCH=${TARGETARCH} plumbing already wires up cross-compile
correctly; only the runtime of the builder stage was wrong.
The -a flag forces rebuild of every package including stdlib,
defeating the build cache. The Makefile already drops it for local
builds; the Dockerfiles were the outliers. Modern kubebuilder v4
scaffolds (this project's PROJECT layout) ship with both fixes by
default — this brings the Dockerfiles in line with the upstream
template.
Expected total CI time for the release workflow: ~25min -> ~5min on
cold cache.
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Hai Huang <huang195@gmail.com>
cwiklik
left a comment
There was a problem hiding this comment.
Clean, consistent change across all three Dockerfiles. --platform=$BUILDPLATFORM pins the builder to host arch for native cross-compilation instead of QEMU emulation, and dropping -a restores build cache. The existing GOARCH=${TARGETARCH} plumbing was already correct — only the builder runtime was wrong.
One nit inline about a stale comment (non-blocking).
| @@ -21,7 +26,7 @@ COPY internal/ internal/ | |||
| # was called. For example, if we call make docker-build in a local env which has the Apple Silicon M1 SO | |||
| # the docker BUILDPLATFORM arg will be linux/arm64 when for Apple x86 it will be linux/amd64. Therefore, | |||
| # by leaving it empty we can ensure that the container and binary shipped on it will have the same platform. | |||
There was a problem hiding this comment.
nit: This comment ("by leaving it empty we can ensure that the container and binary shipped on it will have the same platform") is now stale — it described the old approach of not pinning the builder platform. With --platform=$BUILDPLATFORM on line 7, the builder IS explicitly pinned to host arch and Go cross-compiles via GOARCH=${TARGETARCH}. The new comment at the top of the file explains the correct rationale. Consider removing or updating these three lines to avoid contradicting the new approach.
Summary
--platform=$BUILDPLATFORMto the builderFROMin all three Dockerfiles (operator, agentcard-signer, test-tls-agent) and drop-afromgo build. Pins the builder stage to the host arch so Go cross-compiles to$TARGETARCHinstead of running under QEMU.linux/arm64builder step at 1335s (22m15s) vslinux/amd64at 155s native — a ~9× slowdown caused entirely by emulating the arm64 CPU throughtonistiigi/binfmt. CGO is disabled and there are no per-arch dependencies, so cross-compiling produces a bit-for-bit identical binary.GOARCH=${TARGETARCH}plumbing already wires up cross-compile correctly; only the runtime of the builder stage was wrong.-aforces rebuild of every package including stdlib, defeating the build cache. TheMakefile'sbuildtarget already omits it; the Dockerfiles were the outliers. Modern kubebuilder v4 scaffolds (this project's PROJECT layout) ship with both fixes by default.Expected impact
releaseworkflow (per-image, parallel matrix)linux/arm64builder step (Go compile)Test plan
docker buildx imagetools inspect ghcr.io/kagenti/kagenti-operator/kagenti-operator:latestafter merge reports bothlinux/amd64andlinux/arm64digests (parity with current behavior).e2e-helm-installjob (Kind on amd64) still passes — pulls the same multi-arch manifest, only the build path for amd64 changed (and amd64 was already native, so it should be a no-op for that arch).sha256sumthemanagerbinary out of the new image vs. a prior release; bytes should match for the same toolchain version (Go's content-addressable cache makes-airrelevant for output bytes).Notes
kagenti-webhookDockerfile in akram-kagenti-extensions if/when that repo is upstreamed.Assisted-By: Claude Code