[Carry] Dockerfile: cache mount for flashinfer cubin downloads - #18
Draft
terafin wants to merge 1 commit into
Draft
[Carry] Dockerfile: cache mount for flashinfer cubin downloads#18terafin wants to merge 1 commit into
terafin wants to merge 1 commit into
Conversation
The vllm-base stage's `flashinfer download-cubin` step takes ~9m of
network-bound cubin fetch with no cache, on every build. Add a
BuildKit cache mount so subsequent builds reuse the downloaded cubins:
RUN --mount=type=cache,target=/root/.cache/flashinfer,sharing=locked \
flashinfer show-config && flashinfer download-cubin
`sharing=locked` because cubin downloads are not safe to do
concurrently — concurrent downloaders racing the same cache target
can corrupt partial files.
First cold build still pays the full 9m. Subsequent builds (rebuilds
on the same builder, with the cache mount preserved) skip the
download — typical savings ~7m wall-time off vllm-base.
Intarweb-only carry (fork-only — not filed upstream). Dockerfile-only
diff; carry-drift guard safe.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
intarweb-sync-bot
Bot
force-pushed
the
intarweb-dev
branch
29 times, most recently
from
June 14, 2026 10:09
cbeb0e5 to
5bcea71
Compare
intarweb-sync-bot
Bot
force-pushed
the
intarweb-dev
branch
17 times, most recently
from
June 19, 2026 23:09
f02f651 to
1f3d93f
Compare
intarweb-sync-bot
Bot
force-pushed
the
intarweb-dev
branch
13 times, most recently
from
June 28, 2026 09:26
5eadc10 to
03b0a58
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Internal carry, lands via FORK_CARRIED_COMMITS. Draft for visibility only — not intended to merge.
What
Add a BuildKit cache mount to the
flashinfer download-cubinstep in vllm-base:Why fork-only
We control build cadence on our builder;
--mount=type=cacherequires BuildKit + a persistent builder. Upstream may build in environments where this assumption doesn't hold.Why sharing=locked
Concurrent cubin downloads to the same target can corrupt partial files (race on the same on-disk paths).
sharing=lockedserializes concurrent builders touching this cache target.Risk