fix(cloudflare): serve wrangler secrets via single-shot FIFO#774
Merged
Conversation
The FIFO that feeds resolved env to wrangler re-armed a new writer in a tight loop. A FIFO is an unframed byte stream and wrangler reads the --secrets-file with readFileSync (read-until-EOF), where EOF only occurs once no writer holds the pipe open. Because the loop re-opened a writer immediately after each close, a reader could read several concatenated copies of the JSON before ever seeing EOF, so wrangler intermittently reported the secrets-file contents as invalid (observed in Linux CI, scheduling-dependent). Fixes #739. Writers now serve exactly one copy then exit, guaranteeing a clean EOF after one copy. The dev path, where wrangler re-reads the env file across hot-reloads/restarts, opts into re-arming a fresh single-shot writer after each read; only one writer is ever armed at a time, so copies never overlap. The FIFO is kept (rather than switching to a temp file) so resolved secrets never exist as a file at rest — a failed or aborted deploy leaves no plaintext secrets on disk.
Contributor
|
The changes in this PR will be included in the next version bump.
|
commit: |
wrangler reads these files more than once — `wrangler types` re-reads the --env-file, and the deploy/upload path can re-read the --secrets-file — so a pure one-shot writer left the second read hanging (the cloudflare `types` framework test timed out). Always re-arm a fresh single-shot writer once the previous copy is consumed, for every command rather than dev only. Still only one writer is armed at a time, so copies never overlap (no concatenation), and secrets never touch disk.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
varlock-wrangler deploy/versions uploadintermittently fail in Linux CI with wrangler reporting:Fixes #739.
Root cause
@varlock/cloudflare-integrationfeeds resolved env to wrangler through a temporary FIFO so secrets never touch disk. The FIFO server re-armed a new writer in a tightsetImmediateloop, continuously re-writing the same JSON.A FIFO is an unframed byte stream. Wrangler reads the file with
fs.readFileSync(read-until-EOF), and EOF only occurs once no writer holds the pipe open. Because the loop re-opened a writer immediately after each close, a reader could read several concatenated copies of the JSON ({...}{...}{...}) before ever seeing EOF — which failsJSON.parse, falls back to dotenv, yields 0 keys, and surfaces as "is not valid".It's intermittent and Linux-only because it depends on kernel pipe scheduling; macOS reliably gives a clean EOF between writes, so it never reproduces locally.
Diagnostic signature: the bad read's byte count is always an exact integer multiple of the real payload size (e.g.
721884 = 3 × 240628).Fix
Writers now serve exactly one copy then exit (
process.exit(0)instead ofsetImmediate(serve)), so closing the write fd delivers a clean EOF after one copy — no concatenation possible.Once that copy is consumed (the writer exits cleanly), a fresh single-shot writer is re-armed so the next reader has content available. This is required because wrangler reads these files more than once —
wrangler typesre-reads the--env-file, and the deploy/upload path can re-read the--secrets-file— so a pure one-shot writer would leave the second read hanging. Re-arming is generation-guarded so only one writer is ever armed at a time, meaning copies can never overlap (no concatenation).The FIFO is intentionally kept rather than replaced with a regular temp file: resolved secrets never exist as a file at rest, so a failed or aborted deploy leaves no plaintext secrets on disk.
Verification
Reproduced and verified against the real built package on Linux (the bug does not reproduce on macOS):
varlock-wrangler deploy, 320 runs, 4 parallelvarlock-wrangler types(wrangler re-reads the env file)worker-configuration.d.tsAll 21 cloudflare
wranglerframework tests pass (includingtypes generation, which caught an earlier pure-single-shot attempt that didn't re-arm).typecheckandlintpass. Changeset included (@varlock/cloudflare-integration: patch).Testing notes
No dedicated deploy-concatenation test is added. The fix lives in the shared FIFO serving code used by
deploy,types, anddev, and the existingtypes/devframework scenarios already exercise it end-to-end with real wrangler — thetypesscenario caught the re-arm regression during development. A deploy-specific guard would only be meaningful on Linux and is inherently probabilistic (a scheduling race), so it would risk silently passing on broken code while adding maintenance surface; the existing coverage plus the documented invariant instartServingis the better trade-off.