fix(uxf,payments): bundleCid determinism — lock envelope timestamp across attempts#362
Merged
Conversation
…ross attempts
The CAR root CID of a UXF bundle is the dag-cbor SHA-256 of the
envelope block. The envelope embeds `createdAt` and `updatedAt`. Both
fields were sourced from `Math.floor(Date.now() / 1000)`:
- `UxfPackage.create()` at uxf/UxfPackage.ts:75
- `ingest()` at uxf/UxfPackage.ts:550
- `ingestAll()` at uxf/UxfPackage.ts:686
- `removeToken()` at uxf/UxfPackage.ts:719
- `mergePkg()` at uxf/UxfPackage.ts:1011
This makes the bundleCid non-deterministic across attempts. Two
`UxfPackage.create({...}).ingestAll(...)` sequences straddling a
wall-clock second boundary produce DIFFERENT envelope bytes →
DIFFERENT bundleCids. The flake surfaces in
tests/integration/transfer/crash-recovery.test.ts:726 on slow Node
20 runs (5-min CI hits the boundary often enough to flake the
"bundleCid stable on resume" assertion).
The bug breaks every system keyed on bundleCid for idempotency:
- replay-LRU at the recipient (T.3.A's dedup key)
- IPFS pin reuse (republishing should hit the existing pin, not
create a duplicate)
- sender-side outbox dedup ("same retry, not a new send")
- audit-#333 H3 mergePkg `targetExisting === sourceIncoming`
fast-path
Fix:
- `UxfPackage.create({ createdAt?, updatedAt? })` accepts caller-
locked timestamps. Falls back to `Math.floor(Date.now() / 1000)`
when omitted.
- `UxfPackage.ingest(token, { updatedAt? })` and
`ingestAll(tokens, { updatedAt? })` accept an explicit updatedAt
override. Internal `ingest()`/`ingestAll()` functions widen to
match.
- InstantSenderDeps and ConservativeSenderDeps gain
`bundleCreatedAt?: number` (ms). Senders lock `now =
deps.bundleCreatedAt ?? Date.now()` once at the top, compute
`envelopeStamp = Math.floor(now / 1000)`, and pass it to BOTH
`UxfPackage.create({ createdAt: envelopeStamp })` and
`pkg.ingestAll(..., { updatedAt: envelopeStamp })`. The outbox
entry's `createdAt` already uses the same `now` (just at ms
resolution).
- tests/integration/transfer/crash-recovery.test.ts: each resume
deps now passes `bundleCreatedAt: persisted!.createdAt` so the
re-built bundle reproduces the original bytes — what the test
has always intended to assert.
- New tests/unit/uxf/bundle-cid-determinism.test.ts (6 tests):
forward regression guard for the determinism contract. Includes
a contrast test that verifies the bug exists when locks are
omitted, so a future refactor that accidentally re-introduces
the unlocked path will fail this file specifically.
Production wiring (deferred to a follow-up):
- `PaymentsModule.dispatchUxfInstantSend` / `dispatchUxfConservativeSend`
should pass `bundleCreatedAt` from the resumed outbox entry's
`createdAt`. First-attempt sends (no prior outbox entry) omit it
and the sender computes a single locked timestamp internally.
Tests: 6 new bundleCid determinism tests; existing
crash-recovery.test.ts now correctly verifies idempotency (was a
silent flake); 8651 unit/integration tests pass; tsc clean.
Unblocks: PR #358 (V6-RECOVER test gap) which inherited the Node 20
flake.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The CAR root CID of a UXF bundle is the dag-cbor SHA-256 of the envelope block. The envelope embeds
createdAtandupdatedAt— both sourced fromMath.floor(Date.now() / 1000)at create/ingest/merge time. This makes the bundleCid non-deterministic across attempts: twoUxfPackage.create({...}).ingestAll(...)sequences straddling a wall-clock second boundary (a few ms apart) produce different envelope bytes → different bundleCids.How the bug surfaces
tests/integration/transfer/crash-recovery.test.ts:726asserts:On slow Node 20 CI runs the resume call lands in a different wall-clock second than the first attempt → assertion flakes. Today's PR #358 CI fail (Node 20) → re-run pass demonstrates the flake exactly.
What's broken (besides this one test)
bundleCid stability is a core invariant, not a test convenience. The bug silently degrades:
mergePkgtargetExisting === sourceIncomingfast-path (already in main as of fix(uxf): surface mergePkg skipped tokens + add strict mode (Audit #333 H3) #352 — relies on consistent CIDs)Fix
Public API (additive, fully back-compat)
When the new fields are omitted, behaviour is identical to before. When provided, they lock the envelope timestamps.
Sender wiring
Inside both senders:
Test update
tests/integration/transfer/crash-recovery.test.tsnow passesbundleCreatedAt: persisted!.createdAton each resume deps — what the test had always intended to verify. The 3 idempotency assertions there finally guarantee what they read.New test file — forward regression guard
tests/unit/uxf/bundle-cid-determinism.test.ts(6 tests):createdAtproduce identical CIDscreatedAtproduce different CIDs (sanity: fix didn't strip the timestamp)updatedAtindependently controls envelopeupdatedAtdefaults tocreatedAtThe contrast test is the regression guard: a future refactor that re-introduces unlocked
Date.now()somewhere on the envelope path will fail this specific test, not just flake.Production wiring (deferred)
PaymentsModule.dispatchUxfInstantSend/dispatchUxfConservativeSendshould passbundleCreatedAtfrom the resumed outbox entry'screatedAt. First-attempt sends (no prior outbox entry) omit it and the sender computes a single locked timestamp internally. That's a follow-up — the infrastructure here is sufficient.Test plan
tests/integration/transfer/crash-recovery.test.tstests passtests/unit/uxf/+ 1491tests/unit/payments/transfer/tests passtsc --noEmitcleanRelated