Skip to content

[glue] Verify Reshare Epoch Artifacts on Demand - #4364

Merged
patrick-ogrady merged 22 commits into
mainfrom
cl/precompute-artifacts
Aug 20, 2026
Merged

[glue] Verify Reshare Epoch Artifacts on Demand#4364
patrick-ogrady merged 22 commits into
mainfrom
cl/precompute-artifacts

Conversation

@clabby

@clabby clabby commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Moves reshare ceremony verification off the inclusion actor and onto the shared CPU pool.
  • Starts verification only for a live final-block artifact request or exact canonical finalization, with at most one active verification task.
  • Keeps competing ancestry requests lazy and FIFO, materializing one active request while reusing results for exact dealer-log views.
  • Follows the epoch after a late state-sync floor, where skipped inclusion history cannot prove dealer-log coverage.

Adversarial behavior

The epoch determines the boundary height, while the typed ancestry head determines fork identity. Inclusion derives the boundary parent from that ancestry and rejects missing, unexpected, gapped, or truncated chains. A malicious leader cannot force verification of every growing dealer-log prefix, and a malicious host cannot provide an independent canonical fanout key.

Finalization completes any admitted speculative waiter before reconstructing the canonical artifact. Speculative verification, caches, and queued requests remain phase-local and reconstructable after restart; wire and storage formats are unchanged.

Validation

  • just test -p commonware-glue
  • just test --profile slow -p commonware-glue reshare_e2e_state_sync
  • just clippy -p commonware-glue
  • just check-fmt
  • just check-stability

Closes #4300

@clabby clabby added this to Tracker Jul 30, 2026
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 30, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
commonware-mcp 6ddcbf2 Aug 19 2026, 11:55 PM

@clabby
clabby marked this pull request as ready for review July 30, 2026 02:14
@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown

Benchmark results

Regressions: 0.

✅ `qmdb::merkleize/v=any::unordered::fixed::mmr k=10000 ch=false s=true cc=true` (2/2 gates passed)
Field Value
Package commonware-storage
Benchmark target qmdb_gungraun
Variant qmdb::merkleize/v=any::unordered::fixed::mmr k=10000 ch=false s=true cc=true
Filter *::bench_merkleize::any_unordered_fixed_mmr
Baseline suite commonware-storage
Gates EstimatedCycles should decrease; tolerance 10.00%; blob_reads should decrease; tolerance 10.00%
Cargo flags --features test-traits
Metric Baseline Current Delta Gate
Ir 13,193,266 13,164,382 -0.22% -
L1hits 16,876,759 16,846,221 -0.18% -
LLhits 97,477 95,075 -2.46% -
RamHits 11,263 11,148 -1.02% -
TotalRW 16,985,499 16,952,444 -0.19% -
EstimatedCycles 17,758,349 17,711,776 -0.26% EstimatedCycles should decrease; tolerance 10.00%
blob_reads 158 158 +0.00% blob_reads should decrease; tolerance 10.00%
✅ `qmdb::merkleize/v=current::ordered::fixed::mmb chunk=256 k=10000 ch=false s=true cc=true` (2/2 gates passed)
Field Value
Package commonware-storage
Benchmark target qmdb_gungraun
Variant qmdb::merkleize/v=current::ordered::fixed::mmb chunk=256 k=10000 ch=false s=true cc=true
Filter *::bench_merkleize::current_ordered_fixed_mmb_chunk_256
Baseline suite commonware-storage
Gates EstimatedCycles should decrease; tolerance 10.00%; blob_reads should decrease; tolerance 10.00%
Cargo flags --features test-traits
Metric Baseline Current Delta Gate
Ir 15,506,247 15,485,943 -0.13% -
L1hits 20,403,595 20,374,553 -0.14% -
LLhits 145,436 138,318 -4.89% -
RamHits 14,314 17,349 +21.20% -
TotalRW 20,563,345 20,530,220 -0.16% -
EstimatedCycles 21,631,765 21,673,358 +0.19% EstimatedCycles should decrease; tolerance 10.00%
blob_reads 235 235 +0.00% blob_reads should decrease; tolerance 10.00%

Baseline commit(s): 1ba964dcd45a

Comment thread glue/src/dkg/reshare/actor/inclusion.rs
@clabby
clabby marked this pull request as draft July 30, 2026 14:06
@clabby
clabby force-pushed the cl/precompute-artifacts branch from 2de33c4 to 467bf3d Compare July 30, 2026 14:25
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 30, 2026

Copy link
Copy Markdown

Deploying monorepo with  Cloudflare Pages  Cloudflare Pages

Latest commit: 6ddcbf2
Status: ✅  Deploy successful!
Preview URL: https://651743b4.monorepo-eu0.pages.dev
Branch Preview URL: https://cl-precompute-artifacts.monorepo-eu0.pages.dev

View logs

@clabby
clabby marked this pull request as ready for review July 30, 2026 14:40
@clabby clabby moved this to Ready for Review in Tracker Jul 30, 2026
Comment thread glue/src/dkg/reshare/actor/inclusion.rs Outdated
@clabby
clabby marked this pull request as draft July 30, 2026 14:59
@clabby
clabby marked this pull request as ready for review July 30, 2026 15:05
@clabby clabby moved this from Ready for Review to In Progress in Tracker Jul 30, 2026
@patrick-ogrady patrick-ogrady added this to the v2026.8.0 milestone Aug 2, 2026
# Conflicts:
#	glue/src/dkg/reshare/actor/dealing.rs
#	glue/src/dkg/reshare/actor/inclusion.rs
#	glue/src/dkg/tests/mocks.rs

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit c9b981a. Configure here.

Comment thread glue/src/dkg/reshare/actor/inclusion.rs Outdated
@patrick-ogrady patrick-ogrady changed the title [glue] Verify Reshare Epoch Material Asynchronously [glue] Verify Reshare Epoch Artifacts on Demand Aug 11, 2026
Comment thread glue/src/dkg/reshare/actor/inclusion.rs Outdated
@codecov

codecov Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.05094% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.57%. Comparing base (6e39079) to head (6ddcbf2).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
glue/src/dkg/reshare/mailbox.rs 90.00% 5 Missing and 2 partials ⚠️
glue/src/dkg/reshare/actor/setup.rs 98.50% 1 Missing and 1 partial ⚠️
glue/src/dkg/reshare/application.rs 94.44% 1 Missing and 1 partial ⚠️
@@            Coverage Diff             @@
##             main    #4364      +/-   ##
==========================================
+ Coverage   95.52%   95.57%   +0.05%     
==========================================
  Files         610      611       +1     
  Lines      276780   277515     +735     
  Branches     6646     6655       +9     
==========================================
+ Hits       264381   265237     +856     
+ Misses      10208    10113      -95     
+ Partials     2191     2165      -26     
Files with missing lines Coverage Δ
consensus/src/marshal/ancestry.rs 96.02% <100.00%> (+0.99%) ⬆️
consensus/src/marshal/application/validation.rs 100.00% <ø> (ø)
consensus/src/marshal/coding/validation.rs 92.20% <ø> (ø)
consensus/src/marshal/standard/validation.rs 83.62% <ø> (ø)
glue/src/dkg/reshare/actor/dealing.rs 83.95% <100.00%> (-1.81%) ⬇️
glue/src/dkg/reshare/actor/follower.rs 81.96% <ø> (+81.96%) ⬆️
glue/src/dkg/reshare/actor/inclusion.rs 95.72% <ø> (+4.07%) ⬆️
glue/src/dkg/reshare/actor/mod.rs 92.70% <100.00%> (+3.22%) ⬆️
glue/src/dkg/reshare/store.rs 96.83% <100.00%> (+0.01%) ⬆️
glue/src/dkg/reshare/actor/setup.rs 96.86% <98.50%> (+2.18%) ⬆️
... and 2 more

... and 41 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6e39079...6ddcbf2. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@patrick-ogrady patrick-ogrady left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't call glue::dkg ready for primetime yet but this is definitely a step forward.

@patrick-ogrady
patrick-ogrady merged commit 0bd869f into main Aug 20, 2026
316 checks passed
@patrick-ogrady
patrick-ogrady deleted the cl/precompute-artifacts branch August 20, 2026 13:41
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Tracker Aug 20, 2026
@patrick-ogrady

Copy link
Copy Markdown
Contributor

Post-merge note: this PR also fixes a security-relevant final-block validation gap in reshare follower mode. Previously, EpochInfoResponse::Following delegated final-block verification to the wrapped application even though the follower could not independently reconstruct the epoch artifact. That allowed a follower to approve boundary metadata it had not validated. The merged behavior keeps verification pending instead, so the node can learn the outcome from external finalization without producing a completed application verdict for unverified epoch metadata.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[glue] Verify reshare epoch material asynchronously

2 participants