Skip to content

feat: attach recovery envelopes to atomic DB WAL batches - #51

Merged
feichai0017 merged 4 commits into
mainfrom
fix/attached-recovery-journal
Aug 13, 2026
Merged

feat: attach recovery envelopes to atomic DB WAL batches#51
feichai0017 merged 4 commits into
mainfrom
fix/attached-recovery-journal

Conversation

@feichai0017

Copy link
Copy Markdown
Member

What changed

NoKV needs each canonical recovery record to share the commit boundary of the metadata mutation it describes. This PR adds an attached recovery stream to DB. One opaque envelope and its guarded multi-tree batch share one CRC-covered WAL record.

The file-backed stream stores its checkpoint floor in the format-4 WAL header. Retained WAL records define the current tail. A checkpoint mirrors the anchor before truncating the retained suffix. Memory databases provide the same ordering and paging only during the current process. After stream initialization, Holt rejects ordinary logical writes and requires attached batches.

The branch also fixes routed-compaction accounting for reachable EmptyRoot nodes and prepares Holt 0.9.0.

Compatibility

  • Durability::Wal { sync: true } puts each acknowledged attached batch on a forced sync boundary. sync: false does not provide per-acknowledgement power-loss durability.
  • A checkpoint advances the local retention floor. Scans from an older cursor return Error::JournalPositionExpired.
  • Writable open upgrades a header-only format-3 WAL. Holt replays a nonempty format-3 WAL only in read-only mode and rejects writable open before changing store files.
  • Downgrade from format 4 requires DB::export_checkpoint() and DB::install_checkpoint() into a fresh 0.8.x store.
  • The attached recovery stream is local. It is not a shared log, replication stream, or unbounded change feed.

Test plan

  • cargo build --workspace --all-targets --all-features --locked
  • cargo test --workspace --all-features --lib --tests --examples --locked
  • cargo test --workspace --all-features --doc --locked
  • workspace and soak-tool format and clippy checks
  • rustdoc with warnings denied
  • all four documented examples
  • normal and DB-normal soak smoke runs
  • 512-case property tests
  • cargo publish --dry-run --locked --allow-dirty

Regression tests cover crash replay without a checkpoint, dual anchor-slot corruption, interrupted checkpoint retry, legacy WAL migration, and read-only export. They also cover stream fencing, concurrent anchors, checkpoint lock ordering, bounded paging, oversized records, and reachable-empty-root compaction.

The local host lacks rustup/nightly and cargo-llvm-cov. GitHub Actions runs the fuzz smoke and 88% coverage gates.

A downstream NoKV capacity oracle ran two 500-publish batches against the attached stream. The second batch did not raise Holt's slot high-water mark. Its logical store growth was 0.055 MiB per publish.

Related

This implements the checkpoint-bounded recovery suffix under ROADMAP P1. Live subscription and shared retention remain separate work.

NoKV adoption and existing-store migration remain downstream work.

This maintainer-directed Holt 0.9.0 storage work has no linked issue.

After merge, tag the exact merged main commit as v0.9.0. The release workflow publishes the crate and creates the GitHub release.

Preserve-mode merges can leave EmptyRoot sentinels below inner nodes. Count each emitted sentinel in the routing arena so filter-mode compaction cannot cross its measured boundary.

Signed-off-by: Guocheng(Eric) Song <132118805+feichai0017@users.noreply.github.com>
Applications need canonical recovery records to share the commit boundary of the metadata mutations they describe. Store one opaque envelope and its guarded multi-tree batch in the same WAL record, with checkpoint-bounded local retention.

Signed-off-by: Guocheng(Eric) Song <132118805+feichai0017@users.noreply.github.com>
Document the WAL format boundary and migration path. Update the crate version, install examples, supported-version table, and standalone lockfiles for the 0.9.0 release.

Signed-off-by: Guocheng(Eric) Song <132118805+feichai0017@users.noreply.github.com>
The checkpointer removes retry candidates from the FIFO while it tests them, then restores pinned entries. Assert the durable blob state instead of sampling that transient queue window under coverage instrumentation.

Signed-off-by: Guocheng(Eric) Song <132118805+feichai0017@users.noreply.github.com>
@feichai0017
feichai0017 marked this pull request as ready for review August 13, 2026 19:05
Copilot AI lite review requested due to automatic review settings August 13, 2026 19:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@feichai0017
feichai0017 merged commit 8e5f7da into main Aug 13, 2026
25 checks passed
@feichai0017
feichai0017 deleted the fix/attached-recovery-journal branch August 13, 2026 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants