tiered: daemon-held handle — per-cycle extend instead of rebuild-shaped maintenance

[jamie8johnson]

Follow-up flagged in #1794: when CQS_TIERED_INDEX=1 is active, the watch loop still treats index maintenance as rebuild-shaped — it doesn't hold a persistent tiered handle, so incremental deltas can't flow as per-cycle extend calls. True O(batch) incremental behavior means the daemon owns a long-lived TieredIndex and the watch drain calls extend on it directly, with the cold-start rebuild remaining the only full-construction path (cuVS has no tiered serialize).

Scope: daemon/watch lifecycle only — the backend itself (src/tiered.rs) already supports extend. Gated behind the same tiered-index feature + fork pin, so this doesn't block releases.

[jamie8johnson]

Scoping verdict (2026-07-05): STRIKE — conditional YAGNI, keep open as blocked-on-consumer

A code-grounded scoping pass (high confidence) found the gap is real if tiered runs, but nothing runs it, and the fix shape in the issue contradicts the actual daemon architecture — it can't be built as-scoped.

The gap is genuine but dormant. TieredIndex::extend (O(batch), src/tiered.rs:317) exists and works, but has zero production callers (only internal + a test). The served index is built by TieredBackend::try_open (src/tiered.rs:569), which reconstructs the whole tiered index fresh from the store on every call, cached by checkout_epoch. Any store write bumps PRAGMA data_version → invalidate_mutable_caches → epoch fetch_add → the next search full-rebuilds tiered O(n). So today tiered trades one periodic HNSW rebuild for a full tiered rebuild on every data_version bump — plausibly worse than the path it replaces.

But nothing executes this path. tiered-index is not in default (Cargo.toml:210), is opt-in (:296), won't compile without uncommenting the cuvs fork [patch.crates-io] pin (:383-393), and is runtime-gated on CQS_TIERED_INDEX=1 (shadowed by CAGRA otherwise). The systemd daemon and installed binary build cuda-index, not tiered-index. Nothing breaks today because the path is dead in every built/deployed config.

The issue's fix shape is wrong for the architecture. It assumes the served index lives in the watch loop and a watch-drain can call extend on it directly. It doesn't: the watch loop holds a concrete HnswIndex (src/cli/watch/mod.rs:182), the serve path answers from BatchView::vector_index() (src/cli/batch/view.rs:793), and the two are decoupled through disk + SQLite. Tiered has no disk persistence (src/tiered.rs:36), so a real fix can't use that handoff — it needs an in-memory Arc<TieredIndex> shared across the watch loop and BatchContext, plus special-casing the epoch-invalidation contract (mutate-in-place vs invalidate-and-rebuild-from-disk) for one backend while HNSW/CAGRA keep the rebuild model. That's a redesign of the daemon's cache-coherence core, not a contained lane.

Revive condition (flips STRIKE → BUILD): someone deploys the daemon with --features tiered-index + CQS_TIERED_INDEX=1 on a real corpus and measures the per-invalidation full rebuild as a live regression. Until a running tiered deployment exists, this optimizes a dead path.

If ever built — lane shape: persistent shared Arc<TieredIndex> owned across watch + batch; a VectorIndex-level extend/mutate affordance the epoch path calls instead of drop-and-rebuild for mutable backends; route WatchState deltas into it. Files: src/tiered.rs, src/index.rs (trait), src/cli/batch/{view,context}.rs (invalidation contract), src/cli/watch/{mod,events,rebuild}.rs. Not small.

Leaving open as tracking — the gap is real, just consumer-blocked.