Clear the Vector Set index pointer on the diskless replication stream-in path

[Tiago Nápoli (tiagonapoli)]

Problem

A Vector Set index record persists the primary's native DiskANN handle (IndexPtr) in its value. Diskless sync streams records verbatim into the replica's memory, so the replica receives the primary's pointer.

The only hook that zeroes that field is GarnetRecordTriggers.OnDiskRead, which never fires for records streamed straight into memory. Since NeedsRecreate is exactly indexPtr == 0, a foreign pointer looks healthy: the lazy RecreateIndex rebuild is skipped and the raw value reaches the P/Invoke, faulting the replica with SIGSEGV inside NativeDiskANNMethods.card.

Fix

• Index pointer: NetworkClusterSync clears it as records are deserialized, so the replica rebuilds its own index on first touch.
• Context reservation: a diskless full sync SETs streamed index and ContextMetadata records directly, bypassing the RMW path that maintains the in-memory reservation bitmap. Nothing rebuilt it, so after promotion a fresh Vector Set could be handed a context a streamed set already owns, corrupting both. Streamed records now go through SanitizeAndTrackIngestedRecordIfApplicable, and the reservation is rebuilt via ReconcileRecoveredState once the stream completes. Its requireNoReservedContexts flag fails loudly if a context survived the preceding flush, since the rebuild replaces contextMetadatas wholesale and would otherwise leak that index.
• Flush (found in review): FlushGuard.Dispose reset contextMetadatas but left dirtyContextMetadatas populated, so a concurrent UpdateContextMetadata drain could index out of range, or re-persist metadata for a context the flush just removed. Now cleared with the rest of the invalidated state.

Tests

Adds Garnet.test.cluster.replication.vectorsets, registered in Garnet.slnx and the CI/nightly matrices.

The harness asserts index ownership and element-level equality — matching cardinality alone is too weak, since an aliased index reports the source's count perfectly well. Covers diskless full sync, async replay, multi-database sync, FreshVectorSetDoesNotReuseStreamedContextAfterDisklessFullSync for the reservation regression, and two exception-injection tests for an aborted full sync (recovery after retry, and no orphaned namespaced records left behind).

Deferred (separate PRs)

• Vector Set failover tests.
• RangeIndex diskless-replication coverage.
• Disk-based full sync of a populated Vector Set (distinct, more severe foreign-IndexPtr AVE + disk-based reservation gap).

[Copilot]

Pull request overview

This PR fixes a diskless-replication safety issue for Vector Sets by sanitizing streamed Vector Set index records (clearing the native IndexPtr) and by rebuilding/validating Vector Set context-reservation state after a diskless full sync, preventing replicas from dereferencing foreign pointers or reusing streamed contexts.

Changes:

• Sanitize ingested (recovery + diskless full sync) Vector Set records via VectorManager.SanitizeAndTrackIngestedRecordIfApplicable, and rebuild reservation state via VectorManager.ReconcileRecoveredState.
• Hook sanitization into CLUSTER SYNC stream-in and run reconciliation after diskless full sync completion.
• Add a dedicated cluster replication test project for Vector Sets and wire it into the solution + CI/nightly matrices.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
test/cluster/Garnet.test.cluster/Garnet.test.cluster.csproj	Adds IVT for the new Vector Set replication test assembly.
test/cluster/Garnet.test.cluster/ClusterTestUtils.cs	Adds cluster-formation and replication helpers used by the new tests.
test/cluster/Garnet.test.cluster/ClusterTestContext.cs	Adds a dedicated port assignment and convenience helpers for the new replication tests.
test/cluster/Garnet.test.cluster.replication.vectorsets/TestProjectSetup.cs	Sets a unique base port for the new test assembly.
test/cluster/Garnet.test.cluster.replication.vectorsets/ReplicationTests/VectorSetReplicationTestBase.cs	Adds shared harness utilities for asserting Vector Set replication correctness and index ownership.
test/cluster/Garnet.test.cluster.replication.vectorsets/ReplicationTests/ClusterVectorSetDisklessSyncTests.cs	Adds diskless full-sync / failover / migration / exception-injection coverage for Vector Set replication.
test/cluster/Garnet.test.cluster.replication.vectorsets/Garnet.test.cluster.replication.vectorsets.csproj	Introduces the new test project and references required components.
libs/server/Storage/Functions/GarnetRecordTriggers.cs	Routes recovery snapshot reads through the new sanitize+track helper.
libs/server/Resp/Vector/VectorManager.cs	Adds SanitizeAndTrackIngestedRecordIfApplicable, renames/extends post-recovery logic to ReconcileRecoveredState, and makes recovered bookkeeping reusable.
libs/server/Resp/Vector/VectorManager.ContextMetadata.cs	Clears pending recovered state on flush (and should also clear dirty bookkeeping; see comment).
libs/server/Properties/AssemblyInfo.cs	Adds IVT for the new test assembly.
libs/server/Databases/SingleDatabaseManager.cs	Updates vector-set recovery path to call ReconcileRecoveredState().
libs/server/Databases/MultiDatabaseManager.cs	Updates vector-set recovery path to call ReconcileRecoveredState() across databases.
libs/cluster/Session/RespClusterReplicationCommands.cs	Sanitizes streamed records during CLUSTER SYNC ingestion.
libs/cluster/Server/Replication/ReplicaOps/ReplicaDisklessSync.cs	Reconciles recovered state after diskless full sync completion (and validates reservation is empty on this path).
Garnet.slnx	Registers the new test project in the solution.
.github/workflows/nightly.yml	Adds the new test project to the nightly test matrix.
.github/workflows/ci.yml	Adds the new test project to the CI test matrix.

[kevin-montrose]

The dirty namespaces Copilot seems correct, plus I think some HasNamespace checks are missing (things might still pass in Release, Debug I would expect assert failures).