fix(directory): make rolling upgrade handoffs durable - #10323
Merged
Merged
Conversation
Wait for distributed-directory registration and duplicate cleanup before allowing the legacy owner to remove transferred entries. Coalesce retried handoffs and verify recipient completion precedes sender removal. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4485bcb4-85f7-4611-8db5-ec91b5380f0c
Contributor
There was a problem hiding this comment.
Pull request overview
This PR hardens LocalGrainDirectory → DistributedGrainDirectory rolling-upgrade split-partition handoffs by ensuring the receiver only acknowledges the handoff after registrations are installed and duplicate activations are cleaned up, preventing brief “no authoritative owner” windows which can lead to duplicate activations.
Changes:
- Make
DistributedRemoteGrainDirectory.AcceptSplitPartitionawait durable completion of registration + duplicate-activation cleanup, and coalesce in-flight retries onto an existing batch. - Update the rolling-upgrade test to assert (from structured diagnostics) that receiver completion occurs before sender-side removal, and that a non-empty handoff occurred.
- Extend the test log capture to extract
Silo/AddedSiloandCountfrom structured log state for stronger assertions.
Show a summary per file
| File | Description |
|---|---|
| test/Orleans.GrainDirectory.Tests/GrainDirectory/GrainDirectoryRollingUpgradeTests.cs | Adds structured-log-based assertions validating durable handoff ordering and exercises a non-empty handoff path. |
| src/Orleans.Runtime/GrainDirectory/DistributedRemoteGrainDirectory.cs | Changes split-partition acceptance to complete only after registrations/cleanup finish and deduplicates concurrent retries via a shared completion signal. |
Copilot's findings
- Files reviewed: 2/2 changed files
- Comments generated: 0
This was referenced Aug 11, 2026
This was referenced Aug 28, 2026
Merged
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
During LocalGrainDirectory to DistributedGrainDirectory rolling upgrades, split-partition handoffs acknowledged queued registration work before it completed. The legacy owner could remove its entries first, briefly leaving no authoritative registration and allowing a duplicate activation to be created.
This change makes the handoff acknowledgment complete only after the distributed directory has installed the registrations and cleaned up losing activations. Repeated sender retries coalesce onto the existing transfer, and the rolling-upgrade test now verifies from structured diagnostics that recipient completion precedes sender removal and that a non-empty handoff was exercised.
Microsoft Reviewers: Open in CodeFlow