Skip to content

Skip Interlocked.Exchange and FlushConfig when no update detected - #905

Merged
Vasileios Zois (vazois) merged 5 commits into
microsoft:mainfrom
vazois:vazois/config-update
Jan 9, 2025
Merged

Skip Interlocked.Exchange and FlushConfig when no update detected#905
Vasileios Zois (vazois) merged 5 commits into
microsoft:mainfrom
vazois:vazois/config-update

Conversation

@vazois

@vazois Vasileios Zois (vazois) commented Jan 9, 2025

Copy link
Copy Markdown
Contributor

This PR fixes an issue that caused message flooding across the cluster. This happened because the config object kept being updated even when the received config did not change the local config during merge.

The check for an updated config is happening implicitly by comparing the new and current config objects

var currentCopy = current.Copy();
var next = currentCopy.Merge(senderConfig, workerBanList, logger).HandleConfigEpochCollision(senderConfig, logger);
if (currentCopy == next) return false;
if (Interlocked.CompareExchange(ref currentConfig, next, current) == current)
break;
}
FlushConfig();
return true;

This meant that the call to currentCopy.Merge should return the same object if no update has happened. This occurs correctly when merging the worker info

if (worker.ConfigEpoch <= workers[i].ConfigEpoch) return this;

However, an equivalent check was not there when the object was updated during the merging of the slotMap information.
The PR adds this check to fix the issue of message flooding when the cluster configuration is stable

Fixes #900

@vazois
Vasileios Zois (vazois) force-pushed the vazois/config-update branch 3 times, most recently from 894c344 to 49c0edd Compare January 9, 2025 02:37

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, please update version for a new release as well.

@vazois
Vasileios Zois (vazois) marked this pull request as draft January 9, 2025 19:15
@vazois
Vasileios Zois (vazois) marked this pull request as ready for review January 9, 2025 20:17
@vazois
Vasileios Zois (vazois) merged commit e9d7906 into microsoft:main Jan 9, 2025
@github-actions github-actions Bot locked and limited conversation to collaborators Mar 11, 2025
@vazois
Vasileios Zois (vazois) deleted the vazois/config-update branch June 26, 2025 16:55
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unresponsive Garner server upon multiple cluster meets

2 participants