Clusters autodiscovery #629
Open
+553
−49
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Resolve the issue of leaking keeper watches.
|
This is an automated comment for commit 0575462 with description of existing statuses. It's updated for the latest CI running ❌ Click here to open a full report in a separate page
Successful checks
|
ilejn
reviewed
Feb 21, 2025
| @@ -110,6 +111,54 @@ class ClusterDiscovery::ConcurrentFlags | |||
| bool stop_flag = false; | |||
| }; | |||
|
|
|||
There was a problem hiding this comment.
Please, explain why Flags and ConcurrentFlags coexist.
| /* cluster_secret */ cluster_secret | ||
| ); | ||
|
|
||
| multicluster_discovery_paths->push_back( |
There was a problem hiding this comment.
To me, we overuse shared_ptr in this code (not only in this particular place).
May be it worth getting rid of this extra layer or at least introduce types like FlagsPtr? I think that emplace_back should work effortlessly.
|
|
||
| for (auto & path : (*multicluster_discovery_paths)) | ||
| { | ||
| ++zk_root_index; |
There was a problem hiding this comment.
Please, explain why we start index from 1.
|
|
||
| if (info.count(cluster)) | ||
| { /// Possible with several root paths, it's a configuration error | ||
| LOG_WARNING(log, "Found dynamic duplicate of cluster '{}' in Keeper, skipped record by path {}:{}", |
There was a problem hiding this comment.
Shouldn't we be more strict and prohibit this kind of misconfiguration?
| { | ||
| LOG_ERROR(log, "Unknown cluster '{}'", cluster_name); | ||
| continue; | ||
| if (!info.zk_root_index) |
There was a problem hiding this comment.
Is it actually possible? In which circumstances?
| continue; | ||
| auto p = new_dynamic_clusters_info.find(cluster_name); | ||
| if (p != new_dynamic_clusters_info.end()) | ||
| new_dynamic_clusters_info.erase(p); |
There was a problem hiding this comment.
new_dynamic_clusters_info.erase(cluster_name) is enough,
though I am not sure, may be it is less verbose.
|
|
||
| if (!need_update.exchange(false)) | ||
| auto clusters = dynamic_clusters_to_update->wait(5s, finished); |
There was a problem hiding this comment.
'5s' probably worth moving to header.
| auto cluster_info_it = clusters_info.find(cluster_name); | ||
| if (cluster_info_it == clusters_info.end()) | ||
| { | ||
| LOG_ERROR(log, "Unknown cluster '{}'", cluster_name); |
There was a problem hiding this comment.
May be a bit more verbose message?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Clusters autodiscovery
Cherry-pick of bugfix ClickHouse#74521
Documentation entry for user-facing changes
ClickHouse has a feature to discovery nodes in cluster (see setting
allow_experimental_cluster_discovery).This PR add ability to discover new clusters.
Nodes in new clusters keep the same way to register in Keeper
Added ability to discover all clusters in Keeper node
Cherry-pick of bugfix ClickHouse#74521