Skip to content

Simplify cluster state management in ESQL #131635

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 23, 2025

Conversation

idegtiarenko
Copy link
Contributor

Today we group indices by remote cluster twice. There are also 2 places where remote cluster info is initialized (initializeClusterData or checkForCcsLicense in case of failure).

This change merges them into a single method call as well as ensures groupIndices is only called once.

@idegtiarenko idegtiarenko requested a review from smalyshev July 21, 2025 16:17
@idegtiarenko idegtiarenko added >refactoring Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) :Analytics/ES|QL AKA ESQL v9.2.0 labels Jul 21, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

* @param indices index expression requested by user
* @param indicesGrouper grouper of index expressions by cluster alias
* @param licenseState license state on the querying cluster
* Checks the index expression for the presence of remote clusters.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this function is the joining of checkForCcsLicense and initializeClusterData - which is fine, but then the comment should reflect that. Right now the comment only talks about checking the license but not the rest of it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already added below:

* as well as initialize corresponding cluster state in execution info.

throw EsqlLicenseChecker.invalidLicenseForCcsException(licenseState);
}
}
} catch (NoSuchRemoteClusterException e) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand it right that NoSuchRemoteClusterException can be thrown by groupIndices? If so, I'd have try/catch to only enclose this call maybe? Otherwise the code flow is a bit un-obvious I think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tried that initially, however it looked quiet confusing regarding where the flow could terminate.
I would prefer to keep this version.

assert patterns.size() == 1 : "Only single index pattern is supported";
try {
var groupedIndices = indicesGrouper.groupIndices(
Set.copyOf(indicesGrouper.getConfiguredClusters()),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be useful to add some comment about why we're copying it here, because I personally don't remember it, and I worked with this code a lot :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not clear for me either. I copied it from the original place.
It is backed by the mutable data structure below, so this might guard against possible different results when resolving multiple *. Let me add that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes, I think I vaguely remember now that was a concern - since the original map can be changed by settings, we were worried that it could change mid-request and cause all kinds of weirdness. If it's only within groupIndices it's probably less probable, but better to be safe here.

@idegtiarenko idegtiarenko merged commit 72bd5a0 into elastic:main Jul 23, 2025
33 checks passed
@idegtiarenko idegtiarenko deleted the simplify_esql_ccs branch July 23, 2025 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL >refactoring Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v9.2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants