Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable query cancellation for MSQE + cancel using client-provided id #14823

Merged
merged 34 commits into from
Feb 11, 2025
Merged
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
f7a9488
add cancelClientQuery operation for SingleStageBroker (only numerical…
albertobastos Jan 15, 2025
39a4f94
avoid synchronized BiMap and checkstyle
albertobastos Jan 16, 2025
c969abd
Merge remote-tracking branch 'origin' into cancel-with-cqid
albertobastos Jan 16, 2025
8162bc6
Merge branch 'master' into cancel-with-cqid
albertobastos Jan 24, 2025
aa8c120
Merge branch 'master' of github.com:albertobastos/pinot into cancel-w…
albertobastos Jan 27, 2025
7a5f713
add cancel feature (with queryId and clientQueryId) to MSQE, some ref…
albertobastos Jan 27, 2025
a9d1e49
set and delete clientRequestId on MSQE
albertobastos Jan 27, 2025
97e7b5d
fix unimplemented method
albertobastos Jan 27, 2025
65f73a0
fix I/O parameter and related tests
albertobastos Jan 27, 2025
e3a9a5e
Merge branch 'master' of github.com:albertobastos/pinot into cancel-w…
albertobastos Jan 28, 2025
fe5c846
add clientRequestId on response test
albertobastos Jan 28, 2025
ae3260c
Merge branch 'master' of github.com:albertobastos/pinot into cancel-w…
albertobastos Jan 28, 2025
2eb506e
add sleep and random functions for further tests
albertobastos Jan 28, 2025
5dd5409
Merge branch 'master' of github.com:albertobastos/pinot into cancel-w…
albertobastos Jan 29, 2025
e9bbdac
override test server conf
albertobastos Jan 29, 2025
9dcc393
add missing superclass call
albertobastos Jan 29, 2025
5ac7d9e
add some cancel query test using internal sleep function with a trick
albertobastos Jan 29, 2025
a46659c
Merge branch 'master' of github.com:albertobastos/pinot into cancel-w…
albertobastos Jan 30, 2025
110fb16
bring master
albertobastos Jan 30, 2025
47fb9bb
reuse same broker endpoint for internal and client-based cancellation
albertobastos Jan 31, 2025
52998d3
add javadoc
albertobastos Jan 31, 2025
e2678af
add mapping comments
albertobastos Jan 31, 2025
c201cf4
refactor base broker methods
albertobastos Jan 31, 2025
c60b953
return immutable view instead of copy
albertobastos Jan 31, 2025
6042ad2
enable sleep(ms) function only during testing
albertobastos Jan 31, 2025
b458a5b
reduce unit test wait time
albertobastos Jan 31, 2025
9d0f335
replace constant with literal on test
albertobastos Jan 31, 2025
1e00bf6
linter
albertobastos Jan 31, 2025
d3061ba
Merge branch 'master' of github.com:albertobastos/pinot into cancel-w…
albertobastos Jan 31, 2025
a0e1e83
remove embarassing npe
albertobastos Jan 31, 2025
d0e6393
Merge branch 'master' of github.com:albertobastos/pinot into cancel-w…
albertobastos Feb 3, 2025
5e58284
Merge branch 'master' of github.com:albertobastos/pinot into cancel-w…
albertobastos Feb 7, 2025
f8d23cd
minor pr suggestions
albertobastos Feb 7, 2025
f4ad417
Merge remote-tracking branch 'origin' into cancel-with-cqid
albertobastos Feb 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
bring master
albertobastos committed Jan 30, 2025
commit 110fb16f83807ca0387dba371696ab0b4f78d9ec
Original file line number Diff line number Diff line change
@@ -142,6 +142,8 @@ public abstract class BaseSingleStageBrokerRequestHandler extends BaseBrokerRequ
protected final boolean _enableDistinctCountBitmapOverride;
protected final int _queryResponseLimit;
protected final Map<Long, QueryServers> _serversById;
// if >= 0, then overrides default limit of 10, otherwise setting is ignored
protected final int _defaultQueryLimit;
protected final boolean _enableMultistageMigrationMetric;
protected ExecutorService _multistageCompileExecutor;
protected BlockingQueue<Pair<String, String>> _multistageCompileQueryQueue;
@@ -164,6 +166,10 @@ public BaseSingleStageBrokerRequestHandler(PinotConfiguration config, String bro
} else {
_serversById = null;
}
_defaultQueryLimit = config.getProperty(Broker.CONFIG_OF_BROKER_DEFAULT_QUERY_LIMIT,
Broker.DEFAULT_BROKER_QUERY_LIMIT);
boolean enableQueryCancellation =
Boolean.parseBoolean(config.getProperty(Broker.CONFIG_OF_BROKER_ENABLE_QUERY_CANCELLATION));

_enableMultistageMigrationMetric = _config.getProperty(Broker.CONFIG_OF_BROKER_ENABLE_MULTISTAGE_MIGRATION_METRIC,
Broker.DEFAULT_ENABLE_MULTISTAGE_MIGRATION_METRIC);
@@ -172,10 +178,11 @@ public BaseSingleStageBrokerRequestHandler(PinotConfiguration config, String bro
_multistageCompileQueryQueue = new LinkedBlockingQueue<>(1000);
}

LOGGER.info("Initialized {} with broker id: {}, timeout: {}ms, query response limit: {}, query log max length: {}, "
+ "query log max rate: {}", getClass().getSimpleName(), _brokerId,
_brokerTimeoutMs, _queryResponseLimit, _queryLogger.getMaxQueryLengthToLog(), _queryLogger.getLogRateLimit(),
this.isQueryCancellationEnabled());
LOGGER.info("Initialized {} with broker id: {}, timeout: {}ms, query response limit: {}, "
+ "default query limit {}, query log max length: {}, query log max rate: {}, query cancellation "
+ "enabled: {}", getClass().getSimpleName(), _brokerId, _brokerTimeoutMs, _queryResponseLimit,
_defaultQueryLimit, _queryLogger.getMaxQueryLengthToLog(), _queryLogger.getLogRateLimit(),
enableQueryCancellation);
}

@Override
You are viewing a condensed version of this merge commit. You can view the full changes here.