[Improvement](tablet) Prefetch tablets before opening #47501
+105
−44
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
|
TeamCity be ut coverage result: |
|
run buildall |
TPC-H: Total hot run time: 32511 ms |
TPC-DS: Total hot run time: 184979 ms |
ClickBench: Total hot run time: 30.39 s |
|
TeamCity be ut coverage result: |
HappenLee
reviewed
Feb 6, 2025
|
run buildall |
HappenLee
reviewed
Feb 6, 2025
| @@ -424,6 +427,17 @@ Status OlapScanLocalState::_init_scanners(std::list<vectorized::VScannerSPtr>* s | |||
| return Status::OK(); | |||
| } | |||
|
|
|||
| Status OlapScanLocalState::hold_tablets() { | |||
There was a problem hiding this comment.
not only tablet consider rowset will erase by compaction
|
run buildall |
TPC-H: Total hot run time: 31640 ms |
TPC-DS: Total hot run time: 190643 ms |
ClickBench: Total hot run time: 31.05 s |
|
TeamCity be ut coverage result: |
|
run buildall |
TPC-H: Total hot run time: 31707 ms |
TPC-DS: Total hot run time: 191358 ms |
ClickBench: Total hot run time: 31.12 s |
|
TeamCity be ut coverage result: |
HappenLee
reviewed
Feb 7, 2025
|
|
||
| RETURN_IF_ERROR(_tablets[i].tablet->capture_rs_readers( | ||
| {0, version}, &_read_sources[i].rs_splits, | ||
| RuntimeFilterConsumer::_state->skip_missing_version())); |
There was a problem hiding this comment.
why here need RuntimeFilterConsumer？seems wired？why not directly _state
HappenLee
reviewed
Feb 7, 2025
|
|
||
| return Status::OK(); | ||
| } | ||
|
|
||
| Status OlapScanLocalState::hold_tablets() { |
There was a problem hiding this comment.
if the function cost too mush time, should log warning in BE. the clould mode may have io in pipeline thread may hang the thread, we should know the time cost
|
run buildall |
TPC-H: Total hot run time: 31542 ms |
TPC-DS: Total hot run time: 183810 ms |
ClickBench: Total hot run time: 30.82 s |
|
TeamCity be ut coverage result: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Release note
For a scan operator in a query, FE will assign a tablet id to a specific scan operator and then it will be search by storage engine during execution. But this searching process will be done after runtime filter reached. However, if we wait for a runtime filter for a long time and compaction / balance tasks is done at the same time, a tablet / rowset will be lost and we got an error.
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)