[fix](scanner) fix shared rowset reader in different scanners #47744
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
3 similar comments
|
run buildall |
|
run buildall |
|
run buildall |
TPC-H: Total hot run time: 31679 ms |
TPC-DS: Total hot run time: 190759 ms |
ClickBench: Total hot run time: 31.27 s |
|
TeamCity be ut coverage result: |
Gabriel39
force-pushed
the
fix_0211_scan
branch
from
5225301 to
5b73159
Compare
|
run buildall |
TPC-H: Total hot run time: 31338 ms |
HappenLee
reviewed
Feb 13, 2025
TPC-DS: Total hot run time: 188569 ms |
ClickBench: Total hot run time: 30.33 s |
HappenLee
reviewed
Feb 13, 2025
| @@ -383,6 +384,11 @@ Status OlapScanLocalState::_init_scanners(std::list<vectorized::VScannerSPtr>* s | |||
| } | |||
|
|
|||
| COUNTER_UPDATE(_key_range_counter, scanner_ranges.size()); | |||
There was a problem hiding this comment.
add comment and better be
for (auto& rs : _read_sources[scan_range_idx].rs_splits) { for ( auto& split : splits) { split.rs_reader = split.rs_reader->clone(); } }
here no need vector copy
|
TeamCity be ut coverage result: |
|
run buildall |
TPC-H: Total hot run time: 31791 ms |
TPC-DS: Total hot run time: 184178 ms |
ClickBench: Total hot run time: 31.32 s |
|
TeamCity be ut coverage result: |
github-actions
bot
added
the
approved
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
BiteTheDDDDt
approved these changes
Feb 13, 2025
Gabriel39
added a commit
to Gabriel39/incubator-doris
that referenced
this pull request
Feb 13, 2025
…#47744) *** Query id: b7482b4d3ab94bcf-acdb53e8a8fcff81 *** *** is nereids: 1 *** *** tablet id: 0 *** *** Aborted at 1739024048 (unix time) try "date -d @1739024048" if you are using GNU date *** *** Current BE git commitID: a733a56 *** *** SIGSEGV address not mapped to object (@0x0) received by PID 5943 (TID 7905 OR 0x7fd0b0c80640) from PID 0; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/common/signal_handler.h:421 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so 2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so 3# 0x00007FD4985D8520 in /lib/x86_64-linux-gnu/libc.so.6 4# doris::segment_v2::FileColumnIterator::next_batch(unsigned long*, COW<doris::vectorized::IColumn>::mutable_ptr<doris::vectorized::IColumn>&, bool*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/column_reader.cpp:1271 5# doris::segment_v2::SegmentIterator::_read_columns(std::vector<unsigned int, std::allocator<unsigned int> > const&, std::vector<COW<doris::vectorized::IColumn>::mutable_ptr<doris::vectorized::IColumn>, std::allocator<COW<doris::vectorized::IColumn>::mutable_ptr<doris::vectorized::IColumn> > >&, unsigned long) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1575 6# doris::segment_v2::SegmentIterator::_seek_and_peek(unsigned int) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1253 7# doris::segment_v2::SegmentIterator::_lookup_ordinal_from_sk_index(doris::RowCursor const&, bool, unsigned int, unsigned int*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1148 8# doris::segment_v2::SegmentIterator::_get_row_ranges_by_keys() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:432 9# doris::segment_v2::SegmentIterator::_lazy_init() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:382 10# doris::segment_v2::SegmentIterator::_next_batch_internal(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2012 11# doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1920 12# doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/lazy_init_segment_iterator.h:45 13# doris::vectorized::VUnionIterator::next_batch(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vgeneric_iterators.cpp:405 14# doris::BetaRowsetReader::next_block(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/beta_rowset_reader.cpp:376 15# doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:509 16# doris::vectorized::VCollectIterator::Level0Iterator::init(bool) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:462 17# doris::vectorized::VCollectIterator::build_heap(std::vector<std::shared_ptr<doris::RowsetReader>, std::allocator<std::shared_ptr<doris::RowsetReader> > >&) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:126 18# doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/block_reader.cpp:148 19# doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/block_reader.cpp:222 20# doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:232 21# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:221 22# std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()apache#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291 23# doris::ThreadPool::dispatch_thread() in /mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be 24# doris::Thread::supervise_thread(void*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/thread.cpp:499 25# start_thread at ./nptl/pthread_create.c:442 26# 0x00007FD4986BC850 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:83
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
*** Query id: b7482b4d3ab94bcf-acdb53e8a8fcff81 ***
*** is nereids: 1 ***
*** tablet id: 0 ***
*** Aborted at 1739024048 (unix time) try "date -d @1739024048" if you are using GNU date ***
*** Current BE git commitID: a733a56 ***
*** SIGSEGV address not mapped to object (@0x0) received by PID 5943 (TID 7905 OR 0x7fd0b0c80640) from PID 0; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/common/signal_handler.h:421
1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
3# 0x00007FD4985D8520 in /lib/x86_64-linux-gnu/libc.so.6
4# doris::segment_v2::FileColumnIterator::next_batch(unsigned long*, COWdoris::vectorized::IColumn::mutable_ptrdoris::vectorized::IColumn&, bool*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/column_reader.cpp:1271
5# doris::segment_v2::SegmentIterator::_read_columns(std::vector<unsigned int, std::allocator > const&, std::vector<COWdoris::vectorized::IColumn::mutable_ptrdoris::vectorized::IColumn, std::allocator<COWdoris::vectorized::IColumn::mutable_ptrdoris::vectorized::IColumn > >&, unsigned long) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1575
6# doris::segment_v2::SegmentIterator::_seek_and_peek(unsigned int) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1253
7# doris::segment_v2::SegmentIterator::_lookup_ordinal_from_sk_index(doris::RowCursor const&, bool, unsigned int, unsigned int*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1148
8# doris::segment_v2::SegmentIterator::_get_row_ranges_by_keys() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:432
9# doris::segment_v2::SegmentIterator::_lazy_init() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:382
10# doris::segment_v2::SegmentIterator::_next_batch_internal(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2012
11# doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1920
12# doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/lazy_init_segment_iterator.h:45
13# doris::vectorized::VUnionIterator::next_batch(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vgeneric_iterators.cpp:405
14# doris::BetaRowsetReader::next_block(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/beta_rowset_reader.cpp:376
15# doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:509
16# doris::vectorized::VCollectIterator::Level0Iterator::init(bool) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:462
17# doris::vectorized::VCollectIterator::build_heap(std::vector<std::shared_ptrdoris::RowsetReader, std::allocator<std::shared_ptrdoris::RowsetReader > >&) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:126
18# doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/block_reader.cpp:148
19# doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/block_reader.cpp:222
20# doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:232
21# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptrdoris::vectorized::ScannerContext, std::shared_ptrdoris::vectorized::ScanTask) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:221
22# std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptrdoris::vectorized::ScannerContext, std::shared_ptrdoris::vectorized::ScanTask)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
23# doris::ThreadPool::dispatch_thread() in /mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
24# doris::Thread::supervise_thread(void*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/thread.cpp:499
25# start_thread at ./nptl/pthread_create.c:442
26# 0x00007FD4986BC850 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:83
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)