Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[enhance](PrefetchReader) Make the prefetch timeout one config #27371

Merged
merged 1 commit into from
Nov 24, 2023
Merged

[enhance](PrefetchReader) Make the prefetch timeout one config #27371

merged 1 commit into from
Nov 24, 2023

Conversation

ByteYue
Copy link
Contributor

@ByteYue ByteYue commented Nov 21, 2023

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@ByteYue
Copy link
Contributor Author

ByteYue commented Nov 21, 2023

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.52% (8446/23125)
Line Coverage: 28.85% (68673/238007)
Region Coverage: 27.81% (35507/127675)
Branch Coverage: 24.57% (18110/73712)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c6020c41a0fee4092ade50ee2f5f4493c89a12ca_c6020c41a0fee4092ade50ee2f5f4493c89a12ca/report/index.html

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' 
Tpch sf100 test result on commit c6020c41a0fee4092ade50ee2f5f4493c89a12ca, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4905	4658	4666	4658
q2	360	143	167	143
q3	2020	1946	1961	1946
q4	1380	1260	1225	1225
q5	3971	3939	3958	3939
q6	248	132	136	132
q7	1424	880	911	880
q8	2735	2753	2746	2746
q9	9838	9588	9604	9588
q10	3472	3520	3523	3520
q11	385	249	258	249
q12	446	297	305	297
q13	4558	3841	3800	3800
q14	314	283	292	283
q15	588	540	522	522
q16	664	586	584	584
q17	1120	968	934	934
q18	7756	7301	7342	7301
q19	1657	1699	1673	1673
q20	520	329	310	310
q21	4382	3930	3985	3930
q22	472	375	380	375
Total cold run time: 53215 ms
Total hot run time: 49035 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4592	4563	4561	4561
q2	348	223	261	223
q3	4032	4008	3994	3994
q4	2702	2688	2696	2688
q5	9654	9656	9701	9656
q6	238	125	125	125
q7	3015	2495	2461	2461
q8	4406	4410	4385	4385
q9	13151	13158	13073	13073
q10	4103	4157	4185	4157
q11	832	646	736	646
q12	986	788	817	788
q13	4301	3623	3567	3567
q14	378	348	344	344
q15	586	522	514	514
q16	742	685	671	671
q17	3842	3863	3880	3863
q18	9594	9160	9095	9095
q19	1842	1769	1759	1759
q20	2393	2084	2069	2069
q21	8761	8699	8465	8465
q22	946	799	827	799
Total cold run time: 81444 ms
Total hot run time: 77903 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.7 seconds
stream load tsv: 571 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 34 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.4 seconds inserted 10000000 Rows, about 340K ops/s
storage size: 17098912668 Bytes

dataroaring
dataroaring approved these changes Nov 24, 2023
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 24, 2023
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

sollhui
sollhui approved these changes Nov 24, 2023
View reviewed changes
Copy link
Contributor

@sollhui sollhui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 0a8e3d2 into apache:master Nov 24, 2023
ByteYue added a commit to ByteYue/doris that referenced this pull request Nov 24, 2023
@ByteYue
[enhance](PrefetchReader) Make the prefetch timeout one config (apach…
b63cf50 
…e#27371)
@ByteYue ByteYue mentioned this pull request Nov 24, 2023
Merged
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 28, 2023
@ByteYue @seawinde
[enhance](PrefetchReader) Make the prefetch timeout one config (apach…
a048e90 
…e#27371)
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
@ByteYue
[enhance](PrefetchReader) Make the prefetch timeout one config (apach…
6540bf5 
…e#27371)
xiaokang pushed a commit that referenced this pull request Dec 22, 2023
@ByteYue
[enhance](PrefetchReader) Make the prefetch timeout one config (#27371)…
2bc39d5 
… (#27530)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sollhui sollhui sollhui approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.4-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@ByteYue @doris-robot @sollhui @dataroaring @xiaokang