Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

branch-3.0: [Test](Export) add some debug logs for export #47400 #47474

Open
wants to merge 2 commits into
base: branch-3.0
Choose a base branch
from

Conversation

morningman
Copy link
Contributor

bp #47400

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman
Copy link
Contributor Author

DO NOT MERGE

@morningman
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40971 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3b19ca7380f17c0e3077bb030b8185a91d2b3c89, data reload: false

------ Round 1 ----------------------------------
q1	17579	7320	7228	7228
q2	2060	162	164	162
q3	10617	1075	1233	1075
q4	10525	724	752	724
q5	7740	2840	2827	2827
q6	233	145	143	143
q7	957	605	606	605
q8	9350	1952	2004	1952
q9	6533	6358	6363	6358
q10	7043	2305	2329	2305
q11	464	263	261	261
q12	404	215	209	209
q13	17877	3053	3005	3005
q14	236	210	207	207
q15	573	525	532	525
q16	696	595	604	595
q17	961	546	580	546
q18	7171	6761	6727	6727
q19	1394	1077	1081	1077
q20	454	208	198	198
q21	4082	3268	3246	3246
q22	1083	997	996	996
Total cold run time: 108032 ms
Total hot run time: 40971 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7226	7240	7157	7157
q2	323	237	228	228
q3	2901	2938	2981	2938
q4	2010	1881	1818	1818
q5	5699	5735	5737	5735
q6	229	145	144	144
q7	2235	1832	1846	1832
q8	3379	3531	3484	3484
q9	8824	8839	8890	8839
q10	3591	3578	3534	3534
q11	604	510	491	491
q12	800	624	624	624
q13	9015	3125	3171	3125
q14	291	283	287	283
q15	580	532	525	525
q16	685	655	654	654
q17	1833	1613	1584	1584
q18	8173	7674	7558	7558
q19	1639	1662	1520	1520
q20	2107	1871	1886	1871
q21	5430	5298	5407	5298
q22	1140	1069	1029	1029
Total cold run time: 68714 ms
Total hot run time: 60271 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198758 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3b19ca7380f17c0e3077bb030b8185a91d2b3c89, data reload: false

query1	1327	954	949	949
query2	6243	1998	2053	1998
query3	10830	4166	4372	4166
query4	67654	28792	23944	23944
query5	4978	449	434	434
query6	403	174	168	168
query7	5682	313	315	313
query8	296	215	226	215
query9	9361	2689	2700	2689
query10	468	265	259	259
query11	18100	15728	15804	15728
query12	154	102	105	102
query13	1571	442	431	431
query14	10374	7360	7239	7239
query15	221	193	188	188
query16	7164	471	477	471
query17	1117	601	607	601
query18	2006	330	315	315
query19	273	156	165	156
query20	120	115	116	115
query21	206	106	104	104
query22	4768	4334	4417	4334
query23	34434	34072	34283	34072
query24	6288	3011	2984	2984
query25	530	413	391	391
query26	652	170	180	170
query27	1897	360	361	360
query28	3829	2463	2480	2463
query29	676	441	433	433
query30	251	157	156	156
query31	1027	828	832	828
query32	67	55	54	54
query33	395	291	285	285
query34	928	521	542	521
query35	834	740	746	740
query36	1105	942	999	942
query37	129	79	77	77
query38	4206	4002	3983	3983
query39	1541	1467	1458	1458
query40	197	97	104	97
query41	48	47	51	47
query42	110	102	100	100
query43	541	494	496	494
query44	1196	842	839	839
query45	186	168	167	167
query46	1157	737	773	737
query47	2052	1969	1921	1921
query48	482	381	403	381
query49	750	411	411	411
query50	836	425	425	425
query51	7411	7266	7140	7140
query52	100	87	91	87
query53	250	183	183	183
query54	585	469	455	455
query55	76	79	75	75
query56	266	251	242	242
query57	1209	1111	1102	1102
query58	222	215	213	213
query59	3352	2978	2970	2970
query60	270	259	248	248
query61	114	111	104	104
query62	844	717	731	717
query63	219	189	189	189
query64	1403	672	649	649
query65	3263	3205	3242	3205
query66	711	299	302	299
query67	15799	15895	15533	15533
query68	3311	586	588	586
query69	435	271	268	268
query70	1173	1125	1113	1113
query71	375	261	263	261
query72	6283	4391	4063	4063
query73	748	352	351	351
query74	10293	9048	9061	9048
query75	3380	2642	2689	2642
query76	1514	1105	1115	1105
query77	485	311	284	284
query78	10571	9949	9620	9620
query79	2096	605	608	605
query80	1297	422	428	422
query81	519	249	235	235
query82	1012	123	115	115
query83	235	145	145	145
query84	283	73	84	73
query85	1005	300	289	289
query86	419	279	307	279
query87	4488	4284	4225	4225
query88	3848	2388	2351	2351
query89	420	290	294	290
query90	1929	186	186	186
query91	182	149	148	148
query92	62	50	48	48
query93	2311	561	550	550
query94	876	295	291	291
query95	354	256	261	256
query96	610	279	282	279
query97	3327	3229	3156	3156
query98	230	226	213	213
query99	1706	1470	1395	1395
Total cold run time: 321531 ms
Total hot run time: 198758 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.9 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3b19ca7380f17c0e3077bb030b8185a91d2b3c89, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.62	0.10	0.11
query5	0.52	0.51	0.52
query6	1.14	0.73	0.73
query7	0.02	0.02	0.02
query8	0.03	0.03	0.03
query9	0.55	0.51	0.50
query10	0.56	0.55	0.55
query11	0.15	0.10	0.11
query12	0.14	0.13	0.12
query13	0.61	0.60	0.59
query14	2.72	2.81	2.74
query15	0.89	0.82	0.84
query16	0.40	0.38	0.39
query17	1.04	1.03	1.04
query18	0.24	0.22	0.22
query19	2.00	1.83	1.97
query20	0.01	0.02	0.03
query21	15.36	0.57	0.56
query22	2.76	2.31	1.99
query23	16.81	1.05	0.84
query24	2.95	0.89	1.59
query25	0.21	0.24	0.06
query26	0.47	0.14	0.13
query27	0.06	0.03	0.04
query28	10.43	1.11	1.07
query29	12.60	3.24	3.26
query30	0.24	0.06	0.06
query31	2.87	0.39	0.38
query32	3.24	0.47	0.46
query33	2.99	3.02	3.01
query34	16.89	4.45	4.47
query35	4.50	4.55	4.51
query36	0.66	0.48	0.49
query37	0.09	0.06	0.06
query38	0.05	0.03	0.03
query39	0.04	0.02	0.02
query40	0.17	0.13	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.04
Total cold run time: 106.54 s
Total hot run time: 32.9 s

@morningman
Copy link
Contributor Author

run p0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants