Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

branch-3.0: [feat](binlog) Support getting binlogs in batch #47557 #47638

Merged
merged 2 commits into from
Feb 19, 2025

Conversation

github-actions[bot]
Copy link
Contributor

@github-actions github-actions bot commented Feb 8, 2025

Cherry-picked from #47557

@github-actions github-actions bot requested a review from dataroaring as a code owner February 8, 2025 06:29
@w41ter
Copy link
Contributor

w41ter commented Feb 10, 2025

run buildall

@w41ter w41ter force-pushed the auto-pick-47557-branch-3.0 branch from bf03513 to cb54f5d Compare February 10, 2025 02:14
@w41ter
Copy link
Contributor

w41ter commented Feb 10, 2025

run buildall

@doris-robot
Copy link

TeamCity cloud ut coverage result:
Function Coverage: 82.36% (1060/1287)
Line Coverage: 65.88% (17550/26640)
Region Coverage: 65.39% (8653/13233)
Branch Coverage: 55.24% (4661/8438)
Coverage Report: http://coverage.selectdb-in.cc/coverage/cb54f5d48e8795f4f70b4d10d3c3efd35fcd88b4_cb54f5d48e8795f4f70b4d10d3c3efd35fcd88b4_cloud/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 41293 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cb54f5d48e8795f4f70b4d10d3c3efd35fcd88b4, data reload: false

------ Round 1 ----------------------------------
q1	17565	7399	7267	7267
q2	2069	185	180	180
q3	10519	1090	1134	1090
q4	10469	779	724	724
q5	7750	2929	2855	2855
q6	231	148	147	147
q7	981	627	612	612
q8	9370	1988	2061	1988
q9	6642	6473	6448	6448
q10	7039	2319	2359	2319
q11	463	269	267	267
q12	408	213	216	213
q13	17792	2996	3041	2996
q14	234	211	220	211
q15	575	512	525	512
q16	675	591	608	591
q17	996	580	558	558
q18	7595	6795	6816	6795
q19	1392	1118	1074	1074
q20	488	211	199	199
q21	4050	3243	3322	3243
q22	1103	1012	1004	1004
Total cold run time: 108406 ms
Total hot run time: 41293 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7218	7214	7228	7214
q2	327	228	230	228
q3	2971	2931	2953	2931
q4	2124	1934	1811	1811
q5	5736	5752	5740	5740
q6	226	142	145	142
q7	2280	1826	1829	1826
q8	3365	3629	3527	3527
q9	8890	9033	8909	8909
q10	3644	3585	3578	3578
q11	600	515	521	515
q12	798	585	599	585
q13	10026	3193	3208	3193
q14	319	295	280	280
q15	573	537	524	524
q16	711	652	640	640
q17	1842	1613	1589	1589
q18	8394	7842	7688	7688
q19	1691	1612	1490	1490
q20	2109	1918	1866	1866
q21	5645	5352	5366	5352
q22	1182	1042	1046	1042
Total cold run time: 70671 ms
Total hot run time: 60670 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198540 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cb54f5d48e8795f4f70b4d10d3c3efd35fcd88b4, data reload: false

query1	1313	935	932	932
query2	6275	2093	2010	2010
query3	10940	4543	4386	4386
query4	66683	28563	23814	23814
query5	4931	446	470	446
query6	433	188	179	179
query7	5548	309	318	309
query8	310	217	229	217
query9	8835	2703	2708	2703
query10	461	276	256	256
query11	17532	15178	15933	15178
query12	174	99	103	99
query13	1508	456	442	442
query14	10854	7967	7902	7902
query15	198	187	176	176
query16	7128	510	499	499
query17	1104	615	598	598
query18	1877	326	321	321
query19	212	161	179	161
query20	118	114	116	114
query21	203	102	103	102
query22	4685	4403	4469	4403
query23	34572	34156	33791	33791
query24	6070	2936	2919	2919
query25	550	444	422	422
query26	658	171	176	171
query27	2224	345	357	345
query28	4255	2496	2499	2496
query29	710	464	440	440
query30	282	176	165	165
query31	1008	815	825	815
query32	66	55	55	55
query33	424	290	287	287
query34	943	499	525	499
query35	842	734	731	731
query36	1089	953	963	953
query37	122	75	72	72
query38	4121	4069	4099	4069
query39	1687	1471	1490	1471
query40	205	106	100	100
query41	49	49	47	47
query42	118	104	107	104
query43	504	500	490	490
query44	1197	836	821	821
query45	189	169	175	169
query46	1169	737	709	709
query47	2053	1915	1922	1915
query48	505	387	385	385
query49	724	396	399	396
query50	872	422	417	417
query51	7310	7288	7250	7250
query52	102	90	87	87
query53	260	182	187	182
query54	552	459	457	457
query55	81	77	77	77
query56	266	242	242	242
query57	1205	1092	1089	1089
query58	205	200	208	200
query59	3044	3021	2947	2947
query60	278	255	252	252
query61	108	109	107	107
query62	865	751	732	732
query63	225	190	191	190
query64	1405	707	682	682
query65	3257	3208	3187	3187
query66	636	294	308	294
query67	15736	15590	15337	15337
query68	4042	560	580	560
query69	440	270	262	262
query70	1176	1136	1139	1136
query71	320	259	257	257
query72	6370	3989	4035	3989
query73	751	352	346	346
query74	10591	9043	9273	9043
query75	3375	2620	2641	2620
query76	1969	1089	1001	1001
query77	538	274	272	272
query78	10657	9655	9612	9612
query79	1450	607	580	580
query80	892	427	424	424
query81	523	241	240	240
query82	1269	115	125	115
query83	170	158	161	158
query84	282	78	81	78
query85	884	296	310	296
query86	360	287	300	287
query87	4443	4247	4271	4247
query88	3674	2404	2450	2404
query89	424	291	294	291
query90	2029	182	183	182
query91	178	152	153	152
query92	71	52	50	50
query93	1912	550	558	550
query94	842	299	287	287
query95	351	256	252	252
query96	632	278	289	278
query97	3328	3170	3133	3133
query98	219	219	203	203
query99	1713	1436	1422	1422
Total cold run time: 319940 ms
Total hot run time: 198540 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.5 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cb54f5d48e8795f4f70b4d10d3c3efd35fcd88b4, data reload: false

query1	0.03	0.04	0.03
query2	0.09	0.04	0.05
query3	0.23	0.06	0.05
query4	1.65	0.08	0.08
query5	0.51	0.51	0.51
query6	1.14	0.75	0.75
query7	0.02	0.02	0.02
query8	0.08	0.04	0.05
query9	0.55	0.50	0.49
query10	0.55	0.57	0.56
query11	0.18	0.12	0.12
query12	0.16	0.14	0.13
query13	0.62	0.59	0.59
query14	2.88	2.84	2.86
query15	0.92	0.84	0.83
query16	0.39	0.40	0.38
query17	1.03	0.99	1.04
query18	0.18	0.18	0.18
query19	1.98	1.79	2.05
query20	0.01	0.01	0.01
query21	15.36	0.66	0.66
query22	4.18	7.72	1.67
query23	18.27	1.39	1.38
query24	2.19	0.24	0.22
query25	0.14	0.08	0.08
query26	0.27	0.19	0.18
query27	0.09	0.08	0.08
query28	13.24	1.19	1.13
query29	12.65	3.41	3.37
query30	0.25	0.06	0.06
query31	2.85	0.42	0.40
query32	3.22	0.48	0.49
query33	3.03	3.05	3.05
query34	16.82	4.57	4.55
query35	4.64	4.65	4.64
query36	0.66	0.51	0.50
query37	0.21	0.17	0.16
query38	0.16	0.16	0.16
query39	0.05	0.04	0.04
query40	0.17	0.14	0.12
query41	0.09	0.05	0.06
query42	0.07	0.06	0.05
query43	0.05	0.04	0.04
Total cold run time: 111.86 s
Total hot run time: 33.5 s

@hello-stephen
Copy link
Contributor

run buildall

@doris-robot
Copy link

TeamCity cloud ut coverage result:
Function Coverage: 82.36% (1060/1287)
Line Coverage: 65.83% (17537/26640)
Region Coverage: 65.36% (8649/13233)
Branch Coverage: 55.24% (4661/8438)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4a767383f711edbaaa98ab2c4114b1d290ef9044_4a767383f711edbaaa98ab2c4114b1d290ef9044_cloud/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 41432 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4a767383f711edbaaa98ab2c4114b1d290ef9044, data reload: false

------ Round 1 ----------------------------------
q1	17581	7565	7306	7306
q2	2067	177	174	174
q3	10628	1190	1203	1190
q4	10518	789	795	789
q5	7745	2915	2864	2864
q6	238	151	151	151
q7	1004	634	630	630
q8	9373	1999	2098	1999
q9	6579	6468	6508	6468
q10	7016	2353	2331	2331
q11	475	272	268	268
q12	407	221	216	216
q13	17787	2998	3026	2998
q14	256	222	208	208
q15	576	528	535	528
q16	643	590	620	590
q17	988	592	628	592
q18	7430	6734	6720	6720
q19	1376	1098	1084	1084
q20	482	210	206	206
q21	4141	3350	3141	3141
q22	1092	984	979	979
Total cold run time: 108402 ms
Total hot run time: 41432 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7276	7153	7217	7153
q2	334	242	232	232
q3	2997	2928	2937	2928
q4	2070	1882	1865	1865
q5	5717	5754	5758	5754
q6	237	155	160	155
q7	2281	1892	1837	1837
q8	3418	3614	3459	3459
q9	8944	8954	8870	8870
q10	3590	3577	3611	3577
q11	612	508	524	508
q12	829	647	593	593
q13	9827	3140	3182	3140
q14	304	278	283	278
q15	581	521	538	521
q16	711	662	639	639
q17	1860	1630	1615	1615
q18	8406	7836	7747	7747
q19	1684	1632	1617	1617
q20	2113	1859	1846	1846
q21	5618	5389	5584	5389
q22	1178	1036	1034	1034
Total cold run time: 70587 ms
Total hot run time: 60757 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 199544 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4a767383f711edbaaa98ab2c4114b1d290ef9044, data reload: false

query1	1292	953	921	921
query2	6223	2072	2057	2057
query3	10976	4315	4591	4315
query4	66639	29222	23959	23959
query5	5008	463	452	452
query6	413	175	172	172
query7	5718	325	322	322
query8	313	230	223	223
query9	9512	2721	2728	2721
query10	466	285	261	261
query11	17522	15802	15765	15765
query12	158	101	98	98
query13	1617	454	424	424
query14	10908	7386	7877	7386
query15	211	183	191	183
query16	7116	510	465	465
query17	1122	588	603	588
query18	1882	334	322	322
query19	254	169	168	168
query20	121	124	115	115
query21	223	102	106	102
query22	4753	4691	4508	4508
query23	34399	34315	34436	34315
query24	6186	3003	3036	3003
query25	526	415	432	415
query26	695	181	180	180
query27	1752	359	360	359
query28	4661	2506	2469	2469
query29	754	506	432	432
query30	241	169	163	163
query31	1028	807	853	807
query32	68	60	57	57
query33	429	299	295	295
query34	937	516	514	514
query35	868	744	725	725
query36	1067	996	980	980
query37	119	71	70	70
query38	4123	4048	3973	3973
query39	1677	1474	1462	1462
query40	209	103	100	100
query41	50	45	51	45
query42	122	109	102	102
query43	535	509	498	498
query44	1192	820	844	820
query45	184	170	171	170
query46	1140	750	755	750
query47	2055	1951	1967	1951
query48	481	385	404	385
query49	731	421	395	395
query50	882	440	436	436
query51	7364	7113	7103	7103
query52	102	91	90	90
query53	268	184	197	184
query54	561	462	448	448
query55	80	81	79	79
query56	269	240	240	240
query57	1218	1143	1072	1072
query58	217	209	214	209
query59	3109	2877	2842	2842
query60	300	256	244	244
query61	106	104	107	104
query62	863	708	749	708
query63	215	191	199	191
query64	1408	672	636	636
query65	3304	3233	3164	3164
query66	709	303	320	303
query67	15987	15884	15623	15623
query68	4264	592	616	592
query69	433	267	264	264
query70	1196	1142	1056	1056
query71	347	261	263	261
query72	6446	3974	3982	3974
query73	789	344	350	344
query74	10351	9156	9107	9107
query75	3307	2669	2680	2669
query76	1837	1025	1076	1025
query77	467	272	273	272
query78	10701	9694	9615	9615
query79	1762	619	626	619
query80	1013	442	421	421
query81	553	240	235	235
query82	213	130	116	116
query83	197	146	146	146
query84	277	81	74	74
query85	965	305	290	290
query86	434	299	297	297
query87	4486	4314	4365	4314
query88	4193	2370	2331	2331
query89	410	294	303	294
query90	2045	189	187	187
query91	199	148	151	148
query92	65	50	51	50
query93	2182	572	563	563
query94	897	294	282	282
query95	363	261	261	261
query96	621	278	281	278
query97	3289	3255	3192	3192
query98	221	207	197	197
query99	1690	1463	1432	1432
Total cold run time: 321851 ms
Total hot run time: 199544 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.76 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4a767383f711edbaaa98ab2c4114b1d290ef9044, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.05
query3	0.23	0.06	0.05
query4	1.66	0.09	0.09
query5	0.53	0.53	0.51
query6	1.14	0.74	0.74
query7	0.02	0.01	0.01
query8	0.05	0.05	0.06
query9	0.55	0.50	0.51
query10	0.56	0.54	0.58
query11	0.16	0.12	0.11
query12	0.17	0.12	0.12
query13	0.61	0.60	0.60
query14	2.87	2.89	2.87
query15	0.92	0.84	0.85
query16	0.38	0.38	0.38
query17	1.08	1.05	1.01
query18	0.20	0.19	0.18
query19	1.93	1.84	2.02
query20	0.02	0.01	0.01
query21	15.37	0.67	0.67
query22	4.02	7.39	1.94
query23	18.28	1.36	1.27
query24	2.20	0.22	0.22
query25	0.15	0.09	0.09
query26	0.27	0.18	0.19
query27	0.08	0.08	0.08
query28	13.33	1.16	1.14
query29	12.68	3.41	3.46
query30	0.24	0.06	0.07
query31	2.85	0.41	0.40
query32	3.22	0.50	0.49
query33	3.01	3.01	3.04
query34	17.16	4.57	4.56
query35	4.59	4.62	4.62
query36	0.66	0.48	0.52
query37	0.20	0.16	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.05
query40	0.16	0.13	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 112.08 s
Total hot run time: 33.76 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 323f8c9 into branch-3.0 Feb 19, 2025
25 of 29 checks passed
@github-actions github-actions bot deleted the auto-pick-47557-branch-3.0 branch February 19, 2025 02:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants