Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[refactor]Separate StringOP and SubstringUtil #48054

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

[refactor]Separate StringOP and SubstringUtil #48054

wants to merge 3 commits into from

Conversation

lzyy2024
Copy link
Contributor

What problem does this PR solve?

Separate StringOP and SubstringUtil from /doris/be/src/vec/functions/function_string.cpp

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

StringOP and SubstringUtil are very common, and include function_string.cpp contains many other unnecessary functions

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Feb 19, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@lzyy2024 lzyy2024 force-pushed the Separate-StringOP-and-SubstringUtil branch from c196553 to 0495acb Compare February 19, 2025 05:03
@lzyy2024 lzyy2024 changed the title Separate StringOP and SubstringUtil [refactor]Separate StringOP and SubstringUtil Feb 19, 2025
@lzyy2024
Copy link
Contributor Author

run buildall

@lzyy2024 lzyy2024 force-pushed the Separate-StringOP-and-SubstringUtil branch from 17e1a9f to 4dc3625 Compare February 19, 2025 09:40
@lzyy2024
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31389 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e406e92fe01f2af37d2fd8b1f1e6c02cb9ee9809, data reload: false

------ Round 1 ----------------------------------
q1	17617	5506	5110	5110
q2	2042	298	171	171
q3	10423	1209	740	740
q4	10200	1013	526	526
q5	7504	2437	2325	2325
q6	186	162	136	136
q7	914	744	596	596
q8	9299	1305	1164	1164
q9	4794	4682	4571	4571
q10	6805	2307	1863	1863
q11	481	274	260	260
q12	357	376	233	233
q13	17762	3685	3104	3104
q14	222	230	208	208
q15	508	469	473	469
q16	611	621	579	579
q17	580	847	327	327
q18	6508	6264	6070	6070
q19	1217	946	540	540
q20	316	325	191	191
q21	2742	2264	1912	1912
q22	364	335	294	294
Total cold run time: 101452 ms
Total hot run time: 31389 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5155	5110	5167	5110
q2	238	329	231	231
q3	2166	2715	2294	2294
q4	1401	1799	1365	1365
q5	4256	4133	4178	4133
q6	210	164	125	125
q7	1862	1814	1663	1663
q8	2604	2624	2602	2602
q9	7256	7231	7149	7149
q10	3041	3185	2801	2801
q11	560	525	483	483
q12	685	769	605	605
q13	3572	3894	3371	3371
q14	295	293	278	278
q15	495	465	477	465
q16	652	666	625	625
q17	1118	1617	1315	1315
q18	7548	7341	7306	7306
q19	811	1008	1088	1008
q20	1958	2007	1876	1876
q21	5407	4827	4780	4780
q22	584	585	536	536
Total cold run time: 51874 ms
Total hot run time: 50121 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190455 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e406e92fe01f2af37d2fd8b1f1e6c02cb9ee9809, data reload: false

query1	1334	969	952	952
query2	6241	1884	1827	1827
query3	10983	4443	4365	4365
query4	54759	24668	23293	23293
query5	5254	586	488	488
query6	342	206	188	188
query7	5019	508	297	297
query8	322	243	242	242
query9	5879	2550	2548	2548
query10	397	309	268	268
query11	15197	15010	14791	14791
query12	155	112	111	111
query13	1076	512	370	370
query14	10710	6300	6368	6300
query15	191	212	186	186
query16	7033	647	468	468
query17	1069	706	551	551
query18	1539	397	314	314
query19	195	188	163	163
query20	127	123	129	123
query21	207	120	105	105
query22	4288	4562	4269	4269
query23	34092	33233	33401	33233
query24	5788	2498	2486	2486
query25	475	473	411	411
query26	668	292	156	156
query27	1707	481	341	341
query28	2924	2497	2451	2451
query29	585	590	415	415
query30	220	190	160	160
query31	866	864	843	843
query32	68	62	57	57
query33	451	369	304	304
query34	801	858	513	513
query35	789	836	768	768
query36	962	994	907	907
query37	118	95	81	81
query38	4505	4394	4348	4348
query39	1556	1460	1423	1423
query40	217	114	103	103
query41	53	50	50	50
query42	121	109	108	108
query43	505	510	476	476
query44	1358	837	834	834
query45	182	180	170	170
query46	901	1083	674	674
query47	1791	1841	1804	1804
query48	402	461	321	321
query49	699	499	438	438
query50	732	745	442	442
query51	4346	4311	4251	4251
query52	105	102	100	100
query53	237	265	196	196
query54	494	533	437	437
query55	88	87	83	83
query56	277	276	274	274
query57	1175	1195	1106	1106
query58	255	243	240	240
query59	2759	2749	2729	2729
query60	310	300	271	271
query61	141	135	140	135
query62	741	741	693	693
query63	245	208	202	202
query64	1479	1049	666	666
query65	3233	3137	3123	3123
query66	771	391	322	322
query67	16031	15628	15403	15403
query68	5511	807	539	539
query69	533	299	259	259
query70	1190	1153	1113	1113
query71	434	293	258	258
query72	5803	3770	3831	3770
query73	1196	755	351	351
query74	9401	9153	9110	9110
query75	3194	3159	2709	2709
query76	3915	1163	738	738
query77	547	360	281	281
query78	10004	10007	9422	9422
query79	2393	810	629	629
query80	621	597	448	448
query81	504	277	244	244
query82	480	132	99	99
query83	182	172	148	148
query84	287	98	77	77
query85	760	347	304	304
query86	383	318	295	295
query87	4493	4448	4333	4333
query88	3773	2194	2184	2184
query89	424	333	295	295
query90	1705	198	197	197
query91	130	137	144	137
query92	75	58	59	58
query93	1931	1030	598	598
query94	716	412	307	307
query95	346	274	259	259
query96	491	537	280	280
query97	2826	2859	2757	2757
query98	235	202	196	196
query99	1627	1408	1293	1293
Total cold run time: 294650 ms
Total hot run time: 190455 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.22 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e406e92fe01f2af37d2fd8b1f1e6c02cb9ee9809, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.03	0.03
query3	0.24	0.06	0.06
query4	1.62	0.10	0.10
query5	0.40	0.41	0.42
query6	1.17	0.66	0.66
query7	0.02	0.02	0.01
query8	0.04	0.04	0.03
query9	0.59	0.52	0.53
query10	0.58	0.58	0.57
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.62	0.60	0.60
query14	2.71	2.74	2.75
query15	0.92	0.87	0.86
query16	0.38	0.36	0.39
query17	1.07	1.08	1.01
query18	0.21	0.20	0.19
query19	1.94	1.76	2.03
query20	0.01	0.01	0.02
query21	15.35	0.88	0.54
query22	0.75	1.19	0.73
query23	14.96	1.35	0.58
query24	12.27	1.03	0.35
query25	0.36	0.22	0.14
query26	1.06	0.19	0.14
query27	0.05	0.04	0.05
query28	5.94	0.79	0.44
query29	12.57	3.96	3.32
query30	0.26	0.09	0.06
query31	2.84	0.58	0.38
query32	3.22	0.54	0.48
query33	2.99	3.03	3.02
query34	15.66	5.14	4.51
query35	4.53	4.51	4.51
query36	0.67	0.50	0.49
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.08	0.03	0.02
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.89 s
Total hot run time: 30.22 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 43.83% (11636/26546)
Line Coverage: 33.74% (97526/289035)
Region Coverage: 32.84% (49933/152033)
Branch Coverage: 28.57% (25105/87886)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e406e92fe01f2af37d2fd8b1f1e6c02cb9ee9809_e406e92fe01f2af37d2fd8b1f1e6c02cb9ee9809/report/index.html

@lzyy2024
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31304 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e406e92fe01f2af37d2fd8b1f1e6c02cb9ee9809, data reload: false

------ Round 1 ----------------------------------
q1	17647	5326	5121	5121
q2	2042	292	169	169
q3	10423	1303	723	723
q4	10201	1022	546	546
q5	7563	2468	2258	2258
q6	191	167	132	132
q7	911	742	603	603
q8	9305	1338	1105	1105
q9	4899	4465	4605	4465
q10	6809	2314	1890	1890
q11	466	275	273	273
q12	349	349	219	219
q13	17756	3705	3139	3139
q14	224	222	203	203
q15	511	460	435	435
q16	612	601	578	578
q17	589	857	357	357
q18	6485	6318	6127	6127
q19	1084	947	538	538
q20	309	327	191	191
q21	2816	2146	1932	1932
q22	365	321	300	300
Total cold run time: 101557 ms
Total hot run time: 31304 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5136	5132	5128	5128
q2	233	333	242	242
q3	2138	2618	2302	2302
q4	1369	1828	1338	1338
q5	4192	4109	4122	4109
q6	208	165	122	122
q7	1828	1806	1647	1647
q8	2583	2537	2509	2509
q9	7351	7042	7128	7042
q10	2957	3165	2741	2741
q11	588	525	488	488
q12	727	768	607	607
q13	3491	3990	3265	3265
q14	272	299	259	259
q15	519	468	467	467
q16	627	681	635	635
q17	1106	1601	1323	1323
q18	7526	7417	7300	7300
q19	777	761	790	761
q20	1951	2030	1857	1857
q21	5390	4978	4749	4749
q22	624	558	547	547
Total cold run time: 51593 ms
Total hot run time: 49438 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 184289 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e406e92fe01f2af37d2fd8b1f1e6c02cb9ee9809, data reload: false

query1	966	387	379	379
query2	6524	1840	1817	1817
query3	6795	218	212	212
query4	26601	23761	23780	23761
query5	5281	679	493	493
query6	297	189	171	171
query7	4608	497	292	292
query8	294	246	235	235
query9	8661	2535	2520	2520
query10	520	318	256	256
query11	15854	15139	14814	14814
query12	156	109	108	108
query13	1666	547	398	398
query14	9617	6212	6244	6212
query15	235	196	190	190
query16	7505	654	462	462
query17	1156	717	565	565
query18	1984	408	304	304
query19	189	188	160	160
query20	131	116	115	115
query21	211	123	102	102
query22	4159	4458	4462	4458
query23	34233	33371	32872	32872
query24	8344	2366	2385	2366
query25	503	444	409	409
query26	1237	279	148	148
query27	2672	494	337	337
query28	4247	2410	2385	2385
query29	716	538	406	406
query30	242	182	152	152
query31	947	839	779	779
query32	75	60	57	57
query33	548	352	290	290
query34	755	866	521	521
query35	771	817	741	741
query36	948	988	897	897
query37	112	103	72	72
query38	4117	4190	4201	4190
query39	1443	1403	1407	1403
query40	205	110	96	96
query41	56	51	49	49
query42	124	104	99	99
query43	495	489	473	473
query44	1285	793	797	793
query45	182	171	161	161
query46	858	1029	631	631
query47	1746	1801	1725	1725
query48	383	406	304	304
query49	795	506	419	419
query50	675	717	428	428
query51	4130	4148	4068	4068
query52	108	113	96	96
query53	228	248	204	204
query54	483	479	399	399
query55	82	85	81	81
query56	264	260	255	255
query57	1115	1137	1055	1055
query58	259	224	239	224
query59	2693	2716	2540	2540
query60	331	278	267	267
query61	135	124	120	120
query62	772	728	696	696
query63	226	188	185	185
query64	4261	993	651	651
query65	3198	3123	3137	3123
query66	1066	402	308	308
query67	16541	16313	15777	15777
query68	7974	766	526	526
query69	475	317	256	256
query70	1200	1135	1110	1110
query71	449	295	265	265
query72	5654	3741	3723	3723
query73	718	744	350	350
query74	9168	9100	8908	8908
query75	3202	3159	2662	2662
query76	3310	1166	727	727
query77	549	363	281	281
query78	10021	9994	9203	9203
query79	2306	817	605	605
query80	637	530	452	452
query81	506	272	246	246
query82	477	129	94	94
query83	169	170	153	153
query84	246	101	74	74
query85	752	348	295	295
query86	364	309	286	286
query87	4344	4638	4407	4407
query88	4172	2240	2188	2188
query89	384	321	279	279
query90	1991	187	191	187
query91	128	142	110	110
query92	77	65	57	57
query93	1897	1013	588	588
query94	694	381	292	292
query95	355	270	254	254
query96	489	562	272	272
query97	2798	2824	2705	2705
query98	236	207	204	204
query99	1332	1409	1254	1254
Total cold run time: 275090 ms
Total hot run time: 184289 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.36 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e406e92fe01f2af37d2fd8b1f1e6c02cb9ee9809, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.03	0.03
query3	0.23	0.07	0.07
query4	1.64	0.10	0.10
query5	0.42	0.40	0.40
query6	1.16	0.65	0.66
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.59	0.51	0.55
query10	0.58	0.59	0.57
query11	0.15	0.11	0.10
query12	0.14	0.11	0.13
query13	0.61	0.61	0.60
query14	2.66	2.74	2.68
query15	0.91	0.84	0.84
query16	0.38	0.37	0.38
query17	1.04	1.04	1.03
query18	0.21	0.19	0.19
query19	1.85	1.79	1.96
query20	0.01	0.01	0.02
query21	15.35	0.89	0.54
query22	0.76	1.11	0.99
query23	14.68	1.37	0.66
query24	11.56	1.07	0.37
query25	0.31	0.14	0.10
query26	0.83	0.19	0.15
query27	0.05	0.06	0.04
query28	6.06	0.77	0.42
query29	12.54	3.95	3.28
query30	0.24	0.08	0.06
query31	2.83	0.60	0.38
query32	3.23	0.55	0.48
query33	3.00	2.99	3.11
query34	15.83	5.09	4.48
query35	4.55	4.51	4.52
query36	0.67	0.49	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.02	0.02	0.02
query40	0.16	0.13	0.13
query41	0.07	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 105.71 s
Total hot run time: 30.36 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 43.83% (11636/26546)
Line Coverage: 33.75% (97559/289047)
Region Coverage: 32.84% (49929/152040)
Branch Coverage: 28.55% (25096/87890)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e406e92fe01f2af37d2fd8b1f1e6c02cb9ee9809_e406e92fe01f2af37d2fd8b1f1e6c02cb9ee9809/report/index.html

@zclllyybb
Copy link
Contributor

run buildall

1 similar comment
@lzyy2024
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31428 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a5a3fe5fda3fb9e279c096cc3020a9fa7c02c49d, data reload: false

------ Round 1 ----------------------------------
q1	17642	5327	5073	5073
q2	2075	293	166	166
q3	10445	1295	684	684
q4	10297	981	529	529
q5	8353	2394	2268	2268
q6	201	168	131	131
q7	894	735	601	601
q8	9318	1318	1137	1137
q9	5026	4654	4645	4645
q10	6802	2329	1886	1886
q11	477	277	252	252
q12	346	360	222	222
q13	17767	3691	3137	3137
q14	220	225	216	216
q15	510	472	458	458
q16	614	597	583	583
q17	593	851	329	329
q18	6627	6183	6238	6183
q19	1419	942	538	538
q20	307	318	187	187
q21	2627	2188	1904	1904
q22	358	327	299	299
Total cold run time: 102918 ms
Total hot run time: 31428 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5163	5092	5129	5092
q2	240	334	227	227
q3	2160	2677	2305	2305
q4	1419	1808	1359	1359
q5	4206	4094	4124	4094
q6	211	163	122	122
q7	1834	1836	1664	1664
q8	2580	2552	2555	2552
q9	7210	7228	7245	7228
q10	3021	3218	2734	2734
q11	568	507	480	480
q12	646	738	632	632
q13	3555	3861	3250	3250
q14	283	293	281	281
q15	512	485	451	451
q16	642	671	644	644
q17	1121	1578	1347	1347
q18	7657	7359	7260	7260
q19	772	908	1030	908
q20	1968	2045	1889	1889
q21	5336	4808	4829	4808
q22	619	559	535	535
Total cold run time: 51723 ms
Total hot run time: 49862 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190720 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a5a3fe5fda3fb9e279c096cc3020a9fa7c02c49d, data reload: false

query1	1318	941	961	941
query2	6314	1816	1806	1806
query3	10987	4476	4481	4476
query4	53787	24883	23590	23590
query5	5205	573	524	524
query6	396	205	200	200
query7	5177	507	295	295
query8	341	246	227	227
query9	6927	2556	2539	2539
query10	425	303	264	264
query11	15617	14958	14880	14880
query12	165	108	106	106
query13	1223	521	402	402
query14	10220	6883	6661	6661
query15	210	198	206	198
query16	7153	683	488	488
query17	1140	757	572	572
query18	1728	415	320	320
query19	224	203	163	163
query20	133	126	120	120
query21	225	136	108	108
query22	4405	4451	4499	4451
query23	33926	33393	33390	33390
query24	5931	2479	2421	2421
query25	448	448	401	401
query26	688	284	156	156
query27	1754	494	336	336
query28	2749	2447	2444	2444
query29	560	564	424	424
query30	213	185	156	156
query31	895	860	852	852
query32	84	59	58	58
query33	460	352	339	339
query34	755	877	511	511
query35	796	849	774	774
query36	934	989	875	875
query37	137	103	73	73
query38	4241	4264	4267	4264
query39	1483	1441	1467	1441
query40	218	124	104	104
query41	52	55	50	50
query42	126	117	104	104
query43	495	522	488	488
query44	1325	827	818	818
query45	173	175	166	166
query46	885	1055	678	678
query47	1827	1921	1838	1838
query48	387	430	320	320
query49	699	507	409	409
query50	714	742	425	425
query51	4291	4322	4238	4238
query52	107	109	102	102
query53	234	262	196	196
query54	475	485	410	410
query55	81	80	77	77
query56	256	249	256	249
query57	1079	1125	1047	1047
query58	228	228	229	228
query59	2684	2759	2593	2593
query60	291	274	258	258
query61	116	116	112	112
query62	728	712	636	636
query63	225	186	182	182
query64	1802	1004	661	661
query65	3221	3165	3139	3139
query66	707	380	290	290
query67	15869	15663	15350	15350
query68	5352	775	535	535
query69	525	302	257	257
query70	1160	1174	1121	1121
query71	424	306	260	260
query72	5949	3561	3696	3561
query73	1080	751	354	354
query74	8849	9187	9166	9166
query75	3224	3136	2710	2710
query76	3751	1172	726	726
query77	562	362	279	279
query78	10077	10260	9280	9280
query79	2041	821	610	610
query80	846	541	481	481
query81	535	278	236	236
query82	405	126	92	92
query83	189	171	156	156
query84	294	88	68	68
query85	744	351	293	293
query86	379	308	283	283
query87	4392	4425	4376	4376
query88	2956	2233	2199	2199
query89	388	318	284	284
query90	1659	185	198	185
query91	135	137	106	106
query92	61	60	57	57
query93	2303	1004	588	588
query94	638	399	304	304
query95	345	262	255	255
query96	489	544	274	274
query97	2732	2810	2729	2729
query98	220	202	202	202
query99	1320	1413	1279	1279
Total cold run time: 292693 ms
Total hot run time: 190720 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.1 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a5a3fe5fda3fb9e279c096cc3020a9fa7c02c49d, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.06
query4	1.62	0.10	0.10
query5	0.42	0.41	0.42
query6	1.16	0.67	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.59	0.52	0.52
query10	0.58	0.57	0.57
query11	0.16	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.61	0.59
query14	2.72	2.69	2.71
query15	0.91	0.85	0.85
query16	0.38	0.38	0.37
query17	1.01	1.03	1.02
query18	0.21	0.20	0.19
query19	1.84	1.81	2.00
query20	0.02	0.01	0.02
query21	15.35	0.94	0.56
query22	0.77	1.22	0.73
query23	14.88	1.39	0.65
query24	11.40	0.96	0.33
query25	0.32	0.09	0.17
query26	0.96	0.18	0.13
query27	0.06	0.06	0.05
query28	6.46	0.78	0.42
query29	12.64	3.94	3.30
query30	0.25	0.09	0.06
query31	2.82	0.59	0.39
query32	3.23	0.53	0.46
query33	2.99	3.07	3.01
query34	15.82	5.12	4.47
query35	4.51	4.51	4.55
query36	0.67	0.49	0.48
query37	0.09	0.07	0.06
query38	0.06	0.04	0.04
query39	0.04	0.03	0.02
query40	0.18	0.12	0.13
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 106.43 s
Total hot run time: 30.1 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 43.82% (11636/26556)
Line Coverage: 33.74% (97533/289090)
Region Coverage: 32.84% (49935/152061)
Branch Coverage: 28.56% (25105/87900)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a5a3fe5fda3fb9e279c096cc3020a9fa7c02c49d_a5a3fe5fda3fb9e279c096cc3020a9fa7c02c49d/report/index.html

@lzyy2024 lzyy2024 force-pushed the Separate-StringOP-and-SubstringUtil branch from a5a3fe5 to e406e92 Compare February 20, 2025 07:16
@lzyy2024
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31604 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e406e92fe01f2af37d2fd8b1f1e6c02cb9ee9809, data reload: false

------ Round 1 ----------------------------------
q1	17595	5337	5097	5097
q2	2048	309	178	178
q3	10393	1298	729	729
q4	10213	999	532	532
q5	7537	2410	2309	2309
q6	187	174	138	138
q7	922	758	608	608
q8	9309	1316	1135	1135
q9	4823	4640	4830	4640
q10	6815	2297	1897	1897
q11	478	274	271	271
q12	356	363	226	226
q13	17762	3666	3094	3094
q14	223	230	216	216
q15	505	468	460	460
q16	629	608	572	572
q17	598	863	328	328
q18	6478	6337	6166	6166
q19	1075	938	559	559
q20	308	327	190	190
q21	2800	2146	1933	1933
q22	369	335	326	326
Total cold run time: 101423 ms
Total hot run time: 31604 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5109	5093	5096	5093
q2	232	329	234	234
q3	2146	2642	2291	2291
q4	1380	1842	1357	1357
q5	4249	4112	4118	4112
q6	208	161	123	123
q7	1856	1835	1676	1676
q8	2610	2573	2537	2537
q9	7331	7184	7141	7141
q10	2969	3195	2746	2746
q11	567	537	489	489
q12	694	742	638	638
q13	3478	3869	3262	3262
q14	289	300	257	257
q15	515	465	455	455
q16	662	675	636	636
q17	1126	1618	1367	1367
q18	7495	7461	7232	7232
q19	778	796	885	796
q20	2005	2003	1897	1897
q21	5494	4991	4919	4919
q22	687	568	537	537
Total cold run time: 51880 ms
Total hot run time: 49795 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 183161 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e406e92fe01f2af37d2fd8b1f1e6c02cb9ee9809, data reload: false

query1	961	361	354	354
query2	6547	1813	1829	1813
query3	6798	210	209	209
query4	26364	23685	23594	23594
query5	4475	665	480	480
query6	304	187	173	173
query7	4608	492	293	293
query8	295	227	216	216
query9	8621	2518	2524	2518
query10	458	317	239	239
query11	15309	15136	14865	14865
query12	157	110	106	106
query13	1655	522	405	405
query14	9109	6280	6118	6118
query15	201	196	178	178
query16	7474	609	475	475
query17	1201	714	561	561
query18	1968	417	292	292
query19	187	179	151	151
query20	116	120	112	112
query21	212	119	102	102
query22	4440	4179	4400	4179
query23	34201	33482	32888	32888
query24	8088	2358	2337	2337
query25	514	454	376	376
query26	1227	273	146	146
query27	2607	489	329	329
query28	4294	2411	2395	2395
query29	732	526	432	432
query30	226	183	151	151
query31	906	860	761	761
query32	69	64	63	63
query33	545	341	308	308
query34	794	841	508	508
query35	795	814	714	714
query36	981	954	874	874
query37	113	93	73	73
query38	4137	4124	4130	4124
query39	1443	1368	1388	1368
query40	206	111	100	100
query41	57	50	48	48
query42	113	107	107	107
query43	495	491	480	480
query44	1251	785	788	785
query45	178	172	156	156
query46	875	1017	634	634
query47	1765	1815	1726	1726
query48	389	421	308	308
query49	773	494	414	414
query50	685	729	437	437
query51	4145	4187	4078	4078
query52	116	104	93	93
query53	234	269	183	183
query54	462	493	406	406
query55	81	76	76	76
query56	257	258	231	231
query57	1130	1152	1076	1076
query58	245	233	243	233
query59	2655	2709	2604	2604
query60	302	299	269	269
query61	151	118	113	113
query62	770	754	640	640
query63	226	191	190	190
query64	4352	986	661	661
query65	3219	3146	3149	3146
query66	1098	394	351	351
query67	15762	15512	15331	15331
query68	5069	807	535	535
query69	487	306	263	263
query70	1170	1077	1138	1077
query71	385	288	262	262
query72	5731	3523	3800	3523
query73	722	725	370	370
query74	9320	9067	8959	8959
query75	3126	3144	2662	2662
query76	3246	1146	726	726
query77	467	380	277	277
query78	9964	10005	9266	9266
query79	1250	882	634	634
query80	614	538	448	448
query81	485	264	233	233
query82	192	127	96	96
query83	178	171	156	156
query84	238	92	71	71
query85	747	338	294	294
query86	314	303	288	288
query87	4358	4544	4424	4424
query88	2855	2209	2174	2174
query89	391	318	279	279
query90	1790	193	188	188
query91	134	141	110	110
query92	61	60	55	55
query93	1108	992	586	586
query94	586	396	303	303
query95	346	307	264	264
query96	501	534	275	275
query97	2759	2804	2691	2691
query98	220	211	199	199
query99	1305	1412	1317	1317
Total cold run time: 265243 ms
Total hot run time: 183161 ms

@lzyy2024
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31378 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit aaf9bd6d9a4b16637d5b0dbdbb251fe4fd3bcd98, data reload: false

------ Round 1 ----------------------------------
q1	17617	5294	5042	5042
q2	2052	316	198	198
q3	10366	1316	719	719
q4	10210	1012	521	521
q5	7494	2468	2301	2301
q6	192	167	133	133
q7	917	751	602	602
q8	9305	1322	1228	1228
q9	4987	4519	4557	4519
q10	6885	2306	1883	1883
q11	474	277	258	258
q12	351	346	226	226
q13	17782	3679	3056	3056
q14	239	223	212	212
q15	511	472	439	439
q16	639	621	570	570
q17	634	880	343	343
q18	6892	6121	6273	6121
q19	1210	948	554	554
q20	321	331	192	192
q21	2816	2097	1954	1954
q22	370	339	307	307
Total cold run time: 102264 ms
Total hot run time: 31378 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5116	5149	5130	5130
q2	241	340	238	238
q3	2174	2685	2282	2282
q4	1434	1824	1342	1342
q5	4251	4142	4216	4142
q6	206	163	123	123
q7	1856	1830	1672	1672
q8	2506	2545	2478	2478
q9	6866	6904	6839	6839
q10	2914	3074	2625	2625
q11	579	511	476	476
q12	646	710	568	568
q13	3274	3720	3064	3064
q14	283	277	272	272
q15	493	459	457	457
q16	622	655	631	631
q17	1082	1576	1290	1290
q18	7180	7090	7023	7023
q19	780	831	887	831
q20	1888	2016	1814	1814
q21	5142	4764	4615	4615
q22	599	566	513	513
Total cold run time: 50132 ms
Total hot run time: 48425 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 182839 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit aaf9bd6d9a4b16637d5b0dbdbb251fe4fd3bcd98, data reload: false

query1	985	389	382	382
query2	2689	1849	1840	1840
query3	6707	219	214	214
query4	25488	23689	23576	23576
query5	2329	667	487	487
query6	283	182	173	173
query7	4545	509	292	292
query8	280	226	246	226
query9	8597	2528	2504	2504
query10	481	316	252	252
query11	15609	15061	14842	14842
query12	151	109	107	107
query13	1664	540	393	393
query14	10121	6252	6107	6107
query15	210	194	195	194
query16	5931	636	501	501
query17	1556	757	581	581
query18	1956	409	312	312
query19	250	183	161	161
query20	119	115	117	115
query21	196	120	104	104
query22	4081	4309	4257	4257
query23	33843	33009	32801	32801
query24	7048	2391	2366	2366
query25	476	451	410	410
query26	853	285	163	163
query27	2457	485	349	349
query28	4278	2423	2411	2411
query29	695	541	426	426
query30	283	182	166	166
query31	936	839	803	803
query32	81	69	63	63
query33	525	349	292	292
query34	750	844	531	531
query35	791	802	729	729
query36	951	988	879	879
query37	121	98	70	70
query38	4178	4200	4143	4143
query39	1449	1394	1387	1387
query40	207	114	101	101
query41	55	53	51	51
query42	121	107	106	106
query43	480	473	474	473
query44	1292	800	797	797
query45	175	170	157	157
query46	861	1035	649	649
query47	1761	1769	1697	1697
query48	398	417	317	317
query49	850	522	404	404
query50	698	737	423	423
query51	4222	4236	4116	4116
query52	102	104	96	96
query53	236	268	194	194
query54	488	476	408	408
query55	80	79	78	78
query56	271	280	241	241
query57	1113	1126	1065	1065
query58	240	225	237	225
query59	2539	2584	2606	2584
query60	292	263	263	263
query61	119	116	144	116
query62	786	757	658	658
query63	239	192	192	192
query64	3310	1039	658	658
query65	3248	3153	3174	3153
query66	1093	405	290	290
query67	15733	15506	15591	15506
query68	3306	801	539	539
query69	502	292	271	271
query70	1204	1150	1062	1062
query71	439	302	287	287
query72	6106	2618	3697	2618
query73	716	749	362	362
query74	9104	9163	8820	8820
query75	3159	3153	2688	2688
query76	3323	1173	743	743
query77	506	358	301	301
query78	9997	10193	9348	9348
query79	1910	831	613	613
query80	1038	627	462	462
query81	505	277	238	238
query82	399	129	99	99
query83	277	187	150	150
query84	236	92	72	72
query85	718	342	297	297
query86	330	317	291	291
query87	4479	4564	4469	4469
query88	2995	2233	2199	2199
query89	404	315	283	283
query90	1799	195	199	195
query91	135	136	113	113
query92	64	64	53	53
query93	1169	1016	600	600
query94	608	397	290	290
query95	334	280	260	260
query96	498	551	273	273
query97	2787	2870	2740	2740
query98	224	210	206	206
query99	1311	1411	1270	1270
Total cold run time: 255534 ms
Total hot run time: 182839 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.21 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit aaf9bd6d9a4b16637d5b0dbdbb251fe4fd3bcd98, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.61	0.11	0.10
query5	0.41	0.40	0.39
query6	1.18	0.67	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.60	0.51	0.51
query10	0.56	0.57	0.57
query11	0.15	0.10	0.10
query12	0.15	0.10	0.11
query13	0.62	0.59	0.61
query14	2.69	2.70	2.72
query15	0.93	0.87	0.86
query16	0.39	0.38	0.38
query17	1.06	1.02	1.00
query18	0.21	0.20	0.19
query19	1.87	1.78	1.98
query20	0.01	0.01	0.01
query21	15.35	0.94	0.54
query22	0.74	1.15	0.80
query23	14.81	1.33	0.62
query24	12.03	0.96	0.40
query25	0.33	0.14	0.09
query26	0.89	0.19	0.15
query27	0.05	0.05	0.04
query28	5.96	0.78	0.44
query29	12.51	3.95	3.28
query30	0.25	0.09	0.06
query31	2.83	0.58	0.38
query32	3.22	0.54	0.45
query33	3.00	3.09	3.08
query34	15.79	5.11	4.45
query35	4.51	4.49	4.51
query36	0.66	0.50	0.48
query37	0.10	0.06	0.07
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.14	0.14
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.26 s
Total hot run time: 30.21 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 43.83% (11640/26556)
Line Coverage: 33.76% (97599/289093)
Region Coverage: 32.86% (49966/152064)
Branch Coverage: 28.57% (25117/87902)
Coverage Report: http://coverage.selectdb-in.cc/coverage/aaf9bd6d9a4b16637d5b0dbdbb251fe4fd3bcd98_aaf9bd6d9a4b16637d5b0dbdbb251fe4fd3bcd98/report/index.html

zclllyybb
zclllyybb approved these changes Feb 20, 2025
View reviewed changes
Copy link
Contributor

@zclllyybb zclllyybb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

zclllyybb
zclllyybb reviewed Feb 20, 2025
View reviewed changes
Copy link
Contributor

@zclllyybb zclllyybb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add BEUT for StringOP's functions by the way. thank you !

@lzyy2024 lzyy2024 force-pushed the Separate-StringOP-and-SubstringUtil branch from e054482 to ceb6232 Compare February 23, 2025 06:56
@lzyy2024
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31315 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cfce5191bd3285a98796f5de94d091e84cc81675, data reload: false

------ Round 1 ----------------------------------
q1	17812	5383	5166	5166
q2	2044	294	165	165
q3	10453	1243	730	730
q4	10203	1011	549	549
q5	7534	2483	2224	2224
q6	189	168	133	133
q7	921	752	602	602
q8	9296	1354	1159	1159
q9	4881	4567	4650	4567
q10	6818	2307	1895	1895
q11	486	272	262	262
q12	351	368	231	231
q13	17770	3634	3035	3035
q14	233	234	207	207
q15	518	471	462	462
q16	610	600	580	580
q17	585	868	344	344
q18	6952	6107	6200	6107
q19	1210	968	549	549
q20	309	309	185	185
q21	2732	2100	1875	1875
q22	347	328	288	288
Total cold run time: 102254 ms
Total hot run time: 31315 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5109	5138	5106	5106
q2	227	332	227	227
q3	2161	2660	2356	2356
q4	1391	1800	1354	1354
q5	4218	4119	4130	4119
q6	211	161	122	122
q7	1880	1820	1614	1614
q8	2562	2549	2529	2529
q9	7297	7202	7225	7202
q10	3010	3238	2784	2784
q11	592	508	496	496
q12	692	781	657	657
q13	3439	3872	3324	3324
q14	271	302	277	277
q15	516	455	473	455
q16	670	694	655	655
q17	1119	1543	1357	1357
q18	7512	7377	7271	7271
q19	818	910	1148	910
q20	1978	2009	1890	1890
q21	5313	4919	4786	4786
q22	625	566	548	548
Total cold run time: 51611 ms
Total hot run time: 50039 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190618 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cfce5191bd3285a98796f5de94d091e84cc81675, data reload: false

query1	1283	959	936	936
query2	6227	1929	1942	1929
query3	10994	4535	4452	4452
query4	56500	25638	23516	23516
query5	5181	515	478	478
query6	364	182	173	173
query7	5134	497	286	286
query8	329	244	236	236
query9	6470	2577	2590	2577
query10	444	329	251	251
query11	15121	15020	14971	14971
query12	167	105	108	105
query13	1146	530	397	397
query14	10475	6785	6391	6391
query15	207	210	177	177
query16	7081	615	428	428
query17	1094	725	594	594
query18	1581	416	322	322
query19	240	190	174	174
query20	144	131	123	123
query21	211	129	112	112
query22	4554	4675	4459	4459
query23	33975	33188	33421	33188
query24	5592	2448	2415	2415
query25	451	459	405	405
query26	691	274	166	166
query27	1758	464	352	352
query28	2807	2529	2447	2447
query29	553	588	430	430
query30	216	189	151	151
query31	896	885	830	830
query32	75	66	66	66
query33	434	366	304	304
query34	746	852	530	530
query35	809	842	756	756
query36	945	978	914	914
query37	118	108	72	72
query38	4271	4279	4196	4196
query39	1459	1432	1418	1418
query40	213	127	106	106
query41	51	50	47	47
query42	124	108	110	108
query43	512	516	489	489
query44	1311	795	791	791
query45	184	172	168	168
query46	881	1048	674	674
query47	1860	1898	1820	1820
query48	403	426	318	318
query49	711	495	399	399
query50	704	750	451	451
query51	4264	4363	4242	4242
query52	109	105	97	97
query53	229	258	187	187
query54	482	484	433	433
query55	101	79	81	79
query56	265	284	261	261
query57	1217	1177	1140	1140
query58	240	258	259	258
query59	2734	2859	2803	2803
query60	300	285	262	262
query61	119	121	122	121
query62	744	739	674	674
query63	234	190	188	188
query64	1496	1059	720	720
query65	3292	3170	3147	3147
query66	707	397	308	308
query67	15729	15786	15442	15442
query68	4698	768	512	512
query69	497	306	263	263
query70	1235	1128	1121	1121
query71	418	288	269	269
query72	6467	3661	3648	3648
query73	790	740	348	348
query74	9302	9095	8993	8993
query75	3290	3135	2647	2647
query76	3570	1172	733	733
query77	540	379	272	272
query78	10014	10066	9254	9254
query79	2348	829	605	605
query80	660	524	436	436
query81	497	277	234	234
query82	239	129	93	93
query83	184	170	147	147
query84	285	98	71	71
query85	803	346	308	308
query86	412	326	288	288
query87	4436	4511	4433	4433
query88	3794	2282	2214	2214
query89	407	312	279	279
query90	1916	190	192	190
query91	133	136	107	107
query92	72	57	54	54
query93	2300	986	573	573
query94	682	370	296	296
query95	347	267	253	253
query96	504	565	275	275
query97	2854	2887	2764	2764
query98	229	203	194	194
query99	1303	1396	1284	1284
Total cold run time: 295185 ms
Total hot run time: 190618 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.55 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cfce5191bd3285a98796f5de94d091e84cc81675, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.03	0.04
query3	0.24	0.07	0.07
query4	1.62	0.10	0.11
query5	0.42	0.42	0.40
query6	1.18	0.66	0.65
query7	0.03	0.02	0.01
query8	0.04	0.04	0.03
query9	0.60	0.52	0.54
query10	0.56	0.56	0.57
query11	0.15	0.10	0.10
query12	0.15	0.11	0.11
query13	0.60	0.61	0.60
query14	2.68	2.72	2.72
query15	0.91	0.84	0.84
query16	0.37	0.38	0.38
query17	1.04	1.02	1.06
query18	0.21	0.19	0.20
query19	1.89	1.81	1.97
query20	0.02	0.00	0.02
query21	15.35	0.90	0.53
query22	0.74	1.19	0.66
query23	14.97	1.37	0.64
query24	6.93	1.16	0.89
query25	0.53	0.25	0.09
query26	0.59	0.15	0.15
query27	0.06	0.05	0.04
query28	10.07	0.86	0.42
query29	12.59	3.95	3.28
query30	0.25	0.10	0.07
query31	2.81	0.57	0.38
query32	3.22	0.54	0.46
query33	2.99	3.01	3.05
query34	15.84	5.13	4.46
query35	4.54	4.49	4.52
query36	0.67	0.50	0.49
query37	0.09	0.06	0.07
query38	0.06	0.04	0.04
query39	0.03	0.02	0.02
query40	0.18	0.14	0.13
query41	0.08	0.02	0.03
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 105.49 s
Total hot run time: 30.55 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zclllyybb zclllyybb zclllyybb approved these changes

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@lzyy2024 @Thearas @doris-robot @zclllyybb