[improvement](json load) Compatible with "$" and "$." that appear during json load #56703

0AyanamiRei · 2025-10-04T09:54:32Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

For JSON-related parameters, both $ and $. are supported for json_root in json load

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-10-04T09:54:38Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

0AyanamiRei · 2025-10-04T09:57:50Z

run buildall

0AyanamiRei · 2025-10-04T11:07:59Z

run buildall

doris-robot · 2025-10-04T11:41:58Z

TPC-DS: Total hot run time: 190518 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f83b4a78c8db8b31cad5d650f6ad112f8c962ff0, data reload: false

query1	1024	444	409	409
query2	6561	1703	1697	1697
query3	6753	231	220	220
query4	25800	23574	23409	23409
query5	4854	644	496	496
query6	383	242	222	222
query7	4652	499	292	292
query8	301	266	250	250
query9	8722	2591	2608	2591
query10	488	347	288	288
query11	15766	15159	14924	14924
query12	186	127	115	115
query13	1668	559	432	432
query14	11540	9439	9541	9439
query15	211	190	177	177
query16	7774	754	586	586
query17	1405	828	632	632
query18	2230	447	374	374
query19	223	219	221	219
query20	133	132	131	131
query21	219	156	127	127
query22	4676	4628	4550	4550
query23	34567	33784	33306	33306
query24	8433	2389	2412	2389
query25	572	507	446	446
query26	1213	273	160	160
query27	2740	487	367	367
query28	4393	2169	2141	2141
query29	777	633	513	513
query30	294	232	206	206
query31	921	792	749	749
query32	90	75	72	72
query33	612	403	344	344
query34	805	847	522	522
query35	807	858	751	751
query36	974	1031	954	954
query37	126	118	93	93
query38	3519	3493	3576	3493
query39	1500	1464	1398	1398
query40	227	133	116	116
query41	64	59	63	59
query42	121	114	114	114
query43	506	506	476	476
query44	1341	843	834	834
query45	187	186	178	178
query46	835	1006	643	643
query47	1736	1815	1742	1742
query48	388	422	310	310
query49	766	511	423	423
query50	664	679	411	411
query51	3854	3998	3824	3824
query52	107	109	99	99
query53	243	274	191	191
query54	600	600	525	525
query55	87	87	87	87
query56	329	336	325	325
query57	1161	1198	1127	1127
query58	280	285	285	285
query59	2575	2647	2564	2564
query60	357	344	332	332
query61	157	152	153	152
query62	802	736	668	668
query63	237	197	196	196
query64	4365	1146	831	831
query65	4037	3980	3999	3980
query66	1098	463	349	349
query67	15650	15272	15142	15142
query68	7926	943	600	600
query69	506	344	307	307
query70	1376	1218	1304	1218
query71	504	344	343	343
query72	5610	4889	4860	4860
query73	670	572	363	363
query74	9365	9145	9121	9121
query75	4017	3378	2837	2837
query76	3742	1167	744	744
query77	830	433	321	321
query78	9536	9879	8952	8952
query79	2024	885	608	608
query80	707	558	521	521
query81	499	266	233	233
query82	483	157	140	140
query83	293	271	253	253
query84	313	104	98	98
query85	878	463	426	426
query86	342	334	304	304
query87	3856	3747	3766	3747
query88	2960	2260	2276	2260
query89	417	337	307	307
query90	2019	225	231	225
query91	171	162	132	132
query92	81	76	71	71
query93	1139	991	650	650
query94	703	443	334	334
query95	401	322	318	318
query96	493	571	280	280
query97	2985	2984	2934	2934
query98	233	216	215	215
query99	1440	1449	1329	1329
Total cold run time: 277205 ms
Total hot run time: 190518 ms

doris-robot · 2025-10-04T11:47:28Z

ClickBench: Total hot run time: 30.35 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f83b4a78c8db8b31cad5d650f6ad112f8c962ff0, data reload: false

query1	0.06	0.04	0.05
query2	0.09	0.06	0.06
query3	0.26	0.09	0.08
query4	1.61	0.12	0.12
query5	0.27	0.27	0.25
query6	1.18	0.64	0.65
query7	0.03	0.03	0.02
query8	0.06	0.05	0.05
query9	0.60	0.52	0.52
query10	0.58	0.60	0.57
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.64	0.63	0.63
query14	1.03	1.03	1.02
query15	0.87	0.88	0.88
query16	0.40	0.39	0.40
query17	1.05	1.07	1.05
query18	0.22	0.21	0.20
query19	2.01	1.85	1.82
query20	0.01	0.01	0.02
query21	15.46	1.00	0.60
query22	0.76	1.19	0.65
query23	15.04	1.42	0.62
query24	6.93	2.03	0.64
query25	0.50	0.16	0.13
query26	0.61	0.17	0.13
query27	0.07	0.06	0.06
query28	9.11	1.41	0.95
query29	12.56	3.98	3.27
query30	0.28	0.13	0.12
query31	2.83	0.59	0.39
query32	3.24	0.58	0.48
query33	3.16	3.10	3.10
query34	16.20	5.54	4.86
query35	4.97	4.88	4.93
query36	0.71	0.52	0.51
query37	0.11	0.08	0.07
query38	0.07	0.05	0.04
query39	0.04	0.03	0.03
query40	0.18	0.16	0.14
query41	0.08	0.03	0.03
query42	0.04	0.04	0.03
query43	0.04	0.04	0.04
Total cold run time: 104.27 s
Total hot run time: 30.35 s

doris-robot · 2025-10-04T12:09:42Z

BE UT Coverage Report

Increment line coverage 0.00% (0/14) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.49% (17708/33739)
Line Coverage	37.66% (160763/426880)
Region Coverage	32.17% (122858/381920)
Branch Coverage	33.55% (53857/160547)

hello-stephen · 2025-10-04T13:44:08Z

BE Regression && UT Coverage Report

Increment line coverage 71.43% (10/14) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.24% (23551/33057)
Line Coverage	57.69% (246003/426449)
Region Coverage	53.00% (204968/386748)
Branch Coverage	54.65% (88177/161358)

0AyanamiRei · 2025-10-09T03:22:23Z

run buildall

doris-robot · 2025-10-09T04:02:15Z

TPC-DS: Total hot run time: 190156 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9ba220e65f46b916d5ddea8218167aaaf15f8baa, data reload: false

query1	1078	428	402	402
query2	6547	1675	1693	1675
query3	6761	223	221	221
query4	26428	23729	23538	23538
query5	5232	654	490	490
query6	343	264	223	223
query7	4670	489	314	314
query8	314	271	254	254
query9	8743	2561	2578	2561
query10	532	359	299	299
query11	15580	15318	14833	14833
query12	187	121	120	120
query13	1690	578	450	450
query14	12092	9261	9173	9173
query15	252	202	179	179
query16	7777	686	522	522
query17	1620	776	631	631
query18	2077	457	341	341
query19	302	217	209	209
query20	141	129	130	129
query21	217	134	116	116
query22	4796	4822	4747	4747
query23	35270	33757	34059	33757
query24	8362	2511	2490	2490
query25	643	527	454	454
query26	1250	277	165	165
query27	2832	520	370	370
query28	4353	2180	2191	2180
query29	802	697	528	528
query30	299	235	306	235
query31	929	850	760	760
query32	86	75	66	66
query33	589	415	344	344
query34	878	876	528	528
query35	824	917	775	775
query36	959	1033	961	961
query37	139	117	94	94
query38	3563	3641	3528	3528
query39	1457	1430	1439	1430
query40	222	125	117	117
query41	63	57	56	56
query42	128	107	111	107
query43	496	498	446	446
query44	1302	826	814	814
query45	181	184	172	172
query46	832	998	631	631
query47	1773	1841	1743	1743
query48	392	418	335	335
query49	750	508	407	407
query50	634	687	405	405
query51	3961	3874	3884	3874
query52	110	105	101	101
query53	236	259	199	199
query54	619	580	517	517
query55	87	85	83	83
query56	321	307	336	307
query57	1180	1211	1128	1128
query58	280	274	256	256
query59	2564	2641	2562	2562
query60	350	347	341	341
query61	155	155	149	149
query62	805	720	686	686
query63	230	187	190	187
query64	4433	1143	894	894
query65	4088	3962	3971	3962
query66	1082	433	346	346
query67	15384	15336	15093	15093
query68	8744	955	598	598
query69	486	316	294	294
query70	1425	1285	1275	1275
query71	555	343	306	306
query72	6022	4909	5004	4909
query73	717	624	358	358
query74	9158	9234	8621	8621
query75	4110	3403	2863	2863
query76	3780	1167	740	740
query77	820	403	312	312
query78	9693	9692	8855	8855
query79	2281	852	594	594
query80	752	558	493	493
query81	492	265	230	230
query82	494	158	127	127
query83	283	277	240	240
query84	258	115	93	93
query85	887	472	425	425
query86	382	374	283	283
query87	3766	3841	3694	3694
query88	3545	2235	2203	2203
query89	388	313	297	297
query90	1999	220	219	219
query91	162	169	135	135
query92	92	68	67	67
query93	1780	959	638	638
query94	701	444	319	319
query95	406	323	318	318
query96	491	565	276	276
query97	2941	2958	2893	2893
query98	250	214	209	209
query99	1618	1426	1262	1262
Total cold run time: 282750 ms
Total hot run time: 190156 ms

doris-robot · 2025-10-09T04:07:44Z

ClickBench: Total hot run time: 30.28 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9ba220e65f46b916d5ddea8218167aaaf15f8baa, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.05	0.06
query3	0.25	0.08	0.08
query4	1.61	0.12	0.12
query5	0.28	0.27	0.25
query6	1.17	0.66	0.63
query7	0.03	0.02	0.02
query8	0.06	0.04	0.04
query9	0.63	0.53	0.52
query10	0.58	0.60	0.58
query11	0.17	0.14	0.12
query12	0.15	0.12	0.12
query13	0.64	0.62	0.61
query14	1.05	1.03	1.01
query15	0.86	0.84	0.84
query16	0.40	0.40	0.40
query17	1.07	1.05	1.06
query18	0.23	0.21	0.20
query19	1.92	1.86	1.81
query20	0.02	0.02	0.02
query21	15.41	0.89	0.57
query22	0.77	1.09	0.78
query23	14.91	1.40	0.69
query24	7.11	1.31	0.45
query25	0.50	0.14	0.12
query26	0.55	0.16	0.14
query27	0.05	0.06	0.06
query28	9.82	1.40	0.94
query29	12.62	3.93	3.26
query30	0.27	0.14	0.13
query31	2.82	0.58	0.39
query32	3.24	0.56	0.50
query33	3.06	3.04	3.08
query34	16.15	5.45	4.88
query35	4.89	4.93	4.94
query36	0.69	0.52	0.50
query37	0.10	0.07	0.08
query38	0.06	0.04	0.04
query39	0.04	0.03	0.04
query40	0.18	0.15	0.15
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 104.69 s
Total hot run time: 30.28 s

hello-stephen · 2025-10-09T04:31:58Z

BE UT Coverage Report

Increment line coverage 0.00% (0/5) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.49% (17709/33739)
Line Coverage	37.66% (160774/426876)
Region Coverage	32.18% (122928/381952)
Branch Coverage	33.55% (53865/160537)

hello-stephen · 2025-10-09T05:53:25Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (5/5) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.15% (23520/33057)
Line Coverage	57.60% (245616/426445)
Region Coverage	52.83% (204352/386780)
Branch Coverage	54.55% (88020/161348)

regression-test/suites/json_p0/test_json_root.groovy

0AyanamiRei · 2025-10-10T05:02:52Z

run buildall

doris-robot · 2025-10-10T06:12:04Z

TPC-DS: Total hot run time: 190738 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4d0c2b9b77e2bd464e10a48529ad4ef44218e537, data reload: false

query1	1083	443	404	404
query2	6570	1752	1685	1685
query3	6755	226	219	219
query4	26430	23420	23067	23067
query5	4453	632	476	476
query6	333	267	220	220
query7	4650	504	300	300
query8	302	270	256	256
query9	8716	2582	2563	2563
query10	493	349	299	299
query11	15981	15142	14930	14930
query12	194	117	113	113
query13	1670	558	439	439
query14	11673	9409	9452	9409
query15	212	190	183	183
query16	7672	698	526	526
query17	1298	757	663	663
query18	2030	461	396	396
query19	253	222	179	179
query20	141	134	138	134
query21	231	142	121	121
query22	4662	4641	4619	4619
query23	35622	34064	33945	33945
query24	8476	2514	2476	2476
query25	590	540	490	490
query26	1295	307	167	167
query27	2746	519	373	373
query28	4390	2212	2199	2199
query29	813	639	533	533
query30	349	242	210	210
query31	971	875	788	788
query32	84	85	74	74
query33	605	417	353	353
query34	854	869	557	557
query35	858	879	782	782
query36	997	1020	945	945
query37	123	114	88	88
query38	3571	3542	3578	3542
query39	1527	1441	1404	1404
query40	227	137	115	115
query41	60	58	58	58
query42	121	115	117	115
query43	500	516	454	454
query44	1339	850	834	834
query45	190	177	178	177
query46	841	994	644	644
query47	1762	1792	1753	1753
query48	403	448	317	317
query49	779	511	423	423
query50	655	704	411	411
query51	3986	3924	3829	3829
query52	109	108	101	101
query53	240	267	200	200
query54	602	604	535	535
query55	91	82	87	82
query56	336	343	365	343
query57	1196	1186	1143	1143
query58	296	275	274	274
query59	2529	2634	2591	2591
query60	347	344	331	331
query61	166	160	149	149
query62	809	735	670	670
query63	235	194	193	193
query64	4435	1182	856	856
query65	4088	3983	4002	3983
query66	1123	463	359	359
query67	15722	15196	15153	15153
query68	7555	894	613	613
query69	503	339	306	306
query70	1376	1313	1283	1283
query71	442	349	400	349
query72	5755	4888	4842	4842
query73	630	577	364	364
query74	8853	8861	8669	8669
query75	3335	3412	2851	2851
query76	3160	1167	750	750
query77	623	416	329	329
query78	9674	9771	8907	8907
query79	2045	830	589	589
query80	628	578	507	507
query81	494	261	237	237
query82	466	170	134	134
query83	275	276	252	252
query84	258	114	94	94
query85	894	527	456	456
query86	382	335	276	276
query87	3801	3856	3688	3688
query88	3718	2261	2247	2247
query89	400	357	295	295
query90	1990	227	228	227
query91	170	165	133	133
query92	83	75	66	66
query93	1753	983	646	646
query94	752	442	355	355
query95	415	345	322	322
query96	485	576	279	279
query97	2937	3018	2916	2916
query98	245	222	222	222
query99	1658	1458	1314	1314
Total cold run time: 278615 ms
Total hot run time: 190738 ms

doris-robot · 2025-10-10T06:17:34Z

ClickBench: Total hot run time: 30.26 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4d0c2b9b77e2bd464e10a48529ad4ef44218e537, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.06	0.06
query3	0.25	0.08	0.08
query4	1.62	0.12	0.12
query5	0.27	0.27	0.25
query6	1.17	0.65	0.66
query7	0.03	0.03	0.02
query8	0.06	0.05	0.04
query9	0.62	0.53	0.52
query10	0.58	0.57	0.59
query11	0.16	0.12	0.11
query12	0.16	0.12	0.13
query13	0.64	0.62	0.62
query14	1.04	1.03	1.04
query15	0.89	0.85	0.90
query16	0.41	0.41	0.40
query17	1.09	1.07	1.03
query18	0.21	0.20	0.20
query19	1.99	1.89	1.81
query20	0.01	0.02	0.01
query21	15.47	0.96	0.57
query22	0.76	1.28	0.67
query23	14.88	1.39	0.63
query24	6.99	1.73	0.46
query25	0.53	0.21	0.13
query26	0.64	0.17	0.13
query27	0.07	0.06	0.06
query28	9.15	1.37	0.94
query29	12.60	4.00	3.27
query30	0.29	0.13	0.12
query31	2.85	0.60	0.39
query32	3.26	0.58	0.48
query33	3.07	3.19	3.19
query34	16.21	5.45	4.86
query35	4.89	4.93	4.98
query36	0.73	0.52	0.52
query37	0.10	0.08	0.07
query38	0.07	0.05	0.04
query39	0.03	0.03	0.03
query40	0.18	0.16	0.14
query41	0.09	0.03	0.04
query42	0.04	0.03	0.03
query43	0.05	0.03	0.03
Total cold run time: 104.3 s
Total hot run time: 30.26 s

doris-robot · 2025-10-10T07:13:46Z

BE UT Coverage Report

Increment line coverage 0.00% (0/10) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.50% (17715/33742)
Line Coverage	37.68% (160867/426963)
Region Coverage	32.20% (122993/382014)
Branch Coverage	33.56% (53892/160587)

hello-stephen · 2025-10-10T08:16:56Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (10/10) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.19% (23536/33060)
Line Coverage	57.61% (245722/426532)
Region Coverage	52.70% (203862/386842)
Branch Coverage	54.47% (87921/161398)

eldenmoon

LGTM

github-actions · 2025-10-11T07:18:52Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-10-11T07:18:54Z

PR approved by anyone and no changes requested.

dataroaring

LGTM

dataroaring

LGTM

…ing json load (#56703) For JSON-related parameters, both `$` and `$.` are supported for `json_root` in json load

…t appear during json load #56703 (#56866) Cherry-picked from #56703 Co-authored-by: Refrain <[email protected]>

json root union

ab932c1

remove

f83b4a7

0AyanamiRei changed the title ~~[improvement](json load) Compatible with "$" and "$." that appear during JSON import and query processes.~~ [improvement](json load) Compatible with "$" and "$." that appear during json load Oct 4, 2025

fix

9ba220e

liaoxin01 reviewed Oct 10, 2025

View reviewed changes

regression-test/suites/json_p0/test_json_root.groovy Show resolved Hide resolved

add jsonpaths change

4d0c2b9

eldenmoon approved these changes Oct 11, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 11, 2025

github-actions bot added the reviewed label Oct 11, 2025

dataroaring approved these changes Oct 11, 2025

View reviewed changes

dataroaring added the dev/4.0.x label Oct 11, 2025

dataroaring approved these changes Oct 11, 2025

View reviewed changes

dataroaring merged commit d20036e into apache:master Oct 11, 2025
28 of 31 checks passed

github-actions bot pushed a commit that referenced this pull request Oct 11, 2025

[improvement](json load) Compatible with "$" and "$." that appear dur…

affe439

…ing json load (#56703) For JSON-related parameters, both `$` and `$.` are supported for `json_root` in json load

github-actions bot mentioned this pull request Oct 11, 2025

branch-4.0: [improvement](json load) Compatible with "$" and "$." that appear during json load #56703 #56866

Merged

yiguolei pushed a commit that referenced this pull request Oct 12, 2025

branch-4.0: [improvement](json load) Compatible with "$" and "$." tha…

19894b7

…t appear during json load #56703 (#56866) Cherry-picked from #56703 Co-authored-by: Refrain <[email protected]>

yiguolei added dev/4.0.1-merged and removed dev/4.0.x labels Oct 12, 2025

0AyanamiRei deleted the jsonRoot branch October 13, 2025 02:59

[improvement](json load) Compatible with "$" and "$." that appear during json load #56703

[improvement](json load) Compatible with "$" and "$." that appear during json load #56703

Uh oh!

Conversation

0AyanamiRei commented Oct 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Oct 4, 2025

Uh oh!

0AyanamiRei commented Oct 4, 2025

Uh oh!

0AyanamiRei commented Oct 4, 2025

Uh oh!

doris-robot commented Oct 4, 2025

Uh oh!

doris-robot commented Oct 4, 2025

Uh oh!

doris-robot commented Oct 4, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Oct 4, 2025

BE Regression && UT Coverage Report

Uh oh!

0AyanamiRei commented Oct 9, 2025

Uh oh!

doris-robot commented Oct 9, 2025

Uh oh!

doris-robot commented Oct 9, 2025

Uh oh!

hello-stephen commented Oct 9, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Oct 9, 2025

BE Regression && UT Coverage Report

Uh oh!

Uh oh!

0AyanamiRei commented Oct 10, 2025

Uh oh!

doris-robot commented Oct 10, 2025

Uh oh!

doris-robot commented Oct 10, 2025

Uh oh!

doris-robot commented Oct 10, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Oct 10, 2025

BE Regression && UT Coverage Report

Uh oh!

eldenmoon left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 11, 2025

Uh oh!

github-actions bot commented Oct 11, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

0AyanamiRei commented Oct 4, 2025 •

edited

Loading