Fix JSON paths by Vladilen · Pull Request #30243 · ydb-platform/ydb

Vladilen · 2025-12-05T17:51:47Z

Changelog entry

...

Changelog category

Not for changelog (changelog entry is not required)

Description for reviewers

...

ydbot · 2025-12-05T17:53:16Z

Run Extra Tests

Run additional tests for this PR. You can customize:

Test Size: small, medium, large (default: all)
Test Targets: any directory path (default: ydb/)
Sanitizers: ASAN, MSAN, TSAN
Coredumps: enable for debugging (default: off)
Additional args: custom ya make arguments

Copilot

Pull request overview

Copilot reviewed 26 out of 26 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

github-actions · 2025-12-08T15:05:48Z

⚪ 2025-12-08 15:05:47 UTC Pre-commit check linux-x86_64-relwithdebinfo for 940e5ef has started.
⚪ 2025-12-08 15:06:05 UTC Artifacts will be uploaded here
⚪ 2025-12-08 15:08:14 UTC ya make is running...
🟡 2025-12-08 17:13:34 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
39479	36553	0	3	2903	20

⚪ 2025-12-08 17:13:48 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-12-08 17:23:29 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
42 (only retried tests)	29	0	0	0	13

🟢 2025-12-08 17:23:35 UTC Build successful.
🟡 2025-12-08 17:23:54 UTC ydbd size 2.3 GiB changed* by +220.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `fe86628`	merge: `940e5ef`	diff	diff %
ydbd size	2 467 827 848 Bytes	2 468 053 880 Bytes	+220.7 KiB	+0.009%
ydbd stripped size	525 130 592 Bytes	525 150 048 Bytes	+19.0 KiB	+0.004%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-12-08T15:07:04Z

⚪ 2025-12-08 15:07:04 UTC Pre-commit check linux-x86_64-release-asan for 940e5ef has started.
⚪ 2025-12-08 15:07:20 UTC Artifacts will be uploaded here
⚪ 2025-12-08 15:09:28 UTC ya make is running...
🟡 2025-12-08 16:51:06 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
12064	11992	0	55	7	10

🟢 2025-12-08 16:51:15 UTC Build successful.
🟡 2025-12-08 16:51:38 UTC ydbd size 3.8 GiB changed* by +310.9 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `fe86628`	merge: `940e5ef`	diff	diff %
ydbd size	4 131 363 280 Bytes	4 131 681 632 Bytes	+310.9 KiB	+0.008%
ydbd stripped size	1 533 585 400 Bytes	1 533 654 072 Bytes	+67.1 KiB	+0.004%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

Copilot

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot

Copilot reviewed 25 out of 25 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

dorooleg · 2025-12-09T09:29:19Z

+    auto idx = accessor->GetCookie();
    if (!idx) {
-        return std::make_shared<TSparsedArray>(nullptr, arrow::binary(), recordsCount);
+        return std::shared_ptr<TJsonPathAccessor>{};


А в каком случае это возможно? И как тут раньше SparsedArray появлялся?

sparsedArray хранил колчиество записей и можно было иметь доступ к каждой, чтобы она возвращала null. Я так и не понял зачем это нужно.
Вообще индекс легко может не найтись, если путь будет несуществующий, перепроверю что на это есть тест

github-actions · 2025-12-09T16:24:02Z

⚪ 2025-12-09 16:24:01 UTC Pre-commit check linux-x86_64-relwithdebinfo for 9daa5f6 has started.
⚪ 2025-12-09 16:24:36 UTC Artifacts will be uploaded here
⚪ 2025-12-09 16:26:31 UTC ya make is running...
🟡 2025-12-09 18:34:13 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40191	37253	0	5	2902	31

⚪ 2025-12-09 18:34:26 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-12-09 18:52:34 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
53 (only retried tests)	39	0	0	0	14

🟢 2025-12-09 18:52:41 UTC Build successful.
🟡 2025-12-09 18:53:07 UTC ydbd size 2.3 GiB changed* by +215.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `76cdf70`	merge: `9daa5f6`	diff	diff %
ydbd size	2 467 367 680 Bytes	2 467 588 584 Bytes	+215.7 KiB	+0.009%
ydbd stripped size	525 092 928 Bytes	525 107 648 Bytes	+14.4 KiB	+0.003%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-12-09T16:26:16Z

⚪ 2025-12-09 16:26:15 UTC Pre-commit check linux-x86_64-release-asan for 9daa5f6 has started.
⚪ 2025-12-09 16:26:32 UTC Artifacts will be uploaded here
⚪ 2025-12-09 16:28:42 UTC ya make is running...
🟡 2025-12-09 18:10:44 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
12105	12036	0	55	7	7

🟢 2025-12-09 18:10:53 UTC Build successful.
🟡 2025-12-09 18:11:25 UTC ydbd size 3.8 GiB changed* by +304.9 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `76cdf70`	merge: `9daa5f6`	diff	diff %
ydbd size	4 131 816 288 Bytes	4 132 128 496 Bytes	+304.9 KiB	+0.008%
ydbd stripped size	1 533 780 216 Bytes	1 533 850 456 Bytes	+68.6 KiB	+0.005%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

Copilot

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

ydbot · 2025-12-10T09:05:30Z

Backport

To backport this PR, click the button next to the target branch and then click "Run workflow" in the Run Actions UI.

Branch	Run
`stable-25-2, stable-25-2-1, stable-25-3, stable-25-3-1`
`stable-25-3, stable-25-3-1`
`stable-25-3`

Vladilen added 22 commits November 17, 2025 11:19

Remove double BinaryJson parsing

f7a297c

Add array and object parsing

608b53c

Refactoring

1fba8e7

Fix review notes

dfd73df

Fix review notes

10a497b

Fix review notes

85196e1

Fix review notes

c8f8bb1

Merge remote-tracking branch 'official/main' into json-binary-parsing

2e7191a

Fix first level double parsing

db6d2f6

Merge remote-tracking branch 'official/main' into json-binary-parsing

1064744

Json optimisations WIP

925bbbf

Add tests and some parsing fixes

fa8f848

wip

5ce2cf2

Fix bugs and add tests

91d9044

Refactor tests

f6e551f

Implement in accessor

bf5362f

Merge remote-tracking branch 'official/main' into tmp

caa1c87

Fix some tests

d4f9f62

Merge remote-tracking branch 'official/main' into tmp

f66a3b4

Fix some tests

b2321e4

Fix all tests

d5c8e94

Removed excess changes

501a614

Copilot AI review requested due to automatic review settings December 5, 2025 17:51

Vladilen requested review from a team as code owners December 5, 2025 17:51

github-actions Bot added the not-for-changelog label Dec 5, 2025

Vladilen requested review from XJIE6 and dorooleg December 5, 2025 17:52

Copilot started reviewing on behalf of Vladilen December 5, 2025 17:54 View session

dorooleg requested a review from Copilot December 8, 2025 13:22

dorooleg reviewed Dec 8, 2025

View reviewed changes

Comment thread ydb/core/tx/columnshard/engines/reader/common_reader/iterator/default_fetching.h Outdated

Copilot started reviewing on behalf of dorooleg December 8, 2025 13:29 View session

Copilot AI reviewed Dec 8, 2025

View reviewed changes

Fix review notes

4414a91

Vladilen requested a review from Copilot December 8, 2025 15:02

Copilot started reviewing on behalf of Vladilen December 8, 2025 15:04 View session

Copilot AI reviewed Dec 8, 2025

View reviewed changes

Comment thread ydb/core/formats/arrow/accessor/sub_columns/partial.cpp

Copilot AI reviewed Dec 8, 2025

View reviewed changes

Vladilen requested a review from Copilot December 9, 2025 08:37

Copilot started reviewing on behalf of Vladilen December 9, 2025 08:38 View session

Copilot AI reviewed Dec 9, 2025

View reviewed changes

dorooleg requested changes Dec 9, 2025

View reviewed changes

Fix review notes

b151ccb

Vladilen requested review from Copilot and dorooleg December 9, 2025 16:23

Copilot started reviewing on behalf of Vladilen December 9, 2025 16:24 View session

Copilot AI reviewed Dec 9, 2025

View reviewed changes

Comment thread ydb/core/formats/arrow/accessor/sub_columns/json_value_path.cpp

Comment thread ydb/core/formats/arrow/accessor/sub_columns/json_value_path.cpp

Comment thread ydb/core/formats/arrow/accessor/sub_columns/json_value_path.h

Copilot AI reviewed Dec 9, 2025

View reviewed changes

dorooleg approved these changes Dec 10, 2025

View reviewed changes

denis0x0D approved these changes Dec 10, 2025

View reviewed changes

Vladilen merged commit b890600 into ydb-platform:main Dec 10, 2025
15 checks passed

Vladilen mentioned this pull request Dec 10, 2025

JSON_VALUE(json_payload, "$.\"host\"") = "fqdn" not working on CL EXP #19125

Closed

This was referenced Dec 10, 2025

Make max deadline for put in DSProxy customizable via ICB #30501

Merged

Add external canceller for TEvPut requests #30553

Merged

Conversation

Vladilen commented Dec 5, 2025

Changelog entry

Changelog category

Description for reviewers

Uh oh!

ydbot commented Dec 5, 2025

Run Extra Tests

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dorooleg Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

Vladilen Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ydbot commented Dec 10, 2025

Backport

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

github-actions Bot commented Dec 8, 2025 •

edited

Loading

github-actions Bot commented Dec 8, 2025 •

edited

Loading

github-actions Bot commented Dec 9, 2025 •

edited

Loading

github-actions Bot commented Dec 9, 2025 •

edited

Loading