Skip to content

Conversation

@stebloev
Copy link
Member

@stebloev stebloev commented Dec 5, 2025

Refined workload vector init

Minor changes and now workload vector init specifies partition settings for data table

Changelog category

  • Not for changelog

Description for reviewers

...

@stebloev stebloev self-assigned this Dec 5, 2025
@stebloev stebloev linked an issue Dec 5, 2025 that may be closed by this pull request
@github-actions
Copy link

github-actions bot commented Dec 5, 2025

2025-12-05 14:01:10 UTC Pre-commit check linux-x86_64-relwithdebinfo for ee750ea has started.
2025-12-05 14:01:26 UTC Artifacts will be uploaded here
2025-12-05 14:03:35 UTC ya make is running...
🟢 2025-12-05 14:46:31 UTC Tests successful.

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
4650 4542 0 0 95 13

🟢 2025-12-05 14:46:37 UTC Build successful.
🟢 2025-12-05 14:46:55 UTC ydbd size 2.3 GiB changed* by 0 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash main: b90d453 merge: ee750ea diff diff %
ydbd size 2 466 095 936 Bytes 2 466 095 936 Bytes 0 Bytes 0.000%
ydbd stripped size 524 834 240 Bytes 524 834 240 Bytes 0 Bytes 0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Dec 5, 2025

2025-12-05 14:01:23 UTC Pre-commit check linux-x86_64-release-asan for ee750ea has started.
2025-12-05 14:01:40 UTC Artifacts will be uploaded here
2025-12-05 14:03:58 UTC ya make is running...
🟡 2025-12-05 14:38:16 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
1447 1424 0 16 1 6

🟢 2025-12-05 14:38:23 UTC Build successful.
🟢 2025-12-05 14:38:42 UTC ydbd size 3.8 GiB changed* by -96 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash main: b90d453 merge: ee750ea diff diff %
ydbd size 4 127 729 816 Bytes 4 127 729 720 Bytes -96 Bytes -0.000%
ydbd stripped size 1 532 666 648 Bytes 1 532 666 584 Bytes -64 Bytes -0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@stebloev stebloev requested a review from Copilot December 5, 2025 14:01
@ydbot
Copy link
Collaborator

ydbot commented Dec 5, 2025

Run Extra Tests

Run additional tests for this PR. You can customize:

  • Test Size: small, medium, large (default: all)
  • Test Targets: any directory path (default: ydb/)
  • Sanitizers: ASAN, MSAN, TSAN
  • Coredumps: enable for debugging (default: off)
  • Additional args: custom ya make arguments

▶  Run tests

@github-actions
Copy link

github-actions bot commented Dec 5, 2025

🟢 2025-12-09 13:18:10 UTC The validation of the Pull Request description is successful.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for configuring table partitioning options when initializing tables using the workload vector init command. The changes refactor the configuration structure to use dedicated option structs and expose partitioning settings to users.

  • Refactored configuration to use TTableOpts, TTablePartitioningOpts, and TVectorOpts structs
  • Added command-line options for controlling table partitioning behavior (min partitions, partition size, auto-partitioning by size/load)
  • Updated DDL generation to include user-specified partitioning settings

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
ydb/library/workload/vector/vector_workload_params.h Reorganized member variables to use new structured option types (TTableOpts, TTablePartitioningOpts, TVectorOpts)
ydb/library/workload/vector/vector_workload_params.cpp Replaced manual init parameter configuration with calls to new Configure*Opts functions; updated all references from TableName to TableOpts.Name
ydb/library/workload/vector/configure_opts.h Defined new structs for table options and partitioning options with default values
ydb/library/workload/vector/configure_opts.cpp Implemented configuration functions for table and partitioning options with command-line parameter definitions
ydb/library/workload/vector/vector_workload_generator.cpp Updated DDL query generation to dynamically include partitioning settings based on user configuration
ydb/library/workload/vector/vector_sql.cpp Updated table reference from TableName to TableOpts.Name
ydb/library/workload/vector/vector_sampler.cpp Updated table references from TableName to TableOpts.Name
ydb/library/workload/vector/vector_recall_evaluator.cpp Updated table reference from TableName to TableOpts.Name
ydb/library/workload/vector/vector_data_generator.cpp Updated all table name references from TableName to TableOpts.Name
ydb/library/workload/vector/vector_command_index.cpp Updated table name references from TableName to TableOpts.Name
ydb/apps/ydb/CHANGELOG.md Added changelog entry documenting the new partitioning settings support

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@stebloev stebloev marked this pull request as ready for review December 5, 2025 15:22
@stebloev stebloev requested a review from a team as a code owner December 5, 2025 15:22
@github-actions
Copy link

github-actions bot commented Dec 5, 2025

2025-12-05 15:23:34 UTC Pre-commit check linux-x86_64-release-asan for 99b354b has started.
2025-12-05 15:23:51 UTC Artifacts will be uploaded here
2025-12-05 15:26:00 UTC ya make is running...
🟡 2025-12-05 16:03:46 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
1447 1425 0 12 1 9

🟢 2025-12-05 16:03:52 UTC Build successful.
🟢 2025-12-05 16:04:12 UTC ydbd size 3.8 GiB changed* by -96 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash main: 1d6c232 merge: 99b354b diff diff %
ydbd size 4 127 809 296 Bytes 4 127 809 200 Bytes -96 Bytes -0.000%
ydbd stripped size 1 532 692 760 Bytes 1 532 692 696 Bytes -64 Bytes -0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@stebloev stebloev requested a review from pnv1 December 5, 2025 15:24
@github-actions
Copy link

github-actions bot commented Dec 5, 2025

2025-12-05 15:25:57 UTC Pre-commit check linux-x86_64-relwithdebinfo for 99b354b has started.
2025-12-05 15:26:14 UTC Artifacts will be uploaded here
2025-12-05 15:28:22 UTC ya make is running...
🟡 2025-12-05 16:08:13 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
4650 4538 0 1 95 16

2025-12-05 16:08:20 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-12-05 16:22:11 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
30 (only retried tests) 22 0 0 0 8

🟢 2025-12-05 16:22:18 UTC Build successful.
🟢 2025-12-05 16:22:35 UTC ydbd size 2.3 GiB changed* by 0 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash main: 1d6c232 merge: 99b354b diff diff %
ydbd size 2 466 152 008 Bytes 2 466 152 008 Bytes 0 Bytes 0.000%
ydbd stripped size 524 843 136 Bytes 524 843 136 Bytes 0 Bytes 0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Dec 9, 2025

2025-12-09 11:00:55 UTC Pre-commit check linux-x86_64-relwithdebinfo for bf5f357 has started.
2025-12-09 11:01:11 UTC Artifacts will be uploaded here
2025-12-09 11:03:18 UTC ya make is running...
2025-12-09 11:20:49 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Dec 9, 2025

2025-12-09 11:01:32 UTC Pre-commit check linux-x86_64-release-asan for bf5f357 has started.
2025-12-09 11:01:50 UTC Artifacts will be uploaded here
2025-12-09 11:04:05 UTC ya make is running...
2025-12-09 11:20:51 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Dec 9, 2025

2025-12-09 11:22:18 UTC Pre-commit check linux-x86_64-release-asan for 5ec8a8e has started.
2025-12-09 11:22:35 UTC Artifacts will be uploaded here
2025-12-09 11:24:48 UTC ya make is running...
🟡 2025-12-09 12:00:18 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
1447 1421 0 13 1 12

🟢 2025-12-09 12:00:26 UTC Build successful.
🟡 2025-12-09 12:00:48 UTC ydbd size 3.8 GiB changed* by +399.3 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: d43030a merge: 5ec8a8e diff diff %
ydbd size 4 131 194 304 Bytes 4 131 603 184 Bytes +399.3 KiB +0.010%
ydbd stripped size 1 533 582 744 Bytes 1 533 705 112 Bytes +119.5 KiB +0.008%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Dec 9, 2025

2025-12-09 11:23:53 UTC Pre-commit check linux-x86_64-relwithdebinfo for 5ec8a8e has started.
2025-12-09 11:24:31 UTC Artifacts will be uploaded here
2025-12-09 11:27:32 UTC ya make is running...
🟡 2025-12-09 12:10:58 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
4609 4499 0 1 93 16

2025-12-09 12:11:07 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-12-09 12:31:48 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
28 (only retried tests) 18 0 0 0 10

🟢 2025-12-09 12:31:56 UTC Build successful.
🟡 2025-12-09 12:32:19 UTC ydbd size 2.3 GiB changed* by +267.9 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: d43030a merge: 5ec8a8e diff diff %
ydbd size 2 466 964 168 Bytes 2 467 238 480 Bytes +267.9 KiB +0.011%
ydbd stripped size 525 014 272 Bytes 525 062 560 Bytes +47.2 KiB +0.009%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@stebloev stebloev changed the title Add partitioning options for workload vector init Refine workload vector init Dec 9, 2025
@stebloev stebloev merged commit 76cdf70 into ydb-platform:main Dec 9, 2025
11 checks passed
@stebloev stebloev deleted the workload-vector-init-refinement branch December 9, 2025 15:32
@ydbot
Copy link
Collaborator

ydbot commented Dec 9, 2025

Backport

To backport this PR, click the button next to the target branch and then click "Run workflow" in the Run Actions UI.

Branch Run
stable-25-2, stable-25-2-1, stable-25-3, stable-25-3-1 ▶  Backport
stable-25-3, stable-25-3-1 ▶  Backport
stable-25-3 ▶  Backport

▶  Backport manual

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Vector index workload

3 participants