Skip to content

Export / import s3 for topics configuration docs #30232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
stanislav-shchetinin wants to merge 2 commits into
base: main
Choose a base branch
from
+11 −2
Open

Export / import s3 for topics configuration docs #30232

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
806cc08
topics in export
stanislav-shchetinin Dec 5, 2025
8085aef
cli
stanislav-shchetinin Dec 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions ydb/docs/en/core/reference/ydb-cli/export-import/_includes/file_structure.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,9 @@ For each table in the database, there's a same-name directory in the file struct
- The `changefeed_description.pb` file describing the changefeed in the [text protobuf](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.text_format) format
- The `topic_description.pb` file describing the underlying topic in the [text protobuf](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.text_format) format

## Topics {#topics}

For each topic in the database, there's a same-name directory in the file structure's directory hierarchy that includes the create_topic.pb file. This file provides information about the topic parameters, partitioning settings, and consumers in the [text protobuf](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.text_format) format.
Copy link

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The filename create_topic.pb should be formatted with backticks for consistency with other file references in the documentation (e.g., changefeed_description.pb and topic_description.pb on lines 73-74).

Suggested change
For each topic in the database, there's a same-name directory in the file structure's directory hierarchy that includes the create_topic.pb file. This file provides information about the topic parameters, partitioning settings, and consumers in the [text protobuf](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.text_format) format.
For each topic in the database, there's a same-name directory in the file structure's directory hierarchy that includes the `create_topic.pb` file. This file provides information about the topic parameters, partitioning settings, and consumers in the [text protobuf](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.text_format) format.

Copilot uses AI. Check for mistakes.

## Files with data {#datafiles}

Expand Down
1 change: 1 addition & 0 deletions ydb/docs/en/core/reference/ydb-cli/export-import/_includes/s3_export.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ The export feature is available only for objects of the following types:
- [Row-oriented table](../../../../concepts/datamodel/table.md#row-oriented-tables)
- [Secondary index](../../../../concepts/glossary.md#secondary-index)
- [Vector index](../../../../concepts/glossary.md#vector-index)
- [Topic](../../../../concepts/datamodel/topic.md) ()
Copy link

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parentheses appear to be incomplete. Based on the Russian version of this file, this should read "(only the schema, without messages)" to match the description "только схема, без сообщений".

Suggested change
- [Topic](../../../../concepts/datamodel/topic.md) ()
- [Topic](../../../../concepts/datamodel/topic.md) (only the schema, without messages)

Copilot uses AI. Check for mistakes.

{% endnote %}

Expand Down
5 changes: 3 additions & 2 deletions ydb/docs/ru/core/reference/ydb-cli/export-import/_includes/export-s3.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@

- [директория](../../../../concepts/datamodel/dir.md);
- [строковая таблица](../../../../concepts/datamodel/table.md#row-oriented-tables);
- [вторичный индекс](../../../../concepts/glossary.md#secondary-index).
- [векторный индекс](../../../../concepts/glossary.md#vector-index).
- [вторичный индекс](../../../../concepts/glossary.md#secondary-index);
- [векторный индекс](../../../../concepts/glossary.md#vector-index);
- [топик](../../../../concepts/datamodel/topic.md) (only the schema, without messages).
Copy link

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inconsistent use of language. The line contains Russian text "только схема, без сообщений" within English documentation. This should be translated to English: "(only the schema, without messages)".

Suggested change
- [топик](../../../../concepts/datamodel/topic.md) (only the schema, without messages).
- [топик](../../../../concepts/datamodel/topic.md) (только схема, без сообщений).

Copilot uses AI. Check for mistakes.
Copy link
Member

@CyberROFL CyberROFL Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Поддерживаю.

Copy link
Member

@CyberROFL CyberROFL Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- [топик](../../../../concepts/datamodel/topic.md) (only the schema, without messages).
- [топик](../../../../concepts/datamodel/topic.md) (только схема).


Для более простого экспорта одиночных строковых и колоночных таблиц в S3-совместимое хранилище данных можно использовать [внешние источники данных](../../../../concepts/datamodel/external_data_source.md). Подробнее см. в статье [{#T}](../../../../concepts/federated_query/s3/write_data.md#export-to-s3).

Expand Down
4 changes: 4 additions & 0 deletions ydb/docs/ru/core/reference/ydb-cli/export-import/_includes/file-structure.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,10 @@
- Файл `changefeed_description.pb`, содержащий информацию о потоке изменений в формате [text protobuf](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.text_format)
- Файл `topic_description.pb`, содержащий информацию о нижележащем топике в формате [text protobuf](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.text_format)

## Топики {#topics}

Каждому топику в базе данных также соответствует одноименная директория в иерархии директорий файловой структуры, в которой находится файл `create_topic.pb`, содержащий информацию о параметрах топика, настройках партиционирования и консьюмерах в формате [text protobuf](https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.text_format)

## Файлы с данными {#datafiles}

Формат файлов с данными - `.csv`, одна строка соответствует одной записи в таблице, без строки с заголовками колонок. Для строк применяется представление в urlencoded формате. Например, строка файла для таблицы с колонками uint64 и utf8, содержащая число 1 и строку "Привет" соответственно, выглядит таким образом:
Expand Down
Loading