From fc0634f3b8eac484ebfa63ac8e369b1cdc37e6db Mon Sep 17 00:00:00 2001 From: Jianjun Liao Date: Mon, 10 Mar 2025 12:35:56 +0800 Subject: [PATCH 1/4] draft Signed-off-by: Jianjun Liao --- br/backup-and-restore-use-cases.md | 4 +++- br/br-pitr-guide.md | 4 +++- br/br-pitr-manual.md | 4 +++- br/br-snapshot-guide.md | 25 ++++++++++++++++++++++--- br/br-snapshot-manual.md | 4 +++- 5 files changed, 34 insertions(+), 7 deletions(-) diff --git a/br/backup-and-restore-use-cases.md b/br/backup-and-restore-use-cases.md index 1bb46bd970603..3408ce5d19b18 100644 --- a/br/backup-and-restore-use-cases.md +++ b/br/backup-and-restore-use-cases.md @@ -145,7 +145,9 @@ tiup br restore point --pd="${PD_IP}:2379" \ --full-backup-storage='s3://tidb-pitr-bucket/backup-data/snapshot-20220514000000' \ --restored-ts '2022-05-15 18:00:00+0800' -Full Restore <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Split&Scatter Region <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Download&Ingest SST <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Restore Pipeline <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% [2022/05/29 18:15:39.132 +08:00] [INFO] [collector.go:69] ["Full Restore success summary"] [total-ranges=12] [ranges-succeed=xxx] [ranges-failed=0] [split-region=xxx.xxxµs] [restore-ranges=xxx] [total-take=xxx.xxxs] [restore-data-size(after-compressed)=xxx.xxx] [Size=xxxx] [BackupTS={TS}] [total-kv=xxx] [total-kv-size=xxx] [average-speed=xxx] Restore Meta Files <--------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% Restore KV Files <----------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% diff --git a/br/br-pitr-guide.md b/br/br-pitr-guide.md index ef0f24f3d8bde..5feeb9450a097 100644 --- a/br/br-pitr-guide.md +++ b/br/br-pitr-guide.md @@ -69,7 +69,9 @@ tiup br restore point --pd "${PD_IP}:2379" \ During data restore, you can view the progress through the progress bar in the terminal. The restore is divided into two phases, full restore and log restore (restore meta files and restore KV files). After each phase is completed, `br` outputs information such as restore time and data size. ```shell -Full Restore <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Split&Scatter Region <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Download&Ingest SST <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Restore Pipeline <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% *** ["Full Restore success summary"] ****** [total-take=xxx.xxxs] [restore-data-size(after-compressed)=xxx.xxx] [Size=xxxx] [BackupTS={TS}] [total-kv=xxx] [total-kv-size=xxx] [average-speed=xxx] Restore Meta Files <--------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% Restore KV Files <----------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% diff --git a/br/br-pitr-manual.md b/br/br-pitr-manual.md index 22711ce5dffdd..e36d345c3384e 100644 --- a/br/br-pitr-manual.md +++ b/br/br-pitr-manual.md @@ -453,7 +453,9 @@ tiup br restore point --pd="${PD_IP}:2379" --storage='s3://backup-101/logbackup?access-key=${access-key}&secret-access-key=${secret-access-key}' --full-backup-storage='s3://backup-101/snapshot-202205120000?access-key=${access-key}&secret-access-key=${secret-access-key}' -Full Restore <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Split&Scatter Region <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Download&Ingest SST <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% +Restore Pipeline <--------------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% *** ***["Full Restore success summary"] ****** [total-take=3.112928252s] [restore-data-size(after-compressed)=5.056kB] [Size=5056] [BackupTS=434693927394607136] [total-kv=4] [total-kv-size=290B] [average-speed=93.16B/s] Restore Meta Files <--------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% Restore KV Files <----------------------------------------------------------------------------------------------------------------------------------------------------> 100.00% diff --git a/br/br-snapshot-guide.md b/br/br-snapshot-guide.md index 4ef4a957630bc..4204f79efb2a1 100644 --- a/br/br-snapshot-guide.md +++ b/br/br-snapshot-guide.md @@ -39,10 +39,17 @@ In the preceding command: During backup, a progress bar is displayed in the terminal as shown below. When the progress bar advances to 100%, the backup task is completed and statistics such as total backup time, average backup speed, and backup data size are displayed. +- `total-ranges` indicates the total number of files backed up +- `ranges-succeed` indicates the number of files successfully backed up +- `ranges-failed` indicates the number of files that failed to be backed up +- `backup-total-ranges` indicates the number of tables (including partitions) and indexes backed up +- `write-CF-files` indicates the number of backed up SSTs containing `write CF` data +- `default-CF-files` indicates the number of backed up SSTs containing `default CF` data + ```shell Full Backup <-------------------------------------------------------------------------------> 100.00% Checksum <----------------------------------------------------------------------------------> 100.00% -*** ["Full Backup success summary"] *** [backup-checksum=3.597416ms] [backup-fast-checksum=2.36975ms] *** [total-take=4.715509333s] [BackupTS=435844546560000000] [total-kv=1131] [total-kv-size=250kB] [average-speed=53.02kB/s] [backup-data-size(after-compressed)=71.33kB] [Size=71330] +*** ["Full Backup success summary"] *** [total-ranges=20] [ranges-succeed=20] [ranges-failed=0] [backup-checksum=3.597416ms] [backup-fast-checksum=2.36975ms] [backup-total-ranges=11] [backup-total-regions=10] [write-CF-files=14] [default-CF-files=6] [total-take=4.715509333s] [BackupTS=435844546560000000] [total-kv=1131] [total-kv-size=250kB] [average-speed=53.02kB/s] [backup-data-size(after-compressed)=71.33kB] [Size=71330] ``` ## Get the backup time point of a snapshot backup @@ -81,9 +88,21 @@ tiup br restore full --pd "${PD_IP}:2379" \ During restore, a progress bar is displayed in the terminal as shown below. When the progress bar advances to 100%, the restore task is completed and statistics such as total restore time, average restore speed, and total data size are displayed. +- `total-ranges` indicates the total number of files restored +- `ranges-succeed` indicates the number of files successfully restored +- `ranges-failed` indicates the number of files that failed to be restored +- `merge-ranges` indicates the time taken to merge the data range +- `split-region` indicates the time taken to split and scatter regions +- `restore-files` indicates the time it takes TiKV to download and ingest SSTs +- `write-CF-files` indicates the number of restored SSTs containing `write CF` data +- `default-CF-files` indicates the number of restored SSTs containing `default CF` data +- `split-keys` indicates the number of keys generated for splitting regions + ```shell -Full Restore <------------------------------------------------------------------------------> 100.00% -*** ["Full Restore success summary"] *** [total-take=4.344617542s] [total-kv=5] [total-kv-size=327B] [average-speed=75.27B/s] [restore-data-size(after-compressed)=4.813kB] [Size=4813] [BackupTS=435844901803917314] +Split&Scatter Region <--------------------------------------------------------------------> 100.00% +Download&Ingest SST <---------------------------------------------------------------------> 100.00% +Restore Pipeline <------------------------------------------------------------------------> 100.00% +*** ["Full Restore success summary"] [total-ranges=20] [ranges-succeed=20] [ranges-failed=0] [merge-ranges=7.546971ms] [split-region=343.594072ms] [restore-files=1.57662s] [default-CF-files=6] [write-CF-files=14] [split-keys=9] [total-take=4.344617542s] [total-kv=5] [total-kv-size=327B] [average-speed=75.27B/s] [restore-data-size(after-compressed)=4.813kB] [Size=4813] [BackupTS=435844901803917314] ``` ### Restore a database or a table diff --git a/br/br-snapshot-manual.md b/br/br-snapshot-manual.md index c4c11246dbc92..aeccb16e55a6c 100644 --- a/br/br-snapshot-manual.md +++ b/br/br-snapshot-manual.md @@ -181,7 +181,9 @@ In the preceding command: During restore, a progress bar is displayed in the terminal as shown below. When the progress bar advances to 100%, the restore task is completed. Then `br` will verify the restored data to ensure data security. ```shell -Full Restore <---------/...............................................> 17.12%. +Split&Scatter Region <--------------------------------------------------------------------> 100.00% +Download&Ingest SST <---------------------------------------------------------------------> 100.00% +Restore Pipeline <-------------------------/...............................................> 17.12% ``` ## Restore a database or a table From 1a4f407999443cedaed8af94d87bbad1fdbfc940 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 14 Mar 2025 09:15:19 +0800 Subject: [PATCH 2/4] Apply suggestions from code review --- br/br-snapshot-guide.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/br/br-snapshot-guide.md b/br/br-snapshot-guide.md index 4204f79efb2a1..2ea10338f0c9f 100644 --- a/br/br-snapshot-guide.md +++ b/br/br-snapshot-guide.md @@ -88,15 +88,15 @@ tiup br restore full --pd "${PD_IP}:2379" \ During restore, a progress bar is displayed in the terminal as shown below. When the progress bar advances to 100%, the restore task is completed and statistics such as total restore time, average restore speed, and total data size are displayed. -- `total-ranges` indicates the total number of files restored -- `ranges-succeed` indicates the number of files successfully restored -- `ranges-failed` indicates the number of files that failed to be restored -- `merge-ranges` indicates the time taken to merge the data range -- `split-region` indicates the time taken to split and scatter regions -- `restore-files` indicates the time it takes TiKV to download and ingest SSTs -- `write-CF-files` indicates the number of restored SSTs containing `write CF` data -- `default-CF-files` indicates the number of restored SSTs containing `default CF` data -- `split-keys` indicates the number of keys generated for splitting regions +- `total-ranges`: indicates the total number of files that are restored +- `ranges-succeed`: indicates the number of files that are successfully restored +- `ranges-failed`: indicates the number of files that failed to be restored +- `merge-ranges`: indicates the time taken to merge the data range +- `split-region`: indicates the time taken to split and scatter regions +- `restore-files`: indicates the time it takes TiKV to download and ingest SST files +- `write-CF-files`: indicates the number of restored SST files that contain `write CF` data +- `default-CF-files`: indicates the number of restored SST files that contain `default CF` data +- `split-keys`: indicates the number of keys generated for splitting regions ```shell Split&Scatter Region <--------------------------------------------------------------------> 100.00% From 7b0765e6ac532461dec541d56d2a8378ff61e8e3 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 14 Mar 2025 09:15:33 +0800 Subject: [PATCH 3/4] Apply suggestions from code review --- br/br-snapshot-guide.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/br/br-snapshot-guide.md b/br/br-snapshot-guide.md index 2ea10338f0c9f..a5385dbc2bca5 100644 --- a/br/br-snapshot-guide.md +++ b/br/br-snapshot-guide.md @@ -39,12 +39,12 @@ In the preceding command: During backup, a progress bar is displayed in the terminal as shown below. When the progress bar advances to 100%, the backup task is completed and statistics such as total backup time, average backup speed, and backup data size are displayed. -- `total-ranges` indicates the total number of files backed up -- `ranges-succeed` indicates the number of files successfully backed up -- `ranges-failed` indicates the number of files that failed to be backed up -- `backup-total-ranges` indicates the number of tables (including partitions) and indexes backed up -- `write-CF-files` indicates the number of backed up SSTs containing `write CF` data -- `default-CF-files` indicates the number of backed up SSTs containing `default CF` data +- `total-ranges`: indicates the total number of files that are backed up +- `ranges-succeed`: indicates the number of files that are successfully backed up +- `ranges-failed`: indicates the number of files that failed to be backed up +- `backup-total-ranges`: indicates the number of tables (including partitions) and indexes that are backed up +- `write-CF-files`: indicates the number of backup SST files that contain `write CF` data +- `default-CF-files`: indicates the number of backup SST files that contain `default CF` data ```shell Full Backup <-------------------------------------------------------------------------------> 100.00% From 2122b4f81f1a9c2b572f35a07fa563506e2dded0 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Mon, 17 Mar 2025 11:50:24 +0800 Subject: [PATCH 4/4] Apply suggestions from code review Co-authored-by: Grace Cai --- br/br-snapshot-guide.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/br/br-snapshot-guide.md b/br/br-snapshot-guide.md index a5385dbc2bca5..4bb9bc9f1582d 100644 --- a/br/br-snapshot-guide.md +++ b/br/br-snapshot-guide.md @@ -39,12 +39,12 @@ In the preceding command: During backup, a progress bar is displayed in the terminal as shown below. When the progress bar advances to 100%, the backup task is completed and statistics such as total backup time, average backup speed, and backup data size are displayed. -- `total-ranges`: indicates the total number of files that are backed up -- `ranges-succeed`: indicates the number of files that are successfully backed up -- `ranges-failed`: indicates the number of files that failed to be backed up -- `backup-total-ranges`: indicates the number of tables (including partitions) and indexes that are backed up -- `write-CF-files`: indicates the number of backup SST files that contain `write CF` data -- `default-CF-files`: indicates the number of backup SST files that contain `default CF` data +- `total-ranges`: indicates the total number of files to be backed up. +- `ranges-succeed`: indicates the number of files that are successfully backed up. +- `ranges-failed`: indicates the number of files that failed to be backed up. +- `backup-total-ranges`: indicates the number of tables (including partitions) and indexes that are to be backed up. +- `write-CF-files`: indicates the number of backup SST files that contain `write CF` data. +- `default-CF-files`: indicates the number of backup SST files that contain `default CF` data. ```shell Full Backup <-------------------------------------------------------------------------------> 100.00% @@ -88,15 +88,15 @@ tiup br restore full --pd "${PD_IP}:2379" \ During restore, a progress bar is displayed in the terminal as shown below. When the progress bar advances to 100%, the restore task is completed and statistics such as total restore time, average restore speed, and total data size are displayed. -- `total-ranges`: indicates the total number of files that are restored -- `ranges-succeed`: indicates the number of files that are successfully restored -- `ranges-failed`: indicates the number of files that failed to be restored -- `merge-ranges`: indicates the time taken to merge the data range -- `split-region`: indicates the time taken to split and scatter regions -- `restore-files`: indicates the time it takes TiKV to download and ingest SST files -- `write-CF-files`: indicates the number of restored SST files that contain `write CF` data -- `default-CF-files`: indicates the number of restored SST files that contain `default CF` data -- `split-keys`: indicates the number of keys generated for splitting regions +- `total-ranges`: indicates the total number of files that are to be restored. +- `ranges-succeed`: indicates the number of files that are successfully restored. +- `ranges-failed`: indicates the number of files that failed to be restored. +- `merge-ranges`: indicates the time taken to merge the data range. +- `split-region`: indicates the time taken to split and scatter Regions. +- `restore-files`: indicates the time TiKV takes to download and ingest SST files. +- `write-CF-files`: indicates the number of restored SST files that contain `write CF` data. +- `default-CF-files`: indicates the number of restored SST files that contain `default CF` data. +- `split-keys`: indicates the number of keys generated for splitting Regions. ```shell Split&Scatter Region <--------------------------------------------------------------------> 100.00%