Skip to content

Commit a113f53

Browse files
leto-bleto-b
authored
supplement param mods same as mods.enable to data sync (#971)
1 parent f1243f3 commit a113f53

16 files changed

Lines changed: 136 additions & 124 deletions

File tree

‎src/UserGuide/Master/Table/User-Manual/Data-Sync_apache.md‎

Lines changed: 6 additions & 5 deletions
Large diffs are not rendered by default.

‎src/UserGuide/Master/Table/User-Manual/Data-Sync_timecho.md‎

Lines changed: 6 additions & 7 deletions
Large diffs are not rendered by default.

‎src/UserGuide/Master/Tree/User-Manual/Data-Sync_apache.md‎

Lines changed: 11 additions & 11 deletions
Large diffs are not rendered by default.

‎src/UserGuide/Master/Tree/User-Manual/Data-Sync_timecho.md‎

Lines changed: 13 additions & 13 deletions
Large diffs are not rendered by default.

‎src/UserGuide/latest-Table/User-Manual/Data-Sync_apache.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -485,6 +485,7 @@ pipe_all_sinks_rate_limit_bytes_per_second=-1
485485
| table-name | When the user connects with `sql_dialect` set to `table`, this parameter can be specified. Determines the scope of data capture, affecting the `data` in `inclusion`. Specifies the table name to filter. It can be a specific table name or a Java-style regular expression to match multiple tables. By default, all tables are matched. | String: Data table name or data table regular expression pattern string, which can be uncreated or non - existent tables. | No | ".*" |
486486
| start-time | Determines the scope of data capture, affecting the `data` in `inclusion`. Data with an event time **greater than or equal to** this parameter will be selected for stream processing in the pipe. | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp)orString: ISO format timestamp supported by IoTDB | No | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp) |
487487
| end-time | Determines the scope of data capture, affecting the `data` in `inclusion`. Data with an event time **less than or equal to** this parameter will be selected for stream processing in the pipe. | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp)orString: ISO format timestamp supported by IoTDB | No | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp) |
488+
| mods | Same as mods.enable, whether to send the MODS file for TSFile. | Boolean: true / false | Optional | false |
488489

489490
> 💎 **Note:** The difference between the values of true and false for the data extraction mode `mode.streaming`
490491
>

‎src/UserGuide/latest-Table/User-Manual/Data-Sync_timecho.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -570,6 +570,7 @@ pipe_all_sinks_rate_limit_bytes_per_second=-1
570570
| start-time | Determines the scope of data capture, affecting the `data` in `inclusion`. Data with an event time **greater than or equal to** this parameter will be selected for stream processing in the pipe. | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp)orString: ISO format timestamp supported by IoTDB | No | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp) |
571571
| end-time | Determines the scope of data capture, affecting the `data` in `inclusion`. Data with an event time **less than or equal to** this parameter will be selected for stream processing in the pipe. | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp)orString: ISO format timestamp supported by IoTDB | No | Long: [Long.MIN_VALUE, Long.MAX_VALUE](Unix bare timestamp) |
572572
| mode.double-living | Whether to enable full dual-active mode. When enabled, the system will ignore the `-sql_dialect` connection method to capture all tree-table model data and not forward data synced from another pipe (to avoid circular synchronization). | Boolean: true / false | No | false |
573+
| mods | Same as mods.enable, whether to send the MODS file for TSFile. | Boolean: true / false | Optional | false |
573574

574575
> 💎 **Note:** The difference between the values of true and false for the data extraction mode `mode.streaming`
575576
>

‎src/UserGuide/latest/User-Manual/Data-Sync_apache.md‎

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -483,18 +483,19 @@ pipe_all_sinks_rate_limit_bytes_per_second=-1
483483

484484
### 5.1 source parameter
485485

486-
| key | value | value range | required or not | default value |
487-
| :------------------------------ | :----------------------------------------------------------- | :------------------------------------- | :------- | :------------- |
488-
| source | iotdb-source | String: iotdb-source | Required | - |
486+
| key | value | value range | required or not | default value |
487+
| :------------------------------ | :----------------------------------------------------------- |:-----------------------------------------------------------------------| :------- | :------------- |
488+
| source | iotdb-source | String: iotdb-source | Required | - |
489489
| inclusion | Used to specify the range of data to be synchronized in the data synchronization task, including data, schema, and auth | String:all, data(insert,delete), schema(database,timeseries,ttl), auth | Optional | data.insert |
490490
| inclusion.exclusion | Used to exclude specific operations from the range specified by inclusion, reducing the amount of data synchronized | String:all, data(insert,delete), schema(database,timeseries,ttl), auth | Optional | - |
491-
| mode.streaming | Specifies the capture source for time-series data writes. Applicable when mode.streamingis false, determining the source for capturing data.insertspecified in inclusion. Offers two strategies:- true: ​​Dynamic capture selection.​​ The system adaptively chooses between capturing individual write requests or only TsFile sealing requests based on downstream processing speed. Prioritizes capturing write requests for lower latency when processing is fast; captures only file sealing requests to avoid backlog when slow. Suitable for most scenarios, balancing latency and throughput optimally.- false: ​​Fixed batch capture.​​ Captures only TsFile sealing requests. Suitable for resource-constrained scenarios to reduce system load. Note: The snapshot data captured upon pipe startup is only provided to downstream processing in file format. | Boolean: true / false | 否 | true |
491+
| mode.streaming | Specifies the capture source for time-series data writes. Applicable when mode.streamingis false, determining the source for capturing data.insertspecified in inclusion. Offers two strategies:- true: ​​Dynamic capture selection.​​ The system adaptively chooses between capturing individual write requests or only TsFile sealing requests based on downstream processing speed. Prioritizes capturing write requests for lower latency when processing is fast; captures only file sealing requests to avoid backlog when slow. Suitable for most scenarios, balancing latency and throughput optimally.- false: ​​Fixed batch capture.​​ Captures only TsFile sealing requests. Suitable for resource-constrained scenarios to reduce system load. Note: The snapshot data captured upon pipe startup is only provided to downstream processing in file format. | Boolean: true / false | Optional | true |
492492
| mode.strict | Determines the strictness when filtering data using time/ path/ database-name/ table-nameparameters:- true: ​​Strict filtering.​​ The system strictly filters captured data according to the given conditions, ensuring only matching data is selected.- false: ​​Non-strict filtering.​​ The system may include some extra data during filtering. Suitable for performance-sensitive scenarios to reduce CPU and I/O consumption. | Boolean: true / false | Optional | true |
493493
| mode.snapshot | Determines the capture mode for time-series data, affecting the dataspecified in inclusion. Offers two modes:- true: ​​Static data capture.​​ Upon pipe startup, a one-time data snapshot is captured. ​​The pipe will automatically terminate (DROP PIPE SQL is executed automatically) after the snapshot data is fully consumed.​​- false: ​​Dynamic data capture.​​ In addition to capturing a snapshot upon startup, the pipe continuously captures subsequent data changes. The pipe runs continuously to handle the dynamic data stream. | Boolean: true / false | Optional | false |
494-
| path | Can be specified when the user connects with sql_dialectset to tree. For upgraded user pipes, the default sql_dialectis tree. This parameter determines the capture scope for time-series data, affecting the dataspecified in inclusion, as well as some sequence-related metadata. Data is selected into the streaming pipe if its tree model path matches the specified path. | String: IoTDB-standard tree path pattern, wildcards allowed | Optional | root.** |
495-
| start-time | The start event time for synchronizing all data, including start-time | Long: [Long.MIN_VALUE, Long.MAX_VALUE] | Optional | Long.MIN_VALUE |
496-
| end-time | The end event time for synchronizing all data, including end-time | Long: [Long.MIN_VALUE, Long.MAX_VALUE] | Optional | Long.MAX_VALUE |
497-
| forwarding-pipe-requests | Whether to forward data written by other Pipes (usually data synchronization) | Boolean: true | Optional | true |
494+
| path | Can be specified when the user connects with sql_dialectset to tree. For upgraded user pipes, the default sql_dialectis tree. This parameter determines the capture scope for time-series data, affecting the dataspecified in inclusion, as well as some sequence-related metadata. Data is selected into the streaming pipe if its tree model path matches the specified path. | String: IoTDB-standard tree path pattern, wildcards allowed | Optional | root.** |
495+
| start-time | The start event time for synchronizing all data, including start-time | Long: [Long.MIN_VALUE, Long.MAX_VALUE] | Optional | Long.MIN_VALUE |
496+
| end-time | The end event time for synchronizing all data, including end-time | Long: [Long.MIN_VALUE, Long.MAX_VALUE] | Optional | Long.MAX_VALUE |
497+
| forwarding-pipe-requests | Whether to forward data written by other Pipes (usually data synchronization) | Boolean: true /false | Optional | true |
498+
| mods | Same as mods.enable, whether to send the MODS file for TSFile. | Boolean: true / false | Optional | false |
498499

499500
> 💎 **Note:** The difference between the values of true and false for the data extraction mode `mode.streaming`
500501
>

0 commit comments

Comments
 (0)