Skip to content

Commit 24d99ef

Browse files
authored
[IOTDB-4829] Let NoMoreTsBlockEvent RPC is called in async way (apache#7911)
1 parent 523e82a commit 24d99ef

4 files changed

Lines changed: 108 additions & 82 deletions

File tree

docs/UserGuide/Reference/Syntax-Conventions.md

Lines changed: 19 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,11 @@
2020
-->
2121

2222

23-
# Literal Values
23+
## Literal Values
2424

2525
This section describes how to write literal values in IoTDB. These include strings, numbers, timestamp values, boolean values, and NULL.
2626

27-
## String Literals
27+
### String Literals
2828

2929
> We refer to MySQL's definition of string:A string is a sequence of bytes or characters, enclosed within either single quote (`'`) or double quote (`"`) characters.
3030
@@ -111,7 +111,7 @@ Usages of string literals:
111111
- The key/value of an attribute can be String Literal and identifier, more details can be found at **key-value pair** part.
112112

113113

114-
### How to use quotation marks in String Literals
114+
#### How to use quotation marks in String Literals
115115

116116
There are several ways to include quote characters within a string:
117117

@@ -132,7 +132,7 @@ The following examples demonstrate how quoting and escaping work:
132132
"""string" // "string
133133
```
134134

135-
## Numeric Literals
135+
### Numeric Literals
136136

137137
Number literals include integer (exact-value) literals and floating-point (approximate-value) literals.
138138

@@ -146,27 +146,27 @@ The `FLOAT` and `DOUBLE` data types are floating-point types and calculations ar
146146

147147
An integer may be used in floating-point context; it is interpreted as the equivalent floating-point number.
148148

149-
## Timestamp Literals
149+
### Timestamp Literals
150150

151151
The timestamp is the time point at which data is produced. It includes absolute timestamps and relative timestamps in IoTDB. For information about timestamp support in IoTDB, see [Data Type Doc](../Data-Concept/Data-Type.md).
152152

153153
Specially, `NOW()` represents a constant timestamp that indicates the system time at which the statement began to execute.
154154

155-
## Boolean Literals
155+
### Boolean Literals
156156

157157
The constants `TRUE` and `FALSE` evaluate to 1 and 0, respectively. The constant names can be written in any lettercase.
158158

159-
## NULL Values
159+
### NULL Values
160160

161161
The `NULL` value means “no data.” `NULL` can be written in any lettercase.
162162

163-
# Identifiers
163+
## Identifiers
164164

165-
## Usage scenarios
165+
### Usage scenarios
166166

167167
Certain objects within IoTDB, including `TRIGGER`, `FUNCTION`(UDF), `CONTINUOUS QUERY`, `SCHEMA TEMPLATE`, `USER`, `ROLE`,`Pipe`,`PipeSink`,`alias` and other object names are known as identifiers.
168168

169-
## Constraints
169+
### Constraints
170170

171171
Below are basic constraints of identifiers, specific identifiers may have other constraints, for example, `user` should consists of more than 4 characters.
172172

@@ -182,7 +182,7 @@ Below are basic constraints of identifiers, specific identifiers may have other
182182
- Identifier contains special characters.
183183
- Identifier that is a real number.
184184

185-
## How to use quotations marks in quoted identifiers
185+
### How to use quotations marks in quoted identifiers
186186

187187
`'` and `"` can be used directly in quoted identifiers.
188188

@@ -198,7 +198,7 @@ create schema template `t1``t`
198198
(temperature FLOAT encoding=RLE, status BOOLEAN encoding=PLAIN compression=SNAPPY)
199199
```
200200

201-
## Examples
201+
### Examples
202202

203203
Examples of case in which quoted identifier is used :
204204

@@ -279,11 +279,11 @@ Examples of case in which quoted identifier is used :
279279
- The key/value of an attribute can be String Literal and identifier, more details can be found at **key-value pair** part.
280280

281281

282-
# Node Names in Path
282+
## Node Names in Path
283283

284284
Node name is a special identifier, it can also be wildcard `*` and `**`. When creating timeseries, node name can not be wildcard. In query statment, you can use wildcard to match one or more nodes of path.
285285

286-
## Wildcard
286+
### Wildcard
287287

288288
`*` represents one node. For example, `root.vehicle.*.sensor1` represents a 4-node path which is prefixed with `root.vehicle` and suffixed with `sensor1`.
289289

@@ -315,7 +315,7 @@ select a*b from root.sg
315315
|Time|root.sg.a * root.sg.b|
316316
```
317317

318-
## Identifier
318+
### Identifier
319319

320320
When node name is not wildcard, it is a identifier, which means the constraints on it is the same as described in Identifier part.
321321

@@ -378,7 +378,7 @@ Results:
378378
+-----------------------------+-----------+
379379
```
380380

381-
# Key-Value Pair
381+
## Key-Value Pair
382382

383383
**The key/value of an attribute can be constant(including string) and identifier. **
384384

@@ -477,13 +477,13 @@ CREATE PIPE my_pipe TO my_iotdb FROM
477477
(select ** from root WHERE time>=yyyy-mm-dd HH:MM:SS) WITH 'SyncDelOp' = 'true'
478478
```
479479

480-
# Keywords and Reserved Words
480+
## Keywords and Reserved Words
481481

482482
Keywords are words that have significance in SQL. Keywords can be used as an identifier. Certain keywords, such as TIME/TIMESTAMP and ROOT, are reserved and cannot use as identifiers.
483483

484484
[Keywords and Reserved Words](Keywords.md) shows the keywords and reserved words in IoTDB.
485485

486-
# Session、TsFile API
486+
## Session、TsFile API
487487

488488
When using the Session and TsFile APIs, if the method you call requires parameters such as measurement, device, storage group, path in the form of String, **please ensure that the parameters passed in the input string is the same as when using the SQL statement**, here are some examples to help you understand. Code example could be found at: `example/session/src/main/java/org/apache/iotdb/SyntaxConventionRelatedExample.java`
489489

@@ -582,7 +582,7 @@ String[] paths = new String[]{"root.sg.a", "root.sg.`a.``\"b`", "root.sg.`111`"}
582582
List<String> pathList = Arrays.asList(paths);
583583
```
584584

585-
# Detailed Definitions of Lexical and Grammar
585+
## Detailed Definitions of Lexical and Grammar
586586

587587
Please read the lexical and grammar description files in our code repository:
588588

docs/zh/UserGuide/Reference/Syntax-Conventions.md

Lines changed: 20 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,11 @@
2020
-->
2121

2222

23-
# 字面值常量
23+
## 字面值常量
2424

2525
该部分对 IoTDB 中支持的字面值常量进行说明,包括字符串常量、数值型常量、时间戳常量、布尔型常量和空值。
2626

27-
## 字符串常量
27+
### 字符串常量
2828

2929
> 我们参照了 MySQL 对 字符串的定义:A string is a sequence of bytes or characters, enclosed within either single quote (`'`) or double quote (`"`) characters.
3030
@@ -37,7 +37,7 @@ MySQL 对字符串的定义可以参考:[MySQL :: MySQL 8.0 Reference Manual :
3737
"another string"
3838
```
3939

40-
### 使用场景
40+
#### 使用场景
4141

4242
- `INSERT` 或者 `SELECT` 中用于表达 `TEXT` 类型数据的场景。
4343

@@ -108,7 +108,7 @@ MySQL 对字符串的定义可以参考:[MySQL :: MySQL 8.0 Reference Manual :
108108

109109
- 用于表示键值对,键值对的键和值可以被定义成常量(包括字符串)或者标识符,具体请参考键值对章节。
110110

111-
### 如何在字符串内使用引号
111+
#### 如何在字符串内使用引号
112112

113113
- 在单引号引起的字符串内,双引号无需特殊处理。同理,在双引号引起的字符串内,单引号无需特殊处理。
114114
- 在单引号引起的字符串里,可以通过双写单引号来表示一个单引号,即单引号 ' 可以表示为 ''。
@@ -128,7 +128,7 @@ MySQL 对字符串的定义可以参考:[MySQL :: MySQL 8.0 Reference Manual :
128128
"""string" // "string
129129
```
130130

131-
## 数值型常量
131+
### 数值型常量
132132

133133
数值型常量包括整型和浮点型。
134134

@@ -140,27 +140,27 @@ MySQL 对字符串的定义可以参考:[MySQL :: MySQL 8.0 Reference Manual :
140140

141141
在浮点上下文中可以使用整数,它会被解释为等效的浮点数。
142142

143-
## 时间戳常量
143+
### 时间戳常量
144144

145145
时间戳是一个数据到来的时间点,在 IoTDB 中分为绝对时间戳和相对时间戳。详细信息可参考 [数据类型文档](https://iotdb.apache.org/zh/UserGuide/Master/Data-Concept/Data-Type.html)
146146

147147
特别地,`NOW()`表示语句开始执行时的服务端系统时间戳。
148148

149-
## 布尔型常量
149+
### 布尔型常量
150150

151151
布尔值常量 `TRUE``FALSE` 分别等价于 `1``0`,它们对大小写不敏感。
152152

153-
## 空值
153+
### 空值
154154

155155
`NULL`值表示没有数据。`NULL`对大小写不敏感。
156156

157-
# 标识符
157+
## 标识符
158158

159-
## 使用场景
159+
### 使用场景
160160

161161
在 IoTDB 中,触发器名称、UDF函数名、元数据模板名称、用户与角色名、连续查询标识、Pipe、PipeSink、键值对中的键和值、别名等可以作为标识符。
162162

163-
## 约束
163+
### 约束
164164

165165
请注意,此处约束是标识符的通用约束,具体标识符可能还附带其它约束条件,如用户名限制字符数大于等于4,更严格的约束请参考具体标识符相关的说明文档。
166166

@@ -181,7 +181,7 @@ MySQL 对字符串的定义可以参考:[MySQL :: MySQL 8.0 Reference Manual :
181181
- 标识符包含不允许的特殊字符。
182182
- 标识符为实数。
183183

184-
## 如何在反引号引起的标识符中使用引号
184+
### 如何在反引号引起的标识符中使用引号
185185

186186
**在反引号引起的标识符中可以直接使用单引号和双引号。**
187187

@@ -197,7 +197,7 @@ create schema template `t1't"t`
197197
(temperature FLOAT encoding=RLE, status BOOLEAN encoding=PLAIN compression=SNAPPY)
198198
```
199199

200-
## 特殊情况示例
200+
### 特殊情况示例
201201

202202
需要使用反引号进行引用的部分情况示例:
203203

@@ -277,11 +277,11 @@ create schema template `t1't"t`
277277
- 用于表示键值对,键值对的键和值可以被定义成常量(包括字符串)或者标识符,具体请参考键值对章节。
278278

279279

280-
# 路径结点名
280+
## 路径结点名
281281

282282
路径结点名是特殊的标识符,其还可以是通配符 \*\*\*。在创建时间序列时,各层级的路径结点名不能为通配符 \*\*\*。在查询语句中,可以用通配符 \*\*\* 来表示路径结点名,以匹配一层或多层路径。
283283

284-
## 通配符
284+
### 通配符
285285

286286
`*`在路径中表示一层。例如`root.vehicle.*.sensor1`代表的是以`root.vehicle`为前缀,以`sensor1`为后缀,层次等于 4 层的路径。
287287

@@ -312,7 +312,7 @@ select a*b from root.sg
312312
|Time|root.sg.a * root.sg.b|
313313
```
314314

315-
## 标识符
315+
### 标识符
316316

317317
路径结点名不为通配符时,使用方法和标识符一致。**在 SQL 中需要使用反引号引用的路径结点,在结果集中也会用反引号引起。**
318318

@@ -377,7 +377,7 @@ select `111` from root.sg
377377
+-----------------------------+-------------+
378378
```
379379

380-
# 键值对
380+
## 键值对
381381

382382
**键值对的键和值可以被定义为标识符或者常量。**
383383

@@ -476,13 +476,13 @@ CREATE PIPE my_pipe TO my_iotdb FROM
476476
(select ** from root WHERE time>=yyyy-mm-dd HH:MM:SS) WITH 'SyncDelOp' = 'true'
477477
```
478478

479-
# 关键字和保留字
479+
## 关键字和保留字
480480

481481
关键字是在 SQL 具有特定含义的词,可以作为标识符。保留字是关键字的一个子集,保留字不能用于标识符。
482482

483483
关于 IoTDB 的关键字和保留字列表,可以查看 [关键字和保留字](https://iotdb.apache.org/zh/UserGuide/Master/Reference/Keywords.html)
484484

485-
# Session、TsFile API
485+
## Session、TsFile API
486486

487487
在使用Session、TsFIle API时,如果您调用的方法需要以字符串形式传入物理量(measurement)、设备(device)、存储组(storage group)、路径(path)等参数,**请保证所传入字符串与使用 SQL 语句时的写法一致**,下面是一些帮助您理解的例子。具体代码示例可以参考:`example/session/src/main/java/org/apache/iotdb/SyntaxConventionRelatedExample.java`
488488

@@ -581,7 +581,7 @@ String[] paths = new String[]{"root.sg.a", "root.sg.`a.``\"b`", "root.sg.`111`"}
581581
List<String> pathList = Arrays.asList(paths);
582582
```
583583

584-
# 词法与文法详细定义
584+
## 词法与文法详细定义
585585

586586
请阅读代码仓库中的词法和语法描述文件:
587587

server/src/main/java/org/apache/iotdb/db/mpp/execution/exchange/SinkHandle.java

Lines changed: 48 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -170,48 +170,13 @@ public synchronized void send(int partition, List<TsBlock> tsBlocks) {
170170
throw new UnsupportedOperationException();
171171
}
172172

173-
private void sendEndOfDataBlockEvent() throws Exception {
174-
logger.debug("[NotifyNoMoreTsBlock]");
175-
int attempt = 0;
176-
TEndOfDataBlockEvent endOfDataBlockEvent =
177-
new TEndOfDataBlockEvent(
178-
remoteFragmentInstanceId,
179-
remotePlanNodeId,
180-
localFragmentInstanceId,
181-
nextSequenceId - 1);
182-
while (attempt < MAX_ATTEMPT_TIMES) {
183-
attempt += 1;
184-
try (SyncDataNodeMPPDataExchangeServiceClient client =
185-
mppDataExchangeServiceClientManager.borrowClient(remoteEndpoint)) {
186-
client.onEndOfDataBlockEvent(endOfDataBlockEvent);
187-
break;
188-
} catch (Throwable e) {
189-
logger.error("Failed to send end of data block event, attempt times: {}", attempt, e);
190-
if (attempt == MAX_ATTEMPT_TIMES) {
191-
throw e;
192-
}
193-
Thread.sleep(retryIntervalInMs);
194-
}
195-
}
196-
}
197-
198173
@Override
199174
public synchronized void setNoMoreTsBlocks() {
200175
logger.debug("[StartSetNoMoreTsBlocks]");
201176
if (aborted || closed) {
202177
return;
203178
}
204-
try {
205-
sendEndOfDataBlockEvent();
206-
} catch (Exception e) {
207-
throw new RuntimeException("Send EndOfDataBlockEvent failed", e);
208-
}
209-
noMoreTsBlocks = true;
210-
211-
if (isFinished()) {
212-
sinkHandleListener.onFinish(this);
213-
}
214-
sinkHandleListener.onEndOfBlocks(this);
179+
executorService.submit(new SendEndOfDataBlockEventTask());
215180
}
216181

217182
@Override
@@ -413,4 +378,51 @@ public void run() {
413378
}
414379
}
415380
}
381+
382+
/**
383+
* Send a {@link org.apache.iotdb.mpp.rpc.thrift.TEndOfDataBlockEvent} to downstream fragment
384+
* instance.
385+
*/
386+
class SendEndOfDataBlockEventTask implements Runnable {
387+
388+
@Override
389+
public void run() {
390+
try (SetThreadName sinkHandleName = new SetThreadName(threadName)) {
391+
logger.debug("[NotifyNoMoreTsBlock]");
392+
int attempt = 0;
393+
TEndOfDataBlockEvent endOfDataBlockEvent =
394+
new TEndOfDataBlockEvent(
395+
remoteFragmentInstanceId,
396+
remotePlanNodeId,
397+
localFragmentInstanceId,
398+
nextSequenceId - 1);
399+
while (attempt < MAX_ATTEMPT_TIMES) {
400+
attempt += 1;
401+
try (SyncDataNodeMPPDataExchangeServiceClient client =
402+
mppDataExchangeServiceClientManager.borrowClient(remoteEndpoint)) {
403+
client.onEndOfDataBlockEvent(endOfDataBlockEvent);
404+
break;
405+
} catch (Throwable e) {
406+
logger.error("Failed to send end of data block event, attempt times: {}", attempt, e);
407+
if (attempt == MAX_ATTEMPT_TIMES) {
408+
logger.error("Failed to send end of data block event after all retry", e);
409+
sinkHandleListener.onFailure(SinkHandle.this, e);
410+
return;
411+
}
412+
try {
413+
Thread.sleep(retryIntervalInMs);
414+
} catch (InterruptedException ex) {
415+
Thread.currentThread().interrupt();
416+
sinkHandleListener.onFailure(SinkHandle.this, e);
417+
}
418+
}
419+
}
420+
noMoreTsBlocks = true;
421+
if (isFinished()) {
422+
sinkHandleListener.onFinish(SinkHandle.this);
423+
}
424+
sinkHandleListener.onEndOfBlocks(SinkHandle.this);
425+
}
426+
}
427+
}
416428
}

0 commit comments

Comments
 (0)