Skip to content

Commit 74d5997

Browse files
committed
add doc
Signed-off-by: Kai Huang <ahkcs@amazon.com>
1 parent 342331e commit 74d5997

1 file changed

Lines changed: 216 additions & 4 deletions

File tree

docs/user/ppl/cmd/reverse.md

Lines changed: 216 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,16 @@ The `reverse` command reverses the display order of the search results. It retur
55

66
> **Note**: The `reverse` command processes the entire dataset. If applied directly to millions of records, it consumes significant coordinating node memory resources. Only apply the `reverse` command to smaller datasets, typically after aggregation operations.
77
8+
## Performance optimization
9+
10+
The `reverse` command uses an optimized implementation that intelligently reverses existing sort collations instead of using a `ROW_NUMBER()` approach. The behavior depends on the context:
11+
12+
1. **Existing sort collation**: If a preceding `sort` command is detected, `reverse` flips the sort direction of each field (e.g., ASC becomes DESC and vice versa). This leverages database-native sort reversal for significantly better performance.
13+
2. **`@timestamp` field**: If no explicit sort exists but the data source has an `@timestamp` field, `reverse` sorts by `@timestamp` in descending order.
14+
3. **No sort or `@timestamp`**: If neither an explicit sort nor an `@timestamp` field is found, `reverse` is a no-op (ignored).
15+
16+
The optimization also supports **backtracking** through non-blocking operators like `where`, `eval`, and `fields` to find an upstream sort. However, blocking operators such as `stats` (aggregation), `join`, and set operations destroy the collation, so `reverse` after these operators is a no-op unless a new `sort` is added after them.
17+
818
## Syntax
919

1020
The `reverse` command has the following syntax:
@@ -116,19 +126,19 @@ fetched rows / total rows = 4/4
116126
```
117127

118128

119-
## Example 5: Use the reverse command with a complex pipeline
129+
## Example 5: Use the reverse command with a complex pipeline
120130

121131
The following query uses the `reverse` command with filtering and field selection:
122-
132+
123133
```ppl
124134
source=accounts
125135
| where age > 30
126136
| fields account_number, age
127137
| reverse
128138
```
129-
139+
130140
The query returns the following results:
131-
141+
132142
```text
133143
fetched rows / total rows = 3/3
134144
+----------------+-----+
@@ -140,3 +150,205 @@ fetched rows / total rows = 3/3
140150
+----------------+-----+
141151
```
142152

153+
## Example 6: Reverse with descending sort
154+
155+
The following query reverses a descending sort, effectively producing ascending order:
156+
157+
```ppl
158+
source=accounts
159+
| sort - account_number
160+
| fields account_number
161+
| reverse
162+
```
163+
164+
The query returns the following results:
165+
166+
```text
167+
fetched rows / total rows = 7/7
168+
+----------------+
169+
| account_number |
170+
|----------------|
171+
| 1 |
172+
| 6 |
173+
| 13 |
174+
| 18 |
175+
| 20 |
176+
| 25 |
177+
| 32 |
178+
+----------------+
179+
```
180+
181+
## Example 7: Reverse with mixed sort directions
182+
183+
The following query reverses a multi-field sort with mixed directions. Each field's sort direction is individually flipped:
184+
185+
```ppl
186+
source=accounts
187+
| sort - account_number, + firstname
188+
| fields account_number, firstname
189+
| reverse
190+
```
191+
192+
The query returns the following results:
193+
194+
```text
195+
fetched rows / total rows = 7/7
196+
+----------------+--------------+
197+
| account_number | firstname |
198+
|----------------+--------------|
199+
| 1 | Amber JOHnny |
200+
| 6 | Hattie |
201+
| 13 | Nanette |
202+
| 18 | Dale |
203+
| 20 | Elinor |
204+
| 25 | Virginia |
205+
| 32 | Dillard |
206+
+----------------+--------------+
207+
```
208+
209+
## Example 8: Reverse with @timestamp field
210+
211+
When no explicit sort exists but the data source has an `@timestamp` field, `reverse` sorts by `@timestamp` in descending order:
212+
213+
```ppl
214+
source=time_test_data
215+
| fields value, category, `@timestamp`
216+
| reverse
217+
| head 5
218+
```
219+
220+
The query returns the following results:
221+
222+
```text
223+
fetched rows / total rows = 5/5
224+
+-------+----------+---------------------+
225+
| value | category | @timestamp |
226+
|-------+----------+---------------------|
227+
| 8762 | A | 2025-08-01 03:47:41 |
228+
| 7348 | C | 2025-08-01 02:00:56 |
229+
| 9015 | B | 2025-08-01 01:14:11 |
230+
| 6489 | D | 2025-08-01 00:27:26 |
231+
| 8676 | A | 2025-07-31 23:40:33 |
232+
+-------+----------+---------------------+
233+
```
234+
235+
## Example 9: Reverse is ignored without sort or @timestamp
236+
237+
When there is no explicit sort and the data source has no `@timestamp` field, `reverse` is a no-op and data remains in its natural order:
238+
239+
```ppl
240+
source=accounts
241+
| fields account_number
242+
| reverse
243+
| head 3
244+
```
245+
246+
The query returns the following results:
247+
248+
```text
249+
fetched rows / total rows = 3/3
250+
+----------------+
251+
| account_number |
252+
|----------------|
253+
| 1 |
254+
| 6 |
255+
| 13 |
256+
+----------------+
257+
```
258+
259+
## Example 10: Reverse backtracks through filter and eval
260+
261+
The `reverse` command can detect sort collations through non-blocking operators like `where` and `eval`:
262+
263+
```ppl
264+
source=accounts
265+
| sort account_number
266+
| where balance > 30000
267+
| fields account_number, balance
268+
| reverse
269+
```
270+
271+
The query returns the following results:
272+
273+
```text
274+
fetched rows / total rows = 4/4
275+
+----------------+---------+
276+
| account_number | balance |
277+
|----------------+---------|
278+
| 32 | 48086 |
279+
| 25 | 40540 |
280+
| 13 | 32838 |
281+
| 1 | 39225 |
282+
+----------------+---------+
283+
```
284+
285+
## Example 11: Reverse is a no-op after aggregation
286+
287+
Aggregation (`stats`) destroys input ordering, so `reverse` after aggregation without a subsequent `sort` is a no-op:
288+
289+
```ppl
290+
source=accounts
291+
| stats count() as c by gender
292+
| reverse
293+
```
294+
295+
The query returns the following results (order not guaranteed):
296+
297+
```text
298+
fetched rows / total rows = 2/2
299+
+---+--------+
300+
| c | gender |
301+
|---+--------|
302+
| 4 | M |
303+
| 3 | F |
304+
+---+--------+
305+
```
306+
307+
## Example 12: Reverse works with sort after aggregation
308+
309+
Adding a `sort` after aggregation restores collation, allowing `reverse` to work:
310+
311+
```ppl
312+
source=accounts
313+
| stats count() as c by gender
314+
| sort gender
315+
| reverse
316+
```
317+
318+
The query returns the following results:
319+
320+
```text
321+
fetched rows / total rows = 2/2
322+
+---+--------+
323+
| c | gender |
324+
|---+--------|
325+
| 4 | M |
326+
| 3 | F |
327+
+---+--------+
328+
```
329+
330+
## Example 13: Reverse with timechart
331+
332+
The `timechart` command adds a sort on the time field, so `reverse` flips it to return results in reverse chronological order:
333+
334+
```ppl
335+
source=events
336+
| timechart span=1m count()
337+
| reverse
338+
```
339+
340+
The query returns the following results:
341+
342+
```text
343+
fetched rows / total rows = 5/5
344+
+---------------------+----------+
345+
| @timestamp | count() |
346+
|---------------------+----------|
347+
| 2024-07-01 00:04:00 | 1 |
348+
| 2024-07-01 00:03:00 | 1 |
349+
| 2024-07-01 00:02:00 | 1 |
350+
| 2024-07-01 00:01:00 | 1 |
351+
| 2024-07-01 00:00:00 | 1 |
352+
+---------------------+----------+
353+
```
354+

0 commit comments

Comments
 (0)