Skip to content

Commit ede7d07

Browse files
authored
[Feature] fieldformat command implementation (opensearch-project#5080)
* fielfformat changes Signed-off-by: Asif Bashar <asif.bashar@gmail.com> # Conflicts: # ppl/src/main/antlr/OpenSearchPPLLexer.g4 # ppl/src/main/antlr/OpenSearchPPLParser.g4 # Conflicts: # ppl/src/main/antlr/OpenSearchPPLParser.g4 * added field format with concatenation of string , added more examples Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added field format with concatenation of string , added more examples Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * ci failure fix test Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * ci failure fix test Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * ci failure fix test Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * ci failure fix test Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added missing test files Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added missing test files Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * doc fix Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * doc fix Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * doc fix Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added test Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * [Feature] implement transpose command as in the roadmap opensearch-project#4786 (opensearch-project#5011) * transpose command implementation Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * transpose rows to columns Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added argument type missing map and hashmap Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added more validations Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added validation Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * index.md formatting fix Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * doc format Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * coderabbit review fixes Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added recommended changes Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added recommended changes Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * for cross cluster failure debugging Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * for cross cluster failure debugging Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * for cross cluster failure debugging Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * trim columnName Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * per review moved to class varialble. Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * per review moved to class varialble. Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added field resolution Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * fix by removing metadata field Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * fixed explain test after removing of metadata fields in transpose result Signed-off-by: Asif Bashar <asif.bashar@gmail.com> --------- Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added test Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * [Feature] implement transpose command as in the roadmap opensearch-project#4786 (opensearch-project#5011) * transpose command implementation Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * transpose rows to columns Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added argument type missing map and hashmap Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added more validations Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added validation Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * index.md formatting fix Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * doc format Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * coderabbit review fixes Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added recommended changes Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added recommended changes Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * for cross cluster failure debugging Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * for cross cluster failure debugging Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * for cross cluster failure debugging Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * trim columnName Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * per review moved to class varialble. Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * per review moved to class varialble. Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added field resolution Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * fix by removing metadata field Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * fixed explain test after removing of metadata fields in transpose result Signed-off-by: Asif Bashar <asif.bashar@gmail.com> --------- Signed-off-by: Asif Bashar <asif.bashar@gmail.com> # Conflicts: # integ-test/src/test/java/org/opensearch/sql/ppl/NewAddedCommandsIT.java # ppl/src/main/antlr/OpenSearchPPLLexer.g4 * fielfformat changes Signed-off-by: Asif Bashar <asif.bashar@gmail.com> # Conflicts: # ppl/src/main/antlr/OpenSearchPPLLexer.g4 # ppl/src/main/antlr/OpenSearchPPLParser.g4 * added field format with concatenation of string , added more examples Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added test Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * [Feature] implement transpose command as in the roadmap opensearch-project#4786 (opensearch-project#5011) * transpose command implementation Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * transpose rows to columns Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added argument type missing map and hashmap Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added tests Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added more validations Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added validation Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * index.md formatting fix Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * doc format Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * coderabbit review fixes Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added recommended changes Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added recommended changes Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * for cross cluster failure debugging Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * for cross cluster failure debugging Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * for cross cluster failure debugging Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * trim columnName Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * per review moved to class varialble. Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * per review moved to class varialble. Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added field resolution Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * fix by removing metadata field Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * fixed explain test after removing of metadata fields in transpose result Signed-off-by: Asif Bashar <asif.bashar@gmail.com> --------- Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added test Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * merge from main Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * merge conflict issues Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * doc fix Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * test fix Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * test fix Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * coderabbit recommendations Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * resolve conflicts Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * resolve conflicts Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * resolve conflicts Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * resolve conflicts Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * removed ambigous sentence Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * added missing test files Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * spotlessApply Signed-off-by: Asif Bashar <asif.bashar@gmail.com> * merge conflict fix Signed-off-by: Asif Bashar <asif.bashar@gmail.com> --------- Signed-off-by: Asif Bashar <asif.bashar@gmail.com>
1 parent 019cd26 commit ede7d07

18 files changed

Lines changed: 809 additions & 1 deletion

File tree

core/src/main/java/org/opensearch/sql/analysis/Analyzer.java

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -526,6 +526,12 @@ public LogicalPlan visitEval(Eval node, AnalysisContext context) {
526526
return new LogicalEval(child, expressionsBuilder.build());
527527
}
528528

529+
/** Build {@link LogicalEval}. */
530+
@Override
531+
public LogicalPlan visitFieldFormat(Eval node, AnalysisContext context) {
532+
throw getOnlyForCalciteException("fieldformat");
533+
}
534+
529535
@Override
530536
public LogicalPlan visitAddTotals(AddTotals node, AnalysisContext context) {
531537
throw getOnlyForCalciteException("addtotals");

core/src/main/java/org/opensearch/sql/ast/AbstractNodeVisitor.java

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -264,6 +264,10 @@ public T visitEval(Eval node, C context) {
264264
return visitChildren(node, context);
265265
}
266266

267+
public T visitFieldFormat(Eval node, C context) {
268+
return visitChildren(node, context);
269+
}
270+
267271
public T visitParse(Parse node, C context) {
268272
return visitChildren(node, context);
269273
}

core/src/main/java/org/opensearch/sql/ast/analysis/FieldResolutionVisitor.java

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -626,6 +626,12 @@ public Node visitAddColTotals(AddColTotals node, FieldResolutionContext context)
626626
return node;
627627
}
628628

629+
@Override
630+
public Node visitFieldFormat(Eval node, FieldResolutionContext context) {
631+
visitChildren(node, context);
632+
return node;
633+
}
634+
629635
@Override
630636
public Node visitExpand(Expand node, FieldResolutionContext context) {
631637
Set<String> expandFields = extractFieldsFromExpression(node.getField());

core/src/main/java/org/opensearch/sql/ast/expression/Let.java

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@
2020
public class Let extends UnresolvedExpression {
2121
private final Field var;
2222
private final UnresolvedExpression expression;
23+
private final Literal concatPrefix;
24+
private final Literal concatSuffix;
2325

2426
public Let(Field var, UnresolvedExpression expression) {
2527
String varName = var.getField().toString();
@@ -29,6 +31,21 @@ public Let(Field var, UnresolvedExpression expression) {
2931
}
3032
this.var = var;
3133
this.expression = expression;
34+
this.concatPrefix = null;
35+
this.concatSuffix = null;
36+
}
37+
38+
public Let(
39+
Field var, UnresolvedExpression expression, Literal concatPrefix, Literal concatSuffix) {
40+
String varName = var.getField().toString();
41+
if (OpenSearchConstants.METADATAFIELD_TYPE_MAP.containsKey(varName)) {
42+
throw new IllegalArgumentException(
43+
String.format("Cannot use metadata field [%s] as the eval field.", varName));
44+
}
45+
this.var = var;
46+
this.expression = expression;
47+
this.concatPrefix = concatPrefix;
48+
this.concatSuffix = concatSuffix;
3249
}
3350

3451
@Override

core/src/main/java/org/opensearch/sql/calcite/CalciteRexNodeVisitor.java

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -398,6 +398,28 @@ public RexNode visitLambdaFunction(LambdaFunction node, CalcitePlanContext conte
398398
@Override
399399
public RexNode visitLet(Let node, CalcitePlanContext context) {
400400
RexNode expr = analyze(node.getExpression(), context);
401+
if (node.getConcatPrefix() != null) {
402+
403+
expr =
404+
context.rexBuilder.makeCall(
405+
SqlStdOperatorTable.CONCAT,
406+
context.rexBuilder.makeLiteral(
407+
node.getConcatPrefix().getValue(),
408+
context.rexBuilder.getTypeFactory().createSqlType(SqlTypeName.VARCHAR),
409+
true),
410+
expr);
411+
}
412+
if (node.getConcatSuffix() != null) {
413+
414+
expr =
415+
context.rexBuilder.makeCall(
416+
SqlStdOperatorTable.CONCAT,
417+
expr,
418+
context.rexBuilder.makeLiteral(
419+
node.getConcatSuffix().getValue(),
420+
context.rexBuilder.getTypeFactory().createSqlType(SqlTypeName.VARCHAR),
421+
true));
422+
}
401423
return context.relBuilder.alias(expr, node.getVar().getField().toString());
402424
}
403425

docs/category.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
"user/ppl/cmd/describe.md",
1818
"user/ppl/cmd/eventstats.md",
1919
"user/ppl/cmd/eval.md",
20+
"user/ppl/cmd/fieldformat.md",
2021
"user/ppl/cmd/fields.md",
2122
"user/ppl/cmd/fillnull.md",
2223
"user/ppl/cmd/grok.md",

docs/user/ppl/cmd/fieldformat.md

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
2+
# fieldformat
3+
4+
The `fieldformat` command sets the value to a field with the specified expression and appends the field with evaluated result to the search results. The command is an alias of eval command.
5+
Additionally, it also provides string concatenation dot operator followed by and/or follows a string that will be concatenated to the expression.
6+
7+
8+
## Syntax
9+
10+
The `fieldformat` command has the following syntax:
11+
12+
```syntax
13+
fieldformat <field>=[(prefix).]<expression>[.(suffix)] ["," <field>=[(prefix).]<expression>[.(suffix)] ]...
14+
15+
```
16+
17+
## Parameters
18+
19+
The `fieldformat` command supports the following parameters.
20+
21+
| Parameter| Required/Optional | Description |
22+
|----------------|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|
23+
| `<field>` | Required | The name of the field to create or update. If the field does not exist, a new field is added. If it already exists, its value is overwritten. |
24+
| `<expression>` | Required | The expression to evaluate. The expression can have a prefix and/or suffix string part that will be concatenated to the expression. |
25+
| `prefix` | Optional | A string before the expression followed by dot operator which will be concatenated as prefix to the evaluated expression value. |
26+
| `suffix` | Optional | A string that follows the expression and dot operator which will be concatenated as suffix to the evaluated expression value. |
27+
28+
29+
## Example 1: Create a new field
30+
31+
The following query creates a new `doubleAge` field for each document:
32+
33+
```ppl
34+
source=accounts
35+
| fieldformat doubleAge = age * 2
36+
| fields age, doubleAge
37+
```
38+
39+
The query returns the following results:
40+
41+
```text
42+
fetched rows / total rows = 4/4
43+
+-----+-----------+
44+
| age | doubleAge |
45+
|-----+-----------|
46+
| 32 | 64 |
47+
| 36 | 72 |
48+
| 28 | 56 |
49+
| 33 | 66 |
50+
+-----+-----------+
51+
```
52+
53+
54+
## Example 2: Override an existing field
55+
56+
The following query overrides the `age` field by adding `1` to its value:
57+
58+
```ppl
59+
source=accounts
60+
| fieldformat age = age + 1
61+
| fields age
62+
```
63+
64+
The query returns the following results:
65+
66+
```text
67+
fetched rows / total rows = 4/4
68+
+-----+
69+
| age |
70+
|-----|
71+
| 33 |
72+
| 37 |
73+
| 29 |
74+
| 34 |
75+
+-----+
76+
```
77+
78+
79+
80+
81+
## Example 3: String concatenation with prefix
82+
83+
The following query uses the `.` (dot) operator for string concatenation. You can concatenate string literals and field values as follows:
84+
85+
```ppl
86+
source=accounts
87+
| fieldformat greeting = 'Hello '.tostring( firstname)
88+
| fields firstname, greeting
89+
```
90+
91+
The query returns the following results:
92+
93+
```text
94+
fetched rows / total rows = 4/4
95+
+-----------+---------------+
96+
| firstname | greeting |
97+
|-----------+---------------|
98+
| Amber | Hello Amber |
99+
| Hattie | Hello Hattie |
100+
| Nanette | Hello Nanette |
101+
| Dale | Hello Dale |
102+
+-----------+---------------+
103+
```
104+
105+
106+
## Example 4: String concatenation with dot operator, prefix and suffix
107+
108+
The following query performs prefix and suffix string concatenation operations using dot operator:
109+
110+
```ppl
111+
source=accounts | fieldformat age_info = 'Age: '.CAST(age AS STRING).' years.' | fields firstname, age, age_info
112+
```
113+
114+
The query returns the following results:
115+
116+
```text
117+
fetched rows / total rows = 4/4
118+
+-----------+-----+----------------+
119+
| firstname | age | age_info |
120+
|-----------+-----+----------------|
121+
| Amber | 32 | Age: 32 years. |
122+
| Hattie | 36 | Age: 36 years. |
123+
| Nanette | 28 | Age: 28 years. |
124+
| Dale | 33 | Age: 33 years. |
125+
+-----------+-----+----------------+
126+
```
127+
128+

integ-test/src/test/java/org/opensearch/sql/calcite/CalciteNoPushdownIT.java

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@
3434
CalciteDedupCommandIT.class,
3535
CalciteDescribeCommandIT.class,
3636
CalciteExpandCommandIT.class,
37+
CalciteFieldFormatCommandIT.class,
3738
CalciteFieldsCommandIT.class,
3839
CalciteFillNullCommandIT.class,
3940
CalciteFlattenCommandIT.class,

integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalciteExplainIT.java

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@
2626

2727
import java.io.IOException;
2828
import java.util.Locale;
29+
import org.apache.commons.text.StringEscapeUtils;
2930
import org.junit.Assert;
3031
import org.junit.Ignore;
3132
import org.junit.Test;
@@ -2516,6 +2517,21 @@ public void testAggFilterOnNestedFields() throws IOException {
25162517
TEST_INDEX_CASCADED_NESTED)));
25172518
}
25182519

2520+
@Test
2521+
public void testFieldFormatExplain() throws Exception {
2522+
2523+
enabledOnlyWhenPushdownIsEnabled();
2524+
String expected = loadExpectedPlan("explain_field_format.yaml");
2525+
assertYamlEqualsIgnoreId(
2526+
expected,
2527+
explainQueryYaml(
2528+
StringEscapeUtils.escapeJson(
2529+
StringUtils.format(
2530+
"source=%s | head 5| fieldformat formatted_balance ="
2531+
+ " \"$\".tostring(balance,\"commas\") ",
2532+
TEST_INDEX_ACCOUNT))));
2533+
}
2534+
25192535
@Test
25202536
public void testExplainMvCombine() throws IOException {
25212537
String query =
Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
/*
2+
* Copyright OpenSearch Contributors
3+
* SPDX-License-Identifier: Apache-2.0
4+
*/
5+
6+
package org.opensearch.sql.calcite.remote;
7+
8+
import static org.opensearch.sql.util.MatcherUtils.*;
9+
10+
import java.io.IOException;
11+
import org.apache.commons.text.StringEscapeUtils;
12+
import org.json.JSONObject;
13+
import org.junit.jupiter.api.Test;
14+
import org.opensearch.client.Request;
15+
import org.opensearch.sql.ppl.PPLIntegTestCase;
16+
17+
public class CalciteFieldFormatCommandIT extends PPLIntegTestCase {
18+
19+
@Override
20+
public void init() throws Exception {
21+
super.init();
22+
enableCalcite();
23+
24+
loadIndex(Index.BANK);
25+
26+
// Create test data for string concatenation
27+
Request request1 = new Request("PUT", "/test_eval/_doc/1?refresh=true");
28+
request1.setJsonEntity("{\"name\": \"Alice\", \"age\": 25, \"title\": \"Engineer\"}");
29+
client().performRequest(request1);
30+
31+
Request request2 = new Request("PUT", "/test_eval/_doc/2?refresh=true");
32+
request2.setJsonEntity("{\"name\": \"Bob\", \"age\": 30, \"title\": \"Manager\"}");
33+
client().performRequest(request2);
34+
35+
Request request3 = new Request("PUT", "/test_eval/_doc/3?refresh=true");
36+
request3.setJsonEntity("{\"name\": \"Charlie\", \"age\": null, \"title\": \"Analyst\"}");
37+
client().performRequest(request3);
38+
}
39+
40+
@Test
41+
public void testFieldFormatStringConcatenation() throws IOException {
42+
JSONObject result =
43+
executeQuery(
44+
StringEscapeUtils.escapeJson(
45+
"source=test_eval | fieldformat greeting = 'Hello ' + name"));
46+
verifySchema(
47+
result,
48+
schema("name", "string"),
49+
schema("title", "string"),
50+
schema("age", "bigint"),
51+
schema("greeting", "string"));
52+
verifyDataRows(
53+
result,
54+
rows("Alice", "Engineer", 25, "Hello Alice"),
55+
rows("Bob", "Manager", 30, "Hello Bob"),
56+
rows("Charlie", "Analyst", null, "Hello Charlie"));
57+
}
58+
59+
@Test
60+
public void testFieldFormatStringConcatenationWithNullFieldToString() throws IOException {
61+
JSONObject result =
62+
executeQuery(
63+
StringEscapeUtils.escapeJson(
64+
"source=test_eval | fieldformat age_desc = \"Age: \".tostring(age,\"commas\") |"
65+
+ " fields name, age, age_desc"));
66+
verifySchema(
67+
result, schema("name", "string"), schema("age", "bigint"), schema("age_desc", "string"));
68+
verifyDataRows(
69+
result,
70+
rows("Alice", 25, "Age: 25"),
71+
rows("Bob", 30, "Age: 30"),
72+
rows("Charlie", null, null));
73+
}
74+
75+
@Test
76+
public void testFieldFormatStringConcatenationWithNullField() throws IOException {
77+
JSONObject result =
78+
executeQuery(
79+
StringEscapeUtils.escapeJson(
80+
"source=test_eval | fieldformat age_desc = \"Age: \".CAST(age AS STRING) | fields"
81+
+ " name, age, age_desc"));
82+
verifySchema(
83+
result, schema("name", "string"), schema("age", "bigint"), schema("age_desc", "string"));
84+
verifyDataRows(
85+
result,
86+
rows("Alice", 25, "Age: 25"),
87+
rows("Bob", 30, "Age: 30"),
88+
rows("Charlie", null, null));
89+
}
90+
91+
@Test
92+
public void testFieldFormatStringConcatWithSuffix() throws IOException {
93+
JSONObject result =
94+
executeQuery(
95+
StringEscapeUtils.escapeJson(
96+
"source=test_eval | fieldformat age_desc = CAST(age AS STRING).\" years\" | fields"
97+
+ " name, age, age_desc"));
98+
verifySchema(
99+
result, schema("name", "string"), schema("age", "bigint"), schema("age_desc", "string"));
100+
verifyDataRows(
101+
result,
102+
rows("Alice", 25, "25 years"),
103+
rows("Bob", 30, "30 years"),
104+
rows("Charlie", null, null));
105+
}
106+
107+
@Test
108+
public void testFieldFormatStringConcatWithPrefixSuffix() throws IOException {
109+
JSONObject result =
110+
executeQuery(
111+
StringEscapeUtils.escapeJson(
112+
"source=test_eval | fieldformat age_desc = \"Age: \".CAST(age AS STRING).\" years\""
113+
+ " | fields name, age, age_desc"));
114+
verifySchema(
115+
result, schema("name", "string"), schema("age", "bigint"), schema("age_desc", "string"));
116+
verifyDataRows(
117+
result,
118+
rows("Alice", 25, "Age: 25 years"),
119+
rows("Bob", 30, "Age: 30 years"),
120+
rows("Charlie", null, null));
121+
}
122+
}

0 commit comments

Comments
 (0)