diff --git a/config/_default/menus/main.en.yaml b/config/_default/menus/main.en.yaml index 3612af878dc..95610e758d3 100644 --- a/config/_default/menus/main.en.yaml +++ b/config/_default/menus/main.en.yaml @@ -6099,9 +6099,9 @@ menu: identifier: observability_pipelines_packs_zscaler_zia_web_logs weight: 520 - name: Search Syntax - url: observability_pipelines/search_syntax/ + url: observability_pipelines/search_syntax/logs/ parent: observability_pipelines - identifier: observability_pipelines_search_syntax + identifier: observability_pipelines_logs_search_syntax weight: 6 - name: Scaling and Performance url: observability_pipelines/scaling_and_performance/ diff --git a/content/en/observability_pipelines/search_syntax/_index.md b/content/en/observability_pipelines/search_syntax/_index.md new file mode 100644 index 00000000000..0816c75d816 --- /dev/null +++ b/content/en/observability_pipelines/search_syntax/_index.md @@ -0,0 +1,4 @@ +--- +title: Search Syntax +type: multi-code-lang +--- \ No newline at end of file diff --git a/content/en/observability_pipelines/search_syntax.md b/content/en/observability_pipelines/search_syntax/logs.md similarity index 97% rename from content/en/observability_pipelines/search_syntax.md rename to content/en/observability_pipelines/search_syntax/logs.md index 8c98c5bff68..31412fae3ff 100644 --- a/content/en/observability_pipelines/search_syntax.md +++ b/content/en/observability_pipelines/search_syntax/logs.md @@ -1,7 +1,12 @@ --- -title: Search Syntax -description: Learn the new filter query search syntax for your Observability Pipelines processors. +title: Logs Search Syntax +description: Learn how to use logs search syntax for your Observability Pipelines processors filter queries. disable_toc: false +aliases: + - /observability_pipelines/search_syntax/ +code_lang: logs +type: multi-code-lang +weight: 1 --- ## Overview diff --git a/content/en/observability_pipelines/search_syntax/metrics.md b/content/en/observability_pipelines/search_syntax/metrics.md new file mode 100644 index 00000000000..d37810afcb4 --- /dev/null +++ b/content/en/observability_pipelines/search_syntax/metrics.md @@ -0,0 +1,140 @@ +--- +title: Metrics Search Syntax +description: Learn about how to use metrics search syntax for your Observability Pipelines processors filter queries. +disable_toc: false +code_lang: metrics +type: multi-code-lang +weight: 2 +--- + +## Overview + +When you add a processor to a pipeline, you can filter metrics ({{< tooltip glossary="preview" case="title" >}}) to only process a defined subset. This document goes over the following information: + +- [Free text search](#free-text-search) +- [Attribute search](#attribute-search) +- [Tags search](#tags-search) +- [Boolean operators](#boolean-operators) +- [Wildcards](#wildcards) + +## Free text search + +Free text search for metrics only searches the `name` field and is case sensitive. + +An example of a free text search: `system.cpu.user`, which matches metrics with `name:system.cpu.user`. + +## Attribute search + +You can search the attribute keys: + +- `kind`: Whether the metrics is `absolute` or `incremental`. +- `value`: The metric type. + - `counter` + - `gauge` + - `distribution` + - `histogram` + +Here are some attribute search syntax examples: + +`kind:absolute` +: Matches metrics with `kind:absolute`. + +`value:counter` +: Matches `counter` metrics. + +**Note**: Attribute searches are case sensitive. + +## Tags search + +You can search a metric's `tags`. Special characters and spaces in a tag must be escaped in the search syntax. + +### Escape special characters and spaces + +The following characters are considered special and must be escaped with a backslash (`\`): + +`-` `!` `&&` `||` `>` `>=` `<` `<=` `(` `)` `{` `}` `[` `]` `"` `*` `?` `:` `#`, and spaces. + +**Notes**: + +- `/` is not considered a special character and doesn't need to be escaped. +- You can search for special characters in a tag. See [Search for a tag that contains special characters](#search-for-a-tag-that-contains-special-characters). +- If you want to match metrics tags that contain the special character `!` , use the attribute search syntax: `tags:*\:!*.` + +### Search for a tag that contains special characters + +Searching for a tag that contains special characters requires escaping or double quotes. For example, to search for a tag `my_app` with the value `hello:world`, use one of the following search syntaxes: + +- `tags:my_app\:hello\:world` +- `tags:my_app\:"hello:world"` + +### Match a single special character + +To match a single special character or space, use the `?` wildcard. For example, the search syntax: `"tags:message\:hello?world"` matches metrics with either of these tags: + +- `message:"hello world"` +- `message:"hello-world"` + +### Examples + +To learn how to escape special characters in a tag search, let's look at a metric example: + +``` +{ +"name":"datadog.agent.retry_queue_duration.bytes_per_sec", + "tags":{ + "agent":"core", + "domain":"https://7-72-3-app.agent.datadoghq.com", + "host":"COMP-YGVQDJG75L", + "source_type_name":"System", + "env:prod" + }, + "timestamp":"2025-11-28T13:03:09Z", + "kind":"absolute", + "gauge":{"value":454.1372767857143} +} +``` + +The following are search syntax examples that escape special characters in the metric example: + +`tags:env*` +: Matches metrics with `tags` containing the `env` key. + +`tags:(env\:prod OR env\:test)` +: Matches metrics with `env:prod` or `env:test` in `tags`. +: This query can also be written as `tags:("env:prod" OR "env:test")`. + +`tags:env\:prod AND -tags:version\:beta` +: Matches metrics that have `env:prod` and does not have `version:beta` in `tags`. +: This query can also be written as `tags:"env:prod" AND -tags:"version:beta"`. + +## Boolean operators + +You can use the following case sensitive Boolean operators to combine multiple terms in a search query. + +The follow are example queries that use Boolean operators: + +`NOT system.cpu.user` +: Matches metrics that do not have `name:system.cpu.user`. +: This query can also be written as `-system.cpu.user`. + +`system.cpu.user OR system.cpu.user.total` +: Matches metrics that have either `name:system.cpu.user` or `name:system.cpu.user.total`. + +`value:counter AND (tags:env\:staging OR tags:env\:dev)` +: Matches metrics that have both `counter:{value:}` and the either the tag `env:prod` or `env:dev` + +## Wildcards + +​​You can use `*` for wildcard searches. The following are wildcard search examples: + +`*system.cpu.user*` +: Matches metrics with a `name` value that contains `system.cpu.user`. + +`system.cpu.user*` +: Matches metrics with a `name` value that starts with `system.cpu.user`. + +`*system.cpu.user` +: Matches metrics with a `name` value that ends with `system.cpu.user`. + +`tags:*\:bin` +: Matches metrics that have a tag with the value `bin`, regardless of what the tag key is.