diff --git a/deploy-manage/monitor/autoops.md b/deploy-manage/monitor/autoops.md index d817cbc24e..ed1e952a82 100644 --- a/deploy-manage/monitor/autoops.md +++ b/deploy-manage/monitor/autoops.md @@ -75,4 +75,4 @@ In this section, you'll find the following information: * [Regions](/deploy-manage/monitor/autoops/ec-autoops-regions.md) where AutoOps is available. * Which [views](/deploy-manage/monitor/autoops/views.md) AutoOps offers to gain insight into your deployment. * What AutoOps [events](/deploy-manage/monitor/autoops/ec-autoops-events.md) are and how you can configure [event settings](/deploy-manage/monitor/autoops/ec-autoops-event-settings.md) and [notifications](/deploy-manage/monitor/autoops/ec-autoops-notifications-settings.md). -* [Frequently asked questions](/deploy-manage/monitor/autoops/ec-autoops-faq.md) about AutoOps. +* [Frequently asked questions](/deploy-manage/monitor/autoops/ec-autoops-faq.md) about AutoOps. \ No newline at end of file diff --git a/deploy-manage/monitor/logging-configuration/query-logs.md b/deploy-manage/monitor/logging-configuration/query-logs.md index 21c9f9c722..be6a68a089 100644 --- a/deploy-manage/monitor/logging-configuration/query-logs.md +++ b/deploy-manage/monitor/logging-configuration/query-logs.md @@ -358,3 +358,4 @@ Each query language may also include its own fields, prefixed with `elasticsearc - [Query activity](/deploy-manage/monitor/query-activity.md) - [Tune for search speed](/deploy-manage/production-guidance/optimize-performance/search-speed.md) +- [Optimize {{esql}} query performance](elasticsearch://reference/query-languages/esql/esql-query-performance.md) diff --git a/deploy-manage/monitor/logging-configuration/slow-logs.md b/deploy-manage/monitor/logging-configuration/slow-logs.md index d006a4804b..66f91f7a00 100644 --- a/deploy-manage/monitor/logging-configuration/slow-logs.md +++ b/deploy-manage/monitor/logging-configuration/slow-logs.md @@ -263,4 +263,4 @@ Slow logging checks each event against the reporting threshold when the event is ## Learn more [_learn_more] -To learn about other ways to optimize your search and indexing requests, refer to [tune for search speed](/deploy-manage/production-guidance/optimize-performance/search-speed.md) and [tune for indexing speed](/deploy-manage/production-guidance/optimize-performance/indexing-speed.md). \ No newline at end of file +To learn about other ways to optimize your search and indexing requests, refer to [tune for search speed](/deploy-manage/production-guidance/optimize-performance/search-speed.md) and [tune for indexing speed](/deploy-manage/production-guidance/optimize-performance/indexing-speed.md). For {{esql}}-specific guidance, refer to [Optimize {{esql}} query performance](elasticsearch://reference/query-languages/esql/esql-query-performance.md). \ No newline at end of file diff --git a/deploy-manage/monitor/query-activity.md b/deploy-manage/monitor/query-activity.md index f37f71dfee..3f7eed9b2c 100644 --- a/deploy-manage/monitor/query-activity.md +++ b/deploy-manage/monitor/query-activity.md @@ -131,3 +131,4 @@ To change this threshold: - [](/deploy-manage/monitor/logging-configuration/query-logs.md) - [](/deploy-manage/monitor/logging-configuration/slow-logs.md) - [](/deploy-manage/production-guidance/optimize-performance/search-speed.md) +- [Optimize {{esql}} query performance](elasticsearch://reference/query-languages/esql/esql-query-performance.md) diff --git a/deploy-manage/production-guidance/optimize-performance.md b/deploy-manage/production-guidance/optimize-performance.md index 1012ca025d..d1b14fda40 100644 --- a/deploy-manage/production-guidance/optimize-performance.md +++ b/deploy-manage/production-guidance/optimize-performance.md @@ -23,6 +23,9 @@ Use the following topics to explore relevant strategies: * [General recommendations](general-recommendations.md) * [Tune for indexing speed](optimize-performance/indexing-speed.md) * [Tune for search speed](optimize-performance/search-speed.md) + + For additional guidance specific to {{esql}} queries, refer to [Optimize {{esql}} query performance](elasticsearch://reference/query-languages/esql/esql-query-performance.md). + * [Tune approximate kNN search](optimize-performance/approximate-knn-search.md) * [Tune for disk usage](optimize-performance/disk-usage.md) * [Size your shards](optimize-performance/size-shards.md) diff --git a/deploy-manage/production-guidance/optimize-performance/search-speed.md b/deploy-manage/production-guidance/optimize-performance/search-speed.md index afe69af135..dfda9e8c66 100644 --- a/deploy-manage/production-guidance/optimize-performance/search-speed.md +++ b/deploy-manage/production-guidance/optimize-performance/search-speed.md @@ -365,6 +365,10 @@ Now imagine that you have a 2-shards index and two nodes. In one case, the numbe So what is the right number of replicas? If you have a cluster that has `num_nodes` nodes, `num_primaries` primary shards *in total* and if you want to be able to cope with `max_failures` node failures at once at most, then the right number of replicas for you is `max(max_failures, ceil(num_nodes / num_primaries) - 1)`. +## Optimize {{esql}} queries [_optimize_esql_queries] + +For {{esql}}-specific performance guidance, including common anti-patterns and techniques for reducing scan size, refer to [Optimize {{esql}} query performance](elasticsearch://reference/query-languages/esql/esql-query-performance.md). + ## Tune your queries with the Search Profiler [_tune_your_queries_with_the_search_profiler] The [Profile API](elasticsearch://reference/elasticsearch/rest-apis/search-profile.md) provides detailed information about how each component of your queries and aggregations impacts the time it takes to process the request. diff --git a/explore-analyze/discover/try-esql.md b/explore-analyze/discover/try-esql.md index 35d52ed270..b44bb450a7 100644 --- a/explore-analyze/discover/try-esql.md +++ b/explore-analyze/discover/try-esql.md @@ -19,9 +19,13 @@ Elasticsearch Query Language ({{esql}}) helps you explore and analyze your {{pro - You must have data in {{product.elasticsearch}}. The examples on this page use the {{product.kibana}} sample web logs to explore data and create visualizations. You can install sample data by following [Add sample data](../index.md#gs-get-data-into-kibana). -::::{tip} -For the complete {{esql}} documentation, including all supported commands, functions, and operators, refer to the [{{esql}} reference](elasticsearch://reference/query-languages/esql/esql-syntax-reference.md). For a more detailed overview of {{esql}} in {{product.kibana}}, refer to [Use {{esql}} in Kibana](../query-filter/languages/esql-kibana.md). -:::: +## Resources + +This tutorial covers the basics of querying data with {{esql}} in Discover. For more information, refer to: + +* [{{esql}} reference](elasticsearch://reference/query-languages/esql/esql-syntax-reference.md): Complete list of commands, functions, and operators +* [Use {{esql}} in Kibana](../query-filter/languages/esql-kibana.md): Detailed overview of {{esql}} features in {{product.kibana}} +* [Optimize {{esql}} query performance](elasticsearch://reference/query-languages/esql/esql-query-performance.md): Techniques for writing fast queries ## Get started with {{esql}} in Discover [tutorial-try-esql] diff --git a/explore-analyze/query-filter/languages/esql-kibana.md b/explore-analyze/query-filter/languages/esql-kibana.md index 46c347a53f..453e825aa1 100644 --- a/explore-analyze/query-filter/languages/esql-kibana.md +++ b/explore-analyze/query-filter/languages/esql-kibana.md @@ -401,3 +401,4 @@ The first time a query references an unmapped field, the editor shows a warning - [{{esql}} visualizations](/explore-analyze/visualize/esorql.md): Create and edit {{esql}}-based visualizations in dashboards. - [Dashboard controls](/explore-analyze/dashboards/add-controls.md): Add {{esql}}-powered controls to dashboards. - {applies_to}`stack: ga 9.4` {applies_to}`serverless: ga` [Custom Vega visualizations](/explore-analyze/visualize/custom-visualizations-with-vega.md#vega-esql-queries): Use {{esql}} queries as a data source in Vega and Vega-Lite visualizations. +- [Optimize {{esql}} query performance](elasticsearch://reference/query-languages/esql/esql-query-performance.md): Techniques for writing fast queries and identifying slow ones. diff --git a/explore-analyze/visualize/esorql.md b/explore-analyze/visualize/esorql.md index d0677a69c1..f3c8ed96cb 100644 --- a/explore-analyze/visualize/esorql.md +++ b/explore-analyze/visualize/esorql.md @@ -41,7 +41,7 @@ You can then **Save** and add it to an existing or a new dashboard using the sav 2. Choose **ES|QL** under **Visualizations**. An ES|QL editor appears and lets you configure your query and its associated visualization. The **Suggestions** panel can help you find alternative ways to configure the visualization. ::::{tip} - Check the [ES|QL reference](elasticsearch://reference/query-languages/esql.md) to get familiar with the syntax and optimize your query. + Check the [ES|QL reference](elasticsearch://reference/query-languages/esql.md) to get familiar with the syntax and [optimize your query](elasticsearch://reference/query-languages/esql/esql-query-performance.md). :::: 3. When editing your query or its configuration, run the query to update the preview of the visualization. diff --git a/solutions/search/esql-for-search.md b/solutions/search/esql-for-search.md index d228ebfd40..ac40c335ff 100644 --- a/solutions/search/esql-for-search.md +++ b/solutions/search/esql-for-search.md @@ -197,6 +197,7 @@ The [`MMR` command](elasticsearch://reference/query-languages/esql/commands/mmr. ### Tutorials and how-to guides [esql-for-search-tutorials] - [Search and filter with {{esql}}](elasticsearch://reference/query-languages/esql/esql-search-tutorial.md): Hands-on tutorial for getting started with search tools in {{esql}}, with concrete examples of the functionalities described in this page +- [Optimize {{esql}} query performance](elasticsearch://reference/query-languages/esql/esql-query-performance.md): Techniques for writing fast queries, including using full-text search functions instead of `LIKE` or `RLIKE` ### Technical reference [esql-for-search-reference] diff --git a/solutions/security/esql-for-security.md b/solutions/security/esql-for-security.md index 7286270e9b..e9d256f32d 100644 --- a/solutions/security/esql-for-security.md +++ b/solutions/security/esql-for-security.md @@ -22,4 +22,5 @@ Learn how to: - [Generate and understand {{esql}} queries](/solutions/security/ai/generate-customize-learn-about-esorql-queries.md) using the AI Assistant - [Investigate events in Timeline](/solutions/security/investigate/timeline.md#esql-in-timeline) using {{esql}} - [Create detection rules](/solutions/security/detect-and-alert/esql.md) using {{esql}} -- [Convert Splunk SPL rules to {{esql}}](/solutions/security/get-started/automatic-migration.md) with Automatic Migration \ No newline at end of file +- [Convert Splunk SPL rules to {{esql}}](/solutions/security/get-started/automatic-migration.md) with Automatic Migration +- [Optimize {{esql}} query performance](elasticsearch://reference/query-languages/esql/esql-query-performance.md) to write faster queries for threat hunting and detection rules \ No newline at end of file diff --git a/solutions/security/investigate/timeline.md b/solutions/security/investigate/timeline.md index 10e72326a7..b0428bfbfe 100644 --- a/solutions/security/investigate/timeline.md +++ b/solutions/security/investigate/timeline.md @@ -284,3 +284,5 @@ You can use {{esql}} in Timeline by opening the **{{esql}}** tab. From there, yo To get started using {{esql}}, read the tutorial for [using {{esql}} in {{kib}}](/explore-analyze/query-filter/languages/esql-kibana.md). Much of the functionality available in {{kib}} is also available in Timeline. To find examples of using {{esql}} for threat hunting, check out [our blog](https://www.elastic.co/blog/introduction-to-esql-new-query-language-flexible-iterative-analytics). + +For tips on writing faster {{esql}} queries, refer to [Optimize {{esql}} query performance](elasticsearch://reference/query-languages/esql/esql-query-performance.md). diff --git a/troubleshoot/elasticsearch/circuit-breaker-errors.md b/troubleshoot/elasticsearch/circuit-breaker-errors.md index 29808e30f3..c2825f70e2 100644 --- a/troubleshoot/elasticsearch/circuit-breaker-errors.md +++ b/troubleshoot/elasticsearch/circuit-breaker-errors.md @@ -111,6 +111,10 @@ If you’ve triggered the fielddata circuit breaker and can’t disable fielddat POST _cache/clear?fielddata=true ``` +**Optimize expensive queries** + +Both [Query DSL](../../explore-analyze/query-filter/languages/querydsl.md) and {{esql}} queries can trigger circuit breaker errors when they consume large amounts of memory. For {{esql}}, high-cardinality `STATS BY` groupings are a common cause. Refer to [Optimize {{esql}} query performance](elasticsearch://reference/query-languages/esql/esql-query-performance.md) for techniques to reduce memory usage. + ## Memory evaluation Circuit breakers may either directly evaluate memory usage estimates or indirectly limit operations that are likely to cause excessive memory consumption. For example, the `script` circuit breaker checks memory indirectly by rate-limiting Painless/Mustache script compilations. However, even with circuit breakers in place, nodes can still encounter out-of-memory (OOM) conditions. This can occur, for example, because: diff --git a/troubleshoot/elasticsearch/high-jvm-memory-pressure.md b/troubleshoot/elasticsearch/high-jvm-memory-pressure.md index d4e45ff428..709fba26cc 100644 --- a/troubleshoot/elasticsearch/high-jvm-memory-pressure.md +++ b/troubleshoot/elasticsearch/high-jvm-memory-pressure.md @@ -193,6 +193,10 @@ PUT _cluster/settings } ``` +::::{tip} +For {{esql}}-specific guidance on writing efficient queries, refer to [Optimize {{esql}} query performance](elasticsearch://reference/query-languages/esql/esql-query-performance.md). +:::: + #### Prevent mapping explosion [reduce-jvm-memory-pressure-setup-mapping] Defining too many fields or nesting fields too deeply can lead to [mapping explosions](/troubleshoot/elasticsearch/mapping-explosion.md) that use large amounts of memory. To prevent mapping explosions, use the [mapping limit settings](elasticsearch://reference/elasticsearch/index-settings/mapping-limit.md) to limit the number of field mappings. diff --git a/troubleshoot/elasticsearch/troubleshooting-searches.md b/troubleshoot/elasticsearch/troubleshooting-searches.md index 9444b6f937..d6aa65afb8 100644 --- a/troubleshoot/elasticsearch/troubleshooting-searches.md +++ b/troubleshoot/elasticsearch/troubleshooting-searches.md @@ -243,3 +243,5 @@ xpack.security.audit.logfile.events.emit_request_body: true ``` Refer to [Advanced tuning: finding and fixing slow Elasticsearch queries](https://www.elastic.co/blog/advanced-tuning-finding-and-fixing-slow-elasticsearch-queries) for more information. + +For {{esql}}-specific slow query diagnosis and prevention, refer to [Optimize {{esql}} query performance](elasticsearch://reference/query-languages/esql/esql-query-performance.md).