From 22e084f4593653e5d82b1be40d0d2bd2ef3dd904 Mon Sep 17 00:00:00 2001 From: Hodayah Zargari Date: Wed, 3 Dec 2025 11:49:35 +0200 Subject: [PATCH 1/6] Troubleshooting query acceleration - reformatting --- ...ery-acceleration-troubleshooting-guide.md} | 181 ++++++++---------- 1 file changed, 84 insertions(+), 97 deletions(-) rename data-explorer/kusto/management/{query-acceleration-tsg.md => query-acceleration-troubleshooting-guide.md} (68%) diff --git a/data-explorer/kusto/management/query-acceleration-tsg.md b/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md similarity index 68% rename from data-explorer/kusto/management/query-acceleration-tsg.md rename to data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md index 8a727a3225..708ade1b21 100644 --- a/data-explorer/kusto/management/query-acceleration-tsg.md +++ b/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md @@ -1,9 +1,9 @@ --- -title: query acceleration troubleshooting guide -description: Learn how to troubleshoot query acceleration issues +title: Query Acceleration Troubleshooting Guide +description: Learn how to troubleshoot for common errors encountered in query acceleration. ms.reviewer: urishapira ms.topic: reference -ms.date: 12/01/2025 +ms.date: 12/03/2025 --- # Troubleshoot query acceleration over external delta tables @@ -18,7 +18,7 @@ The query acceleration feature consists of the following components: - A background job that caches delta table data files. - Query-time enhancements that utilize the catalog and the cached data. -To understand why things aren't working as expected, it's important to identify which of these components isn't functioning properly. +To understand why things aren't working as expected, you need to identify which of these components isn't functioning properly. This article helps you troubleshoot scenarios where: @@ -34,42 +34,43 @@ This article helps you troubleshoot scenarios where: | project isnotnull(Policy) and todynamic(Policy).IsEnabled ``` - If this command returns `false`, enable the query acceleration policy using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md). + If this command returns `false`, enable the query acceleration policy by using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md). -2. **Ensure the delta table complies with the Delta protocol.** +1. **Ensure the delta table complies with the Delta protocol.** - The query acceleration feature assumes a delta table that complies with the Delta protocol. Manual operations executed directly on the delta table (for example, editing transaction logs or parquet files) aren't supported and may result in unexpected behavior. + The query acceleration feature assumes a delta table that complies with the Delta protocol. Manual operations executed directly on the delta table (for example, editing transaction logs or parquet files) aren't supported and might result in unexpected behavior. - If such operations have been executed on the delta table, recreate the external table and re-enable the query acceleration policy. + If you executed such operations on the delta table, recreate the external table and re-enable the query acceleration policy. +## Common errors -## Query is returning stale data +### Query returns stale data -This is a data freshness issue: query results don't reflect the latest data from the underlying delta table. +This issue occurs when query results don't reflect the latest data from the underlying delta table. Query acceleration refreshes the accelerated data periodically, so that results are no older than the configured `MaxAge` value in the policy. -By design, queries over accelerated external tables may return data that lags behind the latest delta table version by up to `MaxAge`. Set `MaxAge` to the maximum data staleness that is acceptable at query time. +By design, queries over accelerated external tables might return data that lags behind the latest delta table version by up to `MaxAge`. Set `MaxAge` to the maximum data staleness that is acceptable at query time. You can control the effective `MaxAge` in two ways: -- Configure the `MaxAge` property in the query acceleration policy using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md). +- Configure the `MaxAge` property in the query acceleration policy by using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md). - Override `MaxAge` per query by using the [`external_table()` operator's](../query/external-table-function.md) `MaxAgeOverride` parameter. -## Query isn't running fast enough +### Query latency -This is a performance issue: query is slower than expected, and acceleration doesn't appear to improve performance. +This issue is a performance problem: the query is slower than expected, and acceleration doesn't seem to improve performance. -There are a few reasons why this could happen: +Several reasons can cause this issue: -- The query acceleration catalog is unusable (out-of-date or never built) - see section [Check if catalog is unusable](#check-if-catalog-is-unusable) -- The query scans nonaccelerated data - see section [Check if query is over nonaccelerated data](#check-if-query-is-over-nonaccelerated-data) -- The query doesn't comply with KQL best practices - see [KQL best practices](../query/best-practices.md) +- The query acceleration catalog is unusable (out-of-date or never built). To resolve this issue see the [Check if catalog is unusable](#check-if-catalog-is-unusable) section. +- The query scans nonaccelerated data. To resolve this issue see the [Check if query is over nonaccelerated data](#check-if-query-is-over-nonaccelerated-data) section. +- The query doesn't comply with KQL best practices. To resolve this issue see the [KQL best practices](../query/best-practices.md) guide. -## Check if catalog is unusable +## Unusable catalogs -Query acceleration uses a local catalog for the external table containing a snapshot of the delta table metadata. If this catalog hasn't been updated within the configured `MaxAge` (see the query acceleration policy's `MaxAge` property), it's considered **unusable** and isn't used at query time. In that case, queries fall back to reading the remote delta table directly, which can be significantly slower. +Query acceleration uses a local catalog for the external table that contains a snapshot of the delta table metadata. If this catalog isn't updated within the configured `MaxAge` (see the query acceleration policy's `MaxAge` property), it's considered **unusable** and isn't used at query time. In this case, queries fall back to reading the remote delta table directly, which can be significantly slower. -Fetch the current state of the catalog using the following command: +Fetch the current state of the catalog with the following command: ```kusto .show external table [ETName] details @@ -77,43 +78,52 @@ Fetch the current state of the catalog using the following command: | project IsCatalogUnusable = MinimumUpdateTime > todatetime(todynamic(QueryAccelerationState).LastUpdatedDateTime) ``` -`IsCatalogUnusable == true` indicates the catalog is stale and query acceleration won't be used. +`IsCatalogUnusable == true` indicates the catalog is stale and query acceleration isn't used. -### Troubleshoot unusable catalogs +### Nonaccelerated data -To understand why a catalog is unusable, first check if the query acceleration state is healthy and resolve unhealthy reasons as needed. +To get the full benefit of query acceleration, you must run queries over **accelerated data**. If your query runs over non-accelerated data, it reads data directly from the remote delta table, which can cause significant latency. -Run: +Use the following command and filter on a time frame that includes the relevant query: ```kusto -.show external table [ETName] details -| project state = todynamic(QueryAccelerationState) -| project IsHealthy = state.IsHealthy, UnhealthyReason = state.NotHealthyReason +.show queries +| where StartedOn > ago(1h) +| extend ExternalDataStats = OverallQueryStats.input_dataset_statistics.external_data ``` -- If the state is **healthy** but the catalog is still stale, it could be that the query acceleration policy was enabled recently. See [Query acceleration policy was enabled recently](#query-acceleration-policy-was-enabled-recently) -- If the state is **unhealthy**, refer to [Query acceleration unhealthy state – understanding and mitigating](#query-acceleration-unhealthy-state--understanding-and-mitigating). +If `ExternalDataStats.iterated_artifacts` or `ExternalDataStats.downloaded_items` are greater than `0`, the query reads data from the remote delta table (non-accelerated path). +## Troubleshooting -#### Query acceleration policy was enabled recently +### Troubleshoot unusable catalogs -When the query acceleration policy is enabled for the first time, building the initial catalog needs to complete before it can be used in queries. During this period, the `LastUpdatedDateTime` value is empty: +To understand why a catalog is unusable, first check if the query acceleration state is healthy and resolve unhealthy reasons as needed. + +Run: ```kusto .show external table [ETName] details -| project todynamic(QueryAccelerationState).LastUpdatedDateTime +| project state = todynamic(QueryAccelerationState) +| project IsHealthy = state.IsHealthy, UnhealthyReason = state.NotHealthyReason ``` -If `LastUpdatedDateTime` is empty, allow some time for the first update to complete. This usually takes up to several minutes. Subsequent updates are expected to be significantly faster. - -#### Query acceleration unhealthy state – understanding and mitigating +- If the state is **healthy** but the catalog is still stale, it could be that the query acceleration policy was enabled recently. + When you enable the query acceleration policy for the first time, the initial catalog needs to be built before you can use it in queries. During this period, the `LastUpdatedDateTime` value is empty: + + ```kusto + .show external table [ETName] details + | project todynamic(QueryAccelerationState).LastUpdatedDateTime + ``` + + If `LastUpdatedDateTime` is empty, allow some time for the first update to complete. This process usually takes up to several minutes. Subsequent updates are significantly faster. -When an external table's query acceleration is unhealthy, you can retrieve the unhealthy reason using the following command: +- If the state is **unhealthy**, you can retrieve the unhealthy reason by using the following command: -```kusto -.show external table [ETName] details -| project todynamic(QueryAccelerationState).NotHealthyReason -``` + ```kusto + .show external table [ETName] details + | project todynamic(QueryAccelerationState).NotHealthyReason + ``` Use the following table to understand and mitigate common unhealthy states. @@ -126,7 +136,6 @@ Use the following table to understand and mitigate common unhealthy states. > .alter-merge external table [ETName] policy query_acceleration @'{"IsEnabled":true}' >``` - ::: moniker range="azure-data-explorer" | Unhealthy reason | Example `NotHealthyReason` | Action | @@ -156,60 +165,38 @@ Use the following table to understand and mitigate common unhealthy states. ::: moniker-end - -## Check if query is over nonaccelerated data - -To fully benefit from query acceleration, queries must be executed over **accelerated data**. Non-accelerated data is read directly from the remote delta table, which may result in significant latency. - -Use the following command and filter on a time frame that includes the relevant query: - -```kusto -.show queries -| where StartedOn > ago(1h) -| extend ExternalDataStats = OverallQueryStats.input_dataset_statistics.external_data -``` - -If `ExternalDataStats.iterated_artifacts` or `ExternalDataStats.downloaded_items` are greater than `0`, it means data was read from the remote delta table (non-accelerated path). -The following section helps you understand why. - ### Troubleshoot queries over nonaccelerated data -There are two main reasons why a query might read non-accelerated data: +A query might read non-accelerated data for two main reasons: - The query-time filter isn't fully within the query acceleration hot period or hot windows. -- The data within the policy hot period isn't fully cached. - -#### Query filter isn't fully within the hot period or hot windows - -Run the following command to view the hot caching properties and make sure the query filters match them: - -```kusto -.show external table [ETName] policy query_acceleration -| project Policy = todynamic(Policy) -| project Policy.Hot, Policy.HotWindows -``` - -Ensure your query's time filter is fully contained within the configured `Hot` period or the defined `HotWindows`. + Run the following command to view the hot caching properties and make sure the query filters match them: -If it's a one-time query, policy change isn't recommended. However, if you anticipate running multiple queries over the same time range that lies outside the configured `Hot` period or defined `HotWindows` and require improved performance, alter the policy by: - -- Increasing the hot period, and/or -- Adding additional hot windows that match your query patterns. - -#### Data within the hot period isn't fully cached - -Use the following command to check the acceleration progress: - -```kusto -.show external table [ETName] details -| project state = todynamic(QueryAccelerationState) -| project state.CompletionPercentage, state.PendingDataFilesCount -``` - -- If `CompletionPercentage < 100`, allow more time for data to be accelerated. -- If `CompletionPercentage` doesn't increase over time, follow the guidance in [Understanding and mitigating data acceleration issues](#understanding-and-mitigating-data-acceleration-issues). + ```kusto + .show external table [ETName] policy query_acceleration + | project Policy = todynamic(Policy) + | project Policy.Hot, Policy.HotWindows + ``` + + Ensure your query's time filter is fully contained within the configured `Hot` period or the defined `HotWindows`. + + If it's a one-time query, don't change the policy. However, if you anticipate running multiple queries over the same time range that lies outside the configured `Hot` period or defined `HotWindows` and require improved performance, alter the policy by: + + - Increasing the hot period, and/or + - Adding additional hot windows that match your query patterns. +- The data within the policy hot period isn't fully cached. + Use the following command to check the acceleration progress: + + ```kusto + .show external table [ETName] details + | project state = todynamic(QueryAccelerationState) + | project state.CompletionPercentage, state.PendingDataFilesCount + ``` + + - If `CompletionPercentage < 100`, allow more time for data to be accelerated. + - If `CompletionPercentage` doesn't increase over time, follow the guidance in [Understanding and mitigating data acceleration issues](#understanding-and-mitigating-data-acceleration-issues). -### Understanding and mitigating data acceleration issues +## Understanding and mitigating data acceleration issues Unaccelerated data (`CompletionPercentage < 100`) can stem from several issues. @@ -217,18 +204,18 @@ Unaccelerated data (`CompletionPercentage < 100`) can stem from several issues. Data acceleration might take time, especially when: -- A query acceleration policy has recently been enabled, or -- A significant amount of data was recently added to the delta table, or -- The delta table has undergone an optimization operation such as `OPTIMIZE` that results in many deleted and recreated files. +- You recently enabled a query acceleration policy. +- You recently added a significant amount of data to the delta table. +- You ran an optimization operation such as `OPTIMIZE` on the delta table that results in many deleted and recreated files. -Frequently running `OPTIMIZE` or `MERGE` operations on the source delta table that cause large-scale rewrites of data files can negatively affect acceleration performance because data files are repeatedly rewritten, and need to be accelerated. +Frequently running `OPTIMIZE` or `MERGE` operations on the source delta table that cause large-scale rewrites of data files can negatively affect acceleration performance because data files are repeatedly rewritten and need to be accelerated. #### Data files aren't eligible for acceleration -Parquet data files larger than **1 GB** won't be cached. +Parquet data files larger than **1 GB** aren't cached. If your delta table includes many large files, consider adjusting your data generation or optimization strategy to produce smaller parquet files. -If this requires recreating the Delta table, make sure you recreate the external table and re-enable query acceleration policy. +If this adjustment requires recreating the Delta table, make sure you recreate the external table and re-enable the query acceleration policy. #### Insufficient cluster capacity or resources @@ -245,5 +232,5 @@ Run the following command to view the remaining capacity: - If `Remaining == 0` consistently and `CompletionPercentage` isn't increasing, consider: - Scaling the cluster out or up to provide more resources. - - Increasing the `QueryAcceleration` capacity by [altering the capacity policy](alter-capacity-policy-command.md). NOTE: altering the capacity policy may have adverse effects on other operations. Alter the policy as a last resort at your own discretion. + - Increasing the `QueryAcceleration` capacity by [altering the capacity policy](alter-capacity-policy-command.md). NOTE: altering the capacity policy might have adverse effects on other operations. Alter the policy as a last resort at your own discretion. From 58961f20ae24c6660013b9022f0d0e7e77a69b23 Mon Sep 17 00:00:00 2001 From: Hodayah Zargari Date: Wed, 3 Dec 2025 11:55:14 +0200 Subject: [PATCH 2/6] fixed warning issues --- data-explorer/kusto/management/query-acceleration-policy.md | 2 +- .../management/query-acceleration-troubleshooting-guide.md | 5 +++-- data-explorer/kusto/management/toc.yml | 2 +- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/data-explorer/kusto/management/query-acceleration-policy.md b/data-explorer/kusto/management/query-acceleration-policy.md index 17bb45cde4..e68831621f 100644 --- a/data-explorer/kusto/management/query-acceleration-policy.md +++ b/data-explorer/kusto/management/query-acceleration-policy.md @@ -22,7 +22,7 @@ To enable query acceleration in the Fabric UI, see [Query acceleration over OneL ::: moniker-end > [!IMPORTANT] -> To troubleshoot problems with query acceleration, see [Troubleshoot query acceleration over external delta tables](query-acceleration-tsg.md). +> To troubleshoot problems with query acceleration, see [Troubleshoot query acceleration over external delta tables](query-acceleration-troubleshooting-guide.md). ## Limitations diff --git a/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md b/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md index 708ade1b21..91fd7f82f8 100644 --- a/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md +++ b/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md @@ -62,8 +62,9 @@ This issue is a performance problem: the query is slower than expected, and acce Several reasons can cause this issue: -- The query acceleration catalog is unusable (out-of-date or never built). To resolve this issue see the [Check if catalog is unusable](#check-if-catalog-is-unusable) section. -- The query scans nonaccelerated data. To resolve this issue see the [Check if query is over nonaccelerated data](#check-if-query-is-over-nonaccelerated-data) section. +- The query acceleration catalog is unusable (out-of-date or never built). To resolve this issue see the [Troubleshoot unusable catalogs](#troubleshoot-unusable-catalogs) section. +- The query scans nonaccelerated data. To resolve this issue see the [Troubleshoot queries over nonaccelerated data +](#troubleshoot-queries-over-nonaccelerated-data) section. - The query doesn't comply with KQL best practices. To resolve this issue see the [KQL best practices](../query/best-practices.md) guide. ## Unusable catalogs diff --git a/data-explorer/kusto/management/toc.yml b/data-explorer/kusto/management/toc.yml index 98d7e137d6..59cb1c4f41 100644 --- a/data-explorer/kusto/management/toc.yml +++ b/data-explorer/kusto/management/toc.yml @@ -587,7 +587,7 @@ items: href: query-acceleration-policy.md displayName: external table, delta table - name: Query acceleration troubleshooting guide - href: query-acceleration-tsg.md + href: query-acceleration-troubleshooting-guide.md displayName: external table, delta table - name: .alter query acceleration policy command href: alter-query-acceleration-policy-command.md From 61bd5f44c9e8bbe8dd9fe4aee39b6fb8f7b43358 Mon Sep 17 00:00:00 2001 From: Hodayah Zargari Date: Wed, 3 Dec 2025 12:08:33 +0200 Subject: [PATCH 3/6] read-through --- ...uery-acceleration-troubleshooting-guide.md | 22 +++++++++---------- 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md b/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md index 91fd7f82f8..c28eca2c7d 100644 --- a/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md +++ b/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md @@ -58,20 +58,17 @@ You can control the effective `MaxAge` in two ways: ### Query latency -This issue is a performance problem: the query is slower than expected, and acceleration doesn't seem to improve performance. +If a query is performing slower than anticipated and query acceleration doesn't seem to enhance performance, consider the following potential causes: -Several reasons can cause this issue: +- **Unusable query acceleration catalog**: The catalog may be outdated or not yet built. Refer to the [Troubleshoot unusable catalogs](#troubleshoot-unusable-catalogs) section for resolution steps. +- **Query accessing non-accelerated data**: The query might be scanning data that hasn't been cached. See the [Troubleshoot queries over nonaccelerated data](#troubleshoot-queries-over-nonaccelerated-data) section for guidance. +- **Non-compliance with KQL best practices**: Ensure the query adheres to KQL best practices. Refer to the [KQL best practices](../query/best-practices.md) guide for optimization techniques. -- The query acceleration catalog is unusable (out-of-date or never built). To resolve this issue see the [Troubleshoot unusable catalogs](#troubleshoot-unusable-catalogs) section. -- The query scans nonaccelerated data. To resolve this issue see the [Troubleshoot queries over nonaccelerated data -](#troubleshoot-queries-over-nonaccelerated-data) section. -- The query doesn't comply with KQL best practices. To resolve this issue see the [KQL best practices](../query/best-practices.md) guide. - -## Unusable catalogs +### Unusable catalogs Query acceleration uses a local catalog for the external table that contains a snapshot of the delta table metadata. If this catalog isn't updated within the configured `MaxAge` (see the query acceleration policy's `MaxAge` property), it's considered **unusable** and isn't used at query time. In this case, queries fall back to reading the remote delta table directly, which can be significantly slower. -Fetch the current state of the catalog with the following command: +Retrieve the current state of the catalog with the following command: ```kusto .show external table [ETName] details @@ -81,9 +78,8 @@ Fetch the current state of the catalog with the following command: `IsCatalogUnusable == true` indicates the catalog is stale and query acceleration isn't used. -### Nonaccelerated data -To get the full benefit of query acceleration, you must run queries over **accelerated data**. If your query runs over non-accelerated data, it reads data directly from the remote delta table, which can cause significant latency. +To maximize the benefits of query acceleration, ensure your queries target **accelerated data**. Queries that access non-accelerated data retrieve information directly from the remote delta table, which might result in higher latency. Use the following command and filter on a time frame that includes the relevant query: @@ -233,5 +229,7 @@ Run the following command to view the remaining capacity: - If `Remaining == 0` consistently and `CompletionPercentage` isn't increasing, consider: - Scaling the cluster out or up to provide more resources. - - Increasing the `QueryAcceleration` capacity by [altering the capacity policy](alter-capacity-policy-command.md). NOTE: altering the capacity policy might have adverse effects on other operations. Alter the policy as a last resort at your own discretion. + - Increasing the `QueryAcceleration` capacity by [altering the capacity policy](alter-capacity-policy-command.md). + > [!NOTE] + > Altering the capacity policy might have adverse effects on other operations. Alter the policy as a last resort at your own discretion. From 4912b84ed4ef70d6e5a4e71827126c371dfcc6c1 Mon Sep 17 00:00:00 2001 From: Hodayah Zargari Date: Mon, 8 Dec 2025 12:09:03 +0200 Subject: [PATCH 4/6] reformatting --- ...uery-acceleration-troubleshooting-guide.md | 57 +++++++++---------- 1 file changed, 27 insertions(+), 30 deletions(-) diff --git a/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md b/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md index c28eca2c7d..68569f07e2 100644 --- a/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md +++ b/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md @@ -3,7 +3,7 @@ title: Query Acceleration Troubleshooting Guide description: Learn how to troubleshoot for common errors encountered in query acceleration. ms.reviewer: urishapira ms.topic: reference -ms.date: 12/03/2025 +ms.date: 12/08/2025 --- # Troubleshoot query acceleration over external delta tables @@ -40,7 +40,7 @@ This article helps you troubleshoot scenarios where: The query acceleration feature assumes a delta table that complies with the Delta protocol. Manual operations executed directly on the delta table (for example, editing transaction logs or parquet files) aren't supported and might result in unexpected behavior. - If you executed such operations on the delta table, recreate the external table and re-enable the query acceleration policy. + If such operations have been executed on the delta table, recreate the external table and re-enable the query acceleration policy. ## Common errors @@ -56,15 +56,17 @@ You can control the effective `MaxAge` in two ways: - Configure the `MaxAge` property in the query acceleration policy by using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md). - Override `MaxAge` per query by using the [`external_table()` operator's](../query/external-table-function.md) `MaxAgeOverride` parameter. -### Query latency +### Query over an accelerated external delta table is *slower* than expected If a query is performing slower than anticipated and query acceleration doesn't seem to enhance performance, consider the following potential causes: -- **Unusable query acceleration catalog**: The catalog may be outdated or not yet built. Refer to the [Troubleshoot unusable catalogs](#troubleshoot-unusable-catalogs) section for resolution steps. -- **Query accessing non-accelerated data**: The query might be scanning data that hasn't been cached. See the [Troubleshoot queries over nonaccelerated data](#troubleshoot-queries-over-nonaccelerated-data) section for guidance. +- **Unusable query acceleration catalog**: The catalog may be outdated or not yet built. Refer to the [Troubleshoot unusable catalogs](#troubleshoot-unusable-catalogs) section to check if this is the case. +- **Query accessing non-accelerated data**: The query might be scanning data that hasn't been cached. See the [Troubleshoot queries over nonaccelerated data](#troubleshoot-queries-over-nonaccelerated-data) section to check if this is the case. - **Non-compliance with KQL best practices**: Ensure the query adheres to KQL best practices. Refer to the [KQL best practices](../query/best-practices.md) guide for optimization techniques. -### Unusable catalogs +## Troubleshooting steps + +### Step 1: Check for unusable catalogs Query acceleration uses a local catalog for the external table that contains a snapshot of the delta table metadata. If this catalog isn't updated within the configured `MaxAge` (see the query acceleration policy's `MaxAge` property), it's considered **unusable** and isn't used at query time. In this case, queries fall back to reading the remote delta table directly, which can be significantly slower. @@ -78,24 +80,9 @@ Retrieve the current state of the catalog with the following command: `IsCatalogUnusable == true` indicates the catalog is stale and query acceleration isn't used. +### Step 2: Check query acceleration state health -To maximize the benefits of query acceleration, ensure your queries target **accelerated data**. Queries that access non-accelerated data retrieve information directly from the remote delta table, which might result in higher latency. - -Use the following command and filter on a time frame that includes the relevant query: - -```kusto -.show queries -| where StartedOn > ago(1h) -| extend ExternalDataStats = OverallQueryStats.input_dataset_statistics.external_data -``` - -If `ExternalDataStats.iterated_artifacts` or `ExternalDataStats.downloaded_items` are greater than `0`, the query reads data from the remote delta table (non-accelerated path). - -## Troubleshooting - -### Troubleshoot unusable catalogs - -To understand why a catalog is unusable, first check if the query acceleration state is healthy and resolve unhealthy reasons as needed. +To understand why a catalog is unusable, check if the query acceleration state is healthy and resolve unhealthy reasons as needed. Run: @@ -162,11 +149,21 @@ Use the following table to understand and mitigate common unhealthy states. ::: moniker-end -### Troubleshoot queries over nonaccelerated data +### Step 3: Check if query is over nonaccelerated data + +To fully benefit from query acceleration, queries must be executed over accelerated data. Non-accelerated data is read directly from the remote delta table, which may result in significant latency. +Use the following command and filter on a time frame that includes the relevant query: + +```kusto +.show queries +| where StartedOn > ago(1h) +| extend ExternalDataStats = OverallQueryStats.input_dataset_statistics.external_data +``` +If ExternalDataStats.iterated_artifacts or ExternalDataStats.downloaded_items are greater than 0, it means data was read from the remote delta table (non-accelerated path). The following section helps you understand why. -A query might read non-accelerated data for two main reasons: +A query might read nonaccelerated data for two main reasons: -- The query-time filter isn't fully within the query acceleration hot period or hot windows. +- **The query-time filter isn't fully within the query acceleration hot period or hot windows**. Run the following command to view the hot caching properties and make sure the query filters match them: ```kusto @@ -179,9 +176,9 @@ A query might read non-accelerated data for two main reasons: If it's a one-time query, don't change the policy. However, if you anticipate running multiple queries over the same time range that lies outside the configured `Hot` period or defined `HotWindows` and require improved performance, alter the policy by: - - Increasing the hot period, and/or - - Adding additional hot windows that match your query patterns. -- The data within the policy hot period isn't fully cached. + - Increasing the hot period, and/or + - Adding additional hot windows that match your query patterns. +- **The data within the policy hot period isn't fully cached.** Use the following command to check the acceleration progress: ```kusto @@ -193,7 +190,7 @@ A query might read non-accelerated data for two main reasons: - If `CompletionPercentage < 100`, allow more time for data to be accelerated. - If `CompletionPercentage` doesn't increase over time, follow the guidance in [Understanding and mitigating data acceleration issues](#understanding-and-mitigating-data-acceleration-issues). -## Understanding and mitigating data acceleration issues +### Step 4: Understanding and mitigating data acceleration issues Unaccelerated data (`CompletionPercentage < 100`) can stem from several issues. From c5551d907de113d38fbee58634a0fb6371d13fd6 Mon Sep 17 00:00:00 2001 From: Hodayah Zargari Date: Mon, 8 Dec 2025 12:20:17 +0200 Subject: [PATCH 5/6] links fix --- .../query-acceleration-troubleshooting-guide.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md b/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md index 68569f07e2..4e05ce064f 100644 --- a/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md +++ b/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md @@ -66,7 +66,9 @@ If a query is performing slower than anticipated and query acceleration doesn't ## Troubleshooting steps -### Step 1: Check for unusable catalogs +### Troubleshoot unusable catalogs + +#### Step 1: Check for unusable catalogs Query acceleration uses a local catalog for the external table that contains a snapshot of the delta table metadata. If this catalog isn't updated within the configured `MaxAge` (see the query acceleration policy's `MaxAge` property), it's considered **unusable** and isn't used at query time. In this case, queries fall back to reading the remote delta table directly, which can be significantly slower. @@ -80,7 +82,7 @@ Retrieve the current state of the catalog with the following command: `IsCatalogUnusable == true` indicates the catalog is stale and query acceleration isn't used. -### Step 2: Check query acceleration state health +#### Step 2: Check query acceleration state health To understand why a catalog is unusable, check if the query acceleration state is healthy and resolve unhealthy reasons as needed. @@ -149,7 +151,9 @@ Use the following table to understand and mitigate common unhealthy states. ::: moniker-end -### Step 3: Check if query is over nonaccelerated data +### Troubleshoot queries over nonaccelerated data + +#### Step 3: Check if query is over nonaccelerated data To fully benefit from query acceleration, queries must be executed over accelerated data. Non-accelerated data is read directly from the remote delta table, which may result in significant latency. Use the following command and filter on a time frame that includes the relevant query: @@ -159,6 +163,7 @@ Use the following command and filter on a time frame that includes the relevant | where StartedOn > ago(1h) | extend ExternalDataStats = OverallQueryStats.input_dataset_statistics.external_data ``` + If ExternalDataStats.iterated_artifacts or ExternalDataStats.downloaded_items are greater than 0, it means data was read from the remote delta table (non-accelerated path). The following section helps you understand why. A query might read nonaccelerated data for two main reasons: @@ -190,7 +195,7 @@ A query might read nonaccelerated data for two main reasons: - If `CompletionPercentage < 100`, allow more time for data to be accelerated. - If `CompletionPercentage` doesn't increase over time, follow the guidance in [Understanding and mitigating data acceleration issues](#understanding-and-mitigating-data-acceleration-issues). -### Step 4: Understanding and mitigating data acceleration issues +#### Step 4: Understanding and mitigating data acceleration issues Unaccelerated data (`CompletionPercentage < 100`) can stem from several issues. From 26c73230d81ffb6f347a312d532f4f03f74b78a3 Mon Sep 17 00:00:00 2001 From: Hodayah Zargari Date: Mon, 8 Dec 2025 14:00:47 +0200 Subject: [PATCH 6/6] broken link --- .../management/query-acceleration-troubleshooting-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md b/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md index 4e05ce064f..600db4c098 100644 --- a/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md +++ b/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md @@ -193,7 +193,7 @@ A query might read nonaccelerated data for two main reasons: ``` - If `CompletionPercentage < 100`, allow more time for data to be accelerated. - - If `CompletionPercentage` doesn't increase over time, follow the guidance in [Understanding and mitigating data acceleration issues](#understanding-and-mitigating-data-acceleration-issues). + - If `CompletionPercentage` doesn't increase over time, follow the guidance in [Understanding and mitigating data acceleration issues](#step-4-understanding-and-mitigating-data-acceleration-issues). #### Step 4: Understanding and mitigating data acceleration issues