From e7cf23af993615532c5a5e3dd6283ad7637adb55 Mon Sep 17 00:00:00 2001 From: "urishapira@microsoft.com" Date: Wed, 19 Nov 2025 11:00:50 +0200 Subject: [PATCH 01/16] updates --- .../management/query-acceleration-tsg.md | 240 ++++++++++++++++++ data-explorer/kusto/management/toc.yml | 3 + 2 files changed, 243 insertions(+) create mode 100644 data-explorer/kusto/management/query-acceleration-tsg.md diff --git a/data-explorer/kusto/management/query-acceleration-tsg.md b/data-explorer/kusto/management/query-acceleration-tsg.md new file mode 100644 index 0000000000..fbcd8acf7e --- /dev/null +++ b/data-explorer/kusto/management/query-acceleration-tsg.md @@ -0,0 +1,240 @@ +# Troubleshoot query acceleration over external delta tables +[!INCLUDE applies] [!INCLUDE fabric] [!INCLUDE azure-data-explorer] + +The [query acceleration policy](query-acceleration-policy.md) enables accelerating queries over external delta tables by caching delta table metadata and data files. The policy defines which data ranges (number of days back and hot windows) are accelerated so that queries over those ranges can run significantly faster. + +The query acceleration feature consists of the following components: + +- A background job that maintains a local snapshot (**catalog**) of the delta table metadata. +- A background job that accelerates delta table data files. +- Query-time enhancements that utilize accelerated data. + +To understand why things aren't working as expected, it's important to identify which of these components isn't functioning properly. + +This article helps you troubleshoot scenarios where: + +- Queries over accelerated external delta tables return **old data**, or +- Queries over accelerated external delta tables are **not significantly faster** than non-accelerated queries. + +## Prerequisites + +1. **Ensure query acceleration is enabled on the external table** by running the following command: + + ```kusto + .show external table policy query_acceleration + | project todynamic(Policy).IsEnabled + ``` + + If this command returns `false`, enable the query acceleration policy using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md). + +2. **Ensure the delta table complies with the Delta protocol.** + + The query acceleration feature assumes a delta table that complies with the Delta protocol. Manual operations executed directly on the delta table (for example, editing transaction logs or parquet files) aren't supported and may result in unexpected behavior. + + If such operations have been executed on the delta table, first recreate the external table and re-enable the query acceleration policy. + +## Troubleshooting tree + +Use the following logical flow to identify and mitigate query acceleration issues. + +- **Query acceleration issues** + - **[Query is returning old data](#query-is-returning-old-data)** + Result freshness issue: query results don't reflect the latest data from the underlying delta table. + - **Query is not running fast enough** + Performance issue: query is slower than expected, and acceleration doesn't appear to improve performance. + - **[Check if catalog is stale](#check-if-catalog-is-stale)** + Is the query acceleration catalog older than the configured `MaxAge` and therefore not used? + - **[Troubleshoot stale catalogs](#troubleshoot-stale-catalogs)** + Diagnose why the catalog isn't refreshing (for example, unhealthy state, frequent changes, or recent enablement). + - **[Query acceleration unhealthy state – understanding and mitigating](#query-acceleration-unhealthy-state--understanding-and-mitigating)** + Is query acceleration unhealthy due to configuration or schema issues? + - **[Check if query is over non-accelerated data](#check-if-query-is-over-non-accelerated-data)** + Is the query reading data directly from the remote delta table? + - **[Troubleshoot queries over non-accelerated-data](#troubleshoot-queries-over-non-accelerated-data)** + Align query filters with the hot period or hot windows and verify that data within those ranges is fully cached. + - **[Understanding and mitigating data acceleration issues](#understanding-and-mitigating-data-acceleration-issues)** + Investigate incomplete acceleration due to ongoing caching, large parquet files, or insufficient cluster capacity. + +## Query is returning old data + +Query acceleration refreshes the accelerated data so that results are no older than the configured `MaxAge` value in the policy. Set `MaxAge` to the maximum data staleness that is acceptable at query time. + +You can control the effective `MaxAge` in two ways: + +1. Configure the `MaxAge` property in the query acceleration policy using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md). +2. Override `MaxAge` per query by using the [`external_table()` operator's](../query/external-table-function.md) `MaxAgeOverride` parameter. + +## Check if catalog is stale + +Query acceleration uses a local catalog for the external table containing a snapshot of the delta table metadata. If this catalog hasn't been updated within the configured `MaxAge` (see the query acceleration policy's `MaxAge` property), it's considered **stale** and won't be used at query time. In that case, queries fall back to reading the remote delta table directly, which can be significantly slower. + +Fetch the current state of the catalog using the following command: + +```kusto +.show external table [ETName] details +| extend MinimumUpdateTime = now() - totimespan(todynamic(QueryAccelerationPolicy).MaxAge) +| project IsCatalogStale = MinimumUpdateTime < todatetime(todynamic(QueryAccelerationState).LastUpdatedDateTime) +``` + +`IsCatalogStale == true` indicates the catalog is stale and query acceleration won't be used. + +### Troubleshoot stale catalogs + +To understand why a catalog is stale, first check whether the query acceleration state is healthy and resolve unhealthy reasons as needed. + +Run: + +```kusto +.show external table [ETName] details +| project state = todynamic(QueryAccelerationState) +| project IsHealthy = state.IsHealthy, UnhealthyReason = state.NotHealthyReason +``` + +- If the state is **unhealthy**, refer to [Query acceleration unhealthy state – understanding and mitigating](#query-acceleration-unhealthy-state--understanding-and-mitigating). +- If the state is **healthy** but the catalog is still stale, consider the following cases. + +#### Query acceleration policy was enabled recently + +When the query acceleration policy is enabled for the first time, the initial catalog build needs to complete before the catalog becomes usable. During this period, you might see an empty `LastUpdatedDateTime` value: + +```kusto +.show external table [ETName] details +| project todynamic(QueryAccelerationState).LastUpdatedDateTime +``` + +If `LastUpdatedDateTime` is empty, allow some time for the first update to complete. Subsequent updates are expected to be significantly faster. + +#### The delta table is frequently changing + +A `MaxAge` value (default is five minutes) that is too low for a frequently changing delta table can result in a constantly stale catalog. For example, if the delta table undergoes frequent `OPTIMIZE` or `MERGE` operations that rewrite a large portion of the underlying parquet files (such as aggressive compaction or large upserts), the catalog might lag behind. + +If the queries can tolerate slightly older data, consider increasing the `MaxAge` value using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md). + +## Query acceleration unhealthy state – understanding and mitigating + +When an external table's query acceleration is unhealthy, you can retrieve the unhealthy reason using the following command: + +```kusto +.show external table [ETName] details +| project todynamic(QueryAccelerationState).NotHealthyReason +``` + +Use the following tables to understand and mitigate common unhealthy states. + +::: moniker range="azure-data-explorer" + +| Unhealthy reason | Example `NotHealthyReason` | Action | +| --------------------------------------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| External table access is forbidden | `InaccessibleDeltaTable: Access to Delta table is forbidden` | Alter the external table connection string with a valid authentication method. | +| External table connection string doesn't point to a valid delta table | `DeltaTableNotFound: Delta table does not exist` | Alter the external table connection string to point to a valid delta table location. | +| Delta table column mapping mode has changed | `ColumnMappingModeChange: Column mapping mode has changed. Previous: 'None', New: 'Name'` | Recreate the external table with the new column mapping mode and re-enable the query acceleration policy. | +| Delta table column mappings have changed | `NewColumnMapping: New column mapping was introduced. Column 'Col1' is now mapped to 'Col2'` | Recreate the external table and re-enable the query acceleration policy so that column mappings are aligned with the delta table. | +| Delta table column type has changed | `ColumnTypeMismatch: Column 'Col1' type has changed. Previous delta type: 'long', New type: 'string'. Respective external table type: long` | Recreate (or alter) the external table so that its schema is aligned with the delta table column types, and then re-enable query acceleration. | +| Managed identity error | *Managed identity must be specified for external tables with impersonation authentication.* | Ensure that the query acceleration policy contains a valid managed identity that has:
• Appropriate permissions on the delta table
• The `AutomatedFlows` usage type in the cluster or database managed identity policy. | +| Hot datetime column not found | `HotDateTimeColumn 'Col1' does not exist as a datetime column in the Delta table schema` | Alter the query acceleration policy to include a valid `HotDateTimeColumn`, or leave the property empty if not required. | +| Delta table has unsupported features for query acceleration | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. | + +::: moniker-end + +::: moniker range="microsoft-fabric" + +| Unhealthy reason | Example `NotHealthyReason` | Action | +| --------------------------------------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| External table access is forbidden | `InaccessibleDeltaTable: Access to Delta table is forbidden` | Alter the external table connection string with a valid authentication method. | +| External table connection string doesn't point to a valid delta table | `DeltaTableNotFound: Delta table does not exist` | Alter the external table connection string to point to a valid delta table location. | +| Delta table column mapping mode has changed | `ColumnMappingModeChange: Column mapping mode has changed. Previous: 'None', New: 'Name'` | Recreate the external table with the new column mapping mode and re-enable the query acceleration policy. | +| Delta table column mappings have changed | `NewColumnMapping: New column mapping was introduced. Column 'Col1' is now mapped to 'Col2'` | Recreate the external table and re-enable the query acceleration policy so that column mappings are aligned with the delta table. | +| Delta table column type has changed | `ColumnTypeMismatch: Column 'Col1' type has changed. Previous delta type: 'long', New type: 'string'. Respective external table type: long` | Recreate (or alter) the external table so that its schema is aligned with the delta table column types, and then re-enable query acceleration. | +| Hot datetime column not found | `HotDateTimeColumn 'Col1' does not exist as a datetime column in the Delta table schema` | Alter the query acceleration policy to include a valid `HotDateTimeColumn`, or leave the property empty if not required. | +| Delta table has unsupported features for query acceleration | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. | + +::: moniker-end + + +## Check if query is over non-accelerated data + +To fully benefit from query acceleration, queries must be executed over **accelerated data**. Non-accelerated data is read directly from the remote delta table, which may result in significant latency. + +Use the following command and filter on a time frame that includes the relevant query: + +```kusto +.show queries +| where StartedOn > ago(1h) +| extend ExternalDataStats = ['OverallQueryStats']['input_dataset_statistics']['external_data'] +``` + +If `ExternalDataStats.iterated_artifacts` or `ExternalDataStats.downloaded_items` are greater than `0`, it means data is being read from the remote delta table (non-accelerated path). + +### Troubleshoot queries over non-accelerated-data + +There are two main reasons why a query might read non-accelerated data: + +1. **The query-time filter isn't fully within the query acceleration hot period or hot windows.** +2. **The data within the policy hot period isn't fully cached.** + +#### Query filter isn't fully within the hot period or hot windows + +Run the following command to view the hot caching properties and make sure the query filters match them: + +```kusto +.show external table [ETName] policy query_acceleration +| project Policy = todynamic(Policy) +| project Policy.Hot, Policy.HotWindows +``` + +- Ensure your query's time filter is fully contained within the configured `Hot` period or the defined `HotWindows`. + +If the query needs to access data outside the configured hot period or hot windows, alter the policy by: + +- Increasing the hot period, or +- Adding additional hot windows that match your query patterns. + +#### Data within the hot period isn't fully cached + +Use the following command to check the acceleration progress: + +```kusto +.show external table [ETName] details +| project state = todynamic(QueryAccelerationState) +| project state.CompletionPercentage, state.PendingDataFilesCount +``` + +- If `CompletionPercentage < 100`, allow more time for data to be accelerated. +- If `CompletionPercentage` doesn't increase over time, follow the guidance in [Understanding and mitigating data acceleration issues](#understanding-and-mitigating-data-acceleration-issues). + +## Understanding and mitigating data acceleration issues + +Unaccelerated data (`CompletionPercentage < 100`) can stem from several issues. + +### Data is currently being accelerated + +Data acceleration might take time, especially when: + +- A query acceleration policy has recently been enabled, or +- The delta table has undergone an optimization operation such as `OPTIMIZE` that results in many deleted and recreated files. + +Frequently running `OPTIMIZE` or `MERGE` operations on the source delta table that cause large-scale rewrites of data files can negatively affect acceleration performance because data files are repeatedly rewritten. + +### Data files aren't eligible for acceleration + +Parquet data files with a **compressed size greater than 1 GB** won't be cached. + +If your delta table includes many large files, consider adjusting your data generation or optimization strategy to produce smaller parquet files. +If this requires recreating the Delta table, make sure you recreate the external table and reenable query acceleration policy. + +### Insufficient cluster capacity or resources + +Query acceleration operations are restricted by the cluster's available query acceleration capacity. + +Run the following command to view the remaining capacity: + +```kusto +.show capacity +| where Resource == 'QueryAcceleration' +| project Remaining +``` + +- If `Remaining == 0` consistently and `CompletionPercentage` isn't increasing, consider: + + - Increasing the `QueryAcceleration` capacity by altering the capacity policy. + - Scaling the cluster out or up to provide more resources. diff --git a/data-explorer/kusto/management/toc.yml b/data-explorer/kusto/management/toc.yml index 84ade8be98..98d7e137d6 100644 --- a/data-explorer/kusto/management/toc.yml +++ b/data-explorer/kusto/management/toc.yml @@ -586,6 +586,9 @@ items: - name: Query acceleration policy href: query-acceleration-policy.md displayName: external table, delta table + - name: Query acceleration troubleshooting guide + href: query-acceleration-tsg.md + displayName: external table, delta table - name: .alter query acceleration policy command href: alter-query-acceleration-policy-command.md displayName: external table, delta table From 07d19484c41dc2ad9af5d339842abe6a8886092b Mon Sep 17 00:00:00 2001 From: "urishapira@microsoft.com" Date: Wed, 19 Nov 2025 11:12:25 +0200 Subject: [PATCH 02/16] add link in general page --- data-explorer/kusto/management/query-acceleration-policy.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/data-explorer/kusto/management/query-acceleration-policy.md b/data-explorer/kusto/management/query-acceleration-policy.md index 87ac974818..17bb45cde4 100644 --- a/data-explorer/kusto/management/query-acceleration-policy.md +++ b/data-explorer/kusto/management/query-acceleration-policy.md @@ -21,6 +21,9 @@ Query acceleration is supported in Eventhouse over OneLake, Azure Data Lake Stor To enable query acceleration in the Fabric UI, see [Query acceleration over OneLake shortcuts](https://go.microsoft.com/fwlink/?linkid=2296674). ::: moniker-end +> [!IMPORTANT] +> To troubleshoot problems with query acceleration, see [Troubleshoot query acceleration over external delta tables](query-acceleration-tsg.md). + ## Limitations * The number of columns in the external table can't exceed 900. From 58d9c282995eb942805ac2597c7f4207362e7e24 Mon Sep 17 00:00:00 2001 From: "urishapira@microsoft.com" Date: Wed, 19 Nov 2025 11:20:43 +0200 Subject: [PATCH 03/16] fix applies to --- data-explorer/kusto/management/query-acceleration-tsg.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/data-explorer/kusto/management/query-acceleration-tsg.md b/data-explorer/kusto/management/query-acceleration-tsg.md index fbcd8acf7e..6aeea7ea3f 100644 --- a/data-explorer/kusto/management/query-acceleration-tsg.md +++ b/data-explorer/kusto/management/query-acceleration-tsg.md @@ -1,5 +1,6 @@ # Troubleshoot query acceleration over external delta tables -[!INCLUDE applies] [!INCLUDE fabric] [!INCLUDE azure-data-explorer] + +> [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] The [query acceleration policy](query-acceleration-policy.md) enables accelerating queries over external delta tables by caching delta table metadata and data files. The policy defines which data ranges (number of days back and hot windows) are accelerated so that queries over those ranges can run significantly faster. From 3a4cc9c683d5b69a7f2e3a4d46aa2455585339dc Mon Sep 17 00:00:00 2001 From: urishapira <89069415+urishapira@users.noreply.github.com> Date: Wed, 26 Nov 2025 22:08:46 +0200 Subject: [PATCH 04/16] Update data-explorer/kusto/management/query-acceleration-tsg.md Co-authored-by: Slavik N <34075198+slavikn84@users.noreply.github.com> --- data-explorer/kusto/management/query-acceleration-tsg.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data-explorer/kusto/management/query-acceleration-tsg.md b/data-explorer/kusto/management/query-acceleration-tsg.md index 6aeea7ea3f..3c28b76315 100644 --- a/data-explorer/kusto/management/query-acceleration-tsg.md +++ b/data-explorer/kusto/management/query-acceleration-tsg.md @@ -8,7 +8,7 @@ The query acceleration feature consists of the following components: - A background job that maintains a local snapshot (**catalog**) of the delta table metadata. - A background job that accelerates delta table data files. -- Query-time enhancements that utilize accelerated data. +- Query-time enhancements that utilize the catalog and the cached data. To understand why things aren't working as expected, it's important to identify which of these components isn't functioning properly. From 49b9da63eb6f78e2b346d302220b21fc555a48c5 Mon Sep 17 00:00:00 2001 From: urishapira <89069415+urishapira@users.noreply.github.com> Date: Wed, 26 Nov 2025 22:09:23 +0200 Subject: [PATCH 05/16] Update data-explorer/kusto/management/query-acceleration-tsg.md Co-authored-by: Slavik N <34075198+slavikn84@users.noreply.github.com> --- data-explorer/kusto/management/query-acceleration-tsg.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data-explorer/kusto/management/query-acceleration-tsg.md b/data-explorer/kusto/management/query-acceleration-tsg.md index 3c28b76315..67aa86f570 100644 --- a/data-explorer/kusto/management/query-acceleration-tsg.md +++ b/data-explorer/kusto/management/query-acceleration-tsg.md @@ -14,7 +14,7 @@ To understand why things aren't working as expected, it's important to identify This article helps you troubleshoot scenarios where: -- Queries over accelerated external delta tables return **old data**, or +- Queries over accelerated external delta tables return **stale data**, or - Queries over accelerated external delta tables are **not significantly faster** than non-accelerated queries. ## Prerequisites From 93afd866903c0c2d740a4db3a1ae1be49b7e1b01 Mon Sep 17 00:00:00 2001 From: "urishapira@microsoft.com" Date: Wed, 26 Nov 2025 23:27:34 +0200 Subject: [PATCH 06/16] pr comments --- .../management/query-acceleration-tsg.md | 51 ++++++++----------- 1 file changed, 21 insertions(+), 30 deletions(-) diff --git a/data-explorer/kusto/management/query-acceleration-tsg.md b/data-explorer/kusto/management/query-acceleration-tsg.md index 67aa86f570..c576ca26e9 100644 --- a/data-explorer/kusto/management/query-acceleration-tsg.md +++ b/data-explorer/kusto/management/query-acceleration-tsg.md @@ -7,7 +7,7 @@ The [query acceleration policy](query-acceleration-policy.md) enables accelerati The query acceleration feature consists of the following components: - A background job that maintains a local snapshot (**catalog**) of the delta table metadata. -- A background job that accelerates delta table data files. +- A background job that caches delta table data files. - Query-time enhancements that utilize the catalog and the cached data. To understand why things aren't working as expected, it's important to identify which of these components isn't functioning properly. @@ -15,7 +15,7 @@ To understand why things aren't working as expected, it's important to identify This article helps you troubleshoot scenarios where: - Queries over accelerated external delta tables return **stale data**, or -- Queries over accelerated external delta tables are **not significantly faster** than non-accelerated queries. +- Queries over accelerated external delta tables are **slower than expected** ## Prerequisites @@ -34,7 +34,7 @@ This article helps you troubleshoot scenarios where: If such operations have been executed on the delta table, first recreate the external table and re-enable the query acceleration policy. -## Troubleshooting tree +## Troubleshooting flow Use the following logical flow to identify and mitigate query acceleration issues. @@ -55,10 +55,13 @@ Use the following logical flow to identify and mitigate query acceleration issue Align query filters with the hot period or hot windows and verify that data within those ranges is fully cached. - **[Understanding and mitigating data acceleration issues](#understanding-and-mitigating-data-acceleration-issues)** Investigate incomplete acceleration due to ongoing caching, large parquet files, or insufficient cluster capacity. + - **[Ensure query complies with KQL best practices](../query/best-practices.md)**
+ Optimize the query as is instructed in the KQL best practices document ## Query is returning old data -Query acceleration refreshes the accelerated data so that results are no older than the configured `MaxAge` value in the policy. Set `MaxAge` to the maximum data staleness that is acceptable at query time. +Query acceleration refreshes the accelerated data so that results are no older than the configured `MaxAge` value in the policy. +By design, queries over accelerated external tables may return data that lags behind the latest delta table version by up to `MaxAge`. Set `MaxAge` to the maximum data staleness that is acceptable at query time. You can control the effective `MaxAge` in two ways: @@ -96,14 +99,14 @@ Run: #### Query acceleration policy was enabled recently -When the query acceleration policy is enabled for the first time, the initial catalog build needs to complete before the catalog becomes usable. During this period, you might see an empty `LastUpdatedDateTime` value: +When the query acceleration policy is enabled for the first time, building the initial catalog needs to complete before it can be used in queries. During this period, the `LastUpdatedDateTime` value will be empty: ```kusto .show external table [ETName] details | project todynamic(QueryAccelerationState).LastUpdatedDateTime ``` -If `LastUpdatedDateTime` is empty, allow some time for the first update to complete. Subsequent updates are expected to be significantly faster. +If `LastUpdatedDateTime` is empty, allow some time for the first update to complete. This usually takes several minutes. Subsequent updates are expected to be significantly faster. #### The delta table is frequently changing @@ -120,20 +123,14 @@ When an external table's query acceleration is unhealthy, you can retrieve the u | project todynamic(QueryAccelerationState).NotHealthyReason ``` -Use the following tables to understand and mitigate common unhealthy states. +Use the following table to understand and mitigate common unhealthy states. ::: moniker range="azure-data-explorer" | Unhealthy reason | Example `NotHealthyReason` | Action | | --------------------------------------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| External table access is forbidden | `InaccessibleDeltaTable: Access to Delta table is forbidden` | Alter the external table connection string with a valid authentication method. | -| External table connection string doesn't point to a valid delta table | `DeltaTableNotFound: Delta table does not exist` | Alter the external table connection string to point to a valid delta table location. | -| Delta table column mapping mode has changed | `ColumnMappingModeChange: Column mapping mode has changed. Previous: 'None', New: 'Name'` | Recreate the external table with the new column mapping mode and re-enable the query acceleration policy. | -| Delta table column mappings have changed | `NewColumnMapping: New column mapping was introduced. Column 'Col1' is now mapped to 'Col2'` | Recreate the external table and re-enable the query acceleration policy so that column mappings are aligned with the delta table. | -| Delta table column type has changed | `ColumnTypeMismatch: Column 'Col1' type has changed. Previous delta type: 'long', New type: 'string'. Respective external table type: long` | Recreate (or alter) the external table so that its schema is aligned with the delta table column types, and then re-enable query acceleration. | -| Managed identity error | *Managed identity must be specified for external tables with impersonation authentication.* | Ensure that the query acceleration policy contains a valid managed identity that has:
• Appropriate permissions on the delta table
• The `AutomatedFlows` usage type in the cluster or database managed identity policy. | -| Hot datetime column not found | `HotDateTimeColumn 'Col1' does not exist as a datetime column in the Delta table schema` | Alter the query acceleration policy to include a valid `HotDateTimeColumn`, or leave the property empty if not required. | -| Delta table has unsupported features for query acceleration | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. | +[!INCLUDE [query-acceleration-unhealthy-reasons-table](query-acceleration-unhealthy-reasons.md)] +| Managed identity error | *Managed identity must be specified for external tables with impersonation authentication.* | Ensure that the query acceleration policy contains a valid managed identity that has:
• Appropriate permissions on the Delta table
• The `AutomatedFlows` usage type in the cluster or database managed identity policy. | ::: moniker-end @@ -141,13 +138,7 @@ Use the following tables to understand and mitigate common unhealthy states. | Unhealthy reason | Example `NotHealthyReason` | Action | | --------------------------------------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| External table access is forbidden | `InaccessibleDeltaTable: Access to Delta table is forbidden` | Alter the external table connection string with a valid authentication method. | -| External table connection string doesn't point to a valid delta table | `DeltaTableNotFound: Delta table does not exist` | Alter the external table connection string to point to a valid delta table location. | -| Delta table column mapping mode has changed | `ColumnMappingModeChange: Column mapping mode has changed. Previous: 'None', New: 'Name'` | Recreate the external table with the new column mapping mode and re-enable the query acceleration policy. | -| Delta table column mappings have changed | `NewColumnMapping: New column mapping was introduced. Column 'Col1' is now mapped to 'Col2'` | Recreate the external table and re-enable the query acceleration policy so that column mappings are aligned with the delta table. | -| Delta table column type has changed | `ColumnTypeMismatch: Column 'Col1' type has changed. Previous delta type: 'long', New type: 'string'. Respective external table type: long` | Recreate (or alter) the external table so that its schema is aligned with the delta table column types, and then re-enable query acceleration. | -| Hot datetime column not found | `HotDateTimeColumn 'Col1' does not exist as a datetime column in the Delta table schema` | Alter the query acceleration policy to include a valid `HotDateTimeColumn`, or leave the property empty if not required. | -| Delta table has unsupported features for query acceleration | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. | +[!INCLUDE [query-acceleration-unhealthy-reasons-table](query-acceleration-unhealthy-reasons.md)] ::: moniker-end @@ -161,10 +152,10 @@ Use the following command and filter on a time frame that includes the relevant ```kusto .show queries | where StartedOn > ago(1h) -| extend ExternalDataStats = ['OverallQueryStats']['input_dataset_statistics']['external_data'] +| extend ExternalDataStats = OverallQueryStats.input_dataset_statistics.external_data ``` -If `ExternalDataStats.iterated_artifacts` or `ExternalDataStats.downloaded_items` are greater than `0`, it means data is being read from the remote delta table (non-accelerated path). +If `ExternalDataStats.iterated_artifacts` or `ExternalDataStats.downloaded_items` are greater than `0`, it means data was read from the remote delta table (non-accelerated path). ### Troubleshoot queries over non-accelerated-data @@ -183,11 +174,11 @@ Run the following command to view the hot caching properties and make sure the q | project Policy.Hot, Policy.HotWindows ``` -- Ensure your query's time filter is fully contained within the configured `Hot` period or the defined `HotWindows`. +Ensure your query's time filter is fully contained within the configured `Hot` period or the defined `HotWindows`. -If the query needs to access data outside the configured hot period or hot windows, alter the policy by: +If it's a one-time query, policy change is not recommended. However, if you anticipate running multiple queries over the same time range that lies outside the configured `Hot` period or defined `HotWindows` and require improved performance, alter the policy by: -- Increasing the hot period, or +- Increasing the hot period, and/or - Adding additional hot windows that match your query patterns. #### Data within the hot period isn't fully cached @@ -214,11 +205,11 @@ Data acceleration might take time, especially when: - A query acceleration policy has recently been enabled, or - The delta table has undergone an optimization operation such as `OPTIMIZE` that results in many deleted and recreated files. -Frequently running `OPTIMIZE` or `MERGE` operations on the source delta table that cause large-scale rewrites of data files can negatively affect acceleration performance because data files are repeatedly rewritten. +Frequently running `OPTIMIZE` or `MERGE` operations on the source delta table that cause large-scale rewrites of data files can negatively affect acceleration performance because data files are repeatedly rewritten, and need to be accelerated. ### Data files aren't eligible for acceleration -Parquet data files with a **compressed size greater than 1 GB** won't be cached. +Parquet data files larger than **1 GB** won't be cached. If your delta table includes many large files, consider adjusting your data generation or optimization strategy to produce smaller parquet files. If this requires recreating the Delta table, make sure you recreate the external table and reenable query acceleration policy. @@ -237,5 +228,5 @@ Run the following command to view the remaining capacity: - If `Remaining == 0` consistently and `CompletionPercentage` isn't increasing, consider: - - Increasing the `QueryAcceleration` capacity by altering the capacity policy. + - Increasing the `QueryAcceleration` capacity by [altering the capacity policy](alter-capacity-policy-command.md). - Scaling the cluster out or up to provide more resources. From 498b900fb890b7baab0381dee03391adf52e9022 Mon Sep 17 00:00:00 2001 From: "urishapira@microsoft.com" Date: Wed, 26 Nov 2025 23:28:00 +0200 Subject: [PATCH 07/16] adding includes --- .../management/query-acceleration-unhealthy-reasons.md | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 data-explorer/kusto/management/query-acceleration-unhealthy-reasons.md diff --git a/data-explorer/kusto/management/query-acceleration-unhealthy-reasons.md b/data-explorer/kusto/management/query-acceleration-unhealthy-reasons.md new file mode 100644 index 0000000000..b23090efc0 --- /dev/null +++ b/data-explorer/kusto/management/query-acceleration-unhealthy-reasons.md @@ -0,0 +1,8 @@ + +| External table access is forbidden | `InaccessibleDeltaTable: Access to Delta table is forbidden` | Verify the connection string of the external table, including the authentication method, and that the permissions on the underlying storage are correct. | +| External table connection string doesn't point to a valid delta table | `DeltaTableNotFound: Delta table does not exist` | Alter the external table’s connection string so it targets a valid Delta table location. Make sure the path points to the table’s root folder - not the _delta_log directory. | +| Delta table column mapping mode has changed | `ColumnMappingModeChange: Column mapping mode has changed. Previous: 'None', New: 'Name'` | Recreate the external table with the new column mapping mode and re-enable the query acceleration policy. | +| Delta table column mappings have changed | `NewColumnMapping: New column mapping was introduced. Column 'Col1' is now mapped to 'Col2'` | Recreate the external table and re-enable the query acceleration policy so that column mappings are aligned with the delta table. | +| Delta table column type has changed | `ColumnTypeMismatch: Column 'Col1' type has changed. Previous delta type: 'long', New type: 'string'. Respective external table type: long` | Recreate (or alter) the external table so that its schema is aligned with the delta table column types, and then re-enable query acceleration. | +| Hot datetime column not found | `HotDateTimeColumn 'Col1' does not exist as a datetime column in the Delta table schema` | Alter the query acceleration policy to include a valid `HotDateTimeColumn` (a column of type `datetime` in the delta table), or leave the property empty if not required. | +| Delta table has one of the following unsupported features for query acceleration
• Column mapping mode 'Name'
• AddFile transactions referencing files with absoulte path
• deletion vectors with absoulte path | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. | From 56752072d3593c7e018ba658d2d8e2920ecb3a7c Mon Sep 17 00:00:00 2001 From: "urishapira@microsoft.com" Date: Wed, 26 Nov 2025 23:37:43 +0200 Subject: [PATCH 08/16] changes --- .../kusto/management/query-acceleration-tsg.md | 16 ++++++++++++++-- .../query-acceleration-unhealthy-reasons.md | 8 -------- 2 files changed, 14 insertions(+), 10 deletions(-) delete mode 100644 data-explorer/kusto/management/query-acceleration-unhealthy-reasons.md diff --git a/data-explorer/kusto/management/query-acceleration-tsg.md b/data-explorer/kusto/management/query-acceleration-tsg.md index c576ca26e9..f973e99587 100644 --- a/data-explorer/kusto/management/query-acceleration-tsg.md +++ b/data-explorer/kusto/management/query-acceleration-tsg.md @@ -129,7 +129,13 @@ Use the following table to understand and mitigate common unhealthy states. | Unhealthy reason | Example `NotHealthyReason` | Action | | --------------------------------------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -[!INCLUDE [query-acceleration-unhealthy-reasons-table](query-acceleration-unhealthy-reasons.md)] +| External table access is forbidden | `InaccessibleDeltaTable: Access to Delta table is forbidden` | Verify the connection string of the external table, including the authentication method, and that the permissions on the underlying storage are correct. | +| External table connection string doesn't point to a valid delta table | `DeltaTableNotFound: Delta table does not exist` | Alter the external table’s connection string so it targets a valid Delta table location. Make sure the path points to the table’s root folder - not the _delta_log directory. | +| Delta table column mapping mode has changed | `ColumnMappingModeChange: Column mapping mode has changed. Previous: 'None', New: 'Name'` | Recreate the external table with the new column mapping mode and re-enable the query acceleration policy. | +| Delta table column mappings have changed | `NewColumnMapping: New column mapping was introduced. Column 'Col1' is now mapped to 'Col2'` | Recreate the external table and re-enable the query acceleration policy so that column mappings are aligned with the delta table. | +| Delta table column type has changed | `ColumnTypeMismatch: Column 'Col1' type has changed. Previous delta type: 'long', New type: 'string'. Respective external table type: long` | Recreate (or alter) the external table so that its schema is aligned with the delta table column types, and then re-enable query acceleration. | +| Hot datetime column not found | `HotDateTimeColumn 'Col1' does not exist as a datetime column in the Delta table schema` | Alter the query acceleration policy to include a valid `HotDateTimeColumn` (a column of type `datetime` in the delta table), or leave the property empty if not required. | +| Delta table has one of the following unsupported features for query acceleration
• Column mapping mode 'Name'
• AddFile transactions referencing files with absoulte path
• deletion vectors with absoulte path | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. | | Managed identity error | *Managed identity must be specified for external tables with impersonation authentication.* | Ensure that the query acceleration policy contains a valid managed identity that has:
• Appropriate permissions on the Delta table
• The `AutomatedFlows` usage type in the cluster or database managed identity policy. | ::: moniker-end @@ -138,7 +144,13 @@ Use the following table to understand and mitigate common unhealthy states. | Unhealthy reason | Example `NotHealthyReason` | Action | | --------------------------------------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -[!INCLUDE [query-acceleration-unhealthy-reasons-table](query-acceleration-unhealthy-reasons.md)] +| External table access is forbidden | `InaccessibleDeltaTable: Access to Delta table is forbidden` | Verify the connection string of the external table, including the authentication method, and that the permissions on the underlying storage are correct. | +| External table connection string doesn't point to a valid delta table | `DeltaTableNotFound: Delta table does not exist` | Alter the external table’s connection string so it targets a valid Delta table location. Make sure the path points to the table’s root folder - not the _delta_log directory. | +| Delta table column mapping mode has changed | `ColumnMappingModeChange: Column mapping mode has changed. Previous: 'None', New: 'Name'` | Recreate the external table with the new column mapping mode and re-enable the query acceleration policy. | +| Delta table column mappings have changed | `NewColumnMapping: New column mapping was introduced. Column 'Col1' is now mapped to 'Col2'` | Recreate the external table and re-enable the query acceleration policy so that column mappings are aligned with the delta table. | +| Delta table column type has changed | `ColumnTypeMismatch: Column 'Col1' type has changed. Previous delta type: 'long', New type: 'string'. Respective external table type: long` | Recreate (or alter) the external table so that its schema is aligned with the delta table column types, and then re-enable query acceleration. | +| Hot datetime column not found | `HotDateTimeColumn 'Col1' does not exist as a datetime column in the Delta table schema` | Alter the query acceleration policy to include a valid `HotDateTimeColumn` (a column of type `datetime` in the delta table), or leave the property empty if not required. | +| Delta table has one of the following unsupported features for query acceleration
• Column mapping mode 'Name'
• AddFile transactions referencing files with absoulte path
• deletion vectors with absoulte path | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. | ::: moniker-end diff --git a/data-explorer/kusto/management/query-acceleration-unhealthy-reasons.md b/data-explorer/kusto/management/query-acceleration-unhealthy-reasons.md deleted file mode 100644 index b23090efc0..0000000000 --- a/data-explorer/kusto/management/query-acceleration-unhealthy-reasons.md +++ /dev/null @@ -1,8 +0,0 @@ - -| External table access is forbidden | `InaccessibleDeltaTable: Access to Delta table is forbidden` | Verify the connection string of the external table, including the authentication method, and that the permissions on the underlying storage are correct. | -| External table connection string doesn't point to a valid delta table | `DeltaTableNotFound: Delta table does not exist` | Alter the external table’s connection string so it targets a valid Delta table location. Make sure the path points to the table’s root folder - not the _delta_log directory. | -| Delta table column mapping mode has changed | `ColumnMappingModeChange: Column mapping mode has changed. Previous: 'None', New: 'Name'` | Recreate the external table with the new column mapping mode and re-enable the query acceleration policy. | -| Delta table column mappings have changed | `NewColumnMapping: New column mapping was introduced. Column 'Col1' is now mapped to 'Col2'` | Recreate the external table and re-enable the query acceleration policy so that column mappings are aligned with the delta table. | -| Delta table column type has changed | `ColumnTypeMismatch: Column 'Col1' type has changed. Previous delta type: 'long', New type: 'string'. Respective external table type: long` | Recreate (or alter) the external table so that its schema is aligned with the delta table column types, and then re-enable query acceleration. | -| Hot datetime column not found | `HotDateTimeColumn 'Col1' does not exist as a datetime column in the Delta table schema` | Alter the query acceleration policy to include a valid `HotDateTimeColumn` (a column of type `datetime` in the delta table), or leave the property empty if not required. | -| Delta table has one of the following unsupported features for query acceleration
• Column mapping mode 'Name'
• AddFile transactions referencing files with absoulte path
• deletion vectors with absoulte path | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. | From e2b266c9f92154f9c0fc39bcc21800752495d2ce Mon Sep 17 00:00:00 2001 From: "urishapira@microsoft.com" Date: Wed, 26 Nov 2025 23:45:57 +0200 Subject: [PATCH 09/16] nit changes --- data-explorer/kusto/management/query-acceleration-tsg.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/data-explorer/kusto/management/query-acceleration-tsg.md b/data-explorer/kusto/management/query-acceleration-tsg.md index f973e99587..e35032510c 100644 --- a/data-explorer/kusto/management/query-acceleration-tsg.md +++ b/data-explorer/kusto/management/query-acceleration-tsg.md @@ -136,7 +136,7 @@ Use the following table to understand and mitigate common unhealthy states. | Delta table column type has changed | `ColumnTypeMismatch: Column 'Col1' type has changed. Previous delta type: 'long', New type: 'string'. Respective external table type: long` | Recreate (or alter) the external table so that its schema is aligned with the delta table column types, and then re-enable query acceleration. | | Hot datetime column not found | `HotDateTimeColumn 'Col1' does not exist as a datetime column in the Delta table schema` | Alter the query acceleration policy to include a valid `HotDateTimeColumn` (a column of type `datetime` in the delta table), or leave the property empty if not required. | | Delta table has one of the following unsupported features for query acceleration
• Column mapping mode 'Name'
• AddFile transactions referencing files with absoulte path
• deletion vectors with absoulte path | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. | -| Managed identity error | *Managed identity must be specified for external tables with impersonation authentication.* | Ensure that the query acceleration policy contains a valid managed identity that has:
• Appropriate permissions on the Delta table
• The `AutomatedFlows` usage type in the cluster or database managed identity policy. | +| Managed identity error | `Managed identity must be specified for external tables with impersonation authentication.` | Ensure that the query acceleration policy contains a valid managed identity that has:
• Appropriate permissions on the Delta table
• The `AutomatedFlows` usage type in the cluster or database managed identity policy. | ::: moniker-end @@ -168,6 +168,7 @@ Use the following command and filter on a time frame that includes the relevant ``` If `ExternalDataStats.iterated_artifacts` or `ExternalDataStats.downloaded_items` are greater than `0`, it means data was read from the remote delta table (non-accelerated path). +The following sections will help you troubleshoot this. ### Troubleshoot queries over non-accelerated-data @@ -215,6 +216,7 @@ Unaccelerated data (`CompletionPercentage < 100`) can stem from several issues. Data acceleration might take time, especially when: - A query acceleration policy has recently been enabled, or +- A significant amount of data was recently added to the delta table, or - The delta table has undergone an optimization operation such as `OPTIMIZE` that results in many deleted and recreated files. Frequently running `OPTIMIZE` or `MERGE` operations on the source delta table that cause large-scale rewrites of data files can negatively affect acceleration performance because data files are repeatedly rewritten, and need to be accelerated. From cf1a16d01de2401ba8a7343dede9e7c55b50f566 Mon Sep 17 00:00:00 2001 From: "urishapira@microsoft.com" Date: Fri, 28 Nov 2025 12:55:23 +0200 Subject: [PATCH 10/16] pr comments --- .../management/query-acceleration-tsg.md | 79 ++++++++----------- 1 file changed, 31 insertions(+), 48 deletions(-) diff --git a/data-explorer/kusto/management/query-acceleration-tsg.md b/data-explorer/kusto/management/query-acceleration-tsg.md index e35032510c..be45e393b0 100644 --- a/data-explorer/kusto/management/query-acceleration-tsg.md +++ b/data-explorer/kusto/management/query-acceleration-tsg.md @@ -14,16 +14,16 @@ To understand why things aren't working as expected, it's important to identify This article helps you troubleshoot scenarios where: -- Queries over accelerated external delta tables return **stale data**, or -- Queries over accelerated external delta tables are **slower than expected** +- Query over an accelerated external delta table returns **stale data**, or +- Query over an accelerated external delta table is **slower than expected** -## Prerequisites +# Prerequisites 1. **Ensure query acceleration is enabled on the external table** by running the following command: ```kusto .show external table policy query_acceleration - | project todynamic(Policy).IsEnabled + | project isnotnull(Policy) and todynamic(Policy).IsEnabled ``` If this command returns `false`, enable the query acceleration policy using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md). @@ -34,31 +34,10 @@ This article helps you troubleshoot scenarios where: If such operations have been executed on the delta table, first recreate the external table and re-enable the query acceleration policy. -## Troubleshooting flow - -Use the following logical flow to identify and mitigate query acceleration issues. - -- **Query acceleration issues** - - **[Query is returning old data](#query-is-returning-old-data)** - Result freshness issue: query results don't reflect the latest data from the underlying delta table. - - **Query is not running fast enough** - Performance issue: query is slower than expected, and acceleration doesn't appear to improve performance. - - **[Check if catalog is stale](#check-if-catalog-is-stale)** - Is the query acceleration catalog older than the configured `MaxAge` and therefore not used? - - **[Troubleshoot stale catalogs](#troubleshoot-stale-catalogs)** - Diagnose why the catalog isn't refreshing (for example, unhealthy state, frequent changes, or recent enablement). - - **[Query acceleration unhealthy state – understanding and mitigating](#query-acceleration-unhealthy-state--understanding-and-mitigating)** - Is query acceleration unhealthy due to configuration or schema issues? - - **[Check if query is over non-accelerated data](#check-if-query-is-over-non-accelerated-data)** - Is the query reading data directly from the remote delta table? - - **[Troubleshoot queries over non-accelerated-data](#troubleshoot-queries-over-non-accelerated-data)** - Align query filters with the hot period or hot windows and verify that data within those ranges is fully cached. - - **[Understanding and mitigating data acceleration issues](#understanding-and-mitigating-data-acceleration-issues)** - Investigate incomplete acceleration due to ongoing caching, large parquet files, or insufficient cluster capacity. - - **[Ensure query complies with KQL best practices](../query/best-practices.md)**
- Optimize the query as is instructed in the KQL best practices document - -## Query is returning old data + +# Query is returning stale data + +This is a data freshness issue: query results don't reflect the latest data from the underlying delta table. Query acceleration refreshes the accelerated data so that results are no older than the configured `MaxAge` value in the policy. By design, queries over accelerated external tables may return data that lags behind the latest delta table version by up to `MaxAge`. Set `MaxAge` to the maximum data staleness that is acceptable at query time. @@ -68,23 +47,32 @@ You can control the effective `MaxAge` in two ways: 1. Configure the `MaxAge` property in the query acceleration policy using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md). 2. Override `MaxAge` per query by using the [`external_table()` operator's](../query/external-table-function.md) `MaxAgeOverride` parameter. -## Check if catalog is stale +# Query is not running fast enough + +This is a performance issue: query is slower than expected, and acceleration doesn't appear to improve performance. + +There are a few reasons why this could happen: +1. The query acceleration catalog is unusable (out-of-date or never built) - see section [Check if catalog is unusable](#check-if-catalog-is-unusable) +2. The query scans non-accelerated data - see section [Check if query is over non-accelerated data](#check-if-query-is-over-non-accelerated-data) +3. The query does not comply with KQL best practices - see [KQL best practices](../query/best-practices.md) + +## Check if catalog is unusable -Query acceleration uses a local catalog for the external table containing a snapshot of the delta table metadata. If this catalog hasn't been updated within the configured `MaxAge` (see the query acceleration policy's `MaxAge` property), it's considered **stale** and won't be used at query time. In that case, queries fall back to reading the remote delta table directly, which can be significantly slower. +Query acceleration uses a local catalog for the external table containing a snapshot of the delta table metadata. If this catalog hasn't been updated within the configured `MaxAge` (see the query acceleration policy's `MaxAge` property), it's considered **unusable** and won't be used at query time. In that case, queries fall back to reading the remote delta table directly, which can be significantly slower. Fetch the current state of the catalog using the following command: ```kusto .show external table [ETName] details | extend MinimumUpdateTime = now() - totimespan(todynamic(QueryAccelerationPolicy).MaxAge) -| project IsCatalogStale = MinimumUpdateTime < todatetime(todynamic(QueryAccelerationState).LastUpdatedDateTime) +| project IsCatalogUnusable = MinimumUpdateTime < todatetime(todynamic(QueryAccelerationState).LastUpdatedDateTime) ``` -`IsCatalogStale == true` indicates the catalog is stale and query acceleration won't be used. +`IsCatalogUnusable == true` indicates the catalog is stale and query acceleration won't be used. -### Troubleshoot stale catalogs +### Troubleshoot unusable catalogs -To understand why a catalog is stale, first check whether the query acceleration state is healthy and resolve unhealthy reasons as needed. +To understand why a catalog is unusable, first check if the query acceleration state is healthy and resolve unhealthy reasons as needed. Run: @@ -94,8 +82,9 @@ Run: | project IsHealthy = state.IsHealthy, UnhealthyReason = state.NotHealthyReason ``` +- If the state is **healthy** but the catalog is still stale, it could be that the query acceleration policy was enabled recently. - If the state is **unhealthy**, refer to [Query acceleration unhealthy state – understanding and mitigating](#query-acceleration-unhealthy-state--understanding-and-mitigating). -- If the state is **healthy** but the catalog is still stale, consider the following cases. + #### Query acceleration policy was enabled recently @@ -108,13 +97,7 @@ When the query acceleration policy is enabled for the first time, building the i If `LastUpdatedDateTime` is empty, allow some time for the first update to complete. This usually takes several minutes. Subsequent updates are expected to be significantly faster. -#### The delta table is frequently changing - -A `MaxAge` value (default is five minutes) that is too low for a frequently changing delta table can result in a constantly stale catalog. For example, if the delta table undergoes frequent `OPTIMIZE` or `MERGE` operations that rewrite a large portion of the underlying parquet files (such as aggressive compaction or large upserts), the catalog might lag behind. - -If the queries can tolerate slightly older data, consider increasing the `MaxAge` value using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md). - -## Query acceleration unhealthy state – understanding and mitigating +#### Query acceleration unhealthy state – understanding and mitigating When an external table's query acceleration is unhealthy, you can retrieve the unhealthy reason using the following command: @@ -168,7 +151,7 @@ Use the following command and filter on a time frame that includes the relevant ``` If `ExternalDataStats.iterated_artifacts` or `ExternalDataStats.downloaded_items` are greater than `0`, it means data was read from the remote delta table (non-accelerated path). -The following sections will help you troubleshoot this. +The following section will help you understand why. ### Troubleshoot queries over non-accelerated-data @@ -207,11 +190,11 @@ Use the following command to check the acceleration progress: - If `CompletionPercentage < 100`, allow more time for data to be accelerated. - If `CompletionPercentage` doesn't increase over time, follow the guidance in [Understanding and mitigating data acceleration issues](#understanding-and-mitigating-data-acceleration-issues). -## Understanding and mitigating data acceleration issues +### Understanding and mitigating data acceleration issues Unaccelerated data (`CompletionPercentage < 100`) can stem from several issues. -### Data is currently being accelerated +#### Data is currently being accelerated Data acceleration might take time, especially when: @@ -221,14 +204,14 @@ Data acceleration might take time, especially when: Frequently running `OPTIMIZE` or `MERGE` operations on the source delta table that cause large-scale rewrites of data files can negatively affect acceleration performance because data files are repeatedly rewritten, and need to be accelerated. -### Data files aren't eligible for acceleration +#### Data files aren't eligible for acceleration Parquet data files larger than **1 GB** won't be cached. If your delta table includes many large files, consider adjusting your data generation or optimization strategy to produce smaller parquet files. If this requires recreating the Delta table, make sure you recreate the external table and reenable query acceleration policy. -### Insufficient cluster capacity or resources +#### Insufficient cluster capacity or resources Query acceleration operations are restricted by the cluster's available query acceleration capacity. From 49150a668be69fc2785a8151db48608e23adb70b Mon Sep 17 00:00:00 2001 From: "urishapira@microsoft.com" Date: Fri, 28 Nov 2025 13:14:23 +0200 Subject: [PATCH 11/16] changes --- .../management/query-acceleration-tsg.md | 40 ++++++++++++------- 1 file changed, 25 insertions(+), 15 deletions(-) diff --git a/data-explorer/kusto/management/query-acceleration-tsg.md b/data-explorer/kusto/management/query-acceleration-tsg.md index be45e393b0..d35802b821 100644 --- a/data-explorer/kusto/management/query-acceleration-tsg.md +++ b/data-explorer/kusto/management/query-acceleration-tsg.md @@ -2,7 +2,7 @@ > [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] -The [query acceleration policy](query-acceleration-policy.md) enables accelerating queries over external delta tables by caching delta table metadata and data files. The policy defines which data ranges (number of days back and hot windows) are accelerated so that queries over those ranges can run significantly faster. +The [query acceleration policy](query-acceleration-policy.md) enables accelerating queries over external delta tables by caching delta table metadata and data files. The policy defines which data ranges (number of days back and hot windows) are accelerated so that queries over those ranges can run faster. The query acceleration feature consists of the following components: @@ -47,18 +47,18 @@ You can control the effective `MaxAge` in two ways: 1. Configure the `MaxAge` property in the query acceleration policy using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md). 2. Override `MaxAge` per query by using the [`external_table()` operator's](../query/external-table-function.md) `MaxAgeOverride` parameter. -# Query is not running fast enough +# Query isn't running fast enough This is a performance issue: query is slower than expected, and acceleration doesn't appear to improve performance. There are a few reasons why this could happen: 1. The query acceleration catalog is unusable (out-of-date or never built) - see section [Check if catalog is unusable](#check-if-catalog-is-unusable) -2. The query scans non-accelerated data - see section [Check if query is over non-accelerated data](#check-if-query-is-over-non-accelerated-data) -3. The query does not comply with KQL best practices - see [KQL best practices](../query/best-practices.md) +2. The query scans nonaccelerated data - see section [Check if query is over nonaccelerated data](#check-if-query-is-over-non-accelerated-data) +3. The query doesn't comply with KQL best practices - see [KQL best practices](../query/best-practices.md) ## Check if catalog is unusable -Query acceleration uses a local catalog for the external table containing a snapshot of the delta table metadata. If this catalog hasn't been updated within the configured `MaxAge` (see the query acceleration policy's `MaxAge` property), it's considered **unusable** and won't be used at query time. In that case, queries fall back to reading the remote delta table directly, which can be significantly slower. +Query acceleration uses a local catalog for the external table containing a snapshot of the delta table metadata. If this catalog hasn't been updated within the configured `MaxAge` (see the query acceleration policy's `MaxAge` property), it's considered **unusable** and isn't used at query time. In that case, queries fall back to reading the remote delta table directly, which can be significantly slower. Fetch the current state of the catalog using the following command: @@ -88,7 +88,7 @@ Run: #### Query acceleration policy was enabled recently -When the query acceleration policy is enabled for the first time, building the initial catalog needs to complete before it can be used in queries. During this period, the `LastUpdatedDateTime` value will be empty: +When the query acceleration policy is enabled for the first time, building the initial catalog needs to complete before it can be used in queries. During this period, the `LastUpdatedDateTime` value is empty: ```kusto .show external table [ETName] details @@ -108,17 +108,27 @@ When an external table's query acceleration is unhealthy, you can retrieve the u Use the following table to understand and mitigate common unhealthy states. +> [!NOTE] +> +> To re-enable an external table's query acceleration policy, run the following commands: +>```kusto +> .alter-merge external table [ETName] policy query_acceleration @'{"IsEnabled":false}' +> +> .alter-merge external table [ETName] policy query_acceleration @'{"IsEnabled":true}' +>``` + + ::: moniker range="azure-data-explorer" | Unhealthy reason | Example `NotHealthyReason` | Action | | --------------------------------------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | External table access is forbidden | `InaccessibleDeltaTable: Access to Delta table is forbidden` | Verify the connection string of the external table, including the authentication method, and that the permissions on the underlying storage are correct. | -| External table connection string doesn't point to a valid delta table | `DeltaTableNotFound: Delta table does not exist` | Alter the external table’s connection string so it targets a valid Delta table location. Make sure the path points to the table’s root folder - not the _delta_log directory. | -| Delta table column mapping mode has changed | `ColumnMappingModeChange: Column mapping mode has changed. Previous: 'None', New: 'Name'` | Recreate the external table with the new column mapping mode and re-enable the query acceleration policy. | +| External table connection string doesn't point to a valid delta table | `DeltaTableNotFound: Delta table does not exist` | Alter the external table’s connection string so it targets a valid Delta table location. Make sure the path points to the table’s root folder - not the _delta_log directory. | +| Delta table column mapping mode has changed | `ColumnMappingModeChange: Column mapping mode has changed. Previous: 'None', New: 'Name'` | Recreate the external table and re-enable the query acceleration policy. | | Delta table column mappings have changed | `NewColumnMapping: New column mapping was introduced. Column 'Col1' is now mapped to 'Col2'` | Recreate the external table and re-enable the query acceleration policy so that column mappings are aligned with the delta table. | | Delta table column type has changed | `ColumnTypeMismatch: Column 'Col1' type has changed. Previous delta type: 'long', New type: 'string'. Respective external table type: long` | Recreate (or alter) the external table so that its schema is aligned with the delta table column types, and then re-enable query acceleration. | | Hot datetime column not found | `HotDateTimeColumn 'Col1' does not exist as a datetime column in the Delta table schema` | Alter the query acceleration policy to include a valid `HotDateTimeColumn` (a column of type `datetime` in the delta table), or leave the property empty if not required. | -| Delta table has one of the following unsupported features for query acceleration
• Column mapping mode 'Name'
• AddFile transactions referencing files with absoulte path
• deletion vectors with absoulte path | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. | +| Delta table has one of the following unsupported features for query acceleration
• Column mapping mode 'Name'
• AddFile transactions referencing files with absolute path
• deletion vectors with absolute path | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. | | Managed identity error | `Managed identity must be specified for external tables with impersonation authentication.` | Ensure that the query acceleration policy contains a valid managed identity that has:
• Appropriate permissions on the Delta table
• The `AutomatedFlows` usage type in the cluster or database managed identity policy. | ::: moniker-end @@ -133,12 +143,12 @@ Use the following table to understand and mitigate common unhealthy states. | Delta table column mappings have changed | `NewColumnMapping: New column mapping was introduced. Column 'Col1' is now mapped to 'Col2'` | Recreate the external table and re-enable the query acceleration policy so that column mappings are aligned with the delta table. | | Delta table column type has changed | `ColumnTypeMismatch: Column 'Col1' type has changed. Previous delta type: 'long', New type: 'string'. Respective external table type: long` | Recreate (or alter) the external table so that its schema is aligned with the delta table column types, and then re-enable query acceleration. | | Hot datetime column not found | `HotDateTimeColumn 'Col1' does not exist as a datetime column in the Delta table schema` | Alter the query acceleration policy to include a valid `HotDateTimeColumn` (a column of type `datetime` in the delta table), or leave the property empty if not required. | -| Delta table has one of the following unsupported features for query acceleration
• Column mapping mode 'Name'
• AddFile transactions referencing files with absoulte path
• deletion vectors with absoulte path | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. | +| Delta table has one of the following unsupported features for query acceleration
• Column mapping mode 'Name'
• AddFile transactions referencing files with absolute path
• deletion vectors with absolute path | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. | ::: moniker-end -## Check if query is over non-accelerated data +## Check if query is over nonaccelerated data To fully benefit from query acceleration, queries must be executed over **accelerated data**. Non-accelerated data is read directly from the remote delta table, which may result in significant latency. @@ -151,9 +161,9 @@ Use the following command and filter on a time frame that includes the relevant ``` If `ExternalDataStats.iterated_artifacts` or `ExternalDataStats.downloaded_items` are greater than `0`, it means data was read from the remote delta table (non-accelerated path). -The following section will help you understand why. +The following section helps you understand why. -### Troubleshoot queries over non-accelerated-data +### Troubleshoot queries over nonaccelerated data There are two main reasons why a query might read non-accelerated data: @@ -172,7 +182,7 @@ Run the following command to view the hot caching properties and make sure the q Ensure your query's time filter is fully contained within the configured `Hot` period or the defined `HotWindows`. -If it's a one-time query, policy change is not recommended. However, if you anticipate running multiple queries over the same time range that lies outside the configured `Hot` period or defined `HotWindows` and require improved performance, alter the policy by: +If it's a one-time query, policy change isn't recommended. However, if you anticipate running multiple queries over the same time range that lies outside the configured `Hot` period or defined `HotWindows` and require improved performance, alter the policy by: - Increasing the hot period, and/or - Adding additional hot windows that match your query patterns. @@ -209,7 +219,7 @@ Frequently running `OPTIMIZE` or `MERGE` operations on the source delta table th Parquet data files larger than **1 GB** won't be cached. If your delta table includes many large files, consider adjusting your data generation or optimization strategy to produce smaller parquet files. -If this requires recreating the Delta table, make sure you recreate the external table and reenable query acceleration policy. +If this requires recreating the Delta table, make sure you recreate the external table and re-enable query acceleration policy. #### Insufficient cluster capacity or resources From e70f496dd06ffec22f8b6ab04b6bd218605f8c03 Mon Sep 17 00:00:00 2001 From: "urishapira@microsoft.com" Date: Sun, 30 Nov 2025 11:17:57 +0200 Subject: [PATCH 12/16] Solutions git --- data-explorer/kusto/management/query-acceleration-tsg.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/data-explorer/kusto/management/query-acceleration-tsg.md b/data-explorer/kusto/management/query-acceleration-tsg.md index d35802b821..46b1f17938 100644 --- a/data-explorer/kusto/management/query-acceleration-tsg.md +++ b/data-explorer/kusto/management/query-acceleration-tsg.md @@ -139,9 +139,9 @@ Use the following table to understand and mitigate common unhealthy states. | --------------------------------------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | External table access is forbidden | `InaccessibleDeltaTable: Access to Delta table is forbidden` | Verify the connection string of the external table, including the authentication method, and that the permissions on the underlying storage are correct. | | External table connection string doesn't point to a valid delta table | `DeltaTableNotFound: Delta table does not exist` | Alter the external table’s connection string so it targets a valid Delta table location. Make sure the path points to the table’s root folder - not the _delta_log directory. | -| Delta table column mapping mode has changed | `ColumnMappingModeChange: Column mapping mode has changed. Previous: 'None', New: 'Name'` | Recreate the external table with the new column mapping mode and re-enable the query acceleration policy. | +| Delta table column mapping mode has changed | `ColumnMappingModeChange: Column mapping mode has changed. Previous: 'None', New: 'Name'` | Recreate the external table and re-enable the query acceleration policy. | | Delta table column mappings have changed | `NewColumnMapping: New column mapping was introduced. Column 'Col1' is now mapped to 'Col2'` | Recreate the external table and re-enable the query acceleration policy so that column mappings are aligned with the delta table. | -| Delta table column type has changed | `ColumnTypeMismatch: Column 'Col1' type has changed. Previous delta type: 'long', New type: 'string'. Respective external table type: long` | Recreate (or alter) the external table so that its schema is aligned with the delta table column types, and then re-enable query acceleration. | +| Delta table column type has changed | `ColumnTypeMismatch: Column 'Col1' type has changed. Previous delta type: 'long', New type: 'string'. Respective external table type: long` | Recreate the external table so that its schema is aligned with the delta table column types, and then re-enable query acceleration. | | Hot datetime column not found | `HotDateTimeColumn 'Col1' does not exist as a datetime column in the Delta table schema` | Alter the query acceleration policy to include a valid `HotDateTimeColumn` (a column of type `datetime` in the delta table), or leave the property empty if not required. | | Delta table has one of the following unsupported features for query acceleration
• Column mapping mode 'Name'
• AddFile transactions referencing files with absolute path
• deletion vectors with absolute path | `Unsupported feature: Column mapping of type 'Id'` | Recreate the delta table with a supported configuration (for example, using `Name` column mapping type), and then re-enable query acceleration. | From adf44eee4b47f92fdbd7978c3ab7583c3d0f5860 Mon Sep 17 00:00:00 2001 From: urishapira <89069415+urishapira@users.noreply.github.com> Date: Mon, 1 Dec 2025 02:34:19 +0200 Subject: [PATCH 13/16] Apply suggestions from code review Co-authored-by: Slavik N <34075198+slavikn84@users.noreply.github.com> --- .../kusto/management/query-acceleration-tsg.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/data-explorer/kusto/management/query-acceleration-tsg.md b/data-explorer/kusto/management/query-acceleration-tsg.md index 46b1f17938..d6a52308ae 100644 --- a/data-explorer/kusto/management/query-acceleration-tsg.md +++ b/data-explorer/kusto/management/query-acceleration-tsg.md @@ -2,7 +2,7 @@ > [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] -The [query acceleration policy](query-acceleration-policy.md) enables accelerating queries over external delta tables by caching delta table metadata and data files. The policy defines which data ranges (number of days back and hot windows) are accelerated so that queries over those ranges can run faster. +The [query acceleration policy](query-acceleration-policy.md) enables accelerating queries over external delta tables by caching delta table metadata and data files. The policy defines which date ranges (number of days back and hot windows) are accelerated so that queries over those ranges can run faster. The query acceleration feature consists of the following components: @@ -32,14 +32,14 @@ This article helps you troubleshoot scenarios where: The query acceleration feature assumes a delta table that complies with the Delta protocol. Manual operations executed directly on the delta table (for example, editing transaction logs or parquet files) aren't supported and may result in unexpected behavior. - If such operations have been executed on the delta table, first recreate the external table and re-enable the query acceleration policy. + If such operations have been executed on the delta table, recreate the external table and re-enable the query acceleration policy. # Query is returning stale data This is a data freshness issue: query results don't reflect the latest data from the underlying delta table. -Query acceleration refreshes the accelerated data so that results are no older than the configured `MaxAge` value in the policy. +Query acceleration refreshes the accelerated data periodically, so that results are no older than the configured `MaxAge` value in the policy. By design, queries over accelerated external tables may return data that lags behind the latest delta table version by up to `MaxAge`. Set `MaxAge` to the maximum data staleness that is acceptable at query time. You can control the effective `MaxAge` in two ways: @@ -65,7 +65,7 @@ Fetch the current state of the catalog using the following command: ```kusto .show external table [ETName] details | extend MinimumUpdateTime = now() - totimespan(todynamic(QueryAccelerationPolicy).MaxAge) -| project IsCatalogUnusable = MinimumUpdateTime < todatetime(todynamic(QueryAccelerationState).LastUpdatedDateTime) +| project IsCatalogUnusable = MinimumUpdateTime > todatetime(todynamic(QueryAccelerationState).LastUpdatedDateTime) ``` `IsCatalogUnusable == true` indicates the catalog is stale and query acceleration won't be used. @@ -95,7 +95,7 @@ When the query acceleration policy is enabled for the first time, building the i | project todynamic(QueryAccelerationState).LastUpdatedDateTime ``` -If `LastUpdatedDateTime` is empty, allow some time for the first update to complete. This usually takes several minutes. Subsequent updates are expected to be significantly faster. +If `LastUpdatedDateTime` is empty, allow some time for the first update to complete. This usually takes up to several minutes. Subsequent updates are expected to be significantly faster. #### Query acceleration unhealthy state – understanding and mitigating @@ -112,7 +112,9 @@ Use the following table to understand and mitigate common unhealthy states. > > To re-enable an external table's query acceleration policy, run the following commands: >```kusto +> .execute database script <| > .alter-merge external table [ETName] policy query_acceleration @'{"IsEnabled":false}' +> .alter-merge external table [ETName] policy query_acceleration @'{"IsEnabled":true}' > > .alter-merge external table [ETName] policy query_acceleration @'{"IsEnabled":true}' >``` @@ -167,8 +169,8 @@ The following section helps you understand why. There are two main reasons why a query might read non-accelerated data: -1. **The query-time filter isn't fully within the query acceleration hot period or hot windows.** -2. **The data within the policy hot period isn't fully cached.** +1. The query-time filter isn't fully within the query acceleration hot period or hot windows. +2. The data within the policy hot period isn't fully cached. #### Query filter isn't fully within the hot period or hot windows From cf861c38f3fb68ea11bab778fe516f9da8afdffb Mon Sep 17 00:00:00 2001 From: "urishapira@microsoft.com" Date: Mon, 1 Dec 2025 02:40:41 +0200 Subject: [PATCH 14/16] pr comments --- .../management/query-acceleration-tsg.md | 20 ++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/data-explorer/kusto/management/query-acceleration-tsg.md b/data-explorer/kusto/management/query-acceleration-tsg.md index d6a52308ae..1ccd46df6d 100644 --- a/data-explorer/kusto/management/query-acceleration-tsg.md +++ b/data-explorer/kusto/management/query-acceleration-tsg.md @@ -1,3 +1,11 @@ +--- +title: query acceleration troubleshooting guide +description: Learn how to troubleshoot query acceleration issues +ms.reviewer: urishapira +ms.topic: reference +ms.date: 12/01/2025 +--- + # Troubleshoot query acceleration over external delta tables > [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] @@ -17,7 +25,7 @@ This article helps you troubleshoot scenarios where: - Query over an accelerated external delta table returns **stale data**, or - Query over an accelerated external delta table is **slower than expected** -# Prerequisites +## Prerequisites 1. **Ensure query acceleration is enabled on the external table** by running the following command: @@ -35,7 +43,7 @@ This article helps you troubleshoot scenarios where: If such operations have been executed on the delta table, recreate the external table and re-enable the query acceleration policy. -# Query is returning stale data +## Query is returning stale data This is a data freshness issue: query results don't reflect the latest data from the underlying delta table. @@ -47,13 +55,13 @@ You can control the effective `MaxAge` in two ways: 1. Configure the `MaxAge` property in the query acceleration policy using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md). 2. Override `MaxAge` per query by using the [`external_table()` operator's](../query/external-table-function.md) `MaxAgeOverride` parameter. -# Query isn't running fast enough +## Query isn't running fast enough This is a performance issue: query is slower than expected, and acceleration doesn't appear to improve performance. There are a few reasons why this could happen: 1. The query acceleration catalog is unusable (out-of-date or never built) - see section [Check if catalog is unusable](#check-if-catalog-is-unusable) -2. The query scans nonaccelerated data - see section [Check if query is over nonaccelerated data](#check-if-query-is-over-non-accelerated-data) +2. The query scans nonaccelerated data - see section [Check if query is over nonaccelerated data](#check-if-query-is-over-nonaccelerated-data) 3. The query doesn't comply with KQL best practices - see [KQL best practices](../query/best-practices.md) ## Check if catalog is unusable @@ -82,7 +90,7 @@ Run: | project IsHealthy = state.IsHealthy, UnhealthyReason = state.NotHealthyReason ``` -- If the state is **healthy** but the catalog is still stale, it could be that the query acceleration policy was enabled recently. +- If the state is **healthy** but the catalog is still stale, it could be that the query acceleration policy was enabled recently. See [Query acceleration policy was enabled recently](#query-acceleration-policy-was-enabled-recently) - If the state is **unhealthy**, refer to [Query acceleration unhealthy state – understanding and mitigating](#query-acceleration-unhealthy-state--understanding-and-mitigating). @@ -115,8 +123,6 @@ Use the following table to understand and mitigate common unhealthy states. > .execute database script <| > .alter-merge external table [ETName] policy query_acceleration @'{"IsEnabled":false}' > .alter-merge external table [ETName] policy query_acceleration @'{"IsEnabled":true}' -> -> .alter-merge external table [ETName] policy query_acceleration @'{"IsEnabled":true}' >``` From e84144ac1058d6c988d2a791b4cfab79bdd30d73 Mon Sep 17 00:00:00 2001 From: "urishapira@microsoft.com" Date: Mon, 1 Dec 2025 13:07:56 +0200 Subject: [PATCH 15/16] add capacity note --- data-explorer/kusto/management/query-acceleration-tsg.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/data-explorer/kusto/management/query-acceleration-tsg.md b/data-explorer/kusto/management/query-acceleration-tsg.md index 1ccd46df6d..209f927289 100644 --- a/data-explorer/kusto/management/query-acceleration-tsg.md +++ b/data-explorer/kusto/management/query-acceleration-tsg.md @@ -243,5 +243,6 @@ Run the following command to view the remaining capacity: - If `Remaining == 0` consistently and `CompletionPercentage` isn't increasing, consider: - - Increasing the `QueryAcceleration` capacity by [altering the capacity policy](alter-capacity-policy-command.md). - Scaling the cluster out or up to provide more resources. + - Increasing the `QueryAcceleration` capacity by [altering the capacity policy](alter-capacity-policy-command.md). NOTE: altering the capacity policy may have adverse effects on other operations. Alter the policy as a last resort at your own discretion. + From 45055ab944005fb337d259d5c1d677f89da7c248 Mon Sep 17 00:00:00 2001 From: Dennis Rea Date: Mon, 1 Dec 2025 10:06:28 -0800 Subject: [PATCH 16/16] Bullet nonsequential list items --- .../kusto/management/query-acceleration-tsg.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/data-explorer/kusto/management/query-acceleration-tsg.md b/data-explorer/kusto/management/query-acceleration-tsg.md index 209f927289..8a727a3225 100644 --- a/data-explorer/kusto/management/query-acceleration-tsg.md +++ b/data-explorer/kusto/management/query-acceleration-tsg.md @@ -52,17 +52,18 @@ By design, queries over accelerated external tables may return data that lags be You can control the effective `MaxAge` in two ways: -1. Configure the `MaxAge` property in the query acceleration policy using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md). -2. Override `MaxAge` per query by using the [`external_table()` operator's](../query/external-table-function.md) `MaxAgeOverride` parameter. +- Configure the `MaxAge` property in the query acceleration policy using the [`.alter query acceleration policy` command](alter-query-acceleration-policy-command.md). +- Override `MaxAge` per query by using the [`external_table()` operator's](../query/external-table-function.md) `MaxAgeOverride` parameter. ## Query isn't running fast enough This is a performance issue: query is slower than expected, and acceleration doesn't appear to improve performance. There are a few reasons why this could happen: -1. The query acceleration catalog is unusable (out-of-date or never built) - see section [Check if catalog is unusable](#check-if-catalog-is-unusable) -2. The query scans nonaccelerated data - see section [Check if query is over nonaccelerated data](#check-if-query-is-over-nonaccelerated-data) -3. The query doesn't comply with KQL best practices - see [KQL best practices](../query/best-practices.md) + +- The query acceleration catalog is unusable (out-of-date or never built) - see section [Check if catalog is unusable](#check-if-catalog-is-unusable) +- The query scans nonaccelerated data - see section [Check if query is over nonaccelerated data](#check-if-query-is-over-nonaccelerated-data) +- The query doesn't comply with KQL best practices - see [KQL best practices](../query/best-practices.md) ## Check if catalog is unusable @@ -175,8 +176,8 @@ The following section helps you understand why. There are two main reasons why a query might read non-accelerated data: -1. The query-time filter isn't fully within the query acceleration hot period or hot windows. -2. The data within the policy hot period isn't fully cached. +- The query-time filter isn't fully within the query acceleration hot period or hot windows. +- The data within the policy hot period isn't fully cached. #### Query filter isn't fully within the hot period or hot windows