Main view hits 500k KQL limit for a 30 day period, thus not showing data anymore

[rahalan]

🐛 Problem

When viewing a month in FinOps hub, I get the error message:

"Underlying Error: QueryUserError
More details: Expression.Error: Query execution has exceeded the allowed limits (80DA0003): The results of this query exceed the set limit of 500000 records, so not all records were returned (E_QUERY_RESULT_SET_TOO_LARGE, 0x80DA0003). See https://aka.ms/kustoquerylimits for more information and possible solutions..
[0]Kusto.Data.Exceptions.KustoServicePartialQueryFailureLimitsExceededException: Query execution has exceeded the allowed limits (80DA0003): The results of this query exceed the set limit of 500000 records, so not all records were returned (E_QUERY_RESULT_SET_TOO_LARGE, 0x80DA0003). See https://aka.ms/kustoquerylimits for more information and possible solutions.."

👣 Repro steps

1. Go to Resources
2. Select time period for 30 days
3. See error: both visuals are shown as error

🤔 Expected

To see the chart and entries with their costs

📷 Screenshots

🔧 Environment

• FinOps hub version: (e.g., 0.10, 0.11, 12.0, 13.0)
• Billing account type: EA (Enterprise Agreement)
• Power BI report type: ADX (Azure Data Explorer)
• Cost Management export: FOCUS 1.0 r2

ℹ️ Additional context

It worked in the past, but since this month it looks like to be too much data

🙋‍♀️ Ask for the community

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

[microsoft-github-policy-service]

Thank you for submitting this issue! We try to respond within 3 business days. In the meantime:

• 📖 Check the troubleshooting guide for common issues
• 💬 Ask questions in Discussions
• 📅 Join our biweekly office hours for live help from the team

[RolandKrummenacher]

Thanks for the report and screenshot, @rahalan — that's very helpful.

From the image, the KPI cards (Effective cost / Total savings / ESR) render fine while the two per-resource detail visuals on the Resources page error out. That's the signature of Kusto's default result-row limit: in the ADX-based Power BI report (the variant connected to Azure Data Explorer via DirectQuery), the Costs table is a DirectQuery partition, and for a full month on a large EA tenant the folded Kusto query returns more rows than Kusto's default cap — so it returns E_QUERY_RESULT_SET_TOO_LARGE (0x80DA0003). The aggregate cards fold to tiny result sets, which is why only the detail visuals fail. (The page's "top 1000 resources" note is a visual-level TopN — Kusto still scans/aggregates the full set to compute it, so it doesn't dodge the limit.) This also matches "it worked in the past" — the data for that window has simply grown past the cap.

One data point would help us dig in: roughly how many rows back that page for the month? e.g. Costs_v1_2 | where ChargePeriodStart >= startofmonth(now()) | count against the cluster.

Workaround in the meantime: reduce the Number of Months / set granularity to Monthly, and narrow page filters (subscription / resource group) so the Resources page query stays under the limit.

[RolandKrummenacher]

🔍 Root-cause analysis (engineering)

Surface: ADX-based Power BI report (src/power-bi/kql/), Costs table.

Root cause: The Costs partition is DirectQuery (Costs.tmdl:1867) querying server-side Costs_v1_2() (:1889) via AzureDataExplorer.Contents(…, [MaxRows=null, MaxSize=null, NoTruncate=null, AdditionalSetStatements=null]) (:1971). Nothing in the repo raises Kusto's result-row truncation (repo-wide search: no notruncation/truncationmaxrecords enabled; Costs_v1_2() doesn't lift it), so the engine default applies and row-grain Resources-page visuals exceeding it return 0x80DA0003. Note: the "500k" figure is Kusto's documented default, not a value set in the toolkit source.

Scope / impact: ADX-based Power BI report only (storage-based unaffected). Large tenants whose per-period resource line items exceed the cap; primarily the row-grain Resources page. Worsens as data grows.

Fix options:

1. Push the page's TopN into the Kusto query so only ~N rows return — medium effort, aligns with the existing "top 1000" note, fixes the failing visuals cleanly.
2. Pre-aggregate the Resources page server-side (summarize by resource) so line items aren't materialized — higher effort (report change), best long-term scalability.
3. Raise the connector limit via AdditionalSetStatements (set truncationmaxrecords / notruncation) — low effort, but shifts the ceiling and strains Power BI / gateway memory; partial.
4. Document the workaround only — lowest effort, no fix.

Recommendation: Fix. Option 1 primary; option 4 as interim guidance; option 2 longer-term. Avoid option 3 alone — it moves the limit rather than removing it.