docs: run FinOps hubs on your own hardware#2187
Open
MSBrett wants to merge 64 commits into
Open
Conversation
Local install of FinOps Hubs: runs the toolkit's ADX KQL against a local
Kusto Emulator (Kustainer) with the real two-database topology
(Ingestion + Hub), so dashboards / Power BI / the query catalog run
unchanged against a local endpoint.
T-5000.1 — load-ftk-kql.ps1 creates the Ingestion + Hub databases and loads
them by combining the toolkit's own finops-hub/.build.config fabric-setup
bundles (real v1_0 included; database('Ingestion'). cross-DB references
preserved; zero object renames — strict name parity). No hand-coded load
order, no strip hack, no v1_0 stubs.
T-5000.2 — ingest.ps1 -> Ingestion; ftk.ps1 -> Hub; run-parity-checks.ps1 is
DB-aware (raw/final checks on Ingestion, Hub-view checks 9/10 on Hub) and
adds a name-parity check. Verified end-to-end on R&D data: Costs and Prices
ingested at exact parity (BilledCost preserved to $0.00; Prices raw==final
12,735,587), Hub cross-DB queries resolve, parity suite 10 pass / 0 fail /
2 manual.
Prototype docs/notes are intentionally not migrated here yet — they are
authored cleanly (sanitized, two-DB-accurate) in the docs task (T-5000.7).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Red-team caught that `make kql` defaulted to Ingestion, so Hub view-function queries (e.g. Costs_v1_2) failed with 400 unless DB=Hub was passed. The DB var was shared with the parity recipe, which needs Ingestion as its base DB. Split into KQL_DB (?= Hub, used by `make kql`) and PARITY_DB (?= Ingestion, passed to run-parity-checks.ps1). ingest-status now passes KQL_DB=Ingestion explicitly since it inspects Ingest_Manifest in Ingestion. Header/help/topology comments updated. ingest unaffected (ingest.ps1 defaults to Ingestion). Verified: `make kql QUERY='Costs_v1_2 | count'` -> 1350561 (Hub) with no override; KQL_DB=Ingestion override works; make parity still 10/0/2 + name parity PASS; make ingest-status reads Ingest_Manifest on Ingestion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…00.4)
Document the data-in path: Cost Management export configuration via the
toolkit's New-/Get-/Start-FinOpsCostExport cmdlets (Azure-only; ftklocal
consumes the produced FOCUS parquet), and the local staging contract that
ingest.ps1 consumes — export/{scope}/{type}/{period}/{run-uuid}/ with
manifest.json, the ms--focus-cost / ms--pricesheet dataset tokens, latest-run
selection, SHA-256 idempotency, and supersede semantics (correct/replace data
by staging a NEW run-uuid; same run-uuid + changed file double-ingests).
Maps the msexports/ (no hub) and ingestion/ (hub deployed) source modes;
auto-download bridge deferred (T-5000.9). Contract cross-checked against
ingest.ps1.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…on (T-5000.10) Add a sanitized settings.example.json (HubInstance schema: scopes + retention) and scripts/settings.ps1 (init/show/get) for local-hub fidelity; gitignore the real settings.json. Wire load-ftk-kql.ps1 to derive the raw soft-delete window ($$rawRetentionInDays$$) from settings.json retention.raw.days with a 3650 fallback and a guard (0/negative/non-numeric -> fallback) plus a warning when a positive value is surprisingly short (<90 days) for local analysis. Document the config-container decision: Mode A (ingestion container) needs no config; Mode B (msexports) would need the msexports->ingestion conversion + the in-repo Exports/schemas (deferred to the bridge, T-5000.9). The ADX/KQL layer consumes no config schemas; load-ftk-kql.ps1 reads settings.json only for retention. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Localize the FinOps hub ADX dashboard for the local Kusto emulator: point the single data source at http://localhost:8082 / database Hub (135 tiles, 87 queries unchanged from upstream). Add notes/dashboard.md documenting the proven ftk.ps1 CLI path, the ADX web UI as best-effort with the HTTPS->HTTP mixed-content caveat, and an ngrok HTTPS-tunnel workaround. Power BI dropped (local Mac, no AAD endpoint). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ocal Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e-ingest guard (T-5000.3) Add proactive per-extent chunked backfill for large final tables (default threshold 2,000,000 rows OR >4 source extents, both env-configurable via BACKFILL_CHUNK_ROW_THRESHOLD / BACKFILL_CHUNK_EXTENT_THRESHOLD), so the Prices transform never attempts a single-pass backfill that OOMs; Costs stays single-pass with the reactive chunked fallback intact. Add a fail-fast guard: re-staging a file under the same (scope,type,period,run-uuid) with a changed checksum now halts with a non-zero exit and a clear message instead of silently appending duplicate rows. Harden Invoke-KustoPost strict-mode catch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…sers/ literal) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…resolution (T-5000.8) Add the finops-hub-local agent skill (SKILL.md thin router, README, local-vs-remote reference) that drives the local Kusto-emulator FinOps hub via the ftk.ps1 CLI and the finops-hub-local Makefile, reusing the published query catalog (no new KQL). Also fix ftk.ps1 catalog discovery for the in-repo layout: Get-FtkRepo assumed the old prototype's sibling-folder layout (<parent>/finops-toolkit) and failed to find src/queries/catalog when ftklocal lives inside the repo, breaking 'ftk list' and 'ftk run'. Resolve the toolkit root in-repo (three levels up from the script dir) with a git-toplevel fallback and the legacy sibling layout as last resort. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Under Set-StrictMode + $ErrorActionPreference='Stop', calling '& git' when git is absent throws CommandNotFoundException and skips the legacy sibling-layout fallback. Guard the git-toplevel probe with Get-Command so a missing git degrades gracefully. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…alog fix Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…+ harden Prices value parity (T-5000.3) Per architect-duo ruling: drop the extents>4 proactive-chunk criterion (OOM is row-driven; >4 extents mis-fires on every multi-part export and would chunk Costs, which single-passes safely at 15 extents). Gate proactive chunking on rawCount > 2,000,000 alone; keep the reactive single-pass->chunked fallback. Extent count is now informational logging only. Remove the BACKFILL_CHUNK_EXTENT_THRESHOLD knob; note that adding a new RawToFinalPolicy table needs a fresh single-pass OOM measurement. Fix the Makefile threshold comment (3,000,000 -> 2,000,000). Harden parity CHECK 13 (Prices value parity): emit delta_pct so the runner enforces a 0.1% relative tolerance, with a sentinel that fails on zero rows or row-count mismatch so the check cannot pass vacuously. Verified: real data 9,804,381 rows / sums equal at $58.39B -> PASS; x1.01 corruption -> FAIL; zero rows -> FAIL. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…uble-ingest guard, Prices value parity) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ss-db strip in ftk.ps1 (T-5000.7)
Add user-facing docs under src/templates/finops-hub-local/: README.md (quickstart,
two-database Ingestion+Hub architecture, ingest with the fail-fast double-ingest guard
and automatic chunked backfill, querying via ftk.ps1 + the agent skill, dashboard
best-effort, measured limits table, troubleshooting), notes/ftk-kql-adaptations.md (the
load-time vs ftk.ps1 on-read adapter layers), notes/parity-gaps.md (the 13-check parity
suite, grounded in a live run).
Also remove the now-stale database('Ingestion'). strip from the ftk.ps1 on-read adapter:
on the two-database topology cross-database references resolve correctly, so stripping
them would break any catalog query that legitimately references an Ingestion table.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…trip removal Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add notes/performance.md with measured memory characterization on MEM_LIMIT=16g (amd64-on-Rosetta), instrumented via cgroup v2: cold loaded floor ~1.12 GiB (7%), hot post-ingest working set 13.99 GiB (87.4%), Prices chunked-backfill peak 15.03 GiB (94%), and the single-pass Prices transform ceiling where Kusto self-terminates at 15.12 GiB (~94.5%). Includes the row-count to memory curve, the ingest wall-clock (raw 7.0 min, full 14.0 min), and an operating-states table that answers when the engine approaches the limit (only during ingest+transform of a 28M-row dataset). Reconcile the README limits table with these measured numbers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…red) Replace the speculative 'mixed-content blocked / not proven' caveat with measured findings: the Kusto emulator returns CORS headers for https://dataexplorer.azure.com (OPTIONS 204 + POST 200 with Access-Control-Allow-Origin), real dashboard tile queries return renderable data over that path, and dashboard.json (schema v60, 135 tiles / 87 queries / 12 pages) has zero dangling references so it imports cleanly. localhost is exempt from mixed-content blocking; the real browser gate is the Edge 143+/Chrome 142+ Local Network Access permission prompt (click Allow), with browser-flag and ngrok fallbacks documented for locked-down environments. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… + cross-platform bring-up Add per-OS install guides (notes/install-windows.md primary, notes/install-mac.md secondary) and make the README quickstart cross-platform: 'docker compose up -d --wait' (honors the container healthcheck) plus the self-contained 'pwsh scripts/*.ps1' commands replace the Unix-only 'make' front door, which remains a macOS/Linux convenience. The scripts already self-resolve HOST_PORT from .env, so they run standalone on Windows. Windows guide covers Docker Desktop + WSL 2, PowerShell 7 (not Windows PowerShell 5.1), WSL 2 memory tuning, forward-slash EXPORT_DIR, and Windows-specific troubleshooting; notes that linux/amd64 runs natively on Windows x64 (no emulation) so it outperforms the Apple-Silicon numbers in performance.md. Supporting correctness: add .gitattributes (LF for .ps1/.sh/.kql/.json/Makefile) to prevent CRLF corruption on Windows checkout, and '#Requires -Version 7.0' to the entry scripts so PowerShell 5.1 fails with a clear message instead of cryptic parser errors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Run repo Prettier (.prettierrc) over the finops-hub-local markdown so the docs conform to the toolkit coding standard (docs-wiki/Coding-guidelines.md): aligned table columns and underscore emphasis. Formatting only — no content, measured numbers, or links changed. 'prettier --check' now passes for all finops-hub-local markdown. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Matches the repo-wide markdown convention (all .md stored LF) and prevents CRLF drift for Windows contributors under core.autocrlf=true. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add notes/build-by-hand.md — the manual runbook for standing up a local FinOps hub: six steps, about a dozen raw commands (start emulator, create the two databases, load the schema, ingest Parquet, build the final tables, query). Each schema bundle is a single .execute database script command, so loading is one POST per database; the doc shows the exact curl/KQL for every step, verified against the emulator (118/118 statements load in one call). A closing table maps each manual step to the helper that automates it, so the scripts are no longer a black box. Link it from the README. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The engine runs a whole bundle natively via '.execute database script with (ContinueOnErrors=true)', returning a per-statement result table. So wrap each built bundle and POST it once instead of splitting it into ~157 commands and posting each in a loop. Removes the command splitter and per-command loop (~48 lines); per-statement diagnostics are preserved by parsing the result table. Verified: 118/118 Ingestion and 39/39 Hub statements load in one call each, schema reload is non-destructive (data intact at the known total), and the dry-run path is unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…FinOpsHubLocal command, and guide Replace the rejected agent skill with a tested PowerShell command and a build-generated open-data artifact for the Run hubs locally guide. Verified end-to-end against a real Kusto emulator with real FOCUS data (7/7 integration tests, no mocks). - Add finops-hub-local-opendata.kql build artifact (.build.config) so the guide downloads it instead of hand-coding externaldata schemas. - Add Initialize-FinOpsHubLocal (downloads released setup scripts, applies retention, loads open data with verify-and-retry); no Docker management. - Order ingest prices-before-costs so the cost transform enriches from Prices_final_v1_2. - Add reference page, command list, TOC, and changelog entries; remove the ftklocal agent skill. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
microsoft-github-policy-service
Bot
added
Needs: Review 👀
# Conflicts: # docs-mslearn/toolkit/changelog.md
6-tab live FinOps dashboard for local Kusto emulator (port 8082): - Overview: total cost KPIs, top-10 services/regions, daily trend - Allocation: cost by resource group, subscription, department - Rate optimization: savings summary, commitment utilization (incl. col-5/col-7 grid fix) - Usage & unit economics: core-hours, unit cost trends - Anomalies & forecast: spend anomalies, 30-day forecast - AI tokenomics: token usage, model cost breakdown Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
- Add kpiThreshold() helper: maps pct to threshold-green/amber/red - Add isPartialMonth() helper: true when today < last day of current month - Extend kpiCard() with optional 5th thresholdClass param - Overview: Untagged cost gets threshold-green/amber/red badge (<10/10-25/>25%) - Overview: Commitment coverage meta updated to 'of compute spend' - Overview: Latest month appends 'partial month' (amber) when month is incomplete - Rate tab: Effective savings rate meta adds 'vs. list price' sub-label - Rate tab: Commitment waste gets threshold badge (<10/10-20/>20%) + 'of commitment spend' sub-label - Allocation tab: Untagged cost gets same threshold badge as overview - CSS: add .threshold-green/.threshold-amber/.threshold-red border-color rules Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add 3-tile triage strip (Anomalies | Overspend | Savings Opportunities) above the KPI row on the Overview tab only - Each tile shows count, severity badge, and one-line action cue - Tile color uses threshold-green/amber/red classes from Task 1 - Anomaly count: reads from state.cache['anomaly'] when loaded - Overspend count: months in trend data where effective cost rose >20% MoM - Savings count: underutilized commitments from state.cache['rate'] when loaded - Clicking any tile fires switchTab() to deep-link to the relevant tab - Zero count renders 'None' in green; unloaded cache renders '—' (neutral) - Extract switchTab() from wireControls so triage tiles can reuse it - Add responsive 1-column layout at ≤640 px Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e strip - Bug 1: Fix Overspend tile navigating to 'anomaly' tab instead of 'usage' - Bug 2: Replace null-check pattern for anomaly/savings counts with || [] pattern Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add module-level queryState { rows, health, refreshedAt, dataset }
- Add fmtRelativeTime() using Intl.RelativeTimeFormat (no dependencies)
- Add renderDiagnosticRail() — populates #diagnostic-rail below content
- Update updateChrome(): populate queryState from payload, call rail;
health=warn on 0 rows, health=error on fetch failure, health=ok otherwise
- Simplify #source-line to cluster URI only; clear row/refresh data from
#footer-meta (window range kept); all data metadata moves to rail
- Add #diagnostic-rail div to index.html between </main> and <footer>
- Add .diagnostic-rail CSS: 24px max-height, muted text, flex layout;
.rail-health--ok/warn/error colour-coded; dotted underline on time tooltip
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- openKqlDialog(panelId): reads PANEL_KQL, populates textarea, calls showModal() - executeKql(): POST /api/kql, renders result table inside dialog - showKqlResult(): displays up to 200 rows in scrollable table - wireControls(): wires kql-close, kql-copy, kql-run, and .kql-btn delegation - panelHtml(): adds data-panel-id to button, fixes aria-label to 'View KQL query' - All 26 PANEL_KQL panels migrated to panelHtml() (tokenomics, anomaly, usage, rate, allocation) - index.html: remove readonly from textarea (edit+run AC requires editable) - app.css: add .btn-primary, .kql-result, .kql-result-scroll, .kql-dialog-footer styles Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… guard - executeKql() now closes the dialog on success and calls renderKqlResultInPanel() which injects the result table directly into the matching panel's .panel-body (found via [data-panel-id] DOM attribute), satisfying AC3 - Replaced showKqlResult() (in-dialog table injection) with renderKqlResultInPanel() - Added res.ok guard before res.json() so HTTP 4xx/5xx surfaces as 'Server error N' Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ence KPIs (T-5001.3) - Add kpi--primary and kpi--reference CSS classes to distinguish KPI importance - Modify kpiCard() to auto-detect primary/reference status based on KPI label - Primary KPIs: Untagged cost, Commitment waste, Effective savings rate (grid-column: span 2, accent border) - Reference KPIs: Commitment coverage, Compute coverage, Total tokens, Anomaly days (85% opacity, muted border) - Primary KPIs render at full size with accent borders for visibility - Reference KPIs render smaller/muted for context without distraction - Integrates with Task 1's threshold badge logic (coexist in same class string) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…001.3) - Reorder KPI arrays in renderOverview, renderRate, renderTokenomics, renderAnomaly, renderAllocation so primaries appear first, then references, then supporting KPIs - Remove grid-column:span 2 from .kpi--primary to eliminate auto-fit layout holes on narrow viewports - Fix double top-border: .kpi--primary now uses thicker inset box-shadow (4px) instead of a competing border-top; .kpi--reference uses --kpi-accent:var(--border-muted) + opacity:0.75 instead of border-top - Add --border-muted custom property to :root with GitHub token fallback Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…-size rule (T-5001.3) - Replace --kpi-accent re-assignment in .kpi--reference with explicit box-shadow override - Remove ineffective font-size: 1.05em from .kpi--primary (children use absolute px units) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…01.5) - Add runQuery import from kusto.mjs - Add /api/kql POST handler for open KQL escape hatch - Extract logError + sendQueryError helpers - Sanitize error messages to avoid leaking stack traces to browser - Harden unhandled rejection path (res.headersSent guard) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ted-format-string) Pass context as a separate argument to console.error so it is never used as a printf-style format string. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…alog - Rate tab: commitment-utilization-score (formal KPI per catalog) per-commitment table + grand-total score; top-commitment-transactions table (top 10 RI/SP purchases) - Overview tab: macc-consumption-vs-commitment KPI card (graceful N/A when no Transactions() data) - Tokenomics tab: ai-cost-by-application showback table by app/team/env/cost-center Closes 5 catalog query gaps identified in INDEX.md + KPI.md coverage review. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Filter state: state.filters {dim -> string[]} + filterKey()/cacheKey()/filterParam()
- hbar(): filterable rows get data-filter-dim/val attrs, CSS classes hbar-filterable/selected/dimmed
- renderFilterBar(): renders chips (×-removable) + Reset all button in #filter-bar
- wireControls(): event delegation for hbar click, chip remove, reset
- Server-side KQL injection: buildFilterWhere() in kusto.mjs appends | where Col in (...)
- extension.mjs parseFilters() reads ?filters=<JSON> and forwards to all 6 getters
- Cache keyed by preset+filterKey() so filtered/unfiltered results are cached separately
- tagKeys hbar opts out of filtering (synthetic key column)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
P1: remove KPI hero-metric top-stripe (box-shadow inset); replace with
threshold value-text coloring (threshold-green/amber/red → .value)
P1: remove false hover affordance from non-filterable hbar rows
(.hbar:hover removed from opacity rule)
P2: fix triage-title uppercase/tracking eyebrow anti-pattern
(text-transform:uppercase + letter-spacing removed)
P2: replace loading text with shimmer skeleton grid
P3: restructure error state with copyable Initialize-FinOpsHubLocal
command block and collapsible error detail
P3: add filter-bar show/hide CSS transition
P3: normalize all non-standard font-weights (680→700, 650/640/620→600,
580→500, var fallback 650→600)
P3: remove dotted underline from .rail-time (misleading affordance)
P3: add tab overflow-x:auto + white-space:nowrap for narrow canvas widths
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- AbortController: cancel in-flight fetch when tab/preset changes; discard AbortError silently - aria-busy on #content during load; aria-live="polite" on main element (screen reader announcements) - Render error boundary: catch any renderXxx() throw and show structured error instead of blank screen - fmtMonth/fmtDayRange: null/type guards to prevent crash on sparse Kusto data - Filterable hbar rows: tabindex="0" + role="button" + aria-label for keyboard access; Enter/Space delegation in wireControls - KQL copy button: success/failure feedback with 1.5s label swap - svgEl: optional aria-label parameter for chart accessibility Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…a-pressed, KQL button - P1: add KPI_TIPS map with threshold rationale for 7 KPIs; render as native title-tooltip '?' button (kpi-tip) on each labelled card — no JS required - P2: add aria-pressed to filterable hbar rows so AT announces toggle state - P2: replace KQL button icon-only '</>' with text label 'KQL'; bump default opacity to 0.85 (was 0.7) so the button is always discoverable - P2: triage arrival context — anomaly tab shows amber callout banner when anomalous days detected; rate tab shows callout when underutilized commitments exist; both echo the triage-tile finding to anchor the user Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…container restarts Without -v /kustodata, databases are stored in the container's writable layer. docker rm (or Docker Desktop recreation) wipes everything, forcing the user to redo the full setup. Add $dataPath + the volume flag so persistence is durable by default. Also document the restart path (docker start). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🛠️ Description
Adds and UAT-hardens a self-hosted path for FinOps hubs: a documented way to run the hub analytics layer on your own hardware using the Kusto emulator. Customers, consultants, contributors, and agents can exercise the same hub KQL, schema, transforms, update policies, and open-data enrichment without deploying the full Azure-hosted stack.
This PR is no longer docs-only. Based on review feedback (see below), it now also ships the supporting automation so the guide stays correct and easy to follow:
finops-hub-local-opendata.kqlbuild artifact — the open-data load script is generated by the toolkit build from the same source a deployed hub uses (.build.configcombineKql), so the guide downloads and runs it instead of hand-codingexternaldataschemas that could silently drift.Initialize-FinOpsHubLocalPowerShell command — sets up a local hub against a running emulator in one call: creates theIngestionandHubdatabases, downloads the released setup scripts, applies raw retention, and loads open data with verify-and-retry. It does not install or manage Docker, and it does not ingest cost data.Prices_final_v1_2.Why this changed from the original docs-only PR
@flanakin requested removing the agent skill (anti-pattern) and asked whether the fragile inline blocks could be build-generated and driven by a PowerShell command. This PR does exactly that: the skill is removed, the open-data load is build-generated, and the deterministic steps are a tested command — no Docker management in the module.
Verification
Verified end-to-end against a real Kusto emulator (Docker
kustainer-linux) with real release artifacts served over HTTP and real FOCUS data (Cost Management export, billing account 8611537, 2025-04). No mocks.src/powershell/Tests/Integration/Initialize-FinOpsHubLocal.Tests.ps1: 7/7 passed.Costs_raw117,559 →Costs_final_v1_2117,559;Prices_raw/Prices_final_v1_2853,636; prices present before the costs phase;Costs_rawretention90.00:00:00; cost enrichment 117,559/117,559.externaldataread after setup can silently return 0 rows; the command and the guide now load, verify, and retry.Boundaries
The Kusto emulator is not for production: no security, no managed ingestion, limited by host capacity. The local hub is an analysis and validation tier, not a replacement for an Azure Data Explorer or Microsoft Fabric hub.
No linked issue.
📷 Screenshots
Not applicable.
📋 Checklist
🔬 How did you test this change?
📦 Deploy to test?
🙋♀️ Do any of the following that apply?
📑 Did you update
docs/changelog.md?📖 Did you update documentation?