Skip to content

FOCUS 1.4 phase 1: add FOCUS 1.3 ingestion (IngestionSetup_v1_3.kql) #2124

@flanakin

Description

@flanakin

Parent: #2120

Goal

Add FOCUS 1.3 ingestion support: a new IngestionSetup_v1_3.kql that defines _v1_3 raw transforms and final tables, plus the new columns added by FOCUS 1.3 in the IngestionSetup_RawTables.kql raw schema.

This phase covers Cost and Usage only. The new Contract Commitment dataset is its own phase (3).

FOCUS 1.3 Cost and Usage column additions

These are new in 1.3 and need handling in the 1_3 transform / final table:

Column Type Mandatory Notes
AllocatedMethodId String Conditional Data generator-calculated split cost allocation method ID
AllocatedMethodDetails JSON Recommended Allocation factors
AllocatedResourceId String Conditional Resource ID of allocated charge
AllocatedResourceName String Conditional Resource name of allocated charge
AllocatedTags JSON Conditional Tags of allocated charge
ContractApplied JSON Conditional Per-row contract commitment application
ServiceProviderName String Mandatory Replacement for deprecated ProviderName
HostProviderName String Mandatory (nullable) Underlying infrastructure provider

Deprecations to handle

  • ProviderName — deprecated in 1.3, removed in 1.4. Keep in v1_3 schema with same value as ServiceProviderName for backward compatibility.
  • PublisherName — deprecated in 1.3, removed in 1.4. Keep in v1_3 schema (legacy consumers).

Scope

File: src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_v1_3.kql (new)

  1. Copy IngestionSetup_v1_2.kql as the starting point.
  2. Rename functions/tables/policies: _v1_2_v1_3.
  3. Rename comments referencing FOCUS 1.2 → FOCUS 1.3.
  4. Update Costs_final_v1_3 table schema to include the 8 new columns.
  5. Update Costs_transform_v1_3() function to populate the 8 new columns from the raw table (mostly direct passthrough since Cost Management hasn't shipped 1.3 exports yet — null defaults are OK).
  6. Update raw-table update policy: the transform function should target _v1_3 final tables, and the v1_2 update policy should be deleted/redirected per the existing pattern.

File: IngestionSetup_RawTables.kql

Add new columns under each affected raw table (Costs_raw at minimum) with version annotation comments matching the existing convention (e.g., // FOCUS 1.3+).

Out of scope for this phase

  • New ContractCommitment_raw table and transform → phase 3
  • HubSetup_v1_3.kql with versioned hub functions → phase 2
  • Cost Management focuscost_1.3.json schema → out of scope (CM-owned, only updated when CM ships an official 1.3 export)
  • Open data FocusCost_1.3.json → phase 6

Branch

flanakin/focus14-phase1-ingestion-1.3 off flanakin/focus14. PR target: flanakin/focus14.

Done when

  • New file compiles cleanly (no KQL parse errors via the existing test pattern).
  • Raw tables include 1.3 column annotations.
  • Existing v1_2 ingestion still parses (no regression).
  • PR opened against flanakin/focus14.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions