From 5b32714f9f0960889c1b704cba524630eee88e5a Mon Sep 17 00:00:00 2001 From: Roland Krummenacher Date: Sun, 28 Jun 2026 14:10:11 +0200 Subject: [PATCH 1/5] feat(open-data): Add instance size flexibility dataset from Catalogs API (#2090) Adds a new InstanceSizeFlexibility open data dataset to replace the deprecated ISF CSVs hosted on ccmstorageprod.blob.core.windows.net (AutofitComboMeterData.csv and isfratioblob.csv / aka.ms/isf), which Microsoft is retiring (no updates after 9 May 2026, removed 30 Aug 2026). - Update-InstanceSizeFlexibility.ps1 extracts ISF groups/ratios from the authenticated Azure Reservations Catalogs API (Microsoft.Capacity/catalogs), unioning the four ISF-bearing reserved resource types (VirtualMachines, BlockBlob, RedisCache, DedicatedHost) across a broad region set. - InstanceSizeFlexibility.csv is populated with 2,437 SKUs / 558 groups. - Weekly workflow refreshes the data via OIDC (maintainers must configure the AZURE_CLIENT_ID/TENANT_ID/SUBSCRIPTION_ID secrets; see workflow header). - Raw Microsoft ratios are kept for drop-in parity with the deprecated files (normalization available via -Normalize). Foundation only: Power BI and Optimization Engine consumers will be repointed to this dataset in a follow-up once it ships in a release. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../opendata-instance-size-flexibility.yml | 78 + docs/open-data.md | 6 + src/open-data/InstanceSizeFlexibility.csv | 2438 +++++++++++++++++ src/open-data/README.md | 25 + .../Update-InstanceSizeFlexibility.ps1 | 341 +++ 5 files changed, 2888 insertions(+) create mode 100644 .github/workflows/opendata-instance-size-flexibility.yml create mode 100644 src/open-data/InstanceSizeFlexibility.csv create mode 100644 src/scripts/Update-InstanceSizeFlexibility.ps1 diff --git a/.github/workflows/opendata-instance-size-flexibility.yml b/.github/workflows/opendata-instance-size-flexibility.yml new file mode 100644 index 000000000..25fffff88 --- /dev/null +++ b/.github/workflows/opendata-instance-size-flexibility.yml @@ -0,0 +1,78 @@ +name: Update Instance Size Flexibility + +on: + schedule: + - cron: '0 6 * * 1' # Every Monday at 06:00 UTC + workflow_dispatch: + +# NOTE: Unlike the other open data workflows, the Catalogs API is authenticated and requires +# Microsoft.Capacity/catalogs/read. Maintainers must configure an OIDC service principal and +# set the AZURE_CLIENT_ID, AZURE_TENANT_ID, and AZURE_SUBSCRIPTION_ID repository secrets before +# this workflow can run. See https://github.com/Azure/login#configure-a-federated-credential. +permissions: + contents: write + pull-requests: write + id-token: write + +jobs: + update: + runs-on: ubuntu-latest + timeout-minutes: 60 + steps: + - uses: actions/checkout@v4 + with: + ref: dev + + - name: Azure login (OIDC) + uses: azure/login@v2 + with: + client-id: ${{ secrets.AZURE_CLIENT_ID }} + tenant-id: ${{ secrets.AZURE_TENANT_ID }} + subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }} + enable-AzPSSession: true + + - name: Fetch ISF ratios from Azure Catalogs API + uses: azure/powershell@v2 + with: + azPSVersion: latest + inlineScript: src/scripts/Update-InstanceSizeFlexibility.ps1 + + - name: Create PR if data changed + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PR_BODY: | + ## Summary + + Weekly automated update of instance size flexibility (ISF) ratios from the + [Azure Reservations Catalogs API](https://learn.microsoft.com/azure/cost-management-billing/reservations/instance-size-flexibility#extract-instance-size-flexibility-ratios-using-azure-catalogs-api). + + This file maps each ARM SKU to its instance size flexibility group and ratio, + replacing the deprecated `AutofitComboMeterData.csv` and `isfratioblob.csv` files + that were hosted on `ccmstorageprod.blob.core.windows.net` (retired 30 Aug 2026). + + ## Test plan + + - [ ] Review the diff for unexpected changes (large drops in group/SKU count, etc.) + - [ ] Spot-check a few SKUs against the Azure Pricing Calculator + + πŸ€– Generated by the **Update Instance Size Flexibility** workflow + run: | + # Check for changes (handles both new file and modified file) + if git diff --quiet --exit-code -- src/open-data/InstanceSizeFlexibility.csv 2>/dev/null && \ + ! git ls-files --others --exclude-standard | grep -q 'InstanceSizeFlexibility.csv'; then + echo "No changes detected" + exit 0 + fi + + BRANCH="opendata/instance-size-flexibility-$(date +%Y%m%d)" + git checkout -b "$BRANCH" + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + git add src/open-data/InstanceSizeFlexibility.csv + git commit -m "chore: Update instance size flexibility ratios" + git push origin "$BRANCH" + + gh pr create \ + --title "[Open Data] Update instance size flexibility ratios ($(date +'%B %Y'))" \ + --base dev \ + --body "$PR_BODY" diff --git a/docs/open-data.md b/docs/open-data.md index fd0fd59ab..ed191680b 100644 --- a/docs/open-data.md +++ b/docs/open-data.md @@ -34,6 +34,12 @@ FinOps toolkit open data is used to transform Cost Management actual and amortiz ## Download open data