diff --git a/.claude/commands/announce.md b/.claude/commands/announce.md new file mode 100644 index 000000000..01179d0fc --- /dev/null +++ b/.claude/commands/announce.md @@ -0,0 +1,103 @@ +--- +description: Draft a blog announcement for a release, tool, or feature +allowed-tools: Read, Grep, Glob, Write, Edit, WebFetch, Bash(git status*), Bash(git diff*), Bash(git log*), Bash(ls*) +--- + +# Announce + +Drafts a blog post in `announce/{year}/{yyyy-MM}_{slug}.md` for publishing to the [FinOps Tech Community blog](https://techcommunity.microsoft.com/tag/finops%20toolkit?nodeId=board%3AFinOpsBlog). + +**Input:** Optional topic (e.g., `v14`, `commitment discount eligibility data`, `claude plugin`). If omitted, the command identifies candidates automatically. + +## Phase 1: Pick a topic + +If the user provided a topic, skip to **Phase 2**. + +Otherwise: + +1. Fetch the recent blog list from `https://techcommunity.microsoft.com/tag/finops%20toolkit?nodeId=board%3AFinOpsBlog` to see what was already announced. +2. Read [@docs-mslearn/toolkit/changelog.md](../../docs-mslearn/toolkit/changelog.md). Identify what's new since the last announced release. +3. Pick **up to 4 candidates**, prioritized in this order: + 1. **New version** — the latest `## v##` section if it hasn't been announced. + 2. **New tools** — H3 sections in this release that didn't exist in the previous release. + 3. **New/added features** — sub-bullets under "Added" that introduce notable capability. + 4. **Other notable changes** — sub-bullets under "Changed" or "Fixed" that address visible pain. +4. Present via AskUserQuestion as a single question with the candidates as options. Always include a "(Recommended)" tag on the highest-priority candidate. The default "Other" option lets the user type something custom. +5. If the user picks "Other" with no clear topic, ask one console question for clarity. Do **not** keep guessing. + +## Phase 2: Research + +Gather context for the topic: + +1. Read the relevant section(s) of [@docs-mslearn/toolkit/changelog.md](../../docs-mslearn/toolkit/changelog.md). +2. Read related docs in `docs-mslearn/` and `docs/` (use Glob/Grep to find them by tool/feature name). +3. Only fall back to source code under `src/` if no docs cover the feature. +4. For release announcements, also read the **prior** release's announcement file in `announce/` for tone reference. + +If gaps remain after research, batch up to 4 questions in a **single** AskUserQuestion call. Each question must offer a recommended option. Discovery questions should be about clarifying scope or filling factual gaps — not stylistic preferences. + +Only use a second round of AskUserQuestion if the first round didn't give enough clarity. If you still can't get clarity, ask via the console rather than guessing further. + +## Phase 3: Outline + +Propose an outline. + +### For release announcements + +The outline is **fixed** — do not invent new sections. Use this structure (matches `announce/2025/2025-08_v12.md` and `announce/2026/2026-02_v13.md`): + +1. **Title** — `What's new in FinOps toolkit {N} – {Month} {Year}` +2. **Preview snippet** — 1–2 sentences shown in blog list previews and search results. Lead with the most newsworthy item. No links, no markdown, under ~300 chars. Place under the title as a markdown blockquote (prefix the line with `>`) so it's clearly separable from the intro. +3. **Intro paragraph** — One paragraph: who this release is for, what it emphasizes (themes, not full list), and a "read on for details" hook. +4. **New to the FinOps toolkit?** — Boilerplate intro for new readers. Reuse from the prior release; update links if they changed. +5. **Featured sections** (3–5) — Themed deep-dives on the biggest changes from the changelog. Each gets its own H2 with a benefit-led title (not just the feature name). This is the only part that varies per release. Inline contributor thanks belong here when relevant (e.g., "Thanks to @username for contributing this feature."). +6. **Other new and noteworthy updates** — Bulleted list grouped by tool, covering everything else worth mentioning that didn't get a featured section. Extract from the changelog, rephrase to be user-friendly. +7. **What's next** — Forward-looking paragraph. Reuse and refresh from the prior release. +8. **Social media appendix** (see Phase 4). + +The **only** thing to confirm with the user during scoping is **which featured sections to include and in what order**. Generate 3–4 concrete, terminal options up front so the user picks once and is done. Each option must fully describe the resulting outline — no follow-up question needed. + +Build the options like this: + +1. **Option 1 (Recommended):** the proposed order as a `→`-joined list of section titles. +2. **Option 2:** a meaningful alternative (e.g., reordered to lead with a different theme, or dropping the weakest section to keep it tight at 3). +3. **Option 3:** another meaningful alternative (e.g., swap one featured section for a candidate that didn't make the cut, or extend to 5 by promoting a notable item). +4. **Option 4 (only if relevant):** a third alternative if there's a clear close-call worth offering. + +Each option's `description` field must include the full section list it represents — the user reads the full outline in the option, not just a label. The user-supplied "Other" fallback handles freeform changes. Do NOT use vague options like "Reorder them", "Swap one", "Approve", or "Edit" — they all require a follow-up question. + +### For non-release posts + +Propose an outline tailored to the topic. Aim for 3–6 H2 sections following news/blog best practices: + +- **Hook** — Why this matters now (1–2 paragraphs). +- **Problem or context** — What pain or opportunity drives this. +- **The thing** — What's new, with concrete details. +- **How to use it / next steps** — Action-oriented. +- **Related reading** — Links. + +Present the proposed outline via AskUserQuestion with options to approve, reorder, or rewrite a specific section. Same rule: no generic "Approve" options. + +## Phase 4: Draft + +1. Determine the publish month. Default to the current month unless context (release date, scheduled publish date) clearly implies a future month. Don't ask just to confirm — only ask if there's actual ambiguity. +2. Determine the slug. For release announcements, use `v{N}` (e.g., `v14`). Otherwise, use a short kebab-cased topic (e.g., `azure-openai-costs`). +3. Write the draft to `announce/{year}/{yyyy-MM}_{slug}.md` using `Write`. +4. **Tone:** active voice, benefit-led, friendly but not breathless. Match the voice of recent releases in `announce/`. +5. **Microsoft style:** sentence casing for headings (not Title Case), no end punctuation on short headings, follow [Coding-guidelines.md](../../docs-wiki/Coding-guidelines.md) content rules. +6. **Length:** release announcements run ~6–10 minute reads. Featured sections are 2–4 short paragraphs each. Don't pad. +7. **Links:** prefer `aka.ms` short links when they exist (check the changelog for examples). Use full Learn URLs for deep documentation. + +### Social media appendix + +Append a `## Social media` section at the end of the file with three short blurbs: + +- **LinkedIn** — 2–3 sentences, professional tone, includes the blog URL placeholder `{{POST_URL}}` and 1–3 relevant hashtags (#FinOps, #Azure, #FOCUS). +- **Twitter/X** — Under 280 characters, includes `{{POST_URL}}` and 1–2 hashtags. +- **Internal** — One sentence + URL for posting in F2 Slack or similar internal channels. + +The blog URL isn't known at draft time, so use `{{POST_URL}}` as a placeholder for the user to fill in after publishing. + +## Phase 5: Wrap up + +Report the file path and a one-line summary of what was written. Do not commit — the user reviews and commits manually. diff --git a/announce/2025/2025-08_non-breaking-changes-in-hubs-12.md b/announce/2025/2025-08_non-breaking-changes-in-hubs-12.md new file mode 100644 index 000000000..6203a93b4 --- /dev/null +++ b/announce/2025/2025-08_non-breaking-changes-in-hubs-12.md @@ -0,0 +1,115 @@ +# Introducing non-breaking "breaking" changes in FinOps hubs 12 + +_Published: 2025-08-05 · [Original post](https://techcommunity.microsoft.com/blog/finopsblog/introducing-non-breaking-%E2%80%9Cbreaking%E2%80%9D-changes-in-finops-hubs-12/4438554)_ + +Before I explain this, I want to say that I'm extremely excited about this update. FinOps hubs was designed to solve a common versioning challenge many organizations face where they need data coming from new columns but can't update because of breaking changes in other columns. FinOps hubs solves this by introducing these breaking changes in a non-breaking way, giving you the control and flexibility to update when and where you need while leaving your foundational reports and integration points untouched and running just as smoothly as before. + +FinOps hubs 12 is the first release to fully realize the value of this non-breaking, "breaking" changes approach since the architecture was established late last year. This approach ensures the FinOps hubs platform will not break reports, will not stagnate with historical baggage, and will also avoid getting bloated with duplicate columns and data, like you might see in certain Cost and Usage Reports out there. But let me take a step back and walk you through it… + +## How schema versioning works in FinOps hubs + +FinOps hubs 0.7 added a custom, FOCUS-aligned schema for all supported datasets. When data is ingested into FinOps hubs with Azure Data Explorer or Microsoft Fabric, the native schemas are transformed into a FOCUS-like dataset to provide forward-looking datasets aligned to the future direction of FinOps across the industry. The data is also augmented to with extra columns and missing data to facilitate common FinOps tasks and goals we hear from organizations big and small. We refer to this as the v1_0 schema because all our tables and functions are named *_v1_0 to be clear about what schema version they use. + +Some of you may be using the non-versioned functions, like Costs and Prices. These are wrappers around the corresponding versioned functions, like Costs_v1_0 and Prices_v1_0. The non-versioned functions are for ad-hoc use when you need a quick answer and don't want to think about what version you need. These always return the latest version. And until FinOps hubs 12, this was always v1_0. + +Now, FinOps hubs 12 includes a new v1_2 dataset that aligns to FOCUS 1.2 and includes even more augmented columns to support new scenarios. This gives you three options when querying the system. Let's use cost as an example: + +- **Costs** is that ad-hoc function where you don't have to think about what version you need. This now uses the v1_2 schema. +- **Costs_v1_0** is the original FOCUS 1.0 schema that was implemented in FinOps hubs 0.7. This has not changed and will not change. +- **Costs_v1_2** is the new schema that aligns to FOCUS 1.2 and includes additional columns to support other scenarios like commitment discount utilization, Azure Hybrid Benefit analysis, and more. + +If you followed our guidance, then your reports, dashboards, and integration points should all use the versioned functions, like Costs_v1_0. In that case, upgrading to FinOps hubs 12 shouldn't impact you at all. All your reports and dashboards will continue to function as they have before. If you find you used non-versioned functions, like Costs, simply change to the versioned functions and you should revert back to the same behavior you were seeing before. + +## Working with older FOCUS exports + +Microsoft Cost Management has four different dataset versions for their FOCUS exports: + +- **1.0-preview(v1)** is aligned to FOCUS 1.0 preview from November 2023. This was the first public release. +- **1.0** is fundamentally the same as 1.0-preview(v1) except with changes in the official FOCUS columns to align to the FOCUS 1.0 GA. +- **1.0r2** is the same as 1.0 except the date columns, like ChargePeriodStart and ChargePeriodEnd, are formatted with seconds. That's it. Older versions use "2025-01-01T00:00" and 1.0r2 going forward use "2025-01-01T00:00:00". The only difference is the added ":00" to support some systems which weren't able to parse dates without seconds. +- **1.2-preview** is aligned to FOCUS 1.2, except there are a few gaps that have not been filled, so it's flagged as a preview. Once those gaps are filled, you'll see a new "1.2" release. + +FinOps hubs can work with any of these versions. When you export an older version of the data, FinOps hubs simply transforms it to the latest schema version. This means, if you're working on top of a 1.0-preview(v1) export, that data will now be fully converted to FOCUS 1.2, even if Cost Management didn't provide the new columns. If you're still using the v1_0 schema, you won't even notice the difference. But as soon as you need to leverage one of the newer columns in the v1_2 schema, it's right there for you, ready when you are. And the best thing is, you don't need to reprocess any of the data. All your historical data is immediately accessible using either the v1_0 or v1_2 schema. + +I'll leave it at this for now, but please do leave comments if you're curious about the inner workings of this and how we implemented it. I'm happy to write a more detailed blog post to share the inner workings. In the meantime, refer to [FinOps hub data model](https://learn.microsoft.com/cloud-computing/finops/toolkit/hubs/data-model) to learn more. + +## What's new in Costs_v1_2 + +Each of the datasets supported by FinOps hubs were updated. The Costs dataset had the most updates, so we'll cover those first. The first difference you'll notice in Costs_v1_2 is support for the latest version of FOCUS: + +- Added CapacityReservationId +- Added CapacityReservationStatus +- Added CommitmentDiscountQuantity +- Added CommitmentDiscountUnit +- Added ServiceSubcategory +- Added SkuPriceDetails based on x_SkuDetails, changed to align to FOCUS 1.2 requirements +- Renamed x_InvoiceId to InvoiceId +- Renamed x_PricingCurrency to PricingCurrency +- Renamed x_SkuMeterName to SkuMeter + +You'll also see new columns coming from Microsoft Cost Management: + +- x_AmortizationClass to help filter out principal charges that can be duplicated when summing ListCost and ContractedCost. +- x_CommitmentDiscountNormalizedRatio for the instance size flexibility ratio needed to support CommitmentDiscountQuantity calculations. +- x_ServiceModel to indicate what service model the charge is (i.e., IaaS, PaaS, SaaS). +- x_SkuPlanName for the Marketplace plan name. + +Note that some of the above columns are empty coming from Cost Management. FinOps hubs populates most of the missing columns, like the new capacity reservation and commitment discount columns. We added a new x_SourceValues column to track the column changes happening during FinOps hubs data ingestion. If you're curious about any of the customizations applied on top of Cost Management data, review the properties in x_SourceValues. Any value that is changed is first backed up in x_SourceValues with its original column name and value to help source data quality issues. + +While not a new column, one other change you may notice is that x_SkuTier is now being populated across all Cost Management FOCUS versions. This is an important one because you cannot get this information from actual and amortized cost datasets. You will only see the tier in FOCUS datasets. That's just one more reason to switch to FOCUS. + +Looking beyond the columns coming from Cost Management, you'll also see extended columns for Alibaba and Tencent Cloud. This completes our native cloud FOCUS dataset support alongside AWS, GCP, and OCI, which are already supported. (Note we don't ingest the cost automatically. We added support for the data once it's been dropped into Azure storage.) This includes the following new columns: + +- Alibaba: x_BillingItemCode, x_BillingItemName, x_CommodityCode, x_CommodityName, x_InstanceID +- Tencent: x_ComponentName, x_ComponentType, x_ExportTime, x_OwnerAccountID, x_SubproductName + +FinOps hubs also added new columns to support scenarios covered in Power BI reports and the Data Explorer dashboard. With these columns promoted to the database level, reports will render faster and more consistently. This includes: + +- Discount percentage columns: x_NegotiatedDiscountPercent, x_CommitmentDiscountPercent, x_TotalDiscountPercent. +- Savings columns: x_NegotiatedDiscountSavings, x_CommitmentDiscountSavings, x_TotalSavings. +- Commitment discount utilization columns: x_CommitmentDiscountUtilizationAmount, x_CommitmentDiscountUtilizationPotential. +- Azure Hybrid Benefit columns: x_SkuLicenseQuantity, x_SkuLicenseStatus, x_SkuLicenseType, x_SkuLicenseUnit. +- SKU property columns: x_SkuCoreCount, x_SkuInstanceType, x_SkuOperatingSystem. +- x_ConsumedCoreHours to track total core hours for the charge by multiplying ConsumedQuantity by x_SkuCoreCount. + +## FOCUS updates for other v1_2 datasets + +While updating the Costs dataset, we also updated the other datasets to align to FOCUS 1.2 changes. Changes in other tables weren't as big, but pair well and will be important to note if you're using those functions: + +- CommitmentDiscountUsage + - Added CommitmentDiscountUnit + - Renamed x_CommitmentDiscountQuantity to CommitmentDiscountQuantity +- Prices + - Renamed x_PricingCurrency to PricingCurrency + - Renamed x_SkuMeterName to SkuMeter +- Transactions + - Renamed x_InvoiceId to InvoiceId + +## Recommendations changes for the future + +In addition to aligning to FOCUS 1.2, we updated the Recommendations dataset schema to account for future plans to ingest Azure Advisor recommendations and also generate custom recommendations. This includes the following new columns: + +- ResourceId +- ResourceName +- ResourceType +- SubAccountName +- x_RecommendationCategory +- x_RecommendationDescription +- x_RecommendationId +- x_ResourceGroupName + +These columns are empty today, but will be populated in a future release when the Azure Advisor integration is complete. + +## Decimal columns switched to the Real datatype + +In our initial Data Explorer release, we set all floating-point columns, like prices and costs, to use the decimal datatype. Later, we learned that real is preferred when remaining under a certain level of precision. While we couldn't make the change in a non-breaking way within the v1_0 schema version, adopting a new schema version offered the perfect chance to address this. + +Starting in v1_2, all floating-point columns will use the real datatype. If you're extending the tables, functions, or building any custom extensions, be sure to switch from decimal to real when you switch to the v1_2 schema. If you opt to remain on v1_0, you can disregard this as v1_0 will continue to use the decimal datatype going forward and will not change based on our non-breaking promise. For those of you who do switch, you may notice a slight performance improvement when working with numbers at scale. + +## Next steps + +Some may look at this update and see it as a simple update to align to FOCUS 1.2, while others may see it as a major shift in how FinOps hubs work and how that impacts the data being ingested. The truth is it's somewhere in the middle. FinOps hubs were designed to scale beyond a single FOCUS dataset version. And while FinOps hubs have always supported multiple dataset versions with 1.0-preview(v1), 1.0, and 1.0r2, this is the first time when the schema version has seen such a big change, leveraging the inherent benefits of the architecture. + +We hope you're as excited about this as we are. You've already taken the first step to adopt FOCUS and now you'll be able to decide when you're ready to take the next step to FOCUS 1.2 when and where you need it, while keeping all other reports and integrations steady on 1.0. Minimal impact, maximum potential. + +To learn more about managed datasets in FinOps hubs, see [FinOps hub data model](https://learn.microsoft.com/cloud-computing/finops/toolkit/hubs/data-model). And if you're looking for more, I'm working on a set of premium services designed to help organizations deploy, customize, and scale the FinOps hubs with confidence. Whether you need help getting started, tailoring the tools to your environment, or ensuring long-term success, these services are built to meet you where you are – strategic, secure, and ready to deliver value from day one. Connect with me directly on [LinkedIn](https://linkedin.com/in/flanakin) or Slack to learn more. diff --git a/announce/2025/2025-08_v12.md b/announce/2025/2025-08_v12.md new file mode 100644 index 000000000..79f616dae --- /dev/null +++ b/announce/2025/2025-08_v12.md @@ -0,0 +1,114 @@ +# What's new in FinOps toolkit 12 – July 2025 + +_Published: 2025-08 · [Original post](https://techcommunity.microsoft.com/blog/finopsblog/what%E2%80%99s-new-in-finops-toolkit-12-%E2%80%93-july-2025/4438562)_ + +Whether you consider yourself a FinOps practitioner, someone who's enthusiastic about driving cloud efficiency and maximizing the value you get from the cloud or were just asked to look at ways to reduce cost, the FinOps toolkit has something for you. This month, you'll find support for FOCUS 1.2, autostart in FinOps hubs which can reduce your hub costs, a new page in the Cost summary Power BI report, and various small fixes, improvements, and documentation updates across the board. Read on for details. + +## New to the FinOps toolkit? + +In case you haven't heard, the FinOps toolkit is an open-source collection of tools and resources that help you learn, adopt, and implement FinOps in the Microsoft Cloud. The foundation of the toolkit is the [Implementing FinOps guide](http://aka.ms/finops/guide) that helps you get started with FinOps whether you're using native tools in the Azure portal, looking for ways to automate and extend those tools, or if you're looking to build your own FinOps tools and reports. To learn more about the toolkit, how to provide feedback, or how to contribute, see [FinOps toolkit documentation](https://aka.ms/ftk/docs). + +## FinOps Open Cost and Usage Specification (FOCUS) version 1.2 + +As you may have seen, [Microsoft Cost Management released support for FOCUS 1.2](https://techcommunity.microsoft.com/blog/finopsblog/focus-1-2-in-microsoft-cost-management-unified-multi-cloud-multi-currency-report/4423852) last month. This month, you'll find updated FOCUS documentation, Power BI reports, FinOps hubs data ingestion, and a new FOCUS 1.2 metadata file. + +The FOCUS documentation provides a high-level overview and summary of what FOCUS is, why it's important, how to transition to it, and any conformance gaps between the Cost Management dataset and the official specification. These updates cover: + +- New columns, like ServiceSubcategory. +- Renamed columns, like InvoiceId, PricingCurrency, and SkuMeter. +- Future columns that are not yet populated, like CapacityReservationId, CapacityReservationStatus, CommitmentDiscountQuantity, CommitmentDiscountUnit, and SkuPriceDetails. + +If you're migrating reports from native Cost Management schemas to FOCUS, converting your own data, or comparing FOCUS data to native schemas, you can learn more with the following articles: + +- [Update reports to use FOCUS columns](https://learn.microsoft.com/cloud-computing/finops/focus/mapping) +- [Convert cost and usage data to FOCUS](https://learn.microsoft.com/cloud-computing/finops/focus/convert) +- [Validate FOCUS data](https://learn.microsoft.com/cloud-computing/finops/focus/validate) + +Power BI reports were also updated to account for the new and renamed columns. Storage reports include both new and old names with values copied to/from both, so no matter what you use, you'll continue to get the right values whether you're using FOCUS 1.0 or FOCUS 1.2 data. This includes: + +- InvoiceId and x_InvoiceId +- PricingCurrency and x_PricingCurrency +- SkuMeter and x_SkuMeterName + +KQL reports go a step further and provide some of the additional columns that are not yet provided in Cost Management FOCUS exports. This includes CapacityReservationId, CapacityReservationStatus, CommitmentDiscountQuantity, CommitmentDiscountUnit, SkuPriceDetails, and more. I'll cover this more in the next section. + +Lastly, FOCUS defines a metadata file that identifies and describes the columns included in a FOCUS dataset. You can download a [JSON version of these metadata files](https://github.com/microsoft/finops-toolkit/releases/latest/download/dataset-metadata.zip) or learn more in [Details about the FOCUS dataset](https://learn.microsoft.com/cloud-computing/finops/focus/metadata). + +## New FinOps hubs v1_2 schema + +Data is at the heart of every FinOps practice. Whether you're a hobbyist or part of a large team, you can't "FinOps" without data. And when you start to "FinOps" at scale, you realize you need a lot of different data from a lot of different data sources. And these data sources often have different characteristics and change over time. Luckily, FinOps hubs were designed to handle ever-evolving datasets through it's "schema version" concept. + +FinOps hubs schema versions are contracts you can rely on to always present a consistent version of the data. Today, FinOps hubs use a "v1_0" schema version that aligns with FOCUS 1.0 (hence the name). As of FinOps hubs 12, you will see a new "v1_2" schema version, which aligns with FOCUS 1.2. And you'll notice that your data is populated in the FOCUS 1.2 format regardless of what dataset version you exported from Cost Management. And the best part is, this optimization won't impact you at all as long as your reports and integrations are setup to use the versioned functions (e.g., Costs_v1_0). If you find you have code using non-versioned functions (e.g., Costs), simply switch to the versioned function to get all data – old and new – using the previous v1_0 schema version. + +To learn more, see [Introducing non-breaking "breaking" changes in FinOps hubs 12](https://techcommunity.microsoft.com/blog/finopsblog/introducing-non-breaking-%E2%80%9Cbreaking%E2%80%9D-changes-in-finops-hubs-12/4438554). + +## Unlocking broader compatibility with flexible CIDR support + +When setting up private endpoints in FinOps hubs, defining the right virtual network (VNet) configuration is essential to ensure secure and scalable connectivity. A key part of this setup involves specifying a CIDR (Classless Inter-Domain Routing) block, which determines the size and structure of the IP address space within the VNet. Based on community feedback, FinOps hubs expanded CIDR validation to support any subnet size ranging from /8 to /26 for maximum flexibility. (/26 is the minimum required to support native FinOps hubs functionality.) + +This update reflects a more flexible and inclusive approach to network planning. By allowing a wider range of CIDR sizes, FinOps hubs better accommodate diverse deployment scenarios, such as those involving Power BI VNet data gateways that require larger address spaces. This change not only resolves upgrade blockers but also empowers you to tailor your deployments to meet your unique needs and without unnecessary constraints. + +## Balance cost and freshness with Data Explorer auto-start + +One of the most effective ways to reduce cloud spend is by scheduling workloads to run only when needed, and Azure Data Explorer clusters are a prime candidate for this strategy. Data Explorer powers ingestion and analysis, but usage is billed hourly, which means you're being charged for it even when not actively querying or ingesting data. To optimize this, you can configure Data Explorer to shut down automatically after a period of inactivity, but historically, this has introduced a problem because the cluster was shut down when new data needs to be ingested. With FinOps hubs 12, the ingestion pipeline automatically checks the status of the Data Explorer cluster and starts it if it's stopped when new data arrives. When initiating the Data Explorer cluster, the startup process typically takes approximately 10 minutes. While this delay incurs pipeline execution costs, it remains significantly more cost-effective than maintaining a continuously running cluster (24x7). During this startup phase, data ingestion tasks are paused until the cluster is successfully provisioned. + +To get started with scheduling Data Explorer, see [Automatic stop of inactive Azure Data Explorer clusters](https://learn.microsoft.com/azure/data-explorer/auto-stop-clusters). + +## Strengthened security with flexible export permissions + +FinOps hubs leverage Cost Management exports, which run on a predetermined schedule that can't be set or changed. But some organizations are looking for more control over when exports run and data gets ingested into FinOps hubs. This is where managed exports come in. Managed exports automate the delivery of Cost Management data, enabling seamless and timely integration with analytics pipelines and reporting tools. Managed exports work by granting your FinOps hub instance access to create and run exports on your behalf. The challenge here is that Cost Management requires the User Access Administrator role to create exports, which is an elevated permission that some organizations restrict due to security policies. To increase the default security posture of FinOps hubs, managed exports are now disabled by default in FinOps hubs 12. If you'd like to use managed exports, simply check the **Enable managed exports** checkbox on the Advanced tab (or the set the **enableManagedExports** template parameter to **true**). + +By leaving this option disabled or explicitly setting the parameter to **false**, you can now deploy FinOps hubs without requiring elevated permissions for role assignments. This makes it easier to deploy in secure or restricted environments while still maintaining core functionality. It's a small but powerful change that simplifies onboarding and aligns with enterprise-grade security practices. + +## New executive summary in the Cost summary report + +Understanding where your cloud spend is coming from is critical – not just for FinOps practitioners, but for executive stakeholders as well. That's why the FinOps toolkit 12 release introduces a new **Summary** page in the Cost summary report, designed specifically to provide a high-level, executive-friendly view of cost drivers across your environment. + +This new page highlights the top 10 subscriptions, resource groups, and service categories by spend for the selected period, making it easy to identify where costs are concentrated. It also includes a breakdown of monthly usage by pricing category and a daily spend trend for the last 30 days to facilitate quick pattern recognition and anomaly detection at a glance. To make room for this new executive summary, the original summary page has been renamed to **Running total**, which continues to provide a cumulative view of costs over time. Together, these updates offer a more complete and accessible picture of your cloud financials for both technical and non-technical audiences. + +## PowerShell updates for Cost Management exports + +To ensure compatibility with the latest features and improvements in Microsoft Cost Management, the FinOps toolkit PowerShell commands for Cost Management exports were all updated to default to the latest GA API version, 2025-03-01. This keeps the toolkit aligned with the most recent capabilities and ensures long-term support for evolving services. + +In addition, the default export version for FOCUS exports has been updated to 1.2-preview in the [New-FinOpsCostExport command](https://aka.ms/ftk/New-FinOpsCostExport). This version includes the latest industry and data quality improvements, helping teams get more value from their exported cost data. These updates make it easier to stay aligned with the latest standards while reducing the need for manual configuration. If you need an older dataset version, simply set the -DatasetVersion parameter to the desired version. + +## New versioning strategy for a growing toolkit + +What started as a modest effort to simplify FinOps operations with a single tool has grown into a suite of nine tools, each addressing a different aspect of FinOps. While we've been driving towards a "1.0" release of FinOps hubs, the FinOps toolkit scope has grown, but our approach to versioning hasn't. With this release, we're shifting to a versioning strategy that better represents the ever-growing nature of the toolkit as a collection of tools. Instead of "0.12" which only represents the state of a single tool, we're dropping the "0." to reflect the collective progress as version "12" of the toolkit as a whole. + +## Other new and noteworthy updates + +Many small improvements and bug fixes go into each release, so covering everything in detail can be a lot to take in. But I do want to call out a few other small things that you may be interested in. + +**In [FinOps hubs](https://aka.ms/finops/hubs):** + +- Document how savings are calculated and how to troubleshoot negative savings. +- Updated the Data Explorer dashboard to use the v1_2 schema. + +**In [Power BI reports](https://aka.ms/ftk/pbi):** + +- Added an invoice ID filter to the Microsoft Customer Agreement (MCA) invoice recon page in the Invoicing and chargeback report. +- Updated KQL reports to use the FinOps hubs v1_2 schema. +- Correctly detect the hub version in the Data ingestion report. + +**In [Azure optimization engine](https://aka.ms/ftk/aoe):** + +- Link to recommendations Power BI report in MS Learn documentation (#1614). +- Aka.ms links to point to the latest version of documentation articles in MS Learn. + +**In [open data](https://aka.ms/ftk/data):** + +- Added 20 new and updated 7 existing resource types. +- Added 2 new service mappings. + +## What's next + +Despite many changes in Microsoft, I'm more excited than ever about what's coming. The FinOps toolkit has grown into a vibrant community of contributors and users, all passionate about making FinOps more accessible, scalable, and impactful. And we're just getting started. + +Looking ahead, you can expect continued innovation across the toolkit: + +- FinOps hubs will add recommendations, a new extensibility model, and continued data quality improvements. +- Power BI and FinOps workbooks will continue to expand with recurring updates and deeper FinOps insights. +- Azure Optimization Engine will evolve as its capabilities are integrated into FinOps hubs. + +To support this momentum, I'm also working on a set of premium services designed to help organizations deploy, customize, and scale the FinOps toolkit with confidence. Whether you need help getting started, tailoring the tools to your environment, or ensuring long-term success, these services are built to meet you where you are – strategic, secure, and ready to deliver value from day one. Connect with me directly on [LinkedIn](https://linkedin.com/in/flanakin) or Slack to learn more. diff --git a/announce/2026/2026-01_azure-openai-costs.md b/announce/2026/2026-01_azure-openai-costs.md new file mode 100644 index 000000000..1db07879d --- /dev/null +++ b/announce/2026/2026-01_azure-openai-costs.md @@ -0,0 +1,208 @@ +# Managing Azure OpenAI costs with the FinOps toolkit and FOCUS: Turning tokens into unit economics + +_Published: 2026-01-22 · By Robb Dilallo · [Original post](https://techcommunity.microsoft.com/blog/finopsblog/managing-azure-openai-costs-with-the-finops-toolkit-and-focus-turning-tokens-int/4413886)_ + +## Introduction + +As organizations rapidly adopt generative AI, Azure OpenAI usage is growing—and so are the complexities of managing its costs. Unlike traditional cloud services billed per compute hour or storage GB, Azure OpenAI charges based on **token usage**. + +For FinOps practitioners, this introduces a new frontier: understanding _AI unit economics_ and managing costs where the consumed unit is a token. + +This article explains how to leverage the **Microsoft FinOps toolkit** and the **FinOps Open Cost and Usage Specification (FOCUS)** to gain visibility, allocate costs, and calculate unit economics for Azure OpenAI workloads. + +## Why Azure OpenAI cost management is different + +AI services break many traditional cost management assumptions: + +- **Billed by token usage** (input + output tokens). +- **Model choices matter** (e.g., GPT-3.5 vs. GPT-4 Turbo vs. GPT-4o). +- **Prompt engineering impacts cost** (longer context = more tokens). +- **Bursty usage patterns** complicate forecasting. + +Without proper visibility and unit cost tracking, it's difficult to optimize spend or align costs to business value. + +## Step 1: Get visibility with the FinOps toolkit + +The **Microsoft FinOps toolkit** provides pre-built modules and patterns for analyzing Azure cost data. + +**Key tools include:** + +- **Microsoft Cost Management exports** — Export daily usage and cost data in a FOCUS-aligned format. +- **FinOps hubs** — Infrastructure-as-Code solution to ingest, transform, and serve cost data. +- **Power BI templates** — Pre-built reports conformed to FOCUS for easy analysis. + +**Pro tip:** Start by connecting your Microsoft Cost Management exports to a FinOps hub. Then, use the toolkit's Power BI FOCUS templates to begin reporting. + +[Learn more about the FinOps toolkit](https://github.com/microsoft/finops-toolkit) + +## Step 2: Normalize data with FOCUS + +The **FinOps Open Cost and Usage Specification (FOCUS)** standardizes billing data across providers—including Azure OpenAI. + +| FOCUS Column | Purpose | Azure Cost Management Field | +|---|---|---| +| ServiceName | Cloud service (e.g., Azure OpenAI Service) | ServiceName | +| ConsumedQuantity | Number of tokens consumed | Quantity | +| PricingUnit | Unit type, should align to "tokens" | DistinctUnits | +| BilledCost | Actual cost billed | CostInBillingCurrency | +| ChargeCategory | Identifies consumption vs. reservation | ChargeType | +| ResourceId | Links to specific deployments or apps | ResourceId | +| Tags | Maps usage to teams, projects, or environments | Tags | +| UsageType / Usage Details | Further SKU-level detail | Sku Meter Subcategory, Sku Meter Name | + +**Why it matters:** Azure's native billing schema can vary across services and time. FOCUS ensures consistency and enables cross-cloud comparisons. + +_Tip:_ If you use custom deployment IDs or user metadata, apply them as **tags** to improve allocation and unit economics. + +[Review the FOCUS specification](https://focus.finops.org) + +## Step 3: Calculate unit economics + +Unit cost per token = BilledCost ÷ ConsumedQuantity + +### Real-world example: Calculating unit cost in Power BI + +A recent Power BI report breaks down Azure OpenAI usage by: + +- **SKU Meter Category** → e.g., _Azure OpenAI_ +- **SKU Meter Subcategory** → e.g., _gpt 4o 0513 Input global Tokens_ +- **SKU Meter Name** → detailed SKU info (input/output, model version, etc.) + +| GPT Model | Usage Type | Effective Cost | +|---|---|---| +| gpt 4o 0513 Input global Tokens | Input | $292.77 | +| gpt 4o 0513 Output global Tokens | Output | $23.40 | + +Unit Cost Formula: `Unit Cost = EffectiveCost ÷ ConsumedQuantity` + +Power BI Measure Example: `Unit Cost = SUM(EffectiveCost) / SUM(ConsumedQuantity)` + +**Pro tip:** Break out input and output token costs by model version to: + +- Track which workloads are driving spend. +- Benchmark cost per token across GPT models. +- Attribute costs back to teams or product features using Tags or ResourceId. + +### Power BI tip: Building a GPT cost breakdown matrix + +To easily calculate token unit costs by GPT model and usage type, build a **Matrix visual** in Power BI using this hierarchy: + +**Rows:** + +- SKU Meter Category +- SKU Meter Subcategory +- SKU Meter Name + +**Values:** + +- EffectiveCost (sum) +- ConsumedQuantity (sum) +- Unit Cost (calculated measure) + +`Unit Cost = SUM('Costs'[EffectiveCost]) / SUM('Costs'[ConsumedQuantity])` + +**Hierarchy Example:** + +``` +Azure OpenAI + ├── GPT 4o Input global Tokens + ├── GPT 4o Output global Tokens + ├── GPT 4.5 Input global Tokens + └── etc. +``` + +### What you can see at the token level + +| Metric | Description | Data Source | +|---|---|---| +| **Token Volume** | Total tokens consumed | Consumed Quantity | +| **Effective Cost** | Actual billed cost | BilledCost / Cost | +| **Unit Cost per Token** | Cost divided by token quantity | Effective Unit Price | +| **SKU Category & Subcategory** | Model, version, and token type (input/output) | Sku Meter Category, Subcategory, Meter Name | +| **Resource Group / Business Unit** | Logical or organizational grouping | Resource Group, Business Unit | +| **Application** | Application or workload responsible for usage | Application (tag) | + +This visibility allows teams to: + +- Benchmark cost efficiency across GPT models. +- Track token costs over time. +- Allocate AI costs to business units or features. +- Detect usage anomalies and optimize workload design. + +_Tip:_ Apply consistent tagging (Cost Center, Application, Environment) to Azure OpenAI resources to enhance allocation and unit economics reporting. + +### How the FinOps Foundation's AI working group informs this approach + +The **FinOps for AI overview**, developed by the [FinOps Foundation's AI working group](https://www.finops.org/wg/finops-for-ai-overview), highlights unique challenges in managing AI-related cloud costs, including: + +- Complex cost drivers (tokens, models, compute hours, data transfer). +- Cross-functional collaboration between Finance, Engineering, and ML Ops teams. +- The importance of tracking **AI unit economics** to connect spend with value. + +By combining the **FinOps toolkit**, **FOCUS-conformed data**, and **Power BI reporting**, practitioners can implement many of the AI Working Group's recommendations: + +- Establish **token-level unit cost metrics**. +- Allocate costs to **teams, models, and AI features**. +- Detect cost anomalies specific to AI usage patterns. +- Improve forecasting accuracy despite AI workload variability. + +_Tip:_ Applying consistent tagging to AI workloads (model version, environment, business unit, and experiment ID) significantly improves cost allocation and reporting maturity. + +## Step 4: Allocate and report costs + +With FOCUS + FinOps toolkit: + +- **Allocate** costs to teams, projects, or business units using Tags, ResourceId, or custom dimensions. +- **Showback/Chargeback** AI usage costs to stakeholders. +- **Detect anomalies** using the toolkit's patterns or integrate with Azure Monitor. + +**Tagging tip:** Add metadata to Azure OpenAI deployments for easier allocation and unit cost reporting. Example: + +```yaml +tags: + CostCenter: AI-Research + Environment: Production + Feature: Chatbot +``` + +## Step 5: Iterate using FinOps best practices + +| FinOps capability | Relevance | +|---|---| +| Reporting & analytics | Visualize token costs and trends | +| Allocation | Assign costs to teams or workloads | +| Unit economics | Track cost per token or business output | +| Forecasting | Predict future AI costs | +| Anomaly management | Identify unexpected usage spikes | + +Start small (**Crawl**), expand as you mature (**Walk → Run**). + +[Learn about the FinOps Framework](https://www.finops.org/framework) + +## Next steps + +Ready to take control of your Azure OpenAI costs? + +1. **Deploy the Microsoft FinOps toolkit** — Start ingesting and analyzing your Azure billing data. [Get started](https://aka.ms/finops/toolkit) +2. **Adopt FOCUS** — Normalize your cost data for clarity and cross-cloud consistency. [Explore FOCUS](https://aka.ms/finops/focus) +3. **Calculate AI unit economics** — Track token consumption and unit costs using Power BI. +4. **Customize Power BI reports** — Extend toolkit templates to include token-based unit economics. +5. **Join the conversation** — Share insights or questions with the [FinOps community on TechCommunity](https://aka.ms/finops/discuss) or in the FinOps Foundation Slack. +6. **Advance your skills** — Consider the [FinOps Certified FOCUS Analyst certification](https://learn.finops.org/page/focus). + +## Further reading + +- [Managing the cost of AI: Understanding AI workload cost considerations](https://techcommunity.microsoft.com/blog/finopsblog/understanding-ai-workload-cost-considerations/4400844) +- [Microsoft FinOps toolkit](https://aka.ms/finops/toolkit) +- [Learn about FOCUS](https://aka.ms/finops/focus) +- [Microsoft Cost Management + Billing](https://learn.microsoft.com/azure/cost-management-billing/) +- [FinOps Foundation](https://www.finops.org) + +## Appendix: FOCUS column glossary + +- **ConsumedQuantity** — The number of tokens or units consumed for a given SKU. This is the key measure of usage. +- **ConsumedUnit** — The type of unit being consumed, such as 'tokens', 'GB', or 'vCPU hours'. Often appears as 'Units' in Azure exports for OpenAI workloads. +- **PricingUnit** — The unit of measure used for pricing. Should match 'ConsumedUnit', e.g., 'tokens'. +- **EffectiveCost** — Final cost after amortization of reservations, discounts, and prepaid credits. Often derived from billing data. +- **BilledCost** — The invoiced charge before applying commitment discounts or amortization. +- **PricingQuantity** — The volume of usage after applying pricing rules such as tiered or block pricing. Used to calculate cost when multiplied by unit price. diff --git a/announce/2026/2026-02_v13.md b/announce/2026/2026-02_v13.md new file mode 100644 index 000000000..5ee62d111 --- /dev/null +++ b/announce/2026/2026-02_v13.md @@ -0,0 +1,79 @@ +# What's new in FinOps toolkit 13 – January 2026 + +_Published: 2026-02-09 · [Original post](https://techcommunity.microsoft.com/blog/finopsblog/what%E2%80%99s-new-in-finops-toolkit-13-%E2%80%93-january-2026/4493090)_ + +Whether you consider yourself a FinOps practitioner, someone enthusiastic about driving cloud efficiency, or were tasked with reducing costs, the FinOps toolkit has something for you. This release emphasizes stability and usability improvements across FinOps hubs, Power BI reports, and Azure Optimization Engine, with key additions including Key Vault purge protection configuration and Parquet format support for Cost Management exports via PowerShell. + +## New to the FinOps toolkit? + +The FinOps toolkit is an open-source collection of tools and resources helping organizations learn, adopt, and implement FinOps in the Microsoft Cloud. The foundation includes the [Implementing FinOps guide](http://aka.ms/finops/guide) for getting started with native Azure portal tools or custom FinOps solutions. Learn more at the [FinOps toolkit documentation](https://aka.ms/ftk/docs). + +## Faster troubleshooting with expanded documentation + +This release addresses documentation gaps through several improvements: + +- New [Data Lake Storage connectivity guide](https://github.com/microsoft/finops-toolkit/blob/dev/docs-mslearn/toolkit/data-lake-storage-connectivity.md) for connecting FinOps hubs data to tools beyond Power BI +- Updated [Configure scopes](https://github.com/microsoft/finops-toolkit/blob/dev/docs-mslearn/toolkit/hubs/configure-scopes.md) documentation clarifying support for multiple Azure scopes and cross-cloud data ingestion +- New guidance on [removing private networking](https://github.com/microsoft/finops-toolkit/blob/dev/docs-mslearn/toolkit/hubs/private-networking.md#removing-private-networking) to reduce costs +- Expanded troubleshooting guidance for common Data Explorer ingestion errors +- Power BI report documentation clarifying required Cost Management exports +- Updated FOCUS converter documentation covering newly added 1.2-preview fields: ServiceSubcategory, InvoiceId, PricingCurrency, and SkuMeter + +[Share community tips](https://aka.ms/ftk/discuss) to help others and provide feedback. + +## Join us for FinOps toolkit office hours + +Community engagement is central to the FinOps toolkit's success. The team is launching office hours to strengthen this connection. + +FinOps toolkit office hours occur every other Wednesday from 8:30–9:00 AM Pacific (4:30–5:00 PM UTC). Drop in for deployment help, discuss feature requests, share feedback, or connect with others. No agenda or preparation required; all experience levels welcome. + +[Join the next session](https://aka.ms/ftk/officehours) or [add it to your calendar](https://github.com/microsoft/finops-toolkit/releases/latest/download/office-hours.ics). + +## Scaling FinOps hubs to meet enterprise demands + +As adoption grows, so do enterprise demands. This release addresses compliance, reliability, and extensibility needs. + +**Compliance:** Organizations with Azure Landing Zone policies requiring Key Vault purge protection can now use a new optional **enablePurgeProtection** parameter, enabling it when needed while keeping it off by default for development and testing flexibility. The release also tightens security by replacing the User Access Administrator role with the more restrictive RBAC Administrator role, scoped specifically to the Managed Exports app. + +**Reliability:** Improvements address pipeline issues causing deployment failures or data inconsistencies, including ADF triggers not starting after deployment, InitializeHub pipeline failures, and duplicate records in certain scenarios. + +**Extensibility:** The release reorganizes Bicep templates into separate application modules, laying groundwork for more flexible deployment options. The vision is to "deploy only the components you need, extend with your own, and integrate more seamlessly with existing infrastructure." + +[Start a discussion](https://aka.ms/ftk/discuss) to suggest new extensions. + +## Streamline export automation with Parquet + +Cost Management exports can grow large with complex environments or long retention periods. CSV files work but are verbose and slow at scale. The FinOps toolkit recommends Parquet exports with snappy compression for fastest downloads and ingestion. + +The PowerShell module now supports Parquet format through a new **-Format** parameter in the [New-FinOpsCostExport](https://learn.microsoft.com/azure/cost-management-billing/finops/toolkit/powershell/cost/new-finopscostexport) command. Parquet files are columnar, compact, and significantly faster to query in Azure Data Explorer, Synapse, and Fabric. A new **-CompressionMode** parameter provides gzip and snappy compression control to further reduce storage costs and transfer times. + +Thanks to [gorkomikus](https://github.com/gorkomikus) on GitHub for contributing this feature. + +## Other new and noteworthy updates + +**In Power BI reports:** + +- Fixed tag expansion when tag names contain special characters like colons +- Fixed unattached disks count in the workload optimization report to show only truly unattached disks +- Fixed "Number of Months" parameter calculation that was excluding the first 5 days of data +- Fixed EA department scope failing on pricesheet export by skipping unsupported scopes + +**In FinOps workbooks:** + +- Fixed SQL Managed Instance vCores displaying incorrect values in the AHB workbook + +**In Azure optimization engine:** + +- Changed default SQL database backup redundancy to LRS for improved cost efficiency and compatibility with non-paired Azure regions +- Fixed reservations-related workbooks by replacing Instance Size Flexibility ratios CSV vanity URL +- Fixed underutilized disks recommendations not being generated for Premium SSD V2 disks + +## What's next + +Looking ahead, expect continued innovation: + +- AI automation on top of Cost Management and FinOps hubs offering more options for getting answers +- FinOps hubs will add recommendations, a new extensibility model, and continued data quality improvements +- Azure Optimization Engine will evolve as capabilities integrate into FinOps hubs + +Additionally, premium paid services are in development to help organizations deploy, customize, and scale the FinOps toolkit with confidence. Connect on [LinkedIn](https://linkedin.com/in/flanakin) or [FinOps Foundation Slack](https://www.finops.org/community/) to learn more. diff --git a/announce/README.md b/announce/README.md new file mode 100644 index 000000000..b4ffddf3c --- /dev/null +++ b/announce/README.md @@ -0,0 +1,26 @@ +# Announcements + +FinOps toolkit blog posts and announcements published to the [FinOps Tech Community blog](https://techcommunity.microsoft.com/tag/finops%20toolkit?nodeId=board%3AFinOpsBlog) and other channels. Used as a reference for tone, structure, and what's already been published. + +## Folder layout + +Files are organized by year, named `YYYY-MM_kebab-cased-name.md`. The name is usually the version (e.g., `v14`) for release announcements or a short topic for everything else. + +``` +announce/ +├── 2025/ +│ └── 2025-08_v12.md +└── 2026/ + ├── 2026-01_azure-openai-costs.md + └── 2026-02_v13.md +``` + +## Creating a new announcement + +Run `/announce` in Claude Code. Pass a topic (e.g., `/announce v14 release`) or invoke without arguments to have Claude scan the [changelog](../docs-mslearn/toolkit/changelog.md) and [recent blog posts](https://techcommunity.microsoft.com/tag/finops%20toolkit?nodeId=board%3AFinOpsBlog) to suggest the top candidates that haven't been announced yet. + +The command walks through three phases: + +1. **Discovery** — gather context from the changelog, MS Learn docs, and code (in that order), asking clarifying questions only when needed. +2. **Scoping** — propose an outline. Release announcements use a fixed structure documented in the command; other posts adapt based on the topic. +3. **Drafting** — write the post under `announce/{year}/`, including a short social media blurb appendix that links back to the published post.