From 6dd5ce435b1abd579b9012b2cb592c9594a4a9ed Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Fri, 1 Aug 2025 11:27:18 -0700 Subject: [PATCH 01/69] Fix workbooks deployment permissions documentation (#1629) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: KevDLR <86437159+KevDLR@users.noreply.github.com> Co-authored-by: flanakin <399533+flanakin@users.noreply.github.com> Co-authored-by: Michael Flanakin --- .../toolkit/workbooks/customize-workbooks.md | 8 ++++++-- .../workbooks/finops-workbooks-overview.md | 20 +++++++++++++++---- docs/workbooks.md | 4 +++- src/templates/finops-workbooks/README.md | 5 ++++- 4 files changed, 29 insertions(+), 8 deletions(-) diff --git a/docs-mslearn/toolkit/workbooks/customize-workbooks.md b/docs-mslearn/toolkit/workbooks/customize-workbooks.md index 4194fec33..d835620a5 100644 --- a/docs-mslearn/toolkit/workbooks/customize-workbooks.md +++ b/docs-mslearn/toolkit/workbooks/customize-workbooks.md @@ -45,8 +45,12 @@ If you want to make modifications to the Cost optimization workbook or use other First, confirm you have the following least-privileged roles to deploy and use the workbook. -- [Workbook Contributor](/azure/role-based-access-control/built-in-roles#workbook-contributor) - allows you to import, save, and deploy the workbook. -- [Reader](/azure/role-based-access-control/built-in-roles#reader) allows you to view all the workbook tabs without saving. +- **Contributor** or a role with both `Microsoft.Resources/deployments/validate/action` and `Microsoft.Resources/deployments/write` permissions is required for ARM template deployments. +- [Workbook Contributor](/azure/role-based-access-control/built-in-roles#workbook-contributor) on the target resource group allows you to edit and save the workbook after deployment. +- [Reader](/azure/role-based-access-control/built-in-roles#reader) is required on all subscriptions you will monitor to access resource information. + +> [!NOTE] +> If you only have Reader, you can create a new workbook and upload the `workbook.json` file to view and edit it, but you won't be able to save it. You still need Reader access to all subscriptions that you want to monitor. FinOps workbooks use multiple `workbook.json` files, which you can find in the `workbooks` folder of the FinOps workbooks download for the [latest release](https://aka.ms/ftk/latest). Deploy the FinOps workbooks template with one of the following options: diff --git a/docs-mslearn/toolkit/workbooks/finops-workbooks-overview.md b/docs-mslearn/toolkit/workbooks/finops-workbooks-overview.md index aca3346c9..7d675f71b 100644 --- a/docs-mslearn/toolkit/workbooks/finops-workbooks-overview.md +++ b/docs-mslearn/toolkit/workbooks/finops-workbooks-overview.md @@ -27,13 +27,16 @@ This template includes the following workbooks: 1. To deploy and use the workbook, confirm you have the following least-privileged roles: - - **Workbook Contributor** allows you to deploy the workbook. - - **Reader** view all of the workbook tabs. + - **Contributor** or a role with both `Microsoft.Resources/deployments/validate/action` and `Microsoft.Resources/deployments/write` permissions is required for ARM template deployments. + - [Workbook Contributor](/azure/role-based-access-control/built-in-roles#workbook-contributor) on the target resource group allows you to edit and save the workbook after deployment. + - [Reader](/azure/role-based-access-control/built-in-roles#reader) is required on all subscriptions you will monitor to access resource information. > [!NOTE] - > If you only have read access, you can still import your workbook directly into Azure Monitor. You will not be able to save it, but you can view all tabs. + > If you only have read access, you can still import your workbook directly into Azure Monitor. You will not be able to save it, but you can view all tabs. You can download the workbook JSON files from the FinOps workbooks download in the [latest release](https://aka.ms/ftk/latest). -2. Deploy the **finops-workbooks** template. +2. Deploy the **finops-workbooks** template using one of the following methods: + + **Option 1:** Deploy using ARM template (requires Microsoft.Resources/deployments permissions, Workbook Contributor, and Reader) Deploy To Azure   @@ -43,6 +46,15 @@ This template includes the following workbooks: Deploy To Azure China --> + **Option 2:** Import JSON files directly (works with Reader access) + + 1. Download FinOps workbooks from the [latest release](https://aka.ms/ftk/latest). + 2. Navigate to [Azure Monitor Workbooks](https://portal.azure.com/#view/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/~/workbooks) in the Azure portal + 3. Click on **+ New** and select **Advanced editory** + 4. Copy the text from the desired workbook.json from the downloaded ZIP file, paste it into the editor, and select **Apply**. + 5. Select **Done editing** to view the workbook. + 6. Repeat steps 3-5 for each workbook. +
## Give feedback diff --git a/docs/workbooks.md b/docs/workbooks.md index 3df52e0ee..296cad816 100644 --- a/docs/workbooks.md +++ b/docs/workbooks.md @@ -67,7 +67,9 @@ FinOps workbooks are Azure workbooks that provide a series of tools to help engi ## Deploy FinOps workbooks -FinOps workbooks require the Workbook Contributor role to deploy and Reader to view all tabs. +FinOps workbooks require the Contributor role or a role with both Microsoft.Resources/deployments/validate/action and Microsoft.Resources/deployments/write permissions for ARM template deployments, Workbook Contributor role to save changes, and Reader on all subscriptions you want to monitor. + +> If you only have Reader access, you can download the workbook JSON files from the finops-workbooks.zip package available in the [latest release](https://aka.ms/ftk/latest), and then import them directly into Azure Monitor Workbooks.
Deploy to Azure diff --git a/src/templates/finops-workbooks/README.md b/src/templates/finops-workbooks/README.md index 1d6ca8044..a5e2fb1c1 100644 --- a/src/templates/finops-workbooks/README.md +++ b/src/templates/finops-workbooks/README.md @@ -38,7 +38,10 @@ To learn more about FinOps workbooks, the roadmap, or how to contribute , see [F ## 📋 Prerequisites -Azure Monitor workbooks provide direct access to Azure resource details. To deploy workbooks, you must have the **Workbook Contributor** role. To use workbooks, you need read access to the resources being monitored. The exact permissions vary by resource and service. +To deploy workbooks using ARM templates, you must have: +- **Contributor** role or a role with both `Microsoft.Resources/deployments/validate/action` and `Microsoft.Resources/deployments/write` permissions. +- **Workbook Contributor** role to edit and save workbooks. +- **Reader** role on all subscriptions you want to monitor. If you run into any issues, see [Troubleshooting FinOps toolkit solutions](https://aka.ms/ftk/tsg). From ef76aa78fb80f40f00865c958a529f9ff25ff163 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Fri, 1 Aug 2025 22:55:56 -0700 Subject: [PATCH 02/69] Add guidance to PS documentation for connecting to Power BI Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com> --- docs-mslearn/toolkit/power-bi/setup.md | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/docs-mslearn/toolkit/power-bi/setup.md b/docs-mslearn/toolkit/power-bi/setup.md index 1006f858c..4a712860d 100644 --- a/docs-mslearn/toolkit/power-bi/setup.md +++ b/docs-mslearn/toolkit/power-bi/setup.md @@ -18,6 +18,20 @@ The FinOps toolkit Power BI reports provide a great starting point for your FinO FinOps toolkit reports support several ways to connect to your cost data. We generally recommend starting with Cost Management exports, which support up to $2-5 million in monthly spend. If you experience data refresh timeouts or need to report on data across multiple directories or tenants, use [FinOps hubs](../hubs/finops-hubs-overview.md). It's a data pipeline solution that optimizes data and offers more functionality. For more information about choosing the right backend, see [Help me choose](help-me-choose.md). +## Datasets and compatible tools + +The following table shows the different datasets available and which reports and tools can access them: + +| Dataset | Description | Compatible reports and tools | Notes | +|---------|-------------|------------------------------|-------| +| Cost Management exports | Raw cost and usage data exported to Azure Data Lake Storage Gen2 | • Power BI storage reports
• Custom Power BI reports
• Direct storage access | Recommended for accounts with less than $2M monthly spend | +| FinOps hubs + Data Explorer (ADX) | Cost data processed and stored in Azure Data Explorer clusters | • Power BI KQL reports
• Data Explorer dashboards
• Azure Monitor workbooks
• Direct KQL queries
• Custom applications via Kusto API | Recommended for accounts with more than $100K total spend | +| FinOps hubs + Microsoft Fabric RTI | Cost data processed and stored in Microsoft Fabric Real-Time Intelligence | • Power BI KQL reports
• Fabric Real-Time dashboards
• Direct KQL queries
• Custom applications via Kusto API | Best performance and capabilities option | +| Microsoft Fabric OneLake | Raw exports stored in Microsoft Fabric OneLake | • Custom Fabric notebooks
• Custom Power BI reports
• Fabric data pipelines | For organizations already using Microsoft Fabric | +| Cost Management connector | Direct Power BI connection to Cost Management APIs | • Cost Management connector Power BI reports | Deprecated - not recommended for new implementations | + +Each dataset offers different capabilities and is optimized for different use cases. For detailed guidance on choosing the right option, see [Help me choose](help-me-choose.md). + Support for the [Cost Management connector for Power BI](/power-bi/connect-data/desktop-connect-azure-cost-management) is available for backwards compatibility, but isn't recommended. The Microsoft Cost Management team is no longer updating the Cost Management connector and instead recommends exporting data. Use following information to connect and customize FinOps toolkit and other Power BI reports.
@@ -42,7 +56,9 @@ The FinOps toolkit Power BI reports include preconfigured visuals, but aren't co - On the **Overview** page, under **Properties**, confirm **Access tier** is set to **Hot**. - If not, select the link and change the access tier to "Hot". - Other access tiers have not been tested and are not recommended due to the performance impact. -2. Configure Cost Management exports for any data you would like to include in reports, including: +2. Configure Cost Management exports for any data you would like to include in reports. + + You can create exports manually in the Azure portal or programmatically using the [`New-FinOpsCostExport`](../powershell/cost/new-finopscostexport.md) PowerShell command. Include the following datasets: | Dataset | Version | Notes | | --------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------- | From 00ca9efa652c93ffe1c238d0665fd54833682799 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Fri, 1 Aug 2025 23:22:38 -0700 Subject: [PATCH 03/69] [Hubs] Add billing account ID finding instructions to configure scopes documentation (#1785) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com> --- docs-mslearn/toolkit/hubs/configure-scopes.md | 78 +++++++++++++++---- 1 file changed, 61 insertions(+), 17 deletions(-) diff --git a/docs-mslearn/toolkit/hubs/configure-scopes.md b/docs-mslearn/toolkit/hubs/configure-scopes.md index 0582b05fb..8456573bb 100644 --- a/docs-mslearn/toolkit/hubs/configure-scopes.md +++ b/docs-mslearn/toolkit/hubs/configure-scopes.md @@ -12,11 +12,12 @@ ms.reviewer: micflan --- + # Configure scopes Connect FinOps hubs to your billing accounts and subscriptions by configuring Cost Management exports manually or granting FinOps hubs access to manage exports for you. -FinOps hubs use Cost Management exports to import cost data for the billing accounts and subscriptions you want to monitor. You can either configure Cost Management exports manually or grant FinOps hubs access to manage exports for you. +FinOps hubs use Cost Management exports to import cost data for the billing accounts and subscriptions you want to monitor. You can either configure Cost Management exports manually or grant FinOps hubs access to manage exports for you. For information about identifying your billing account and scope IDs, see [Find your billing account and scope IDs](#find-your-billing-account-and-scope-ids).
@@ -33,12 +34,57 @@ This walkthrough will trigger the following indirect costs:
+## Find your billing account and scope IDs + +Before configuring exports, you need to identify the billing account and scope IDs you want to monitor. The specific ID format depends on your billing account type. + +### Enterprise Agreement (EA) accounts + +For EA accounts, you need your enrollment number (billing account ID): + +1. Sign in to the [Azure portal](https://portal.azure.com). +2. Search for and select **Cost Management + Billing**. +3. Select **Billing scopes** from the left menu. +4. Select your billing account. +5. On the **Properties** page, copy the **Billing account ID** (enrollment number). + - Use this format for billing account scope: `/providers/Microsoft.Billing/billingAccounts/{enrollment-number}` + - For departments, append the department ID: `/providers/Microsoft.Billing/billingAccounts/{enrollment-number}/departments/{department-id}` + +### Microsoft Customer Agreement (MCA) accounts + +For MCA accounts, you should set up exports using the billing profile, not the billing account. You need your billing profile ID: + +1. Sign in to the [Azure portal](https://portal.azure.com). +2. Search for and select **Cost Management + Billing**. +3. Select **Billing scopes** from the left menu. +4. Select your billing account, then select the billing profile you want to monitor. +5. On the billing profile **Properties** page, copy the **Billing profile ID**. + - Use this format: `/providers/Microsoft.Billing/billingAccounts/{billing-account-id}/billingProfiles/{billing-profile-id}` + +> [!IMPORTANT] +> For MCA contracts, always use the billing profile scope for exports, not the billing account scope. Certain datasets (price sheets, reservation recommendations, and reservation details) are only available at the billing profile level. + +### Subscriptions and resource groups + +For subscriptions and resource groups: + +1. Sign in to the [Azure portal](https://portal.azure.com). +2. Search for and select **Subscriptions**. +3. Select the subscription you want to monitor. +4. On the **Overview** page, copy the **Subscription ID**. + - For subscription scope: `/subscriptions/{subscription-id}` + - For resource group scope: `/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}` + +For more information about finding your billing account and scope information, see [View all your billing accounts in Azure portal](/azure/cost-management-billing/manage/view-all-accounts). + +
+ ## About Cost Management exports Cost Management provides the following 5 types of exports: - Cost and usage details (FOCUS) - - Exports all costs using the FOCUS version of the cost and usage details file as they're defined in the FinOps Open Cost and Usage Specification (FOCUS) project. + - Exports all costs using the FOCUS version of the cost and usage details file as they're defined in the FinOps Open Cost and Usage Specification (FOCUS) project. - Maps to the Costs folder in the ingestion container. - Price sheet - Exports prices for your Azure services. @@ -71,24 +117,26 @@ For the most seamless experience, we recommend [allowing FinOps hubs to manage e If you can't grant permissions for your scope, you can create Cost Management exports manually to accomplish the same goal. 1. Determine the scope for your data export. + - For information about finding your billing account and scope IDs, see [Find your billing account and scope IDs](#find-your-billing-account-and-scope-ids). - We recommend exporting from either an **EA billing account** or **MCA billing profile** scope to access additional datasets, including price sheets and reservation recommendations. - Price sheet exports are required to populate missing prices and costs. - Reservation recommendation exports are used on the Rate optimization Reservation recommendations page. - + > [!IMPORTANT] > **Microsoft Customer Agreement (MCA) scope requirements** - > + > > For MCA contracts, certain datasets are **only available at the billing profile level**, not at the billing account level: + > > - Price sheet data - > - Reservation recommendations + > - Reservation recommendations > - Reservation details - > + > > You must use the billing profile scope (`/providers/Microsoft.Billing/billingAccounts/###/billingProfiles/###`) for these exports. This is a Cost Management limitation. - - We recommend creating daily exports for each export type supported at your chosen billing scope: - - Enterprise Agreement billing account: FocusCosts, Pricesheet, ReservationTransactions, ReservationDetails, ReservationRecommendations - - Microsoft Customer Agreement billing profile: FocusCosts, Pricesheet, ReservationTransactions, ReservationDetails, ReservationRecommendations - - Subscription: FocusCosts + - Enterprise Agreement billing account: FocusCosts, Pricesheet, ReservationTransactions, ReservationDetails, ReservationRecommendations + - Microsoft Customer Agreement billing profile: FocusCosts, Pricesheet, ReservationTransactions, ReservationDetails, ReservationRecommendations + - Subscription: FocusCosts + 2. [Create a new FOCUS cost export](/azure/cost-management-billing/costs/tutorial-export-acm-data) using the following settings: - **Type of data** = `Cost and usage details (FOCUS)`¹ - **Dataset version** = `1.0` or `1.0r2`² @@ -146,7 +194,6 @@ Managed exports allow FinOps hubs to set up and maintain Cost Management exports Managed exports use a managed identity (MI) to configure the exports automatically. To set it up, use the following steps: 1. **Grant access to Azure Data Factory.** - - From the FinOps hub resource group, navigate to **Deployments** > **hub** > **Outputs**, and make note of the values for **managedIdentityId** and **managedIdentityTenantId**. You'll use them in the next step. - Use the following guides to assign access to each scope you want to monitor: - EA enrollments – [Assign enrollment reader role permission](/azure/cost-management-billing/manage/assign-roles-azure-service-principals#assign-enrollment-account-role-permission-to-the-spn). @@ -155,18 +202,18 @@ Managed exports use a managed identity (MI) to configure the exports automatical 2. **Add the desired scopes.** - 1. From the FinOps hub resource group, open the storage account and navigate to **Storage browser** > **Blob containers** > **config**. 2. Select the **settings.json** file, then select **⋯** > **View/edit** to open the file. 3. Update the **scopes** property to include the scopes you want to monitor. For more information, see [Settings.json scope examples](#settingsjson-scope-examples). @@ -195,7 +241,6 @@ Managed exports use a managed identity (MI) to configure the exports automatical Use the **config_RunBackfillJob** pipeline to process historical data after it's been exported. For more information about running Azure Data Factory pipelines, see [Azure Data Factory pipelines](/azure/data-factory/concepts-pipelines-activities). To run the pipeline from the Azure portal: - 1. From the FinOps hub resource group, open the Data Factory instance, select **Launch Studio**, and navigate to **Author** > **Pipelines** > **config_RunBackfillJob**. 2. Select **Debug** in the command bar to run the pipeline. The total run time varies depending on the retention period and number of scopes you're monitoring. @@ -216,7 +261,6 @@ Managed exports use a managed identity (MI) to configure the exports automatical #### Option 2: Using Cost Management exports You can backfill multiple months of data directly using the Cost Management UI. Learn more about exports in the [Cost Management exports documentation](/azure/cost-management-billing/costs/tutorial-export-acm-data). - 1. Open the Azure portal and navigate to **Cost Management** > **Exports**. 2. Select the managed export created by your FinOps hub. 3. Select **Export selected dates** from the top menu. From cd5569a458febff734b9c770ad74cc6c060e818a Mon Sep 17 00:00:00 2001 From: Brett Wilson Date: Sat, 2 Aug 2025 11:03:31 -0700 Subject: [PATCH 04/69] add IngestionSetup_v1_2.kql to build process (#1777) Co-authored-by: msbrett --- src/templates/finops-hub/.build.config | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/templates/finops-hub/.build.config b/src/templates/finops-hub/.build.config index d35e0bac8..0a7ea8c24 100644 --- a/src/templates/finops-hub/.build.config +++ b/src/templates/finops-hub/.build.config @@ -29,7 +29,8 @@ "modules/scripts/Common.kql", "modules/scripts/IngestionSetup_HubInfra.kql", "modules/scripts/IngestionSetup_RawTables.kql", - "modules/scripts/IngestionSetup_v1_0.kql" + "modules/scripts/IngestionSetup_v1_0.kql", + "modules/scripts/IngestionSetup_v1_2.kql" ] }, { From f4202c53819261c24930773a9f74cb4f0818dff1 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Sun, 3 Aug 2025 06:05:58 -0700 Subject: [PATCH 05/69] Fix PowerBI "We cannot convert the value null to type Table" error for blob storage URLs (#1791) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com> --- docs-mslearn/toolkit/help/errors.md | 22 +++++++++++++++++++ docs-mslearn/toolkit/power-bi/setup.md | 6 +++-- .../definition/expressions.tmdl | 2 +- .../definition/expressions.tmdl | 5 +++-- 4 files changed, 30 insertions(+), 5 deletions(-) diff --git a/docs-mslearn/toolkit/help/errors.md b/docs-mslearn/toolkit/help/errors.md index 83b8e9829..e774b2da8 100644 --- a/docs-mslearn/toolkit/help/errors.md +++ b/docs-mslearn/toolkit/help/errors.md @@ -714,6 +714,28 @@ Applicable versions: **0.1 - 0.1.1** (fixed in **0.2**)
+## Power BI: We cannot convert the value null to type Table + +Severity: Critical + +This error can occur when connecting Power BI storage reports to a storage account using the Blob endpoint instead of the required Data Lake Storage (DFS) endpoint. + +**Mitigation**: Update your Storage URL parameter to use the Data Lake Storage endpoint: + +1. Change `.blob.core.windows.net` to `.dfs.core.windows.net` in your Storage URL. +2. If using the Azure portal: + - Open your storage account + - Select **Settings** > **Endpoints** + - Copy the **Data Lake Storage** URL (not the Blob service URL) + +For example: +- ❌ Incorrect: `https://mystorageaccount.blob.core.windows.net/container` +- ✅ Correct: `https://mystorageaccount.dfs.core.windows.net/container` + +If you're still experiencing issues after using the correct endpoint, see [FinOps hubs: We can't convert the value null to type Table](#finops-hubs-we-cant-convert-the-value-null-to-type-table) for additional troubleshooting steps. + +
+ ## FinOps hubs: We can't convert the value null to type Table This error typically indicates that data wasn't ingested into the **ingestion** container. diff --git a/docs-mslearn/toolkit/power-bi/setup.md b/docs-mslearn/toolkit/power-bi/setup.md index 4a712860d..2ac21ea43 100644 --- a/docs-mslearn/toolkit/power-bi/setup.md +++ b/docs-mslearn/toolkit/power-bi/setup.md @@ -100,7 +100,9 @@ The FinOps toolkit Power BI reports include preconfigured visuals, but aren't co - If connecting directly to Cost Management exports in storage: 1. Open the desired storage account in the Azure portal. 2. Select **Settings** > **Endpoints** in the menu. - 3. Copy the **Data Lake Storage** URL. + 3. Copy the **Data Lake Storage** URL (not the Blob service URL). + > [!IMPORTANT] + > Make sure to use the Data Lake Storage endpoint (contains `.dfs.core.windows.net`), not the Blob service endpoint (contains `.blob.core.windows.net`). Power BI storage reports require the DFS endpoint to function properly. 4. Append the container and export path, if applicable. - **Number of Months** – Optional number of closed months you would like to report on if you want to always show a specific number of recent months. If not specified, the report will include all data in storage. - **RangeStart** / **RangeEnd** – Optional date range you would like to limit to. If not specified, the report will include all data in storage. @@ -209,7 +211,7 @@ If using exports or FinOps hubs, you use the Azure Data Lake Storage Gen2 connec - If using raw exports, copy the URL from the storage account: 1. Open the desired storage account in the Azure portal. 2. Select **Settings** > **Endpoints** in the menu. - 3. Copy the **Data Lake Storage** URL. + 3. Copy the **Data Lake Storage** URL (not the Blob service URL). 4. Append the container and export path, if applicable. 5. Select **OK**. > [!WARNING] diff --git a/src/power-bi/kql/Shared.Dataset/definition/expressions.tmdl b/src/power-bi/kql/Shared.Dataset/definition/expressions.tmdl index dc0dc0f4f..8462dbca5 100644 --- a/src/power-bi/kql/Shared.Dataset/definition/expressions.tmdl +++ b/src/power-bi/kql/Shared.Dataset/definition/expressions.tmdl @@ -1,4 +1,4 @@ -/// Name of the Azure DataLake Gen2 storage account to pull data from. +/// URL for the Azure Data Lake Storage Gen2 account with your data. Must use the DFS endpoint (.dfs.core.windows.net), not the Blob endpoint (.blob.core.windows.net). expression 'Storage URL' = "https://demohubupzaljui2bxfm.dfs.core.windows.net/ingestion" meta [IsParameterQuery=true, Type="Text", IsParameterQueryRequired=false] lineageTag: 421c1232-0e40-45a4-bc59-257ce648478c queryGroup: '🛠️ Setup' diff --git a/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl b/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl index 1c5d2a64e..b6685af8f 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl @@ -1,4 +1,4 @@ -/// Name of the Azure DataLake Gen2 storage account to pull data from. +/// URL for the Azure Data Lake Storage Gen2 account with your data. Must use the DFS endpoint (.dfs.core.windows.net), not the Blob endpoint (.blob.core.windows.net). expression 'Storage URL' = "https://demohubupzaljui2bxfm.dfs.core.windows.net/ingestion" meta [IsParameterQuery=true, Type="Text", IsParameterQueryRequired=false] lineageTag: 421c1232-0e40-45a4-bc59-257ce648478c queryGroup: '🛠️ Setup' @@ -32,6 +32,7 @@ expression '▶️ START HERE' = ``` StorageCheck = if StorageUrl = "" or StorageUrl = null then {"✖ Invalid", "Storage URL not specified."} else if Text.StartsWith(StorageUrl, "https://") = false then {"✖ Invalid", "Storage URL must be a valid HTTPS path."} + else if Text.Contains(StorageUrl, ".blob.core.windows.net") then {"✖ Invalid", "Storage URL must use the Data Lake Storage (DFS) endpoint, not the Blob endpoint. Replace '.blob.core.windows.net' with '.dfs.core.windows.net' in your URL. Find the correct URL in your storage account under Settings > Endpoints > Data Lake Storage."} else if (StorageUrlSegmentCount = 4 and Text.Lower(StorageUrlSegments{3}) = "ingestion") or (StorageUrlSegmentCount = 7 and Text.Lower(StorageUrlSegments{5}) = "Files" and Text.Lower(StorageUrlSegments{6}) = "ingestion") then ( {"✔️ Specified", "Will use FinOps hub storage: " & Text.Split(StorageUrl, "."){0} & "."} // TODO: Consider validating the hub version @@ -88,7 +89,7 @@ expression '▶️ START HERE' = ``` [ Instructions = " If using exports:", Value = "", Status = "", Message = "" ], [ Instructions = " ➖ Go to your storage account", Value = "", Status = "", Message = "" ], [ Instructions = " ➖ Open Endpoints", Value = "", Status = "", Message = "" ], - [ Instructions = " ➖ Copy the 'Data Lake Storage' value", Value = "", Status = "", Message = "" ], + [ Instructions = " ➖ Copy the 'Data Lake Storage' value (NOT Blob service)", Value = "", Status = "", Message = "" ], [ Instructions = " ➖ Optionally add a container or file path", Value = "", Status = "", Message = "" ], [ Instructions = " ", Value = "", Status = "", Message = "" ], [ Instructions = "② Optional: Set the desired date range", Value = DateValue, Status = DateCheck{0}, Message = DateCheck{1} ], From dc33e6e55d666760ad7fd226c93a10bc05ed2be1 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Sun, 3 Aug 2025 06:07:18 -0700 Subject: [PATCH 06/69] Clarify Bicep CLI requirement terminology for PowerShell deployments (#1790) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com> --- docs-mslearn/toolkit/hubs/deploy.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs-mslearn/toolkit/hubs/deploy.md b/docs-mslearn/toolkit/hubs/deploy.md index 0139fc94d..810b72843 100644 --- a/docs-mslearn/toolkit/hubs/deploy.md +++ b/docs-mslearn/toolkit/hubs/deploy.md @@ -37,7 +37,7 @@ In this tutorial, you learn how to create a new or update an existing FinOps hub - MCA billing scopes: Contributor on the billing account, billing profile, or invoice section. - MPA billing scopes: Contributor on the billing account, billing profile, or customer. - Optional: Access to Power BI or a Microsoft Fabric workspace with Contributor or Member permissions to create resources and publish reports. -- Optional: PowerShell 7 or Azure Cloud Shell with the [FinOps toolkit PowerShell module](../powershell/powershell-commands.md) installed and imported. +- Optional: PowerShell 7 or Azure Cloud Shell with the [FinOps toolkit PowerShell module](../powershell/powershell-commands.md) installed and imported, and Bicep CLI installed ([installation guide](https://learn.microsoft.com/azure/azure-resource-manager/bicep/install)). More permissions are covered as part of the tutorial. @@ -163,6 +163,9 @@ The core engine for FinOps hubs is deployed via an Azure Resource Manager deploy The following command is part of the FinOps toolkit PowerShell module. To install the module, see [Install the FinOps toolkit PowerShell module](../powershell/powershell-commands.md#install-the-module). +> [!IMPORTANT] +> PowerShell deployment requires Bicep CLI to be installed and available in PATH. See [Install Azure Bicep](https://learn.microsoft.com/azure/azure-resource-manager/bicep/install) for installation instructions. + ```powershell # Deploying to Azure Data Explorer Deploy-FinOpsHub ` From 3d33d01f2a77a2a4be286f05b986a487889e8834 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Sun, 3 Aug 2025 06:10:58 -0700 Subject: [PATCH 07/69] Document multiple scopes capabilities and add overlap warnings in FinOps hubs (#1788) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com> Co-authored-by: flanakin <399533+flanakin@users.noreply.github.com> Co-authored-by: Brett Wilson --- docs-mslearn/toolkit/changelog.md | 21 +++++++ docs-mslearn/toolkit/hubs/configure-scopes.md | 56 ++++++++++++++++++- 2 files changed, 74 insertions(+), 3 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index fd11dd309..050ec518b 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -28,8 +28,29 @@ The following section lists features and enhancements that are currently in deve - Cost Management export modules for subscriptions and resource groups. +### Documentation improvements +
+## v13 + +_Released August 2025_ + +### [FinOps hubs](hubs/finops-hubs-overview.md) v13 + +- **Changed** + - Enhanced [Configure scopes documentation](hubs/configure-scopes.md) to explicitly clarify that FinOps hubs support: + - Multiple Azure scopes (billing accounts, subscriptions, resource groups) in a single hub instance + - Cross-cloud data ingestion through FOCUS format support + - Extensible platform capabilities for custom data sources + +> [!div class="nextstepaction"] +> [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v13) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v12...v13) + +
+ ## v12 _Released July 16, 2025_ diff --git a/docs-mslearn/toolkit/hubs/configure-scopes.md b/docs-mslearn/toolkit/hubs/configure-scopes.md index 8456573bb..0ba104178 100644 --- a/docs-mslearn/toolkit/hubs/configure-scopes.md +++ b/docs-mslearn/toolkit/hubs/configure-scopes.md @@ -17,7 +17,23 @@ ms.reviewer: micflan Connect FinOps hubs to your billing accounts and subscriptions by configuring Cost Management exports manually or granting FinOps hubs access to manage exports for you. -FinOps hubs use Cost Management exports to import cost data for the billing accounts and subscriptions you want to monitor. You can either configure Cost Management exports manually or grant FinOps hubs access to manage exports for you. For information about identifying your billing account and scope IDs, see [Find your billing account and scope IDs](#find-your-billing-account-and-scope-ids). +FinOps hubs use Cost Management exports to import cost data for the billing accounts and subscriptions you want to monitor. **You can configure multiple billing accounts, subscriptions, and even data from other cloud providers** within a single FinOps hub instance. You can either configure Cost Management exports manually or grant FinOps hubs access to manage exports for you. + +
+ +## Multi-scope and multi-cloud capabilities + +FinOps hubs are designed to handle multiple scopes and even data from multiple cloud providers: + +- **Multiple Azure scopes**: You can configure a single FinOps hub to monitor multiple EA billing accounts, MCA billing profiles, subscriptions, and resource groups simultaneously. +- **Cross-cloud support**: FinOps hubs support the [FinOps Open Cost and Usage Specification (FOCUS)](https://focus.finops.org), which enables ingestion of cost data from other cloud providers like AWS, Google Cloud, and others. +- **Extensible platform**: The open architecture allows you to extend FinOps hubs to ingest custom data sources beyond standard cloud billing data. + +> [!TIP] +> When configuring multiple scopes, ensure each scope has a unique directory path in your exports to avoid data conflicts. See the [Settings.json scope examples](#settingsjson-scope-examples) section for detailed configuration guidance. + +> [!WARNING] +> Avoid configuring overlapping export scopes as this leads to duplicate cost data. For example, if you configure both a billing account-level export and a subscription-level export for the same subscription, costs for that subscription will be duplicated in your hub. Always ensure your export scopes are mutually exclusive.
@@ -168,7 +184,10 @@ If you can't grant permissions for your scope, you can create Cost Management ex - Use the **Run now** command at the top of the Cost Management Exports page. - Your data should be available within 15 minutes or so, depending on how large your account is. - If you want to backfill data, open the export details and select the **Export selected dates** command to export one month at a time or use the [Start-FinOpsCostExport PowerShell command](../powershell/cost/Start-FinOpsCostExport.md) to export a larger date range with either the `-Backfill` parameter or specific start and end dates. -6. Repeat steps 1-4 for each scope you want to monitor. +6. **Repeat steps 1-5 for each additional scope you want to monitor** (multiple billing accounts, subscriptions, etc.). + +> [!IMPORTANT] +> **Configuring multiple scopes**: When setting up multiple scopes, ensure each has a unique directory path to prevent data conflicts. You can monitor multiple EA billing accounts, MCA billing profiles, subscriptions, and resource groups within a single FinOps hub instance. _¹ FinOps hubs 0.2 and later requires FOCUS cost data. As of July 2024, the option to export FOCUS cost data is only accessible from the central Cost Management experience in the Azure portal. If you don't see this option, search for or navigate to [Cost Management Exports](https://portal.azure.com/#blade/Microsoft_Azure_CostManagement/Menu/open/exports)._ @@ -325,7 +344,36 @@ Managed exports use a managed identity (MI) to configure the exports automatical "scope": "/subscriptions/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e" }, { - "scope": "subscriptions/bbbb1b1b-cc2c-dd3d-ee4e-ffffff5f5f5f" + "scope": "/subscriptions/bbbb1b1b-cc2c-dd3d-ee4e-ffffff5f5f5f" + } + ] + ``` + +- Multiple EA billing accounts + + ```json + "scopes": [ + { + "scope": "/providers/Microsoft.Billing/billingAccounts/1234567" + }, + { + "scope": "/providers/Microsoft.Billing/billingAccounts/7654321" + } + ] + ``` + +- Mixed scopes (EA billing account and subscriptions) + + ```json + "scopes": [ + { + "scope": "/providers/Microsoft.Billing/billingAccounts/1234567" + }, + { + "scope": "/subscriptions/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e" + }, + { + "scope": "/subscriptions/bbbb1b1b-cc2c-dd3d-ee4e-ffffff5f5f5f" } ] ``` @@ -376,6 +424,8 @@ If you're looking for something specific, vote for an existing or create a new i > [!div class="nextstepaction"] > [Vote on or suggest ideas](https://github.com/microsoft/finops-toolkit/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22Tool%3A%20FinOps%20hubs%22%20sort%3Areactions-%2B1-desc) +
+ ## Related content - [Connect to Power BI](../power-bi/setup.md) From 16dcc8b4c6bc96eacd9886912b9250d1960ef64c Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Sun, 3 Aug 2025 11:46:01 -0700 Subject: [PATCH 08/69] Fix tag expansion in Power BI reports when tag names contain special characters (#1773) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com> Co-authored-by: flanakin <399533+flanakin@users.noreply.github.com> Co-authored-by: Brett Wilson --- .github/copilot-instructions.md | 16 +++++++++++++--- docs-mslearn/toolkit/changelog.md | 14 ++++++++++---- .../Shared.Dataset/definition/tables/Costs.tmdl | 4 ++-- .../Shared.Dataset/definition/tables/Costs.tmdl | 2 +- 4 files changed, 26 insertions(+), 10 deletions(-) diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index ab3b5afbb..48e34b230 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -13,9 +13,19 @@ This is the Microsoft FinOps toolkit - an open-source collection of tools and re - **Test coverage**: Ensure changes don't break existing functionality - **Microsoft standards**: Follow Microsoft style guide and development practices - **Document changes**: Always document changes in the [changelog](../docs-mslearn/toolkit/changelog.md) - - Changelog entries must be under the correct tool and version (e.g., FinOps hubs v12) - - Check the current version in [package.json](../package.json) - - Remove `-dev` from the version, if included + - **Every change must have a changelog entry** - no exceptions for bug fixes, features, or improvements + - **Document in the next release section**, not the last release: + - Check the current version in [package.json](../package.json) (e.g., if version is "12.0.0", document changes under v13) + - Remove `-dev` from the version number when determining the next release + - Create the next version section if it doesn't exist yet + - Changelog entries must be under the correct tool and version (e.g., FinOps hubs v13) + - For releases, include download and changelog links at the end of each release section using the format: + ```markdown + > [!div class="nextstepaction"] + > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/vX) + > [!div class="nextstepaction"] + > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/vX-1...vX) + ``` ### Code style guidelines diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 050ec518b..b1131fd11 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -30,11 +30,11 @@ The following section lists features and enhancements that are currently in deve ### Documentation improvements -
-## v13 +### [Power BI reports](power-bi/reports.md) v13 -_Released August 2025_ +- **Fixed** + - Fixed tag expansion in Power BI reports when tag names contain special characters like colons. ### [FinOps hubs](hubs/finops-hubs-overview.md) v13 @@ -42,7 +42,13 @@ _Released August 2025_ - Enhanced [Configure scopes documentation](hubs/configure-scopes.md) to explicitly clarify that FinOps hubs support: - Multiple Azure scopes (billing accounts, subscriptions, resource groups) in a single hub instance - Cross-cloud data ingestion through FOCUS format support - - Extensible platform capabilities for custom data sources + +
+ +## v13 + +_Released August 2025_ + > [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v13) diff --git a/src/power-bi/kql/Shared.Dataset/definition/tables/Costs.tmdl b/src/power-bi/kql/Shared.Dataset/definition/tables/Costs.tmdl index 1b24e36f0..cb665ee6f 100644 --- a/src/power-bi/kql/Shared.Dataset/definition/tables/Costs.tmdl +++ b/src/power-bi/kql/Shared.Dataset/definition/tables/Costs.tmdl @@ -1945,10 +1945,10 @@ table Costs else if Trim_spaces_from_tags then "parse_json(replace_string(keys, ' ', ''))" else "keys" ) & " - | extend " & Text.Combine(List.Transform(PromotedTags, each "tag_" & Text.Replace(_, " ", "") & " = iff(array_index_of(cleankeys, tolower('" & _ & "')) < 0, '', tostring(Tags[tostring(keys[toint(array_index_of(cleankeys, tolower('" & _ & "')))])]))"), ", ") & " + | extend " & Text.Combine(List.Transform(PromotedTags, each "tag_" & Text.Replace(Text.Replace(_, ":", "_"), " ", "") & " = iff(array_index_of(cleankeys, tolower('" & _ & "')) < 0, '', tostring(Tags[tostring(keys[toint(array_index_of(cleankeys, tolower('" & _ & "')))])]))"), ", ") & " | project-away keys, cleankeys" ) else (" - | extend " & Text.Combine(List.Transform(PromotedTags, each "tag_" & Text.Replace(_, " ", "") & " = tostring(Tags['" & _ & "'])"), ", ") + | extend " & Text.Combine(List.Transform(PromotedTags, each "tag_" & Text.Replace(Text.Replace(_, ":", "_"), " ", "") & " = tostring(Tags['" & _ & "'])"), ", ") ) ) else "") & " // diff --git a/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl b/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl index 996ee889b..904fd4447 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl @@ -1642,7 +1642,7 @@ table Costs Table.ReplaceErrorValues(Table.AddColumn(FTK, "tmp_Tags", each [x_TagsDictionary]), {{"tmp_Tags", null}}), "tmp_Tags", PromotedTags, - List.Transform(PromotedTags, each "tag_" & Text.Replace(_, " ", "")) + List.Transform(PromotedTags, each "tag_" & Text.Replace(Text.Replace(_, ":", "_"), " ", "")) ), // AHB columns From 2367310bd84c71c7f7e22423655f00d120626cf6 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Thu, 7 Aug 2025 08:38:30 -0700 Subject: [PATCH 09/69] [Hubs] Document how to move from private to public access and remove costly resources (#1787) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com> --- docs-mslearn/toolkit/changelog.md | 5 ++ .../toolkit/hubs/private-networking.md | 74 +++++++++++++++++++ 2 files changed, 79 insertions(+) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index b1131fd11..4e4134883 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -24,6 +24,11 @@ This article summarizes the features and enhancements in each release of the Fin The following section lists features and enhancements that are currently in development. +### [FinOps hubs](hubs/finops-hubs-overview.md) + +- **Added** + - Document [how to remove private networking](hubs/private-networking.md#removing-private-networking) and switch back to public access to reduce costs ([#1342](https://github.com/microsoft/finops-toolkit/issues/1342)). + ### Bicep Registry module pending updates - Cost Management export modules for subscriptions and resource groups. diff --git a/docs-mslearn/toolkit/hubs/private-networking.md b/docs-mslearn/toolkit/hubs/private-networking.md index bbc41d29f..300ecc0d0 100644 --- a/docs-mslearn/toolkit/hubs/private-networking.md +++ b/docs-mslearn/toolkit/hubs/private-networking.md @@ -74,6 +74,80 @@ Before enabling private access, review the networking details on this page to un
+## Removing private networking + +If you need to reduce costs or simplify your FinOps hub deployment, you can remove private networking and switch back to public access. This change will: + +- Remove the virtual network and associated networking costs +- Disable private endpoints and DNS zones +- Configure storage, Data Explorer, and Key Vault to use public access +- Switch Azure Data Factory back to the public integration runtime + +> [!WARNING] +> Removing private networking is a significant change that will affect how you access your FinOps hub. Ensure all stakeholders understand the security implications before proceeding. + +### Steps to remove private networking + +1. **Plan the transition**: + - Identify all users and systems currently accessing the hub via private networking + - Coordinate with your network administrators about the change + - Schedule maintenance window as the hub will be temporarily inaccessible during the transition + +2. **Update the FinOps hub deployment**: + + You have two options to redeploy your FinOps hub with public access: + + **Option 1: Redeploy from existing deployment** + - Navigate to your FinOps hub resource group in the Azure portal + - Go to the **Deployments** tab on the resource group + - Find and open the original FinOps hub deployment + - Click **Redeploy** + - On the **Advanced** tab, set **Access** to **Public** + - Review all other settings to ensure they remain as desired + - Deploy the updated configuration + + **Option 2: Deploy latest toolkit version** + - Install the latest current version of the FinOps toolkit + - Use the same resource group name, hub name, and Data Explorer cluster name as your existing deployment + - These values can be obtained from the original deployment template or the config.json file in your hub storage account + - On the **Advanced** tab, set **Access** to **Public** + - Deploy with the same configuration to update your existing hub + +3. **Verify the changes**: + - Confirm that storage accounts, Data Explorer, and Key Vault are accessible via public endpoints + - Test data access from Power BI and other connected systems + - Verify that Azure Data Factory pipelines continue to run successfully + +4. **Clean up networking resources** (optional): + - Once you've confirmed the hub is working correctly with public access, you can delete the networking resources to stop incurring networking costs + - Delete resources in the following order to avoid dependency conflicts: + 1. Private endpoints + 2. Private DNS zones + 3. Virtual network and network security groups (NSGs) + - Be cautious when deleting resources manually - ensure they're not being used by other systems + +5. **Remove Azure Data Factory managed integration runtime** (optional): + - When private networking was enabled, Azure Data Factory may have created a managed integration runtime for secure data processing + - While leaving the managed integration runtime won't break functionality, it does carry ongoing costs + - To remove the managed integration runtime: + 1. Navigate to your Azure Data Factory instance in the Azure portal + 2. Go to **Manage** > **Integration runtimes** + 3. Identify any managed integration runtimes that were created for private networking (typically named with your hub instance) + 4. Stop and delete the managed integration runtime if it's no longer needed + 5. Verify that your data pipelines continue to work with the public integration runtime + - Only remove managed integration runtimes that were specifically created for the FinOps hub private networking setup + +> [!NOTE] +> After removing private networking, your FinOps hub data will be accessible over the internet, though still protected by role-based access control (RBAC) and transport layer security (TLS). Review your organization's security policies to ensure this meets your requirements. +> +> **Security recommendations:** +> - Check the security settings on storage accounts and Azure Data Explorer clusters to ensure they align with your security requirements +> - Consider using network security groups (NSGs) or firewall rules to restrict access to well-known IP addresses such as your corporate firewall, VPN endpoints, or specific office locations +> - Review and configure storage account network access rules to limit access from trusted networks if needed +> - Verify that Azure Data Explorer cluster network settings are properly configured for your access requirements + +
+ ## FinOps hub virtual network When private access is selected, your FinOps hub instance includes a virtual network to ensure communication between its various components remain private. From 5e97a9fe3b2137a16b0869722a390c0e09dc69b4 Mon Sep 17 00:00:00 2001 From: Mike Pritchard <20865962+mpritchard2@users.noreply.github.com> Date: Wed, 1 Oct 2025 09:29:30 -0500 Subject: [PATCH 10/69] Update AOE documentation to show automation variable name for multiple workspaces (#1828) --- .../toolkit/optimization-engine/configure-workspaces.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs-mslearn/toolkit/optimization-engine/configure-workspaces.md b/docs-mslearn/toolkit/optimization-engine/configure-workspaces.md index 9ecb76fa2..ab1a59324 100644 --- a/docs-mslearn/toolkit/optimization-engine/configure-workspaces.md +++ b/docs-mslearn/toolkit/optimization-engine/configure-workspaces.md @@ -79,7 +79,7 @@ In summary, a Windows VM generates, in average, 245 bytes per performance counte ## Using multiple workspaces for performance logs -To include VMs from multiple Log Analytics workspaces in the VM right-size recommendations report, add a new variable to the AOE Azure Automation account. You can add any workspace to the scope of AOE, provided the AOE Managed Identity has Reader permissions over that workspace. The workspace can be in the same subscription or in any other subscription in the same tenant or even in a different tenant ([with the help of Lighthouse](./customize.md#widen-the-engine-scope)). +To include VMs from multiple Log Analytics workspaces in the VM right-size recommendations report, add a new variable named `AzureOptimization_RightSizeAdditionalPerfWorkspaces` to the AOE Azure Automation account. You can add any workspace to the scope of AOE, provided the AOE Managed Identity has Reader permissions over that workspace. The workspace can be in the same subscription or in any other subscription in the same tenant or even in a different tenant ([with the help of Lighthouse](./customize.md#widen-the-engine-scope)). :::image type="content" source="./media/configure-workspaces/log-analytics-additional-performance-workspaces.png" border="true" alt-text="Screenshot showing adding an Automation Account variable with a list of additional workspace IDs VM right-size recommendations." lightbox="./media/configure-workspaces/log-analytics-additional-performance-workspaces.png"::: From 207c817e6a242d151c57b4d4a96ae12710a12c2f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?H=C3=A9lder=20Pinto?= Date: Wed, 1 Oct 2025 21:22:47 +0100 Subject: [PATCH 11/69] [AOE] Fixes underutilized Premium disk recommendations in customers with Premium SSD V2 disks (#1832) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Hélder Pinto --- docs-mslearn/toolkit/changelog.md | 5 +++++ .../Recommend-DiskOptimizationsToBlobStorage.ps1 | 4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 4e4134883..7c5d94e4b 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -29,6 +29,11 @@ The following section lists features and enhancements that are currently in deve - **Added** - Document [how to remove private networking](hubs/private-networking.md#removing-private-networking) and switch back to public access to reduce costs ([#1342](https://github.com/microsoft/finops-toolkit/issues/1342)). +### [Optimization engine](optimization-engine/overview.md) + +- **Fixed** + - Underutilized disks recommendations were not being generated when customer environment has Premium SSD V2 disks ([#1831](https://github.com/microsoft/finops-toolkit/issues/1831)). + ### Bicep Registry module pending updates - Cost Management export modules for subscriptions and resource groups. diff --git a/src/optimization-engine/runbooks/recommendations/Recommend-DiskOptimizationsToBlobStorage.ps1 b/src/optimization-engine/runbooks/recommendations/Recommend-DiskOptimizationsToBlobStorage.ps1 index 86a484745..364b3b303 100644 --- a/src/optimization-engine/runbooks/recommendations/Recommend-DiskOptimizationsToBlobStorage.ps1 +++ b/src/optimization-engine/runbooks/recommendations/Recommend-DiskOptimizationsToBlobStorage.ps1 @@ -224,7 +224,7 @@ $baseQuery = @" | summarize MaxIOPSMetric = max(todouble(MetricValue_s)) by ResourceId | join kind=inner ( $disksTableName - | where TimeGenerated > ago(1d) and DiskState_s =~ 'Attached' and SKU_s startswith 'Premium' + | where TimeGenerated > ago(1d) and DiskState_s =~ 'Attached' and SKU_s startswith 'Premium' and SKU_s !contains "V2" | extend DiskTier_s = strcat(DiskTier_s, ' ', tostring(split(SKU_s, '_')[1])) | project ResourceId=InstanceId_s, DiskName_s, ResourceGroup = ResourceGroupName_s, SubscriptionId = SubscriptionGuid_g, Cloud_s, TenantGuid_g, Tags_s, MaxIOPSDisk=toint(DiskIOPS_s), DiskSizeGB_s, SKU_s, DiskTier_s, DiskType_s ) on ResourceId @@ -237,7 +237,7 @@ $baseQuery = @" | summarize MaxMBsMetric = max(todouble(MetricValue_s)/1024/1024) by ResourceId | join kind=inner ( $disksTableName - | where TimeGenerated > ago(1d) and DiskState_s =~ 'Attached' and SKU_s startswith 'Premium' + | where TimeGenerated > ago(1d) and DiskState_s =~ 'Attached' and SKU_s startswith 'Premium' and SKU_s !contains "V2" | extend DiskTier_s = strcat(DiskTier_s, ' ', tostring(split(SKU_s, '_')[1])) | project ResourceId=InstanceId_s, DiskName_s, ResourceGroup = ResourceGroupName_s, SubscriptionId = SubscriptionGuid_g, Cloud_s, TenantGuid_g, Tags_s, MaxMBsDisk=toint(DiskThroughput_s), DiskSizeGB_s, SKU_s, DiskTier_s, DiskType_s ) on ResourceId From b5a65bee568a439d2f7af00eab0b1ba23c678e90 Mon Sep 17 00:00:00 2001 From: Nicolas Teyan Date: Sat, 11 Oct 2025 02:53:22 +0200 Subject: [PATCH 12/69] Compute best practices documentation update (#1088) Co-authored-by: Michael Flanakin Co-authored-by: msbrett Co-authored-by: Brett Wilson Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> From ce22272a5645bfdcd29a4f2158e988bbfe4dec9d Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Fri, 10 Oct 2025 20:34:31 -0700 Subject: [PATCH 13/69] Break out bicep modules for hubs extensibility (#1800) Co-authored-by: msbrett --- .gitignore | 5 + .vscode/settings.json | 38 +- CLAUDE.md | 182 + docs-mslearn/toolkit/changelog.md | 119 +- src/scripts/Build-OpenData.ps1 | 2 +- src/scripts/Build-Toolkit.ps1 | 155 +- src/scripts/Deploy-Toolkit.ps1 | 31 +- src/templates/finops-hub/.build.config | 32 +- .../Exports/app.bicep | 1720 ++++++ .../schemas/actualcost_c360-2025-04.json | 0 .../schemas/amortizedcost_c360-2025-04.json | 0 .../schemas/focuscost_1.0-preview(v1).json | 0 .../Exports}/schemas/focuscost_1.0.json | 0 .../Exports}/schemas/focuscost_1.0r2.json | 0 .../schemas/focuscost_1.2-preview.json | 0 .../Exports}/schemas/focuscost_1.2.json | 0 .../schemas/pricesheet_2023-05-01_ea.json | 0 .../schemas/pricesheet_2023-05-01_mca.json | 0 .../reservationdetails_2023-03-01.json | 0 ...ervationrecommendations_2023-05-01_ea.json | 0 ...rvationrecommendations_2023-05-01_mca.json | 0 ...reservationtransactions_2023-05-01_ea.json | 0 ...eservationtransactions_2023-05-01_mca.json | 0 .../ManagedExports/app.bicep | 1620 +++++ .../ManagedExports/timeZones.bicep} | 0 .../Microsoft.FinOpsHubs/Analytics/app.bicep | 1909 ++++++ .../Analytics}/dataExplorerEndpoints.bicep | 0 .../Analytics}/scripts/Common.kql | 0 .../Analytics}/scripts/HubSetup_Latest.kql | 0 .../Analytics}/scripts/HubSetup_OpenData.kql | 0 .../Analytics}/scripts/HubSetup_v1_0.kql | 0 .../Analytics}/scripts/HubSetup_v1_2.kql | 0 .../scripts/IngestionSetup_HubInfra.kql | 0 .../scripts/IngestionSetup_RawTables.kql | 0 .../scripts/IngestionSetup_v1_0.kql | 0 .../scripts/IngestionSetup_v1_2.kql | 0 .../Analytics}/scripts/OpenDataFunctions.kql | 0 .../OpenDataFunctions_resource_type_1.kql | 0 .../OpenDataFunctions_resource_type_2.kql | 0 .../OpenDataFunctions_resource_type_3.kql | 0 .../OpenDataFunctions_resource_type_4.kql | 0 .../OpenDataFunctions_resource_type_5.kql | 0 .../Core}/Copy-FileToAzureBlob.ps1 | 0 .../Core/app.bicep} | 179 +- .../Core}/infrastructure.bicep | 6 +- .../Microsoft.FinOpsHubs/Core}/settings.json | 0 .../Microsoft.FinOpsHubs/RemoteHub/app.bicep | 159 + src/templates/finops-hub/modules/README.md | 24 +- .../finops-hub/modules/cm-exports.bicep | 82 - .../finops-hub/modules/dataExplorer.bicep | 501 -- .../finops-hub/modules/dataFactory.bicep | 5251 ----------------- .../finops-hub/modules/{ => fx}/ftkver.txt | 0 .../finops-hub/modules/fx/hub-app.bicep | 576 ++ .../modules/{ => fx}/hub-database.bicep | 1 + .../{ => fx}/hub-deploymentScript.bicep | 5 +- .../hub-eventTrigger.bicep} | 0 .../modules/{ => fx}/hub-identity.bicep | 4 +- .../modules/fx/hub-initialize.bicep | 76 + .../modules/{ => fx}/hub-storage.bicep | 12 +- .../modules/{ => fx}/hub-types.bicep | 121 +- .../modules/{ => fx}/hub-vault.bicep | 0 .../modules/{ => fx}/keyVaultEndpoints.bicep | 0 .../modules/fx/scripts/Init-DataFactory.ps1 | 73 + .../{ => fx}/scripts/Remove-OldResources.ps1 | 0 .../{ => fx}/scripts/Upload-StorageFile.ps1 | 0 .../modules/{ => fx}/storageEndpoints.bicep | 0 .../finops-hub/modules/hub-app.bicep | 332 -- src/templates/finops-hub/modules/hub.bicep | 207 +- .../finops-hub/modules/remoteHub.bicep | 59 - .../modules/scripts/Start-Triggers.ps1 | 65 - src/templates/finops-hub/schemas/README.md | 8 - 71 files changed, 6875 insertions(+), 6679 deletions(-) create mode 100644 CLAUDE.md create mode 100644 src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep rename src/templates/finops-hub/{ => modules/Microsoft.CostManagement/Exports}/schemas/actualcost_c360-2025-04.json (100%) rename src/templates/finops-hub/{ => modules/Microsoft.CostManagement/Exports}/schemas/amortizedcost_c360-2025-04.json (100%) rename src/templates/finops-hub/{ => modules/Microsoft.CostManagement/Exports}/schemas/focuscost_1.0-preview(v1).json (100%) rename src/templates/finops-hub/{ => modules/Microsoft.CostManagement/Exports}/schemas/focuscost_1.0.json (100%) rename src/templates/finops-hub/{ => modules/Microsoft.CostManagement/Exports}/schemas/focuscost_1.0r2.json (100%) rename src/templates/finops-hub/{ => modules/Microsoft.CostManagement/Exports}/schemas/focuscost_1.2-preview.json (100%) rename src/templates/finops-hub/{ => modules/Microsoft.CostManagement/Exports}/schemas/focuscost_1.2.json (100%) rename src/templates/finops-hub/{ => modules/Microsoft.CostManagement/Exports}/schemas/pricesheet_2023-05-01_ea.json (100%) rename src/templates/finops-hub/{ => modules/Microsoft.CostManagement/Exports}/schemas/pricesheet_2023-05-01_mca.json (100%) rename src/templates/finops-hub/{ => modules/Microsoft.CostManagement/Exports}/schemas/reservationdetails_2023-03-01.json (100%) rename src/templates/finops-hub/{ => modules/Microsoft.CostManagement/Exports}/schemas/reservationrecommendations_2023-05-01_ea.json (100%) rename src/templates/finops-hub/{ => modules/Microsoft.CostManagement/Exports}/schemas/reservationrecommendations_2023-05-01_mca.json (100%) rename src/templates/finops-hub/{ => modules/Microsoft.CostManagement/Exports}/schemas/reservationtransactions_2023-05-01_ea.json (100%) rename src/templates/finops-hub/{ => modules/Microsoft.CostManagement/Exports}/schemas/reservationtransactions_2023-05-01_mca.json (100%) create mode 100644 src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep rename src/templates/finops-hub/modules/{azuretimezones.bicep => Microsoft.CostManagement/ManagedExports/timeZones.bicep} (100%) create mode 100644 src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/dataExplorerEndpoints.bicep (100%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/scripts/Common.kql (100%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/scripts/HubSetup_Latest.kql (100%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/scripts/HubSetup_OpenData.kql (100%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/scripts/HubSetup_v1_0.kql (100%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/scripts/HubSetup_v1_2.kql (100%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/scripts/IngestionSetup_HubInfra.kql (100%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/scripts/IngestionSetup_RawTables.kql (100%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/scripts/IngestionSetup_v1_0.kql (100%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/scripts/IngestionSetup_v1_2.kql (100%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/scripts/OpenDataFunctions.kql (100%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/scripts/OpenDataFunctions_resource_type_1.kql (100%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/scripts/OpenDataFunctions_resource_type_2.kql (100%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/scripts/OpenDataFunctions_resource_type_3.kql (100%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/scripts/OpenDataFunctions_resource_type_4.kql (100%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Analytics}/scripts/OpenDataFunctions_resource_type_5.kql (100%) rename src/templates/finops-hub/modules/{scripts => Microsoft.FinOpsHubs/Core}/Copy-FileToAzureBlob.ps1 (100%) rename src/templates/finops-hub/modules/{core.bicep => Microsoft.FinOpsHubs/Core/app.bicep} (50%) rename src/templates/finops-hub/modules/{ => Microsoft.FinOpsHubs/Core}/infrastructure.bicep (97%) rename src/templates/finops-hub/{schemas => modules/Microsoft.FinOpsHubs/Core}/settings.json (100%) create mode 100644 src/templates/finops-hub/modules/Microsoft.FinOpsHubs/RemoteHub/app.bicep delete mode 100644 src/templates/finops-hub/modules/cm-exports.bicep delete mode 100644 src/templates/finops-hub/modules/dataExplorer.bicep delete mode 100644 src/templates/finops-hub/modules/dataFactory.bicep rename src/templates/finops-hub/modules/{ => fx}/ftkver.txt (100%) create mode 100644 src/templates/finops-hub/modules/fx/hub-app.bicep rename src/templates/finops-hub/modules/{ => fx}/hub-database.bicep (93%) rename src/templates/finops-hub/modules/{ => fx}/hub-deploymentScript.bicep (95%) rename src/templates/finops-hub/modules/{hub-event-trigger.bicep => fx/hub-eventTrigger.bicep} (100%) rename src/templates/finops-hub/modules/{ => fx}/hub-identity.bicep (92%) create mode 100644 src/templates/finops-hub/modules/fx/hub-initialize.bicep rename src/templates/finops-hub/modules/{ => fx}/hub-storage.bicep (93%) rename src/templates/finops-hub/modules/{ => fx}/hub-types.bicep (77%) rename src/templates/finops-hub/modules/{ => fx}/hub-vault.bicep (100%) rename src/templates/finops-hub/modules/{ => fx}/keyVaultEndpoints.bicep (100%) create mode 100644 src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 rename src/templates/finops-hub/modules/{ => fx}/scripts/Remove-OldResources.ps1 (100%) rename src/templates/finops-hub/modules/{ => fx}/scripts/Upload-StorageFile.ps1 (100%) rename src/templates/finops-hub/modules/{ => fx}/storageEndpoints.bicep (100%) delete mode 100644 src/templates/finops-hub/modules/hub-app.bicep delete mode 100644 src/templates/finops-hub/modules/remoteHub.bicep delete mode 100644 src/templates/finops-hub/modules/scripts/Start-Triggers.ps1 delete mode 100644 src/templates/finops-hub/schemas/README.md diff --git a/.gitignore b/.gitignore index 189b956d5..835f03ec4 100644 --- a/.gitignore +++ b/.gitignore @@ -368,3 +368,8 @@ venv/ ENV/ env/ +# Claude settings +.claude/ +claude.settings.local.json +settings.local.json + diff --git a/.vscode/settings.json b/.vscode/settings.json index 574991ca3..6b6db9a8d 100644 --- a/.vscode/settings.json +++ b/.vscode/settings.json @@ -1,4 +1,23 @@ { + "[bicep]": { + "editor.formatOnSave": true, + "editor.defaultFormatter": "esbenp.prettier-vscode" + }, + "[json]": { + "editor.formatOnSave": true, + "editor.defaultFormatter": "esbenp.prettier-vscode" + }, + "[markdown]": { + "editor.formatOnSave": true, + "editor.defaultFormatter": "esbenp.prettier-vscode" + }, + "[powerquery]": { + "editor.formatOnSave": false + }, + "[powershell]": { + "editor.formatOnSave": true, + "editor.defaultFormatter": "ms-vscode.powershell" + }, "cSpell.words": [ "ADLS", "architecting", @@ -51,10 +70,15 @@ "Unhide", "westus" ], - "markdown.preview.typographer": false, + "editor.detectIndentation": false, + "editor.formatOnSave": true, + "editor.codeActionsOnSave": { + "source.fixAll": "explicit" + }, "markdown.extension.italic.indicator": "_", "markdown.extension.toc.levels": "2..2", "markdown.extension.toc.orderedList": false, + "markdown.preview.typographer": false, "markdown.updateLinksOnFileMove.enabled": "prompt", "markdownlint.config": { "line-length": false, @@ -70,9 +94,13 @@ "powershell.codeFormatting.whitespaceBeforeOpenBrace": true, "powershell.codeFormatting.whitespaceBeforeOpenParen": true, "powershell.codeFormatting.whitespaceInsideBrace": true, - "[powerquery]": { - "editor.formatOnSave": false - }, "powerquery.general.experimental": false, - "editor.detectIndentation": false + "terminal.integrated.defaultProfile.osx": "pwsh", + "terminal.integrated.profiles.osx": { + "pwsh": { + "path": "pwsh", + "args": ["-NoLogo"] + } + }, + "workbench.editor.enablePreview": false } diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 000000000..3de4262b1 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,182 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Repository Overview + +The FinOps Toolkit is an open-source collection of tools for adopting and implementing FinOps capabilities in the Microsoft Cloud. It contains templates, PowerShell modules, workbooks, optimization engines, and supporting documentation organized in a modular architecture. + +## Common Commands + +### Building and Development + +```bash +# Build entire toolkit +npm run build +# or +pwsh -Command ./src/scripts/Build-Toolkit + +# Build FinOps hubs +pwsh -Command ./src/scripts/Build-Toolkit finops-hub + +# Build specific components +npm run build-ps # PowerShell module only +pwsh -Command ./src/scripts/Build-Bicep # Bicep templates +pwsh -Command ./src/scripts/Build-Workbook # Azure Monitor workbooks +pwsh -Command ./src/scripts/Build-OpenData # Open data files + +# Deploy for testing +npm run deploy-test +# or +pwsh -Command ./src/scripts/Deploy-Toolkit -Build -Test + +# Package for release +npm run package +# or +pwsh -Command ./src/scripts/Package-Toolkit -Build +``` + +### Testing + +```bash +# Run PowerShell unit tests +npm run pester +# or +pwsh -Command Invoke-Pester -Output Detailed -Path ./src/powershell/Tests/Unit/* + +# Run integration tests +pwsh -Command ./src/scripts/Test-PowerShell -Integration + +# Run specific test categories +pwsh -Command ./src/scripts/Test-PowerShell -Hubs -Exports + +# Lint PowerShell code +pwsh -Command ./src/scripts/Test-PowerShell -Lint +``` + +### Bicep Development + +```bash +# Validate Bicep templates +bicep build path/to/template.bicep --stdout + +# Test template deployment +az deployment group what-if --resource-group myRG --template-file template.bicep +``` + +## Architecture and Code Organization + +### High-Level Structure + +- **`/src/templates/`** - ARM/Bicep infrastructure templates with modular namespace organization +- **`/src/powershell/`** - PowerShell module with public/private functions and comprehensive tests +- **`/src/optimization-engine/`** - Azure Optimization Engine for cost recommendations +- **`/src/workbooks/`** - Azure Monitor workbooks for governance and optimization +- **`/src/open-data/`** - Reference data (pricing, regions, services) with utilities +- **`/src/scripts/`** - Build automation and development tools +- **`/docs/`** - Jekyll documentation website +- **`/docs-mslearn/`** - Microsoft Learn documentation website +- **`/docs-wiki/`** - GitHub wiki documentation + +### Current Architectural Reorganization + +The FinOps hubs solution is actively migrating to a namespace-based modular structure: + +- **`Microsoft.FinOpsHubs/`** - Core FinOps Hub infrastructure modules +- **`Microsoft.CostManagement/`** - Cost management exports and schemas +- **`fx/`** - Shared foundation components (hub-types, scripts, utilities) + +### Template Architecture + +Templates use a multi-target build system that generates: + +- Azure Quickstart Templates (ARM JSON) +- Bicep Registry modules +- Standalone deployments +- Azure portal UI definitions + +Key patterns: + +- **`.build.config`** files control build behavior per template +- **`settings.json`** contains component-specific configuration +- **`ftkver.txt`** files maintain version synchronization +- **Conditional resource deployment** based on parameters + +### PowerShell Module Structure + +- **`Public/`** - User-facing cmdlets (Get-_, Set-_, New-\*, etc.) +- **`Private/`** - Internal utilities and helpers +- **`Tests/Unit/`** - Pester unit tests with mocking +- **`Tests/Integration/`** - End-to-end Azure integration tests +- **Module manifest** defines exports and dependencies + +### Data Flow and Integration + +- **Open data** provides reference information consumed by templates and PowerShell +- **Build scripts** orchestrate compilation across all components +- **Version management** is centralized through `Update-Version.ps1` +- **Templates reference** shared schemas and types from `fx/` namespace + +## Key Development Patterns + +### Template Development + +- Use `newApp()` and `newHub()` functions from `fx/hub-types.bicep` for consistent resource naming +- Follow the conditional deployment pattern: `resource foo 'type' = if (condition) { ... }` +- Implement proper parameter validation with `@allowed`, `@minValue`, `@maxValue` +- Include telemetry tracking via `defaultTelemetry` parameter + +### PowerShell Development + +- All public functions must have comment-based help +- Use approved verbs from `Get-Verb` +- Implement comprehensive parameter validation +- Support `-WhatIf` and `-Confirm` for destructive operations +- Include Pester tests for all functions + +### Testing Strategy + +- **Lint tests** validate syntax and coding standards +- **Unit tests** test isolated function behavior with mocks +- **Integration tests** perform end-to-end validation against Azure +- **Template validation** uses `bicep build` and ARM what-if deployments + +### Build System Integration + +The PowerShell-based build system: + +- Compiles templates to multiple target formats +- Validates all code before packaging +- Maintains version consistency across components +- Generates release artifacts automatically + +### Version Management + +- Central version in `package.json` (currently 12.0.0) +- Synchronized across all components via build scripts +- Individual `ftkver.txt` files distributed to modules +- Git tags correspond to release versions + +## Repository Conventions + +### Branch Strategy + +- **`dev`** - Main integration branch +- Feature branches merge into `dev` +- Releases are tagged from `dev` + +### File Organization + +- Templates follow namespace/module/component structure +- PowerShell follows standard module layout +- Documentation uses Jekyll conventions +- Build artifacts are generated, not checked in + +### Coding Standards + +- Always follow the content and coding standards defined in `docs-wiki/Coding-guidelines.md` +- Content (text strings): Follow the Microsoft style guide and always use sentence casing except for proper nouns +- Bicep: Follow Azure Bicep style guide +- PowerShell: Use PowerShell best practices and approved verbs +- Documentation: Use markdown with consistent formatting +- Commit messages: Use conventional commit format diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 7c5d94e4b..1593a5f0c 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -13,7 +13,8 @@ ms.reviewer: micflan - + + # FinOps toolkit changelog This article summarizes the features and enhancements in each release of the FinOps toolkit. @@ -38,32 +39,29 @@ The following section lists features and enhancements that are currently in deve - Cost Management export modules for subscriptions and resource groups. -### Documentation improvements - +
-### [Power BI reports](power-bi/reports.md) v13 +## v13 -- **Fixed** - - Fixed tag expansion in Power BI reports when tag names contain special characters like colons. +_Released August 2025_ ### [FinOps hubs](hubs/finops-hubs-overview.md) v13 - **Changed** + - Reorganized Bicep modules into separate apps. - Enhanced [Configure scopes documentation](hubs/configure-scopes.md) to explicitly clarify that FinOps hubs support: - Multiple Azure scopes (billing accounts, subscriptions, resource groups) in a single hub instance - Cross-cloud data ingestion through FOCUS format support +- **Fixed** + - Fixed all Bicep compilation errors and warnings with inline suppressions and descriptive comments. + - Fixed Build-Toolkit.ps1 bicep generate-params command bug. -
- -## v13 - -_Released August 2025_ +### [Power BI reports](power-bi/reports.md) v13 +- **Fixed** + - Fixed tag expansion in Power BI reports when tag names contain special characters like colons. -> [!div class="nextstepaction"] -> [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v13) -> [!div class="nextstepaction"] -> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v12...v13) +> [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v13) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v12...v13)
@@ -167,10 +165,7 @@ _Released July 16, 2025_ - microsoft.durabletask/schedulers - microsoft.edge/contexts -> [!div class="nextstepaction"] -> [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v12) -> [!div class="nextstepaction"] -> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.11...v12) +> [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v12) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.11...v12)
@@ -303,11 +298,8 @@ _Released June 2, 2025_ - microsoft.synapse/workspaces/kustopools - microsoft.synapse/workspaces/sqlpools - microsoft.web/sites/slots - -> [!div class="nextstepaction"] -> [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.11) -> [!div class="nextstepaction"] -> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.10...v0.11) + +> [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.11) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.10...v0.11)
@@ -411,7 +403,7 @@ _Released May 4, 2025_ - microsoft.sentinelplatformservices/sentinelplatformservices - oracle.database/networkanchors - oracle.database/resourceanchors - **Changed** +- **Changed** - Updated the following resource types: - dell.storage/filesystems - lambdatest.hyperexecute/organizations @@ -424,10 +416,7 @@ _Released May 4, 2025_ - microsoft.liftrpilot/organizations - mongodb.atlas/organizations -> [!div class="nextstepaction"] -> [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.10) -> [!div class="nextstepaction"] -> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.9...v0.10) +> [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.10) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.9...v0.10)
@@ -441,7 +430,7 @@ This release is a minor patch to fix the FinOps hub deployment and the Power BI - **Fixed** - Removed a reference to old columns that are no longer applicable. - - This may have caused new deployments to fail on April 4. Upgrades were not affected. + - This may have caused new deployments to fail on April 4. Upgrades were not affected. ### [Power BI reports](power-bi/reports.md) v0.9 Update 1 @@ -610,10 +599,7 @@ _Released April 4, 2025_ - **Added** - Added sample data for MCA reservation exports. -> [!div class="nextstepaction"] -> [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.9) -> [!div class="nextstepaction"] -> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.8...v0.9) +> [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.9) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.8...v0.9)
@@ -672,6 +658,7 @@ _Released February 12, 2025_ ### [FinOps hubs](hubs/finops-hubs-overview.md) v0.8 + - **Added** - Added Data Explorer dashboard template. - Added new KQL functions in Data Explorer: @@ -705,10 +692,10 @@ _Released February 12, 2025_ #### [Optimization workbook](workbooks/optimization.md) v0.8 - **Added** - - Azure Arc Windows license management under the **Commitment Discounts** tab. + - Azure Arc Windows license management under the **Commitment Discounts** tab. - **Fixed** - Enabled "Export to CSV" option on the **Idle backups** query. - - Corrected VM processor details on the **Compute** tab query. + - Corrected VM processor details on the **Compute** tab query. ### [Optimization engine](optimization-engine/overview.md) v0.8 @@ -824,10 +811,7 @@ _Released February 12, 2025_ - microsoft.iotoperations/instances - microsoft.networkcloud/baremetalmachines -> [!div class="nextstepaction"] -> [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.8) -> [!div class="nextstepaction"] -> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.7...v0.8) +> [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.8) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.7...v0.8)
@@ -880,7 +864,7 @@ _Released December 1, 2024_ - Updated supported spend estimates in the Power BI documentation. - **Fixed** - Fixed EffectiveCost for savings plan purchases to work around a bug in exported data. - + ### [FinOps hubs](hubs/finops-hubs-overview.md) v0.7 _**Breaking change**_ @@ -982,10 +966,7 @@ _**Breaking change**_ - microsoft.healthdataaiservices/deidservices - microsoft.insights/datacollectionrules -> [!div class="nextstepaction"] -> [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.7) -> [!div class="nextstepaction"] -> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.6...v0.7) +> [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.7) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.6...v0.7)
@@ -1135,10 +1116,7 @@ _Released October 2, 2024_ - microsoft.sql/longtermretentionservers - microsoft.verifiedid/authorities -> [!div class="nextstepaction"] -> [Download v0.6](https://github.com/microsoft/finops-toolkit/releases/tag/v0.6) -> [!div class="nextstepaction"] -> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.5...v0.6) +> [!div class="nextstepaction"] > [Download v0.6](https://github.com/microsoft/finops-toolkit/releases/tag/v0.6) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.5...v0.6)
@@ -1370,7 +1348,7 @@ _Released September 1, 2024_ - Updated multiple resource types for the following resource providers: **microsoft.awsconnector**. - Changed the following resource providers to be GA: **microsoft.modsimworkbench**. - **Removed** - - Removed internal "microsoft.cognitiveservices/browse*" resource types. + - Removed internal "microsoft.cognitiveservices/browse\*" resource types. #### [Services v0.5](open-data.md#services) @@ -1407,10 +1385,7 @@ _Released September 1, 2024_ - Move Microsoft Defender for Endpoint from the **Multicloud** service category to **Security**. - Move StorSimple from the **Multicloud** service category to **Storage**. -> [!div class="nextstepaction"] -> [Download v0.5](https://github.com/microsoft/finops-toolkit/releases/tag/v0.5) -> [!div class="nextstepaction"] -> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.4...v0.5) +> [!div class="nextstepaction"] > [Download v0.5](https://github.com/microsoft/finops-toolkit/releases/tag/v0.5) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.4...v0.5)
@@ -1532,10 +1507,7 @@ _Released July 12, 2024_ - Changed the primary columns in the [Regions](open-data.md#regions) and [Services](open-data.md#services) open data files to be lowercase. - Updated all [sample exports](open-data.md#dataset-examples) to use the same date range as the FOCUS 1.0 dataset. -> [!div class="nextstepaction"] -> [Download v0.4](https://github.com/microsoft/finops-toolkit/releases/tag/v0.4) -> [!div class="nextstepaction"] -> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.3...v0.4) +> [!div class="nextstepaction"] > [Download v0.4](https://github.com/microsoft/finops-toolkit/releases/tag/v0.4) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.3...v0.4)
@@ -1623,10 +1595,7 @@ _Released March 28, 2024_ - Added ServiceModel and Environment columns to the [services](open-data.md#services) data ([#585](https://github.com/microsoft/finops-toolkit/issues/585)). - New and updated [resource types](open-data.md#resource-types) and icons. -> [!div class="nextstepaction"] -> [Download v0.3](https://github.com/microsoft/finops-toolkit/releases/tag/v0.3) -> [!div class="nextstepaction"] -> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.2...v0.3) +> [!div class="nextstepaction"] > [Download v0.3](https://github.com/microsoft/finops-toolkit/releases/tag/v0.3) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.2...v0.3)
@@ -1715,10 +1684,7 @@ _**Breaking change**_ - **Added** - [FinOps Open Cost and Usage Specification (FOCUS) details](../focus/what-is-focus.md). -> [!div class="nextstepaction"] -> [Download v0.2](https://github.com/microsoft/finops-toolkit/releases/tag/v0.2) -> [!div class="nextstepaction"] -> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.1.1...v0.2) +> [!div class="nextstepaction"] > [Download v0.2](https://github.com/microsoft/finops-toolkit/releases/tag/v0.2) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.1.1...v0.2)
@@ -1753,10 +1719,7 @@ _Released October 26, 2023_ - [Register-FinOpsHubProviders](powershell/hubs/Register-FinOpsHubProviders.md) - [Remove-FinOpsHub](powershell/hubs/Remove-FinOpsHub.md) -> [!div class="nextstepaction"] -> [Download v0.1.1](https://github.com/microsoft/finops-toolkit/releases/tag/v0.1.1) -> [!div class="nextstepaction"] -> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.1...v0.1.1) +> [!div class="nextstepaction"] > [Download v0.1.1](https://github.com/microsoft/finops-toolkit/releases/tag/v0.1.1) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.1...v0.1.1)
@@ -1804,10 +1767,7 @@ _Released October 22, 2023_ - [Regions](open-data.md#regions) to map historical resource location values in Microsoft Cost Management to standard Azure regions. - [Services](open-data.md#services) to map all resource types to FOCUS service names and categories. -> [!div class="nextstepaction"] -> [Download v0.1](https://github.com/microsoft/finops-toolkit/releases/tag/v0.1) -> [!div class="nextstepaction"] -> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.0.1...v0.1) +> [!div class="nextstepaction"] > [Download v0.1](https://github.com/microsoft/finops-toolkit/releases/tag/v0.1) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.0.1...v0.1)
@@ -1832,10 +1792,7 @@ _Released May 27, 2023_ - **Added** - [Cost optimization workbook](workbooks/optimization.md) to centralize cost optimization. -> [!div class="nextstepaction"] -> [Download v0.0.1](https://github.com/microsoft/finops-toolkit/releases/tag/v0.0.1) -> [!div class="nextstepaction"] -> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/878e4864ca785db4fc13bdd2ec3a6a00058688c3...v0.0.1) +> [!div class="nextstepaction"] > [Download v0.0.1](https://github.com/microsoft/finops-toolkit/releases/tag/v0.0.1) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/878e4864ca785db4fc13bdd2ec3a6a00058688c3...v0.0.1)
@@ -1843,12 +1800,10 @@ _Released May 27, 2023_ Let us know how we're doing with a quick review. We use these reviews to improve and expand FinOps tools and resources. -> [!div class="nextstepaction"] -> [Give feedback](https://portal.azure.com/#view/HubsExtension/InProductFeedbackBlade/extensionName/FinOpsToolkit/cesQuestion/How%20easy%20or%20hard%20is%20it%20to%20use%20FinOps%20toolkit%20tools%20and%20resources%3F/cvaQuestion/How%20valuable%20is%20the%20FinOps%20toolkit%3F/surveyId/FTK/bladeName/Toolkit/featureName/Changelog) +> [!div class="nextstepaction"] > [Give feedback](https://portal.azure.com/#view/HubsExtension/InProductFeedbackBlade/extensionName/FinOpsToolkit/cesQuestion/How%20easy%20or%20hard%20is%20it%20to%20use%20FinOps%20toolkit%20tools%20and%20resources%3F/cvaQuestion/How%20valuable%20is%20the%20FinOps%20toolkit%3F/surveyId/FTK/bladeName/Toolkit/featureName/Changelog) If you're looking for something specific, vote for an existing or create a new idea. Share ideas with others to get more votes. We focus on ideas with the most votes. -> [!div class="nextstepaction"] -> [Vote on or suggest ideas](https://github.com/microsoft/finops-toolkit/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%252B1-desc) +> [!div class="nextstepaction"] > [Vote on or suggest ideas](https://github.com/microsoft/finops-toolkit/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%252B1-desc)
diff --git a/src/scripts/Build-OpenData.ps1 b/src/scripts/Build-OpenData.ps1 index be8ffe2ec..c0a961a45 100644 --- a/src/scripts/Build-OpenData.ps1 +++ b/src/scripts/Build-OpenData.ps1 @@ -173,7 +173,7 @@ function Write-KqlWrapperFunction($Function, $Parts) Write-Output "}" } -$hubsDir = "$PSScriptRoot/../templates/finops-hub/modules/scripts" +$hubsDir = "$PSScriptRoot/../templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts" $psDir = "$PSScriptRoot/../powershell" $srcDir = "$PSScriptRoot/../open-data" $svgDir = "$PSScriptRoot/../../docs/svg" diff --git a/src/scripts/Build-Toolkit.ps1 b/src/scripts/Build-Toolkit.ps1 index fbee8dd7e..7cad6cf61 100644 --- a/src/scripts/Build-Toolkit.ps1 +++ b/src/scripts/Build-Toolkit.ps1 @@ -21,6 +21,8 @@ .LINK https://github.com/microsoft/finops-toolkit/blob/dev/src/scripts/README.md#-build-toolkit #> + +[CmdletBinding()] param( [Parameter(Position = 0)][string]$Template = "*", [switch]$Major, @@ -32,11 +34,15 @@ param( # Create output directory $outDir = "$PSScriptRoot/../../release" -& "$PSScriptRoot/New-Directory" $outDir +Write-Verbose "Creating output directory: $outDir" +& "$PSScriptRoot/New-Directory.ps1" $outDir + +Write-Verbose "Starting build for template pattern: '$Template'" # Update version Write-Host '' -$ver = & "$PSScriptRoot/Update-Version" -Major:$Major -Minor:$Minor -Patch:$Patch -Prerelease:$Prerelease -Label $Label +Write-Verbose "Updating version information..." +$ver = & "$PSScriptRoot/Update-Version.ps1" -Major:$Major -Minor:$Minor -Patch:$Patch -Prerelease:$Prerelease -Label $Label if ($Major -or $Minor -or $Patch -or $Prerelease) { Write-Host "Updated version to $ver" @@ -48,56 +54,109 @@ else Write-Host '' # Generate Bicep Registry modules -Get-ChildItem "$PSScriptRoot/../bicep-registry/$($Template -replace '(subscription|resourceGroup|managementGroup|tenant)-', '')*" -Directory -ErrorAction SilentlyContinue ` +Write-Verbose "Searching for Bicep Registry modules..." +$bicepModules = Get-ChildItem "$PSScriptRoot/../bicep-registry/$($Template -replace '(subscription|resourceGroup|managementGroup|tenant)-', '')*" -Directory -ErrorAction SilentlyContinue ` | Where-Object { $_.Name -ne '.scaffold' } -| ForEach-Object { - ./Build-Bicep $_.Name + +if ($bicepModules) +{ + Write-Verbose "Found $($bicepModules.Count) Bicep Registry module(s) to build" + $bicepModules | ForEach-Object { + Write-Verbose "Building Bicep module: $($_.Name)" + & "$PSScriptRoot/Build-Bicep.ps1" $_.Name + } +} +else +{ + Write-Verbose "No Bicep Registry modules found matching pattern: $Template" } # Generate deployment files from main.bicep in the target directory function Build-MainBicep($dir) { Write-Host " Generating parameters..." + Write-Verbose " Building Bicep template: $dir/main.bicep" bicep build "$dir/main.bicep" --outfile "$dir/azuredeploy.json" - bicep generate-params "$dir/main.bicep" --outfile "$dir/azuredeploy.json" + + Write-Verbose " Generating parameter template: $dir/azuredeploy.parameters.json" + bicep generate-params "$dir/main.bicep" --outfile "$dir/azuredeploy.parameters.json" + $paramFilePath = "$dir/azuredeploy.parameters.json" + Write-Verbose " Processing parameter placeholders in: $paramFilePath" $params = Get-Content $paramFilePath -Raw | ConvertFrom-Json - $params.parameters.psobject.Properties ` - | ForEach-Object { + + $parameterCount = 0 + $params.parameters.psobject.Properties | ForEach-Object { # Add placeholder values for required parameters # See AQT docs for allowed values: https://github.com/Azure/azure-quickstart-templates/tree/4a6e5eae3c860208bf1731b392ae2b8a5fb24f4b/1-CONTRIBUTION-GUIDE#azure-devops-ci - if ($_ -and $_.Name.EndsWith('Name')) { $_.Value.value = "GEN-UNIQUE" } + if ($_ -and $_.Name.EndsWith('Name')) + { + Write-Verbose " Setting placeholder for parameter: $($_.Name)" + $_.Value.value = "GEN-UNIQUE" + $parameterCount++ + } } + + Write-Verbose " Updated $parameterCount parameter placeholder(s)" $params | ConvertTo-Json -Depth 100 | Out-File $paramFilePath } # Generate workbook templates -Get-ChildItem "$PSScriptRoot/../workbooks/*" -Directory ` +Write-Verbose "Searching for workbook templates..." +$workbooks = Get-ChildItem "$PSScriptRoot/../workbooks/*" -Directory ` | Where-Object { $_.Name -ne '.scaffold' -and ($Template -eq "*" -or $Template -eq $_.Name -or $Template -eq "$($_.Name)-workbook" -or $Template -eq "finops-workbooks") } -| ForEach-Object { - $workbook = $_.Name - Write-Host "Building workbook $workbook..." - & "$PSScriptRoot/Build-Workbook" $workbook - Build-MainBicep "$outdir/$workbook-workbook" - $ver | Out-File "$outdir/$workbook-workbook/ftkver.txt" -NoNewline - Write-Host '' + +if ($workbooks) +{ + Write-Verbose "Found $($workbooks.Count) workbook template(s) to build" + $workbooks | ForEach-Object { + $workbook = $_.Name + Write-Host "Building workbook $workbook..." + Write-Verbose " Building workbook: $workbook" + & "$PSScriptRoot/Build-Workbook.ps1" $workbook + + Write-Verbose " Generating deployment files for: $workbook-workbook" + Build-MainBicep "$outdir/$workbook-workbook" + + Write-Verbose " Writing version file: $outdir/$workbook-workbook/ftkver.txt" + $ver | Out-File "$outdir/$workbook-workbook/ftkver.txt" -NoNewline + Write-Host '' + } +} +else +{ + Write-Verbose "No workbook templates found matching pattern: $Template" } # Package templates -Get-ChildItem -Path "$PSScriptRoot/../templates/*", "$PSScriptRoot/../optimization-engine*" -Directory -ErrorAction SilentlyContinue ` -| ForEach-Object { +Write-Verbose "Searching for templates to package..." +$templates = Get-ChildItem -Path "$PSScriptRoot/../templates/*", "$PSScriptRoot/../optimization-engine*" -Directory -ErrorAction SilentlyContinue + +if ($templates) +{ + Write-Verbose "Found $($templates.Count) template(s) to process" +} +else +{ + Write-Verbose "No templates found to package" +} + +$templates | ForEach-Object { $srcDir = $_ $templateName = $srcDir.Name # Skip if not the specified template if ($Template -ne "*" -and $Template -ne $templateName) { + Write-Verbose "Skipping template '$templateName' (doesn't match pattern '$Template')" return } Write-Host "Building template $templateName..." + Write-Verbose " Processing template: $templateName from $($srcDir.FullName)" # Get custom build configuration + Write-Verbose " Loading build configuration from: $($srcDir.FullName)/.build.config" $buildConfig = Get-Content "$_/.build.config" -ErrorAction SilentlyContinue | ConvertFrom-Json -Depth 10 # Backfill config options to avoid null references @@ -110,59 +169,89 @@ Get-ChildItem -Path "$PSScriptRoot/../templates/*", "$PSScriptRoot/../optimizati # Create target directory $destDir = "$outdir/$templateName" + Write-Verbose " Creating target directory: $destDir" Remove-Item $destDir -Recurse -ErrorAction SilentlyContinue - & "$PSScriptRoot/New-Directory" $destDir + & "$PSScriptRoot/New-Directory.ps1" $destDir # Copy required files Write-Host " Copying files..." - Get-ChildItem $srcDir | Copy-Item -Destination $destDir -Recurse -Exclude ".build.config,.buildignore,scaffold.json" + $sourceFiles = Get-ChildItem $srcDir | Where-Object { $_.Name -notin @(".build.config", ".buildignore", "scaffold.json") } + Write-Verbose " Copying $($sourceFiles.Count) items from source to destination" + $sourceFiles | Copy-Item -Destination $destDir -Recurse # Remove ignored files $ignoredFiles = (Get-Content "$srcDir/.buildignore" -ErrorAction SilentlyContinue) + $buildConfig.ignore if ($ignoredFiles.Length) { Write-Host " Removing ignored files..." - $ignoredFiles ` - | ForEach-Object { + Write-Verbose " Processing $($ignoredFiles.Length) ignore pattern(s)" + $removedCount = 0 + $ignoredFiles | ForEach-Object { $file = $_ if (Test-Path "$destDir/$file") { - Write-Verbose "Removing $file" + Write-Verbose " Removing: $file" Remove-Item "$destDir/$file" -Recurse -Force + $removedCount++ } } + Write-Verbose " Removed $removedCount ignored file(s)" + } + else + { + Write-Verbose " No files to ignore" } # Combine KQL files, if specified if ($buildConfig.combineKql.Length) { Write-Host " Combining KQL files..." + Write-Verbose " Processing $($buildConfig.combineKql.Length) KQL combination(s)" $buildConfig.combineKql | ForEach-Object { + Write-Verbose " Combining $($_.files.Length) files into $($_.name)" $combinedScript = ".execute database script with (ContinueOnErrors=true)`n<|`n//`n" $_.files | ForEach-Object { + Write-Verbose " Including file: $_" $combinedScript += Get-Content "$srcDir/$_" -Raw } $combinedScript = $combinedScript -replace '(\r?\n)(\r?\n)+', '$1//$1' - $combinedScript | Out-File "$destDir/../$($_.name)" -Encoding utf8 -Force - Write-Verbose "Combined $($_.files.Length) files into $($_.name)" + $outputPath = "$destDir/../$($_.name)" + Write-Verbose " Writing combined KQL to: $outputPath" + $combinedScript | Out-File $outputPath -Encoding utf8 -Force } } + else + { + Write-Verbose " No KQL files to combine" + } # Update placeholder variables if ($buildConfig.variableExpansion.Length) { Write-Host " Expanding variables..." + Write-Verbose " Processing $($buildConfig.variableExpansion.Length) file(s) for variable expansion" + $expandedCount = 0 $buildConfig.variableExpansion | ForEach-Object { if (Test-Path "$destDir/$_") { - Write-Verbose "Updating $_" + Write-Verbose " Expanding variables in: $_" (Get-Content "$destDir/$_" -Raw) ` -replace '\$\$ftkver\$\$', $ver ` -replace '\$\$build-date\$\$', (Get-Date -Format 'yyyy-MM-dd') ` -replace '\$\$build-month\$\$', (Get-Date -Format 'MMMM yyyy') ` | Out-File "$destDir/$_" -Encoding utf8 -Force + $expandedCount++ + } + else + { + Write-Verbose " File not found for variable expansion: $_" } } + Write-Verbose " Expanded variables in $expandedCount file(s)" + } + else + { + Write-Verbose " No variable expansion configured" } # Move files, if specified @@ -184,11 +273,21 @@ Get-ChildItem -Path "$PSScriptRoot/../templates/*", "$PSScriptRoot/../optimizati # Build main.bicep, if applicable if (Test-Path "$srcDir/main.bicep") { + Write-Verbose " Found main.bicep, generating deployment files" Build-MainBicep $destDir } + else + { + Write-Verbose " No main.bicep found, skipping deployment file generation" + } # Update version in ftkver.txt files - Get-ChildItem $destDir -Include ftkver.txt -Recurse | ForEach-Object { $ver | Out-File $_ -NoNewline } + $versionFiles = Get-ChildItem $destDir -Include ftkver.txt -Recurse + Write-Verbose " Updating $($versionFiles.Count) version file(s) with version: $ver" + $versionFiles | ForEach-Object { + Write-Verbose " Updating version in: $($_.FullName)" + $ver | Out-File $_ -NoNewline + } Write-Host '' } diff --git a/src/scripts/Deploy-Toolkit.ps1 b/src/scripts/Deploy-Toolkit.ps1 index bf2828a94..371c088f6 100644 --- a/src/scripts/Deploy-Toolkit.ps1 +++ b/src/scripts/Deploy-Toolkit.ps1 @@ -78,9 +78,10 @@ if ($Test -and $Demo) # Generates a unique name based on the signed in username and computer name for local testing function Get-UniqueName() { + # Cross-platform name detection (PowerShell 7+) # NOTE: For some reason, using variables directly does not get the value until we write them - $c = $env:ComputerName - $u = $env:USERNAME + $u = $env:USERNAME ?? $env:USER ?? "unknown" + $c = ($env:COMPUTERNAME ?? $env:HOSTNAME ?? "local").Trim() $c | Out-Null $u | Out-Null return "ftk-$u-$c".ToLower() @@ -104,13 +105,27 @@ if (Test-Path "$PSScriptRoot/../workbooks/$Template") # Find bicep file # NOTE: Include templates after release to account for test templates, which are not included in release builds -@("$PSScriptRoot/../../release") ` -| ForEach-Object { Get-Item (Join-Path $_ $Template (iff $Test test/main.test.bicep main.bicep)) -ErrorAction SilentlyContinue } ` +# Resolve template file candidates first to avoid silent exit when none are found +$templateFileCandidates = @("$PSScriptRoot/../../release") ` +| ForEach-Object { Get-Item (Join-Path $_ $Template (iff $Test test/main.test.bicep main.bicep)) -ErrorAction SilentlyContinue } +if (-not $templateFileCandidates) +{ + Write-Error "Template '$Template' not found under '$PSScriptRoot/../../release'." + return +} + +$templateFileCandidates ` | ForEach-Object { $templateFile = $_ $templateName = iff $Test ($templateFile.Directory.Parent.Name + "/test") $templateFile.Directory.Name $parentFolder = iff $Test $templateFile.Directory.Parent.Parent.Name $templateFile.Directory.Parent.Name - $targetScope = (Get-Content $templateFile | Select-String "targetScope = '([^']+)'").Matches[0].Captures[0].Groups[1].Value + $tsMatch = (Get-Content $templateFile | Select-String "targetScope = '([^']+)'") + if ($null -eq $tsMatch -or $tsMatch.Matches.Count -eq 0) + { + Write-Error "Could not determine targetScope in $($templateFile.FullName). Expected: targetScope = 'resourceGroup'|'subscription'|'tenant'." + return + } + $targetScope = $tsMatch.Matches[0].Groups[1].Value # Fall back to default parameters if none were provided $Parameters = iff ($null -eq $Parameters) $defaultParameters["$templateName$(iff $Demo '/demo' '')"] $Parameters @@ -145,11 +160,11 @@ if (Test-Path "$PSScriptRoot/../workbooks/$Template") else { # Create resource group if it doesn't exist - Write-Verbose 'Checking resource group $ResourceGroup...' + Write-Verbose "Checking resource group $ResourceGroup..." $rg = Get-AzResourceGroup $ResourceGroup -ErrorAction SilentlyContinue if ($null -eq $rg) { - Write-Verbose 'Creating resource group $ResourceGroup...' + Write-Verbose "Creating resource group $ResourceGroup..." New-AzResourceGroup ` -Name $ResourceGroup ` -Location $Location ` @@ -205,7 +220,7 @@ if (Test-Path "$PSScriptRoot/../workbooks/$Template") Write-Verbose 'Starting tenant deployment...' $azContext = (Get-AzContext).Tenant - Write-Host " → [tenant] $(iff ([string]::IsNullOrWhitespace($azContext.Name)) $azContext.Id $azContext.Name)..." + Write-Host " → [tenant] $(iff ([string]::IsNullOrWhiteSpace($azContext.Name)) $azContext.Id $azContext.Name)..." $Parameters.Keys | ForEach-Object { Write-Host " $($_) = $($Parameters[$_])" } if ($Debug) diff --git a/src/templates/finops-hub/.build.config b/src/templates/finops-hub/.build.config index 0a7ea8c24..11a298117 100644 --- a/src/templates/finops-hub/.build.config +++ b/src/templates/finops-hub/.build.config @@ -20,27 +20,27 @@ { "name": "finops-hub-fabric-setup-Ingestion.kql", "files": [ - "modules/scripts/OpenDataFunctions_resource_type_1.kql", - "modules/scripts/OpenDataFunctions_resource_type_2.kql", - "modules/scripts/OpenDataFunctions_resource_type_3.kql", - "modules/scripts/OpenDataFunctions_resource_type_4.kql", - "modules/scripts/OpenDataFunctions_resource_type_5.kql", - "modules/scripts/OpenDataFunctions.kql", - "modules/scripts/Common.kql", - "modules/scripts/IngestionSetup_HubInfra.kql", - "modules/scripts/IngestionSetup_RawTables.kql", - "modules/scripts/IngestionSetup_v1_0.kql", - "modules/scripts/IngestionSetup_v1_2.kql" + "modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions_resource_type_1.kql", + "modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions_resource_type_2.kql", + "modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions_resource_type_3.kql", + "modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions_resource_type_4.kql", + "modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions_resource_type_5.kql", + "modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions.kql", + "modules/Microsoft.FinOpsHubs/Analytics/scripts/Common.kql", + "modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_HubInfra.kql", + "modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_RawTables.kql", + "modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_v1_0.kql", + "modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_v1_2.kql" ] }, { "name": "finops-hub-fabric-setup-Hub.kql", "files": [ - "modules/scripts/Common.kql", - "modules/scripts/HubSetup_OpenData.kql", - "modules/scripts/HubSetup_v1_0.kql", - "modules/scripts/HubSetup_v1_2.kql", - "modules/scripts/HubSetup_Latest.kql" + "modules/Microsoft.FinOpsHubs/Analytics/scripts/Common.kql", + "modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_OpenData.kql", + "modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_v1_0.kql", + "modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_v1_2.kql", + "modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_Latest.kql" ] } ] diff --git a/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep new file mode 100644 index 000000000..e020a8031 --- /dev/null +++ b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep @@ -0,0 +1,1720 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { finOpsToolkitVersion, HubAppProperties } from '../../fx/hub-types.bicep' + + +//============================================================================== +// Parameters +//============================================================================== + +@description('Required. FinOps hub app getting deployed.') +param app HubAppProperties + + +//============================================================================== +// Variables +//============================================================================== + +var CONFIG = 'config' +var INGESTION = 'ingestion' +var MSEXPORTS = 'msexports' + +// Separator used to separate ingestion ID from file name for ingested files +var ingestionIdFileNameSeparator = '__' + + +//============================================================================== +// Resources +//============================================================================== + +// Register app +module appRegistration '../../fx/hub-app.bicep' = { + name: 'Microsoft.CostManagement.Exports_Register' + params: { + app: app + version: finOpsToolkitVersion + features: [ + 'Storage' // msexports + schema files + 'DataFactory' // Move files from msexports to ingestion + ] + storageRoles: [ + // User Access Administrator -- https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#user-access-administrator + // Used to create Cost Management exports (which require access to grant access) + '18d7d88d-d35e-4fb5-a5c3-7773c20a72d9' + ] + } +} + +//------------------------------------------------------------------------------ +// Storage +//------------------------------------------------------------------------------ + +// Upload schema files +module schemaFiles '../../fx/hub-storage.bicep' = { + name: 'Microsoft.CostManagement.Exports_Storage.SchemaFiles' + dependsOn: [ + appRegistration + ] + params: { + app: app + container: 'config' + files: { + // cSpell:ignore actualcost, amortizedcost, focuscost, pricesheet, reservationdetails, reservationrecommendations, reservationtransactions + 'schemas/actualcost_c360-2025-04.json': loadTextContent('./schemas/actualcost_c360-2025-04.json') + 'schemas/amortizedcost_c360-2025-04.json': loadTextContent('./schemas/amortizedcost_c360-2025-04.json') + 'schemas/focuscost_1.2.json': loadTextContent('./schemas/focuscost_1.2.json') + 'schemas/focuscost_1.2-preview.json': loadTextContent('./schemas/focuscost_1.2-preview.json') + 'schemas/focuscost_1.0r2.json': loadTextContent('./schemas/focuscost_1.0r2.json') + 'schemas/focuscost_1.0.json': loadTextContent('./schemas/focuscost_1.0.json') + 'schemas/focuscost_1.0-preview(v1).json': loadTextContent('./schemas/focuscost_1.0-preview(v1).json') + 'schemas/pricesheet_2023-05-01_ea.json': loadTextContent('./schemas/pricesheet_2023-05-01_ea.json') + 'schemas/pricesheet_2023-05-01_mca.json': loadTextContent('./schemas/pricesheet_2023-05-01_mca.json') + 'schemas/reservationdetails_2023-03-01.json': loadTextContent('./schemas/reservationdetails_2023-03-01.json') + 'schemas/reservationrecommendations_2023-05-01_ea.json': loadTextContent('./schemas/reservationrecommendations_2023-05-01_ea.json') + 'schemas/reservationrecommendations_2023-05-01_mca.json': loadTextContent('./schemas/reservationrecommendations_2023-05-01_mca.json') + 'schemas/reservationtransactions_2023-05-01_ea.json': loadTextContent('./schemas/reservationtransactions_2023-05-01_ea.json') + 'schemas/reservationtransactions_2023-05-01_mca.json': loadTextContent('./schemas/reservationtransactions_2023-05-01_mca.json') + } + } +} + +// Create msexports container +module exportContainer '../../fx/hub-storage.bicep' = { + name: 'Microsoft.CostManagement.Exports_Storage.ExportContainer' + dependsOn: [ + appRegistration + ] + params: { + app: app + container: MSEXPORTS + } +} + +//------------------------------------------------------------------------------ +// Data Factory +//------------------------------------------------------------------------------ + +resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { + name: app.dataFactory + dependsOn: [ + appRegistration + ] + + // cSpell:ignore linkedservices + resource linkedService_storageAccount 'linkedservices' existing = { + name: app.storage + } + + resource dataset_config 'datasets' existing = { + name: CONFIG + } + + resource dataset_ingestion 'datasets' existing = { + name: INGESTION + } + + resource dataset_ingestion_files 'datasets' existing = { + name: '${INGESTION}_files' + } + + resource dataset_manifest 'datasets' = { + name: 'manifest' + properties: { + parameters: { + fileName: { + type: 'String' + defaultValue: 'manifest.json' + } + folderPath: { + type: 'String' + defaultValue: MSEXPORTS + } + } + type: 'Json' + typeProperties: { + location: { + type: 'AzureBlobFSLocation' + fileName: { + value: '@{dataset().fileName}' + type: 'Expression' + } + folderPath: { + value: '@{dataset().folderPath}' + type: 'Expression' + } + } + } + linkedServiceName: { + // TODO: Should linked service names/references be part of settings? Should datasets be hub modules? + referenceName: app.storage + type: 'LinkedServiceReference' + } + } + } + + resource dataset_msexports 'datasets' = { + name: replace('${MSEXPORTS}', '-', '_') + properties: { + parameters: { + blobPath: { + type: 'String' + } + } + type: 'DelimitedText' + typeProperties: { + location: { + type: 'AzureBlobFSLocation' + fileName: { + value: '@{dataset().blobPath}' + type: 'Expression' + } + fileSystem: exportContainer.outputs.containerName + } + columnDelimiter: ',' + escapeChar: '"' + quoteChar: '"' + firstRowAsHeader: true + } + linkedServiceName: { + referenceName: linkedService_storageAccount.name + type: 'LinkedServiceReference' + } + } + } + + resource dataset_msexports_gzip 'datasets' = { + name: '${MSEXPORTS}_gzip' + properties: { + parameters: { + blobPath: { + type: 'String' + } + } + type: 'DelimitedText' + typeProperties: { + location: { + type: 'AzureBlobFSLocation' + fileName: { + value: '@{dataset().blobPath}' + type: 'Expression' + } + fileSystem: MSEXPORTS + } + columnDelimiter: ',' + escapeChar: '"' + quoteChar: '"' + firstRowAsHeader: true + compressionCodec: 'Gzip' + } + linkedServiceName: { + referenceName: linkedService_storageAccount.name + type: 'LinkedServiceReference' + } + } + } + + resource dataset_msexports_parquet 'datasets' = { + name: '${MSEXPORTS}_parquet' + properties: { + parameters: { + blobPath: { + type: 'String' + } + } + type: 'Parquet' + typeProperties: { + location: { + type: 'AzureBlobFSLocation' + fileName: { + value: '@{dataset().blobPath}' + type: 'Expression' + } + fileSystem: MSEXPORTS + } + } + linkedServiceName: { + referenceName: linkedService_storageAccount.name + type: 'LinkedServiceReference' + } + } + } + + //--------------------------------------------------------------------------- + // msexports_ExecuteETL pipeline + // Triggered by msexports_ManifestAdded trigger + //--------------------------------------------------------------------------- + resource pipeline_ExecuteExportsETL 'pipelines' = { + name: '${MSEXPORTS}_ExecuteETL' + properties: { + activities: [ + { // Wait + name: 'Wait' + description: 'Files may not be available immediately after being created.' + type: 'Wait' + dependsOn: [] + userProperties: [] + typeProperties: { + waitTimeInSeconds: 60 + } + } + { // Read Manifest + name: 'Read Manifest' + description: 'Load the export manifest to determine the scope, dataset, and date range.' + type: 'Lookup' + dependsOn: [ + { + activity: 'Wait' + dependencyConditions: ['Completed'] + } + ] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'JsonSource' + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: true + enablePartitionDiscovery: false + } + formatSettings: { + type: 'JsonReadSettings' + } + } + dataset: { + referenceName: dataFactory::dataset_manifest.name + type: 'DatasetReference' + parameters: { + fileName: { + value: '@pipeline().parameters.fileName' + type: 'Expression' + } + folderPath: { + value: '@pipeline().parameters.folderPath' + type: 'Expression' + } + } + } + } + } + { // Set Has No Rows + name: 'Set Has No Rows' + description: 'Check the row count ' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Read Manifest' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'hasNoRows' + value: { + value: '@or(equals(activity(\'Read Manifest\').output.firstRow.blobCount, null), equals(activity(\'Read Manifest\').output.firstRow.blobCount, 0))' + type: 'Expression' + } + } + } + { // Set Export Dataset Type + name: 'Set Export Dataset Type' + description: 'Save the dataset type from the export manifest.' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Read Manifest' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'exportDatasetType' + value: { + value: '@activity(\'Read Manifest\').output.firstRow.exportConfig.type' + type: 'Expression' + } + } + } + { // Set MCA Column + name: 'Set MCA Column' + description: 'Determines if the dataset schema has channel-specific columns and saves the column name that only exists in MCA to determine if it is an MCA dataset.' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Set Export Dataset Type' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'mcaColumnToCheck' + value: { + // cSpell:ignore pricesheet, reservationtransactions, reservationrecommendations + value: '@if(contains(createArray(\'pricesheet\', \'reservationtransactions\'), toLower(variables(\'exportDatasetType\'))), \'BillingProfileId\', if(equals(toLower(variables(\'exportDatasetType\')), \'reservationrecommendations\'), \'Net Savings\', null))' + type: 'Expression' + } + } + } + { // Set Export Dataset Version + name: 'Set Export Dataset Version' + description: 'Save the dataset version from the export manifest.' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Read Manifest' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'exportDatasetVersion' + value: { + value: '@activity(\'Read Manifest\').output.firstRow.exportConfig.dataVersion' + type: 'Expression' + } + } + } + { // Detect Channel + name: 'Detect Channel' + description: 'Determines what channel this export is from. Switch statement handles the different file types if the mcaColumnToCheck variable is set.' + type: 'Switch' + dependsOn: [ + { + activity: 'Set Has No Rows' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Set MCA Column' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Set Export Dataset Version' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + on: { + value: '@if(or(empty(variables(\'mcaColumnToCheck\')), variables(\'hasNoRows\')), \'ignore\', last(array(split(activity(\'Read Manifest\').output.firstRow.blobs[0].blobName, \'.\'))))' + type: 'Expression' + } + cases: [ + { // csv + value: 'csv' + activities: [ + { + name: 'Check for MCA Column in CSV' + description: 'Checks the dataset to determine if the applicable MCA-specific column exists.' + type: 'Lookup' + dependsOn: [] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'DelimitedTextSource' + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: false + enablePartitionDiscovery: false + } + formatSettings: { + type: 'DelimitedTextReadSettings' + } + } + dataset: { + referenceName: dataFactory::dataset_msexports.name + type: 'DatasetReference' + parameters: { + blobPath: { + value: '@activity(\'Read Manifest\').output.firstRow.blobs[0].blobName' + type: 'Expression' + } + } + } + } + } + { + name: 'Set Schema File with Channel in CSV' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Check for MCA Column in CSV' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'schemaFile' + value: { + value: '@toLower(concat(variables(\'exportDatasetType\'), \'_\', variables(\'exportDatasetVersion\'), if(and(contains(activity(\'Check for MCA Column in CSV\').output, \'firstRow\'), contains(activity(\'Check for MCA Column in CSV\').output.firstRow, variables(\'mcaColumnToCheck\'))), \'_mca\', \'_ea\'), \'.json\'))' + type: 'Expression' + } + } + } + ] + } + { // gz + value: 'gz' + activities: [ + { + name: 'Check for MCA Column in Gzip CSV' + description: 'Checks the dataset to determine if the applicable MCA-specific column exists.' + type: 'Lookup' + dependsOn: [] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'DelimitedTextSource' + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: false + enablePartitionDiscovery: false + } + formatSettings: { + type: 'DelimitedTextReadSettings' + } + } + dataset: { + referenceName: dataFactory::dataset_msexports_gzip.name + type: 'DatasetReference' + parameters: { + blobPath: { + value: '@activity(\'Read Manifest\').output.firstRow.blobs[0].blobName' + type: 'Expression' + } + } + } + } + } + { + name: 'Set Schema File with Channel in Gzip CSV' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Check for MCA Column in Gzip CSV' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'schemaFile' + value: { + value: '@toLower(concat(variables(\'exportDatasetType\'), \'_\', variables(\'exportDatasetVersion\'), if(and(contains(activity(\'Check for MCA Column in Gzip CSV\').output, \'firstRow\'), contains(activity(\'Check for MCA Column in Gzip CSV\').output.firstRow, variables(\'mcaColumnToCheck\'))), \'_mca\', \'_ea\'), \'.json\'))' + type: 'Expression' + } + } + } + ] + } + { // parquet + value: 'parquet' + activities: [ + { + name: 'Check for MCA Column in Parquet' + description: 'Checks the dataset to determine if the applicable MCA-specific column exists.' + type: 'Lookup' + dependsOn: [] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'ParquetSource' + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: false + enablePartitionDiscovery: false + } + formatSettings: { + type: 'ParquetReadSettings' + } + } + dataset: { + referenceName: dataFactory::dataset_msexports_parquet.name + type: 'DatasetReference' + parameters: { + blobPath: { + value: '@activity(\'Read Manifest\').output.firstRow.blobs[0].blobName' + type: 'Expression' + } + } + } + } + } + { + name: 'Set Schema File with Channel for Parquet' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Check for MCA Column in Parquet' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'schemaFile' + value: { + value: '@toLower(concat(variables(\'exportDatasetType\'), \'_\', variables(\'exportDatasetVersion\'), if(and(contains(activity(\'Check for MCA Column in Parquet\').output, \'firstRow\'), contains(activity(\'Check for MCA Column in Parquet\').output.firstRow, variables(\'mcaColumnToCheck\'))), \'_mca\', \'_ea\'), \'.json\'))' + type: 'Expression' + } + } + } + ] + } + ] + defaultActivities: [ + { + name: 'Set Schema File' + type: 'SetVariable' + dependsOn: [] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'schemaFile' + value: { + value: '@toLower(concat(variables(\'exportDatasetType\'), \'_\', variables(\'exportDatasetVersion\'), \'.json\'))' + type: 'Expression' + } + } + } + ] + } + } + { // Set Scope + name: 'Set Scope' + description: 'Save the scope from the export manifest.' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Read Manifest' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'scope' + value: { + value: '@split(toLower(activity(\'Read Manifest\').output.firstRow.exportConfig.resourceId), \'/providers/microsoft.costmanagement/exports/\')[0]' + type: 'Expression' + } + } + } + { // Set Date + name: 'Set Date' + description: 'Save the exported month from the export manifest.' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Read Manifest' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'date' + value: { + value: '@replace(substring(activity(\'Read Manifest\').output.firstRow.runInfo.startDate, 0, 7), \'-\', \'\')' + type: 'Expression' + } + } + } + { // Error: ManifestReadFailed + name: 'Failed to Read Manifest' + type: 'Fail' + dependsOn: [ + { + activity: 'Set Date' + dependencyConditions: ['Failed'] + } + { + activity: 'Set Export Dataset Type' + dependencyConditions: ['Failed'] + } + { + activity: 'Set Scope' + dependencyConditions: ['Failed'] + } + { + activity: 'Read Manifest' + dependencyConditions: ['Failed'] + } + { + activity: 'Set Export Dataset Version' + dependencyConditions: ['Failed'] + } + { + activity: 'Detect Channel' + dependencyConditions: ['Failed'] + } + ] + userProperties: [] + typeProperties: { + message: { + value: '@concat(\'Failed to read the manifest file for this export run. Manifest path: \', pipeline().parameters.folderPath)' + type: 'Expression' + } + errorCode: 'ManifestReadFailed' + } + } + { // Check Schema + name: 'Check Schema' + description: 'Verify that the schema file exists in storage.' + type: 'GetMetadata' + dependsOn: [ + { + activity: 'Set Scope' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Set Date' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Detect Channel' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + dataset: { + referenceName: dataFactory::dataset_config.name + type: 'DatasetReference' + parameters: { + fileName: { + value: '@variables(\'schemaFile\')' + type: 'Expression' + } + folderPath: '${schemaFiles.outputs.containerName}/schemas' + } + } + fieldList: ['exists'] + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: true + enablePartitionDiscovery: false + } + formatSettings: { + type: 'JsonReadSettings' + } + } + } + { // Error: SchemaNotFound + name: 'Schema Not Found' + type: 'Fail' + dependsOn: [ + { + activity: 'Check Schema' + dependencyConditions: ['Failed'] + } + ] + userProperties: [] + typeProperties: { + message: { + value: '@concat(\'The \', variables(\'schemaFile\'), \' schema mapping file was not found. Please confirm version \', variables(\'exportDatasetVersion\'), \' of the \', variables(\'exportDatasetType\'), \' dataset is supported by this version of FinOps hubs. You may need to upgrade to a newer release. To add support for another dataset, you can create a custom mapping file.\')' + type: 'Expression' + } + errorCode: 'SchemaNotFound' + } + } + { // Set Hub Dataset + name: 'Set Hub Dataset' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Set Export Dataset Type' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'hubDataset' + value: { + value: '@if(equals(toLower(variables(\'exportDatasetType\')), \'focuscost\'), \'Costs\', if(equals(toLower(variables(\'exportDatasetType\')), \'pricesheet\'), \'Prices\', if(equals(toLower(variables(\'exportDatasetType\')), \'reservationdetails\'), \'CommitmentDiscountUsage\', if(equals(toLower(variables(\'exportDatasetType\')), \'reservationrecommendations\'), \'Recommendations\', if(equals(toLower(variables(\'exportDatasetType\')), \'reservationtransactions\'), \'Transactions\', if(equals(toLower(variables(\'exportDatasetType\')), \'actualcost\'), \'ActualCosts\', if(equals(toLower(variables(\'exportDatasetType\')), \'amortizedcost\'), \'AmortizedCosts\', toLower(variables(\'exportDatasetType\')))))))))' + type: 'Expression' + } + } + } + { // Set Destination Folder + name: 'Set Destination Folder' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Check Schema' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Set Hub Dataset' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'destinationFolder' + value: { + value: '@replace(concat(variables(\'hubDataset\'),\'/\',substring(variables(\'date\'), 0, 4),\'/\',substring(variables(\'date\'), 4, 2),\'/\',toLower(variables(\'scope\')), if(equals(variables(\'hubDataset\'), \'Recommendations\'), activity(\'Read Manifest\').output.firstRow.exportConfig.exportName, \'\')),\'//\',\'/\')' + type: 'Expression' + } + } + } + { // For Each Blob + name: 'For Each Blob' + description: 'Loop thru each exported file listed in the manifest.' + type: 'ForEach' + dependsOn: [ + { + activity: 'Set Destination Folder' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + items: { + value: '@if(variables(\'hasNoRows\'), json(\'[]\'), activity(\'Read Manifest\').output.firstRow.blobs)' + type: 'Expression' + } + batchCount: app.hub.options.privateRouting ? 4 : 30 // so we don't overload the managed runtime + isSequential: false + activities: [ + { // Execute + name: 'Execute' + description: 'Run the ingestion ETL pipeline.' + type: 'ExecutePipeline' + dependsOn: [] + policy: { + secureInput: false + } + userProperties: [] + typeProperties: { + pipeline: { + referenceName: pipeline_ToIngestion.name + type: 'PipelineReference' + } + waitOnCompletion: true + parameters: { + blobPath: { + value: '@item().blobName' + type: 'Expression' + } + destinationFolder: { + value: '@variables(\'destinationFolder\')' + type: 'Expression' + } + destinationFile: { + value: '@last(array(split(replace(replace(item().blobName, \'.gz\', \'\'), \'.csv\', \'.parquet\'), \'/\')))' + type: 'Expression' + } + ingestionId: { + value: '@activity(\'Read Manifest\').output.firstRow.runInfo.runId' + type: 'Expression' + } + schemaFile: { + value: '@variables(\'schemaFile\')' + type: 'Expression' + } + exportDatasetType: { + value: '@variables(\'exportDatasetType\')' + type: 'Expression' + } + exportDatasetVersion: { + value: '@variables(\'exportDatasetVersion\')' + type: 'Expression' + } + } + } + } + ] + } + } + { // Copy Manifest + name: 'Copy Manifest' + description: 'Copy the manifest to the ingestion container to trigger ADX ingestion' + type: 'Copy' + dependsOn: [ + { + activity: 'For Each Blob' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'JsonSource' + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: true + enablePartitionDiscovery: false + } + formatSettings: { + type: 'JsonReadSettings' + } + } + sink: { + type: 'JsonSink' + storeSettings: { + type: 'AzureBlobFSWriteSettings' + } + formatSettings: { + type: 'JsonWriteSettings' + } + } + enableStaging: false + } + inputs: [ + { + referenceName: dataFactory::dataset_manifest.name + type: 'DatasetReference' + parameters: { + fileName: 'manifest.json' + folderPath: { + value: '@pipeline().parameters.folderPath' + type: 'Expression' + } + } + } + ] + outputs: [ + { + referenceName: dataFactory::dataset_manifest.name + type: 'DatasetReference' + parameters: { + fileName: 'manifest.json' + folderPath: { + value: '@concat(\'${INGESTION}/\', variables(\'destinationFolder\'))' + type: 'Expression' + } + } + } + ] + } + ] + parameters: { + folderPath: { + type: 'string' + } + fileName: { + type: 'string' + } + } + variables: { + date: { + type: 'String' + } + destinationFolder: { + type: 'String' + } + exportDatasetType: { + type: 'String' + } + exportDatasetVersion: { + type: 'String' + } + hasNoRows: { + type: 'Boolean' + } + hubDataset: { + type: 'String' + } + mcaColumnToCheck: { + type: 'String' + } + schemaFile: { + type: 'String' + } + scope: { + type: 'String' + } + } + annotations: [ + 'New export' + ] + } + } + + //--------------------------------------------------------------------------- + // msexports_ETL_ingestion pipeline + // Triggered by msexports_ExecuteETL + //--------------------------------------------------------------------------- + resource pipeline_ToIngestion 'pipelines' = { + name: '${MSEXPORTS}_ETL_${INGESTION}' + properties: { + activities: [ + { // Get Existing Parquet Files + name: 'Get Existing Parquet Files' + description: 'Get the previously ingested files so we can remove any older data. This is necessary to avoid data duplication in reports.' + type: 'GetMetadata' + dependsOn: [] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + dataset: { + referenceName: dataFactory::dataset_ingestion_files.name + type: 'DatasetReference' + parameters: { + folderPath: '@pipeline().parameters.destinationFolder' + } + } + fieldList: [ + 'childItems' + ] + storeSettings: { + type: 'AzureBlobFSReadSettings' + enablePartitionDiscovery: false + } + formatSettings: { + type: 'ParquetReadSettings' + } + } + } + { // Filter Out Current Exports + name: 'Filter Out Current Exports' + description: 'Remove existing files from the current export so those files do not get deleted.' + type: 'Filter' + dependsOn: [ + { + activity: 'Get Existing Parquet Files' + dependencyConditions: [ + 'Completed' + ] + } + ] + userProperties: [] + typeProperties: { + items: { + value: '@if(contains(activity(\'Get Existing Parquet Files\').output, \'childItems\'), activity(\'Get Existing Parquet Files\').output.childItems, json(\'[]\'))' + type: 'Expression' + } + condition: { + // cSpell:ignore endswith + value: '@and(endswith(item().name, \'.parquet\'), not(startswith(item().name, concat(pipeline().parameters.ingestionId, \'${ingestionIdFileNameSeparator}\'))))' + type: 'Expression' + } + } + } + { // Load Schema Mappings + name: 'Load Schema Mappings' + description: 'Get schema mapping file to use for the CSV to parquet conversion.' + type: 'Lookup' + dependsOn: [] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'JsonSource' + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: true + enablePartitionDiscovery: false + } + formatSettings: { + type: 'JsonReadSettings' + } + } + dataset: { + referenceName: dataFactory::dataset_config.name + type: 'DatasetReference' + parameters: { + fileName: { + value: '@toLower(pipeline().parameters.schemaFile)' + type: 'Expression' + } + folderPath: '${CONFIG}/schemas' + } + } + } + } + { // Error: SchemaLoadFailed + name: 'Failed to Load Schema' + type: 'Fail' + dependsOn: [ + { + activity: 'Load Schema Mappings' + dependencyConditions: [ + 'Failed' + ] + } + ] + userProperties: [] + typeProperties: { + message: { + value: '@concat(\'Unable to load the \', pipeline().parameters.schemaFile, \' schema file. Please confirm the schema and version are supported for FinOps hubs ingestion. Unsupported files will remain in the msexports container.\')' + type: 'Expression' + } + errorCode: 'SchemaLoadFailed' + } + } + { // Set Additional Columns + name: 'Set Additional Columns' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Load Schema Mappings' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'additionalColumns' + value: { + value: '@intersection(array(json(concat(\'[{"name":"x_SourceProvider","value":"Microsoft"},{"name":"x_SourceName","value":"Cost Management"},{"name":"x_SourceType","value":"\', pipeline().parameters.exportDatasetVersion, \'"},{"name":"x_SourceVersion","value":"\', pipeline().parameters.exportDatasetVersion, \'"}\'))), activity(\'Load Schema Mappings\').output.firstRow.additionalColumns)' + type: 'Expression' + } + } + } + { // For Each Old File + name: 'For Each Old File' + description: 'Loop thru each of the existing files from previous exports.' + type: 'ForEach' + dependsOn: [ + { + activity: 'Convert to Parquet' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Filter Out Current Exports' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + items: { + value: '@activity(\'Filter Out Current Exports\').output.Value' + type: 'Expression' + } + activities: [ + { // Delete Old Ingested File + name: 'Delete Old Ingested File' + description: 'Delete the previously ingested files from older exports.' + type: 'Delete' + dependsOn: [] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + dataset: { + referenceName: dataFactory::dataset_ingestion.name + type: 'DatasetReference' + parameters: { + blobPath: { + value: '@concat(pipeline().parameters.destinationFolder, \'/\', item().name)' + type: 'Expression' + } + } + } + enableLogging: false + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: false + enablePartitionDiscovery: false + } + } + } + ] + } + } + { // Set Destination Path + name: 'Set Destination Path' + type: 'SetVariable' + dependsOn: [] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'destinationPath' + value: { + value: '@concat(pipeline().parameters.destinationFolder, \'/\', pipeline().parameters.ingestionId, \'${ingestionIdFileNameSeparator}\', pipeline().parameters.destinationFile)' + type: 'Expression' + } + } + } + { // Convert to Parquet + name: 'Convert to Parquet' + description: 'Convert CSV to parquet and move the file to the ${INGESTION} container.' + type: 'Switch' + dependsOn: [ + { + activity: 'Set Destination Path' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Load Schema Mappings' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Set Additional Columns' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + on: { + value: '@last(array(split(pipeline().parameters.blobPath, \'.\')))' + type: 'Expression' + } + cases: [ + { // CSV + value: 'csv' + activities: [ + { // Convert CSV File + name: 'Convert CSV File' + type: 'Copy' + dependsOn: [] + policy: { + timeout: '0.00:10:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'DelimitedTextSource' + additionalColumns: { + value: '@variables(\'additionalColumns\')' + type: 'Expression' + } + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: true + enablePartitionDiscovery: false + } + formatSettings: { + type: 'DelimitedTextReadSettings' + } + } + sink: { + type: 'ParquetSink' + storeSettings: { + type: 'AzureBlobFSWriteSettings' + } + formatSettings: { + type: 'ParquetWriteSettings' + fileExtension: '.parquet' + } + } + enableStaging: false + parallelCopies: 1 + validateDataConsistency: false + translator: { + value: '@activity(\'Load Schema Mappings\').output.firstRow.translator' + type: 'Expression' + } + } + inputs: [ + { + referenceName: dataFactory::dataset_msexports.name + type: 'DatasetReference' + parameters: { + blobPath: { + value: '@pipeline().parameters.blobPath' + type: 'Expression' + } + } + } + ] + outputs: [ + { + referenceName: dataFactory::dataset_ingestion.name + type: 'DatasetReference' + parameters: { + blobPath: { + value: '@variables(\'destinationPath\')' + type: 'Expression' + } + } + } + ] + } + ] + } + { // GZ + value: 'gz' + activities: [ + { // Convert GZip CSV File + name: 'Convert GZip CSV File' + type: 'Copy' + dependsOn: [] + policy: { + timeout: '0.00:10:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'DelimitedTextSource' + additionalColumns: { + value: '@variables(\'additionalColumns\')' + type: 'Expression' + } + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: true + enablePartitionDiscovery: false + } + formatSettings: { + type: 'DelimitedTextReadSettings' + } + } + sink: { + type: 'ParquetSink' + storeSettings: { + type: 'AzureBlobFSWriteSettings' + } + formatSettings: { + type: 'ParquetWriteSettings' + fileExtension: '.parquet' + } + } + enableStaging: false + parallelCopies: 1 + validateDataConsistency: false + translator: { + value: '@activity(\'Load Schema Mappings\').output.firstRow.translator' + type: 'Expression' + } + } + inputs: [ + { + referenceName: dataFactory::dataset_msexports_gzip.name + type: 'DatasetReference' + parameters: { + blobPath: { + value: '@pipeline().parameters.blobPath' + type: 'Expression' + } + } + } + ] + outputs: [ + { + referenceName: dataFactory::dataset_ingestion.name + type: 'DatasetReference' + parameters: { + blobPath: { + value: '@variables(\'destinationPath\')' + type: 'Expression' + } + } + } + ] + } + ] + } + { // Parquet + value: 'parquet' + activities: [ + { // Move Parquet File + name: 'Move Parquet File' + type: 'Copy' + dependsOn: [] + policy: { + timeout: '0.00:05:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'ParquetSource' + additionalColumns: { + value: '@variables(\'additionalColumns\')' + type: 'Expression' + } + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: true + enablePartitionDiscovery: false + } + formatSettings: { + type: 'ParquetReadSettings' + } + } + sink: { + type: 'ParquetSink' + storeSettings: { + type: 'AzureBlobFSWriteSettings' + } + formatSettings: { + type: 'ParquetWriteSettings' + fileExtension: '.parquet' + } + } + enableStaging: false + parallelCopies: 1 + validateDataConsistency: false + } + inputs: [ + { + referenceName: dataFactory::dataset_msexports_parquet.name + type: 'DatasetReference' + parameters: { + blobPath: { + value: '@pipeline().parameters.blobPath' + type: 'Expression' + } + } + } + ] + outputs: [ + { + referenceName: dataFactory::dataset_ingestion.name + type: 'DatasetReference' + parameters: { + blobPath: { + value: '@variables(\'destinationPath\')' + type: 'Expression' + } + } + } + ] + } + ] + } + ] + defaultActivities: [ + { // Error: UnsupportedFileType + name: 'Unsupported File Type' + type: 'Fail' + dependsOn: [] + userProperties: [] + typeProperties: { + message: { + value: '@concat(\'Unable to ingest the specified export file because the file type is not supported. File: \', pipeline().parameters.blobPath)' + type: 'Expression' + } + errorCode: 'UnsupportedExportFileType' + } + } + ] + } + } + { // Read Hub Config + name: 'Read Hub Config' + description: 'Read the hub config to determine if the export should be retained.' + type: 'Lookup' + dependsOn: [] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'JsonSource' + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: false + enablePartitionDiscovery: false + } + formatSettings: { + type: 'JsonReadSettings' + } + } + dataset: { + referenceName: dataFactory::dataset_config.name + type: 'DatasetReference' + parameters: { + fileName: 'settings.json' + folderPath: CONFIG + } + } + } + } + { // If Not Retaining Exports + name: 'If Not Retaining Exports' + description: 'If the msexports retention period <= 0, delete the source file. The main reason to keep the source file is to allow for troubleshooting and reprocessing in the future.' + type: 'IfCondition' + dependsOn: [ + { + activity: 'Convert to Parquet' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Read Hub Config' + dependencyConditions: [ + 'Completed' + ] + } + ] + userProperties: [] + typeProperties: { + expression: { + value: '@lessOrEquals(coalesce(activity(\'Read Hub Config\').output.firstRow.retention.msexports.days, 0), 0)' + type: 'Expression' + } + ifTrueActivities: [ + { // Delete Source File + name: 'Delete Source File' + description: 'Delete the exported data file to keep storage costs down. This file is not referenced by any reporting systems.' + type: 'Delete' + dependsOn: [] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + dataset: { + referenceName: dataFactory::dataset_msexports_parquet.name + type: 'DatasetReference' + parameters: { + blobPath: { + value: '@pipeline().parameters.blobPath' + type: 'Expression' + } + } + } + enableLogging: false + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: true + enablePartitionDiscovery: false + } + } + } + ] + } + } + ] + parameters: { + blobPath: { + type: 'String' + } + destinationFile: { + type: 'string' + } + destinationFolder: { + type: 'string' + } + ingestionId: { + type: 'string' + } + schemaFile: { + type: 'string' + } + exportDatasetType: { + type: 'string' + } + exportDatasetVersion: { + type: 'string' + } + } + variables: { + additionalColumns: { + type: 'Array' + } + destinationPath: { + type: 'String' + } + } + } + } +} + +// msexports_ManifestAdded trigger -> msexports_ExecuteETL pipeline +module trigger_ExportManifestAdded '../../fx/hub-eventTrigger.bicep' = { + name: 'Microsoft.CostManagement.Exports_ADF.ExportManifestTrigger' + params: { + dataFactoryName: dataFactory.name + triggerName: '${MSEXPORTS}_ManifestAdded' + + // TODO: Replace pipeline with event: 'Microsoft.CostManagement.Exports.ManifestAdded' + pipelineName: dataFactory::pipeline_ExecuteExportsETL.name + pipelineParameters: { + folderPath: '@triggerBody().folderPath' + fileName: '@triggerBody().fileName' + } + + storageAccountName: app.storage + storageContainer: MSEXPORTS + storagePathEndsWith: 'manifest.json' + } +} + + +//============================================================================== +// Outputs +//============================================================================== + +@description('Properties of the hub app.') +output app HubAppProperties = app + +@description('Name of the container used for Cost Management exports.') +output exportContainer string = exportContainer.outputs.containerName + +@description('Number of schema files uploaded.') +output schemaFilesUploaded int = schemaFiles.outputs.filesUploaded diff --git a/src/templates/finops-hub/schemas/actualcost_c360-2025-04.json b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/actualcost_c360-2025-04.json similarity index 100% rename from src/templates/finops-hub/schemas/actualcost_c360-2025-04.json rename to src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/actualcost_c360-2025-04.json diff --git a/src/templates/finops-hub/schemas/amortizedcost_c360-2025-04.json b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/amortizedcost_c360-2025-04.json similarity index 100% rename from src/templates/finops-hub/schemas/amortizedcost_c360-2025-04.json rename to src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/amortizedcost_c360-2025-04.json diff --git a/src/templates/finops-hub/schemas/focuscost_1.0-preview(v1).json b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/focuscost_1.0-preview(v1).json similarity index 100% rename from src/templates/finops-hub/schemas/focuscost_1.0-preview(v1).json rename to src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/focuscost_1.0-preview(v1).json diff --git a/src/templates/finops-hub/schemas/focuscost_1.0.json b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/focuscost_1.0.json similarity index 100% rename from src/templates/finops-hub/schemas/focuscost_1.0.json rename to src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/focuscost_1.0.json diff --git a/src/templates/finops-hub/schemas/focuscost_1.0r2.json b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/focuscost_1.0r2.json similarity index 100% rename from src/templates/finops-hub/schemas/focuscost_1.0r2.json rename to src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/focuscost_1.0r2.json diff --git a/src/templates/finops-hub/schemas/focuscost_1.2-preview.json b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/focuscost_1.2-preview.json similarity index 100% rename from src/templates/finops-hub/schemas/focuscost_1.2-preview.json rename to src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/focuscost_1.2-preview.json diff --git a/src/templates/finops-hub/schemas/focuscost_1.2.json b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/focuscost_1.2.json similarity index 100% rename from src/templates/finops-hub/schemas/focuscost_1.2.json rename to src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/focuscost_1.2.json diff --git a/src/templates/finops-hub/schemas/pricesheet_2023-05-01_ea.json b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/pricesheet_2023-05-01_ea.json similarity index 100% rename from src/templates/finops-hub/schemas/pricesheet_2023-05-01_ea.json rename to src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/pricesheet_2023-05-01_ea.json diff --git a/src/templates/finops-hub/schemas/pricesheet_2023-05-01_mca.json b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/pricesheet_2023-05-01_mca.json similarity index 100% rename from src/templates/finops-hub/schemas/pricesheet_2023-05-01_mca.json rename to src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/pricesheet_2023-05-01_mca.json diff --git a/src/templates/finops-hub/schemas/reservationdetails_2023-03-01.json b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/reservationdetails_2023-03-01.json similarity index 100% rename from src/templates/finops-hub/schemas/reservationdetails_2023-03-01.json rename to src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/reservationdetails_2023-03-01.json diff --git a/src/templates/finops-hub/schemas/reservationrecommendations_2023-05-01_ea.json b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/reservationrecommendations_2023-05-01_ea.json similarity index 100% rename from src/templates/finops-hub/schemas/reservationrecommendations_2023-05-01_ea.json rename to src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/reservationrecommendations_2023-05-01_ea.json diff --git a/src/templates/finops-hub/schemas/reservationrecommendations_2023-05-01_mca.json b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/reservationrecommendations_2023-05-01_mca.json similarity index 100% rename from src/templates/finops-hub/schemas/reservationrecommendations_2023-05-01_mca.json rename to src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/reservationrecommendations_2023-05-01_mca.json diff --git a/src/templates/finops-hub/schemas/reservationtransactions_2023-05-01_ea.json b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/reservationtransactions_2023-05-01_ea.json similarity index 100% rename from src/templates/finops-hub/schemas/reservationtransactions_2023-05-01_ea.json rename to src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/reservationtransactions_2023-05-01_ea.json diff --git a/src/templates/finops-hub/schemas/reservationtransactions_2023-05-01_mca.json b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/reservationtransactions_2023-05-01_mca.json similarity index 100% rename from src/templates/finops-hub/schemas/reservationtransactions_2023-05-01_mca.json rename to src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/schemas/reservationtransactions_2023-05-01_mca.json diff --git a/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep b/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep new file mode 100644 index 000000000..848fc8aac --- /dev/null +++ b/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep @@ -0,0 +1,1620 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { finOpsToolkitVersion, HubAppProperties } from '../../fx/hub-types.bicep' + + +//============================================================================== +// Parameters +//============================================================================== + +@description('Required. FinOps hub app getting deployed.') +param app HubAppProperties + + +//============================================================================== +// Variables +//============================================================================== + +var CONFIG = 'config' +var MSEXPORTS = 'msexports' + +var exportsApiVersion = '2023-07-01-preview' +var exportDataVersions = { + focuscost: '1.2-preview' + pricesheet: '2023-03-01' + reservationdetails: '2023-03-01' + reservationrecommendations: '2023-05-01' + reservationtransactions: '2023-05-01' +} + +// cSpell:ignore timeframe +// Function to generate the body for a Cost Management export +func getExportBody(exportContainerName string, datasetType string, schemaVersion string, isMonthly bool, exportFormat string, compressionMode string, partitionData string, dataOverwriteBehavior string) string => '{ "properties": { "definition": { "dataSet": { "configuration": { "dataVersion": "${schemaVersion}", "filters": [] }, "granularity": "Daily" }, "timeframe": "${isMonthly ? 'TheLastMonth': 'MonthToDate' }", "type": "${datasetType}" }, "deliveryInfo": { "destination": { "container": "${exportContainerName}", "rootFolderPath": "@{if(startswith(item().scope, \'/\'), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}", "type": "AzureBlob", "resourceId": "@{variables(\'storageAccountId\')}" } }, "schedule": { "recurrence": "${ isMonthly ? 'Monthly' : 'Daily'}", "recurrencePeriod": { "from": "2024-01-01T00:00:00.000Z", "to": "2050-02-01T00:00:00.000Z" }, "status": "Inactive" }, "format": "${exportFormat}", "partitionData": "${partitionData}", "dataOverwriteBehavior": "${dataOverwriteBehavior}", "compressionMode": "${compressionMode}" }, "id": "@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{variables(\'exportName\')}", "name": "@{variables(\'exportName\')}", "type": "Microsoft.CostManagement/reports", "identity": { "type": "systemAssigned" }, "location": "global" }' + +func getExportBodyV2(exportContainerName string, datasetType string, isMonthly bool, exportFormat string, compressionMode string, partitionData string, dataOverwriteBehavior string, recommendationScope string, recommendationLookbackPeriod string, resourceType string) string => /* + */ toLower(datasetType) == 'focuscost' ? /* + */ '{ "properties": { "definition": { "dataSet": { "configuration": { "dataVersion": "${exportDataVersions[toLower(datasetType)]}", "filters": [] }, "granularity": "Daily" }, "timeframe": "${isMonthly ? 'TheLastMonth': 'MonthToDate' }", "type": "${datasetType}" }, "deliveryInfo": { "destination": { "container": "${exportContainerName}", "rootFolderPath": "@{if(startswith(item().scope, \'/\'), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}", "type": "AzureBlob", "resourceId": "@{variables(\'storageAccountId\')}" } }, "schedule": { "recurrence": "${ isMonthly ? 'Monthly' : 'Daily'}", "recurrencePeriod": { "from": "2024-01-01T00:00:00.000Z", "to": "2050-02-01T00:00:00.000Z" }, "status": "Inactive" }, "format": "${exportFormat}", "partitionData": "${partitionData}", "dataOverwriteBehavior": "${dataOverwriteBehavior}", "compressionMode": "${compressionMode}" }, "id": "@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-costdetails\'))}", "name": "@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-costdetails\'))}", "type": "Microsoft.CostManagement/reports", "identity": { "type": "systemAssigned" }, "location": "global" }' /* + */ : toLower(datasetType) == 'reservationdetails' ? /* + */ '{ "properties": { "definition": { "dataSet": { "configuration": { "dataVersion": "${exportDataVersions[toLower(datasetType)]}", "filters": [] }, "granularity": "Daily" }, "timeframe": "${isMonthly ? 'TheLastMonth': 'MonthToDate' }", "type": "${datasetType}" }, "deliveryInfo": { "destination": { "container": "${exportContainerName}", "rootFolderPath": "@{if(startswith(item().scope, \'/\'), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}", "type": "AzureBlob", "resourceId": "@{variables(\'storageAccountId\')}" } }, "schedule": { "recurrence": "${ isMonthly ? 'Monthly' : 'Daily'}", "recurrencePeriod": { "from": "2024-01-01T00:00:00.000Z", "to": "2050-02-01T00:00:00.000Z" }, "status": "Inactive" }, "format": "${exportFormat}", "partitionData": "${partitionData}", "dataOverwriteBehavior": "${dataOverwriteBehavior}", "compressionMode": "${compressionMode}" }, "id": "@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-${toLower(datasetType)}\'))}", "name": "@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-${toLower(datasetType)}\'))}", "type": "Microsoft.CostManagement/reports", "identity": { "type": "systemAssigned" }, "location": "global" }' /* + */ : (toLower(datasetType) == 'pricesheet') || (toLower(datasetType) == 'reservationtransactions') ? /* + */ '{ "properties": { "definition": { "dataSet": { "configuration": { "dataVersion": "${exportDataVersions[toLower(datasetType)]}", "filters": [] }}, "timeframe": "${isMonthly ? 'TheCurrentMonth': 'MonthToDate' }", "type": "${datasetType}" }, "deliveryInfo": { "destination": { "container": "${exportContainerName}", "rootFolderPath": "@{if(startswith(item().scope, \'/\'), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}", "type": "AzureBlob", "resourceId": "@{variables(\'storageAccountId\')}" } }, "schedule": { "recurrence": "${ isMonthly ? 'Monthly' : 'Daily'}", "recurrencePeriod": { "from": "2024-01-01T00:00:00.000Z", "to": "2050-02-01T00:00:00.000Z" }, "status": "Inactive" }, "format": "${exportFormat}", "partitionData": "${partitionData}", "dataOverwriteBehavior": "${dataOverwriteBehavior}", "compressionMode": "${compressionMode}" }, "id": "@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-${toLower(datasetType)}\'))}", "name": "@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-${toLower(datasetType)}\'))}", "type": "Microsoft.CostManagement/reports", "identity": { "type": "systemAssigned" }, "location": "global" }' /* + */ : toLower(datasetType) == 'reservationrecommendations' ? /* + */ '{ "properties": { "definition": { "dataSet": { "configuration": { "dataVersion": "${exportDataVersions[toLower(datasetType)]}", "filters": [ { "name": "reservationScope", "value": "${recommendationScope}" }, { "name": "resourceType", "value": "${resourceType}" }, { "name": "lookBackPeriod", "value": "${recommendationLookbackPeriod}" }] }}, "timeframe": "${isMonthly ? 'TheLastMonth': 'MonthToDate' }", "type": "${datasetType}" }, "deliveryInfo": { "destination": { "container": "${exportContainerName}", "rootFolderPath": "@{if(startswith(item().scope, \'/\'), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}", "type": "AzureBlob", "resourceId": "@{variables(\'storageAccountId\')}" } }, "schedule": { "recurrence": "${ isMonthly ? 'Monthly' : 'Daily'}", "recurrencePeriod": { "from": "2024-01-01T00:00:00.000Z", "to": "2050-02-01T00:00:00.000Z" }, "status": "Inactive" }, "format": "${exportFormat}", "partitionData": "${partitionData}", "dataOverwriteBehavior": "${dataOverwriteBehavior}", "compressionMode": "${compressionMode}" }, "id": "@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-costdetails\'))}", "name": "@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-costdetails\'))}", "type": "Microsoft.CostManagement/reports", "identity": { "type": "systemAssigned" }, "location": "global" }' /* + */ : 'undefined' + +//============================================================================== +// Resources +//============================================================================== + +// Register app +module appRegistration '../../fx/hub-app.bicep' = { + name: 'Microsoft.CostManagement.ManagedExports_Register' + params: { + app: app + version: finOpsToolkitVersion + features: [ + 'DataFactory' + ] + } +} + +resource storageAccount 'Microsoft.Storage/storageAccounts@2022-09-01' existing = { + name: app.storage +} + +resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { + name: app.dataFactory + + resource dataset_config 'datasets' existing = { + name: CONFIG + } + + resource trigger_DailySchedule 'triggers' = { + name: '${CONFIG}_DailySchedule' + properties: { + pipelines: [ + { + pipelineReference: { + referenceName: dataFactory::pipeline_StartExportProcess.name + type: 'PipelineReference' + } + parameters: { + Recurrence: 'Daily' + } + } + ] + type: 'ScheduleTrigger' + typeProperties: { + recurrence: { + frequency: 'Hour' + interval: 24 + startTime: '2023-01-01T01:01:00' + timeZone: timeZones.outputs.Timezone + } + } + } + } + + resource trigger_MonthlySchedule 'triggers' = { + name: '${CONFIG}_MonthlySchedule' + properties: { + pipelines: [ + { + pipelineReference: { + referenceName: dataFactory::pipeline_StartExportProcess.name + type: 'PipelineReference' + } + parameters: { + Recurrence: 'Monthly' + } + } + ] + type: 'ScheduleTrigger' + typeProperties: { + recurrence: { + frequency: 'Month' + interval: 1 + startTime: '2023-01-05T01:11:00' + timeZone: timeZones.outputs.Timezone + schedule: { + monthDays: [ + 2 + 5 + 19 + ] + } + } + } + } + } + + //---------------------------------------------------------------------------- + // config_StartBackfillProcess pipeline + //---------------------------------------------------------------------------- + resource pipeline_StartBackfillProcess 'pipelines' = { + name: '${CONFIG}_StartBackfillProcess' + properties: { + activities: [ + { // Get Config + name: 'Get Config' + type: 'Lookup' + dependsOn: [] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'JsonSource' + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: true + enablePartitionDiscovery: false + } + formatSettings: { + type: 'JsonReadSettings' + } + } + dataset: { + referenceName: dataset_config.name + type: 'DatasetReference' + parameters: { + fileName: { + value: '@variables(\'fileName\')' + type: 'Expression' + } + folderPath: { + value: '@variables(\'folderPath\')' + type: 'Expression' + } + } + } + } + } + { // Set backfill end date + name: 'Set backfill end date' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Get Config' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + variableName: 'endDate' + value: { + value: '@addDays(startOfMonth(utcNow()), -1)' + type: 'Expression' + } + } + } + { // Set backfill start date + name: 'Set backfill start date' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Get Config' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + variableName: 'startDate' + value: { + value: '@subtractFromTime(startOfMonth(utcNow()), activity(\'Get Config\').output.firstRow.retention.ingestion.months, \'Month\')' + type: 'Expression' + } + } + } + { // Set export start date + name: 'Set export start date' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Set backfill start date' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + variableName: 'thisMonth' + value: { + value: '@startOfMonth(variables(\'endDate\'))' + type: 'Expression' + } + } + } + { // Set export end date + name: 'Set export end date' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Set export start date' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + variableName: 'nextMonth' + value: { + value: '@startOfMonth(subtractFromTime(variables(\'thisMonth\'), 1, \'Month\'))' + type: 'Expression' + } + } + } + { // Every Month + name: 'Every Month' + type: 'Until' + dependsOn: [ + { + activity: 'Set export end date' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Set backfill end date' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + expression: { + value: '@less(variables(\'thisMonth\'), variables(\'startDate\'))' + type: 'Expression' + } + activities: [ + { + name: 'Update export start date' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Backfill data' + dependencyConditions: [ + 'Completed' + ] + } + ] + userProperties: [] + typeProperties: { + variableName: 'thisMonth' + value: { + value: '@variables(\'nextMonth\')' + type: 'Expression' + } + } + } + { + name: 'Update export end date' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Update export start date' + dependencyConditions: [ + 'Completed' + ] + } + ] + userProperties: [] + typeProperties: { + variableName: 'nextMonth' + value: { + value: '@subtractFromTime(variables(\'thisMonth\'), 1, \'Month\')' + type: 'Expression' + } + } + } + { + name: 'Backfill data' + type: 'ExecutePipeline' + dependsOn: [] + userProperties: [] + typeProperties: { + pipeline: { + referenceName: pipeline_RunBackfillJob.name + type: 'PipelineReference' + } + waitOnCompletion: true + parameters: { + StartDate: { + value: '@variables(\'thisMonth\')' + type: 'Expression' + } + EndDate: { + value: '@addDays(addToTime(variables(\'thisMonth\'), 1, \'Month\'), -1)' + type: 'Expression' + } + } + } + } + ] + timeout: '0.02:00:00' + } + } + ] + concurrency: 1 + variables: { + exportName: { + type: 'String' + } + storageAccountId: { + type: 'String' + defaultValue: storageAccount.id + } + finOpsHub: { + type: 'String' + defaultValue: app.hub.name + } + resourceManagementUri: { + type: 'String' + defaultValue: environment().resourceManager + } + fileName: { + type: 'String' + defaultValue: 'settings.json' + } + folderPath: { + type: 'String' + defaultValue: CONFIG + } + endDate: { + type: 'String' + } + startDate: { + type: 'String' + } + thisMonth: { + type: 'String' + } + nextMonth: { + type: 'String' + } + } + } + } + + //---------------------------------------------------------------------------- + // config_RunBackfillJob pipeline + // Triggered by config_StartBackfillProcess pipeline + //---------------------------------------------------------------------------- + resource pipeline_RunBackfillJob 'pipelines' = { + name: '${CONFIG}_RunBackfillJob' + properties: { + activities: [ + { // Get Config + name: 'Get Config' + type: 'Lookup' + dependsOn: [] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'JsonSource' + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: true + enablePartitionDiscovery: false + } + formatSettings: { + type: 'JsonReadSettings' + } + } + dataset: { + referenceName: dataset_config.name + type: 'DatasetReference' + parameters: { + fileName: { + value: '@variables(\'fileName\')' + type: 'Expression' + } + folderPath: { + value: '@variables(\'folderPath\')' + type: 'Expression' + } + } + } + } + } + { // Set Scopes + name: 'Set Scopes' + description: 'Save scopes to test if it is an array' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Get Config' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'scopesArray' + value: { + value: '@activity(\'Get Config\').output.firstRow.scopes' + type: 'Expression' + } + } + } + { // Set Scopes as Array + name: 'Set Scopes as Array' + description: 'Wraps a single scope object into an array to work around the PowerShell bug where single-item arrays are sometimes written as a single object instead of an array.' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Set Scopes' + dependencyConditions: [ + 'Failed' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'scopesArray' + value: { + value: '@createArray(activity(\'Get Config\').output.firstRow.scopes)' + type: 'Expression' + } + } + } + { // Filter Invalid Scopes + name: 'Filter Invalid Scopes' + description: 'Remove any invalid scopes to avoid errors.' + type: 'Filter' + dependsOn: [ + { + activity: 'Set Scopes' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Set Scopes as Array' + dependencyConditions: [ + 'Skipped' + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + items: { + value: '@variables(\'scopesArray\')' + type: 'Expression' + } + condition: { + value: '@and(not(empty(item().scope)), not(equals(item().scope, \'/\')))' + type: 'Expression' + } + } + } + { // ForEach Export Scope + name: 'ForEach Export Scope' + type: 'ForEach' + dependsOn: [ + { + activity: 'Filter Invalid Scopes' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + items: { + value: '@activity(\'Filter Invalid Scopes\').output.Value' + type: 'Expression' + } + isSequential: true + activities: [ + { + name: 'Set backfill export name' + type: 'SetVariable' + dependsOn: [] + userProperties: [] + typeProperties: { + variableName: 'exportName' + value: { + // cSpell:ignore costdetails + value: '@toLower(concat(variables(\'finOpsHub\'), \'-monthly-costdetails\'))' + type: 'Expression' + } + } + } + { + name: 'Trigger backfill export' + type: 'WebActivity' + dependsOn: [ + { + activity: 'Set backfill export name' + dependencyConditions: [ + 'Completed' + ] + } + ] + policy: { + timeout: '0.00:05:00' + retry: 1 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + url: { + value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{variables(\'exportName\')}/run?api-version=${exportsApiVersion}' + type: 'Expression' + } + method: 'POST' + headers: { + 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunBackfill@${finOpsToolkitVersion}' + 'Content-Type': 'application/json' + ClientType: 'FinOpsToolkit.Hubs@${finOpsToolkitVersion}' + } + body: '{"timePeriod" : { "from" : "@{pipeline().parameters.StartDate}", "to" : "@{pipeline().parameters.EndDate}" }}' + authentication: { + type: 'MSI' + resource: { + value: '@variables(\'resourceManagementUri\')' + type: 'Expression' + } + } + } + } + ] + } + } + ] + concurrency: 1 + parameters: { + StartDate: { + type: 'string' + } + EndDate: { + type: 'string' + } + } + variables: { + exportName: { + type: 'String' + } + storageAccountId: { + type: 'String' + defaultValue: storageAccount.id + } + finOpsHub: { + type: 'String' + defaultValue: app.hub.name + } + resourceManagementUri: { + type: 'String' + defaultValue: environment().resourceManager + } + fileName: { + type: 'String' + defaultValue: 'settings.json' + } + folderPath: { + type: 'String' + defaultValue: CONFIG + } + scopesArray: { + type: 'Array' + } + } + } + } + + //---------------------------------------------------------------------------- + // config_StartExportProcess pipeline + // Triggered by config_DailySchedule/MonthlySchedule triggers + //---------------------------------------------------------------------------- + resource pipeline_StartExportProcess 'pipelines' = { + name: '${CONFIG}_StartExportProcess' + properties: { + activities: [ + { // Get Config + name: 'Get Config' + type: 'Lookup' + dependsOn: [] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'JsonSource' + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: true + enablePartitionDiscovery: false + } + formatSettings: { + type: 'JsonReadSettings' + } + } + dataset: { + referenceName: dataset_config.name + type: 'DatasetReference' + parameters: { + fileName: { + value: '@variables(\'fileName\')' + type: 'Expression' + } + folderPath: { + value: '@variables(\'folderPath\')' + type: 'Expression' + } + } + } + } + } + { // Set Scopes + name: 'Set Scopes' + description: 'Save scopes to test if it is an array' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Get Config' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'scopesArray' + value: { + value: '@activity(\'Get Config\').output.firstRow.scopes' + type: 'Expression' + } + } + } + { // Set Scopes as Array + name: 'Set Scopes as Array' + description: 'Wraps a single scope object into an array to work around the PowerShell bug where single-item arrays are sometimes written as a single object instead of an array.' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Set Scopes' + dependencyConditions: [ + 'Failed' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'scopesArray' + value: { + value: '@createArray(activity(\'Get Config\').output.firstRow.scopes)' + type: 'Expression' + } + } + } + { // Filter Invalid Scopes + name: 'Filter Invalid Scopes' + description: 'Remove any invalid scopes to avoid errors.' + type: 'Filter' + dependsOn: [ + { + activity: 'Set Scopes' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Set Scopes as Array' + dependencyConditions: [ + 'Succeeded' + 'Skipped' + ] + } + ] + userProperties: [] + typeProperties: { + items: { + value: '@variables(\'scopesArray\')' + type: 'Expression' + } + condition: { + value: '@and(not(empty(item().scope)), not(equals(item().scope, \'/\')))' + type: 'Expression' + } + } + } + { // ForEach Export Scope + name: 'ForEach Export Scope' + type: 'ForEach' + dependsOn: [ + { + activity: 'Filter Invalid Scopes' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + items: { + value: '@activity(\'Filter Invalid Scopes\').output.Value' + type: 'Expression' + } + isSequential: true + activities: [ + { + name: 'Get exports for scope' + type: 'WebActivity' + dependsOn: [] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + url: { + value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports?api-version=${exportsApiVersion}' + type: 'Expression' + } + method: 'GET' + authentication: { + type: 'MSI' + resource: { + value: '@variables(\'resourceManagementUri\')' + type: 'Expression' + } + } + } + } + { + name: 'Run exports for scope' + type: 'ExecutePipeline' + dependsOn: [ + { + activity: 'Get exports for scope' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + pipeline: { + referenceName: pipeline_RunExportJobs.name + type: 'PipelineReference' + } + waitOnCompletion: true + parameters: { + ExportScopes: { + value: '@activity(\'Get exports for scope\').output.value' + type: 'Expression' + } + Recurrence: { + value: '@pipeline().parameters.Recurrence' + type: 'Expression' + } + } + } + } + ] + } + } + ] + concurrency: 1 + parameters: { + Recurrence: { + type: 'string' + defaultValue: 'Daily' + } + } + variables: { + fileName: { + type: 'String' + defaultValue: 'settings.json' + } + folderPath: { + type: 'String' + defaultValue: CONFIG + } + finOpsHub: { + type: 'String' + defaultValue: app.hub.name + } + resourceManagementUri: { + type: 'String' + defaultValue: environment().resourceManager + } + scopesArray: { + type: 'Array' + } + } + } + } + + //---------------------------------------------------------------------------- + // config_RunExportJobs pipeline + // Triggered by pipeline_StartExportProcess pipeline + //---------------------------------------------------------------------------- + resource pipeline_RunExportJobs 'pipelines' = { + name: '${CONFIG}_RunExportJobs' + dependsOn: [ + dataset_config + ] + properties: { + activities: [ + { + name: 'ForEach export scope' + type: 'ForEach' + dependsOn: [] + userProperties: [] + typeProperties: { + items: { + value: '@pipeline().parameters.exportScopes' + type: 'Expression' + } + isSequential: true + activities: [ + { + name: 'If scheduled' + type: 'IfCondition' + dependsOn: [] + userProperties: [] + typeProperties: { + expression: { + value: '@and( startswith(toLower(item().name), toLower(variables(\'hubName\'))), and(contains(string(item().properties.schedule), \'recurrence\'), equals(toLower(item().properties.schedule.recurrence), toLower(pipeline().parameters.Recurrence))))' + type: 'Expression' + } + ifTrueActivities: [ + { + name: 'Trigger export' + type: 'WebActivity' + dependsOn: [] + policy: { + timeout: '0.00:05:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + method: 'POST' + url: { + value: '@{replace(toLower(concat(variables(\'resourceManagementUri\'),item().id)), \'com//\', \'com/\')}/run?api-version=${exportsApiVersion}' + type: 'Expression' + } + headers: { + 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs@${finOpsToolkitVersion}' + ClientType: 'FinOpsToolkit.Hubs@${finOpsToolkitVersion}' + } + body: ' ' + authentication: { + type: 'MSI' + resource: { + value: '@variables(\'resourceManagementUri\')' + type: 'Expression' + } + } + } + } + ] + } + } + ] + } + } + ] + concurrency: 1 + parameters: { + ExportScopes: { + type: 'array' + } + Recurrence: { + type: 'string' + defaultValue: 'Daily' + } + } + variables: { + resourceManagementUri: { + type: 'String' + defaultValue: environment().resourceManager + } + hubName: { + type: 'String' + defaultValue: app.hub.name + } + } + } + } + + //---------------------------------------------------------------------------- + // config_ConfigureExports pipeline + // Triggered by config_SettingsUpdated trigger + //---------------------------------------------------------------------------- + resource pipeline_ConfigureExports 'pipelines' = { + name: '${CONFIG}_ConfigureExports' + properties: { + activities: [ + { // Get Config + name: 'Get Config' + type: 'Lookup' + dependsOn: [] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'JsonSource' + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: true + enablePartitionDiscovery: false + } + formatSettings: { + type: 'JsonReadSettings' + } + } + dataset: { + referenceName: dataset_config.name + type: 'DatasetReference' + parameters: { + fileName: { + value: '@variables(\'fileName\')' + type: 'Expression' + } + folderPath: { + value: '@variables(\'folderPath\')' + type: 'Expression' + } + } + } + } + } + { // Save Scopes + name: 'Save Scopes' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Get Config' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'scopesArray' + value: { + value: '@activity(\'Get Config\').output.firstRow.scopes' + type: 'Expression' + } + } + } + { // Save Scopes as Array + name: 'Save Scopes as Array' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Save Scopes' + dependencyConditions: [ + 'Failed' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'scopesArray' + value: { + value: '@array(activity(\'Get Config\').output.firstRow.scopes)' + type: 'Expression' + } + } + } + { // Filter Invalid Scopes + name: 'Filter Invalid Scopes' + type: 'Filter' + dependsOn: [ + { + activity: 'Save Scopes' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Save Scopes as Array' + dependencyConditions: [ + 'Skipped' + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + items: { + value: '@variables(\'scopesArray\')' + type: 'Expression' + } + condition: { + value: '@and(not(empty(item().scope)), not(equals(item().scope, \'/\')))' + type: 'Expression' + } + } + } + { // ForEach Export Scope + name: 'ForEach Export Scope' + type: 'ForEach' + dependsOn: [ + { + activity: 'Filter Invalid Scopes' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + items: { + value: '@activity(\'Filter Invalid Scopes\').output.value' + type: 'Expression' + } + isSequential: true + activities: [ + { + name: 'Set Export Type' + type: 'SetVariable' + dependsOn: [] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'exportScopeType' + value: { + value: '@if(contains(toLower(item().scope), \'providers/microsoft.billing/billingaccounts\'), if(contains(toLower(item().scope), \':\'), \'mca\', \'ea\'), if(contains(toLower(item().scope), \'subscriptions/\'), \'subscription\', \'undefined\'))' + type: 'Expression' + } + } + } + { + name: 'Switch Export Type' + type: 'Switch' + dependsOn: [ + { + activity: 'Set Export Type' + dependencyConditions: [ 'Succeeded' ] + } + ] + userProperties: [] + typeProperties: { + on: { + value: '@toLower(variables(\'exportScopeType\'))' + type: 'Expression' + } + cases: [ + { // EA + value: 'ea' + activities: [ + { // 'Open month focus export' + name: 'Open month focus export' + type: 'WebActivity' + dependsOn: [ + ] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + url: { + value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-daily-costdetails\'))}?api-version=${exportsApiVersion}' + type: 'Expression' + } + method: 'PUT' + body: { + value: getExportBodyV2(MSEXPORTS, 'FocusCost', false, 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '') + type: 'Expression' + } + headers: { + 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.CostsDaily@${finOpsToolkitVersion}' + ClientType: 'FinOpsToolkit.Hubs@${finOpsToolkitVersion}' + } + authentication: { + type: 'MSI' + resource: { + value: '@variables(\'resourceManagementUri\')' + type: 'Expression' + } + } + } + } + { // 'Closed month focus export' + name: 'Closed month focus export' + type: 'WebActivity' + dependsOn: [ + { + activity: 'Open month focus export' + dependencyConditions: [ 'Succeeded' ] + } + ] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + url: { + value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-monthly-costdetails\'))}?api-version=${exportsApiVersion}' + type: 'Expression' + } + method: 'PUT' + body: { + value: getExportBodyV2(MSEXPORTS, 'FocusCost', true, 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '') + type: 'Expression' + } + headers: { + 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.CostsMonthly@${finOpsToolkitVersion}' + ClientType: 'FinOpsToolkit.Hubs@${finOpsToolkitVersion}' + } + authentication: { + type: 'MSI' + resource: { + value: '@variables(\'resourceManagementUri\')' + type: 'Expression' + } + } + } + } + { // 'Monthly pricesheet export' + name: 'Monthly pricesheet export' + type: 'WebActivity' + dependsOn: [ + { + activity: 'Closed month focus export' + dependencyConditions: [ 'Succeeded' ] + } + ] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + url: { + value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-monthly-pricesheet\'))}?api-version=${exportsApiVersion}' + type: 'Expression' + } + method: 'PUT' + body: { + value: getExportBodyV2(MSEXPORTS, 'Pricesheet', true, 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '') + type: 'Expression' + } + headers: { + 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.Prices@${finOpsToolkitVersion}' + ClientType: 'FinOpsToolkit.Hubs@${finOpsToolkitVersion}' + } + authentication: { + type: 'MSI' + resource: { + value: '@variables(\'resourceManagementUri\')' + type: 'Expression' + } + } + } + } + { + name: 'Trigger EA monthly pricesheet export' + type: 'WebActivity' + dependsOn: [ + { + activity: 'Monthly pricesheet export' + dependencyConditions: [ 'Succeeded' ] + } + ] + policy: { + timeout: '0.00:05:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + method: 'POST' + url: { + value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-monthly-pricesheet\'))}/run?api-version=${exportsApiVersion}' + type: 'Expression' + } + headers: { + 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.Prices@${finOpsToolkitVersion}' + ClientType: 'FinOpsToolkit.Hubs@${finOpsToolkitVersion}' + } + body: ' ' + authentication: { + type: 'MSI' + resource: { + value: '@variables(\'resourceManagementUri\')' + type: 'Expression' + } + } + } + } + { // 'Daily reservation details export' + name: 'Daily reservation details export' + type: 'WebActivity' + dependsOn: [ + { + activity: 'Monthly pricesheet export' + dependencyConditions: [ 'Succeeded' ] + } + ] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + url: { + value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-daily-reservationdetails\'))}?api-version=${exportsApiVersion}' + type: 'Expression' + } + method: 'PUT' + body: { + value: getExportBodyV2(MSEXPORTS, 'ReservationDetails', false, 'CSV', 'None', 'true', 'CreateNewReport', '', '', '') + type: 'Expression' + } + headers: { + 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.ReservationDetails@${finOpsToolkitVersion}' + ClientType: 'FinOpsToolkit.Hubs@${finOpsToolkitVersion}' + } + authentication: { + type: 'MSI' + resource: { + value: '@variables(\'resourceManagementUri\')' + type: 'Expression' + } + } + } + } + { // 'Daily reservation transactions export' + name: 'Daily reservation transactions export' + type: 'WebActivity' + dependsOn: [ + { + activity: 'Daily reservation details export' + dependencyConditions: [ 'Succeeded' ] + } + ] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + url: { + value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-daily-reservationtransactions\'))}?api-version=${exportsApiVersion}' + type: 'Expression' + } + method: 'PUT' + body: { + value: getExportBodyV2(MSEXPORTS, 'ReservationTransactions', false, 'CSV', 'None', 'true', 'CreateNewReport', '', '', '') + type: 'Expression' + } + headers: { + 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.ReservationTransactions@${finOpsToolkitVersion}' + ClientType: 'FinOpsToolkit.Hubs@${finOpsToolkitVersion}' + } + authentication: { + type: 'MSI' + resource: { + value: '@variables(\'resourceManagementUri\')' + type: 'Expression' + } + } + } + } + { // 'Daily recommendations shared last30day virtual machines export' + name: 'Daily shared 30day virtual machines' + type: 'WebActivity' + dependsOn: [ + { + activity: 'Daily reservation transactions export' + dependencyConditions: [ 'Succeeded' ] + } + ] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + url: { + value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-daily-recommendations-shared-last30days-virtualmachines\'))}?api-version=${exportsApiVersion}' + type: 'Expression' + } + method: 'PUT' + body: { + value: getExportBodyV2(MSEXPORTS, 'ReservationRecommendations', false, 'CSV', 'None', 'true', 'CreateNewReport', 'Shared', 'Last30Days', 'VirtualMachines') + type: 'Expression' + } + headers: { + 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.ReservationRecommendations.VM.Shared.30d@${finOpsToolkitVersion}' + ClientType: 'FinOpsToolkit.Hubs@${finOpsToolkitVersion}' + } + authentication: { + type: 'MSI' + resource: { + value: '@variables(\'resourceManagementUri\')' + type: 'Expression' + } + } + } + } + ] + } + { // subscription + value: 'subscription' + activities: [ + { // 'Subscription open month focus export' + name: 'Subscription open month focus export' + type: 'WebActivity' + dependsOn: [ + ] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + url: { + value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-daily-costdetails\'))}?api-version=${exportsApiVersion}' + type: 'Expression' + } + method: 'PUT' + body: { + value: getExportBodyV2(MSEXPORTS, 'FocusCost', false, 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '') + type: 'Expression' + } + headers: { + 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.CostsDaily@${finOpsToolkitVersion}' + ClientType: 'FinOpsToolkit.Hubs@${finOpsToolkitVersion}' + } + authentication: { + type: 'MSI' + resource: { + value: '@variables(\'resourceManagementUri\')' + type: 'Expression' + } + } + } + } + { // 'Subscription closed month focus export' + name: 'Subscription closed month focus export' + type: 'WebActivity' + dependsOn: [ + { + activity: 'Subscription open month focus export' + dependencyConditions: [ 'Succeeded' ] + } + ] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + url: { + value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-monthly-costdetails\'))}?api-version=${exportsApiVersion}' + type: 'Expression' + } + method: 'PUT' + body: { + value: getExportBodyV2(MSEXPORTS, 'FocusCost', true, 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '') + type: 'Expression' + } + headers: { + 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.CostsMonthly@${finOpsToolkitVersion}' + ClientType: 'FinOpsToolkit.Hubs@${finOpsToolkitVersion}' + } + authentication: { + type: 'MSI' + resource: { + value: '@variables(\'resourceManagementUri\')' + type: 'Expression' + } + } + } + } + ] + } + { // MCA + value: 'mca' + activities: [ + { + name: 'Export Type Unsupported Error' + type: 'Fail' + dependsOn: [] + userProperties: [] + typeProperties: { + message: { + value: '@concat(\'MCA agreements are not supported for managed exports :\',variables(\'exportScope\'))' + type: 'Expression' + } + errorCode: 'ExportTypeUnsupported' + } + } + ] + } + ] + defaultActivities: [ + { + name: 'Export Type Not Defined Error' + type: 'Fail' + dependsOn: [] + userProperties: [] + typeProperties: { + message: { + value: '@concat(\'Unable to determine the export scope type for :\',variables(\'exportScope\'))' + type: 'Expression' + } + errorCode: 'ExportTypeNotDefined' + } + } + ] + } + } + ] + } + } + ] + concurrency: 1 + variables: { + scopesArray: { + type: 'Array' + } + exportName: { + type: 'String' + } + exportScope: { + type: 'String' + } + exportScopeType: { + type: 'String' + } + storageAccountId: { + type: 'String' + defaultValue: storageAccount.id + } + finOpsHub: { + type: 'String' + defaultValue: app.hub.name + } + resourceManagementUri: { + type: 'String' + defaultValue: environment().resourceManager + } + fileName: { + type: 'String' + defaultValue: 'settings.json' + } + folderPath: { + type: 'String' + defaultValue: CONFIG + } + } + } + } +} + +// TODO: Can we move this into hub-types.bicep or merge it here? +module timeZones 'timeZones.bicep' = { + name: 'Microsoft.CostManagement.ManagedExports_TimeZones' + params: { + location: app.hub.location + } +} + +module trigger_SettingsUpdated '../../fx/hub-eventTrigger.bicep' = { + name: 'Microsoft.FinOpsHubs.Core_SettingsUpdatedTrigger' + params: { + dataFactoryName: dataFactory.name + triggerName: '${CONFIG}_SettingsUpdated' + + // TODO: Replace pipeline with event: 'Microsoft.FinOpsHubs.Core.SettingsUpdated' + pipelineName: dataFactory::pipeline_ConfigureExports.name + pipelineParameters: {} + + storageAccountName: app.storage + storageContainer: CONFIG + // TODO: Change this to startswith + storagePathEndsWith: 'settings.json' + } +} + + +//============================================================================== +// Outputs +//============================================================================== + +// None diff --git a/src/templates/finops-hub/modules/azuretimezones.bicep b/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/timeZones.bicep similarity index 100% rename from src/templates/finops-hub/modules/azuretimezones.bicep rename to src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/timeZones.bicep diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep new file mode 100644 index 000000000..6e290e2da --- /dev/null +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep @@ -0,0 +1,1909 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { finOpsToolkitVersion, HubAppProperties, privateRoutingForLinkedServices } from '../../fx/hub-types.bicep' + + +//============================================================================== +// Parameters +//============================================================================== + +@description('Required. FinOps hub app getting deployed.') +param app HubAppProperties + +@description('Optional. Name of the Azure Data Explorer cluster to use for advanced analytics. If empty, Azure Data Explorer will not be deployed. Required to use with Power BI if you have more than $2-5M/mo in costs being monitored. Default: "" (do not use).') +@maxLength(22) +param clusterName string = '' + +// https://learn.microsoft.com/azure/templates/microsoft.kusto/clusters?pivots=deployment-language-bicep#azuresku +@description('Optional. Name of the Azure Data Explorer SKU. Default: "Dev(No SLA)_Standard_E2a_v4".') +@allowed([ + 'Dev(No SLA)_Standard_E2a_v4' // 2 CPU, 16GB RAM, 24GB cache, $110/mo + 'Dev(No SLA)_Standard_D11_v2' // 2 CPU, 14GB RAM, 78GB cache, $121/mo + 'Standard_D11_v2' // 2 CPU, 14GB RAM, 78GB cache, $245/mo + 'Standard_D12_v2' + 'Standard_D13_v2' + 'Standard_D14_v2' + 'Standard_D16d_v5' + 'Standard_D32d_v4' + 'Standard_D32d_v5' + 'Standard_DS13_v2+1TB_PS' + 'Standard_DS13_v2+2TB_PS' + 'Standard_DS14_v2+3TB_PS' + 'Standard_DS14_v2+4TB_PS' + 'Standard_E2a_v4' // 2 CPU, 14GB RAM, 78GB cache, $220/mo + 'Standard_E2ads_v5' + 'Standard_E2d_v4' + 'Standard_E2d_v5' + 'Standard_E4a_v4' + 'Standard_E4ads_v5' + 'Standard_E4d_v4' + 'Standard_E4d_v5' + 'Standard_E8a_v4' + 'Standard_E8ads_v5' + 'Standard_E8as_v4+1TB_PS' + 'Standard_E8as_v4+2TB_PS' + 'Standard_E8as_v5+1TB_PS' + 'Standard_E8as_v5+2TB_PS' + 'Standard_E8d_v4' + 'Standard_E8d_v5' + 'Standard_E8s_v4+1TB_PS' + 'Standard_E8s_v4+2TB_PS' + 'Standard_E8s_v5+1TB_PS' + 'Standard_E8s_v5+2TB_PS' + 'Standard_E16a_v4' + 'Standard_E16ads_v5' + 'Standard_E16as_v4+3TB_PS' + 'Standard_E16as_v4+4TB_PS' + 'Standard_E16as_v5+3TB_PS' + 'Standard_E16as_v5+4TB_PS' + 'Standard_E16d_v4' + 'Standard_E16d_v5' + 'Standard_E16s_v4+3TB_PS' + 'Standard_E16s_v4+4TB_PS' + 'Standard_E16s_v5+3TB_PS' + 'Standard_E16s_v5+4TB_PS' + 'Standard_E64i_v3' + 'Standard_E80ids_v4' + 'Standard_EC8ads_v5' + 'Standard_EC8as_v5+1TB_PS' + 'Standard_EC8as_v5+2TB_PS' + 'Standard_EC16ads_v5' + 'Standard_EC16as_v5+3TB_PS' + 'Standard_EC16as_v5+4TB_PS' + 'Standard_L4s' + 'Standard_L8as_v3' + 'Standard_L8s' + 'Standard_L8s_v2' + 'Standard_L8s_v3' + 'Standard_L16as_v3' + 'Standard_L16s' + 'Standard_L16s_v2' + 'Standard_L16s_v3' + 'Standard_L32as_v3' + 'Standard_L32s_v3' +]) +param clusterSku string = 'Dev(No SLA)_Standard_E2a_v4' + +@description('Optional. Number of nodes to use in the cluster. Allowed values: 1 for the Basic SKU tier and 2-1000 for Standard. Default: 1 for dev/test SKUs, 2 for standard SKUs.') +@minValue(1) +@maxValue(1000) +param clusterCapacity int = 1 + +// TODO: Figure out why this is breaking upgrades +// @description('Optional. Array of external tenant IDs that should have access to the cluster. Default: empty (no external access).') +// param clusterTrustedExternalTenants string[] = [] + +// cSpell:ignore eventhouse +@description('Optional. Microsoft Fabric eventhouse query URI. Default: "" (do not use).') +param fabricQueryUri string = '' + +@description('Optional. Number of capacity units for the Microsoft Fabric capacity. This is the number in your Fabric SKU (e.g., Trial = 1, F2 = 2, F64 = 64). This is used to manage parallelization in data pipelines. If you change capacity, please redeploy the template. Allowed values: 1 for the Fabric trial and 2-2048 based on the assigned Fabric capacity (e.g., F2-F2048). Default: 2.') +@minValue(1) +@maxValue(2048) +param fabricCapacityUnits int = 2 + +@description('Optional. Forces the table to be updated if different from the last time it was deployed.') +param forceUpdateTag string = utcNow() + +@description('Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false.') +param continueOnErrors bool = false + +@description('Required. Number of days of data to retain in the Data Explorer *_raw tables.') +param rawRetentionInDays int + + +//============================================================================== +// Variables +//============================================================================== + +var CONFIG = 'config' +var HUB_DATA_EXPLORER = 'hubDataExplorer' +var HUB_DB = 'Hub' +var INGESTION = 'ingestion' +var INGESTION_DB = 'Ingestion' +var INGESTION_ID_SEPARATOR = '__' + +var ftkReleaseUri = endsWith(finOpsToolkitVersion, '-dev') + ? 'https://github.com/microsoft/finops-toolkit/releases/latest/download' + : 'https://github.com/microsoft/finops-toolkit/releases/download/v${finOpsToolkitVersion}' + +var useFabric = !empty(fabricQueryUri) +var useAzure = !useFabric && !empty(clusterName) + +// cSpell:ignore ftkver, privatelink +var dataExplorerPrivateDnsZoneName = replace('privatelink.${app.hub.location}.${replace(environment().suffixes.storage, 'core', 'kusto')}', '..', '.') + +// Actual = Minimum(ClusterMaximumConcurrentOperations, Number of nodes in cluster * Maximum(1, Core count per node * CoreUtilizationCoefficient)) +var ingestionCapacity = { + 'Dev(No SLA)_Standard_E2a_v4': 1 + 'Dev(No SLA)_Standard_D11_v2': 1 + Standard_D11_v2: 2 + Standard_D12_v2: 4 + Standard_D13_v2: 8 + Standard_D14_v2: 16 + Standard_D16d_v5: 16 + Standard_D32d_v4: 32 + Standard_D32d_v5: 32 + 'Standard_DS13_v2+1TB_PS': 8 + 'Standard_DS13_v2+2TB_PS': 8 + 'Standard_DS14_v2+3TB_PS': 16 + 'Standard_DS14_v2+4TB_PS': 16 + Standard_E2a_v4: 2 + Standard_E2ads_v5: 2 + Standard_E2d_v4: 2 + Standard_E2d_v5: 2 + Standard_E4a_v4: 4 + Standard_E4ads_v5: 4 + Standard_E4d_v4: 4 + Standard_E4d_v5: 4 + Standard_E8a_v4: 8 + Standard_E8ads_v5: 8 + 'Standard_E8as_v4+1TB_PS': 8 + 'Standard_E8as_v4+2TB_PS': 8 + 'Standard_E8as_v5+1TB_PS': 8 + 'Standard_E8as_v5+2TB_PS': 8 + Standard_E8d_v4: 8 + Standard_E8d_v5: 8 + 'Standard_E8s_v4+1TB_PS': 8 + 'Standard_E8s_v4+2TB_PS': 8 + 'Standard_E8s_v5+1TB_PS': 8 + 'Standard_E8s_v5+2TB_PS': 8 + Standard_E16a_v4: 16 + Standard_E16ads_v5: 16 + 'Standard_E16as_v4+3TB_PS': 16 + 'Standard_E16as_v4+4TB_PS': 16 + 'Standard_E16as_v5+3TB_PS': 16 + 'Standard_E16as_v5+4TB_PS': 16 + Standard_E16d_v4: 16 + Standard_E16d_v5: 16 + 'Standard_E16s_v4+3TB_PS': 16 + 'Standard_E16s_v4+4TB_PS': 16 + 'Standard_E16s_v5+3TB_PS': 16 + 'Standard_E16s_v5+4TB_PS': 16 + Standard_E64i_v3: 64 + Standard_E80ids_v4: 80 + Standard_EC8ads_v5: 8 + 'Standard_EC8as_v5+1TB_PS': 8 + 'Standard_EC8as_v5+2TB_PS': 8 + Standard_EC16ads_v5: 16 + 'Standard_EC16as_v5+3TB_PS': 16 + 'Standard_EC16as_v5+4TB_PS': 16 + Standard_L4s: 4 + Standard_L8as_v3: 8 + Standard_L8s: 8 + Standard_L8s_v2: 8 + Standard_L8s_v3: 8 + Standard_L16as_v3: 16 + Standard_L16s: 16 + Standard_L16s_v2: 16 + Standard_L16s_v3: 16 + Standard_L32as_v3: 32 + Standard_L32s_v3: 32 +} + +var dataExplorerIngestionCapacity = useFabric + ? fabricCapacityUnits + : (!useAzure ? 1 : ingestionCapacity[?clusterSku] ?? 1) + +// WORKAROUND: Direct property access fails on cluster updates due to ARM bug +// See: https://github.com/Azure/azure-resource-manager-templates/issues/[issue-number] +var dataExplorerUri = useFabric ? fabricQueryUri : 'https://${cluster.name}.${app.hub.location}.kusto.windows.net' + +//============================================================================== +// Resources +//============================================================================== + +// App registration +module appRegistration '../../fx/hub-app.bicep' = { + name: 'Microsoft.FinOpsHubs.Analytics_Register' + params: { + app: app + version: finOpsToolkitVersion + features: [ + 'DataFactory' + 'Storage' + ] + } +} + +//------------------------------------------------------------------------------ +// Dependencies +//------------------------------------------------------------------------------ + +// Get data factory instance +resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { + name: app.dataFactory + dependsOn: [ + appRegistration + ] +} + +resource blobPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = { + name: 'privatelink.blob.${environment().suffixes.storage}' + dependsOn: [ + appRegistration + ] +} + +resource queuePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = { + name: 'privatelink.queue.${environment().suffixes.storage}' + dependsOn: [ + appRegistration + ] +} + +resource tablePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = { + name: 'privatelink.table.${environment().suffixes.storage}' + dependsOn: [ + appRegistration + ] +} + +resource storage 'Microsoft.Storage/storageAccounts@2022-09-01' existing = { + name: app.storage + dependsOn: [ + appRegistration + ] +} + +//------------------------------------------------------------------------------ +// Cluster + databases +//------------------------------------------------------------------------------ + +// Kusto cluster +resource cluster 'Microsoft.Kusto/clusters@2023-08-15' = if (useAzure) { + name: replace(clusterName, '_', '-') + dependsOn: [ + appRegistration + ] + location: app.hub.location + tags: union(app.tags, app.hub.tagsByResource[?'Microsoft.Kusto/clusters'] ?? {}) + sku: { + name: clusterSku + tier: startsWith(clusterSku, 'Dev(No SLA)_') ? 'Basic' : 'Standard' + capacity: startsWith(clusterSku, 'Dev(No SLA)_') ? 1 : (clusterCapacity == 1 ? 2 : clusterCapacity) + } + identity: { + type: 'SystemAssigned' + } + properties: { + enableStreamingIngest: true + enableAutoStop: false + publicNetworkAccess: app.hub.options.privateRouting ? 'Disabled' : 'Enabled' + // TODO: Figure out why this is breaking upgrades + // trustedExternalTenants: [for tenantId in clusterTrustedExternalTenants: { + // value: tenantId + // }] + } + + resource adfClusterAdmin 'principalAssignments' = { + name: 'adf-mi-cluster-admin' + properties: { + principalType: 'App' + principalId: dataFactory.identity.principalId + tenantId: dataFactory.identity.tenantId + role: 'AllDatabasesAdmin' + } + } + + resource ingestionDb 'databases' = { + name: INGESTION_DB + location: app.hub.location + kind: 'ReadWrite' + } + + resource hubDb 'databases' = { + name: HUB_DB + location: app.hub.location + kind: 'ReadWrite' + } +} + +module ingestion_OpenDataInternalScripts '../../fx/hub-database.bicep' = if (useAzure) { + name: 'Microsoft.FinOpsHubs.Analytics_ADX.IngestionOpenDataInternal' + params: { + clusterName: cluster.name + databaseName: cluster::ingestionDb.name + scripts: { + OpenDataFunctions_resource_type_1: loadTextContent('scripts/OpenDataFunctions_resource_type_1.kql') + OpenDataFunctions_resource_type_2: loadTextContent('scripts/OpenDataFunctions_resource_type_2.kql') + OpenDataFunctions_resource_type_3: loadTextContent('scripts/OpenDataFunctions_resource_type_3.kql') + OpenDataFunctions_resource_type_4: loadTextContent('scripts/OpenDataFunctions_resource_type_4.kql') + OpenDataFunctions_resource_type_5: loadTextContent('scripts/OpenDataFunctions_resource_type_5.kql') + } + continueOnErrors: continueOnErrors + forceUpdateTag: forceUpdateTag + } +} + +module ingestion_InitScripts '../../fx/hub-database.bicep' = if (useAzure) { + name: 'Microsoft.FinOpsHubs.Analytics_ADX.IngestionInit' + dependsOn: [ + ingestion_OpenDataInternalScripts + ] + params: { + clusterName: cluster.name + databaseName: cluster::ingestionDb.name + scripts: { + openData: loadTextContent('scripts/OpenDataFunctions.kql') + common: loadTextContent('scripts/Common.kql') + infra: loadTextContent('scripts/IngestionSetup_HubInfra.kql') + rawTables: replace(loadTextContent('scripts/IngestionSetup_RawTables.kql'), '$$rawRetentionInDays$$', string(rawRetentionInDays)) + } + continueOnErrors: continueOnErrors + forceUpdateTag: forceUpdateTag + } +} + +module ingestion_VersionedScripts '../../fx/hub-database.bicep' = if (useAzure) { + name: 'Microsoft.FinOpsHubs.Analytics_ADX.IngestionVersioned' + dependsOn: [ + ingestion_InitScripts + ] + params: { + clusterName: cluster.name + databaseName: cluster::ingestionDb.name + scripts: { + v1_0: loadTextContent('scripts/IngestionSetup_v1_0.kql') + v1_2: loadTextContent('scripts/IngestionSetup_v1_2.kql') + } + continueOnErrors: continueOnErrors + forceUpdateTag: forceUpdateTag + } +} + +module hub_InitScripts '../../fx/hub-database.bicep' = if (useAzure) { + name: 'Microsoft.FinOpsHubs.Analytics_ADX.HubInit' + dependsOn: [ + ingestion_InitScripts + ] + params: { + clusterName: cluster.name + databaseName: cluster::hubDb.name + scripts: { + common: loadTextContent('scripts/Common.kql') + openData: loadTextContent('scripts/HubSetup_OpenData.kql') + } + continueOnErrors: continueOnErrors + forceUpdateTag: forceUpdateTag + } +} + +module hub_VersionedScripts '../../fx/hub-database.bicep' = if (useAzure) { + name: 'Microsoft.FinOpsHubs.Analytics_ADX.HubVersioned' + dependsOn: [ + ingestion_VersionedScripts + hub_InitScripts + ] + params: { + clusterName: cluster.name + databaseName: cluster::hubDb.name + scripts: { + v1_0: loadTextContent('scripts/HubSetup_v1_0.kql') + v1_2: loadTextContent('scripts/HubSetup_v1_2.kql') + } + continueOnErrors: continueOnErrors + forceUpdateTag: forceUpdateTag + } +} + +module hub_LatestScripts '../../fx/hub-database.bicep' = if (useAzure) { + name: 'Microsoft.FinOpsHubs.Analytics_ADX.HubLatest' + dependsOn: [ + hub_VersionedScripts + ] + params: { + clusterName: cluster.name + databaseName: cluster::hubDb.name + scripts: { + latest: loadTextContent('scripts/HubSetup_Latest.kql') + } + continueOnErrors: continueOnErrors + forceUpdateTag: forceUpdateTag + } +} + +// Authorize Kusto Cluster to read storage +resource clusterStorageAccess 'Microsoft.Authorization/roleAssignments@2022-04-01' = if (useAzure) { + name: guid(cluster.name, subscription().id, 'Storage Blob Data Contributor') + scope: storage + properties: { + description: 'Give "Storage Blob Data Contributor" to the cluster' + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access + principalId: cluster.identity.principalId + // Required in case principal not ready when deploying the assignment + principalType: 'ServicePrincipal' + roleDefinitionId: subscriptionResourceId( + 'Microsoft.Authorization/roleDefinitions', + 'ba92f5b4-2d11-453d-a403-e96b0029c9fe' // Storage Blob Data Contributor -- https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#storage + ) + } +} + +// DNS zone +resource dataExplorerPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (useAzure && app.hub.options.privateRouting) { + name: dataExplorerPrivateDnsZoneName + location: 'global' + tags: union(app.tags, app.hub.tagsByResource[?'Microsoft.Network/privateDnsZones'] ?? {}) + properties: {} +} + +// Link DNS zone to VNet +resource dataExplorerPrivateDnsZoneLink 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2024-06-01' = if (useAzure && app.hub.options.privateRouting) { + name: '${replace(dataExplorerPrivateDnsZone.name, '.', '-')}-link' + location: 'global' + parent: dataExplorerPrivateDnsZone + tags: union(app.tags, app.hub.tagsByResource[?'Microsoft.Network/privateDnsZones/virtualNetworkLinks'] ?? {}) + properties: { + virtualNetwork: { + id: app.hub.routing.networkId + } + registrationEnabled: false + } +} + +// Private endpoint +resource dataExplorerEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (useAzure && app.hub.options.privateRouting) { + name: '${cluster.name}-ep' + location: app.hub.location + tags: union(app.tags, app.hub.tagsByResource[?'Microsoft.Network/privateEndpoints'] ?? {}) + properties: { + subnet: { + id: app.hub.routing.subnets.dataExplorer + } + privateLinkServiceConnections: [ + { + name: 'dataExplorerLink' + properties: { + privateLinkServiceId: cluster.id + groupIds: ['cluster'] + } + } + ] + } +} + +// DNS records for private endpoint +resource dataExplorerPrivateDnsZoneGroup 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups@2023-11-01' = if (useAzure && app.hub.options.privateRouting) { + name: 'dataExplorer-endpoint-zone' + parent: dataExplorerEndpoint + properties: { + privateDnsZoneConfigs: [ + { + name: 'privatelink-westus-kusto-net' + properties: { + privateDnsZoneId: dataExplorerPrivateDnsZone.id + } + } + { + name: 'privatelink-blob-core-windows-net' + properties: { + privateDnsZoneId: blobPrivateDnsZone.id + } + } + { + name: 'privatelink-table-core-windows-net' + properties: { + privateDnsZoneId: tablePrivateDnsZone.id + } + } + { + name: 'privatelink-queue-core-windows-net' + properties: { + privateDnsZoneId: queuePrivateDnsZone.id + } + } + ] + } +} + +//------------------------------------------------------------------------------ +// Data Factory setup +// cSpell:ignore linkedservices +//------------------------------------------------------------------------------ + +resource dataFactoryVNet 'Microsoft.DataFactory/factories/managedVirtualNetworks@2018-06-01' existing = if (useAzure && app.hub.options.privateRouting) { + name: 'default' + parent: dataFactory + + resource dataExplorerManagedPrivateEndpoint 'managedPrivateEndpoints' = { + name: HUB_DATA_EXPLORER + properties: { + name: HUB_DATA_EXPLORER + groupId: 'cluster' + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access + privateLinkResourceId: cluster.id + fqdns: [ + 'https://${replace(clusterName, '_', '-')}.${app.hub.location}.kusto.windows.net' + ] + } + } +} + +module getDataExplorerPrivateEndpointConnections 'dataExplorerEndpoints.bicep' = if (useAzure && app.hub.options.privateRouting) { + name: 'GetDataExplorerPrivateEndpointConnections' + dependsOn: [ + dataFactoryVNet::dataExplorerManagedPrivateEndpoint + ] + params: { + dataExplorerName: cluster.name + } +} + +module approveDataExplorerPrivateEndpointConnections 'dataExplorerEndpoints.bicep' = if (useAzure && app.hub.options.privateRouting) { + name: 'ApproveDataExplorerPrivateEndpointConnections' + params: { + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access + dataExplorerName: cluster.name + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access + privateEndpointConnections: getDataExplorerPrivateEndpointConnections.outputs.privateEndpointConnections + } +} + +// ADX/Fabric linked service +resource linkedService_dataExplorer 'Microsoft.DataFactory/factories/linkedservices@2018-06-01' = if (useAzure || useFabric) { + name: HUB_DATA_EXPLORER + parent: dataFactory + properties: { + type: 'AzureDataExplorer' + parameters: { + database: { + type: 'String' + defaultValue: INGESTION_DB + } + } + typeProperties: { + endpoint: dataExplorerUri + database: '@{linkedService().database}' + tenant: dataFactory.identity.tenantId + servicePrincipalId: dataFactory.identity.principalId + } + ...privateRoutingForLinkedServices(app.hub) + } +} + +// GitHub repository linked service for FTK open data +resource linkedService_ftkRepo 'Microsoft.DataFactory/factories/linkedservices@2018-06-01' = { + name: 'ftkRepo' + parent: dataFactory + properties: { + type: 'HttpServer' + parameters: { + filePath: { + type: 'string' + } + } + typeProperties: { + url: '@concat(\'https://gitapp.hub.com/microsoft/finops-toolkit/\', linkedService().filePath)' + enableServerCertificateValidation: true + authenticationType: 'Anonymous' + } + ...privateRoutingForLinkedServices(app.hub) + } +} + +resource dataset_dataExplorer 'Microsoft.DataFactory/factories/datasets@2018-06-01' = { + name: HUB_DATA_EXPLORER + parent: dataFactory + properties: { + type: 'AzureDataExplorerTable' + linkedServiceName: { + parameters: { + database: '@dataset().database' + } + referenceName: linkedService_dataExplorer.name + type: 'LinkedServiceReference' + } + parameters: { + database: { + type: 'String' + defaultValue: INGESTION_DB // Do not use dynamic reference since that won't work with Fabric + } + table: { type: 'String' } + } + typeProperties: { + table: { + value: '@dataset().table' + type: 'Expression' + } + } + } +} + +resource dataset_ftkReleaseFile 'Microsoft.DataFactory/factories/datasets@2018-06-01' = { + name: 'ftkReleaseFile' + parent: dataFactory + properties: { + linkedServiceName: { + referenceName: linkedService_ftkRepo.name + type: 'LinkedServiceReference' + } + parameters: { + fileName: { + type: 'string' + } + version: { + type: 'string' + defaultValue: finOpsToolkitVersion + } + } + annotations: [] + type: 'DelimitedText' + typeProperties: { + location: { + type: 'HttpServerLocation' + relativeUrl: { + value: '@concat(\'releases/download/v\', dataset().version, \'/\', dataset().fileName)' + type: 'Expression' + } + } + columnDelimiter: ',' + escapeChar: '\\' + firstRowAsHeader: true + quoteChar: '"' + } + schema: [] + } +} + +module trigger_IngestionManifestAdded '../../fx/hub-eventTrigger.bicep' = { + name: 'Microsoft.FinOpsHubs.Core_IngestionManifestAddedTrigger' + params: { + dataFactoryName: dataFactory.name + triggerName: '${INGESTION}_ManifestAdded' + + // TODO: Replace pipeline with event: 'Microsoft.FinOpsHubs.Core.IngestionManifestAdded' + pipelineName: pipeline_ExecuteIngestionETL.name + pipelineParameters: { + folderPath: '@triggerBody().folderPath' + } + + storageAccountName: app.storage + storageContainer: INGESTION + storagePathEndsWith: 'manifest.json' + } +} + +//------------------------------------------------------------------------------ +// config_InitializeHub pipeline +//------------------------------------------------------------------------------ +@description('Initializes the hub instance based on the configuration settings.') +resource pipeline_InitializeHub 'Microsoft.DataFactory/factories/pipelines@2018-06-01' = { + name: '${CONFIG}_InitializeHub' + parent: dataFactory + properties: { + activities: [ + { // Get Config + name: 'Get Config' + type: 'Lookup' + dependsOn: [] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'JsonSource' + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: true + enablePartitionDiscovery: false + } + formatSettings: { + type: 'JsonReadSettings' + } + } + dataset: { + referenceName: CONFIG + type: 'DatasetReference' + } + } + } + { // Set Version + name: 'Set Version' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Get Config' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + variableName: 'version' + value: { + value: '@activity(\'Get Config\').output.firstRow.version' + type: 'Expression' + } + } + } + { // Set Scopes + name: 'Set Scopes' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Get Config' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + variableName: 'scopes' + value: { + value: '@string(activity(\'Get Config\').output.firstRow.scopes)' + type: 'Expression' + } + } + } + { // Set Retention + name: 'Set Retention' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Get Config' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + variableName: 'retention' + value: { + value: '@string(activity(\'Get Config\').output.firstRow.retention)' + type: 'Expression' + } + } + } + { // Until Capacity Is Available + name: 'Until Capacity Is Available' + type: 'Until' + dependsOn: [ + { + activity: 'Set Version' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Set Scopes' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Set Retention' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + expression: { + value: '@equals(variables(\'tryAgain\'), false)' + type: 'Expression' + } + activities: [ + { // Confirm Ingestion Capacity + name: 'Confirm Ingestion Capacity' + type: 'AzureDataExplorerCommand' + dependsOn: [] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + // cSpell:ignore Ingestions + command: '.show capacity | where Resource == \'Ingestions\' | project Remaining' + commandTimeout: '00:20:00' + } + linkedServiceName: { + referenceName: linkedService_dataExplorer.name + type: 'LinkedServiceReference' + parameters: { + database: INGESTION_DB // Do not use dynamic reference since that won't work with Fabric + } + } + } + { // If Has Capacity + name: 'If Has Capacity' + type: 'IfCondition' + dependsOn: [ + { + activity: 'Confirm Ingestion Capacity' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + expression: { + value: '@or(equals(activity(\'Confirm Ingestion Capacity\').output.count, 0), greater(activity(\'Confirm Ingestion Capacity\').output.value[0].Remaining, 0))' + type: 'Expression' + } + ifFalseActivities: [ + { // Wait for Ingestion + name: 'Wait for Ingestion' + type: 'Wait' + dependsOn: [] + userProperties: [] + typeProperties: { + waitTimeInSeconds: 15 + } + } + { // Try Again + name: 'Try Again' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Wait for Ingestion' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'tryAgain' + value: true + } + } + ] + ifTrueActivities: [ + { // Save ingestion policy in ADX + name: 'Set ingestion policy in ADX' + type: 'AzureDataExplorerCommand' + dependsOn: [] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + command: { + // Do not attempt to set the ingestion policy if using Fabric; use a simple query as a placeholder + value: useFabric + ? '.show database ${INGESTION_DB} policy managed_identity' + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access + : '.alter-merge database ${INGESTION_DB} policy managed_identity "[ { \'ObjectId\' : \'${cluster.identity.principalId}\', \'AllowedUsages\' : \'NativeIngestion\' }]"' + type: 'Expression' + } + commandTimeout: '00:20:00' + } + linkedServiceName: { + referenceName: linkedService_dataExplorer.name + type: 'LinkedServiceReference' + parameters: { + database: INGESTION_DB // Do not use dynamic reference since that won't work with Fabric + } + } + } + { // Save Hub Settings in ADX + name: 'Save Hub Settings in ADX' + type: 'AzureDataExplorerCommand' + dependsOn: [ + { + activity: 'Set ingestion policy in ADX' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + command: { + // cSpell:ignore isnull, isnotempty + value: '@concat(\'.append HubSettingsLog <| print version="\', variables(\'version\'), \'",scopes=dynamic(\', variables(\'scopes\'), \'),retention=dynamic(\', variables(\'retention\'), \') | extend scopes = iff(isnull(scopes[0]), pack_array(scopes), scopes) | mv-apply scopeObj = scopes on (where isnotempty(scopeObj.scope) | summarize scopes = make_set(scopeObj.scope))\')' + type: 'Expression' + } + commandTimeout: '00:20:00' + } + linkedServiceName: { + referenceName: linkedService_dataExplorer.name + type: 'LinkedServiceReference' + parameters: { + database: INGESTION_DB // Do not use dynamic reference since that won't work with Fabric + } + } + } + { // Update PricingUnits in ADX + name: 'Update PricingUnits in ADX' + type: 'AzureDataExplorerCommand' + dependsOn: [ + { + activity: 'Save Hub Settings in ADX' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + // cSpell:ignore externaldata + command: '.set-or-replace PricingUnits <| externaldata(x_PricingUnitDescription: string, AccountTypes: string, x_PricingBlockSize: decimal, PricingUnit: string)[@"${ftkReleaseUri}/PricingUnits.csv"] with (format="csv", ignoreFirstRecord=true) | project-away AccountTypes' + commandTimeout: '00:20:00' + } + linkedServiceName: { + referenceName: linkedService_dataExplorer.name + type: 'LinkedServiceReference' + parameters: { + database: INGESTION_DB // Do not use dynamic reference since that won't work with Fabric + } + } + } + { // Update Regions in ADX + name: 'Update Regions in ADX' + type: 'AzureDataExplorerCommand' + dependsOn: [ + { + activity: 'Update PricingUnits in ADX' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + command: '.set-or-replace Regions <| externaldata(ResourceLocation: string, RegionId: string, RegionName: string)[@"${ftkReleaseUri}/Regions.csv"] with (format="csv", ignoreFirstRecord=true)' + commandTimeout: '00:20:00' + } + linkedServiceName: { + referenceName: linkedService_dataExplorer.name + type: 'LinkedServiceReference' + parameters: { + database: INGESTION_DB // Do not use dynamic reference since that won't work with Fabric + } + } + } + { // Update ResourceTypes in ADX + name: 'Update ResourceTypes in ADX' + type: 'AzureDataExplorerCommand' + dependsOn: [ + { + activity: 'Update Regions in ADX' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + command: '.set-or-replace ResourceTypes <| externaldata(x_ResourceType: string, SingularDisplayName: string, PluralDisplayName: string, LowerSingularDisplayName: string, LowerPluralDisplayName: string, IsPreview: bool, Description: string, IconUri: string, Links: string)[@"${ftkReleaseUri}/ResourceTypes.csv"] with (format="csv", ignoreFirstRecord=true) | project-away Links' + commandTimeout: '00:20:00' + } + linkedServiceName: { + referenceName: linkedService_dataExplorer.name + type: 'LinkedServiceReference' + parameters: { + database: INGESTION_DB // Do not use dynamic reference since that won't work with Fabric + } + } + } + { // Update Services in ADX + name: 'Update Services in ADX' + type: 'AzureDataExplorerCommand' + dependsOn: [ + { + activity: 'Update ResourceTypes in ADX' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + command: '.set-or-replace Services <| externaldata(x_ConsumedService: string, x_ResourceType: string, ServiceName: string, ServiceCategory: string, ServiceSubcategory: string, PublisherName: string, x_PublisherCategory: string, x_Environment: string, x_ServiceModel: string)[@"${ftkReleaseUri}/Services.csv"] with (format="csv", ignoreFirstRecord=true)' + commandTimeout: '00:20:00' + } + linkedServiceName: { + referenceName: linkedService_dataExplorer.name + type: 'LinkedServiceReference' + parameters: { + database: INGESTION_DB // Do not use dynamic reference since that won't work with Fabric + } + } + } + { // Ingestion Complete + name: 'Ingestion Complete' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Update Services in ADX' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'tryAgain' + value: false + } + } + ] + } + } + { // Abort On Error + name: 'Abort On Error' + type: 'SetVariable' + dependsOn: [ + { + activity: 'If Has Capacity' + dependencyConditions: [ + 'Failed' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'tryAgain' + value: false + } + } + ] + timeout: '0.02:00:00' + } + } + { // Timeout Error + name: 'Timeout Error' + type: 'Fail' + dependsOn: [ + { + activity: 'Until Capacity Is Available' + dependencyConditions: [ + 'Failed' + ] + } + ] + userProperties: [] + typeProperties: { + message: 'Data Explorer ingestion timed out after 2 hours while waiting for available capacity. Please re-run this pipeline to re-attempt ingestion. If you continue to see this error, please report an issue at https://aka.ms/ftk/ideas.' + errorCode: 'DataExplorerIngestionTimeout' + } + } + ] + concurrency: 1 + variables: { + version: { + type: 'String' + } + scopes: { + type: 'String' + } + retention: { + type: 'String' + } + tryAgain: { + type: 'Boolean' + defaultValue: true + } + } + } +} + +//------------------------------------------------------------------------------ +// ingestion_ETL_dataExplorer pipeline +// Triggered by ingestion_ExecuteETL +//------------------------------------------------------------------------------ +@description('Ingests parquet data into an Azure Data Explorer cluster.') +resource pipeline_ToDataExplorer 'Microsoft.DataFactory/factories/pipelines@2018-06-01' = if (useAzure || useFabric) { + name: '${INGESTION}_ETL_dataExplorer' + parent: dataFactory + properties: { + activities: [ + { // Read Hub Config + name: 'Read Hub Config' + description: 'Read the hub config to determine how long data should be retained.' + type: 'Lookup' + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + source: { + type: 'JsonSource' + storeSettings: { + type: 'AzureBlobFSReadSettings' + recursive: false + enablePartitionDiscovery: false + } + formatSettings: { + type: 'JsonReadSettings' + } + } + dataset: { + referenceName: CONFIG + type: 'DatasetReference' + parameters: { + fileName: 'settings.json' + folderPath: CONFIG + } + } + } + } + { // Set Final Retention Months + name: 'Set Final Retention Months' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Read Hub Config' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'finalRetentionMonths' + value: { + value: '@coalesce(activity(\'Read Hub Config\').output.firstRow.retention.final.months, 999)' + type: 'Expression' + } + } + } + { // Until Capacity Is Available + name: 'Until Capacity Is Available' + type: 'Until' + dependsOn: [ + { + activity: 'Set Final Retention Months' + dependencyConditions: [ + 'Completed' + 'Skipped' + ] + } + ] + userProperties: [] + typeProperties: { + expression: { + value: '@equals(variables(\'tryAgain\'), false)' + type: 'Expression' + } + activities: [ + { // Confirm Ingestion Capacity + name: 'Confirm Ingestion Capacity' + type: 'AzureDataExplorerCommand' + dependsOn: [] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + command: '.show capacity | where Resource == \'Ingestions\' | project Remaining' + commandTimeout: '00:20:00' + } + linkedServiceName: { + referenceName: linkedService_dataExplorer.name + type: 'LinkedServiceReference' + } + } + { // If Has Capacity + name: 'If Has Capacity' + type: 'IfCondition' + dependsOn: [ + { + activity: 'Confirm Ingestion Capacity' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + expression: { + value: '@or(equals(activity(\'Confirm Ingestion Capacity\').output.count, 0), greater(activity(\'Confirm Ingestion Capacity\').output.value[0].Remaining, 0))' + type: 'Expression' + } + ifFalseActivities: [ + { // Wait for Ingestion + name: 'Wait for Ingestion' + type: 'Wait' + dependsOn: [] + userProperties: [] + typeProperties: { + waitTimeInSeconds: 15 + } + } + { // Try Again + name: 'Try Again' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Wait for Ingestion' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'tryAgain' + value: true + } + } + ] + ifTrueActivities: [ + { // Pre-Ingest Cleanup + name: 'Pre-Ingest Cleanup' + description: 'Cost Management exports include all month-to-date data from the previous export run. To ensure data is not double-reported, it must be dropped from the raw table before ingestion completes. Remove previous ingestions into the raw table for the month and any previous runs of the current ingestion month file in any table.' + type: 'AzureDataExplorerCommand' + dependsOn: [] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + typeProperties: { + command: { + value: '@concat(\'.drop extents <| .show extents | where (TableName == "\', pipeline().parameters.table, \'" and Tags !has "drop-by:\', pipeline().parameters.ingestionId, \'" and Tags has "drop-by:\', pipeline().parameters.folderPath, \'") or (Tags has "drop-by:\', pipeline().parameters.ingestionId, \'" and Tags has "drop-by:\', pipeline().parameters.folderPath, \'/\', pipeline().parameters.originalFileName, \'")\')' + type: 'Expression' + } + commandTimeout: '00:20:00' + } + linkedServiceName: { + referenceName: linkedService_dataExplorer.name + type: 'LinkedServiceReference' + parameters: { + database: INGESTION_DB // Do not use dynamic reference since that won't work with Fabric + } + } + } + { // Ingest Data + name: 'Ingest Data' + type: 'AzureDataExplorerCommand' + dependsOn: [ + { + activity: 'Pre-Ingest Cleanup' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + timeout: '0.12:00:00' + retry: 3 + retryIntervalInSeconds: 120 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + command: { + // cSpell:ignore abfss, toscalar + value: '@concat(\'.ingest into table \', pipeline().parameters.table, \' ("abfss://${INGESTION}@${app.storage}.dfs.${environment().suffixes.storage}/\', pipeline().parameters.folderPath, \'/\', pipeline().parameters.fileName, \';${useFabric ? 'impersonate' : 'managed_identity=system'}") with (format="parquet", ingestionMappingReference="\', pipeline().parameters.table, \'_mapping", tags="[\\"drop-by:\', pipeline().parameters.ingestionId, \'\\", \\"drop-by:\', pipeline().parameters.folderPath, \'/\', pipeline().parameters.originalFileName, \'\\", \\"drop-by:ftk-version-${finOpsToolkitVersion}\\"]"); print Success = assert(iff(toscalar($command_results | project-keep HasErrors) == false, true, false), "Ingestion Failed")\')' + type: 'Expression' + } + commandTimeout: '01:00:00' + } + linkedServiceName: { + referenceName: linkedService_dataExplorer.name + type: 'LinkedServiceReference' + parameters: { + database: INGESTION_DB // Do not use dynamic reference since that won't work with Fabric + } + } + } + { // Post-Ingest Cleanup + name: 'Post-Ingest Cleanup' + description: 'Cost Management exports include all month-to-date data from the previous export run. To ensure data is not double-reported, it must be dropped after ingestion completes. Remove the current ingestion month file from raw and any old ingestions for the month from the final table.' + type: 'AzureDataExplorerCommand' + dependsOn: [ + { + activity: 'Ingest Data' + dependencyConditions: [ + 'Completed' + ] + } + ] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + typeProperties: { + command: { + // cSpell:ignore startofmonth, strcat, todatetime + value: '@concat(\'.drop extents <| .show extents | extend isOldFinalData = (TableName startswith "\', replace(pipeline().parameters.table, \'_raw\', \'_final_v\'), \'" and Tags !has "drop-by:\', pipeline().parameters.ingestionId, \'" and Tags has "drop-by:\', pipeline().parameters.folderPath, \'") | extend isPastFinalRetention = (TableName startswith "\', replace(pipeline().parameters.table, \'_raw\', \'_final_v\'), \'" and todatetime(substring(strcat(replace_string(extract("drop-by:[A-Za-z]+/(\\\\d{4}/\\\\d{2}(/\\\\d{2})?)", 1, Tags), "/", "-"), "-01"), 0, 10)) < datetime_add("month", -\', if(lessOrEquals(variables(\'finalRetentionMonths\'), 0), 0, variables(\'finalRetentionMonths\')), \', startofmonth(now()))) | where isOldFinalData or isPastFinalRetention\')' + type: 'Expression' + } + commandTimeout: '00:20:00' + } + linkedServiceName: { + referenceName: linkedService_dataExplorer.name + type: 'LinkedServiceReference' + parameters: { + database: INGESTION_DB // Do not use dynamic reference since that won't work with Fabric + } + } + } + { // Ingestion Complete + name: 'Ingestion Complete' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Post-Ingest Cleanup' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'tryAgain' + value: false + } + } + { // Abort On Ingestion Error + name: 'Abort On Ingestion Error' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Ingest Data' + dependencyConditions: [ + 'Failed' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'tryAgain' + value: false + } + } + { // Error: DataExplorerIngestionFailed + name: 'Ingestion Failed Error' + type: 'Fail' + dependsOn: [ + { + activity: 'Abort On Ingestion Error' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + message: { + value: '@concat(\'Data Explorer ingestion into the \', pipeline().parameters.table, \' table failed. Please fix the error and rerun ingestion for the following folder path: "\', pipeline().parameters.folderPath, \'". File: \', pipeline().parameters.originalFileName, \'. Error: \', if(greater(length(activity(\'Ingest Data\').output.errors), 0), activity(\'Ingest Data\').output.errors[0].Message, \'Unknown\'), \' (Code: \', if(greater(length(activity(\'Ingest Data\').output.errors), 0), activity(\'Ingest Data\').output.errors[0].Code, \'None\'), \')\')' + type: 'Expression' + } + errorCode: 'DataExplorerIngestionFailed' + } + } + { // Abort On Pre-Ingest Drop Error + name: 'Abort On Pre-Ingest Drop Error' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Pre-Ingest Cleanup' + dependencyConditions: [ + 'Failed' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'tryAgain' + value: false + } + } + { // Error: DataExplorerPreIngestionDropFailed + name: 'Pre-Ingest Drop Failed Error' + type: 'Fail' + dependsOn: [ + { + activity: 'Abort On Pre-Ingest Drop Error' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + message: { + value: '@concat(\'Data Explorer pre-ingestion cleanup (drop extents from raw table) for the \', pipeline().parameters.table, \' table failed. Ingestion was not completed. Please fix the error and rerun ingestion for the following folder path: "\', pipeline().parameters.folderPath, \'". File: \', pipeline().parameters.originalFileName, \'. Error: \', if(greater(length(activity(\'Pre-Ingest Cleanup\').output.errors), 0), activity(\'Pre-Ingest Cleanup\').output.errors[0].Message, \'Unknown\'), \' (Code: \', if(greater(length(activity(\'Pre-Ingest Cleanup\').output.errors), 0), activity(\'Pre-Ingest Cleanup\').output.errors[0].Code, \'None\'), \')\')' + type: 'Expression' + } + errorCode: 'DataExplorerPreIngestionDropFailed' + } + } + { // Abort On Post-Ingest Drop Error + name: 'Abort On Post-Ingest Drop Error' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Post-Ingest Cleanup' + dependencyConditions: [ + 'Failed' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'tryAgain' + value: false + } + } + { // Error: DataExplorerPostIngestionDropFailed + name: 'Post-Ingest Drop Failed Error' + type: 'Fail' + dependsOn: [ + { + activity: 'Abort On Post-Ingest Drop Error' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + message: { + value: '@concat(\'Data Explorer post-ingestion cleanup (drop extents from final tables) for the \', replace(pipeline().parameters.table, \'_raw\', \'_final_*\'), \' table failed. Please fix the error and rerun ingestion for the following folder path: "\', pipeline().parameters.folderPath, \'". File: \', pipeline().parameters.originalFileName, \'. Error: \', if(greater(length(activity(\'Post-Ingest Cleanup\').output.errors), 0), activity(\'Post-Ingest Cleanup\').output.errors[0].Message, \'Unknown\'), \' (Code: \', if(greater(length(activity(\'Post-Ingest Cleanup\').output.errors), 0), activity(\'Post-Ingest Cleanup\').output.errors[0].Code, \'None\'), \')\')' + type: 'Expression' + } + errorCode: 'DataExplorerPostIngestionDropFailed' + } + } + ] + } + } + ] + timeout: '0.02:00:00' + } + } + ] + parameters: { + folderPath: { + type: 'string' + } + fileName: { + type: 'string' + } + originalFileName: { + type: 'string' + } + ingestionId: { + type: 'string' + } + table: { + type: 'string' + } + } + variables: { + tryAgain: { + type: 'Boolean' + defaultValue: true + } + logRetentionDays: { + type: 'Integer' + defaultValue: 0 + } + finalRetentionMonths: { + type: 'Integer' + defaultValue: 999 + } + } + annotations: [] + } +} + +//------------------------------------------------------------------------------ +// ingestion_ExecuteETL pipeline +// Triggered by ingestion_ManifestAdded trigger +//------------------------------------------------------------------------------ +@description('Queues the ingestion_ETL_dataExplorer pipeline to account for Data Factory pipeline trigger limits.') +resource pipeline_ExecuteIngestionETL 'Microsoft.DataFactory/factories/pipelines@2018-06-01' = if (useAzure || useFabric) { + name: '${INGESTION}_ExecuteETL' + parent: dataFactory + properties: { + concurrency: 1 + activities: [ + { // Wait + name: 'Wait' + description: 'Files may not be available immediately after being created.' + type: 'Wait' + dependsOn: [] + userProperties: [] + typeProperties: { + waitTimeInSeconds: 60 + } + } + { // Set Container Folder Path + name: 'Set Container Folder Path' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Wait' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'containerFolderPath' + value: { + value: '@join(skip(array(split(pipeline().parameters.folderPath, \'/\')), 1), \'/\')' + type: 'Expression' + } + } + } + { // Get Existing Parquet Files + name: 'Get Existing Parquet Files' + description: 'Get the previously ingested files so we can get file paths.' + type: 'GetMetadata' + dependsOn: [ + { + activity: 'Set Container Folder Path' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + timeout: '0.12:00:00' + retry: 0 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + dataset: { + referenceName: 'ingestion_files' + type: 'DatasetReference' + parameters: { + folderPath: '@variables(\'containerFolderPath\')' + } + } + fieldList: [ + 'childItems' + ] + storeSettings: { + type: 'AzureBlobFSReadSettings' + enablePartitionDiscovery: false + } + formatSettings: { + type: 'ParquetReadSettings' + } + } + } + { // Filter Out Folders and manifest files + name: 'Filter Out Folders' + description: 'Remove any folders or manifest files.' + type: 'Filter' + dependsOn: [ + { + activity: 'Get Existing Parquet Files' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + items: { + value: '@if(contains(activity(\'Get Existing Parquet Files\').output, \'childItems\'), activity(\'Get Existing Parquet Files\').output.childItems, json(\'[]\'))' + type: 'Expression' + } + condition: { + value: '@and(equals(item().type, \'File\'), not(contains(toLower(item().name), \'manifest.json\')))' + type: 'Expression' + } + } + } + { // Set Ingestion Timestamp + name: 'Set Ingestion Timestamp' + type: 'SetVariable' + dependsOn: [ + { + activity: 'Wait' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + policy: { + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + variableName: 'timestamp' + value: { + value: '@utcNow()' + type: 'Expression' + } + } + } + { // For Each Old File + name: 'For Each Old File' + description: 'Loop thru each of the existing files.' + type: 'ForEach' + dependsOn: [ + { + activity: 'Filter Out Folders' + dependencyConditions: [ + 'Succeeded' + ] + } + { + activity: 'Set Ingestion Timestamp' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + batchCount: dataExplorerIngestionCapacity // Concurrency limit + items: { + value: '@activity(\'Filter Out Folders\').output.Value' + type: 'Expression' + } + activities: [ + { // Execute + name: 'Execute' + description: 'Run the ADX ETL pipeline.' + type: 'ExecutePipeline' + dependsOn: [] + policy: { + secureInput: false + } + userProperties: [] + typeProperties: { + pipeline: { + referenceName: pipeline_ToDataExplorer.name + type: 'PipelineReference' + } + waitOnCompletion: true + parameters: { + folderPath: { + value: '@variables(\'containerFolderPath\')' + type: 'Expression' + } + fileName: { + value: '@item().name' + type: 'Expression' + } + originalFileName: { + value: '@last(array(split(item().name, \'${INGESTION_ID_SEPARATOR}\')))' + type: 'Expression' + } + ingestionId: { + value: '@concat(first(array(split(item().name, \'${INGESTION_ID_SEPARATOR}\'))), \'_\', variables(\'timestamp\'))' + type: 'Expression' + } + table: { + value: '@concat(first(array(split(variables(\'containerFolderPath\'), \'/\'))), \'_raw\')' + type: 'Expression' + } + } + } + } + ] + } + } + { // If No Files + name: 'If No Files' + description: 'If there are no files found, fail the pipeline.' + type: 'IfCondition' + dependsOn: [ + { + activity: 'Filter Out Folders' + dependencyConditions: [ + 'Succeeded' + ] + } + ] + userProperties: [] + typeProperties: { + expression: { + value: '@equals(length(activity(\'Filter Out Folders\').output.Value), 0)' + type: 'Expression' + } + ifTrueActivities: [ + { // Error: IngestionFilesNotFound + name: 'Files Not Found' + type: 'Fail' + dependsOn: [] + userProperties: [] + typeProperties: { + message: { + value: '@concat(\'Unable to locate parquet files to ingest from the \', pipeline().parameters.folderPath, \' path. Please confirm the folder path is the full path, including the "ingestion" container and not starting with or ending with a slash ("/").\')' + type: 'Expression' + } + errorCode: 'IngestionFilesNotFound' + } + } + ] + } + } + ] + parameters: { + folderPath: { + type: 'string' + } + } + variables: { + containerFolderPath: { + type: 'string' + } + timestamp: { + type: 'string' + } + } + annotations: [ + 'New ingestion' + ] + } +} + +// Run initialization pipeline after everything is deployed +module runInitializationPipeline '../../fx/hub-initialize.bicep' = if (useAzure || useFabric) { + name: 'Microsoft.FinOpsHubs.Analytics_InitializeHub' + params: { + app: app + dataFactoryInstances: [ + app.dataFactory + ] + identityName: appRegistration.outputs.triggerManagerIdentityName + startPipelines: [ + pipeline_InitializeHub.name + ] + } +} + + +//============================================================================== +// Outputs +//============================================================================== + +@description('The resource ID of the cluster.') +#disable-next-line BCP318 // Null safety warning for conditional resource access +output clusterId string = useFabric ? '' : cluster.id + +@description('The ID of the cluster system assigned managed identity.') +#disable-next-line BCP318 // Null safety warning for conditional resource access +output principalId string = useFabric ? '' : cluster.identity.principalId + +@description('The name of the cluster.') +#disable-next-line BCP318 // Null safety warning for conditional resource access +output clusterName string = useFabric ? '' : cluster.name + +@description('The URI of the cluster.') +output clusterUri string = dataExplorerUri + +@description('The name of the database for data ingestion.') +output ingestionDbName string = INGESTION_DB // Don't use cluster DB reference since that won't work for Fabric + +@description('The name of the database for queries.') +output hubDbName string = HUB_DB // Don't use cluster DB reference since that won't work for Fabric + +@description('Max ingestion capacity of the cluster.') +output clusterIngestionCapacity int = dataExplorerIngestionCapacity diff --git a/src/templates/finops-hub/modules/dataExplorerEndpoints.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/dataExplorerEndpoints.bicep similarity index 100% rename from src/templates/finops-hub/modules/dataExplorerEndpoints.bicep rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/dataExplorerEndpoints.bicep diff --git a/src/templates/finops-hub/modules/scripts/Common.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/Common.kql similarity index 100% rename from src/templates/finops-hub/modules/scripts/Common.kql rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/Common.kql diff --git a/src/templates/finops-hub/modules/scripts/HubSetup_Latest.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_Latest.kql similarity index 100% rename from src/templates/finops-hub/modules/scripts/HubSetup_Latest.kql rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_Latest.kql diff --git a/src/templates/finops-hub/modules/scripts/HubSetup_OpenData.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_OpenData.kql similarity index 100% rename from src/templates/finops-hub/modules/scripts/HubSetup_OpenData.kql rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_OpenData.kql diff --git a/src/templates/finops-hub/modules/scripts/HubSetup_v1_0.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_v1_0.kql similarity index 100% rename from src/templates/finops-hub/modules/scripts/HubSetup_v1_0.kql rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_v1_0.kql diff --git a/src/templates/finops-hub/modules/scripts/HubSetup_v1_2.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_v1_2.kql similarity index 100% rename from src/templates/finops-hub/modules/scripts/HubSetup_v1_2.kql rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_v1_2.kql diff --git a/src/templates/finops-hub/modules/scripts/IngestionSetup_HubInfra.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_HubInfra.kql similarity index 100% rename from src/templates/finops-hub/modules/scripts/IngestionSetup_HubInfra.kql rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_HubInfra.kql diff --git a/src/templates/finops-hub/modules/scripts/IngestionSetup_RawTables.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_RawTables.kql similarity index 100% rename from src/templates/finops-hub/modules/scripts/IngestionSetup_RawTables.kql rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_RawTables.kql diff --git a/src/templates/finops-hub/modules/scripts/IngestionSetup_v1_0.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_v1_0.kql similarity index 100% rename from src/templates/finops-hub/modules/scripts/IngestionSetup_v1_0.kql rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_v1_0.kql diff --git a/src/templates/finops-hub/modules/scripts/IngestionSetup_v1_2.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_v1_2.kql similarity index 100% rename from src/templates/finops-hub/modules/scripts/IngestionSetup_v1_2.kql rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_v1_2.kql diff --git a/src/templates/finops-hub/modules/scripts/OpenDataFunctions.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions.kql similarity index 100% rename from src/templates/finops-hub/modules/scripts/OpenDataFunctions.kql rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions.kql diff --git a/src/templates/finops-hub/modules/scripts/OpenDataFunctions_resource_type_1.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions_resource_type_1.kql similarity index 100% rename from src/templates/finops-hub/modules/scripts/OpenDataFunctions_resource_type_1.kql rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions_resource_type_1.kql diff --git a/src/templates/finops-hub/modules/scripts/OpenDataFunctions_resource_type_2.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions_resource_type_2.kql similarity index 100% rename from src/templates/finops-hub/modules/scripts/OpenDataFunctions_resource_type_2.kql rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions_resource_type_2.kql diff --git a/src/templates/finops-hub/modules/scripts/OpenDataFunctions_resource_type_3.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions_resource_type_3.kql similarity index 100% rename from src/templates/finops-hub/modules/scripts/OpenDataFunctions_resource_type_3.kql rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions_resource_type_3.kql diff --git a/src/templates/finops-hub/modules/scripts/OpenDataFunctions_resource_type_4.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions_resource_type_4.kql similarity index 100% rename from src/templates/finops-hub/modules/scripts/OpenDataFunctions_resource_type_4.kql rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions_resource_type_4.kql diff --git a/src/templates/finops-hub/modules/scripts/OpenDataFunctions_resource_type_5.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions_resource_type_5.kql similarity index 100% rename from src/templates/finops-hub/modules/scripts/OpenDataFunctions_resource_type_5.kql rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/OpenDataFunctions_resource_type_5.kql diff --git a/src/templates/finops-hub/modules/scripts/Copy-FileToAzureBlob.ps1 b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/Copy-FileToAzureBlob.ps1 similarity index 100% rename from src/templates/finops-hub/modules/scripts/Copy-FileToAzureBlob.ps1 rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/Copy-FileToAzureBlob.ps1 diff --git a/src/templates/finops-hub/modules/core.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/app.bicep similarity index 50% rename from src/templates/finops-hub/modules/core.bicep rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/app.bicep index 46d3cb0a8..5a903c536 100644 --- a/src/templates/finops-hub/modules/core.bicep +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/app.bicep @@ -1,15 +1,15 @@ // Copyright (c) Microsoft Corporation. // Licensed under the MIT License. -import { HubProperties } from 'hub-types.bicep' +import { finOpsToolkitVersion, HubAppProperties } from '../../fx/hub-types.bicep' //============================================================================== // Parameters //============================================================================== -@description('Required. FinOps hub instance to deploy the app to.') -param hub HubProperties +@description('Required. FinOps hub app getting deployed.') +param app HubAppProperties @description('Optional. List of scope IDs to monitor and ingest cost for.') param scopesToMonitor array @@ -29,75 +29,83 @@ param rawRetentionInDays int = 0 param finalRetentionInMonths int = 13 -//------------------------------------------------------------------------------ -// Temporary parameters that should be removed in the future -//------------------------------------------------------------------------------ - -// TODO: Consider moving telemetryString generation to hub-types.bicep -@description('Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed.') -param telemetryString string = '' - - //============================================================================== // Variables //============================================================================== -var app = appRegistration.outputs.app +var CONFIG = 'config' +var INGESTION = 'ingestion' //============================================================================== // Resources //============================================================================== +// Networking infrastructure +module infrastructure 'infrastructure.bicep' = { + name: 'Microsoft.FinOpsHubs.Core_Infrastructure' + params: { + hub: app.hub + } +} + // Register app -module appRegistration 'hub-app.bicep' = { +module appRegistration '../../fx/hub-app.bicep' = { name: 'Microsoft.FinOpsHubs.Core_Register' + dependsOn: [ + infrastructure + ] params: { - hub: hub - publisher: 'Microsoft FinOps hubs' - namespace: 'Microsoft.FinOpsHubs' - appName: 'Core' - displayName: 'FinOps hub core' - appVersion: loadTextContent('ftkver.txt') // cSpell:ignore ftkver + app: app + version: finOpsToolkitVersion features: [ 'DataFactory' 'Storage' ] - telemetryString: telemetryString } } // Create config container -module configContainer 'hub-storage.bicep' = { +module configContainer '../../fx/hub-storage.bicep' = { name: 'Microsoft.FinOpsHubs.Core_Storage.ConfigContainer' + dependsOn: [ + appRegistration + ] params: { app: app - container: 'config' + container: CONFIG forceCreateBlobManagerIdentity: true } } // Create ingestion container -module ingestionContainer 'hub-storage.bicep' = { +module ingestionContainer '../../fx/hub-storage.bicep' = { name: 'Microsoft.FinOpsHubs.Core_Storage.IngestionContainer' + dependsOn: [ + appRegistration + ] params: { app: app - container: 'ingestion' + container: INGESTION } } // Create/update Settings.json -module uploadSettings 'hub-deploymentScript.bicep' = { +module uploadSettings '../../fx/hub-deploymentScript.bicep' = { name: 'Microsoft.FinOpsHubs.Core_Storage.UpdateSettings' + dependsOn: [ + appRegistration + ] params: { app: app identityName: configContainer.outputs.identityName scriptName: '${app.storage}_uploadSettings' + scriptContent: loadTextContent('Copy-FileToAzureBlob.ps1') environmentVariables: [ { // cSpell:ignore ftkver name: 'ftkVersion' - value: loadTextContent('./ftkver.txt') + value: finOpsToolkitVersion } { name: 'scopes' @@ -128,10 +136,107 @@ module uploadSettings 'hub-deploymentScript.bicep' = { value: 'config' } ] - scriptContent: loadTextContent('./scripts/Copy-FileToAzureBlob.ps1') } } +// Data Factory +resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { + name: app.dataFactory + dependsOn: [ + appRegistration + ] + + // Config dataset + resource dataset_config 'datasets' = { + name: CONFIG + properties: { + linkedServiceName: { + referenceName: app.storage + type: 'LinkedServiceReference' + } + type: 'Json' + typeProperties: { + location: { + type: 'AzureBlobFSLocation' + fileName: { + value: '@{dataset().fileName}' + type: 'Expression' + } + folderPath: { + value: '@{dataset().folderPath}' + type: 'Expression' + } + } + } + parameters: { + fileName: { + type: 'String' + defaultValue: 'settings.json' + } + folderPath: { + type: 'String' + defaultValue: configContainer.outputs.containerName + } + } + } + } + + resource dataset_ingestion 'datasets' = { + name: INGESTION + properties: { + annotations: [] + parameters: { + blobPath: { + type: 'String' + } + } + type: 'Parquet' + typeProperties: { + location: { + type: 'AzureBlobFSLocation' + fileName: { + value: '@{dataset().blobPath}' + type: 'Expression' + } + fileSystem: ingestionContainer.outputs.containerName + } + } + linkedServiceName: { + parameters: {} + referenceName: app.storage + type: 'LinkedServiceReference' + } + } + } + + resource dataset_ingestion_files 'datasets' = { + name: '${INGESTION}_files' + properties: { + annotations: [] + parameters: { + folderPath: { + type: 'String' + } + } + type: 'Parquet' + typeProperties: { + location: { + type: 'AzureBlobFSLocation' + fileSystem: ingestionContainer.outputs.containerName + folderPath: { + value: '@dataset().folderPath' + type: 'Expression' + } + } + } + linkedServiceName: { + parameters: {} + referenceName: app.storage + type: 'LinkedServiceReference' + } + } + } +} //============================================================================== // Outputs @@ -139,24 +244,20 @@ module uploadSettings 'hub-deploymentScript.bicep' = { // TODO: Review the use of these outputs and deprecate the ones that aren't needed, remove them in 3 months +@description('Properties of the hub app.') +output app HubAppProperties = app + @description('Name of the Data Factory.') output dataFactoryName string = app.dataFactory @description('Name of the storage account created for the hub instance. This must be used when connecting FinOps toolkit Power BI reports to your data.') output storageAccountName string = app.storage -@description('The name of the container used for configuration settings.') -output configContainer string = configContainer.outputs.containerName - -@description('The name of the container used for normalized data ingestion.') -output ingestionContainer string = ingestionContainer.outputs.containerName - @description('URL to use when connecting custom Power BI reports to your data.') -output storageUrlForPowerBI string = 'https://${app.storage}.dfs.${environment().suffixes.storage}/${ingestionContainer.outputs.containerName}' +output storageUrlForPowerBI string = 'https://${app.storage}.dfs.${environment().suffixes.storage}/${INGESTION}' @description('Object ID of the Data Factory managed identity. This will be needed when configuring managed exports.') -output principalId string = appRegistration.outputs.principalId +output principalId string = dataFactory.identity.principalId -// TODO: Remove this output -@description('Tags for the FinOps hub publisher.') -output publisherTags object = app.publisher.tags +@description('Name of the managed identity used to create and stop ADF triggers.') +output triggerManagerIdentityName string = appRegistration.outputs.triggerManagerIdentityName diff --git a/src/templates/finops-hub/modules/infrastructure.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/infrastructure.bicep similarity index 97% rename from src/templates/finops-hub/modules/infrastructure.bicep rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/infrastructure.bicep index 27e994fb0..0cf23a808 100644 --- a/src/templates/finops-hub/modules/infrastructure.bicep +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/infrastructure.bicep @@ -1,7 +1,7 @@ // Copyright (c) Microsoft Corporation. // Licensed under the MIT License. -import { getHubTags, HubProperties } from 'hub-types.bicep' +import { getHubTags, HubProperties } from '../../fx/hub-types.bicep' //============================================================================== @@ -293,7 +293,7 @@ resource tablePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if //------------------------------------------------------------------------------ resource scriptStorageAccount 'Microsoft.Storage/storageAccounts@2022-09-01' = if (hub.options.privateRouting) { - name: string(hub.routing.scriptStorage) + name: hub.routing.scriptStorage dependsOn: [ vNet::scriptSubnet ] @@ -372,9 +372,11 @@ output config HubProperties = hub output vNetId string = !hub.options.privateRouting ? '' : vNet.id @description('Virtual network address prefixes.') +#disable-next-line BCP318 // Null safety warning for conditional resource access output vNetAddressSpace array = !hub.options.privateRouting ? [] : vNet.properties.addressSpace.addressPrefixes @description('Virtual network subnets.') +#disable-next-line BCP318 // Null safety warning for conditional resource access output vNetSubnets array = !hub.options.privateRouting ? [] : vNet.properties.subnets @description('Resource ID of the FinOps hub network subnet.') diff --git a/src/templates/finops-hub/schemas/settings.json b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/settings.json similarity index 100% rename from src/templates/finops-hub/schemas/settings.json rename to src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/settings.json diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/RemoteHub/app.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/RemoteHub/app.bicep new file mode 100644 index 000000000..798cb420d --- /dev/null +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/RemoteHub/app.bicep @@ -0,0 +1,159 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { finOpsToolkitVersion, HubAppProperties, privateRoutingForLinkedServices } from '../../fx/hub-types.bicep' + + +//============================================================================== +// Parameters +//============================================================================== + +@description('Required. FinOps hub app getting deployed.') +param app HubAppProperties + +@description('Required. Create and store a key for a remote storage account.') +@secure() +param remoteStorageKey string + +@description('Required. Remote storage account for ingestion dataset.') +param remoteHubStorageUri string + +@description('Optional. Name of the ingestion container. Default: ingestion.') +param ingestionContainerName string = 'ingestion' + + +//============================================================================== +// Variables +//============================================================================== + +var storageKeySecretName = '${toLower(app.hub.name)}-storage-key' + + +//============================================================================== +// Resources +//============================================================================== + +// App registration +module appRegistration '../../fx/hub-app.bicep' = { + name: 'Microsoft.FinOpsHubs.RemoteHub_Register' + params: { + app: app + version: finOpsToolkitVersion + features: [ + 'DataFactory' + 'KeyVault' + 'Storage' + ] + } +} + +// Key Vault secret +module keyVault_secret '../../fx/hub-vault.bicep' = { + name: 'keyVault_secret' + params: { + vaultName: app.keyVault + secretName: storageKeySecretName + secretValue: remoteStorageKey + secretExpirationInSeconds: 1702648632 + secretNotBeforeInSeconds: 10000 + } +} + +// Get key vault instance +resource keyVault 'Microsoft.KeyVault/vaults@2023-02-01' existing = { + name: app.keyVault +} + +// Get data factory instance +resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { + name: app.dataFactory + + // cSpell:ignore linkedservices + resource linkedService_remoteHubStorage 'linkedservices' = { + name: 'remoteHubStorage' + properties: { + annotations: [] + parameters: {} + type: 'AzureBlobFS' + typeProperties: { + url: remoteHubStorageUri + accountKey: { + type: 'AzureKeyVaultSecret' + store: { + // TODO: Should the key vault linked service name/reference be part of hub settings? + referenceName: keyVault.name + type: 'LinkedServiceReference' + } + secretName: storageKeySecretName + } + } + ...privateRoutingForLinkedServices(app.hub) + } + } + + // Replace the ingestion dataset + resource dataset_ingestion 'datasets' = { + name: ingestionContainerName + properties: { + annotations: [] + parameters: { + blobPath: { + type: 'String' + } + } + type: 'Parquet' + typeProperties: { + location: { + type: 'AzureBlobFSLocation' + fileName: { + value: '@{dataset().blobPath}' + type: 'Expression' + } + fileSystem: ingestionContainerName + } + } + linkedServiceName: { + parameters: {} + referenceName: linkedService_remoteHubStorage.name + type: 'LinkedServiceReference' + } + } + } + + // Replace the ingestion_files dataset + resource dataset_ingestion_files 'datasets' = { + name: '${ingestionContainerName}_files' + properties: { + annotations: [] + parameters: { + folderPath: { + type: 'String' + } + } + type: 'Parquet' + typeProperties: { + location: { + type: 'AzureBlobFSLocation' + fileSystem: ingestionContainerName + folderPath: { + value: '@dataset().folderPath' + type: 'Expression' + } + } + } + linkedServiceName: { + parameters: {} + referenceName: linkedService_remoteHubStorage.name + type: 'LinkedServiceReference' + } + } + } +} + + +//============================================================================== +// Outputs +//============================================================================== + +@description('Name of the Key Vault instance.') +output keyVaultName string = app.keyVault diff --git a/src/templates/finops-hub/modules/README.md b/src/templates/finops-hub/modules/README.md index ba1070efe..f6067314d 100644 --- a/src/templates/finops-hub/modules/README.md +++ b/src/templates/finops-hub/modules/README.md @@ -1,13 +1,21 @@ -# 📦 FinOps hub modules +# 📦 FinOps hub modules and apps -All FinOps hub module source is available at the root of this directory. +FinOps hubs consist of reusable `fx` modules and a set of apps, separated by publisher. -Modules: +- Publishers define ownership, accountability, and act as a security boundary. +- Apps should have a single responsibility and provide a complete, comprehensive, and \[generally] self-contained capability. +- Prefer separate apps to maximize modularity and allow customers to enable or disable holistic features. +- Apps may rely on functionality from other apps. This is the goal of the extensibility model. -- [hub.bicep](./hub.bicep) orchestrates the creation of all required resources. -- [storage.bicep](./storage.bicep) creates the storage account, containers, and settings.json file. -- [keyVault.bicep](./keyVault.bicep) creates the Key Vault instance and stored secrets. -- [dataFactory.bicep](./dataFactory.bicep) creates Data Factory pipelines, triggers, etc. -- [dataExplorer.bicep](./dataExplorer.bicep) creates Data Explorer cluster, database, etc. +Use the following to help guide decisions about the publisher and app to use for new functionality: + +- Who owns and will (or should) maintain the app? + - For core FinOps hubs contributors, use `Microsoft.FinOpsHubs`. + - For Microsoft product teams, use `Microsoft.{service}` where `{service}` is the owning engineering team. + > _NOTE: This includes functionality that would ideally be managed by a service team that is not engaged due to the complexity of the solution. Not every feature should be managed by a separate engineering team. Use your best judgement._ + - For community-supported features, use `FinOpsToolkit.{area}` where `{area}` is a specific domain or area of responsibility and not a broad customer segment. Community-supported apps will not have the same level of support. +- Does the functionality share the same security boundary as others (e.g., compliance, data access, permissions)? + - If so, use the publisher with the same precise security boundary. + - If not, use a new publisher.
diff --git a/src/templates/finops-hub/modules/cm-exports.bicep b/src/templates/finops-hub/modules/cm-exports.bicep deleted file mode 100644 index e2126a4d8..000000000 --- a/src/templates/finops-hub/modules/cm-exports.bicep +++ /dev/null @@ -1,82 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -import { HubProperties } from 'hub-types.bicep' - - -//============================================================================== -// Parameters -//============================================================================== - -@description('Required. FinOps hub instance to deploy the app to.') -param hub HubProperties - - -//============================================================================== -// Resources -//============================================================================== - -// Register app -module appRegistration 'hub-app.bicep' = { - name: 'Microsoft.CostManagement.Exports_Register' - params: { - hub: hub - publisher: 'Microsoft FinOps hubs' - namespace: 'Microsoft.FinOpsHubs' - appName: 'Core' - displayName: 'FinOps hub core' - appVersion: loadTextContent('ftkver.txt') // cSpell:ignore ftkver - features: [ - 'DataFactory' - 'Storage' - ] - } -} - -// Upload schema files -module schemaFiles 'hub-storage.bicep' = { - name: 'Microsoft.CostManagement.Exports_Storage.SchemaFiles' - params: { - app: appRegistration.outputs.app - container: 'config' - files: { - // cSpell:ignore actualcost, amortizedcost, focuscost, pricesheet, reservationdetails, reservationrecommendations, reservationtransactions - 'schemas/actualcost_c360-2025-04.json': loadTextContent('../schemas/actualcost_c360-2025-04.json') - 'schemas/amortizedcost_c360-2025-04.json': loadTextContent('../schemas/amortizedcost_c360-2025-04.json') - 'schemas/focuscost_1.2.json': loadTextContent('../schemas/focuscost_1.2.json') - 'schemas/focuscost_1.2-preview.json': loadTextContent('../schemas/focuscost_1.2-preview.json') - 'schemas/focuscost_1.0r2.json': loadTextContent('../schemas/focuscost_1.0r2.json') - 'schemas/focuscost_1.0.json': loadTextContent('../schemas/focuscost_1.0.json') - 'schemas/focuscost_1.0-preview(v1).json': loadTextContent('../schemas/focuscost_1.0-preview(v1).json') - 'schemas/pricesheet_2023-05-01_ea.json': loadTextContent('../schemas/pricesheet_2023-05-01_ea.json') - 'schemas/pricesheet_2023-05-01_mca.json': loadTextContent('../schemas/pricesheet_2023-05-01_mca.json') - 'schemas/reservationdetails_2023-03-01.json': loadTextContent('../schemas/reservationdetails_2023-03-01.json') - 'schemas/reservationrecommendations_2023-05-01_ea.json': loadTextContent('../schemas/reservationrecommendations_2023-05-01_ea.json') - 'schemas/reservationrecommendations_2023-05-01_mca.json': loadTextContent('../schemas/reservationrecommendations_2023-05-01_mca.json') - 'schemas/reservationtransactions_2023-05-01_ea.json': loadTextContent('../schemas/reservationtransactions_2023-05-01_ea.json') - 'schemas/reservationtransactions_2023-05-01_mca.json': loadTextContent('../schemas/reservationtransactions_2023-05-01_mca.json') - } - } -} - -// Create msexports container -module exportContainer 'hub-storage.bicep' = { - name: 'Microsoft.CostManagement.Exports_Storage.ExportContainer' - params: { - app: appRegistration.outputs.app - container: 'msexports' - } -} - -// TODO: Add export handling pipelines - - -//============================================================================== -// Outputs -//============================================================================== - -@description('Name of the container used for Cost Management exports.') -output exportContainer string = exportContainer.outputs.containerName - -@description('Number of schema files uploaded.') -output schemaFilesUploaded int = schemaFiles.outputs.filesUploaded diff --git a/src/templates/finops-hub/modules/dataExplorer.bicep b/src/templates/finops-hub/modules/dataExplorer.bicep deleted file mode 100644 index 86a452e48..000000000 --- a/src/templates/finops-hub/modules/dataExplorer.bicep +++ /dev/null @@ -1,501 +0,0 @@ -//============================================================================== -// Parameters -//============================================================================== - -// @description('Required. Name of the FinOps hub instance. Used to ensure unique resource names.') -// param hubName string - -// @description('Required. Suffix to add to the storage account name to ensure uniqueness.') -// @minLength(6) // Min length requirement is to avoid a false positive warning -// param uniqueSuffix string - -@description('Optional. Name of the Azure Data Explorer cluster to use for advanced analytics. If empty, Azure Data Explorer will not be deployed. Required to use with Power BI if you have more than $2-5M/mo in costs being monitored. Default: "" (do not use).') -param clusterName string = '' - -// https://learn.microsoft.com/azure/templates/microsoft.kusto/clusters?pivots=deployment-language-bicep#azuresku -@description('Optional. Name of the Azure Data Explorer SKU. Default: "Dev(No SLA)_Standard_E2a_v4".') -@allowed([ - 'Dev(No SLA)_Standard_E2a_v4' // 2 CPU, 16GB RAM, 24GB cache, $110/mo - 'Dev(No SLA)_Standard_D11_v2' // 2 CPU, 14GB RAM, 78GB cache, $121/mo - 'Standard_D11_v2' // 2 CPU, 14GB RAM, 78GB cache, $245/mo - 'Standard_D12_v2' - 'Standard_D13_v2' - 'Standard_D14_v2' - 'Standard_D16d_v5' - 'Standard_D32d_v4' - 'Standard_D32d_v5' - 'Standard_DS13_v2+1TB_PS' - 'Standard_DS13_v2+2TB_PS' - 'Standard_DS14_v2+3TB_PS' - 'Standard_DS14_v2+4TB_PS' - 'Standard_E2a_v4' // 2 CPU, 14GB RAM, 78GB cache, $220/mo - 'Standard_E2ads_v5' - 'Standard_E2d_v4' - 'Standard_E2d_v5' - 'Standard_E4a_v4' - 'Standard_E4ads_v5' - 'Standard_E4d_v4' - 'Standard_E4d_v5' - 'Standard_E8a_v4' - 'Standard_E8ads_v5' - 'Standard_E8as_v4+1TB_PS' - 'Standard_E8as_v4+2TB_PS' - 'Standard_E8as_v5+1TB_PS' - 'Standard_E8as_v5+2TB_PS' - 'Standard_E8d_v4' - 'Standard_E8d_v5' - 'Standard_E8s_v4+1TB_PS' - 'Standard_E8s_v4+2TB_PS' - 'Standard_E8s_v5+1TB_PS' - 'Standard_E8s_v5+2TB_PS' - 'Standard_E16a_v4' - 'Standard_E16ads_v5' - 'Standard_E16as_v4+3TB_PS' - 'Standard_E16as_v4+4TB_PS' - 'Standard_E16as_v5+3TB_PS' - 'Standard_E16as_v5+4TB_PS' - 'Standard_E16d_v4' - 'Standard_E16d_v5' - 'Standard_E16s_v4+3TB_PS' - 'Standard_E16s_v4+4TB_PS' - 'Standard_E16s_v5+3TB_PS' - 'Standard_E16s_v5+4TB_PS' - 'Standard_E64i_v3' - 'Standard_E80ids_v4' - 'Standard_EC8ads_v5' - 'Standard_EC8as_v5+1TB_PS' - 'Standard_EC8as_v5+2TB_PS' - 'Standard_EC16ads_v5' - 'Standard_EC16as_v5+3TB_PS' - 'Standard_EC16as_v5+4TB_PS' - 'Standard_L4s' - 'Standard_L8as_v3' - 'Standard_L8s' - 'Standard_L8s_v2' - 'Standard_L8s_v3' - 'Standard_L16as_v3' - 'Standard_L16s' - 'Standard_L16s_v2' - 'Standard_L16s_v3' - 'Standard_L32as_v3' - 'Standard_L32s_v3' -]) -param clusterSku string = 'Dev(No SLA)_Standard_E2a_v4' - -@description('Optional. Number of nodes to use in the cluster. Allowed values: 1 for the Basic SKU tier and 2-1000 for Standard. Default: 1 for dev/test SKUs, 2 for standard SKUs.') -@minValue(1) -@maxValue(1000) -param clusterCapacity int = 1 - -// TODO: Figure out why this is breaking upgrades -// @description('Optional. Array of external tenant IDs that should have access to the cluster. Default: empty (no external access).') -// param clusterTrustedExternalTenants string[] = [] - -@description('Optional. Forces the table to be updated if different from the last time it was deployed.') -param forceUpdateTag string = utcNow() - -@description('Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false.') -param continueOnErrors bool = false - -@description('Optional. Azure location to use for the managed identity and deployment script to auto-start triggers. Default: (resource group location).') -param location string = resourceGroup().location - -@description('Optional. Tags to apply to all resources.') -param tags object = {} - -@description('Optional. Tags to apply to resources based on their resource type. Resource type specific tags will be merged with tags for all resources.') -param tagsByResource object = {} - -@description('Required. Name of the Data Factory instance.') -param dataFactoryName string - -@description('Optional. Number of days of data to retain in the Data Explorer *_raw tables. Default: 0.') -param rawRetentionInDays int = 0 - -@description('Required. Name of the storage account to use for data ingestion.') -param storageAccountName string - -@description('Required. Resource ID of the virtual network for private endpoints.') -param virtualNetworkId string - -@description('Required. Resource ID of the subnet for private endpoints.') -param privateEndpointSubnetId string - -@description('Optional. Enable public access.') -param enablePublicAccess bool - -//------------------------------------------------------------------------------ -// Variables -//------------------------------------------------------------------------------ - -// cSpell:ignore ftkver, privatelink -var dataExplorerPrivateDnsZoneName = replace('privatelink.${location}.${replace(environment().suffixes.storage, 'core', 'kusto')}', '..', '.') - -// Actual = Minimum(ClusterMaximumConcurrentOperations, Number of nodes in cluster * Maximum(1, Core count per node * CoreUtilizationCoefficient)) -var ingestionCapacity = { - 'Dev(No SLA)_Standard_E2a_v4': 1 - 'Dev(No SLA)_Standard_D11_v2': 1 - Standard_D11_v2: 2 - Standard_D12_v2: 4 - Standard_D13_v2: 8 - Standard_D14_v2: 16 - Standard_D16d_v5: 16 - Standard_D32d_v4: 32 - Standard_D32d_v5: 32 - 'Standard_DS13_v2+1TB_PS': 8 - 'Standard_DS13_v2+2TB_PS': 8 - 'Standard_DS14_v2+3TB_PS': 16 - 'Standard_DS14_v2+4TB_PS': 16 - Standard_E2a_v4: 2 - Standard_E2ads_v5: 2 - Standard_E2d_v4: 2 - Standard_E2d_v5: 2 - Standard_E4a_v4: 4 - Standard_E4ads_v5: 4 - Standard_E4d_v4: 4 - Standard_E4d_v5: 4 - Standard_E8a_v4: 8 - Standard_E8ads_v5: 8 - 'Standard_E8as_v4+1TB_PS': 8 - 'Standard_E8as_v4+2TB_PS': 8 - 'Standard_E8as_v5+1TB_PS': 8 - 'Standard_E8as_v5+2TB_PS': 8 - Standard_E8d_v4: 8 - Standard_E8d_v5: 8 - 'Standard_E8s_v4+1TB_PS': 8 - 'Standard_E8s_v4+2TB_PS': 8 - 'Standard_E8s_v5+1TB_PS': 8 - 'Standard_E8s_v5+2TB_PS': 8 - Standard_E16a_v4: 16 - Standard_E16ads_v5: 16 - 'Standard_E16as_v4+3TB_PS': 16 - 'Standard_E16as_v4+4TB_PS': 16 - 'Standard_E16as_v5+3TB_PS': 16 - 'Standard_E16as_v5+4TB_PS': 16 - Standard_E16d_v4: 16 - Standard_E16d_v5: 16 - 'Standard_E16s_v4+3TB_PS': 16 - 'Standard_E16s_v4+4TB_PS': 16 - 'Standard_E16s_v5+3TB_PS': 16 - 'Standard_E16s_v5+4TB_PS': 16 - Standard_E64i_v3: 64 - Standard_E80ids_v4: 80 - Standard_EC8ads_v5: 8 - 'Standard_EC8as_v5+1TB_PS': 8 - 'Standard_EC8as_v5+2TB_PS': 8 - Standard_EC16ads_v5: 16 - 'Standard_EC16as_v5+3TB_PS': 16 - 'Standard_EC16as_v5+4TB_PS': 16 - Standard_L4s: 4 - Standard_L8as_v3: 8 - Standard_L8s: 8 - Standard_L8s_v2: 8 - Standard_L8s_v3: 8 - Standard_L16as_v3: 16 - Standard_L16s: 16 - Standard_L16s_v2: 16 - Standard_L16s_v3: 16 - Standard_L32as_v3: 32 - Standard_L32s_v3: 32 -} - -//============================================================================== -// Resources -//============================================================================== - -//------------------------------------------------------------------------------ -// Dependencies -//------------------------------------------------------------------------------ - -// Get data factory instance -resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { - name: dataFactoryName -} - -resource blobPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = { - name: 'privatelink.blob.${environment().suffixes.storage}' -} - -resource queuePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = { - name: 'privatelink.queue.${environment().suffixes.storage}' -} - -resource tablePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = { - name: 'privatelink.table.${environment().suffixes.storage}' -} - -resource storage 'Microsoft.Storage/storageAccounts@2022-09-01' existing = { - name: storageAccountName -} - -//------------------------------------------------------------------------------ -// Cluster + databases -//------------------------------------------------------------------------------ - -// Kusto cluster -resource cluster 'Microsoft.Kusto/clusters@2023-08-15' = { - name: clusterName - location: location - tags: union(tags, tagsByResource[?'Microsoft.Kusto/clusters'] ?? {}) - sku: { - name: clusterSku - tier: startsWith(clusterSku, 'Dev(No SLA)_') ? 'Basic' : 'Standard' - capacity: startsWith(clusterSku, 'Dev(No SLA)_') ? 1 : (clusterCapacity == 1 ? 2 : clusterCapacity) - } - identity: { - type: 'SystemAssigned' - } - properties: { - enableStreamingIngest: true - enableAutoStop: false - publicNetworkAccess: enablePublicAccess ? 'Enabled' : 'Disabled' - // TODO: Figure out why this is breaking upgrades - // trustedExternalTenants: [for tenantId in clusterTrustedExternalTenants: { - // value: tenantId - // }] - } - - resource adfClusterAdmin 'principalAssignments' = { - name: 'adf-mi-cluster-admin' - properties: { - principalType: 'App' - principalId: dataFactory.identity.principalId - tenantId: dataFactory.identity.tenantId - role: 'AllDatabasesAdmin' - } - } - - resource ingestionDb 'databases' = { - name: 'Ingestion' - location: location - kind: 'ReadWrite' - } - - resource hubDb 'databases' = { - name: 'Hub' - location: location - kind: 'ReadWrite' - } -} - -module ingestion_OpenDataInternalScripts 'hub-database.bicep' = { - name: 'ingestion_OpenDataInternalScripts' - params: { - clusterName: cluster.name - databaseName: cluster::ingestionDb.name - scripts: { - OpenDataFunctions_resource_type_1: loadTextContent('scripts/OpenDataFunctions_resource_type_1.kql') - OpenDataFunctions_resource_type_2: loadTextContent('scripts/OpenDataFunctions_resource_type_2.kql') - OpenDataFunctions_resource_type_3: loadTextContent('scripts/OpenDataFunctions_resource_type_3.kql') - OpenDataFunctions_resource_type_4: loadTextContent('scripts/OpenDataFunctions_resource_type_4.kql') - OpenDataFunctions_resource_type_5: loadTextContent('scripts/OpenDataFunctions_resource_type_5.kql') - } - continueOnErrors: continueOnErrors - forceUpdateTag: forceUpdateTag - } -} - -module ingestion_InitScripts 'hub-database.bicep' = { - name: 'ingestion_InitScripts' - dependsOn: [ - ingestion_OpenDataInternalScripts - ] - params: { - clusterName: cluster.name - databaseName: cluster::ingestionDb.name - scripts: { - openData: loadTextContent('scripts/OpenDataFunctions.kql') - common: loadTextContent('scripts/Common.kql') - infra: loadTextContent('scripts/IngestionSetup_HubInfra.kql') - rawTables: replace(loadTextContent('scripts/IngestionSetup_RawTables.kql'), '$$rawRetentionInDays$$', string(rawRetentionInDays)) - } - continueOnErrors: continueOnErrors - forceUpdateTag: forceUpdateTag - } -} - -module ingestion_VersionedScripts 'hub-database.bicep' = { - name: 'ingestion_VersionedScripts' - dependsOn: [ - ingestion_InitScripts - ] - params: { - clusterName: cluster.name - databaseName: cluster::ingestionDb.name - scripts: { - v1_0: loadTextContent('scripts/IngestionSetup_v1_0.kql') - v1_2: loadTextContent('scripts/IngestionSetup_v1_2.kql') - } - continueOnErrors: continueOnErrors - forceUpdateTag: forceUpdateTag - } -} - -module hub_InitScripts 'hub-database.bicep' = { - name: 'hub_InitScripts' - dependsOn: [ - ingestion_InitScripts - ] - params: { - clusterName: cluster.name - databaseName: cluster::hubDb.name - scripts: { - common: loadTextContent('scripts/Common.kql') - openData: loadTextContent('scripts/HubSetup_OpenData.kql') - } - continueOnErrors: continueOnErrors - forceUpdateTag: forceUpdateTag - } -} - -module hub_VersionedScripts 'hub-database.bicep' = { - name: 'hub_VersionedScripts' - dependsOn: [ - ingestion_VersionedScripts - hub_InitScripts - ] - params: { - clusterName: cluster.name - databaseName: cluster::hubDb.name - scripts: { - v1_0: loadTextContent('scripts/HubSetup_v1_0.kql') - v1_2: loadTextContent('scripts/HubSetup_v1_2.kql') - } - continueOnErrors: continueOnErrors - forceUpdateTag: forceUpdateTag - } -} - -module hub_LatestScripts 'hub-database.bicep' = { - name: 'hub_LatestScripts' - dependsOn: [ - hub_VersionedScripts - ] - params: { - clusterName: cluster.name - databaseName: cluster::hubDb.name - scripts: { - latest: loadTextContent('scripts/HubSetup_Latest.kql') - } - continueOnErrors: continueOnErrors - forceUpdateTag: forceUpdateTag - } -} - -// Authorize Kusto Cluster to read storage -resource clusterStorageAccess 'Microsoft.Authorization/roleAssignments@2022-04-01' = { - name: guid(cluster.name, subscription().id, 'Storage Blob Data Contributor') - scope: storage - properties: { - description: 'Give "Storage Blob Data Contributor" to the cluster' - principalId: cluster.identity.principalId - // Required in case principal not ready when deploying the assignment - principalType: 'ServicePrincipal' - roleDefinitionId: subscriptionResourceId( - 'Microsoft.Authorization/roleDefinitions', - 'ba92f5b4-2d11-453d-a403-e96b0029c9fe' // Storage Blob Data Contributor -- https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#storage - ) - } -} - -// DNS zone -resource dataExplorerPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (!enablePublicAccess) { - name: dataExplorerPrivateDnsZoneName - location: 'global' - tags: union(tags, tagsByResource[?'Microsoft.Network/privateDnsZones'] ?? {}) - properties: {} -} - -// Link DNS zone to VNet -resource dataExplorerPrivateDnsZoneLink 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2024-06-01' = if (!enablePublicAccess) { - name: '${replace(dataExplorerPrivateDnsZone.name, '.', '-')}-link' - location: 'global' - parent: dataExplorerPrivateDnsZone - tags: union(tags, tagsByResource[?'Microsoft.Network/privateDnsZones/virtualNetworkLinks'] ?? {}) - properties: { - virtualNetwork: { - id: virtualNetworkId - } - registrationEnabled: false - } -} - -// Private endpoint -resource dataExplorerEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (!enablePublicAccess) { - name: '${cluster.name}-ep' - location: location - tags: union(tags, tagsByResource[?'Microsoft.Network/privateEndpoints'] ?? {}) - properties: { - subnet: { - id: privateEndpointSubnetId - } - privateLinkServiceConnections: [ - { - name: 'dataExplorerLink' - properties: { - privateLinkServiceId: cluster.id - groupIds: ['cluster'] - } - } - ] - } -} - -// DNS records for private endpoint -resource dataExplorerPrivateDnsZoneGroup 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups@2023-11-01' = if (!enablePublicAccess) { - name: 'dataExplorer-endpoint-zone' - parent: dataExplorerEndpoint - properties: { - privateDnsZoneConfigs: [ - { - name: 'privatelink-westus-kusto-net' - properties: { - privateDnsZoneId: dataExplorerPrivateDnsZone.id - } - } - { - name: 'privatelink-blob-core-windows-net' - properties: { - privateDnsZoneId: blobPrivateDnsZone.id - } - } - { - name: 'privatelink-table-core-windows-net' - properties: { - privateDnsZoneId: tablePrivateDnsZone.id - } - } - { - name: 'privatelink-queue-core-windows-net' - properties: { - privateDnsZoneId: queuePrivateDnsZone.id - } - } - ] - } -} - -//============================================================================== -// Outputs -//============================================================================== - -@description('The resource ID of the cluster.') -output clusterId string = cluster.id - -@description('The ID of the cluster system assigned managed identity.') -output principalId string = cluster.identity.principalId - -@description('The name of the cluster.') -output clusterName string = cluster.name - -@description('The URI of the cluster.') -output clusterUri string = cluster.properties.uri - -@description('The name of the database for data ingestion.') -output ingestionDbName string = cluster::ingestionDb.name - -@description('The name of the database for queries.') -output hubDbName string = cluster::hubDb.name - -@description('Max ingestion capacity of the cluster.') -output clusterIngestionCapacity int = ingestionCapacity[?clusterSku] ?? 1 diff --git a/src/templates/finops-hub/modules/dataFactory.bicep b/src/templates/finops-hub/modules/dataFactory.bicep deleted file mode 100644 index 2c7fe53e5..000000000 --- a/src/templates/finops-hub/modules/dataFactory.bicep +++ /dev/null @@ -1,5251 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -import { HubAppProperties } from 'hub-types.bicep' - - -//============================================================================== -// Parameters -//============================================================================== - -@description('Required. Temporary app placeholder for the deployments module.') -param app HubAppProperties - -@description('Required. Name of the FinOps hub instance.') -param hubName string - -@description('Required. Name of the Data Factory instance.') -param dataFactoryName string - -@description('Required. The name of the Azure Key Vault instance.') -param keyVaultName string - -@description('Required. The name of the Azure storage account instance.') -param storageAccountName string - -@description('Required. The name of the container where Cost Management data is exported.') -param exportContainerName string - -@description('Required. The name of the container where normalized data is ingested.') -param ingestionContainerName string - -@description('Required. The name of the container where normalized data is ingested.') -param configContainerName string - -@description('Optional. Name of the Azure Data Explorer cluster to use for advanced analytics, if applicable.') -param dataExplorerName string = '' - -@description('Optional. Resource ID of the Azure Data Explorer cluster to use for advanced analytics, if applicable.') -param dataExplorerId string = '' - -@description('Optional. ID of the Azure Data Explorer cluster system assigned managed identity, if applicable.') -param dataExplorerPrincipalId string = '' - -// cSpell:ignore eventhouse -@description('Optional. URI of the Azure Data Explorer cluster or Microsoft Fabric eventhouse query endpoint to use for advanced analytics, if applicable.') -param dataExplorerUri string = '' - -@description('Optional. Name of the Azure Data Explorer ingestion database. Default: "ingestion".') -param dataExplorerIngestionDatabase string = 'Ingestion' - -@description('Optional. Azure Data Explorer ingestion capacity or Microsoft Fabric capacity units. Increase for non-dev/trial SKUs. Default: 1') -param dataExplorerIngestionCapacity int = 1 - -@description('Optional. The location to use for the managed identity and deployment script to auto-start triggers. Default = (resource group location).') -param location string = resourceGroup().location - -@description('Optional. Remote storage account for ingestion dataset.') -param remoteHubStorageUri string - -@description('Optional. Tags to apply to all resources.') -param tags object = {} - -@description('Optional. Tags to apply to resources based on their resource type. Resource type specific tags will be merged with tags for all resources.') -param tagsByResource object = {} - -@description('Optional. Enable managed exports where your FinOps hub instance will create and run Cost Management exports on your behalf. Not supported for Microsoft Customer Agreement (MCA) billing profiles. Requires the ability to grant User Access Administrator role to FinOps hubs, which is required to create Cost Management exports. Default: true.') -param enableManagedExports bool = true - -@description('Required. Enable public access.') -param enablePublicAccess bool - -//------------------------------------------------------------------------------ -// Variables -//------------------------------------------------------------------------------ - -var focusSchemaVersion = '1.0' -var exportSchemaVersion = '2023-05-01' -var reservationDetailsSchemaVersion = '2023-03-01' -// cSpell:ignore ftkver -var ftkVersion = loadTextContent('ftkver.txt') -var ftkReleaseUri = endsWith(ftkVersion, '-dev') - ? 'https://github.com/microsoft/finops-toolkit/releases/latest/download' - : 'https://github.com/microsoft/finops-toolkit/releases/download/v${ftkVersion}' -var exportApiVersion = '2023-07-01-preview' -var hubDataExplorerName = 'hubDataExplorer' - -// cSpell:ignore timeframe -// Function to generate the body for a Cost Management export -func getExportBody(exportContainerName string, datasetType string, schemaVersion string, isMonthly bool, exportFormat string, compressionMode string, partitionData string, dataOverwriteBehavior string) string => '{ "properties": { "definition": { "dataSet": { "configuration": { "dataVersion": "${schemaVersion}", "filters": [] }, "granularity": "Daily" }, "timeframe": "${isMonthly ? 'TheLastMonth': 'MonthToDate' }", "type": "${datasetType}" }, "deliveryInfo": { "destination": { "container": "${exportContainerName}", "rootFolderPath": "@{if(startswith(item().scope, \'/\'), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}", "type": "AzureBlob", "resourceId": "@{variables(\'storageAccountId\')}" } }, "schedule": { "recurrence": "${ isMonthly ? 'Monthly' : 'Daily'}", "recurrencePeriod": { "from": "2024-01-01T00:00:00.000Z", "to": "2050-02-01T00:00:00.000Z" }, "status": "Inactive" }, "format": "${exportFormat}", "partitionData": "${partitionData}", "dataOverwriteBehavior": "${dataOverwriteBehavior}", "compressionMode": "${compressionMode}" }, "id": "@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{variables(\'exportName\')}", "name": "@{variables(\'exportName\')}", "type": "Microsoft.CostManagement/reports", "identity": { "type": "systemAssigned" }, "location": "global" }' - -func getExportBodyV2(exportContainerName string, datasetType string, schemaVersion string, isMonthly bool, exportFormat string, compressionMode string, partitionData string, dataOverwriteBehavior string, recommendationScope string, recommendationLookbackPeriod string, resourceType string) string => /* - */ toLower(datasetType) == 'focuscost' ? /* - */ '{ "properties": { "definition": { "dataSet": { "configuration": { "dataVersion": "${schemaVersion}", "filters": [] }, "granularity": "Daily" }, "timeframe": "${isMonthly ? 'TheLastMonth': 'MonthToDate' }", "type": "${datasetType}" }, "deliveryInfo": { "destination": { "container": "${exportContainerName}", "rootFolderPath": "@{if(startswith(item().scope, \'/\'), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}", "type": "AzureBlob", "resourceId": "@{variables(\'storageAccountId\')}" } }, "schedule": { "recurrence": "${ isMonthly ? 'Monthly' : 'Daily'}", "recurrencePeriod": { "from": "2024-01-01T00:00:00.000Z", "to": "2050-02-01T00:00:00.000Z" }, "status": "Inactive" }, "format": "${exportFormat}", "partitionData": "${partitionData}", "dataOverwriteBehavior": "${dataOverwriteBehavior}", "compressionMode": "${compressionMode}" }, "id": "@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-costdetails\'))}", "name": "@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-costdetails\'))}", "type": "Microsoft.CostManagement/reports", "identity": { "type": "systemAssigned" }, "location": "global" }' /* - */ : toLower(datasetType) == 'reservationdetails' ? /* - */ '{ "properties": { "definition": { "dataSet": { "configuration": { "dataVersion": "${schemaVersion}", "filters": [] }, "granularity": "Daily" }, "timeframe": "${isMonthly ? 'TheLastMonth': 'MonthToDate' }", "type": "${datasetType}" }, "deliveryInfo": { "destination": { "container": "${exportContainerName}", "rootFolderPath": "@{if(startswith(item().scope, \'/\'), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}", "type": "AzureBlob", "resourceId": "@{variables(\'storageAccountId\')}" } }, "schedule": { "recurrence": "${ isMonthly ? 'Monthly' : 'Daily'}", "recurrencePeriod": { "from": "2024-01-01T00:00:00.000Z", "to": "2050-02-01T00:00:00.000Z" }, "status": "Inactive" }, "format": "${exportFormat}", "partitionData": "${partitionData}", "dataOverwriteBehavior": "${dataOverwriteBehavior}", "compressionMode": "${compressionMode}" }, "id": "@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-${toLower(datasetType)}\'))}", "name": "@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-${toLower(datasetType)}\'))}", "type": "Microsoft.CostManagement/reports", "identity": { "type": "systemAssigned" }, "location": "global" }' /* - */ : (toLower(datasetType) == 'pricesheet') || (toLower(datasetType) == 'reservationtransactions') ? /* - */ '{ "properties": { "definition": { "dataSet": { "configuration": { "dataVersion": "${schemaVersion}", "filters": [] }}, "timeframe": "${isMonthly ? 'TheCurrentMonth': 'MonthToDate' }", "type": "${datasetType}" }, "deliveryInfo": { "destination": { "container": "${exportContainerName}", "rootFolderPath": "@{if(startswith(item().scope, \'/\'), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}", "type": "AzureBlob", "resourceId": "@{variables(\'storageAccountId\')}" } }, "schedule": { "recurrence": "${ isMonthly ? 'Monthly' : 'Daily'}", "recurrencePeriod": { "from": "2024-01-01T00:00:00.000Z", "to": "2050-02-01T00:00:00.000Z" }, "status": "Inactive" }, "format": "${exportFormat}", "partitionData": "${partitionData}", "dataOverwriteBehavior": "${dataOverwriteBehavior}", "compressionMode": "${compressionMode}" }, "id": "@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-${toLower(datasetType)}\'))}", "name": "@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-${toLower(datasetType)}\'))}", "type": "Microsoft.CostManagement/reports", "identity": { "type": "systemAssigned" }, "location": "global" }' /* - */ : toLower(datasetType) == 'reservationrecommendations' ? /* - */ '{ "properties": { "definition": { "dataSet": { "configuration": { "dataVersion": "${schemaVersion}", "filters": [ { "name": "reservationScope", "value": "${recommendationScope}" }, { "name": "resourceType", "value": "${resourceType}" }, { "name": "lookBackPeriod", "value": "${recommendationLookbackPeriod}" }] }}, "timeframe": "${isMonthly ? 'TheLastMonth': 'MonthToDate' }", "type": "${datasetType}" }, "deliveryInfo": { "destination": { "container": "${exportContainerName}", "rootFolderPath": "@{if(startswith(item().scope, \'/\'), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}", "type": "AzureBlob", "resourceId": "@{variables(\'storageAccountId\')}" } }, "schedule": { "recurrence": "${ isMonthly ? 'Monthly' : 'Daily'}", "recurrencePeriod": { "from": "2024-01-01T00:00:00.000Z", "to": "2050-02-01T00:00:00.000Z" }, "status": "Inactive" }, "format": "${exportFormat}", "partitionData": "${partitionData}", "dataOverwriteBehavior": "${dataOverwriteBehavior}", "compressionMode": "${compressionMode}" }, "id": "@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-costdetails\'))}", "name": "@{toLower(concat(variables(\'finOpsHub\'), \'-${ isMonthly ? 'monthly' : 'daily'}-costdetails\'))}", "type": "Microsoft.CostManagement/reports", "identity": { "type": "systemAssigned" }, "location": "global" }' /* - */ : 'undefined' - -var deployDataExplorer = !empty(dataExplorerId) -var useFabric = !deployDataExplorer && !empty(dataExplorerUri) - -var datasetPropsDefault = { - location: { - type: 'AzureBlobFSLocation' - fileName: { - value: '@{dataset().fileName}' - type: 'Expression' - } - folderPath: { - value: '@{dataset().folderPath}' - type: 'Expression' - } - } -} - -var safeExportContainerName = replace('${exportContainerName}', '-', '_') -var safeIngestionContainerName = replace('${ingestionContainerName}', '-', '_') -var safeConfigContainerName = replace('${configContainerName}', '-', '_') -// cSpell:ignore vnet -var managedVnetName = 'default' - -// Separator used to separate ingestion ID from file name for ingested files -var ingestionIdFileNameSeparator = '__' - -// All hub triggers (used to auto-start) -var exportManifestAddedTriggerName = '${safeExportContainerName}_ManifestAdded' -var ingestionManifestAddedTriggerName = '${safeIngestionContainerName}_ManifestAdded' -var updateConfigTriggerName = '${safeConfigContainerName}_SettingsUpdated' -var dailyTriggerName = '${safeConfigContainerName}_DailySchedule' -var monthlyTriggerName = '${safeConfigContainerName}_MonthlySchedule' -var allHubTriggers = [ - exportManifestAddedTriggerName - ingestionManifestAddedTriggerName - updateConfigTriggerName - dailyTriggerName - monthlyTriggerName -] - -// Roles needed to auto-start triggers -var autoStartRbacRoles = [ - // Data Factory contributor -- https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#data-factory-contributor - // Used to start/stop triggers and delete old pipelines/triggers - '673868aa-7521-48a0-acc6-0f60742d39f5' -] - -// Roles for ADF to manage data in storage -// Does not include roles assignments needed against the export scope -var storageRbacRoles = union ( - [ - // Storage Account Contributor -- https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#storage-account-contributor - // Used to move files from the msexports to ingestion container - '17d1049b-9a84-46fb-8f53-869881c3d3ab' - // Storage Blob Data Contributor -- https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#storage-blob-data-contributor - 'ba92f5b4-2d11-453d-a403-e96b0029c9fe' - // Reader -- https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#reader - 'acdd72a7-3385-48ef-bd42-f606fba81ae7' - ], - // Only use User Access Administrator if managed exports are enabled for least privileged access - !enableManagedExports ? [] : [ - // User Access Administrator -- https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#user-access-administrator - '18d7d88d-d35e-4fb5-a5c3-7773c20a72d9' - ] -) - -// Roles for ADF to to start check ADX cluster and to start cluster if stopped -var adxRbacRoles = [ - 'b24988ac-6180-42a0-ab88-20f7382dd24c' // Contributor permissions on the cluster -] - -//============================================================================== -// Resources -//============================================================================== - -// Get data factory instance -resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { - name: dataFactoryName -} - -// Get storage account instance -resource storageAccount 'Microsoft.Storage/storageAccounts@2022-09-01' existing = { - name: storageAccountName -} - -// Get keyvault instance -resource keyVault 'Microsoft.KeyVault/vaults@2023-02-01' existing = if (!empty(remoteHubStorageUri)) { - name: keyVaultName -} - -// Get ADX cluster instance -resource dataExplorerCluster 'Microsoft.Kusto/clusters@2023-08-15' existing = if (deployDataExplorer) { - name: dataExplorerName -} - -// cSpell:ignore azuretimezones -module azuretimezones 'azuretimezones.bicep' = { - name: 'azuretimezones' - params: { - location: location - } -} - -resource managedVirtualNetwork 'Microsoft.DataFactory/factories/managedVirtualNetworks@2018-06-01' = if (!enablePublicAccess) { - name: managedVnetName - parent: dataFactory - properties: {} -} - -resource managedIntegrationRuntime 'Microsoft.DataFactory/factories/integrationRuntimes@2018-06-01' = if (!enablePublicAccess) { - name: 'ManagedIntegrationRuntime' - parent: dataFactory - properties: { - type: 'Managed' - managedVirtualNetwork: { - referenceName: managedVnetName - type: 'ManagedVirtualNetworkReference' - } - typeProperties: { - computeProperties: { - location: location - dataFlowProperties: { - computeType: 'General' - coreCount: 8 - timeToLive: 10 - cleanup: false - customProperties: [] - } - copyComputeScaleProperties: { - dataIntegrationUnit: 16 - timeToLive: 30 - } - pipelineExternalComputeScaleProperties: { - timeToLive: 30 - numberOfPipelineNodes: 1 - numberOfExternalNodes: 1 - } - } - } - } - dependsOn: [ - managedVirtualNetwork - ] -} - -resource storageManagedPrivateEndpoint 'Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints@2018-06-01' = if (!enablePublicAccess) { - name: storageAccount.name - parent: managedVirtualNetwork - properties: { - name: storageAccount.name - groupId: 'dfs' - privateLinkResourceId: storageAccount.id - fqdns: [ - storageAccount.properties.primaryEndpoints.dfs - ] - } -} - -module getStoragePrivateEndpointConnections 'storageEndpoints.bicep' = if (!enablePublicAccess) { - name: 'GetStoragePrivateEndpointConnections' - dependsOn: [ - storageManagedPrivateEndpoint - ] - params: { - storageAccountName: storageAccount.name - } -} - -module approveStoragePrivateEndpointConnections 'storageEndpoints.bicep' = if (!enablePublicAccess) { - name: 'ApproveStoragePrivateEndpointConnections' - params: { - storageAccountName: storageAccount.name - privateEndpointConnections: getStoragePrivateEndpointConnections.outputs.privateEndpointConnections - } -} - -resource keyVaultManagedPrivateEndpoint 'Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints@2018-06-01' = if (!empty(remoteHubStorageUri) && !enablePublicAccess) { - name: keyVault.name - parent: managedVirtualNetwork - properties: { - name: keyVault.name - groupId: 'vault' - privateLinkResourceId: keyVault.id - fqdns: [ - keyVault.properties.vaultUri - ] - } -} - -module getKeyVaultPrivateEndpointConnections 'keyVaultEndpoints.bicep' = if (!empty(remoteHubStorageUri) && !enablePublicAccess) { - name: 'GetKeyVaultPrivateEndpointConnections' - dependsOn: [ - keyVaultManagedPrivateEndpoint - ] - params: { - keyVaultName: keyVault.name - } -} - -module approveKeyVaultPrivateEndpointConnections 'keyVaultEndpoints.bicep' = if (!empty(remoteHubStorageUri) && !enablePublicAccess) { - name: 'ApproveKeyVaultPrivateEndpointConnections' - params: { - keyVaultName: keyVault.name - privateEndpointConnections: getKeyVaultPrivateEndpointConnections.outputs.privateEndpointConnections - } -} - -resource dataExplorerManagedPrivateEndpoint 'Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints@2018-06-01' = if (deployDataExplorer && !enablePublicAccess) { - name: hubDataExplorerName - parent: managedVirtualNetwork - properties: { - name: hubDataExplorerName - groupId: 'cluster' - privateLinkResourceId: dataExplorerId - fqdns: [ - dataExplorerUri - ] - } -} - -module getDataExplorerPrivateEndpointConnections 'dataExplorerEndpoints.bicep' = if (deployDataExplorer && !enablePublicAccess) { - name: 'GetDataExplorerPrivateEndpointConnections' - dependsOn: [ - dataExplorerManagedPrivateEndpoint - ] - params: { - dataExplorerName: dataExplorerName - } -} - -module approveDataExplorerPrivateEndpointConnections 'dataExplorerEndpoints.bicep' = if (deployDataExplorer && !enablePublicAccess) { - name: 'ApproveDataExplorerPrivateEndpointConnections' - params: { - dataExplorerName: dataExplorerName - privateEndpointConnections: getDataExplorerPrivateEndpointConnections.outputs.privateEndpointConnections - } -} - -//------------------------------------------------------------------------------ -// Identities and RBAC -//------------------------------------------------------------------------------ - -// Create managed identity to start/stop triggers -resource triggerManagerIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = { - name: '${dataFactory.name}_triggerManager' - location: location - tags: union(tags, tagsByResource[?'Microsoft.ManagedIdentity/userAssignedIdentities'] ?? {}) -} - -resource triggerManagerRoleAssignments 'Microsoft.Authorization/roleAssignments@2022-04-01' = [for role in autoStartRbacRoles: { - name: guid(dataFactory.id, role, triggerManagerIdentity.id) - scope: dataFactory - properties: { - roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', role) - principalId: triggerManagerIdentity.properties.principalId - principalType: 'ServicePrincipal' - } -}] - -// Grant ADF identity access to manage data in storage -resource factoryIdentityStorageRoleAssignments 'Microsoft.Authorization/roleAssignments@2022-04-01' = [for role in storageRbacRoles: { - name: guid(storageAccount.id, role, dataFactory.id) - scope: storageAccount - properties: { - roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', role) - principalId: dataFactory.identity.principalId - principalType: 'ServicePrincipal' - } -}] - -// Grant ADF identity access to manage ADX cluster -resource factoryIdentityDataExplorerRoleAssignments 'Microsoft.Authorization/roleAssignments@2022-04-01' = [for role in adxRbacRoles: if (deployDataExplorer) { - name: guid(dataExplorerCluster.id, role, dataFactory.id) - scope: dataExplorerCluster - properties: { - roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', role) - principalId: dataFactory.identity.principalId - principalType: 'ServicePrincipal' - } -}] - -//------------------------------------------------------------------------------ -// Delete old triggers and pipelines -//------------------------------------------------------------------------------ - -module deleteOldResources 'hub-deploymentScript.bicep' = { - name: 'Microsoft.FinOpsHubs.Core_ADF.DeleteOldResources' - dependsOn: [ - triggerManagerRoleAssignments - stopTriggers - ] - params: { - app: app - identityName: triggerManagerIdentity.name - scriptContent: loadTextContent('./scripts/Remove-OldResources.ps1') - environmentVariables: [ - { - name: 'DataFactorySubscriptionId' - value: subscription().id - } - { - name: 'DataFactoryResourceGroup' - value: resourceGroup().name - } - { - name: 'DataFactoryName' - value: dataFactory.name - } - ] - } -} - -//------------------------------------------------------------------------------ -// Stop all triggers before deploying -//------------------------------------------------------------------------------ - -module stopTriggers 'hub-deploymentScript.bicep' = { - name: 'Microsoft.FinOpsHubs.Core_ADF.StopTriggers' - dependsOn: [ - triggerManagerRoleAssignments - ] - params: { - app: app - identityName: triggerManagerIdentity.name - scriptContent: loadTextContent('./scripts/Start-Triggers.ps1') - arguments: '-Stop' - environmentVariables: [ - { - name: 'DataFactorySubscriptionId' - value: subscription().id - } - { - name: 'DataFactoryResourceGroup' - value: resourceGroup().name - } - { - name: 'DataFactoryName' - value: dataFactory.name - } - { - name: 'Triggers' - value: join(allHubTriggers, '|') - } - ] - } -} - -//------------------------------------------------------------------------------ -// Linked services -//------------------------------------------------------------------------------ - -// cSpell:ignore linkedservices -// TODO: Move to the hub-app module -resource linkedService_keyVault 'Microsoft.DataFactory/factories/linkedservices@2018-06-01' = if (!empty(remoteHubStorageUri)) { - name: keyVault.name - parent: dataFactory - dependsOn: enablePublicAccess ? [] : [managedIntegrationRuntime] - properties: { - annotations: [] - parameters: {} - type: 'AzureKeyVault' - typeProperties: { - baseUrl: reference('Microsoft.KeyVault/vaults/${keyVault.name}', '2023-02-01').vaultUri - } - connectVia: enablePublicAccess ? null : { - referenceName: managedIntegrationRuntime.name - type: 'IntegrationRuntimeReference' - } - } -} - -// TODO: Move to the hub-app module -resource linkedService_storageAccount 'Microsoft.DataFactory/factories/linkedservices@2018-06-01' = { - name: storageAccount.name - parent: dataFactory - dependsOn: enablePublicAccess ? [] : [managedIntegrationRuntime] - properties: { - annotations: [] - parameters: {} - type: 'AzureBlobFS' - typeProperties: { - url: reference('Microsoft.Storage/storageAccounts/${storageAccount.name}', '2021-08-01').primaryEndpoints.dfs - } - connectVia: enablePublicAccess ? null : { - referenceName: managedIntegrationRuntime.name - type: 'IntegrationRuntimeReference' - } - } -} - -resource linkedService_dataExplorer 'Microsoft.DataFactory/factories/linkedservices@2018-06-01' = if (deployDataExplorer || useFabric) { - name: hubDataExplorerName - parent: dataFactory - dependsOn: enablePublicAccess ? [] : [managedIntegrationRuntime] - properties: { - type: 'AzureDataExplorer' - parameters: { - database: { - type: 'String' - defaultValue: dataExplorerIngestionDatabase - } - } - typeProperties: { - endpoint: dataExplorerUri - database: '@{linkedService().database}' - tenant: dataFactory.identity.tenantId - servicePrincipalId: dataFactory.identity.principalId - } - connectVia: enablePublicAccess ? null : { - referenceName: managedIntegrationRuntime.name - type: 'IntegrationRuntimeReference' - } - } -} - -resource linkedService_remoteHubStorage 'Microsoft.DataFactory/factories/linkedservices@2018-06-01' = if (!empty(remoteHubStorageUri)) { - name: 'remoteHubStorage' - parent: dataFactory - dependsOn: enablePublicAccess ? [] : [managedIntegrationRuntime] - properties: { - annotations: [] - parameters: {} - type: 'AzureBlobFS' - typeProperties: { - url: remoteHubStorageUri - accountKey: { - type: 'AzureKeyVaultSecret' - store: { - referenceName: linkedService_keyVault.name - type: 'LinkedServiceReference' - } - secretName: '${toLower(hubName)}-storage-key' - } - } - connectVia: enablePublicAccess ? null : { - referenceName: managedIntegrationRuntime.name - type: 'IntegrationRuntimeReference' - } - } -} - -resource linkedService_ftkRepo 'Microsoft.DataFactory/factories/linkedservices@2018-06-01' = { - name: 'ftkRepo' - parent: dataFactory - dependsOn: enablePublicAccess ? [] : [managedIntegrationRuntime] - properties: { - parameters: { - filePath: { - type: 'string' - } - } - annotations: [] - type: 'HttpServer' - typeProperties: { - url: '@concat(\'https://github.com/microsoft/finops-toolkit/\', linkedService().filePath)' - enableServerCertificateValidation: true - authenticationType: 'Anonymous' - } - connectVia: enablePublicAccess ? null : { - referenceName: managedIntegrationRuntime.name - type: 'IntegrationRuntimeReference' - } - } -} - -//------------------------------------------------------------------------------ -// Datasets -//------------------------------------------------------------------------------ - -resource dataset_config 'Microsoft.DataFactory/factories/datasets@2018-06-01' = { - name: safeConfigContainerName - parent: dataFactory - properties: { - annotations: [] - parameters: { - fileName: { - type: 'String' - defaultValue: 'settings.json' - } - folderPath: { - type: 'String' - defaultValue: configContainerName - } - } - type: 'Json' - typeProperties: datasetPropsDefault - linkedServiceName: { - parameters: {} - referenceName: linkedService_storageAccount.name - type: 'LinkedServiceReference' - } - } -} - -resource dataset_manifest 'Microsoft.DataFactory/factories/datasets@2018-06-01' = { - name: 'manifest' - parent: dataFactory - properties: { - annotations: [] - parameters: { - fileName: { - type: 'String' - defaultValue: 'manifest.json' - } - folderPath: { - type: 'String' - defaultValue: exportContainerName - } - } - type: 'Json' - typeProperties: datasetPropsDefault - linkedServiceName: { - parameters: {} - referenceName: linkedService_storageAccount.name - type: 'LinkedServiceReference' - } - } -} - -resource dataset_msexports 'Microsoft.DataFactory/factories/datasets@2018-06-01' = { - name: safeExportContainerName - parent: dataFactory - properties: { - annotations: [] - parameters: { - blobPath: { - type: 'String' - } - } - type: 'DelimitedText' - typeProperties: { - location: { - type: 'AzureBlobFSLocation' - fileName: { - value: '@{dataset().blobPath}' - type: 'Expression' - } - fileSystem: safeExportContainerName - } - columnDelimiter: ',' - escapeChar: '"' - quoteChar: '"' - firstRowAsHeader: true - } - linkedServiceName: { - parameters: {} - referenceName: linkedService_storageAccount.name - type: 'LinkedServiceReference' - } - } -} - -resource dataset_msexports_gzip 'Microsoft.DataFactory/factories/datasets@2018-06-01' = { - name: '${safeExportContainerName}_gzip' - parent: dataFactory - properties: { - annotations: [] - parameters: { - blobPath: { - type: 'String' - } - } - type: 'DelimitedText' - typeProperties: { - location: { - type: 'AzureBlobFSLocation' - fileName: { - value: '@{dataset().blobPath}' - type: 'Expression' - } - fileSystem: safeExportContainerName - } - columnDelimiter: ',' - escapeChar: '"' - quoteChar: '"' - firstRowAsHeader: true - compressionCodec: 'Gzip' - } - linkedServiceName: { - parameters: {} - referenceName: linkedService_storageAccount.name - type: 'LinkedServiceReference' - } - } -} - -resource dataset_msexports_parquet 'Microsoft.DataFactory/factories/datasets@2018-06-01' = { - name: '${safeExportContainerName}_parquet' - parent: dataFactory - properties: { - annotations: [] - parameters: { - blobPath: { - type: 'String' - } - } - type: 'Parquet' - typeProperties: { - location: { - type: 'AzureBlobFSLocation' - fileName: { - value: '@{dataset().blobPath}' - type: 'Expression' - } - fileSystem: safeExportContainerName - } - } - linkedServiceName: { - parameters: {} - referenceName: linkedService_storageAccount.name - type: 'LinkedServiceReference' - } - } -} - -resource dataset_ingestion 'Microsoft.DataFactory/factories/datasets@2018-06-01' = { - name: safeIngestionContainerName - parent: dataFactory - properties: { - annotations: [] - parameters: { - blobPath: { - type: 'String' - } - } - type: 'Parquet' - typeProperties: { - location: { - type: 'AzureBlobFSLocation' - fileName: { - value: '@{dataset().blobPath}' - type: 'Expression' - } - fileSystem: safeIngestionContainerName - } - } - linkedServiceName: { - parameters: {} - referenceName: empty(remoteHubStorageUri) ? linkedService_storageAccount.name : linkedService_remoteHubStorage.name - type: 'LinkedServiceReference' - } - } -} - -resource dataset_ingestion_files 'Microsoft.DataFactory/factories/datasets@2018-06-01' = { - name: '${safeIngestionContainerName}_files' - parent: dataFactory - properties: { - annotations: [] - parameters: { - folderPath: { - type: 'String' - } - } - type: 'Parquet' - typeProperties: { - location: { - type: 'AzureBlobFSLocation' - fileSystem: safeIngestionContainerName - folderPath: { - value: '@dataset().folderPath' - type: 'Expression' - } - } - } - linkedServiceName: { - parameters: {} - referenceName: empty(remoteHubStorageUri) ? linkedService_storageAccount.name : linkedService_remoteHubStorage.name - type: 'LinkedServiceReference' - } - } -} - -resource dataset_dataExplorer 'Microsoft.DataFactory/factories/datasets@2018-06-01' = if (deployDataExplorer || useFabric) { - name: hubDataExplorerName - parent: dataFactory - properties: { - type: 'AzureDataExplorerTable' - linkedServiceName: { - parameters: { - database: '@dataset().database' - } - referenceName: linkedService_dataExplorer.name - type: 'LinkedServiceReference' - } - parameters: { - database: { - type: 'String' - defaultValue: dataExplorerIngestionDatabase - } - table: { type: 'String' } - } - typeProperties: { - table: { - value: '@dataset().table' - type: 'Expression' - } - } - } -} - -resource dataset_ftkReleaseFile 'Microsoft.DataFactory/factories/datasets@2018-06-01' = { - name: 'ftkReleaseFile' - parent: dataFactory - properties: { - linkedServiceName: { - referenceName: linkedService_ftkRepo.name - type: 'LinkedServiceReference' - } - parameters: { - fileName: { - type: 'string' - } - version: { - type: 'string' - defaultValue: ftkVersion - } - } - annotations: [] - type: 'DelimitedText' - typeProperties: { - location: { - type: 'HttpServerLocation' - relativeUrl: { - value: '@concat(\'releases/download/v\', dataset().version, \'/\', dataset().fileName)' - type: 'Expression' - } - } - columnDelimiter: ',' - escapeChar: '\\' - firstRowAsHeader: true - quoteChar: '"' - } - schema: [] - } -} - -//------------------------------------------------------------------------------ -// Triggers -//------------------------------------------------------------------------------ - -// TODO: Create apps_PublishEvent pipeline { event, properties } - -module trigger_ExportManifestAdded 'hub-event-trigger.bicep' = { - name: 'Microsoft.FinOpsHubs.Core_ExportManifestAddedTrigger' - dependsOn: [ - stopTriggers - ] - params: { - dataFactoryName: dataFactory.name - triggerName: exportManifestAddedTriggerName - - // TODO: Replace pipeline with event: 'Microsoft.CostManagement.Exports.ManifestAdded' - pipelineName: pipeline_ExecuteExportsETL.name - pipelineParameters: { - folderPath: '@triggerBody().folderPath' - fileName: '@triggerBody().fileName' - } - - storageAccountName: storageAccount.name - storageContainer: exportContainerName - storagePathEndsWith: 'manifest.json' - } -} - -module trigger_IngestionManifestAdded 'hub-event-trigger.bicep' = if (deployDataExplorer || useFabric) { - name: 'Microsoft.FinOpsHubs.Core_IngestionManifestAddedTrigger' - dependsOn: [ - stopTriggers - ] - params: { - dataFactoryName: dataFactory.name - triggerName: ingestionManifestAddedTriggerName - - // TODO: Replace pipeline with event: 'Microsoft.FinOpsHubs.Core.IngestionManifestAdded' - pipelineName: pipeline_ExecuteIngestionETL.name - pipelineParameters: { - folderPath: '@triggerBody().folderPath' - } - - storageAccountName: storageAccount.name - storageContainer: ingestionContainerName - storagePathEndsWith: 'manifest.json' - } -} - -module trigger_SettingsUpdated 'hub-event-trigger.bicep' = if (enableManagedExports) { - name: 'Microsoft.FinOpsHubs.Core_SettingsUpdatedTrigger' - dependsOn: [ - stopTriggers - ] - params: { - dataFactoryName: dataFactory.name - triggerName: updateConfigTriggerName - - // TODO: Replace pipeline with event: 'Microsoft.FinOpsHubs.Core.SettingsUpdated' - pipelineName: pipeline_ConfigureExports.name - pipelineParameters: {} - - storageAccountName: storageAccount.name - storageContainer: configContainerName - // TODO: Change this to startswith - storagePathEndsWith: 'settings.json' - } -} - -resource trigger_DailySchedule 'Microsoft.DataFactory/factories/triggers@2018-06-01' = if (enableManagedExports) { - name: dailyTriggerName - parent: dataFactory - dependsOn: [ - stopTriggers - ] - properties: { - pipelines: [ - { - pipelineReference: { - referenceName: pipeline_StartExportProcess.name - type: 'PipelineReference' - } - parameters: { - Recurrence: 'Daily' - } - } - ] - type: 'ScheduleTrigger' - typeProperties: { - recurrence: { - frequency: 'Hour' - interval: 24 - startTime: '2023-01-01T01:01:00' - timeZone: azuretimezones.outputs.Timezone - } - } - } -} - -resource trigger_MonthlySchedule 'Microsoft.DataFactory/factories/triggers@2018-06-01' = if (enableManagedExports) { - name: monthlyTriggerName - parent: dataFactory - dependsOn: [ - stopTriggers - ] - properties: { - pipelines: [ - { - pipelineReference: { - referenceName: pipeline_StartExportProcess.name - type: 'PipelineReference' - } - parameters: { - Recurrence: 'Monthly' - } - } - ] - type: 'ScheduleTrigger' - typeProperties: { - recurrence: { - frequency: 'Month' - interval: 1 - startTime: '2023-01-05T01:11:00' - timeZone: azuretimezones.outputs.Timezone - schedule: { - monthDays: [ - 2 - 5 - 19 - ] - } - } - } - } -} - -//------------------------------------------------------------------------------ -// Pipelines -//------------------------------------------------------------------------------ - -//------------------------------------------------------------------------------ -// config_InitializeHub pipeline -//------------------------------------------------------------------------------ -@description('Initializes the hub instance based on the configuration settings.') -resource pipeline_InitializeHub 'Microsoft.DataFactory/factories/pipelines@2018-06-01' = if (deployDataExplorer || useFabric) { - name: '${safeConfigContainerName}_InitializeHub' - parent: dataFactory - properties: { - activities: [ - { // Get Config - name: 'Get Config' - type: 'Lookup' - dependsOn: [] - policy: { - timeout: '0.00:05:00' - retry: 2 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'JsonSource' - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: true - enablePartitionDiscovery: false - } - formatSettings: { - type: 'JsonReadSettings' - } - } - dataset: { - referenceName: dataset_config.name - type: 'DatasetReference' - } - } - } - { // Set Version - name: 'Set Version' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Get Config' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - variableName: 'version' - value: { - value: '@activity(\'Get Config\').output.firstRow.version' - type: 'Expression' - } - } - } - { // Set Scopes - name: 'Set Scopes' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Get Config' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - variableName: 'scopes' - value: { - value: '@string(activity(\'Get Config\').output.firstRow.scopes)' - type: 'Expression' - } - } - } - { // Set Retention - name: 'Set Retention' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Get Config' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - variableName: 'retention' - value: { - value: '@string(activity(\'Get Config\').output.firstRow.retention)' - type: 'Expression' - } - } - } - { // Until Capacity Is Available - name: 'Until Capacity Is Available' - type: 'Until' - dependsOn: [ - { - activity: 'Set Version' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Set Scopes' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Set Retention' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - expression: { - value: '@equals(variables(\'tryAgain\'), false)' - type: 'Expression' - } - activities: [ - { // Confirm Ingestion Capacity - name: 'Confirm Ingestion Capacity' - type: 'AzureDataExplorerCommand' - dependsOn: [] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - // cSpell:ignore Ingestions - command: '.show capacity | where Resource == \'Ingestions\' | project Remaining' - commandTimeout: '00:20:00' - } - linkedServiceName: { - referenceName: linkedService_dataExplorer.name - type: 'LinkedServiceReference' - parameters: { - database: dataExplorerIngestionDatabase - } - } - } - { // If Has Capacity - name: 'If Has Capacity' - type: 'IfCondition' - dependsOn: [ - { - activity: 'Confirm Ingestion Capacity' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - expression: { - value: '@or(equals(activity(\'Confirm Ingestion Capacity\').output.count, 0), greater(activity(\'Confirm Ingestion Capacity\').output.value[0].Remaining, 0))' - type: 'Expression' - } - ifFalseActivities: [ - { // Wait for Ingestion - name: 'Wait for Ingestion' - type: 'Wait' - dependsOn: [] - userProperties: [] - typeProperties: { - waitTimeInSeconds: 15 - } - } - { // Try Again - name: 'Try Again' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Wait for Ingestion' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'tryAgain' - value: true - } - } - ] - ifTrueActivities: [ - { // Save ingestion policy in ADX - name: 'Set ingestion policy in ADX' - type: 'AzureDataExplorerCommand' - dependsOn: [] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - command: { - // Do not attempt to set the ingestion policy if using Fabric; use a simple query as a placeholder - value: useFabric - ? '.show database ${dataExplorerIngestionDatabase} policy managed_identity' - : '.alter-merge database ${dataExplorerIngestionDatabase} policy managed_identity "[ { \'ObjectId\' : \'${dataExplorerPrincipalId}\', \'AllowedUsages\' : \'NativeIngestion\' }]"' - type: 'Expression' - } - commandTimeout: '00:20:00' - } - linkedServiceName: { - referenceName: linkedService_dataExplorer.name - type: 'LinkedServiceReference' - parameters: { - database: dataExplorerIngestionDatabase - } - } - } - { // Save Hub Settings in ADX - name: 'Save Hub Settings in ADX' - type: 'AzureDataExplorerCommand' - dependsOn: [ - { - activity: 'Set ingestion policy in ADX' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - command: { - // cSpell:ignore isnull, isnotempty - value: '@concat(\'.append HubSettingsLog <| print version="\', variables(\'version\'), \'",scopes=dynamic(\', variables(\'scopes\'), \'),retention=dynamic(\', variables(\'retention\'), \') | extend scopes = iff(isnull(scopes[0]), pack_array(scopes), scopes) | mv-apply scopeObj = scopes on (where isnotempty(scopeObj.scope) | summarize scopes = make_set(scopeObj.scope))\')' - type: 'Expression' - } - commandTimeout: '00:20:00' - } - linkedServiceName: { - referenceName: linkedService_dataExplorer.name - type: 'LinkedServiceReference' - parameters: { - database: dataExplorerIngestionDatabase - } - } - } - { // Update PricingUnits in ADX - name: 'Update PricingUnits in ADX' - type: 'AzureDataExplorerCommand' - dependsOn: [ - { - activity: 'Save Hub Settings in ADX' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - // cSpell:ignore externaldata - command: '.set-or-replace PricingUnits <| externaldata(x_PricingUnitDescription: string, AccountTypes: string, x_PricingBlockSize: decimal, PricingUnit: string)[@"${ftkReleaseUri}/PricingUnits.csv"] with (format="csv", ignoreFirstRecord=true) | project-away AccountTypes' - commandTimeout: '00:20:00' - } - linkedServiceName: { - referenceName: linkedService_dataExplorer.name - type: 'LinkedServiceReference' - parameters: { - database: dataExplorerIngestionDatabase - } - } - } - { // Update Regions in ADX - name: 'Update Regions in ADX' - type: 'AzureDataExplorerCommand' - dependsOn: [ - { - activity: 'Update PricingUnits in ADX' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - command: '.set-or-replace Regions <| externaldata(ResourceLocation: string, RegionId: string, RegionName: string)[@"${ftkReleaseUri}/Regions.csv"] with (format="csv", ignoreFirstRecord=true)' - commandTimeout: '00:20:00' - } - linkedServiceName: { - referenceName: linkedService_dataExplorer.name - type: 'LinkedServiceReference' - parameters: { - database: dataExplorerIngestionDatabase - } - } - } - { // Update ResourceTypes in ADX - name: 'Update ResourceTypes in ADX' - type: 'AzureDataExplorerCommand' - dependsOn: [ - { - activity: 'Update Regions in ADX' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - command: '.set-or-replace ResourceTypes <| externaldata(x_ResourceType: string, SingularDisplayName: string, PluralDisplayName: string, LowerSingularDisplayName: string, LowerPluralDisplayName: string, IsPreview: bool, Description: string, IconUri: string, Links: string)[@"${ftkReleaseUri}/ResourceTypes.csv"] with (format="csv", ignoreFirstRecord=true) | project-away Links' - commandTimeout: '00:20:00' - } - linkedServiceName: { - referenceName: linkedService_dataExplorer.name - type: 'LinkedServiceReference' - parameters: { - database: dataExplorerIngestionDatabase - } - } - } - { // Update Services in ADX - name: 'Update Services in ADX' - type: 'AzureDataExplorerCommand' - dependsOn: [ - { - activity: 'Update ResourceTypes in ADX' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - command: '.set-or-replace Services <| externaldata(x_ConsumedService: string, x_ResourceType: string, ServiceName: string, ServiceCategory: string, ServiceSubcategory: string, PublisherName: string, x_PublisherCategory: string, x_Environment: string, x_ServiceModel: string)[@"${ftkReleaseUri}/Services.csv"] with (format="csv", ignoreFirstRecord=true)' - commandTimeout: '00:20:00' - } - linkedServiceName: { - referenceName: linkedService_dataExplorer.name - type: 'LinkedServiceReference' - parameters: { - database: dataExplorerIngestionDatabase - } - } - } - { // Ingestion Complete - name: 'Ingestion Complete' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Update Services in ADX' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'tryAgain' - value: false - } - } - ] - } - } - { // Abort On Error - name: 'Abort On Error' - type: 'SetVariable' - dependsOn: [ - { - activity: 'If Has Capacity' - dependencyConditions: [ - 'Failed' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'tryAgain' - value: false - } - } - ] - timeout: '0.02:00:00' - } - } - { // Timeout Error - name: 'Timeout Error' - type: 'Fail' - dependsOn: [ - { - activity: 'Until Capacity Is Available' - dependencyConditions: [ - 'Failed' - ] - } - ] - userProperties: [] - typeProperties: { - message: 'Data Explorer ingestion timed out after 2 hours while waiting for available capacity. Please re-run this pipeline to re-attempt ingestion. If you continue to see this error, please report an issue at https://aka.ms/ftk/ideas.' - errorCode: 'DataExplorerIngestionTimeout' - } - } - ] - concurrency: 1 - variables: { - version: { - type: 'String' - } - scopes: { - type: 'String' - } - retention: { - type: 'String' - } - tryAgain: { - type: 'Boolean' - defaultValue: true - } - } - } -} - -//------------------------------------------------------------------------------ -// config_StartBackfillProcess pipeline -//------------------------------------------------------------------------------ -@description('Runs the backfill job for each month based on retention settings.') -resource pipeline_StartBackfillProcess 'Microsoft.DataFactory/factories/pipelines@2018-06-01' = if (enableManagedExports) { - name: '${safeConfigContainerName}_StartBackfillProcess' - parent: dataFactory - properties: { - activities: [ - { // Get Config - name: 'Get Config' - type: 'Lookup' - dependsOn: [] - policy: { - timeout: '0.00:05:00' - retry: 2 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'JsonSource' - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: true - enablePartitionDiscovery: false - } - formatSettings: { - type: 'JsonReadSettings' - } - } - dataset: { - referenceName: dataset_config.name - type: 'DatasetReference' - parameters: { - fileName: { - value: '@variables(\'fileName\')' - type: 'Expression' - } - folderPath: { - value: '@variables(\'folderPath\')' - type: 'Expression' - } - } - } - } - } - { // Set backfill end date - name: 'Set backfill end date' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Get Config' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - variableName: 'endDate' - value: { - value: '@addDays(startOfMonth(utcNow()), -1)' - type: 'Expression' - } - } - } - { // Set backfill start date - name: 'Set backfill start date' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Get Config' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - variableName: 'startDate' - value: { - value: '@subtractFromTime(startOfMonth(utcNow()), activity(\'Get Config\').output.firstRow.retention.ingestion.months, \'Month\')' - type: 'Expression' - } - } - } - { // Set export start date - name: 'Set export start date' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Set backfill start date' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - variableName: 'thisMonth' - value: { - value: '@startOfMonth(variables(\'endDate\'))' - type: 'Expression' - } - } - } - { // Set export end date - name: 'Set export end date' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Set export start date' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - variableName: 'nextMonth' - value: { - value: '@startOfMonth(subtractFromTime(variables(\'thisMonth\'), 1, \'Month\'))' - type: 'Expression' - } - } - } - { // Every Month - name: 'Every Month' - type: 'Until' - dependsOn: [ - { - activity: 'Set export end date' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Set backfill end date' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - expression: { - value: '@less(variables(\'thisMonth\'), variables(\'startDate\'))' - type: 'Expression' - } - activities: [ - { - name: 'Update export start date' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Backfill data' - dependencyConditions: [ - 'Completed' - ] - } - ] - userProperties: [] - typeProperties: { - variableName: 'thisMonth' - value: { - value: '@variables(\'nextMonth\')' - type: 'Expression' - } - } - } - { - name: 'Update export end date' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Update export start date' - dependencyConditions: [ - 'Completed' - ] - } - ] - userProperties: [] - typeProperties: { - variableName: 'nextMonth' - value: { - value: '@subtractFromTime(variables(\'thisMonth\'), 1, \'Month\')' - type: 'Expression' - } - } - } - { - name: 'Backfill data' - type: 'ExecutePipeline' - dependsOn: [] - userProperties: [] - typeProperties: { - pipeline: { - referenceName: pipeline_RunBackfillJob.name - type: 'PipelineReference' - } - waitOnCompletion: true - parameters: { - StartDate: { - value: '@variables(\'thisMonth\')' - type: 'Expression' - } - EndDate: { - value: '@addDays(addToTime(variables(\'thisMonth\'), 1, \'Month\'), -1)' - type: 'Expression' - } - } - } - } - ] - timeout: '0.02:00:00' - } - } - ] - concurrency: 1 - variables: { - exportName: { - type: 'String' - } - storageAccountId: { - type: 'String' - defaultValue: storageAccount.id - } - finOpsHub: { - type: 'String' - defaultValue: hubName - } - resourceManagementUri: { - type: 'String' - defaultValue: environment().resourceManager - } - fileName: { - type: 'String' - defaultValue: 'settings.json' - } - folderPath: { - type: 'String' - defaultValue: configContainerName - } - endDate: { - type: 'String' - } - startDate: { - type: 'String' - } - thisMonth: { - type: 'String' - } - nextMonth: { - type: 'String' - } - } - } -} - -//------------------------------------------------------------------------------ -// config_RunBackfillJob pipeline -// Triggered by config_StartBackfillProcess pipeline -//------------------------------------------------------------------------------ -@description('Creates and triggers exports for all defined scopes for the specified date range.') -resource pipeline_RunBackfillJob 'Microsoft.DataFactory/factories/pipelines@2018-06-01' = if (enableManagedExports) { - name: '${safeConfigContainerName}_RunBackfillJob' - parent: dataFactory - properties: { - activities: [ - { // Get Config - name: 'Get Config' - type: 'Lookup' - dependsOn: [] - policy: { - timeout: '0.00:05:00' - retry: 2 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'JsonSource' - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: true - enablePartitionDiscovery: false - } - formatSettings: { - type: 'JsonReadSettings' - } - } - dataset: { - referenceName: dataset_config.name - type: 'DatasetReference' - parameters: { - fileName: { - value: '@variables(\'fileName\')' - type: 'Expression' - } - folderPath: { - value: '@variables(\'folderPath\')' - type: 'Expression' - } - } - } - } - } - { // Set Scopes - name: 'Set Scopes' - description: 'Save scopes to test if it is an array' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Get Config' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'scopesArray' - value: { - value: '@activity(\'Get Config\').output.firstRow.scopes' - type: 'Expression' - } - } - } - { // Set Scopes as Array - name: 'Set Scopes as Array' - description: 'Wraps a single scope object into an array to work around the PowerShell bug where single-item arrays are sometimes written as a single object instead of an array.' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Set Scopes' - dependencyConditions: [ - 'Failed' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'scopesArray' - value: { - value: '@createArray(activity(\'Get Config\').output.firstRow.scopes)' - type: 'Expression' - } - } - } - { // Filter Invalid Scopes - name: 'Filter Invalid Scopes' - description: 'Remove any invalid scopes to avoid errors.' - type: 'Filter' - dependsOn: [ - { - activity: 'Set Scopes' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Set Scopes as Array' - dependencyConditions: [ - 'Skipped' - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - items: { - value: '@variables(\'scopesArray\')' - type: 'Expression' - } - condition: { - value: '@and(not(empty(item().scope)), not(equals(item().scope, \'/\')))' - type: 'Expression' - } - } - } - { // ForEach Export Scope - name: 'ForEach Export Scope' - type: 'ForEach' - dependsOn: [ - { - activity: 'Filter Invalid Scopes' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - items: { - value: '@activity(\'Filter Invalid Scopes\').output.Value' - type: 'Expression' - } - isSequential: true - activities: [ - { - name: 'Set backfill export name' - type: 'SetVariable' - dependsOn: [] - userProperties: [] - typeProperties: { - variableName: 'exportName' - value: { - // cSpell:ignore costdetails - value: '@toLower(concat(variables(\'finOpsHub\'), \'-monthly-costdetails\'))' - type: 'Expression' - } - } - } - { - name: 'Trigger backfill export' - type: 'WebActivity' - dependsOn: [ - { - activity: 'Set backfill export name' - dependencyConditions: [ - 'Completed' - ] - } - ] - policy: { - timeout: '0.00:05:00' - retry: 1 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - url: { - value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{variables(\'exportName\')}/run?api-version=${exportApiVersion}' - type: 'Expression' - } - method: 'POST' - headers: { - 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunBackfill@${ftkVersion}' - 'Content-Type': 'application/json' - ClientType: 'FinOpsToolkit.Hubs@${ftkVersion}' - } - body: '{"timePeriod" : { "from" : "@{pipeline().parameters.StartDate}", "to" : "@{pipeline().parameters.EndDate}" }}' - authentication: { - type: 'MSI' - resource: { - value: '@variables(\'resourceManagementUri\')' - type: 'Expression' - } - } - } - } - ] - } - } - ] - concurrency: 1 - parameters: { - StartDate: { - type: 'string' - } - EndDate: { - type: 'string' - } - } - variables: { - exportName: { - type: 'String' - } - storageAccountId: { - type: 'String' - defaultValue: storageAccount.id - } - finOpsHub: { - type: 'String' - defaultValue: hubName - } - resourceManagementUri: { - type: 'String' - defaultValue: environment().resourceManager - } - fileName: { - type: 'String' - defaultValue: 'settings.json' - } - folderPath: { - type: 'String' - defaultValue: configContainerName - } - scopesArray: { - type: 'Array' - } - } - } -} - -//------------------------------------------------------------------------------ -// config_StartExportProcess pipeline -// Triggered by config_DailySchedule/MonthlySchedule triggers -//------------------------------------------------------------------------------ -@description('Gets a list of all Cost Management exports configured for this hub based on the scopes defined in settings.json, then runs each export using the config_RunExportJobs pipeline.') -resource pipeline_StartExportProcess 'Microsoft.DataFactory/factories/pipelines@2018-06-01' = if (enableManagedExports) { - name: '${safeConfigContainerName}_StartExportProcess' - parent: dataFactory - properties: { - activities: [ - { // Get Config - name: 'Get Config' - type: 'Lookup' - dependsOn: [] - policy: { - timeout: '0.00:05:00' - retry: 2 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'JsonSource' - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: true - enablePartitionDiscovery: false - } - formatSettings: { - type: 'JsonReadSettings' - } - } - dataset: { - referenceName: dataset_config.name - type: 'DatasetReference' - parameters: { - fileName: { - value: '@variables(\'fileName\')' - type: 'Expression' - } - folderPath: { - value: '@variables(\'folderPath\')' - type: 'Expression' - } - } - } - } - } - { // Set Scopes - name: 'Set Scopes' - description: 'Save scopes to test if it is an array' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Get Config' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'scopesArray' - value: { - value: '@activity(\'Get Config\').output.firstRow.scopes' - type: 'Expression' - } - } - } - { // Set Scopes as Array - name: 'Set Scopes as Array' - description: 'Wraps a single scope object into an array to work around the PowerShell bug where single-item arrays are sometimes written as a single object instead of an array.' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Set Scopes' - dependencyConditions: [ - 'Failed' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'scopesArray' - value: { - value: '@createArray(activity(\'Get Config\').output.firstRow.scopes)' - type: 'Expression' - } - } - } - { // Filter Invalid Scopes - name: 'Filter Invalid Scopes' - description: 'Remove any invalid scopes to avoid errors.' - type: 'Filter' - dependsOn: [ - { - activity: 'Set Scopes' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Set Scopes as Array' - dependencyConditions: [ - 'Succeeded' - 'Skipped' - ] - } - ] - userProperties: [] - typeProperties: { - items: { - value: '@variables(\'scopesArray\')' - type: 'Expression' - } - condition: { - value: '@and(not(empty(item().scope)), not(equals(item().scope, \'/\')))' - type: 'Expression' - } - } - } - { // ForEach Export Scope - name: 'ForEach Export Scope' - type: 'ForEach' - dependsOn: [ - { - activity: 'Filter Invalid Scopes' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - items: { - value: '@activity(\'Filter Invalid Scopes\').output.Value' - type: 'Expression' - } - isSequential: true - activities: [ - { - name: 'Get exports for scope' - type: 'WebActivity' - dependsOn: [] - policy: { - timeout: '0.00:05:00' - retry: 2 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - url: { - value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports?api-version=${exportApiVersion}' - type: 'Expression' - } - method: 'GET' - authentication: { - type: 'MSI' - resource: { - value: '@variables(\'resourceManagementUri\')' - type: 'Expression' - } - } - } - } - { - name: 'Run exports for scope' - type: 'ExecutePipeline' - dependsOn: [ - { - activity: 'Get exports for scope' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - pipeline: { - referenceName: pipeline_RunExportJobs.name - type: 'PipelineReference' - } - waitOnCompletion: true - parameters: { - ExportScopes: { - value: '@activity(\'Get exports for scope\').output.value' - type: 'Expression' - } - Recurrence: { - value: '@pipeline().parameters.Recurrence' - type: 'Expression' - } - } - } - } - ] - } - } - ] - concurrency: 1 - parameters: { - Recurrence: { - type: 'string' - defaultValue: 'Daily' - } - } - variables: { - fileName: { - type: 'String' - defaultValue: 'settings.json' - } - folderPath: { - type: 'String' - defaultValue: configContainerName - } - finOpsHub: { - type: 'String' - defaultValue: hubName - } - resourceManagementUri: { - type: 'String' - defaultValue: environment().resourceManager - } - scopesArray: { - type: 'Array' - } - } - } -} - -//------------------------------------------------------------------------------ -// config_RunExportJobs pipeline -// Triggered by pipeline_StartExportProcess pipeline -//------------------------------------------------------------------------------ -@description('Runs the specified Cost Management exports.') -resource pipeline_RunExportJobs 'Microsoft.DataFactory/factories/pipelines@2018-06-01' = if (enableManagedExports) { - name: '${safeConfigContainerName}_RunExportJobs' - parent: dataFactory - dependsOn: [ - dataset_config - ] - properties: { - activities: [ - { - name: 'ForEach export scope' - type: 'ForEach' - dependsOn: [] - userProperties: [] - typeProperties: { - items: { - value: '@pipeline().parameters.exportScopes' - type: 'Expression' - } - isSequential: true - activities: [ - { - name: 'If scheduled' - type: 'IfCondition' - dependsOn: [] - userProperties: [] - typeProperties: { - expression: { - value: '@and( startswith(toLower(item().name), toLower(variables(\'hubName\'))), and(contains(string(item().properties.schedule), \'recurrence\'), equals(toLower(item().properties.schedule.recurrence), toLower(pipeline().parameters.Recurrence))))' - type: 'Expression' - } - ifTrueActivities: [ - { - name: 'Trigger export' - type: 'WebActivity' - dependsOn: [] - policy: { - timeout: '0.00:05:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - method: 'POST' - url: { - value: '@{replace(toLower(concat(variables(\'resourceManagementUri\'),item().id)), \'com//\', \'com/\')}/run?api-version=${exportApiVersion}' - type: 'Expression' - } - headers: { - 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs@${ftkVersion}' - ClientType: 'FinOpsToolkit.Hubs@${ftkVersion}' - } - body: ' ' - authentication: { - type: 'MSI' - resource: { - value: '@variables(\'resourceManagementUri\')' - type: 'Expression' - } - } - } - } - ] - } - } - ] - } - } - ] - concurrency: 1 - parameters: { - ExportScopes: { - type: 'array' - } - Recurrence: { - type: 'string' - defaultValue: 'Daily' - } - } - variables: { - resourceManagementUri: { - type: 'String' - defaultValue: environment().resourceManager - } - hubName: { - type: 'String' - defaultValue: hubName - } - } - } -} - -//------------------------------------------------------------------------------ -// config_ConfigureExports pipeline -// Triggered by config_SettingsUpdated trigger -//------------------------------------------------------------------------------ -@description('Creates Cost Management exports for supported scopes.') -resource pipeline_ConfigureExports 'Microsoft.DataFactory/factories/pipelines@2018-06-01' = if (enableManagedExports) { - name: '${safeConfigContainerName}_ConfigureExports' - parent: dataFactory - properties: { - activities: [ - { // Get Config - name: 'Get Config' - type: 'Lookup' - dependsOn: [] - policy: { - timeout: '0.00:05:00' - retry: 2 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'JsonSource' - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: true - enablePartitionDiscovery: false - } - formatSettings: { - type: 'JsonReadSettings' - } - } - dataset: { - referenceName: dataset_config.name - type: 'DatasetReference' - parameters: { - fileName: { - value: '@variables(\'fileName\')' - type: 'Expression' - } - folderPath: { - value: '@variables(\'folderPath\')' - type: 'Expression' - } - } - } - } - } - { // Save Scopes - name: 'Save Scopes' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Get Config' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'scopesArray' - value: { - value: '@activity(\'Get Config\').output.firstRow.scopes' - type: 'Expression' - } - } - } - { // Save Scopes as Array - name: 'Save Scopes as Array' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Save Scopes' - dependencyConditions: [ - 'Failed' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'scopesArray' - value: { - value: '@array(activity(\'Get Config\').output.firstRow.scopes)' - type: 'Expression' - } - } - } - { // Filter Invalid Scopes - name: 'Filter Invalid Scopes' - type: 'Filter' - dependsOn: [ - { - activity: 'Save Scopes' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Save Scopes as Array' - dependencyConditions: [ - 'Skipped' - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - items: { - value: '@variables(\'scopesArray\')' - type: 'Expression' - } - condition: { - value: '@and(not(empty(item().scope)), not(equals(item().scope, \'/\')))' - type: 'Expression' - } - } - } - { // ForEach Export Scope - name: 'ForEach Export Scope' - type: 'ForEach' - dependsOn: [ - { - activity: 'Filter Invalid Scopes' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - items: { - value: '@activity(\'Filter Invalid Scopes\').output.value' - type: 'Expression' - } - isSequential: true - activities: [ - { - name: 'Set Export Type' - type: 'SetVariable' - dependsOn: [] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'exportScopeType' - value: { - value: '@if(contains(toLower(item().scope), \'providers/microsoft.billing/billingaccounts\'), if(contains(toLower(item().scope), \':\'), \'mca\', \'ea\'), if(contains(toLower(item().scope), \'subscriptions/\'), \'subscription\', \'undefined\'))' - type: 'Expression' - } - } - } - { - name: 'Switch Export Type' - type: 'Switch' - dependsOn: [ - { - activity: 'Set Export Type' - dependencyConditions: [ 'Succeeded' ] - } - ] - userProperties: [] - typeProperties: { - on: { - value: '@toLower(variables(\'exportScopeType\'))' - type: 'Expression' - } - cases: [ - { // EA - value: 'ea' - activities: [ - { // 'EA open month focus export' - name: 'EA open month focus export' - type: 'WebActivity' - dependsOn: [ - ] - policy: { - timeout: '0.00:05:00' - retry: 2 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - url: { - value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-daily-costdetails\'))}?api-version=${exportApiVersion}' - type: 'Expression' - } - method: 'PUT' - body: { - value: getExportBodyV2(exportContainerName, 'FocusCost', focusSchemaVersion, false, 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '') - type: 'Expression' - } - headers: { - 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.CostsDaily@${ftkVersion}' - ClientType: 'FinOpsToolkit.Hubs@${ftkVersion}' - } - authentication: { - type: 'MSI' - resource: { - value: '@variables(\'resourceManagementUri\')' - type: 'Expression' - } - } - } - } - { // 'EA closed month focus export' - name: 'EA closed month focus export' - type: 'WebActivity' - dependsOn: [ - { - activity: 'EA open month focus export' - dependencyConditions: [ 'Succeeded' ] - } - ] - policy: { - timeout: '0.00:05:00' - retry: 2 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - url: { - value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-monthly-costdetails\'))}?api-version=${exportApiVersion}' - type: 'Expression' - } - method: 'PUT' - body: { - value: getExportBodyV2(exportContainerName, 'FocusCost', focusSchemaVersion, true, 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '') - type: 'Expression' - } - headers: { - 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.CostsMonthly@${ftkVersion}' - ClientType: 'FinOpsToolkit.Hubs@${ftkVersion}' - } - authentication: { - type: 'MSI' - resource: { - value: '@variables(\'resourceManagementUri\')' - type: 'Expression' - } - } - } - } - { // 'EA monthly pricesheet export' - name: 'EA monthly pricesheet export' - type: 'WebActivity' - dependsOn: [ - { - activity: 'EA closed month focus export' - dependencyConditions: [ 'Succeeded' ] - } - ] - policy: { - timeout: '0.00:05:00' - retry: 2 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - url: { - value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-monthly-pricesheet\'))}?api-version=${exportApiVersion}' - type: 'Expression' - } - method: 'PUT' - body: { - value: getExportBodyV2(exportContainerName, 'Pricesheet', exportSchemaVersion, true, 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '') - type: 'Expression' - } - headers: { - 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.Prices@${ftkVersion}' - ClientType: 'FinOpsToolkit.Hubs@${ftkVersion}' - } - authentication: { - type: 'MSI' - resource: { - value: '@variables(\'resourceManagementUri\')' - type: 'Expression' - } - } - } - } - { - name: 'Trigger EA monthly pricesheet export' - type: 'WebActivity' - dependsOn: [ - { - activity: 'EA monthly pricesheet export' - dependencyConditions: [ 'Succeeded' ] - } - ] - policy: { - timeout: '0.00:05:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - method: 'POST' - url: { - value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-monthly-pricesheet\'))}/run?api-version=${exportApiVersion}' - type: 'Expression' - } - headers: { - 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.Prices@${ftkVersion}' - ClientType: 'FinOpsToolkit.Hubs@${ftkVersion}' - } - body: ' ' - authentication: { - type: 'MSI' - resource: { - value: '@variables(\'resourceManagementUri\')' - type: 'Expression' - } - } - } - } - { // 'EA daily reservation details export' - name: 'EA daily reservation details export' - type: 'WebActivity' - dependsOn: [ - { - activity: 'EA monthly pricesheet export' - dependencyConditions: [ 'Succeeded' ] - } - ] - policy: { - timeout: '0.00:05:00' - retry: 2 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - url: { - value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-daily-reservationdetails\'))}?api-version=${exportApiVersion}' - type: 'Expression' - } - method: 'PUT' - body: { - value: getExportBodyV2(exportContainerName, 'ReservationDetails', reservationDetailsSchemaVersion, false, 'CSV', 'None', 'true', 'CreateNewReport', '', '', '') - type: 'Expression' - } - headers: { - 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.ReservationDetails@${ftkVersion}' - ClientType: 'FinOpsToolkit.Hubs@${ftkVersion}' - } - authentication: { - type: 'MSI' - resource: { - value: '@variables(\'resourceManagementUri\')' - type: 'Expression' - } - } - } - } - { // 'EA daily reservation transactions export' - name: 'EA daily reservation transactions export' - type: 'WebActivity' - dependsOn: [ - { - activity: 'EA daily reservation details export' - dependencyConditions: [ 'Succeeded' ] - } - ] - policy: { - timeout: '0.00:05:00' - retry: 2 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - url: { - value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-daily-reservationtransactions\'))}?api-version=${exportApiVersion}' - type: 'Expression' - } - method: 'PUT' - body: { - value: getExportBodyV2(exportContainerName, 'ReservationTransactions', exportSchemaVersion, false, 'CSV', 'None', 'true', 'CreateNewReport', '', '', '') - type: 'Expression' - } - headers: { - 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.ReservationTransactions@${ftkVersion}' - ClientType: 'FinOpsToolkit.Hubs@${ftkVersion}' - } - authentication: { - type: 'MSI' - resource: { - value: '@variables(\'resourceManagementUri\')' - type: 'Expression' - } - } - } - } - { // 'EA daily recommendations shared last30day virtualmachines export' - name: 'EA daily shared 30day virtualmachines' - type: 'WebActivity' - dependsOn: [ - { - activity: 'EA daily reservation transactions export' - dependencyConditions: [ 'Succeeded' ] - } - ] - policy: { - timeout: '0.00:05:00' - retry: 2 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - url: { - value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-daily-recommendations-shared-last30days-virtualmachines\'))}?api-version=${exportApiVersion}' - type: 'Expression' - } - method: 'PUT' - body: { - value: getExportBodyV2(exportContainerName, 'ReservationRecommendations', exportSchemaVersion, false, 'CSV', 'None', 'true', 'CreateNewReport', 'Shared', 'Last30Days', 'VirtualMachines') - type: 'Expression' - } - headers: { - 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.ReservationRecommendations.VM.Shared.30d@${ftkVersion}' - ClientType: 'FinOpsToolkit.Hubs@${ftkVersion}' - } - authentication: { - type: 'MSI' - resource: { - value: '@variables(\'resourceManagementUri\')' - type: 'Expression' - } - } - } - } - ] - } - { // subscription - value: 'subscription' - activities: [ - { // 'Subscription open month focus export' - name: 'Subscription open month focus export' - type: 'WebActivity' - dependsOn: [ - ] - policy: { - timeout: '0.00:05:00' - retry: 2 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - url: { - value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-daily-costdetails\'))}?api-version=${exportApiVersion}' - type: 'Expression' - } - method: 'PUT' - body: { - value: getExportBodyV2(exportContainerName, 'FocusCost', focusSchemaVersion, false, 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '') - type: 'Expression' - } - headers: { - 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.CostsDaily@${ftkVersion}' - ClientType: 'FinOpsToolkit.Hubs@${ftkVersion}' - } - authentication: { - type: 'MSI' - resource: { - value: '@variables(\'resourceManagementUri\')' - type: 'Expression' - } - } - } - } - { // 'Subscription closed month focus export' - name: 'Subscription closed month focus export' - type: 'WebActivity' - dependsOn: [ - { - activity: 'Subscription open month focus export' - dependencyConditions: [ 'Succeeded' ] - } - ] - policy: { - timeout: '0.00:05:00' - retry: 2 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - url: { - value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-monthly-costdetails\'))}?api-version=${exportApiVersion}' - type: 'Expression' - } - method: 'PUT' - body: { - value: getExportBodyV2(exportContainerName, 'FocusCost', focusSchemaVersion, true, 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '') - type: 'Expression' - } - headers: { - 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.CostsMonthly@${ftkVersion}' - ClientType: 'FinOpsToolkit.Hubs@${ftkVersion}' - } - authentication: { - type: 'MSI' - resource: { - value: '@variables(\'resourceManagementUri\')' - type: 'Expression' - } - } - } - } - ] - } - { // MCA - value: 'mca' - activities: [ - { - name: 'Export Type Unsupported Error' - type: 'Fail' - dependsOn: [] - userProperties: [] - typeProperties: { - message: { - value: '@concat(\'MCA agreements are not supported for managed exports :\',variables(\'exportScope\'))' - type: 'Expression' - } - errorCode: 'ExportTypeUnsupported' - } - } - ] - } - ] - defaultActivities: [ - { - name: 'Export Type Not Defined Error' - type: 'Fail' - dependsOn: [] - userProperties: [] - typeProperties: { - message: { - value: '@concat(\'Unable to determine the export scope type for :\',variables(\'exportScope\'))' - type: 'Expression' - } - errorCode: 'ExportTypeNotDefined' - } - } - ] - } - } - ] - } - } - ] - concurrency: 1 - variables: { - scopesArray: { - type: 'Array' - } - exportName: { - type: 'String' - } - exportScope: { - type: 'String' - } - exportScopeType: { - type: 'String' - } - storageAccountId: { - type: 'String' - defaultValue: storageAccount.id - } - finOpsHub: { - type: 'String' - defaultValue: hubName - } - resourceManagementUri: { - type: 'String' - defaultValue: environment().resourceManager - } - fileName: { - type: 'String' - defaultValue: 'settings.json' - } - folderPath: { - type: 'String' - defaultValue: configContainerName - } - } - } -} - -//------------------------------------------------------------------------------ -// msexports_ExecuteETL pipeline -// Triggered by msexports_ManifestAdded trigger -//------------------------------------------------------------------------------ -@description('Queues the msexports_ETL_ingestion pipeline.') -resource pipeline_ExecuteExportsETL 'Microsoft.DataFactory/factories/pipelines@2018-06-01' = { - name: '${safeExportContainerName}_ExecuteETL' - parent: dataFactory - properties: { - activities: [ - { // Wait - name: 'Wait' - description: 'Files may not be available immediately after being created.' - type: 'Wait' - dependsOn: [] - userProperties: [] - typeProperties: { - waitTimeInSeconds: 60 - } - } - { // Read Manifest - name: 'Read Manifest' - description: 'Load the export manifest to determine the scope, dataset, and date range.' - type: 'Lookup' - dependsOn: [ - { - activity: 'Wait' - dependencyConditions: ['Completed'] - } - ] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'JsonSource' - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: true - enablePartitionDiscovery: false - } - formatSettings: { - type: 'JsonReadSettings' - } - } - dataset: { - referenceName: dataset_manifest.name - type: 'DatasetReference' - parameters: { - fileName: { - value: '@pipeline().parameters.fileName' - type: 'Expression' - } - folderPath: { - value: '@pipeline().parameters.folderPath' - type: 'Expression' - } - } - } - } - } - { // Set Has No Rows - name: 'Set Has No Rows' - description: 'Check the row count ' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Read Manifest' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'hasNoRows' - value: { - value: '@or(equals(activity(\'Read Manifest\').output.firstRow.blobCount, null), equals(activity(\'Read Manifest\').output.firstRow.blobCount, 0))' - type: 'Expression' - } - } - } - { // Set Export Dataset Type - name: 'Set Export Dataset Type' - description: 'Save the dataset type from the export manifest.' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Read Manifest' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'exportDatasetType' - value: { - value: '@activity(\'Read Manifest\').output.firstRow.exportConfig.type' - type: 'Expression' - } - } - } - { // Set MCA Column - name: 'Set MCA Column' - description: 'Determines if the dataset schema has channel-specific columns and saves the column name that only exists in MCA to determine if it is an MCA dataset.' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Set Export Dataset Type' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'mcaColumnToCheck' - value: { - // cSpell:ignore pricesheet, reservationtransactions, reservationrecommendations - value: '@if(contains(createArray(\'pricesheet\', \'reservationtransactions\'), toLower(variables(\'exportDatasetType\'))), \'BillingProfileId\', if(equals(toLower(variables(\'exportDatasetType\')), \'reservationrecommendations\'), \'Net Savings\', null))' - type: 'Expression' - } - } - } - { // Set Export Dataset Version - name: 'Set Export Dataset Version' - description: 'Save the dataset version from the export manifest.' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Read Manifest' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'exportDatasetVersion' - value: { - value: '@activity(\'Read Manifest\').output.firstRow.exportConfig.dataVersion' - type: 'Expression' - } - } - } - { // Detect Channel - name: 'Detect Channel' - description: 'Determines what channel this export is from. Switch statement handles the different file types if the mcaColumnToCheck variable is set.' - type: 'Switch' - dependsOn: [ - { - activity: 'Set Has No Rows' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Set MCA Column' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Set Export Dataset Version' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - on: { - value: '@if(or(empty(variables(\'mcaColumnToCheck\')), variables(\'hasNoRows\')), \'ignore\', last(array(split(activity(\'Read Manifest\').output.firstRow.blobs[0].blobName, \'.\'))))' - type: 'Expression' - } - cases: [ - { // csv - value: 'csv' - activities: [ - { - name: 'Check for MCA Column in CSV' - description: 'Checks the dataset to determine if the applicable MCA-specific column exists.' - type: 'Lookup' - dependsOn: [] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'DelimitedTextSource' - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: false - enablePartitionDiscovery: false - } - formatSettings: { - type: 'DelimitedTextReadSettings' - } - } - dataset: { - referenceName: dataset_msexports.name - type: 'DatasetReference' - parameters: { - blobPath: { - value: '@activity(\'Read Manifest\').output.firstRow.blobs[0].blobName' - type: 'Expression' - } - } - } - } - } - { - name: 'Set Schema File with Channel in CSV' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Check for MCA Column in CSV' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'schemaFile' - value: { - value: '@toLower(concat(variables(\'exportDatasetType\'), \'_\', variables(\'exportDatasetVersion\'), if(and(contains(activity(\'Check for MCA Column in CSV\').output, \'firstRow\'), contains(activity(\'Check for MCA Column in CSV\').output.firstRow, variables(\'mcaColumnToCheck\'))), \'_mca\', \'_ea\'), \'.json\'))' - type: 'Expression' - } - } - } - ] - } - { // gz - value: 'gz' - activities: [ - { - name: 'Check for MCA Column in Gzip CSV' - description: 'Checks the dataset to determine if the applicable MCA-specific column exists.' - type: 'Lookup' - dependsOn: [] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'DelimitedTextSource' - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: false - enablePartitionDiscovery: false - } - formatSettings: { - type: 'DelimitedTextReadSettings' - } - } - dataset: { - referenceName: dataset_msexports_gzip.name - type: 'DatasetReference' - parameters: { - blobPath: { - value: '@activity(\'Read Manifest\').output.firstRow.blobs[0].blobName' - type: 'Expression' - } - } - } - } - } - { - name: 'Set Schema File with Channel in Gzip CSV' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Check for MCA Column in Gzip CSV' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'schemaFile' - value: { - value: '@toLower(concat(variables(\'exportDatasetType\'), \'_\', variables(\'exportDatasetVersion\'), if(and(contains(activity(\'Check for MCA Column in Gzip CSV\').output, \'firstRow\'), contains(activity(\'Check for MCA Column in Gzip CSV\').output.firstRow, variables(\'mcaColumnToCheck\'))), \'_mca\', \'_ea\'), \'.json\'))' - type: 'Expression' - } - } - } - ] - } - { // parquet - value: 'parquet' - activities: [ - { - name: 'Check for MCA Column in Parquet' - description: 'Checks the dataset to determine if the applicable MCA-specific column exists.' - type: 'Lookup' - dependsOn: [] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'ParquetSource' - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: false - enablePartitionDiscovery: false - } - formatSettings: { - type: 'ParquetReadSettings' - } - } - dataset: { - referenceName: dataset_msexports_parquet.name - type: 'DatasetReference' - parameters: { - blobPath: { - value: '@activity(\'Read Manifest\').output.firstRow.blobs[0].blobName' - type: 'Expression' - } - } - } - } - } - { - name: 'Set Schema File with Channel for Parquet' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Check for MCA Column in Parquet' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'schemaFile' - value: { - value: '@toLower(concat(variables(\'exportDatasetType\'), \'_\', variables(\'exportDatasetVersion\'), if(and(contains(activity(\'Check for MCA Column in Parquet\').output, \'firstRow\'), contains(activity(\'Check for MCA Column in Parquet\').output.firstRow, variables(\'mcaColumnToCheck\'))), \'_mca\', \'_ea\'), \'.json\'))' - type: 'Expression' - } - } - } - ] - } - ] - defaultActivities: [ - { - name: 'Set Schema File' - type: 'SetVariable' - dependsOn: [] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'schemaFile' - value: { - value: '@toLower(concat(variables(\'exportDatasetType\'), \'_\', variables(\'exportDatasetVersion\'), \'.json\'))' - type: 'Expression' - } - } - } - ] - } - } - { // Set Scope - name: 'Set Scope' - description: 'Save the scope from the export manifest.' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Read Manifest' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'scope' - value: { - value: '@split(toLower(activity(\'Read Manifest\').output.firstRow.exportConfig.resourceId), \'/providers/microsoft.costmanagement/exports/\')[0]' - type: 'Expression' - } - } - } - { // Set Date - name: 'Set Date' - description: 'Save the exported month from the export manifest.' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Read Manifest' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'date' - value: { - value: '@replace(substring(activity(\'Read Manifest\').output.firstRow.runInfo.startDate, 0, 7), \'-\', \'\')' - type: 'Expression' - } - } - } - { // Error: ManifestReadFailed - name: 'Failed to Read Manifest' - type: 'Fail' - dependsOn: [ - { - activity: 'Set Date' - dependencyConditions: ['Failed'] - } - { - activity: 'Set Export Dataset Type' - dependencyConditions: ['Failed'] - } - { - activity: 'Set Scope' - dependencyConditions: ['Failed'] - } - { - activity: 'Read Manifest' - dependencyConditions: ['Failed'] - } - { - activity: 'Set Export Dataset Version' - dependencyConditions: ['Failed'] - } - { - activity: 'Detect Channel' - dependencyConditions: ['Failed'] - } - ] - userProperties: [] - typeProperties: { - message: { - value: '@concat(\'Failed to read the manifest file for this export run. Manifest path: \', pipeline().parameters.folderPath)' - type: 'Expression' - } - errorCode: 'ManifestReadFailed' - } - } - { // Check Schema - name: 'Check Schema' - description: 'Verify that the schema file exists in storage.' - type: 'GetMetadata' - dependsOn: [ - { - activity: 'Set Scope' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Set Date' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Detect Channel' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - dataset: { - referenceName: dataset_config.name - type: 'DatasetReference' - parameters: { - fileName: { - value: '@variables(\'schemaFile\')' - type: 'Expression' - } - folderPath: '${configContainerName}/schemas' - } - } - fieldList: ['exists'] - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: true - enablePartitionDiscovery: false - } - formatSettings: { - type: 'JsonReadSettings' - } - } - } - { // Error: SchemaNotFound - name: 'Schema Not Found' - type: 'Fail' - dependsOn: [ - { - activity: 'Check Schema' - dependencyConditions: ['Failed'] - } - ] - userProperties: [] - typeProperties: { - message: { - value: '@concat(\'The \', variables(\'schemaFile\'), \' schema mapping file was not found. Please confirm version \', variables(\'exportDatasetVersion\'), \' of the \', variables(\'exportDatasetType\'), \' dataset is supported by this version of FinOps hubs. You may need to upgrade to a newer release. To add support for another dataset, you can create a custom mapping file.\')' - type: 'Expression' - } - errorCode: 'SchemaNotFound' - } - } - { // Set Hub Dataset - name: 'Set Hub Dataset' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Set Export Dataset Type' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'hubDataset' - value: { - value: '@if(equals(toLower(variables(\'exportDatasetType\')), \'focuscost\'), \'Costs\', if(equals(toLower(variables(\'exportDatasetType\')), \'pricesheet\'), \'Prices\', if(equals(toLower(variables(\'exportDatasetType\')), \'reservationdetails\'), \'CommitmentDiscountUsage\', if(equals(toLower(variables(\'exportDatasetType\')), \'reservationrecommendations\'), \'Recommendations\', if(equals(toLower(variables(\'exportDatasetType\')), \'reservationtransactions\'), \'Transactions\', if(equals(toLower(variables(\'exportDatasetType\')), \'actualcost\'), \'ActualCosts\', if(equals(toLower(variables(\'exportDatasetType\')), \'amortizedcost\'), \'AmortizedCosts\', toLower(variables(\'exportDatasetType\')))))))))' - type: 'Expression' - } - } - } - { // Set Destination Folder - name: 'Set Destination Folder' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Check Schema' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Set Hub Dataset' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'destinationFolder' - value: { - value: '@replace(concat(variables(\'hubDataset\'),\'/\',substring(variables(\'date\'), 0, 4),\'/\',substring(variables(\'date\'), 4, 2),\'/\',toLower(variables(\'scope\')), if(equals(variables(\'hubDataset\'), \'Recommendations\'), activity(\'Read Manifest\').output.firstRow.exportConfig.exportName, \'\')),\'//\',\'/\')' - type: 'Expression' - } - } - } - { // For Each Blob - name: 'For Each Blob' - description: 'Loop thru each exported file listed in the manifest.' - type: 'ForEach' - dependsOn: [ - { - activity: 'Set Destination Folder' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - items: { - value: '@if(variables(\'hasNoRows\'), json(\'[]\'), activity(\'Read Manifest\').output.firstRow.blobs)' - type: 'Expression' - } - batchCount: enablePublicAccess ? 30 : 4 // so we don't overload the managed runtime - isSequential: false - activities: [ - { // Execute - name: 'Execute' - description: 'Run the ingestion ETL pipeline.' - type: 'ExecutePipeline' - dependsOn: [] - policy: { - secureInput: false - } - userProperties: [] - typeProperties: { - pipeline: { - referenceName: pipeline_ToIngestion.name - type: 'PipelineReference' - } - waitOnCompletion: true - parameters: { - blobPath: { - value: '@item().blobName' - type: 'Expression' - } - destinationFolder: { - value: '@variables(\'destinationFolder\')' - type: 'Expression' - } - destinationFile: { - value: '@last(array(split(replace(replace(item().blobName, \'.gz\', \'\'), \'.csv\', \'.parquet\'), \'/\')))' - type: 'Expression' - } - ingestionId: { - value: '@activity(\'Read Manifest\').output.firstRow.runInfo.runId' - type: 'Expression' - } - schemaFile: { - value: '@variables(\'schemaFile\')' - type: 'Expression' - } - exportDatasetType: { - value: '@variables(\'exportDatasetType\')' - type: 'Expression' - } - exportDatasetVersion: { - value: '@variables(\'exportDatasetVersion\')' - type: 'Expression' - } - } - } - } - ] - } - } - { // Copy Manifest - name: 'Copy Manifest' - description: 'Copy the manifest to the ingestion container to trigger ADX ingestion' - type: 'Copy' - dependsOn: [ - { - activity: 'For Each Blob' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'JsonSource' - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: true - enablePartitionDiscovery: false - } - formatSettings: { - type: 'JsonReadSettings' - } - } - sink: { - type: 'JsonSink' - storeSettings: { - type: 'AzureBlobFSWriteSettings' - } - formatSettings: { - type: 'JsonWriteSettings' - } - } - enableStaging: false - } - inputs: [ - { - referenceName: dataset_manifest.name - type: 'DatasetReference' - parameters: { - fileName: 'manifest.json' - folderPath: { - value: '@pipeline().parameters.folderPath' - type: 'Expression' - } - } - } - ] - outputs: [ - { - referenceName: dataset_manifest.name - type: 'DatasetReference' - parameters: { - fileName: 'manifest.json' - folderPath: { - value: '@concat(\'${ingestionContainerName}/\', variables(\'destinationFolder\'))' - type: 'Expression' - } - } - } - ] - } - ] - parameters: { - folderPath: { - type: 'string' - } - fileName: { - type: 'string' - } - } - variables: { - date: { - type: 'String' - } - destinationFolder: { - type: 'String' - } - exportDatasetType: { - type: 'String' - } - exportDatasetVersion: { - type: 'String' - } - hasNoRows: { - type: 'Boolean' - } - hubDataset: { - type: 'String' - } - mcaColumnToCheck: { - type: 'String' - } - schemaFile: { - type: 'String' - } - scope: { - type: 'String' - } - } - annotations: [ - 'New export' - ] - } -} - -//------------------------------------------------------------------------------ -// msexports_ETL_ingestion pipeline -// Triggered by msexports_ExecuteETL -//------------------------------------------------------------------------------ -@description('Transforms CSV data to a standard schema and converts to Parquet.') -resource pipeline_ToIngestion 'Microsoft.DataFactory/factories/pipelines@2018-06-01' = { - name: '${safeExportContainerName}_ETL_${safeIngestionContainerName}' - parent: dataFactory - properties: { - activities: [ - { // Get Existing Parquet Files - name: 'Get Existing Parquet Files' - description: 'Get the previously ingested files so we can remove any older data. This is necessary to avoid data duplication in reports.' - type: 'GetMetadata' - dependsOn: [] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - dataset: { - referenceName: dataset_ingestion_files.name - type: 'DatasetReference' - parameters: { - folderPath: '@pipeline().parameters.destinationFolder' - } - } - fieldList: [ - 'childItems' - ] - storeSettings: { - type: 'AzureBlobFSReadSettings' - enablePartitionDiscovery: false - } - formatSettings: { - type: 'ParquetReadSettings' - } - } - } - { // Filter Out Current Exports - name: 'Filter Out Current Exports' - description: 'Remove existing files from the current export so those files do not get deleted.' - type: 'Filter' - dependsOn: [ - { - activity: 'Get Existing Parquet Files' - dependencyConditions: [ - 'Completed' - ] - } - ] - userProperties: [] - typeProperties: { - items: { - value: '@if(contains(activity(\'Get Existing Parquet Files\').output, \'childItems\'), activity(\'Get Existing Parquet Files\').output.childItems, json(\'[]\'))' - type: 'Expression' - } - condition: { - // cSpell:ignore endswith - value: '@and(endswith(item().name, \'.parquet\'), not(startswith(item().name, concat(pipeline().parameters.ingestionId, \'${ingestionIdFileNameSeparator}\'))))' - type: 'Expression' - } - } - } - { // Load Schema Mappings - name: 'Load Schema Mappings' - description: 'Get schema mapping file to use for the CSV to parquet conversion.' - type: 'Lookup' - dependsOn: [] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'JsonSource' - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: true - enablePartitionDiscovery: false - } - formatSettings: { - type: 'JsonReadSettings' - } - } - dataset: { - referenceName: dataset_config.name - type: 'DatasetReference' - parameters: { - fileName: { - value: '@toLower(pipeline().parameters.schemaFile)' - type: 'Expression' - } - folderPath: '${configContainerName}/schemas' - } - } - } - } - { // Error: SchemaLoadFailed - name: 'Failed to Load Schema' - type: 'Fail' - dependsOn: [ - { - activity: 'Load Schema Mappings' - dependencyConditions: [ - 'Failed' - ] - } - ] - userProperties: [] - typeProperties: { - message: { - value: '@concat(\'Unable to load the \', pipeline().parameters.schemaFile, \' schema file. Please confirm the schema and version are supported for FinOps hubs ingestion. Unsupported files will remain in the msexports container.\')' - type: 'Expression' - } - errorCode: 'SchemaLoadFailed' - } - } - { // Set Additional Columns - name: 'Set Additional Columns' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Load Schema Mappings' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'additionalColumns' - value: { - value: '@intersection(array(json(concat(\'[{"name":"x_SourceProvider","value":"Microsoft"},{"name":"x_SourceName","value":"Cost Management"},{"name":"x_SourceType","value":"\', pipeline().parameters.exportDatasetVersion, \'"},{"name":"x_SourceVersion","value":"\', pipeline().parameters.exportDatasetVersion, \'"}\'))), activity(\'Load Schema Mappings\').output.firstRow.additionalColumns)' - type: 'Expression' - } - } - } - { // For Each Old File - name: 'For Each Old File' - description: 'Loop thru each of the existing files from previous exports.' - type: 'ForEach' - dependsOn: [ - { - activity: 'Convert to Parquet' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Filter Out Current Exports' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - items: { - value: '@activity(\'Filter Out Current Exports\').output.Value' - type: 'Expression' - } - activities: [ - { // Delete Old Ingested File - name: 'Delete Old Ingested File' - description: 'Delete the previously ingested files from older exports.' - type: 'Delete' - dependsOn: [] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - dataset: { - referenceName: dataset_ingestion.name - type: 'DatasetReference' - parameters: { - blobPath: { - value: '@concat(pipeline().parameters.destinationFolder, \'/\', item().name)' - type: 'Expression' - } - } - } - enableLogging: false - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: false - enablePartitionDiscovery: false - } - } - } - ] - } - } - { // Set Destination Path - name: 'Set Destination Path' - type: 'SetVariable' - dependsOn: [] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'destinationPath' - value: { - value: '@concat(pipeline().parameters.destinationFolder, \'/\', pipeline().parameters.ingestionId, \'${ingestionIdFileNameSeparator}\', pipeline().parameters.destinationFile)' - type: 'Expression' - } - } - } - { // Convert to Parquet - name: 'Convert to Parquet' - description: 'Convert CSV to parquet and move the file to the ${ingestionContainerName} container.' - type: 'Switch' - dependsOn: [ - { - activity: 'Set Destination Path' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Load Schema Mappings' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Set Additional Columns' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - on: { - value: '@last(array(split(pipeline().parameters.blobPath, \'.\')))' - type: 'Expression' - } - cases: [ - { // CSV - value: 'csv' - activities: [ - { // Convert CSV File - name: 'Convert CSV File' - type: 'Copy' - dependsOn: [] - policy: { - timeout: '0.00:10:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'DelimitedTextSource' - additionalColumns: { - value: '@variables(\'additionalColumns\')' - type: 'Expression' - } - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: true - enablePartitionDiscovery: false - } - formatSettings: { - type: 'DelimitedTextReadSettings' - } - } - sink: { - type: 'ParquetSink' - storeSettings: { - type: 'AzureBlobFSWriteSettings' - } - formatSettings: { - type: 'ParquetWriteSettings' - fileExtension: '.parquet' - } - } - enableStaging: false - parallelCopies: 1 - validateDataConsistency: false - translator: { - value: '@activity(\'Load Schema Mappings\').output.firstRow.translator' - type: 'Expression' - } - } - inputs: [ - { - referenceName: dataset_msexports.name - type: 'DatasetReference' - parameters: { - blobPath: { - value: '@pipeline().parameters.blobPath' - type: 'Expression' - } - } - } - ] - outputs: [ - { - referenceName: dataset_ingestion.name - type: 'DatasetReference' - parameters: { - blobPath: { - value: '@variables(\'destinationPath\')' - type: 'Expression' - } - } - } - ] - } - ] - } - { // GZ - value: 'gz' - activities: [ - { // Convert GZip CSV File - name: 'Convert GZip CSV File' - type: 'Copy' - dependsOn: [] - policy: { - timeout: '0.00:10:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'DelimitedTextSource' - additionalColumns: { - value: '@variables(\'additionalColumns\')' - type: 'Expression' - } - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: true - enablePartitionDiscovery: false - } - formatSettings: { - type: 'DelimitedTextReadSettings' - } - } - sink: { - type: 'ParquetSink' - storeSettings: { - type: 'AzureBlobFSWriteSettings' - } - formatSettings: { - type: 'ParquetWriteSettings' - fileExtension: '.parquet' - } - } - enableStaging: false - parallelCopies: 1 - validateDataConsistency: false - translator: { - value: '@activity(\'Load Schema Mappings\').output.firstRow.translator' - type: 'Expression' - } - } - inputs: [ - { - referenceName: dataset_msexports_gzip.name - type: 'DatasetReference' - parameters: { - blobPath: { - value: '@pipeline().parameters.blobPath' - type: 'Expression' - } - } - } - ] - outputs: [ - { - referenceName: dataset_ingestion.name - type: 'DatasetReference' - parameters: { - blobPath: { - value: '@variables(\'destinationPath\')' - type: 'Expression' - } - } - } - ] - } - ] - } - { // Parquet - value: 'parquet' - activities: [ - { // Move Parquet File - name: 'Move Parquet File' - type: 'Copy' - dependsOn: [] - policy: { - timeout: '0.00:05:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'ParquetSource' - additionalColumns: { - value: '@variables(\'additionalColumns\')' - type: 'Expression' - } - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: true - enablePartitionDiscovery: false - } - formatSettings: { - type: 'ParquetReadSettings' - } - } - sink: { - type: 'ParquetSink' - storeSettings: { - type: 'AzureBlobFSWriteSettings' - } - formatSettings: { - type: 'ParquetWriteSettings' - fileExtension: '.parquet' - } - } - enableStaging: false - parallelCopies: 1 - validateDataConsistency: false - } - inputs: [ - { - referenceName: dataset_msexports_parquet.name - type: 'DatasetReference' - parameters: { - blobPath: { - value: '@pipeline().parameters.blobPath' - type: 'Expression' - } - } - } - ] - outputs: [ - { - referenceName: dataset_ingestion.name - type: 'DatasetReference' - parameters: { - blobPath: { - value: '@variables(\'destinationPath\')' - type: 'Expression' - } - } - } - ] - } - ] - } - ] - defaultActivities: [ - { // Error: UnsupportedFileType - name: 'Unsupported File Type' - type: 'Fail' - dependsOn: [] - userProperties: [] - typeProperties: { - message: { - value: '@concat(\'Unable to ingest the specified export file because the file type is not supported. File: \', pipeline().parameters.blobPath)' - type: 'Expression' - } - errorCode: 'UnsupportedExportFileType' - } - } - ] - } - } - { // Read Hub Config - name: 'Read Hub Config' - description: 'Read the hub config to determine if the export should be retained.' - type: 'Lookup' - dependsOn: [] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'JsonSource' - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: false - enablePartitionDiscovery: false - } - formatSettings: { - type: 'JsonReadSettings' - } - } - dataset: { - referenceName: dataset_config.name - type: 'DatasetReference' - parameters: { - fileName: 'settings.json' - folderPath: configContainerName - } - } - } - } - { // If Not Retaining Exports - name: 'If Not Retaining Exports' - description: 'If the msexports retention period <= 0, delete the source file. The main reason to keep the source file is to allow for troubleshooting and reprocessing in the future.' - type: 'IfCondition' - dependsOn: [ - { - activity: 'Convert to Parquet' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Read Hub Config' - dependencyConditions: [ - 'Completed' - ] - } - ] - userProperties: [] - typeProperties: { - expression: { - value: '@lessOrEquals(coalesce(activity(\'Read Hub Config\').output.firstRow.retention.msexports.days, 0), 0)' - type: 'Expression' - } - ifTrueActivities: [ - { // Delete Source File - name: 'Delete Source File' - description: 'Delete the exported data file to keep storage costs down. This file is not referenced by any reporting systems.' - type: 'Delete' - dependsOn: [] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - dataset: { - referenceName: dataset_msexports_parquet.name - type: 'DatasetReference' - parameters: { - blobPath: { - value: '@pipeline().parameters.blobPath' - type: 'Expression' - } - } - } - enableLogging: false - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: true - enablePartitionDiscovery: false - } - } - } - ] - } - } - ] - parameters: { - blobPath: { - type: 'String' - } - destinationFile: { - type: 'string' - } - destinationFolder: { - type: 'string' - } - ingestionId: { - type: 'string' - } - schemaFile: { - type: 'string' - } - exportDatasetType: { - type: 'string' - } - exportDatasetVersion: { - type: 'string' - } - } - variables: { - additionalColumns: { - type: 'Array' - } - destinationPath: { - type: 'String' - } - } - annotations: [] - } -} - -//------------------------------------------------------------------------------ -// ingestion_ETL_dataExplorer pipeline -// Triggered by ingestion_ExecuteETL -//------------------------------------------------------------------------------ -@description('Ingests parquet data into an Azure Data Explorer cluster.') -resource pipeline_ToDataExplorer 'Microsoft.DataFactory/factories/pipelines@2018-06-01' = if (deployDataExplorer || useFabric) { - name: '${safeIngestionContainerName}_ETL_dataExplorer' - parent: dataFactory - properties: { - activities: [ - { // Read Hub Config - name: 'Read Hub Config' - description: 'Read the hub config to determine how long data should be retained.' - type: 'Lookup' - dependsOn: [ - ] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - source: { - type: 'JsonSource' - storeSettings: { - type: 'AzureBlobFSReadSettings' - recursive: false - enablePartitionDiscovery: false - } - formatSettings: { - type: 'JsonReadSettings' - } - } - dataset: { - referenceName: dataset_config.name - type: 'DatasetReference' - parameters: { - fileName: 'settings.json' - folderPath: configContainerName - } - } - } - } - { // Set Final Retention Months - name: 'Set Final Retention Months' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Read Hub Config' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'finalRetentionMonths' - value: { - value: '@coalesce(activity(\'Read Hub Config\').output.firstRow.retention.final.months, 999)' - type: 'Expression' - } - } - } - { // Until Capacity Is Available - name: 'Until Capacity Is Available' - type: 'Until' - dependsOn: [ - { - activity: 'Set Final Retention Months' - dependencyConditions: [ - 'Completed' - 'Skipped' - ] - } - ] - userProperties: [] - typeProperties: { - expression: { - value: '@equals(variables(\'tryAgain\'), false)' - type: 'Expression' - } - activities: [ - { // Confirm Ingestion Capacity - name: 'Confirm Ingestion Capacity' - type: 'AzureDataExplorerCommand' - dependsOn: [] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - command: '.show capacity | where Resource == \'Ingestions\' | project Remaining' - commandTimeout: '00:20:00' - } - linkedServiceName: { - referenceName: linkedService_dataExplorer.name - type: 'LinkedServiceReference' - } - } - { // If Has Capacity - name: 'If Has Capacity' - type: 'IfCondition' - dependsOn: [ - { - activity: 'Confirm Ingestion Capacity' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - expression: { - value: '@or(equals(activity(\'Confirm Ingestion Capacity\').output.count, 0), greater(activity(\'Confirm Ingestion Capacity\').output.value[0].Remaining, 0))' - type: 'Expression' - } - ifFalseActivities: [ - { // Wait for Ingestion - name: 'Wait for Ingestion' - type: 'Wait' - dependsOn: [] - userProperties: [] - typeProperties: { - waitTimeInSeconds: 15 - } - } - { // Try Again - name: 'Try Again' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Wait for Ingestion' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'tryAgain' - value: true - } - } - ] - ifTrueActivities: [ - { // Pre-Ingest Cleanup - name: 'Pre-Ingest Cleanup' - description: 'Cost Management exports include all month-to-date data from the previous export run. To ensure data is not double-reported, it must be dropped from the raw table before ingestion completes. Remove previous ingestions into the raw table for the month and any previous runs of the current ingestion month file in any table.' - type: 'AzureDataExplorerCommand' - dependsOn: [] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - typeProperties: { - command: { - value: '@concat(\'.drop extents <| .show extents | where (TableName == "\', pipeline().parameters.table, \'" and Tags !has "drop-by:\', pipeline().parameters.ingestionId, \'" and Tags has "drop-by:\', pipeline().parameters.folderPath, \'") or (Tags has "drop-by:\', pipeline().parameters.ingestionId, \'" and Tags has "drop-by:\', pipeline().parameters.folderPath, \'/\', pipeline().parameters.originalFileName, \'")\')' - type: 'Expression' - } - commandTimeout: '00:20:00' - } - linkedServiceName: { - referenceName: linkedService_dataExplorer.name - type: 'LinkedServiceReference' - parameters: { - database: dataExplorerIngestionDatabase - } - } - } - { // Ingest Data - name: 'Ingest Data' - type: 'AzureDataExplorerCommand' - dependsOn: [ - { - activity: 'Pre-Ingest Cleanup' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - timeout: '0.12:00:00' - retry: 3 - retryIntervalInSeconds: 120 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - command: { - // cSpell:ignore abfss, toscalar - value: '@concat(\'.ingest into table \', pipeline().parameters.table, \' ("abfss://${ingestionContainerName}@${storageAccount.name}.dfs.${environment().suffixes.storage}/\', pipeline().parameters.folderPath, \'/\', pipeline().parameters.fileName, \';${useFabric ? 'impersonate' : 'managed_identity=system'}") with (format="parquet", ingestionMappingReference="\', pipeline().parameters.table, \'_mapping", tags="[\\"drop-by:\', pipeline().parameters.ingestionId, \'\\", \\"drop-by:\', pipeline().parameters.folderPath, \'/\', pipeline().parameters.originalFileName, \'\\", \\"drop-by:ftk-version-${ftkVersion}\\"]"); print Success = assert(iff(toscalar($command_results | project-keep HasErrors) == false, true, false), "Ingestion Failed")\')' - type: 'Expression' - } - commandTimeout: '01:00:00' - } - linkedServiceName: { - referenceName: linkedService_dataExplorer.name - type: 'LinkedServiceReference' - parameters: { - database: dataExplorerIngestionDatabase - } - } - } - { // Post-Ingest Cleanup - name: 'Post-Ingest Cleanup' - description: 'Cost Management exports include all month-to-date data from the previous export run. To ensure data is not double-reported, it must be dropped after ingestion completes. Remove the current ingestion month file from raw and any old ingestions for the month from the final table.' - type: 'AzureDataExplorerCommand' - dependsOn: [ - { - activity: 'Ingest Data' - dependencyConditions: [ - 'Completed' - ] - } - ] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - typeProperties: { - command: { - // cSpell:ignore startofmonth, strcat, todatetime - value: '@concat(\'.drop extents <| .show extents | extend isOldFinalData = (TableName startswith "\', replace(pipeline().parameters.table, \'_raw\', \'_final_v\'), \'" and Tags !has "drop-by:\', pipeline().parameters.ingestionId, \'" and Tags has "drop-by:\', pipeline().parameters.folderPath, \'") | extend isPastFinalRetention = (TableName startswith "\', replace(pipeline().parameters.table, \'_raw\', \'_final_v\'), \'" and todatetime(substring(strcat(replace_string(extract("drop-by:[A-Za-z]+/(\\\\d{4}/\\\\d{2}(/\\\\d{2})?)", 1, Tags), "/", "-"), "-01"), 0, 10)) < datetime_add("month", -\', if(lessOrEquals(variables(\'finalRetentionMonths\'), 0), 0, variables(\'finalRetentionMonths\')), \', startofmonth(now()))) | where isOldFinalData or isPastFinalRetention\')' - type: 'Expression' - } - commandTimeout: '00:20:00' - } - linkedServiceName: { - referenceName: linkedService_dataExplorer.name - type: 'LinkedServiceReference' - parameters: { - database: dataExplorerIngestionDatabase - } - } - } - { // Ingestion Complete - name: 'Ingestion Complete' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Post-Ingest Cleanup' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'tryAgain' - value: false - } - } - { // Abort On Ingestion Error - name: 'Abort On Ingestion Error' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Ingest Data' - dependencyConditions: [ - 'Failed' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'tryAgain' - value: false - } - } - { // Error: DataExplorerIngestionFailed - name: 'Ingestion Failed Error' - type: 'Fail' - dependsOn: [ - { - activity: 'Abort On Ingestion Error' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - message: { - value: '@concat(\'Data Explorer ingestion into the \', pipeline().parameters.table, \' table failed. Please fix the error and rerun ingestion for the following folder path: "\', pipeline().parameters.folderPath, \'". File: \', pipeline().parameters.originalFileName, \'. Error: \', if(greater(length(activity(\'Ingest Data\').output.errors), 0), activity(\'Ingest Data\').output.errors[0].Message, \'Unknown\'), \' (Code: \', if(greater(length(activity(\'Ingest Data\').output.errors), 0), activity(\'Ingest Data\').output.errors[0].Code, \'None\'), \')\')' - type: 'Expression' - } - errorCode: 'DataExplorerIngestionFailed' - } - } - { // Abort On Pre-Ingest Drop Error - name: 'Abort On Pre-Ingest Drop Error' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Pre-Ingest Cleanup' - dependencyConditions: [ - 'Failed' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'tryAgain' - value: false - } - } - { // Error: DataExplorerPreIngestionDropFailed - name: 'Pre-Ingest Drop Failed Error' - type: 'Fail' - dependsOn: [ - { - activity: 'Abort On Pre-Ingest Drop Error' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - message: { - value: '@concat(\'Data Explorer pre-ingestion cleanup (drop extents from raw table) for the \', pipeline().parameters.table, \' table failed. Ingestion was not completed. Please fix the error and rerun ingestion for the following folder path: "\', pipeline().parameters.folderPath, \'". File: \', pipeline().parameters.originalFileName, \'. Error: \', if(greater(length(activity(\'Pre-Ingest Cleanup\').output.errors), 0), activity(\'Pre-Ingest Cleanup\').output.errors[0].Message, \'Unknown\'), \' (Code: \', if(greater(length(activity(\'Pre-Ingest Cleanup\').output.errors), 0), activity(\'Pre-Ingest Cleanup\').output.errors[0].Code, \'None\'), \')\')' - type: 'Expression' - } - errorCode: 'DataExplorerPreIngestionDropFailed' - } - } - { // Abort On Post-Ingest Drop Error - name: 'Abort On Post-Ingest Drop Error' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Post-Ingest Cleanup' - dependencyConditions: [ - 'Failed' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'tryAgain' - value: false - } - } - { // Error: DataExplorerPostIngestionDropFailed - name: 'Post-Ingest Drop Failed Error' - type: 'Fail' - dependsOn: [ - { - activity: 'Abort On Post-Ingest Drop Error' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - message: { - value: '@concat(\'Data Explorer post-ingestion cleanup (drop extents from final tables) for the \', replace(pipeline().parameters.table, \'_raw\', \'_final_*\'), \' table failed. Please fix the error and rerun ingestion for the following folder path: "\', pipeline().parameters.folderPath, \'". File: \', pipeline().parameters.originalFileName, \'. Error: \', if(greater(length(activity(\'Post-Ingest Cleanup\').output.errors), 0), activity(\'Post-Ingest Cleanup\').output.errors[0].Message, \'Unknown\'), \' (Code: \', if(greater(length(activity(\'Post-Ingest Cleanup\').output.errors), 0), activity(\'Post-Ingest Cleanup\').output.errors[0].Code, \'None\'), \')\')' - type: 'Expression' - } - errorCode: 'DataExplorerPostIngestionDropFailed' - } - } - ] - } - } - ] - timeout: '0.02:00:00' - } - } - ] - parameters: { - folderPath: { - type: 'string' - } - fileName: { - type: 'string' - } - originalFileName: { - type: 'string' - } - ingestionId: { - type: 'string' - } - table: { - type: 'string' - } - } - variables: { - tryAgain: { - type: 'Boolean' - defaultValue: true - } - logRetentionDays: { - type: 'Integer' - defaultValue: 0 - } - finalRetentionMonths: { - type: 'Integer' - defaultValue: 999 - } - } - annotations: [] - } -} - -//------------------------------------------------------------------------------ -// ingestion_ExecuteETL pipeline -// Triggered by ingestion_ManifestAdded trigger -//------------------------------------------------------------------------------ -@description('Queues the ingestion_ETL_dataExplorer pipeline to account for Data Factory pipeline trigger limits.') -resource pipeline_ExecuteIngestionETL 'Microsoft.DataFactory/factories/pipelines@2018-06-01' = if (deployDataExplorer || useFabric) { - name: '${safeIngestionContainerName}_ExecuteETL' - parent: dataFactory - properties: { - concurrency: 1 - activities: [ - { // Wait - name: 'Wait' - description: 'Files may not be available immediately after being created.' - type: 'Wait' - dependsOn: [] - userProperties: [] - typeProperties: { - waitTimeInSeconds: 60 - } - } - { // Set Container Folder Path - name: 'Set Container Folder Path' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Wait' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'containerFolderPath' - value: { - value: '@join(skip(array(split(pipeline().parameters.folderPath, \'/\')), 1), \'/\')' - type: 'Expression' - } - } - } - { // Get Existing Parquet Files - name: 'Get Existing Parquet Files' - description: 'Get the previously ingested files so we can get file paths.' - type: 'GetMetadata' - dependsOn: [ - { - activity: 'Set Container Folder Path' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - dataset: { - referenceName: dataset_ingestion_files.name - type: 'DatasetReference' - parameters: { - folderPath: '@variables(\'containerFolderPath\')' - } - } - fieldList: [ - 'childItems' - ] - storeSettings: { - type: 'AzureBlobFSReadSettings' - enablePartitionDiscovery: false - } - formatSettings: { - type: 'ParquetReadSettings' - } - } - } - { // Filter Out Folders and manifest files - name: 'Filter Out Folders' - description: 'Remove any folders or manifest files.' - type: 'Filter' - dependsOn: [ - { - activity: 'Get Existing Parquet Files' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - items: { - value: '@if(contains(activity(\'Get Existing Parquet Files\').output, \'childItems\'), activity(\'Get Existing Parquet Files\').output.childItems, json(\'[]\'))' - type: 'Expression' - } - condition: { - value: '@and(equals(item().type, \'File\'), not(contains(toLower(item().name), \'manifest.json\')))' - type: 'Expression' - } - } - } - { // Set Ingestion Timestamp - name: 'Set Ingestion Timestamp' - type: 'SetVariable' - dependsOn: [ - { - activity: 'Wait' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - variableName: 'timestamp' - value: { - value: '@utcNow()' - type: 'Expression' - } - } - } - { // For Each Old File - name: 'For Each Old File' - description: 'Loop thru each of the existing files.' - type: 'ForEach' - dependsOn: [ - { - activity: 'Filter Out Folders' - dependencyConditions: [ - 'Succeeded' - ] - } - { - activity: 'Data Explorer validation' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - batchCount: dataExplorerIngestionCapacity // Concurrency limit - items: { - value: '@activity(\'Filter Out Folders\').output.Value' - type: 'Expression' - } - activities: [ - { // Execute - name: 'Execute' - description: 'Run the ADX ETL pipeline.' - type: 'ExecutePipeline' - dependsOn: [] - policy: { - secureInput: false - } - userProperties: [] - typeProperties: { - pipeline: { - referenceName: pipeline_ToDataExplorer.name - type: 'PipelineReference' - } - waitOnCompletion: true - parameters: { - folderPath: { - value: '@variables(\'containerFolderPath\')' - type: 'Expression' - } - fileName: { - value: '@item().name' - type: 'Expression' - } - originalFileName: { - value: '@last(array(split(item().name, \'${ingestionIdFileNameSeparator}\')))' - type: 'Expression' - } - ingestionId: { - value: '@concat(first(array(split(item().name, \'${ingestionIdFileNameSeparator}\'))), \'_\', variables(\'timestamp\'))' - type: 'Expression' - } - table: { - value: '@concat(first(array(split(variables(\'containerFolderPath\'), \'/\'))), \'_raw\')' - type: 'Expression' - } - } - } - } - ] - } - } - { // If No Files - name: 'If No Files' - description: 'If there are no files found, fail the pipeline.' - type: 'IfCondition' - dependsOn: [ - { - activity: 'Filter Out Folders' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - expression: { - value: '@equals(length(activity(\'Filter Out Folders\').output.Value), 0)' - type: 'Expression' - } - ifTrueActivities: [ - { // Error: IngestionFilesNotFound - name: 'Files Not Found' - type: 'Fail' - dependsOn: [] - userProperties: [] - typeProperties: { - message: { - value: '@concat(\'Unable to locate parquet files to ingest from the \', pipeline().parameters.folderPath, \' path. Please confirm the folder path is the full path, including the "ingestion" container and not starting with or ending with a slash ("/").\')' - type: 'Expression' - } - errorCode: 'IngestionFilesNotFound' - } - } - ] - } - } - { - name: 'Data Explorer validation' - description: 'If Data Explorer is stopped, start it' - type: 'IfCondition' - dependsOn: [ - { - activity: 'Set Ingestion Timestamp' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - userProperties: [] - typeProperties: { - expression: { - value: '@equals(${deployDataExplorer}, true)' - type: 'Expression' - } - ifTrueActivities: [ - { - name: 'Start ADX Cluster' - type: 'WebActivity' - dependsOn: [] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - method: 'POST' - url: { - value: '${environment().resourceManager}${dataExplorerCluster.id}/start?api-version=2024-04-13' - type: 'Expression' - } - body: '{}' - authentication: { - type: 'MSI' - resource: { - value: environment().resourceManager - type: 'Expression' - } - } - } - } - { - name: 'Error ADX Start' - type: 'Fail' - dependsOn: [ - { - activity: 'Start ADX Cluster After Error' - dependencyConditions: [ - 'Failed' - ] - } - ] - userProperties: [] - typeProperties: { - message: { - value:'@concat(\'Failed to start the Data Explorer instance. Message: \', activity(\'Start ADX Cluster After Error\').output.error.message)' - type: 'Expression' - } - errorCode: { - value: '@activity(\'Start ADX Cluster After Error\').output.error.code' - type: 'Expression' - } - } - } - { - name: 'Wait ADX Provision State' - type: 'Wait' - dependsOn: [ - { - activity: 'Start ADX Cluster' - dependencyConditions: [ - 'Failed' - ] - } - ] - userProperties: [] - typeProperties: { - waitTimeInSeconds: 600 - } - } - { - name: 'Start ADX Cluster After Error' - type: 'WebActivity' - dependsOn: [ - { - activity: 'Wait ADX Provision State' - dependencyConditions: [ - 'Succeeded' - ] - } - ] - policy: { - timeout: '0.12:00:00' - retry: 0 - retryIntervalInSeconds: 30 - secureOutput: false - secureInput: false - } - userProperties: [] - typeProperties: { - method: 'POST' - url: { - value: '${environment().resourceManager}${dataExplorerCluster.id}/start?api-version=2024-04-13' - type: 'Expression' - body: '{}' - } - authentication: { - type: 'MSI' - resource: { - value: environment().resourceManager - type: 'Expression' - } - } - } - } - ] - } - } - ] - parameters: { - folderPath: { - type: 'string' - } - } - variables: { - containerFolderPath: { - type: 'string' - } - timestamp: { - type: 'string' - } - } - annotations: [ - 'New ingestion' - ] - } -} - -//------------------------------------------------------------------------------ -// Start all triggers -//------------------------------------------------------------------------------ - -module startTriggers 'hub-deploymentScript.bicep' = { - name: 'Microsoft.FinOpsHubs.Core_ADF.StartTriggers' - dependsOn: [ - triggerManagerRoleAssignments - trigger_ExportManifestAdded - trigger_IngestionManifestAdded - trigger_SettingsUpdated - trigger_DailySchedule - trigger_MonthlySchedule - deleteOldResources - ] - params: { - app: app - identityName: triggerManagerIdentity.name - scriptContent: loadTextContent('./scripts/Start-Triggers.ps1') - environmentVariables: [ - { - name: 'DataFactorySubscriptionId' - value: subscription().id - } - { - name: 'DataFactoryResourceGroup' - value: resourceGroup().name - } - { - name: 'DataFactoryName' - value: dataFactory.name - } - { - name: 'Triggers' - value: join(allHubTriggers, '|') - } - { - name: 'Pipelines' - value: join([ pipeline_InitializeHub.name ], '|') - } - ] - } -} - -//============================================================================== -// Outputs -//============================================================================== - -@description('The Resource ID of the Data factory.') -output resourceId string = dataFactory.id - -@description('The Name of the Azure Data Factory instance.') -output name string = dataFactory.name diff --git a/src/templates/finops-hub/modules/ftkver.txt b/src/templates/finops-hub/modules/fx/ftkver.txt similarity index 100% rename from src/templates/finops-hub/modules/ftkver.txt rename to src/templates/finops-hub/modules/fx/ftkver.txt diff --git a/src/templates/finops-hub/modules/fx/hub-app.bicep b/src/templates/finops-hub/modules/fx/hub-app.bicep new file mode 100644 index 000000000..4eefc1a5e --- /dev/null +++ b/src/templates/finops-hub/modules/fx/hub-app.bicep @@ -0,0 +1,576 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { getAppPublisherTags, HubAppProperties, HubAppFeature } from 'hub-types.bicep' + + +//============================================================================== +// Parameters +//============================================================================== + +@description('Required. FinOps hub app getting deployed.') +param app HubAppProperties + +@description('Required. Version number of the FinOps hub app.') +param version string + +// @description('Required. Minimum version number supported by the FinOps hub app.') +// param hubMinVersion string + +// @description('Required. Maximum version number supported by the FinOps hub app.') +// param hubMaxVersion string + +@description('Optional. Indicate which features the app requires. Allowed values: "DataFactory", "KeyVault", "Storage". Default: [] (none).') +param features HubAppFeature[] = [] + +@description('Optional. Indicate which RBAC roles the Data Factory identity needs on the storage account, if created. This is in addition to Storage Blob Data Contributor for reading and managing content. Default: [] (none).') +param storageRoles string[] = [] + +@description('Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed.') +param telemetryString string = '' + + +//============================================================================== +// Variables +//============================================================================== + +// Features +var usesDataFactory = contains(features, 'DataFactory') +var usesKeyVault = contains(features, 'KeyVault') +var usesStorage = contains(features, 'Storage') + +// App telemetry +var telemetryId = 'ftk-hubapp-${app.id}${empty(telemetryString) ? '' : '_'}${telemetryString}' // cSpell:ignore hubapp +var telemetryProps = { + mode: 'Incremental' + template: { + '$schema': 'https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#' + contentVersion: '1.0.0.0' + metadata: { + _generator: { + name: 'FTK: ${app.id}' + version: version + } + } + resources: [] + } +} + +// Roles needed to auto-start Data Factory triggers +var autoStartRbacRoles = [ + // Data Factory contributor -- https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#data-factory-contributor + // Used to start/stop triggers and delete old pipelines/triggers + '673868aa-7521-48a0-acc6-0f60742d39f5' +] + +// Roles for ADF to manage data in storage +// Does not include roles assignments needed against the export scope +var factoryStorageRoles = union(storageRoles, [ + // Storage Account Contributor -- https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#storage-account-contributor + // Used to move files from the msexports to ingestion container + '17d1049b-9a84-46fb-8f53-869881c3d3ab' + // Storage Blob Data Contributor -- https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#storage-blob-data-contributor + 'ba92f5b4-2d11-453d-a403-e96b0029c9fe' + // Reader -- https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#reader + 'acdd72a7-3385-48ef-bd42-f606fba81ae7' +]) + +// Storage infrastructure encryption +var storageInfrastructureEncryptionProperties = !app.hub.options.storageInfrastructureEncryption ? {} : { + encryption: { + keySource: 'Microsoft.Storage' + requireInfrastructureEncryption: app.hub.options.storageInfrastructureEncryption + } +} + +// KeyVault access policies +var keyVaultAccessPolicies = [ + { + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access + objectId: dataFactory.identity.principalId + tenantId: subscription().tenantId + permissions: { secrets: ['get'] } + } +] + + +//============================================================================== +// Resources +//============================================================================== + +// TODO: Get hub instance to verify version compatibility + +//------------------------------------------------------------------------------ +// Telemetry +// Used to anonymously count the number of times the template has been deployed +// and to track and fix deployment bugs to ensure the highest quality. +// No information about you or your cost data is collected. +//------------------------------------------------------------------------------ + +resource appTelemetry 'Microsoft.Resources/deployments@2022-09-01' = if (app.hub.options.enableTelemetry) { + name: length(telemetryId) <= 64 ? telemetryId : substring(telemetryId, 0, 64) + tags: getAppPublisherTags(app, 'Microsoft.Resources/deployments') + properties: telemetryProps +} + +//------------------------------------------------------------------------------ +// Data Factory +//------------------------------------------------------------------------------ + +resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' = if (usesDataFactory) { + name: app.dataFactory + location: app.hub.location + tags: getAppPublisherTags(app, 'Microsoft.DataFactory/factories') + identity: { type: 'SystemAssigned' } + properties: any({ // Using any() to hide the error that gets surfaced because globalConfigurations is not in the ADF schema yet + globalConfigurations: { + PipelineBillingEnabled: 'true' + } + }) + + resource managedVirtualNetwork 'managedVirtualNetworks' = if (app.hub.options.privateRouting) { + name: 'default' + properties: {} + + resource storageManagedPrivateEndpoint 'managedPrivateEndpoints' = if (usesStorage) { + name: storageAccount.name + properties: { + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access + name: storageAccount.name + groupId: 'dfs' + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access + privateLinkResourceId: storageAccount.id + fqdns: [ + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access + storageAccount.properties.primaryEndpoints.dfs + ] + } + } + + resource keyVaultManagedPrivateEndpoint 'managedPrivateEndpoints' = if (usesKeyVault) { + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access + name: keyVault.name + properties: { + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access + name: keyVault.name + groupId: 'vault' + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access + privateLinkResourceId: keyVault.id + fqdns: [ + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access + keyVault.properties.vaultUri + ] + } + } + } + + resource managedIntegrationRuntime 'integrationRuntimes' = if (app.hub.options.privateRouting) { + name: 'ManagedIntegrationRuntime' + + properties: { + type: 'Managed' + managedVirtualNetwork: { + referenceName: dataFactory::managedVirtualNetwork.name + type: 'ManagedVirtualNetworkReference' + } + typeProperties: { + computeProperties: { + location: app.hub.location + dataFlowProperties: { + computeType: 'General' + coreCount: 8 + timeToLive: 10 + cleanup: false + customProperties: [] + } + copyComputeScaleProperties: { + dataIntegrationUnit: 16 + timeToLive: 30 + } + pipelineExternalComputeScaleProperties: { + timeToLive: 30 + numberOfPipelineNodes: 1 + numberOfExternalNodes: 1 + } + } + } + } + } + + // cSpell:ignore linkedservices + resource linkedService_keyVault 'linkedservices' = if (usesKeyVault) { + name: keyVault.name + dependsOn: app.hub.options.privateRouting ? [managedIntegrationRuntime] : [] + properties: { + annotations: [] + parameters: {} + type: 'AzureKeyVault' + typeProperties: { + baseUrl: reference('Microsoft.KeyVault/vaults/${keyVault.name}', '2023-02-01').vaultUri + } + connectVia: app.hub.options.privateRouting + ? { + referenceName: managedIntegrationRuntime.name + type: 'IntegrationRuntimeReference' + } + : null + } + } + + resource linkedService_storageAccount 'linkedservices' = if (usesStorage) { + name: storageAccount.name + dependsOn: app.hub.options.privateRouting ? [managedIntegrationRuntime] : [] + properties: { + annotations: [] + parameters: {} + type: 'AzureBlobFS' + typeProperties: { + url: reference('Microsoft.Storage/storageAccounts/${storageAccount.name}', '2021-08-01').primaryEndpoints.dfs + } + connectVia: app.hub.options.privateRouting + ? { + referenceName: managedIntegrationRuntime.name + type: 'IntegrationRuntimeReference' + } + : null + } + } +} + +// TODO: Consolidate keyVaultEndpoints.bicep into hub-app.bicep +module getKeyVaultPrivateEndpointConnections 'keyVaultEndpoints.bicep' = if (usesDataFactory && usesKeyVault && app.hub.options.privateRouting) { + name: 'GetKeyVaultPrivateEndpointConnections' + dependsOn: [ + dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint + getStoragePrivateEndpointConnections // Queue Key Vault private endpoints after storage since we can only run one deployment at a time with private endpoints + ] + params: { + keyVaultName: keyVault.name + } +} + +module approveKeyVaultPrivateEndpointConnections 'keyVaultEndpoints.bicep' = if (usesDataFactory && usesKeyVault && app.hub.options.privateRouting) { + name: 'ApproveKeyVaultPrivateEndpointConnections' + params: { + keyVaultName: keyVault.name + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access + privateEndpointConnections: getKeyVaultPrivateEndpointConnections.outputs.privateEndpointConnections + } +} + +// TODO: Consolidate storageEndpoints.bicep into hub-app.bicep +module getStoragePrivateEndpointConnections 'storageEndpoints.bicep' = if (usesDataFactory && usesStorage && app.hub.options.privateRouting) { + name: 'GetStoragePrivateEndpointConnections' + dependsOn: [ + dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint + stopTriggers // Queue storage private endpoints after triggers are stopped since we can only run one deployment at a time with private endpoints + ] + params: { + storageAccountName: storageAccount.name + } +} + +module approveStoragePrivateEndpointConnections 'storageEndpoints.bicep' = if (usesDataFactory && usesStorage && app.hub.options.privateRouting) { + name: 'ApproveStoragePrivateEndpointConnections' + params: { + storageAccountName: storageAccount.name + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access + privateEndpointConnections: getStoragePrivateEndpointConnections.outputs.privateEndpointConnections + } +} + +//------------------------------------------------------------------------------ +// Role assignments +//------------------------------------------------------------------------------ + +// Grant ADF identity access to storage +resource storageRoleAssignments 'Microsoft.Authorization/roleAssignments@2022-04-01' = [ + for role in factoryStorageRoles: { + name: guid(storageAccount.id, role, dataFactory.id) + scope: storageAccount + properties: { + roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', role) + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access + principalId: dataFactory.identity.principalId + principalType: 'ServicePrincipal' + } + } +] + +//------------------------------------------------------------------------------ +// Stop triggers and delete old resources +//------------------------------------------------------------------------------ + +// Create managed identity to start/stop triggers +resource triggerManagerIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = if (usesDataFactory) { + name: '${dataFactory.name}_triggerManager' + location: app.hub.location + tags: union(app.tags, app.hub.tagsByResource[?'Microsoft.ManagedIdentity/userAssignedIdentities'] ?? {}) +} + +resource triggerManagerRoleAssignments 'Microsoft.Authorization/roleAssignments@2022-04-01' = [ + for role in autoStartRbacRoles: if (usesDataFactory) { + name: guid(dataFactory.id, role, triggerManagerIdentity.id) + scope: dataFactory + properties: { + roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', role) + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access + principalId: triggerManagerIdentity.properties.principalId + principalType: 'ServicePrincipal' + } + } +] + +// Stop all triggers before deploying triggers +module stopTriggers 'hub-deploymentScript.bicep' = { + name: '${app.publisher}.${app.name}_ADF.StopTriggers' + dependsOn: [ + // TODO: Do we need to make this optional only if private endpoints are enabled and telemetry is enabled? Will it fail when telemetry is disabled? + appTelemetry // Ensure the telemetry deployment is run before stopping triggers since we can only run one deployment at a time with private endpoints + triggerManagerRoleAssignments + ] + params: { + app: app + identityName: triggerManagerIdentity.name + scriptContent: loadTextContent('./scripts/Init-DataFactory.ps1') + arguments: '-Stop' + environmentVariables: [ + { + name: 'DataFactorySubscriptionId' + value: subscription().id + } + { + name: 'DataFactoryResourceGroup' + value: resourceGroup().name + } + { + name: 'DataFactoryName' + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access + value: dataFactory.name + } + ] + } +} + +//------------------------------------------------------------------------------ +// Storage account +//------------------------------------------------------------------------------ + +resource storageAccount 'Microsoft.Storage/storageAccounts@2022-09-01' = if (usesStorage) { + name: app.storage + location: app.hub.location + sku: { + name: app.hub.options.storageSku + } + kind: 'BlockBlobStorage' + tags: getAppPublisherTags(app, 'Microsoft.Storage/storageAccounts') + properties: { + ...storageInfrastructureEncryptionProperties + supportsHttpsTrafficOnly: true + allowSharedKeyAccess: true + isHnsEnabled: true + minimumTlsVersion: 'TLS1_2' + allowBlobPublicAccess: false + publicNetworkAccess: 'Enabled' + networkAcls: { + bypass: 'AzureServices' + defaultAction: app.hub.options.privateRouting ? 'Deny' : 'Allow' + } + } + + resource blobService 'blobServices' = { + name: 'default' + } +} + +resource blobPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = if (usesStorage && app.hub.options.privateRouting) { + name: 'privatelink.blob.${environment().suffixes.storage}' // cSpell:ignore privatelink +} + +resource blobEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (usesStorage && app.hub.options.privateRouting) { + name: '${storageAccount.name}-blob-ep' + location: app.hub.location + tags: getAppPublisherTags(app, 'Microsoft.Network/privateEndpoints') + properties: { + subnet: { + id: app.hub.routing.subnets.storage + } + privateLinkServiceConnections: [ + { + name: 'blobLink' + properties: { + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access + privateLinkServiceId: storageAccount.id + groupIds: ['blob'] + } + } + ] + } + + resource blobPrivateDnsZoneGroup 'privateDnsZoneGroups' = { + name: 'storage-endpoint-zone' + properties: { + privateDnsZoneConfigs: [ + { + name: blobPrivateDnsZone.name + properties: { + privateDnsZoneId: blobPrivateDnsZone.id + } + } + ] + } + } +} + +resource dfsPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = if (usesStorage && app.hub.options.privateRouting) { + name: 'privatelink.dfs.${environment().suffixes.storage}' // cSpell:ignore privatelink +} + +resource dfsEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (usesStorage && app.hub.options.privateRouting) { + name: '${storageAccount.name}-dfs-ep' + location: app.hub.location + tags: getAppPublisherTags(app, 'Microsoft.Network/privateEndpoints') + properties: { + subnet: { + id: app.hub.routing.subnets.storage + } + privateLinkServiceConnections: [ + { + name: 'dfsLink' + properties: { + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access + privateLinkServiceId: storageAccount.id + groupIds: ['dfs'] + } + } + ] + } + + resource dfsPrivateDnsZoneGroup 'privateDnsZoneGroups' = { + name: 'dfs-endpoint-zone' + properties: { + privateDnsZoneConfigs: [ + { + name: dfsPrivateDnsZone.name + properties: { + privateDnsZoneId: dfsPrivateDnsZone.id + } + } + ] + } + } +} + +//------------------------------------------------------------------------------ +// KeyVault for secrets +//------------------------------------------------------------------------------ + +resource keyVault 'Microsoft.KeyVault/vaults@2023-02-01' = if (usesKeyVault) { + name: app.keyVault + location: app.hub.location + tags: getAppPublisherTags(app, 'Microsoft.KeyVault/vaults') + properties: { + sku: any({ + name: app.hub.options.keyVaultSku + family: 'A' + }) + enabledForDeployment: true + enabledForTemplateDeployment: true + enabledForDiskEncryption: true + enableSoftDelete: true + softDeleteRetentionInDays: 90 + enableRbacAuthorization: false + createMode: 'default' + tenantId: subscription().tenantId + accessPolicies: keyVaultAccessPolicies + networkAcls: { + bypass: 'AzureServices' + defaultAction: app.hub.options.privateRouting ? 'Deny' : 'Allow' + } + } + + resource keyVault_accessPolicies 'accessPolicies' = { + name: 'add' + properties: { + accessPolicies: keyVaultAccessPolicies + } + } +} + +resource keyVaultPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (usesKeyVault && app.hub.options.privateRouting) { + name: 'privatelink${replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')}' // cSpell:ignore privatelink, vaultcore + location: 'global' + tags: getAppPublisherTags(app, 'Microsoft.Network/privateDnsZones') + properties: {} + + resource keyVaultPrivateDnsZoneLink 'virtualNetworkLinks@2024-06-01' = { + name: '${replace(keyVaultPrivateDnsZone.name, '.', '-')}-link' + location: 'global' + tags: getAppPublisherTags(app, 'Microsoft.Network/privateDnsZones/virtualNetworkLinks') + properties: { + virtualNetwork: { + id: app.hub.routing.networkId + } + registrationEnabled: false + } + } +} + +resource keyVaultEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (usesKeyVault && app.hub.options.privateRouting) { + name: '${keyVault.name}-ep' + location: app.hub.location + tags: getAppPublisherTags(app, 'Microsoft.Network/privateEndpoints') + properties: { + subnet: { + id: app.hub.routing.subnets.keyVault + } + privateLinkServiceConnections: [ + { + name: 'keyVaultLink' + properties: { + #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access + privateLinkServiceId: keyVault.id + groupIds: ['vault'] + } + } + ] + } + + resource keyVaultPrivateDnsZoneGroup 'privateDnsZoneGroups' = { + name: 'keyvault-endpoint-zone' + properties: { + privateDnsZoneConfigs: [ + { + name: keyVaultPrivateDnsZone.name + properties: { + privateDnsZoneId: keyVaultPrivateDnsZone.id + } + } + ] + } + } +} + + +//============================================================================== +// Outputs +//============================================================================== + +@description('Resource ID of the Data Factory instance used by the FinOps hub app.') +#disable-next-line BCP318 // Null safety warning for conditional resource access +output dataFactoryId string = dataFactory.id + +@description('Resource ID of the Key Vault instance used by the FinOps hub app.') +#disable-next-line BCP318 // Null safety warning for conditional resource access +output keyVaultId string = keyVault.id + +@description('Resource ID of the storage account instance used by the FinOps hub app.') +#disable-next-line BCP318 // Null safety warning for conditional resource access +output storageAccountId string = storageAccount.id + +@description('Principal ID for the managed identity used by Data Factory.') +#disable-next-line BCP318 // Null safety warning for conditional resource access +output principalId string = dataFactory.identity.principalId + +@description('Name of the managed identity used to create and stop ADF triggers.') +output triggerManagerIdentityName string = triggerManagerIdentity.name diff --git a/src/templates/finops-hub/modules/hub-database.bicep b/src/templates/finops-hub/modules/fx/hub-database.bicep similarity index 93% rename from src/templates/finops-hub/modules/hub-database.bicep rename to src/templates/finops-hub/modules/fx/hub-database.bicep index 085e53b0f..871b6a966 100644 --- a/src/templates/finops-hub/modules/hub-database.bicep +++ b/src/templates/finops-hub/modules/fx/hub-database.bicep @@ -34,6 +34,7 @@ resource cluster 'Microsoft.Kusto/clusters@2023-08-15' existing = { resource script 'scripts' = [for scr in items(scripts) : { name: scr.key properties: { + #disable-next-line use-secure-value-for-secure-inputs // KQL scripts don't contain sensitive information scriptContent: scr.value continueOnErrors: continueOnErrors forceUpdateTag: forceUpdateTag diff --git a/src/templates/finops-hub/modules/hub-deploymentScript.bicep b/src/templates/finops-hub/modules/fx/hub-deploymentScript.bicep similarity index 95% rename from src/templates/finops-hub/modules/hub-deploymentScript.bicep rename to src/templates/finops-hub/modules/fx/hub-deploymentScript.bicep index 6fdc804fc..95408c527 100644 --- a/src/templates/finops-hub/modules/hub-deploymentScript.bicep +++ b/src/templates/finops-hub/modules/fx/hub-deploymentScript.bicep @@ -40,12 +40,13 @@ param environmentVariables EnvironmentVariable[] = [] var privateEndpointDeploymentRoles = !app.hub.options.privateRouting ? [] : [ '69566ab7-960f-475b-8e7c-b3118f30c6bd' // Storage File Data Privileged Contributor - https://learn.microsoft.com/azure/role-based-access-control/built-in-roles/storage#storage-file-data-privileged-contributor ] +var containerGroupName = replace(replace(replace(scriptName, '/', '-'), '.', '-'), '_', '-') var privateEndpointDeploymentProperties = !app.hub.options.privateRouting ? {} : { storageAccountSettings: { storageAccountName: app.hub.routing.scriptStorage ?? '' } containerSettings: { - containerGroupName: '${app.hub.routing.scriptStorage}cg' + containerGroupName: length(containerGroupName) > 63 ? substring(containerGroupName, 0, 62) : containerGroupName subnetIds: [ { id: app.hub.routing.subnets.scripts ?? '' @@ -110,7 +111,7 @@ resource script 'Microsoft.Resources/deploymentScripts@2023-08-01' = { } properties: { ...privateEndpointDeploymentProperties - azPowerShellVersion: '9.0' + azPowerShellVersion: '11.0' retentionInterval: 'PT1H' cleanupPreference: 'OnSuccess' scriptContent: scriptContent diff --git a/src/templates/finops-hub/modules/hub-event-trigger.bicep b/src/templates/finops-hub/modules/fx/hub-eventTrigger.bicep similarity index 100% rename from src/templates/finops-hub/modules/hub-event-trigger.bicep rename to src/templates/finops-hub/modules/fx/hub-eventTrigger.bicep diff --git a/src/templates/finops-hub/modules/hub-identity.bicep b/src/templates/finops-hub/modules/fx/hub-identity.bicep similarity index 92% rename from src/templates/finops-hub/modules/hub-identity.bicep rename to src/templates/finops-hub/modules/fx/hub-identity.bicep index f440e0e14..0b12f2776 100644 --- a/src/templates/finops-hub/modules/hub-identity.bicep +++ b/src/templates/finops-hub/modules/fx/hub-identity.bicep @@ -1,7 +1,7 @@ // Copyright (c) Microsoft Corporation. // Licensed under the MIT License. -import { getPublisherTags, HubAppProperties } from 'hub-types.bicep' +import { getAppPublisherTags, HubAppProperties } from 'hub-types.bicep' //============================================================================== @@ -28,7 +28,7 @@ param roles string[] // Create managed identity resource identity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = { name: identityName - tags: getPublisherTags(app, 'Microsoft.ManagedIdentity/userAssignedIdentities') + tags: getAppPublisherTags(app, 'Microsoft.ManagedIdentity/userAssignedIdentities') location: app.hub.location } diff --git a/src/templates/finops-hub/modules/fx/hub-initialize.bicep b/src/templates/finops-hub/modules/fx/hub-initialize.bicep new file mode 100644 index 000000000..b688f8a59 --- /dev/null +++ b/src/templates/finops-hub/modules/fx/hub-initialize.bicep @@ -0,0 +1,76 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { HubAppProperties } from 'hub-types.bicep' + + +//============================================================================== +// Parameters +//============================================================================== + +@description('Required. FinOps hub app getting deployed.') +param app HubAppProperties + +@description('Required. List of Azure Data Factory instances to start triggers for. Can be up to 1 per publisher.') +param dataFactoryInstances string[] + +@description('Required. Name of the managed identity to use when starting the triggers.') +param identityName string + +@description('Optional. Start all triggers for the Data Factory instances. Default: false.') +param startAllTriggers bool = false + +@description('Optional. List of pipelines to run. Default: [] (no pipelines).') +param startPipelines string[] = [] + + +//============================================================================== +// Variables +//============================================================================== + +// Clean up dataFactoryInstances array - remove empty values and duplicates +var uniqueInstances = union(filter(dataFactoryInstances, adf => !empty(adf)), []) + +//============================================================================== +// Resources +//============================================================================== + +// Initialize Data Factory instances (start triggers and/or run pipelines) +module initialize 'hub-deploymentScript.bicep' = [ + for adf in uniqueInstances: { + name: length('Microsoft.FinOpsHubs.Init_${adf}') <= 64 ? 'Microsoft.FinOpsHubs.Init_${adf}' : substring('Microsoft.FinOpsHubs.Init_${adf}', 0, 64) + params: { + app: app + identityName: identityName + scriptContent: loadTextContent('./scripts/Init-DataFactory.ps1') + environmentVariables: [ + { + name: 'DataFactorySubscriptionId' + value: subscription().id + } + { + name: 'DataFactoryResourceGroup' + value: resourceGroup().name + } + { + name: 'DataFactoryName' + value: adf + } + { + name: 'Pipelines' + value: join(startPipelines, '|') + } + { + name: 'StartAllTriggers' + value: string(startAllTriggers) + } + ] + } + } +] + +//============================================================================== +// Outputs +//============================================================================== + +// None diff --git a/src/templates/finops-hub/modules/hub-storage.bicep b/src/templates/finops-hub/modules/fx/hub-storage.bicep similarity index 93% rename from src/templates/finops-hub/modules/hub-storage.bicep rename to src/templates/finops-hub/modules/fx/hub-storage.bicep index 723628f44..8dda03fe0 100644 --- a/src/templates/finops-hub/modules/hub-storage.bicep +++ b/src/templates/finops-hub/modules/fx/hub-storage.bicep @@ -3,7 +3,6 @@ import { HubAppProperties } from 'hub-types.bicep' - //============================================================================== // Parameters //============================================================================== @@ -20,7 +19,6 @@ param files object = {} @description('Optional. Indicates whether to create the blob manager user assigned identity even if files are not being uploaded. Default: false.') param forceCreateBlobManagerIdentity bool = false - //============================================================================== // Variables //============================================================================== @@ -28,7 +26,6 @@ param forceCreateBlobManagerIdentity bool = false var fileCount = length(items(files)) var hasFiles = fileCount > 0 - //============================================================================== // Resources //============================================================================== @@ -36,7 +33,7 @@ var hasFiles = fileCount > 0 // Get storage account instance resource storageAccount 'Microsoft.Storage/storageAccounts@2022-09-01' existing = { name: app.storage - + resource blobService 'blobServices@2022-09-01' existing = { name: 'default' @@ -63,7 +60,7 @@ module identity 'hub-identity.bicep' = if (hasFiles || forceCreateBlobManagerIde // Storage Blob Data Contributor - https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#storage-blob-data-contributor // Used by deployment scripts to write data to blob storage 'ba92f5b4-2d11-453d-a403-e96b0029c9fe' - + // Storage File Data Privileged Contributor - https://learn.microsoft.com/azure/role-based-access-control/built-in-roles/storage#storage-file-data-privileged-contributor // https://learn.microsoft.com/azure/azure-resource-manager/templates/deployment-script-template#use-existing-storage-account '69566ab7-960f-475b-8e7c-b3118f30c6bd' @@ -76,6 +73,7 @@ module uploadFiles 'hub-deploymentScript.bicep' = if (hasFiles) { name: '${deployment().name}.Upload' params: { app: app + #disable-next-line BCP318 // Null safety warning for conditional resource access identityName: identity.outputs.name environmentVariables: [ { @@ -95,7 +93,6 @@ module uploadFiles 'hub-deploymentScript.bicep' = if (hasFiles) { } } - //============================================================================== // Outputs //============================================================================== @@ -107,10 +104,13 @@ output containerName string = storageAccount::blobService::targetContainer.name output filesUploaded int = fileCount @description('Resource ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false.') +#disable-next-line BCP318 // Null safety warning for conditional resource access output identityId string = hasFiles || forceCreateBlobManagerIdentity ? identity.outputs.id : '' @description('Name of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false.') +#disable-next-line BCP318 // Null safety warning for conditional resource access output identityName string = hasFiles || forceCreateBlobManagerIdentity ? identity.outputs.name : '' @description('Principal ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false.') +#disable-next-line BCP318 // Null safety warning for conditional resource access output identityPrincipalId string = hasFiles || forceCreateBlobManagerIdentity ? identity.outputs.principalId : '' diff --git a/src/templates/finops-hub/modules/hub-types.bicep b/src/templates/finops-hub/modules/fx/hub-types.bicep similarity index 77% rename from src/templates/finops-hub/modules/hub-types.bicep rename to src/templates/finops-hub/modules/fx/hub-types.bicep index b4b2fc771..d0072df2c 100644 --- a/src/templates/finops-hub/modules/hub-types.bicep +++ b/src/templates/finops-hub/modules/fx/hub-types.bicep @@ -32,6 +32,7 @@ type IdNameObject = { id: string, name: string } table: 'Resource ID and name for the table storage DNS zone.' } subnets: { + dataExplorer: 'Resource ID of the subnet for the Data Explorer instance.' dataFactory: 'Resource ID of the subnet for Data Factory instances.' keyVault: 'Resource ID of the subnet for Key Vault instances.' scripts: 'Resource ID of the subnet for deployment script storage.' @@ -49,6 +50,7 @@ type HubRoutingProperties = { table: IdNameObject } subnets: { + dataExplorer: string dataFactory: string keyVault: string scripts: string @@ -110,34 +112,26 @@ type HubProperties = { @export() @description('FinOps hub app configuration settings.') @metadata({ - name: 'Short name of the FinOps hub app (not including the publisher namespace).' - displayName: 'Display name of the FinOps hub app.' - tags: 'Tags to apply to all FinOps hub resources for this FinOps hub app.' - publisher: { - name: 'Fully-qualified namespace of the FinOps hub app publisher.' - displayName: 'Display name of the FinOps hub app publisher.' - suffix: 'Unique suffix used for publisher resources.' - tags: 'Tags to apply to all FinOps hub resources for this FinOps hub app publisher.' - } - hub: 'FinOps hub instance the app is deployed to.' + id: 'Fully-qualified name of the publisher and app, separated by a dot.' + name: 'Short name of the FinOps hub app. Last segment of the app ID.' + publisher: 'Fully-qualified namespace of the FinOps hub app publisher.' + suffix: 'Unique suffix used for publisher resources.' + tags: 'Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.' dataFactory: 'Name of the Data Factory instance for this publisher.' keyVault: 'Name of the KeyVault instance for this publisher.' storage: 'Name of the storage account for this publisher.' + hub: 'FinOps hub instance the app is deployed to.' }) type HubAppProperties = { + id: string name: string - displayName: string + publisher: string + suffix: string tags: object - publisher: { - name: string - displayName: string - suffix: string - tags: object - } - hub: HubProperties dataFactory: string keyVault: string storage: string + hub: HubProperties } @export() @@ -149,6 +143,7 @@ type HubAppFeature = 'DataFactory' | 'KeyVault' | 'Storage' // Variables //============================================================================== +@export() @description('Version of the FinOps toolkit.') var finOpsToolkitVersion = loadTextContent('ftkver.txt') // cSpell:ignore ftkver @@ -221,10 +216,11 @@ func newHubInternal( table: enablePublicAccess ? { id:'', name:'' } : dnsZoneIdName('table') } subnets: { - dataFactory: enablePublicAccess ? '' : resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, 'private-endpoint-subnet')! - keyVault: enablePublicAccess ? '' : resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, 'private-endpoint-subnet')! - scripts: enablePublicAccess ? '' : resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, 'script-subnet')! - storage: enablePublicAccess ? '' : resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, 'private-endpoint-subnet')! + dataExplorer: enablePublicAccess ? '' : resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, 'dataExplorer-subnet')! + dataFactory: enablePublicAccess ? '' : resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, 'private-endpoint-subnet')! + keyVault: enablePublicAccess ? '' : resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, 'private-endpoint-subnet')! + scripts: enablePublicAccess ? '' : resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, 'script-subnet')! + storage: enablePublicAccess ? '' : resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, 'private-endpoint-subnet')! } } core: { @@ -268,56 +264,48 @@ func newHub( // Internal function to create a new FinOps hub configuration object that includes extra parameters that are reused within the function. func newAppInternal( hub HubProperties, - publisherName string, - publisherDisplayName string, - publisherSuffix string, - publisherTags object, - appName string, - appDisplayName string, - version string, + id string, + name string, + publisher string, + suffix string, ) HubAppProperties => { - name: appName - displayName: appDisplayName - tags: union(hub.tags, publisherTags, { - 'ftk-hubapp': appName // cSpell:ignore hubapp - 'ftk-hubapp-version': version - }) - publisher: { - name: publisherName - displayName: publisherDisplayName - suffix: publisherSuffix - tags: union(hub.tags, publisherTags) - } + id: id + name: name + publisher: publisher + suffix: suffix + tags: union( + hub.tags, + { 'ftk-hubapp-publisher': publisher } // publisherTags + // TODO: How do we want to handle app-specific tags? + // { + // 'ftk-hubapp': appName // cSpell:ignore hubapp + // 'ftk-hubapp-version': version + // } + ) hub: hub // Globally unique Data Factory name: 3-63 chars; letters, numbers, non-repeating dashes - dataFactory: replace('${take('${replace(hub.name, '_', '-')}-engine', 63 - length(publisherSuffix) - 1)}-${publisherSuffix}', '--', '-') + dataFactory: replace('${take('${replace(hub.name, '_', '-')}-engine', 63 - length(suffix) - 1)}-${suffix}', '--', '-') // Globally unique KeyVault name: 3-24 chars; letters, numbers, dashes - keyVault: replace('${take('${replace(hub.name, '_', '-')}-vault', 24 - length(publisherSuffix) - 1)}-${publisherSuffix}', '--', '-') + keyVault: replace('${take('${replace(hub.name, '_', '-')}-vault', 24 - length(suffix) - 1)}-${suffix}', '--', '-') // Globally unique storage account name: 3-24 chars; lowercase letters/numbers only - storage: '${take(safeStorageName(hub.name), 24 - length(publisherSuffix))}${publisherSuffix}' + storage: '${take(safeStorageName(hub.name), 24 - length(suffix))}${suffix}' } @export() @description('Creates a new FinOps hub app configuration object.') func newApp( hub HubProperties, - publisherDisplayName string, - publisherName string, - appPartialName string, - appDisplayName string, - version string, + publisher string, + app string, ) HubAppProperties => newAppInternal( hub, - publisherName, - publisherDisplayName, - !hub.options.publisherIsolation || publisherName == 'Microsoft.FinOpsHubs' ? hub.core.suffix : uniqueString(publisherName), // publisherSuffix - { 'ftk-hubapp-publisher': publisherName }, // publisherTags - '${publisherName}.${appPartialName}', // appName - appDisplayName, - version + '${publisher}.${app}', // id + app, + publisher, + !hub.options.publisherIsolation || publisher == 'Microsoft.FinOpsHubs' ? hub.core.suffix : uniqueString(publisher) // publisher suffix ) @export() @@ -329,14 +317,21 @@ func getHubTags(hub HubProperties, resourceType string) object => union( @export() @description('Returns a tags dictionary that includes tags for the FinOps hub app publisher.') -func getPublisherTags(app HubAppProperties, resourceType string) object => union( - app.hub.options.publisherIsolation ? app.publisher.tags : app.hub.tags, +func getAppPublisherTags(app HubAppProperties, resourceType string) object => union( + app.hub.options.publisherIsolation ? app.tags : app.hub.tags, app.hub.tagsByResource[?resourceType] ?? {} ) + +//------------------------------------------------------------------------------ +// Private routing +//------------------------------------------------------------------------------ + @export() -@description('Returns a tags dictionary that includes tags for the FinOps hub app.') -func getAppTags(app HubAppProperties, resourceType string, forceAppTags bool?) object => union( - app.hub.options.publisherIsolation || (forceAppTags ?? false) ? app.tags : app.hub.tags, - app.hub.tagsByResource[?resourceType] ?? {} -) +@description('Returns an object that represents the properties needed to enable private routing for linked services. Use property expansion (`...value`) to apply to a linkedServices resource.') +func privateRoutingForLinkedServices(hub HubProperties) object => hub.options.privateRouting ? { + connectVia: { + referenceName: 'ManagedIntegrationRuntime' + type: 'IntegrationRuntimeReference' + } +} : {} diff --git a/src/templates/finops-hub/modules/hub-vault.bicep b/src/templates/finops-hub/modules/fx/hub-vault.bicep similarity index 100% rename from src/templates/finops-hub/modules/hub-vault.bicep rename to src/templates/finops-hub/modules/fx/hub-vault.bicep diff --git a/src/templates/finops-hub/modules/keyVaultEndpoints.bicep b/src/templates/finops-hub/modules/fx/keyVaultEndpoints.bicep similarity index 100% rename from src/templates/finops-hub/modules/keyVaultEndpoints.bicep rename to src/templates/finops-hub/modules/fx/keyVaultEndpoints.bicep diff --git a/src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 b/src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 new file mode 100644 index 000000000..6ec94421c --- /dev/null +++ b/src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 @@ -0,0 +1,73 @@ +# Copyright (c) Microsoft Corporation. +# Licensed under the MIT License. + +param( + [switch] $Stop +) + +# Init outputs +$DeploymentScriptOutputs = @{} + +if (-not $Stop) +{ + Start-Sleep -Seconds 10 +} + +# Loop thru triggers +$triggers = Get-AzDataFactoryV2Trigger ` + -ResourceGroupName $env:DataFactoryResourceGroup ` + -DataFactoryName $env:DataFactoryName + +Write-Output "Found $($triggers.Length) trigger(s)" + +if ($startTriggers) +{ + $triggers | ForEach-Object { + $trigger = $_.Name + if ($Stop) + { + Write-Output "Stopping trigger $trigger..." + $triggerOutput = Stop-AzDataFactoryV2Trigger ` + -ResourceGroupName $env:DataFactoryResourceGroup ` + -DataFactoryName $env:DataFactoryName ` + -Name $trigger ` + -Force ` + -ErrorAction SilentlyContinue # Ignore errors, since the trigger may not exist + } + else + { + Write-Output "Starting trigger $trigger..." + $triggerOutput = Start-AzDataFactoryV2Trigger ` + -ResourceGroupName $env:DataFactoryResourceGroup ` + -DataFactoryName $env:DataFactoryName ` + -Name $trigger ` + -Force + } + if ($triggerOutput) + { + Write-Output "done..." + } + else + { + Write-Output "failed..." + } + $DeploymentScriptOutputs[$trigger] = $triggerOutput + } + + if ($Stop) + { + Start-Sleep -Seconds 10 + } +} + +if (-not [string]::IsNullOrWhiteSpace($env:Pipelines)) +{ + $env:Pipelines.Split('|') ` + | ForEach-Object { + Write-Output "Running the init pipeline..." + Invoke-AzDataFactoryV2Pipeline ` + -ResourceGroupName $env:DataFactoryResourceGroup ` + -DataFactoryName $env:DataFactoryName ` + -PipelineName $_ + } +} diff --git a/src/templates/finops-hub/modules/scripts/Remove-OldResources.ps1 b/src/templates/finops-hub/modules/fx/scripts/Remove-OldResources.ps1 similarity index 100% rename from src/templates/finops-hub/modules/scripts/Remove-OldResources.ps1 rename to src/templates/finops-hub/modules/fx/scripts/Remove-OldResources.ps1 diff --git a/src/templates/finops-hub/modules/scripts/Upload-StorageFile.ps1 b/src/templates/finops-hub/modules/fx/scripts/Upload-StorageFile.ps1 similarity index 100% rename from src/templates/finops-hub/modules/scripts/Upload-StorageFile.ps1 rename to src/templates/finops-hub/modules/fx/scripts/Upload-StorageFile.ps1 diff --git a/src/templates/finops-hub/modules/storageEndpoints.bicep b/src/templates/finops-hub/modules/fx/storageEndpoints.bicep similarity index 100% rename from src/templates/finops-hub/modules/storageEndpoints.bicep rename to src/templates/finops-hub/modules/fx/storageEndpoints.bicep diff --git a/src/templates/finops-hub/modules/hub-app.bicep b/src/templates/finops-hub/modules/hub-app.bicep deleted file mode 100644 index 932e12688..000000000 --- a/src/templates/finops-hub/modules/hub-app.bicep +++ /dev/null @@ -1,332 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -import { getAppTags, getPublisherTags, HubAppProperties, HubAppFeature, HubProperties, newApp } from 'hub-types.bicep' - - -//============================================================================== -// Parameters -//============================================================================== - -@description('Required. FinOps hub instance properties.') -param hub HubProperties - -@description('Required. Display name of the FinOps hub app publisher.') -param publisher string - -@description('Required. Namespace to use for the FinOps hub app publisher. Will be combined with appName to form a fully-qualified identifier. Must be an alphanumeric string without spaces or special characters except for periods. This value should never change and will be used to uniquely identify the publisher. A change would require migrating content to the new publisher. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed.') -param namespace string - -@description('Required. Unique identifier of the FinOps hub app within the publisher namespace. Must be an alphanumeric string without spaces or special characters. This name should never change and will be used with the namespace to fully qualify the app. A change would require migrating content to the new app. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed.') -param appName string - -@description('Required. Display name of the FinOps hub app.') -param displayName string - -@description('Optional. Version number of the FinOps hub app.') -param appVersion string = '' - -// @description('Required. Minimum version number supported by the FinOps hub app.') -// param hubMinVersion string - -// @description('Required. Maximum version number supported by the FinOps hub app.') -// param hubMaxVersion string - -@description('Optional. Indicate which features the app requires. Allowed values: "Storage". Default: [] (none).') -param features HubAppFeature[] = [] - -@description('Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed.') -param telemetryString string = '' - - -//============================================================================== -// Variables -//============================================================================== - -var app = newApp(hub, publisher, namespace, appName, displayName, appVersion) - -// Features -var usesDataFactory = contains(features, 'DataFactory') -var usesKeyVault = contains(features, 'KeyVault') -var usesStorage = contains(features, 'Storage') - -// App telemetry -var telemetryId = 'ftk-hubapp-${app.name}${empty(telemetryString) ? '' : '_'}${telemetryString}' // cSpell:ignore hubapp -var telemetryProps = { - mode: 'Incremental' - template: { - '$schema': 'https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#' - contentVersion: '1.0.0.0' - metadata: { - _generator: { - name: 'FTK: ${publisher} - ${displayName} ${telemetryId}' - version: appVersion - } - } - resources: [] - } -} - -// Storage infrastructure encryption -var storageInfrastructureEncryptionProperties = !hub.options.storageInfrastructureEncryption ? {} : { - encryption: { - keySource: 'Microsoft.Storage' - requireInfrastructureEncryption: hub.options.storageInfrastructureEncryption - } -} - -// KeyVault access policies -var keyVaultAccessPolicies = [ - { - objectId: dataFactory.identity.principalId - tenantId: subscription().tenantId - permissions: { secrets: ['get'] } - } -] - - -//============================================================================== -// Resources -//============================================================================== - -// TODO: Get hub instance to verify version compatibility - -//------------------------------------------------------------------------------ -// Telemetry -// Used to anonymously count the number of times the template has been deployed -// and to track and fix deployment bugs to ensure the highest quality. -// No information about you or your cost data is collected. -//------------------------------------------------------------------------------ - -resource appTelemetry 'Microsoft.Resources/deployments@2022-09-01' = if (hub.options.enableTelemetry) { - name: length(telemetryId) <= 64 ? telemetryId : substring(telemetryId, 0, 64) - tags: getAppTags(app, 'Microsoft.Resources/deployments', true) - properties: telemetryProps -} - -//------------------------------------------------------------------------------ -// TODO: Get hub details -//------------------------------------------------------------------------------ - -//------------------------------------------------------------------------------ -// Data Factory -//------------------------------------------------------------------------------ - -resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' = if (usesDataFactory) { - name: app.dataFactory - location: app.hub.location - tags: getPublisherTags(app, 'Microsoft.DataFactory/factories') - identity: { type: 'SystemAssigned' } - properties: any({ // Using any() to hide the error that gets surfaced because globalConfigurations is not in the ADF schema yet - globalConfigurations: { - PipelineBillingEnabled: 'true' - } - }) -} - -//------------------------------------------------------------------------------ -// Storage account -//------------------------------------------------------------------------------ - -resource storageAccount 'Microsoft.Storage/storageAccounts@2022-09-01' = if (usesStorage) { - name: app.storage - location: hub.location - sku: { - name: hub.options.storageSku - } - kind: 'BlockBlobStorage' - tags: getPublisherTags(app, 'Microsoft.Storage/storageAccounts') - properties: { - ...storageInfrastructureEncryptionProperties - supportsHttpsTrafficOnly: true - allowSharedKeyAccess: true - isHnsEnabled: true - minimumTlsVersion: 'TLS1_2' - allowBlobPublicAccess: false - publicNetworkAccess: 'Enabled' - networkAcls: { - bypass: 'AzureServices' - defaultAction: hub.options.privateRouting ? 'Deny' : 'Allow' - } - } - - resource blobService 'blobServices' = { - name: 'default' - } -} - -resource blobPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = if (usesStorage && hub.options.privateRouting) { - name: 'privatelink.blob.${environment().suffixes.storage}' // cSpell:ignore privatelink -} - -resource blobEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (usesStorage && hub.options.privateRouting) { - name: '${storageAccount.name}-blob-ep' - location: hub.location - tags: getPublisherTags(app, 'Microsoft.Network/privateEndpoints') - properties: { - subnet: { - id: hub.routing.subnets.storage - } - privateLinkServiceConnections: [ - { - name: 'blobLink' - properties: { - privateLinkServiceId: storageAccount.id - groupIds: ['blob'] - } - } - ] - } - - resource blobPrivateDnsZoneGroup 'privateDnsZoneGroups' = { - name: 'storage-endpoint-zone' - properties: { - privateDnsZoneConfigs: [ - { - name: blobPrivateDnsZone.name - properties: { - privateDnsZoneId: blobPrivateDnsZone.id - } - } - ] - } - } -} - -resource dfsPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = if (usesStorage && hub.options.privateRouting) { - name: 'privatelink.dfs.${environment().suffixes.storage}' // cSpell:ignore privatelink -} - -resource dfsEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (usesStorage && hub.options.privateRouting) { - name: '${storageAccount.name}-dfs-ep' - location: hub.location - tags: getPublisherTags(app, 'Microsoft.Network/privateEndpoints') - properties: { - subnet: { - id: hub.routing.subnets.storage - } - privateLinkServiceConnections: [ - { - name: 'dfsLink' - properties: { - privateLinkServiceId: storageAccount.id - groupIds: ['dfs'] - } - } - ] - } - - resource dfsPrivateDnsZoneGroup 'privateDnsZoneGroups' = { - name: 'dfs-endpoint-zone' - properties: { - privateDnsZoneConfigs: [ - { - name: dfsPrivateDnsZone.name - properties: { - privateDnsZoneId: dfsPrivateDnsZone.id - } - } - ] - } - } -} - -//------------------------------------------------------------------------------ -// KeyVault for secrets -//------------------------------------------------------------------------------ - -resource keyVault 'Microsoft.KeyVault/vaults@2023-02-01' = if (usesKeyVault) { - name: app.keyVault - location: hub.location - tags: getPublisherTags(app, 'Microsoft.KeyVault/vaults') - properties: { - sku: any({ - name: hub.options.keyVaultSku - family: 'A' - }) - enabledForDeployment: true - enabledForTemplateDeployment: true - enabledForDiskEncryption: true - enableSoftDelete: true - softDeleteRetentionInDays: 90 - enableRbacAuthorization: false - createMode: 'default' - tenantId: subscription().tenantId - accessPolicies: keyVaultAccessPolicies - networkAcls: { - bypass: 'AzureServices' - defaultAction: hub.options.privateRouting ? 'Deny' : 'Allow' - } - } - - resource keyVault_accessPolicies 'accessPolicies' = { - name: 'add' - properties: { - accessPolicies: keyVaultAccessPolicies - } - } -} - -resource keyVaultPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (usesKeyVault && hub.options.privateRouting) { - name: 'privatelink${replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')}' // cSpell:ignore privatelink, vaultcore - location: 'global' - tags: getPublisherTags(app, 'Microsoft.Network/privateDnsZones') - properties: {} - - resource keyVaultPrivateDnsZoneLink 'virtualNetworkLinks@2024-06-01' = { - name: '${replace(keyVaultPrivateDnsZone.name, '.', '-')}-link' - location: 'global' - tags: getPublisherTags(app, 'Microsoft.Network/privateDnsZones/virtualNetworkLinks') - properties: { - virtualNetwork: { - id: hub.routing.networkId - } - registrationEnabled: false - } - } -} - -resource keyVaultEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (usesKeyVault && hub.options.privateRouting) { - name: '${keyVault.name}-ep' - location: hub.location - tags: getPublisherTags(app, 'Microsoft.Network/privateEndpoints') - properties: { - subnet: { - id: hub.routing.subnets.keyVault - } - privateLinkServiceConnections: [ - { - name: 'keyVaultLink' - properties: { - privateLinkServiceId: keyVault.id - groupIds: ['vault'] - } - } - ] - } - - resource keyVaultPrivateDnsZoneGroup 'privateDnsZoneGroups' = { - name: 'keyvault-endpoint-zone' - properties: { - privateDnsZoneConfigs: [ - { - name: keyVaultPrivateDnsZone.name - properties: { - privateDnsZoneId: keyVaultPrivateDnsZone.id - } - } - ] - } - } -} - - -//============================================================================== -// Outputs -//============================================================================== - -@description('FinOps hub app configuration.') -output app HubAppProperties = app - -@description('Principal ID for the managed identity used by Data Factory.') -output principalId string = dataFactory.identity.principalId diff --git a/src/templates/finops-hub/modules/hub.bicep b/src/templates/finops-hub/modules/hub.bicep index 5929bda87..5c5793106 100644 --- a/src/templates/finops-hub/modules/hub.bicep +++ b/src/templates/finops-hub/modules/hub.bicep @@ -1,7 +1,7 @@ // Copyright (c) Microsoft Corporation. // Licensed under the MIT License. -import { getHubTags, newApp, newHub } from 'hub-types.bicep' +import { getHubTags, newApp, newHub } from 'fx/hub-types.bicep' //============================================================================== @@ -40,7 +40,7 @@ param remoteHubStorageUri string = '' @description('Optional. Storage account key for remote storage account.') @secure() param remoteHubStorageKey string = '' - + @description('Optional. Enable managed exports where your FinOps hub instance will create and run Cost Management exports on your behalf. Not supported for Microsoft Customer Agreement (MCA) billing profiles. Requires the ability to grant User Access Administrator role to FinOps hubs, which is required to create Cost Management exports. Default: true.') param enableManagedExports bool = true @@ -72,7 +72,7 @@ param dataExplorerName string = '' 'Standard_DS13_v2+2TB_PS' 'Standard_DS14_v2+3TB_PS' 'Standard_DS14_v2+4TB_PS' - 'Standard_E2a_v4' // 2 CPU, 14GB RAM, 78GB cache, $220/mo + 'Standard_E2a_v4' // 2 CPU, 14GB RAM, 78GB cache, $220/mo 'Standard_E2ads_v5' 'Standard_E2d_v4' 'Standard_E2d_v5' @@ -185,45 +185,8 @@ var hub = newHub( enableDefaultTelemetry ) -// Do not reference these deployments directly or indirectly to avoid a DeploymentNotFound error var useFabric = !empty(fabricQueryUri) -var deployDataExplorer = !useFabric && !empty(dataExplorerName) -var safeDataExplorerName = !deployDataExplorer ? '' : dataExplorer.outputs.clusterName -var safeDataExplorerUri = useFabric ? fabricQueryUri : (!deployDataExplorer ? '' : dataExplorer.outputs.clusterUri) -var safeDataExplorerId = !deployDataExplorer ? '' : dataExplorer.outputs.clusterId -var safeDataExplorerIngestionDb = useFabric ? 'Ingestion' : (!deployDataExplorer ? '' : dataExplorer.outputs.ingestionDbName) -var safeDataExplorerIngestionCapacity = useFabric ? fabricCapacityUnits : (!deployDataExplorer ? 1 : dataExplorer.outputs.clusterIngestionCapacity) -var safeDataExplorerPrincipalId = !deployDataExplorer ? '' : dataExplorer.outputs.principalId -var safeVnetId = enablePublicAccess ? '' : infrastructure.outputs.vNetId -var safeDataExplorerSubnetId = enablePublicAccess ? '' : infrastructure.outputs.dataExplorerSubnetId -// var safeFinopsHubSubnetId = enablePublicAccess ? '' : infrastructure.outputs.finopsHubSubnetId -// var safeScriptSubnetId = enablePublicAccess ? '' : infrastructure.outputs.scriptSubnetId - -// cSpell:ignore eventgrid -// var eventGridName = 'finops-hub-eventgrid-${config.hub.suffix}' - -// var eventGridPrefix = '${replace(hubName, '_', '-')}-ns' -// var eventGridSuffix = '-${config.hub.suffix}' -// var eventGridName = replace( -// '${take(eventGridPrefix, 50 - length(eventGridSuffix))}${eventGridSuffix}', -// '--', -// '-' -// ) - -// EventGrid Contributor role -// var eventGridContributorRoleId = '1e241071-0855-49ea-94dc-649edcd759de' - -// cSpell:ignore israelcentral, uaenorth, italynorth, switzerlandnorth, mexicocentral, southcentralus, polandcentral, swedencentral, spaincentral, francecentral, usdodeast, usdodcentral -// Find a fallback region for EventGrid -// var eventGridLocationFallback = { -// israelcentral: 'uaenorth' -// italynorth: 'switzerlandnorth' -// mexicocentral: 'southcentralus' -// polandcentral: 'swedencentral' -// spaincentral: 'francecentral' -// usdodeast: 'usdodcentral' -// } -// var finalEventGridLocation = eventGridLocation != null && !empty(eventGridLocation) ? eventGridLocation : (eventGridLocationFallback[?location] ?? location) +var useAzureDataExplorer = !useFabric && !empty(dataExplorerName) // Prefer Fabric over Azure Data Explorer // The last segment of the GUID in the telemetryId (40b) is used to identify this module // Remaining characters identify settings; must be <= 12 chars -- Example: (guid)_RLXD##x1000P @@ -236,11 +199,11 @@ var telemetryString = join([ // F = Fabric enabled !useFabric ? '' : 'F${fabricCapacityUnits}' // X = ADX enabled + D (dev) or S (standard) SKU - !deployDataExplorer ? '' : 'X${substring(dataExplorerSku, 0, 1)}' + !useAzureDataExplorer ? '' : 'X${substring(dataExplorerSku, 0, 1)}' // Number of cores in the VM size - !deployDataExplorer ? '' : replace(replace(replace(replace(replace(replace(replace(replace(split(split(dataExplorerSku, 'Standard_')[1], '_')[0], 'C', ''), 'D', ''), 'E', ''), 'L', ''), 'a', ''), 'd', ''), 'i', ''), 's', '') + !useAzureDataExplorer ? '' : replace(replace(replace(replace(replace(replace(replace(replace(split(split(dataExplorerSku, 'Standard_')[1], '_')[0], 'C', ''), 'D', ''), 'E', ''), 'L', ''), 'a', ''), 'd', ''), 'i', ''), 's', '') // Number of nodes in the cluster - !deployDataExplorer || dataExplorerCapacity == 1 ? '' : 'x${dataExplorerCapacity}' + !useAzureDataExplorer || dataExplorerCapacity == 1 ? '' : 'x${dataExplorerCapacity}' // P = private endpoints enabled enablePublicAccess ? '' : 'P' ], '') @@ -265,7 +228,7 @@ resource telemetry 'Microsoft.Resources/deployments@2022-09-01' = if (enableDefa metadata: { _generator: { name: 'FinOps toolkit' - version: loadTextContent('ftkver.txt') // cSpell:ignore ftkver + version: loadTextContent('fx/ftkver.txt') // cSpell:ignore ftkver } } resources: [] @@ -273,30 +236,14 @@ resource telemetry 'Microsoft.Resources/deployments@2022-09-01' = if (enableDefa } } -//------------------------------------------------------------------------------ -// Base resources needed for hub apps -//------------------------------------------------------------------------------ - -// TODO: Can this be merged into core.bicep? -module infrastructure 'infrastructure.bicep' = { - name: 'Microsoft.FinOpsHubs.Infrastructure' - params: { - hub: hub - } -} - //------------------------------------------------------------------------------ // Hub core app //------------------------------------------------------------------------------ -module core 'core.bicep' = { +module core 'Microsoft.FinOpsHubs/Core/app.bicep' = { name: 'Microsoft.FinOpsHubs.Core' - dependsOn: [ - infrastructure - ] params: { - hub: hub - telemetryString: telemetryString + app: newApp(hub, 'Microsoft.FinOpsHubs', 'Core') scopesToMonitor: scopesToMonitor msexportRetentionInDays: exportRetentionInDays // cSpell:ignore msexport ingestionRetentionInMonths: ingestionRetentionInMonths @@ -306,16 +253,26 @@ module core 'core.bicep' = { } //------------------------------------------------------------------------------ -// ADLSv2 storage account for staging and archive +// Cost Management //------------------------------------------------------------------------------ -module cmExports 'cm-exports.bicep' = { +module cmExports 'Microsoft.CostManagement/Exports/app.bicep' = { name: 'Microsoft.CostManagement.Exports' dependsOn: [ core ] params: { - hub: hub + app: newApp(hub, 'Microsoft.CostManagement', 'Exports') + } +} + +module cmManagedExports 'Microsoft.CostManagement/ManagedExports/app.bicep' = if (enableManagedExports) { + name: 'Microsoft.CostManagement.ManagedExports' + dependsOn: [ + cmExports + ] + params: { + app: newApp(hub, 'Microsoft.CostManagement', 'ManagedExports') } } @@ -323,75 +280,83 @@ module cmExports 'cm-exports.bicep' = { // Data Explorer for analytics //------------------------------------------------------------------------------ -module dataExplorer 'dataExplorer.bicep' = if (deployDataExplorer) { - name: 'dataExplorer' +module analytics 'Microsoft.FinOpsHubs/Analytics/app.bicep' = if (useFabric || useAzureDataExplorer) { + name: 'Microsoft.FinOpsHubs.Analytics' + dependsOn: hub.options.privateRouting ? [ + core + // When private endpoints are enabled, we need to explicitly block on anything that uses deployment scripts to guarantee only one deployment script runs at a time + cmExports + deleteOldResources + ] : [ + core + ] params: { + app: newApp(hub, 'Microsoft.FinOpsHubs', 'Analytics') + fabricQueryUri: fabricQueryUri + fabricCapacityUnits: fabricCapacityUnits clusterName: dataExplorerName clusterSku: dataExplorerSku clusterCapacity: dataExplorerCapacity - // TODO: Figure out why this is breaking upgrades -- clusterTrustedExternalTenants: dataExplorerTrustedExternalTenants - location: location - tags: hub.tags - tagsByResource: tagsByResource - dataFactoryName: core.outputs.dataFactoryName rawRetentionInDays: dataExplorerRawRetentionInDays - virtualNetworkId: safeVnetId // cSpell:ignore vnet - privateEndpointSubnetId: safeDataExplorerSubnetId - enablePublicAccess: enablePublicAccess - storageAccountName: core.outputs.storageAccountName + // TODO: Figure out why this is breaking upgrades -- clusterTrustedExternalTenants: dataExplorerTrustedExternalTenants } } //------------------------------------------------------------------------------ -// Data Factory and pipelines +// Remote hub app //------------------------------------------------------------------------------ -module dataFactoryResources 'dataFactory.bicep' = { - name: 'dataFactoryResources' +module remoteHub 'Microsoft.FinOpsHubs/RemoteHub/app.bicep' = if (!empty(remoteHubStorageKey)) { + name: 'Microsoft.FinOpsHubs.RemoteHub' + dependsOn: [ + core + ] params: { - // TODO: Split dataFactory.bicep into its separate apps - app: newApp( - hub, - 'Microsoft FinOps hubs', - 'Microsoft.FinOpsHubs', - 'DataFactory', - 'FinOps hub engine', - loadTextContent('ftkver.txt') - ) - - hubName: hubName - dataFactoryName: core.outputs.dataFactoryName - location: location - tags: core.outputs.publisherTags - tagsByResource: tagsByResource - storageAccountName: core.outputs.storageAccountName - exportContainerName: cmExports.outputs.exportContainer - configContainerName: core.outputs.configContainer - ingestionContainerName: core.outputs.ingestionContainer - dataExplorerName: safeDataExplorerName - dataExplorerPrincipalId: safeDataExplorerPrincipalId - dataExplorerIngestionDatabase: safeDataExplorerIngestionDb - dataExplorerIngestionCapacity: safeDataExplorerIngestionCapacity - dataExplorerUri: safeDataExplorerUri - dataExplorerId: safeDataExplorerId - enableManagedExports: enableManagedExports - enablePublicAccess: enablePublicAccess - - // TODO: Move to remoteHub.bicep - keyVaultName: empty(remoteHubStorageKey) ? '' : remoteHub.outputs.keyVaultName + app: newApp(hub, 'Microsoft.FinOpsHubs', 'RemoteHub') + remoteStorageKey: remoteHubStorageKey remoteHubStorageUri: remoteHubStorageUri } } //------------------------------------------------------------------------------ -// Remote hub app +// Final touches //------------------------------------------------------------------------------ -module remoteHub 'remoteHub.bicep' = if (!empty(remoteHubStorageKey)) { - name: 'Microsoft.FinOpsHubs.RemoteHub' +// Delete old triggers and pipelines +module deleteOldResources 'fx/hub-deploymentScript.bicep' = { + name: 'Microsoft.FinOpsHubs.DeleteOldResources' params: { - hub: hub - remoteStorageKey: remoteHubStorageKey + app: core.outputs.app + identityName: core.outputs.triggerManagerIdentityName + scriptContent: loadTextContent('fx/scripts/Remove-OldResources.ps1') + environmentVariables: [ + { + name: 'DataFactorySubscriptionId' + value: subscription().id + } + { + name: 'DataFactoryResourceGroup' + value: resourceGroup().name + } + { + name: 'DataFactoryName' + value: core.outputs.app.dataFactory + } + ] + } +} + +// Start all ADF triggers +module startTriggers 'fx/hub-initialize.bicep' = { + name: 'Microsoft.FinOpsHubs.StartTriggers' + params: { + app: core.outputs.app + dataFactoryInstances: [ + core.outputs.app.dataFactory // Microsoft.FinOpsHubs + cmExports.outputs.app.dataFactory // Microsoft.CostManagement + ] + identityName: core.outputs.triggerManagerIdentityName + startAllTriggers: true } } @@ -418,16 +383,20 @@ output storageAccountName string = core.outputs.storageAccountName output storageUrlForPowerBI string = core.outputs.storageUrlForPowerBI @description('The resource ID of the Data Explorer cluster.') -output clusterId string = !deployDataExplorer ? '' : dataExplorer.outputs.clusterId +#disable-next-line BCP318 // Null safety warning for conditional resource access +output clusterId string = !useAzureDataExplorer ? '' : analytics.outputs.clusterId @description('The URI of the Data Explorer cluster.') -output clusterUri string = useFabric ? fabricQueryUri : (!deployDataExplorer ? '' : dataExplorer.outputs.clusterUri) +#disable-next-line BCP318 // Null safety warning for conditional resource access +output clusterUri string = useFabric ? fabricQueryUri : (!useAzureDataExplorer ? '' : analytics.outputs.clusterUri) @description('The name of the Data Explorer database used for ingesting data.') -output ingestionDbName string = useFabric ? 'Ingestion' : (!deployDataExplorer ? '' : dataExplorer.outputs.ingestionDbName) +#disable-next-line BCP318 // Null safety warning for conditional resource access +output ingestionDbName string = useFabric || useAzureDataExplorer ? analytics.outputs.ingestionDbName : '' @description('The name of the Data Explorer database used for querying data.') -output hubDbName string = useFabric ? 'Hub' : (!deployDataExplorer ? '' : dataExplorer.outputs.hubDbName) +#disable-next-line BCP318 // Null safety warning for conditional resource access +output hubDbName string = useFabric || useAzureDataExplorer ? analytics.outputs.hubDbName : '' @description('Object ID of the Data Factory managed identity. This will be needed when configuring managed exports.') output managedIdentityId string = core.outputs.principalId diff --git a/src/templates/finops-hub/modules/remoteHub.bicep b/src/templates/finops-hub/modules/remoteHub.bicep deleted file mode 100644 index 41175d497..000000000 --- a/src/templates/finops-hub/modules/remoteHub.bicep +++ /dev/null @@ -1,59 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -import { HubProperties } from 'hub-types.bicep' - - -//============================================================================== -// Parameters -//============================================================================== - -@description('Required. FinOps hub instance properties.') -param hub HubProperties - -@description('Required. Create and store a key for a remote storage account.') -@secure() -param remoteStorageKey string - - -//============================================================================== -// Resources -//============================================================================== - -// App registration -module appRegistration 'hub-app.bicep' = { - name: 'Microsoft.FinOpsHubs.RemoteHub_Register' - params: { - hub: hub - publisher: 'Microsoft FinOps hubs' - namespace: 'Microsoft.FinOpsHubs' - appName: 'RemoteHub' - displayName: 'FinOps hub remote relay' - appVersion: loadTextContent('ftkver.txt') // cSpell:ignore ftkver - features: [ - // TODO: Add pipeline -- 'DataFactory' - 'KeyVault' - 'Storage' - ] - } -} - -// Key Vault secret -module keyVault_secret 'hub-vault.bicep' = { - name: 'keyVault_secret' - params: { - vaultName: appRegistration.outputs.app.keyVault - secretName: '${toLower(appRegistration.outputs.app.hub.name)}-storage-key' - secretValue: remoteStorageKey - secretExpirationInSeconds: 1702648632 - secretNotBeforeInSeconds: 10000 - } -} - - -//============================================================================== -// Outputs -//============================================================================== - -@description('Name of the Key Vault instance.') -output keyVaultName string = appRegistration.outputs.app.keyVault diff --git a/src/templates/finops-hub/modules/scripts/Start-Triggers.ps1 b/src/templates/finops-hub/modules/scripts/Start-Triggers.ps1 deleted file mode 100644 index f462d811e..000000000 --- a/src/templates/finops-hub/modules/scripts/Start-Triggers.ps1 +++ /dev/null @@ -1,65 +0,0 @@ -# Copyright (c) Microsoft Corporation. -# Licensed under the MIT License. - -Param( - [switch] $Stop -) - -# Init outputs -$DeploymentScriptOutputs = @{} - -if (-not $Stop) -{ - Start-Sleep -Seconds 10 -} - -# Loop thru triggers -$env:Triggers.Split('|') ` -| ForEach-Object { - $trigger = $_ - if ($Stop) - { - Write-Output "Stopping trigger $trigger..." - $triggerOutput = Stop-AzDataFactoryV2Trigger ` - -ResourceGroupName $env:DataFactoryResourceGroup ` - -DataFactoryName $env:DataFactoryName ` - -Name $trigger ` - -Force ` - -ErrorAction SilentlyContinue # Ignore errors, since the trigger may not exist - } - else - { - Write-Output "Starting trigger $trigger..." - $triggerOutput = Start-AzDataFactoryV2Trigger ` - -ResourceGroupName $env:DataFactoryResourceGroup ` - -DataFactoryName $env:DataFactoryName ` - -Name $trigger ` - -Force - } - if ($triggerOutput) - { - Write-Output "done..." - } - else - { - Write-Output "failed..." - } - $DeploymentScriptOutputs[$trigger] = $triggerOutput -} - -if ($Stop) -{ - Start-Sleep -Seconds 10 -} - -if (-not [string]::IsNullOrWhiteSpace($env:Pipelines)) -{ - $env:Pipelines.Split('|') ` - | ForEach-Object { - Write-Output "Running the init pipeline..." - Invoke-AzDataFactoryV2Pipeline ` - -ResourceGroupName $env:DataFactoryResourceGroup ` - -DataFactoryName $env:DataFactoryName ` - -PipelineName $_ - } -} diff --git a/src/templates/finops-hub/schemas/README.md b/src/templates/finops-hub/schemas/README.md deleted file mode 100644 index 346cf385c..000000000 --- a/src/templates/finops-hub/schemas/README.md +++ /dev/null @@ -1,8 +0,0 @@ -# 📦 FinOps hub schemas - -These schemas are used to indicate data types for the CSV to parquet conversion. - -- [focuscost_1.0.json](./focuscost_1.0.json) -- [focuscost_1.0-preview(v1).json](./focuscost_1.0-preview(v1).json) - -
From eb6087bbd6b694e43a369b465e4b4ad2d2fafc4e Mon Sep 17 00:00:00 2001 From: Mike Pritchard <20865962+mpritchard2@users.noreply.github.com> Date: Fri, 10 Oct 2025 23:22:27 -0500 Subject: [PATCH 14/69] Fix: Reset-AutomationSchedules incorrectly checking if provided time is at least one hour in the future (#1834) Co-authored-by: msbrett Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- .../Reset-AutomationSchedules.ps1 | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/src/optimization-engine/Reset-AutomationSchedules.ps1 b/src/optimization-engine/Reset-AutomationSchedules.ps1 index 7762f06eb..459d3da0f 100644 --- a/src/optimization-engine/Reset-AutomationSchedules.ps1 +++ b/src/optimization-engine/Reset-AutomationSchedules.ps1 @@ -77,21 +77,26 @@ else { $newBaseTimeStr = Read-Host "Please, enter a new base time for the *weekly* schedules in UTC (YYYY-MM-dd HH:mm:ss). If you want to keep the current one, just press ENTER" if (-not($newBaseTimeStr)) { $newBaseTimeStr = $baseTimeStr + $baseTimeOffset = [DateTimeOffset]::Parse($newBaseTimeStr) } else { + $minAllowedTime = [DateTimeOffset]::UtcNow.AddHours(1) + try { - $newBaseTimeStr += "Z" - $newBaseTime = [DateTime]::Parse($newBaseTimeStr) + $baseTimeOffset = [DateTimeOffset]::Parse($newBaseTimeStr + "Z") } catch { throw "$newBaseTimeStr is an invalid base time. Use the following format: YYYY-MM-dd HH:mm:ss. For example: 1977-09-08 06:14:15" } - if ($newBaseTime -lt (Get-Date).ToUniversalTime().AddHours(-1)) { - throw "$newBaseTimeStr is an invalid base time. It can't be sooner than $((Get-Date).ToUniversalTime().AddHours(-1).ToString('u'))" + + if ($baseTimeOffset -lt $minAllowedTime) { + throw "$($baseTimeOffset.ToString('u')) is an invalid base time. It must be at least 1 hour in the future (after $($minAllowedTime.ToString('u')))" } + + $newBaseTimeStr = $baseTimeOffset.ToString("yyyy-MM-dd HH:mm:ss") + "Z" } -$baseTimeUtc = [DateTime]::Parse($newBaseTimeStr).ToUniversalTime() +$baseTimeUtc = $baseTimeOffset.UtcDateTime if ($newBaseTimeStr -ne $baseTimeStr) { Write-Host "Updating current base schedule to every $($baseTimeUtc.DayOfWeek) at $($baseTimeUtc.TimeOfDay.ToString()) UTC..." -ForegroundColor Green @@ -205,4 +210,4 @@ else Write-Host "Kept current Hybrid Worker option: $hybridWorkerOption" -ForegroundColor Green } -Write-Host "DONE" -ForegroundColor Green \ No newline at end of file +Write-Host "DONE" -ForegroundColor Green From ef381d880f53ef3dc86baff142c4de1cda64d0a0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?H=C3=A9lder=20Pinto?= Date: Sat, 11 Oct 2025 05:49:04 +0100 Subject: [PATCH 15/69] [AOE] Fixes Reservations-related workbooks link to ISF ratio CSV (#1815) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Hélder Pinto --- docs-mslearn/toolkit/changelog.md | 1 + .../views/workbooks/benefits-simulation.json | 16 ++++++++-------- .../views/workbooks/reservations-potential.json | 10 +++++----- .../views/workbooks/reservations-usage.json | 16 ++++++++-------- 4 files changed, 22 insertions(+), 21 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 1593a5f0c..7a370d4d1 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -33,6 +33,7 @@ The following section lists features and enhancements that are currently in deve ### [Optimization engine](optimization-engine/overview.md) - **Fixed** + - Reservations-related workbooks fixed by replacing Instance Size Flexibility ratios CSV vanity URL with actual one to work around Log Analytics externaldata limitation ([#1810](https://github.com/microsoft/finops-toolkit/issues/1810)). - Underutilized disks recommendations were not being generated when customer environment has Premium SSD V2 disks ([#1831](https://github.com/microsoft/finops-toolkit/issues/1831)). ### Bicep Registry module pending updates diff --git a/src/optimization-engine/views/workbooks/benefits-simulation.json b/src/optimization-engine/views/workbooks/benefits-simulation.json index 5e13b7db9..40708a1b7 100644 --- a/src/optimization-engine/views/workbooks/benefits-simulation.json +++ b/src/optimization-engine/views/workbooks/benefits-simulation.json @@ -101,7 +101,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s)['ServiceType'])\r\n| extend SKUName = strcat(VMSize, ' ', ResourceLocation_s)\r\n| project Date_s, QtyHours=todouble(Quantity_s), BillingMeter=MeterName_s, ResourceLocation_s, MeterId=MeterId_g, ResourceId, OnDemandPrice=todouble(iif(todouble(UnitPrice_s) > 0.0, UnitPrice_s, EffectivePrice_s)), SubscriptionName, VMSize, SKUName\r\n| join kind=leftouter ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| extend ISFGroup = strcat(ISFGroup, ' ', ResourceLocation_s)\r\n| extend OnDemandCost = QtyHours * OnDemandPrice\r\n| summarize HourlyCost=sum(OnDemandCost)/24 by bin(todatetime(Date_s), 1d), {GroupBy}", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s)['ServiceType'])\r\n| extend SKUName = strcat(VMSize, ' ', ResourceLocation_s)\r\n| project Date_s, QtyHours=todouble(Quantity_s), BillingMeter=MeterName_s, ResourceLocation_s, MeterId=MeterId_g, ResourceId, OnDemandPrice=todouble(iif(todouble(UnitPrice_s) > 0.0, UnitPrice_s, EffectivePrice_s)), SubscriptionName, VMSize, SKUName\r\n| join kind=leftouter ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| extend ISFGroup = strcat(ISFGroup, ' ', ResourceLocation_s)\r\n| extend OnDemandCost = QtyHours * OnDemandPrice\r\n| summarize HourlyCost=sum(OnDemandCost)/24 by bin(todatetime(Date_s), 1d), {GroupBy}", "size": 1, "aggregation": 3, "title": "Average On-Demand hourly usage (actual cost)", @@ -206,7 +206,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nlet LinuxOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption' and MeterSubCategory_s !endswith \"Windows\"\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet OnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxOnDemandPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend OnDemandPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, OnDemandPrice;\r\nlet LinuxSavingsPlanPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s in ('Savings Plan','SavingsPlan') and MeterSubCategory_s !endswith \"Windows\" and Term_s == iif('{SavingsPlanTerm}' == '3 Years','P3Y','P1Y')\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet SavingsPlanPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s in ('Savings Plan','SavingsPlan') and Term_s == iif('{SavingsPlanTerm}' == '3 Years','P3Y','P1Y')\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxSavingsPlanPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend SavingsPlanPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, SavingsPlanPrice;\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| join kind=inner ( OnDemandPriceSheet ) on $left.MeterId_g == $right.MeterID_g\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s)['ServiceType'])\r\n| extend ResourceLocation_s=tolower(ResourceLocation_s)\r\n| extend SKUName = strcat(VMSize, ' ', ResourceLocation_s)\r\n| project Date_s, QtyHours=todouble(Quantity_s), BillingMeter=MeterName_s, ResourceLocation_s, MeterId=MeterId_g, ResourceId, OnDemandPrice, SubscriptionName, VMSize, SKUName\r\n| join kind=leftouter ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| extend ISFGroup = strcat(ISFGroup, ' ', ResourceLocation_s)\r\n| join kind=inner ( SavingsPlanPriceSheet ) on $left.MeterId == $right.MeterID_g\r\n| extend SavingsPlanCost = QtyHours * SavingsPlanPrice\r\n| summarize HourlyCost=sum(SavingsPlanCost)/24 by todatetime(Date_s), {GroupBy}", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nlet LinuxOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption' and MeterSubCategory_s !endswith \"Windows\"\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet OnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxOnDemandPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend OnDemandPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, OnDemandPrice;\r\nlet LinuxSavingsPlanPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s in ('Savings Plan','SavingsPlan') and MeterSubCategory_s !endswith \"Windows\" and Term_s == iif('{SavingsPlanTerm}' == '3 Years','P3Y','P1Y')\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet SavingsPlanPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s in ('Savings Plan','SavingsPlan') and Term_s == iif('{SavingsPlanTerm}' == '3 Years','P3Y','P1Y')\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxSavingsPlanPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend SavingsPlanPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, SavingsPlanPrice;\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| join kind=inner ( OnDemandPriceSheet ) on $left.MeterId_g == $right.MeterID_g\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s)['ServiceType'])\r\n| extend ResourceLocation_s=tolower(ResourceLocation_s)\r\n| extend SKUName = strcat(VMSize, ' ', ResourceLocation_s)\r\n| project Date_s, QtyHours=todouble(Quantity_s), BillingMeter=MeterName_s, ResourceLocation_s, MeterId=MeterId_g, ResourceId, OnDemandPrice, SubscriptionName, VMSize, SKUName\r\n| join kind=leftouter ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| extend ISFGroup = strcat(ISFGroup, ' ', ResourceLocation_s)\r\n| join kind=inner ( SavingsPlanPriceSheet ) on $left.MeterId == $right.MeterID_g\r\n| extend SavingsPlanCost = QtyHours * SavingsPlanPrice\r\n| summarize HourlyCost=sum(SavingsPlanCost)/24 by todatetime(Date_s), {GroupBy}", "size": 1, "aggregation": 3, "title": "Average On-Demand hourly usage (Savings Plan prices)", @@ -406,7 +406,7 @@ "label": "Size", "type": 2, "isRequired": true, - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| extend ServiceType = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| join kind=inner ( ISFGroups ) on $left.ServiceType == $right.ArmSKUName\r\n| distinct ServiceType\r\n| order by ServiceType asc", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| extend ServiceType = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| join kind=inner ( ISFGroups ) on $left.ServiceType == $right.ArmSKUName\r\n| distinct ServiceType\r\n| order by ServiceType asc", "typeSettings": { "additionalResourceOptions": [], "showDefault": false @@ -426,7 +426,7 @@ "label": "Region", "type": 2, "isRequired": true, - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| extend ServiceType = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| join kind=inner ( ISFGroups ) on $left.ServiceType == $right.ArmSKUName\r\n| where ServiceType == '{VMSize}'\r\n| extend ResourceLocation_s=tolower(ResourceLocation_s)\r\n| distinct ResourceLocation_s\r\n| order by ResourceLocation_s asc", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| extend ServiceType = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| join kind=inner ( ISFGroups ) on $left.ServiceType == $right.ArmSKUName\r\n| where ServiceType == '{VMSize}'\r\n| extend ResourceLocation_s=tolower(ResourceLocation_s)\r\n| distinct ResourceLocation_s\r\n| order by ResourceLocation_s asc", "typeSettings": { "additionalResourceOptions": [], "showDefault": false @@ -463,7 +463,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = materialize(externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName));\r\nlet SelectedISFGroup = toscalar(ISFGroups | where ArmSKUName == '{VMSize}' | project ISFGroup);\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| extend ServiceType = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| join kind=inner ( ISFGroups ) on $left.ServiceType == $right.ArmSKUName\r\n| project Date_s, QtyHours=todouble(Quantity_s), ArmRegion=tolower(ResourceLocation_s), MeterId=MeterId_g, ResourceId, ServiceType, SubscriptionName, ISFGroup\r\n| where iif('{UseISF}' == 'Yes', ISFGroup == SelectedISFGroup, ServiceType == '{VMSize}')\r\n| where ArmRegion == '{VMRegion}'\r\n| extend ReservationSKU = strcat(ServiceType, ' ', ArmRegion)\r\n| summarize HourlyVMs=sum(QtyHours/24) by bin(todatetime(Date_s), 1d), ReservationSKU", + "query": "let ISFGroups = materialize(externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName));\r\nlet SelectedISFGroup = toscalar(ISFGroups | where ArmSKUName == '{VMSize}' | project ISFGroup);\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| extend ServiceType = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| join kind=inner ( ISFGroups ) on $left.ServiceType == $right.ArmSKUName\r\n| project Date_s, QtyHours=todouble(Quantity_s), ArmRegion=tolower(ResourceLocation_s), MeterId=MeterId_g, ResourceId, ServiceType, SubscriptionName, ISFGroup\r\n| where iif('{UseISF}' == 'Yes', ISFGroup == SelectedISFGroup, ServiceType == '{VMSize}')\r\n| where ArmRegion == '{VMRegion}'\r\n| extend ReservationSKU = strcat(ServiceType, ' ', ArmRegion)\r\n| summarize HourlyVMs=sum(QtyHours/24) by bin(todatetime(Date_s), 1d), ReservationSKU", "size": 1, "aggregation": 3, "title": "Average On-Demand usage (VMs #)", @@ -489,7 +489,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = materialize(externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName));\r\nlet SelectedISFGroup = toscalar(ISFGroups | where ArmSKUName == '{VMSize}' | project ISFGroup);\r\nlet ISFRatio = toscalar(ISFGroups | where ArmSKUName == '{VMSize}' | project Ratio);\r\nlet TermDivider = iif('{ReservationTerm}' == '3 Years', 3, 1);\r\nlet VMQuantity = {VMQuantity};\r\nlet LinuxOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption' and MeterSubCategory_s !endswith \"Windows\"\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet OnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxOnDemandPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend OnDemandPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, OnDemandPrice;\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| extend ServiceType = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| join kind=inner ( OnDemandPriceSheet ) on $left.MeterId_g == $right.MeterID_g\r\n| project Date_s, QtyHours=todouble(Quantity_s), ArmRegion=tolower(ResourceLocation_s), ServiceType, MeterId=MeterId_g, ResourceId, OnDemandPrice, SubscriptionName\r\n| join kind=inner ( ISFGroups ) on $left.ServiceType == $right.ArmSKUName\r\n| where iif('{UseISF}' == 'Yes', ISFGroup == SelectedISFGroup, ServiceType == '{VMSize}')\r\n| where ArmRegion == '{VMRegion}'\r\n| join kind=inner ( \r\n AzureOptimizationReservationsPriceV1_CL\r\n | where TimeGenerated > ago(14d)\r\n | where serviceName_s == 'Virtual Machines' and reservationTerm_s == '{ReservationTerm}'\r\n | extend ReservationPrice = todouble(replace_string(unitPrice_s,',','.'))/TermDivider/12/730\r\n | summarize arg_max(TimeGenerated, ReservationPrice) by ArmRegion=tolower(armRegionName_s), ServiceType=tolower(armSkuName_s)\r\n) on ArmRegion and ServiceType\r\n| order by todatetime(Date_s)\r\n| extend RIConsumed = row_cumsum(QtyHours * Ratio / ISFRatio / 24, prev(Date_s) != Date_s)\r\n| extend CutOff = iif(prev(Date_s) != Date_s or prev(RIConsumed, 1, 0) < VMQuantity, false, true)\r\n| where not(CutOff)\r\n| extend RICost = QtyHours * ReservationPrice\r\n| extend OnDemandCost = QtyHours * OnDemandPrice\r\n| summarize DailyRICost=sum(RICost), DailyOnDemandCost=sum(OnDemandCost), MaxRIConsumed=max(RIConsumed) by Date_s\r\n| extend RIAllocationPercentage = iif(MaxRIConsumed < VMQuantity, 1.0, 1 - (MaxRIConsumed - VMQuantity) / MaxRIConsumed)\r\n| project todatetime(Date_s), SavedAmount = (DailyOnDemandCost-DailyRICost) * RIAllocationPercentage\r\n", + "query": "let ISFGroups = materialize(externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName));\r\nlet SelectedISFGroup = toscalar(ISFGroups | where ArmSKUName == '{VMSize}' | project ISFGroup);\r\nlet ISFRatio = toscalar(ISFGroups | where ArmSKUName == '{VMSize}' | project Ratio);\r\nlet TermDivider = iif('{ReservationTerm}' == '3 Years', 3, 1);\r\nlet VMQuantity = {VMQuantity};\r\nlet LinuxOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption' and MeterSubCategory_s !endswith \"Windows\"\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet OnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxOnDemandPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend OnDemandPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, OnDemandPrice;\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| extend ServiceType = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| join kind=inner ( OnDemandPriceSheet ) on $left.MeterId_g == $right.MeterID_g\r\n| project Date_s, QtyHours=todouble(Quantity_s), ArmRegion=tolower(ResourceLocation_s), ServiceType, MeterId=MeterId_g, ResourceId, OnDemandPrice, SubscriptionName\r\n| join kind=inner ( ISFGroups ) on $left.ServiceType == $right.ArmSKUName\r\n| where iif('{UseISF}' == 'Yes', ISFGroup == SelectedISFGroup, ServiceType == '{VMSize}')\r\n| where ArmRegion == '{VMRegion}'\r\n| join kind=inner ( \r\n AzureOptimizationReservationsPriceV1_CL\r\n | where TimeGenerated > ago(14d)\r\n | where serviceName_s == 'Virtual Machines' and reservationTerm_s == '{ReservationTerm}'\r\n | extend ReservationPrice = todouble(replace_string(unitPrice_s,',','.'))/TermDivider/12/730\r\n | summarize arg_max(TimeGenerated, ReservationPrice) by ArmRegion=tolower(armRegionName_s), ServiceType=tolower(armSkuName_s)\r\n) on ArmRegion and ServiceType\r\n| order by todatetime(Date_s)\r\n| extend RIConsumed = row_cumsum(QtyHours * Ratio / ISFRatio / 24, prev(Date_s) != Date_s)\r\n| extend CutOff = iif(prev(Date_s) != Date_s or prev(RIConsumed, 1, 0) < VMQuantity, false, true)\r\n| where not(CutOff)\r\n| extend RICost = QtyHours * ReservationPrice\r\n| extend OnDemandCost = QtyHours * OnDemandPrice\r\n| summarize DailyRICost=sum(RICost), DailyOnDemandCost=sum(OnDemandCost), MaxRIConsumed=max(RIConsumed) by Date_s\r\n| extend RIAllocationPercentage = iif(MaxRIConsumed < VMQuantity, 1.0, 1 - (MaxRIConsumed - VMQuantity) / MaxRIConsumed)\r\n| project todatetime(Date_s), SavedAmount = (DailyOnDemandCost-DailyRICost) * RIAllocationPercentage\r\n", "size": 1, "title": "Estimated savings (in your billing currency)", "queryType": 0, @@ -518,7 +518,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = materialize(externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName));\r\nlet SelectedISFGroup = toscalar(ISFGroups | where ArmSKUName == '{VMSize}' | project ISFGroup);\r\nlet ISFRatio = toscalar(ISFGroups | where ArmSKUName == '{VMSize}' | project Ratio);\r\nlet TermDivider = iif('{ReservationTerm}' == '3 Years', 3, 1);\r\nlet VMQuantity = {VMQuantity};\r\nlet LinuxOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption' and MeterSubCategory_s !endswith \"Windows\"\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet OnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxOnDemandPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend OnDemandPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, OnDemandPrice;\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| extend ServiceType = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| join kind=inner ( OnDemandPriceSheet ) on $left.MeterId_g == $right.MeterID_g\r\n| project Date_s, QtyHours=todouble(Quantity_s), ArmRegion=tolower(ResourceLocation_s), ServiceType, MeterId=MeterId_g, ResourceId, OnDemandPrice, SubscriptionName\r\n| join kind=inner ( ISFGroups ) on $left.ServiceType == $right.ArmSKUName\r\n| where iif('{UseISF}' == 'Yes', ISFGroup == SelectedISFGroup, ServiceType == '{VMSize}')\r\n| where ArmRegion == '{VMRegion}'\r\n| join kind=inner ( \r\n AzureOptimizationReservationsPriceV1_CL\r\n | where TimeGenerated > ago(14d)\r\n | where serviceName_s == 'Virtual Machines' and reservationTerm_s == '{ReservationTerm}'\r\n | extend ReservationPrice = todouble(replace_string(unitPrice_s,',','.'))/TermDivider/12/730\r\n | summarize arg_max(TimeGenerated, ReservationPrice) by ArmRegion=tolower(armRegionName_s), ServiceType=tolower(armSkuName_s)\r\n) on ArmRegion and ServiceType\r\n| order by todatetime(Date_s)\r\n| extend RIConsumed = row_cumsum(QtyHours * Ratio / ISFRatio / 24, prev(Date_s) != Date_s)\r\n| extend CutOff = iif(prev(Date_s) != Date_s or prev(RIConsumed, 1, 0) < VMQuantity, false, true)\r\n| where not(CutOff)\r\n| extend RICost = QtyHours * ReservationPrice\r\n| extend OnDemandCost = QtyHours * OnDemandPrice\r\n| summarize DailyRICost=sum(RICost), DailyOnDemandCost=sum(OnDemandCost), MaxRIConsumed=max(RIConsumed) by Date_s\r\n| extend RIAllocationPercentage = iif(MaxRIConsumed < VMQuantity, 1.0, 1 - (MaxRIConsumed - VMQuantity) / MaxRIConsumed)\r\n| project Date_s, SavedAmount = (DailyOnDemandCost-DailyRICost) * RIAllocationPercentage, AmountWouldSpendOnDemand = DailyOnDemandCost * RIAllocationPercentage\r\n| extend SavedPercentage = SavedAmount / AmountWouldSpendOnDemand\r\n| project SavedPercentage, todatetime(Date_s)", + "query": "let ISFGroups = materialize(externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName));\r\nlet SelectedISFGroup = toscalar(ISFGroups | where ArmSKUName == '{VMSize}' | project ISFGroup);\r\nlet ISFRatio = toscalar(ISFGroups | where ArmSKUName == '{VMSize}' | project Ratio);\r\nlet TermDivider = iif('{ReservationTerm}' == '3 Years', 3, 1);\r\nlet VMQuantity = {VMQuantity};\r\nlet LinuxOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption' and MeterSubCategory_s !endswith \"Windows\"\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet OnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxOnDemandPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend OnDemandPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, OnDemandPrice;\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| extend ServiceType = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| join kind=inner ( OnDemandPriceSheet ) on $left.MeterId_g == $right.MeterID_g\r\n| project Date_s, QtyHours=todouble(Quantity_s), ArmRegion=tolower(ResourceLocation_s), ServiceType, MeterId=MeterId_g, ResourceId, OnDemandPrice, SubscriptionName\r\n| join kind=inner ( ISFGroups ) on $left.ServiceType == $right.ArmSKUName\r\n| where iif('{UseISF}' == 'Yes', ISFGroup == SelectedISFGroup, ServiceType == '{VMSize}')\r\n| where ArmRegion == '{VMRegion}'\r\n| join kind=inner ( \r\n AzureOptimizationReservationsPriceV1_CL\r\n | where TimeGenerated > ago(14d)\r\n | where serviceName_s == 'Virtual Machines' and reservationTerm_s == '{ReservationTerm}'\r\n | extend ReservationPrice = todouble(replace_string(unitPrice_s,',','.'))/TermDivider/12/730\r\n | summarize arg_max(TimeGenerated, ReservationPrice) by ArmRegion=tolower(armRegionName_s), ServiceType=tolower(armSkuName_s)\r\n) on ArmRegion and ServiceType\r\n| order by todatetime(Date_s)\r\n| extend RIConsumed = row_cumsum(QtyHours * Ratio / ISFRatio / 24, prev(Date_s) != Date_s)\r\n| extend CutOff = iif(prev(Date_s) != Date_s or prev(RIConsumed, 1, 0) < VMQuantity, false, true)\r\n| where not(CutOff)\r\n| extend RICost = QtyHours * ReservationPrice\r\n| extend OnDemandCost = QtyHours * OnDemandPrice\r\n| summarize DailyRICost=sum(RICost), DailyOnDemandCost=sum(OnDemandCost), MaxRIConsumed=max(RIConsumed) by Date_s\r\n| extend RIAllocationPercentage = iif(MaxRIConsumed < VMQuantity, 1.0, 1 - (MaxRIConsumed - VMQuantity) / MaxRIConsumed)\r\n| project Date_s, SavedAmount = (DailyOnDemandCost-DailyRICost) * RIAllocationPercentage, AmountWouldSpendOnDemand = DailyOnDemandCost * RIAllocationPercentage\r\n| extend SavedPercentage = SavedAmount / AmountWouldSpendOnDemand\r\n| project SavedPercentage, todatetime(Date_s)", "size": 1, "aggregation": 3, "title": "Estimated savings (percentage)", @@ -557,7 +557,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = materialize(externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName));\r\nlet SelectedISFGroup = toscalar(ISFGroups | where ArmSKUName == '{VMSize}' | project ISFGroup);\r\nlet ISFRatio = toscalar(ISFGroups | where ArmSKUName == '{VMSize}' | project Ratio);\r\nlet TermDivider = iif('{ReservationTerm}' == '3 Years', 3, 1);\r\nlet VMQuantity = {VMQuantity};\r\nlet LinuxOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption' and MeterSubCategory_s !endswith \"Windows\"\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet OnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxOnDemandPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend OnDemandPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, OnDemandPrice;\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| extend ServiceType = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| join kind=inner ( OnDemandPriceSheet ) on $left.MeterId_g == $right.MeterID_g\r\n| project Date_s, QtyHours=todouble(Quantity_s), ArmRegion=tolower(ResourceLocation_s), ServiceType, MeterId=MeterId_g, ResourceId, OnDemandPrice, SubscriptionName\r\n| join kind=inner ( ISFGroups ) on $left.ServiceType == $right.ArmSKUName\r\n| where iif('{UseISF}' == 'Yes', ISFGroup == SelectedISFGroup, ServiceType == '{VMSize}')\r\n| where ArmRegion == '{VMRegion}'\r\n| join kind=inner ( \r\n AzureOptimizationReservationsPriceV1_CL\r\n | where TimeGenerated > ago(14d)\r\n | where serviceName_s == 'Virtual Machines' and reservationTerm_s == '{ReservationTerm}'\r\n | extend ReservationPrice = todouble(replace_string(unitPrice_s,',','.'))/TermDivider/12/730\r\n | summarize arg_max(TimeGenerated, ReservationPrice) by ArmRegion=tolower(armRegionName_s), ServiceType=tolower(armSkuName_s)\r\n) on ArmRegion and ServiceType\r\n| order by todatetime(Date_s)\r\n| extend RIConsumed = row_cumsum(QtyHours * Ratio / ISFRatio / 24, prev(Date_s) != Date_s)\r\n| extend CutOff = iif(prev(Date_s) != Date_s or prev(RIConsumed, 1, 0) < VMQuantity, false, true)\r\n| where not(CutOff)\r\n| summarize MaxRIConsumed=max(RIConsumed) by todatetime(Date_s)\r\n| extend RIUsagePercentage = iif(MaxRIConsumed >= VMQuantity, 1.0, MaxRIConsumed / VMQuantity)\r\n| project-away MaxRIConsumed", + "query": "let ISFGroups = materialize(externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName));\r\nlet SelectedISFGroup = toscalar(ISFGroups | where ArmSKUName == '{VMSize}' | project ISFGroup);\r\nlet ISFRatio = toscalar(ISFGroups | where ArmSKUName == '{VMSize}' | project Ratio);\r\nlet TermDivider = iif('{ReservationTerm}' == '3 Years', 3, 1);\r\nlet VMQuantity = {VMQuantity};\r\nlet LinuxOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption' and MeterSubCategory_s !endswith \"Windows\"\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet OnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxOnDemandPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend OnDemandPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, OnDemandPrice;\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend SubscriptionName = iif(isnotempty(SubscriptionName_s), SubscriptionName_s, SubscriptionName)\r\n| where SubscriptionName in ({Subscriptions})\r\n| extend ServiceType = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| join kind=inner ( OnDemandPriceSheet ) on $left.MeterId_g == $right.MeterID_g\r\n| project Date_s, QtyHours=todouble(Quantity_s), ArmRegion=tolower(ResourceLocation_s), ServiceType, MeterId=MeterId_g, ResourceId, OnDemandPrice, SubscriptionName\r\n| join kind=inner ( ISFGroups ) on $left.ServiceType == $right.ArmSKUName\r\n| where iif('{UseISF}' == 'Yes', ISFGroup == SelectedISFGroup, ServiceType == '{VMSize}')\r\n| where ArmRegion == '{VMRegion}'\r\n| join kind=inner ( \r\n AzureOptimizationReservationsPriceV1_CL\r\n | where TimeGenerated > ago(14d)\r\n | where serviceName_s == 'Virtual Machines' and reservationTerm_s == '{ReservationTerm}'\r\n | extend ReservationPrice = todouble(replace_string(unitPrice_s,',','.'))/TermDivider/12/730\r\n | summarize arg_max(TimeGenerated, ReservationPrice) by ArmRegion=tolower(armRegionName_s), ServiceType=tolower(armSkuName_s)\r\n) on ArmRegion and ServiceType\r\n| order by todatetime(Date_s)\r\n| extend RIConsumed = row_cumsum(QtyHours * Ratio / ISFRatio / 24, prev(Date_s) != Date_s)\r\n| extend CutOff = iif(prev(Date_s) != Date_s or prev(RIConsumed, 1, 0) < VMQuantity, false, true)\r\n| where not(CutOff)\r\n| summarize MaxRIConsumed=max(RIConsumed) by todatetime(Date_s)\r\n| extend RIUsagePercentage = iif(MaxRIConsumed >= VMQuantity, 1.0, MaxRIConsumed / VMQuantity)\r\n| project-away MaxRIConsumed", "size": 1, "aggregation": 3, "title": "Estimated efficiency", diff --git a/src/optimization-engine/views/workbooks/reservations-potential.json b/src/optimization-engine/views/workbooks/reservations-potential.json index 5c787f54a..495ed1deb 100644 --- a/src/optimization-engine/views/workbooks/reservations-potential.json +++ b/src/optimization-engine/views/workbooks/reservations-potential.json @@ -89,7 +89,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s)['ServiceType'])\r\n| extend SKUName = strcat(VMSize, ' ', ResourceLocation_s)\r\n| project Date_s, QtyHours=todouble(Quantity_s), BillingMeter=MeterName_s, ResourceLocation_s, MeterId=MeterId_g, ResourceId, OnDemandPrice=todouble(iif(todouble(UnitPrice_s) > 0.0, UnitPrice_s, EffectivePrice_s)), SubscriptionName, VMSize, SKUName\r\n| join kind=leftouter ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| extend ISFGroup = strcat(ISFGroup, ' ', ResourceLocation_s)\r\n| extend OnDemandCost = QtyHours * OnDemandPrice\r\n| summarize DailyCost=sum(OnDemandCost) by bin(todatetime(Date_s), 1d), iif(\"{UseISF}\" == \"Yes\", ISFGroup, SKUName)\r\n| order by DailyCost", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where todatetime(Date_s) > todatetime('{LookbackPeriod:startISO}') and todatetime(Date_s) < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and MeterCategory_s == 'Virtual Machines'\r\n| join kind=leftouter (\r\n AzureOptimizationResourceContainersV1_CL\r\n | where TimeGenerated > ago(2d)\r\n | where ContainerType_s == 'microsoft.resources/subscriptions'\r\n | distinct SubscriptionId=SubscriptionGuid_g, SubscriptionName=ContainerName_s, Cloud=Cloud_s\r\n) on SubscriptionId\r\n| where Cloud == 'AzureCloud'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s)['ServiceType'])\r\n| extend SKUName = strcat(VMSize, ' ', ResourceLocation_s)\r\n| project Date_s, QtyHours=todouble(Quantity_s), BillingMeter=MeterName_s, ResourceLocation_s, MeterId=MeterId_g, ResourceId, OnDemandPrice=todouble(iif(todouble(UnitPrice_s) > 0.0, UnitPrice_s, EffectivePrice_s)), SubscriptionName, VMSize, SKUName\r\n| join kind=leftouter ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| extend ISFGroup = strcat(ISFGroup, ' ', ResourceLocation_s)\r\n| extend OnDemandCost = QtyHours * OnDemandPrice\r\n| summarize DailyCost=sum(OnDemandCost) by bin(todatetime(Date_s), 1d), iif(\"{UseISF}\" == \"Yes\", ISFGroup, SKUName)\r\n| order by DailyCost", "size": 0, "title": "Average on-demand (PAYG) daily consumption (actual cost - Virtual Machines only)", "queryType": 0, @@ -473,7 +473,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and ChargeType_s == 'Usage' and ResourceLocation_s !startswith 'china'\r\n| where ConsumedService_s in ('Microsoft.Compute','Microsoft.ClassicCompute','Microsoft.Batch','Microsoft.MachineLearningServices','Microsoft.Kusto')\r\n| where MeterCategory_s != 'Virtual Machines Licenses'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend VMSize=tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| extend ResourceLocation_s = tolower(ResourceLocation_s)\r\n| where isnotempty(VMSize)\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| summarize UsedQuantity = sum(todouble(Quantity_s)) by ResourceId, Date_s, VMSize, ISFGroup, Ratio, ResourceLocation_s\r\n| summarize RIPotential=sum(UsedQuantity/24*Ratio), AvgSizeUsageHours=avg(UsedQuantity) by Date_s, ISFGroup, ResourceLocation_s\r\n| summarize RIPotential=round(avg(RIPotential),1), AvgSizeUsageHours=round(avg(AvgSizeUsageHours)) by ISFGroup, ResourceLocation_s\r\n| extend Fragmentation = case(AvgSizeUsageHours >= 24.0, 0.0, AvgSizeUsageHours >= 18.0 and AvgSizeUsageHours < 24.0, 0.25, AvgSizeUsageHours >= 12.0 and AvgSizeUsageHours < 18.0, 0.5, AvgSizeUsageHours >= 6.0 and AvgSizeUsageHours < 12.0, 0.75, 1.0)\r\n| project-reorder ISFGroup, ResourceLocation_s, RIPotential, Fragmentation\r\n| order by Fragmentation asc, RIPotential desc", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and ChargeType_s == 'Usage' and ResourceLocation_s !startswith 'china'\r\n| where ConsumedService_s in ('Microsoft.Compute','Microsoft.ClassicCompute','Microsoft.Batch','Microsoft.MachineLearningServices','Microsoft.Kusto')\r\n| where MeterCategory_s != 'Virtual Machines Licenses'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend VMSize=tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| extend ResourceLocation_s = tolower(ResourceLocation_s)\r\n| where isnotempty(VMSize)\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| summarize UsedQuantity = sum(todouble(Quantity_s)) by ResourceId, Date_s, VMSize, ISFGroup, Ratio, ResourceLocation_s\r\n| summarize RIPotential=sum(UsedQuantity/24*Ratio), AvgSizeUsageHours=avg(UsedQuantity) by Date_s, ISFGroup, ResourceLocation_s\r\n| summarize RIPotential=round(avg(RIPotential),1), AvgSizeUsageHours=round(avg(AvgSizeUsageHours)) by ISFGroup, ResourceLocation_s\r\n| extend Fragmentation = case(AvgSizeUsageHours >= 24.0, 0.0, AvgSizeUsageHours >= 18.0 and AvgSizeUsageHours < 24.0, 0.25, AvgSizeUsageHours >= 12.0 and AvgSizeUsageHours < 18.0, 0.5, AvgSizeUsageHours >= 6.0 and AvgSizeUsageHours < 12.0, 0.75, 1.0)\r\n| project-reorder ISFGroup, ResourceLocation_s, RIPotential, Fragmentation\r\n| order by Fragmentation asc, RIPotential desc", "size": 0, "title": "On-demand ISF group usage and RI potential/fragmentation (click on a line for more details)", "exportedParameters": [ @@ -597,7 +597,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and ChargeType_s == 'Usage'\r\n| where ConsumedService_s in ('Microsoft.Compute','Microsoft.ClassicCompute','Microsoft.Batch','Microsoft.MachineLearningServices','Microsoft.Kusto')\r\n| where MeterCategory_s != 'Virtual Machines Licenses'\r\n| extend VMSize=tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| extend ResourceLocation_s = tolower(ResourceLocation_s)\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| where ISFGroup == '{ISFGroup}' and ResourceLocation_s =~ '{Location}'\r\n| summarize UsedQuantity = round(sum(todouble(Quantity_s)/24*Ratio)) by todatetime(Date_s)\r\n| extend RIPotential = {RIPotential}\r\n| render timechart", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and ChargeType_s == 'Usage'\r\n| where ConsumedService_s in ('Microsoft.Compute','Microsoft.ClassicCompute','Microsoft.Batch','Microsoft.MachineLearningServices','Microsoft.Kusto')\r\n| where MeterCategory_s != 'Virtual Machines Licenses'\r\n| extend VMSize=tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| extend ResourceLocation_s = tolower(ResourceLocation_s)\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| where ISFGroup == '{ISFGroup}' and ResourceLocation_s =~ '{Location}'\r\n| summarize UsedQuantity = round(sum(todouble(Quantity_s)/24*Ratio)) by todatetime(Date_s)\r\n| extend RIPotential = {RIPotential}\r\n| render timechart", "size": 0, "aggregation": 3, "title": "Instance count for selected ISF Group/location (click on a line in the table at the left)", @@ -623,7 +623,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and ChargeType_s == 'Usage'\r\n| where ConsumedService_s in ('Microsoft.Compute','Microsoft.ClassicCompute','Microsoft.Batch','Microsoft.MachineLearningServices','Microsoft.Kusto')\r\n| where MeterCategory_s != 'Virtual Machines Licenses'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend VMSize=tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| extend ResourceLocation_s = tolower(ResourceLocation_s)\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| where ISFGroup == '{ISFGroup}' and ResourceLocation_s =~ '{Location}'\r\n| summarize UsedQuantity = sum(todouble(Quantity_s)) by Date_s, ResourceId, SubscriptionId, VMSize, Ratio\r\n| summarize AvgUsedQuantity = round(avg(UsedQuantity),1) by ResourceId, SubscriptionId, VMSize, Ratio\r\n| join kind=leftouter ( AzureOptimizationResourceContainersV1_CL | where TimeGenerated > ago(1d) and ContainerType_s =~ 'microsoft.resources/subscriptions' | project SubscriptionName=ContainerName_s, SubscriptionId=SubscriptionGuid_g) on SubscriptionId\r\n| project-away SubscriptionId*\r\n| project-reorder ResourceId, SubscriptionName\r\n| extend AvgUsedVMs = AvgUsedQuantity / 24\r\n| order by ResourceId asc", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and ChargeType_s == 'Usage'\r\n| where ConsumedService_s in ('Microsoft.Compute','Microsoft.ClassicCompute','Microsoft.Batch','Microsoft.MachineLearningServices','Microsoft.Kusto')\r\n| where MeterCategory_s != 'Virtual Machines Licenses'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend VMSize=tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| extend ResourceLocation_s = tolower(ResourceLocation_s)\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| where ISFGroup == '{ISFGroup}' and ResourceLocation_s =~ '{Location}'\r\n| summarize UsedQuantity = sum(todouble(Quantity_s)) by Date_s, ResourceId, SubscriptionId, VMSize, Ratio\r\n| summarize AvgUsedQuantity = round(avg(UsedQuantity),1) by ResourceId, SubscriptionId, VMSize, Ratio\r\n| join kind=leftouter ( AzureOptimizationResourceContainersV1_CL | where TimeGenerated > ago(1d) and ContainerType_s =~ 'microsoft.resources/subscriptions' | project SubscriptionName=ContainerName_s, SubscriptionId=SubscriptionGuid_g) on SubscriptionId\r\n| project-away SubscriptionId*\r\n| project-reorder ResourceId, SubscriptionName\r\n| extend AvgUsedVMs = AvgUsedQuantity / 24\r\n| order by ResourceId asc", "size": 1, "title": "Daily on-demand usage for selected ISF group/location by resource (click on a line in the table above)", "showExportToExcel": true, @@ -717,7 +717,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and ChargeType_s == 'Usage'\r\n| where ConsumedService_s in ('Microsoft.Compute','Microsoft.ClassicCompute','Microsoft.Batch','Microsoft.MachineLearningServices','Microsoft.Kusto')\r\n| where MeterCategory_s != 'Virtual Machines Licenses'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend VMSize=tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| extend ResourceLocation_s = tolower(ResourceLocation_s)\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| where ISFGroup == '{ISFGroup}' and ResourceLocation_s =~ '{Location}'\r\n| extend Tags_s = iif(Tags_s startswith \"{\", Tags_s, strcat(\"{\", Tags_s, \"}\"))\r\n| extend AggregatorTag = tostring(parse_json(Tags_s)['{AggregatorTag}'])\r\n| summarize UsedQuantity = sum(todouble(Quantity_s)) by Date_s, AggregatorTag, SubscriptionId, VMSize, Ratio\r\n| summarize AvgUsedQuantity = round(avg(UsedQuantity),1) by AggregatorTag, SubscriptionId, VMSize, Ratio\r\n| join kind=leftouter ( AzureOptimizationResourceContainersV1_CL | where TimeGenerated > ago(1d) and ContainerType_s =~ 'microsoft.resources/subscriptions' | project SubscriptionName=ContainerName_s, SubscriptionId=SubscriptionGuid_g) on SubscriptionId\r\n| project-away SubscriptionId*\r\n| project-reorder AggregatorTag, SubscriptionName\r\n| extend AvgUsedVMs = AvgUsedQuantity / 24\r\n| order by AggregatorTag asc", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and ChargeType_s == 'Usage'\r\n| where ConsumedService_s in ('Microsoft.Compute','Microsoft.ClassicCompute','Microsoft.Batch','Microsoft.MachineLearningServices','Microsoft.Kusto')\r\n| where MeterCategory_s != 'Virtual Machines Licenses'\r\n| extend PricingModel = iif(isnotempty(PricingModel_s), PricingModel_s, iif(isnotempty(ReservationName_s), 'Reservation', iif(MeterName_s endswith 'Spot', 'Spot', iif(isnotempty(benefitName_s), 'SavingsPlan', 'OnDemand'))))\r\n| where PricingModel == 'OnDemand'\r\n| extend VMSize=tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| extend ResourceLocation_s = tolower(ResourceLocation_s)\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| where ISFGroup == '{ISFGroup}' and ResourceLocation_s =~ '{Location}'\r\n| extend Tags_s = iif(Tags_s startswith \"{\", Tags_s, strcat(\"{\", Tags_s, \"}\"))\r\n| extend AggregatorTag = tostring(parse_json(Tags_s)['{AggregatorTag}'])\r\n| summarize UsedQuantity = sum(todouble(Quantity_s)) by Date_s, AggregatorTag, SubscriptionId, VMSize, Ratio\r\n| summarize AvgUsedQuantity = round(avg(UsedQuantity),1) by AggregatorTag, SubscriptionId, VMSize, Ratio\r\n| join kind=leftouter ( AzureOptimizationResourceContainersV1_CL | where TimeGenerated > ago(1d) and ContainerType_s =~ 'microsoft.resources/subscriptions' | project SubscriptionName=ContainerName_s, SubscriptionId=SubscriptionGuid_g) on SubscriptionId\r\n| project-away SubscriptionId*\r\n| project-reorder AggregatorTag, SubscriptionName\r\n| extend AvgUsedVMs = AvgUsedQuantity / 24\r\n| order by AggregatorTag asc", "size": 1, "title": "Daily on-demand usage for selected ISF group/location by tag (click on a line in the table above)", "showExportToExcel": true, diff --git a/src/optimization-engine/views/workbooks/reservations-usage.json b/src/optimization-engine/views/workbooks/reservations-usage.json index 575512cef..828bec869 100644 --- a/src/optimization-engine/views/workbooks/reservations-usage.json +++ b/src/optimization-engine/views/workbooks/reservations-usage.json @@ -536,7 +536,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nlet LinuxOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption' and MeterSubCategory_s !endswith \"Windows\"\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet VMOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxOnDemandPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend OnDemandUnitPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, OnDemandUnitPrice;\r\nlet OnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s != 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| summarize OnDemandUnitPrice = max(todouble(UnitPrice_s)) by MeterID_g\r\n| distinct MeterID_g, OnDemandUnitPrice\r\n| union (VMOnDemandPriceSheet)\r\n| summarize OnDemandUnitPrice=min(OnDemandUnitPrice) by MeterID_g;\r\nlet ReservationPricesheet = AzureOptimizationReservationsPriceV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| extend Term_s = iif(reservationTerm_s == '3 Years', 'P3Y', 'P1Y')\r\n| extend TermDivider = iif(reservationTerm_s == '3 Years', 3, 1)\r\n| extend ReservationPrice = todouble(replace_string(unitPrice_s,',','.'))/TermDivider/12/730\r\n| summarize arg_max(TimeGenerated, ReservationPrice) by Location_s=tolower(armRegionName_s), SkuName=tolower(armSkuName_s), Term_s;\r\nlet ReservationOnDemandMeters = AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| extend SkuName = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| distinct ReservationId_g, MeterId_g, SkuName;\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| where ReservationId_g in ({Reservation:value})\r\n| extend RINormalizationRatio = tostring(parse_json(AdditionalInfo_s).RINormalizationRatio)\r\n| extend UsedRIs = todouble(Quantity_s) * todouble(RINormalizationRatio) / 24\r\n| summarize UsedRIsDaily=round(sum(UsedRIs),2) by Date_s, ReservationId_g\r\n| summarize AvgRIsUsedDaily=round(avg(UsedRIsDaily),2) by ReservationId_g\r\n| join kind=rightouter (\r\n AzureOptimizationReservationsUsageV1_CL\r\n | where TimeGenerated > ago(1d)\r\n | where ReservationId_g in ({Reservation:value})\r\n | summarize arg_max(TimeGenerated, *) by ReservationId_g\r\n | where ProvisioningState_s in ('Succeeded','Expiring')\r\n | extend UsedQuantity = todouble(TotalReservedQuantity_s) * todouble(Util7Days_s) / 100\r\n | extend UsedQuantity30d = todouble(TotalReservedQuantity_s) * todouble(Util30Days_s) / 100\r\n | extend SKUName_s=tolower(SKUName_s)\r\n | project ReservationId_g, ReservationName_s=DisplayName_s, SKUName_s, Location_s, UsedQuantity, UsedQuantity30d, TotalReservedQuantity_s, Term_s, AppliedScopeType_s\r\n) on ReservationId_g\r\n| project ReservationId_g=ReservationId_g1, ReservationName_s, TotalReservedQuantity_s, SKUName_s, Location_s, AvgRIsUsedDaily=iif(isempty(AvgRIsUsedDaily), 0.0, AvgRIsUsedDaily), UsedQuantity, UsedQuantity30d, Term_s, AppliedScopeType_s\r\n| join kind=inner ( ISFGroups ) on $left.SKUName_s == $right.ArmSKUName\r\n| join kind=leftouter ( ReservationOnDemandMeters ) on ReservationId_g\r\n| summarize arg_max(MeterId_g, *) by ReservationId_g\r\n| join kind=leftouter ( ReservationPricesheet ) on SkuName and Location_s and Term_s\r\n| join kind=leftouter ( OnDemandPriceSheet ) on $left.MeterId_g == $right.MeterID_g\r\n| extend DiscountPercent = (1 - ReservationPrice/OnDemandUnitPrice) * 100\r\n| extend AvgRIsUsedInSmallestRatio = Ratio * AvgRIsUsedDaily\r\n| summarize TotalReservedQuantity_s=sum(todouble(TotalReservedQuantity_s)*Ratio), AvgRIsUsedDaily=sum(AvgRIsUsedInSmallestRatio), UsedQuantity=sum(UsedQuantity*Ratio), UsedQuantity30d=sum(UsedQuantity30d*Ratio), AvgDiscountPercent=avg(DiscountPercent) by ISFGroup, Location_s, Term_s, AppliedScopeType_s\r\n| extend Util7Days_s = UsedQuantity/TotalReservedQuantity_s*100, Util30Days_s = UsedQuantity30d/TotalReservedQuantity_s*100\r\n| extend AvgRIUsagePercentInSmallestRatio = round(AvgRIsUsedDaily / TotalReservedQuantity_s * 100, 1)\r\n| extend AvgDiscountPercent=iif(AvgDiscountPercent > 0.0, AvgDiscountPercent, 0.0)\r\n| extend SavingsMargin=round(todouble(Util7Days_s))-100.0+AvgDiscountPercent \r\n| project-away AvgRIUsagePercentInSmallestRatio, AvgRIsUsedDaily\r\n| project-reorder ISFGroup, Location_s, Term_s, AppliedScopeType_s, TotalReservedQuantity_s, Util7Days_s, UsedQuantity, Util30Days_s, UsedQuantity30d\r\n| order by Util7Days_s asc", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nlet LinuxOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption' and MeterSubCategory_s !endswith \"Windows\"\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet VMOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxOnDemandPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend OnDemandUnitPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, OnDemandUnitPrice;\r\nlet OnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s != 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| summarize OnDemandUnitPrice = max(todouble(UnitPrice_s)) by MeterID_g\r\n| distinct MeterID_g, OnDemandUnitPrice\r\n| union (VMOnDemandPriceSheet)\r\n| summarize OnDemandUnitPrice=min(OnDemandUnitPrice) by MeterID_g;\r\nlet ReservationPricesheet = AzureOptimizationReservationsPriceV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| extend Term_s = iif(reservationTerm_s == '3 Years', 'P3Y', 'P1Y')\r\n| extend TermDivider = iif(reservationTerm_s == '3 Years', 3, 1)\r\n| extend ReservationPrice = todouble(replace_string(unitPrice_s,',','.'))/TermDivider/12/730\r\n| summarize arg_max(TimeGenerated, ReservationPrice) by Location_s=tolower(armRegionName_s), SkuName=tolower(armSkuName_s), Term_s;\r\nlet ReservationOnDemandMeters = AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| extend SkuName = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| distinct ReservationId_g, MeterId_g, SkuName;\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| where ReservationId_g in ({Reservation:value})\r\n| extend RINormalizationRatio = tostring(parse_json(AdditionalInfo_s).RINormalizationRatio)\r\n| extend UsedRIs = todouble(Quantity_s) * todouble(RINormalizationRatio) / 24\r\n| summarize UsedRIsDaily=round(sum(UsedRIs),2) by Date_s, ReservationId_g\r\n| summarize AvgRIsUsedDaily=round(avg(UsedRIsDaily),2) by ReservationId_g\r\n| join kind=rightouter (\r\n AzureOptimizationReservationsUsageV1_CL\r\n | where TimeGenerated > ago(1d)\r\n | where ReservationId_g in ({Reservation:value})\r\n | summarize arg_max(TimeGenerated, *) by ReservationId_g\r\n | where ProvisioningState_s in ('Succeeded','Expiring')\r\n | extend UsedQuantity = todouble(TotalReservedQuantity_s) * todouble(Util7Days_s) / 100\r\n | extend UsedQuantity30d = todouble(TotalReservedQuantity_s) * todouble(Util30Days_s) / 100\r\n | extend SKUName_s=tolower(SKUName_s)\r\n | project ReservationId_g, ReservationName_s=DisplayName_s, SKUName_s, Location_s, UsedQuantity, UsedQuantity30d, TotalReservedQuantity_s, Term_s, AppliedScopeType_s\r\n) on ReservationId_g\r\n| project ReservationId_g=ReservationId_g1, ReservationName_s, TotalReservedQuantity_s, SKUName_s, Location_s, AvgRIsUsedDaily=iif(isempty(AvgRIsUsedDaily), 0.0, AvgRIsUsedDaily), UsedQuantity, UsedQuantity30d, Term_s, AppliedScopeType_s\r\n| join kind=inner ( ISFGroups ) on $left.SKUName_s == $right.ArmSKUName\r\n| join kind=leftouter ( ReservationOnDemandMeters ) on ReservationId_g\r\n| summarize arg_max(MeterId_g, *) by ReservationId_g\r\n| join kind=leftouter ( ReservationPricesheet ) on SkuName and Location_s and Term_s\r\n| join kind=leftouter ( OnDemandPriceSheet ) on $left.MeterId_g == $right.MeterID_g\r\n| extend DiscountPercent = (1 - ReservationPrice/OnDemandUnitPrice) * 100\r\n| extend AvgRIsUsedInSmallestRatio = Ratio * AvgRIsUsedDaily\r\n| summarize TotalReservedQuantity_s=sum(todouble(TotalReservedQuantity_s)*Ratio), AvgRIsUsedDaily=sum(AvgRIsUsedInSmallestRatio), UsedQuantity=sum(UsedQuantity*Ratio), UsedQuantity30d=sum(UsedQuantity30d*Ratio), AvgDiscountPercent=avg(DiscountPercent) by ISFGroup, Location_s, Term_s, AppliedScopeType_s\r\n| extend Util7Days_s = UsedQuantity/TotalReservedQuantity_s*100, Util30Days_s = UsedQuantity30d/TotalReservedQuantity_s*100\r\n| extend AvgRIUsagePercentInSmallestRatio = round(AvgRIsUsedDaily / TotalReservedQuantity_s * 100, 1)\r\n| extend AvgDiscountPercent=iif(AvgDiscountPercent > 0.0, AvgDiscountPercent, 0.0)\r\n| extend SavingsMargin=round(todouble(Util7Days_s))-100.0+AvgDiscountPercent \r\n| project-away AvgRIUsagePercentInSmallestRatio, AvgRIsUsedDaily\r\n| project-reorder ISFGroup, Location_s, Term_s, AppliedScopeType_s, TotalReservedQuantity_s, Util7Days_s, UsedQuantity, Util30Days_s, UsedQuantity30d\r\n| order by Util7Days_s asc", "size": 0, "showAnalytics": true, "title": "Reservation Usage Details grouped by ISF group (click on a line for more details)", @@ -817,7 +817,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| where ResourceLocation_s =~ '{selectedRegion}'\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| where ISFGroup == '{selectedISFGroup:value}'\r\n| extend RINormalizationRatio = tostring(parse_json(AdditionalInfo_s).RINormalizationRatio)\r\n| extend ConsumedSize = iif(isnotempty(VMSize), VMSize, strcat(MeterSubCategory_s, ' ', MeterName_s))\r\n| extend UsedRIs = todouble(Quantity_s) / 24 * Ratio\r\n| summarize round(sum(UsedRIs),1) by todatetime(Date_s), ConsumedSize", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| where ResourceLocation_s =~ '{selectedRegion}'\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| where ISFGroup == '{selectedISFGroup:value}'\r\n| extend RINormalizationRatio = tostring(parse_json(AdditionalInfo_s).RINormalizationRatio)\r\n| extend ConsumedSize = iif(isnotempty(VMSize), VMSize, strcat(MeterSubCategory_s, ' ', MeterName_s))\r\n| extend UsedRIs = todouble(Quantity_s) / 24 * Ratio\r\n| summarize round(sum(UsedRIs),1) by todatetime(Date_s), ConsumedSize", "size": 0, "aggregation": 3, "showAnalytics": true, @@ -851,7 +851,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nlet absoluteRIBought = toscalar(AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}')\r\n and TimeGenerated < todatetime('{LookbackPeriod:endISO}')\r\n and ChargeType_s == 'Usage'\r\n and isnotempty(ReservationName_s)\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| where ResourceLocation_s =~ '{selectedRegion}'\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| where ISFGroup == '{selectedISFGroup:value}'\r\n| summarize (datetime_diff('Day',max(todatetime(Date_s)),min(todatetime(Date_s)))+1)*{selectedQuantity});\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| where ResourceLocation_s =~ '{selectedRegion}'\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| where ISFGroup == '{selectedISFGroup:value}'\r\n| extend UsedRIs = todouble(Quantity_s) * Ratio / 24\r\n| summarize UsedRIPercentage=round(sum(UsedRIs) / absoluteRIBought * 100, 2) by ResourceId, SubscriptionId\r\n| join kind=leftouter ( \r\n AzureOptimizationResourceContainersV1_CL \r\n | where TimeGenerated > ago(1d) and ContainerType_s == 'microsoft.resources/subscriptions'\r\n | project Subscription=ContainerName_s, SubscriptionId=SubscriptionGuid_g\r\n) on SubscriptionId\r\n| project-away SubscriptionId*\r\n| project-reorder ResourceId, Subscription, UsedRIPercentage\r\n| order by ResourceId asc, UsedRIPercentage", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nlet absoluteRIBought = toscalar(AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}')\r\n and TimeGenerated < todatetime('{LookbackPeriod:endISO}')\r\n and ChargeType_s == 'Usage'\r\n and isnotempty(ReservationName_s)\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| where ResourceLocation_s =~ '{selectedRegion}'\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| where ISFGroup == '{selectedISFGroup:value}'\r\n| summarize (datetime_diff('Day',max(todatetime(Date_s)),min(todatetime(Date_s)))+1)*{selectedQuantity});\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| where ResourceLocation_s =~ '{selectedRegion}'\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| where ISFGroup == '{selectedISFGroup:value}'\r\n| extend UsedRIs = todouble(Quantity_s) * Ratio / 24\r\n| summarize UsedRIPercentage=round(sum(UsedRIs) / absoluteRIBought * 100, 2) by ResourceId, SubscriptionId\r\n| join kind=leftouter ( \r\n AzureOptimizationResourceContainersV1_CL \r\n | where TimeGenerated > ago(1d) and ContainerType_s == 'microsoft.resources/subscriptions'\r\n | project Subscription=ContainerName_s, SubscriptionId=SubscriptionGuid_g\r\n) on SubscriptionId\r\n| project-away SubscriptionId*\r\n| project-reorder ResourceId, Subscription, UsedRIPercentage\r\n| order by ResourceId asc, UsedRIPercentage", "size": 1, "showAnalytics": true, "title": "Average Daily Reservation Usage by Resource (click on a line in the table above)", @@ -892,7 +892,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nlet absoluteRIBought = toscalar(AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}')\r\n and TimeGenerated < todatetime('{LookbackPeriod:endISO}')\r\n and ChargeType_s == 'Usage'\r\n and isnotempty(ReservationName_s)\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| where ResourceLocation_s =~ '{selectedRegion}'\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| where ISFGroup == '{selectedISFGroup:value}'\r\n| summarize (datetime_diff('Day',max(todatetime(Date_s)),min(todatetime(Date_s)))+1)*{selectedQuantity});\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| where ResourceLocation_s =~ '{selectedRegion}'\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| where ISFGroup == '{selectedISFGroup:value}'\r\n| extend Tags_s = iif(Tags_s startswith \"{\", Tags_s, strcat(\"{\", Tags_s, \"}\"))\r\n| extend AggregatorTag = tostring(parse_json(Tags_s)['{Aggregator}'])\r\n| extend UsedRIs = todouble(Quantity_s) * Ratio / 24\r\n| summarize UsedRIPercentage=round(sum(UsedRIs) / absoluteRIBought * 100, 2) by AggregatorTag, SubscriptionId\r\n| join kind=leftouter ( \r\n AzureOptimizationResourceContainersV1_CL \r\n | where TimeGenerated > ago(1d) and ContainerType_s == 'microsoft.resources/subscriptions'\r\n | project Subscription=ContainerName_s, SubscriptionId=SubscriptionGuid_g\r\n) on SubscriptionId\r\n| project-away SubscriptionId*\r\n| project-reorder AggregatorTag, Subscription, UsedRIPercentage\r\n| order by AggregatorTag asc, UsedRIPercentage", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nlet absoluteRIBought = toscalar(AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}')\r\n and TimeGenerated < todatetime('{LookbackPeriod:endISO}')\r\n and ChargeType_s == 'Usage'\r\n and isnotempty(ReservationName_s)\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| where ResourceLocation_s =~ '{selectedRegion}'\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| where ISFGroup == '{selectedISFGroup:value}'\r\n| summarize (datetime_diff('Day',max(todatetime(Date_s)),min(todatetime(Date_s)))+1)*{selectedQuantity});\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| extend VMSize = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| where ResourceLocation_s =~ '{selectedRegion}'\r\n| join kind=inner ( ISFGroups ) on $left.VMSize == $right.ArmSKUName\r\n| where ISFGroup == '{selectedISFGroup:value}'\r\n| extend Tags_s = iif(Tags_s startswith \"{\", Tags_s, strcat(\"{\", Tags_s, \"}\"))\r\n| extend AggregatorTag = tostring(parse_json(Tags_s)['{Aggregator}'])\r\n| extend UsedRIs = todouble(Quantity_s) * Ratio / 24\r\n| summarize UsedRIPercentage=round(sum(UsedRIs) / absoluteRIBought * 100, 2) by AggregatorTag, SubscriptionId\r\n| join kind=leftouter ( \r\n AzureOptimizationResourceContainersV1_CL \r\n | where TimeGenerated > ago(1d) and ContainerType_s == 'microsoft.resources/subscriptions'\r\n | project Subscription=ContainerName_s, SubscriptionId=SubscriptionGuid_g\r\n) on SubscriptionId\r\n| project-away SubscriptionId*\r\n| project-reorder AggregatorTag, Subscription, UsedRIPercentage\r\n| order by AggregatorTag asc, UsedRIPercentage", "size": 1, "showAnalytics": true, "title": "Average Daily Reservation Usage by Tag (click on a line in the table above)", @@ -1057,7 +1057,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'UnusedReservation'\r\n| where ReservationId_g in ({Reservation:value})\r\n| extend UnusedCost = todouble(CostInBillingCurrency_s)\r\n| join kind=leftouter (\r\n AzureOptimizationReservationsUsageV1_CL\r\n | where TimeGenerated > ago(1d)\r\n | where ProvisioningState_s in ('Succeeded','Expiring')\r\n | extend UnusedQuantity = todouble(TotalReservedQuantity_s) - (todouble(TotalReservedQuantity_s) * todouble(Util7Days_s) / 100)\r\n | extend UnusedQuantity30d = todouble(TotalReservedQuantity_s) - (todouble(TotalReservedQuantity_s) * todouble(Util30Days_s) / 100)\r\n | project ReservationId_g, SKUName_s=tolower(SKUName_s), Location_s\r\n) on ReservationId_g\r\n| join kind=leftouter ( ISFGroups ) on $left.SKUName_s == $right.ArmSKUName\r\n| extend SKUName = iif('{UseISF}' == 'Yes', strcat(ISFGroup, \" \", Location_s), strcat(SKUName_s, \" \", Location_s))\r\n| extend SKUName = iif(strlen(SKUName) < 2, 'Canceled RIs', SKUName)\r\n| summarize sum(UnusedCost) by todatetime(Date_s), SKUName", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'UnusedReservation'\r\n| where ReservationId_g in ({Reservation:value})\r\n| extend UnusedCost = todouble(CostInBillingCurrency_s)\r\n| join kind=leftouter (\r\n AzureOptimizationReservationsUsageV1_CL\r\n | where TimeGenerated > ago(1d)\r\n | where ProvisioningState_s in ('Succeeded','Expiring')\r\n | extend UnusedQuantity = todouble(TotalReservedQuantity_s) - (todouble(TotalReservedQuantity_s) * todouble(Util7Days_s) / 100)\r\n | extend UnusedQuantity30d = todouble(TotalReservedQuantity_s) - (todouble(TotalReservedQuantity_s) * todouble(Util30Days_s) / 100)\r\n | project ReservationId_g, SKUName_s=tolower(SKUName_s), Location_s\r\n) on ReservationId_g\r\n| join kind=leftouter ( ISFGroups ) on $left.SKUName_s == $right.ArmSKUName\r\n| extend SKUName = iif('{UseISF}' == 'Yes', strcat(ISFGroup, \" \", Location_s), strcat(SKUName_s, \" \", Location_s))\r\n| extend SKUName = iif(strlen(SKUName) < 2, 'Canceled RIs', SKUName)\r\n| summarize sum(UnusedCost) by todatetime(Date_s), SKUName", "size": 0, "showAnalytics": true, "title": "Cost of Unused Reservations over time (by SKU)", @@ -1090,7 +1090,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'UnusedReservation'\r\n| where ReservationId_g in ({Reservation:value})\r\n| extend UnusedHours = todouble(Quantity_s)\r\n| join kind=inner (\r\n AzureOptimizationReservationsUsageV1_CL\r\n | where TimeGenerated > ago(1d)\r\n | where ProvisioningState_s in ('Succeeded','Expiring')\r\n | extend UnusedQuantity = todouble(TotalReservedQuantity_s) - (todouble(TotalReservedQuantity_s) * todouble(Util7Days_s) / 100)\r\n | extend UnusedQuantity30d = todouble(TotalReservedQuantity_s) - (todouble(TotalReservedQuantity_s) * todouble(Util30Days_s) / 100)\r\n | project ReservationId_g, SKUName_s=tolower(SKUName_s), Location_s\r\n) on ReservationId_g\r\n| join kind=leftouter ( ISFGroups ) on $left.SKUName_s == $right.ArmSKUName\r\n| extend SKUName = iif('{UseISF}' == 'Yes', strcat(ISFGroup, \" \", Location_s), strcat(SKUName_s, \" \", Location_s))\r\n| extend UnusedVMs = iif('{UseISF}' == 'Yes', UnusedHours * Ratio / 24, UnusedHours / 24)\r\n| summarize sum(UnusedVMs) by todatetime(Date_s), SKUName", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'UnusedReservation'\r\n| where ReservationId_g in ({Reservation:value})\r\n| extend UnusedHours = todouble(Quantity_s)\r\n| join kind=inner (\r\n AzureOptimizationReservationsUsageV1_CL\r\n | where TimeGenerated > ago(1d)\r\n | where ProvisioningState_s in ('Succeeded','Expiring')\r\n | extend UnusedQuantity = todouble(TotalReservedQuantity_s) - (todouble(TotalReservedQuantity_s) * todouble(Util7Days_s) / 100)\r\n | extend UnusedQuantity30d = todouble(TotalReservedQuantity_s) - (todouble(TotalReservedQuantity_s) * todouble(Util30Days_s) / 100)\r\n | project ReservationId_g, SKUName_s=tolower(SKUName_s), Location_s\r\n) on ReservationId_g\r\n| join kind=leftouter ( ISFGroups ) on $left.SKUName_s == $right.ArmSKUName\r\n| extend SKUName = iif('{UseISF}' == 'Yes', strcat(ISFGroup, \" \", Location_s), strcat(SKUName_s, \" \", Location_s))\r\n| extend UnusedVMs = iif('{UseISF}' == 'Yes', UnusedHours * Ratio / 24, UnusedHours / 24)\r\n| summarize sum(UnusedVMs) by todatetime(Date_s), SKUName", "size": 0, "showAnalytics": true, "title": "Unused Reservations over time (by VM count)", @@ -1310,7 +1310,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'UnusedReservation'\r\n| where ReservationId_g in ({Reservation:value})\r\n| summarize TotalUnusedCost = sum(todouble(CostInBillingCurrency_s)) by ReservationId_g\r\n| where round(TotalUnusedCost) > 0\r\n| join kind=inner (\r\n AzureOptimizationReservationsUsageV1_CL\r\n | where TimeGenerated > ago(1d)\r\n | where ProvisioningState_s in ('Succeeded','Expiring')\r\n | extend UnusedQuantity = todouble(TotalReservedQuantity_s) - (todouble(TotalReservedQuantity_s) * todouble(Util7Days_s) / 100)\r\n | extend UnusedQuantity30d = todouble(TotalReservedQuantity_s) - (todouble(TotalReservedQuantity_s) * todouble(Util30Days_s) / 100)\r\n | project ReservationId_g, ReservationName_s=DisplayName_s, SKUName_s=tolower(SKUName_s), Location_s, TotalReservedQuantity_s, Term_s, AppliedScopeType_s, UnusedQuantity, UnusedQuantity30d\r\n) on ReservationId_g\r\n| project-away ReservationId_g1\r\n| join kind=inner ( ISFGroups ) on $left.SKUName_s == $right.ArmSKUName\r\n| summarize TotalUnusedCost=sum(TotalUnusedCost), TotalReservedQuantity_s=sum(todouble(TotalReservedQuantity_s)*Ratio), UnusedQuantity=sum(UnusedQuantity*Ratio), UnusedQuantity30d=sum(UnusedQuantity30d*Ratio) by ISFGroup, Location_s, Term_s, AppliedScopeType_s\r\n| extend Util7Days_s = (1-UnusedQuantity/TotalReservedQuantity_s)*100, Util30Days_s = (1-UnusedQuantity30d/TotalReservedQuantity_s)*100\r\n| project-reorder ISFGroup, TotalUnusedCost, Location_s, Term_s, AppliedScopeType_s, TotalReservedQuantity_s, Util7Days_s, UnusedQuantity, Util30Days_s\r\n| order by TotalUnusedCost", + "query": "let ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend ArmSKUName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'UnusedReservation'\r\n| where ReservationId_g in ({Reservation:value})\r\n| summarize TotalUnusedCost = sum(todouble(CostInBillingCurrency_s)) by ReservationId_g\r\n| where round(TotalUnusedCost) > 0\r\n| join kind=inner (\r\n AzureOptimizationReservationsUsageV1_CL\r\n | where TimeGenerated > ago(1d)\r\n | where ProvisioningState_s in ('Succeeded','Expiring')\r\n | extend UnusedQuantity = todouble(TotalReservedQuantity_s) - (todouble(TotalReservedQuantity_s) * todouble(Util7Days_s) / 100)\r\n | extend UnusedQuantity30d = todouble(TotalReservedQuantity_s) - (todouble(TotalReservedQuantity_s) * todouble(Util30Days_s) / 100)\r\n | project ReservationId_g, ReservationName_s=DisplayName_s, SKUName_s=tolower(SKUName_s), Location_s, TotalReservedQuantity_s, Term_s, AppliedScopeType_s, UnusedQuantity, UnusedQuantity30d\r\n) on ReservationId_g\r\n| project-away ReservationId_g1\r\n| join kind=inner ( ISFGroups ) on $left.SKUName_s == $right.ArmSKUName\r\n| summarize TotalUnusedCost=sum(TotalUnusedCost), TotalReservedQuantity_s=sum(todouble(TotalReservedQuantity_s)*Ratio), UnusedQuantity=sum(UnusedQuantity*Ratio), UnusedQuantity30d=sum(UnusedQuantity30d*Ratio) by ISFGroup, Location_s, Term_s, AppliedScopeType_s\r\n| extend Util7Days_s = (1-UnusedQuantity/TotalReservedQuantity_s)*100, Util30Days_s = (1-UnusedQuantity30d/TotalReservedQuantity_s)*100\r\n| project-reorder ISFGroup, TotalUnusedCost, Location_s, Term_s, AppliedScopeType_s, TotalReservedQuantity_s, Util7Days_s, UnusedQuantity, Util30Days_s\r\n| order by TotalUnusedCost", "size": 2, "showAnalytics": true, "title": "Unused Reservations Details (grouped by Instance Size Flexibility group)", @@ -1485,7 +1485,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "let LinuxOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption' and MeterSubCategory_s !endswith \"Windows\"\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet VMOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxOnDemandPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend OnDemandUnitPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, OnDemandUnitPrice;\r\nlet OnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s != 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| summarize OnDemandUnitPrice = max(todouble(UnitPrice_s)) by MeterID_g\r\n| union (VMOnDemandPriceSheet)\r\n| summarize OnDemandUnitPrice=min(OnDemandUnitPrice) by MeterID_g;\r\nlet ReservationPricesheet = materialize(AzureOptimizationReservationsPriceV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| extend Term_s = iif(reservationTerm_s == '3 Years', 'P3Y', 'P1Y')\r\n| extend TermDivider = iif(reservationTerm_s == '3 Years', 3, 1)\r\n| extend ReservationPrice = todouble(replace_string(unitPrice_s,',','.'))/TermDivider/12/730\r\n| summarize arg_max(TimeGenerated, ReservationPrice) by Location_s=tolower(armRegionName_s), SkuName=tolower(armSkuName_s), Term_s);\r\nlet ReservationOnDemandMeters = AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| extend SkuName = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| distinct ReservationId_g, MeterId_g, SkuName;\r\nlet ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://aka.ms/isf\"] with(ignoreFirstRecord=true) | extend skuName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| where ReservationId_g in ({Reservation:value})\r\n| extend RINormalizationRatio = tostring(parse_json(AdditionalInfo_s).RINormalizationRatio)\r\n| extend UsedRIs = todouble(Quantity_s) * todouble(RINormalizationRatio) / 24\r\n| summarize UsedRIsDaily=round(sum(UsedRIs),2) by Date_s, ReservationId_g\r\n| summarize AvgRIsUsedDaily=round(avg(UsedRIsDaily),2) by ReservationId_g\r\n| join kind=rightouter (\r\n AzureOptimizationReservationsUsageV1_CL\r\n | where TimeGenerated > ago(1d)\r\n | where ReservationId_g in ({Reservation:value})\r\n | summarize arg_max(TimeGenerated, *) by ReservationId_g\r\n | where ProvisioningState_s in ('Succeeded','Expiring')\r\n | project ReservationId_g, ReservationName_s=DisplayName_s, SKUName_s, Location_s, Util7Days_s, Util30Days_s, TotalReservedQuantity_s, Term_s, ExpiryDate_s\r\n) on ReservationId_g\r\n| project ReservationId_g=ReservationId_g1, ReservationName_s, TotalReservedQuantity_s, SKUName_s=tolower(SKUName_s), Location_s, AvgRIsUsedDaily=iif(isempty(AvgRIsUsedDaily), 0.0, AvgRIsUsedDaily), Util7Days_s, Util30Days_s, Term_s, ExpiryDate_s\r\n| join kind=leftouter ( ISFGroups ) on $left.SKUName_s==$right.skuName\r\n| join kind=leftouter (ReservationOnDemandMeters) on ReservationId_g\r\n| summarize arg_max(MeterId_g, *) by ReservationId_g\r\n| join kind=leftouter (ReservationPricesheet) on SkuName and Location_s and Term_s\r\n| join kind=leftouter (OnDemandPriceSheet) on $left.MeterId_g == $right.MeterID_g\r\n| extend DiscountPercent = (1 - ReservationPrice/OnDemandUnitPrice) * 100\r\n| project-away ReservationPrice\r\n| join kind=leftouter (ReservationPricesheet) on $left.SKUName_s==$right.SkuName and Location_s and Term_s\r\n| extend HoursUntilExpiry=(todatetime(ExpiryDate_s)-now())/1h\r\n| extend TotalReservedHoursToConsume=todouble(TotalReservedQuantity_s)*HoursUntilExpiry\r\n| extend AmountRemainingToConsume = round(TotalReservedHoursToConsume * ReservationPrice, 2)\r\n| project ReservationId_g, ReservationName_s, TotalReservedQuantity_s=round(todouble(TotalReservedQuantity_s)), SKUName_s, ISFGroup, Location_s, Term_s, ExpiryDate_s, AmountRemainingToConsume, AvgRIsUsedDaily, Util7Days_s=round(todouble(Util7Days_s)), Util30Days_s=round(todouble(Util30Days_s)), DiscountPercent, SavingsMargin=round(todouble(Util7Days_s))-100.0+DiscountPercent \r\n| order by Util7Days_s asc", + "query": "let LinuxOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption' and MeterSubCategory_s !endswith \"Windows\"\r\n| extend MeterSubCategory_s = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Linux'))\r\n| summarize LinuxUnitPrice=max(todouble(UnitPrice_s)) by LinuxMeterId=MeterID_g, MeterName_s, MeterSubCategory_s, MeterRegion_s, LinuxUnitOfMeasure=UnitOfMeasure_s;\r\nlet VMOnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s == 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| extend NonWindowsMeterSubcategory = substring(MeterSubCategory_s, 0, indexof(MeterSubCategory_s, ' Windows'))\r\n| extend WindowsMeterSubCategory = MeterSubCategory_s\r\n| extend NonWindowsMeterSubcategory = substring(NonWindowsMeterSubcategory, 0, indexof(NonWindowsMeterSubcategory, ' Linux'))\r\n| summarize UnitPrice_s=max(todouble(UnitPrice_s)) by MeterID_g, MeterName_s, NonWindowsMeterSubcategory, WindowsMeterSubCategory, MeterRegion_s, UnitOfMeasure_s\r\n| join kind=leftouter ( LinuxOnDemandPriceSheet ) on MeterName_s, MeterRegion_s, $left.NonWindowsMeterSubcategory == $right.MeterSubCategory_s\r\n| extend PricesheetPrice = iif(isnotempty(LinuxUnitPrice), LinuxUnitPrice, UnitPrice_s)\r\n| extend PricesheetUnitOfMeasure = iif(isnotempty(LinuxUnitOfMeasure), LinuxUnitOfMeasure, UnitOfMeasure_s)\r\n| extend UnitHrs = toint(substring(PricesheetUnitOfMeasure, 0, indexof(PricesheetUnitOfMeasure, 'Hour')-1))\r\n| extend OnDemandUnitPrice = PricesheetPrice/UnitHrs\r\n| distinct MeterID_g, OnDemandUnitPrice;\r\nlet OnDemandPriceSheet = AzureOptimizationPricesheetV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| where MeterCategory_s != 'Virtual Machines' and PriceType_s == 'Consumption'\r\n| summarize OnDemandUnitPrice = max(todouble(UnitPrice_s)) by MeterID_g\r\n| union (VMOnDemandPriceSheet)\r\n| summarize OnDemandUnitPrice=min(OnDemandUnitPrice) by MeterID_g;\r\nlet ReservationPricesheet = materialize(AzureOptimizationReservationsPriceV1_CL\r\n| where TimeGenerated > ago(14d)\r\n| extend Term_s = iif(reservationTerm_s == '3 Years', 'P3Y', 'P1Y')\r\n| extend TermDivider = iif(reservationTerm_s == '3 Years', 3, 1)\r\n| extend ReservationPrice = todouble(replace_string(unitPrice_s,',','.'))/TermDivider/12/730\r\n| summarize arg_max(TimeGenerated, ReservationPrice) by Location_s=tolower(armRegionName_s), SkuName=tolower(armSkuName_s), Term_s);\r\nlet ReservationOnDemandMeters = AzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| extend SkuName = tolower(parse_json(AdditionalInfo_s).ServiceType)\r\n| distinct ReservationId_g, MeterId_g, SkuName;\r\nlet ISFGroups = externaldata(ISFGroup:string, ArmSKUName:string, Ratio:double) [@\"https://ccmstorageprod.blob.core.windows.net/instancesizeflexibility-data/isfratioblob.csv\"] with(ignoreFirstRecord=true) | extend skuName=tolower(ArmSKUName);\r\nAzureOptimizationConsumptionV1_CL\r\n| where TimeGenerated > todatetime('{LookbackPeriod:startISO}') and TimeGenerated < todatetime('{LookbackPeriod:endISO}') and ChargeType_s == 'Usage' and isnotempty(ReservationName_s)\r\n| where ReservationId_g in ({Reservation:value})\r\n| extend RINormalizationRatio = tostring(parse_json(AdditionalInfo_s).RINormalizationRatio)\r\n| extend UsedRIs = todouble(Quantity_s) * todouble(RINormalizationRatio) / 24\r\n| summarize UsedRIsDaily=round(sum(UsedRIs),2) by Date_s, ReservationId_g\r\n| summarize AvgRIsUsedDaily=round(avg(UsedRIsDaily),2) by ReservationId_g\r\n| join kind=rightouter (\r\n AzureOptimizationReservationsUsageV1_CL\r\n | where TimeGenerated > ago(1d)\r\n | where ReservationId_g in ({Reservation:value})\r\n | summarize arg_max(TimeGenerated, *) by ReservationId_g\r\n | where ProvisioningState_s in ('Succeeded','Expiring')\r\n | project ReservationId_g, ReservationName_s=DisplayName_s, SKUName_s, Location_s, Util7Days_s, Util30Days_s, TotalReservedQuantity_s, Term_s, ExpiryDate_s\r\n) on ReservationId_g\r\n| project ReservationId_g=ReservationId_g1, ReservationName_s, TotalReservedQuantity_s, SKUName_s=tolower(SKUName_s), Location_s, AvgRIsUsedDaily=iif(isempty(AvgRIsUsedDaily), 0.0, AvgRIsUsedDaily), Util7Days_s, Util30Days_s, Term_s, ExpiryDate_s\r\n| join kind=leftouter ( ISFGroups ) on $left.SKUName_s==$right.skuName\r\n| join kind=leftouter (ReservationOnDemandMeters) on ReservationId_g\r\n| summarize arg_max(MeterId_g, *) by ReservationId_g\r\n| join kind=leftouter (ReservationPricesheet) on SkuName and Location_s and Term_s\r\n| join kind=leftouter (OnDemandPriceSheet) on $left.MeterId_g == $right.MeterID_g\r\n| extend DiscountPercent = (1 - ReservationPrice/OnDemandUnitPrice) * 100\r\n| project-away ReservationPrice\r\n| join kind=leftouter (ReservationPricesheet) on $left.SKUName_s==$right.SkuName and Location_s and Term_s\r\n| extend HoursUntilExpiry=(todatetime(ExpiryDate_s)-now())/1h\r\n| extend TotalReservedHoursToConsume=todouble(TotalReservedQuantity_s)*HoursUntilExpiry\r\n| extend AmountRemainingToConsume = round(TotalReservedHoursToConsume * ReservationPrice, 2)\r\n| project ReservationId_g, ReservationName_s, TotalReservedQuantity_s=round(todouble(TotalReservedQuantity_s)), SKUName_s, ISFGroup, Location_s, Term_s, ExpiryDate_s, AmountRemainingToConsume, AvgRIsUsedDaily, Util7Days_s=round(todouble(Util7Days_s)), Util30Days_s=round(todouble(Util30Days_s)), DiscountPercent, SavingsMargin=round(todouble(Util7Days_s))-100.0+DiscountPercent \r\n| order by Util7Days_s asc", "size": 0, "showAnalytics": true, "timeContextFromParameter": "LookbackPeriod", From e5eb47b5943c3d1a296bfaefb2c0b27c31257138 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Fri, 10 Oct 2025 21:59:36 -0700 Subject: [PATCH 16/69] Restore Azure Migrate functionality descriptions in FinOps documentation (#1797) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: arthurclares <53261392+arthurclares@users.noreply.github.com> Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com> --- docs-mslearn/toolkit/changelog.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 7a370d4d1..ea78bcc8a 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -64,6 +64,11 @@ _Released August 2025_ > [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v13) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v12...v13) +### [Implementing FinOps guide](../implementing-finops-guide.md) v13 + +- **Changed** + - Updated FinOps framework documentation to prepare for Azure TCO calculator retirement scheduled for August 31, 2025. Azure Migrate cost estimation functionality remains available. +
## v12 From 03bb80eed11cd0d3feb061ca0fe568c0c56e9cdb Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Fri, 10 Oct 2025 22:02:07 -0700 Subject: [PATCH 17/69] [Power BI] Document export requirements for each report (#1786) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com> Co-authored-by: Brett Wilson Co-authored-by: msbrett --- docs-mslearn/toolkit/changelog.md | 6 ++++++ docs-mslearn/toolkit/power-bi/cost-summary.md | 15 +++++++++++++ .../toolkit/power-bi/data-ingestion.md | 15 +++++++++++++ docs-mslearn/toolkit/power-bi/governance.md | 14 +++++++++++++ docs-mslearn/toolkit/power-bi/invoicing.md | 15 +++++++++++++ .../toolkit/power-bi/rate-optimization.md | 19 +++++++++++++++++ docs-mslearn/toolkit/power-bi/reports.md | 21 +++++++++++++------ .../toolkit/power-bi/workload-optimization.md | 14 +++++++++++++ 8 files changed, 113 insertions(+), 6 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index ea78bcc8a..edb42dc42 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -40,6 +40,12 @@ The following section lists features and enhancements that are currently in deve - Cost Management export modules for subscriptions and resource groups. +### [Power BI reports](power-bi/reports.md) + +- **Added** + - Added export requirements sections to all Power BI report documentation pages to clarify which Cost Management exports are needed for each report. + - Added Azure Resource Graph as an explicit requirement for governance and workload optimization reports. +
## v13 diff --git a/docs-mslearn/toolkit/power-bi/cost-summary.md b/docs-mslearn/toolkit/power-bi/cost-summary.md index eae3f8499..5016cb27f 100644 --- a/docs-mslearn/toolkit/power-bi/cost-summary.md +++ b/docs-mslearn/toolkit/power-bi/cost-summary.md @@ -34,6 +34,21 @@ This article contains images showing example data. Any price data is for test pu
+## Export requirements + +Before using this report, you need to configure Cost Management exports to provide the necessary data. The following exports are required or recommended: + +| Dataset | Version | Requirement | Notes | +| --------------------------- | ---------------- | ----------- | ------------------------------------------------------------------------------------------------- | +| Cost and usage (FOCUS) | `1.0` or `1.0r2` | **Required** | Provides the primary cost and usage data for all report functionality. | +| Price sheet | `2023-05-01` | Recommended | Required to populate missing prices for EA and MCA accounts to show accurate cost calculations. | +| Reservation details | `2023-03-01` | Optional | Provides additional reservation usage details if you use reservations. | +| Reservation transactions | `2023-05-01` | Optional | Provides reservation purchase and refund details if you use reservations. | + +For instructions on how to create these exports, see [Create and manage exports](/azure/cost-management-billing/costs/tutorial-improved-exports). If using FinOps hubs, these exports can be configured automatically. + +
+ ## Working with this report This report includes the following filters on each page: diff --git a/docs-mslearn/toolkit/power-bi/data-ingestion.md b/docs-mslearn/toolkit/power-bi/data-ingestion.md index bc02ccd59..482454fa9 100644 --- a/docs-mslearn/toolkit/power-bi/data-ingestion.md +++ b/docs-mslearn/toolkit/power-bi/data-ingestion.md @@ -28,6 +28,21 @@ Power BI reports are provided as template (.PBIT) files. Template files are not
+## Export requirements + +This report is specifically designed for **FinOps hubs** deployments and requires different data sources than other reports: + +| Data source | Requirement | Notes | +| --------------------------- | ----------- | ------------------------------------------------------------------------------------------------- | +| FinOps hubs Data Explorer | **Required** | Provides ingestion monitoring data and hub cost analysis. Only available with KQL reports. | +| Cost and usage (FOCUS) | **Required** | Provides cost data to analyze FinOps hubs infrastructure costs. | + +This report is **only available for KQL reports** connecting to FinOps hubs with Azure Data Explorer. It cannot be used with storage reports or direct Cost Management exports. + +For instructions on how to deploy FinOps hubs, see [FinOps hubs overview](../hubs/finops-hubs-overview.md). + +
+ ## Get started The **Get started** page includes a basic introduction to the report with other links to learn more. diff --git a/docs-mslearn/toolkit/power-bi/governance.md b/docs-mslearn/toolkit/power-bi/governance.md index 2246fd834..b341037ee 100644 --- a/docs-mslearn/toolkit/power-bi/governance.md +++ b/docs-mslearn/toolkit/power-bi/governance.md @@ -41,6 +41,20 @@ Power BI reports are provided as template (.PBIT) files. Template files are not
+## Export requirements + +Before using this report, you need to configure Cost Management exports to provide the necessary data. The following exports are required or recommended: + +| Dataset | Version | Requirement | Notes | +| --------------------------- | ---------------- | ----------- | ------------------------------------------------------------------------------------------------- | +| Cost and usage (FOCUS) | `1.0` or `1.0r2` | **Required** | Provides the primary cost and usage data for governance analysis. | +| Price sheet | `2023-05-01` | Recommended | Required to populate missing prices for EA and MCA accounts to show accurate cost calculations. | +| Azure Resource Graph | Latest | **Required** | Required to gather resource metadata and governance information for compliance analysis. | + +For instructions on how to create Cost Management exports, see [Create and manage exports](/azure/cost-management-billing/costs/tutorial-improved-exports). If using FinOps hubs, these exports can be configured automatically. + +
+ ## Get started The **Get started** page includes a basic introduction to the report with links to learn more. diff --git a/docs-mslearn/toolkit/power-bi/invoicing.md b/docs-mslearn/toolkit/power-bi/invoicing.md index 4cc648c1f..b6b248193 100644 --- a/docs-mslearn/toolkit/power-bi/invoicing.md +++ b/docs-mslearn/toolkit/power-bi/invoicing.md @@ -30,6 +30,21 @@ This article contains images showing example data. Any price data is for test pu
+## Export requirements + +Before using this report, you need to configure Cost Management exports to provide the necessary data. The following exports are required or recommended: + +| Dataset | Version | Requirement | Notes | +| --------------------------- | ---------------- | ----------- | ------------------------------------------------------------------------------------------------- | +| Cost and usage (FOCUS) | `1.0` or `1.0r2` | **Required** | Provides the primary cost and usage data for all report functionality. | +| Price sheet | `2023-05-01` | Recommended | Required to populate missing prices for EA and MCA accounts to show accurate billed costs. | +| Reservation details | `2023-03-01` | Optional | Provides additional reservation usage details if you use reservations. | +| Reservation transactions | `2023-05-01` | Optional | Provides reservation purchase and refund details if you use reservations. | + +For instructions on how to create these exports, see [Create and manage exports](/azure/cost-management-billing/costs/tutorial-improved-exports). If using FinOps hubs, these exports can be configured automatically. + +
+ ## Working with this report This report includes the following filters on each page: diff --git a/docs-mslearn/toolkit/power-bi/rate-optimization.md b/docs-mslearn/toolkit/power-bi/rate-optimization.md index 0a2756e8d..c93ded728 100644 --- a/docs-mslearn/toolkit/power-bi/rate-optimization.md +++ b/docs-mslearn/toolkit/power-bi/rate-optimization.md @@ -37,6 +37,25 @@ Power BI reports are provided as template (.PBIT) files. Template files are not
+## Export requirements + +Before using this report, you need to configure Cost Management exports to provide the necessary data. The following exports are required or recommended: + +| Dataset | Version | Requirement | Notes | +| --------------------------- | ---------------- | ----------- | ------------------------------------------------------------------------------------------------- | +| Cost and usage (FOCUS) | `1.0` or `1.0r2` | **Required** | Provides the primary cost and usage data for all report functionality. | +| Price sheet | `2023-05-01` | **Required** | Required to calculate accurate savings and show missing list and contracted prices. | +| Reservation details | `2023-03-01` | Recommended | Provides detailed reservation usage data for utilization analysis. | +| Reservation recommendations | `2023-05-01` | **Required** | Required to display reservation purchase recommendations in the Recommendations page. | +| Reservation transactions | `2023-05-01` | Optional | Provides reservation purchase and refund details. | + +For instructions on how to create these exports, see [Create and manage exports](/azure/cost-management-billing/costs/tutorial-improved-exports). If using FinOps hubs, these exports can be configured automatically. + +> [!IMPORTANT] +> Microsoft Cost Management does not include the list and contracted prices for all accounts. To calculate accurate and complete savings, you will need to export prices. If using storage reports, enable the "Experimental: Populate Missing Prices" parameter in each report. If using KQL reports, missing prices will be populated automatically when prices are exported. + +
+ ## Working with this report This report includes the following filters on each page: diff --git a/docs-mslearn/toolkit/power-bi/reports.md b/docs-mslearn/toolkit/power-bi/reports.md index 2c7cea800..50b32ae0d 100644 --- a/docs-mslearn/toolkit/power-bi/reports.md +++ b/docs-mslearn/toolkit/power-bi/reports.md @@ -87,13 +87,22 @@ Reports are provided as Power BI template (.pbit) files that do not include samp | Exports in storage (including FinOps hubs) | [Storage reports](https://github.com/microsoft/finops-toolkit/releases/latest/download/PowerBI-storage.zip) | Not recommended when monitoring more than $2 million per month. | | Cost Management connector | [Cost Management connector report](https://github.com/microsoft/finops-toolkit/releases/latest/download/CostManagementConnector.zip) | Not recommended when monitoring more than $1M in total cost or accounts that contain savings plan usage. | -Configure FinOps hubs or Cost Management exports with KQL or storage reports. For FinOps hubs, refer to [Configure scopes](../hubs/configure-scopes.md). For Cost Management exports, refer to [How to create exports](/azure/cost-management-billing/costs/tutorial-improved-exports). Power BI reports use the following export types: +Configure FinOps hubs or Cost Management exports with KQL or storage reports. For FinOps hubs, refer to [Configure scopes](../hubs/configure-scopes.md). For Cost Management exports, refer to [How to create exports](/azure/cost-management-billing/costs/tutorial-improved-exports). -- Cost and usage (FOCUS) – Required for all reports. -- Price sheet -- Reservation details -- Reservation recommendations – Required to see reservation recommendations in the Rate optimization report. -- Reservation transactions +### Export requirements + +**Before using any Power BI report**, you need to configure the appropriate Cost Management exports. Different reports require different datasets: + +| Dataset | Version | Required for reports | Notes | +| --------------------------- | ---------------- | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | +| Cost and usage (FOCUS) | `1.0` or `1.0r2` | **All reports** | Primary cost and usage data. Required for all functionality. | +| Price sheet | `2023-05-01` | All reports (recommended for accurate pricing) | Required to populate missing prices for EA and MCA accounts. | +| Reservation details | `2023-03-01` | Rate optimization (recommended) | Provides detailed reservation usage data for utilization analysis. | +| Reservation recommendations | `2023-05-01` | **Rate optimization** (required for recommendations) | Required to display reservation purchase recommendations. | +| Reservation transactions | `2023-05-01` | Rate optimization, Invoicing (optional) | Provides reservation purchase and refund details. | + +> [!IMPORTANT] +> Each report documentation page includes specific export requirements. Review the "Export requirements" section on each report page before downloading to ensure you have the necessary data configured. For more information, see [How to setup Power BI](setup.md#set-up-your-first-report). diff --git a/docs-mslearn/toolkit/power-bi/workload-optimization.md b/docs-mslearn/toolkit/power-bi/workload-optimization.md index d32097b6f..a7d279b18 100644 --- a/docs-mslearn/toolkit/power-bi/workload-optimization.md +++ b/docs-mslearn/toolkit/power-bi/workload-optimization.md @@ -35,6 +35,20 @@ Power BI reports are provided as template (.PBIT) files. Template files are not
+## Export requirements + +Before using this report, you need to configure Cost Management exports to provide the necessary data. The following exports are required or recommended: + +| Dataset | Version | Requirement | Notes | +| --------------------------- | ---------------- | ----------- | ------------------------------------------------------------------------------------------------- | +| Cost and usage (FOCUS) | `1.0` or `1.0r2` | **Required** | Provides the primary cost and usage data for resource cost analysis. | +| Price sheet | `2023-05-01` | Recommended | Required to populate missing prices for EA and MCA accounts to show accurate cost calculations. | +| Azure Resource Graph | Latest | **Required** | Required to gather resource metadata for workload optimization analysis. | + +For instructions on how to create Cost Management exports, see [Create and manage exports](/azure/cost-management-billing/costs/tutorial-improved-exports). If using FinOps hubs, these exports can be configured automatically. + +
+ ## Get started The **Get started** page includes a basic introduction to the report with links to learn more. From 4a05789bf6398f4c3bc3b937575acf3de0853b73 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Fri, 10 Oct 2025 22:04:42 -0700 Subject: [PATCH 18/69] Document how to use Data Lake Storage outside of Power BI (#1784) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com> Co-authored-by: msbrett --- docs-mslearn/toolkit/changelog.md | 5 + .../toolkit/data-lake-storage-connectivity.md | 205 ++++++++++++++++++ 2 files changed, 210 insertions(+) create mode 100644 docs-mslearn/toolkit/data-lake-storage-connectivity.md diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index edb42dc42..5fde3cb86 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -46,6 +46,11 @@ The following section lists features and enhancements that are currently in deve - Added export requirements sections to all Power BI report documentation pages to clarify which Cost Management exports are needed for each report. - Added Azure Resource Graph as an explicit requirement for governance and workload optimization reports. +### Documentation improvements + +- **Added** + - Created comprehensive [Data Lake Storage connectivity options](data-lake-storage-connectivity.md) documentation covering tools and services beyond Power BI including Azure Data Explorer, Microsoft Fabric, Azure Synapse Analytics, Azure Databricks, and custom applications. +
## v13 diff --git a/docs-mslearn/toolkit/data-lake-storage-connectivity.md b/docs-mslearn/toolkit/data-lake-storage-connectivity.md new file mode 100644 index 000000000..ab03ada38 --- /dev/null +++ b/docs-mslearn/toolkit/data-lake-storage-connectivity.md @@ -0,0 +1,205 @@ +--- +title: Data Lake Storage connectivity options +description: Learn about tools and services that can connect to Data Lake Storage for FinOps analytics beyond Power BI, including Azure Data Explorer, Microsoft Fabric, and Azure Synapse Analytics. +author: flanakin +ms.author: micflan +ms.date: 04/02/2025 +ms.topic: concept-article +ms.service: finops +ms.subservice: finops-toolkit +ms.reviewer: micflan +#customer intent: As a FinOps practitioner, I want to understand Data Lake Storage connectivity options outside of Power BI so that I can build custom reports that meet my organizational needs. +--- + + +# Data Lake Storage connectivity options + +As a FinOps practitioner, you may need to build custom reports and analytics solutions outside of Power BI to meet specific organizational requirements. Azure Data Lake Storage provides a central repository for your FinOps data that can be accessed by multiple tools and services for advanced analytics, custom applications, and integration scenarios. + +This article covers the primary tools and services that can connect to Data Lake Storage for FinOps analytics and reporting. + +
+ +## Azure Data Explorer (ADX) + +Azure Data Explorer is a fast, highly scalable data exploration service for log and telemetry data that provides powerful analytics capabilities for your FinOps data. + +If you're using [FinOps hubs](hubs/finops-hubs-overview.md), Azure Data Explorer is automatically configured with pre-built data ingestion pipelines, optimized data models, sample dashboards and queries, and automated data processing. + +> [!div class="nextstepaction"] +> [Configure Data Explorer dashboards](hubs/configure-dashboards.md) + +
+ +## Microsoft Fabric + +Microsoft Fabric is an all-in-one analytics solution that combines data integration, data engineering, data warehousing, data science, real-time analytics, and business intelligence into a unified platform. + +Fabric provides a unified analytics platform with OneLake storage, AI and machine learning capabilities, seamless Power BI integration, and support for real-time insights. + +> [!div class="nextstepaction"] +> [Create a Fabric workspace for FinOps](../fabric/create-fabric-workspace-finops.md) + +
+ +## Azure Synapse Analytics + +Azure Synapse Analytics is an enterprise data warehouse solution that combines big data and data warehousing capabilities. + +### Benefits for FinOps + +- **Scalable data warehouse**: Handle large volumes of historical FinOps data +- **SQL and Spark support**: Use familiar SQL or Apache Spark for data processing +- **Integrated machine learning**: Build predictive models for cost forecasting +- **Data lake integration**: Native integration with Data Lake Storage +- **Enterprise security**: Advanced security and governance features + +### Getting started with Synapse + +Azure Synapse Analytics provides comprehensive documentation for connecting to and querying data in Data Lake Storage. + +> [!div class="nextstepaction"] +> [Query data in Azure Data Lake Storage with Synapse SQL](https://learn.microsoft.com/azure/synapse-analytics/sql/query-data-storage) + +> [!div class="nextstepaction"] +> [Create external tables in Synapse SQL](https://learn.microsoft.com/azure/synapse-analytics/sql/create-external-table-as-select) + +
+ +## Azure Databricks + +Azure Databricks is a unified analytics platform that provides collaborative Apache Spark-based analytics for advanced data science and machine learning scenarios. + +### Benefits for FinOps + +- **Advanced analytics**: Perform complex cost modeling and forecasting +- **Machine learning**: Build predictive models for cost optimization +- **Collaborative notebooks**: Share analysis with data science teams +- **Delta Lake support**: ACID transactions and versioning for data quality +- **Integration capabilities**: Connect to multiple data sources and tools + +### Getting started with Databricks + +Azure Databricks provides comprehensive documentation for connecting to and analyzing data in Data Lake Storage. + +> [!div class="nextstepaction"] +> [Connect to Azure Data Lake Storage from Databricks](https://learn.microsoft.com/azure/databricks/storage/azure-storage) + +> [!div class="nextstepaction"] +> [Machine learning with Databricks](https://learn.microsoft.com/azure/databricks/machine-learning/) + +
+ +## Azure Machine Learning + +Azure Machine Learning provides enterprise-grade machine learning capabilities for building advanced cost optimization and forecasting models. + +### Benefits for FinOps + +- **MLOps capabilities**: End-to-end machine learning lifecycle management +- **Automated ML**: Automatically build and optimize cost prediction models +- **Model deployment**: Deploy models as web services for real-time predictions +- **Responsible AI**: Built-in tools for model interpretability and fairness +- **Integration**: Connect with other Azure services and tools + +### Use cases for FinOps + +- **Cost forecasting**: Predict future spending based on historical patterns +- **Anomaly detection**: Identify unusual cost spikes or patterns +- **Optimization recommendations**: Generate automated cost optimization suggestions +- **Budget planning**: Support budget planning with predictive insights + +
+ +## Custom applications and APIs + +Data Lake Storage provides REST APIs and SDKs that enable you to build custom applications and integrate FinOps data with existing systems. + +### Benefits + +- **Custom integrations**: Build integrations with existing business systems +- **Automated reporting**: Create automated report generation and distribution +- **Real-time monitoring**: Build custom monitoring and alerting solutions +- **API access**: Programmatic access to FinOps data for any application + +### Getting started with custom applications + +Azure Data Lake Storage provides comprehensive SDKs and REST APIs for building custom applications. + +> [!div class="nextstepaction"] +> [Azure Data Lake Storage REST API](https://learn.microsoft.com/rest/api/storageservices/data-lake-storage-gen2) + +> [!div class="nextstepaction"] +> [Azure Data Lake Storage client libraries](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-directory-file-acl-dotnet) + +
+ +## Choosing the right tool + +The choice of tool depends on your specific requirements: + +| Tool | Best for | Complexity | Cost model | +|------|----------|------------|------------| +| **Azure Data Explorer** | Real-time analytics, KQL queries, built-in dashboards | Medium | Pay-per-use | +| **Microsoft Fabric** | Unified analytics platform, AI/ML integration | Medium-High | Capacity-based | +| **Azure Synapse** | Data warehousing, large-scale ETL, enterprise scenarios | High | Pay-per-use or dedicated | +| **Azure Databricks** | Advanced analytics, machine learning, data science | High | Pay-per-use | +| **Azure Machine Learning** | MLOps, automated ML, model deployment | High | Pay-per-use | +| **Custom applications** | Specific integrations, custom workflows | Variable | Development cost | + +
+ +## Security and governance + +When connecting to Data Lake Storage, ensure proper security and governance: + +- **Authentication**: Use Azure Active Directory and managed identities +- **Authorization**: Implement role-based access control (RBAC) +- **Network security**: Configure private endpoints and network restrictions +- **Data classification**: Classify and protect sensitive financial data +- **Auditing**: Enable audit logging for access and operations + +
+ +## Give feedback + +Let us know how we're doing with a quick review. We use these reviews to improve and expand FinOps tools and resources. + +> [!div class="nextstepaction"] +> [Give feedback](https://portal.azure.com/#view/HubsExtension/InProductFeedbackBlade/extensionName/FinOpsToolkit/cesQuestion/How%20easy%20or%20hard%20is%20it%20to%20use%20Data%20Lake%20Storage%20connectivity%20options%3F/cvaQuestion/How%20valuable%20are%20the%20Data%20Lake%20Storage%20connectivity%20options%3F/surveyId/FTK/bladeName/DataLakeConnectivity/featureName/Documentation) + +If you're looking for something specific, vote for an existing or create a new idea. Share ideas with others to get more votes. We focus on ideas with the most votes. + +> [!div class="nextstepaction"] +> [Vote on or suggest ideas](https://github.com/microsoft/finops-toolkit/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc) + +
+ +## Related content + +Related resources: + +- [FinOps Open Cost and Usage Specification (FOCUS)](../focus/what-is-focus.md) +- [Data ingestion best practices](../framework/understand/ingestion.md) + +Related FinOps capabilities: + +- [Reporting and analytics](../framework/understand/reporting.md) +- [Data analysis](../framework/understand/analysis.md) + +Related products: + +- [Azure Data Explorer](/azure/data-explorer/) +- [Microsoft Fabric](/fabric/get-started/microsoft-fabric-overview) +- [Azure Synapse Analytics](/azure/synapse-analytics/) +- [Azure Databricks](/azure/databricks/) +- [Azure Machine Learning](/azure/machine-learning/) +- [Azure Data Lake Storage](/azure/storage/blobs/data-lake-storage-introduction) + +Related solutions: + +- [FinOps hubs](hubs/finops-hubs-overview.md) +- [FinOps toolkit Power BI reports](power-bi/reports.md) +- [FinOps toolkit open data](open-data.md) + +
\ No newline at end of file From 4040d451a4ed4f83f48ece1771582eceeca3a86c Mon Sep 17 00:00:00 2001 From: msbrett Date: Sat, 11 Oct 2025 08:13:05 -0700 Subject: [PATCH 19/69] Remove CLAUDE.md from repository - security incident remediation --- CLAUDE.md | 182 ------------------------------------------------------ 1 file changed, 182 deletions(-) delete mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index 3de4262b1..000000000 --- a/CLAUDE.md +++ /dev/null @@ -1,182 +0,0 @@ -# CLAUDE.md - -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - -## Repository Overview - -The FinOps Toolkit is an open-source collection of tools for adopting and implementing FinOps capabilities in the Microsoft Cloud. It contains templates, PowerShell modules, workbooks, optimization engines, and supporting documentation organized in a modular architecture. - -## Common Commands - -### Building and Development - -```bash -# Build entire toolkit -npm run build -# or -pwsh -Command ./src/scripts/Build-Toolkit - -# Build FinOps hubs -pwsh -Command ./src/scripts/Build-Toolkit finops-hub - -# Build specific components -npm run build-ps # PowerShell module only -pwsh -Command ./src/scripts/Build-Bicep # Bicep templates -pwsh -Command ./src/scripts/Build-Workbook # Azure Monitor workbooks -pwsh -Command ./src/scripts/Build-OpenData # Open data files - -# Deploy for testing -npm run deploy-test -# or -pwsh -Command ./src/scripts/Deploy-Toolkit -Build -Test - -# Package for release -npm run package -# or -pwsh -Command ./src/scripts/Package-Toolkit -Build -``` - -### Testing - -```bash -# Run PowerShell unit tests -npm run pester -# or -pwsh -Command Invoke-Pester -Output Detailed -Path ./src/powershell/Tests/Unit/* - -# Run integration tests -pwsh -Command ./src/scripts/Test-PowerShell -Integration - -# Run specific test categories -pwsh -Command ./src/scripts/Test-PowerShell -Hubs -Exports - -# Lint PowerShell code -pwsh -Command ./src/scripts/Test-PowerShell -Lint -``` - -### Bicep Development - -```bash -# Validate Bicep templates -bicep build path/to/template.bicep --stdout - -# Test template deployment -az deployment group what-if --resource-group myRG --template-file template.bicep -``` - -## Architecture and Code Organization - -### High-Level Structure - -- **`/src/templates/`** - ARM/Bicep infrastructure templates with modular namespace organization -- **`/src/powershell/`** - PowerShell module with public/private functions and comprehensive tests -- **`/src/optimization-engine/`** - Azure Optimization Engine for cost recommendations -- **`/src/workbooks/`** - Azure Monitor workbooks for governance and optimization -- **`/src/open-data/`** - Reference data (pricing, regions, services) with utilities -- **`/src/scripts/`** - Build automation and development tools -- **`/docs/`** - Jekyll documentation website -- **`/docs-mslearn/`** - Microsoft Learn documentation website -- **`/docs-wiki/`** - GitHub wiki documentation - -### Current Architectural Reorganization - -The FinOps hubs solution is actively migrating to a namespace-based modular structure: - -- **`Microsoft.FinOpsHubs/`** - Core FinOps Hub infrastructure modules -- **`Microsoft.CostManagement/`** - Cost management exports and schemas -- **`fx/`** - Shared foundation components (hub-types, scripts, utilities) - -### Template Architecture - -Templates use a multi-target build system that generates: - -- Azure Quickstart Templates (ARM JSON) -- Bicep Registry modules -- Standalone deployments -- Azure portal UI definitions - -Key patterns: - -- **`.build.config`** files control build behavior per template -- **`settings.json`** contains component-specific configuration -- **`ftkver.txt`** files maintain version synchronization -- **Conditional resource deployment** based on parameters - -### PowerShell Module Structure - -- **`Public/`** - User-facing cmdlets (Get-_, Set-_, New-\*, etc.) -- **`Private/`** - Internal utilities and helpers -- **`Tests/Unit/`** - Pester unit tests with mocking -- **`Tests/Integration/`** - End-to-end Azure integration tests -- **Module manifest** defines exports and dependencies - -### Data Flow and Integration - -- **Open data** provides reference information consumed by templates and PowerShell -- **Build scripts** orchestrate compilation across all components -- **Version management** is centralized through `Update-Version.ps1` -- **Templates reference** shared schemas and types from `fx/` namespace - -## Key Development Patterns - -### Template Development - -- Use `newApp()` and `newHub()` functions from `fx/hub-types.bicep` for consistent resource naming -- Follow the conditional deployment pattern: `resource foo 'type' = if (condition) { ... }` -- Implement proper parameter validation with `@allowed`, `@minValue`, `@maxValue` -- Include telemetry tracking via `defaultTelemetry` parameter - -### PowerShell Development - -- All public functions must have comment-based help -- Use approved verbs from `Get-Verb` -- Implement comprehensive parameter validation -- Support `-WhatIf` and `-Confirm` for destructive operations -- Include Pester tests for all functions - -### Testing Strategy - -- **Lint tests** validate syntax and coding standards -- **Unit tests** test isolated function behavior with mocks -- **Integration tests** perform end-to-end validation against Azure -- **Template validation** uses `bicep build` and ARM what-if deployments - -### Build System Integration - -The PowerShell-based build system: - -- Compiles templates to multiple target formats -- Validates all code before packaging -- Maintains version consistency across components -- Generates release artifacts automatically - -### Version Management - -- Central version in `package.json` (currently 12.0.0) -- Synchronized across all components via build scripts -- Individual `ftkver.txt` files distributed to modules -- Git tags correspond to release versions - -## Repository Conventions - -### Branch Strategy - -- **`dev`** - Main integration branch -- Feature branches merge into `dev` -- Releases are tagged from `dev` - -### File Organization - -- Templates follow namespace/module/component structure -- PowerShell follows standard module layout -- Documentation uses Jekyll conventions -- Build artifacts are generated, not checked in - -### Coding Standards - -- Always follow the content and coding standards defined in `docs-wiki/Coding-guidelines.md` -- Content (text strings): Follow the Microsoft style guide and always use sentence casing except for proper nouns -- Bicep: Follow Azure Bicep style guide -- PowerShell: Use PowerShell best practices and approved verbs -- Documentation: Use markdown with consistent formatting -- Commit messages: Use conventional commit format From a7b2bd418656c4fcde962ddcd608494d774d7aae Mon Sep 17 00:00:00 2001 From: msbrett Date: Sat, 11 Oct 2025 08:15:58 -0700 Subject: [PATCH 20/69] Revert "Remove CLAUDE.md from repository - security incident remediation" This reverts commit 4040d451a4ed4f83f48ece1771582eceeca3a86c. --- CLAUDE.md | 182 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 182 insertions(+) create mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 000000000..3de4262b1 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,182 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Repository Overview + +The FinOps Toolkit is an open-source collection of tools for adopting and implementing FinOps capabilities in the Microsoft Cloud. It contains templates, PowerShell modules, workbooks, optimization engines, and supporting documentation organized in a modular architecture. + +## Common Commands + +### Building and Development + +```bash +# Build entire toolkit +npm run build +# or +pwsh -Command ./src/scripts/Build-Toolkit + +# Build FinOps hubs +pwsh -Command ./src/scripts/Build-Toolkit finops-hub + +# Build specific components +npm run build-ps # PowerShell module only +pwsh -Command ./src/scripts/Build-Bicep # Bicep templates +pwsh -Command ./src/scripts/Build-Workbook # Azure Monitor workbooks +pwsh -Command ./src/scripts/Build-OpenData # Open data files + +# Deploy for testing +npm run deploy-test +# or +pwsh -Command ./src/scripts/Deploy-Toolkit -Build -Test + +# Package for release +npm run package +# or +pwsh -Command ./src/scripts/Package-Toolkit -Build +``` + +### Testing + +```bash +# Run PowerShell unit tests +npm run pester +# or +pwsh -Command Invoke-Pester -Output Detailed -Path ./src/powershell/Tests/Unit/* + +# Run integration tests +pwsh -Command ./src/scripts/Test-PowerShell -Integration + +# Run specific test categories +pwsh -Command ./src/scripts/Test-PowerShell -Hubs -Exports + +# Lint PowerShell code +pwsh -Command ./src/scripts/Test-PowerShell -Lint +``` + +### Bicep Development + +```bash +# Validate Bicep templates +bicep build path/to/template.bicep --stdout + +# Test template deployment +az deployment group what-if --resource-group myRG --template-file template.bicep +``` + +## Architecture and Code Organization + +### High-Level Structure + +- **`/src/templates/`** - ARM/Bicep infrastructure templates with modular namespace organization +- **`/src/powershell/`** - PowerShell module with public/private functions and comprehensive tests +- **`/src/optimization-engine/`** - Azure Optimization Engine for cost recommendations +- **`/src/workbooks/`** - Azure Monitor workbooks for governance and optimization +- **`/src/open-data/`** - Reference data (pricing, regions, services) with utilities +- **`/src/scripts/`** - Build automation and development tools +- **`/docs/`** - Jekyll documentation website +- **`/docs-mslearn/`** - Microsoft Learn documentation website +- **`/docs-wiki/`** - GitHub wiki documentation + +### Current Architectural Reorganization + +The FinOps hubs solution is actively migrating to a namespace-based modular structure: + +- **`Microsoft.FinOpsHubs/`** - Core FinOps Hub infrastructure modules +- **`Microsoft.CostManagement/`** - Cost management exports and schemas +- **`fx/`** - Shared foundation components (hub-types, scripts, utilities) + +### Template Architecture + +Templates use a multi-target build system that generates: + +- Azure Quickstart Templates (ARM JSON) +- Bicep Registry modules +- Standalone deployments +- Azure portal UI definitions + +Key patterns: + +- **`.build.config`** files control build behavior per template +- **`settings.json`** contains component-specific configuration +- **`ftkver.txt`** files maintain version synchronization +- **Conditional resource deployment** based on parameters + +### PowerShell Module Structure + +- **`Public/`** - User-facing cmdlets (Get-_, Set-_, New-\*, etc.) +- **`Private/`** - Internal utilities and helpers +- **`Tests/Unit/`** - Pester unit tests with mocking +- **`Tests/Integration/`** - End-to-end Azure integration tests +- **Module manifest** defines exports and dependencies + +### Data Flow and Integration + +- **Open data** provides reference information consumed by templates and PowerShell +- **Build scripts** orchestrate compilation across all components +- **Version management** is centralized through `Update-Version.ps1` +- **Templates reference** shared schemas and types from `fx/` namespace + +## Key Development Patterns + +### Template Development + +- Use `newApp()` and `newHub()` functions from `fx/hub-types.bicep` for consistent resource naming +- Follow the conditional deployment pattern: `resource foo 'type' = if (condition) { ... }` +- Implement proper parameter validation with `@allowed`, `@minValue`, `@maxValue` +- Include telemetry tracking via `defaultTelemetry` parameter + +### PowerShell Development + +- All public functions must have comment-based help +- Use approved verbs from `Get-Verb` +- Implement comprehensive parameter validation +- Support `-WhatIf` and `-Confirm` for destructive operations +- Include Pester tests for all functions + +### Testing Strategy + +- **Lint tests** validate syntax and coding standards +- **Unit tests** test isolated function behavior with mocks +- **Integration tests** perform end-to-end validation against Azure +- **Template validation** uses `bicep build` and ARM what-if deployments + +### Build System Integration + +The PowerShell-based build system: + +- Compiles templates to multiple target formats +- Validates all code before packaging +- Maintains version consistency across components +- Generates release artifacts automatically + +### Version Management + +- Central version in `package.json` (currently 12.0.0) +- Synchronized across all components via build scripts +- Individual `ftkver.txt` files distributed to modules +- Git tags correspond to release versions + +## Repository Conventions + +### Branch Strategy + +- **`dev`** - Main integration branch +- Feature branches merge into `dev` +- Releases are tagged from `dev` + +### File Organization + +- Templates follow namespace/module/component structure +- PowerShell follows standard module layout +- Documentation uses Jekyll conventions +- Build artifacts are generated, not checked in + +### Coding Standards + +- Always follow the content and coding standards defined in `docs-wiki/Coding-guidelines.md` +- Content (text strings): Follow the Microsoft style guide and always use sentence casing except for proper nouns +- Bicep: Follow Azure Bicep style guide +- PowerShell: Use PowerShell best practices and approved verbs +- Documentation: Use markdown with consistent formatting +- Commit messages: Use conventional commit format From df9904de0d54dba56055c92fe8723257375fe974 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Sat, 11 Oct 2025 13:09:43 -0700 Subject: [PATCH 21/69] Add comprehensive troubleshooting guide for Data Explorer SEM0080 ingestion errors (#1857) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com> Co-authored-by: msbrett --- docs-mslearn/toolkit/changelog.md | 2 + docs-mslearn/toolkit/help/errors.md | 95 ++++++++++++++++++++++++++++- 2 files changed, 96 insertions(+), 1 deletion(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 5fde3cb86..452ff770b 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -50,6 +50,8 @@ The following section lists features and enhancements that are currently in deve - **Added** - Created comprehensive [Data Lake Storage connectivity options](data-lake-storage-connectivity.md) documentation covering tools and services beyond Power BI including Azure Data Explorer, Microsoft Fabric, Azure Synapse Analytics, Azure Databricks, and custom applications. + - Enhanced troubleshooting guidance for [Data Explorer ingestion errors](help/errors.md#dataexploreringestionfailed) with detailed steps for diagnosing and resolving the common SEM0080 "Ingestion Failed" semantic error, including schema mismatch detection, ingestion mapping verification, and diagnostic query examples. +
diff --git a/docs-mslearn/toolkit/help/errors.md b/docs-mslearn/toolkit/help/errors.md index e774b2da8..589a110d1 100644 --- a/docs-mslearn/toolkit/help/errors.md +++ b/docs-mslearn/toolkit/help/errors.md @@ -125,7 +125,100 @@ This error message is not related to the FinOps toolkit. Data Explorer ingestion failed. The new data will not be available for reporting. -**Mitigation**: Review the Data Explorer error message and resolve the issue. Rerun data ingestion for the specified folder using the ingestion_ExecuteETL pipeline in Azure Data Factory. Report unresolved issues at https://aka.ms/ftk/ideas. +### Common error: SEM0080 assert() has failed with message 'Ingestion Failed' + +If you see the following semantic error in the Azure Data Factory pipeline: + +> _Semantic error: Relop semantic error: SEM0080: assert() has failed with message: 'Ingestion Failed'_ + +This error indicates that the Data Explorer `.ingest` command detected errors during the ingestion process. The ingestion command includes an assertion check (`assert(iff(toscalar($command_results | project-keep HasErrors) == false, true, false), "Ingestion Failed")`) that verifies the `HasErrors` column in the command results. When `HasErrors` is `true`, the assertion fails and triggers this error. + +**Common root causes**: + +1. **Empty parquet file**: The parquet file contains no data rows. This is the most common cause. + - Cost Management export generated an empty file (no data for the time period) + - ETL pipeline created an empty parquet file during transformation + - File was created but data write operation failed + +2. **Schema mismatch**: The parquet file schema doesn't match the ingestion mapping reference for the target table. + - Columns in the parquet file may have different names or data types than expected + - The ingestion mapping (e.g., `_mapping`) may be outdated or incorrect + - New columns were added to the export schema that aren't in the mapping + +3. **Corrupted or invalid parquet files**: The source file may be malformed, corrupted, or not a valid parquet file. + +4. **Missing or incorrect ingestion mapping**: The referenced mapping (e.g., `Costs_raw_mapping`) doesn't exist or has incorrect column definitions. + +5. **Data type conversion errors**: Data in the parquet file can't be converted to the target column types defined in the table schema. + +6. **File access issues**: Data Explorer can't access the parquet file in storage due to permissions or network issues. + +**Mitigation steps**: + +1. **Check ingestion failures in Data Explorer**: + - Connect to your Data Explorer cluster/database + - Run the following query to see detailed error information: + ```kusto + .show ingestion failures + | where FailedOn > ago(4h) and Database == "" + | project FailedOn, Table, IngestionSourcePath, ErrorCode, Details + ``` + - Review the `Details` column for specific error messages about empty files, schema mismatches, or data issues + - Look for error codes like `BadRequest_NoRecordsOrWrongFormat` which indicates an empty file + +2. **Check if the parquet file is empty**: + - Download the problematic parquet file from the ingestion container (path is in the error message) + - Use a parquet viewer tool or Azure Storage Explorer to inspect the file + - Check the file size - if it's very small (< 1KB), it's likely empty + - Verify the file contains data rows + - **If empty**: This is expected behavior when there's no data for the time period. The file can be safely deleted from the ingestion container. Cost Management may export empty files for months with no usage. + +3. **Verify the ingestion mapping exists and is correct**: + - Run this query in Data Explorer to check if the mapping exists: + ```kusto + .show table ingestion mappings + ``` + - If the mapping is missing, it needs to be recreated. Check the FinOps hub deployment logs for mapping creation errors. + - If the mapping exists, verify it matches the expected schema for your data source + +4. **Check for schema changes**: + - If you recently updated Cost Management exports or changed export versions (e.g., from FOCUS 1.0 to 1.2), the schema may have changed + - Verify the export dataset version in the manifest.json file in the msexports container + - Confirm FinOps hubs supports the dataset version - see [supported datasets](../hubs/data-processing.md#datasets) + +5. **Check Data Explorer diagnostics**: + - In the Azure portal, navigate to your Data Explorer cluster + - Go to **Monitoring** > **Diagnostic settings** + - Enable `FailedIngestion` diagnostic logs if not already enabled + - Review logs in Log Analytics for detailed error information + +6. **Redeploy FinOps hubs if mappings are missing**: + - If ingestion mappings are missing or corrupted, redeploy FinOps hubs to recreate them + - This will recreate all tables, mappings, and functions without data loss + +7. **Review Azure Data Explorer metrics**: + - Check the **Ingestion result** metric in Azure Monitor + - Filter by status to see success vs failure rates + - See [Monitor queued ingestion](https://learn.microsoft.com/azure/data-explorer/monitor-queued-ingestion) for more details + +8. **Rerun ingestion after fixing the issue**: + - After resolving the root cause, rerun the `ingestion_ExecuteETL` pipeline + - Specify the folder path from the error message as the parameter + - Monitor the pipeline execution to confirm successful ingestion + - Note: Empty files do not need to be reingested - they can be safely ignored + +**Additional resources**: + +- [Azure Data Explorer ingestion error codes](https://learn.microsoft.com/azure/data-explorer/error-codes) +- [Ingestion behavior of invalid data](https://learn.microsoft.com/azure/data-explorer/ingest-invalid-data) +- [Data Explorer ingestion overview](https://learn.microsoft.com/azure/data-explorer/ingest-data-overview) +- [Kusto ingestion failures command](https://learn.microsoft.com/kusto/management/ingestion-failures) + +If you continue to experience this error after following these steps, please [report the issue](https://aka.ms/ftk/ideas) with the following information: +- Complete error message from the ADF pipeline +- Output from the `.show ingestion failures` query +- Dataset type and version from the manifest.json file +- FinOps hubs version
From 7fa789623ad69e16e345cf0cba8e0ba9b9aa8609 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Sat, 11 Oct 2025 14:12:15 -0700 Subject: [PATCH 22/69] [Hubs] Document remote hubs support and add UI configuration (#1724) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: flanakin <399533+flanakin@users.noreply.github.com> Co-authored-by: Brett Wilson --- .../toolkit/hubs/configure-remote-hubs.md | 109 ++++++++++++++++++ docs-mslearn/toolkit/hubs/deploy.md | 17 +++ docs-mslearn/toolkit/hubs/template.md | 36 +++--- .../powershell/hubs/deploy-finopshub.md | 35 ++++-- docs/hubs.md | 4 + .../Tests/Unit/Deploy-FinOpsHub.Tests.ps1 | 28 +++++ .../finops-hub/createUiDefinition.json | 47 ++++++++ 7 files changed, 249 insertions(+), 27 deletions(-) create mode 100644 docs-mslearn/toolkit/hubs/configure-remote-hubs.md diff --git a/docs-mslearn/toolkit/hubs/configure-remote-hubs.md b/docs-mslearn/toolkit/hubs/configure-remote-hubs.md new file mode 100644 index 000000000..c5f8d2ce6 --- /dev/null +++ b/docs-mslearn/toolkit/hubs/configure-remote-hubs.md @@ -0,0 +1,109 @@ +--- +ms.service: finops +ms.author: flanakin +author: flanakin +ms.date: 11/01/2024 +ms.topic: how-to +title: Configure remote hubs +description: Learn how to configure FinOps hubs to collect cost data across multiple Azure tenants and clouds using remote hub functionality. +--- + +# Configure remote hubs + +Remote hubs enable cross-tenant cost data collection scenarios where a central tenant aggregates cost data from multiple tenants or subscriptions. In this setup, "satellite" FinOps hubs in different tenants send their processed data to a central "primary" hub for consolidated reporting and analysis. + +Remote hubs work across different Azure clouds, supporting: +- Azure Commercial +- Azure Government +- Azure China + +
+ +## When to use remote hubs + +Consider remote hubs when you have: + +- Multiple Azure tenants with separate billing relationships +- A centralized FinOps team that needs visibility across multiple organizations +- Subsidiaries or business units in separate tenants +- Partners or customers who want to contribute cost data to a shared analysis +- Multi-cloud scenarios where you need cost data from different Azure cloud environments + +
+ +## Architecture overview + +In a remote hub configuration: + +1. **Primary hub**: Central FinOps hub that receives and stores aggregated data from all tenants +2. **Remote (satellite) hubs**: FinOps hubs in remote tenants that process local cost data and send it to the primary hub + +
+ +## Configure the primary hub + +1. Deploy a standard FinOps hub in your central tenant using the regular deployment process +2. Note the storage account name (found in the resource group after deployment) +3. Get the Data Lake storage endpoint: + - Navigate to the storage account in the Azure portal + - Select **Settings** > **Endpoints** + - Copy the **Data Lake storage** URL (format: `https://storageaccount.dfs.core.windows.net/`) +4. Get the storage account access key: + - Navigate to **Security + networking** > **Access keys** + - Copy **key1** or **key2** value + +
+ +## Configure remote hubs + +When deploying remote hubs, provide the primary hub's storage details: + +### [Azure portal](#tab/azure-portal) + +1. When deploying the FinOps hub template, navigate to the **Advanced** tab +2. Expand **Remote hub configuration** +3. Enter the **Remote hub storage URI** from the primary hub (copy from the primary hub's storage account Settings > Endpoints > Data Lake storage) +4. Enter the **Remote hub storage key** from the primary hub (copy from the primary hub's storage account Security + networking > Access keys > key1/2 > Key) +5. Complete the deployment normally + +### [PowerShell](#tab/powershell) + +```powershell +Deploy-FinOpsHub ` + -Name MyRemoteHub ` + -ResourceGroup MyRemoteHubResourceGroup ` + -Location westus ` + -RemoteHubStorageUri "https://primaryhubstore123.dfs.core.windows.net/" ` + -RemoteHubStorageKey "abc123...xyz789==" +``` + +--- + +
+ +## Security considerations + +- **Version requirement**: Remote hubs support requires FinOps hub template version 0.4 or later +- **Storage keys**: Treat storage keys as secrets. They provide full access to the storage account +- **Network access**: Consider using private networking for both primary and remote hubs +- **Key rotation**: Regularly rotate storage keys and update remote hub configurations +- **Least privilege**: The storage key provides broad access; consider using Azure AD authentication when available + +
+ +## Data flow and processing + +Remote hubs process data locally and then send processed (not raw) cost data to the primary hub. This approach: + +- Reduces data transfer costs +- Maintains data sovereignty for initial processing +- Centralizes only the final, processed cost data +- Preserves full granularity in the primary hub + +
+ +## Next steps + +- [Deploy a FinOps hub](deploy.md) +- [Configure private networking](private-networking.md) +- [Upgrade FinOps hubs](upgrade.md) \ No newline at end of file diff --git a/docs-mslearn/toolkit/hubs/deploy.md b/docs-mslearn/toolkit/hubs/deploy.md index 810b72843..1d488120e 100644 --- a/docs-mslearn/toolkit/hubs/deploy.md +++ b/docs-mslearn/toolkit/hubs/deploy.md @@ -89,6 +89,19 @@ Public routing doesn't require configuration. If you opt for private routing, wo
+## Plan for multiple tenants and clouds + +FinOps hubs support remote hub functionality for collecting cost data across multiple Azure tenants and clouds. Remote hubs enable centralized cost management for organizations with distributed billing relationships while maintaining data sovereignty during processing. + +Remote hubs can be used for multiple tenants in the same Azure cloud or tenants in different Azure clouds. Supported clouds are: +- Azure Commercial +- Azure Government +- Azure China + +To configure remote hubs, see [Configure remote hubs](configure-remote-hubs.md). + +
+ ## Optional: Set up Microsoft Fabric Many organizations adopt Microsoft Fabric as a unified data platform to streamline data analytics, storage, and processing. FinOps hubs can use Microsoft Fabric Real-Time Intelligence (RTI) as either a primary or secondary data store. This section only applies when configuring Microsoft Fabric as a primary data store instead of Azure Data Explorer. @@ -154,6 +167,10 @@ The core engine for FinOps hubs is deployed via an Azure Resource Manager deploy 12. Indicate is you want public or private network routing. [Learn more](private-networking.md). 13. If you selected private, specify the desired private network address prefix. 14. Select the **Next** button at the bottom of the form. + - **Optional**: For remote hub configuration (cross-tenant scenarios), expand **Remote hub configuration** and: + - Enter the **Remote hub storage URI** from the primary hub + - Enter the **Remote hub storage key** from the primary hub + - For details, see [Configure remote hubs](configure-remote-hubs.md) 15. If desired, specify more tags to add to resources. 16. Select the **Next** button at the bottom of the form. 17. Review the configuration summary and select the **Create** button at the bottom of the form. diff --git a/docs-mslearn/toolkit/hubs/template.md b/docs-mslearn/toolkit/hubs/template.md index b0d6171c9..db0ea830c 100644 --- a/docs-mslearn/toolkit/hubs/template.md +++ b/docs-mslearn/toolkit/hubs/template.md @@ -74,24 +74,24 @@ Ensure the following prerequisites are met before you deploy the template: Here are the parameters you can use to customize the deployment: -| Parameter | Type | Description | Default value | -| -------------------------------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | -| **hubName** | String | Optional. Name of the hub. Used to ensure unique resource names. | "finops-hub" | -| **location** | String | Optional. Azure location where all resources should be created. See https://aka.ms/azureregions. | Same as deployment | -| **storageSku** | String | Optional. Storage SKU to use. LRS = Lowest cost, ZRS = High availability. Note Standard SKUs are not available for Data Lake gen2 storage. Allowed: `Premium_LRS`, `Premium_ZRS`. | "Premium_LRS" | -| **dataExplorerName** | String | Optional. Name of the Azure Data Explorer cluster to use for advanced analytics. If empty, Azure Data Explorer will not be deployed. Required to use with Power BI if you have more than $2-5M/mo in costs being monitored. Default: "" (do not use). | | -| **dataExplorerSkuName** | String | Optional. Name of the Azure Data Explorer SKU. Default: "Dev(No SLA)_Standard_E2a_v4". | | -| **dataExplorerSkuTier** | String | Optional. SKU tier for the Azure Data Explorer cluster. Use Basic for the lowest cost with no SLA (due to a single node). Use Standard for high availability and improved performance. Allowed values: Basic, Standard. Default: "Basic". | | -| **dataExplorerSkuCapacity** | Int | Optional. Number of nodes to use in the cluster. Allowed values: 1 for the Basic SKU tier and 2-1000 for Standard. Default: 1. | | -| **tags** | Object | Optional. Tags to apply to all resources. We will also add the `cm-resource-parent` tag for improved cost roll-ups in Cost Management. | | -| **tagsByResource** | Object | Optional. Tags to apply to resources based on their resource type. Resource type specific tags will be merged with tags for all resources. | | -| **scopesToMonitor** | Array | Optional. List of scope IDs to monitor and ingest cost for. | | -| **exportRetentionInDays** | Int | Optional. Number of days of data to retain in the msexports container. | 0 | -| **ingestionRetentionInMonths** | Int | Optional. Number of months of data to retain in the ingestion container. | 13 | -| **dataExplorerLogRetentionInDays** | Int | Optional. Number of days of data to retain in the Data Explorer \*_log tables. | 0 | -| **dataExplorerFinalRetentionInMonths** | Int | Optional. Number of months of data to retain in the Data Explorer \*_final_v\* tables. | 13 | -| **remoteHubStorageUri** | String | Optional. Storage account to push data to for ingestion into a remote hub. | | -| **remoteHubStorageKey** | String | Optional. Storage account key to use when pushing data to a remote hub. | | +| Parameter | Type | Description | Default value | +| -------------------------------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | +| **hubName** | String | Optional. Name of the hub. Used to ensure unique resource names. | "finops-hub" | +| **location** | String | Optional. Azure location where all resources should be created. See https://aka.ms/azureregions. | Same as deployment | +| **storageSku** | String | Optional. Storage SKU to use. LRS = Lowest cost, ZRS = High availability. Note Standard SKUs are not available for Data Lake gen2 storage. Allowed: `Premium_LRS`, `Premium_ZRS`. | "Premium_LRS" | +| **dataExplorerName** | String | Optional. Name of the Azure Data Explorer cluster to use for advanced analytics. If empty, Azure Data Explorer will not be deployed. Required to use with Power BI if you have more than $2-5M/mo in costs being monitored. Default: "" (do not use). | | +| **dataExplorerSkuName** | String | Optional. Name of the Azure Data Explorer SKU. Default: "Dev(No SLA)_Standard_E2a_v4". | | +| **dataExplorerSkuTier** | String | Optional. SKU tier for the Azure Data Explorer cluster. Use Basic for the lowest cost with no SLA (due to a single node). Use Standard for high availability and improved performance. Allowed values: Basic, Standard. Default: "Basic". | | +| **dataExplorerSkuCapacity** | Int | Optional. Number of nodes to use in the cluster. Allowed values: 1 for the Basic SKU tier and 2-1000 for Standard. Default: 1. | | +| **tags** | Object | Optional. Tags to apply to all resources. We will also add the `cm-resource-parent` tag for improved cost roll-ups in Cost Management. | | +| **tagsByResource** | Object | Optional. Tags to apply to resources based on their resource type. Resource type specific tags will be merged with tags for all resources. | | +| **scopesToMonitor** | Array | Optional. List of scope IDs to monitor and ingest cost for. | | +| **exportRetentionInDays** | Int | Optional. Number of days of data to retain in the msexports container. | 0 | +| **ingestionRetentionInMonths** | Int | Optional. Number of months of data to retain in the ingestion container. | 13 | +| **dataExplorerLogRetentionInDays** | Int | Optional. Number of days of data to retain in the Data Explorer \*_log tables. | 0 | +| **dataExplorerFinalRetentionInMonths** | Int | Optional. Number of months of data to retain in the Data Explorer \*_final_v\* tables. | 13 | +| **remoteHubStorageUri** | String | Optional. Data Lake storage endpoint from the remote (primary) hub storage account. Used for cross-tenant cost data collection where this hub sends processed data to a central hub. Example: `https://primaryhub.dfs.core.windows.net/` | | +| **remoteHubStorageKey** | String | Optional. Storage account access key for the remote (primary) hub. Used with remoteHubStorageUri for cross-tenant scenarios. Must be kept secure as it provides full storage access. | | | **enableManagedExports** | Bool | Optional. Enable managed exports where your FinOps hub instance will create and run Cost Management exports on your behalf. Not supported for Microsoft Customer Agreement (MCA) billing profiles. Requires the ability to grant User Access Administrator role to FinOps hubs, which is required to create Cost Management exports. | True | | **enablePublicAccess** | Bool | Optional. Disable public access to the data lake (storage firewall). | True | | **virtualNetworkAddressPrefix** | String | Optional. IP Address range for the private virtual network used by FinOps hubs. Accepts any subnet size from `/8` to `/26` with a minimum of `/26` required. `/26` is recommended to avoid wasting IPs unless you need additional address space for services like Power BI VNet Data Gateway. Internally, the following subnets will be created: `/28` for private endpoints, another `/28` subnet for temporary deployment scripts (container instances), and `/27` for Azure Data Explorer, if enabled. | '10.20.30.0/26' | diff --git a/docs-mslearn/toolkit/powershell/hubs/deploy-finopshub.md b/docs-mslearn/toolkit/powershell/hubs/deploy-finopshub.md index 3437a6527..f64bdf653 100644 --- a/docs-mslearn/toolkit/powershell/hubs/deploy-finopshub.md +++ b/docs-mslearn/toolkit/powershell/hubs/deploy-finopshub.md @@ -30,6 +30,8 @@ Deploy-FinOpsHub ` [-Version ] ` [-Preview] ` [-StorageSku ] ` + [-RemoteHubStorageUri ] ` + [-RemoteHubStorageKey ] ` [-Tags ] ` [] ``` @@ -38,15 +40,17 @@ Deploy-FinOpsHub ` ## Parameters -| Name | Description | -| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `‑Name` | Required. Name of the FinOps hub instance. | -| `‑ResourceGroup` | Required. Name of the resource group to deploy to. It gets created if it doesn't exist. | -| `‑Location` | Required. Azure location to execute the deployment from. | -| `‑Version` | Optional. Version of the FinOps hub template to use. Default = "latest". | -| `‑Preview` | Optional. Indicates that preview releases should also be included. Default = false. | -| `‑StorageSku` | Optional. Storage account SKU. Premium_LRS = Lowest cost, Premium_ZRS = High availability. Note Standard SKUs aren't available for Data Lake gen2 storage. Default = "Premium_LRS". | -| `‑Tags` | Optional. Tags for all resources. | +| Name | Description | +| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `‑Name` | Required. Name of the FinOps hub instance. | +| `‑ResourceGroup` | Required. Name of the resource group to deploy to. It gets created if it doesn't exist. | +| `‑Location` | Required. Azure location to execute the deployment from. | +| `‑Version` | Optional. Version of the FinOps hub template to use. Default = "latest". | +| `‑Preview` | Optional. Indicates that preview releases should also be included. Default = false. | +| `‑StorageSku` | Optional. Storage account SKU. Premium_LRS = Lowest cost, Premium_ZRS = High availability. Note Standard SKUs aren't available for Data Lake gen2 storage. Default = "Premium_LRS". | +| `‑RemoteHubStorageUri` | Optional. Data Lake storage endpoint from the remote hub storage account. Used for cross-tenant cost data collection scenarios. Example: `https://primaryhub.dfs.core.windows.net/` | +| `‑RemoteHubStorageKey` | Optional. Storage account access key for the remote hub. Used for cross-tenant cost data collection scenarios. Must be kept secure as it provides full storage access. | +| `‑Tags` | Optional. Tags for all resources. |
@@ -77,6 +81,19 @@ Deploy-FinOpsHub ` Deploys a FinOps hub instance named MyHub to the MyExistingResourceGroup resource group using version 0.1.1 of the template. This version is required for Microsoft Online Services Agreement (MOSA) subscriptions since FOCUS exports aren't available from Cost Management. If the resource group doesn't exist, it gets created. If the hub already exists, it gets updated to version 0.1.1. +### Deploy with remote hub configuration + +```powershell +Deploy-FinOpsHub ` + -Name MyRemoteHub ` + -ResourceGroup MyRemoteHubResourceGroup ` + -Location westus ` + -RemoteHubStorageUri "https://centralfinooshub123.dfs.core.windows.net/" ` + -RemoteHubStorageKey "abc123...xyz789==" +``` + +Deploys a FinOps hub instance named MyRemoteHub configured to send data to a remote (central) hub. The remote hub storage URI and key enable cross-tenant data collection scenarios where a central tenant aggregates cost data from multiple tenants. The RemoteHubStorageUri should be copied from the central hub's storage account Settings > Endpoints > Data Lake storage, and the RemoteHubStorageKey should be copied from Security + networking > Access keys. Remote hubs require template version 0.4 or later. +
## Give feedback diff --git a/docs/hubs.md b/docs/hubs.md index 220209fe8..f64689ed4 100644 --- a/docs/hubs.md +++ b/docs/hubs.md @@ -211,6 +211,10 @@ Create a new or update an existing FinOps hub instance.
Details about what's included in the FinOps hub deployment template.
 
+
+ +
Configure cross-tenant data collection where satellite hubs send data to a central hub.
+
Identify breaking changes in each release that may require additional work when upgrading.
diff --git a/src/powershell/Tests/Unit/Deploy-FinOpsHub.Tests.ps1 b/src/powershell/Tests/Unit/Deploy-FinOpsHub.Tests.ps1 index aba1cc039..445bb8689 100644 --- a/src/powershell/Tests/Unit/Deploy-FinOpsHub.Tests.ps1 +++ b/src/powershell/Tests/Unit/Deploy-FinOpsHub.Tests.ps1 @@ -161,6 +161,34 @@ InModuleScope 'FinOpsToolkit' { } } -Times 1 } + + It 'Should deploy the template with RemoteHubStorageUri' { + $remoteHubStorageUri = 'https://primaryhub.dfs.core.windows.net/' + { Deploy-FinOpsHub -Name $hubName -ResourceGroup $rgName -Location $location -RemoteHubStorageUri $remoteHubStorageUri -Version 'latest' } | Should -Not -Throw + Assert-MockCalled -CommandName 'Get-ChildItem' -Times 1 + Assert-MockCalled -CommandName 'New-AzResourceGroupDeployment' -ParameterFilter { + $TemplateParameterObject.remoteHubStorageUri -eq $remoteHubStorageUri + } -Times 1 + } + + It 'Should deploy the template with RemoteHubStorageKey' { + $remoteHubStorageKey = 'abc123...xyz789==' + { Deploy-FinOpsHub -Name $hubName -ResourceGroup $rgName -Location $location -RemoteHubStorageKey $remoteHubStorageKey -Version 'latest' } | Should -Not -Throw + Assert-MockCalled -CommandName 'Get-ChildItem' -Times 1 + Assert-MockCalled -CommandName 'New-AzResourceGroupDeployment' -ParameterFilter { + $TemplateParameterObject.remoteHubStorageKey -eq $remoteHubStorageKey + } -Times 1 + } + + It 'Should deploy the template with both RemoteHub parameters' { + $remoteHubStorageUri = 'https://primaryhub.dfs.core.windows.net/' + $remoteHubStorageKey = 'abc123...xyz789==' + { Deploy-FinOpsHub -Name $hubName -ResourceGroup $rgName -Location $location -RemoteHubStorageUri $remoteHubStorageUri -RemoteHubStorageKey $remoteHubStorageKey -Version 'latest' } | Should -Not -Throw + Assert-MockCalled -CommandName 'Get-ChildItem' -Times 1 + Assert-MockCalled -CommandName 'New-AzResourceGroupDeployment' -ParameterFilter { + $TemplateParameterObject.remoteHubStorageUri -eq $remoteHubStorageUri -and $TemplateParameterObject.remoteHubStorageKey -eq $remoteHubStorageKey + } -Times 1 + } } } } diff --git a/src/templates/finops-hub/createUiDefinition.json b/src/templates/finops-hub/createUiDefinition.json index 16b32e2ef..201aefd47 100644 --- a/src/templates/finops-hub/createUiDefinition.json +++ b/src/templates/finops-hub/createUiDefinition.json @@ -696,6 +696,51 @@ } ], "visible": true + }, + { + "name": "remoteHub", + "type": "Microsoft.Common.Section", + "label": "Remote hub configuration", + "elements": [ + { + "name": "remoteHubIntro", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "Configure this hub to send data to a remote FinOps hub in another tenant or subscription. This enables cross-tenant cost management scenarios where a central tenant collects cost data from multiple tenants. Leave these fields empty if this is not a remote hub setup." + } + }, + { + "name": "remoteHubStorageUri", + "type": "Microsoft.Common.TextBox", + "label": "Remote hub storage URI", + "toolTip": "Data Lake storage endpoint from the remote hub storage account. Copy from the storage account Settings > Endpoints > Data Lake storage. Example: https://myremotehub.dfs.core.windows.net/", + "constraints": { + "required": false, + "regex": "^$|^https://.*\\.dfs\\.core\\.windows\\.net/?$", + "validationMessage": "Must be a valid Data Lake storage endpoint URL in the format: https://storageaccount.dfs.core.windows.net/" + }, + "visible": true + }, + { + "name": "remoteHubStorageKey", + "type": "Microsoft.Common.PasswordBox", + "label": { + "password": "Remote hub storage key" + }, + "toolTip": "Storage account access key for the remote hub. Copy from the remote hub storage account Security + networking > Access keys > key1/2 > Key.", + "constraints": { + "required": false, + "regex": "^$|^[A-Za-z0-9+/]{86}==$", + "validationMessage": "Must be a valid storage account access key (base64 encoded, ending with ==)" + }, + "options": { + "hideConfirmation": true + }, + "visible": true + } + ], + "visible": true } ] }, @@ -739,6 +784,8 @@ "ingestionRetentionInMonths": "[steps('retention').storage.ingestionMonths]", "dataExplorerRawRetentionInDays": "[steps('retention').dataExplorer.rawDays]", "dataExplorerFinalRetentionInMonths": "[steps('retention').dataExplorer.finalMonths]", + "remoteHubStorageUri": "[steps('advanced').remoteHub.remoteHubStorageUri]", + "remoteHubStorageKey": "[steps('advanced').remoteHub.remoteHubStorageKey]", "tagsByResource": "[steps('tags').tagsByResource]" } } From a45450848b522552da0977c2d3e8849731d4b41a Mon Sep 17 00:00:00 2001 From: Brett Wilson Date: Sat, 11 Oct 2025 15:11:35 -0700 Subject: [PATCH 23/69] Fix finops-workbooks Bicep validation - correct .build.config paths (#1863) Co-authored-by: msbrett --- src/templates/finops-workbooks/.build.config | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/templates/finops-workbooks/.build.config b/src/templates/finops-workbooks/.build.config index e60e6defc..a5b5943dd 100644 --- a/src/templates/finops-workbooks/.build.config +++ b/src/templates/finops-workbooks/.build.config @@ -6,7 +6,7 @@ "variableExpansion": [], "move": [ { - "path": "../governance-workbook", + "path": "../../release/governance-workbook", "destination": "workbooks/governance", "ignore": [ "azuredeploy*.json", @@ -17,7 +17,7 @@ ] }, { - "path": "../optimization-workbook", + "path": "../../release/optimization-workbook", "destination": "workbooks/optimization", "ignore": [ "azuredeploy*.json", From c91b2e96da29f45741b446392a14514517527924 Mon Sep 17 00:00:00 2001 From: Brett Wilson Date: Sat, 11 Oct 2025 16:53:33 -0700 Subject: [PATCH 24/69] Fix BCP318 warnings in finops-workbooks conditional outputs (#1865) Co-authored-by: msbrett --- src/templates/finops-workbooks/main.bicep | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/templates/finops-workbooks/main.bicep b/src/templates/finops-workbooks/main.bicep index ebd9bb532..6dfe815cf 100644 --- a/src/templates/finops-workbooks/main.bicep +++ b/src/templates/finops-workbooks/main.bicep @@ -104,13 +104,13 @@ module governance 'workbooks/governance/main.bicep' = if (includeGovernance) { //============================================================================== @sys.description('Optimization workbook resource ID.') -output optimizationId string = optimization.outputs.workbookId +output optimizationId string = includeOptimization ? optimization!.outputs.workbookId : '' @sys.description('Optimization workbook Azure portal link.') -output optimizationUrl string = optimization.outputs.workbookUrl +output optimizationUrl string = includeOptimization ? optimization!.outputs.workbookUrl : '' @sys.description('Governance workbook resource ID.') -output governanceId string = governance.outputs.workbookId +output governanceId string = includeGovernance ? governance!.outputs.workbookId : '' @sys.description('Governance workbook Azure portal link.') -output governanceUrl string = governance.outputs.workbookUrl +output governanceUrl string = includeGovernance ? governance!.outputs.workbookUrl : '' From 7a99fbfd495b38160eba584a46adb00e8ab490f4 Mon Sep 17 00:00:00 2001 From: Brett Wilson Date: Sat, 11 Oct 2025 20:19:47 -0700 Subject: [PATCH 25/69] Move merged v13 changes from Unreleased to v13 in changelog (#1866) Co-authored-by: msbrett --- docs-mslearn/toolkit/changelog.md | 40 +++++++++++++------------------ 1 file changed, 17 insertions(+), 23 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 452ff770b..4f38a7adf 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -25,33 +25,10 @@ This article summarizes the features and enhancements in each release of the Fin The following section lists features and enhancements that are currently in development. -### [FinOps hubs](hubs/finops-hubs-overview.md) - -- **Added** - - Document [how to remove private networking](hubs/private-networking.md#removing-private-networking) and switch back to public access to reduce costs ([#1342](https://github.com/microsoft/finops-toolkit/issues/1342)). - -### [Optimization engine](optimization-engine/overview.md) - -- **Fixed** - - Reservations-related workbooks fixed by replacing Instance Size Flexibility ratios CSV vanity URL with actual one to work around Log Analytics externaldata limitation ([#1810](https://github.com/microsoft/finops-toolkit/issues/1810)). - - Underutilized disks recommendations were not being generated when customer environment has Premium SSD V2 disks ([#1831](https://github.com/microsoft/finops-toolkit/issues/1831)). - ### Bicep Registry module pending updates - Cost Management export modules for subscriptions and resource groups. -### [Power BI reports](power-bi/reports.md) - -- **Added** - - Added export requirements sections to all Power BI report documentation pages to clarify which Cost Management exports are needed for each report. - - Added Azure Resource Graph as an explicit requirement for governance and workload optimization reports. - -### Documentation improvements - -- **Added** - - Created comprehensive [Data Lake Storage connectivity options](data-lake-storage-connectivity.md) documentation covering tools and services beyond Power BI including Azure Data Explorer, Microsoft Fabric, Azure Synapse Analytics, Azure Databricks, and custom applications. - - Enhanced troubleshooting guidance for [Data Explorer ingestion errors](help/errors.md#dataexploreringestionfailed) with detailed steps for diagnosing and resolving the common SEM0080 "Ingestion Failed" semantic error, including schema mismatch detection, ingestion mapping verification, and diagnostic query examples. -
@@ -61,6 +38,8 @@ _Released August 2025_ ### [FinOps hubs](hubs/finops-hubs-overview.md) v13 +- **Added** + - Document [how to remove private networking](hubs/private-networking.md#removing-private-networking) and switch back to public access to reduce costs ([#1342](https://github.com/microsoft/finops-toolkit/issues/1342)). - **Changed** - Reorganized Bicep modules into separate apps. - Enhanced [Configure scopes documentation](hubs/configure-scopes.md) to explicitly clarify that FinOps hubs support: @@ -70,8 +49,17 @@ _Released August 2025_ - Fixed all Bicep compilation errors and warnings with inline suppressions and descriptive comments. - Fixed Build-Toolkit.ps1 bicep generate-params command bug. +### [Optimization engine](optimization-engine/overview.md) v13 + +- **Fixed** + - Reservations-related workbooks fixed by replacing Instance Size Flexibility ratios CSV vanity URL with actual one to work around Log Analytics externaldata limitation ([#1810](https://github.com/microsoft/finops-toolkit/issues/1810)). + - Underutilized disks recommendations were not being generated when customer environment has Premium SSD V2 disks ([#1831](https://github.com/microsoft/finops-toolkit/issues/1831)). + ### [Power BI reports](power-bi/reports.md) v13 +- **Added** + - Added export requirements sections to all Power BI report documentation pages to clarify which Cost Management exports are needed for each report. + - Added Azure Resource Graph as an explicit requirement for governance and workload optimization reports. - **Fixed** - Fixed tag expansion in Power BI reports when tag names contain special characters like colons. @@ -82,6 +70,12 @@ _Released August 2025_ - **Changed** - Updated FinOps framework documentation to prepare for Azure TCO calculator retirement scheduled for August 31, 2025. Azure Migrate cost estimation functionality remains available. +### Documentation improvements v13 + +- **Added** + - Created comprehensive [Data Lake Storage connectivity options](data-lake-storage-connectivity.md) documentation covering tools and services beyond Power BI including Azure Data Explorer, Microsoft Fabric, Azure Synapse Analytics, Azure Databricks, and custom applications. + - Enhanced troubleshooting guidance for [Data Explorer ingestion errors](help/errors.md#dataexploreringestionfailed) with detailed steps for diagnosing and resolving the common SEM0080 "Ingestion Failed" semantic error, including schema mismatch detection, ingestion mapping verification, and diagnostic query examples. +
## v12 From 456fd1cf288454607c4e626e7337f3fa19ff8c43 Mon Sep 17 00:00:00 2001 From: Roland Krummenacher Date: Fri, 12 Dec 2025 11:55:02 +0100 Subject: [PATCH 26/69] fix: decimal to real (#1893) Co-authored-by: Roland Krummenacher --- src/templates/finops-hub/dashboard.json | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/templates/finops-hub/dashboard.json b/src/templates/finops-hub/dashboard.json index 94dcb70b0..7c04b3a88 100644 --- a/src/templates/finops-hub/dashboard.json +++ b/src/templates/finops-hub/dashboard.json @@ -3083,13 +3083,13 @@ }, { "dataSource": { "kind": "inline", "dataSourceId": "23540be2-ffc9-4b61-8c4c-05e493e682a6" }, - "text": "let data = materialize(\n CostsByMonth\n //\n | where isnotempty(CommitmentDiscountStatus)\n //\n // Guarantee there's a row for every combination\n | union (\n print json = dynamic([\n {\"order\": 11, \"CommitmentDiscountType\": \"Reservation\", \"CommitmentDiscountStatus\": \"Used\"},\n {\"order\": 12, \"CommitmentDiscountType\": \"Reservation\", \"CommitmentDiscountStatus\": \"Unused\"},\n {\"order\": 21, \"CommitmentDiscountType\": \"Savings Plan\", \"CommitmentDiscountStatus\": \"Used\"},\n {\"order\": 22, \"CommitmentDiscountType\": \"Savings Plan\", \"CommitmentDiscountStatus\": \"Unused\"}\n ])\n | mv-expand json\n | evaluate bag_unpack(json)\n | extend EffectiveCost = todecimal(0)\n )\n //\n | summarize Value = sum(EffectiveCost), order = sum(order) by CommitmentDiscountStatus, CommitmentDiscountType\n | order by order asc\n | project Label = strcat(CommitmentDiscountStatus, ' ', tolower(CommitmentDiscountType), 's'), Value\n);\ndata", + "text": "let data = materialize(\n CostsByMonth\n //\n | where isnotempty(CommitmentDiscountStatus)\n //\n // Guarantee there's a row for every combination\n | union (\n print json = dynamic([\n {\"order\": 11, \"CommitmentDiscountType\": \"Reservation\", \"CommitmentDiscountStatus\": \"Used\"},\n {\"order\": 12, \"CommitmentDiscountType\": \"Reservation\", \"CommitmentDiscountStatus\": \"Unused\"},\n {\"order\": 21, \"CommitmentDiscountType\": \"Savings Plan\", \"CommitmentDiscountStatus\": \"Used\"},\n {\"order\": 22, \"CommitmentDiscountType\": \"Savings Plan\", \"CommitmentDiscountStatus\": \"Unused\"}\n ])\n | mv-expand json\n | evaluate bag_unpack(json)\n | extend EffectiveCost = toreal(0)\n )\n //\n | summarize Value = sum(EffectiveCost), order = sum(order) by CommitmentDiscountStatus, CommitmentDiscountType\n | order by order asc\n | project Label = strcat(CommitmentDiscountStatus, ' ', tolower(CommitmentDiscountType), 's'), Value\n);\ndata", "id": "7d0c2c1f-338b-4534-b173-36b284779131", "usedVariables": ["CostsByMonth"] }, { "dataSource": { "kind": "inline", "dataSourceId": "23540be2-ffc9-4b61-8c4c-05e493e682a6" }, - "text": "let data = materialize(\n CostsByDay\n //\n // Don't double-count commitment discount purchases\n | where isnotempty(CommitmentDiscountStatus)\n //\n // Guarantee there's a row for every combination\n | union (\n print json = dynamic([\n {\"order\": 11, \"CommitmentDiscountType\": \"Reservation\", \"CommitmentDiscountStatus\": \"Used\"},\n {\"order\": 12, \"CommitmentDiscountType\": \"Reservation\", \"CommitmentDiscountStatus\": \"Unused\"},\n {\"order\": 21, \"CommitmentDiscountType\": \"Savings Plan\", \"CommitmentDiscountStatus\": \"Used\"},\n {\"order\": 22, \"CommitmentDiscountType\": \"Savings Plan\", \"CommitmentDiscountStatus\": \"Unused\"}\n ])\n | mv-expand json\n | evaluate bag_unpack(json)\n | extend EffectiveCost = todecimal(0)\n )\n //\n | summarize Value = sum(EffectiveCost), order = sum(order) by CommitmentDiscountStatus, CommitmentDiscountType\n | order by order asc\n | project Label = strcat(CommitmentDiscountStatus, ' ', tolower(CommitmentDiscountType), 's'), Value\n);\ndata", + "text": "let data = materialize(\n CostsByDay\n //\n // Don't double-count commitment discount purchases\n | where isnotempty(CommitmentDiscountStatus)\n //\n // Guarantee there's a row for every combination\n | union (\n print json = dynamic([\n {\"order\": 11, \"CommitmentDiscountType\": \"Reservation\", \"CommitmentDiscountStatus\": \"Used\"},\n {\"order\": 12, \"CommitmentDiscountType\": \"Reservation\", \"CommitmentDiscountStatus\": \"Unused\"},\n {\"order\": 21, \"CommitmentDiscountType\": \"Savings Plan\", \"CommitmentDiscountStatus\": \"Used\"},\n {\"order\": 22, \"CommitmentDiscountType\": \"Savings Plan\", \"CommitmentDiscountStatus\": \"Unused\"}\n ])\n | mv-expand json\n | evaluate bag_unpack(json)\n | extend EffectiveCost = toreal(0)\n )\n //\n | summarize Value = sum(EffectiveCost), order = sum(order) by CommitmentDiscountStatus, CommitmentDiscountType\n | order by order asc\n | project Label = strcat(CommitmentDiscountStatus, ' ', tolower(CommitmentDiscountType), 's'), Value\n);\ndata", "id": "84ecad69-79ac-45b9-a8af-60be28dcc748", "usedVariables": ["CostsByDay"] }, @@ -3245,7 +3245,7 @@ }, { "dataSource": { "kind": "inline", "dataSourceId": "23540be2-ffc9-4b61-8c4c-05e493e682a6" }, - "text": "CostsByMonth\n| extend x_AmortizationClass = case(\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory), 'Principal',\n isnotempty(CommitmentDiscountCategory), 'Amortized Charge',\n ''\n)\n| extend x_CommitmentDiscountSavings = iff(ContractedCost == 0, decimal(0), ContractedCost - EffectiveCost)\n| extend x_NegotiatedDiscountSavings = iff(ListCost == 0, decimal(0), ListCost - ContractedCost)\n| extend x_TotalSavings = iff(ListCost == 0, decimal(0), ListCost - EffectiveCost)\n| summarize\n ['List cost'] = round(sumif(ListCost, x_AmortizationClass != 'Principal'), 2),\n ['Effective cost'] = round(sum(EffectiveCost), 2),\n Savings = round(sum(x_TotalSavings), 2)\n by\n Account = x_BillingProfileId,\n Month = substring(startofmonth(ChargePeriodStart), 0, 7)\n| extend ESR = percentstring(Savings/ ['List cost'])\n| order by Month desc", + "text": "CostsByMonth\n| extend x_AmortizationClass = case(\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory), 'Principal',\n isnotempty(CommitmentDiscountCategory), 'Amortized Charge',\n ''\n)\n| extend x_CommitmentDiscountSavings = iff(ContractedCost == 0, real(0), ContractedCost - EffectiveCost)\n| extend x_NegotiatedDiscountSavings = iff(ListCost == 0, real(0), ListCost - ContractedCost)\n| extend x_TotalSavings = iff(ListCost == 0, real(0), ListCost - EffectiveCost)\n| summarize\n ['List cost'] = round(sumif(ListCost, x_AmortizationClass != 'Principal'), 2),\n ['Effective cost'] = round(sum(EffectiveCost), 2),\n Savings = round(sum(x_TotalSavings), 2)\n by\n Account = x_BillingProfileId,\n Month = substring(startofmonth(ChargePeriodStart), 0, 7)\n| extend ESR = percentstring(Savings/ ['List cost'])\n| order by Month desc", "id": "3b3f0a58-2d84-4e3e-bebc-3e747a7d5ede", "usedVariables": ["CostsByMonth"] }, @@ -3263,7 +3263,7 @@ }, { "dataSource": { "kind": "inline", "dataSourceId": "23540be2-ffc9-4b61-8c4c-05e493e682a6" }, - "text": "CostsByMonth\n| extend x_AmortizationClass = case(\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory), 'Principal',\n isnotempty(CommitmentDiscountCategory), 'Amortized Charge',\n ''\n)\n| extend x_CommitmentDiscountSavings = iff(ContractedCost == 0, decimal(0), ContractedCost - EffectiveCost)\n| extend x_NegotiatedDiscountSavings = iff(ListCost == 0, decimal(0), ListCost - ContractedCost)\n| extend x_TotalSavings = iff(ListCost == 0, decimal(0), ListCost - EffectiveCost)\n| summarize\n ['On-demand'] = round(sumif(EffectiveCost, PricingCategory == 'Standard'), 2),\n Spot = round(sumif(EffectiveCost, PricingCategory == 'Dynamic'), 2),\n Reservation = round(sumif(EffectiveCost, CommitmentDiscountType == 'Reservation'), 2),\n ['Savings plan'] = round(sumif(EffectiveCost, CommitmentDiscountType == 'Savings Plan'), 2),\n ['Other'] = round(sumif(EffectiveCost, PricingCategory !in ('Standard', 'Dynamic') and isempty(CommitmentDiscountType)), 2)\n by\n Account = x_BillingProfileId,\n Month = substring(startofmonth(ChargePeriodStart), 0, 7)\n| order by Month desc", + "text": "CostsByMonth\n| extend x_AmortizationClass = case(\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory), 'Principal',\n isnotempty(CommitmentDiscountCategory), 'Amortized Charge',\n ''\n)\n| extend x_CommitmentDiscountSavings = iff(ContractedCost == 0, real(0), ContractedCost - EffectiveCost)\n| extend x_NegotiatedDiscountSavings = iff(ListCost == 0, real(0), ListCost - ContractedCost)\n| extend x_TotalSavings = iff(ListCost == 0, real(0), ListCost - EffectiveCost)\n| summarize\n ['On-demand'] = round(sumif(EffectiveCost, PricingCategory == 'Standard'), 2),\n Spot = round(sumif(EffectiveCost, PricingCategory == 'Dynamic'), 2),\n Reservation = round(sumif(EffectiveCost, CommitmentDiscountType == 'Reservation'), 2),\n ['Savings plan'] = round(sumif(EffectiveCost, CommitmentDiscountType == 'Savings Plan'), 2),\n ['Other'] = round(sumif(EffectiveCost, PricingCategory !in ('Standard', 'Dynamic') and isempty(CommitmentDiscountType)), 2)\n by\n Account = x_BillingProfileId,\n Month = substring(startofmonth(ChargePeriodStart), 0, 7)\n| order by Month desc", "id": "0341b3e4-eccc-4924-9555-9835b128c543", "usedVariables": ["CostsByMonth"] }, From 8a13be511354e64a9a4c0f5775628ffb178c2e94 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Fri, 12 Dec 2025 03:14:39 -0800 Subject: [PATCH 27/69] docs: Add changelog entry for decimal to real bug fix (#1901) Co-authored-by: Roland Krummenacher Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: RolandKrummenacher <1803486+RolandKrummenacher@users.noreply.github.com> --- docs-mslearn/toolkit/changelog.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 4f38a7adf..db543056d 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -48,6 +48,7 @@ _Released August 2025_ - **Fixed** - Fixed all Bicep compilation errors and warnings with inline suppressions and descriptive comments. - Fixed Build-Toolkit.ps1 bicep generate-params command bug. + - Fixed Azure Data Explorer dashboard queries by converting `todecimal(0)` to `toreal(0)` to ensure compatibility with KQL type system ([#1893](https://github.com/microsoft/finops-toolkit/issues/1893)). ### [Optimization engine](optimization-engine/overview.md) v13 From 91aa821ead68525f34a868dc4bc21bcf6b3089c6 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Tue, 6 Jan 2026 01:48:27 -0800 Subject: [PATCH 28/69] Fix ADF pipeline dependency logic for scope array handling (#1913) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: RolandKrummenacher <1803486+RolandKrummenacher@users.noreply.github.com> --- docs-mslearn/toolkit/changelog.md | 1 + .../modules/Microsoft.CostManagement/ManagedExports/app.bicep | 3 +++ 2 files changed, 4 insertions(+) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index db543056d..fcc5c69dd 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -49,6 +49,7 @@ _Released August 2025_ - Fixed all Bicep compilation errors and warnings with inline suppressions and descriptive comments. - Fixed Build-Toolkit.ps1 bicep generate-params command bug. - Fixed Azure Data Explorer dashboard queries by converting `todecimal(0)` to `toreal(0)` to ensure compatibility with KQL type system ([#1893](https://github.com/microsoft/finops-toolkit/issues/1893)). + - Fixed ADF pipeline dependency logic in config_RunBackfillJob, config_StartExportProcess, and config_ConfigureExports pipelines to properly handle both array and non-array scope configurations by adding 'Failed' condition to 'Save/Set Scopes' activity dependencies. ### [Optimization engine](optimization-engine/overview.md) v13 diff --git a/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep b/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep index 848fc8aac..9d9151ed1 100644 --- a/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep +++ b/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep @@ -496,6 +496,7 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { activity: 'Set Scopes' dependencyConditions: [ 'Succeeded' + 'Failed' ] } { @@ -742,6 +743,7 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { activity: 'Set Scopes' dependencyConditions: [ 'Succeeded' + 'Failed' ] } { @@ -1075,6 +1077,7 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { activity: 'Save Scopes' dependencyConditions: [ 'Succeeded' + 'Failed' ] } { From cdd589df26edba7d7fd21a82ce1728ca836c8372 Mon Sep 17 00:00:00 2001 From: Sander Naus Date: Tue, 6 Jan 2026 10:55:15 +0100 Subject: [PATCH 29/69] Fix scopesToMonitor parsing in script (#1882) --- docs-mslearn/toolkit/changelog.md | 3 +++ .../Microsoft.FinOpsHubs/Core/Copy-FileToAzureBlob.ps1 | 6 +++--- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index fcc5c69dd..45bb53d4b 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -29,6 +29,9 @@ The following section lists features and enhancements that are currently in deve - Cost Management export modules for subscriptions and resource groups. +### [FinOps hubs](hubs/finops-hubs-overview.md) +- **Fixed** + - Fixed logic to properly generate the scopes to monitor.
diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/Copy-FileToAzureBlob.ps1 b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/Copy-FileToAzureBlob.ps1 index 8e835d816..ffea9d454 100644 --- a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/Copy-FileToAzureBlob.ps1 +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/Copy-FileToAzureBlob.ps1 @@ -5,12 +5,12 @@ Write-Output "Updating settings.json file..." Write-Output " Storage account: $env:storageAccountName" Write-Output " Container: $env:containerName" -$validateScopes = { $_.Length -gt 45 } +$validateScopes = { $_.scope.Length -gt 45 } # Initialize variables $fileName = 'settings.json' $filePath = Join-Path -Path . -ChildPath $fileName -$newScopes = $env:scopes.Split('|') | Where-Object $validateScopes | ForEach-Object { @{ scope = $_ } } +$newScopes = $env:scopes.Split('|') | ForEach-Object { [PSCustomObject]@{ scope = $_ } } | Where-Object $validateScopes # Get storage context $storageContext = @{ @@ -157,7 +157,7 @@ else # Updating settings Write-Output "Updating version to $env:ftkVersion..." $json.version = $env:ftkVersion -$json.scopes = (@() + $json.scopes + $newScopes) | Select-Object -Unique +$json.scopes = ($json.scopes + $newScopes) | Sort-Object scope -Unique if ($null -eq $json.scopes) { $json.scopes = @() } $text = $json | ConvertTo-Json Write-Output "---------" From 624d365a4bcd06a430511a03fd3debb71244b15d Mon Sep 17 00:00:00 2001 From: "allcontributors[bot]" <46447321+allcontributors[bot]@users.noreply.github.com> Date: Tue, 6 Jan 2026 01:55:51 -0800 Subject: [PATCH 30/69] add sandernaus as a contributor for code (#1917) Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com> --- .all-contributorsrc | 9 +++++++++ README.md | 1 + docs/README.md | 1 + 3 files changed, 11 insertions(+) diff --git a/.all-contributorsrc b/.all-contributorsrc index 466ea0723..2884189d3 100644 --- a/.all-contributorsrc +++ b/.all-contributorsrc @@ -833,6 +833,15 @@ "contributions": [ "review" ] + }, + { + "login": "sandernaus", + "name": "Sander Naus", + "avatar_url": "https://avatars.githubusercontent.com/u/714779?v=4", + "profile": "http://www.sandernaus.com", + "contributions": [ + "code" + ] } ], "commitType": "docs", diff --git a/README.md b/README.md index 84712363d..fd1375aa3 100644 --- a/README.md +++ b/README.md @@ -139,6 +139,7 @@ There are many ways to participate. From reporting bugs and requesting features
+
Adeel Aziz
Adeel Aziz
💻		 shasulin
shasulin
👀		Sander Naus
Sander Naus
💻
diff --git a/docs/README.md b/docs/README.md index 15fbf8c6d..ce23968dd 100644 --- a/docs/README.md +++ b/docs/README.md @@ -209,6 +209,7 @@ Whether you're looking for a little assistance or are interested in contributing Adeel Aziz
Adeel Aziz
💻 shasulin
shasulin
👀 + Sander Naus
Sander Naus
💻 From 58cd97291d5691601234def6a72e78b9f98f0865 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Tue, 6 Jan 2026 02:36:21 -0800 Subject: [PATCH 31/69] Fix datatype mismatch in InitializeHub pipeline for PricingUnits ingestion (#1912) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: RolandKrummenacher <1803486+RolandKrummenacher@users.noreply.github.com> Co-authored-by: Michael Flanakin --- docs-mslearn/toolkit/changelog.md | 8 +++----- .../modules/Microsoft.FinOpsHubs/Analytics/app.bicep | 2 +- 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 45bb53d4b..3696aca84 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -29,15 +29,11 @@ The following section lists features and enhancements that are currently in deve - Cost Management export modules for subscriptions and resource groups. -### [FinOps hubs](hubs/finops-hubs-overview.md) -- **Fixed** - - Fixed logic to properly generate the scopes to monitor. -
## v13 -_Released August 2025_ +_Released January 2026_ ### [FinOps hubs](hubs/finops-hubs-overview.md) v13 @@ -52,6 +48,8 @@ _Released August 2025_ - Fixed all Bicep compilation errors and warnings with inline suppressions and descriptive comments. - Fixed Build-Toolkit.ps1 bicep generate-params command bug. - Fixed Azure Data Explorer dashboard queries by converting `todecimal(0)` to `toreal(0)` to ensure compatibility with KQL type system ([#1893](https://github.com/microsoft/finops-toolkit/issues/1893)). + - Fixed logic to properly generate the scopes to monitor. + - Fixed datatype mismatch in InitializeHub pipeline by changing `x_PricingBlockSize` from `decimal` to `real` to match PricingUnits table schema. - Fixed ADF pipeline dependency logic in config_RunBackfillJob, config_StartExportProcess, and config_ConfigureExports pipelines to properly handle both array and non-array scope configurations by adding 'Failed' condition to 'Save/Set Scopes' activity dependencies. ### [Optimization engine](optimization-engine/overview.md) v13 diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep index 6e290e2da..b689fc184 100644 --- a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep @@ -976,7 +976,7 @@ resource pipeline_InitializeHub 'Microsoft.DataFactory/factories/pipelines@2018- userProperties: [] typeProperties: { // cSpell:ignore externaldata - command: '.set-or-replace PricingUnits <| externaldata(x_PricingUnitDescription: string, AccountTypes: string, x_PricingBlockSize: decimal, PricingUnit: string)[@"${ftkReleaseUri}/PricingUnits.csv"] with (format="csv", ignoreFirstRecord=true) | project-away AccountTypes' + command: '.set-or-replace PricingUnits <| externaldata(x_PricingUnitDescription: string, AccountTypes: string, x_PricingBlockSize: real, PricingUnit: string)[@"${ftkReleaseUri}/PricingUnits.csv"] with (format="csv", ignoreFirstRecord=true) | project-away AccountTypes' commandTimeout: '00:20:00' } linkedServiceName: { From 8b570f236498fedceda022b8de27b152797d4a11 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Tue, 6 Jan 2026 02:45:44 -0800 Subject: [PATCH 32/69] Fix backward compatibility for pre-FOCUS 1.2 cost exports in Costs_transform_v1_2() (#1906) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: RolandKrummenacher <1803486+RolandKrummenacher@users.noreply.github.com> Co-authored-by: Michael Flanakin --- docs-mslearn/toolkit/changelog.md | 1 + .../Analytics/scripts/IngestionSetup_v1_2.kql | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 3696aca84..9ed9ac625 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -51,6 +51,7 @@ _Released January 2026_ - Fixed logic to properly generate the scopes to monitor. - Fixed datatype mismatch in InitializeHub pipeline by changing `x_PricingBlockSize` from `decimal` to `real` to match PricingUnits table schema. - Fixed ADF pipeline dependency logic in config_RunBackfillJob, config_StartExportProcess, and config_ConfigureExports pipelines to properly handle both array and non-array scope configurations by adding 'Failed' condition to 'Save/Set Scopes' activity dependencies. + - Fixed backward compatibility in `Costs_transform_v1_2()` to support Cost Management exports that predate FOCUS 1.2 by adding fallback mappings for `PricingCurrency` and `SkuMeter` columns to their legacy `x_` counterparts. ### [Optimization engine](optimization-engine/overview.md) v13 diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_v1_2.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_v1_2.kql index ae5a5d95c..2b36351ea 100644 --- a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_v1_2.kql +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_v1_2.kql @@ -748,7 +748,7 @@ Costs_transform_v1_2() ListCost, ListUnitPrice, PricingCategory, - PricingCurrency, + PricingCurrency = coalesce(PricingCurrency, x_PricingCurrency), PricingQuantity, PricingUnit, ProviderName, @@ -762,7 +762,7 @@ Costs_transform_v1_2() ServiceName, ServiceSubcategory, // TODO: Populate ServiceSubcategory from ServiceName when missing SkuId, - SkuMeter, + SkuMeter = coalesce(SkuMeter, x_SkuMeterName), SkuPriceDetails, SkuPriceId, SubAccountId, From 75f79ee9a6d6b08dd8b93201fc3fed167c2e8a44 Mon Sep 17 00:00:00 2001 From: "allcontributors[bot]" <46447321+allcontributors[bot]@users.noreply.github.com> Date: Tue, 6 Jan 2026 02:48:57 -0800 Subject: [PATCH 33/69] add gorkomikus as a contributor for code (#1918) Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com> --- .all-contributorsrc | 9 +++++++++ README.md | 1 + docs/README.md | 1 + 3 files changed, 11 insertions(+) diff --git a/.all-contributorsrc b/.all-contributorsrc index 2884189d3..76bce1622 100644 --- a/.all-contributorsrc +++ b/.all-contributorsrc @@ -842,6 +842,15 @@ "contributions": [ "code" ] + }, + { + "login": "gorkomikus", + "name": "gorkomikus", + "avatar_url": "https://avatars.githubusercontent.com/u/76739221?v=4", + "profile": "https://github.com/gorkomikus", + "contributions": [ + "code" + ] } ], "commitType": "docs", diff --git a/README.md b/README.md index fd1375aa3..40760a7b1 100644 --- a/README.md +++ b/README.md @@ -140,6 +140,7 @@ There are many ways to participate. From reporting bugs and requesting features Adeel Aziz
Adeel Aziz
💻 shasulin
shasulin
👀 Sander Naus
Sander Naus
💻 + gorkomikus
gorkomikus
💻 diff --git a/docs/README.md b/docs/README.md index ce23968dd..e8eba22b0 100644 --- a/docs/README.md +++ b/docs/README.md @@ -210,6 +210,7 @@ Whether you're looking for a little assistance or are interested in contributing Adeel Aziz
Adeel Aziz
💻 shasulin
shasulin
👀 Sander Naus
Sander Naus
💻 + gorkomikus
gorkomikus
💻 From 23294446ac51b6ab0604dcb61db1b5b16340145c Mon Sep 17 00:00:00 2001 From: Eskil Uhlving Larsen <7443949+picccard@users.noreply.github.com> Date: Tue, 6 Jan 2026 11:49:46 +0100 Subject: [PATCH 34/69] chore: update doc paths in PR template (#1872) --- .github/PULL_REQUEST_TEMPLATE.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 1a985d93c..9cf64e12b 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -39,6 +39,8 @@ Fixes # ### 📖 Did you update documentation? > - [ ] ✅ Public docs in `docs` (required for `dev`) +> - [ ] ✅ Public docs in `docs-mslearn` (required for `dev`) +> - [ ] ✅ Internal dev docs in `docs-wiki` (required for `dev`) > - [ ] ✅ Internal dev docs in `src` (required for `dev`) > - [ ] ➡️ Will add docs in a future PR (feature branch PRs only) > - [ ] ❎ Docs not needed (small/internal change) From 5320f693f90ba132366eeb8a4ffb4e7a200d890b Mon Sep 17 00:00:00 2001 From: Eskil Uhlving Larsen <7443949+picccard@users.noreply.github.com> Date: Tue, 6 Jan 2026 11:53:37 +0100 Subject: [PATCH 35/69] docs: normalize and update broken links in wiki (#1871) --- .../-internal-only--release-checklist.md | 7 ++--- README.md | 2 +- docs-wiki/About.md | 2 +- docs-wiki/Branching-strategy.md | 8 ++--- docs-wiki/Build-and-test.md | 12 ++++---- docs-wiki/Home.md | 8 ++--- docs-wiki/Release-process.md | 14 ++++----- docs-wiki/Support-escalations.md | 4 +-- docs-wiki/_Footer.md | 4 +-- docs-wiki/architecture.md | 29 ++++++++++--------- 10 files changed, 45 insertions(+), 45 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/-internal-only--release-checklist.md b/.github/ISSUE_TEMPLATE/-internal-only--release-checklist.md index c16ae4475..a1ec9d21c 100644 --- a/.github/ISSUE_TEMPLATE/-internal-only--release-checklist.md +++ b/.github/ISSUE_TEMPLATE/-internal-only--release-checklist.md @@ -71,10 +71,9 @@ Status icons: - [ ] Confirm all tests pass: `/src/scripts/Test-PowerShell.ps1 -Unit -Integration -Build`. - [ ] Confirm if all features are code complete and not missing any functionality required for release. > _Once in `dev`, the feature is considered part of the next release and can be pushed out at any time. Any broken features will be reverted._ - - [ ] Confirm new or updated functionality is documented in the [changelog](https://github.com/microsoft/finops-toolkit/blob/dev/docs/changelog.md). - > _See [Changelog guidance](#-changelog-guidance) for details about changelog requirements._ + - [ ] Confirm new or updated functionality is documented in the [changelog](https://github.com/microsoft/finops-toolkit/blob/dev/docs-mslearn/toolkit/changelog.md). - [ ] Confirm new or updated functionality must be documented in the [documentation](https://github.com/microsoft/finops-toolkit/blob/dev/docs). - - [ ] If adding a new tool, update the [list of available tools](https://github.com/microsoft/finops-toolkit/tree/dev/docs#-available-tools) on the documentation home page. + - [ ] If adding a new tool, update the [list of available tools](https://aka.ms/finops/toolkit#available-tools) on the documentation home page. - [ ] Merge any feature branches that are ready to `dev`. - [ ] Create a PR to merge the feature branch into `dev`. - [ ] Follow the normal PR process to merge the PR. @@ -83,7 +82,7 @@ Status icons: ## 🔜 Finalize release -- [ ] Review the [changelog](../docs/_resources/changelog.md) to ensure it encapsulates all changes. +- [ ] Review the [changelog](https://github.com/microsoft/finops-toolkit/blob/dev/docs-mslearn/toolkit/changelog.md) to ensure it encapsulates all changes. - Move all released changes to an official numbered version section. - If there are committed changes in a feature branch that you want to mention, add them to an "Unreleased" section. - [ ] Update the version. diff --git a/README.md b/README.md index 40760a7b1..5478d4f65 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ The FinOps toolkit is an open-source collection of customizable tools and resour - Advanced solutions to facilitate building custom capabilities. - Learning resources and best practices about how to implement FinOps. -[🧰 Learn about the tools](https://aka.ms/finops/toolkit#-available-tools) +[🧰 Learn about the tools](https://aka.ms/finops/toolkit#available-tools)
diff --git a/docs-wiki/About.md b/docs-wiki/About.md index 01bed4c32..a01241d12 100644 --- a/docs-wiki/About.md +++ b/docs-wiki/About.md @@ -24,7 +24,7 @@ To empower organizations to quantify and maximize the value they get from the Mi The FinOps toolkit is a labor of love from passionate people across the globe both inside and outside of Microsoft. There are many ways to contribute from feedback and discussions to submitting and reviewing documentation updates and code changes. We value any contribution, regardless of its size. -To learn more, refer to our [contribution guide](https://github.com/microsoft/finops-toolkit/blob/dev/CONTRIBUTING.md) +To learn more, refer to our [contribution guide](../tree/dev/CONTRIBUTING.md)
diff --git a/docs-wiki/Branching-strategy.md b/docs-wiki/Branching-strategy.md index 71fc0ef16..dc2d8b14f 100644 --- a/docs-wiki/Branching-strategy.md +++ b/docs-wiki/Branching-strategy.md @@ -1,7 +1,7 @@ -- [main](https://github.com/microsoft/finops-toolkit/tree/main) includes the latest stable release. -- [dev](https://github.com/microsoft/finops-toolkit/tree/dev) includes the latest changes that will go into the next release. +- [main](../tree/main) includes the latest stable release. +- [dev](../tree/dev) includes the latest changes that will go into the next release. - Feature branches (`features/`) are used for any in-progress features that are not yet ready for release. - Personal branches (`/`) are intended for a single developer and typically not shared. Use these for small changes that can easily be integrated into the next release. @@ -35,7 +35,7 @@ On this page: - `dev` is for the next release. All PRs to this branch must meet the following requirements: - Changes must be complete and validated. No partial commits. - Applicable documentation in [docs](../tree/dev/docs) must be updated. - - External-facing changes must be covered in the [changelog](../tree/dev/docs/changelog.md). + - External-facing changes must be covered in the [changelog](../tree/dev/docs-mslearn/toolkit/changelog.md). - `features/workbookv2` is for the Cost optimization workbook's next release. Target version: `0.1` (TBD). - `features/governance` is for a new Governance workbook. Target version: `0.1` (TBD). - `features/powershell` is for PowerShell automation. Target version: `0.1.*`. @@ -53,7 +53,7 @@ If contributing to an in-progress feature, switch to the feature branch and subm If contributing a new feature, switch to the `dev` branch and submit a PR back to the main repo's `dev` branch. You are free to invite others to contribute within your fork as needed. -If you run into any issues, please reach out to us on [Discussions](https://github.com/microsoft/finops-toolkit/discussions). We're happy to help! +If you run into any issues, please reach out to us on [Discussions](../discussions). We're happy to help!
diff --git a/docs-wiki/Build-and-test.md b/docs-wiki/Build-and-test.md index 1548bee3e..8410afcd9 100644 --- a/docs-wiki/Build-and-test.md +++ b/docs-wiki/Build-and-test.md @@ -32,7 +32,7 @@ cd "" src/scripts/Build-Toolkit "" ``` -To build all templates and modules, simply remove the template name or run `npm run build`, if you have NPM setup. To learn more about the build script, see [Build-Toolkit](https://github.com/microsoft/finops-toolkit/blob/dev/src/scripts/README.md#-build-toolkit). +To build all templates and modules, simply remove the template name or run `npm run build`, if you have NPM setup. To learn more about the build script, see [Build-Toolkit](../tree/dev/src/scripts/README.md#-build-toolkit). To build and deploy templates, run: @@ -41,9 +41,9 @@ cd "" src/scripts/Deploy-Toolkit "" -Build ``` -For more local deployment options, see [Deploy-Toolkit](https://github.com/microsoft/finops-toolkit/blob/dev/src/scripts/README.md#-deploy-toolkit). +For more local deployment options, see [Deploy-Toolkit](../tree/dev/src/scripts/README.md#-deploy-toolkit). -To learn more, see [FinOps toolkit templates](https://github.com/microsoft/finops-toolkit/blob/dev/src/templates/README.md). +To learn more, see [FinOps toolkit templates](../tree/dev/src/templates/README.md). ### Building workbooks @@ -65,7 +65,7 @@ src/scripts/Deploy-Toolkit "-workbook" -Build The Build-Toolkit script calls an internal Build-Workbook script, which does all the work. You can also call this directly; however, we recommend running the Build-Toolkit script for a complete build process. -To learn more, see [Workbook modules](https://github.com/microsoft/finops-toolkit/blob/dev/src/workbooks/README.md). +To learn more, see [Workbook modules](../tree/dev/src/workbooks/README.md). ### Building Bicep Registry modules @@ -92,7 +92,7 @@ cd "" src/scripts/Deploy-Toolkit "" -Build -Test ``` -To learn more, see [Bicep Registry modules](https://github.com/microsoft/finops-toolkit/blob/dev/src/bicep-registry/README.md). +To learn more, see [Bicep Registry modules](../tree/dev/src/bicep-registry/README.md). ### Building open data files @@ -104,7 +104,7 @@ To build open data, run: src/Build-OpenData ``` -To learn more about open data, see [Open data](https://github.com/microsoft/finops-toolkit/blob/dev/src/open-data/README.md). +To learn more about open data, see [Open data](../tree/dev/src/open-data/README.md). ### Building PowerShell diff --git a/docs-wiki/Home.md b/docs-wiki/Home.md index 21dfd7a06..acdd5d665 100644 --- a/docs-wiki/Home.md +++ b/docs-wiki/Home.md @@ -4,12 +4,12 @@ - Every folder has a README that explains its purpose. - If you want to know how to deploy a FinOps toolkit solution, start with the [documentation](https://aka.ms/finops/toolkit). -- If you want to know how you can contribute, check out the [contribution guide](https://github.com/microsoft/finops-toolkit/tree/dev/CONTRIBUTING.md). -- If you want to get started with the code, start in the [wiki](https://github.com/microsoft/finops-toolkit/wiki).   **← YOU ARE HERE** +- If you want to know how you can contribute, check out the [contribution guide](../tree/dev/CONTRIBUTING.md). +- If you want to get started with the code, start in the [wiki](../wiki).   **← YOU ARE HERE** - Read about our [[Architecture]] for context on technologies and structure. - Review our [[Coding guidelines]] before you write/review code. - Review the guidance below for how to contribute code. -- When you're ready to dig into code, you'll check the [src](https://github.com/microsoft/finops-toolkit/tree/dev/src) folder. +- When you're ready to dig into code, you'll check the [src](../tree/dev/src) folder.
@@ -27,7 +27,7 @@ On this page: ## 🛫 Get started -There are many ways to contribute to the FinOps toolkit project, like reporting issues, suggesting features, and submitting or reviewing pull requests. For an overview, refer to the [contribution guide](../CONTRIBUTING.md). This page covers how to contribute to the code. +There are many ways to contribute to the FinOps toolkit project, like reporting issues, suggesting features, and submitting or reviewing pull requests. For an overview, refer to the [contribution guide](../tree/dev/CONTRIBUTING.md). This page covers how to contribute to the code. After cloning and building the repo, check out the [issues list](../issues): diff --git a/docs-wiki/Release-process.md b/docs-wiki/Release-process.md index 3c758689f..55deb704f 100644 --- a/docs-wiki/Release-process.md +++ b/docs-wiki/Release-process.md @@ -18,10 +18,10 @@ When a feature branch is code-complete, it can be merged into `dev`. Before proc > _The PR will be blocked if they don't._ 2. Any new features are code complete and not missing any functionality required for release. > _Once in `dev`, the feature is considered part of the next release and can be pushed out at any time. Any broken features will be reverted._ -3. All new or updated functionality must be documented in the [changelog](https://github.com/microsoft/finops-toolkit/blob/dev/docs/changelog.md). +3. All new or updated functionality must be documented in the [changelog](../tree/dev/docs-mslearn/toolkit/changelog.md). > _See [Changelog guidance](#-changelog-guidance) for details about changelog requirements._ -4. All new or updated functionality must be documented in the [documentation](https://github.com/microsoft/finops-toolkit/blob/dev/docs). -5. Update the [list of available tools](https://github.com/microsoft/finops-toolkit/tree/dev/docs#-available-tools) on the documentation home page. +4. All new or updated functionality must be documented in the [documentation](../tree/dev/docs). +5. Update the [list of available tools](https://aka.ms/finops/toolkit#available-tools) on the documentation home page. Once the above requirements have been met, the feature branch can be merged into `dev` using the following steps: @@ -34,7 +34,7 @@ Once the above requirements have been met, the feature branch can be merged into ## 🚀 Publishing an official release -1. Review the [changelog](../docs/_resources/changelog.md) to ensure it encapsulates all changes. +1. Review the [changelog](../tree/dev/docs-mslearn/toolkit/changelog.md) to ensure it encapsulates all changes. - Move all released changes to an official numbered version section. - If there are committed changes in a feature branch that you want to mention, add them to an "Unreleased" section. 2. Update the version. @@ -86,7 +86,7 @@ Once the above requirements have been met, the feature branch can be merged into 5. Finalize the release. - 1. Update the [milestone](https://github.com/microsoft/finops-toolkit/milestones). + 1. Update the [milestone](../milestones). 1. Review all issues in the milestone, move anything that needs to be pushed, and close any completed items. 2. Close the milestone when all issues have been closed or moved. @@ -119,7 +119,7 @@ Once the above requirements have been met, the feature branch can be merged into 5. Verify [documentation](https://aka.ms/finops/toolkit) updated correctly. - > _The documentation site may take 5 minutes to update after the merge is committed. If not updated, look at [GitHub actions](https://github.com/microsoft/finops-toolkit/actions/workflows/pages/pages-build-deployment) to see if there are any failures._ + > _The documentation site may take 5 minutes to update after the merge is committed. If not updated, look at [GitHub actions](../actions/workflows/pages/pages-build-deployment) to see if there are any failures._ 6. Run `Package-Toolkit -Build -PowerBI` script. - For each Power BI report: @@ -136,7 +136,7 @@ Once the above requirements have been met, the feature branch can be merged into 6. Save PBIX again in the release folder. > ⚠️ _**DO NOT** save the above changes back to the Power BI project files!_ 7. Copy the first paragraph from the **Get started** page and export a template (PBIT file) in the release folder. Use the copied text for the description and add "Learn more at https://aka.ms/ftk/{report-name}" as a separate paragraph in the description. - 7. Tag and publish a [new release](https://github.com/microsoft/finops-toolkit/releases/new): + 7. Tag and publish a [new release](../releases/new): 1. Create a tag on publish using the "vX.X" format. 2. Set the **Target** to `main`. 3. Set the **Previous tag** to the previous release tag. diff --git a/docs-wiki/Support-escalations.md b/docs-wiki/Support-escalations.md index 211950874..d74553448 100644 --- a/docs-wiki/Support-escalations.md +++ b/docs-wiki/Support-escalations.md @@ -18,11 +18,11 @@ Tools and resources within the FinOps toolkit are provided as-is without any exp If you run into an issue, we recommend taking the following actions: -1. **Report security issues securely.**
If you believe you've found a security vulnerability, refer to [Reporting security issues](https://github.com/microsoft/finops-toolkit/blob/dev/SECURITY.md).
  +1. **Report security issues securely.**
If you believe you've found a security vulnerability, refer to [Reporting security issues](../tree/dev/SECURITY.md).
  2. **Confirm all setup instructions were completed in order.**
9 out of 10 issues are due to missing steps. Please follow instructions carefully.
  3. **Review the [troubleshooting guide](https://aka.ms/ftk/trouble).**
The most common issues and their solutions are documented and should be able to be resolved indepdentently.
  4. **Identify the source of the issue.**
For error messages, what product is showing the error? Does the error refer to another product? For missing or incorrect data, is the data generated in Power BI report or does it come directly from a product, like Cost Management?
  -5. **Create support requests for product issues.**
If the source of the issue is a managed product (including data from Cost Management), create a Microsoft support request for that specific product. If you're not sure, ask in the [Q&A discussion forum](https://github.com/microsoft/finops-toolkit/discussions/categories/q-a).
  +5. **Create support requests for product issues.**
If the source of the issue is a managed product (including data from Cost Management), create a Microsoft support request for that specific product. If you're not sure, ask in the [Q&A discussion forum](../discussions/categories/q-a).
  6. **Create an issue in GitHub.**
Whether you submit a support request or not, we recommend [creating an issue](https://aka.ms/ftk/idea) to let us know about the problems you're facing. Even if the issue is a product bug, we would like to document it to help others. We try to respond to issues and discussions within 2 business days but there can sometimes be unanticipated delays. If you've completed all steps above and the issue has not been resolved within a week, we should set up a Teams call for you to share your screen so we can troubleshoot the issue together. diff --git a/docs-wiki/_Footer.md b/docs-wiki/_Footer.md index 7c6ccb24e..71238efae 100644 --- a/docs-wiki/_Footer.md +++ b/docs-wiki/_Footer.md @@ -1,5 +1,5 @@ -Have a question or suggestion? [Start a discussion](https://github.com/microsoft/finops-toolkit/discussions/new?category=general) and let us know you think. +Have a question or suggestion? [Start a discussion](../discussions/new?category=general) and let us know you think. -Find a doc bug? [Update docs-wiki](https://github.com/microsoft/finops-toolkit/tree/dev/docs-wiki) and submit a PR. +Find a doc bug? [Update docs-wiki](../tree/dev/docs-wiki) and submit a PR. diff --git a/docs-wiki/architecture.md b/docs-wiki/architecture.md index 8e115907b..19240c0a1 100644 --- a/docs-wiki/architecture.md +++ b/docs-wiki/architecture.md @@ -14,20 +14,21 @@ On this page: ## 📂 Folder structure -| Name | Description | -| -------------------------------------------------------------------- | -------------------------------- | -| [docs](../docs) | Public-facing toolkit docs. | -| [docs-wiki](../docs-wiki) | Repo wiki for internal dev docs. | -| [src](../src) | Source code and dev docs. | -| ├─ [bicep-registry](../src/bicep-registry) | Bicep registry modules. | -| ├─ [open-data](../src/open-data) | Open data. | -| ├─ [power-bi](../src/power-bi) | Power BI reports. | -| ├─ [powershell](../src/powershell) | PowerShell module functions. | -| ├─ [templates](../src/templates) | ARM deployment templates. | -| │    └─ [finops-hub](../src/templates/finops-hub) | FinOps hub template. | -| └─ [workbooks](../src/workbooks) | Azure Monitor workbooks. | -|      ├─ [governance](../src/templates/governance) | Governance workbook. | -|      └─ [optimization](../src/templates/optimization) | Optimization workbook. | +| Name | Description | +| ----------------------------------------------------------------------------- | -------------------------------- | +| [docs](../tree/dev/docs) | Public-facing toolkit docs. | +| [docs-mslearn](../tree/dev/docs-mslearn) | Public-facing docs on mslearn. | +| [docs-wiki](../tree/dev/docs-wiki) | Repo wiki for internal dev docs. | +| [src](../tree/dev/src) | Source code and dev docs. | +| ├─ [bicep-registry](../tree/dev/src/bicep-registry) | Bicep registry modules. | +| ├─ [open-data](../tree/dev/src/open-data) | Open data. | +| ├─ [power-bi](../tree/dev/src/power-bi) | Power BI reports. | +| ├─ [powershell](../tree/dev/src/powershell) | PowerShell module functions. | +| ├─ [templates](../tree/dev/src/templates) | ARM deployment templates. | +| │    └─ [finops-hub](../tree/dev/src/templates/finops-hub) | FinOps hub template. | +| └─ [workbooks](../tree/dev/src/workbooks) | Azure Monitor workbooks. | +|      ├─ [governance](../tree/dev/src/workbooks/governance) | Governance workbook. | +|      └─ [optimization](../tree/dev/src/workbooks/optimization) | Optimization workbook. | Files and folders should use kebab casing (for example, `this-is-my-folder`). The only exception is for RP namespaces in module paths. From a1e2833591d8683659cd5df921c9e7c20786b7e4 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Tue, 6 Jan 2026 13:29:48 +0100 Subject: [PATCH 36/69] Fix config_InitializeHub failure caused by HTTP redirects in Azure Data Explorer externaldata commands (#1908) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: RolandKrummenacher <1803486+RolandKrummenacher@users.noreply.github.com> Co-authored-by: Roland Krummenacher --- docs-mslearn/toolkit/changelog.md | 2 ++ .../modules/Microsoft.FinOpsHubs/Analytics/app.bicep | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 9ed9ac625..4fea614bf 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -48,11 +48,13 @@ _Released January 2026_ - Fixed all Bicep compilation errors and warnings with inline suppressions and descriptive comments. - Fixed Build-Toolkit.ps1 bicep generate-params command bug. - Fixed Azure Data Explorer dashboard queries by converting `todecimal(0)` to `toreal(0)` to ensure compatibility with KQL type system ([#1893](https://github.com/microsoft/finops-toolkit/issues/1893)). + - Fixed `config_InitializeHub` pipeline failure in Azure Data Explorer caused by HTTP redirects when loading open data CSV files (PricingUnits, Regions, ResourceTypes, Services) from GitHub releases. Updated to use raw.githubusercontent.com URLs that do not redirect ([#1886](https://github.com/microsoft/finops-toolkit/issues/1886)). - Fixed logic to properly generate the scopes to monitor. - Fixed datatype mismatch in InitializeHub pipeline by changing `x_PricingBlockSize` from `decimal` to `real` to match PricingUnits table schema. - Fixed ADF pipeline dependency logic in config_RunBackfillJob, config_StartExportProcess, and config_ConfigureExports pipelines to properly handle both array and non-array scope configurations by adding 'Failed' condition to 'Save/Set Scopes' activity dependencies. - Fixed backward compatibility in `Costs_transform_v1_2()` to support Cost Management exports that predate FOCUS 1.2 by adding fallback mappings for `PricingCurrency` and `SkuMeter` columns to their legacy `x_` counterparts. + ### [Optimization engine](optimization-engine/overview.md) v13 - **Fixed** diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep index b689fc184..02f3b694e 100644 --- a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep @@ -125,8 +125,8 @@ var INGESTION_DB = 'Ingestion' var INGESTION_ID_SEPARATOR = '__' var ftkReleaseUri = endsWith(finOpsToolkitVersion, '-dev') - ? 'https://github.com/microsoft/finops-toolkit/releases/latest/download' - : 'https://github.com/microsoft/finops-toolkit/releases/download/v${finOpsToolkitVersion}' + ? 'https://raw.githubusercontent.com/microsoft/finops-toolkit/refs/heads/dev/src/open-data' + : 'https://raw.githubusercontent.com/microsoft/finops-toolkit/refs/tags/v${finOpsToolkitVersion}/src/open-data' var useFabric = !empty(fabricQueryUri) var useAzure = !useFabric && !empty(clusterName) From b29059748e267be110ce8943b28bacd5948c5d82 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Tue, 6 Jan 2026 13:48:44 +0100 Subject: [PATCH 37/69] Fix Costs_v1_2 duplicate records from Services table join (#1907) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: RolandKrummenacher <1803486+RolandKrummenacher@users.noreply.github.com> Co-authored-by: Roland Krummenacher --- docs-mslearn/toolkit/changelog.md | 2 +- .../Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_v1_2.kql | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 4fea614bf..d2271388e 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -48,13 +48,13 @@ _Released January 2026_ - Fixed all Bicep compilation errors and warnings with inline suppressions and descriptive comments. - Fixed Build-Toolkit.ps1 bicep generate-params command bug. - Fixed Azure Data Explorer dashboard queries by converting `todecimal(0)` to `toreal(0)` to ensure compatibility with KQL type system ([#1893](https://github.com/microsoft/finops-toolkit/issues/1893)). + - Fixed Costs_v1_2 function producing duplicate records when Services table contains multiple entries for the same resource type, causing cost discrepancies between dashboards. - Fixed `config_InitializeHub` pipeline failure in Azure Data Explorer caused by HTTP redirects when loading open data CSV files (PricingUnits, Regions, ResourceTypes, Services) from GitHub releases. Updated to use raw.githubusercontent.com URLs that do not redirect ([#1886](https://github.com/microsoft/finops-toolkit/issues/1886)). - Fixed logic to properly generate the scopes to monitor. - Fixed datatype mismatch in InitializeHub pipeline by changing `x_PricingBlockSize` from `decimal` to `real` to match PricingUnits table schema. - Fixed ADF pipeline dependency logic in config_RunBackfillJob, config_StartExportProcess, and config_ConfigureExports pipelines to properly handle both array and non-array scope configurations by adding 'Failed' condition to 'Save/Set Scopes' activity dependencies. - Fixed backward compatibility in `Costs_transform_v1_2()` to support Cost Management exports that predate FOCUS 1.2 by adding fallback mappings for `PricingCurrency` and `SkuMeter` columns to their legacy `x_` counterparts. - ### [Optimization engine](optimization-engine/overview.md) v13 - **Fixed** diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_v1_2.kql b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_v1_2.kql index f3dd467cf..65775bb04 100644 --- a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_v1_2.kql +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_v1_2.kql @@ -99,7 +99,7 @@ Costs_v1_2() PricingCurrency = x_PricingCurrency, SkuMeter = x_SkuMeterName // Add new columns - | join kind=leftouter (Services | where isnotempty(x_ResourceType) | project x_ResourceType, ServiceSubcategory, x_ServiceModel) on x_ResourceType + | lookup kind=leftouter (Services | where isnotempty(x_ResourceType) | summarize take_any(ServiceSubcategory), take_any(x_ServiceModel) by x_ResourceType) on x_ResourceType | extend CapacityReservationId = tostring(x_SkuDetails.VMCapacityReservationId) | extend CapacityReservationStatus = case( isempty(CapacityReservationId), '', From 3733c9698c783c1d9d2af180996ff5ec0f46f44f Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Wed, 7 Jan 2026 17:59:54 +0100 Subject: [PATCH 38/69] Fix broken finops-hub-copilot.zip download link in AI configuration docs (#1804) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: flanakin <399533+flanakin@users.noreply.github.com> Co-authored-by: Michael Flanakin --- docs-mslearn/toolkit/changelog.md | 1 + src/scripts/Package-Toolkit.ps1 | 23 ++++++++++++++++++- .../finops-hub-copilot/.build.config | 3 +++ 3 files changed, 26 insertions(+), 1 deletion(-) create mode 100644 src/templates/finops-hub-copilot/.build.config diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index d2271388e..4a0a838b2 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -54,6 +54,7 @@ _Released January 2026_ - Fixed datatype mismatch in InitializeHub pipeline by changing `x_PricingBlockSize` from `decimal` to `real` to match PricingUnits table schema. - Fixed ADF pipeline dependency logic in config_RunBackfillJob, config_StartExportProcess, and config_ConfigureExports pipelines to properly handle both array and non-array scope configurations by adding 'Failed' condition to 'Save/Set Scopes' activity dependencies. - Fixed backward compatibility in `Costs_transform_v1_2()` to support Cost Management exports that predate FOCUS 1.2 by adding fallback mappings for `PricingCurrency` and `SkuMeter` columns to their legacy `x_` counterparts. + - Fixed broken link for GitHub Copilot instructions download in [Configure AI documentation](hubs/configure-ai.md). The packaging process now respects the `unversionedZip` property in `.build.config` to create unversioned ZIP files for finops-hub-copilot, enabling stable download links ([#1803](https://github.com/microsoft/finops-toolkit/issues/1803)). ### [Optimization engine](optimization-engine/overview.md) v13 diff --git a/src/scripts/Package-Toolkit.ps1 b/src/scripts/Package-Toolkit.ps1 index c325d4654..a5d112a72 100644 --- a/src/scripts/Package-Toolkit.ps1 +++ b/src/scripts/Package-Toolkit.ps1 @@ -95,7 +95,28 @@ function Copy-TemplateFiles() $srcPath = $_ $templateName = $srcPath.Name $versionSubFolder = (Join-Path $srcPath $version) - $zip = Join-Path (Get-Item $relDir) "$templateName-v$version.zip" + + # Check if template should use an unversioned ZIP filename + $buildConfigPath = Join-Path $PSScriptRoot ".." "templates" $templateName ".build.config" + $unversionedZip = $false + if (Test-Path $buildConfigPath) + { + try + { + $buildConfig = Get-Content $buildConfigPath -Raw -ErrorAction Stop | ConvertFrom-Json -ErrorAction Stop + $unversionedZip = $buildConfig.unversionedZip -eq $true + } + catch + { + Write-Warning "Failed to read .build.config for $templateName : $_" + } + } + + $zip = if ($unversionedZip) { + Join-Path (Get-Item $relDir) "$templateName.zip" + } else { + Join-Path (Get-Item $relDir) "$templateName-v$version.zip" + } Write-Verbose "Checking for a nested version folder: $versionSubFolder" if ((Test-Path -Path $versionSubFolder -PathType Container) -eq $true) diff --git a/src/templates/finops-hub-copilot/.build.config b/src/templates/finops-hub-copilot/.build.config new file mode 100644 index 000000000..d48d65e3f --- /dev/null +++ b/src/templates/finops-hub-copilot/.build.config @@ -0,0 +1,3 @@ +{ + "unversionedZip": true +} From 804895293506332bbdc30f87958852c78fde0103 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Thu, 8 Jan 2026 08:07:45 +0100 Subject: [PATCH 39/69] docs: Add comprehensive troubleshooting for Data Explorer ingestion pipeline errors (#1850) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com> Co-authored-by: Roland Krummenacher --- docs-mslearn/toolkit/changelog.md | 3 + docs-mslearn/toolkit/help/errors.md | 124 +++++++++++++++++++++++++++- 2 files changed, 125 insertions(+), 2 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 4a0a838b2..4f2900cc6 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -39,6 +39,9 @@ _Released January 2026_ - **Added** - Document [how to remove private networking](hubs/private-networking.md#removing-private-networking) and switch back to public access to reduce costs ([#1342](https://github.com/microsoft/finops-toolkit/issues/1342)). + - Added comprehensive troubleshooting guidance for [ErrorCodeNotString](help/errors.md#errorcodenotstring) error that occurs when Azure Data Factory Fail activities cannot evaluate dynamic expressions. + - Enhanced [DataExplorerPostIngestionDropFailed](help/errors.md#dataexplorerpostingestiondropfailed) error documentation with detailed troubleshooting steps, common scenarios, and links to Microsoft Learn resources. + - Enhanced [DataExplorerPreIngestionDropFailed](help/errors.md#dataexplorerpreingestiondropfailed) error documentation with troubleshooting guidance and cross-references. - **Changed** - Reorganized Bicep modules into separate apps. - Enhanced [Configure scopes documentation](hubs/configure-scopes.md) to explicitly clarify that FinOps hubs support: diff --git a/docs-mslearn/toolkit/help/errors.md b/docs-mslearn/toolkit/help/errors.md index 589a110d1..cd04425f4 100644 --- a/docs-mslearn/toolkit/help/errors.md +++ b/docs-mslearn/toolkit/help/errors.md @@ -248,7 +248,45 @@ Data Explorer ingestion timed out after 2 hours while waiting for available capa Data Explorer post-ingestion cleanup (drop extents from the final table) failed. Data from a previous ingestion may be present in reporting, which could result in duplicated and inaccurate costs. -**Mitigation**: Review the Data Explorer error message and resolve the issue. Rerun data ingestion for the specified folder using the `ingestion_ExecuteETL` pipeline in Azure Data Factory. Report unresolved issues at https://aka.ms/ftk/ideas. +This error can occur when: +- The Data Explorer cluster is experiencing capacity issues or high resource utilization +- The drop extents command encounters an invalid expression or syntax error +- There are permission issues accessing the Data Explorer database +- Network connectivity issues between Data Factory and Data Explorer + +**Mitigation**: + +1. **Review the detailed error message**: Navigate to Azure Data Factory > Monitor > Pipeline runs > Click on the failed run > View the "Post-Ingest Drop Failed Error" activity to see the specific Data Explorer error code and message. + +2. **Common solutions based on error type**: + - **If you see "Failed to interpret Post-Ingest Drop Failed Error fail message or error code"**: This indicates the dynamic expression in the Fail activity couldn't be evaluated. This typically means: + - The `Post-Ingest Cleanup` activity failed but didn't return error details in the expected format + - Check the `Post-Ingest Cleanup` activity output for the actual Data Explorer error + - See [ErrorCodeNotString](#errorcodenotstring) for more details on this specific error pattern + + - **For capacity/resource issues**: + - Wait a few minutes and rerun the pipeline + - Check Data Explorer cluster metrics in Azure Monitor + - Consider scaling up the cluster if consistently hitting capacity limits + + - **For permission issues**: + - Verify the Data Factory managed identity has proper permissions on the Data Explorer database + - Ensure the managed identity has at least "Database Ingestor" and "Database Admin" roles + + - **For syntax/expression errors**: + - Review the Data Explorer command logs using `.show commands` in the Data Explorer query editor + - Check for recent schema changes that might affect the drop extents query + +3. **Rerun ingestion**: Once the issue is resolved, rerun data ingestion for the specified folder using the `ingestion_ExecuteETL` pipeline in Azure Data Factory. + +4. **Prevent data duplication**: If the error persists, you may need to manually clean up duplicate extents using Data Explorer commands before rerunning ingestion. Contact support for assistance. + +For more information, see: +- [Azure Data Factory Fail activity error codes](https://learn.microsoft.com/azure/data-factory/control-flow-fail-activity#understand-the-fail-activity-error-code) +- [Troubleshoot Azure Data Explorer connector](https://learn.microsoft.com/azure/data-factory/connector-troubleshoot-azure-data-explorer) +- [Monitor Azure Data Explorer ingestion](https://learn.microsoft.com/azure/data-explorer/monitor-data-explorer) + +Report unresolved issues at https://aka.ms/ftk/ideas.
@@ -258,7 +296,30 @@ Data Explorer post-ingestion cleanup (drop extents from the final table) failed. Data Explorer pre-ingestion cleanup (drop extents from the raw table) failed. Ingestion was not completed. -**Mitigation**: Review the Data Explorer error message and resolve the issue. Rerun data ingestion for the specified folder using the `ingestion_ExecuteETL` pipeline in Azure Data Factory. Report unresolved issues at https://aka.ms/ftk/ideas. +This error occurs when the Data Explorer cleanup step that runs before ingesting new data fails. This cleanup is necessary to prevent duplicate data in the raw tables. + +**Mitigation**: + +1. **Review the detailed error message**: Navigate to Azure Data Factory > Monitor > Pipeline runs > Click on the failed run > View the "Pre-Ingest Drop Failed Error" activity to see the specific Data Explorer error code and message. + +2. **Common solutions based on error type**: + - **If you see "Failed to interpret Pre-Ingest Drop Failed Error fail message or error code"**: See [ErrorCodeNotString](#errorcodenotstring) for troubleshooting steps. + + - **For capacity/resource issues**: + - Wait a few minutes and rerun the pipeline + - Check Data Explorer cluster metrics in Azure Monitor + + - **For permission issues**: + - Verify the Data Factory managed identity has "Database Admin" role on the Data Explorer database + + - **For syntax/expression errors**: + - Review the Data Explorer command logs using `.show commands` in the Data Explorer query editor + +3. **Rerun ingestion**: Once the issue is resolved, rerun data ingestion for the specified folder using the `ingestion_ExecuteETL` pipeline in Azure Data Factory. + +For more information, see the mitigation steps for [DataExplorerPostIngestionDropFailed](#dataexplorerpostingestiondropfailed). + +Report unresolved issues at https://aka.ms/ftk/ideas.
@@ -306,6 +367,65 @@ Microsoft Customer Agreements are not supported for managed exports.
+## ErrorCodeNotString + +Severity: Critical + +This error occurs when an Azure Data Factory Fail activity cannot evaluate its dynamic error message or error code expression to a valid string. The error message typically appears as "Failed to interpret *[activity_name]* fail message or error code" with error code `ErrorCodeNotString`. + +**Common scenarios**: +- A parent activity (like `Post-Ingest Cleanup`, `Pre-Ingest Cleanup`, or `Ingest Data`) failed but didn't produce error output in the expected format +- The dynamic expression tries to access a property that doesn't exist in the activity output +- The activity output is null, empty, or not in the expected JSON structure + +**Mitigation**: + +1. **Identify the root cause activity**: Look at which activity triggered the Fail activity (e.g., if you see "Post-Ingest Drop Failed Error", check the "Post-Ingest Cleanup" activity). + +2. **Review the parent activity output**: + - Navigate to Azure Data Factory > Monitor > Pipeline runs + - Click on the failed pipeline run + - Find and click on the activity that ran just before the Fail activity + - Review the "Output" tab to see the actual error details + - Look for any error messages or codes that explain why the activity failed + +3. **Check for Data Explorer-specific issues** (for ingestion pipeline errors): + - **Resource capacity**: The Data Explorer cluster might be at capacity. Check cluster metrics in Azure Monitor. + - **Command syntax errors**: Review Data Explorer command history using `.show commands` in the query editor. + - **Permission issues**: Verify the managed identity has proper database permissions. + - **Network connectivity**: Ensure Data Factory can reach the Data Explorer cluster. + +4. **Common Data Explorer troubleshooting commands**: + ```kusto + // Check recent failed operations + .show operations + | where StartedOn > ago(4h) and State == "Failed" + + // Check ingestion failures + .show ingestion failures + | where FailedOn > ago(4h) + + // Check command history + .show commands + | where StartedOn > ago(4h) + ``` + +5. **After resolving the underlying issue**: Rerun the failed pipeline from Azure Data Factory. + +**Related errors**: This error is often seen in conjunction with: +- [DataExplorerPostIngestionDropFailed](#dataexplorerpostingestiondropfailed) +- [DataExplorerPreIngestionDropFailed](#dataexplorerpreingestiondropfailed) +- [DataExplorerIngestionFailed](#dataexploreringestionfailed) + +For more information, see: +- [Azure Data Factory Fail activity documentation](https://learn.microsoft.com/azure/data-factory/control-flow-fail-activity#understand-the-fail-activity-error-code) +- [Troubleshoot Azure Data Factory pipelines](https://learn.microsoft.com/azure/data-factory/data-factory-troubleshoot-guide) +- [Azure Data Explorer troubleshooting guide](https://learn.microsoft.com/azure/data-explorer/troubleshoot-database-table) + +Report unresolved issues at https://aka.ms/ftk/ideas. + +
+ ## HubDataNotFound Severity: Critical From 79e441d9beb6f7f3b61a3c4019e6971d275ea393 Mon Sep 17 00:00:00 2001 From: "allcontributors[bot]" <46447321+allcontributors[bot]@users.noreply.github.com> Date: Thu, 8 Jan 2026 08:08:29 +0100 Subject: [PATCH 40/69] add mpritchard2 as a contributor for doc (#1830) Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com> Co-authored-by: Roland Krummenacher Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- .all-contributorsrc | 10 ++++++++++ README.md | 1 + docs/README.md | 1 + 3 files changed, 12 insertions(+) diff --git a/.all-contributorsrc b/.all-contributorsrc index 76bce1622..65d82962c 100644 --- a/.all-contributorsrc +++ b/.all-contributorsrc @@ -834,6 +834,16 @@ "review" ] }, + { + "login": "mpritchard2", + "name": "Mike Pritchard", + "avatar_url": "https://avatars.githubusercontent.com/u/20865962?v=4", + "profile": "https://github.com/mpritchard2", + "contributions": [ + "doc", + "code" + ] + }, { "login": "sandernaus", "name": "Sander Naus", diff --git a/README.md b/README.md index 5478d4f65..ca656c5b8 100644 --- a/README.md +++ b/README.md @@ -139,6 +139,7 @@ There are many ways to participate. From reporting bugs and requesting features Adeel Aziz
Adeel Aziz
💻 shasulin
shasulin
👀 + Mike Pritchard
Mike Pritchard
📖 💻 Sander Naus
Sander Naus
💻 gorkomikus
gorkomikus
💻 diff --git a/docs/README.md b/docs/README.md index e8eba22b0..e2743c5fe 100644 --- a/docs/README.md +++ b/docs/README.md @@ -209,6 +209,7 @@ Whether you're looking for a little assistance or are interested in contributing Adeel Aziz
Adeel Aziz
💻 shasulin
shasulin
👀 + Mike Pritchard
Mike Pritchard
📖 💻 Sander Naus
Sander Naus
💻 gorkomikus
gorkomikus
💻 From cffbbfa2450e9cc2e34ff7a344ee75b58a9a9937 Mon Sep 17 00:00:00 2001 From: Brett Wilson Date: Wed, 7 Jan 2026 23:14:55 -0800 Subject: [PATCH 41/69] Update marketing pages with v13 what's new (#1867) Co-authored-by: msbrett --- docs/README.md | 4 ++-- docs/hubs.md | 4 ++-- docs/optimization-engine.md | 6 ++---- docs/power-bi.md | 4 ++-- 4 files changed, 8 insertions(+), 10 deletions(-) diff --git a/docs/README.md b/docs/README.md index e2743c5fe..394cacbc8 100644 --- a/docs/README.md +++ b/docs/README.md @@ -17,9 +17,9 @@ Automate and extend the Microsoft Cloud with starter kits, scripts, and advanced ---
-

What's new in July 2025v12

+

What's new in August 2025v13

- July introduces support for FOCUS 1.2, autostart in FinOps hubs, a new page in the Cost summary Power BI report, and various small fixes, improvements, and documentation updates across the board. + August brings modular Bicep reorganization, enhanced multi-cloud documentation, Optimization Engine bug fixes, Power BI export requirements documentation, and comprehensive Data Lake connectivity guides.

See all changes

diff --git a/docs/hubs.md b/docs/hubs.md index f64689ed4..4bba14c74 100644 --- a/docs/hubs.md +++ b/docs/hubs.md @@ -20,9 +20,9 @@ Open, extensible, and scalable cost governance for the enterprise. FinOps hubs are a reliable, trustworthy platform for cost analytics, insights, and optimization – virtual command centers for leaders throughout the organization to report on, monitor, and optimize cost based on their organizational needs.
-

What's new in July 2025v12

+

What's new in August 2025v13

- In July, FinOps hubs introduced a new v1_2 schema version with support for FOCUS 1.2 and performance improvements, added support to start Data Explorer if stopped, made managed exports optional, expanded supported VNet CIDR block sizes, and added support for Alibaba and Tencent cloud columns. + In August, FinOps hubs reorganized Bicep modules into separate apps, enhanced scope configuration documentation to clarify multi-account and cross-cloud support, fixed all Bicep compilation warnings, and added documentation for removing private networking.

See all changes

diff --git a/docs/optimization-engine.md b/docs/optimization-engine.md index 23825d7b5..f7b2db16d 100644 --- a/docs/optimization-engine.md +++ b/docs/optimization-engine.md @@ -19,15 +19,13 @@ Optimize your Azure environment. The Azure Optimization Engine (AOE) is an extensible solution designed to generate optimization recommendations for your Azure environment. See it like a fully customizable Azure Advisor. - ## Azure Optimization Engine features diff --git a/docs/power-bi.md b/docs/power-bi.md index a9dae9a1c..97ca293d5 100644 --- a/docs/power-bi.md +++ b/docs/power-bi.md @@ -20,9 +20,9 @@ Accelerate your analytics efforts with simple, targeted reports. Summarize and b FinOps toolkit Power BI reports provide a great starting point for FinOps reporting. Customize and augment reports with your own data to facilitate organizational requirements.
-

What's new in July 2025v12

+

What's new in August 2025v13

- In July, Power BI reports were updated to add a new Summary page in the Cost summary report, add an invoice ID filter in the Invoicing and chargeback report, and update all KQL reports to use the latest FinOps hubs v1_2 schema. + In August, Power BI reports added export requirements documentation to all report pages, added Azure Resource Graph as an explicit requirement for governance and workload optimization reports, and fixed tag expansion for tag names with special characters.

See all changes

From aeb66397e85f9333284eba62a1405fd5c66d4e84 Mon Sep 17 00:00:00 2001 From: "allcontributors[bot]" <46447321+allcontributors[bot]@users.noreply.github.com> Date: Thu, 8 Jan 2026 08:20:25 +0100 Subject: [PATCH 42/69] add kasimrehman as a contributor for code (#1838) Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com> Co-authored-by: Roland Krummenacher Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- .all-contributorsrc | 9 +++++++++ README.md | 1 + docs/README.md | 1 + 3 files changed, 11 insertions(+) diff --git a/.all-contributorsrc b/.all-contributorsrc index 65d82962c..723f8001f 100644 --- a/.all-contributorsrc +++ b/.all-contributorsrc @@ -834,6 +834,15 @@ "review" ] }, + { + "login": "kasimrehman", + "name": "Kasim Rehman", + "avatar_url": "https://avatars.githubusercontent.com/u/57490380?v=4", + "profile": "https://github.com/kasimrehman", + "contributions": [ + "code" + ] + }, { "login": "mpritchard2", "name": "Mike Pritchard", diff --git a/README.md b/README.md index ca656c5b8..bb3c48554 100644 --- a/README.md +++ b/README.md @@ -139,6 +139,7 @@ There are many ways to participate. From reporting bugs and requesting features Adeel Aziz
Adeel Aziz
💻 shasulin
shasulin
👀 + Kasim Rehman
Kasim Rehman
💻 Mike Pritchard
Mike Pritchard
📖 💻 Sander Naus
Sander Naus
💻 gorkomikus
gorkomikus
💻 diff --git a/docs/README.md b/docs/README.md index 394cacbc8..042c6995b 100644 --- a/docs/README.md +++ b/docs/README.md @@ -209,6 +209,7 @@ Whether you're looking for a little assistance or are interested in contributing Adeel Aziz
Adeel Aziz
💻 shasulin
shasulin
👀 + Kasim Rehman
Kasim Rehman
💻 Mike Pritchard
Mike Pritchard
📖 💻 Sander Naus
Sander Naus
💻 gorkomikus
gorkomikus
💻 From 71b8f309b90006596d3b2005a99174180e16a70c Mon Sep 17 00:00:00 2001 From: Roland Krummenacher Date: Thu, 8 Jan 2026 09:53:25 +0100 Subject: [PATCH 43/69] Fix RemoteHub manifest not copied to remote storage (#1926) Co-authored-by: Roland Krummenacher --- docs-mslearn/toolkit/changelog.md | 2 + .../Exports/app.bicep | 45 ++++++++++++++++--- .../Microsoft.FinOpsHubs/RemoteHub/app.bicep | 40 +++++++++++++++++ .../finops-hub/modules/fx/hub-app.bicep | 7 --- .../fx/scripts/Remove-OldResources.ps1 | 5 +++ 5 files changed, 86 insertions(+), 13 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 4f2900cc6..da87ab99f 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -48,6 +48,7 @@ _Released January 2026_ - Multiple Azure scopes (billing accounts, subscriptions, resource groups) in a single hub instance - Cross-cloud data ingestion through FOCUS format support - **Fixed** + - Fixed duplicate Key Vault deployment in RemoteHub by removing redundant accessPolicies nested resource and adding proper dependencies. - Fixed all Bicep compilation errors and warnings with inline suppressions and descriptive comments. - Fixed Build-Toolkit.ps1 bicep generate-params command bug. - Fixed Azure Data Explorer dashboard queries by converting `todecimal(0)` to `toreal(0)` to ensure compatibility with KQL type system ([#1893](https://github.com/microsoft/finops-toolkit/issues/1893)). @@ -57,6 +58,7 @@ _Released January 2026_ - Fixed datatype mismatch in InitializeHub pipeline by changing `x_PricingBlockSize` from `decimal` to `real` to match PricingUnits table schema. - Fixed ADF pipeline dependency logic in config_RunBackfillJob, config_StartExportProcess, and config_ConfigureExports pipelines to properly handle both array and non-array scope configurations by adding 'Failed' condition to 'Save/Set Scopes' activity dependencies. - Fixed backward compatibility in `Costs_transform_v1_2()` to support Cost Management exports that predate FOCUS 1.2 by adding fallback mappings for `PricingCurrency` and `SkuMeter` columns to their legacy `x_` counterparts. + - Fixed RemoteHub manifest file not being copied to remote storage by splitting manifest dataset into separate source and sink datasets, allowing RemoteHub to override the sink to point to remote storage. - Fixed broken link for GitHub Copilot instructions download in [Configure AI documentation](hubs/configure-ai.md). The packaging process now respects the `unversionedZip` property in `.build.config` to create unversioned ZIP files for finops-hub-copilot, enabling stable download links ([#1803](https://github.com/microsoft/finops-toolkit/issues/1803)). ### [Optimization engine](optimization-engine/overview.md) v13 diff --git a/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep index e020a8031..aeb777492 100644 --- a/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep +++ b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep @@ -118,8 +118,8 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { name: '${INGESTION}_files' } - resource dataset_manifest 'datasets' = { - name: 'manifest' + resource dataset_manifest_source 'datasets' = { + name: 'manifest_source' properties: { parameters: { fileName: { @@ -146,7 +146,40 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { } } linkedServiceName: { - // TODO: Should linked service names/references be part of settings? Should datasets be hub modules? + referenceName: app.storage + type: 'LinkedServiceReference' + } + } + } + + resource dataset_manifest_sink 'datasets' = { + name: 'manifest_sink' + properties: { + parameters: { + fileName: { + type: 'String' + defaultValue: 'manifest.json' + } + folderPath: { + type: 'String' + defaultValue: INGESTION + } + } + type: 'Json' + typeProperties: { + location: { + type: 'AzureBlobFSLocation' + fileName: { + value: '@{dataset().fileName}' + type: 'Expression' + } + folderPath: { + value: '@{dataset().folderPath}' + type: 'Expression' + } + } + } + linkedServiceName: { referenceName: app.storage type: 'LinkedServiceReference' } @@ -289,7 +322,7 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { } } dataset: { - referenceName: dataFactory::dataset_manifest.name + referenceName: dataFactory::dataset_manifest_source.name type: 'DatasetReference' parameters: { fileName: { @@ -987,7 +1020,7 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { } inputs: [ { - referenceName: dataFactory::dataset_manifest.name + referenceName: dataFactory::dataset_manifest_source.name type: 'DatasetReference' parameters: { fileName: 'manifest.json' @@ -1000,7 +1033,7 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { ] outputs: [ { - referenceName: dataFactory::dataset_manifest.name + referenceName: dataFactory::dataset_manifest_sink.name type: 'DatasetReference' parameters: { fileName: 'manifest.json' diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/RemoteHub/app.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/RemoteHub/app.bicep index 798cb420d..75bf8190b 100644 --- a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/RemoteHub/app.bicep +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/RemoteHub/app.bicep @@ -50,6 +50,7 @@ module appRegistration '../../fx/hub-app.bicep' = { // Key Vault secret module keyVault_secret '../../fx/hub-vault.bicep' = { name: 'keyVault_secret' + dependsOn: [appRegistration] // Wait for Key Vault to be created params: { vaultName: app.keyVault secretName: storageKeySecretName @@ -62,11 +63,13 @@ module keyVault_secret '../../fx/hub-vault.bicep' = { // Get key vault instance resource keyVault 'Microsoft.KeyVault/vaults@2023-02-01' existing = { name: app.keyVault + dependsOn: [appRegistration] // Wait for Key Vault to be created } // Get data factory instance resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { name: app.dataFactory + dependsOn: [appRegistration] // Wait for Key Vault to be created // cSpell:ignore linkedservices resource linkedService_remoteHubStorage 'linkedservices' = { @@ -148,6 +151,43 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { } } } + + // Replace the manifest_sink dataset to write manifests to remote hub + resource dataset_manifest_sink 'datasets' = { + name: 'manifest_sink' + properties: { + annotations: [] + parameters: { + fileName: { + type: 'String' + defaultValue: 'manifest.json' + } + folderPath: { + type: 'String' + defaultValue: ingestionContainerName + } + } + type: 'Json' + typeProperties: { + location: { + type: 'AzureBlobFSLocation' + fileName: { + value: '@{dataset().fileName}' + type: 'Expression' + } + folderPath: { + value: '@{dataset().folderPath}' + type: 'Expression' + } + } + } + linkedServiceName: { + parameters: {} + referenceName: linkedService_remoteHubStorage.name + type: 'LinkedServiceReference' + } + } + } } diff --git a/src/templates/finops-hub/modules/fx/hub-app.bicep b/src/templates/finops-hub/modules/fx/hub-app.bicep index 4eefc1a5e..3229ab70a 100644 --- a/src/templates/finops-hub/modules/fx/hub-app.bicep +++ b/src/templates/finops-hub/modules/fx/hub-app.bicep @@ -488,13 +488,6 @@ resource keyVault 'Microsoft.KeyVault/vaults@2023-02-01' = if (usesKeyVault) { defaultAction: app.hub.options.privateRouting ? 'Deny' : 'Allow' } } - - resource keyVault_accessPolicies 'accessPolicies' = { - name: 'add' - properties: { - accessPolicies: keyVaultAccessPolicies - } - } } resource keyVaultPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (usesKeyVault && app.hub.options.privateRouting) { diff --git a/src/templates/finops-hub/modules/fx/scripts/Remove-OldResources.ps1 b/src/templates/finops-hub/modules/fx/scripts/Remove-OldResources.ps1 index cf52e53e1..826c8e633 100644 --- a/src/templates/finops-hub/modules/fx/scripts/Remove-OldResources.ps1 +++ b/src/templates/finops-hub/modules/fx/scripts/Remove-OldResources.ps1 @@ -20,3 +20,8 @@ $DeploymentScriptOutputs["deleteTriggers"] = $triggers | Remove-AzDataFactoryV2T $DeploymentScriptOutputs["pipelines"] = Get-AzDataFactoryV2Pipeline @adfParams -ErrorAction SilentlyContinue ` | Where-Object { $_.Name -match '^(msexports_(backfill|extract|fill|get|run|setup|transform)|config_(BackfillData|ExportData|RunBackfill|RunExports))$' } ` | Remove-AzDataFactoryV2Pipeline -Force -ErrorAction SilentlyContinue + +# Delete old datasets +$DeploymentScriptOutputs["datasets"] = Get-AzDataFactoryV2Dataset @adfParams -ErrorAction SilentlyContinue ` +| Where-Object { $_.Name -eq 'manifest' } ` +| Remove-AzDataFactoryV2Dataset -Force -ErrorAction SilentlyContinue From 6d4e7de7fba14068637ba15befeebb23ccc0775c Mon Sep 17 00:00:00 2001 From: Roland Krummenacher Date: Thu, 8 Jan 2026 10:14:50 +0100 Subject: [PATCH 44/69] [RemoteHub]: Remove redundant accessPolicies nested resource in hub-app.bicep, adding dependencies (#1924) Co-authored-by: Roland Krummenacher From 70946718e004107263bba428a3764ee93f93de07 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Thu, 8 Jan 2026 10:21:48 +0100 Subject: [PATCH 45/69] Update v13 release date to January 2026 and enhance What's new sections (#1919) Co-authored-by: msbrett Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: flanakin <399533+flanakin@users.noreply.github.com> Co-authored-by: Roland Krummenacher Co-authored-by: RolandKrummenacher <1803486+RolandKrummenacher@users.noreply.github.com> --- docs/README.md | 4 ++-- docs/hubs.md | 4 ++-- docs/optimization-engine.md | 4 ++-- docs/power-bi.md | 4 ++-- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/README.md b/docs/README.md index 042c6995b..e6750c0ff 100644 --- a/docs/README.md +++ b/docs/README.md @@ -17,9 +17,9 @@ Automate and extend the Microsoft Cloud with starter kits, scripts, and advanced ---
-

What's new in August 2025v13

+

What's new in January 2026v13

- August brings modular Bicep reorganization, enhanced multi-cloud documentation, Optimization Engine bug fixes, Power BI export requirements documentation, and comprehensive Data Lake connectivity guides. + January brings modular Bicep reorganization, enhanced multi-cloud documentation, comprehensive troubleshooting for Data Explorer ingestion errors, Optimization Engine bug fixes, Power BI export requirements documentation, and Data Lake connectivity guides covering tools like Azure Data Explorer, Microsoft Fabric, and Azure Synapse Analytics.

See all changes

diff --git a/docs/hubs.md b/docs/hubs.md index 4bba14c74..08e0f6d56 100644 --- a/docs/hubs.md +++ b/docs/hubs.md @@ -20,9 +20,9 @@ Open, extensible, and scalable cost governance for the enterprise. FinOps hubs are a reliable, trustworthy platform for cost analytics, insights, and optimization – virtual command centers for leaders throughout the organization to report on, monitor, and optimize cost based on their organizational needs.
-

What's new in August 2025v13

+

What's new in January 2026v13

- In August, FinOps hubs reorganized Bicep modules into separate apps, enhanced scope configuration documentation to clarify multi-account and cross-cloud support, fixed all Bicep compilation warnings, and added documentation for removing private networking. + In January, FinOps hubs reorganized Bicep modules into separate apps, enhanced scope configuration documentation to clarify multi-account and cross-cloud support, added comprehensive troubleshooting for Data Explorer ingestion errors, improved KQL function reliability, fixed all Bicep compilation warnings, and added documentation for removing private networking.

See all changes

diff --git a/docs/optimization-engine.md b/docs/optimization-engine.md index f7b2db16d..05cc7a185 100644 --- a/docs/optimization-engine.md +++ b/docs/optimization-engine.md @@ -20,9 +20,9 @@ Optimize your Azure environment. The Azure Optimization Engine (AOE) is an extensible solution designed to generate optimization recommendations for your Azure environment. See it like a fully customizable Azure Advisor.
-

What's new in August 2025v13

+

What's new in January 2026v13

- In August, Azure Optimization Engine fixed reservations workbooks to work around Log Analytics limitations and corrected underutilized disk recommendations for environments with Premium SSD V2 disks. + In January, Azure Optimization Engine fixed reservations workbooks to work around Log Analytics limitations and corrected underutilized disk recommendations for environments with Premium SSD V2 disks.

See all changes

diff --git a/docs/power-bi.md b/docs/power-bi.md index 97ca293d5..c3a6f30d2 100644 --- a/docs/power-bi.md +++ b/docs/power-bi.md @@ -20,9 +20,9 @@ Accelerate your analytics efforts with simple, targeted reports. Summarize and b FinOps toolkit Power BI reports provide a great starting point for FinOps reporting. Customize and augment reports with your own data to facilitate organizational requirements.
-

What's new in August 2025v13

+

What's new in January 2026v13

- In August, Power BI reports added export requirements documentation to all report pages, added Azure Resource Graph as an explicit requirement for governance and workload optimization reports, and fixed tag expansion for tag names with special characters. + In January, Power BI reports added export requirements documentation to all report pages, added Azure Resource Graph as an explicit requirement for governance and workload optimization reports, and fixed tag expansion for tag names with special characters.

See all changes

From 9064ae07e20b4ccf2de9e339948576ffce049169 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Thu, 8 Jan 2026 01:29:50 -0800 Subject: [PATCH 46/69] Fix Power BI "Number of Months" parameter causing incorrect date range calculation (#1852) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: MSBrett <24294904+MSBrett@users.noreply.github.com> Co-authored-by: Michael Flanakin --- docs-mslearn/toolkit/changelog.md | 1 + src/power-bi/kql/Shared.Dataset/definition/expressions.tmdl | 2 +- .../kql/Shared.Dataset/definition/tables/StorageData.tmdl | 2 +- .../storage/Shared.Dataset/definition/expressions.tmdl | 4 ++-- .../storage/Shared.Dataset/definition/tables/StorageData.tmdl | 2 +- 5 files changed, 6 insertions(+), 5 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index da87ab99f..74426c29f 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -74,6 +74,7 @@ _Released January 2026_ - Added Azure Resource Graph as an explicit requirement for governance and workload optimization reports. - **Fixed** - Fixed tag expansion in Power BI reports when tag names contain special characters like colons. + - Fixed "Number of Months" parameter calculation that was excluding the first 5 days of data when set to 3 months ([#1833](https://github.com/microsoft/finops-toolkit/issues/1833)). > [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v13) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v12...v13) diff --git a/src/power-bi/kql/Shared.Dataset/definition/expressions.tmdl b/src/power-bi/kql/Shared.Dataset/definition/expressions.tmdl index 8462dbca5..81cf35cf5 100644 --- a/src/power-bi/kql/Shared.Dataset/definition/expressions.tmdl +++ b/src/power-bi/kql/Shared.Dataset/definition/expressions.tmdl @@ -22,7 +22,7 @@ expression '▶️ START HERE' = ``` DateEnd = null, DateStart = null, DateMonths = #"Number of Months", - DateStartEffective = if DateStart <> null and DateStart <> "" then DateStart else Date.AddMonths(if DateEnd <> null and DateEnd <> "" then DateEnd else Date.AddMonths(Date.StartOfMonth(DateTime.LocalNow()), 1), -DateMonths+1), + DateStartEffective = if DateStart <> null and DateStart <> "" then DateStart else Date.AddMonths(if DateEnd <> null and DateEnd <> "" then DateEnd else Date.AddMonths(Date.StartOfMonth(DateTime.LocalNow()), 1), -DateMonths), _formatDate = (d) => try DateTime.ToText(d, [Format="MMM d, yyyy"]) otherwise d, DateStartFormat = if (DateStart = null or DateStart = "") and (DateMonths = null or DateMonths < 1) then "*" diff --git a/src/power-bi/kql/Shared.Dataset/definition/tables/StorageData.tmdl b/src/power-bi/kql/Shared.Dataset/definition/tables/StorageData.tmdl index 71037ccca..7c617c9a1 100644 --- a/src/power-bi/kql/Shared.Dataset/definition/tables/StorageData.tmdl +++ b/src/power-bi/kql/Shared.Dataset/definition/tables/StorageData.tmdl @@ -266,7 +266,7 @@ table StorageData end = if RangeEnd <> null then Date.From(RangeEnd, Culture.Current) else null, start = if RangeStart <> null then Date.From(RangeStart, Culture.Current) - else if #"Number of Months" <> null and #"Number of Months" > 0 then Date.AddMonths(if end <> null then end else Date.StartOfMonth(Date.From(DateTime.LocalNow())), -(#"Number of Months"+1)) + else if #"Number of Months" <> null and #"Number of Months" > 0 then Date.AddMonths(if end <> null then end else Date.StartOfMonth(Date.From(DateTime.LocalNow())), -#"Number of Months") else null, FilterFilesByDate = if start = null and end = null then InitialDetails diff --git a/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl b/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl index b6685af8f..98873d2dc 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl @@ -58,7 +58,7 @@ expression '▶️ START HERE' = ``` DateEnd = RangeEnd, DateStart = RangeStart, DateMonths = #"Number of Months", - DateStartEffective = if DateStart <> null and DateStart <> "" then DateStart else Date.AddMonths(if DateEnd <> null and DateEnd <> "" then DateEnd else Date.AddMonths(Date.StartOfMonth(DateTime.LocalNow()), 1), -DateMonths+1), + DateStartEffective = if DateStart <> null and DateStart <> "" then DateStart else Date.AddMonths(if DateEnd <> null and DateEnd <> "" then DateEnd else Date.AddMonths(Date.StartOfMonth(DateTime.LocalNow()), 1), -DateMonths), _formatDate = (d) => try DateTime.ToText(d, [Format="MMM d, yyyy"]) otherwise d, DateStartFormat = if (DateStart = null or DateStart = "") and (DateMonths = null or DateMonths < 1) then "*" @@ -127,7 +127,7 @@ expression ftk_Storage = ``` end = if RangeEnd <> null then Date.From(RangeEnd, Culture.Current) else null, start = if RangeStart <> null then Date.From(RangeStart, Culture.Current) - else if #"Number of Months" <> null and #"Number of Months" > 0 then Date.AddMonths(if end <> null then end else Date.StartOfMonth(Date.From(DateTime.LocalNow())), -(#"Number of Months"+1)) + else if #"Number of Months" <> null and #"Number of Months" > 0 then Date.AddMonths(if end <> null then end else Date.StartOfMonth(Date.From(DateTime.LocalNow())), -#"Number of Months") else null, data = if datasetType <> null and datasetType <> "" then Text.Lower(datasetType) else "focuscost", diff --git a/src/power-bi/storage/Shared.Dataset/definition/tables/StorageData.tmdl b/src/power-bi/storage/Shared.Dataset/definition/tables/StorageData.tmdl index 43eb716b3..8c1d26051 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/tables/StorageData.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/tables/StorageData.tmdl @@ -266,7 +266,7 @@ table StorageData end = if RangeEnd <> null then Date.From(RangeEnd, Culture.Current) else null, start = if RangeStart <> null then Date.From(RangeStart, Culture.Current) - else if #"Number of Months" <> null and #"Number of Months" > 0 then Date.AddMonths(if end <> null then end else Date.StartOfMonth(Date.From(DateTime.LocalNow())), -(#"Number of Months"+1)) + else if #"Number of Months" <> null and #"Number of Months" > 0 then Date.AddMonths(if end <> null then end else Date.StartOfMonth(Date.From(DateTime.LocalNow())), -#"Number of Months") else null, FilterFilesByDate = if start = null and end = null then InitialDetails From a31d180a88ee592a96657ad4a98de4ce1c5163b8 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Thu, 8 Jan 2026 01:43:02 -0800 Subject: [PATCH 47/69] Update FOCUS converter documentation for 1.2-preview specification (#1823) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: arthurclares <53261392+arthurclares@users.noreply.github.com> Co-authored-by: flanakin <399533+flanakin@users.noreply.github.com> Co-authored-by: Michael Flanakin Co-authored-by: RolandKrummenacher <1803486+RolandKrummenacher@users.noreply.github.com> --- docs-mslearn/focus/convert.md | 39 +++++++++++++++++++++---------- docs-mslearn/toolkit/changelog.md | 20 +++++++--------- 2 files changed, 36 insertions(+), 23 deletions(-) diff --git a/docs-mslearn/focus/convert.md b/docs-mslearn/focus/convert.md index d4a94abf3..1f20dc277 100644 --- a/docs-mslearn/focus/convert.md +++ b/docs-mslearn/focus/convert.md @@ -14,7 +14,7 @@ ms.reviewer: micflan # Convert Cost Management data to FOCUS -This document provides guidance for converting Cost Management actual and amortized datasets to the FinOps Open Cost and Usage Specification (FOCUS). To learn more about FOCUS, refer to the [FOCUS overview](what-is-focus.md). +This document provides guidance for converting Cost Management actual and amortized datasets to the FinOps Open Cost and Usage Specification (FOCUS). This includes mappings for FOCUS 1.0, 1.1, and 1.2-preview specifications. To learn more about FOCUS, refer to the [FOCUS overview](what-is-focus.md).
@@ -32,12 +32,12 @@ Apply the following logic to all of the rows: | BilledCost | CostInBillingCurrency | If ChargeType == "Usage" and PricingModel == "Reservation" or "SavingsPlan", then `0`; otherwise, use CostInBillingCurrency. | | BillingAccountId | Enterprise Agreement: BillingAccountId

Microsoft Customer Agreement: BillingProfileId | None | | BillingAccountName | Enterprise Agreement: BillingAccountName

Microsoft Customer Agreement: BillingProfileName | None | -| BillingAccountType | Enterprise Agreement: `Billing Account`

Microsoft Customer Agreement: `Billing Profile` | None | +| BillingAccountType | Enterprise Agreement: `Billing Account`

Microsoft Customer Agreement: `Billing Profile` | New in FOCUS 1.2. | | BillingCurrency | Enterprise Agreement: BillingCurrencyCode

Microsoft Customer Agreement: BillingCurrency | None | | BillingPeriodEnd | BillingPeriodEndDate | Add one day for the exclusive end date. | | BillingPeriodStart | BillingPeriodStartDate | None | -| CapacityReservationId | AdditionalInfo.VMCapacityReservationId | None | -| CapacityReservationStatus | AdditionalInfo.VMCapacityReservationId | If AdditionalInfo.VMCapacityReservationId is null or empty, null; if x_ResourceType == `microsoft.compute/capacityreservationgroups/capacityreservations`, `Unused`; otherwise, `Used`. | +| CapacityReservationId | AdditionalInfo.VMCapacityReservationId | New in FOCUS 1.1. | +| CapacityReservationStatus | AdditionalInfo.VMCapacityReservationId | If AdditionalInfo.VMCapacityReservationId is null or empty, null; if x_ResourceType == `microsoft.compute/capacityreservationgroups/capacityreservations`, `Unused`; otherwise, `Used`. New in FOCUS 1.1. | | ChargeCategory | ChargeType | If `Usage`, `Purchase`, `Credit`, or `Tax`, same value; if `UnusedReservation` or `UnusedSavingsPlan`, then `Usage`; if `Refund`, `Purchase`; otherwise, `Adjustment`. | | ChargeClass | ChargeType | If `Refund`, then use `Correction`. | | ChargeDescription | ProductName | None | @@ -49,19 +49,19 @@ Apply the following logic to all of the rows: | CommitmentDiscountName | BenefitName | None | | CommitmentDiscountStatus | ChargeType | If `UnusedReservation` or `UnusedSavingsPlan`, then `Unused`; else if PricingModel == `Reservation` or `SavingsPlan`, then `Used`; otherwise, null. | | CommitmentDiscountType | BenefitId | If BenefitId contains `/microsoft.capacity/` (case-insensitive), `Reservation`; if it contains `/microsoft.billingbenefits/`, `Savings Plan`; otherwise, null. | -| CommitmentDiscountQuantity | Not available | If focus:CommitmentDiscountCategory == `Spend`, focus:EffectiveCost / focus:x_BillingExchangeRate; if focus:CommitmentDiscountCategory == `Usage`, (focus:PricingQuantity / focus:x_PricingBlockSize) * (normalized ratio); otherwise, null. | -| CommitmentDiscountUnit | Not available | If focus:CommitmentDiscountCategory == `Spend`, focus:PricingCurrency; if focus:CommitmentDiscountCategory == `Usage` and the SKU uses instance size flexibility, `Normalized {focus:ConsumedUnit}`; if focus:CommitmentDiscountCategory == `Usage`, focus:ConsumedUnit; otherwise, null. | +| CommitmentDiscountQuantity | Not available | If focus:CommitmentDiscountCategory == `Spend`, focus:EffectiveCost / focus:x_BillingExchangeRate; if focus:CommitmentDiscountCategory == `Usage`, (focus:PricingQuantity / focus:x_PricingBlockSize) * (normalized ratio); otherwise, null. New in FOCUS 1.1. | +| CommitmentDiscountUnit | Not available | If focus:CommitmentDiscountCategory == `Spend`, focus:PricingCurrency; if focus:CommitmentDiscountCategory == `Usage` and the SKU uses instance size flexibility, `Normalized {focus:ConsumedUnit}`; if focus:CommitmentDiscountCategory == `Usage`, focus:ConsumedUnit; otherwise, null. New in FOCUS 1.1. | | ConsumedQuantity | Quantity | If ChargeType == `Usage`, then Quantity; otherwise, null. | | ConsumedUnit | UnitOfMeasure | If ChargeType == `Usage`, then map using [Pricing units data file](../toolkit/open-data.md#pricing-units) ; otherwise, null. | | ContractedCost | UnitPrice * Quantity / focus:x_PricingBlockSize | Note that x_PricingBlockSize requires a mapping. See column notes for details. | | ContractedUnitPrice | UnitPrice | None | | EffectiveCost | CostInBillingCurrency | If ChargeType == "Purchase" or "Refund" and PricingModel == "Reservation" or "SavingsPlan", then `0`; otherwise, use CostInBillingCurrency. | -| InvoiceId | InvoiceId | None | +| InvoiceId | InvoiceId | New in FOCUS 1.2. Renamed from x_InvoiceId. | | InvoiceIssuerName | PartnerName | If PartnerName is empty, use `Microsoft` | | ListCost | Enterprise Agreement: Not available

Microsoft Customer Agreement: PaygCostInBillingCurrency | None | | ListUnitPrice | Enterprise Agreement: PayGPrice

Microsoft Customer Agreement: PayGPrice \* ExchangeRate | None | | PricingCategory | PricingModel | If `OnDemand`, then `Standard`; if `Spot`, then `Dynamic`; if `Reservation` or `Savings Plan`, then `Committed`; otherwise, null. | -| PricingCurrency | Enterprise Agreement: BillingCurrencyCode

Microsoft Customer Agreement: PricingCurrency | None | +| PricingCurrency | Enterprise Agreement: BillingCurrencyCode

Microsoft Customer Agreement: PricingCurrency | New in FOCUS 1.2. Renamed from x_PricingCurrency. | | PricingQuantity | Quantity / focus:x_PricingBlockSize | Note that x_PricingBlockSize requires a mapping. See column notes for details. | | PricingUnit | DistinctUnits (lookup) | Map UnitOfMeasure to DistinctUnits using [Pricing units data file](../toolkit/open-data.md#pricing-units). | | ProviderName | `Microsoft` | None | @@ -73,20 +73,35 @@ Apply the following logic to all of the rows: | ResourceType | SingularDisplayName (lookup) | Map ResourceType to SingularDisplayName using [Resource types data file](../toolkit/open-data.md#resource-types). | | ServiceCategory | ServiceCategory (lookup) | Map ConsumedService and ResourceType to ServiceCategory using [Services data file](../toolkit/open-data.md#services). | | ServiceName | ServiceName (lookup) | Map ConsumedService and ResourceType to ServiceName using [Services data file](../toolkit/open-data.md#services). | -| ServiceSubcategory | ServiceSubcategory (lookup) | Map ConsumedService and ResourceType to ServiceSubcategory using [Services data file](../toolkit/open-data.md#services). | +| ServiceSubcategory | ServiceSubcategory (lookup) | Map ConsumedService and ResourceType to ServiceSubcategory using [Services data file](../toolkit/open-data.md#services). New in FOCUS 1.1. | | SkuId | Enterprise Agreement: Not available

Microsoft Customer Agreement: ProductId | None | -| SkuMeter | MeterName | None | -| SkuPriceDetails | AdditionalInfo | Prefix all property names with `x_`. | +| SkuMeter | MeterName | New in FOCUS 1.1. | +| SkuPriceDetails | AdditionalInfo | Prefix all property names with `x_`. New in FOCUS 1.1. | | SkuPriceId | Not available | None | | SubAccountId | SubscriptionId | None | | SubAccountName | SubscriptionName | None | -| SubAccountType | `Subscription` | None | +| SubAccountType | `Subscription` | New in FOCUS 1.2. | | Tags | Tags | Wrap in `{` and `}` if needed. | _¹ Quantity in Cost Management is the consumed (usage) quantity._ _² While RegionName is a direct mapping of ResourceLocation, Cost Management and FinOps toolkit reports do additional data cleansing to ensure consistency in values based on the [Regions data file](../toolkit/open-data.md#regions)._ +**Note for FOCUS 1.1 users:** The following columns were added in FOCUS 1.1: +- `CapacityReservationId` +- `CapacityReservationStatus` +- `CommitmentDiscountQuantity` +- `CommitmentDiscountUnit` +- `ServiceSubcategory` +- `SkuMeter` (renamed from `x_SkuMeterName`) +- `SkuPriceDetails` + +**Note for FOCUS 1.2 users:** The following columns were added or renamed in FOCUS 1.2: +- `BillingAccountType` (new column) +- `InvoiceId` (promoted from x_InvoiceId) +- `PricingCurrency` (promoted from x_PricingCurrency) +- `SubAccountType` (new column) +
## Feedback about FOCUS columns diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 74426c29f..020b99355 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -35,6 +35,15 @@ The following section lists features and enhancements that are currently in deve _Released January 2026_ +### [Implementing FinOps guide](../implementing-finops-guide.md) v13 + +- **Added** + - Created comprehensive [Data Lake Storage connectivity options](data-lake-storage-connectivity.md) documentation covering tools and services beyond Power BI including Azure Data Explorer, Microsoft Fabric, Azure Synapse Analytics, Azure Databricks, and custom applications. + - Enhanced troubleshooting guidance for [Data Explorer ingestion errors](help/errors.md#dataexploreringestionfailed) with detailed steps for diagnosing and resolving the common SEM0080 "Ingestion Failed" semantic error, including schema mismatch detection, ingestion mapping verification, and diagnostic query examples. +- **Changed** + - Updated FinOps framework documentation to prepare for Azure TCO calculator retirement scheduled for August 31, 2025. Azure Migrate cost estimation functionality remains available. + - Updated FOCUS converter documentation to include newly added fields in FOCUS 1.2-preview specification, including ServiceSubcategory and renamed columns (InvoiceId, PricingCurrency, SkuMeter). + ### [FinOps hubs](hubs/finops-hubs-overview.md) v13 - **Added** @@ -78,17 +87,6 @@ _Released January 2026_ > [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v13) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v12...v13) -### [Implementing FinOps guide](../implementing-finops-guide.md) v13 - -- **Changed** - - Updated FinOps framework documentation to prepare for Azure TCO calculator retirement scheduled for August 31, 2025. Azure Migrate cost estimation functionality remains available. - -### Documentation improvements v13 - -- **Added** - - Created comprehensive [Data Lake Storage connectivity options](data-lake-storage-connectivity.md) documentation covering tools and services beyond Power BI including Azure Data Explorer, Microsoft Fabric, Azure Synapse Analytics, Azure Databricks, and custom applications. - - Enhanced troubleshooting guidance for [Data Explorer ingestion errors](help/errors.md#dataexploreringestionfailed) with detailed steps for diagnosing and resolving the common SEM0080 "Ingestion Failed" semantic error, including schema mismatch detection, ingestion mapping verification, and diagnostic query examples. -
## v12 From 56f5998e6dede21bb84ddafad012a00ee1b2e086 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Thu, 8 Jan 2026 14:30:58 +0100 Subject: [PATCH 48/69] Add optional Key Vault purge protection parameter (#1922) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: RolandKrummenacher <1803486+RolandKrummenacher@users.noreply.github.com> --- docs-mslearn/toolkit/changelog.md | 1 + src/templates/finops-hub/main.bicep | 4 ++++ src/templates/finops-hub/modules/fx/hub-app.bicep | 2 ++ src/templates/finops-hub/modules/fx/hub-types.bicep | 6 ++++++ src/templates/finops-hub/modules/hub.bicep | 4 ++++ 5 files changed, 17 insertions(+) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 020b99355..ce96bfed7 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -47,6 +47,7 @@ _Released January 2026_ ### [FinOps hubs](hubs/finops-hubs-overview.md) v13 - **Added** + - Added optional `enablePurgeProtection` parameter (default: `false`) to enable purge protection on the Key Vault for compliance with enterprise-scale Azure Landing Zone policies. Available in both `main.bicep` and `modules/hub.bicep` ([#1067](https://github.com/microsoft/finops-toolkit/issues/1067)). - Document [how to remove private networking](hubs/private-networking.md#removing-private-networking) and switch back to public access to reduce costs ([#1342](https://github.com/microsoft/finops-toolkit/issues/1342)). - Added comprehensive troubleshooting guidance for [ErrorCodeNotString](help/errors.md#errorcodenotstring) error that occurs when Azure Data Factory Fail activities cannot evaluate dynamic expressions. - Enhanced [DataExplorerPostIngestionDropFailed](help/errors.md#dataexplorerpostingestiondropfailed) error documentation with detailed troubleshooting steps, common scenarios, and links to Microsoft Learn resources. diff --git a/src/templates/finops-hub/main.bicep b/src/templates/finops-hub/main.bicep index 4fc0be19f..033b19e6f 100644 --- a/src/templates/finops-hub/main.bicep +++ b/src/templates/finops-hub/main.bicep @@ -26,6 +26,9 @@ param storageSku string = 'Premium_LRS' @description('Optional. Enable infrastructure encryption on the storage account. Default = false.') param enableInfrastructureEncryption bool = false +@description('Optional. Enable purge protection for the Key Vault. Default: false.') +param enablePurgeProtection bool = false + @description('Optional. Storage account to push data to for ingestion into a remote hub.') param remoteHubStorageUri string = '' @@ -163,6 +166,7 @@ module hub 'modules/hub.bicep' = { // eventGridLocation: eventGridLocation storageSku: storageSku enableInfrastructureEncryption: enableInfrastructureEncryption + enablePurgeProtection: enablePurgeProtection enableManagedExports: enableManagedExports dataExplorerName: dataExplorerName dataExplorerSku: dataExplorerSku diff --git a/src/templates/finops-hub/modules/fx/hub-app.bicep b/src/templates/finops-hub/modules/fx/hub-app.bicep index 3229ab70a..561bbf6f1 100644 --- a/src/templates/finops-hub/modules/fx/hub-app.bicep +++ b/src/templates/finops-hub/modules/fx/hub-app.bicep @@ -479,6 +479,8 @@ resource keyVault 'Microsoft.KeyVault/vaults@2023-02-01' = if (usesKeyVault) { enabledForDiskEncryption: true enableSoftDelete: true softDeleteRetentionInDays: 90 + // Use null instead of false when purge protection is disabled - Azure requires null to indicate the property should not be set + enablePurgeProtection: app.hub.options.keyVaultEnablePurgeProtection ? true : null enableRbacAuthorization: false createMode: 'default' tenantId: subscription().tenantId diff --git a/src/templates/finops-hub/modules/fx/hub-types.bicep b/src/templates/finops-hub/modules/fx/hub-types.bicep index d0072df2c..0c4eb84be 100644 --- a/src/templates/finops-hub/modules/fx/hub-types.bicep +++ b/src/templates/finops-hub/modules/fx/hub-types.bicep @@ -70,6 +70,7 @@ type HubRoutingProperties = { options: { enableTelemetry: 'Indicates whether telemetry should be enabled for deployments.' keyVaultSku: 'KeyVault SKU. Allowed values: "standard", "premium".' + keyVaultEnablePurgeProtection: 'Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.' networkAddressPrefix: 'Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.' privateRouting: 'Indicates whether private network routing is enabled.' publisherIsolation: 'Indicates whether FinOps hub resources should be separated by publisher for advanced security.' @@ -93,6 +94,7 @@ type HubProperties = { options: { enableTelemetry: bool keyVaultSku: string + keyVaultEnablePurgeProtection: bool networkAddressPrefix: string privateRouting: bool publisherIsolation: bool @@ -180,6 +182,7 @@ func newHubInternal( tagsByResource object, storageSku string, keyVaultSku string, + keyVaultEnablePurgeProtection bool, enableInfrastructureEncryption bool, enablePublicAccess bool, networkName string, @@ -199,6 +202,7 @@ func newHubInternal( options: { enableTelemetry: isTelemetryEnabled ?? true keyVaultSku: keyVaultSku + keyVaultEnablePurgeProtection: keyVaultEnablePurgeProtection networkAddressPrefix: networkAddressPrefix privateRouting: !enablePublicAccess publisherIsolation: false // TODO: Expose publisher isolation option @@ -237,6 +241,7 @@ func newHub( tagsByResource object, storageSku string, keyVaultSku string, + keyVaultEnablePurgeProtection bool, enableInfrastructureEncryption bool, enablePublicAccess bool, networkAddressPrefix string, @@ -250,6 +255,7 @@ func newHub( tagsByResource, storageSku, keyVaultSku, + keyVaultEnablePurgeProtection, enableInfrastructureEncryption, enablePublicAccess, '${safeStorageName(name)}-vnet-${location}', // networkName, cSpell:ignore vnet diff --git a/src/templates/finops-hub/modules/hub.bicep b/src/templates/finops-hub/modules/hub.bicep index 5c5793106..cff8112e7 100644 --- a/src/templates/finops-hub/modules/hub.bicep +++ b/src/templates/finops-hub/modules/hub.bicep @@ -34,6 +34,9 @@ param enableInfrastructureEncryption bool = false ]) param keyVaultSku string = 'premium' +@description('Optional. Enable purge protection for the Key Vault. Default: false.') +param enablePurgeProtection bool = false + @description('Optional. Remote storage account for ingestion dataset.') param remoteHubStorageUri string = '' @@ -179,6 +182,7 @@ var hub = newHub( tagsByResource, storageSku, keyVaultSku, + enablePurgeProtection, enableInfrastructureEncryption, enablePublicAccess, virtualNetworkAddressPrefix, From 71d77df8cf18d0a56393cdfefecc8f8f4fa42a0f Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Fri, 9 Jan 2026 09:54:50 +0100 Subject: [PATCH 49/69] Fix unattached disks card counting all disks instead of filtering by state (#1927) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: RolandKrummenacher <1803486+RolandKrummenacher@users.noreply.github.com> Co-authored-by: Roland Krummenacher --- docs-mslearn/toolkit/changelog.md | 1 + src/power-bi/kql/WorkloadOptimization.Report/report.json | 6 +++--- .../storage/WorkloadOptimization.Report/report.json | 6 +++--- 3 files changed, 7 insertions(+), 6 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index ce96bfed7..1f5ea3429 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -84,6 +84,7 @@ _Released January 2026_ - Added Azure Resource Graph as an explicit requirement for governance and workload optimization reports. - **Fixed** - Fixed tag expansion in Power BI reports when tag names contain special characters like colons. + - Fixed unattached disks count in the workload optimization report to show only truly unattached disks instead of all disks. The card visual now filters disks where (managedBy is empty and diskState is not ActiveSAS) or (diskState is Unattached and not ActiveSAS) ([#1896](https://github.com/microsoft/finops-toolkit/issues/1896)). - Fixed "Number of Months" parameter calculation that was excluding the first 5 days of data when set to 3 months ([#1833](https://github.com/microsoft/finops-toolkit/issues/1833)). > [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v13) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v12...v13) diff --git a/src/power-bi/kql/WorkloadOptimization.Report/report.json b/src/power-bi/kql/WorkloadOptimization.Report/report.json index 722e60a6a..71851c270 100644 --- a/src/power-bi/kql/WorkloadOptimization.Report/report.json +++ b/src/power-bi/kql/WorkloadOptimization.Report/report.json @@ -108,7 +108,7 @@ "config": "{\"visibility\":0}", "displayName": "Unattached disks", "displayOption": 1, - "filters": "[{\"name\":\"b16f3941051eb8c4adae\",\"expression\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Entity\":\"Disks\"}},\"Property\":\"properties.diskState\"}},\"filter\":{\"Version\":2,\"From\":[{\"Name\":\"d\",\"Entity\":\"Disks\",\"Type\":0}],\"Where\":[{\"Condition\":{\"In\":{\"Expressions\":[{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"d\"}},\"Property\":\"properties.diskState\"}}],\"Values\":[[{\"Literal\":{\"Value\":\"'Unattached'\"}}]]}}}]},\"type\":\"Categorical\",\"howCreated\":1,\"objects\":{}}]", + "filters": "[]", "height": 720.00, "name": "1b65e65e85a5204b0700", "ordinal": 2, @@ -186,7 +186,7 @@ }, { "config": "{\"name\":\"32f8b73c37026ee61a51\",\"layouts\":[{\"id\":0,\"position\":{\"x\":215.99999999999997,\"y\":8,\"z\":4000,\"width\":184,\"height\":88,\"tabOrder\":0}}],\"singleVisual\":{\"visualType\":\"card\",\"projections\":{\"Values\":[{\"queryRef\":\"Min(Disks.id)\"}]},\"prototypeQuery\":{\"Version\":2,\"From\":[{\"Name\":\"d\",\"Entity\":\"Disks\",\"Type\":0}],\"Select\":[{\"Aggregation\":{\"Expression\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"d\"}},\"Property\":\"id\"}},\"Function\":2},\"Name\":\"Min(Disks.id)\",\"NativeReferenceName\":\"Unattached disks\"}],\"OrderBy\":[{\"Direction\":2,\"Expression\":{\"Aggregation\":{\"Expression\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"d\"}},\"Property\":\"id\"}},\"Function\":2}}}]},\"columnProperties\":{\"Min(Disks.id)\":{\"displayName\":\"Unattached disks\"}},\"drillFilterOtherVisuals\":true,\"hasDefaultSort\":true,\"objects\":{\"categoryLabels\":[{\"properties\":{\"show\":{\"expr\":{\"Literal\":{\"Value\":\"false\"}}}}}],\"labels\":[{\"properties\":{\"labelDisplayUnits\":{\"expr\":{\"Literal\":{\"Value\":\"0D\"}}},\"fontSize\":{\"expr\":{\"Literal\":{\"Value\":\"32D\"}}}}}]},\"vcObjects\":{\"visualTooltip\":[{\"properties\":{\"show\":{\"expr\":{\"Literal\":{\"Value\":\"true\"}}}}}],\"title\":[{\"properties\":{\"show\":{\"expr\":{\"Literal\":{\"Value\":\"true\"}}},\"text\":{\"expr\":{\"Literal\":{\"Value\":\"'Unattached disks'\"}}},\"alignment\":{\"expr\":{\"Literal\":{\"Value\":\"'center'\"}}},\"fontSize\":{\"expr\":{\"Literal\":{\"Value\":\"12D\"}}}}}]}}}", - "filters": "[]", + "filters": "[{\"name\":\"92f6760209b85a7a8921\",\"expression\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Entity\":\"Disks\"}},\"Property\":\"diskState\"}},\"filter\":{\"Version\":2,\"From\":[{\"Name\":\"d\",\"Entity\":\"Disks\",\"Type\":0}],\"Where\":[{\"Condition\":{\"Comparison\":{\"ComparisonKind\":0,\"Left\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"d\"}},\"Property\":\"diskState\"}},\"Right\":{\"Literal\":{\"Value\":\"'Unattached'\"}}}}}]},\"type\":\"Advanced\",\"howCreated\":1}]", "height": 88.00, "width": 184.00, "x": 216.00, @@ -1317,4 +1317,4 @@ } ], "theme": "Microsoft_FinOps_light_theme6448607457324711.json" -} \ No newline at end of file +} diff --git a/src/power-bi/storage/WorkloadOptimization.Report/report.json b/src/power-bi/storage/WorkloadOptimization.Report/report.json index 4fdc97bef..6d5cb4c5f 100644 --- a/src/power-bi/storage/WorkloadOptimization.Report/report.json +++ b/src/power-bi/storage/WorkloadOptimization.Report/report.json @@ -475,7 +475,7 @@ "config": "{\"filterSortOrder\":3}", "displayName": "Unattached disks", "displayOption": 1, - "filters": "[{\"name\":\"a8b6fd6ed46c12bd0666\",\"expression\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Entity\":\"Disks\"}},\"Property\":\"diskState\"}},\"filter\":{\"Version\":2,\"From\":[{\"Name\":\"d\",\"Entity\":\"Disks\",\"Type\":0}],\"Where\":[{\"Condition\":{\"In\":{\"Expressions\":[{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"d\"}},\"Property\":\"diskState\"}}],\"Values\":[[{\"Literal\":{\"Value\":\"'Unattached'\"}}]]}}}]},\"type\":\"Categorical\",\"howCreated\":1,\"objects\":{},\"ordinal\":0}]", + "filters": "[]", "height": 720.00, "name": "8cbb99e1095a3882680b", "ordinal": 2, @@ -722,7 +722,7 @@ }, { "config": "{\"name\":\"c0ec2c095e8226780de0\",\"layouts\":[{\"id\":0,\"position\":{\"x\":215.99999999999997,\"y\":8,\"z\":4000,\"width\":184,\"height\":88,\"tabOrder\":0}}],\"singleVisual\":{\"visualType\":\"card\",\"projections\":{\"Values\":[{\"queryRef\":\"Min(Disks.id)\"}]},\"prototypeQuery\":{\"Version\":2,\"From\":[{\"Name\":\"d\",\"Entity\":\"Disks\",\"Type\":0}],\"Select\":[{\"Aggregation\":{\"Expression\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"d\"}},\"Property\":\"id\"}},\"Function\":2},\"Name\":\"Min(Disks.id)\",\"NativeReferenceName\":\"Unattached disks\"}],\"OrderBy\":[{\"Direction\":2,\"Expression\":{\"Aggregation\":{\"Expression\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"d\"}},\"Property\":\"id\"}},\"Function\":2}}}]},\"columnProperties\":{\"Min(Disks.id)\":{\"displayName\":\"Unattached disks\"}},\"drillFilterOtherVisuals\":true,\"hasDefaultSort\":true,\"objects\":{\"categoryLabels\":[{\"properties\":{\"show\":{\"expr\":{\"Literal\":{\"Value\":\"false\"}}}}}],\"labels\":[{\"properties\":{\"labelDisplayUnits\":{\"expr\":{\"Literal\":{\"Value\":\"0D\"}}},\"fontSize\":{\"expr\":{\"Literal\":{\"Value\":\"32D\"}}}}}]},\"vcObjects\":{\"visualTooltip\":[{\"properties\":{\"show\":{\"expr\":{\"Literal\":{\"Value\":\"true\"}}}}}],\"title\":[{\"properties\":{\"show\":{\"expr\":{\"Literal\":{\"Value\":\"true\"}}},\"text\":{\"expr\":{\"Literal\":{\"Value\":\"'Unattached disks'\"}}},\"alignment\":{\"expr\":{\"Literal\":{\"Value\":\"'center'\"}}},\"fontSize\":{\"expr\":{\"Literal\":{\"Value\":\"12D\"}}}}}]}}}", - "filters": "[]", + "filters": "[{\"name\":\"92f6760209b85a7a8921\",\"expression\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Entity\":\"Disks\"}},\"Property\":\"diskState\"}},\"filter\":{\"Version\":2,\"From\":[{\"Name\":\"d\",\"Entity\":\"Disks\",\"Type\":0}],\"Where\":[{\"Condition\":{\"Comparison\":{\"ComparisonKind\":0,\"Left\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"d\"}},\"Property\":\"diskState\"}},\"Right\":{\"Literal\":{\"Value\":\"'Unattached'\"}}}}}]},\"type\":\"Advanced\",\"howCreated\":1}]", "height": 88.00, "width": 184.00, "x": 216.00, @@ -1333,4 +1333,4 @@ } ], "theme": "Microsoft_FinOps_light_theme4289017974727012.json" -} \ No newline at end of file +} From 0ec84449f83ba86f8bcd25c4c887d883ac5953cd Mon Sep 17 00:00:00 2001 From: gorkomikus <76739221+gorkomikus@users.noreply.github.com> Date: Fri, 9 Jan 2026 10:51:53 +0100 Subject: [PATCH 50/69] Add format and compressionmode support for New-FinOpsCostExport (#1884) Co-authored-by: Michael Flanakin --- docs-mslearn/toolkit/changelog.md | 11 +++++ .../Public/New-FinOpsCostExport.ps1 | 20 +++++++- .../Tests/Unit/New-FinOpsCostExport.Tests.ps1 | 46 +++++++++++++++++++ 3 files changed, 75 insertions(+), 2 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 1f5ea3429..026e891ee 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -44,6 +44,12 @@ _Released January 2026_ - Updated FinOps framework documentation to prepare for Azure TCO calculator retirement scheduled for August 31, 2025. Azure Migrate cost estimation functionality remains available. - Updated FOCUS converter documentation to include newly added fields in FOCUS 1.2-preview specification, including ServiceSubcategory and renamed columns (InvoiceId, PricingCurrency, SkuMeter). +### [Implementing FinOps guide](../implementing-finops-guide.md) v13 + +- **Added** + - Created comprehensive [Data Lake Storage connectivity options](data-lake-storage-connectivity.md) documentation covering tools and services beyond Power BI including Azure Data Explorer, Microsoft Fabric, Azure Synapse Analytics, Azure Databricks, and custom applications. + - Enhanced troubleshooting guidance for [Data Explorer ingestion errors](help/errors.md#dataexploreringestionfailed) with detailed steps for diagnosing and resolving the common SEM0080 "Ingestion Failed" semantic error, including schema mismatch detection, ingestion mapping verification, and diagnostic query examples. + ### [FinOps hubs](hubs/finops-hubs-overview.md) v13 - **Added** @@ -87,6 +93,11 @@ _Released January 2026_ - Fixed unattached disks count in the workload optimization report to show only truly unattached disks instead of all disks. The card visual now filters disks where (managedBy is empty and diskState is not ActiveSAS) or (diskState is Unattached and not ActiveSAS) ([#1896](https://github.com/microsoft/finops-toolkit/issues/1896)). - Fixed "Number of Months" parameter calculation that was excluding the first 5 days of data when set to 3 months ([#1833](https://github.com/microsoft/finops-toolkit/issues/1833)). +### [PowerShell module](powershell/powershell-commands.md) v13 + +- **Added** + - Added `-Format` and `-CompressionMode` parameters to [New-FinOpsCostExport](powershell/cost/New-FinOpsCostExport.md) to support Parquet format and gzip/snappy compression ([#1074](https://github.com/microsoft/finops-toolkit/issues/1074)). + > [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v13) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v12...v13)
diff --git a/src/powershell/Public/New-FinOpsCostExport.ps1 b/src/powershell/Public/New-FinOpsCostExport.ps1 index 32f42e977..527946264 100644 --- a/src/powershell/Public/New-FinOpsCostExport.ps1 +++ b/src/powershell/Public/New-FinOpsCostExport.ps1 @@ -22,6 +22,12 @@ .PARAMETER Dataset Optional. Dataset to export. Allowed values = "ActualCost", "AmortizedCost", "FocusCost", "PriceSheet", "ReservationDetails", "ReservationRecommendations", "ReservationTransactions". Default = "FocusCost". + .PARAMETER Format + Optional. Format of the export files. Allowed values = "Csv", "Parquet". Default = "Csv". + + .PARAMETER CompressionMode + Optional. Compression used for exported files. Allowed values = "None", "GZip", "Snappy". Default = "None". + .PARAMETER DatasetVersion Optional. Schema version of the dataset to export. Default = "1.2-preview" (applies to FocusCost only). @@ -146,6 +152,16 @@ function New-FinOpsCostExport [string] $Dataset = "FocusCost", + [Parameter()] + [ValidateSet("Csv", "Parquet")] + [string] + $Format = "Csv", + + [Parameter()] + [ValidateSet("None", "GZip", "Snappy")] + [string] + $CompressionMode = "None", + [Parameter()] [string] $DatasetVersion, @@ -278,7 +294,7 @@ function New-FinOpsCostExport } } schedule = @{ status = "Inactive" } - format = "Csv" + format = $Format deliveryInfo = @{ destination = @{ resourceId = $StorageAccountId @@ -359,7 +375,7 @@ function New-FinOpsCostExport $props | Add-Member -Name name -Value $Name -MemberType NoteProperty -Force $props.properties = $props.properties | Add-Member -Name exportDescription -Value $Description -MemberType NoteProperty -Force -PassThru $props.properties = $props.properties | Add-Member -Name dataOverwriteBehavior -Value "$(if ($DoNotOverwrite) { "CreateNewReport" } else { "OverwritePreviousReport" })" -MemberType NoteProperty -Force -PassThru - $props.properties = $props.properties | Add-Member -Name compressionMode -Value "None" -MemberType NoteProperty -Force -PassThru + $props.properties = $props.properties | Add-Member -Name compressionMode -Value $CompressionMode -MemberType NoteProperty -Force -PassThru $props.properties.definition.dataSet.configuration = $props.properties.definition.dataSet.configuration | Add-Member -Name dataVersion -Value $DatasetVersion -MemberType NoteProperty -Force -PassThru $props.properties.deliveryInfo.destination.type = "AzureBlob" diff --git a/src/powershell/Tests/Unit/New-FinOpsCostExport.Tests.ps1 b/src/powershell/Tests/Unit/New-FinOpsCostExport.Tests.ps1 index b48940c99..776b337d6 100644 --- a/src/powershell/Tests/Unit/New-FinOpsCostExport.Tests.ps1 +++ b/src/powershell/Tests/Unit/New-FinOpsCostExport.Tests.ps1 @@ -209,6 +209,52 @@ InModuleScope 'FinOpsToolkit' { } } + Describe 'Format and compression' { + It 'Should default to CSV format' { + # Arrange + # Act + New-FinOpsCostExport @newExportParams + + # Assert + Assert-MockCalled -ModuleName FinOpsToolkit -CommandName 'Invoke-Rest' -Times 1 -ParameterFilter { + $Body.properties.format -eq 'Csv' + } + } + + It 'Should set explicit export format' { + # Arrange + # Act + New-FinOpsCostExport @newExportParams -Format 'Parquet' + + # Assert + Assert-MockCalled -ModuleName FinOpsToolkit -CommandName 'Invoke-Rest' -Times 1 -ParameterFilter { + $Body.properties.format -eq 'Parquet' + } + } + + It 'Should default to no compression' { + # Arrange + # Act + New-FinOpsCostExport @newExportParams + + # Assert + Assert-MockCalled -ModuleName FinOpsToolkit -CommandName 'Invoke-Rest' -Times 1 -ParameterFilter { + $Body.properties.compressionMode -eq 'None' + } + } + + It 'Should set explicit compression mode' { + # Arrange + # Act + New-FinOpsCostExport @newExportParams -CompressionMode 'Snappy' + + # Assert + Assert-MockCalled -ModuleName FinOpsToolkit -CommandName 'Invoke-Rest' -Times 1 -ParameterFilter { + $Body.properties.compressionMode -eq 'Snappy' + } + } + } + Describe 'Storage Path Handling' { It 'Should use scope as default storage path without colons' { # Arrange From 41f3985da6435e4da93bfaf9bafe78e5b9ac33a5 Mon Sep 17 00:00:00 2001 From: Roland Krummenacher Date: Tue, 13 Jan 2026 09:31:04 +0100 Subject: [PATCH 51/69] Fix ADF triggers not starting after deployment (#1931) Co-authored-by: Roland Krummenacher --- docs-mslearn/toolkit/changelog.md | 1 + .../finops-hub/modules/fx/scripts/Init-DataFactory.ps1 | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 026e891ee..d9346ae2b 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -76,6 +76,7 @@ _Released January 2026_ - Fixed backward compatibility in `Costs_transform_v1_2()` to support Cost Management exports that predate FOCUS 1.2 by adding fallback mappings for `PricingCurrency` and `SkuMeter` columns to their legacy `x_` counterparts. - Fixed RemoteHub manifest file not being copied to remote storage by splitting manifest dataset into separate source and sink datasets, allowing RemoteHub to override the sink to point to remote storage. - Fixed broken link for GitHub Copilot instructions download in [Configure AI documentation](hubs/configure-ai.md). The packaging process now respects the `unversionedZip` property in `.build.config` to create unversioned ZIP files for finops-hub-copilot, enabling stable download links ([#1803](https://github.com/microsoft/finops-toolkit/issues/1803)). + - Fixed ADF triggers not starting after deployment due to `$startTriggers` variable not being set from the `StartAllTriggers` environment variable in the Init-DataFactory.ps1 script. ### [Optimization engine](optimization-engine/overview.md) v13 diff --git a/src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 b/src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 index 6ec94421c..ac2cdd7b6 100644 --- a/src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 +++ b/src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 @@ -8,6 +8,9 @@ param( # Init outputs $DeploymentScriptOutputs = @{} +# Convert environment variable to boolean +$startTriggers = $env:StartAllTriggers -eq 'true' -or $env:StartAllTriggers -eq 'True' + if (-not $Stop) { Start-Sleep -Seconds 10 @@ -19,6 +22,7 @@ $triggers = Get-AzDataFactoryV2Trigger ` -DataFactoryName $env:DataFactoryName Write-Output "Found $($triggers.Length) trigger(s)" +Write-Output "StartAllTriggers: $startTriggers" if ($startTriggers) { From e6a5d9a039c75ecacdc7ddb6c5af03ab33913586 Mon Sep 17 00:00:00 2001 From: Roland Krummenacher Date: Tue, 13 Jan 2026 09:40:29 +0100 Subject: [PATCH 52/69] Fix RemoteHub manifest dataset handling and improve naming clarity (#1934) Co-authored-by: Roland Krummenacher --- docs-mslearn/toolkit/changelog.md | 2 +- .../Exports/app.bicep | 46 ++++--------------- .../Microsoft.FinOpsHubs/Core/app.bicep | 36 +++++++++++++++ .../Microsoft.FinOpsHubs/RemoteHub/app.bicep | 6 +-- 4 files changed, 48 insertions(+), 42 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index d9346ae2b..21f951e08 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -74,7 +74,7 @@ _Released January 2026_ - Fixed datatype mismatch in InitializeHub pipeline by changing `x_PricingBlockSize` from `decimal` to `real` to match PricingUnits table schema. - Fixed ADF pipeline dependency logic in config_RunBackfillJob, config_StartExportProcess, and config_ConfigureExports pipelines to properly handle both array and non-array scope configurations by adding 'Failed' condition to 'Save/Set Scopes' activity dependencies. - Fixed backward compatibility in `Costs_transform_v1_2()` to support Cost Management exports that predate FOCUS 1.2 by adding fallback mappings for `PricingCurrency` and `SkuMeter` columns to their legacy `x_` counterparts. - - Fixed RemoteHub manifest file not being copied to remote storage by splitting manifest dataset into separate source and sink datasets, allowing RemoteHub to override the sink to point to remote storage. + - Fixed RemoteHub manifest file not being copied to remote storage. The ingestion_manifest dataset is now consistently handled like other ingestion datasets (ingestion, ingestion_files) - created by the Core module and overridden by the RemoteHub module when configured, ensuring manifests are written to the correct storage location. Also renamed manifest_source to msexports_manifest and manifest_sink to ingestion_manifest for clarity. - Fixed broken link for GitHub Copilot instructions download in [Configure AI documentation](hubs/configure-ai.md). The packaging process now respects the `unversionedZip` property in `.build.config` to create unversioned ZIP files for finops-hub-copilot, enabling stable download links ([#1803](https://github.com/microsoft/finops-toolkit/issues/1803)). - Fixed ADF triggers not starting after deployment due to `$startTriggers` variable not being set from the `StartAllTriggers` environment variable in the Init-DataFactory.ps1 script. diff --git a/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep index aeb777492..547bc9486 100644 --- a/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep +++ b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep @@ -118,42 +118,12 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { name: '${INGESTION}_files' } - resource dataset_manifest_source 'datasets' = { - name: 'manifest_source' - properties: { - parameters: { - fileName: { - type: 'String' - defaultValue: 'manifest.json' - } - folderPath: { - type: 'String' - defaultValue: MSEXPORTS - } - } - type: 'Json' - typeProperties: { - location: { - type: 'AzureBlobFSLocation' - fileName: { - value: '@{dataset().fileName}' - type: 'Expression' - } - folderPath: { - value: '@{dataset().folderPath}' - type: 'Expression' - } - } - } - linkedServiceName: { - referenceName: app.storage - type: 'LinkedServiceReference' - } - } + resource dataset_ingestion_manifest 'datasets' existing = { + name: 'ingestion_manifest' } - resource dataset_manifest_sink 'datasets' = { - name: 'manifest_sink' + resource dataset_msexports_manifest 'datasets' = { + name: 'msexports_manifest' properties: { parameters: { fileName: { @@ -162,7 +132,7 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { } folderPath: { type: 'String' - defaultValue: INGESTION + defaultValue: MSEXPORTS } } type: 'Json' @@ -322,7 +292,7 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { } } dataset: { - referenceName: dataFactory::dataset_manifest_source.name + referenceName: dataFactory::dataset_msexports_manifest.name type: 'DatasetReference' parameters: { fileName: { @@ -1020,7 +990,7 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { } inputs: [ { - referenceName: dataFactory::dataset_manifest_source.name + referenceName: dataFactory::dataset_msexports_manifest.name type: 'DatasetReference' parameters: { fileName: 'manifest.json' @@ -1033,7 +1003,7 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { ] outputs: [ { - referenceName: dataFactory::dataset_manifest_sink.name + referenceName: dataFactory::dataset_ingestion_manifest.name type: 'DatasetReference' parameters: { fileName: 'manifest.json' diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/app.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/app.bicep index 5a903c536..4e30b883f 100644 --- a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/app.bicep +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/app.bicep @@ -236,6 +236,42 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { } } } + + resource dataset_ingestion_manifest 'datasets' = { + name: 'ingestion_manifest' + properties: { + annotations: [] + parameters: { + fileName: { + type: 'String' + defaultValue: 'manifest.json' + } + folderPath: { + type: 'String' + defaultValue: INGESTION + } + } + type: 'Json' + typeProperties: { + location: { + type: 'AzureBlobFSLocation' + fileName: { + value: '@{dataset().fileName}' + type: 'Expression' + } + folderPath: { + value: '@{dataset().folderPath}' + type: 'Expression' + } + } + } + linkedServiceName: { + parameters: {} + referenceName: app.storage + type: 'LinkedServiceReference' + } + } + } } //============================================================================== diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/RemoteHub/app.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/RemoteHub/app.bicep index 75bf8190b..7c13eb207 100644 --- a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/RemoteHub/app.bicep +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/RemoteHub/app.bicep @@ -152,9 +152,9 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { } } - // Replace the manifest_sink dataset to write manifests to remote hub - resource dataset_manifest_sink 'datasets' = { - name: 'manifest_sink' + // Replace the ingestion_manifest dataset to write manifests to remote hub + resource dataset_ingestion_manifest 'datasets' = { + name: 'ingestion_manifest' properties: { annotations: [] parameters: { From 3665e363a77a7292182c1848c6169071cc6a2b4c Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Tue, 13 Jan 2026 11:11:02 +0000 Subject: [PATCH 53/69] Fix Power BI scheduled refresh failures for Governance and WorkloadOptimization (#1937) Co-authored-by: Claude Opus 4.5 --- src/power-bi/kql/Shared.Dataset/definition/expressions.tmdl | 2 +- .../definition/tables/AdvisorRecommendations.tmdl | 2 +- .../kql/Shared.Dataset/definition/tables/Subscriptions.tmdl | 4 ++-- .../storage/Shared.Dataset/definition/expressions.tmdl | 2 +- .../definition/tables/AdvisorRecommendations.tmdl | 2 +- .../definition/tables/AdvisorReservationRecommendations.tmdl | 2 +- .../Shared.Dataset/definition/tables/Subscriptions.tmdl | 4 ++-- 7 files changed, 9 insertions(+), 9 deletions(-) diff --git a/src/power-bi/kql/Shared.Dataset/definition/expressions.tmdl b/src/power-bi/kql/Shared.Dataset/definition/expressions.tmdl index 81cf35cf5..7094545d6 100644 --- a/src/power-bi/kql/Shared.Dataset/definition/expressions.tmdl +++ b/src/power-bi/kql/Shared.Dataset/definition/expressions.tmdl @@ -123,7 +123,7 @@ expression PolicyDefinitions = NullHandling = if Table.HasColumns(Source, "Results") and Table.RowCount(Source) = 1 then #table( - { "id", "name", "displayName", "description", "version" }, + { "subscriptionId", "id", "name", "displayName", "description", "version" }, {} ) else Source diff --git a/src/power-bi/kql/Shared.Dataset/definition/tables/AdvisorRecommendations.tmdl b/src/power-bi/kql/Shared.Dataset/definition/tables/AdvisorRecommendations.tmdl index 8b5d530b7..3cd11b074 100644 --- a/src/power-bi/kql/Shared.Dataset/definition/tables/AdvisorRecommendations.tmdl +++ b/src/power-bi/kql/Shared.Dataset/definition/tables/AdvisorRecommendations.tmdl @@ -529,7 +529,7 @@ table AdvisorRecommendations NullHandling = if Table.HasColumns(Source, "Results") and Table.RowCount(Source) = 1 then #table( - { "id", "name", "properties.recommendationTypeId", "properties.shortDescription", "properties.resourceMetadata", "properties.suppressionIds", "properties.impactedField", "properties.impactedValue", "properties.lastUpdated", "properties.category", "properties.metadata", "properties.impact", "properties.shortDescription.problem", "properties.shortDescription.solution", "properties.extendedProperties.ObservationPeriodStartDate", "properties.extendedProperties.ObservationPeriodEndDate", "properties.extendedProperties.annualSavingsAmount", "properties.extendedProperties.HasRecommendation", "properties.extendedProperties.savingsCurrency", "properties.extendedProperties.IsInReplication", "properties.extendedProperties.Recommended_DTU", "properties.extendedProperties.Recommended_SKU", "properties.extendedProperties.ResourceGroup", "properties.extendedProperties.savingsAmount", "properties.extendedProperties.DatabaseSize", "properties.extendedProperties.DatabaseName", "properties.extendedProperties.ServerName", "properties.extendedProperties.Region", "properties.extendedProperties.lookbackPeriod", "properties.extendedProperties.subId", "properties.extendedProperties.scope", "properties.extendedProperties.term", "properties.extendedProperties.sku", "properties.extendedProperties.commitment", "properties.extendedProperties.currentSku", "properties.extendedProperties.targetSku", "properties.extendedProperties.recommendationMessage", "properties.resourceMetadata.resourceId", "properties.resourceMetadata.singular", "properties.resourceMetadata.plural", "properties.resourceMetadata.action", "properties.resourceMetadata.source", "SortOrder" }, + { "id", "name", "type", "properties.recommendationTypeId", "properties.shortDescription", "properties.resourceMetadata", "properties.suppressionIds", "properties.impactedField", "properties.impactedValue", "properties.lastUpdated", "properties.category", "properties.metadata", "properties.impact", "properties.shortDescription.problem", "properties.shortDescription.solution", "properties.extendedProperties.ObservationPeriodStartDate", "properties.extendedProperties.ObservationPeriodEndDate", "properties.extendedProperties.annualSavingsAmount", "properties.extendedProperties.HasRecommendation", "properties.extendedProperties.savingsCurrency", "properties.extendedProperties.IsInReplication", "properties.extendedProperties.Recommended_DTU", "properties.extendedProperties.Recommended_SKU", "properties.extendedProperties.ResourceGroup", "properties.extendedProperties.savingsAmount", "properties.extendedProperties.DatabaseSize", "properties.extendedProperties.DatabaseName", "properties.extendedProperties.ServerName", "properties.extendedProperties.Region", "properties.extendedProperties.lookbackPeriod", "properties.extendedProperties.subId", "properties.extendedProperties.scope", "properties.extendedProperties.term", "properties.extendedProperties.sku", "properties.extendedProperties.commitment", "properties.extendedProperties.currentSku", "properties.extendedProperties.targetSku", "properties.extendedProperties.recommendationMessage", "properties.resourceMetadata.resourceId", "properties.resourceMetadata.singular", "properties.resourceMetadata.plural", "properties.resourceMetadata.action", "properties.resourceMetadata.source", "SortOrder" }, {} ) else Source, diff --git a/src/power-bi/kql/Shared.Dataset/definition/tables/Subscriptions.tmdl b/src/power-bi/kql/Shared.Dataset/definition/tables/Subscriptions.tmdl index 332d26c5f..67afd70bc 100644 --- a/src/power-bi/kql/Shared.Dataset/definition/tables/Subscriptions.tmdl +++ b/src/power-bi/kql/Shared.Dataset/definition/tables/Subscriptions.tmdl @@ -161,12 +161,12 @@ table Subscriptions NullHandling = if Table.HasColumns(Source, "Results") and Table.RowCount(Source) = 1 then #table( - { "id", "name", "subscriptionId", "tenantId", "properties.managementGroupAncestorsChain", "properties.state" }, + { "id", "name", "type", "subscriptionId", "tenantId", "properties.managementGroupAncestorsChain", "properties.state" }, {} ) else Source in - Source + NullHandling ``` annotation PBI_NavigationStepName = Navigation diff --git a/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl b/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl index 98873d2dc..69c6f549c 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/expressions.tmdl @@ -530,7 +530,7 @@ expression PolicyDefinitions = NullHandling = if Table.HasColumns(Source, "Results") and Table.RowCount(Source) = 1 then #table( - { "id", "name", "displayName", "description", "version" }, + { "subscriptionId", "id", "name", "displayName", "description", "version" }, {} ) else Source diff --git a/src/power-bi/storage/Shared.Dataset/definition/tables/AdvisorRecommendations.tmdl b/src/power-bi/storage/Shared.Dataset/definition/tables/AdvisorRecommendations.tmdl index 1bd3b611d..922e08ebe 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/tables/AdvisorRecommendations.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/tables/AdvisorRecommendations.tmdl @@ -527,7 +527,7 @@ table AdvisorRecommendations NullHandling = if Table.HasColumns(Source, "Results") and Table.RowCount(Source) = 1 then #table( - { "id", "name", "properties.recommendationTypeId", "properties.shortDescription", "properties.resourceMetadata", "properties.suppressionIds", "properties.impactedField", "properties.impactedValue", "properties.lastUpdated", "properties.category", "properties.metadata", "properties.impact", "properties.shortDescription.problem", "properties.shortDescription.solution", "properties.extendedProperties.ObservationPeriodStartDate", "properties.extendedProperties.ObservationPeriodEndDate", "properties.extendedProperties.annualSavingsAmount", "properties.extendedProperties.HasRecommendation", "properties.extendedProperties.savingsCurrency", "properties.extendedProperties.IsInReplication", "properties.extendedProperties.Recommended_DTU", "properties.extendedProperties.Recommended_SKU", "properties.extendedProperties.ResourceGroup", "properties.extendedProperties.savingsAmount", "properties.extendedProperties.DatabaseSize", "properties.extendedProperties.DatabaseName", "properties.extendedProperties.ServerName", "properties.extendedProperties.Region", "properties.extendedProperties.lookbackPeriod", "properties.extendedProperties.subId", "properties.extendedProperties.scope", "properties.extendedProperties.term", "properties.extendedProperties.sku", "properties.extendedProperties.commitment", "properties.extendedProperties.currentSku", "properties.extendedProperties.targetSku", "properties.extendedProperties.recommendationMessage", "properties.resourceMetadata.resourceId", "properties.resourceMetadata.singular", "properties.resourceMetadata.plural", "properties.resourceMetadata.action", "properties.resourceMetadata.source", "SortOrder" }, + { "id", "name", "type", "properties.recommendationTypeId", "properties.shortDescription", "properties.resourceMetadata", "properties.suppressionIds", "properties.impactedField", "properties.impactedValue", "properties.lastUpdated", "properties.category", "properties.metadata", "properties.impact", "properties.shortDescription.problem", "properties.shortDescription.solution", "properties.extendedProperties.ObservationPeriodStartDate", "properties.extendedProperties.ObservationPeriodEndDate", "properties.extendedProperties.annualSavingsAmount", "properties.extendedProperties.HasRecommendation", "properties.extendedProperties.savingsCurrency", "properties.extendedProperties.IsInReplication", "properties.extendedProperties.Recommended_DTU", "properties.extendedProperties.Recommended_SKU", "properties.extendedProperties.ResourceGroup", "properties.extendedProperties.savingsAmount", "properties.extendedProperties.DatabaseSize", "properties.extendedProperties.DatabaseName", "properties.extendedProperties.ServerName", "properties.extendedProperties.Region", "properties.extendedProperties.lookbackPeriod", "properties.extendedProperties.subId", "properties.extendedProperties.scope", "properties.extendedProperties.term", "properties.extendedProperties.sku", "properties.extendedProperties.commitment", "properties.extendedProperties.currentSku", "properties.extendedProperties.targetSku", "properties.extendedProperties.recommendationMessage", "properties.resourceMetadata.resourceId", "properties.resourceMetadata.singular", "properties.resourceMetadata.plural", "properties.resourceMetadata.action", "properties.resourceMetadata.source", "SortOrder" }, {} ) else Source, diff --git a/src/power-bi/storage/Shared.Dataset/definition/tables/AdvisorReservationRecommendations.tmdl b/src/power-bi/storage/Shared.Dataset/definition/tables/AdvisorReservationRecommendations.tmdl index fea8fcdf7..2ffb4878e 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/tables/AdvisorReservationRecommendations.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/tables/AdvisorReservationRecommendations.tmdl @@ -525,7 +525,7 @@ table AdvisorReservationRecommendations NullHandling = if Table.HasColumns(Source, "Results") and Table.RowCount(Source) = 1 then #table( - { "id", "name", "properties.recommendationTypeId", "properties.shortDescription", "properties.resourceMetadata", "properties.suppressionIds", "properties.impactedField", "properties.impactedValue", "properties.lastUpdated", "properties.category", "properties.metadata", "properties.impact", "properties.shortDescription.problem", "properties.shortDescription.solution", "properties.extendedProperties.ObservationPeriodStartDate", "properties.extendedProperties.ObservationPeriodEndDate", "properties.extendedProperties.annualSavingsAmount", "properties.extendedProperties.HasRecommendation", "properties.extendedProperties.savingsCurrency", "properties.extendedProperties.IsInReplication", "properties.extendedProperties.Recommended_DTU", "properties.extendedProperties.Recommended_SKU", "properties.extendedProperties.ResourceGroup", "properties.extendedProperties.savingsAmount", "properties.extendedProperties.DatabaseSize", "properties.extendedProperties.DatabaseName", "properties.extendedProperties.ServerName", "properties.extendedProperties.Region", "properties.extendedProperties.lookbackPeriod", "properties.extendedProperties.subId", "properties.extendedProperties.scope", "properties.extendedProperties.term", "properties.extendedProperties.sku", "properties.extendedProperties.commitment", "properties.extendedProperties.currentSku", "properties.extendedProperties.targetSku", "properties.extendedProperties.recommendationMessage", "properties.resourceMetadata.resourceId", "properties.resourceMetadata.singular", "properties.resourceMetadata.plural", "properties.resourceMetadata.action", "properties.resourceMetadata.source", "SortOrder" }, + { "id", "name", "type", "properties.recommendationTypeId", "properties.shortDescription", "properties.resourceMetadata", "properties.suppressionIds", "properties.impactedField", "properties.impactedValue", "properties.lastUpdated", "properties.category", "properties.metadata", "properties.impact", "properties.shortDescription.problem", "properties.shortDescription.solution", "properties.extendedProperties.ObservationPeriodStartDate", "properties.extendedProperties.ObservationPeriodEndDate", "properties.extendedProperties.annualSavingsAmount", "properties.extendedProperties.HasRecommendation", "properties.extendedProperties.savingsCurrency", "properties.extendedProperties.IsInReplication", "properties.extendedProperties.Recommended_DTU", "properties.extendedProperties.Recommended_SKU", "properties.extendedProperties.ResourceGroup", "properties.extendedProperties.savingsAmount", "properties.extendedProperties.DatabaseSize", "properties.extendedProperties.DatabaseName", "properties.extendedProperties.ServerName", "properties.extendedProperties.Region", "properties.extendedProperties.lookbackPeriod", "properties.extendedProperties.subId", "properties.extendedProperties.scope", "properties.extendedProperties.term", "properties.extendedProperties.sku", "properties.extendedProperties.commitment", "properties.extendedProperties.currentSku", "properties.extendedProperties.targetSku", "properties.extendedProperties.recommendationMessage", "properties.resourceMetadata.resourceId", "properties.resourceMetadata.singular", "properties.resourceMetadata.plural", "properties.resourceMetadata.action", "properties.resourceMetadata.source", "SortOrder" }, {} ) else Source, diff --git a/src/power-bi/storage/Shared.Dataset/definition/tables/Subscriptions.tmdl b/src/power-bi/storage/Shared.Dataset/definition/tables/Subscriptions.tmdl index 56e371ad5..28d8ec2da 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/tables/Subscriptions.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/tables/Subscriptions.tmdl @@ -161,12 +161,12 @@ table Subscriptions NullHandling = if Table.HasColumns(Source, "Results") and Table.RowCount(Source) = 1 then #table( - { "id", "name", "subscriptionId", "tenantId", "properties.managementGroupAncestorsChain", "properties.state" }, + { "id", "name", "type", "subscriptionId", "tenantId", "properties.managementGroupAncestorsChain", "properties.state" }, {} ) else Source in - Source + NullHandling ``` annotation PBI_NavigationStepName = Navigation From af60ebfa9b8ba98acd3d8b36083cb67e68145ea6 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Tue, 13 Jan 2026 11:14:09 +0000 Subject: [PATCH 54/69] Fix EA department scope failing on pricesheet export (#1935) Co-authored-by: Claude Opus 4.5 --- .../ManagedExports/app.bicep | 84 ++++++++++++++++++- 1 file changed, 83 insertions(+), 1 deletion(-) diff --git a/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep b/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep index 9d9151ed1..e75840fd0 100644 --- a/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep +++ b/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep @@ -1131,7 +1131,8 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { typeProperties: { variableName: 'exportScopeType' value: { - value: '@if(contains(toLower(item().scope), \'providers/microsoft.billing/billingaccounts\'), if(contains(toLower(item().scope), \':\'), \'mca\', \'ea\'), if(contains(toLower(item().scope), \'subscriptions/\'), \'subscription\', \'undefined\'))' + // Detect scope type: mca (has colon), ea-department (has /departments/), ea (billing account), subscription, or undefined + value: '@if(contains(toLower(item().scope), \'providers/microsoft.billing/billingaccounts\'), if(contains(toLower(item().scope), \':\'), \'mca\', if(contains(toLower(item().scope), \'/departments/\'), \'ea-department\', \'ea\')), if(contains(toLower(item().scope), \'subscriptions/\'), \'subscription\', \'undefined\'))' type: 'Expression' } } @@ -1430,6 +1431,87 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { } ] } + { // EA Department - only cost details are supported at department scope (no pricesheet, reservation details/transactions, or recommendations) + value: 'ea-department' + activities: [ + { // 'EA Department open month focus export' + name: 'EA Department open month focus export' + type: 'WebActivity' + dependsOn: [ + ] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + url: { + value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-daily-costdetails\'))}?api-version=${exportsApiVersion}' + type: 'Expression' + } + method: 'PUT' + body: { + value: getExportBodyV2(MSEXPORTS, 'FocusCost', false, 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '') + type: 'Expression' + } + headers: { + 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.CostsDaily@${finOpsToolkitVersion}' + ClientType: 'FinOpsToolkit.Hubs@${finOpsToolkitVersion}' + } + authentication: { + type: 'MSI' + resource: { + value: '@variables(\'resourceManagementUri\')' + type: 'Expression' + } + } + } + } + { // 'EA Department closed month focus export' + name: 'EA Department closed month focus export' + type: 'WebActivity' + dependsOn: [ + { + activity: 'EA Department open month focus export' + dependencyConditions: [ 'Succeeded' ] + } + ] + policy: { + timeout: '0.00:05:00' + retry: 2 + retryIntervalInSeconds: 30 + secureOutput: false + secureInput: false + } + userProperties: [] + typeProperties: { + url: { + value: '@{variables(\'resourceManagementUri\')}@{item().scope}/providers/Microsoft.CostManagement/exports/@{toLower(concat(variables(\'finOpsHub\'), \'-monthly-costdetails\'))}?api-version=${exportsApiVersion}' + type: 'Expression' + } + method: 'PUT' + body: { + value: getExportBodyV2(MSEXPORTS, 'FocusCost', true, 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '') + type: 'Expression' + } + headers: { + 'x-ms-command-name': 'FinOpsToolkit.Hubs.config_RunExportJobs.CostsMonthly@${finOpsToolkitVersion}' + ClientType: 'FinOpsToolkit.Hubs@${finOpsToolkitVersion}' + } + authentication: { + type: 'MSI' + resource: { + value: '@variables(\'resourceManagementUri\')' + type: 'Expression' + } + } + } + } + ] + } { // subscription value: 'subscription' activities: [ From 803bdad78f30fca60491816376f3132dcacc6151 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Tue, 13 Jan 2026 11:28:14 +0000 Subject: [PATCH 55/69] Fix SQL MI vCores showing incorrect count in AHB workbook (#1936) Co-authored-by: Claude Opus 4.5 --- docs-mslearn/toolkit/changelog.md | 26 ++++++++++--------- src/workbooks/optimization/AHB/AHB.workbook | 20 +++++++------- .../optimization/Compute/AHB.workbook | 20 +++++++------- 3 files changed, 34 insertions(+), 32 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 21f951e08..096536344 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -44,12 +44,6 @@ _Released January 2026_ - Updated FinOps framework documentation to prepare for Azure TCO calculator retirement scheduled for August 31, 2025. Azure Migrate cost estimation functionality remains available. - Updated FOCUS converter documentation to include newly added fields in FOCUS 1.2-preview specification, including ServiceSubcategory and renamed columns (InvoiceId, PricingCurrency, SkuMeter). -### [Implementing FinOps guide](../implementing-finops-guide.md) v13 - -- **Added** - - Created comprehensive [Data Lake Storage connectivity options](data-lake-storage-connectivity.md) documentation covering tools and services beyond Power BI including Azure Data Explorer, Microsoft Fabric, Azure Synapse Analytics, Azure Databricks, and custom applications. - - Enhanced troubleshooting guidance for [Data Explorer ingestion errors](help/errors.md#dataexploreringestionfailed) with detailed steps for diagnosing and resolving the common SEM0080 "Ingestion Failed" semantic error, including schema mismatch detection, ingestion mapping verification, and diagnostic query examples. - ### [FinOps hubs](hubs/finops-hubs-overview.md) v13 - **Added** @@ -78,12 +72,6 @@ _Released January 2026_ - Fixed broken link for GitHub Copilot instructions download in [Configure AI documentation](hubs/configure-ai.md). The packaging process now respects the `unversionedZip` property in `.build.config` to create unversioned ZIP files for finops-hub-copilot, enabling stable download links ([#1803](https://github.com/microsoft/finops-toolkit/issues/1803)). - Fixed ADF triggers not starting after deployment due to `$startTriggers` variable not being set from the `StartAllTriggers` environment variable in the Init-DataFactory.ps1 script. -### [Optimization engine](optimization-engine/overview.md) v13 - -- **Fixed** - - Reservations-related workbooks fixed by replacing Instance Size Flexibility ratios CSV vanity URL with actual one to work around Log Analytics externaldata limitation ([#1810](https://github.com/microsoft/finops-toolkit/issues/1810)). - - Underutilized disks recommendations were not being generated when customer environment has Premium SSD V2 disks ([#1831](https://github.com/microsoft/finops-toolkit/issues/1831)). - ### [Power BI reports](power-bi/reports.md) v13 - **Added** @@ -93,6 +81,20 @@ _Released January 2026_ - Fixed tag expansion in Power BI reports when tag names contain special characters like colons. - Fixed unattached disks count in the workload optimization report to show only truly unattached disks instead of all disks. The card visual now filters disks where (managedBy is empty and diskState is not ActiveSAS) or (diskState is Unattached and not ActiveSAS) ([#1896](https://github.com/microsoft/finops-toolkit/issues/1896)). - Fixed "Number of Months" parameter calculation that was excluding the first 5 days of data when set to 3 months ([#1833](https://github.com/microsoft/finops-toolkit/issues/1833)). + - Fixed EA department scope failing on pricesheet export by skipping pricesheet exports for scopes that don't support them ([#1870](https://github.com/microsoft/finops-toolkit/issues/1870)). + +### [Optimization engine](optimization-engine/overview.md) v13 + +- **Fixed** + - Reservations-related workbooks fixed by replacing Instance Size Flexibility ratios CSV vanity URL with actual one to work around Log Analytics externaldata limitation ([#1810](https://github.com/microsoft/finops-toolkit/issues/1810)). + - Underutilized disks recommendations were not being generated when customer environment has Premium SSD V2 disks ([#1831](https://github.com/microsoft/finops-toolkit/issues/1831)). + +### [FinOps workbooks](workbooks/finops-workbooks-overview.md) v13 + +#### [Optimization workbook](workbooks/optimization.md) v13 + +- **Fixed** + - Fixed SQL Managed Instance vCores displaying incorrect values in the AHB workbook by changing vCores from string to integer type ([#1877](https://github.com/microsoft/finops-toolkit/issues/1877)). ### [PowerShell module](powershell/powershell-commands.md) v13 diff --git a/src/workbooks/optimization/AHB/AHB.workbook b/src/workbooks/optimization/AHB/AHB.workbook index 119de4c53..9c0248b15 100644 --- a/src/workbooks/optimization/AHB/AHB.workbook +++ b/src/workbooks/optimization/AHB/AHB.workbook @@ -2878,7 +2878,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and tostring(properties.['licenseType']) == 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=tostring(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/databases', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n )\r\n) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, CheckSQLDBAHB,SQLLocation, LicenseType, StorageAccountType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and tostring(properties.['licenseType']) == 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/databases', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n )\r\n) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, CheckSQLDBAHB,SQLLocation, LicenseType, StorageAccountType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n", "size": 0, "title": "AHB Disabled", "queryType": 1, @@ -2898,7 +2898,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and tostring(properties.['licenseType']) != 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=sku.name, SKUTier=sku.tier, SQLLocation = location, vCores=tostring(sku.capacity), LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, SQLLocation, LicenseType, StorageAccountType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n\r\n", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and tostring(properties.['licenseType']) != 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=sku.name, SKUTier=sku.tier, SQLLocation = location, vCores=toint(sku.capacity), LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, SQLLocation, LicenseType, StorageAccountType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n\r\n", "size": 0, "title": "AHB Enabled", "queryType": 1, @@ -3525,7 +3525,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and tostring(properties.['licenseType']) == 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=tostring(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n )\r\n) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, CheckSQLDBAHB,SQLLocation, LicenseType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and tostring(properties.['licenseType']) == 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n )\r\n) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, CheckSQLDBAHB,SQLLocation, LicenseType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n", "size": 0, "title": "AHB Disabled", "queryType": 1, @@ -3545,7 +3545,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and tostring(properties.['licenseType']) != 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=tostring(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, SQLLocation, LicenseType, CheckSQLDBAHB, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n\r\n", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and tostring(properties.['licenseType']) != 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, SQLLocation, LicenseType, CheckSQLDBAHB, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n\r\n", "size": 0, "title": "AHB Enabled", "queryType": 1, @@ -3589,7 +3589,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=tostring(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by SubscriptionName, CheckSQLDBAHB", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by SubscriptionName, CheckSQLDBAHB", "size": 0, "title": "Summary of SQL Databases with or without AHB per Subscription", "showRefreshButton": true, @@ -3675,7 +3675,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=tostring(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by CheckSQLDBAHB", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by CheckSQLDBAHB", "size": 0, "title": "Summary of SQL Databases with or without AHB", "showRefreshButton": true, @@ -4168,7 +4168,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' and tostring(properties.['licenseType']) == 'LicenseIncluded'\r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=tostring(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n | project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n | where SQLRG in ({ResourceGroup})\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' and tostring(properties.['licenseType']) == 'LicenseIncluded'\r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n | project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n | where SQLRG in ({ResourceGroup})\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance", "size": 0, "queryType": 1, "resourceType": "microsoft.resourcegraph/resources", @@ -4187,7 +4187,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' and tostring(properties.['licenseType']) != 'LicenseIncluded'\r\n | extend ManagedInstance=id, SQLName=name, SQLRG=resourceGroup, SQLLocation=location, vCores=tostring(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n | project ManagedInstance, SQLName, SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n | where SQLRG in ({ResourceGroup})\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n ", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' and tostring(properties.['licenseType']) != 'LicenseIncluded'\r\n | extend ManagedInstance=id, SQLName=name, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n | project ManagedInstance, SQLName, SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n | where SQLRG in ({ResourceGroup})\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n ", "size": 0, "queryType": 1, "resourceType": "microsoft.resourcegraph/resources", @@ -4237,7 +4237,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' \r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=tostring(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n| summarize count() by SubscriptionName, CheckSQLMIAHB", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' \r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n| summarize count() by SubscriptionName, CheckSQLMIAHB", "size": 0, "title": "Summary of SQL MI with or without AHB per Subscription", "showRefreshButton": true, @@ -4286,7 +4286,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" \r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances'\r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=tostring(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n| summarize count() by SubscriptionName, CheckSQLMIAHB", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" \r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances'\r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n| summarize count() by SubscriptionName, CheckSQLMIAHB", "size": 0, "title": "Summary of SQL Managed Instance with or without AHB", "showRefreshButton": true, diff --git a/src/workbooks/optimization/Compute/AHB.workbook b/src/workbooks/optimization/Compute/AHB.workbook index 8792c25d2..b20d6bc51 100644 --- a/src/workbooks/optimization/Compute/AHB.workbook +++ b/src/workbooks/optimization/Compute/AHB.workbook @@ -1855,7 +1855,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and tostring(properties.['licenseType']) == 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=tostring(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/databases', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n )\r\n) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, CheckSQLDBAHB,SQLLocation, LicenseType, StorageAccountType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and tostring(properties.['licenseType']) == 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/databases', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n )\r\n) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, CheckSQLDBAHB,SQLLocation, LicenseType, StorageAccountType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n", "size": 0, "title": "AHB Disabled", "queryType": 1, @@ -1875,7 +1875,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and tostring(properties.['licenseType']) != 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=sku.name, SKUTier=sku.tier, SQLLocation = location, vCores=tostring(sku.capacity), LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, SQLLocation, LicenseType, StorageAccountType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n\r\n", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and tostring(properties.['licenseType']) != 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=sku.name, SKUTier=sku.tier, SQLLocation = location, vCores=toint(sku.capacity), LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, SQLLocation, LicenseType, StorageAccountType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n\r\n", "size": 0, "title": "AHB Enabled", "queryType": 1, @@ -2502,7 +2502,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and tostring(properties.['licenseType']) == 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=tostring(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n )\r\n) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, CheckSQLDBAHB,SQLLocation, LicenseType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and tostring(properties.['licenseType']) == 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n )\r\n) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, CheckSQLDBAHB,SQLLocation, LicenseType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n", "size": 0, "title": "AHB Disabled", "queryType": 1, @@ -2522,7 +2522,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and tostring(properties.['licenseType']) != 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=tostring(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, SQLLocation, LicenseType, CheckSQLDBAHB, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n\r\n", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and tostring(properties.['licenseType']) != 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, SQLLocation, LicenseType, CheckSQLDBAHB, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n\r\n", "size": 0, "title": "AHB Enabled", "queryType": 1, @@ -2566,7 +2566,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=tostring(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by SubscriptionName, CheckSQLDBAHB", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by SubscriptionName, CheckSQLDBAHB", "size": 0, "title": "Summary of SQL Databases with or without AHB per Subscription", "showRefreshButton": true, @@ -2652,7 +2652,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=tostring(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by CheckSQLDBAHB", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by CheckSQLDBAHB", "size": 0, "title": "Summary of SQL Databases with or without AHB", "showRefreshButton": true, @@ -3145,7 +3145,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' and tostring(properties.['licenseType']) == 'LicenseIncluded'\r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=tostring(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n | project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n | where SQLRG in ({ResourceGroup})\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' and tostring(properties.['licenseType']) == 'LicenseIncluded'\r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n | project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n | where SQLRG in ({ResourceGroup})\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance", "size": 0, "queryType": 1, "resourceType": "microsoft.resourcegraph/resources", @@ -3164,7 +3164,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' and tostring(properties.['licenseType']) != 'LicenseIncluded'\r\n | extend ManagedInstance=id, SQLName=name, SQLRG=resourceGroup, SQLLocation=location, vCores=tostring(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n | project ManagedInstance, SQLName, SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n | where SQLRG in ({ResourceGroup})\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n ", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' and tostring(properties.['licenseType']) != 'LicenseIncluded'\r\n | extend ManagedInstance=id, SQLName=name, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n | project ManagedInstance, SQLName, SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n | where SQLRG in ({ResourceGroup})\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n ", "size": 0, "queryType": 1, "resourceType": "microsoft.resourcegraph/resources", @@ -3214,7 +3214,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' \r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=tostring(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n| summarize count() by SubscriptionName, CheckSQLMIAHB", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' \r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n| summarize count() by SubscriptionName, CheckSQLMIAHB", "size": 0, "title": "Summary of SQL MI with or without AHB per Subscription", "showRefreshButton": true, @@ -3263,7 +3263,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" \r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances'\r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=tostring(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n| summarize count() by SubscriptionName, CheckSQLMIAHB", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" \r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances'\r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n| summarize count() by SubscriptionName, CheckSQLMIAHB", "size": 0, "title": "Summary of SQL Managed Instance with or without AHB", "showRefreshButton": true, From 1f4e7f7f5e7fa41756fbc0a81f2d241b53bea3da Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Tue, 13 Jan 2026 12:19:51 +0000 Subject: [PATCH 56/69] Fix version format mismatch in ftkver.txt (#1940) Co-authored-by: Claude Opus 4.5 --- docs-mslearn/toolkit/changelog.md | 1 + docs/_includes/ftkver.txt | 2 +- src/scripts/Get-Version.ps1 | 3 ++- 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 096536344..277541d2f 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -71,6 +71,7 @@ _Released January 2026_ - Fixed RemoteHub manifest file not being copied to remote storage. The ingestion_manifest dataset is now consistently handled like other ingestion datasets (ingestion, ingestion_files) - created by the Core module and overridden by the RemoteHub module when configured, ensuring manifests are written to the correct storage location. Also renamed manifest_source to msexports_manifest and manifest_sink to ingestion_manifest for clarity. - Fixed broken link for GitHub Copilot instructions download in [Configure AI documentation](hubs/configure-ai.md). The packaging process now respects the `unversionedZip` property in `.build.config` to create unversioned ZIP files for finops-hub-copilot, enabling stable download links ([#1803](https://github.com/microsoft/finops-toolkit/issues/1803)). - Fixed ADF triggers not starting after deployment due to `$startTriggers` variable not being set from the `StartAllTriggers` environment variable in the Init-DataFactory.ps1 script. + - Fixed version format mismatch causing `config_InitializeHub` pipeline to fail when loading open data files from GitHub. The version in ftkver.txt (e.g., `12.0`) now matches the git tag format (e.g., `v12`) ([#1885](https://github.com/microsoft/finops-toolkit/issues/1885)). ### [Power BI reports](power-bi/reports.md) v13 diff --git a/docs/_includes/ftkver.txt b/docs/_includes/ftkver.txt index 8bafbd775..3cacc0b93 100644 --- a/docs/_includes/ftkver.txt +++ b/docs/_includes/ftkver.txt @@ -1 +1 @@ -12.0 \ No newline at end of file +12 \ No newline at end of file diff --git a/src/scripts/Get-Version.ps1 b/src/scripts/Get-Version.ps1 index cdbe5af3b..fd4cac5e3 100644 --- a/src/scripts/Get-Version.ps1 +++ b/src/scripts/Get-Version.ps1 @@ -29,7 +29,8 @@ param( $ver = (Get-Content (Join-Path $PSScriptRoot ../../package.json) | ConvertFrom-Json).version ` -replace '^[^\d]*((\d+\.\d+)(\.\d+)?(-[a-z]+)?(\.\d+)?)[^\d]*$', '$1' ` -replace '^(\d+\.\d+)(\.\d+)?(-[a-z]+)?(\.0)?$', '$1$2$3' ` - -replace '^(\d+\.\d+)(\.0)?(-[a-z]+)?(\.\d+)?$', '$1$3$4' + -replace '^(\d+\.\d+)(\.0)?(-[a-z]+)?(\.\d+)?$', '$1$3$4' ` + -replace '^(\d+)(\.0)$', '$1' if ($AsDotNetVersion -and $ver.Contains('-')) { From bc665403cb75bc421eabbf44291bb3d5db87301b Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Tue, 20 Jan 2026 13:52:05 +0000 Subject: [PATCH 57/69] Fix start trigger script (#1943) Co-authored-by: Claude Opus 4.5 --- .../finops-hub/modules/fx/hub-app.bicep | 21 ++------ .../modules/fx/hub-initialize.bicep | 28 +++-------- .../modules/fx/scripts/Init-DataFactory.ps1 | 49 ++++++++++--------- src/templates/finops-hub/modules/hub.bicep | 5 ++ 4 files changed, 42 insertions(+), 61 deletions(-) diff --git a/src/templates/finops-hub/modules/fx/hub-app.bicep b/src/templates/finops-hub/modules/fx/hub-app.bicep index 561bbf6f1..3478030cf 100644 --- a/src/templates/finops-hub/modules/fx/hub-app.bicep +++ b/src/templates/finops-hub/modules/fx/hub-app.bicep @@ -333,22 +333,11 @@ module stopTriggers 'hub-deploymentScript.bicep' = { app: app identityName: triggerManagerIdentity.name scriptContent: loadTextContent('./scripts/Init-DataFactory.ps1') - arguments: '-Stop' - environmentVariables: [ - { - name: 'DataFactorySubscriptionId' - value: subscription().id - } - { - name: 'DataFactoryResourceGroup' - value: resourceGroup().name - } - { - name: 'DataFactoryName' - #disable-next-line BCP318 // Null safety warning for conditional resource access // Null safety warning for conditional resource access // Null safety warning for conditional resource access - value: dataFactory.name - } - ] + arguments: join([ + '-DataFactoryResourceGroup "${resourceGroup().name}"' + '-DataFactoryName "${dataFactory.name}"' + '-StopTriggers' + ], ' ') } } diff --git a/src/templates/finops-hub/modules/fx/hub-initialize.bicep b/src/templates/finops-hub/modules/fx/hub-initialize.bicep index b688f8a59..1805172a8 100644 --- a/src/templates/finops-hub/modules/fx/hub-initialize.bicep +++ b/src/templates/finops-hub/modules/fx/hub-initialize.bicep @@ -43,28 +43,12 @@ module initialize 'hub-deploymentScript.bicep' = [ app: app identityName: identityName scriptContent: loadTextContent('./scripts/Init-DataFactory.ps1') - environmentVariables: [ - { - name: 'DataFactorySubscriptionId' - value: subscription().id - } - { - name: 'DataFactoryResourceGroup' - value: resourceGroup().name - } - { - name: 'DataFactoryName' - value: adf - } - { - name: 'Pipelines' - value: join(startPipelines, '|') - } - { - name: 'StartAllTriggers' - value: string(startAllTriggers) - } - ] + arguments: join(filter([ + '-DataFactoryResourceGroup "${resourceGroup().name}"' + '-DataFactoryName "${adf}"' + !empty(startPipelines) ? '-Pipelines "${join(startPipelines, '|')}"' : '' + startAllTriggers ? '-StartTriggers' : '' + ], arg => !empty(arg)), ' ') } } ] diff --git a/src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 b/src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 index ac2cdd7b6..c6159a816 100644 --- a/src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 +++ b/src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 @@ -2,38 +2,41 @@ # Licensed under the MIT License. param( - [switch] $Stop + [string] $DataFactoryResourceGroup, + [string] $DataFactoryName, + [string] $Pipelines = "", + [switch] $StartTriggers, + [switch] $StopTriggers ) # Init outputs $DeploymentScriptOutputs = @{} -# Convert environment variable to boolean -$startTriggers = $env:StartAllTriggers -eq 'true' -or $env:StartAllTriggers -eq 'True' +$RunPipelines = -not [string]::IsNullOrWhiteSpace($Pipelines) -if (-not $Stop) +if ($StartTriggers -or $RunPipelines) { Start-Sleep -Seconds 10 } -# Loop thru triggers -$triggers = Get-AzDataFactoryV2Trigger ` - -ResourceGroupName $env:DataFactoryResourceGroup ` - -DataFactoryName $env:DataFactoryName - -Write-Output "Found $($triggers.Length) trigger(s)" -Write-Output "StartAllTriggers: $startTriggers" - -if ($startTriggers) +if ($StartTriggers -or $StopTriggers) { + # Loop thru triggers + $triggers = Get-AzDataFactoryV2Trigger ` + -ResourceGroupName $DataFactoryResourceGroup ` + -DataFactoryName $DataFactoryName + + Write-Output "Found $($triggers.Length) trigger(s)" + Write-Output "StartTriggers: $StartTriggers" + $triggers | ForEach-Object { $trigger = $_.Name - if ($Stop) + if ($StopTriggers) { Write-Output "Stopping trigger $trigger..." $triggerOutput = Stop-AzDataFactoryV2Trigger ` - -ResourceGroupName $env:DataFactoryResourceGroup ` - -DataFactoryName $env:DataFactoryName ` + -ResourceGroupName $DataFactoryResourceGroup ` + -DataFactoryName $DataFactoryName ` -Name $trigger ` -Force ` -ErrorAction SilentlyContinue # Ignore errors, since the trigger may not exist @@ -42,8 +45,8 @@ if ($startTriggers) { Write-Output "Starting trigger $trigger..." $triggerOutput = Start-AzDataFactoryV2Trigger ` - -ResourceGroupName $env:DataFactoryResourceGroup ` - -DataFactoryName $env:DataFactoryName ` + -ResourceGroupName $DataFactoryResourceGroup ` + -DataFactoryName $DataFactoryName ` -Name $trigger ` -Force } @@ -58,20 +61,20 @@ if ($startTriggers) $DeploymentScriptOutputs[$trigger] = $triggerOutput } - if ($Stop) + if ($StopTriggers) { Start-Sleep -Seconds 10 } } -if (-not [string]::IsNullOrWhiteSpace($env:Pipelines)) +if ($RunPipelines) { - $env:Pipelines.Split('|') ` + $Pipelines.Split('|') ` | ForEach-Object { Write-Output "Running the init pipeline..." Invoke-AzDataFactoryV2Pipeline ` - -ResourceGroupName $env:DataFactoryResourceGroup ` - -DataFactoryName $env:DataFactoryName ` + -ResourceGroupName $DataFactoryResourceGroup ` + -DataFactoryName $DataFactoryName ` -PipelineName $_ } } diff --git a/src/templates/finops-hub/modules/hub.bicep b/src/templates/finops-hub/modules/hub.bicep index cff8112e7..440aeca64 100644 --- a/src/templates/finops-hub/modules/hub.bicep +++ b/src/templates/finops-hub/modules/hub.bicep @@ -353,6 +353,11 @@ module deleteOldResources 'fx/hub-deploymentScript.bicep' = { // Start all ADF triggers module startTriggers 'fx/hub-initialize.bicep' = { name: 'Microsoft.FinOpsHubs.StartTriggers' + dependsOn: [ + analytics + deleteOldResources + remoteHub + ] params: { app: core.outputs.app dataFactoryInstances: [ From fa12d824f394befe0a256978798be5d94add0623 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Wed, 21 Jan 2026 03:53:50 +0000 Subject: [PATCH 58/69] Update board and contributors (#1945) Co-authored-by: Claude Opus 4.5 --- .all-contributorsrc | 177 ++++++++++++++++++----------------- README.md | 36 +++---- docs-wiki/About.md | 23 ++--- docs-wiki/Governing-board.md | 25 +++-- docs/README.md | 36 +++---- 5 files changed, 147 insertions(+), 150 deletions(-) diff --git a/.all-contributorsrc b/.all-contributorsrc index 723f8001f..65afc192c 100644 --- a/.all-contributorsrc +++ b/.all-contributorsrc @@ -9,12 +9,15 @@ "commitConvention": "none", "contributors": [ { - "login": "ms:nelson", - "name": "Nelson Pereira", - "avatar_url": "https://media.licdn.com/dms/image/v2/C5603AQGWN7nsODstqA/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1516244319322?e=1754524800&v=beta&t=EHH5lzBH8qFaqZHcVIn0ovzgkugW-QRlmEnEfgyawOU#", - "profile": "https://www.linkedin.com/in/nelsonmpereira/", + "login": "MSBrett", + "name": "Brett Wilson", + "avatar_url": "https://avatars.githubusercontent.com/u/24294904?v=4", + "profile": "https://github.com/MSBrett", "contributions": [ - "leader" + "leader", + "code", + "doc", + "review" ] }, { @@ -32,47 +35,15 @@ ] }, { - "login": "arthurclares", - "name": "Arthur Clares", - "avatar_url": "https://avatars.githubusercontent.com/u/53261392?v=4", - "profile": "https://github.com/arthurclares", + "login": "RolandKrummenacher", + "name": "Roland Krummenacher", + "avatar_url": "https://avatars.githubusercontent.com/u/1803486?v=4", + "profile": "https://github.com/RolandKrummenacher", "contributions": [ "leader", "code", - "doc", "review", - "mentoring", - "promotion" - ] - }, - { - "login": "scuffy", - "name": "Sonia Cuff", - "avatar_url": "https://avatars.githubusercontent.com/u/41356020?v=4", - "profile": "https://github.com/scuffy", - "contributions": [ - "leader", - "promotion" - ] - }, - { - "login": "tshah2808", - "name": "Tanuja Shah", - "avatar_url": "https://avatars.githubusercontent.com/u/67380293?v=4", - "profile": "https://github.com/tshah2808", - "contributions": [ - "leader" - ] - }, - { - "login": "ms:fernandovas", - "name": "Fernando Vasconcellos", - "avatar_url": "https://media.licdn.com/dms/image/v2/C4D03AQGOHJHlq3nhpQ/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1571181473582?e=1754524800&v=beta&t=dzXv18_LMfbGr2BgOKVVHfxD7OxudO0prln_zP0OEvw#", - "profile": "https://www.linkedin.com/in/fernando-vasconcellos-ba398b191/", - "contributions": [ - "leader", - "doc", - "promotion" + "bug" ] }, { @@ -85,14 +56,13 @@ ] }, { - "login": "MSBrett", - "name": "Brett Wilson", - "avatar_url": "https://avatars.githubusercontent.com/u/24294904?v=4", - "profile": "https://github.com/MSBrett", + "login": "scuffy", + "name": "Sonia Cuff", + "avatar_url": "https://avatars.githubusercontent.com/u/41356020?v=4", + "profile": "https://github.com/scuffy", "contributions": [ - "code", - "doc", - "review" + "leader", + "promotion" ] }, { @@ -107,6 +77,20 @@ "bug" ] }, + { + "login": "arthurclares", + "name": "Arthur Clares", + "avatar_url": "https://avatars.githubusercontent.com/u/53261392?v=4", + "profile": "https://github.com/arthurclares", + "contributions": [ + "leader", + "code", + "doc", + "review", + "mentoring", + "promotion" + ] + }, { "login": "nteyan", "name": "Nicolas Teyan", @@ -118,15 +102,6 @@ "review" ] }, - { - "login": "bandersmsft", - "name": "Bill Anderson", - "avatar_url": "https://avatars.githubusercontent.com/u/9596428?v=4", - "profile": "https://github.com/bandersmsft", - "contributions": [ - "doc" - ] - }, { "login": "ro100e", "name": "Robel", @@ -138,6 +113,18 @@ "review" ] }, + { + "login": "DUeffing", + "name": "Daniel Ueffing", + "avatar_url": "https://avatars.githubusercontent.com/u/94981829?v=4", + "profile": "https://github.com/DUeffing", + "contributions": [ + "ideas", + "code", + "doc", + "bug" + ] + }, { "login": "Springstone", "name": "Sacha Narinx", @@ -160,6 +147,15 @@ "review" ] }, + { + "login": "bandersmsft", + "name": "Bill Anderson", + "avatar_url": "https://avatars.githubusercontent.com/u/9596428?v=4", + "profile": "https://github.com/bandersmsft", + "contributions": [ + "doc" + ] + }, { "login": "grantxyzou", "name": "grantxyzou", @@ -178,17 +174,6 @@ "design" ] }, - { - "login": "RolandKrummenacher", - "name": "Roland Krummenacher", - "avatar_url": "https://avatars.githubusercontent.com/u/1803486?v=4", - "profile": "https://github.com/RolandKrummenacher", - "contributions": [ - "code", - "review", - "bug" - ] - }, { "login": "didayal-msft", "name": "Divyadeep Dayal", @@ -208,6 +193,17 @@ "review" ] }, + { + "login": "chris-bowman", + "name": "Chris Bowman", + "avatar_url": "https://avatars.githubusercontent.com/u/20289947?v=4", + "profile": "https://github.com/chris-bowman", + "contributions": [ + "bug", + "review", + "code" + ] + }, { "login": "akiskips", "name": "Orthodoxos Kipouridis", @@ -321,18 +317,6 @@ "bug" ] }, - { - "login": "DUeffing", - "name": "Daniel Ueffing", - "avatar_url": "https://avatars.githubusercontent.com/u/94981829?v=4", - "profile": "https://github.com/DUeffing", - "contributions": [ - "ideas", - "code", - "doc", - "bug" - ] - }, { "login": "sebassem", "name": "Seif Bassem", @@ -391,14 +375,31 @@ ] }, { - "login": "chris-bowman", - "name": "Chris Bowman", - "avatar_url": "https://avatars.githubusercontent.com/u/20289947?v=4", - "profile": "https://github.com/chris-bowman", + "login": "ms:nelson", + "name": "Nelson Pereira", + "avatar_url": "https://github.com/user-attachments/assets/024a2024-c1da-4668-ba84-c61615c609d5", + "profile": "https://www.linkedin.com/in/nelsonmpereira/", "contributions": [ - "bug", - "review", - "code" + "promotion" + ] + }, + { + "login": "tshah2808", + "name": "Tanuja Shah", + "avatar_url": "https://avatars.githubusercontent.com/u/67380293?v=4", + "profile": "https://github.com/tshah2808", + "contributions": [ + "promotion" + ] + }, + { + "login": "ms:fernandovas", + "name": "Fernando Vasconcellos", + "avatar_url": "https://github.com/user-attachments/assets/15d008be-7819-42a2-beac-92daab361b40", + "profile": "https://www.linkedin.com/in/fernando-vasconcellos-ba398b191/", + "contributions": [ + "doc", + "promotion" ] }, { diff --git a/README.md b/README.md index bb3c48554..6cc99c362 100644 --- a/README.md +++ b/README.md @@ -25,57 +25,57 @@ There are many ways to participate. From reporting bugs and requesting features - + - + + - - + - - - + - - - + + + + - + + - + - + - - - - - + + + + + diff --git a/docs-wiki/About.md b/docs-wiki/About.md index a01241d12..78c15fe19 100644 --- a/docs-wiki/About.md +++ b/docs-wiki/About.md @@ -43,15 +43,13 @@ Current board members:
Nelson Pereira
Nelson Pereira
🌟		Brett Wilson
Brett Wilson
🌟 💻 📖 👀		 Michael Flanakin
Michael Flanakin
🌟 💻 📖 👀 🧑‍🏫 📣		Arthur Clares
Arthur Clares
🌟 💻 📖 👀 🧑‍🏫 📣		Roland Krummenacher
Roland Krummenacher
🌟 💻 👀 🐛		Dirk Brinkmann
Dirk Brinkmann
🌟		 Sonia Cuff
Sonia Cuff
🌟 📣		Tanuja Shah
Tanuja Shah
🌟		Fernando Vasconcellos
Fernando Vasconcellos
🌟 📖 📣		Hélder Pinto
Hélder Pinto
💻 📖 👀 🐛
Dirk Brinkmann
Dirk Brinkmann
🌟		Brett Wilson
Brett Wilson
💻 📖 👀		Hélder Pinto
Hélder Pinto
💻 📖 👀 🐛		Arthur Clares
Arthur Clares
🌟 💻 📖 👀 🧑‍🏫 📣		 Nicolas Teyan
Nicolas Teyan
💻 📖 👀		Bill Anderson
Bill Anderson
📖		 Robel
Robel
💻 📖 👀
Daniel Ueffing
Daniel Ueffing
🤔 💻 📖 🐛		 Sacha Narinx
Sacha Narinx
💻 📖 👀		 Anthony Romano
Anthony Romano
💻 📖 👀
Bill Anderson
Bill Anderson
📖		 grantxyzou
grantxyzou
🎨		 lmoscinski
lmoscinski
🎨		Roland Krummenacher
Roland Krummenacher
💻 👀 🐛		 Divyadeep Dayal
Divyadeep Dayal
💻		jamelachahbar
jamelachahbar
💻 👀		Chris Bowman
Chris Bowman
🐛 👀 💻
jamelachahbar
jamelachahbar
💻 👀		 Orthodoxos Kipouridis
Orthodoxos Kipouridis
💻 📖		 Ben Shy
Ben Shy
💻 👀		 Kevin De La Rosa
Kevin De La Rosa
📖		 bwatts64
bwatts64
💻 👀		 Dany Hoter
Dany Hoter
💻 👀		Joseph John
Joseph John
📖 👀
Joseph John
Joseph John
📖 👀		 ripadrao
ripadrao
📖		 Pedro Sousa
Pedro Sousa
📖		 Sourav Bera
Sourav Bera
📖		 J.R. Phillips
J.R. Phillips
💻		 Saad Mahmood
Saad Mahmood
💻		simonarbel
simonarbel
🐛
simonarbel
simonarbel
🐛		Daniel Ueffing
Daniel Ueffing
🤔 💻 📖 🐛		 Seif Bassem
Seif Bassem
💻		 Arjen Huitema
Arjen Huitema
💻		 Yuan Zhang
Yuan Zhang
💻 👀		 ymehdimsft
ymehdimsft
💻
srilatha inavolu
srilatha inavolu
💻 👀		 soumyananda
soumyananda
💻 👀		Chris Bowman
Chris Bowman
🐛 👀 💻
Nelson Pereira
Nelson Pereira
📣		Tanuja Shah
Tanuja Shah
📣		Fernando Vasconcellos
Fernando Vasconcellos
📖 📣		 Trey Morgan
Trey Morgan
💻		 Travis Silvers
Travis Silvers
👀		 Travis Silvers
Travis Silvers
👀
- - - - + + + - - - + + @@ -60,20 +58,19 @@ To learn more, see [[Governing board]]. ### Advisory council -The FinOps toolkit advisory council represents contributors, their contributions, and end consumers for each of the tools and resources included in the FinOps toolkit. The advisory council advocates for diverse perspectives and fosters a collaborative environment, bridging the gap between the governing board and the contributors and consumers. Advisory council members are typically owners of one or more tools and resources that have been contributed to the FinOps toolkit. +The FinOps toolkit advisory council represents contributors, their contributions, and the practitioners for each of the tools and resources included in the FinOps toolkit. The advisory council advocates for diverse perspectives and fosters a collaborative environment, bridging the gap between the governing board and the contributors and practitioners. Advisory council members are typically owners of one or more tools and resources that have been contributed to the FinOps toolkit. Current advisory council members:
Nelson Pereira
Nelson Pereira
Business sponsor		Michael Flanakin
Michael Flanakin
Product		Arthur Clares
Arthur Clares
Engineering		Sonia Cuff
Sonia Cuff
Community		Brett Wilson
Brett Wilson
Engineering		Michael Flanakin
Michael Flanakin
Product		Roland Krummenacher
Roland Krummenacher
Consulting
Manfred Simonis
Manfred Simonis
Consulting		Cedric Dupui
Cedric Dupui
FinOps Certified Professional		Mark Aggar
Mark Aggar
Internal customer		Dirk Brinkmann
Dirk Brinkmann
FinOps Certified Professional		Sonia Cuff
Sonia Cuff
Community
- - - + + + - - + diff --git a/docs-wiki/Governing-board.md b/docs-wiki/Governing-board.md index 9b8bb2f25..a12284fb6 100644 --- a/docs-wiki/Governing-board.md +++ b/docs-wiki/Governing-board.md @@ -67,19 +67,18 @@ If consensus cannot be reached within 14 days and the board believes the issue i The governing board consists of the following representative seats: -| Seat | Member | Time zone | -| ----------------------------- | --------------------- | --------- | -| Business sponsor | Nelson Pereira | UTC | -| Community | Sonia Cuff | UTC+10 | -| Consulting | Tanuja Shah | UTC-5 | -| Engineering | Arthur Clares | UTC | -| Marketing | Fernando Vasconcellos | UTC-5 | -| Product | Michael Flanakin | UTC-8 | -| FinOps Certified Professional | Dirk Brinkmann | UTC+1 | -| External | Vacant | | -| External | Vacant | | -| External | Vacant | | -| Advisory Council | Vacant | | +| Seat | Member | Time zone | +| ----------------------------- | ------------------- | --------- | +| Engineering | Brett Wilson | UTC-8 | +| FinOps Certified Professional | Dirk Brinkmann | UTC+1 | +| Community | Sonia Cuff | UTC+10 | +| Product (External) | Michael Flanakin | UTC-8 | +| Consulting (External) | Roland Krummenacher | UTC+1 | +| Marketing | Vacant | | +| External | Vacant | | +| External | Vacant | | +| External | Vacant | | +| Advisory Council | Vacant | | Individuals must meet the following criteria in order to attain a board seat: diff --git a/docs/README.md b/docs/README.md index e6750c0ff..0a847bb04 100644 --- a/docs/README.md +++ b/docs/README.md @@ -95,57 +95,57 @@ Whether you're looking for a little assistance or are interested in contributing
Brett Wilson
Brett Wilson
FinOps hubs		Seif Bassem
Seif Bassem
Cost opt workbook		Anthony Romano
Anthony Romano
PowerShell		Hélder Pinto
Hélder Pinto
Azure Opt Engine		Arthur Clares
Arthur Clares
Cost opt workbook		Nicolas Teyan
Nicolas Teyan
Gov workbook
Nicolas Teyan
Nicolas Teyan
Gov workbook		Hélder Pinto
Hélder Pinto
Azure Opt Engine		Anthony Romano
Anthony Romano
PowerShell		 Fernando Vasconcellos
Fernando Vasconcellos
Learning resources
- + - + + - - + - - - + - - - + + + + - + + - + - + - - - - - + + + + + From d7754eb733140b78d070825d1320196c41ee3c83 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Wed, 21 Jan 2026 05:54:20 +0000 Subject: [PATCH 59/69] Document contrib sync and office hours (#1944) Co-authored-by: Claude Opus 4.5 --- .github/policies/issues-01-new.yml | 7 +++ CONTRIBUTING.md | 4 ++ SUPPORT.md | 12 +++- .../toolkit/finops-toolkit-overview.md | 2 + docs-mslearn/toolkit/help/contributors.md | 2 + docs-mslearn/toolkit/help/errors.md | 2 +- docs-mslearn/toolkit/help/help-options.md | 2 + docs-mslearn/toolkit/help/support.md | 3 +- docs-mslearn/toolkit/help/troubleshooting.md | 10 ++- docs-wiki/Advisory-council.md | 18 +++++- docs-wiki/Governing-board.md | 30 ++++++--- docs-wiki/Home.md | 4 ++ docs-wiki/Support-escalations.md | 2 +- docs/contributor-sync.ics | 14 +++++ docs/contributor-sync.md | 63 +++++++++++++++++++ docs/help.md | 22 ++++--- docs/office-hours.ics | 14 +++++ docs/office-hours.md | 53 ++++++++++++++++ 18 files changed, 240 insertions(+), 24 deletions(-) create mode 100644 docs/contributor-sync.ics create mode 100644 docs/contributor-sync.md create mode 100644 docs/office-hours.ics create mode 100644 docs/office-hours.md diff --git a/.github/policies/issues-01-new.yml b/.github/policies/issues-01-new.yml index ebbf37e3e..96767bf2f 100644 --- a/.github/policies/issues-01-new.yml +++ b/.github/policies/issues-01-new.yml @@ -81,5 +81,12 @@ configuration: then: - addLabel: label: 'Needs: Triage 🔍' + - addReply: + reply: | + Thank you for submitting this issue! We try to respond within 3 business days. In the meantime: + + - 📖 Check the [troubleshooting guide](https://aka.ms/ftk/trouble) for common issues + - 💬 Ask questions in [Discussions](https://github.com/microsoft/finops-toolkit/discussions) + - 📅 Join our [biweekly office hours](https://aka.ms/ftk/office-hours) for live help from the team onFailure: onSuccess: diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e3005d35e..8f21334b1 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -26,6 +26,8 @@ Perhaps the simplest way to help make the FinOps toolkit better is simply to sha Have a question? Please ask in [Discussions](../../discussions). Avoid asking questions in issues to keep issues clean and focused on product improvements. If not sure, start a discussion and create an issue based on that discussion later, if needed. +You can also join our [biweekly office hours](https://aka.ms/ftk/office-hours) to get live help from the team on Wednesdays at 8:30 AM Pacific / 4:30 PM UTC. +
## ℹ️ Suggesting features and reporting bugs @@ -54,6 +56,8 @@ If you want to contribute code changes, but aren't sure where to start, scan thr If you find an issue that's assigned to someone, add a comment to check their status – there's a chance they may not have started yet. If you find an issue that isn't assigned, you are welcome to open a PR with a fix. +Want to stay connected with the team? Join our [weekly contributor sync](https://aka.ms/ftk/contrib-sync) on Wednesdays at 8 AM Pacific / 4 PM UTC. + diff --git a/SUPPORT.md b/SUPPORT.md index 3a554e68f..134cf7cee 100644 --- a/SUPPORT.md +++ b/SUPPORT.md @@ -4,10 +4,16 @@ This project uses GitHub issues to track bugs and feature requests. Please search the existing issues before filing new issues to avoid duplicates. For new issues, file your bug or feature request as a new issue. - +For help and questions about using this project: + +- 📖 Review the [troubleshooting guide](https://aka.ms/ftk/trouble) for common issues +- 💬 Ask questions in [Discussions](https://aka.ms/ftk/discuss) +- 📅 Join our [biweekly office hours](https://aka.ms/ftk/office-hours) for live help from the team + +We try to respond to issues and discussions within three business days. ## Microsoft Support Policy +Tools and resources within the FinOps toolkit are provided as-is without any express or implied warranties. The underlying Azure products used by tools in the toolkit are officially supported through standard Microsoft support channels. + Support for this project is limited to the resources listed above. diff --git a/docs-mslearn/toolkit/finops-toolkit-overview.md b/docs-mslearn/toolkit/finops-toolkit-overview.md index 2f62a3ccd..32ee19acc 100644 --- a/docs-mslearn/toolkit/finops-toolkit-overview.md +++ b/docs-mslearn/toolkit/finops-toolkit-overview.md @@ -61,6 +61,8 @@ If you have any questions or comments on past or future releases, [join the conv The FinOps toolkit is an open source project. We have many ideas on the long-term vision, but are more interested in learning from the community to drive the product direction. There are many ways you can contribute to the project. You can participate in discussions and request features to review and submitting pull requests. +Join our [biweekly office hours](https://aka.ms/ftk/office-hours) to ask questions and get live help from the team. Contributors can also join the [weekly contributor sync](https://aka.ms/ftk/contrib-sync) on Wednesdays at 8 AM Pacific / 4 PM UTC. + > [!div class="nextstepaction"] > [How to contribute](https://github.com/microsoft/finops-toolkit/blob/main/CONTRIBUTING.md) diff --git a/docs-mslearn/toolkit/help/contributors.md b/docs-mslearn/toolkit/help/contributors.md index 6bab7c87b..65107b349 100644 --- a/docs-mslearn/toolkit/help/contributors.md +++ b/docs-mslearn/toolkit/help/contributors.md @@ -26,6 +26,8 @@ There are several ways you can contribute to the FinOps toolkit: We appreciate your interest in contributing to the FinOps toolkit! +Want to stay connected with the team? Join our [weekly contributor sync](https://aka.ms/ftk/contrib-sync) on Wednesdays at 8 AM Pacific / 4 PM UTC or [biweekly office hours](https://aka.ms/ftk/office-hours) at 8:30 AM Pacific / 4:30 PM UTC. +
## Work on existing bugs diff --git a/docs-mslearn/toolkit/help/errors.md b/docs-mslearn/toolkit/help/errors.md index cd04425f4..aa9b76cc3 100644 --- a/docs-mslearn/toolkit/help/errors.md +++ b/docs-mslearn/toolkit/help/errors.md @@ -612,7 +612,7 @@ If the manifest properties look good and it was a new or upgraded FinOps hub ins 6. If the error persists, create a [discussion](https://aka.ms/ftk/discuss) to see if anyone else if facing an issue or knows of a possible workaround (especially for policy issues). 7. If the error is clearly a bug or feature gap, [create a bug or feature request issue in GitHub](https://aka.ms/ftk/ideas). -We try to respond to issues and discussions within two business days. +We try to respond to issues and discussions within three business days. Need live help? Join our [biweekly office hours](https://aka.ms/ftk/office-hours). + # How to get support for the FinOps toolkit Tools and resources within the FinOps toolkit are provided as-is without any express or implied warranties. Microsoft Support doesn't handle support requests for the FinOps toolkit. However, the underlying products used by tools in the toolkit are officially supported. @@ -57,6 +58,6 @@ If the source of the issue is a managed product (including data from Cost Manage Whether you submit a support request or not, we recommend [creating an issue](https://aka.ms/ftk/ideas) to let us know about the problems you're facing. Even if the issue is a product bug, we would like to document it to help others. -We try to respond to issues and discussions within two business days but there can sometimes be unanticipated delays. If you completed all of the preceding steps and the issue wasn't resolved within a week, we should set up a Teams call for you. Then you can share your screen so we can troubleshoot the issue together. +We try to respond to issues and discussions within three business days but there can sometimes be unanticipated delays. If you completed all of the preceding steps and the issue wasn't resolved within a week, join our [biweekly office hours](https://aka.ms/ftk/office-hours) to get live help from the team. If you need more hands-on support, you can request a paid, community-driven advisory session or consulting delivery during the office hours call.
diff --git a/docs-mslearn/toolkit/help/troubleshooting.md b/docs-mslearn/toolkit/help/troubleshooting.md index 5215f7efa..8fb9041a8 100644 --- a/docs-mslearn/toolkit/help/troubleshooting.md +++ b/docs-mslearn/toolkit/help/troubleshooting.md @@ -12,6 +12,7 @@ ms.reviewer: micflan --- + # FinOps toolkit troubleshooting guide This article describes how to validate FinOps toolkit solutions were deployed and configured correctly. If you have a specific error code, review [common errors](errors.md) for details and mitigation steps. If you need a more thorough walkthrough to validate your configuration, use the following steps that apply to you. @@ -31,6 +32,7 @@ If you have a specific error code, we recommend starting with [common errors](er ## Validate your FinOps hub deployment Use the following steps to validate your FinOps hub deployment: + +## Still need help? + +If you've followed the troubleshooting steps and still need assistance, join our [biweekly office hours](https://aka.ms/ftk/office-hours) to get live help from the team. If you need more hands-on support, you can request a paid, community-driven advisory session or consulting delivery during the office hours call. + +
+ ## Give feedback Let us know how we're doing with a quick review. We use these reviews to improve and expand FinOps tools and resources. diff --git a/docs-wiki/Advisory-council.md b/docs-wiki/Advisory-council.md index 9d8b1a5e7..d303d84bc 100644 --- a/docs-wiki/Advisory-council.md +++ b/docs-wiki/Advisory-council.md @@ -92,8 +92,22 @@ Individuals must meet the following criteria in order to attain a council seat: ## 📑 Meetings -The advisory council does not have dedicated meetings at this time. The advisory council is expected to participate (synchronously or asynchronously) in weekly FinOps toolkit sync calls on Wednesdays at 3 PM UTC or FinOps hubs sync calls on Mondays at 3:30 PM UTC. Meetings will be recorded to support asynchronous collaboration for those not able to join. Contributors and active community members may request to join these calls or view recordings as desired. +The advisory council does not have dedicated meetings at this time. The advisory council is expected to participate (synchronously or asynchronously) in the following meetings. Meetings are recorded to support asynchronous collaboration for those not able to join. Contributors and active community members may request to join these calls or view recordings as desired. -The advisory council is also invited to the monthly governing board meetings; however, attendance is optional. +- **Weekly contributor sync** + + > The weekly contributor sync is held on Wednesdays at 8 AM Pacific / 4 PM UTC. This is the primary meeting for FinOps toolkit contributors to discuss progress, blockers, and upcoming work. + > + > [Add to calendar](https://aka.ms/ftk/contrib-sync) + +- **Biweekly office hours** + + > Office hours are held every other Wednesday at 8:30 AM Pacific / 4:30 PM UTC. This is an open forum for anyone to ask questions, get help with issues, or discuss FinOps toolkit topics. + > + > [Add to calendar](https://aka.ms/ftk/office-hours) + +- **Biweekly governing board meetings** + + > The advisory council is also invited to the biweekly governing board meetings; however, attendance is optional.
diff --git a/docs-wiki/Governing-board.md b/docs-wiki/Governing-board.md index a12284fb6..fb96a8ff8 100644 --- a/docs-wiki/Governing-board.md +++ b/docs-wiki/Governing-board.md @@ -94,14 +94,30 @@ Individuals must meet the following criteria in order to attain a board seat: ## 📑 Meetings -Governing board meetings are held monthly. Meetings are recorded to support asynchronous collaboration for those not able to join. Advisory council members, contributors, and active community members may request to join these calls or view recordings. Recordings are only stored for 30 days. +Board members are expected to participate (synchronously or asynchronously) in the following meetings. Meetings are recorded to support asynchronous collaboration for those not able to join. Advisory council members, contributors, and active community members may request to join these calls or view recordings. Recordings are only stored for 30 days. -Governing board meetings have the following agenda: +- **Biweekly governing board meetings** -- Monthly updates (5m) -- Milestone progress, blockers, and risks (5-10m) -- OKR and KPI review (5-10m) -- New and ongoing initiatives (30m) -- Action items and next steps (5m) + > Governing board meetings are held biweekly with the following agenda: + > + > - Monthly updates (5m) + > - Milestone progress, blockers, and risks (5-10m) + > - OKR and KPI review (5-10m) + > - New and ongoing initiatives (30m) + > - Action items and next steps (5m) + > + > Governing board members are expected to attend or participate offline at least monthly. + +- **Weekly contributor sync** + + > The [weekly contributor sync](https://aka.ms/ftk/contrib-sync) is held on Wednesdays at 8 AM Pacific / 4 PM UTC. This is the primary meeting for FinOps toolkit contributors to discuss progress, blockers, and upcoming work. + > + > Governing board members are encouraged but not expected to join the contributor syncs. + +- **Biweekly office hours** + + > [Biweekly office hours](https://aka.ms/ftk/office-hours) are held every other Wednesday at 8:30 AM Pacific / 4:30 PM UTC. This is an open forum for anyone to ask questions, get help with issues, or discuss FinOps toolkit topics. + > + > Governing board members are encouraged but not expected to join the office hours.
diff --git a/docs-wiki/Home.md b/docs-wiki/Home.md index acdd5d665..f01ea041c 100644 --- a/docs-wiki/Home.md +++ b/docs-wiki/Home.md @@ -2,6 +2,8 @@ **Welcome aboard!** 👋 If this is your first time to our repo, here are a few tips: +> 💡 **Join us!** We host a weekly contributor sync on Wednesdays at 8 AM Pacific / 4 PM UTC. [Add to calendar][contributor-sync] to stay connected with the team. + - Every folder has a README that explains its purpose. - If you want to know how to deploy a FinOps toolkit solution, start with the [documentation](https://aka.ms/finops/toolkit). - If you want to know how you can contribute, check out the [contribution guide](../tree/dev/CONTRIBUTING.md). @@ -185,3 +187,5 @@ As a reminder, smaller PRs are closed quicker. If your PR has less than 20 lines Congratulations on your first PR! Hopefully it won't be your last! Once your PR is merged, changes are usually deployed the following week in the Azure portal. Note that changes behind a feature flag must be manually enabled or enabled for rollout within the host portal. In general, all new features are rolled out via experimentation within the Azure portal, so they may not be available immediately. + +[contributor-sync]: https://aka.ms/ftk/contrib-sync diff --git a/docs-wiki/Support-escalations.md b/docs-wiki/Support-escalations.md index d74553448..8f358c9c8 100644 --- a/docs-wiki/Support-escalations.md +++ b/docs-wiki/Support-escalations.md @@ -25,7 +25,7 @@ If you run into an issue, we recommend taking the following actions: 5. **Create support requests for product issues.**
If the source of the issue is a managed product (including data from Cost Management), create a Microsoft support request for that specific product. If you're not sure, ask in the [Q&A discussion forum](../discussions/categories/q-a).
  6. **Create an issue in GitHub.**
Whether you submit a support request or not, we recommend [creating an issue](https://aka.ms/ftk/idea) to let us know about the problems you're facing. Even if the issue is a product bug, we would like to document it to help others. -We try to respond to issues and discussions within 2 business days but there can sometimes be unanticipated delays. If you've completed all steps above and the issue has not been resolved within a week, we should set up a Teams call for you to share your screen so we can troubleshoot the issue together. +We try to respond to issues and discussions within 2 business days but there can sometimes be unanticipated delays. If you've completed all steps above and the issue has not been resolved within a week, join our [biweekly office hours](https://aka.ms/ftk/office-hours) to get live help from the team. If you need more hands-on support, you can request a paid, community-driven advisory session or consulting delivery during the office hours call. > [!NOTE] > Internal Microsoft employees are encouraged to utlize this same process for any issues. However, you are welcome to ping Microsoft employees in Microsoft Teams. diff --git a/docs/contributor-sync.ics b/docs/contributor-sync.ics new file mode 100644 index 000000000..971eedda5 --- /dev/null +++ b/docs/contributor-sync.ics @@ -0,0 +1,14 @@ +BEGIN:VCALENDAR +VERSION:2.0 +PRODID:-//Microsoft//FinOps toolkit//EN +METHOD:PUBLISH +BEGIN:VEVENT +DTSTART:20260107T160000Z +DTEND:20260107T163000Z +RRULE:FREQ=WEEKLY;BYDAY=WE +SUMMARY:FinOps toolkit contributor sync +DESCRIPTION:Working session for FinOps toolkit contributors to align on changes\, review proposals\, unblock work\, and coordinate contributions.\n\nThis is for: Contributors and maintainers working on code\, architecture\, docs\, or standards.\n\nThis is not for: Customer support\, demos\, or general community discussion.\n\nCome ready to collaborate\, share ideas\, and build together! +LOCATION:https://teams.microsoft.com/l/meetup-join/19%3ameeting_Nzg3OGYwMDEtNmJjNi00MzQ2LTliNWEtZmI5OGIzYTgyYTkz%40thread.v2/0?context=%7b%22Tid%22%3a%2210a66b8f-838f-4f58-a72b-419c6528f569%22%2c%22Oid%22%3a%22997adb29-26c9-4d91-b52a-58afd612dfdc%22%7d +UID:finops-toolkit-contributor-sync@microsoft.com +END:VEVENT +END:VCALENDAR diff --git a/docs/contributor-sync.md b/docs/contributor-sync.md new file mode 100644 index 000000000..5f7d1001f --- /dev/null +++ b/docs/contributor-sync.md @@ -0,0 +1,63 @@ +--- +layout: default +title: Contributor sync +nav_order: 997 +nav_exclude: true +description: 'Join the weekly contributor sync for FinOps toolkit contributors to align on changes, review proposals, unblock work, and coordinate contributions.' +permalink: /contributor-sync +--- + +FinOps toolkit contributor sync +Weekly working session for contributors to coordinate and collaborate. +{: .fs-6 .fw-300 } + +--- + +Working session for FinOps toolkit contributors to align on changes, review proposals, unblock work, and coordinate contributions. + +**This is for**: Contributors and maintainers working on code, architecture, docs, or standards. + +**This is not for**: Customer support, demos, or general community discussion. For help with FinOps toolkit issues, join our [biweekly office hours](./office-hours) instead. + +Come ready to collaborate, share ideas, and build together! + +
+ +## Schedule + +The contributor sync is held **every Wednesday** from **8:00-8:30 AM Pacific (4:00-4:30 PM UTC)**. + +Join the meeting +Add to calendar + +
+ +## What to expect + +Each sync typically covers: + +- Progress updates on current work +- Review of open pull requests +- Discussion of proposals and architecture decisions +- Unblocking contributors facing issues +- Coordination on upcoming releases + +
+ +## Before you join + +To make the most of your time: + +1. Review the [contribution guide](https://github.com/microsoft/finops-toolkit/blob/main/CONTRIBUTING.md) +2. Check the [GitHub wiki](https://github.com/microsoft/finops-toolkit/wiki) for development setup +3. Review open [pull requests](https://github.com/microsoft/finops-toolkit/pulls) and [issues](https://github.com/microsoft/finops-toolkit/issues) + +
+ +## Resources + +- [How to contribute](https://github.com/microsoft/finops-toolkit/blob/main/CONTRIBUTING.md) +- [FinOps toolkit office hours](./office-hours) +- [About the FinOps toolkit](https://aka.ms/finops/toolkit) + +
diff --git a/docs/help.md b/docs/help.md index 2c0c8363d..e72f47774 100644 --- a/docs/help.md +++ b/docs/help.md @@ -16,13 +16,8 @@ Have a question? We're here to help! Tools and resources within the FinOps toolkit are provided as-is without any express or implied warranties. Microsoft Support doesn't handle support requests for the FinOps toolkit. However, the underlying products used by tools in the toolkit are officially supported. -
- - Official Microsoft Support is coming to the FinOps toolkit! Expect official support to be available in early 2025. In the meantime, please use the resources below. - -
- +
@@ -152,7 +147,7 @@ Tools and resources within the FinOps toolkit are provided as-is without any exp
- +

Whether you submit a support request or not, create an issue to let us know about the problems you're facing. Even if the issue is a product bug, we would like to document it to help others. @@ -162,9 +157,20 @@ Tools and resources within the FinOps toolkit are provided as-is without any exp

+
+ +
+

+ Still need help? Join our biweekly office hours to ask questions and get live support from the FinOps toolkit team. +

+

+ Add to calendar +

+
+
-We try to respond to issues and discussions within two business days but there can sometimes be unanticipated delays. If you completed all of the preceding steps and the issue wasn't resolved within a week, we should set up a Teams call for you. Then you can share your screen so we can troubleshoot the issue together. +We try to respond to issues and discussions within three business days but there can sometimes be unanticipated delays. If you completed all of the preceding steps and the issue wasn't resolved within a week, join our [biweekly office hours](https://aka.ms/ftk/office-hours) to get live help from the team. If you need more hands-on support, you can request a paid, community-driven advisory session or consulting delivery during the office hours call. 💜 Give feedback diff --git a/docs/office-hours.ics b/docs/office-hours.ics new file mode 100644 index 000000000..af94282f4 --- /dev/null +++ b/docs/office-hours.ics @@ -0,0 +1,14 @@ +BEGIN:VCALENDAR +VERSION:2.0 +PRODID:-//Microsoft//FinOps toolkit//EN +METHOD:PUBLISH +BEGIN:VEVENT +DTSTART:20260121T163000Z +DTEND:20260121T170000Z +RRULE:FREQ=WEEKLY;INTERVAL=2;BYDAY=WE +SUMMARY:FinOps toolkit office hours +DESCRIPTION:Join the FinOps toolkit team for open office hours! Bring your questions about FinOps hubs\, Power BI reports\, or anything toolkit-related. Time permitting\, we can also help with other FinOps questions.\n\n- Get help with deployment issues\n- Discuss feature requests and roadmap\n- Share feedback and ideas\n- Connect with the community\n\nNo agenda. Just drop in! All experience levels are welcome!\n\nResources:\n- About the FinOps toolkit: https://aka.ms/finops/toolkit\n- Troubleshooting guide: https://learn.microsoft.com/cloud-computing/finops/toolkit/help/troubleshooting +LOCATION:https://teams.microsoft.com/l/meetup-join/19%3ameeting_ZjA0MzljMGUtOGU2My00MjYxLTlmM2EtYzM1YjRlNWQzMTQ0%40thread.v2/0?context=%7b%22Tid%22%3a%2210a66b8f-838f-4f58-a72b-419c6528f569%22%2c%22Oid%22%3a%22997adb29-26c9-4d91-b52a-58afd612dfdc%22%7d +UID:finops-toolkit-office-hours@microsoft.com +END:VEVENT +END:VCALENDAR diff --git a/docs/office-hours.md b/docs/office-hours.md new file mode 100644 index 000000000..94a18c52f --- /dev/null +++ b/docs/office-hours.md @@ -0,0 +1,53 @@ +--- +layout: default +title: Office hours +nav_order: 998 +nav_exclude: true +description: 'Join the FinOps toolkit team for open office hours to get help with deployment issues, discuss feature requests, and connect with the community.' +permalink: /office-hours +--- + +FinOps toolkit office hours +Join us for open office hours to get live help from the team. +{: .fs-6 .fw-300 } + +--- + +Join the FinOps toolkit team for open office hours! Bring your questions about FinOps hubs, Power BI reports, or anything toolkit-related. Time permitting, we can also help with other FinOps questions. + +- Get help with deployment issues +- Discuss feature requests and roadmap +- Share feedback and ideas +- Connect with the community + +No agenda. Just drop in! All experience levels are welcome! + +
+ +## Schedule + +Office hours are held **every other Wednesday** from **8:30-9:00 AM Pacific (4:30-5:00 PM UTC)**. + +Join the meeting +Add to calendar + +
+ +## Before you join + +To make the most of your time, we recommend: + +1. Review the [troubleshooting guide](https://aka.ms/ftk/trouble) for common issues +2. Check [common errors](https://learn.microsoft.com/cloud-computing/finops/toolkit/help/errors) if you have a specific error code +3. Search [existing issues](https://github.com/microsoft/finops-toolkit/issues) to see if your problem is known +4. Prepare specific questions or screenshots to share + +
+ +## Resources + +- [About the FinOps toolkit](https://aka.ms/finops/toolkit) +- [Troubleshooting guide](https://aka.ms/ftk/trouble) +- [How to get support](./help) + +
From 29f99536ec1474370d12d5a6d9025178b9fe5822 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 22 Jan 2026 17:17:34 -0800 Subject: [PATCH 60/69] Bump lodash from 4.17.21 to 4.17.23 (#1948) Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- package-lock.json | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/package-lock.json b/package-lock.json index eb7a2de9c..410c1a19e 100644 --- a/package-lock.json +++ b/package-lock.json @@ -339,9 +339,9 @@ } }, "node_modules/lodash": { - "version": "4.17.21", - "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz", - "integrity": "sha512-v2kDEe57lecTulaDIuNTPy3Ry4gLGJ6Z1O3vE1krgXZNrsQ+LFTGHVxVjcXPs17LhbZVGedAJv8XZ1tvj5FvSg==", + "version": "4.17.23", + "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.23.tgz", + "integrity": "sha512-LgVTMpQtIopCi79SJeDiP0TfWi5CNEc/L/aRdTh3yIvmZXTnheWpKjSZhnvMl8iXbC1tFg9gdHHDMLoV7CnG+w==", "dev": true }, "node_modules/mimic-fn": { @@ -964,9 +964,9 @@ } }, "lodash": { - "version": "4.17.21", - "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz", - "integrity": "sha512-v2kDEe57lecTulaDIuNTPy3Ry4gLGJ6Z1O3vE1krgXZNrsQ+LFTGHVxVjcXPs17LhbZVGedAJv8XZ1tvj5FvSg==", + "version": "4.17.23", + "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.23.tgz", + "integrity": "sha512-LgVTMpQtIopCi79SJeDiP0TfWi5CNEc/L/aRdTh3yIvmZXTnheWpKjSZhnvMl8iXbC1tFg9gdHHDMLoV7CnG+w==", "dev": true }, "mimic-fn": { From 1cff8cb3d7a15166ab236f1cc85995ac33076d5b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?H=C3=A9lder=20Pinto?= Date: Mon, 26 Jan 2026 22:03:48 +0000 Subject: [PATCH 61/69] [AOE] Changed default SQL database backup redundancy to LRS (#1951) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Hélder Pinto --- docs-mslearn/toolkit/changelog.md | 2 ++ src/optimization-engine/azuredeploy-nested.bicep | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 277541d2f..96acb5ed4 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -86,6 +86,8 @@ _Released January 2026_ ### [Optimization engine](optimization-engine/overview.md) v13 +- **Changed** + - Changed default SQL database backup redundancy to LRS, for improved cost efficiency and compatibility with deployments in non-paired Azure regions. - **Fixed** - Reservations-related workbooks fixed by replacing Instance Size Flexibility ratios CSV vanity URL with actual one to work around Log Analytics externaldata limitation ([#1810](https://github.com/microsoft/finops-toolkit/issues/1810)). - Underutilized disks recommendations were not being generated when customer environment has Premium SSD V2 disks ([#1831](https://github.com/microsoft/finops-toolkit/issues/1831)). diff --git a/src/optimization-engine/azuredeploy-nested.bicep b/src/optimization-engine/azuredeploy-nested.bicep index d05e2408a..0e12a764d 100644 --- a/src/optimization-engine/azuredeploy-nested.bicep +++ b/src/optimization-engine/azuredeploy-nested.bicep @@ -1761,7 +1761,7 @@ resource sqlDatabase 'Microsoft.Sql/servers/databases@2022-05-01-preview' = { zoneRedundant: false readScale: 'Disabled' autoPauseDelay: 60 - requestedBackupStorageRedundancy: 'Geo' + requestedBackupStorageRedundancy: 'Local' } } From 6f5552faab4600c8b9031bc2e60c26d34d6c2512 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?H=C3=A9lder=20Pinto?= Date: Tue, 27 Jan 2026 09:06:17 +0000 Subject: [PATCH 62/69] [AOE] Documentation improvements (#1954) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Hélder Pinto --- docs-mslearn/toolkit/changelog.md | 1 + .../toolkit/optimization-engine/configure-workspaces.md | 2 +- docs-mslearn/toolkit/optimization-engine/reports.md | 5 ++++- 3 files changed, 6 insertions(+), 2 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 96acb5ed4..5603a8b92 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -91,6 +91,7 @@ _Released January 2026_ - **Fixed** - Reservations-related workbooks fixed by replacing Instance Size Flexibility ratios CSV vanity URL with actual one to work around Log Analytics externaldata limitation ([#1810](https://github.com/microsoft/finops-toolkit/issues/1810)). - Underutilized disks recommendations were not being generated when customer environment has Premium SSD V2 disks ([#1831](https://github.com/microsoft/finops-toolkit/issues/1831)). + - Small documentation improvements and fixes to broken links. ### [FinOps workbooks](workbooks/finops-workbooks-overview.md) v13 diff --git a/docs-mslearn/toolkit/optimization-engine/configure-workspaces.md b/docs-mslearn/toolkit/optimization-engine/configure-workspaces.md index ab1a59324..1fe2af0c4 100644 --- a/docs-mslearn/toolkit/optimization-engine/configure-workspaces.md +++ b/docs-mslearn/toolkit/optimization-engine/configure-workspaces.md @@ -79,7 +79,7 @@ In summary, a Windows VM generates, in average, 245 bytes per performance counte ## Using multiple workspaces for performance logs -To include VMs from multiple Log Analytics workspaces in the VM right-size recommendations report, add a new variable named `AzureOptimization_RightSizeAdditionalPerfWorkspaces` to the AOE Azure Automation account. You can add any workspace to the scope of AOE, provided the AOE Managed Identity has Reader permissions over that workspace. The workspace can be in the same subscription or in any other subscription in the same tenant or even in a different tenant ([with the help of Lighthouse](./customize.md#widen-the-engine-scope)). +To include VMs from multiple Log Analytics workspaces in the VM right-size recommendations report, add a new variable named `AzureOptimization_RightSizeAdditionalPerfWorkspaces` to the AOE Azure Automation account. The variable value should be a comma-separated list of workspace IDs. You can add any workspace to the scope of AOE, provided the AOE Managed Identity has Reader permissions over that workspace. The workspace can be in the same subscription or in any other subscription in the same tenant or even in a different tenant ([with the help of Lighthouse](./customize.md#widen-the-engine-scope)). :::image type="content" source="./media/configure-workspaces/log-analytics-additional-performance-workspaces.png" border="true" alt-text="Screenshot showing adding an Automation Account variable with a list of additional workspace IDs VM right-size recommendations." lightbox="./media/configure-workspaces/log-analytics-additional-performance-workspaces.png"::: diff --git a/docs-mslearn/toolkit/optimization-engine/reports.md b/docs-mslearn/toolkit/optimization-engine/reports.md index 338915ae8..63838fbb5 100644 --- a/docs-mslearn/toolkit/optimization-engine/reports.md +++ b/docs-mslearn/toolkit/optimization-engine/reports.md @@ -20,7 +20,10 @@ This article explains reporting options available within Azure optimization engi ## Power BI recommendations report -AOE includes a [Power BI report](https://aka.ms/AzureOptimizationEngine/powerbi) for visualizing recommendations. To use it, you have first to change the data source connection to the SQL Database you deployed with the AOE. In the Power BI top menu, select **Transform Data** > **Data source settings**. +> [!div class="nextstepaction"] +> [Download](https://aka.ms/AzureOptimizationEngine/powerbi) + +AOE includes a Power BI report for visualizing recommendations. To use it, you have first to change the data source connection to the SQL Database you deployed with the AOE. In the Power BI top menu, select **Transform Data** > **Data source settings**. :::image type="content" source="./media/reports/power-bi-transform-data-menu.png" border="true" alt-text="Screenshot showing navigation to the Data source settings menu item." lightbox="./media/reports/power-bi-transform-data-menu.png"::: From cfc9c228aae5a78f4716b88439dbe8e21edc0716 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Wed, 28 Jan 2026 16:42:49 +0000 Subject: [PATCH 63/69] Change User Access Administrator to RBAC Administrator role (#1949) Co-authored-by: Roland Krummenacher --- docs-mslearn/toolkit/changelog.md | 2 + .../Exports/app.bicep | 5 - .../ManagedExports/app.bicep | 5 + .../Microsoft.FinOpsHubs/Analytics/app.bicep | 5 + .../finops-hub/modules/fx/hub-app.bicep | 2 +- .../modules/fx/scripts/Init-DataFactory.ps1 | 164 ++++++++++++++---- src/templates/finops-hub/modules/hub.bicep | 1 + 7 files changed, 142 insertions(+), 42 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 5603a8b92..750d11c9f 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -54,9 +54,11 @@ _Released January 2026_ - Enhanced [DataExplorerPreIngestionDropFailed](help/errors.md#dataexplorerpreingestiondropfailed) error documentation with troubleshooting guidance and cross-references. - **Changed** - Reorganized Bicep modules into separate apps. + - Changed the User Access Administrator role to RBAC Administrator and moved it to the Managed Exports app ([#1946](https://github.com/microsoft/finops-toolkit/issues/1946)). - Enhanced [Configure scopes documentation](hubs/configure-scopes.md) to explicitly clarify that FinOps hubs support: - Multiple Azure scopes (billing accounts, subscriptions, resource groups) in a single hub instance - Cross-cloud data ingestion through FOCUS format support + - Optimize trigger management script with retry logic and improved logging. - **Fixed** - Fixed duplicate Key Vault deployment in RemoteHub by removing redundant accessPolicies nested resource and adding proper dependencies. - Fixed all Bicep compilation errors and warnings with inline suppressions and descriptive comments. diff --git a/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep index 547bc9486..a3dae7ebf 100644 --- a/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep +++ b/src/templates/finops-hub/modules/Microsoft.CostManagement/Exports/app.bicep @@ -38,11 +38,6 @@ module appRegistration '../../fx/hub-app.bicep' = { 'Storage' // msexports + schema files 'DataFactory' // Move files from msexports to ingestion ] - storageRoles: [ - // User Access Administrator -- https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#user-access-administrator - // Used to create Cost Management exports (which require access to grant access) - '18d7d88d-d35e-4fb5-a5c3-7773c20a72d9' - ] } } diff --git a/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep b/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep index e75840fd0..c19b1cf05 100644 --- a/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep +++ b/src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep @@ -56,6 +56,11 @@ module appRegistration '../../fx/hub-app.bicep' = { features: [ 'DataFactory' ] + storageRoles: [ + // RBAC Administrator -- https://learn.microsoft.com/azure/role-based-access-control/built-in-roles/privileged#role-based-access-control-administrator + // Used to create Cost Management exports (which require access to grant access) + 'f58310d9-a9f6-439a-9e8d-f62e7b41a168' + ] } } diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep index 02f3b694e..521731d37 100644 --- a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep @@ -1867,6 +1867,11 @@ resource pipeline_ExecuteIngestionETL 'Microsoft.DataFactory/factories/pipelines // Run initialization pipeline after everything is deployed module runInitializationPipeline '../../fx/hub-initialize.bicep' = if (useAzure || useFabric) { name: 'Microsoft.FinOpsHubs.Analytics_InitializeHub' + dependsOn: [ + ingestion_InitScripts + ingestion_OpenDataInternalScripts + ingestion_VersionedScripts + ] params: { app: app dataFactoryInstances: [ diff --git a/src/templates/finops-hub/modules/fx/hub-app.bicep b/src/templates/finops-hub/modules/fx/hub-app.bicep index 3478030cf..68556d6a1 100644 --- a/src/templates/finops-hub/modules/fx/hub-app.bicep +++ b/src/templates/finops-hub/modules/fx/hub-app.bicep @@ -322,7 +322,7 @@ resource triggerManagerRoleAssignments 'Microsoft.Authorization/roleAssignments@ ] // Stop all triggers before deploying triggers -module stopTriggers 'hub-deploymentScript.bicep' = { +module stopTriggers 'hub-deploymentScript.bicep' = if (usesDataFactory) { name: '${app.publisher}.${app.name}_ADF.StopTriggers' dependsOn: [ // TODO: Do we need to make this optional only if private endpoints are enabled and telemetry is enabled? Will it fail when telemetry is disabled? diff --git a/src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 b/src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 index c6159a816..b5be36aea 100644 --- a/src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 +++ b/src/templates/finops-hub/modules/fx/scripts/Init-DataFactory.ps1 @@ -1,6 +1,47 @@ # Copyright (c) Microsoft Corporation. # Licensed under the MIT License. +<# +.SYNOPSIS +Manages Azure Data Factory triggers and pipelines during FinOps hub deployment. + +.DESCRIPTION +This script is called by Bicep deployment scripts to start/stop Data Factory triggers +and optionally run pipelines. It handles two types of triggers: + +1. Schedule triggers - Can be started/stopped directly +2. BlobEventsTriggers - Require Event Grid subscription management before start/stop + +BlobEventsTriggers use Event Grid to listen for storage blob events. Before stopping, +the Event Grid subscription must be removed (unsubscribed). Before starting, the +subscription must be added and fully provisioned. This script handles this automatically +by detecting BlobEventsTriggers via the BlobPathBeginsWith property. + +The script uses retry logic with linear backoff to handle transient API failures and +wait for Event Grid subscription provisioning/deprovisioning to complete. + +.PARAMETER DataFactoryResourceGroup +The resource group containing the Data Factory. + +.PARAMETER DataFactoryName +The name of the Data Factory instance. + +.PARAMETER Pipelines +Pipe-delimited list of pipeline names to run (e.g., "pipeline1|pipeline2"). + +.PARAMETER StartTriggers +Switch to start all stopped triggers (with Event Grid subscription for blob triggers). + +.PARAMETER StopTriggers +Switch to stop all running triggers (with Event Grid unsubscription for blob triggers). + +.EXAMPLE +.\Init-DataFactory.ps1 -DataFactoryResourceGroup "rg-hub" -DataFactoryName "adf-hub" -StopTriggers + +.EXAMPLE +.\Init-DataFactory.ps1 -DataFactoryResourceGroup "rg-hub" -DataFactoryName "adf-hub" -StartTriggers +#> + param( [string] $DataFactoryResourceGroup, [string] $DataFactoryName, @@ -9,69 +50,120 @@ param( [switch] $StopTriggers ) -# Init outputs +$MAX_RETRIES = 20 $DeploymentScriptOutputs = @{} -$RunPipelines = -not [string]::IsNullOrWhiteSpace($Pipelines) +function Write-Log($Message) +{ + Write-Output "$(Get-Date -Format 'HH:mm:ss') $Message" +} -if ($StartTriggers -or $RunPipelines) +function Invoke-WithRetry([scriptblock]$Action, [string]$Name, [int]$Delay = 5) { - Start-Sleep -Seconds 10 + for ($i = 1; $i -le $MAX_RETRIES; $i++) + { + try { return & $Action } + catch + { + Write-Log "$Name failed (attempt $i/${MAX_RETRIES}): $($_.Exception.Message)" + if ($i -eq $MAX_RETRIES) { throw } + Start-Sleep -Seconds ($Delay * $i) + } + } } -if ($StartTriggers -or $StopTriggers) +function Set-BlobTriggerSubscription([string]$TriggerName, [switch]$Subscribe) { - # Loop thru triggers - $triggers = Get-AzDataFactoryV2Trigger ` - -ResourceGroupName $DataFactoryResourceGroup ` - -DataFactoryName $DataFactoryName - - Write-Output "Found $($triggers.Length) trigger(s)" - Write-Output "StartTriggers: $StartTriggers" + $targetStatus = if ($Subscribe) { 'Enabled' } else { 'Disabled' } + $action = if ($Subscribe) { 'Subscribing' } else { 'Unsubscribing' } - $triggers | ForEach-Object { - $trigger = $_.Name - if ($StopTriggers) + Write-Log "$action $TriggerName to events..." + Invoke-WithRetry -Name "$action $TriggerName" -Delay 5 -Action { + if ($Subscribe) { - Write-Output "Stopping trigger $trigger..." - $triggerOutput = Stop-AzDataFactoryV2Trigger ` + Add-AzDataFactoryV2TriggerSubscription ` -ResourceGroupName $DataFactoryResourceGroup ` -DataFactoryName $DataFactoryName ` - -Name $trigger ` - -Force ` - -ErrorAction SilentlyContinue # Ignore errors, since the trigger may not exist + -Name $TriggerName | Out-Null } else { - Write-Output "Starting trigger $trigger..." - $triggerOutput = Start-AzDataFactoryV2Trigger ` + Remove-AzDataFactoryV2TriggerSubscription ` -ResourceGroupName $DataFactoryResourceGroup ` -DataFactoryName $DataFactoryName ` - -Name $trigger ` - -Force + -Name $TriggerName | Out-Null + } + + $status = Get-AzDataFactoryV2TriggerSubscriptionStatus ` + -ResourceGroupName $DataFactoryResourceGroup ` + -DataFactoryName $DataFactoryName ` + -Name $TriggerName + if ($status.Status -ne $targetStatus) + { + throw "Subscription status is $($status.Status), expected $targetStatus" + } + } +} + +if ($StartTriggers -or $StopTriggers) +{ + $triggers = Invoke-WithRetry -Name "Get triggers" -Action { + Get-AzDataFactoryV2Trigger ` + -ResourceGroupName $DataFactoryResourceGroup ` + -DataFactoryName $DataFactoryName ` + | Where-Object { + ($StartTriggers -and $_.Properties.RuntimeState -ne "Started") ` + -or ($StopTriggers -and $_.Properties.RuntimeState -ne "Stopped") } - if ($triggerOutput) + } + + Write-Log "Found $($triggers.Count) trigger(s) to $(if ($StartTriggers) { 'start' } else { 'stop' })" + + $triggers | ForEach-Object { + $triggerName = $_.Name + $isBlobTrigger = $null -ne $_.Properties.BlobPathBeginsWith + + if ($StopTriggers) { - Write-Output "done..." + if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName } + Write-Log "Stopping trigger $triggerName..." + Invoke-WithRetry -Name "Stop $triggerName" -Action { + Stop-AzDataFactoryV2Trigger ` + -ResourceGroupName $DataFactoryResourceGroup ` + -DataFactoryName $DataFactoryName ` + -Name $triggerName -Force + } } else { - Write-Output "failed..." + if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName -Subscribe } + Write-Log "Starting trigger $triggerName..." + Invoke-WithRetry -Name "Start $triggerName" -Action { + Start-AzDataFactoryV2Trigger ` + -ResourceGroupName $DataFactoryResourceGroup ` + -DataFactoryName $DataFactoryName ` + -Name $triggerName -Force + } } - $DeploymentScriptOutputs[$trigger] = $triggerOutput - } - if ($StopTriggers) - { - Start-Sleep -Seconds 10 + Invoke-WithRetry -Name "Wait for $triggerName" -Action { + $state = (Get-AzDataFactoryV2Trigger ` + -ResourceGroupName $DataFactoryResourceGroup ` + -DataFactoryName $DataFactoryName ` + -Name $triggerName).Properties.RuntimeState + $expected = if ($StartTriggers) { 'Started' } else { 'Stopped' } + if ($state -ne $expected) { throw "Trigger is $state, expected $expected" } + } + + Write-Log "...done" + $DeploymentScriptOutputs[$triggerName] = $true } } -if ($RunPipelines) +if (-not [string]::IsNullOrWhiteSpace($Pipelines)) { - $Pipelines.Split('|') ` - | ForEach-Object { - Write-Output "Running the init pipeline..." + $Pipelines.Split('|') | ForEach-Object { + Write-Log "Running pipeline $_..." Invoke-AzDataFactoryV2Pipeline ` -ResourceGroupName $DataFactoryResourceGroup ` -DataFactoryName $DataFactoryName ` diff --git a/src/templates/finops-hub/modules/hub.bicep b/src/templates/finops-hub/modules/hub.bicep index 440aeca64..4cfa43646 100644 --- a/src/templates/finops-hub/modules/hub.bicep +++ b/src/templates/finops-hub/modules/hub.bicep @@ -357,6 +357,7 @@ module startTriggers 'fx/hub-initialize.bicep' = { analytics deleteOldResources remoteHub + cmManagedExports ] params: { app: core.outputs.app From 6c520a125cc22497903826405302efaf501d796c Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Sat, 31 Jan 2026 07:18:53 +0000 Subject: [PATCH 64/69] v13 release readiness (#1956) Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs-mslearn/toolkit/changelog.md | 58 +- docs-mslearn/toolkit/hubs/compatibility.md | 2 +- docs-mslearn/toolkit/hubs/upgrade.md | 2 +- docs/_includes/ftktag.txt | 1 + docs/_includes/ftkver.txt | 2 +- docs/deploy/finops-alerts-13.0.json | 4384 ++ docs/deploy/finops-alerts-13.0.ui.json | 72 + docs/deploy/finops-alerts-latest.json | 12 +- docs/deploy/finops-hub-13.0.json | 24950 ++++++++++++ docs/deploy/finops-hub-13.0.ui.json | 792 + docs/deploy/finops-hub-latest.json | 33571 +++++++++------- docs/deploy/finops-hub-latest.ui.json | 47 + docs/deploy/finops-workbooks-13.0.json | 20933 ++++++++++ docs/deploy/finops-workbooks-13.0.ui.json | 65 + docs/deploy/finops-workbooks-latest.json | 50 +- .../13.0/azuredeploy-nested.bicep | 2158 + .../13.0/azuredeploy.bicep | 82 + .../Export-AADObjectsToBlobStorage.ps1 | 519 + ...t-ARGAppGatewayPropertiesToBlobStorage.ps1 | 231 + ...GAppServicePlanPropertiesToBlobStorage.ps1 | 209 + ...AvailabilitySetPropertiesToBlobStorage.ps1 | 198 + ...ARGLoadBalancerPropertiesToBlobStorage.ps1 | 222 + ...ARGManagedDisksPropertiesToBlobStorage.ps1 | 232 + .../Export-ARGNICPropertiesToBlobStorage.ps1 | 235 + .../Export-ARGNSGPropertiesToBlobStorage.ps1 | 217 + ...ort-ARGPublicIpPropertiesToBlobStorage.ps1 | 275 + ...ourceContainersPropertiesToBlobStorage.ps1 | 272 + ...-ARGSqlDatabasePropertiesToBlobStorage.ps1 | 204 + ...GUnmanagedDisksPropertiesToBlobStorage.ps1 | 236 + .../Export-ARGVMSSPropertiesToBlobStorage.ps1 | 239 + .../Export-ARGVNetPropertiesToBlobStorage.ps1 | 308 + ...VirtualMachinesPropertiesToBlobStorage.ps1 | 340 + ...rt-AdvisorRecommendationsToBlobStorage.ps1 | 247 + .../Export-AzMonitorMetricsToBlobStorage.ps1 | 296 + .../Export-ConsumptionToBlobStorage.ps1 | 875 + .../Export-PolicyComplianceToBlobStorage.ps1 | 644 + .../Export-PriceSheetToBlobStorage.ps1 | 452 + .../Export-RBACAssignmentsToBlobStorage.ps1 | 268 + .../Export-ReservationsPriceToBlobStorage.ps1 | 146 + .../Export-ReservationsUsageToBlobStorage.ps1 | 304 + .../Export-SavingsPlansUsageToBlobStorage.ps1 | 263 + ...t-OptimizationCSVExportsToLogAnalytics.ps1 | 344 + ...anUp-OlderRecommendationsFromSqlServer.ps1 | 84 + .../Ingest-RecommendationsToLogAnalytics.ps1 | 325 + .../Ingest-RecommendationsToSQLServer.ps1 | 291 + .../Ingest-SuppressionsToLogAnalytics.ps1 | 249 + ...nd-AADExpiringCredentialsToBlobStorage.ps1 | 371 + ...ecommend-ARMOptimizationsToBlobStorage.ps1 | 517 + .../Recommend-AdvisorAsIsToBlobStorage.ps1 | 315 + ...mend-AdvisorCostAugmentedToBlobStorage.ps1 | 903 + ...d-AppServiceOptimizationsToBlobStorage.ps1 | 695 + ...commend-DiskOptimizationsToBlobStorage.ps1 | 539 + ...ommend-SqlDbOptimizationsToBlobStorage.ps1 | 447 + ...orageAccountOptimizationsToBlobStorage.ps1 | 330 + ...Recommend-UnattachedDisksToBlobStorage.ps1 | 272 + .../Recommend-UnusedAppGWsToBlobStorage.ps1 | 273 + ...mmend-UnusedLoadBalancersToBlobStorage.ps1 | 404 + ...Recommend-VMOptimizationsToBlobStorage.ps1 | 461 + ...commend-VMSSOptimizationsToBlobStorage.ps1 | 797 + ...mmend-VMsHighAvailabilityToBlobStorage.ps1 | 1476 + ...commend-VNetOptimizationsToBlobStorage.ps1 | 1329 + .../Remediate-AdvisorRightSizeFiltered.ps1 | 225 + .../Remediate-LongDeallocatedVMsFiltered.ps1 | 306 + .../Remediate-UnattachedDisksFiltered.ps1 | 286 + .../latest/azuredeploy-nested.bicep | 2 +- ...commend-DiskOptimizationsToBlobStorage.ps1 | 4 +- docs/guide.md | 6 +- docs/hubs.md | 2 +- docs/open-data.md | 6 +- docs/powershell.md | 6 +- docs/workbooks.md | 6 +- package-lock.json | 4 +- package.json | 2 +- src/optimization-engine/ftkver.txt | 2 +- .../definition/tables/Costs.tmdl | 19 +- src/powershell/Private/Get-VersionNumber.ps1 | 2 +- .../Private/Save-FinOpsHubTemplate.ps1 | 2 +- src/powershell/Public/Deploy-FinOpsHub.ps1 | 2 +- .../Tests/Integration/CostExports.Tests.ps1 | 6 +- .../Tests/Integration/Hubs.Tests.ps1 | 2 +- .../Tests/Integration/New-Directory.Tests.ps1 | 2 +- .../Tests/Integration/Toolkit.Tests.ps1 | 14 +- .../Tests/Unit/Deploy-FinOpsHub.Tests.ps1 | 2 +- src/scripts/Build-OpenData.ps1 | 86 +- src/scripts/Get-Version.ps1 | 6 +- src/scripts/Update-Version.ps1 | 17 +- .../finops-alerts/modules/ftkver.txt | 2 +- .../Microsoft.FinOpsHubs/Analytics/app.bicep | 7 +- .../finops-hub/modules/fx/ftktag.txt | 1 + .../finops-hub/modules/fx/ftkver.txt | 2 +- src/templates/finops-workbooks/ftkver.txt | 2 +- src/workbooks/.scaffold/ftkver.txt | 2 +- 92 files changed, 91898 insertions(+), 14170 deletions(-) create mode 100644 docs/_includes/ftktag.txt create mode 100644 docs/deploy/finops-alerts-13.0.json create mode 100644 docs/deploy/finops-alerts-13.0.ui.json create mode 100644 docs/deploy/finops-hub-13.0.json create mode 100644 docs/deploy/finops-hub-13.0.ui.json create mode 100644 docs/deploy/finops-workbooks-13.0.json create mode 100644 docs/deploy/finops-workbooks-13.0.ui.json create mode 100644 docs/deploy/optimization-engine/13.0/azuredeploy-nested.bicep create mode 100644 docs/deploy/optimization-engine/13.0/azuredeploy.bicep create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-AADObjectsToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGAppGatewayPropertiesToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGAppServicePlanPropertiesToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGAvailabilitySetPropertiesToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGLoadBalancerPropertiesToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGManagedDisksPropertiesToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGNICPropertiesToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGNSGPropertiesToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGPublicIpPropertiesToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGResourceContainersPropertiesToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGSqlDatabasePropertiesToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGUnmanagedDisksPropertiesToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGVMSSPropertiesToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGVNetPropertiesToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGVirtualMachinesPropertiesToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-AdvisorRecommendationsToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-AzMonitorMetricsToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ConsumptionToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-PolicyComplianceToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-PriceSheetToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-RBACAssignmentsToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ReservationsPriceToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ReservationsUsageToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-SavingsPlansUsageToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/maintenance/CleanUp-OlderRecommendationsFromSqlServer.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-RecommendationsToSQLServer.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AADExpiringCredentialsToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-ARMOptimizationsToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AdvisorAsIsToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AdvisorCostAugmentedToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AppServiceOptimizationsToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-DiskOptimizationsToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-SqlDbOptimizationsToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-StorageAccountOptimizationsToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-UnattachedDisksToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-UnusedAppGWsToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-UnusedLoadBalancersToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VMOptimizationsToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VMSSOptimizationsToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VMsHighAvailabilityToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VNetOptimizationsToBlobStorage.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/remediations/Remediate-AdvisorRightSizeFiltered.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/remediations/Remediate-LongDeallocatedVMsFiltered.ps1 create mode 100644 docs/deploy/optimization-engine/13.0/runbooks/remediations/Remediate-UnattachedDisksFiltered.ps1 create mode 100644 src/templates/finops-hub/modules/fx/ftktag.txt diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 750d11c9f..838f33028 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -38,42 +38,32 @@ _Released January 2026_ ### [Implementing FinOps guide](../implementing-finops-guide.md) v13 - **Added** - - Created comprehensive [Data Lake Storage connectivity options](data-lake-storage-connectivity.md) documentation covering tools and services beyond Power BI including Azure Data Explorer, Microsoft Fabric, Azure Synapse Analytics, Azure Databricks, and custom applications. - - Enhanced troubleshooting guidance for [Data Explorer ingestion errors](help/errors.md#dataexploreringestionfailed) with detailed steps for diagnosing and resolving the common SEM0080 "Ingestion Failed" semantic error, including schema mismatch detection, ingestion mapping verification, and diagnostic query examples. + - Added [Data Lake Storage connectivity options](data-lake-storage-connectivity.md) documentation covering tools and services beyond Power BI. + - Added troubleshooting guidance for [Data Explorer ingestion errors](help/errors.md#dataexploreringestionfailed), including the SEM0080 "Ingestion Failed" semantic error. - **Changed** - - Updated FinOps framework documentation to prepare for Azure TCO calculator retirement scheduled for August 31, 2025. Azure Migrate cost estimation functionality remains available. - - Updated FOCUS converter documentation to include newly added fields in FOCUS 1.2-preview specification, including ServiceSubcategory and renamed columns (InvoiceId, PricingCurrency, SkuMeter). + - Updated FOCUS converter documentation for newly added fields in the FOCUS 1.2-preview specification, including ServiceSubcategory, InvoiceId, PricingCurrency, and SkuMeter. + - Removed Azure TCO calculator references for tool retirement on August 31, 2025. ### [FinOps hubs](hubs/finops-hubs-overview.md) v13 - **Added** - - Added optional `enablePurgeProtection` parameter (default: `false`) to enable purge protection on the Key Vault for compliance with enterprise-scale Azure Landing Zone policies. Available in both `main.bicep` and `modules/hub.bicep` ([#1067](https://github.com/microsoft/finops-toolkit/issues/1067)). - - Document [how to remove private networking](hubs/private-networking.md#removing-private-networking) and switch back to public access to reduce costs ([#1342](https://github.com/microsoft/finops-toolkit/issues/1342)). - - Added comprehensive troubleshooting guidance for [ErrorCodeNotString](help/errors.md#errorcodenotstring) error that occurs when Azure Data Factory Fail activities cannot evaluate dynamic expressions. - - Enhanced [DataExplorerPostIngestionDropFailed](help/errors.md#dataexplorerpostingestiondropfailed) error documentation with detailed troubleshooting steps, common scenarios, and links to Microsoft Learn resources. - - Enhanced [DataExplorerPreIngestionDropFailed](help/errors.md#dataexplorerpreingestiondropfailed) error documentation with troubleshooting guidance and cross-references. + - Added optional `enablePurgeProtection` parameter (default: `false`) to enable Key Vault purge protection for compliance with Azure Landing Zone policies ([#1067](https://github.com/microsoft/finops-toolkit/issues/1067)). + - Added documentation for [how to remove private networking](hubs/private-networking.md#removing-private-networking) and switch back to public access to reduce costs ([#1342](https://github.com/microsoft/finops-toolkit/issues/1342)). + - Added troubleshooting guidance for [ErrorCodeNotString](help/errors.md#errorcodenotstring), [DataExplorerPostIngestionDropFailed](help/errors.md#dataexplorerpostingestiondropfailed), and [DataExplorerPreIngestionDropFailed](help/errors.md#dataexplorerpreingestiondropfailed) errors. - **Changed** - Reorganized Bicep modules into separate apps. - Changed the User Access Administrator role to RBAC Administrator and moved it to the Managed Exports app ([#1946](https://github.com/microsoft/finops-toolkit/issues/1946)). - - Enhanced [Configure scopes documentation](hubs/configure-scopes.md) to explicitly clarify that FinOps hubs support: - - Multiple Azure scopes (billing accounts, subscriptions, resource groups) in a single hub instance - - Cross-cloud data ingestion through FOCUS format support - - Optimize trigger management script with retry logic and improved logging. + - Updated [Configure scopes documentation](hubs/configure-scopes.md) to clarify support for multiple Azure scopes and cross-cloud data ingestion. + - Optimized trigger management script with retry logic and improved logging. - **Fixed** - - Fixed duplicate Key Vault deployment in RemoteHub by removing redundant accessPolicies nested resource and adding proper dependencies. - - Fixed all Bicep compilation errors and warnings with inline suppressions and descriptive comments. - - Fixed Build-Toolkit.ps1 bicep generate-params command bug. - - Fixed Azure Data Explorer dashboard queries by converting `todecimal(0)` to `toreal(0)` to ensure compatibility with KQL type system ([#1893](https://github.com/microsoft/finops-toolkit/issues/1893)). - - Fixed Costs_v1_2 function producing duplicate records when Services table contains multiple entries for the same resource type, causing cost discrepancies between dashboards. - - Fixed `config_InitializeHub` pipeline failure in Azure Data Explorer caused by HTTP redirects when loading open data CSV files (PricingUnits, Regions, ResourceTypes, Services) from GitHub releases. Updated to use raw.githubusercontent.com URLs that do not redirect ([#1886](https://github.com/microsoft/finops-toolkit/issues/1886)). - - Fixed logic to properly generate the scopes to monitor. - - Fixed datatype mismatch in InitializeHub pipeline by changing `x_PricingBlockSize` from `decimal` to `real` to match PricingUnits table schema. - - Fixed ADF pipeline dependency logic in config_RunBackfillJob, config_StartExportProcess, and config_ConfigureExports pipelines to properly handle both array and non-array scope configurations by adding 'Failed' condition to 'Save/Set Scopes' activity dependencies. - - Fixed backward compatibility in `Costs_transform_v1_2()` to support Cost Management exports that predate FOCUS 1.2 by adding fallback mappings for `PricingCurrency` and `SkuMeter` columns to their legacy `x_` counterparts. - - Fixed RemoteHub manifest file not being copied to remote storage. The ingestion_manifest dataset is now consistently handled like other ingestion datasets (ingestion, ingestion_files) - created by the Core module and overridden by the RemoteHub module when configured, ensuring manifests are written to the correct storage location. Also renamed manifest_source to msexports_manifest and manifest_sink to ingestion_manifest for clarity. - - Fixed broken link for GitHub Copilot instructions download in [Configure AI documentation](hubs/configure-ai.md). The packaging process now respects the `unversionedZip` property in `.build.config` to create unversioned ZIP files for finops-hub-copilot, enabling stable download links ([#1803](https://github.com/microsoft/finops-toolkit/issues/1803)). - - Fixed ADF triggers not starting after deployment due to `$startTriggers` variable not being set from the `StartAllTriggers` environment variable in the Init-DataFactory.ps1 script. - - Fixed version format mismatch causing `config_InitializeHub` pipeline to fail when loading open data files from GitHub. The version in ftkver.txt (e.g., `12.0`) now matches the git tag format (e.g., `v12`) ([#1885](https://github.com/microsoft/finops-toolkit/issues/1885)). + - Fixed ADF triggers not starting after deployment. + - Fixed ADF pipeline dependency logic to properly handle both array and non-array scope configurations. + - Fixed `config_InitializeHub` pipeline failures caused by GitHub URL redirects and version format mismatches ([#1885](https://github.com/microsoft/finops-toolkit/issues/1885), [#1886](https://github.com/microsoft/finops-toolkit/issues/1886)). + - Fixed scope generation logic and datatype mismatch (`x_PricingBlockSize`) in InitializeHub pipeline. + - Fixed Azure Data Explorer dashboard queries by converting `todecimal(0)` to `toreal(0)` for KQL compatibility ([#1893](https://github.com/microsoft/finops-toolkit/issues/1893)). + - Fixed `Costs_v1_2` function producing duplicate records when Services table contains multiple entries for the same resource type. + - Fixed backward compatibility in `Costs_transform_v1_2()` to support Cost Management exports that predate FOCUS 1.2. + - Fixed broken link for GitHub Copilot instructions download in [Configure AI documentation](hubs/configure-ai.md) ([#1803](https://github.com/microsoft/finops-toolkit/issues/1803)). ### [Power BI reports](power-bi/reports.md) v13 @@ -81,18 +71,18 @@ _Released January 2026_ - Added export requirements sections to all Power BI report documentation pages to clarify which Cost Management exports are needed for each report. - Added Azure Resource Graph as an explicit requirement for governance and workload optimization reports. - **Fixed** - - Fixed tag expansion in Power BI reports when tag names contain special characters like colons. - - Fixed unattached disks count in the workload optimization report to show only truly unattached disks instead of all disks. The card visual now filters disks where (managedBy is empty and diskState is not ActiveSAS) or (diskState is Unattached and not ActiveSAS) ([#1896](https://github.com/microsoft/finops-toolkit/issues/1896)). - - Fixed "Number of Months" parameter calculation that was excluding the first 5 days of data when set to 3 months ([#1833](https://github.com/microsoft/finops-toolkit/issues/1833)). - - Fixed EA department scope failing on pricesheet export by skipping pricesheet exports for scopes that don't support them ([#1870](https://github.com/microsoft/finops-toolkit/issues/1870)). + - Fixed tag expansion when tag names contain special characters like colons. + - Fixed unattached disks count in the workload optimization report to show only truly unattached disks ([#1896](https://github.com/microsoft/finops-toolkit/issues/1896)). + - Fixed "Number of Months" parameter calculation that was excluding the first 5 days of data ([#1833](https://github.com/microsoft/finops-toolkit/issues/1833)). + - Fixed EA department scope failing on pricesheet export by skipping unsupported scopes ([#1870](https://github.com/microsoft/finops-toolkit/issues/1870)). ### [Optimization engine](optimization-engine/overview.md) v13 - **Changed** - - Changed default SQL database backup redundancy to LRS, for improved cost efficiency and compatibility with deployments in non-paired Azure regions. + - Changed default SQL database backup redundancy to LRS for improved cost efficiency and compatibility with non-paired Azure regions. - **Fixed** - - Reservations-related workbooks fixed by replacing Instance Size Flexibility ratios CSV vanity URL with actual one to work around Log Analytics externaldata limitation ([#1810](https://github.com/microsoft/finops-toolkit/issues/1810)). - - Underutilized disks recommendations were not being generated when customer environment has Premium SSD V2 disks ([#1831](https://github.com/microsoft/finops-toolkit/issues/1831)). + - Fixed reservations-related workbooks by replacing Instance Size Flexibility ratios CSV vanity URL ([#1810](https://github.com/microsoft/finops-toolkit/issues/1810)). + - Fixed underutilized disks recommendations not being generated for Premium SSD V2 disks ([#1831](https://github.com/microsoft/finops-toolkit/issues/1831)). - Small documentation improvements and fixes to broken links. ### [FinOps workbooks](workbooks/finops-workbooks-overview.md) v13 diff --git a/docs-mslearn/toolkit/hubs/compatibility.md b/docs-mslearn/toolkit/hubs/compatibility.md index 5ff974f72..58d2232d9 100644 --- a/docs-mslearn/toolkit/hubs/compatibility.md +++ b/docs-mslearn/toolkit/hubs/compatibility.md @@ -3,7 +3,7 @@ title: Compatibility guide description: Learn which Power BI report versions are compatible with each FinOps hubs version to ensure seamless upgrades and data integrity. author: flanakin ms.author: micflan -ms.date: 05/06/2025 +ms.date: 01/29/2026 ms.topic: concept-article ms.service: finops ms.subservice: finops-toolkit diff --git a/docs-mslearn/toolkit/hubs/upgrade.md b/docs-mslearn/toolkit/hubs/upgrade.md index 69fd19ca3..1b6486453 100644 --- a/docs-mslearn/toolkit/hubs/upgrade.md +++ b/docs-mslearn/toolkit/hubs/upgrade.md @@ -3,7 +3,7 @@ title: Upgrade your FinOps hubs description: Learn how to upgrade your existing FinOps hub instance to the latest version, including necessary steps and considerations. author: flanakin ms.author: micflan -ms.date: 04/02/2025 +ms.date: 01/29/2026 ms.topic: how-to ms.service: finops ms.subservice: finops-toolkit diff --git a/docs/_includes/ftktag.txt b/docs/_includes/ftktag.txt new file mode 100644 index 000000000..ca7bf83ac --- /dev/null +++ b/docs/_includes/ftktag.txt @@ -0,0 +1 @@ +13 \ No newline at end of file diff --git a/docs/_includes/ftkver.txt b/docs/_includes/ftkver.txt index 3cacc0b93..7f27d6b1d 100644 --- a/docs/_includes/ftkver.txt +++ b/docs/_includes/ftkver.txt @@ -1 +1 @@ -12 \ No newline at end of file +13.0 \ No newline at end of file diff --git a/docs/deploy/finops-alerts-13.0.json b/docs/deploy/finops-alerts-13.0.json new file mode 100644 index 000000000..88401106b --- /dev/null +++ b/docs/deploy/finops-alerts-13.0.json @@ -0,0 +1,4384 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "12460451954863791061" + } + }, + "parameters": { + "location": { + "type": "string", + "defaultValue": "[resourceGroup().location]", + "metadata": { + "description": "Azure location where resources should be created" + } + }, + "appName": { + "type": "string", + "defaultValue": "finops-alerts", + "minLength": 1, + "maxLength": 20, + "metadata": { + "description": "Name of the logic app" + } + }, + "recurrenceFrequency": { + "type": "string", + "defaultValue": "Week", + "metadata": { + "description": "Specifies the frequency of the recurrence trigger. Possible values are Week, Day or Hour." + } + }, + "recurrenceInterval": { + "type": "int", + "defaultValue": 1, + "metadata": { + "description": "Specifies the interval for the recurrence trigger. Represents the number of frequency units." + } + }, + "recurrenceType": { + "type": "string", + "defaultValue": "Recurrence", + "metadata": { + "description": "Specifies the type of the trigger. For this example, it is a recurrence trigger." + } + }, + "logicAppSubscriptionId": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "The Id of the subscription to deploy the logic app in." + } + }, + "resourceGroupName": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "The name of the resource group." + } + }, + "enableDefaultTelemetry": { + "type": "bool", + "defaultValue": true, + "metadata": { + "description": "Optional. Enable telemetry to track anonymous module usage trends, monitor for bugs, and improve future releases." + } + } + }, + "variables": { + "$fxv#0": "13.0" + }, + "resources": [ + { + "condition": "[parameters('enableDefaultTelemetry')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "[format('pid-00f120b5-2007-6120-0000-a7e122500000-{0}', uniqueString(deployment().name, parameters('location')))]", + "properties": { + "mode": "Incremental", + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "FinOps toolkit", + "version": "[variables('$fxv#0')]" + } + }, + "resources": [] + } + } + }, + { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('logicApp-{0}', uniqueString(deployment().name, parameters('location'), parameters('appName')))]", + "subscriptionId": "[parameters('logicAppSubscriptionId')]", + "resourceGroup": "[parameters('resourceGroupName')]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "appName": { + "value": "[parameters('appName')]" + }, + "location": { + "value": "[parameters('location')]" + }, + "recurrenceFrequency": { + "value": "[parameters('recurrenceFrequency')]" + }, + "recurrenceInterval": { + "value": "[parameters('recurrenceInterval')]" + }, + "recurrenceType": { + "value": "[parameters('recurrenceType')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "10958230289205929" + } + }, + "parameters": { + "location": { + "type": "string", + "defaultValue": "[resourceGroup().location]", + "metadata": { + "description": "Azure location where resources should be created" + } + }, + "appName": { + "type": "string", + "defaultValue": "finops-alerts", + "minLength": 1, + "maxLength": 20, + "metadata": { + "description": "Name of the logic app" + } + }, + "recurrenceFrequency": { + "type": "string", + "defaultValue": "Week", + "metadata": { + "description": "Specifies the frequency of the recurrence trigger. Possible values are Week, Day or Hour." + } + }, + "recurrenceInterval": { + "type": "int", + "defaultValue": 1, + "metadata": { + "description": "Specifies the interval for the recurrence trigger. Represents the number of frequency units." + } + }, + "recurrenceType": { + "type": "string", + "defaultValue": "Recurrence", + "metadata": { + "description": "Specifies the type of the trigger. For this example, it is a recurrence trigger." + } + } + }, + "variables": { + "safeSuffix": "[replace(replace(toLower(parameters('appName')), '-', ''), '_', '')]", + "connectionName": "[format('{0}-connection', variables('safeSuffix'))]", + "displayName": "[format('{0}-connection', variables('safeSuffix'))]", + "actionKeys": [ + "Send_an_email_V2" + ] + }, + "resources": [ + { + "type": "Microsoft.Logic/workflows", + "apiVersion": "2019-05-01", + "name": "[parameters('appName')]", + "identity": { + "type": "SystemAssigned" + }, + "properties": { + "state": "Enabled", + "definition": { + "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "$connections": { + "defaultValue": {}, + "type": "Object" + } + }, + "triggers": { + "Recurrence": { + "recurrence": { + "frequency": "[parameters('recurrenceFrequency')]", + "interval": "[parameters('recurrenceInterval')]" + }, + "evaluatedRecurrence": { + "frequency": "[parameters('recurrenceFrequency')]", + "interval": "[parameters('recurrenceInterval')]" + }, + "type": "[parameters('recurrenceType')]" + } + }, + "actions": { + "For_each_App_GW": { + "foreach": "@body('Parse_idle_App_Gateways')?['data']", + "actions": { + "Set_App_Gateways_URI": { + "type": "SetVariable", + "inputs": { + "name": "AppGwURI", + "value": "@{concat('https://portal.azure.com/#@',items('For_each_App_GW')?['tenantId'],'/resource',items('For_each_App_GW')?['id'])}" + } + }, + "Compose_AppGw": { + "runAfter": { + "Set_App_Gateways_URI": [ + "Succeeded" + ] + }, + "type": "Compose", + "inputs": "@variables('AppGwURI')" + }, + "Append_to_App_Gateway_HTML": { + "runAfter": { + "Compose_AppGw": [ + "Succeeded" + ] + }, + "type": "AppendToStringVariable", + "inputs": { + "name": "AppGatewayHTML", + "value": " \n \n \n \n " + } + } + }, + "runAfter": { + "Condition_App_Gateway": [ + "Succeeded" + ] + }, + "type": "Foreach", + "runtimeConfiguration": { + "concurrency": { + "repetitions": 1 + } + } + }, + "For_each_Disk": { + "foreach": "@body('Parse_Idle_disks')?['data']", + "actions": { + "Set_Disk_URI": { + "type": "SetVariable", + "inputs": { + "name": "DiskURI", + "value": "@{concat('https://portal.azure.com/#@',items('For_each_Disk')?['tenantId'],'/resource',items('For_each_Disk')?['DiskId'])}" + } + }, + "Compose_Disk": { + "runAfter": { + "Set_Disk_URI": [ + "Succeeded" + ] + }, + "type": "Compose", + "inputs": "@variables('DiskURI')" + }, + "Append_to_Idle_Disk_HTML": { + "runAfter": { + "Compose_Disk": [ + "Succeeded" + ] + }, + "type": "AppendToStringVariable", + "inputs": { + "name": "IdleDiskHTML", + "value": " \n \n \n \n " + } + } + }, + "runAfter": { + "Condition_Disk": [ + "Succeeded" + ] + }, + "type": "Foreach", + "runtimeConfiguration": { + "concurrency": { + "repetitions": 1 + } + } + }, + "For_each_IP_address": { + "foreach": "@body('Parse_Idle_IP_addresses')?['data']", + "actions": { + "Set_IP_address_URI": { + "type": "SetVariable", + "inputs": { + "name": "IPAddressURI", + "value": "@{concat('https://portal.azure.com/#@',items('For_each_IP_address')?['tenantId'],'/resource',items('For_each_IP_address')?['PublicIpId'])}" + } + }, + "Compose_IP": { + "runAfter": { + "Set_IP_address_URI": [ + "Succeeded" + ] + }, + "type": "Compose", + "inputs": "@variables('IPAddressURI')" + }, + "Append_to_IP_Address_HTML": { + "runAfter": { + "Compose_IP": [ + "Succeeded" + ] + }, + "type": "AppendToStringVariable", + "inputs": { + "name": "IPAddressHTML", + "value": " \n \n \n \n " + } + } + }, + "runAfter": { + "Condition_IP_Address": [ + "Succeeded" + ] + }, + "type": "Foreach", + "runtimeConfiguration": { + "concurrency": { + "repetitions": 1 + } + } + }, + "For_each_Load_Balancer": { + "foreach": "@body('Parse_Idle_Load_Balancers')?['data']", + "actions": { + "Set_Load_Balancer_URI": { + "type": "SetVariable", + "inputs": { + "name": "LoadBalancerURI", + "value": "@{concat('https://portal.azure.com/#@',items('For_each_Load_Balancer')?['tenantId'],'/resource',items('For_each_Load_Balancer')?['loadBalancerid'])}" + } + }, + "Compose_LB": { + "runAfter": { + "Set_Load_Balancer_URI": [ + "Succeeded" + ] + }, + "type": "Compose", + "inputs": "@variables('LoadBalancerURI')" + }, + "Append_to_Load_Balancer_HTML": { + "runAfter": { + "Compose_LB": [ + "Succeeded" + ] + }, + "type": "AppendToStringVariable", + "inputs": { + "name": "LoadBalancerHTML", + "value": " \n \n \n \n " + } + } + }, + "runAfter": { + "Condition_Load_Balancer": [ + "Succeeded" + ] + }, + "type": "Foreach", + "runtimeConfiguration": { + "concurrency": { + "repetitions": 1 + } + } + }, + "For_each_Snapshot": { + "foreach": "@body('Parse_Snapshots')?['data']", + "actions": { + "Set_Snapshot_URI": { + "type": "SetVariable", + "inputs": { + "name": "SnapshotURI", + "value": "@{concat('https://portal.azure.com/#@',items('For_each_Snapshot')?['tenantId'],'/resource',items('For_each_Snapshot')?['SnapshotId'])}\n" + } + }, + "Compose_Snapshot": { + "runAfter": { + "Set_Snapshot_URI": [ + "Succeeded" + ] + }, + "type": "Compose", + "inputs": "@variables('SnapshotURI')" + }, + "Append_to_Disk_Snapshot_HTML": { + "runAfter": { + "Compose_Snapshot": [ + "Succeeded" + ] + }, + "type": "AppendToStringVariable", + "inputs": { + "name": "DiskSnapshotHTML", + "value": " \n \n \n \n " + } + } + }, + "runAfter": { + "Condition_Disk_Snapshots": [ + "Succeeded" + ] + }, + "type": "Foreach", + "runtimeConfiguration": { + "concurrency": { + "repetitions": 1 + } + } + }, + "For_each_Stopped_VM": { + "foreach": "@body('Parse_stopped_VMs')?['data']", + "actions": { + "Set_Stopped_VM_URI": { + "type": "SetVariable", + "inputs": { + "name": "StoppedVMURI", + "value": "@{concat('https://portal.azure.com/#@',items('For_each_Stopped_VM')?['tenantId'],'/resource',items('For_each_Stopped_VM')?['VirtualMachineId'])}" + } + }, + "Compose_VM": { + "runAfter": { + "Set_Stopped_VM_URI": [ + "Succeeded" + ] + }, + "type": "Compose", + "inputs": "@variables('StoppedVMURI')" + }, + "Append_to_Stopped_VM_HTML": { + "runAfter": { + "Compose_VM": [ + "Succeeded" + ] + }, + "type": "AppendToStringVariable", + "inputs": { + "name": "StoppedVMHTML", + "value": " \n \n \n \n " + } + } + }, + "runAfter": { + "Condition_Stopped_VMs": [ + "Succeeded" + ] + }, + "type": "Foreach", + "runtimeConfiguration": { + "concurrency": { + "repetitions": 1 + } + } + }, + "Get_idle_App_Gateways": { + "runAfter": { + "Initialize_resources_table": [ + "Succeeded" + ] + }, + "type": "Http", + "inputs": { + "uri": "[format('{0}//providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01', environment().resourceManager)]", + "method": "POST", + "body": { + "query": "@{variables('resourcesTable')} | where type =~ 'Microsoft.Network/applicationGateways'| extend backendPoolsCount = array_length(properties.backendAddressPools),SKUName= tostring(properties.sku.name), SKUTier=tostring(properties.sku.tier),SKUCapacity=properties.sku.capacity,backendPools=properties.backendAddressPools| join (resources | where type =~ 'Microsoft.Network/applicationGateways'| mvexpand backendPools = properties.backendAddressPools| extend backendIPCount =array_length(backendPools.properties.backendIPConfigurations) | extend backendAddressesCount = array_length(backendPools.properties.backendAddresses) | extend backendPoolName=backendPools.properties.backendAddressPools.name | summarize backendIPCount = sum(backendIPCount) ,backendAddressesCount=sum(backendAddressesCount) by id) on id| project-away id1| where (backendIPCount == 0 or isempty(backendIPCount)) and (backendAddressesCount==0 or isempty(backendAddressesCount))| order by id asc | join kind=leftouter ( resourcecontainers | where type == 'microsoft.resources/subscriptions' | project subscriptionId, subscriptionName = name) on subscriptionId", + "scope": "Tenant" + }, + "authentication": { + "type": "ManagedServiceIdentity" + } + } + }, + "Get_idle_Disks": { + "runAfter": { + "Initialize_resources_table": [ + "Succeeded" + ] + }, + "type": "Http", + "inputs": { + "uri": "[format('{0}//providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01', environment().resourceManager)]", + "method": "POST", + "body": { + "query": "@{variables('resourcesTable')} | where type =~ 'microsoft.compute/disks' and managedBy == ''| extend diskState = tostring(properties.diskState) | where (managedBy == '' and diskState != 'ActiveSAS') or (diskState == 'Unattached' and diskState != 'ActiveSAS') | extend DiskId = id, DiskName = name, SKUName = sku.name, SKUTier = sku.tier, DiskSizeGB = tostring(properties.diskSizeGB), Location = location, TimeCreated = tostring(properties.timeCreated) | order by DiskId asc | project DiskId, DiskName, DiskSizeGB, SKUName, SKUTier, resourceGroup, Location, TimeCreated, subscriptionId | join kind=leftouter (resourcecontainers | where type == 'microsoft.resources/subscriptions' | project subscriptionId, subscriptionName = name) on subscriptionId", + "scope": "Tenant" + }, + "authentication": { + "type": "ManagedServiceIdentity" + } + } + }, + "Get_idle_IP_addresses": { + "runAfter": { + "Initialize_resources_table": [ + "Succeeded" + ] + }, + "type": "Http", + "inputs": { + "uri": "[format('{0}//providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01', environment().resourceManager)]", + "method": "POST", + "body": { + "query": "@{variables('resourcesTable')} | where type =~ 'Microsoft.Network/publicIPAddresses' and isempty(properties.ipConfiguration) and isempty(properties.natGateway) | extend PublicIpId=id, IPName=name, AllocationMethod=tostring(properties.publicIPAllocationMethod), SKUName=sku.name, Location=location | project PublicIpId,IPName, SKUName, resourceGroup, Location, AllocationMethod, subscriptionId, tenantId | union ( @{variables('resourcesTable')} | where type =~ 'microsoft.network/networkinterfaces' and isempty(properties.virtualMachine) and isnull(properties.privateEndpoint) and isnotempty(properties.ipConfigurations) | extend IPconfig = properties.ipConfigurations | mv-expand IPconfig | extend PublicIpId= tostring(IPconfig.properties.publicIPAddress.id) | project PublicIpId | join ( @{variables('resourcesTable')} | where type =~ 'Microsoft.Network/publicIPAddresses' | extend PublicIpId=id, IPName=name, AllocationMethod=tostring(properties.publicIPAllocationMethod), SKUName=sku.name, resourceGroup, Location=location ) on PublicIpId | project PublicIpId,IPName, SKUName, resourceGroup, Location, AllocationMethod, subscriptionId, tenantId) | join kind=leftouter ( resourcecontainers | where type == 'microsoft.resources/subscriptions' | project subscriptionId, subscriptionName = name) on subscriptionId", + "scope": "Tenant" + }, + "authentication": { + "type": "ManagedServiceIdentity" + } + } + }, + "Get_idle_Load_Balancers": { + "runAfter": { + "Initialize_resources_table": [ + "Succeeded" + ] + }, + "type": "Http", + "inputs": { + "uri": "[format('{0}//providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01', environment().resourceManager)]", + "method": "POST", + "body": { + "query": "@{variables('resourcesTable')} | where type =~ 'microsoft.network/loadbalancers' and tostring(properties.backendAddressPools) == '[]' | extend loadBalancerId=id,LBRG=resourceGroup, LoadBalancerName=name, SKU=sku, LBLocation=location | order by loadBalancerId asc | project loadBalancerId,LoadBalancerName, SKU.name,SKU.tier, LBLocation, resourceGroup, subscriptionId | join kind=leftouter ( resourcecontainers | where type == 'microsoft.resources/subscriptions' | project subscriptionId, subscriptionName = name) on subscriptionId", + "scope": "Tenant" + }, + "authentication": { + "type": "ManagedServiceIdentity" + } + } + }, + "Get_Disk_Snapshots_older_than_30_days": { + "runAfter": { + "Initialize_resources_table": [ + "Succeeded" + ] + }, + "type": "Http", + "inputs": { + "uri": "[format('{0}//providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01', environment().resourceManager)]", + "method": "POST", + "body": { + "query": "@{variables('resourcesTable')} | where type =~ 'microsoft.compute/snapshots' | extend TimeCreated = properties.timeCreated | where TimeCreated < ago(30d) | extend SnapshotId=id, Snapshotname=name | order by id asc | project id, SnapshotId, Snapshotname, resourceGroup, location, TimeCreated ,subscriptionId | join kind=leftouter ( resourcecontainers | where type == 'microsoft.resources/subscriptions' | project subscriptionId, subscriptionName = name) on subscriptionId", + "scope": "Tenant" + }, + "authentication": { + "type": "ManagedServiceIdentity" + } + } + }, + "Get_Stopped_VMs": { + "runAfter": { + "Initialize_resources_table": [ + "Succeeded" + ] + }, + "type": "Http", + "inputs": { + "uri": "[format('{0}//providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01', environment().resourceManager)]", + "method": "POST", + "body": { + "query": "@{variables('resourcesTable')} | where type =~ 'microsoft.compute/virtualmachines' and tostring(properties.extended.instanceView.powerState.displayStatus) != 'VM deallocated' and tostring(properties.extended.instanceView.powerState.displayStatus) != 'VM running'| extend VMname=name, PowerState=tostring(properties.extended.instanceView.powerState.displayStatus), VMLocation=location, VirtualMachineId=id| order by VirtualMachineId asc| project VirtualMachineId,VMname, PowerState, VMLocation, resourceGroup, subscriptionId | join kind=leftouter ( resourcecontainers | where type == 'microsoft.resources/subscriptions' | project subscriptionId, subscriptionName = name) on subscriptionId", + "scope": "Tenant" + }, + "authentication": { + "type": "ManagedServiceIdentity" + } + } + }, + "Initialize_App_Gateways_URI": { + "runAfter": { + "Parse_idle_App_Gateways": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "AppGwURI", + "type": "string" + } + ] + } + }, + "Initialize_Disk_URI": { + "runAfter": { + "Parse_Idle_disks": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "DiskURI", + "type": "string" + } + ] + } + }, + "Initialize_IP_addresses_URI": { + "runAfter": { + "Parse_Idle_IP_addresses": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "IPAddressURI", + "type": "string" + } + ] + } + }, + "Initialize_Load_Balancer_URI": { + "runAfter": { + "Parse_Idle_Load_Balancers": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "LoadBalancerURI", + "type": "string" + } + ] + } + }, + "Initialize_Snapshot_URI": { + "runAfter": { + "Parse_Snapshots": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "SnapshotURI", + "type": "string" + } + ] + } + }, + "Initialize_Stopped_VM_URI": { + "runAfter": { + "Parse_stopped_VMs": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "StoppedVMURI", + "type": "string" + } + ] + } + }, + "Excluded_subscriptions": { + "runAfter": { + "Included_subscriptions": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "ExcludedSubscriptions", + "type": "array", + "value": [] + } + ] + } + }, + "Parse_idle_App_Gateways": { + "runAfter": { + "Get_idle_App_Gateways": [ + "Succeeded" + ] + }, + "type": "ParseJson", + "inputs": { + "content": "@body('Get_idle_App_Gateways')", + "schema": { + "properties": { + "properties": { + "properties": { + "count": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "data": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "SKUCapacity": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "SKUName": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "SKUTier": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "backendAddressesCount": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "backendIPCount": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "backendPools": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "etag": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "properties": { + "properties": { + "properties": { + "properties": { + "backendAddresses": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "provisioningState": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "requestRoutingRules": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "backendPoolsCount": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "extendedLocation": { + "properties": {}, + "type": "object" + }, + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "identity": { + "properties": {}, + "type": "object" + }, + "kind": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "location": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "managedBy": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "plan": { + "properties": {}, + "type": "object" + }, + "properties": { + "properties": { + "properties": { + "properties": { + "authenticationCertificates": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "backendAddressPools": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "etag": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "properties": { + "properties": { + "properties": { + "properties": { + "backendAddresses": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "provisioningState": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "requestRoutingRules": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "backendHttpSettingsCollection": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "etag": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "properties": { + "properties": { + "properties": { + "properties": { + "cookieBasedAffinity": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "pickHostNameFromBackendAddress": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "port": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "protocol": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "provisioningState": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "requestRoutingRules": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "requestTimeout": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "enableHttp2": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "frontendIPConfigurations": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "etag": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "properties": { + "properties": { + "properties": { + "properties": { + "httpListeners": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "privateIPAllocationMethod": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "provisioningState": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "subnet": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "frontendPorts": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "etag": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "properties": { + "properties": { + "properties": { + "properties": { + "httpListeners": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "port": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "provisioningState": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "gatewayIPConfigurations": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "etag": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "properties": { + "properties": { + "properties": { + "properties": { + "provisioningState": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "subnet": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "httpListeners": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "etag": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "properties": { + "properties": { + "properties": { + "properties": { + "frontendIPConfiguration": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "frontendPort": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "protocol": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "provisioningState": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "requestRoutingRules": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "requireServerNameIndication": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "operationalState": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "probes": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "provisioningState": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "redirectConfigurations": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "requestRoutingRules": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "etag": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "properties": { + "properties": { + "properties": { + "properties": { + "backendAddressPool": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "backendHttpSettings": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "httpListener": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "provisioningState": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "ruleType": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "resourceGuid": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "rewriteRuleSets": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "sku": { + "properties": { + "properties": { + "properties": { + "capacity": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "tier": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "sslCertificates": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "urlPathMaps": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "resourceGroup": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "sku": { + "properties": {}, + "type": "object" + }, + "subscriptionId": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "tags": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "tenantId": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "zones": { + "properties": {}, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "facets": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "resultTruncated": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "totalRecords": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + } + }, + "Parse_Idle_IP_addresses": { + "runAfter": { + "Get_idle_IP_addresses": [ + "Succeeded" + ] + }, + "type": "ParseJson", + "inputs": { + "content": "@body('Get_idle_IP_addresses')", + "schema": { + "properties": { + "properties": { + "properties": { + "count": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "data": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "extendedLocation": { + "properties": {}, + "type": "object" + }, + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "identity": { + "properties": {}, + "type": "object" + }, + "kind": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "location": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "managedBy": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "plan": { + "properties": {}, + "type": "object" + }, + "properties": { + "properties": { + "properties": { + "properties": { + "idleTimeoutInMinutes": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "ipTags": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "provisioningState": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "publicIPAddressVersion": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "publicIPAllocationMethod": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "resourceGuid": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "resourceGroup": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "sku": { + "properties": { + "properties": { + "properties": { + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "tier": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "subscriptionId": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "tags": { + "properties": {}, + "type": "object" + }, + "tenantId": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "zones": { + "properties": {}, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "facets": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "resultTruncated": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "totalRecords": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + } + }, + "Parse_Idle_Load_Balancers": { + "runAfter": { + "Get_idle_Load_Balancers": [ + "Succeeded" + ] + }, + "type": "ParseJson", + "inputs": { + "content": "@body('Get_idle_Load_Balancers')", + "schema": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "childErrors": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "errorType": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "lineNumber": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "linePosition": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "message": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "path": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "schemaId": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "value": { + "properties": { + "items": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + } + }, + "Parse_Idle_disks": { + "runAfter": { + "Get_idle_Disks": [ + "Succeeded" + ] + }, + "type": "ParseJson", + "inputs": { + "content": "@body('Get_idle_Disks')", + "schema": { + "properties": { + "properties": { + "properties": { + "count": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "data": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "extendedLocation": { + "properties": {}, + "type": "object" + }, + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "identity": { + "properties": {}, + "type": "object" + }, + "kind": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "location": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "managedBy": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "plan": { + "properties": {}, + "type": "object" + }, + "properties": { + "properties": { + "properties": { + "properties": { + "creationData": { + "properties": { + "properties": { + "properties": { + "createOption": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "imageReference": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "diskIOPSReadWrite": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "diskMBpsReadWrite": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "diskSizeBytes": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "diskSizeGB": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "diskState": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "encryption": { + "properties": { + "properties": { + "properties": { + "type": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "hyperVGeneration": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "networkAccessPolicy": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "osType": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "provisioningState": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "publicNetworkAccess": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "supportedCapabilities": { + "properties": { + "properties": { + "properties": { + "architecture": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "timeCreated": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "uniqueId": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "resourceGroup": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "sku": { + "properties": { + "properties": { + "properties": { + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "tier": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "subscriptionId": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "tags": { + "properties": {}, + "type": "object" + }, + "tenantId": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "zones": { + "properties": {}, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "facets": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "resultTruncated": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "totalRecords": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + } + }, + "Parse_Snapshots": { + "runAfter": { + "Get_Disk_Snapshots_older_than_30_days": [ + "Succeeded" + ] + }, + "type": "ParseJson", + "inputs": { + "content": "@body('Get_Disk_Snapshots_older_than_30_days')", + "schema": { + "properties": { + "query": { + "type": "string" + }, + "subscriptions": { + "items": { + "type": "string" + }, + "type": "array" + } + }, + "type": "object" + } + } + }, + "Parse_stopped_VMs": { + "runAfter": { + "Get_Stopped_VMs": [ + "Succeeded" + ] + }, + "type": "ParseJson", + "inputs": { + "content": "@body('Get_Stopped_VMs')", + "schema": { + "properties": { + "properties": { + "properties": { + "count": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "data": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "extendedLocation": { + "properties": {}, + "type": "object" + }, + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "identity": { + "properties": {}, + "type": "object" + }, + "kind": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "location": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "managedBy": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "plan": { + "properties": {}, + "type": "object" + }, + "properties": { + "properties": { + "properties": { + "properties": { + "diagnosticsProfile": { + "properties": { + "properties": { + "properties": { + "bootDiagnostics": { + "properties": { + "properties": { + "properties": { + "enabled": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "extended": { + "properties": { + "properties": { + "properties": { + "instanceView": { + "properties": { + "properties": { + "properties": { + "hyperVGeneration": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "powerState": { + "properties": { + "properties": { + "properties": { + "code": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "displayStatus": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "level": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "hardwareProfile": { + "properties": { + "properties": { + "properties": { + "vmSize": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "networkProfile": { + "properties": { + "properties": { + "properties": { + "networkInterfaces": { + "properties": { + "items": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "osProfile": { + "properties": { + "properties": { + "properties": { + "adminUsername": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "allowExtensionOperations": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "computerName": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "requireGuestProvisionSignal": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "secrets": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "windowsConfiguration": { + "properties": { + "properties": { + "properties": { + "enableAutomaticUpdates": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "patchSettings": { + "properties": { + "properties": { + "properties": { + "assessmentMode": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "enableHotpatching": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "patchMode": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "provisionVMAgent": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "provisioningState": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "storageProfile": { + "properties": { + "properties": { + "properties": { + "dataDisks": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "imageReference": { + "properties": { + "properties": { + "properties": { + "exactVersion": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "offer": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "publisher": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "sku": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "version": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "osDisk": { + "properties": { + "properties": { + "properties": { + "caching": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "createOption": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "deleteOption": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "diskSizeGB": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "managedDisk": { + "properties": { + "properties": { + "properties": { + "id": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "storageAccountType": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "name": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "osType": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "vmId": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "resourceGroup": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "sku": { + "properties": {}, + "type": "object" + }, + "subscriptionId": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "tags": { + "properties": {}, + "type": "object" + }, + "tenantId": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "zones": { + "properties": {}, + "type": "object" + } + }, + "type": "object" + }, + "required": { + "items": { + "type": "string" + }, + "type": "array" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + }, + "facets": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "resultTruncated": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + }, + "totalRecords": { + "properties": { + "type": { + "type": "string" + } + }, + "type": "object" + } + }, + "type": "object" + }, + "type": { + "type": "string" + } + }, + "type": "object" + } + } + }, + "Set_alert_recipient": { + "runAfter": { + "Set_email_subject": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "SendAlertTo", + "type": "string" + } + ] + } + }, + "Initialize_App_Gateway_HTML": { + "runAfter": { + "Initialize_App_Gateways_URI": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "AppGatewayHTML", + "type": "string", + "value": "

Idle Application Gateway Details

" + } + ] + } + }, + "Initialize_Disk_HTML": { + "runAfter": { + "Initialize_Disk_URI": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "IdleDiskHTML", + "type": "string", + "value": "

Idle Disk Details

" + } + ] + } + }, + "Send_an_email_(V2)": { + "runAfter": { + "EmailNotice": [ + "Succeeded" + ] + }, + "type": "ApiConnection", + "inputs": { + "host": { + "connection": { + "name": "@parameters('$connections')['office365']['connectionId']" + } + }, + "method": "post", + "body": { + "To": "@variables('SendAlertTo')", + "Subject": "@variables('SetEmailSubject')", + "Body": "

@{variables('EmailNotice')}

@{variables('AppGatewayHTML')}
@{variables('IdleDiskHTML')}
@{variables('IPAddressHTML')}
@{variables('LoadBalancerHTML')}
@{variables('DiskSnapshotHTML')}
@{variables('StoppedVMHTML')}


📧 About FinOps alerts

FinOps alerts keep you informed about cost optimization opportunities within in your cloud environment. They are fully configurable and can be tailored to run on your desired schedule, ensuring that you receive timely notifications on the scenarios most important to your organization. FinOps alerts are part of the FinOps toolkit, an open-source collection of FinOps solutions that help you manage and optimize your cost, usage, and carbon.

Provide feedback

Give feedback
Vote on or suggest ideas


" + }, + "path": "/v2/Mail" + } + }, + "Initialize_IP_Address_HTML": { + "runAfter": { + "Initialize_IP_addresses_URI": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "IPAddressHTML", + "type": "string", + "value": "

Idle IP Address Details

" + } + ] + } + }, + "Initialize_Load_Balancer_HTML": { + "runAfter": { + "Initialize_Load_Balancer_URI": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "LoadBalancerHTML", + "type": "string", + "value": "

Idle Load Balancer Details

" + } + ] + } + }, + "Initialize_Disk_Snapshot_HTML": { + "runAfter": { + "Initialize_Snapshot_URI": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "DiskSnapshotHTML", + "type": "string", + "value": "

Old Disk Snapshot Details

" + } + ] + } + }, + "Initialize_Stopped_VM_HTML": { + "runAfter": { + "Initialize_Stopped_VM_URI": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "StoppedVMHTML", + "type": "string", + "value": "

Stopped VM Details

" + } + ] + } + }, + "End_to_App_Gateway_HTML": { + "runAfter": { + "For_each_App_GW": [ + "Succeeded" + ] + }, + "type": "AppendToStringVariable", + "inputs": { + "name": "AppGatewayHTML", + "value": "
Nelson Pereira
Nelson Pereira
🌟		Brett Wilson
Brett Wilson
🌟 💻 📖 👀		 Michael Flanakin
Michael Flanakin
🌟 💻 📖 👀 🧑‍🏫 📣		Arthur Clares
Arthur Clares
🌟 💻 📖 👀 🧑‍🏫 📣		Roland Krummenacher
Roland Krummenacher
🌟 💻 👀 🐛		Dirk Brinkmann
Dirk Brinkmann
🌟		 Sonia Cuff
Sonia Cuff
🌟 📣		Tanuja Shah
Tanuja Shah
🌟		Fernando Vasconcellos
Fernando Vasconcellos
🌟 📖 📣		Hélder Pinto
Hélder Pinto
💻 📖 👀 🐛
Dirk Brinkmann
Dirk Brinkmann
🌟		Brett Wilson
Brett Wilson
💻 📖 👀		Hélder Pinto
Hélder Pinto
💻 📖 👀 🐛		Arthur Clares
Arthur Clares
🌟 💻 📖 👀 🧑‍🏫 📣		 Nicolas Teyan
Nicolas Teyan
💻 📖 👀		Bill Anderson
Bill Anderson
📖		 Robel
Robel
💻 📖 👀
Daniel Ueffing
Daniel Ueffing
🤔 💻 📖 🐛		 Sacha Narinx
Sacha Narinx
💻 📖 👀		 Anthony Romano
Anthony Romano
💻 📖 👀
Bill Anderson
Bill Anderson
📖		 grantxyzou
grantxyzou
🎨		 lmoscinski
lmoscinski
🎨		Roland Krummenacher
Roland Krummenacher
💻 👀 🐛		 Divyadeep Dayal
Divyadeep Dayal
💻		jamelachahbar
jamelachahbar
💻 👀		Chris Bowman
Chris Bowman
🐛 👀 💻
jamelachahbar
jamelachahbar
💻 👀		 Orthodoxos Kipouridis
Orthodoxos Kipouridis
💻 📖		 Ben Shy
Ben Shy
💻 👀		 Kevin De La Rosa
Kevin De La Rosa
📖		 bwatts64
bwatts64
💻 👀		 Dany Hoter
Dany Hoter
💻 👀		Joseph John
Joseph John
📖 👀
Joseph John
Joseph John
📖 👀		 ripadrao
ripadrao
📖		 Pedro Sousa
Pedro Sousa
📖		 Sourav Bera
Sourav Bera
📖		 J.R. Phillips
J.R. Phillips
💻		 Saad Mahmood
Saad Mahmood
💻		simonarbel
simonarbel
🐛
simonarbel
simonarbel
🐛		Daniel Ueffing
Daniel Ueffing
🤔 💻 📖 🐛		 Seif Bassem
Seif Bassem
💻		 Arjen Huitema
Arjen Huitema
💻		 Yuan Zhang
Yuan Zhang
💻 👀		 ymehdimsft
ymehdimsft
💻
srilatha inavolu
srilatha inavolu
💻 👀		 soumyananda
soumyananda
💻 👀		Chris Bowman
Chris Bowman
🐛 👀 💻
Nelson Pereira
Nelson Pereira
📣		Tanuja Shah
Tanuja Shah
📣		Fernando Vasconcellos
Fernando Vasconcellos
📖 📣		 Trey Morgan
Trey Morgan
💻		 Travis Silvers
Travis Silvers
👀		 Travis Silvers
Travis Silvers
👀
@{items('For_each_App_GW')?['name']}@{items('For_each_App_GW')?['resourceGroup']}@{items('For_each_App_GW')?['subscriptionName']}
@{items('For_each_Disk')?['DiskName']}@{items('For_each_Disk')?['resourceGroup']}@{items('For_each_Disk')?['subscriptionName']}
@{items('For_each_IP_address')?['IPName']}@{items('For_each_IP_address')?['resourceGroup']}@{items('For_each_IP_address')?['subscriptionName']}
@{items('For_each_Load_Balancer')?['LoadBalancerName']}@{items('For_each_Load_Balancer')?['resourceGroup']}@{items('For_each_Load_Balancer')?['subscriptionName']}
@{items('For_each_Snapshot')?['Snapshotname']}@{items('For_each_Snapshot')?['resourceGroup']}@{items('For_each_Snapshot')?['subscriptionName']}
@{items('For_each_Stopped_VM')?['VMname']}@{items('For_each_Stopped_VM')?['resourceGroup']}@{items('For_each_Stopped_VM')?['subscriptionName']}
" + } + }, + "End_to_IP_Address_HTML": { + "runAfter": { + "For_each_IP_address": [ + "Succeeded" + ] + }, + "type": "AppendToStringVariable", + "inputs": { + "name": "IPAddressHTML", + "value": " " + } + }, + "End_to_Disk_HTML": { + "runAfter": { + "For_each_Disk": [ + "Succeeded" + ] + }, + "type": "AppendToStringVariable", + "inputs": { + "name": "IdleDiskHTML", + "value": "" + } + }, + "End_to_Load_Balancer_HTML": { + "runAfter": { + "For_each_Load_Balancer": [ + "Succeeded" + ] + }, + "type": "AppendToStringVariable", + "inputs": { + "name": "LoadBalancerHTML", + "value": "" + } + }, + "End_to_Disk_Snapshot_HTML": { + "runAfter": { + "For_each_Snapshot": [ + "Succeeded" + ] + }, + "type": "AppendToStringVariable", + "inputs": { + "name": "DiskSnapshotHTML", + "value": "" + } + }, + "End_to_Stopped_VM_HTML": { + "runAfter": { + "For_each_Stopped_VM": [ + "Succeeded" + ] + }, + "type": "AppendToStringVariable", + "inputs": { + "name": "StoppedVMHTML", + "value": "" + } + }, + "Set_email_subject": { + "runAfter": {}, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "SetEmailSubject", + "type": "string" + } + ] + } + }, + "Included_subscriptions": { + "runAfter": { + "Set_alert_recipient": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "IncludedSubscriptions", + "type": "array", + "value": [] + } + ] + } + }, + "Initialize_resources_table": { + "runAfter": { + "Excluded_subscriptions": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "resourcesTable", + "type": "string", + "value": "resources@{if(equals(length(variables('IncludedSubscriptions')), 0), '', concat('| where subscriptionId in (', replace(replace(replace(string(variables('IncludedSubscriptions')), '\n', ''), '[', ''), ']', ''), ')'))}@{if(equals(length(variables('ExcludedSubscriptions')), 0), '', concat('| where subscriptionId !in (', replace(replace(replace(string(variables('ExcludedSubscriptions')), '\n', ''), '[', ''), ']', ''), ')'))}" + } + ] + } + }, + "Condition_App_Gateway": { + "actions": { + "Append_no_App_Gateway_results_text": { + "type": "AppendToStringVariable", + "inputs": { + "name": "AppGatewayHTML", + "value": "No resources are idle." + } + } + }, + "runAfter": { + "Initialize_App_Gateway_HTML": [ + "Succeeded" + ] + }, + "else": { + "actions": { + "Append_App_Gateway_results_in_table": { + "type": "AppendToStringVariable", + "inputs": { + "name": "AppGatewayHTML", + "value": "\n \n \n \n \n \n" + } + } + } + }, + "expression": { + "and": [ + { + "equals": [ + "@length(body('Get_idle_App_Gateways')['data'])", + 0 + ] + } + ] + }, + "type": "If" + }, + "Condition_Stopped_VMs": { + "actions": { + "Append_no_Stopped_VM_results_text_": { + "type": "AppendToStringVariable", + "inputs": { + "name": "StoppedVMHTML", + "value": "No resources are idle." + } + } + }, + "runAfter": { + "Initialize_Stopped_VM_HTML": [ + "Succeeded" + ] + }, + "else": { + "actions": { + "Append_Stopped_VM_result_in_table_": { + "type": "AppendToStringVariable", + "inputs": { + "name": "StoppedVMHTML", + "value": "
NameResource GroupSubscription
\n \n \n \n \n \n" + } + } + } + }, + "expression": { + "and": [ + { + "equals": [ + "@length(body('Get_Stopped_VMs')['data'])", + 0 + ] + } + ] + }, + "type": "If" + }, + "Condition_Disk": { + "actions": { + "Append_no_Disk_results_text": { + "type": "AppendToStringVariable", + "inputs": { + "name": "IdleDiskHTML", + "value": "No resources are idle." + } + } + }, + "runAfter": { + "Initialize_Disk_HTML": [ + "Succeeded" + ] + }, + "else": { + "actions": { + "Append_Disk_results_in_table": { + "type": "AppendToStringVariable", + "inputs": { + "name": "IdleDiskHTML", + "value": "
NameResource GroupSubscription
\n \n \n \n \n \n" + } + } + } + }, + "expression": { + "and": [ + { + "equals": [ + "@length(body('Get_idle_Disks')['data'])", + 0 + ] + } + ] + }, + "type": "If" + }, + "Condition_IP_Address": { + "actions": { + "Append_no_IP_Address_results_text": { + "type": "AppendToStringVariable", + "inputs": { + "name": "IPAddressHTML", + "value": "No resources are idle." + } + } + }, + "runAfter": { + "Initialize_IP_Address_HTML": [ + "Succeeded" + ] + }, + "else": { + "actions": { + "Append_IP_Address_results_in_table": { + "type": "AppendToStringVariable", + "inputs": { + "name": "IPAddressHTML", + "value": "
NameResource GroupSubscription
\n \n \n \n \n \n" + } + } + } + }, + "expression": { + "and": [ + { + "equals": [ + "@length(body('Get_idle_IP_addresses')['data'])", + 0 + ] + } + ] + }, + "type": "If" + }, + "Condition_Load_Balancer": { + "actions": { + "Append_no_Load_Balancer_results_text": { + "type": "AppendToStringVariable", + "inputs": { + "name": "LoadBalancerHTML", + "value": "No resources are idle." + } + } + }, + "runAfter": { + "Initialize_Load_Balancer_HTML": [ + "Succeeded" + ] + }, + "else": { + "actions": { + "Append_Load_Balancer_results_in_table": { + "type": "AppendToStringVariable", + "inputs": { + "name": "LoadBalancerHTML", + "value": "
NameResource GroupSubscription
\n \n \n \n \n \n" + } + } + } + }, + "expression": { + "and": [ + { + "equals": [ + "@length(body('Get_idle_Load_Balancers')['data'])", + 0 + ] + } + ] + }, + "type": "If" + }, + "Condition_Disk_Snapshots": { + "actions": { + "Append_no_Disk_Snapshot_results_text": { + "type": "AppendToStringVariable", + "inputs": { + "name": "DiskSnapshotHTML", + "value": "No resources are idle." + } + } + }, + "runAfter": { + "Initialize_Disk_Snapshot_HTML": [ + "Succeeded" + ] + }, + "else": { + "actions": { + "Append_Disk_Snapshot_results_in_table": { + "type": "AppendToStringVariable", + "inputs": { + "name": "DiskSnapshotHTML", + "value": "
NameResource GroupSubscription
\n \n \n \n \n \n" + } + } + } + }, + "expression": { + "and": [ + { + "equals": [ + "@length(body('Get_Disk_Snapshots_older_than_30_days')['data'])", + 0 + ] + } + ] + }, + "type": "If" + }, + "Condition_App_Gateway_next_steps": { + "actions": {}, + "runAfter": { + "End_to_App_Gateway_HTML": [ + "Succeeded" + ] + }, + "else": { + "actions": { + "Append_to_App_Gateway_table": { + "type": "AppendToStringVariable", + "inputs": { + "name": "AppGatewayHTML", + "value": "
\n 👉 Next steps: Review application gateways which include backend pools with no targets.\n
" + } + } + } + }, + "expression": { + "and": [ + { + "equals": [ + "@length(body('Get_idle_App_Gateways')['data'])", + 0 + ] + } + ] + }, + "type": "If" + }, + "Condition_Disk_next_steps": { + "actions": {}, + "runAfter": { + "End_to_Disk_HTML": [ + "Succeeded" + ] + }, + "else": { + "actions": { + "Append_to_Disk_table": { + "type": "AppendToStringVariable", + "inputs": { + "name": "IdleDiskHTML", + "value": "
\n 👉 Next steps: Review managed disks that are not attached to any virtual machine.\n
" + } + } + } + }, + "expression": { + "and": [ + { + "equals": [ + "@length(body('Get_idle_Disks')['data'])", + 0 + ] + } + ] + }, + "type": "If" + }, + "Condition_IP_Address_next_steps": { + "actions": {}, + "runAfter": { + "End_to_IP_Address_HTML": [ + "Succeeded" + ] + }, + "else": { + "actions": { + "Append_to_IP_Address_table": { + "type": "AppendToStringVariable", + "inputs": { + "name": "IPAddressHTML", + "value": "
\n 👉 Next steps: Review unattached public IP addresses, as they may represent additional cost.\n
" + } + } + } + }, + "expression": { + "and": [ + { + "equals": [ + "@length(body('Get_idle_IP_addresses')['data'])", + 0 + ] + } + ] + }, + "type": "If" + }, + "Condition_Load_Balancer_next_steps": { + "actions": {}, + "runAfter": { + "End_to_Load_Balancer_HTML": [ + "Succeeded" + ] + }, + "else": { + "actions": { + "Append_to_Load_Balancer_table": { + "type": "AppendToStringVariable", + "inputs": { + "name": "LoadBalancerHTML", + "value": "
\n 👉 Next steps: Review load balancers with no backend pools and remove them if not needed.\n
" + } + } + } + }, + "expression": { + "and": [ + { + "equals": [ + "@length(body('Get_idle_Load_Balancers')['data'])", + 0 + ] + } + ] + }, + "type": "If" + }, + "Condition_Disk_Snapshot_next_steps": { + "actions": {}, + "runAfter": { + "End_to_Disk_Snapshot_HTML": [ + "Succeeded" + ] + }, + "else": { + "actions": { + "Append_to_Disk_Snapshot_table": { + "type": "AppendToStringVariable", + "inputs": { + "name": "DiskSnapshotHTML", + "value": "
\n 👉 Next steps: Review managed disk snapshots that are older than 30 days.\n
" + } + } + } + }, + "expression": { + "and": [ + { + "equals": [ + "@length(body('Get_Disk_Snapshots_older_than_30_days')['data'])", + 0 + ] + } + ] + }, + "type": "If" + }, + "Condition_stopped_VM_next_steps": { + "actions": {}, + "runAfter": { + "End_to_Stopped_VM_HTML": [ + "Succeeded" + ] + }, + "else": { + "actions": { + "Append_to_Stopped_VM_table": { + "type": "AppendToStringVariable", + "inputs": { + "name": "StoppedVMHTML", + "value": "
\n 👉 Next steps: Review stopped VMs, as they are billed for the allocated cost.\n
" + } + } + } + }, + "expression": { + "and": [ + { + "equals": [ + "@length(body('Get_Stopped_VMs')['data'])", + 0 + ] + } + ] + }, + "type": "If" + }, + "EmailNotice": { + "runAfter": { + "Condition_App_Gateway_next_steps": [ + "Succeeded" + ], + "Condition_Disk_next_steps": [ + "Succeeded" + ], + "Condition_IP_Address_next_steps": [ + "Succeeded" + ], + "Condition_Load_Balancer_next_steps": [ + "Succeeded" + ], + "Condition_Disk_Snapshot_next_steps": [ + "Succeeded" + ], + "Condition_stopped_VM_next_steps": [ + "Succeeded" + ] + }, + "type": "InitializeVariable", + "inputs": { + "variables": [ + { + "name": "EmailNotice", + "type": "string", + "value": "

The following resources have been identified through FinOps alerts. Please take a moment to review and proceed with the next steps outlined below:


" + } + ] + } + } + }, + "outputs": {} + }, + "parameters": { + "$connections": { + "value": { + "office365": { + "connectionId": "[resourceId('Microsoft.Web/connections', variables('connectionName'))]", + "connectionName": "[variables('connectionName')]", + "id": "[subscriptionResourceId('Microsoft.Web/locations/managedApis', parameters('location'), 'office365')]" + } + } + } + } + }, + "location": "[parameters('location')]", + "tags": { + "displayName": "FinOpsalert" + }, + "dependsOn": [ + "[resourceId('Microsoft.Web/connections', variables('connectionName'))]" + ] + }, + { + "type": "Microsoft.Web/connections", + "apiVersion": "2016-06-01", + "name": "[variables('connectionName')]", + "location": "[parameters('location')]", + "properties": { + "api": { + "id": "[subscriptionResourceId('Microsoft.Web/locations/managedApis', parameters('location'), 'office365')]" + }, + "displayName": "[variables('displayName')]" + } + } + ], + "outputs": { + "logicAppName": { + "type": "string", + "value": "[parameters('appName')]" + }, + "connectionName": { + "type": "string", + "value": "[variables('connectionName')]" + }, + "actionsCount": { + "type": "int", + "value": "[length(variables('actionKeys'))]" + }, + "logicAppResourceId": { + "type": "string", + "value": "[resourceId('Microsoft.Logic/workflows', parameters('appName'))]" + }, + "logicAppPrincipalId": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Logic/workflows', parameters('appName')), '2019-05-01', 'full').identity.principalId]" + }, + "logicAppTriggerUrl": { + "type": "string", + "value": "[format('https://{0}.logic.azure.com:443/workflows/{1}/triggers/Recurrence/run?api-version=2016-10-01', parameters('appName'), parameters('appName'))]" + } + } + } + } + } + ] +} \ No newline at end of file diff --git a/docs/deploy/finops-alerts-13.0.ui.json b/docs/deploy/finops-alerts-13.0.ui.json new file mode 100644 index 000000000..7f519e102 --- /dev/null +++ b/docs/deploy/finops-alerts-13.0.ui.json @@ -0,0 +1,72 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/0.1.2-preview/CreateUIDefinition.MultiVm.json#", + "handler": "Microsoft.Azure.CreateUIDef", + "version": "0.1.2-preview", + "parameters": { + "config": { + "basics": { + "description": "FinOps alerts is an Azure Logic Apps-based automated detection system that identifies idle resources across selected subscriptions on a configurable schedule and sends notifications to admins to investigate and take action", + "location": { + "label": "Location", + "toolTip": "Location of logic app must be in the same region as the resource group.", + "resourceTypes": [ + "Microsoft.Logic/workflows", + "Microsoft.Web/connections" + ] + } + } + }, + "basics": [ + { + "name": "finopsalertsName", + "type": "Microsoft.Common.TextBox", + "label": "FinOps alerts name", + "defaultValue": "finops-alerts", + "toolTip": "Name of the FinOps alerts instance.", + "constraints": { + "required": true, + "regex": "^[a-zA-Z0-9][a-zA-Z0-9\\-]{0,61}[a-zA-Z0-9]$", + "validationMessage": "Name must be between 3 and 63 characters long and can contain only lowercase letters, numbers, and hyphens. The first and last characters in the name must be alphanumeric." + }, + "visible": true + }, + { + "name": "connectionsName", + "type": "Microsoft.Common.TextBox", + "label": "Connection name", + "defaultValue": "finops-alerts-connection", + "toolTip": "Name of the API connection.", + "constraints": { + "required": true, + "regex": "^[a-zA-Z0-9][a-zA-Z0-9\\-]{0,61}[a-zA-Z0-9]$", + "validationMessage": "Name must be between 3 and 63 characters long and can contain only lowercase letters, numbers, and hyphens. The first and last characters in the name must be alphanumeric." + }, + "visible": true + } + ], + "steps": [{ + "name": "tags", + "label": "Tags", + "elements": [ + { + "name": "tagsByResource", + "label": "Tags", + "toolTip": "Tags to apply to resources.", + "type": "Microsoft.Common.TagsByResource", + "resources": [ + "Microsoft.Logic/workflows", + "Microsoft.Web/connections" + ] + } + ] + }], + "outputs": { + "hubName": "[basics('finopsalertsName')]", + "location": "[location()]", + "tagsByResource": "[steps('tags').tagsByResource]"}, + "resourceTypes": [ + "Microsoft.Logic/workflows", + "Microsoft.Web/connections" + ] + } +} diff --git a/docs/deploy/finops-alerts-latest.json b/docs/deploy/finops-alerts-latest.json index 07541d1e9..88401106b 100644 --- a/docs/deploy/finops-alerts-latest.json +++ b/docs/deploy/finops-alerts-latest.json @@ -4,8 +4,8 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "2824696842595158492" + "version": "0.40.2.10011", + "templateHash": "12460451954863791061" } }, "parameters": { @@ -69,7 +69,7 @@ } }, "variables": { - "$fxv#0": "12.0" + "$fxv#0": "13.0" }, "resources": [ { @@ -94,7 +94,7 @@ }, { "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", + "apiVersion": "2025-04-01", "name": "[format('logicApp-{0}', uniqueString(deployment().name, parameters('location'), parameters('appName')))]", "subscriptionId": "[parameters('logicAppSubscriptionId')]", "resourceGroup": "[parameters('resourceGroupName')]", @@ -126,8 +126,8 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "16134864669369766052" + "version": "0.40.2.10011", + "templateHash": "10958230289205929" } }, "parameters": { diff --git a/docs/deploy/finops-hub-13.0.json b/docs/deploy/finops-hub-13.0.json new file mode 100644 index 000000000..56ae568cd --- /dev/null +++ b/docs/deploy/finops-hub-13.0.json @@ -0,0 +1,24950 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "15508602991110868852" + } + }, + "parameters": { + "hubName": { + "type": "string", + "metadata": { + "description": "Optional. Name of the hub. Used to ensure unique resource names. Default: \"finops-hub\"." + } + }, + "location": { + "type": "string", + "defaultValue": "[resourceGroup().location]", + "metadata": { + "description": "Optional. Azure location where all resources should be created. See https://aka.ms/azureregions. Default: Same as deployment." + } + }, + "storageSku": { + "type": "string", + "defaultValue": "Premium_LRS", + "allowedValues": [ + "Premium_LRS", + "Premium_ZRS" + ], + "metadata": { + "description": "Optional. Storage SKU to use. LRS = Lowest cost, ZRS = High availability. Note Standard SKUs are not available for Data Lake gen2 storage. Allowed: Premium_LRS, Premium_ZRS. Default: Premium_LRS." + } + }, + "enableInfrastructureEncryption": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Enable infrastructure encryption on the storage account. Default = false." + } + }, + "enablePurgeProtection": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Enable purge protection for the Key Vault. Default: false." + } + }, + "remoteHubStorageUri": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Storage account to push data to for ingestion into a remote hub." + } + }, + "remoteHubStorageKey": { + "type": "securestring", + "defaultValue": "", + "metadata": { + "description": "Optional. Storage account key to use when pushing data to a remote hub." + } + }, + "enableManagedExports": { + "type": "bool", + "defaultValue": true, + "metadata": { + "description": "Optional. Enable managed exports where your FinOps hub instance will create and run Cost Management exports on your behalf. Not supported for Microsoft Customer Agreement (MCA) billing profiles. Requires the ability to grant User Access Administrator role to FinOps hubs, which is required to create Cost Management exports. Default: true." + } + }, + "dataExplorerName": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Name of the Azure Data Explorer cluster to use for advanced analytics. If empty, Azure Data Explorer will not be deployed. Required to use with Power BI if you have more than $2-5M/mo in costs being monitored. Default: \"\" (do not use)." + } + }, + "dataExplorerSku": { + "type": "string", + "defaultValue": "Dev(No SLA)_Standard_D11_v2", + "allowedValues": [ + "Dev(No SLA)_Standard_E2a_v4", + "Dev(No SLA)_Standard_D11_v2", + "Standard_D11_v2", + "Standard_D12_v2", + "Standard_D13_v2", + "Standard_D14_v2", + "Standard_D16d_v5", + "Standard_D32d_v4", + "Standard_D32d_v5", + "Standard_DS13_v2+1TB_PS", + "Standard_DS13_v2+2TB_PS", + "Standard_DS14_v2+3TB_PS", + "Standard_DS14_v2+4TB_PS", + "Standard_E2a_v4", + "Standard_E2ads_v5", + "Standard_E2d_v4", + "Standard_E2d_v5", + "Standard_E4a_v4", + "Standard_E4ads_v5", + "Standard_E4d_v4", + "Standard_E4d_v5", + "Standard_E8a_v4", + "Standard_E8ads_v5", + "Standard_E8as_v4+1TB_PS", + "Standard_E8as_v4+2TB_PS", + "Standard_E8as_v5+1TB_PS", + "Standard_E8as_v5+2TB_PS", + "Standard_E8d_v4", + "Standard_E8d_v5", + "Standard_E8s_v4+1TB_PS", + "Standard_E8s_v4+2TB_PS", + "Standard_E8s_v5+1TB_PS", + "Standard_E8s_v5+2TB_PS", + "Standard_E16a_v4", + "Standard_E16ads_v5", + "Standard_E16as_v4+3TB_PS", + "Standard_E16as_v4+4TB_PS", + "Standard_E16as_v5+3TB_PS", + "Standard_E16as_v5+4TB_PS", + "Standard_E16d_v4", + "Standard_E16d_v5", + "Standard_E16s_v4+3TB_PS", + "Standard_E16s_v4+4TB_PS", + "Standard_E16s_v5+3TB_PS", + "Standard_E16s_v5+4TB_PS", + "Standard_E64i_v3", + "Standard_E80ids_v4", + "Standard_EC8ads_v5", + "Standard_EC8as_v5+1TB_PS", + "Standard_EC8as_v5+2TB_PS", + "Standard_EC16ads_v5", + "Standard_EC16as_v5+3TB_PS", + "Standard_EC16as_v5+4TB_PS", + "Standard_L4s", + "Standard_L8as_v3", + "Standard_L8s", + "Standard_L8s_v2", + "Standard_L8s_v3", + "Standard_L16as_v3", + "Standard_L16s", + "Standard_L16s_v2", + "Standard_L16s_v3", + "Standard_L32as_v3", + "Standard_L32s_v3" + ], + "metadata": { + "description": "Optional. Name of the Azure Data Explorer SKU. Default: \"Dev(No SLA)_Standard_D11_v2\"." + } + }, + "dataExplorerCapacity": { + "type": "int", + "defaultValue": 1, + "minValue": 1, + "maxValue": 1000, + "metadata": { + "description": "Optional. Number of nodes to use in the cluster. Allowed values: 1 for the Basic SKU tier and 2-1000 for Standard. Default: 1 for dev/test SKUs, 2 for standard SKUs." + } + }, + "fabricQueryUri": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Microsoft Fabric eventhouse query URI. Default: \"\" (do not use)." + } + }, + "fabricCapacityUnits": { + "type": "int", + "defaultValue": 2, + "minValue": 1, + "maxValue": 2048, + "metadata": { + "description": "Optional. Number of capacity units for the Microsoft Fabric capacity. This is the number in your Fabric SKU (e.g., Trial = 1, F2 = 2, F64 = 64). This is used to manage parallelization in data pipelines. If you change capacity, please redeploy the template. Allowed values: 1 for the Fabric trial and 2-2048 based on the assigned Fabric capacity (e.g., F2-F2048). Default: 2." + } + }, + "tags": { + "type": "object", + "defaultValue": {}, + "metadata": { + "description": "Optional. Tags to apply to all resources. We will also add the cm-resource-parent tag for improved cost roll-ups in Cost Management." + } + }, + "tagsByResource": { + "type": "object", + "defaultValue": {}, + "metadata": { + "description": "Optional. Tags to apply to resources based on their resource type. Resource type specific tags will be merged with tags for all resources." + } + }, + "scopesToMonitor": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. List of scope IDs to monitor and ingest cost for." + } + }, + "exportRetentionInDays": { + "type": "int", + "defaultValue": 0, + "metadata": { + "description": "Optional. Number of days of data to retain in the msexports container. Default: 0." + } + }, + "ingestionRetentionInMonths": { + "type": "int", + "defaultValue": 13, + "metadata": { + "description": "Optional. Number of months of data to retain in the ingestion container. Default: 13." + } + }, + "dataExplorerRawRetentionInDays": { + "type": "int", + "defaultValue": 0, + "metadata": { + "description": "Optional. Number of days of data to retain in the Data Explorer *_raw tables. Default: 0." + } + }, + "dataExplorerFinalRetentionInMonths": { + "type": "int", + "defaultValue": 13, + "metadata": { + "description": "Optional. Number of months of data to retain in the Data Explorer *_final_v* tables. Default: 13." + } + }, + "enablePublicAccess": { + "type": "bool", + "defaultValue": true, + "metadata": { + "description": "Optional. Enable public access to FinOps hubs resources. Default: true." + } + }, + "virtualNetworkAddressPrefix": { + "type": "string", + "defaultValue": "10.20.30.0/26", + "metadata": { + "description": "Optional. Address space for the workload. Minimum /26 subnet size is required for the workload. Default: \"10.20.30.0/26\"." + } + } + }, + "resources": [ + { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "hub", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "hubName": { + "value": "[parameters('hubName')]" + }, + "location": { + "value": "[parameters('location')]" + }, + "storageSku": { + "value": "[parameters('storageSku')]" + }, + "enableInfrastructureEncryption": { + "value": "[parameters('enableInfrastructureEncryption')]" + }, + "enablePurgeProtection": { + "value": "[parameters('enablePurgeProtection')]" + }, + "enableManagedExports": { + "value": "[parameters('enableManagedExports')]" + }, + "dataExplorerName": { + "value": "[parameters('dataExplorerName')]" + }, + "dataExplorerSku": { + "value": "[parameters('dataExplorerSku')]" + }, + "dataExplorerCapacity": { + "value": "[parameters('dataExplorerCapacity')]" + }, + "fabricQueryUri": { + "value": "[parameters('fabricQueryUri')]" + }, + "fabricCapacityUnits": { + "value": "[parameters('fabricCapacityUnits')]" + }, + "tags": { + "value": "[parameters('tags')]" + }, + "tagsByResource": { + "value": "[parameters('tagsByResource')]" + }, + "scopesToMonitor": { + "value": "[parameters('scopesToMonitor')]" + }, + "exportRetentionInDays": { + "value": "[parameters('exportRetentionInDays')]" + }, + "ingestionRetentionInMonths": { + "value": "[parameters('ingestionRetentionInMonths')]" + }, + "dataExplorerRawRetentionInDays": { + "value": "[parameters('dataExplorerRawRetentionInDays')]" + }, + "dataExplorerFinalRetentionInMonths": { + "value": "[parameters('dataExplorerFinalRetentionInMonths')]" + }, + "remoteHubStorageUri": { + "value": "[parameters('remoteHubStorageUri')]" + }, + "remoteHubStorageKey": { + "value": "[parameters('remoteHubStorageKey')]" + }, + "enablePublicAccess": { + "value": "[parameters('enablePublicAccess')]" + }, + "virtualNetworkAddressPrefix": { + "value": "[parameters('virtualNetworkAddressPrefix')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "2881649287387479903" + } + }, + "definitions": { + "_1.HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "fx/hub-types.bicep" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "fx/hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "fx/hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "fx/hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "_1", + "members": { + "dnsZoneIdName": { + "parameters": [ + { + "type": "string", + "name": "type" + } + ], + "output": { + "$ref": "#/definitions/_1.IdNameObject", + "value": "[_1.idName(format('privatelink.{0}.{1}', parameters('type'), environment().suffixes.storage), 'Microsoft.Network/privateDnsZones')]" + }, + "metadata": { + "__bicep_imported_from!": { + "sourceTemplate": "fx/hub-types.bicep" + } + } + }, + "idName": { + "parameters": [ + { + "type": "string", + "name": "name" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "$ref": "#/definitions/_1.IdNameObject", + "value": { + "id": "[resourceId(parameters('resourceType'), parameters('name'))]", + "name": "[parameters('name')]" + } + }, + "metadata": { + "__bicep_imported_from!": { + "sourceTemplate": "fx/hub-types.bicep" + } + } + }, + "newAppInternal": { + "parameters": [ + { + "$ref": "#/definitions/_1.HubProperties", + "name": "hub" + }, + { + "type": "string", + "name": "id" + }, + { + "type": "string", + "name": "name" + }, + { + "type": "string", + "name": "publisher" + }, + { + "type": "string", + "name": "suffix" + } + ], + "output": { + "$ref": "#/definitions/_1.HubAppProperties", + "value": { + "id": "[parameters('id')]", + "name": "[parameters('name')]", + "publisher": "[parameters('publisher')]", + "suffix": "[parameters('suffix')]", + "tags": "[union(parameters('hub').tags, createObject('ftk-hubapp-publisher', parameters('publisher')))]", + "hub": "[parameters('hub')]", + "dataFactory": "[replace(format('{0}-{1}', take(format('{0}-engine', replace(parameters('hub').name, '_', '-')), sub(sub(63, length(parameters('suffix'))), 1)), parameters('suffix')), '--', '-')]", + "keyVault": "[replace(format('{0}-{1}', take(format('{0}-vault', replace(parameters('hub').name, '_', '-')), sub(sub(24, length(parameters('suffix'))), 1)), parameters('suffix')), '--', '-')]", + "storage": "[format('{0}{1}', take(_1.safeStorageName(parameters('hub').name), sub(24, length(parameters('suffix')))), parameters('suffix'))]" + } + }, + "metadata": { + "__bicep_imported_from!": { + "sourceTemplate": "fx/hub-types.bicep" + } + } + }, + "newHubInternal": { + "parameters": [ + { + "type": "string", + "name": "id" + }, + { + "type": "string", + "name": "name" + }, + { + "type": "string", + "name": "suffix" + }, + { + "type": "string", + "name": "location" + }, + { + "type": "object", + "name": "tags" + }, + { + "type": "object", + "name": "tagsByResource" + }, + { + "type": "string", + "name": "storageSku" + }, + { + "type": "string", + "name": "keyVaultSku" + }, + { + "type": "bool", + "name": "keyVaultEnablePurgeProtection" + }, + { + "type": "bool", + "name": "enableInfrastructureEncryption" + }, + { + "type": "bool", + "name": "enablePublicAccess" + }, + { + "type": "string", + "name": "networkName" + }, + { + "type": "string", + "name": "networkAddressPrefix" + }, + { + "type": "bool", + "name": "isTelemetryEnabled" + } + ], + "output": { + "$ref": "#/definitions/_1.HubProperties", + "value": { + "id": "[parameters('id')]", + "name": "[parameters('name')]", + "location": "[coalesce(parameters('location'), resourceGroup().location)]", + "tags": "[union(parameters('tags'), createObject('cm-resource-parent', parameters('id'), 'ftk-tool', 'FinOps hubs', 'ftk-version', variables('_1.finOpsToolkitVersion')))]", + "tagsByResource": "[parameters('tagsByResource')]", + "version": "[variables('_1.finOpsToolkitVersion')]", + "options": { + "enableTelemetry": "[coalesce(parameters('isTelemetryEnabled'), true())]", + "keyVaultSku": "[parameters('keyVaultSku')]", + "keyVaultEnablePurgeProtection": "[parameters('keyVaultEnablePurgeProtection')]", + "networkAddressPrefix": "[parameters('networkAddressPrefix')]", + "privateRouting": "[not(parameters('enablePublicAccess'))]", + "publisherIsolation": false, + "storageInfrastructureEncryption": "[parameters('enableInfrastructureEncryption')]", + "storageSku": "[parameters('storageSku')]" + }, + "routing": { + "networkId": "[if(parameters('enablePublicAccess'), '', resourceId('Microsoft.Network/virtualNetworks', parameters('networkName')))]", + "networkName": "[if(parameters('enablePublicAccess'), '', parameters('networkName'))]", + "scriptStorage": "[if(parameters('enablePublicAccess'), '', format('{0}script{1}', take(_1.safeStorageName(parameters('name')), sub(16, length(parameters('suffix')))), parameters('suffix')))]", + "dnsZones": { + "blob": "[if(parameters('enablePublicAccess'), createObject('id', '', 'name', ''), _1.dnsZoneIdName('blob'))]", + "dfs": "[if(parameters('enablePublicAccess'), createObject('id', '', 'name', ''), _1.dnsZoneIdName('dfs'))]", + "queue": "[if(parameters('enablePublicAccess'), createObject('id', '', 'name', ''), _1.dnsZoneIdName('queue'))]", + "table": "[if(parameters('enablePublicAccess'), createObject('id', '', 'name', ''), _1.dnsZoneIdName('table'))]" + }, + "subnets": { + "dataExplorer": "[if(parameters('enablePublicAccess'), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('networkName'), 'dataExplorer-subnet'))]", + "dataFactory": "[if(parameters('enablePublicAccess'), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('networkName'), 'private-endpoint-subnet'))]", + "keyVault": "[if(parameters('enablePublicAccess'), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('networkName'), 'private-endpoint-subnet'))]", + "scripts": "[if(parameters('enablePublicAccess'), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('networkName'), 'script-subnet'))]", + "storage": "[if(parameters('enablePublicAccess'), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('networkName'), 'private-endpoint-subnet'))]" + } + }, + "core": { + "suffix": "[parameters('suffix')]" + } + } + }, + "metadata": { + "__bicep_imported_from!": { + "sourceTemplate": "fx/hub-types.bicep" + } + } + }, + "safeStorageName": { + "parameters": [ + { + "type": "string", + "name": "name" + } + ], + "output": { + "type": "string", + "value": "[replace(replace(toLower(parameters('name')), '-', ''), '_', '')]" + }, + "metadata": { + "__bicep_imported_from!": { + "sourceTemplate": "fx/hub-types.bicep" + } + } + } + } + }, + { + "namespace": "__bicep", + "members": { + "getHubTags": { + "parameters": [ + { + "$ref": "#/definitions/_1.HubProperties", + "name": "hub" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(parameters('hub').tags, coalesce(tryGet(parameters('hub').tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub instance.", + "__bicep_imported_from!": { + "sourceTemplate": "fx/hub-types.bicep" + } + } + }, + "newApp": { + "parameters": [ + { + "$ref": "#/definitions/_1.HubProperties", + "name": "hub" + }, + { + "type": "string", + "name": "publisher" + }, + { + "type": "string", + "name": "app" + } + ], + "output": { + "$ref": "#/definitions/_1.HubAppProperties", + "value": "[_1.newAppInternal(parameters('hub'), format('{0}.{1}', parameters('publisher'), parameters('app')), parameters('app'), parameters('publisher'), if(or(not(parameters('hub').options.publisherIsolation), equals(parameters('publisher'), 'Microsoft.FinOpsHubs')), parameters('hub').core.suffix, uniqueString(parameters('publisher'))))]" + }, + "metadata": { + "description": "Creates a new FinOps hub app configuration object.", + "__bicep_imported_from!": { + "sourceTemplate": "fx/hub-types.bicep" + } + } + }, + "newHub": { + "parameters": [ + { + "type": "string", + "name": "name" + }, + { + "type": "string", + "name": "location" + }, + { + "type": "object", + "name": "tags" + }, + { + "type": "object", + "name": "tagsByResource" + }, + { + "type": "string", + "name": "storageSku" + }, + { + "type": "string", + "name": "keyVaultSku" + }, + { + "type": "bool", + "name": "keyVaultEnablePurgeProtection" + }, + { + "type": "bool", + "name": "enableInfrastructureEncryption" + }, + { + "type": "bool", + "name": "enablePublicAccess" + }, + { + "type": "string", + "name": "networkAddressPrefix" + }, + { + "type": "bool", + "name": "isTelemetryEnabled" + } + ], + "output": { + "$ref": "#/definitions/_1.HubProperties", + "value": "[_1.newHubInternal(format('{0}/providers/Microsoft.Cloud/hubs/{1}', resourceGroup().id, parameters('name')), parameters('name'), uniqueString(parameters('name'), resourceGroup().id), parameters('location'), parameters('tags'), parameters('tagsByResource'), parameters('storageSku'), parameters('keyVaultSku'), parameters('keyVaultEnablePurgeProtection'), parameters('enableInfrastructureEncryption'), parameters('enablePublicAccess'), format('{0}-vnet-{1}', _1.safeStorageName(parameters('name')), parameters('location')), parameters('networkAddressPrefix'), coalesce(parameters('isTelemetryEnabled'), true()))]" + }, + "metadata": { + "description": "Creates a new FinOps hub configuration object.", + "__bicep_imported_from!": { + "sourceTemplate": "fx/hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "hubName": { + "type": "string", + "metadata": { + "description": "Optional. Name of the hub. Used to ensure unique resource names. Default: \"finops-hub\"." + } + }, + "location": { + "type": "string", + "defaultValue": "[resourceGroup().location]", + "metadata": { + "description": "Optional. Azure location where all resources should be created. See https://aka.ms/azureregions. Default: (resource group location)." + } + }, + "storageSku": { + "type": "string", + "defaultValue": "Premium_LRS", + "allowedValues": [ + "Premium_LRS", + "Premium_ZRS" + ], + "metadata": { + "description": "Optional. Storage SKU to use. LRS = Lowest cost, ZRS = High availability. Note Standard SKUs are not available for Data Lake gen2 storage. Allowed: Premium_LRS, Premium_ZRS. Default: Premium_LRS." + } + }, + "enableInfrastructureEncryption": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Enable infrastructure encryption on the storage account. Default = false." + } + }, + "keyVaultSku": { + "type": "string", + "defaultValue": "premium", + "allowedValues": [ + "premium", + "standard" + ], + "metadata": { + "description": "Optional. SKU to use for the KeyVault instance, if enabled. Allowed values: \"standard\", \"premium\". Default: \"premium\"." + } + }, + "enablePurgeProtection": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Enable purge protection for the Key Vault. Default: false." + } + }, + "remoteHubStorageUri": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Remote storage account for ingestion dataset." + } + }, + "remoteHubStorageKey": { + "type": "securestring", + "defaultValue": "", + "metadata": { + "description": "Optional. Storage account key for remote storage account." + } + }, + "enableManagedExports": { + "type": "bool", + "defaultValue": true, + "metadata": { + "description": "Optional. Enable managed exports where your FinOps hub instance will create and run Cost Management exports on your behalf. Not supported for Microsoft Customer Agreement (MCA) billing profiles. Requires the ability to grant User Access Administrator role to FinOps hubs, which is required to create Cost Management exports. Default: true." + } + }, + "fabricQueryUri": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Microsoft Fabric eventhouse query URI. Default: \"\" (do not use)." + } + }, + "fabricCapacityUnits": { + "type": "int", + "defaultValue": 2, + "minValue": 1, + "maxValue": 2048, + "metadata": { + "description": "Optional. Number of capacity units for the Microsoft Fabric capacity. This is the number in your Fabric SKU (e.g., Trial = 1, F2 = 2, F64 = 64). This is used to manage parallelization in data pipelines. If you change capacity, please redeploy the template. Allowed values: 1 for the Fabric trial and 2-2048 based on the assigned Fabric capacity (e.g., F2-F2048). Default: 2." + } + }, + "dataExplorerName": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Name of the Azure Data Explorer cluster to use for advanced analytics. If empty, Azure Data Explorer will not be deployed. Required to use with Power BI if you have more than $2-5M/mo in costs being monitored. Default: \"\" (do not use)." + } + }, + "dataExplorerSku": { + "type": "string", + "defaultValue": "Dev(No SLA)_Standard_D11_v2", + "allowedValues": [ + "Dev(No SLA)_Standard_E2a_v4", + "Dev(No SLA)_Standard_D11_v2", + "Standard_D11_v2", + "Standard_D12_v2", + "Standard_D13_v2", + "Standard_D14_v2", + "Standard_D16d_v5", + "Standard_D32d_v4", + "Standard_D32d_v5", + "Standard_DS13_v2+1TB_PS", + "Standard_DS13_v2+2TB_PS", + "Standard_DS14_v2+3TB_PS", + "Standard_DS14_v2+4TB_PS", + "Standard_E2a_v4", + "Standard_E2ads_v5", + "Standard_E2d_v4", + "Standard_E2d_v5", + "Standard_E4a_v4", + "Standard_E4ads_v5", + "Standard_E4d_v4", + "Standard_E4d_v5", + "Standard_E8a_v4", + "Standard_E8ads_v5", + "Standard_E8as_v4+1TB_PS", + "Standard_E8as_v4+2TB_PS", + "Standard_E8as_v5+1TB_PS", + "Standard_E8as_v5+2TB_PS", + "Standard_E8d_v4", + "Standard_E8d_v5", + "Standard_E8s_v4+1TB_PS", + "Standard_E8s_v4+2TB_PS", + "Standard_E8s_v5+1TB_PS", + "Standard_E8s_v5+2TB_PS", + "Standard_E16a_v4", + "Standard_E16ads_v5", + "Standard_E16as_v4+3TB_PS", + "Standard_E16as_v4+4TB_PS", + "Standard_E16as_v5+3TB_PS", + "Standard_E16as_v5+4TB_PS", + "Standard_E16d_v4", + "Standard_E16d_v5", + "Standard_E16s_v4+3TB_PS", + "Standard_E16s_v4+4TB_PS", + "Standard_E16s_v5+3TB_PS", + "Standard_E16s_v5+4TB_PS", + "Standard_E64i_v3", + "Standard_E80ids_v4", + "Standard_EC8ads_v5", + "Standard_EC8as_v5+1TB_PS", + "Standard_EC8as_v5+2TB_PS", + "Standard_EC16ads_v5", + "Standard_EC16as_v5+3TB_PS", + "Standard_EC16as_v5+4TB_PS", + "Standard_L4s", + "Standard_L8as_v3", + "Standard_L8s", + "Standard_L8s_v2", + "Standard_L8s_v3", + "Standard_L16as_v3", + "Standard_L16s", + "Standard_L16s_v2", + "Standard_L16s_v3", + "Standard_L32as_v3", + "Standard_L32s_v3" + ], + "metadata": { + "description": "Optional. Name of the Azure Data Explorer SKU. Ignore when using Microsoft Fabric or not deploying Data Explorer. Default: \"Dev(No SLA)_Standard_D11_v2\"." + } + }, + "dataExplorerCapacity": { + "type": "int", + "defaultValue": 1, + "minValue": 1, + "maxValue": 1000, + "metadata": { + "description": "Optional. Number of nodes to use in the cluster. This is used to manage parallelization in data pipelines. If you change Fabric SKU, please redeploy the template. Allowed values: 1 for the Basic SKU tier and 2-1000 for Standard. Default: 1 for dev/test SKUs, 2 for standard SKUs." + } + }, + "tags": { + "type": "object", + "defaultValue": {}, + "metadata": { + "description": "Optional. Tags to apply to all resources. We will also add the cm-resource-parent tag for improved cost roll-ups in Cost Management." + } + }, + "tagsByResource": { + "type": "object", + "defaultValue": {}, + "metadata": { + "description": "Optional. Tags to apply to resources based on their resource type. Resource type specific tags will be merged with tags for all resources." + } + }, + "scopesToMonitor": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. List of scope IDs to monitor and ingest cost for." + } + }, + "exportRetentionInDays": { + "type": "int", + "defaultValue": 0, + "metadata": { + "description": "Optional. Number of days of data to retain in the msexports container. Default: 0." + } + }, + "ingestionRetentionInMonths": { + "type": "int", + "defaultValue": 13, + "metadata": { + "description": "Optional. Number of months of data to retain in the ingestion container. Default: 13." + } + }, + "dataExplorerRawRetentionInDays": { + "type": "int", + "defaultValue": 0, + "metadata": { + "description": "Optional. Number of days of data to retain in the Data Explorer *_raw tables. Default: 0." + } + }, + "dataExplorerFinalRetentionInMonths": { + "type": "int", + "defaultValue": 13, + "metadata": { + "description": "Optional. Number of months of data to retain in the Data Explorer *_final_v* tables. Default: 13." + } + }, + "enablePublicAccess": { + "type": "bool", + "defaultValue": true, + "metadata": { + "description": "Optional. Enable public access to the data lake. Default: true." + } + }, + "virtualNetworkAddressPrefix": { + "type": "string", + "defaultValue": "10.20.30.0/26", + "metadata": { + "description": "Optional. Address space for the workload. Minimum /26 subnet size is required for the workload. Default: \"10.20.30.0/26\"." + } + }, + "enableDefaultTelemetry": { + "type": "bool", + "defaultValue": true, + "metadata": { + "description": "Optional. Enable telemetry to track anonymous module usage trends, monitor for bugs, and improve future releases." + } + } + }, + "variables": { + "$fxv#0": "13.0", + "$fxv#1": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n# Init outputs\r\n$DeploymentScriptOutputs = @{}\r\n\r\n#\r\n$adfParams = @{\r\n ResourceGroupName = $env:DataFactoryResourceGroup\r\n DataFactoryName = $env:DataFactoryName\r\n}\r\n\r\n# Delete old triggers\r\n$triggers = Get-AzDataFactoryV2Trigger @adfParams -ErrorAction SilentlyContinue `\r\n| Where-Object { $_.Name -match '^msexports(_(setup|daily|monthly|extract|FileAdded))?$' }\r\n$DeploymentScriptOutputs[\"stopTriggers\"] = $triggers | Stop-AzDataFactoryV2Trigger -Force -ErrorAction SilentlyContinue\r\n$DeploymentScriptOutputs[\"deleteTriggers\"] = $triggers | Remove-AzDataFactoryV2Trigger -Force -ErrorAction SilentlyContinue\r\n\r\n# Delete old pipelines\r\n$DeploymentScriptOutputs[\"pipelines\"] = Get-AzDataFactoryV2Pipeline @adfParams -ErrorAction SilentlyContinue `\r\n| Where-Object { $_.Name -match '^(msexports_(backfill|extract|fill|get|run|setup|transform)|config_(BackfillData|ExportData|RunBackfill|RunExports))$' } `\r\n| Remove-AzDataFactoryV2Pipeline -Force -ErrorAction SilentlyContinue\r\n\r\n# Delete old datasets\r\n$DeploymentScriptOutputs[\"datasets\"] = Get-AzDataFactoryV2Dataset @adfParams -ErrorAction SilentlyContinue `\r\n| Where-Object { $_.Name -eq 'manifest' } `\r\n| Remove-AzDataFactoryV2Dataset -Force -ErrorAction SilentlyContinue\r\n", + "hub": "[__bicep.newHub(parameters('hubName'), parameters('location'), parameters('tags'), parameters('tagsByResource'), parameters('storageSku'), parameters('keyVaultSku'), parameters('enablePurgeProtection'), parameters('enableInfrastructureEncryption'), parameters('enablePublicAccess'), parameters('virtualNetworkAddressPrefix'), parameters('enableDefaultTelemetry'))]", + "useFabric": "[not(empty(parameters('fabricQueryUri')))]", + "useAzureDataExplorer": "[and(not(variables('useFabric')), not(empty(parameters('dataExplorerName'))))]", + "telemetryId": "00f120b5-2007-6120-0000-40b000000000", + "telemetryString": "[join(createArray(if(or(empty(parameters('remoteHubStorageUri')), empty(parameters('remoteHubStorageKey'))), '', 'R'), substring(split(parameters('storageSku'), '_')[1], 0, 1), if(not(variables('useFabric')), '', format('F{0}', parameters('fabricCapacityUnits'))), if(not(variables('useAzureDataExplorer')), '', format('X{0}', substring(parameters('dataExplorerSku'), 0, 1))), if(not(variables('useAzureDataExplorer')), '', replace(replace(replace(replace(replace(replace(replace(replace(split(split(parameters('dataExplorerSku'), 'Standard_')[1], '_')[0], 'C', ''), 'D', ''), 'E', ''), 'L', ''), 'a', ''), 'd', ''), 'i', ''), 's', '')), if(or(not(variables('useAzureDataExplorer')), equals(parameters('dataExplorerCapacity'), 1)), '', format('x{0}', parameters('dataExplorerCapacity'))), if(parameters('enablePublicAccess'), '', 'P')), '')]", + "_1.finOpsToolkitVersion": "13.0" + }, + "resources": { + "telemetry": { + "condition": "[parameters('enableDefaultTelemetry')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "[format('pid-{0}_{1}_{2}', variables('telemetryId'), variables('telemetryString'), uniqueString(deployment().name, parameters('location')))]", + "tags": "[__bicep.getHubTags(variables('hub'), 'Microsoft.Resources/deployments')]", + "properties": { + "mode": "Incremental", + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "FinOps toolkit", + "version": "[variables('$fxv#0')]" + } + }, + "resources": [] + } + } + }, + "core": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[__bicep.newApp(variables('hub'), 'Microsoft.FinOpsHubs', 'Core')]" + }, + "scopesToMonitor": { + "value": "[parameters('scopesToMonitor')]" + }, + "msexportRetentionInDays": { + "value": "[parameters('exportRetentionInDays')]" + }, + "ingestionRetentionInMonths": { + "value": "[parameters('ingestionRetentionInMonths')]" + }, + "rawRetentionInDays": { + "value": "[parameters('dataExplorerRawRetentionInDays')]" + }, + "finalRetentionInMonths": { + "value": "[parameters('dataExplorerFinalRetentionInMonths')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "9360754644605054925" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "scopesToMonitor": { + "type": "array", + "metadata": { + "description": "Optional. List of scope IDs to monitor and ingest cost for." + } + }, + "msexportRetentionInDays": { + "type": "int", + "defaultValue": 0, + "metadata": { + "description": "Optional. Number of days of data to retain in the msexports container. Default: 0." + } + }, + "ingestionRetentionInMonths": { + "type": "int", + "defaultValue": 13, + "metadata": { + "description": "Optional. Number of months of data to retain in the ingestion container. Default: 13." + } + }, + "rawRetentionInDays": { + "type": "int", + "defaultValue": 0, + "metadata": { + "description": "Optional. Number of days of data to retain in the Data Explorer *_raw tables. Default: 0." + } + }, + "finalRetentionInMonths": { + "type": "int", + "defaultValue": 13, + "metadata": { + "description": "Optional. Number of months of data to retain in the Data Explorer *_final_v* tables. Default: 13." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\nWrite-Output \"Updating settings.json file...\"\r\nWrite-Output \" Storage account: $env:storageAccountName\"\r\nWrite-Output \" Container: $env:containerName\"\r\n\r\n$validateScopes = { $_.scope.Length -gt 45 }\r\n\r\n# Initialize variables\r\n$fileName = 'settings.json'\r\n$filePath = Join-Path -Path . -ChildPath $fileName\r\n$newScopes = $env:scopes.Split('|') | ForEach-Object { [PSCustomObject]@{ scope = $_ } } | Where-Object $validateScopes\r\n\r\n# Get storage context\r\n$storageContext = @{\r\n Context = New-AzStorageContext -StorageAccountName $env:storageAccountName -UseConnectedAccount\r\n Container = $env:containerName\r\n}\r\n\r\n# Download existing settings, if they exist\r\n$blob = Get-AzStorageBlobContent @storageContext -Blob $fileName -Destination $filePath -Force\r\nif ($blob)\r\n{\r\n $text = Get-Content $filePath -Raw\r\n Write-Output \"---------\"\r\n Write-Output $text\r\n Write-Output \"---------\"\r\n $json = $text | ConvertFrom-Json\r\n Write-Output \"Existing settings.json file found. Updating...\"\r\n\r\n # Rename exportScopes to scopes + convert to object array\r\n if ($json.exportScopes)\r\n {\r\n Write-Output \" Updating exportScopes...\"\r\n if ($json.exportScopes[0] -is [string])\r\n {\r\n Write-Output \" Converting string array to object array...\"\r\n $json.exportScopes = @($json.exportScopes | Where-Object $validateScopes | ForEach-Object { @{ scope = $_ } })\r\n if (-not ($json.exportScopes -is [array]))\r\n {\r\n Write-Output \" Converting single object to object array...\"\r\n $json.exportScopes = @($json.exportScopes)\r\n }\r\n }\r\n\r\n Write-Output \" Renaming to 'scopes'...\"\r\n $json | Add-Member -MemberType NoteProperty -Name scopes -Value $json.exportScopes\r\n $json.PSObject.Properties.Remove('exportScopes')\r\n }\r\n\r\n # Force string array to object array with unique values\r\n if ($json.scopes)\r\n {\r\n Write-Output \" Converting string array to object array...\"\r\n $scopeArray = @()\r\n $json.scopes | Where-Object $validateScopes | ForEach-Object { $scopeArray += $_ } | Select-Object -Unique\r\n $json.scopes = @() + $scopeArray\r\n }\r\n}\r\n\r\n# Set default if not found\r\nif (!$json)\r\n{\r\n Write-Output \"No existing settings.json file found. Creating new file...\"\r\n $json = [ordered]@{\r\n '$schema' = 'https://aka.ms/finops/hubs/settings-schema'\r\n type = 'HubInstance'\r\n version = ''\r\n learnMore = 'https://aka.ms/finops/hubs'\r\n scopes = @()\r\n retention = @{\r\n 'msexports' = @{\r\n days = 0\r\n }\r\n 'ingestion' = @{\r\n months = 13\r\n }\r\n 'raw' = @{\r\n days = 0\r\n }\r\n 'final' = @{\r\n months = 13\r\n }\r\n }\r\n }\r\n\r\n $text = $json | ConvertTo-Json\r\n Write-Output \"---------\"\r\n Write-Output $text\r\n Write-Output \"---------\"\r\n}\r\n\r\n# Set default retention\r\nif (!($json.retention))\r\n{\r\n # In case the retention object is not present in the settings.json file (versions before 0.4), add it with default values\r\n $retention = @\"\r\n {\r\n \"msexports\": {\r\n \"days\": 0\r\n },\r\n \"ingestion\": {\r\n \"months\": 13\r\n },\r\n \"raw\": {\r\n \"days\": 0\r\n },\r\n \"final\": {\r\n \"months\": 13\r\n }\r\n }\r\n\"@\r\n $json | Add-Member -Name retention -Value (ConvertFrom-Json $retention) -MemberType NoteProperty\r\n}\r\n\r\n# Set or update msexports retention\r\nif (!($json.retention.msexports))\r\n{\r\n $json.retention | Add-Member -Name msexports -Value (ConvertFrom-Json \"{\"\"days\"\":$($env:msexportRetentionInDays)}\") -MemberType NoteProperty\r\n}\r\nelse\r\n{\r\n $json.retention.msexports.days = [Int32]::Parse($env:msexportRetentionInDays)\r\n}\r\n\r\n# Set or update ingestion retention\r\nif (!($json.retention.ingestion))\r\n{\r\n $json.retention | Add-Member -Name ingestion -Value (ConvertFrom-Json \"{\"\"months\"\":$($env:ingestionRetentionInMonths)}\") -MemberType NoteProperty\r\n}\r\nelse\r\n{\r\n $json.retention.ingestion.months = [Int32]::Parse($env:ingestionRetentionInMonths)\r\n}\r\n\r\n# Set or update raw retention\r\nif (!($json.retention.raw))\r\n{\r\n $json.retention | Add-Member -Name raw -Value (ConvertFrom-Json \"{\"\"days\"\":$($env:rawRetentionInDays)}\") -MemberType NoteProperty\r\n}\r\nelse\r\n{\r\n $json.retention.raw.days = [Int32]::Parse($env:rawRetentionInDays)\r\n}\r\n\r\n# Set or update final retention\r\nif (!($json.retention.final))\r\n{\r\n $json.retention | Add-Member -Name final -Value (ConvertFrom-Json \"{\"\"months\"\":$($env:finalRetentionInMonths)}\") -MemberType NoteProperty\r\n}\r\nelse\r\n{\r\n $json.retention.final.months = [Int32]::Parse($env:finalRetentionInMonths)\r\n}\r\n\r\n# Updating settings\r\nWrite-Output \"Updating version to $env:ftkVersion...\"\r\n$json.version = $env:ftkVersion\r\n$json.scopes = ($json.scopes + $newScopes) | Sort-Object scope -Unique\r\nif ($null -eq $json.scopes) { $json.scopes = @() }\r\n$text = $json | ConvertTo-Json\r\nWrite-Output \"---------\"\r\nWrite-Output $text\r\nWrite-Output \"---------\"\r\n$text | Out-File $filePath\r\n\r\n# Upload new/updated settings\r\nWrite-Output \"Uploading settings.json file...\"\r\nSet-AzStorageBlobContent @storageContext -File $filePath -Force | Out-Null\r\n", + "CONFIG": "config", + "INGESTION": "ingestion", + "finOpsToolkitVersion": "13.0" + }, + "resources": { + "dataFactory::dataset_config": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, variables('CONFIG'))]", + "properties": { + "linkedServiceName": { + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" + }, + "type": "Json", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().fileName}", + "type": "Expression" + }, + "folderPath": { + "value": "@{dataset().folderPath}", + "type": "Expression" + } + } + }, + "parameters": { + "fileName": { + "type": "String", + "defaultValue": "settings.json" + }, + "folderPath": { + "type": "String", + "defaultValue": "[reference('configContainer').outputs.containerName.value]" + } + } + }, + "dependsOn": [ + "appRegistration", + "configContainer" + ] + }, + "dataFactory::dataset_ingestion": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, variables('INGESTION'))]", + "properties": { + "annotations": [], + "parameters": { + "blobPath": { + "type": "String" + } + }, + "type": "Parquet", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().blobPath}", + "type": "Expression" + }, + "fileSystem": "[reference('ingestionContainer').outputs.containerName.value]" + } + }, + "linkedServiceName": { + "parameters": {}, + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" + } + }, + "dependsOn": [ + "appRegistration", + "ingestionContainer" + ] + }, + "dataFactory::dataset_ingestion_files": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_files', variables('INGESTION')))]", + "properties": { + "annotations": [], + "parameters": { + "folderPath": { + "type": "String" + } + }, + "type": "Parquet", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileSystem": "[reference('ingestionContainer').outputs.containerName.value]", + "folderPath": { + "value": "@dataset().folderPath", + "type": "Expression" + } + } + }, + "linkedServiceName": { + "parameters": {}, + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" + } + }, + "dependsOn": [ + "appRegistration", + "ingestionContainer" + ] + }, + "dataFactory::dataset_ingestion_manifest": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ingestion_manifest')]", + "properties": { + "annotations": [], + "parameters": { + "fileName": { + "type": "String", + "defaultValue": "manifest.json" + }, + "folderPath": { + "type": "String", + "defaultValue": "[variables('INGESTION')]" + } + }, + "type": "Json", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().fileName}", + "type": "Expression" + }, + "folderPath": { + "value": "@{dataset().folderPath}", + "type": "Expression" + } + } + }, + "linkedServiceName": { + "parameters": {}, + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" + } + }, + "dependsOn": [ + "appRegistration" + ] + }, + "dataFactory": { + "existing": true, + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "dependsOn": [ + "appRegistration" + ] + }, + "infrastructure": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core_Infrastructure", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "hub": { + "value": "[parameters('app').hub]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "14104368317935185659" + } + }, + "definitions": { + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getHubTags": { + "parameters": [ + { + "$ref": "#/definitions/HubProperties", + "name": "hub" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(parameters('hub').tags, coalesce(tryGet(parameters('hub').tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub instance.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "hub": { + "$ref": "#/definitions/HubProperties", + "metadata": { + "description": "Required. FinOps hub instance properties." + } + } + }, + "variables": { + "nsgName": "[format('{0}-nsg', parameters('hub').routing.networkName)]", + "finopsHubSubnetName": "private-endpoint-subnet", + "scriptSubnetName": "script-subnet", + "dataExplorerSubnetName": "dataExplorer-subnet", + "subnets": "[if(not(parameters('hub').options.privateRouting), createArray(), createArray(createObject('name', variables('finopsHubSubnetName'), 'properties', createObject('addressPrefix', cidrSubnet(parameters('hub').options.networkAddressPrefix, 28, 0), 'networkSecurityGroup', createObject('id', resourceId('Microsoft.Network/networkSecurityGroups', variables('nsgName'))), 'serviceEndpoints', createArray(createObject('service', 'Microsoft.Storage')))), createObject('name', variables('scriptSubnetName'), 'properties', createObject('addressPrefix', cidrSubnet(parameters('hub').options.networkAddressPrefix, 28, 1), 'networkSecurityGroup', createObject('id', resourceId('Microsoft.Network/networkSecurityGroups', variables('nsgName'))), 'delegations', createArray(createObject('name', 'Microsoft.ContainerInstance/containerGroups', 'properties', createObject('serviceName', 'Microsoft.ContainerInstance/containerGroups'))), 'serviceEndpoints', createArray(createObject('service', 'Microsoft.Storage')))), createObject('name', variables('dataExplorerSubnetName'), 'properties', createObject('addressPrefix', cidrSubnet(parameters('hub').options.networkAddressPrefix, 27, 1), 'networkSecurityGroup', createObject('id', resourceId('Microsoft.Network/networkSecurityGroups', variables('nsgName')))))))]" + }, + "resources": { + "vNet::finopsHubSubnet": { + "condition": "[parameters('hub').options.privateRouting]", + "existing": true, + "type": "Microsoft.Network/virtualNetworks/subnets", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', parameters('hub').routing.networkName, variables('finopsHubSubnetName'))]", + "dependsOn": [ + "vNet" + ] + }, + "vNet::scriptSubnet": { + "condition": "[parameters('hub').options.privateRouting]", + "existing": true, + "type": "Microsoft.Network/virtualNetworks/subnets", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', parameters('hub').routing.networkName, variables('scriptSubnetName'))]", + "dependsOn": [ + "vNet" + ] + }, + "vNet::dataExplorerSubnet": { + "condition": "[parameters('hub').options.privateRouting]", + "existing": true, + "type": "Microsoft.Network/virtualNetworks/subnets", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', parameters('hub').routing.networkName, variables('dataExplorerSubnetName'))]", + "dependsOn": [ + "vNet" + ] + }, + "blobPrivateDnsZone::blobPrivateDnsZoneLink": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', string(parameters('hub').routing.dnsZones.blob.name), format('{0}-link', replace(string(parameters('hub').routing.dnsZones.blob.name), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "properties": { + "registrationEnabled": false, + "virtualNetwork": { + "id": "[parameters('hub').routing.networkId]" + } + }, + "dependsOn": [ + "blobPrivateDnsZone" + ] + }, + "dfsPrivateDnsZone::dfsPrivateDnsZoneLink": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', string(parameters('hub').routing.dnsZones.dfs.name), format('{0}-link', replace(string(parameters('hub').routing.dnsZones.dfs.name), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "properties": { + "registrationEnabled": false, + "virtualNetwork": { + "id": "[parameters('hub').routing.networkId]" + } + }, + "dependsOn": [ + "dfsPrivateDnsZone" + ] + }, + "queuePrivateDnsZone::queuePrivateDnsZoneLink": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', string(parameters('hub').routing.dnsZones.queue.name), format('{0}-link', replace(string(parameters('hub').routing.dnsZones.queue.name), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "properties": { + "registrationEnabled": false, + "virtualNetwork": { + "id": "[parameters('hub').routing.networkId]" + } + }, + "dependsOn": [ + "queuePrivateDnsZone" + ] + }, + "tablePrivateDnsZone::tablePrivateDnsZoneLink": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', string(parameters('hub').routing.dnsZones.table.name), format('{0}-link', replace(string(parameters('hub').routing.dnsZones.table.name), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "properties": { + "registrationEnabled": false, + "virtualNetwork": { + "id": "[parameters('hub').routing.networkId]" + } + }, + "dependsOn": [ + "tablePrivateDnsZone" + ] + }, + "scriptEndpoint::scriptPrivateDnsZoneGroup": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('hub').routing.scriptStorage), 'blob-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[string(parameters('hub').routing.dnsZones.blob.name)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', string(parameters('hub').routing.dnsZones.blob.name))]" + } + } + ] + }, + "dependsOn": [ + "blobPrivateDnsZone", + "scriptEndpoint" + ] + }, + "nsg": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/networkSecurityGroups", + "apiVersion": "2023-11-01", + "name": "[variables('nsgName')]", + "location": "[parameters('hub').location]", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/networkSecurityGroups')]", + "properties": { + "securityRules": [ + { + "name": "AllowVnetInBound", + "properties": { + "priority": 100, + "direction": "Inbound", + "access": "Allow", + "protocol": "*", + "sourcePortRange": "*", + "destinationPortRange": "*", + "sourceAddressPrefix": "VirtualNetwork", + "destinationAddressPrefix": "VirtualNetwork" + } + }, + { + "name": "AllowAzureLoadBalancerInBound", + "properties": { + "priority": 200, + "direction": "Inbound", + "access": "Allow", + "protocol": "*", + "sourcePortRange": "*", + "destinationPortRange": "*", + "sourceAddressPrefix": "AzureLoadBalancer", + "destinationAddressPrefix": "*" + } + }, + { + "name": "DenyAllInBound", + "properties": { + "priority": 4096, + "direction": "Inbound", + "access": "Deny", + "protocol": "*", + "sourcePortRange": "*", + "destinationPortRange": "*", + "sourceAddressPrefix": "*", + "destinationAddressPrefix": "*" + } + }, + { + "name": "AllowVnetOutBound", + "properties": { + "priority": 100, + "direction": "Outbound", + "access": "Allow", + "protocol": "*", + "sourcePortRange": "*", + "destinationPortRange": "*", + "sourceAddressPrefix": "VirtualNetwork", + "destinationAddressPrefix": "VirtualNetwork" + } + }, + { + "name": "AllowInternetOutBound", + "properties": { + "priority": 200, + "direction": "Outbound", + "access": "Allow", + "protocol": "*", + "sourcePortRange": "*", + "destinationPortRange": "*", + "sourceAddressPrefix": "*", + "destinationAddressPrefix": "Internet" + } + }, + { + "name": "DenyAllOutBound", + "properties": { + "priority": 4096, + "direction": "Outbound", + "access": "Deny", + "protocol": "*", + "sourcePortRange": "*", + "destinationPortRange": "*", + "sourceAddressPrefix": "*", + "destinationAddressPrefix": "*" + } + } + ] + } + }, + "vNet": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/virtualNetworks", + "apiVersion": "2023-11-01", + "name": "[parameters('hub').routing.networkName]", + "location": "[parameters('hub').location]", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/virtualNetworks')]", + "properties": { + "addressSpace": { + "addressPrefixes": [ + "[parameters('hub').options.networkAddressPrefix]" + ] + }, + "subnets": "[variables('subnets')]" + }, + "dependsOn": [ + "nsg" + ] + }, + "blobPrivateDnsZone": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[string(parameters('hub').routing.dnsZones.blob.name)]", + "location": "global", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/privateDnsZones')]", + "properties": {}, + "dependsOn": [ + "vNet" + ] + }, + "dfsPrivateDnsZone": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[string(parameters('hub').routing.dnsZones.dfs.name)]", + "location": "global", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/privateDnsZones')]", + "properties": {}, + "dependsOn": [ + "vNet" + ] + }, + "queuePrivateDnsZone": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[string(parameters('hub').routing.dnsZones.queue.name)]", + "location": "global", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/privateDnsZones')]", + "properties": {}, + "dependsOn": [ + "vNet" + ] + }, + "tablePrivateDnsZone": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[string(parameters('hub').routing.dnsZones.table.name)]", + "location": "global", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/privateDnsZones')]", + "properties": {}, + "dependsOn": [ + "vNet" + ] + }, + "scriptStorageAccount": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('hub').routing.scriptStorage]", + "location": "[parameters('hub').location]", + "sku": { + "name": "Standard_LRS" + }, + "kind": "StorageV2", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/storageAccounts')]", + "properties": { + "supportsHttpsTrafficOnly": true, + "allowSharedKeyAccess": true, + "isHnsEnabled": false, + "minimumTlsVersion": "TLS1_2", + "allowBlobPublicAccess": false, + "publicNetworkAccess": "Enabled", + "networkAcls": { + "bypass": "AzureServices", + "defaultAction": "Deny", + "virtualNetworkRules": [ + { + "id": "[parameters('hub').routing.subnets.scripts]", + "action": "Allow" + } + ] + } + }, + "dependsOn": [ + "vNet::scriptSubnet" + ] + }, + "scriptEndpoint": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-blob-ep', parameters('hub').routing.scriptStorage)]", + "location": "[parameters('hub').location]", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('hub').routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "scriptLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('hub').routing.scriptStorage)]", + "groupIds": [ + "blob" + ] + } + } + ] + }, + "dependsOn": [ + "scriptStorageAccount", + "vNet::scriptSubnet" + ] + } + }, + "outputs": { + "config": { + "$ref": "#/definitions/HubProperties", + "metadata": { + "description": "FinOps hub configuration settings." + }, + "value": "[parameters('hub')]" + }, + "vNetId": { + "type": "string", + "metadata": { + "description": "Resource ID of the virtual network." + }, + "value": "[if(not(parameters('hub').options.privateRouting), '', resourceId('Microsoft.Network/virtualNetworks', parameters('hub').routing.networkName))]" + }, + "vNetAddressSpace": { + "type": "array", + "metadata": { + "description": "Virtual network address prefixes." + }, + "value": "[if(not(parameters('hub').options.privateRouting), createArray(), reference('vNet').addressSpace.addressPrefixes)]" + }, + "vNetSubnets": { + "type": "array", + "metadata": { + "description": "Virtual network subnets." + }, + "value": "[if(not(parameters('hub').options.privateRouting), createArray(), reference('vNet').subnets)]" + }, + "finopsHubSubnetId": { + "type": "string", + "metadata": { + "description": "Resource ID of the FinOps hub network subnet." + }, + "value": "[if(not(parameters('hub').options.privateRouting), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('hub').routing.networkName, variables('finopsHubSubnetName')))]" + }, + "scriptSubnetId": { + "type": "string", + "metadata": { + "description": "Resource ID of the script storage account network subnet." + }, + "value": "[if(not(parameters('hub').options.privateRouting), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('hub').routing.networkName, variables('scriptSubnetName')))]" + }, + "dataExplorerSubnetId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Data Explorer network subnet." + }, + "value": "[if(not(parameters('hub').options.privateRouting), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('hub').routing.networkName, variables('dataExplorerSubnetName')))]" + } + } + } + } + }, + "appRegistration": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core_Register", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "version": { + "value": "[variables('finOpsToolkitVersion')]" + }, + "features": { + "value": [ + "DataFactory", + "Storage" + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "8267413868206701537" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppFeature": { + "type": "string", + "allowedValues": [ + "DataFactory", + "KeyVault", + "Storage" + ], + "metadata": { + "description": "FinOps hub app features.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "version": { + "type": "string", + "metadata": { + "description": "Required. Version number of the FinOps hub app." + } + }, + "features": { + "type": "array", + "items": { + "$ref": "#/definitions/HubAppFeature" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which features the app requires. Allowed values: \"DataFactory\", \"KeyVault\", \"Storage\". Default: [] (none)." + } + }, + "storageRoles": { + "type": "array", + "items": { + "type": "string" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which RBAC roles the Data Factory identity needs on the storage account, if created. This is in addition to Storage Blob Data Contributor for reading and managing content. Default: [] (none)." + } + }, + "telemetryString": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n<#\r\n.SYNOPSIS\r\nManages Azure Data Factory triggers and pipelines during FinOps hub deployment.\r\n\r\n.DESCRIPTION\r\nThis script is called by Bicep deployment scripts to start/stop Data Factory triggers\r\nand optionally run pipelines. It handles two types of triggers:\r\n\r\n1. Schedule triggers - Can be started/stopped directly\r\n2. BlobEventsTriggers - Require Event Grid subscription management before start/stop\r\n\r\nBlobEventsTriggers use Event Grid to listen for storage blob events. Before stopping,\r\nthe Event Grid subscription must be removed (unsubscribed). Before starting, the\r\nsubscription must be added and fully provisioned. This script handles this automatically\r\nby detecting BlobEventsTriggers via the BlobPathBeginsWith property.\r\n\r\nThe script uses retry logic with linear backoff to handle transient API failures and\r\nwait for Event Grid subscription provisioning/deprovisioning to complete.\r\n\r\n.PARAMETER DataFactoryResourceGroup\r\nThe resource group containing the Data Factory.\r\n\r\n.PARAMETER DataFactoryName\r\nThe name of the Data Factory instance.\r\n\r\n.PARAMETER Pipelines\r\nPipe-delimited list of pipeline names to run (e.g., \"pipeline1|pipeline2\").\r\n\r\n.PARAMETER StartTriggers\r\nSwitch to start all stopped triggers (with Event Grid subscription for blob triggers).\r\n\r\n.PARAMETER StopTriggers\r\nSwitch to stop all running triggers (with Event Grid unsubscription for blob triggers).\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StopTriggers\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StartTriggers\r\n#>\r\n\r\nparam(\r\n [string] $DataFactoryResourceGroup,\r\n [string] $DataFactoryName,\r\n [string] $Pipelines = \"\",\r\n [switch] $StartTriggers,\r\n [switch] $StopTriggers\r\n)\r\n\r\n$MAX_RETRIES = 20\r\n$DeploymentScriptOutputs = @{}\r\n\r\nfunction Write-Log($Message)\r\n{\r\n Write-Output \"$(Get-Date -Format 'HH:mm:ss') $Message\"\r\n}\r\n\r\nfunction Invoke-WithRetry([scriptblock]$Action, [string]$Name, [int]$Delay = 5)\r\n{\r\n for ($i = 1; $i -le $MAX_RETRIES; $i++)\r\n {\r\n try { return & $Action }\r\n catch\r\n {\r\n Write-Log \"$Name failed (attempt $i/${MAX_RETRIES}): $($_.Exception.Message)\"\r\n if ($i -eq $MAX_RETRIES) { throw }\r\n Start-Sleep -Seconds ($Delay * $i)\r\n }\r\n }\r\n}\r\n\r\nfunction Set-BlobTriggerSubscription([string]$TriggerName, [switch]$Subscribe)\r\n{\r\n $targetStatus = if ($Subscribe) { 'Enabled' } else { 'Disabled' }\r\n $action = if ($Subscribe) { 'Subscribing' } else { 'Unsubscribing' }\r\n\r\n Write-Log \"$action $TriggerName to events...\"\r\n Invoke-WithRetry -Name \"$action $TriggerName\" -Delay 5 -Action {\r\n if ($Subscribe)\r\n {\r\n Add-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n else\r\n {\r\n Remove-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n\r\n $status = Get-AzDataFactoryV2TriggerSubscriptionStatus `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName\r\n if ($status.Status -ne $targetStatus)\r\n {\r\n throw \"Subscription status is $($status.Status), expected $targetStatus\"\r\n }\r\n }\r\n}\r\n\r\nif ($StartTriggers -or $StopTriggers)\r\n{\r\n $triggers = Invoke-WithRetry -Name \"Get triggers\" -Action {\r\n Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n | Where-Object {\r\n ($StartTriggers -and $_.Properties.RuntimeState -ne \"Started\") `\r\n -or ($StopTriggers -and $_.Properties.RuntimeState -ne \"Stopped\")\r\n }\r\n }\r\n\r\n Write-Log \"Found $($triggers.Count) trigger(s) to $(if ($StartTriggers) { 'start' } else { 'stop' })\"\r\n\r\n $triggers | ForEach-Object {\r\n $triggerName = $_.Name\r\n $isBlobTrigger = $null -ne $_.Properties.BlobPathBeginsWith\r\n\r\n if ($StopTriggers)\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName }\r\n Write-Log \"Stopping trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Stop $triggerName\" -Action {\r\n Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n else\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName -Subscribe }\r\n Write-Log \"Starting trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Start $triggerName\" -Action {\r\n Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n\r\n Invoke-WithRetry -Name \"Wait for $triggerName\" -Action {\r\n $state = (Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName).Properties.RuntimeState\r\n $expected = if ($StartTriggers) { 'Started' } else { 'Stopped' }\r\n if ($state -ne $expected) { throw \"Trigger is $state, expected $expected\" }\r\n }\r\n\r\n Write-Log \"...done\"\r\n $DeploymentScriptOutputs[$triggerName] = $true\r\n }\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($Pipelines))\r\n{\r\n $Pipelines.Split('|') | ForEach-Object {\r\n Write-Log \"Running pipeline $_...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", + "usesDataFactory": "[contains(parameters('features'), 'DataFactory')]", + "usesKeyVault": "[contains(parameters('features'), 'KeyVault')]", + "usesStorage": "[contains(parameters('features'), 'Storage')]", + "telemetryId": "[format('ftk-hubapp-{0}{1}{2}', parameters('app').id, if(empty(parameters('telemetryString')), '', '_'), parameters('telemetryString'))]", + "telemetryProps": { + "mode": "Incremental", + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "[format('FTK: {0}', parameters('app').id)]", + "version": "[parameters('version')]" + } + }, + "resources": [] + } + }, + "autoStartRbacRoles": [ + "673868aa-7521-48a0-acc6-0f60742d39f5" + ], + "factoryStorageRoles": "[union(parameters('storageRoles'), createArray('17d1049b-9a84-46fb-8f53-869881c3d3ab', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe', 'acdd72a7-3385-48ef-bd42-f606fba81ae7'))]", + "storageInfrastructureEncryptionProperties": "[if(not(parameters('app').hub.options.storageInfrastructureEncryption), createObject(), createObject('encryption', createObject('keySource', 'Microsoft.Storage', 'requireInfrastructureEncryption', parameters('app').hub.options.storageInfrastructureEncryption)))]" + }, + "resources": { + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').storage)]", + "properties": { + "name": "[parameters('app').storage]", + "groupId": "dfs", + "privateLinkResourceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "fqdns": [ + "[reference('storageAccount').primaryEndpoints.dfs]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "storageAccount" + ] + }, + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').keyVault)]", + "properties": { + "name": "[parameters('app').keyVault]", + "groupId": "vault", + "privateLinkResourceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "fqdns": [ + "[reference('keyVault').vaultUri]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "keyVault" + ] + }, + "dataFactory::managedVirtualNetwork": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'default')]", + "properties": {}, + "dependsOn": [ + "dataFactory" + ] + }, + "dataFactory::managedIntegrationRuntime": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/integrationRuntimes", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ManagedIntegrationRuntime')]", + "properties": { + "type": "Managed", + "managedVirtualNetwork": { + "referenceName": "default", + "type": "ManagedVirtualNetworkReference" + }, + "typeProperties": { + "computeProperties": { + "location": "[parameters('app').hub.location]", + "dataFlowProperties": { + "computeType": "General", + "coreCount": 8, + "timeToLive": 10, + "cleanup": false, + "customProperties": [] + }, + "copyComputeScaleProperties": { + "dataIntegrationUnit": 16, + "timeToLive": 30 + }, + "pipelineExternalComputeScaleProperties": { + "timeToLive": 30, + "numberOfPipelineNodes": 1, + "numberOfExternalNodes": 1 + } + } + } + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedVirtualNetwork" + ] + }, + "dataFactory::linkedService_keyVault": { + "condition": "[and(variables('usesDataFactory'), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').keyVault)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureKeyVault", + "typeProperties": { + "baseUrl": "[reference(format('Microsoft.KeyVault/vaults/{0}', parameters('app').keyVault), '2023-02-01').vaultUri]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "keyVault" + ] + }, + "dataFactory::linkedService_storageAccount": { + "condition": "[and(variables('usesDataFactory'), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').storage)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureBlobFS", + "typeProperties": { + "url": "[reference(format('Microsoft.Storage/storageAccounts/{0}', parameters('app').storage), '2021-08-01').primaryEndpoints.dfs]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "storageAccount" + ] + }, + "storageAccount::blobService": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]", + "dependsOn": [ + "storageAccount" + ] + }, + "blobEndpoint::blobPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "blobEndpoint" + ] + }, + "dfsEndpoint::dfsPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.dfs.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "dfsEndpoint" + ] + }, + "keyVaultPrivateDnsZone::keyVaultPrivateDnsZoneLink": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), format('{0}-link', replace(format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "properties": { + "virtualNetwork": { + "id": "[parameters('app').hub.routing.networkId]" + }, + "registrationEnabled": false + }, + "dependsOn": [ + "keyVaultPrivateDnsZone" + ] + }, + "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')))]" + } + } + ] + }, + "dependsOn": [ + "keyVaultEndpoint", + "keyVaultPrivateDnsZone" + ] + }, + "appTelemetry": { + "condition": "[parameters('app').hub.options.enableTelemetry]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "[if(lessOrEquals(length(variables('telemetryId')), 64), variables('telemetryId'), substring(variables('telemetryId'), 0, 64))]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Resources/deployments')]", + "properties": "[variables('telemetryProps')]" + }, + "dataFactory": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.DataFactory/factories')]", + "identity": { + "type": "SystemAssigned" + }, + "properties": { + "globalConfigurations": { + "PipelineBillingEnabled": "true" + } + } + }, + "storageRoleAssignments": { + "copy": { + "name": "storageRoleAssignments", + "count": "[length(variables('factoryStorageRoles'))]" + }, + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "name": "[guid(resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage), variables('factoryStorageRoles')[copyIndex()], resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('factoryStorageRoles')[copyIndex()])]", + "principalId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "storageAccount" + ] + }, + "triggerManagerIdentity": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[format('{0}_triggerManager', parameters('app').dataFactory)]", + "location": "[parameters('app').hub.location]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "dependsOn": [ + "dataFactory" + ] + }, + "triggerManagerRoleAssignments": { + "copy": { + "name": "triggerManagerRoleAssignments", + "count": "[length(variables('autoStartRbacRoles'))]" + }, + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]", + "name": "[guid(resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory), variables('autoStartRbacRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', format('{0}_triggerManager', parameters('app').dataFactory)))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('autoStartRbacRoles')[copyIndex()])]", + "principalId": "[reference('triggerManagerIdentity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "triggerManagerIdentity" + ] + }, + "storageAccount": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]", + "location": "[parameters('app').hub.location]", + "sku": { + "name": "[parameters('app').hub.options.storageSku]" + }, + "kind": "BlockBlobStorage", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Storage/storageAccounts')]", + "properties": "[shallowMerge(createArray(variables('storageInfrastructureEncryptionProperties'), createObject('supportsHttpsTrafficOnly', true(), 'allowSharedKeyAccess', true(), 'isHnsEnabled', true(), 'minimumTlsVersion', 'TLS1_2', 'allowBlobPublicAccess', false(), 'publicNetworkAccess', 'Enabled', 'networkAcls', createObject('bypass', 'AzureServices', 'defaultAction', if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')))))]" + }, + "blobPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]" + }, + "blobEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-blob-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "blobLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "blob" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "dfsPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]" + }, + "dfsEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-dfs-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "dfsLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "dfs" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "keyVault": { + "condition": "[variables('usesKeyVault')]", + "type": "Microsoft.KeyVault/vaults", + "apiVersion": "2023-02-01", + "name": "[parameters('app').keyVault]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.KeyVault/vaults')]", + "properties": { + "sku": { + "name": "[parameters('app').hub.options.keyVaultSku]", + "family": "A" + }, + "enabledForDeployment": true, + "enabledForTemplateDeployment": true, + "enabledForDiskEncryption": true, + "enableSoftDelete": true, + "softDeleteRetentionInDays": 90, + "enablePurgeProtection": "[if(parameters('app').hub.options.keyVaultEnablePurgeProtection, true(), null())]", + "enableRbacAuthorization": false, + "createMode": "default", + "tenantId": "[subscription().tenantId]", + "accessPolicies": [ + { + "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "tenantId": "[subscription().tenantId]", + "permissions": { + "secrets": [ + "get" + ] + } + } + ], + "networkAcls": { + "bypass": "AzureServices", + "defaultAction": "[if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')]" + } + }, + "dependsOn": [ + "dataFactory" + ] + }, + "keyVaultPrivateDnsZone": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones')]", + "properties": {} + }, + "keyVaultEndpoint": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-ep', parameters('app').keyVault)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.keyVault]" + }, + "privateLinkServiceConnections": [ + { + "name": "keyVaultLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "groupIds": [ + "vault" + ] + } + } + ] + }, + "dependsOn": [ + "keyVault" + ] + }, + "getKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint", + "getStoragePrivateEndpointConnections", + "keyVault" + ] + }, + "approveKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + }, + "privateEndpointConnections": { + "value": "[reference('getKeyVaultPrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getKeyVaultPrivateEndpointConnections", + "keyVault" + ] + }, + "getStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint", + "stopTriggers", + "storageAccount" + ] + }, + "approveStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + }, + "privateEndpointConnections": { + "value": "[reference('getStoragePrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getStoragePrivateEndpointConnections", + "storageAccount" + ] + }, + "stopTriggers": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.{1}_ADF.StopTriggers', parameters('app').publisher, parameters('app').name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "arguments": { + "value": "[join(createArray(format('-DataFactoryResourceGroup \"{0}\"', resourceGroup().name), format('-DataFactoryName \"{0}\"', parameters('app').dataFactory), '-StopTriggers'), ' ')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + }, + "dependsOn": [ + "appTelemetry", + "dataFactory", + "triggerManagerIdentity", + "triggerManagerRoleAssignments" + ] + } + }, + "outputs": { + "dataFactoryId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Data Factory instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]" + }, + "keyVaultId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Key Vault instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]" + }, + "storageAccountId": { + "type": "string", + "metadata": { + "description": "Resource ID of the storage account instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Principal ID for the managed identity used by Data Factory." + }, + "value": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" + }, + "triggerManagerIdentityName": { + "type": "string", + "metadata": { + "description": "Name of the managed identity used to create and stop ADF triggers." + }, + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + } + } + } + }, + "dependsOn": [ + "infrastructure" + ] + }, + "configContainer": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core_Storage.ConfigContainer", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "container": { + "value": "[variables('CONFIG')]" + }, + "forceCreateBlobManagerIdentity": { + "value": true + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6646280599480816180" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app that storage is getting updated for." + } + }, + "container": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage container to create or update." + } + }, + "files": { + "type": "object", + "defaultValue": {}, + "metadata": { + "description": "Optional. Dictionary of key/value pairs for the files to upload to the specified container. The key is the target path under the container and the value is the contents of the file. Default: {} (no files to upload)." + } + }, + "forceCreateBlobManagerIdentity": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Indicates whether to create the blob manager user assigned identity even if files are not being uploaded. Default: false." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n# Get storage context\r\n$storageContext = @{\r\n Context = New-AzStorageContext -StorageAccountName $env:storageAccountName -UseConnectedAccount\r\n Container = $env:containerName\r\n}\r\n\r\n# Uploading files\r\n$files = $env:files | ConvertFrom-Json -Depth 10\r\nWrite-Output \"Uploading ${$files.PSObject.Properties.Count} files...\"\r\n$files.PSObject.Properties | ForEach-Object {\r\n $filePath = $_.Name\r\n $tempPath = \"./$($filePath -replace \"/\", \"_\")\"\r\n Write-Output \" Uploading $filePath...\"\r\n $_.Value | Out-File $tempPath\r\n Set-AzStorageBlobContent @storageContext -File $tempPath -Blob $filePath -Force | Out-Null\r\n}\r\n", + "fileCount": "[length(items(parameters('files')))]", + "hasFiles": "[greater(variables('fileCount'), 0)]" + }, + "resources": { + "storageAccount::blobService::targetContainer": { + "type": "Microsoft.Storage/storageAccounts/blobServices/containers", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}/{2}', parameters('app').storage, 'default', parameters('container'))]", + "properties": { + "publicAccess": "None", + "metadata": {} + } + }, + "storageAccount::blobService": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]" + }, + "storageAccount": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]" + }, + "identity": { + "condition": "[or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity'))]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.Identity', deployment().name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_blobManager', parameters('app').storage)]" + }, + "roleAssignmentResourceId": { + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "roles": { + "value": [ + "ba92f5b4-2d11-453d-a403-e96b0029c9fe", + "69566ab7-960f-475b-8e7c-b3118f30c6bd" + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "18138592202204453545" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the identity is associated with." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the user assigned identity." + } + }, + "roleAssignmentResourceId": { + "type": "string", + "metadata": { + "description": "Required. Resource ID of the resource access is being granted for." + } + }, + "roles": { + "type": "array", + "items": { + "type": "string" + }, + "metadata": { + "description": "Required. List of RBAC role assignment GUIDs." + } + } + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.ManagedIdentity/userAssignedIdentities')]", + "location": "[parameters('app').hub.location]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(parameters('roles'))]" + }, + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "name": "[guid(parameters('roleAssignmentResourceId'), parameters('roles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', parameters('roles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + } + }, + "outputs": { + "id": { + "type": "string", + "metadata": { + "description": "Resource ID of the user assigned identity." + }, + "value": "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName'))]" + }, + "name": { + "type": "string", + "metadata": { + "description": "Name of the user assigned identity." + }, + "value": "[parameters('identityName')]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Principal ID of the user assigned identity." + }, + "value": "[reference('identity').principalId]" + } + } + } + } + }, + "uploadFiles": { + "condition": "[variables('hasFiles')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.Upload', deployment().name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[reference('identity').outputs.name.value]" + }, + "environmentVariables": { + "value": [ + { + "name": "storageAccountName", + "value": "[parameters('app').storage]" + }, + { + "name": "containerName", + "value": "[parameters('container')]" + }, + { + "name": "files", + "value": "[string(parameters('files'))]" + } + ] + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + }, + "dependsOn": [ + "identity" + ] + } + }, + "outputs": { + "containerName": { + "type": "string", + "metadata": { + "description": "The name of the storage container." + }, + "value": "[parameters('container')]" + }, + "filesUploaded": { + "type": "int", + "metadata": { + "description": "The number of files uploaded to the storage container." + }, + "value": "[variables('fileCount')]" + }, + "identityId": { + "type": "string", + "metadata": { + "description": "Resource ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.id.value, '')]" + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Name of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.name.value, '')]" + }, + "identityPrincipalId": { + "type": "string", + "metadata": { + "description": "Principal ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.principalId.value, '')]" + } + } + } + }, + "dependsOn": [ + "appRegistration" + ] + }, + "ingestionContainer": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core_Storage.IngestionContainer", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "container": { + "value": "[variables('INGESTION')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6646280599480816180" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app that storage is getting updated for." + } + }, + "container": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage container to create or update." + } + }, + "files": { + "type": "object", + "defaultValue": {}, + "metadata": { + "description": "Optional. Dictionary of key/value pairs for the files to upload to the specified container. The key is the target path under the container and the value is the contents of the file. Default: {} (no files to upload)." + } + }, + "forceCreateBlobManagerIdentity": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Indicates whether to create the blob manager user assigned identity even if files are not being uploaded. Default: false." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n# Get storage context\r\n$storageContext = @{\r\n Context = New-AzStorageContext -StorageAccountName $env:storageAccountName -UseConnectedAccount\r\n Container = $env:containerName\r\n}\r\n\r\n# Uploading files\r\n$files = $env:files | ConvertFrom-Json -Depth 10\r\nWrite-Output \"Uploading ${$files.PSObject.Properties.Count} files...\"\r\n$files.PSObject.Properties | ForEach-Object {\r\n $filePath = $_.Name\r\n $tempPath = \"./$($filePath -replace \"/\", \"_\")\"\r\n Write-Output \" Uploading $filePath...\"\r\n $_.Value | Out-File $tempPath\r\n Set-AzStorageBlobContent @storageContext -File $tempPath -Blob $filePath -Force | Out-Null\r\n}\r\n", + "fileCount": "[length(items(parameters('files')))]", + "hasFiles": "[greater(variables('fileCount'), 0)]" + }, + "resources": { + "storageAccount::blobService::targetContainer": { + "type": "Microsoft.Storage/storageAccounts/blobServices/containers", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}/{2}', parameters('app').storage, 'default', parameters('container'))]", + "properties": { + "publicAccess": "None", + "metadata": {} + } + }, + "storageAccount::blobService": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]" + }, + "storageAccount": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]" + }, + "identity": { + "condition": "[or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity'))]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.Identity', deployment().name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_blobManager', parameters('app').storage)]" + }, + "roleAssignmentResourceId": { + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "roles": { + "value": [ + "ba92f5b4-2d11-453d-a403-e96b0029c9fe", + "69566ab7-960f-475b-8e7c-b3118f30c6bd" + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "18138592202204453545" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the identity is associated with." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the user assigned identity." + } + }, + "roleAssignmentResourceId": { + "type": "string", + "metadata": { + "description": "Required. Resource ID of the resource access is being granted for." + } + }, + "roles": { + "type": "array", + "items": { + "type": "string" + }, + "metadata": { + "description": "Required. List of RBAC role assignment GUIDs." + } + } + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.ManagedIdentity/userAssignedIdentities')]", + "location": "[parameters('app').hub.location]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(parameters('roles'))]" + }, + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "name": "[guid(parameters('roleAssignmentResourceId'), parameters('roles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', parameters('roles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + } + }, + "outputs": { + "id": { + "type": "string", + "metadata": { + "description": "Resource ID of the user assigned identity." + }, + "value": "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName'))]" + }, + "name": { + "type": "string", + "metadata": { + "description": "Name of the user assigned identity." + }, + "value": "[parameters('identityName')]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Principal ID of the user assigned identity." + }, + "value": "[reference('identity').principalId]" + } + } + } + } + }, + "uploadFiles": { + "condition": "[variables('hasFiles')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.Upload', deployment().name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[reference('identity').outputs.name.value]" + }, + "environmentVariables": { + "value": [ + { + "name": "storageAccountName", + "value": "[parameters('app').storage]" + }, + { + "name": "containerName", + "value": "[parameters('container')]" + }, + { + "name": "files", + "value": "[string(parameters('files'))]" + } + ] + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + }, + "dependsOn": [ + "identity" + ] + } + }, + "outputs": { + "containerName": { + "type": "string", + "metadata": { + "description": "The name of the storage container." + }, + "value": "[parameters('container')]" + }, + "filesUploaded": { + "type": "int", + "metadata": { + "description": "The number of files uploaded to the storage container." + }, + "value": "[variables('fileCount')]" + }, + "identityId": { + "type": "string", + "metadata": { + "description": "Resource ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.id.value, '')]" + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Name of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.name.value, '')]" + }, + "identityPrincipalId": { + "type": "string", + "metadata": { + "description": "Principal ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.principalId.value, '')]" + } + } + } + }, + "dependsOn": [ + "appRegistration" + ] + }, + "uploadSettings": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core_Storage.UpdateSettings", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[reference('configContainer').outputs.identityName.value]" + }, + "scriptName": { + "value": "[format('{0}_uploadSettings', parameters('app').storage)]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "environmentVariables": { + "value": [ + { + "name": "ftkVersion", + "value": "[variables('finOpsToolkitVersion')]" + }, + { + "name": "scopes", + "value": "[join(parameters('scopesToMonitor'), '|')]" + }, + { + "name": "msexportRetentionInDays", + "value": "[string(parameters('msexportRetentionInDays'))]" + }, + { + "name": "ingestionRetentionInMonths", + "value": "[string(parameters('ingestionRetentionInMonths'))]" + }, + { + "name": "rawRetentionInDays", + "value": "[string(parameters('rawRetentionInDays'))]" + }, + { + "name": "finalRetentionInMonths", + "value": "[string(parameters('finalRetentionInMonths'))]" + }, + { + "name": "storageAccountName", + "value": "[parameters('app').storage]" + }, + { + "name": "containerName", + "value": "config" + } + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + }, + "dependsOn": [ + "appRegistration", + "configContainer" + ] + } + }, + "outputs": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Properties of the hub app." + }, + "value": "[parameters('app')]" + }, + "dataFactoryName": { + "type": "string", + "metadata": { + "description": "Name of the Data Factory." + }, + "value": "[parameters('app').dataFactory]" + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Name of the storage account created for the hub instance. This must be used when connecting FinOps toolkit Power BI reports to your data." + }, + "value": "[parameters('app').storage]" + }, + "storageUrlForPowerBI": { + "type": "string", + "metadata": { + "description": "URL to use when connecting custom Power BI reports to your data." + }, + "value": "[format('https://{0}.dfs.{1}/{2}', parameters('app').storage, environment().suffixes.storage, variables('INGESTION'))]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Object ID of the Data Factory managed identity. This will be needed when configuring managed exports." + }, + "value": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" + }, + "triggerManagerIdentityName": { + "type": "string", + "metadata": { + "description": "Name of the managed identity used to create and stop ADF triggers." + }, + "value": "[reference('appRegistration').outputs.triggerManagerIdentityName.value]" + } + } + } + } + }, + "cmExports": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.Exports", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[__bicep.newApp(variables('hub'), 'Microsoft.CostManagement', 'Exports')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "13738069470076156306" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + } + }, + "variables": { + "$fxv#0": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\":{\"name\":\"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingPeriodStartDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStartDate\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingPeriodEndDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEndDate\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountOwnerId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"SubscriptionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubscriptionId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"SubscriptionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubscriptionName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Date\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"Date\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Product\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Product\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PartNumber\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceFamily\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterCategory\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterSubCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterSubCategory\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterRegion\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Quantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Quantity\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"EffectivePrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectivePrice\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Cost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Cost\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"UnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UnitPrice\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceLocation\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceLocation\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AvailabilityZone\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AvailabilityZone\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ConsumedService\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedService\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceInfo1\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceInfo1\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceInfo2\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceInfo2\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AdditionalInfo\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AdditionalInfo\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSectionId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"InvoiceSection\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSection\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CostCenter\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"UnitOfMeasure\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"UnitOfMeasure\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceGroup\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceGroup\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ReservationId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ReservationName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ProductOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductOrderId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ProductOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductOrderName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"OfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"OfferId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"IsAzureCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"IsAzureCreditEligible\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PlanName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PlanName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ChargeType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeType\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Frequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Frequency\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PublisherType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherType\"}\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#1": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\":{\"name\":\"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingPeriodStartDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStartDate\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingPeriodEndDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEndDate\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountOwnerId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"SubscriptionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubscriptionId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"SubscriptionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubscriptionName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Date\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"Date\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Product\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Product\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PartNumber\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceFamily\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterCategory\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterSubCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterSubCategory\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterRegion\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Quantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Quantity\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"EffectivePrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectivePrice\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Cost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Cost\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"UnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UnitPrice\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceLocation\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceLocation\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AvailabilityZone\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AvailabilityZone\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ConsumedService\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedService\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceInfo1\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceInfo1\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceInfo2\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceInfo2\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AdditionalInfo\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AdditionalInfo\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSectionId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"InvoiceSection\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSection\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CostCenter\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"UnitOfMeasure\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"UnitOfMeasure\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceGroup\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceGroup\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ReservationId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ReservationName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ProductOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductOrderId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ProductOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductOrderName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"OfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"OfferId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"IsAzureCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"IsAzureCreditEligible\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PlanName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PlanName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ChargeType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeType\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Frequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Frequency\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PublisherType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherType\"}\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#10": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"SKU\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SKU\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Location\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Location\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CostWithNoReservedInstances\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CostWithNoReservedInstances\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"FirstUsageDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"FirstUsageDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InstanceFlexibilityRatio\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityRatio\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InstanceFlexibilityGroup\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityGroup\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"LookBackPeriod\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"LookBackPeriod\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"NetSavings\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"NetSavings\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"NormalizedSize\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"NormalizedSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RecommendedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"RecommendedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RecommendedQuantityNormalized\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"RecommendedQuantityNormalized\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Scope\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Scope\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuProperties\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuProperties\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"TotalCostWithReservedInstances\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"TotalCostWithReservedInstances\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#11": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"Cost With No ReservedInstances\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CostWithNoReservedInstancesJson\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"First UsageDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"FirstUsageDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Instance Flexibility Ratio\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityRatio\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Instance Flexibility Group\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityGroup\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Location\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Location\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"LookBackPeriod\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"LookBackPeriod\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterID\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Net Savings\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"NetSavingsJson\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Normalized Size\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"NormalizedSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Recommended Quantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"RecommendedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Recommended Quantity Normalized\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"RecommendedQuantityNormalized\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"scope\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Scope\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Sku Properties\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuProperties\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Total Cost With ReservedInstances\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"TotalCostWithReservedInstancesJson\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#12": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"AccountOwnerEmail\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountOwnerEmail\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Amount\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Amount\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ArmSkuName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ArmSkuName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingMonth\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingMonth\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Currency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Currency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CurrentEnrollmentId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CurrentEnrollmentId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"DepartmentName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"DepartmentName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Description\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Description\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EventDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EventDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EventType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"EventType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MonetaryCommitment\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"MonetaryCommitment\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Overage\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Overage\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PurchasingSubscriptionGuid\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PurchasingSubscriptionGuid\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PurchasingSubscriptionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PurchasingSubscriptionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PurchasingEnrollment\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PurchasingEnrollment\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Quantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Quantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Region\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Region\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#13": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"Amount\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Amount\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ArmSkuName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ArmSkuName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Currency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Currency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Description\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Description\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EventDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EventDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EventType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"EventType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Invoice\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Invoice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PurchasingSubscriptionGuid\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PurchasingSubscriptionGuid\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PurchasingSubscriptionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PurchasingSubscriptionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Quantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Quantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Region\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Region\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#2": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"BilledCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BilledCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CapacityReservationId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CapacityReservationId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CapacityReservationStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CapacityReservationStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ConsumedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectiveCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceIssuerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceIssuerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"PricingQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProviderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProviderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuMeter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuMeter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountOwnerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AmortizationClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AmortizationClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRateDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRateDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ContractedCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ContractedCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostAllocationRuleName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostAllocationRuleName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceIssuerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceIssuerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ListCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ListCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditApplied\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditApplied\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditRate\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingBlockSize\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PricingBlockSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingUnitDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingUnitDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceGroupName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceGroupName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServiceModel\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ServiceModel\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuIsCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_SkuIsCreditEligible\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOfferId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPlanName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPlanName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTerm\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTerm\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTier\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTier\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#3": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"BilledCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BilledCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CapacityReservationId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CapacityReservationId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CapacityReservationStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CapacityReservationStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ConsumedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectiveCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceIssuerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceIssuerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"PricingQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProviderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProviderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuMeter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuMeter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountOwnerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AmortizationClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AmortizationClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRateDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRateDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ContractedCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ContractedCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostAllocationRuleName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostAllocationRuleName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceIssuerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceIssuerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ListCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ListCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditApplied\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditApplied\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditRate\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingBlockSize\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PricingBlockSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingUnitDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingUnitDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceGroupName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceGroupName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServiceModel\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ServiceModel\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuIsCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_SkuIsCreditEligible\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOfferId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPlanName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPlanName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTerm\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTerm\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTier\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTier\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#4": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"BilledCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BilledCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ConsumedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectiveCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceIssuerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceIssuerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"PricingQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProviderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProviderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountOwnerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRateDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRateDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ContractedCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ContractedCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostAllocationRuleName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostAllocationRuleName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceIssuerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceIssuerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ListCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ListCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditApplied\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditApplied\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditRate\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingBlockSize\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PricingBlockSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingUnitDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingUnitDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceGroupName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceGroupName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuIsCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_SkuIsCreditEligible\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOfferId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTerm\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTerm\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTier\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTier\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#5": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"BilledCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BilledCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ConsumedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectiveCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceIssuerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceIssuerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"PricingQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProviderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProviderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountOwnerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRateDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRateDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ContractedCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ContractedCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostAllocationRuleName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostAllocationRuleName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceIssuerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceIssuerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ListCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ListCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditApplied\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditApplied\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditRate\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingBlockSize\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PricingBlockSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingUnitDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingUnitDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceGroupName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceGroupName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuIsCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_SkuIsCreditEligible\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOfferId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTerm\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTerm\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTier\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTier\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#6": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"AvailabilityZone\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AvailabilityZone\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BilledCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BilledCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectiveCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceIssuerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceIssuerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"PricingQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProviderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProviderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Region\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Region\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UsageQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UsageQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UsageUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"UsageUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountOwnerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRateDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRateDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ChargeId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ChargeId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostAllocationRuleName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostAllocationRuleName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceIssuerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceIssuerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_OnDemandCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_OnDemandCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_OnDemandCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_OnDemandCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_OnDemandUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_OnDemandUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditApplied\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditApplied\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingBlockSize\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PricingBlockSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingUnitDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingUnitDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceGroupName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceGroupName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuIsCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_SkuIsCreditEligible\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOfferId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTerm\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTerm\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTier\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTier\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#7": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"EnrollmentNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"EnrollmentNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterID\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterID\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterSubCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterSubCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Product\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Product\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuID\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuID\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProductID\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductID\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UnitOfMeasure\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"UnitOfMeasure\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveStartDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EffectiveStartDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveEndDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EffectiveEndDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BasePrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BasePrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MarketPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"MarketPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CurrencyCode\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CurrencyCode\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"IncludedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"IncludedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"OfferID\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"OfferID\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PriceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PriceType\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#8": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Product\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Product\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProductId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UnitOfMeasure\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"UnitOfMeasure\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterSubCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterSubCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"TierMinimumUnits\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"TierMinimumUnits\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveStartDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EffectiveStartDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveEndDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EffectiveEndDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BasePrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BasePrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MarketPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"MarketPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Currency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Currency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PriceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PriceType\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#9": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"InstanceFlexibilityGroup\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityGroup\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InstanceFlexibilityRatio\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityRatio\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InstanceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InstanceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Kind\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Kind\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservedHours\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ReservedHours\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"TotalReservedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"TotalReservedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UsageDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"UsageDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UsedHours\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UsedHours\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "CONFIG": "config", + "INGESTION": "ingestion", + "MSEXPORTS": "msexports", + "ingestionIdFileNameSeparator": "__", + "finOpsToolkitVersion": "13.0" + }, + "resources": { + "dataFactory::linkedService_storageAccount": { + "existing": true, + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').storage)]", + "dependsOn": [ + "appRegistration" + ] + }, + "dataFactory::dataset_config": { + "existing": true, + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, variables('CONFIG'))]", + "dependsOn": [ + "appRegistration" + ] + }, + "dataFactory::dataset_ingestion": { + "existing": true, + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, variables('INGESTION'))]", + "dependsOn": [ + "appRegistration" + ] + }, + "dataFactory::dataset_ingestion_files": { + "existing": true, + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_files', variables('INGESTION')))]", + "dependsOn": [ + "appRegistration" + ] + }, + "dataFactory::dataset_ingestion_manifest": { + "existing": true, + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ingestion_manifest')]", + "dependsOn": [ + "appRegistration" + ] + }, + "dataFactory::dataset_msexports_manifest": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'msexports_manifest')]", + "properties": { + "parameters": { + "fileName": { + "type": "String", + "defaultValue": "manifest.json" + }, + "folderPath": { + "type": "String", + "defaultValue": "[variables('MSEXPORTS')]" + } + }, + "type": "Json", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().fileName}", + "type": "Expression" + }, + "folderPath": { + "value": "@{dataset().folderPath}", + "type": "Expression" + } + } + }, + "linkedServiceName": { + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" + } + }, + "dependsOn": [ + "appRegistration" + ] + }, + "dataFactory::dataset_msexports": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, replace(format('{0}', variables('MSEXPORTS')), '-', '_'))]", + "properties": { + "parameters": { + "blobPath": { + "type": "String" + } + }, + "type": "DelimitedText", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().blobPath}", + "type": "Expression" + }, + "fileSystem": "[reference('exportContainer').outputs.containerName.value]" + }, + "columnDelimiter": ",", + "escapeChar": "\"", + "quoteChar": "\"", + "firstRowAsHeader": true + }, + "linkedServiceName": { + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" + } + }, + "dependsOn": [ + "appRegistration", + "exportContainer" + ] + }, + "dataFactory::dataset_msexports_gzip": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_gzip', variables('MSEXPORTS')))]", + "properties": { + "parameters": { + "blobPath": { + "type": "String" + } + }, + "type": "DelimitedText", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().blobPath}", + "type": "Expression" + }, + "fileSystem": "[variables('MSEXPORTS')]" + }, + "columnDelimiter": ",", + "escapeChar": "\"", + "quoteChar": "\"", + "firstRowAsHeader": true, + "compressionCodec": "Gzip" + }, + "linkedServiceName": { + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" + } + }, + "dependsOn": [ + "appRegistration" + ] + }, + "dataFactory::dataset_msexports_parquet": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_parquet', variables('MSEXPORTS')))]", + "properties": { + "parameters": { + "blobPath": { + "type": "String" + } + }, + "type": "Parquet", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().blobPath}", + "type": "Expression" + }, + "fileSystem": "[variables('MSEXPORTS')]" + } + }, + "linkedServiceName": { + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" + } + }, + "dependsOn": [ + "appRegistration" + ] + }, + "dataFactory::pipeline_ExecuteExportsETL": { + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_ExecuteETL', variables('MSEXPORTS')))]", + "properties": { + "activities": [ + { + "name": "Wait", + "description": "Files may not be available immediately after being created.", + "type": "Wait", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "waitTimeInSeconds": 60 + } + }, + { + "name": "Read Manifest", + "description": "Load the export manifest to determine the scope, dataset, and date range.", + "type": "Lookup", + "dependsOn": [ + { + "activity": "Wait", + "dependencyConditions": [ + "Completed" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "msexports_manifest", + "type": "DatasetReference", + "parameters": { + "fileName": { + "value": "@pipeline().parameters.fileName", + "type": "Expression" + }, + "folderPath": { + "value": "@pipeline().parameters.folderPath", + "type": "Expression" + } + } + } + } + }, + { + "name": "Set Has No Rows", + "description": "Check the row count ", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Read Manifest", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "hasNoRows", + "value": { + "value": "@or(equals(activity('Read Manifest').output.firstRow.blobCount, null), equals(activity('Read Manifest').output.firstRow.blobCount, 0))", + "type": "Expression" + } + } + }, + { + "name": "Set Export Dataset Type", + "description": "Save the dataset type from the export manifest.", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Read Manifest", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "exportDatasetType", + "value": { + "value": "@activity('Read Manifest').output.firstRow.exportConfig.type", + "type": "Expression" + } + } + }, + { + "name": "Set MCA Column", + "description": "Determines if the dataset schema has channel-specific columns and saves the column name that only exists in MCA to determine if it is an MCA dataset.", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Set Export Dataset Type", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "mcaColumnToCheck", + "value": { + "value": "@if(contains(createArray('pricesheet', 'reservationtransactions'), toLower(variables('exportDatasetType'))), 'BillingProfileId', if(equals(toLower(variables('exportDatasetType')), 'reservationrecommendations'), 'Net Savings', null))", + "type": "Expression" + } + } + }, + { + "name": "Set Export Dataset Version", + "description": "Save the dataset version from the export manifest.", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Read Manifest", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "exportDatasetVersion", + "value": { + "value": "@activity('Read Manifest').output.firstRow.exportConfig.dataVersion", + "type": "Expression" + } + } + }, + { + "name": "Detect Channel", + "description": "Determines what channel this export is from. Switch statement handles the different file types if the mcaColumnToCheck variable is set.", + "type": "Switch", + "dependsOn": [ + { + "activity": "Set Has No Rows", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set MCA Column", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set Export Dataset Version", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "on": { + "value": "@if(or(empty(variables('mcaColumnToCheck')), variables('hasNoRows')), 'ignore', last(array(split(activity('Read Manifest').output.firstRow.blobs[0].blobName, '.'))))", + "type": "Expression" + }, + "cases": [ + { + "value": "csv", + "activities": [ + { + "name": "Check for MCA Column in CSV", + "description": "Checks the dataset to determine if the applicable MCA-specific column exists.", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "DelimitedTextSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": false, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "DelimitedTextReadSettings" + } + }, + "dataset": { + "referenceName": "[replace(format('{0}', variables('MSEXPORTS')), '-', '_')]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@activity('Read Manifest').output.firstRow.blobs[0].blobName", + "type": "Expression" + } + } + } + } + }, + { + "name": "Set Schema File with Channel in CSV", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Check for MCA Column in CSV", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "schemaFile", + "value": { + "value": "@toLower(concat(variables('exportDatasetType'), '_', variables('exportDatasetVersion'), if(and(contains(activity('Check for MCA Column in CSV').output, 'firstRow'), contains(activity('Check for MCA Column in CSV').output.firstRow, variables('mcaColumnToCheck'))), '_mca', '_ea'), '.json'))", + "type": "Expression" + } + } + } + ] + }, + { + "value": "gz", + "activities": [ + { + "name": "Check for MCA Column in Gzip CSV", + "description": "Checks the dataset to determine if the applicable MCA-specific column exists.", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "DelimitedTextSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": false, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "DelimitedTextReadSettings" + } + }, + "dataset": { + "referenceName": "[format('{0}_gzip', variables('MSEXPORTS'))]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@activity('Read Manifest').output.firstRow.blobs[0].blobName", + "type": "Expression" + } + } + } + } + }, + { + "name": "Set Schema File with Channel in Gzip CSV", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Check for MCA Column in Gzip CSV", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "schemaFile", + "value": { + "value": "@toLower(concat(variables('exportDatasetType'), '_', variables('exportDatasetVersion'), if(and(contains(activity('Check for MCA Column in Gzip CSV').output, 'firstRow'), contains(activity('Check for MCA Column in Gzip CSV').output.firstRow, variables('mcaColumnToCheck'))), '_mca', '_ea'), '.json'))", + "type": "Expression" + } + } + } + ] + }, + { + "value": "parquet", + "activities": [ + { + "name": "Check for MCA Column in Parquet", + "description": "Checks the dataset to determine if the applicable MCA-specific column exists.", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "ParquetSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": false, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "ParquetReadSettings" + } + }, + "dataset": { + "referenceName": "[format('{0}_parquet', variables('MSEXPORTS'))]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@activity('Read Manifest').output.firstRow.blobs[0].blobName", + "type": "Expression" + } + } + } + } + }, + { + "name": "Set Schema File with Channel for Parquet", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Check for MCA Column in Parquet", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "schemaFile", + "value": { + "value": "@toLower(concat(variables('exportDatasetType'), '_', variables('exportDatasetVersion'), if(and(contains(activity('Check for MCA Column in Parquet').output, 'firstRow'), contains(activity('Check for MCA Column in Parquet').output.firstRow, variables('mcaColumnToCheck'))), '_mca', '_ea'), '.json'))", + "type": "Expression" + } + } + } + ] + } + ], + "defaultActivities": [ + { + "name": "Set Schema File", + "type": "SetVariable", + "dependsOn": [], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "schemaFile", + "value": { + "value": "@toLower(concat(variables('exportDatasetType'), '_', variables('exportDatasetVersion'), '.json'))", + "type": "Expression" + } + } + } + ] + } + }, + { + "name": "Set Scope", + "description": "Save the scope from the export manifest.", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Read Manifest", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "scope", + "value": { + "value": "@split(toLower(activity('Read Manifest').output.firstRow.exportConfig.resourceId), '/providers/microsoft.costmanagement/exports/')[0]", + "type": "Expression" + } + } + }, + { + "name": "Set Date", + "description": "Save the exported month from the export manifest.", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Read Manifest", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "date", + "value": { + "value": "@replace(substring(activity('Read Manifest').output.firstRow.runInfo.startDate, 0, 7), '-', '')", + "type": "Expression" + } + } + }, + { + "name": "Failed to Read Manifest", + "type": "Fail", + "dependsOn": [ + { + "activity": "Set Date", + "dependencyConditions": [ + "Failed" + ] + }, + { + "activity": "Set Export Dataset Type", + "dependencyConditions": [ + "Failed" + ] + }, + { + "activity": "Set Scope", + "dependencyConditions": [ + "Failed" + ] + }, + { + "activity": "Read Manifest", + "dependencyConditions": [ + "Failed" + ] + }, + { + "activity": "Set Export Dataset Version", + "dependencyConditions": [ + "Failed" + ] + }, + { + "activity": "Detect Channel", + "dependencyConditions": [ + "Failed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Failed to read the manifest file for this export run. Manifest path: ', pipeline().parameters.folderPath)", + "type": "Expression" + }, + "errorCode": "ManifestReadFailed" + } + }, + { + "name": "Check Schema", + "description": "Verify that the schema file exists in storage.", + "type": "GetMetadata", + "dependsOn": [ + { + "activity": "Set Scope", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set Date", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Detect Channel", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference", + "parameters": { + "fileName": { + "value": "@variables('schemaFile')", + "type": "Expression" + }, + "folderPath": "[format('{0}/schemas', reference('schemaFiles').outputs.containerName.value)]" + } + }, + "fieldList": [ + "exists" + ], + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + } + }, + { + "name": "Schema Not Found", + "type": "Fail", + "dependsOn": [ + { + "activity": "Check Schema", + "dependencyConditions": [ + "Failed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('The ', variables('schemaFile'), ' schema mapping file was not found. Please confirm version ', variables('exportDatasetVersion'), ' of the ', variables('exportDatasetType'), ' dataset is supported by this version of FinOps hubs. You may need to upgrade to a newer release. To add support for another dataset, you can create a custom mapping file.')", + "type": "Expression" + }, + "errorCode": "SchemaNotFound" + } + }, + { + "name": "Set Hub Dataset", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Set Export Dataset Type", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "hubDataset", + "value": { + "value": "@if(equals(toLower(variables('exportDatasetType')), 'focuscost'), 'Costs', if(equals(toLower(variables('exportDatasetType')), 'pricesheet'), 'Prices', if(equals(toLower(variables('exportDatasetType')), 'reservationdetails'), 'CommitmentDiscountUsage', if(equals(toLower(variables('exportDatasetType')), 'reservationrecommendations'), 'Recommendations', if(equals(toLower(variables('exportDatasetType')), 'reservationtransactions'), 'Transactions', if(equals(toLower(variables('exportDatasetType')), 'actualcost'), 'ActualCosts', if(equals(toLower(variables('exportDatasetType')), 'amortizedcost'), 'AmortizedCosts', toLower(variables('exportDatasetType')))))))))", + "type": "Expression" + } + } + }, + { + "name": "Set Destination Folder", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Check Schema", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set Hub Dataset", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "destinationFolder", + "value": { + "value": "@replace(concat(variables('hubDataset'),'/',substring(variables('date'), 0, 4),'/',substring(variables('date'), 4, 2),'/',toLower(variables('scope')), if(equals(variables('hubDataset'), 'Recommendations'), activity('Read Manifest').output.firstRow.exportConfig.exportName, '')),'//','/')", + "type": "Expression" + } + } + }, + { + "name": "For Each Blob", + "description": "Loop thru each exported file listed in the manifest.", + "type": "ForEach", + "dependsOn": [ + { + "activity": "Set Destination Folder", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@if(variables('hasNoRows'), json('[]'), activity('Read Manifest').output.firstRow.blobs)", + "type": "Expression" + }, + "batchCount": "[if(parameters('app').hub.options.privateRouting, 4, 30)]", + "isSequential": false, + "activities": [ + { + "name": "Execute", + "description": "Run the ingestion ETL pipeline.", + "type": "ExecutePipeline", + "dependsOn": [], + "policy": { + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "pipeline": { + "referenceName": "[format('{0}_ETL_{1}', variables('MSEXPORTS'), variables('INGESTION'))]", + "type": "PipelineReference" + }, + "waitOnCompletion": true, + "parameters": { + "blobPath": { + "value": "@item().blobName", + "type": "Expression" + }, + "destinationFolder": { + "value": "@variables('destinationFolder')", + "type": "Expression" + }, + "destinationFile": { + "value": "@last(array(split(replace(replace(item().blobName, '.gz', ''), '.csv', '.parquet'), '/')))", + "type": "Expression" + }, + "ingestionId": { + "value": "@activity('Read Manifest').output.firstRow.runInfo.runId", + "type": "Expression" + }, + "schemaFile": { + "value": "@variables('schemaFile')", + "type": "Expression" + }, + "exportDatasetType": { + "value": "@variables('exportDatasetType')", + "type": "Expression" + }, + "exportDatasetVersion": { + "value": "@variables('exportDatasetVersion')", + "type": "Expression" + } + } + } + } + ] + } + }, + { + "name": "Copy Manifest", + "description": "Copy the manifest to the ingestion container to trigger ADX ingestion", + "type": "Copy", + "dependsOn": [ + { + "activity": "For Each Blob", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "sink": { + "type": "JsonSink", + "storeSettings": { + "type": "AzureBlobFSWriteSettings" + }, + "formatSettings": { + "type": "JsonWriteSettings" + } + }, + "enableStaging": false + }, + "inputs": [ + { + "referenceName": "msexports_manifest", + "type": "DatasetReference", + "parameters": { + "fileName": "manifest.json", + "folderPath": { + "value": "@pipeline().parameters.folderPath", + "type": "Expression" + } + } + } + ], + "outputs": [ + { + "referenceName": "ingestion_manifest", + "type": "DatasetReference", + "parameters": { + "fileName": "manifest.json", + "folderPath": { + "value": "[format('@concat(''{0}/'', variables(''destinationFolder''))', variables('INGESTION'))]", + "type": "Expression" + } + } + } + ] + } + ], + "parameters": { + "folderPath": { + "type": "string" + }, + "fileName": { + "type": "string" + } + }, + "variables": { + "date": { + "type": "String" + }, + "destinationFolder": { + "type": "String" + }, + "exportDatasetType": { + "type": "String" + }, + "exportDatasetVersion": { + "type": "String" + }, + "hasNoRows": { + "type": "Boolean" + }, + "hubDataset": { + "type": "String" + }, + "mcaColumnToCheck": { + "type": "String" + }, + "schemaFile": { + "type": "String" + }, + "scope": { + "type": "String" + } + }, + "annotations": [ + "New export" + ] + }, + "dependsOn": [ + "appRegistration", + "dataFactory::dataset_msexports", + "dataFactory::dataset_msexports_gzip", + "dataFactory::dataset_msexports_manifest", + "dataFactory::dataset_msexports_parquet", + "dataFactory::pipeline_ToIngestion", + "schemaFiles" + ] + }, + "dataFactory::pipeline_ToIngestion": { + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_ETL_{1}', variables('MSEXPORTS'), variables('INGESTION')))]", + "properties": { + "activities": [ + { + "name": "Get Existing Parquet Files", + "description": "Get the previously ingested files so we can remove any older data. This is necessary to avoid data duplication in reports.", + "type": "GetMetadata", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "dataset": { + "referenceName": "[format('{0}_files', variables('INGESTION'))]", + "type": "DatasetReference", + "parameters": { + "folderPath": "@pipeline().parameters.destinationFolder" + } + }, + "fieldList": [ + "childItems" + ], + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "ParquetReadSettings" + } + } + }, + { + "name": "Filter Out Current Exports", + "description": "Remove existing files from the current export so those files do not get deleted.", + "type": "Filter", + "dependsOn": [ + { + "activity": "Get Existing Parquet Files", + "dependencyConditions": [ + "Completed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@if(contains(activity('Get Existing Parquet Files').output, 'childItems'), activity('Get Existing Parquet Files').output.childItems, json('[]'))", + "type": "Expression" + }, + "condition": { + "value": "[format('@and(endswith(item().name, ''.parquet''), not(startswith(item().name, concat(pipeline().parameters.ingestionId, ''{0}''))))', variables('ingestionIdFileNameSeparator'))]", + "type": "Expression" + } + } + }, + { + "name": "Load Schema Mappings", + "description": "Get schema mapping file to use for the CSV to parquet conversion.", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference", + "parameters": { + "fileName": { + "value": "@toLower(pipeline().parameters.schemaFile)", + "type": "Expression" + }, + "folderPath": "[format('{0}/schemas', variables('CONFIG'))]" + } + } + } + }, + { + "name": "Failed to Load Schema", + "type": "Fail", + "dependsOn": [ + { + "activity": "Load Schema Mappings", + "dependencyConditions": [ + "Failed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Unable to load the ', pipeline().parameters.schemaFile, ' schema file. Please confirm the schema and version are supported for FinOps hubs ingestion. Unsupported files will remain in the msexports container.')", + "type": "Expression" + }, + "errorCode": "SchemaLoadFailed" + } + }, + { + "name": "Set Additional Columns", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Load Schema Mappings", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "additionalColumns", + "value": { + "value": "@intersection(array(json(concat('[{\"name\":\"x_SourceProvider\",\"value\":\"Microsoft\"},{\"name\":\"x_SourceName\",\"value\":\"Cost Management\"},{\"name\":\"x_SourceType\",\"value\":\"', pipeline().parameters.exportDatasetVersion, '\"},{\"name\":\"x_SourceVersion\",\"value\":\"', pipeline().parameters.exportDatasetVersion, '\"}'))), activity('Load Schema Mappings').output.firstRow.additionalColumns)", + "type": "Expression" + } + } + }, + { + "name": "For Each Old File", + "description": "Loop thru each of the existing files from previous exports.", + "type": "ForEach", + "dependsOn": [ + { + "activity": "Convert to Parquet", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Filter Out Current Exports", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@activity('Filter Out Current Exports').output.Value", + "type": "Expression" + }, + "activities": [ + { + "name": "Delete Old Ingested File", + "description": "Delete the previously ingested files from older exports.", + "type": "Delete", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "dataset": { + "referenceName": "[variables('INGESTION')]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@concat(pipeline().parameters.destinationFolder, '/', item().name)", + "type": "Expression" + } + } + }, + "enableLogging": false, + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": false, + "enablePartitionDiscovery": false + } + } + } + ] + } + }, + { + "name": "Set Destination Path", + "type": "SetVariable", + "dependsOn": [], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "destinationPath", + "value": { + "value": "[format('@concat(pipeline().parameters.destinationFolder, ''/'', pipeline().parameters.ingestionId, ''{0}'', pipeline().parameters.destinationFile)', variables('ingestionIdFileNameSeparator'))]", + "type": "Expression" + } + } + }, + { + "name": "Convert to Parquet", + "description": "[format('Convert CSV to parquet and move the file to the {0} container.', variables('INGESTION'))]", + "type": "Switch", + "dependsOn": [ + { + "activity": "Set Destination Path", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Load Schema Mappings", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set Additional Columns", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "on": { + "value": "@last(array(split(pipeline().parameters.blobPath, '.')))", + "type": "Expression" + }, + "cases": [ + { + "value": "csv", + "activities": [ + { + "name": "Convert CSV File", + "type": "Copy", + "dependsOn": [], + "policy": { + "timeout": "0.00:10:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "DelimitedTextSource", + "additionalColumns": { + "value": "@variables('additionalColumns')", + "type": "Expression" + }, + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "DelimitedTextReadSettings" + } + }, + "sink": { + "type": "ParquetSink", + "storeSettings": { + "type": "AzureBlobFSWriteSettings" + }, + "formatSettings": { + "type": "ParquetWriteSettings", + "fileExtension": ".parquet" + } + }, + "enableStaging": false, + "parallelCopies": 1, + "validateDataConsistency": false, + "translator": { + "value": "@activity('Load Schema Mappings').output.firstRow.translator", + "type": "Expression" + } + }, + "inputs": [ + { + "referenceName": "[replace(format('{0}', variables('MSEXPORTS')), '-', '_')]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@pipeline().parameters.blobPath", + "type": "Expression" + } + } + } + ], + "outputs": [ + { + "referenceName": "[variables('INGESTION')]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@variables('destinationPath')", + "type": "Expression" + } + } + } + ] + } + ] + }, + { + "value": "gz", + "activities": [ + { + "name": "Convert GZip CSV File", + "type": "Copy", + "dependsOn": [], + "policy": { + "timeout": "0.00:10:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "DelimitedTextSource", + "additionalColumns": { + "value": "@variables('additionalColumns')", + "type": "Expression" + }, + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "DelimitedTextReadSettings" + } + }, + "sink": { + "type": "ParquetSink", + "storeSettings": { + "type": "AzureBlobFSWriteSettings" + }, + "formatSettings": { + "type": "ParquetWriteSettings", + "fileExtension": ".parquet" + } + }, + "enableStaging": false, + "parallelCopies": 1, + "validateDataConsistency": false, + "translator": { + "value": "@activity('Load Schema Mappings').output.firstRow.translator", + "type": "Expression" + } + }, + "inputs": [ + { + "referenceName": "[format('{0}_gzip', variables('MSEXPORTS'))]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@pipeline().parameters.blobPath", + "type": "Expression" + } + } + } + ], + "outputs": [ + { + "referenceName": "[variables('INGESTION')]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@variables('destinationPath')", + "type": "Expression" + } + } + } + ] + } + ] + }, + { + "value": "parquet", + "activities": [ + { + "name": "Move Parquet File", + "type": "Copy", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "ParquetSource", + "additionalColumns": { + "value": "@variables('additionalColumns')", + "type": "Expression" + }, + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "ParquetReadSettings" + } + }, + "sink": { + "type": "ParquetSink", + "storeSettings": { + "type": "AzureBlobFSWriteSettings" + }, + "formatSettings": { + "type": "ParquetWriteSettings", + "fileExtension": ".parquet" + } + }, + "enableStaging": false, + "parallelCopies": 1, + "validateDataConsistency": false + }, + "inputs": [ + { + "referenceName": "[format('{0}_parquet', variables('MSEXPORTS'))]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@pipeline().parameters.blobPath", + "type": "Expression" + } + } + } + ], + "outputs": [ + { + "referenceName": "[variables('INGESTION')]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@variables('destinationPath')", + "type": "Expression" + } + } + } + ] + } + ] + } + ], + "defaultActivities": [ + { + "name": "Unsupported File Type", + "type": "Fail", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Unable to ingest the specified export file because the file type is not supported. File: ', pipeline().parameters.blobPath)", + "type": "Expression" + }, + "errorCode": "UnsupportedExportFileType" + } + } + ] + } + }, + { + "name": "Read Hub Config", + "description": "Read the hub config to determine if the export should be retained.", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": false, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference", + "parameters": { + "fileName": "settings.json", + "folderPath": "[variables('CONFIG')]" + } + } + } + }, + { + "name": "If Not Retaining Exports", + "description": "If the msexports retention period <= 0, delete the source file. The main reason to keep the source file is to allow for troubleshooting and reprocessing in the future.", + "type": "IfCondition", + "dependsOn": [ + { + "activity": "Convert to Parquet", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Read Hub Config", + "dependencyConditions": [ + "Completed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "expression": { + "value": "@lessOrEquals(coalesce(activity('Read Hub Config').output.firstRow.retention.msexports.days, 0), 0)", + "type": "Expression" + }, + "ifTrueActivities": [ + { + "name": "Delete Source File", + "description": "Delete the exported data file to keep storage costs down. This file is not referenced by any reporting systems.", + "type": "Delete", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "dataset": { + "referenceName": "[format('{0}_parquet', variables('MSEXPORTS'))]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@pipeline().parameters.blobPath", + "type": "Expression" + } + } + }, + "enableLogging": false, + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + } + } + } + ] + } + } + ], + "parameters": { + "blobPath": { + "type": "String" + }, + "destinationFile": { + "type": "string" + }, + "destinationFolder": { + "type": "string" + }, + "ingestionId": { + "type": "string" + }, + "schemaFile": { + "type": "string" + }, + "exportDatasetType": { + "type": "string" + }, + "exportDatasetVersion": { + "type": "string" + } + }, + "variables": { + "additionalColumns": { + "type": "Array" + }, + "destinationPath": { + "type": "String" + } + } + }, + "dependsOn": [ + "appRegistration", + "dataFactory::dataset_msexports", + "dataFactory::dataset_msexports_gzip", + "dataFactory::dataset_msexports_parquet" + ] + }, + "dataFactory": { + "existing": true, + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "dependsOn": [ + "appRegistration" + ] + }, + "appRegistration": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.Exports_Register", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "version": { + "value": "[variables('finOpsToolkitVersion')]" + }, + "features": { + "value": [ + "Storage", + "DataFactory" + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "8267413868206701537" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppFeature": { + "type": "string", + "allowedValues": [ + "DataFactory", + "KeyVault", + "Storage" + ], + "metadata": { + "description": "FinOps hub app features.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "version": { + "type": "string", + "metadata": { + "description": "Required. Version number of the FinOps hub app." + } + }, + "features": { + "type": "array", + "items": { + "$ref": "#/definitions/HubAppFeature" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which features the app requires. Allowed values: \"DataFactory\", \"KeyVault\", \"Storage\". Default: [] (none)." + } + }, + "storageRoles": { + "type": "array", + "items": { + "type": "string" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which RBAC roles the Data Factory identity needs on the storage account, if created. This is in addition to Storage Blob Data Contributor for reading and managing content. Default: [] (none)." + } + }, + "telemetryString": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n<#\r\n.SYNOPSIS\r\nManages Azure Data Factory triggers and pipelines during FinOps hub deployment.\r\n\r\n.DESCRIPTION\r\nThis script is called by Bicep deployment scripts to start/stop Data Factory triggers\r\nand optionally run pipelines. It handles two types of triggers:\r\n\r\n1. Schedule triggers - Can be started/stopped directly\r\n2. BlobEventsTriggers - Require Event Grid subscription management before start/stop\r\n\r\nBlobEventsTriggers use Event Grid to listen for storage blob events. Before stopping,\r\nthe Event Grid subscription must be removed (unsubscribed). Before starting, the\r\nsubscription must be added and fully provisioned. This script handles this automatically\r\nby detecting BlobEventsTriggers via the BlobPathBeginsWith property.\r\n\r\nThe script uses retry logic with linear backoff to handle transient API failures and\r\nwait for Event Grid subscription provisioning/deprovisioning to complete.\r\n\r\n.PARAMETER DataFactoryResourceGroup\r\nThe resource group containing the Data Factory.\r\n\r\n.PARAMETER DataFactoryName\r\nThe name of the Data Factory instance.\r\n\r\n.PARAMETER Pipelines\r\nPipe-delimited list of pipeline names to run (e.g., \"pipeline1|pipeline2\").\r\n\r\n.PARAMETER StartTriggers\r\nSwitch to start all stopped triggers (with Event Grid subscription for blob triggers).\r\n\r\n.PARAMETER StopTriggers\r\nSwitch to stop all running triggers (with Event Grid unsubscription for blob triggers).\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StopTriggers\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StartTriggers\r\n#>\r\n\r\nparam(\r\n [string] $DataFactoryResourceGroup,\r\n [string] $DataFactoryName,\r\n [string] $Pipelines = \"\",\r\n [switch] $StartTriggers,\r\n [switch] $StopTriggers\r\n)\r\n\r\n$MAX_RETRIES = 20\r\n$DeploymentScriptOutputs = @{}\r\n\r\nfunction Write-Log($Message)\r\n{\r\n Write-Output \"$(Get-Date -Format 'HH:mm:ss') $Message\"\r\n}\r\n\r\nfunction Invoke-WithRetry([scriptblock]$Action, [string]$Name, [int]$Delay = 5)\r\n{\r\n for ($i = 1; $i -le $MAX_RETRIES; $i++)\r\n {\r\n try { return & $Action }\r\n catch\r\n {\r\n Write-Log \"$Name failed (attempt $i/${MAX_RETRIES}): $($_.Exception.Message)\"\r\n if ($i -eq $MAX_RETRIES) { throw }\r\n Start-Sleep -Seconds ($Delay * $i)\r\n }\r\n }\r\n}\r\n\r\nfunction Set-BlobTriggerSubscription([string]$TriggerName, [switch]$Subscribe)\r\n{\r\n $targetStatus = if ($Subscribe) { 'Enabled' } else { 'Disabled' }\r\n $action = if ($Subscribe) { 'Subscribing' } else { 'Unsubscribing' }\r\n\r\n Write-Log \"$action $TriggerName to events...\"\r\n Invoke-WithRetry -Name \"$action $TriggerName\" -Delay 5 -Action {\r\n if ($Subscribe)\r\n {\r\n Add-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n else\r\n {\r\n Remove-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n\r\n $status = Get-AzDataFactoryV2TriggerSubscriptionStatus `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName\r\n if ($status.Status -ne $targetStatus)\r\n {\r\n throw \"Subscription status is $($status.Status), expected $targetStatus\"\r\n }\r\n }\r\n}\r\n\r\nif ($StartTriggers -or $StopTriggers)\r\n{\r\n $triggers = Invoke-WithRetry -Name \"Get triggers\" -Action {\r\n Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n | Where-Object {\r\n ($StartTriggers -and $_.Properties.RuntimeState -ne \"Started\") `\r\n -or ($StopTriggers -and $_.Properties.RuntimeState -ne \"Stopped\")\r\n }\r\n }\r\n\r\n Write-Log \"Found $($triggers.Count) trigger(s) to $(if ($StartTriggers) { 'start' } else { 'stop' })\"\r\n\r\n $triggers | ForEach-Object {\r\n $triggerName = $_.Name\r\n $isBlobTrigger = $null -ne $_.Properties.BlobPathBeginsWith\r\n\r\n if ($StopTriggers)\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName }\r\n Write-Log \"Stopping trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Stop $triggerName\" -Action {\r\n Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n else\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName -Subscribe }\r\n Write-Log \"Starting trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Start $triggerName\" -Action {\r\n Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n\r\n Invoke-WithRetry -Name \"Wait for $triggerName\" -Action {\r\n $state = (Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName).Properties.RuntimeState\r\n $expected = if ($StartTriggers) { 'Started' } else { 'Stopped' }\r\n if ($state -ne $expected) { throw \"Trigger is $state, expected $expected\" }\r\n }\r\n\r\n Write-Log \"...done\"\r\n $DeploymentScriptOutputs[$triggerName] = $true\r\n }\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($Pipelines))\r\n{\r\n $Pipelines.Split('|') | ForEach-Object {\r\n Write-Log \"Running pipeline $_...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", + "usesDataFactory": "[contains(parameters('features'), 'DataFactory')]", + "usesKeyVault": "[contains(parameters('features'), 'KeyVault')]", + "usesStorage": "[contains(parameters('features'), 'Storage')]", + "telemetryId": "[format('ftk-hubapp-{0}{1}{2}', parameters('app').id, if(empty(parameters('telemetryString')), '', '_'), parameters('telemetryString'))]", + "telemetryProps": { + "mode": "Incremental", + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "[format('FTK: {0}', parameters('app').id)]", + "version": "[parameters('version')]" + } + }, + "resources": [] + } + }, + "autoStartRbacRoles": [ + "673868aa-7521-48a0-acc6-0f60742d39f5" + ], + "factoryStorageRoles": "[union(parameters('storageRoles'), createArray('17d1049b-9a84-46fb-8f53-869881c3d3ab', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe', 'acdd72a7-3385-48ef-bd42-f606fba81ae7'))]", + "storageInfrastructureEncryptionProperties": "[if(not(parameters('app').hub.options.storageInfrastructureEncryption), createObject(), createObject('encryption', createObject('keySource', 'Microsoft.Storage', 'requireInfrastructureEncryption', parameters('app').hub.options.storageInfrastructureEncryption)))]" + }, + "resources": { + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').storage)]", + "properties": { + "name": "[parameters('app').storage]", + "groupId": "dfs", + "privateLinkResourceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "fqdns": [ + "[reference('storageAccount').primaryEndpoints.dfs]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "storageAccount" + ] + }, + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').keyVault)]", + "properties": { + "name": "[parameters('app').keyVault]", + "groupId": "vault", + "privateLinkResourceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "fqdns": [ + "[reference('keyVault').vaultUri]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "keyVault" + ] + }, + "dataFactory::managedVirtualNetwork": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'default')]", + "properties": {}, + "dependsOn": [ + "dataFactory" + ] + }, + "dataFactory::managedIntegrationRuntime": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/integrationRuntimes", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ManagedIntegrationRuntime')]", + "properties": { + "type": "Managed", + "managedVirtualNetwork": { + "referenceName": "default", + "type": "ManagedVirtualNetworkReference" + }, + "typeProperties": { + "computeProperties": { + "location": "[parameters('app').hub.location]", + "dataFlowProperties": { + "computeType": "General", + "coreCount": 8, + "timeToLive": 10, + "cleanup": false, + "customProperties": [] + }, + "copyComputeScaleProperties": { + "dataIntegrationUnit": 16, + "timeToLive": 30 + }, + "pipelineExternalComputeScaleProperties": { + "timeToLive": 30, + "numberOfPipelineNodes": 1, + "numberOfExternalNodes": 1 + } + } + } + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedVirtualNetwork" + ] + }, + "dataFactory::linkedService_keyVault": { + "condition": "[and(variables('usesDataFactory'), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').keyVault)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureKeyVault", + "typeProperties": { + "baseUrl": "[reference(format('Microsoft.KeyVault/vaults/{0}', parameters('app').keyVault), '2023-02-01').vaultUri]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "keyVault" + ] + }, + "dataFactory::linkedService_storageAccount": { + "condition": "[and(variables('usesDataFactory'), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').storage)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureBlobFS", + "typeProperties": { + "url": "[reference(format('Microsoft.Storage/storageAccounts/{0}', parameters('app').storage), '2021-08-01').primaryEndpoints.dfs]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "storageAccount" + ] + }, + "storageAccount::blobService": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]", + "dependsOn": [ + "storageAccount" + ] + }, + "blobEndpoint::blobPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "blobEndpoint" + ] + }, + "dfsEndpoint::dfsPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.dfs.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "dfsEndpoint" + ] + }, + "keyVaultPrivateDnsZone::keyVaultPrivateDnsZoneLink": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), format('{0}-link', replace(format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "properties": { + "virtualNetwork": { + "id": "[parameters('app').hub.routing.networkId]" + }, + "registrationEnabled": false + }, + "dependsOn": [ + "keyVaultPrivateDnsZone" + ] + }, + "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')))]" + } + } + ] + }, + "dependsOn": [ + "keyVaultEndpoint", + "keyVaultPrivateDnsZone" + ] + }, + "appTelemetry": { + "condition": "[parameters('app').hub.options.enableTelemetry]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "[if(lessOrEquals(length(variables('telemetryId')), 64), variables('telemetryId'), substring(variables('telemetryId'), 0, 64))]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Resources/deployments')]", + "properties": "[variables('telemetryProps')]" + }, + "dataFactory": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.DataFactory/factories')]", + "identity": { + "type": "SystemAssigned" + }, + "properties": { + "globalConfigurations": { + "PipelineBillingEnabled": "true" + } + } + }, + "storageRoleAssignments": { + "copy": { + "name": "storageRoleAssignments", + "count": "[length(variables('factoryStorageRoles'))]" + }, + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "name": "[guid(resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage), variables('factoryStorageRoles')[copyIndex()], resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('factoryStorageRoles')[copyIndex()])]", + "principalId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "storageAccount" + ] + }, + "triggerManagerIdentity": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[format('{0}_triggerManager', parameters('app').dataFactory)]", + "location": "[parameters('app').hub.location]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "dependsOn": [ + "dataFactory" + ] + }, + "triggerManagerRoleAssignments": { + "copy": { + "name": "triggerManagerRoleAssignments", + "count": "[length(variables('autoStartRbacRoles'))]" + }, + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]", + "name": "[guid(resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory), variables('autoStartRbacRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', format('{0}_triggerManager', parameters('app').dataFactory)))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('autoStartRbacRoles')[copyIndex()])]", + "principalId": "[reference('triggerManagerIdentity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "triggerManagerIdentity" + ] + }, + "storageAccount": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]", + "location": "[parameters('app').hub.location]", + "sku": { + "name": "[parameters('app').hub.options.storageSku]" + }, + "kind": "BlockBlobStorage", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Storage/storageAccounts')]", + "properties": "[shallowMerge(createArray(variables('storageInfrastructureEncryptionProperties'), createObject('supportsHttpsTrafficOnly', true(), 'allowSharedKeyAccess', true(), 'isHnsEnabled', true(), 'minimumTlsVersion', 'TLS1_2', 'allowBlobPublicAccess', false(), 'publicNetworkAccess', 'Enabled', 'networkAcls', createObject('bypass', 'AzureServices', 'defaultAction', if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')))))]" + }, + "blobPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]" + }, + "blobEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-blob-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "blobLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "blob" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "dfsPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]" + }, + "dfsEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-dfs-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "dfsLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "dfs" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "keyVault": { + "condition": "[variables('usesKeyVault')]", + "type": "Microsoft.KeyVault/vaults", + "apiVersion": "2023-02-01", + "name": "[parameters('app').keyVault]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.KeyVault/vaults')]", + "properties": { + "sku": { + "name": "[parameters('app').hub.options.keyVaultSku]", + "family": "A" + }, + "enabledForDeployment": true, + "enabledForTemplateDeployment": true, + "enabledForDiskEncryption": true, + "enableSoftDelete": true, + "softDeleteRetentionInDays": 90, + "enablePurgeProtection": "[if(parameters('app').hub.options.keyVaultEnablePurgeProtection, true(), null())]", + "enableRbacAuthorization": false, + "createMode": "default", + "tenantId": "[subscription().tenantId]", + "accessPolicies": [ + { + "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "tenantId": "[subscription().tenantId]", + "permissions": { + "secrets": [ + "get" + ] + } + } + ], + "networkAcls": { + "bypass": "AzureServices", + "defaultAction": "[if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')]" + } + }, + "dependsOn": [ + "dataFactory" + ] + }, + "keyVaultPrivateDnsZone": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones')]", + "properties": {} + }, + "keyVaultEndpoint": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-ep', parameters('app').keyVault)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.keyVault]" + }, + "privateLinkServiceConnections": [ + { + "name": "keyVaultLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "groupIds": [ + "vault" + ] + } + } + ] + }, + "dependsOn": [ + "keyVault" + ] + }, + "getKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint", + "getStoragePrivateEndpointConnections", + "keyVault" + ] + }, + "approveKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + }, + "privateEndpointConnections": { + "value": "[reference('getKeyVaultPrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getKeyVaultPrivateEndpointConnections", + "keyVault" + ] + }, + "getStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint", + "stopTriggers", + "storageAccount" + ] + }, + "approveStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + }, + "privateEndpointConnections": { + "value": "[reference('getStoragePrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getStoragePrivateEndpointConnections", + "storageAccount" + ] + }, + "stopTriggers": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.{1}_ADF.StopTriggers', parameters('app').publisher, parameters('app').name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "arguments": { + "value": "[join(createArray(format('-DataFactoryResourceGroup \"{0}\"', resourceGroup().name), format('-DataFactoryName \"{0}\"', parameters('app').dataFactory), '-StopTriggers'), ' ')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + }, + "dependsOn": [ + "appTelemetry", + "dataFactory", + "triggerManagerIdentity", + "triggerManagerRoleAssignments" + ] + } + }, + "outputs": { + "dataFactoryId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Data Factory instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]" + }, + "keyVaultId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Key Vault instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]" + }, + "storageAccountId": { + "type": "string", + "metadata": { + "description": "Resource ID of the storage account instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Principal ID for the managed identity used by Data Factory." + }, + "value": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" + }, + "triggerManagerIdentityName": { + "type": "string", + "metadata": { + "description": "Name of the managed identity used to create and stop ADF triggers." + }, + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + } + } + } + } + }, + "schemaFiles": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.Exports_Storage.SchemaFiles", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "container": { + "value": "config" + }, + "files": { + "value": { + "schemas/actualcost_c360-2025-04.json": "[variables('$fxv#0')]", + "schemas/amortizedcost_c360-2025-04.json": "[variables('$fxv#1')]", + "schemas/focuscost_1.2.json": "[variables('$fxv#2')]", + "schemas/focuscost_1.2-preview.json": "[variables('$fxv#3')]", + "schemas/focuscost_1.0r2.json": "[variables('$fxv#4')]", + "schemas/focuscost_1.0.json": "[variables('$fxv#5')]", + "schemas/focuscost_1.0-preview(v1).json": "[variables('$fxv#6')]", + "schemas/pricesheet_2023-05-01_ea.json": "[variables('$fxv#7')]", + "schemas/pricesheet_2023-05-01_mca.json": "[variables('$fxv#8')]", + "schemas/reservationdetails_2023-03-01.json": "[variables('$fxv#9')]", + "schemas/reservationrecommendations_2023-05-01_ea.json": "[variables('$fxv#10')]", + "schemas/reservationrecommendations_2023-05-01_mca.json": "[variables('$fxv#11')]", + "schemas/reservationtransactions_2023-05-01_ea.json": "[variables('$fxv#12')]", + "schemas/reservationtransactions_2023-05-01_mca.json": "[variables('$fxv#13')]" + } + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6646280599480816180" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app that storage is getting updated for." + } + }, + "container": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage container to create or update." + } + }, + "files": { + "type": "object", + "defaultValue": {}, + "metadata": { + "description": "Optional. Dictionary of key/value pairs for the files to upload to the specified container. The key is the target path under the container and the value is the contents of the file. Default: {} (no files to upload)." + } + }, + "forceCreateBlobManagerIdentity": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Indicates whether to create the blob manager user assigned identity even if files are not being uploaded. Default: false." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n# Get storage context\r\n$storageContext = @{\r\n Context = New-AzStorageContext -StorageAccountName $env:storageAccountName -UseConnectedAccount\r\n Container = $env:containerName\r\n}\r\n\r\n# Uploading files\r\n$files = $env:files | ConvertFrom-Json -Depth 10\r\nWrite-Output \"Uploading ${$files.PSObject.Properties.Count} files...\"\r\n$files.PSObject.Properties | ForEach-Object {\r\n $filePath = $_.Name\r\n $tempPath = \"./$($filePath -replace \"/\", \"_\")\"\r\n Write-Output \" Uploading $filePath...\"\r\n $_.Value | Out-File $tempPath\r\n Set-AzStorageBlobContent @storageContext -File $tempPath -Blob $filePath -Force | Out-Null\r\n}\r\n", + "fileCount": "[length(items(parameters('files')))]", + "hasFiles": "[greater(variables('fileCount'), 0)]" + }, + "resources": { + "storageAccount::blobService::targetContainer": { + "type": "Microsoft.Storage/storageAccounts/blobServices/containers", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}/{2}', parameters('app').storage, 'default', parameters('container'))]", + "properties": { + "publicAccess": "None", + "metadata": {} + } + }, + "storageAccount::blobService": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]" + }, + "storageAccount": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]" + }, + "identity": { + "condition": "[or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity'))]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.Identity', deployment().name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_blobManager', parameters('app').storage)]" + }, + "roleAssignmentResourceId": { + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "roles": { + "value": [ + "ba92f5b4-2d11-453d-a403-e96b0029c9fe", + "69566ab7-960f-475b-8e7c-b3118f30c6bd" + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "18138592202204453545" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the identity is associated with." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the user assigned identity." + } + }, + "roleAssignmentResourceId": { + "type": "string", + "metadata": { + "description": "Required. Resource ID of the resource access is being granted for." + } + }, + "roles": { + "type": "array", + "items": { + "type": "string" + }, + "metadata": { + "description": "Required. List of RBAC role assignment GUIDs." + } + } + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.ManagedIdentity/userAssignedIdentities')]", + "location": "[parameters('app').hub.location]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(parameters('roles'))]" + }, + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "name": "[guid(parameters('roleAssignmentResourceId'), parameters('roles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', parameters('roles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + } + }, + "outputs": { + "id": { + "type": "string", + "metadata": { + "description": "Resource ID of the user assigned identity." + }, + "value": "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName'))]" + }, + "name": { + "type": "string", + "metadata": { + "description": "Name of the user assigned identity." + }, + "value": "[parameters('identityName')]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Principal ID of the user assigned identity." + }, + "value": "[reference('identity').principalId]" + } + } + } + } + }, + "uploadFiles": { + "condition": "[variables('hasFiles')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.Upload', deployment().name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[reference('identity').outputs.name.value]" + }, + "environmentVariables": { + "value": [ + { + "name": "storageAccountName", + "value": "[parameters('app').storage]" + }, + { + "name": "containerName", + "value": "[parameters('container')]" + }, + { + "name": "files", + "value": "[string(parameters('files'))]" + } + ] + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + }, + "dependsOn": [ + "identity" + ] + } + }, + "outputs": { + "containerName": { + "type": "string", + "metadata": { + "description": "The name of the storage container." + }, + "value": "[parameters('container')]" + }, + "filesUploaded": { + "type": "int", + "metadata": { + "description": "The number of files uploaded to the storage container." + }, + "value": "[variables('fileCount')]" + }, + "identityId": { + "type": "string", + "metadata": { + "description": "Resource ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.id.value, '')]" + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Name of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.name.value, '')]" + }, + "identityPrincipalId": { + "type": "string", + "metadata": { + "description": "Principal ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.principalId.value, '')]" + } + } + } + }, + "dependsOn": [ + "appRegistration" + ] + }, + "exportContainer": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.Exports_Storage.ExportContainer", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "container": { + "value": "[variables('MSEXPORTS')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6646280599480816180" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app that storage is getting updated for." + } + }, + "container": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage container to create or update." + } + }, + "files": { + "type": "object", + "defaultValue": {}, + "metadata": { + "description": "Optional. Dictionary of key/value pairs for the files to upload to the specified container. The key is the target path under the container and the value is the contents of the file. Default: {} (no files to upload)." + } + }, + "forceCreateBlobManagerIdentity": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Indicates whether to create the blob manager user assigned identity even if files are not being uploaded. Default: false." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n# Get storage context\r\n$storageContext = @{\r\n Context = New-AzStorageContext -StorageAccountName $env:storageAccountName -UseConnectedAccount\r\n Container = $env:containerName\r\n}\r\n\r\n# Uploading files\r\n$files = $env:files | ConvertFrom-Json -Depth 10\r\nWrite-Output \"Uploading ${$files.PSObject.Properties.Count} files...\"\r\n$files.PSObject.Properties | ForEach-Object {\r\n $filePath = $_.Name\r\n $tempPath = \"./$($filePath -replace \"/\", \"_\")\"\r\n Write-Output \" Uploading $filePath...\"\r\n $_.Value | Out-File $tempPath\r\n Set-AzStorageBlobContent @storageContext -File $tempPath -Blob $filePath -Force | Out-Null\r\n}\r\n", + "fileCount": "[length(items(parameters('files')))]", + "hasFiles": "[greater(variables('fileCount'), 0)]" + }, + "resources": { + "storageAccount::blobService::targetContainer": { + "type": "Microsoft.Storage/storageAccounts/blobServices/containers", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}/{2}', parameters('app').storage, 'default', parameters('container'))]", + "properties": { + "publicAccess": "None", + "metadata": {} + } + }, + "storageAccount::blobService": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]" + }, + "storageAccount": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]" + }, + "identity": { + "condition": "[or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity'))]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.Identity', deployment().name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_blobManager', parameters('app').storage)]" + }, + "roleAssignmentResourceId": { + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "roles": { + "value": [ + "ba92f5b4-2d11-453d-a403-e96b0029c9fe", + "69566ab7-960f-475b-8e7c-b3118f30c6bd" + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "18138592202204453545" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the identity is associated with." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the user assigned identity." + } + }, + "roleAssignmentResourceId": { + "type": "string", + "metadata": { + "description": "Required. Resource ID of the resource access is being granted for." + } + }, + "roles": { + "type": "array", + "items": { + "type": "string" + }, + "metadata": { + "description": "Required. List of RBAC role assignment GUIDs." + } + } + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.ManagedIdentity/userAssignedIdentities')]", + "location": "[parameters('app').hub.location]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(parameters('roles'))]" + }, + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "name": "[guid(parameters('roleAssignmentResourceId'), parameters('roles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', parameters('roles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + } + }, + "outputs": { + "id": { + "type": "string", + "metadata": { + "description": "Resource ID of the user assigned identity." + }, + "value": "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName'))]" + }, + "name": { + "type": "string", + "metadata": { + "description": "Name of the user assigned identity." + }, + "value": "[parameters('identityName')]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Principal ID of the user assigned identity." + }, + "value": "[reference('identity').principalId]" + } + } + } + } + }, + "uploadFiles": { + "condition": "[variables('hasFiles')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.Upload', deployment().name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[reference('identity').outputs.name.value]" + }, + "environmentVariables": { + "value": [ + { + "name": "storageAccountName", + "value": "[parameters('app').storage]" + }, + { + "name": "containerName", + "value": "[parameters('container')]" + }, + { + "name": "files", + "value": "[string(parameters('files'))]" + } + ] + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + }, + "dependsOn": [ + "identity" + ] + } + }, + "outputs": { + "containerName": { + "type": "string", + "metadata": { + "description": "The name of the storage container." + }, + "value": "[parameters('container')]" + }, + "filesUploaded": { + "type": "int", + "metadata": { + "description": "The number of files uploaded to the storage container." + }, + "value": "[variables('fileCount')]" + }, + "identityId": { + "type": "string", + "metadata": { + "description": "Resource ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.id.value, '')]" + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Name of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.name.value, '')]" + }, + "identityPrincipalId": { + "type": "string", + "metadata": { + "description": "Principal ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.principalId.value, '')]" + } + } + } + }, + "dependsOn": [ + "appRegistration" + ] + }, + "trigger_ExportManifestAdded": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.Exports_ADF.ExportManifestTrigger", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "dataFactoryName": { + "value": "[parameters('app').dataFactory]" + }, + "triggerName": { + "value": "[format('{0}_ManifestAdded', variables('MSEXPORTS'))]" + }, + "pipelineName": { + "value": "[format('{0}_ExecuteETL', variables('MSEXPORTS'))]" + }, + "pipelineParameters": { + "value": { + "folderPath": "@triggerBody().folderPath", + "fileName": "@triggerBody().fileName" + } + }, + "storageAccountName": { + "value": "[parameters('app').storage]" + }, + "storageContainer": { + "value": "[variables('MSEXPORTS')]" + }, + "storagePathEndsWith": { + "value": "manifest.json" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "12667288354191065454" + } + }, + "parameters": { + "dataFactoryName": { + "type": "string", + "metadata": { + "description": "Required. Name of the publisher-specific Data Factory instance." + } + }, + "triggerName": { + "type": "string", + "metadata": { + "description": "Required. Name of the Data Factory trigger to create or update." + } + }, + "storageAccountName": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Azure storage container to monitor for updates and trigger events for." + } + }, + "storageContainer": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Azure storage container to monitor for updates and trigger events for." + } + }, + "storagePathStartsWith": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Beginning of the storage path within the specified storageContainer to monitor for updates and trigger events for." + } + }, + "storagePathEndsWith": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. End of the storage path to monitor for updates and trigger events for." + } + }, + "pipelineName": { + "type": "string", + "metadata": { + "description": "Required. Name of the Data Factory pipeline to execute when the trigger is executed." + } + }, + "pipelineParameters": { + "type": "object", + "metadata": { + "description": "Required. Parameters to pass to the pipeline when the trigger is executed." + } + } + }, + "resources": [ + { + "condition": "[not(empty(parameters('storageAccountName')))]", + "type": "Microsoft.DataFactory/factories/triggers", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('dataFactoryName'), parameters('triggerName'))]", + "properties": { + "annotations": [], + "pipelines": [ + { + "pipelineReference": { + "referenceName": "[parameters('pipelineName')]", + "type": "PipelineReference" + }, + "parameters": "[parameters('pipelineParameters')]" + } + ], + "type": "BlobEventsTrigger", + "typeProperties": { + "blobPathBeginsWith": "[format('/{0}/blobs/{1}', parameters('storageContainer'), parameters('storagePathStartsWith'))]", + "blobPathEndsWith": "[parameters('storagePathEndsWith')]", + "ignoreEmptyBlobs": true, + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]", + "events": [ + "Microsoft.Storage.BlobCreated" + ] + } + } + } + ] + } + }, + "dependsOn": [ + "appRegistration", + "dataFactory::pipeline_ExecuteExportsETL" + ] + } + }, + "outputs": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Properties of the hub app." + }, + "value": "[parameters('app')]" + }, + "exportContainer": { + "type": "string", + "metadata": { + "description": "Name of the container used for Cost Management exports." + }, + "value": "[reference('exportContainer').outputs.containerName.value]" + }, + "schemaFilesUploaded": { + "type": "int", + "metadata": { + "description": "Number of schema files uploaded." + }, + "value": "[reference('schemaFiles').outputs.filesUploaded.value]" + } + } + } + }, + "dependsOn": [ + "core" + ] + }, + "cmManagedExports": { + "condition": "[parameters('enableManagedExports')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.ManagedExports", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[__bicep.newApp(variables('hub'), 'Microsoft.CostManagement', 'ManagedExports')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "17365536975648713074" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getExportBody": { + "parameters": [ + { + "type": "string", + "name": "exportContainerName" + }, + { + "type": "string", + "name": "datasetType" + }, + { + "type": "string", + "name": "schemaVersion" + }, + { + "type": "bool", + "name": "isMonthly" + }, + { + "type": "string", + "name": "exportFormat" + }, + { + "type": "string", + "name": "compressionMode" + }, + { + "type": "string", + "name": "partitionData" + }, + { + "type": "string", + "name": "dataOverwriteBehavior" + } + ], + "output": { + "type": "string", + "value": "[format('{{ \"properties\": {{ \"definition\": {{ \"dataSet\": {{ \"configuration\": {{ \"dataVersion\": \"{0}\", \"filters\": [] }}, \"granularity\": \"Daily\" }}, \"timeframe\": \"{1}\", \"type\": \"{2}\" }}, \"deliveryInfo\": {{ \"destination\": {{ \"container\": \"{3}\", \"rootFolderPath\": \"@{{if(startswith(item().scope, ''/''), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}}\", \"type\": \"AzureBlob\", \"resourceId\": \"@{{variables(''storageAccountId'')}}\" }} }}, \"schedule\": {{ \"recurrence\": \"{4}\", \"recurrencePeriod\": {{ \"from\": \"2024-01-01T00:00:00.000Z\", \"to\": \"2050-02-01T00:00:00.000Z\" }}, \"status\": \"Inactive\" }}, \"format\": \"{5}\", \"partitionData\": \"{6}\", \"dataOverwriteBehavior\": \"{7}\", \"compressionMode\": \"{8}\" }}, \"id\": \"@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{variables(''exportName'')}}\", \"name\": \"@{{variables(''exportName'')}}\", \"type\": \"Microsoft.CostManagement/reports\", \"identity\": {{ \"type\": \"systemAssigned\" }}, \"location\": \"global\" }}', parameters('schemaVersion'), if(parameters('isMonthly'), 'TheLastMonth', 'MonthToDate'), parameters('datasetType'), parameters('exportContainerName'), if(parameters('isMonthly'), 'Monthly', 'Daily'), parameters('exportFormat'), parameters('partitionData'), parameters('dataOverwriteBehavior'), parameters('compressionMode'))]" + } + }, + "getExportBodyV2": { + "parameters": [ + { + "type": "string", + "name": "exportContainerName" + }, + { + "type": "string", + "name": "datasetType" + }, + { + "type": "bool", + "name": "isMonthly" + }, + { + "type": "string", + "name": "exportFormat" + }, + { + "type": "string", + "name": "compressionMode" + }, + { + "type": "string", + "name": "partitionData" + }, + { + "type": "string", + "name": "dataOverwriteBehavior" + }, + { + "type": "string", + "name": "recommendationScope" + }, + { + "type": "string", + "name": "recommendationLookbackPeriod" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "string", + "value": "[if(equals(toLower(parameters('datasetType')), 'focuscost'), format('{{ \"properties\": {{ \"definition\": {{ \"dataSet\": {{ \"configuration\": {{ \"dataVersion\": \"{0}\", \"filters\": [] }}, \"granularity\": \"Daily\" }}, \"timeframe\": \"{1}\", \"type\": \"{2}\" }}, \"deliveryInfo\": {{ \"destination\": {{ \"container\": \"{3}\", \"rootFolderPath\": \"@{{if(startswith(item().scope, ''/''), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}}\", \"type\": \"AzureBlob\", \"resourceId\": \"@{{variables(''storageAccountId'')}}\" }} }}, \"schedule\": {{ \"recurrence\": \"{4}\", \"recurrencePeriod\": {{ \"from\": \"2024-01-01T00:00:00.000Z\", \"to\": \"2050-02-01T00:00:00.000Z\" }}, \"status\": \"Inactive\" }}, \"format\": \"{5}\", \"partitionData\": \"{6}\", \"dataOverwriteBehavior\": \"{7}\", \"compressionMode\": \"{8}\" }}, \"id\": \"@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-{9}-costdetails''))}}\", \"name\": \"@{{toLower(concat(variables(''finOpsHub''), ''-{10}-costdetails''))}}\", \"type\": \"Microsoft.CostManagement/reports\", \"identity\": {{ \"type\": \"systemAssigned\" }}, \"location\": \"global\" }}', variables('exportDataVersions')[toLower(parameters('datasetType'))], if(parameters('isMonthly'), 'TheLastMonth', 'MonthToDate'), parameters('datasetType'), parameters('exportContainerName'), if(parameters('isMonthly'), 'Monthly', 'Daily'), parameters('exportFormat'), parameters('partitionData'), parameters('dataOverwriteBehavior'), parameters('compressionMode'), if(parameters('isMonthly'), 'monthly', 'daily'), if(parameters('isMonthly'), 'monthly', 'daily')), if(equals(toLower(parameters('datasetType')), 'reservationdetails'), format('{{ \"properties\": {{ \"definition\": {{ \"dataSet\": {{ \"configuration\": {{ \"dataVersion\": \"{0}\", \"filters\": [] }}, \"granularity\": \"Daily\" }}, \"timeframe\": \"{1}\", \"type\": \"{2}\" }}, \"deliveryInfo\": {{ \"destination\": {{ \"container\": \"{3}\", \"rootFolderPath\": \"@{{if(startswith(item().scope, ''/''), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}}\", \"type\": \"AzureBlob\", \"resourceId\": \"@{{variables(''storageAccountId'')}}\" }} }}, \"schedule\": {{ \"recurrence\": \"{4}\", \"recurrencePeriod\": {{ \"from\": \"2024-01-01T00:00:00.000Z\", \"to\": \"2050-02-01T00:00:00.000Z\" }}, \"status\": \"Inactive\" }}, \"format\": \"{5}\", \"partitionData\": \"{6}\", \"dataOverwriteBehavior\": \"{7}\", \"compressionMode\": \"{8}\" }}, \"id\": \"@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-{9}-{10}''))}}\", \"name\": \"@{{toLower(concat(variables(''finOpsHub''), ''-{11}-{12}''))}}\", \"type\": \"Microsoft.CostManagement/reports\", \"identity\": {{ \"type\": \"systemAssigned\" }}, \"location\": \"global\" }}', variables('exportDataVersions')[toLower(parameters('datasetType'))], if(parameters('isMonthly'), 'TheLastMonth', 'MonthToDate'), parameters('datasetType'), parameters('exportContainerName'), if(parameters('isMonthly'), 'Monthly', 'Daily'), parameters('exportFormat'), parameters('partitionData'), parameters('dataOverwriteBehavior'), parameters('compressionMode'), if(parameters('isMonthly'), 'monthly', 'daily'), toLower(parameters('datasetType')), if(parameters('isMonthly'), 'monthly', 'daily'), toLower(parameters('datasetType'))), if(or(equals(toLower(parameters('datasetType')), 'pricesheet'), equals(toLower(parameters('datasetType')), 'reservationtransactions')), format('{{ \"properties\": {{ \"definition\": {{ \"dataSet\": {{ \"configuration\": {{ \"dataVersion\": \"{0}\", \"filters\": [] }}}}, \"timeframe\": \"{1}\", \"type\": \"{2}\" }}, \"deliveryInfo\": {{ \"destination\": {{ \"container\": \"{3}\", \"rootFolderPath\": \"@{{if(startswith(item().scope, ''/''), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}}\", \"type\": \"AzureBlob\", \"resourceId\": \"@{{variables(''storageAccountId'')}}\" }} }}, \"schedule\": {{ \"recurrence\": \"{4}\", \"recurrencePeriod\": {{ \"from\": \"2024-01-01T00:00:00.000Z\", \"to\": \"2050-02-01T00:00:00.000Z\" }}, \"status\": \"Inactive\" }}, \"format\": \"{5}\", \"partitionData\": \"{6}\", \"dataOverwriteBehavior\": \"{7}\", \"compressionMode\": \"{8}\" }}, \"id\": \"@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-{9}-{10}''))}}\", \"name\": \"@{{toLower(concat(variables(''finOpsHub''), ''-{11}-{12}''))}}\", \"type\": \"Microsoft.CostManagement/reports\", \"identity\": {{ \"type\": \"systemAssigned\" }}, \"location\": \"global\" }}', variables('exportDataVersions')[toLower(parameters('datasetType'))], if(parameters('isMonthly'), 'TheCurrentMonth', 'MonthToDate'), parameters('datasetType'), parameters('exportContainerName'), if(parameters('isMonthly'), 'Monthly', 'Daily'), parameters('exportFormat'), parameters('partitionData'), parameters('dataOverwriteBehavior'), parameters('compressionMode'), if(parameters('isMonthly'), 'monthly', 'daily'), toLower(parameters('datasetType')), if(parameters('isMonthly'), 'monthly', 'daily'), toLower(parameters('datasetType'))), if(equals(toLower(parameters('datasetType')), 'reservationrecommendations'), format('{{ \"properties\": {{ \"definition\": {{ \"dataSet\": {{ \"configuration\": {{ \"dataVersion\": \"{0}\", \"filters\": [ {{ \"name\": \"reservationScope\", \"value\": \"{1}\" }}, {{ \"name\": \"resourceType\", \"value\": \"{2}\" }}, {{ \"name\": \"lookBackPeriod\", \"value\": \"{3}\" }}] }}}}, \"timeframe\": \"{4}\", \"type\": \"{5}\" }}, \"deliveryInfo\": {{ \"destination\": {{ \"container\": \"{6}\", \"rootFolderPath\": \"@{{if(startswith(item().scope, ''/''), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}}\", \"type\": \"AzureBlob\", \"resourceId\": \"@{{variables(''storageAccountId'')}}\" }} }}, \"schedule\": {{ \"recurrence\": \"{7}\", \"recurrencePeriod\": {{ \"from\": \"2024-01-01T00:00:00.000Z\", \"to\": \"2050-02-01T00:00:00.000Z\" }}, \"status\": \"Inactive\" }}, \"format\": \"{8}\", \"partitionData\": \"{9}\", \"dataOverwriteBehavior\": \"{10}\", \"compressionMode\": \"{11}\" }}, \"id\": \"@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-{12}-costdetails''))}}\", \"name\": \"@{{toLower(concat(variables(''finOpsHub''), ''-{13}-costdetails''))}}\", \"type\": \"Microsoft.CostManagement/reports\", \"identity\": {{ \"type\": \"systemAssigned\" }}, \"location\": \"global\" }}', variables('exportDataVersions')[toLower(parameters('datasetType'))], parameters('recommendationScope'), parameters('resourceType'), parameters('recommendationLookbackPeriod'), if(parameters('isMonthly'), 'TheLastMonth', 'MonthToDate'), parameters('datasetType'), parameters('exportContainerName'), if(parameters('isMonthly'), 'Monthly', 'Daily'), parameters('exportFormat'), parameters('partitionData'), parameters('dataOverwriteBehavior'), parameters('compressionMode'), if(parameters('isMonthly'), 'monthly', 'daily'), if(parameters('isMonthly'), 'monthly', 'daily')), 'undefined'))))]" + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + } + }, + "variables": { + "CONFIG": "config", + "MSEXPORTS": "msexports", + "exportsApiVersion": "2023-07-01-preview", + "exportDataVersions": { + "focuscost": "1.2-preview", + "pricesheet": "2023-03-01", + "reservationdetails": "2023-03-01", + "reservationrecommendations": "2023-05-01", + "reservationtransactions": "2023-05-01" + }, + "finOpsToolkitVersion": "13.0" + }, + "resources": { + "dataFactory::dataset_config": { + "existing": true, + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, variables('CONFIG'))]" + }, + "dataFactory::trigger_DailySchedule": { + "type": "Microsoft.DataFactory/factories/triggers", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_DailySchedule', variables('CONFIG')))]", + "properties": { + "pipelines": [ + { + "pipelineReference": { + "referenceName": "[format('{0}_StartExportProcess', variables('CONFIG'))]", + "type": "PipelineReference" + }, + "parameters": { + "Recurrence": "Daily" + } + } + ], + "type": "ScheduleTrigger", + "typeProperties": { + "recurrence": { + "frequency": "Hour", + "interval": 24, + "startTime": "2023-01-01T01:01:00", + "timeZone": "[reference('timeZones').outputs.Timezone.value]" + } + } + }, + "dependsOn": [ + "dataFactory::pipeline_StartExportProcess", + "timeZones" + ] + }, + "dataFactory::trigger_MonthlySchedule": { + "type": "Microsoft.DataFactory/factories/triggers", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_MonthlySchedule', variables('CONFIG')))]", + "properties": { + "pipelines": [ + { + "pipelineReference": { + "referenceName": "[format('{0}_StartExportProcess', variables('CONFIG'))]", + "type": "PipelineReference" + }, + "parameters": { + "Recurrence": "Monthly" + } + } + ], + "type": "ScheduleTrigger", + "typeProperties": { + "recurrence": { + "frequency": "Month", + "interval": 1, + "startTime": "2023-01-05T01:11:00", + "timeZone": "[reference('timeZones').outputs.Timezone.value]", + "schedule": { + "monthDays": [ + 2, + 5, + 19 + ] + } + } + } + }, + "dependsOn": [ + "dataFactory::pipeline_StartExportProcess", + "timeZones" + ] + }, + "dataFactory::pipeline_StartBackfillProcess": { + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_StartBackfillProcess', variables('CONFIG')))]", + "properties": { + "activities": [ + { + "name": "Get Config", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference", + "parameters": { + "fileName": { + "value": "@variables('fileName')", + "type": "Expression" + }, + "folderPath": { + "value": "@variables('folderPath')", + "type": "Expression" + } + } + } + } + }, + { + "name": "Set backfill end date", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Get Config", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "variableName": "endDate", + "value": { + "value": "@addDays(startOfMonth(utcNow()), -1)", + "type": "Expression" + } + } + }, + { + "name": "Set backfill start date", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Get Config", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "variableName": "startDate", + "value": { + "value": "@subtractFromTime(startOfMonth(utcNow()), activity('Get Config').output.firstRow.retention.ingestion.months, 'Month')", + "type": "Expression" + } + } + }, + { + "name": "Set export start date", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Set backfill start date", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "variableName": "thisMonth", + "value": { + "value": "@startOfMonth(variables('endDate'))", + "type": "Expression" + } + } + }, + { + "name": "Set export end date", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Set export start date", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "variableName": "nextMonth", + "value": { + "value": "@startOfMonth(subtractFromTime(variables('thisMonth'), 1, 'Month'))", + "type": "Expression" + } + } + }, + { + "name": "Every Month", + "type": "Until", + "dependsOn": [ + { + "activity": "Set export end date", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set backfill end date", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "expression": { + "value": "@less(variables('thisMonth'), variables('startDate'))", + "type": "Expression" + }, + "activities": [ + { + "name": "Update export start date", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Backfill data", + "dependencyConditions": [ + "Completed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "variableName": "thisMonth", + "value": { + "value": "@variables('nextMonth')", + "type": "Expression" + } + } + }, + { + "name": "Update export end date", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Update export start date", + "dependencyConditions": [ + "Completed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "variableName": "nextMonth", + "value": { + "value": "@subtractFromTime(variables('thisMonth'), 1, 'Month')", + "type": "Expression" + } + } + }, + { + "name": "Backfill data", + "type": "ExecutePipeline", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "pipeline": { + "referenceName": "[format('{0}_RunBackfillJob', variables('CONFIG'))]", + "type": "PipelineReference" + }, + "waitOnCompletion": true, + "parameters": { + "StartDate": { + "value": "@variables('thisMonth')", + "type": "Expression" + }, + "EndDate": { + "value": "@addDays(addToTime(variables('thisMonth'), 1, 'Month'), -1)", + "type": "Expression" + } + } + } + } + ], + "timeout": "0.02:00:00" + } + } + ], + "concurrency": 1, + "variables": { + "exportName": { + "type": "String" + }, + "storageAccountId": { + "type": "String", + "defaultValue": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "finOpsHub": { + "type": "String", + "defaultValue": "[parameters('app').hub.name]" + }, + "resourceManagementUri": { + "type": "String", + "defaultValue": "[environment().resourceManager]" + }, + "fileName": { + "type": "String", + "defaultValue": "settings.json" + }, + "folderPath": { + "type": "String", + "defaultValue": "[variables('CONFIG')]" + }, + "endDate": { + "type": "String" + }, + "startDate": { + "type": "String" + }, + "thisMonth": { + "type": "String" + }, + "nextMonth": { + "type": "String" + } + } + }, + "dependsOn": [ + "dataFactory::pipeline_RunBackfillJob" + ] + }, + "dataFactory::pipeline_RunBackfillJob": { + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_RunBackfillJob', variables('CONFIG')))]", + "properties": { + "activities": [ + { + "name": "Get Config", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference", + "parameters": { + "fileName": { + "value": "@variables('fileName')", + "type": "Expression" + }, + "folderPath": { + "value": "@variables('folderPath')", + "type": "Expression" + } + } + } + } + }, + { + "name": "Set Scopes", + "description": "Save scopes to test if it is an array", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Get Config", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "scopesArray", + "value": { + "value": "@activity('Get Config').output.firstRow.scopes", + "type": "Expression" + } + } + }, + { + "name": "Set Scopes as Array", + "description": "Wraps a single scope object into an array to work around the PowerShell bug where single-item arrays are sometimes written as a single object instead of an array.", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Set Scopes", + "dependencyConditions": [ + "Failed" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "scopesArray", + "value": { + "value": "@createArray(activity('Get Config').output.firstRow.scopes)", + "type": "Expression" + } + } + }, + { + "name": "Filter Invalid Scopes", + "description": "Remove any invalid scopes to avoid errors.", + "type": "Filter", + "dependsOn": [ + { + "activity": "Set Scopes", + "dependencyConditions": [ + "Succeeded", + "Failed" + ] + }, + { + "activity": "Set Scopes as Array", + "dependencyConditions": [ + "Skipped", + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@variables('scopesArray')", + "type": "Expression" + }, + "condition": { + "value": "@and(not(empty(item().scope)), not(equals(item().scope, '/')))", + "type": "Expression" + } + } + }, + { + "name": "ForEach Export Scope", + "type": "ForEach", + "dependsOn": [ + { + "activity": "Filter Invalid Scopes", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@activity('Filter Invalid Scopes').output.Value", + "type": "Expression" + }, + "isSequential": true, + "activities": [ + { + "name": "Set backfill export name", + "type": "SetVariable", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "variableName": "exportName", + "value": { + "value": "@toLower(concat(variables('finOpsHub'), '-monthly-costdetails'))", + "type": "Expression" + } + } + }, + { + "name": "Trigger backfill export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Set backfill export name", + "dependencyConditions": [ + "Completed" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 1, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{variables(''exportName'')}}/run?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "POST", + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunBackfill@{0}', variables('finOpsToolkitVersion'))]", + "Content-Type": "application/json", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "body": "{\"timePeriod\" : { \"from\" : \"@{pipeline().parameters.StartDate}\", \"to\" : \"@{pipeline().parameters.EndDate}\" }}", + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + } + ] + } + } + ], + "concurrency": 1, + "parameters": { + "StartDate": { + "type": "string" + }, + "EndDate": { + "type": "string" + } + }, + "variables": { + "exportName": { + "type": "String" + }, + "storageAccountId": { + "type": "String", + "defaultValue": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "finOpsHub": { + "type": "String", + "defaultValue": "[parameters('app').hub.name]" + }, + "resourceManagementUri": { + "type": "String", + "defaultValue": "[environment().resourceManager]" + }, + "fileName": { + "type": "String", + "defaultValue": "settings.json" + }, + "folderPath": { + "type": "String", + "defaultValue": "[variables('CONFIG')]" + }, + "scopesArray": { + "type": "Array" + } + } + } + }, + "dataFactory::pipeline_StartExportProcess": { + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_StartExportProcess', variables('CONFIG')))]", + "properties": { + "activities": [ + { + "name": "Get Config", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference", + "parameters": { + "fileName": { + "value": "@variables('fileName')", + "type": "Expression" + }, + "folderPath": { + "value": "@variables('folderPath')", + "type": "Expression" + } + } + } + } + }, + { + "name": "Set Scopes", + "description": "Save scopes to test if it is an array", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Get Config", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "scopesArray", + "value": { + "value": "@activity('Get Config').output.firstRow.scopes", + "type": "Expression" + } + } + }, + { + "name": "Set Scopes as Array", + "description": "Wraps a single scope object into an array to work around the PowerShell bug where single-item arrays are sometimes written as a single object instead of an array.", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Set Scopes", + "dependencyConditions": [ + "Failed" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "scopesArray", + "value": { + "value": "@createArray(activity('Get Config').output.firstRow.scopes)", + "type": "Expression" + } + } + }, + { + "name": "Filter Invalid Scopes", + "description": "Remove any invalid scopes to avoid errors.", + "type": "Filter", + "dependsOn": [ + { + "activity": "Set Scopes", + "dependencyConditions": [ + "Succeeded", + "Failed" + ] + }, + { + "activity": "Set Scopes as Array", + "dependencyConditions": [ + "Succeeded", + "Skipped" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@variables('scopesArray')", + "type": "Expression" + }, + "condition": { + "value": "@and(not(empty(item().scope)), not(equals(item().scope, '/')))", + "type": "Expression" + } + } + }, + { + "name": "ForEach Export Scope", + "type": "ForEach", + "dependsOn": [ + { + "activity": "Filter Invalid Scopes", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@activity('Filter Invalid Scopes').output.Value", + "type": "Expression" + }, + "isSequential": true, + "activities": [ + { + "name": "Get exports for scope", + "type": "WebActivity", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "GET", + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Run exports for scope", + "type": "ExecutePipeline", + "dependsOn": [ + { + "activity": "Get exports for scope", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "pipeline": { + "referenceName": "[format('{0}_RunExportJobs', variables('CONFIG'))]", + "type": "PipelineReference" + }, + "waitOnCompletion": true, + "parameters": { + "ExportScopes": { + "value": "@activity('Get exports for scope').output.value", + "type": "Expression" + }, + "Recurrence": { + "value": "@pipeline().parameters.Recurrence", + "type": "Expression" + } + } + } + } + ] + } + } + ], + "concurrency": 1, + "parameters": { + "Recurrence": { + "type": "string", + "defaultValue": "Daily" + } + }, + "variables": { + "fileName": { + "type": "String", + "defaultValue": "settings.json" + }, + "folderPath": { + "type": "String", + "defaultValue": "[variables('CONFIG')]" + }, + "finOpsHub": { + "type": "String", + "defaultValue": "[parameters('app').hub.name]" + }, + "resourceManagementUri": { + "type": "String", + "defaultValue": "[environment().resourceManager]" + }, + "scopesArray": { + "type": "Array" + } + } + }, + "dependsOn": [ + "dataFactory::pipeline_RunExportJobs" + ] + }, + "dataFactory::pipeline_RunExportJobs": { + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_RunExportJobs', variables('CONFIG')))]", + "properties": { + "activities": [ + { + "name": "ForEach export scope", + "type": "ForEach", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@pipeline().parameters.exportScopes", + "type": "Expression" + }, + "isSequential": true, + "activities": [ + { + "name": "If scheduled", + "type": "IfCondition", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "expression": { + "value": "@and( startswith(toLower(item().name), toLower(variables('hubName'))), and(contains(string(item().properties.schedule), 'recurrence'), equals(toLower(item().properties.schedule.recurrence), toLower(pipeline().parameters.Recurrence))))", + "type": "Expression" + }, + "ifTrueActivities": [ + { + "name": "Trigger export", + "type": "WebActivity", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "method": "POST", + "url": { + "value": "[format('@{{replace(toLower(concat(variables(''resourceManagementUri''),item().id)), ''com//'', ''com/'')}}/run?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "body": " ", + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + } + ] + } + } + ] + } + } + ], + "concurrency": 1, + "parameters": { + "ExportScopes": { + "type": "array" + }, + "Recurrence": { + "type": "string", + "defaultValue": "Daily" + } + }, + "variables": { + "resourceManagementUri": { + "type": "String", + "defaultValue": "[environment().resourceManager]" + }, + "hubName": { + "type": "String", + "defaultValue": "[parameters('app').hub.name]" + } + } + }, + "dependsOn": [ + "dataFactory::dataset_config" + ] + }, + "dataFactory::pipeline_ConfigureExports": { + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_ConfigureExports', variables('CONFIG')))]", + "properties": { + "activities": [ + { + "name": "Get Config", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference", + "parameters": { + "fileName": { + "value": "@variables('fileName')", + "type": "Expression" + }, + "folderPath": { + "value": "@variables('folderPath')", + "type": "Expression" + } + } + } + } + }, + { + "name": "Save Scopes", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Get Config", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "scopesArray", + "value": { + "value": "@activity('Get Config').output.firstRow.scopes", + "type": "Expression" + } + } + }, + { + "name": "Save Scopes as Array", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Save Scopes", + "dependencyConditions": [ + "Failed" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "scopesArray", + "value": { + "value": "@array(activity('Get Config').output.firstRow.scopes)", + "type": "Expression" + } + } + }, + { + "name": "Filter Invalid Scopes", + "type": "Filter", + "dependsOn": [ + { + "activity": "Save Scopes", + "dependencyConditions": [ + "Succeeded", + "Failed" + ] + }, + { + "activity": "Save Scopes as Array", + "dependencyConditions": [ + "Skipped", + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@variables('scopesArray')", + "type": "Expression" + }, + "condition": { + "value": "@and(not(empty(item().scope)), not(equals(item().scope, '/')))", + "type": "Expression" + } + } + }, + { + "name": "ForEach Export Scope", + "type": "ForEach", + "dependsOn": [ + { + "activity": "Filter Invalid Scopes", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@activity('Filter Invalid Scopes').output.value", + "type": "Expression" + }, + "isSequential": true, + "activities": [ + { + "name": "Set Export Type", + "type": "SetVariable", + "dependsOn": [], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "exportScopeType", + "value": { + "value": "@if(contains(toLower(item().scope), 'providers/microsoft.billing/billingaccounts'), if(contains(toLower(item().scope), ':'), 'mca', if(contains(toLower(item().scope), '/departments/'), 'ea-department', 'ea')), if(contains(toLower(item().scope), 'subscriptions/'), 'subscription', 'undefined'))", + "type": "Expression" + } + } + }, + { + "name": "Switch Export Type", + "type": "Switch", + "dependsOn": [ + { + "activity": "Set Export Type", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "on": { + "value": "@toLower(variables('exportScopeType'))", + "type": "Expression" + }, + "cases": [ + { + "value": "ea", + "activities": [ + { + "name": "Open month focus export", + "type": "WebActivity", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-costdetails''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'FocusCost', false(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsDaily@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Closed month focus export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Open month focus export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-monthly-costdetails''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'FocusCost', true(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsMonthly@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Monthly pricesheet export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Closed month focus export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-monthly-pricesheet''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'Pricesheet', true(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.Prices@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Trigger EA monthly pricesheet export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Monthly pricesheet export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "method": "POST", + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-monthly-pricesheet''))}}/run?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.Prices@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "body": " ", + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Daily reservation details export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Monthly pricesheet export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-reservationdetails''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'ReservationDetails', false(), 'CSV', 'None', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.ReservationDetails@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Daily reservation transactions export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Daily reservation details export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-reservationtransactions''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'ReservationTransactions', false(), 'CSV', 'None', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.ReservationTransactions@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Daily shared 30day virtual machines", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Daily reservation transactions export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-recommendations-shared-last30days-virtualmachines''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'ReservationRecommendations', false(), 'CSV', 'None', 'true', 'CreateNewReport', 'Shared', 'Last30Days', 'VirtualMachines')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.ReservationRecommendations.VM.Shared.30d@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + } + ] + }, + { + "value": "ea-department", + "activities": [ + { + "name": "EA Department open month focus export", + "type": "WebActivity", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-costdetails''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'FocusCost', false(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsDaily@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "EA Department closed month focus export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "EA Department open month focus export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-monthly-costdetails''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'FocusCost', true(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsMonthly@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + } + ] + }, + { + "value": "subscription", + "activities": [ + { + "name": "Subscription open month focus export", + "type": "WebActivity", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-costdetails''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'FocusCost', false(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsDaily@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Subscription closed month focus export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Subscription open month focus export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-monthly-costdetails''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'FocusCost', true(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsMonthly@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + } + ] + }, + { + "value": "mca", + "activities": [ + { + "name": "Export Type Unsupported Error", + "type": "Fail", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('MCA agreements are not supported for managed exports :',variables('exportScope'))", + "type": "Expression" + }, + "errorCode": "ExportTypeUnsupported" + } + } + ] + } + ], + "defaultActivities": [ + { + "name": "Export Type Not Defined Error", + "type": "Fail", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Unable to determine the export scope type for :',variables('exportScope'))", + "type": "Expression" + }, + "errorCode": "ExportTypeNotDefined" + } + } + ] + } + } + ] + } + } + ], + "concurrency": 1, + "variables": { + "scopesArray": { + "type": "Array" + }, + "exportName": { + "type": "String" + }, + "exportScope": { + "type": "String" + }, + "exportScopeType": { + "type": "String" + }, + "storageAccountId": { + "type": "String", + "defaultValue": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "finOpsHub": { + "type": "String", + "defaultValue": "[parameters('app').hub.name]" + }, + "resourceManagementUri": { + "type": "String", + "defaultValue": "[environment().resourceManager]" + }, + "fileName": { + "type": "String", + "defaultValue": "settings.json" + }, + "folderPath": { + "type": "String", + "defaultValue": "[variables('CONFIG')]" + } + } + } + }, + "storageAccount": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]" + }, + "dataFactory": { + "existing": true, + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]" + }, + "appRegistration": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.ManagedExports_Register", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "version": { + "value": "[variables('finOpsToolkitVersion')]" + }, + "features": { + "value": [ + "DataFactory" + ] + }, + "storageRoles": { + "value": [ + "f58310d9-a9f6-439a-9e8d-f62e7b41a168" + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "8267413868206701537" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppFeature": { + "type": "string", + "allowedValues": [ + "DataFactory", + "KeyVault", + "Storage" + ], + "metadata": { + "description": "FinOps hub app features.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "version": { + "type": "string", + "metadata": { + "description": "Required. Version number of the FinOps hub app." + } + }, + "features": { + "type": "array", + "items": { + "$ref": "#/definitions/HubAppFeature" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which features the app requires. Allowed values: \"DataFactory\", \"KeyVault\", \"Storage\". Default: [] (none)." + } + }, + "storageRoles": { + "type": "array", + "items": { + "type": "string" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which RBAC roles the Data Factory identity needs on the storage account, if created. This is in addition to Storage Blob Data Contributor for reading and managing content. Default: [] (none)." + } + }, + "telemetryString": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n<#\r\n.SYNOPSIS\r\nManages Azure Data Factory triggers and pipelines during FinOps hub deployment.\r\n\r\n.DESCRIPTION\r\nThis script is called by Bicep deployment scripts to start/stop Data Factory triggers\r\nand optionally run pipelines. It handles two types of triggers:\r\n\r\n1. Schedule triggers - Can be started/stopped directly\r\n2. BlobEventsTriggers - Require Event Grid subscription management before start/stop\r\n\r\nBlobEventsTriggers use Event Grid to listen for storage blob events. Before stopping,\r\nthe Event Grid subscription must be removed (unsubscribed). Before starting, the\r\nsubscription must be added and fully provisioned. This script handles this automatically\r\nby detecting BlobEventsTriggers via the BlobPathBeginsWith property.\r\n\r\nThe script uses retry logic with linear backoff to handle transient API failures and\r\nwait for Event Grid subscription provisioning/deprovisioning to complete.\r\n\r\n.PARAMETER DataFactoryResourceGroup\r\nThe resource group containing the Data Factory.\r\n\r\n.PARAMETER DataFactoryName\r\nThe name of the Data Factory instance.\r\n\r\n.PARAMETER Pipelines\r\nPipe-delimited list of pipeline names to run (e.g., \"pipeline1|pipeline2\").\r\n\r\n.PARAMETER StartTriggers\r\nSwitch to start all stopped triggers (with Event Grid subscription for blob triggers).\r\n\r\n.PARAMETER StopTriggers\r\nSwitch to stop all running triggers (with Event Grid unsubscription for blob triggers).\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StopTriggers\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StartTriggers\r\n#>\r\n\r\nparam(\r\n [string] $DataFactoryResourceGroup,\r\n [string] $DataFactoryName,\r\n [string] $Pipelines = \"\",\r\n [switch] $StartTriggers,\r\n [switch] $StopTriggers\r\n)\r\n\r\n$MAX_RETRIES = 20\r\n$DeploymentScriptOutputs = @{}\r\n\r\nfunction Write-Log($Message)\r\n{\r\n Write-Output \"$(Get-Date -Format 'HH:mm:ss') $Message\"\r\n}\r\n\r\nfunction Invoke-WithRetry([scriptblock]$Action, [string]$Name, [int]$Delay = 5)\r\n{\r\n for ($i = 1; $i -le $MAX_RETRIES; $i++)\r\n {\r\n try { return & $Action }\r\n catch\r\n {\r\n Write-Log \"$Name failed (attempt $i/${MAX_RETRIES}): $($_.Exception.Message)\"\r\n if ($i -eq $MAX_RETRIES) { throw }\r\n Start-Sleep -Seconds ($Delay * $i)\r\n }\r\n }\r\n}\r\n\r\nfunction Set-BlobTriggerSubscription([string]$TriggerName, [switch]$Subscribe)\r\n{\r\n $targetStatus = if ($Subscribe) { 'Enabled' } else { 'Disabled' }\r\n $action = if ($Subscribe) { 'Subscribing' } else { 'Unsubscribing' }\r\n\r\n Write-Log \"$action $TriggerName to events...\"\r\n Invoke-WithRetry -Name \"$action $TriggerName\" -Delay 5 -Action {\r\n if ($Subscribe)\r\n {\r\n Add-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n else\r\n {\r\n Remove-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n\r\n $status = Get-AzDataFactoryV2TriggerSubscriptionStatus `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName\r\n if ($status.Status -ne $targetStatus)\r\n {\r\n throw \"Subscription status is $($status.Status), expected $targetStatus\"\r\n }\r\n }\r\n}\r\n\r\nif ($StartTriggers -or $StopTriggers)\r\n{\r\n $triggers = Invoke-WithRetry -Name \"Get triggers\" -Action {\r\n Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n | Where-Object {\r\n ($StartTriggers -and $_.Properties.RuntimeState -ne \"Started\") `\r\n -or ($StopTriggers -and $_.Properties.RuntimeState -ne \"Stopped\")\r\n }\r\n }\r\n\r\n Write-Log \"Found $($triggers.Count) trigger(s) to $(if ($StartTriggers) { 'start' } else { 'stop' })\"\r\n\r\n $triggers | ForEach-Object {\r\n $triggerName = $_.Name\r\n $isBlobTrigger = $null -ne $_.Properties.BlobPathBeginsWith\r\n\r\n if ($StopTriggers)\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName }\r\n Write-Log \"Stopping trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Stop $triggerName\" -Action {\r\n Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n else\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName -Subscribe }\r\n Write-Log \"Starting trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Start $triggerName\" -Action {\r\n Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n\r\n Invoke-WithRetry -Name \"Wait for $triggerName\" -Action {\r\n $state = (Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName).Properties.RuntimeState\r\n $expected = if ($StartTriggers) { 'Started' } else { 'Stopped' }\r\n if ($state -ne $expected) { throw \"Trigger is $state, expected $expected\" }\r\n }\r\n\r\n Write-Log \"...done\"\r\n $DeploymentScriptOutputs[$triggerName] = $true\r\n }\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($Pipelines))\r\n{\r\n $Pipelines.Split('|') | ForEach-Object {\r\n Write-Log \"Running pipeline $_...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", + "usesDataFactory": "[contains(parameters('features'), 'DataFactory')]", + "usesKeyVault": "[contains(parameters('features'), 'KeyVault')]", + "usesStorage": "[contains(parameters('features'), 'Storage')]", + "telemetryId": "[format('ftk-hubapp-{0}{1}{2}', parameters('app').id, if(empty(parameters('telemetryString')), '', '_'), parameters('telemetryString'))]", + "telemetryProps": { + "mode": "Incremental", + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "[format('FTK: {0}', parameters('app').id)]", + "version": "[parameters('version')]" + } + }, + "resources": [] + } + }, + "autoStartRbacRoles": [ + "673868aa-7521-48a0-acc6-0f60742d39f5" + ], + "factoryStorageRoles": "[union(parameters('storageRoles'), createArray('17d1049b-9a84-46fb-8f53-869881c3d3ab', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe', 'acdd72a7-3385-48ef-bd42-f606fba81ae7'))]", + "storageInfrastructureEncryptionProperties": "[if(not(parameters('app').hub.options.storageInfrastructureEncryption), createObject(), createObject('encryption', createObject('keySource', 'Microsoft.Storage', 'requireInfrastructureEncryption', parameters('app').hub.options.storageInfrastructureEncryption)))]" + }, + "resources": { + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').storage)]", + "properties": { + "name": "[parameters('app').storage]", + "groupId": "dfs", + "privateLinkResourceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "fqdns": [ + "[reference('storageAccount').primaryEndpoints.dfs]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "storageAccount" + ] + }, + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').keyVault)]", + "properties": { + "name": "[parameters('app').keyVault]", + "groupId": "vault", + "privateLinkResourceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "fqdns": [ + "[reference('keyVault').vaultUri]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "keyVault" + ] + }, + "dataFactory::managedVirtualNetwork": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'default')]", + "properties": {}, + "dependsOn": [ + "dataFactory" + ] + }, + "dataFactory::managedIntegrationRuntime": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/integrationRuntimes", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ManagedIntegrationRuntime')]", + "properties": { + "type": "Managed", + "managedVirtualNetwork": { + "referenceName": "default", + "type": "ManagedVirtualNetworkReference" + }, + "typeProperties": { + "computeProperties": { + "location": "[parameters('app').hub.location]", + "dataFlowProperties": { + "computeType": "General", + "coreCount": 8, + "timeToLive": 10, + "cleanup": false, + "customProperties": [] + }, + "copyComputeScaleProperties": { + "dataIntegrationUnit": 16, + "timeToLive": 30 + }, + "pipelineExternalComputeScaleProperties": { + "timeToLive": 30, + "numberOfPipelineNodes": 1, + "numberOfExternalNodes": 1 + } + } + } + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedVirtualNetwork" + ] + }, + "dataFactory::linkedService_keyVault": { + "condition": "[and(variables('usesDataFactory'), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').keyVault)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureKeyVault", + "typeProperties": { + "baseUrl": "[reference(format('Microsoft.KeyVault/vaults/{0}', parameters('app').keyVault), '2023-02-01').vaultUri]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "keyVault" + ] + }, + "dataFactory::linkedService_storageAccount": { + "condition": "[and(variables('usesDataFactory'), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').storage)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureBlobFS", + "typeProperties": { + "url": "[reference(format('Microsoft.Storage/storageAccounts/{0}', parameters('app').storage), '2021-08-01').primaryEndpoints.dfs]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "storageAccount" + ] + }, + "storageAccount::blobService": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]", + "dependsOn": [ + "storageAccount" + ] + }, + "blobEndpoint::blobPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "blobEndpoint" + ] + }, + "dfsEndpoint::dfsPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.dfs.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "dfsEndpoint" + ] + }, + "keyVaultPrivateDnsZone::keyVaultPrivateDnsZoneLink": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), format('{0}-link', replace(format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "properties": { + "virtualNetwork": { + "id": "[parameters('app').hub.routing.networkId]" + }, + "registrationEnabled": false + }, + "dependsOn": [ + "keyVaultPrivateDnsZone" + ] + }, + "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')))]" + } + } + ] + }, + "dependsOn": [ + "keyVaultEndpoint", + "keyVaultPrivateDnsZone" + ] + }, + "appTelemetry": { + "condition": "[parameters('app').hub.options.enableTelemetry]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "[if(lessOrEquals(length(variables('telemetryId')), 64), variables('telemetryId'), substring(variables('telemetryId'), 0, 64))]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Resources/deployments')]", + "properties": "[variables('telemetryProps')]" + }, + "dataFactory": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.DataFactory/factories')]", + "identity": { + "type": "SystemAssigned" + }, + "properties": { + "globalConfigurations": { + "PipelineBillingEnabled": "true" + } + } + }, + "storageRoleAssignments": { + "copy": { + "name": "storageRoleAssignments", + "count": "[length(variables('factoryStorageRoles'))]" + }, + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "name": "[guid(resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage), variables('factoryStorageRoles')[copyIndex()], resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('factoryStorageRoles')[copyIndex()])]", + "principalId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "storageAccount" + ] + }, + "triggerManagerIdentity": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[format('{0}_triggerManager', parameters('app').dataFactory)]", + "location": "[parameters('app').hub.location]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "dependsOn": [ + "dataFactory" + ] + }, + "triggerManagerRoleAssignments": { + "copy": { + "name": "triggerManagerRoleAssignments", + "count": "[length(variables('autoStartRbacRoles'))]" + }, + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]", + "name": "[guid(resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory), variables('autoStartRbacRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', format('{0}_triggerManager', parameters('app').dataFactory)))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('autoStartRbacRoles')[copyIndex()])]", + "principalId": "[reference('triggerManagerIdentity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "triggerManagerIdentity" + ] + }, + "storageAccount": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]", + "location": "[parameters('app').hub.location]", + "sku": { + "name": "[parameters('app').hub.options.storageSku]" + }, + "kind": "BlockBlobStorage", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Storage/storageAccounts')]", + "properties": "[shallowMerge(createArray(variables('storageInfrastructureEncryptionProperties'), createObject('supportsHttpsTrafficOnly', true(), 'allowSharedKeyAccess', true(), 'isHnsEnabled', true(), 'minimumTlsVersion', 'TLS1_2', 'allowBlobPublicAccess', false(), 'publicNetworkAccess', 'Enabled', 'networkAcls', createObject('bypass', 'AzureServices', 'defaultAction', if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')))))]" + }, + "blobPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]" + }, + "blobEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-blob-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "blobLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "blob" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "dfsPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]" + }, + "dfsEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-dfs-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "dfsLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "dfs" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "keyVault": { + "condition": "[variables('usesKeyVault')]", + "type": "Microsoft.KeyVault/vaults", + "apiVersion": "2023-02-01", + "name": "[parameters('app').keyVault]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.KeyVault/vaults')]", + "properties": { + "sku": { + "name": "[parameters('app').hub.options.keyVaultSku]", + "family": "A" + }, + "enabledForDeployment": true, + "enabledForTemplateDeployment": true, + "enabledForDiskEncryption": true, + "enableSoftDelete": true, + "softDeleteRetentionInDays": 90, + "enablePurgeProtection": "[if(parameters('app').hub.options.keyVaultEnablePurgeProtection, true(), null())]", + "enableRbacAuthorization": false, + "createMode": "default", + "tenantId": "[subscription().tenantId]", + "accessPolicies": [ + { + "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "tenantId": "[subscription().tenantId]", + "permissions": { + "secrets": [ + "get" + ] + } + } + ], + "networkAcls": { + "bypass": "AzureServices", + "defaultAction": "[if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')]" + } + }, + "dependsOn": [ + "dataFactory" + ] + }, + "keyVaultPrivateDnsZone": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones')]", + "properties": {} + }, + "keyVaultEndpoint": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-ep', parameters('app').keyVault)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.keyVault]" + }, + "privateLinkServiceConnections": [ + { + "name": "keyVaultLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "groupIds": [ + "vault" + ] + } + } + ] + }, + "dependsOn": [ + "keyVault" + ] + }, + "getKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint", + "getStoragePrivateEndpointConnections", + "keyVault" + ] + }, + "approveKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + }, + "privateEndpointConnections": { + "value": "[reference('getKeyVaultPrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getKeyVaultPrivateEndpointConnections", + "keyVault" + ] + }, + "getStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint", + "stopTriggers", + "storageAccount" + ] + }, + "approveStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + }, + "privateEndpointConnections": { + "value": "[reference('getStoragePrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getStoragePrivateEndpointConnections", + "storageAccount" + ] + }, + "stopTriggers": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.{1}_ADF.StopTriggers', parameters('app').publisher, parameters('app').name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "arguments": { + "value": "[join(createArray(format('-DataFactoryResourceGroup \"{0}\"', resourceGroup().name), format('-DataFactoryName \"{0}\"', parameters('app').dataFactory), '-StopTriggers'), ' ')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + }, + "dependsOn": [ + "appTelemetry", + "dataFactory", + "triggerManagerIdentity", + "triggerManagerRoleAssignments" + ] + } + }, + "outputs": { + "dataFactoryId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Data Factory instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]" + }, + "keyVaultId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Key Vault instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]" + }, + "storageAccountId": { + "type": "string", + "metadata": { + "description": "Resource ID of the storage account instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Principal ID for the managed identity used by Data Factory." + }, + "value": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" + }, + "triggerManagerIdentityName": { + "type": "string", + "metadata": { + "description": "Name of the managed identity used to create and stop ADF triggers." + }, + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + } + } + } + } + }, + "timeZones": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.ManagedExports_TimeZones", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "location": { + "value": "[parameters('app').hub.location]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "7746202288364701295" + } + }, + "parameters": { + "location": { + "type": "string", + "defaultValue": "[resourceGroup().location]", + "metadata": { + "description": "Optional. The location to use for the managed identity and deployment script to auto-start triggers. Default = (resource group location)." + } + }, + "timezoneobject": { + "type": "object", + "defaultValue": { + "australiaeast": "AUS Eastern Standard Time", + "australiacentral": "AUS Eastern Standard Time", + "australiacentral2": "AUS Eastern Standard Time", + "australiasoutheast": "AUS Eastern Standard Time", + "brazilsouth": "E. South America Standard Time", + "canadacentral": "Central Standard Time", + "canadaeast": "Eastern Standard Time", + "centralindia": "India Standard Time", + "centralus": "Central Standard Time", + "eastasia": "China Standard Time", + "eastus": "Eastern Standard Time", + "eastus2": "Eastern Standard Time", + "francecentral": "W. Europe Standard Time", + "germanynorth": "W. Europe Standard Time", + "germanywestcentral": "W. Europe Standard Time", + "japaneast": "Japan Standard Time", + "japanwest": "Japan Standard Time", + "koreacentral": "Korea Standard Time", + "koreasouth": "Korea Standard Time", + "northcentralus": "Central Standard Time", + "northeurope": "GMT Standard Time", + "norwayeast": "W. Europe Standard Time", + "norwaywest": "W. Europe Standard Time", + "southcentralus": "Central Standard Time", + "southindia": "India Standard Time", + "southeastasia": "Singapore Standard Time", + "switzerlandnorth": "W. Europe Standard Time", + "switzerlandwest": "W. Europe Standard Time", + "uksouth": "GMT Standard Time", + "ukwest": "GMT Standard Time", + "westcentralus": "Central Standard Time", + "westeurope": "W. Europe Standard Time", + "westindia": "India Standard Time", + "westus": "Pacific Standard Time", + "westus2": "Pacific Standard Time" + } + }, + "utchrs": { + "type": "string", + "defaultValue": "[utcNow('hh')]" + }, + "utcmins": { + "type": "string", + "defaultValue": "[utcNow('mm')]" + }, + "utcsecs": { + "type": "string", + "defaultValue": "[utcNow('ss')]" + } + }, + "variables": { + "loc": "[toLower(replace(parameters('location'), ' ', ''))]", + "timezone": "[coalesce(tryGet(parameters('timezoneobject'), variables('loc')), 'Universal Coordinated Time')]" + }, + "resources": [], + "outputs": { + "AzureRegion": { + "type": "string", + "value": "[parameters('location')]" + }, + "Timezone": { + "type": "string", + "value": "[variables('timezone')]" + }, + "UtcHours": { + "type": "string", + "value": "[parameters('utchrs')]" + }, + "UtcMinutes": { + "type": "string", + "value": "[parameters('utcmins')]" + }, + "UtcSeconds": { + "type": "string", + "value": "[parameters('utcsecs')]" + } + } + } + } + }, + "trigger_SettingsUpdated": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core_SettingsUpdatedTrigger", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "dataFactoryName": { + "value": "[parameters('app').dataFactory]" + }, + "triggerName": { + "value": "[format('{0}_SettingsUpdated', variables('CONFIG'))]" + }, + "pipelineName": { + "value": "[format('{0}_ConfigureExports', variables('CONFIG'))]" + }, + "pipelineParameters": { + "value": {} + }, + "storageAccountName": { + "value": "[parameters('app').storage]" + }, + "storageContainer": { + "value": "[variables('CONFIG')]" + }, + "storagePathEndsWith": { + "value": "settings.json" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "12667288354191065454" + } + }, + "parameters": { + "dataFactoryName": { + "type": "string", + "metadata": { + "description": "Required. Name of the publisher-specific Data Factory instance." + } + }, + "triggerName": { + "type": "string", + "metadata": { + "description": "Required. Name of the Data Factory trigger to create or update." + } + }, + "storageAccountName": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Azure storage container to monitor for updates and trigger events for." + } + }, + "storageContainer": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Azure storage container to monitor for updates and trigger events for." + } + }, + "storagePathStartsWith": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Beginning of the storage path within the specified storageContainer to monitor for updates and trigger events for." + } + }, + "storagePathEndsWith": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. End of the storage path to monitor for updates and trigger events for." + } + }, + "pipelineName": { + "type": "string", + "metadata": { + "description": "Required. Name of the Data Factory pipeline to execute when the trigger is executed." + } + }, + "pipelineParameters": { + "type": "object", + "metadata": { + "description": "Required. Parameters to pass to the pipeline when the trigger is executed." + } + } + }, + "resources": [ + { + "condition": "[not(empty(parameters('storageAccountName')))]", + "type": "Microsoft.DataFactory/factories/triggers", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('dataFactoryName'), parameters('triggerName'))]", + "properties": { + "annotations": [], + "pipelines": [ + { + "pipelineReference": { + "referenceName": "[parameters('pipelineName')]", + "type": "PipelineReference" + }, + "parameters": "[parameters('pipelineParameters')]" + } + ], + "type": "BlobEventsTrigger", + "typeProperties": { + "blobPathBeginsWith": "[format('/{0}/blobs/{1}', parameters('storageContainer'), parameters('storagePathStartsWith'))]", + "blobPathEndsWith": "[parameters('storagePathEndsWith')]", + "ignoreEmptyBlobs": true, + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]", + "events": [ + "Microsoft.Storage.BlobCreated" + ] + } + } + } + ] + } + }, + "dependsOn": [ + "dataFactory::pipeline_ConfigureExports" + ] + } + } + } + }, + "dependsOn": [ + "cmExports" + ] + }, + "analytics": { + "condition": "[or(variables('useFabric'), variables('useAzureDataExplorer'))]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[__bicep.newApp(variables('hub'), 'Microsoft.FinOpsHubs', 'Analytics')]" + }, + "fabricQueryUri": { + "value": "[parameters('fabricQueryUri')]" + }, + "fabricCapacityUnits": { + "value": "[parameters('fabricCapacityUnits')]" + }, + "clusterName": { + "value": "[parameters('dataExplorerName')]" + }, + "clusterSku": { + "value": "[parameters('dataExplorerSku')]" + }, + "clusterCapacity": { + "value": "[parameters('dataExplorerCapacity')]" + }, + "rawRetentionInDays": { + "value": "[parameters('dataExplorerRawRetentionInDays')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6187428357849376543" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "privateRoutingForLinkedServices": { + "parameters": [ + { + "$ref": "#/definitions/_1.HubProperties", + "name": "hub" + } + ], + "output": { + "type": "object", + "value": "[if(parameters('hub').options.privateRouting, createObject('connectVia', createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference')), createObject())]" + }, + "metadata": { + "description": "Returns an object that represents the properties needed to enable private routing for linked services. Use property expansion (`...value`) to apply to a linkedServices resource.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "clusterName": { + "type": "string", + "defaultValue": "", + "maxLength": 22, + "metadata": { + "description": "Optional. Name of the Azure Data Explorer cluster to use for advanced analytics. If empty, Azure Data Explorer will not be deployed. Required to use with Power BI if you have more than $2-5M/mo in costs being monitored. Default: \"\" (do not use)." + } + }, + "clusterSku": { + "type": "string", + "defaultValue": "Dev(No SLA)_Standard_E2a_v4", + "allowedValues": [ + "Dev(No SLA)_Standard_E2a_v4", + "Dev(No SLA)_Standard_D11_v2", + "Standard_D11_v2", + "Standard_D12_v2", + "Standard_D13_v2", + "Standard_D14_v2", + "Standard_D16d_v5", + "Standard_D32d_v4", + "Standard_D32d_v5", + "Standard_DS13_v2+1TB_PS", + "Standard_DS13_v2+2TB_PS", + "Standard_DS14_v2+3TB_PS", + "Standard_DS14_v2+4TB_PS", + "Standard_E2a_v4", + "Standard_E2ads_v5", + "Standard_E2d_v4", + "Standard_E2d_v5", + "Standard_E4a_v4", + "Standard_E4ads_v5", + "Standard_E4d_v4", + "Standard_E4d_v5", + "Standard_E8a_v4", + "Standard_E8ads_v5", + "Standard_E8as_v4+1TB_PS", + "Standard_E8as_v4+2TB_PS", + "Standard_E8as_v5+1TB_PS", + "Standard_E8as_v5+2TB_PS", + "Standard_E8d_v4", + "Standard_E8d_v5", + "Standard_E8s_v4+1TB_PS", + "Standard_E8s_v4+2TB_PS", + "Standard_E8s_v5+1TB_PS", + "Standard_E8s_v5+2TB_PS", + "Standard_E16a_v4", + "Standard_E16ads_v5", + "Standard_E16as_v4+3TB_PS", + "Standard_E16as_v4+4TB_PS", + "Standard_E16as_v5+3TB_PS", + "Standard_E16as_v5+4TB_PS", + "Standard_E16d_v4", + "Standard_E16d_v5", + "Standard_E16s_v4+3TB_PS", + "Standard_E16s_v4+4TB_PS", + "Standard_E16s_v5+3TB_PS", + "Standard_E16s_v5+4TB_PS", + "Standard_E64i_v3", + "Standard_E80ids_v4", + "Standard_EC8ads_v5", + "Standard_EC8as_v5+1TB_PS", + "Standard_EC8as_v5+2TB_PS", + "Standard_EC16ads_v5", + "Standard_EC16as_v5+3TB_PS", + "Standard_EC16as_v5+4TB_PS", + "Standard_L4s", + "Standard_L8as_v3", + "Standard_L8s", + "Standard_L8s_v2", + "Standard_L8s_v3", + "Standard_L16as_v3", + "Standard_L16s", + "Standard_L16s_v2", + "Standard_L16s_v3", + "Standard_L32as_v3", + "Standard_L32s_v3" + ], + "metadata": { + "description": "Optional. Name of the Azure Data Explorer SKU. Default: \"Dev(No SLA)_Standard_E2a_v4\"." + } + }, + "clusterCapacity": { + "type": "int", + "defaultValue": 1, + "minValue": 1, + "maxValue": 1000, + "metadata": { + "description": "Optional. Number of nodes to use in the cluster. Allowed values: 1 for the Basic SKU tier and 2-1000 for Standard. Default: 1 for dev/test SKUs, 2 for standard SKUs." + } + }, + "fabricQueryUri": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Microsoft Fabric eventhouse query URI. Default: \"\" (do not use)." + } + }, + "fabricCapacityUnits": { + "type": "int", + "defaultValue": 2, + "minValue": 1, + "maxValue": 2048, + "metadata": { + "description": "Optional. Number of capacity units for the Microsoft Fabric capacity. This is the number in your Fabric SKU (e.g., Trial = 1, F2 = 2, F64 = 64). This is used to manage parallelization in data pipelines. If you change capacity, please redeploy the template. Allowed values: 1 for the Fabric trial and 2-2048 based on the assigned Fabric capacity (e.g., F2-F2048). Default: 2." + } + }, + "forceUpdateTag": { + "type": "string", + "defaultValue": "[utcNow()]", + "metadata": { + "description": "Optional. Forces the table to be updated if different from the last time it was deployed." + } + }, + "continueOnErrors": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + } + }, + "rawRetentionInDays": { + "type": "int", + "metadata": { + "description": "Required. Number of days of data to retain in the Data Explorer *_raw tables." + } + } + }, + "variables": { + "$fxv#0": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData/Internal')\r\n_resource_type_1(id: string) {\r\n dynamic({\r\n \"arizeai.observabilityeval/organizations\": { \"SingularDisplayName\": \"Azure Native Arize AI Cloud Service\" }\r\n ,\"astronomer.astro/organizations\": { \"SingularDisplayName\": \"Astro Organization\" }\r\n ,\"citrix.services/xenappessentials\": { \"SingularDisplayName\": \"Citrix Virtual Apps Essentials\" }\r\n ,\"citrix.services/xendesktopessentials\": { \"SingularDisplayName\": \"Citrix Virtual Desktops Essentials\" }\r\n ,\"commvault.contentstore/cloudaccounts\": { \"SingularDisplayName\": \"Commvault Cloud Account\" }\r\n ,\"commvault.contentstore/cloudaccounts/plans\": { \"SingularDisplayName\": \"Commvault.ContentStore cloud accounts plan\" }\r\n ,\"commvault.contentstore/cloudaccounts/protectiongroups\": { \"SingularDisplayName\": \"Commvault.ContentStore cloud accounts protection group\" }\r\n ,\"commvault.contentstore/cloudaccounts/protectiongroups/protecteditems\": { \"SingularDisplayName\": \"Commvault.ContentStore cloud accounts protection groups protected item\" }\r\n ,\"commvault.contentstore/cloudaccounts/storages\": { \"SingularDisplayName\": \"Commvault.ContentStore cloud accounts storage\" }\r\n ,\"dell.storage/filesystems\": { \"SingularDisplayName\": \"Dell PowerScale\" }\r\n ,\"dynatrace.observability/monitors\": { \"SingularDisplayName\": \"Dynatrace\" }\r\n ,\"github.network/networksettings\": { \"SingularDisplayName\": \"GitHub.Network network setting\" }\r\n ,\"informatica.datamanagement/organizations\": { \"SingularDisplayName\": \"Informatica Organization\" }\r\n ,\"lambdatest.hyperexecute/organizations\": { \"SingularDisplayName\": \"Azure Native LambdaTest - HyperExecute Cloud Service\" }\r\n ,\"microsoft.aad/domainservices\": { \"SingularDisplayName\": \"Microsoft Entra Domain Services\" }\r\n ,\"microsoft.aadiam/diagnosticsettings\": { \"SingularDisplayName\": \"Microsoft.aadiam diagnostic setting\" }\r\n ,\"microsoft.aadiam/privatelinkforazuread\": { \"SingularDisplayName\": \"Private Link for Microsoft Entra ID\" }\r\n ,\"microsoft.advisor/advisorscore\": { \"SingularDisplayName\": \"Microsoft.Advisor advisor score\" }\r\n ,\"microsoft.advisor/assessments\": { \"SingularDisplayName\": \"Microsoft.Advisor assessment\" }\r\n ,\"microsoft.advisor/configurations\": { \"SingularDisplayName\": \"Microsoft.Advisor configuration\" }\r\n ,\"microsoft.advisor/generaterecommendations\": { \"SingularDisplayName\": \"Microsoft.Advisor generate recommendation\" }\r\n ,\"microsoft.advisor/metadata\": { \"SingularDisplayName\": \"Microsoft.Advisor metadata\" }\r\n ,\"microsoft.advisor/recommendations\": { \"SingularDisplayName\": \"Microsoft.Advisor recommendation\" }\r\n ,\"microsoft.advisor/recommendations/suppressions\": { \"SingularDisplayName\": \"Microsoft.Advisor recommendations suppression\" }\r\n ,\"microsoft.advisor/resiliencyreviews\": { \"SingularDisplayName\": \"Microsoft.Advisor resiliency review\" }\r\n ,\"microsoft.agfoodplatform/farmbeats\": { \"SingularDisplayName\": \"Azure Data Manager for Agriculture\" }\r\n ,\"microsoft.agfoodplatform/farmbeatsextensiondefinitions\": { \"SingularDisplayName\": \"Microsoft.AgFoodPlatform farm beats extension definition\" }\r\n ,\"microsoft.agfoodplatform/farmbeatssolutiondefinitions\": { \"SingularDisplayName\": \"Microsoft.AgFoodPlatform farm beats solution definition\" }\r\n ,\"microsoft.agricultureplatform/agriservices\": { \"SingularDisplayName\": \"Agriculture data solutions\" }\r\n ,\"microsoft.akshybrid/agentpools\": { \"SingularDisplayName\": \"Microsoft.AksHybrid agent pool\" }\r\n ,\"microsoft.akshybrid/provisionedclusters\": { \"SingularDisplayName\": \"Microsoft.AksHybrid provisioned cluster\" }\r\n ,\"microsoft.akshybrid/upgradeprofiles\": { \"SingularDisplayName\": \"Microsoft.AksHybrid upgrade profile\" }\r\n ,\"microsoft.alertsmanagement/actionrules\": { \"SingularDisplayName\": \"Alert processing rule\" }\r\n ,\"microsoft.alertsmanagement/alerts\": { \"SingularDisplayName\": \"Microsoft.AlertsManagement alert\" }\r\n ,\"microsoft.alertsmanagement/alerts/enrichments\": { \"SingularDisplayName\": \"Microsoft.AlertsManagement alerts enrichment\" }\r\n ,\"microsoft.alertsmanagement/prometheusrulegroups\": { \"SingularDisplayName\": \"Prometheus rule group\" }\r\n ,\"microsoft.alertsmanagement/smartdetectoralertrules\": { \"SingularDisplayName\": \"Smart detector alert rule\" }\r\n ,\"microsoft.alertsmanagement/smartgroups\": { \"SingularDisplayName\": \"Microsoft.AlertsManagement smart group\" }\r\n ,\"microsoft.alertsmanagement/tenantactivitylogalerts\": { \"SingularDisplayName\": \"Microsoft.AlertsManagement tenant activity log alert\" }\r\n ,\"microsoft.all/arcvirtualmachines\": { \"SingularDisplayName\": \"Azure Arc virtual machine\" }\r\n ,\"microsoft.all/hcivirtualmachines\": { \"SingularDisplayName\": \"Azure Local Virtual Machine - Azure Arc\" }\r\n ,\"microsoft.all/virtualmachines\": { \"SingularDisplayName\": \"Virtual machine\" }\r\n ,\"microsoft.analysisservices/servers\": { \"SingularDisplayName\": \"Analysis Services server\" }\r\n ,\"microsoft.anybuild/clusters\": { \"SingularDisplayName\": \"AnyBuild cluster\" }\r\n ,\"microsoft.apicenter/deletedservices\": { \"SingularDisplayName\": \"Microsoft.ApiCenter deleted service\" }\r\n ,\"microsoft.apicenter/services\": { \"SingularDisplayName\": \"API Center\" }\r\n ,\"microsoft.apicenter/services/workspaces\": { \"SingularDisplayName\": \"Workspace\" }\r\n ,\"microsoft.apimanagement/gateways\": { \"SingularDisplayName\": \"API Management gateway\" }\r\n ,\"microsoft.apimanagement/gateways/configconnections\": { \"SingularDisplayName\": \"Microsoft.ApiManagement gateways config connection\" }\r\n ,\"microsoft.apimanagement/service\": { \"SingularDisplayName\": \"API Management service\" }\r\n ,\"microsoft.apimanagement/service/workspaces\": { \"SingularDisplayName\": \"Workspace\" }\r\n ,\"microsoft.apisecurity/defendersettings\": { \"SingularDisplayName\": \"Microsoft.ApiSecurity defender setting\" }\r\n ,\"microsoft.app/agents\": { \"SingularDisplayName\": \"SRE Agent\" }\r\n ,\"microsoft.app/builders\": { \"SingularDisplayName\": \"Microsoft.App builder\" }\r\n ,\"microsoft.app/builders/builds\": { \"SingularDisplayName\": \"Microsoft.App builders build\" }\r\n ,\"microsoft.app/connectedenvironments\": { \"SingularDisplayName\": \"Container Apps Connected Environment\" }\r\n ,\"microsoft.app/containerapps\": { \"SingularDisplayName\": \"Container App\" }\r\n ,\"microsoft.app/jobs\": { \"SingularDisplayName\": \"Container App Job\" }\r\n ,\"microsoft.app/logicapps\": { \"SingularDisplayName\": \"Logic app\" }\r\n ,\"microsoft.app/logicapps/workflows\": { \"SingularDisplayName\": \"Logic app workflow\" }\r\n ,\"microsoft.app/managedenvironments\": { \"SingularDisplayName\": \"Container Apps Environment\" }\r\n ,\"microsoft.app/sessionpools\": { \"SingularDisplayName\": \"Container App Session Pool\" }\r\n ,\"microsoft.app/spaces\": { \"SingularDisplayName\": \"App Space\" }\r\n ,\"microsoft.appassessment/migrateprojects\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate project\" }\r\n ,\"microsoft.appassessment/migrateprojects/assessments\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects assessment\" }\r\n ,\"microsoft.appassessment/migrateprojects/assessments/assessedapplications\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects assessments assessed application\" }\r\n ,\"microsoft.appassessment/migrateprojects/assessments/assessedmachines\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects assessments assessed machine\" }\r\n ,\"microsoft.appassessment/migrateprojects/assessments/machinestoassess\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects assessments machines to asses\" }\r\n ,\"microsoft.appassessment/migrateprojects/sites\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects site\" }\r\n ,\"microsoft.appassessment/migrateprojects/sites/applianceconfigurations\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects sites appliance configuration\" }\r\n ,\"microsoft.appcomplianceautomation/reports\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation report\" }\r\n ,\"microsoft.appcomplianceautomation/reports/evidences\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation reports evidence\" }\r\n ,\"microsoft.appcomplianceautomation/reports/scopingconfigurations\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation reports scoping configuration\" }\r\n ,\"microsoft.appcomplianceautomation/reports/snapshots\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation reports snapshot\" }\r\n ,\"microsoft.appcomplianceautomation/reports/snapshots/controls\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation reports snapshots control\" }\r\n ,\"microsoft.appcomplianceautomation/reports/webhooks\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation reports webhook\" }\r\n ,\"microsoft.appconfiguration/configurationstores\": { \"SingularDisplayName\": \"App Configuration\" }\r\n ,\"microsoft.applicationmigration/discoveryhubs\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration discovery hub\" }\r\n ,\"microsoft.applicationmigration/discoveryhubs/applications\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration discovery hubs application\" }\r\n ,\"microsoft.applicationmigration/discoveryhubs/applications/members\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration discovery hubs applications member\" }\r\n ,\"microsoft.applicationmigration/pgsqlsites\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration pgsqlsite\" }\r\n ,\"microsoft.applicationmigration/pgsqlsites/agents\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration pgsqlsites agent\" }\r\n ,\"microsoft.applicationmigration/pgsqlsites/pgsqldatabases\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration pgsqlsites pgsqldatabase\" }\r\n ,\"microsoft.applicationmigration/pgsqlsites/pgsqlinstances\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration pgsqlsites pgsqlinstance\" }\r\n ,\"microsoft.appplatform/spring\": { \"SingularDisplayName\": \"Azure Spring Apps\" }\r\n ,\"microsoft.appsecurity/appprotectmanagedrulesetmanifests\": { \"SingularDisplayName\": \"Microsoft.AppSecurity app protect managed rule set manifest\" }\r\n ,\"microsoft.appsecurity/policies\": { \"SingularDisplayName\": \"App Protect Policy\" }\r\n ,\"microsoft.arc/all\": { \"SingularDisplayName\": \"Azure Arc enabled resource\" }\r\n ,\"microsoft.arc/allfairfax\": { \"SingularDisplayName\": \"Azure Arc enabled resource\" }\r\n ,\"microsoft.arc/kubernetesresources\": { \"SingularDisplayName\": \"Azure Arc Kubernetes cluster\" }\r\n ,\"microsoft.arc/kubernetesresourcesfairfax\": { \"SingularDisplayName\": \"Azure Arc Kubernetes cluster\" }\r\n ,\"microsoft.arcnetworking/arcnwloadbalancers\": { \"SingularDisplayName\": \"Microsoft.ArcNetworking arc nw load balancer\" }\r\n ,\"microsoft.aszlabhardware/labservers\": { \"SingularDisplayName\": \"Microsoft.AszLabHardware labserver\" }\r\n ,\"microsoft.aszlabhardware/reservations\": { \"SingularDisplayName\": \"Microsoft.AszLabHardware reservation\" }\r\n ,\"microsoft.aszlabhardware/reservations/servers\": { \"SingularDisplayName\": \"Microsoft.AszLabHardware reservations server\" }\r\n ,\"microsoft.aszlabhardware/servers\": { \"SingularDisplayName\": \"Microsoft.AszLabHardware server\" }\r\n ,\"microsoft.attestation/attestationproviders\": { \"SingularDisplayName\": \"Attestation provider\" }\r\n ,\"microsoft.authorization/accessreviewhistorydefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization access review history definition\" }\r\n ,\"microsoft.authorization/accessreviewscheduledefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization access review schedule definition\" }\r\n ,\"microsoft.authorization/accessreviewscheduledefinitions/instances\": { \"SingularDisplayName\": \"Microsoft.Authorization access review schedule definitions instance\" }\r\n ,\"microsoft.authorization/accessreviewscheduledefinitions/instances/decisions\": { \"SingularDisplayName\": \"Microsoft.Authorization access review schedule definitions instances decision\" }\r\n ,\"microsoft.authorization/accessreviewschedulesettings\": { \"SingularDisplayName\": \"Microsoft.Authorization access review schedule setting\" }\r\n ,\"microsoft.authorization/datapolicymanifests\": { \"SingularDisplayName\": \"Microsoft.Authorization data policy manifest\" }\r\n ,\"microsoft.authorization/denyassignments\": { \"SingularDisplayName\": \"Microsoft.Authorization deny assignment\" }\r\n ,\"microsoft.authorization/locks\": { \"SingularDisplayName\": \"Microsoft.Authorization lock\" }\r\n ,\"microsoft.authorization/policyassignments\": { \"SingularDisplayName\": \"Microsoft.Authorization policy assignment\" }\r\n ,\"microsoft.authorization/policydefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization policy definition\" }\r\n ,\"microsoft.authorization/policydefinitions/versions\": { \"SingularDisplayName\": \"Microsoft.Authorization policy definitions version\" }\r\n ,\"microsoft.authorization/policyexemptions\": { \"SingularDisplayName\": \"Microsoft.Authorization policy exemption\" }\r\n ,\"microsoft.authorization/policysetdefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization policy set definition\" }\r\n ,\"microsoft.authorization/policysetdefinitions/versions\": { \"SingularDisplayName\": \"Microsoft.Authorization policy set definitions version\" }\r\n ,\"microsoft.authorization/privatelinkassociations\": { \"SingularDisplayName\": \"Microsoft.Authorization private link association\" }\r\n ,\"microsoft.authorization/provideroperations\": { \"SingularDisplayName\": \"Microsoft.Authorization provider operation\" }\r\n ,\"microsoft.authorization/resourcemanagementprivatelinks\": { \"SingularDisplayName\": \"Resource management private link\" }\r\n ,\"microsoft.authorization/roleassignmentapprovals\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment approval\" }\r\n ,\"microsoft.authorization/roleassignmentapprovals/stages\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment approvals stage\" }\r\n ,\"microsoft.authorization/roleassignments\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment\" }\r\n ,\"microsoft.authorization/roleassignmentscheduleinstances\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment schedule instance\" }\r\n ,\"microsoft.authorization/roleassignmentschedulerequests\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment schedule request\" }\r\n ,\"microsoft.authorization/roleassignmentschedules\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment schedule\" }\r\n ,\"microsoft.authorization/roledefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization role definition\" }\r\n ,\"microsoft.authorization/roleeligibilityscheduleinstances\": { \"SingularDisplayName\": \"Microsoft.Authorization role eligibility schedule instance\" }\r\n ,\"microsoft.authorization/roleeligibilityschedulerequests\": { \"SingularDisplayName\": \"Microsoft.Authorization role eligibility schedule request\" }\r\n ,\"microsoft.authorization/roleeligibilityschedules\": { \"SingularDisplayName\": \"Microsoft.Authorization role eligibility schedule\" }\r\n ,\"microsoft.authorization/rolemanagementalertconfigurations\": { \"SingularDisplayName\": \"Microsoft.Authorization role management alert configuration\" }\r\n ,\"microsoft.authorization/rolemanagementalertdefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization role management alert definition\" }\r\n ,\"microsoft.authorization/rolemanagementalertoperations\": { \"SingularDisplayName\": \"Microsoft.Authorization role management alert operation\" }\r\n ,\"microsoft.authorization/rolemanagementalerts\": { \"SingularDisplayName\": \"Microsoft.Authorization role management alert\" }\r\n ,\"microsoft.authorization/rolemanagementalerts/alertincidents\": { \"SingularDisplayName\": \"Microsoft.Authorization role management alerts alert incident\" }\r\n ,\"microsoft.authorization/rolemanagementpolicies\": { \"SingularDisplayName\": \"Microsoft.Authorization role management policy\" }\r\n ,\"microsoft.authorization/rolemanagementpolicyassignments\": { \"SingularDisplayName\": \"Microsoft.Authorization role management policy assignment\" }\r\n ,\"microsoft.automanage/bestpractices\": { \"SingularDisplayName\": \"Microsoft.Automanage best practice\" }\r\n ,\"microsoft.automanage/bestpractices/versions\": { \"SingularDisplayName\": \"Microsoft.Automanage best practices version\" }\r\n ,\"microsoft.automanage/configurationprofileassignments\": { \"SingularDisplayName\": \"Microsoft.Automanage configuration profile assignment\" }\r\n ,\"microsoft.automanage/configurationprofileassignments/reports\": { \"SingularDisplayName\": \"Microsoft.Automanage configuration profile assignments report\" }\r\n ,\"microsoft.automanage/configurationprofiles\": { \"SingularDisplayName\": \"Microsoft.Automanage configuration profile\" }\r\n ,\"microsoft.automanage/configurationprofiles/versions\": { \"SingularDisplayName\": \"Microsoft.Automanage configuration profiles version\" }\r\n ,\"microsoft.automanage/serviceprincipals\": { \"SingularDisplayName\": \"ServicePrincipals\" }\r\n ,\"microsoft.automation/automationaccounts\": { \"SingularDisplayName\": \"Automation account\" }\r\n ,\"microsoft.automation/automationaccounts/hybridrunbookworkergroups\": { \"SingularDisplayName\": \"Automation hybrid worker group\" }\r\n ,\"microsoft.automation/automationaccounts/runbooks\": { \"SingularDisplayName\": \"Automation runbook\" }\r\n ,\"microsoft.autonomousdevelopmentplatform/accounts\": { \"SingularDisplayName\": \"Microsoft.AutonomousDevelopmentPlatform account\" }\r\n ,\"microsoft.autonomousdevelopmentplatform/accounts/datapools\": { \"SingularDisplayName\": \"Microsoft.AutonomousDevelopmentPlatform accounts data pool\" }\r\n ,\"microsoft.autonomousdevelopmentplatform/workspaces\": { \"SingularDisplayName\": \"Microsoft.AutonomousDevelopmentPlatform workspace\" }\r\n ,\"microsoft.avs/privateclouds\": { \"SingularDisplayName\": \"Azure VMware Solution private cloud\" }\r\n ,\"microsoft.awsconnector/accessanalyzeranalyzers\": { \"SingularDisplayName\": \"Access Analyzer Analyzer\" }\r\n ,\"microsoft.awsconnector/acmcertificatesummaries\": { \"SingularDisplayName\": \"ACM Certificate Summary\" }\r\n ,\"microsoft.awsconnector/apigatewayrestapis\": { \"SingularDisplayName\": \"Api Gateway Rest Api\" }\r\n ,\"microsoft.awsconnector/apigatewaystages\": { \"SingularDisplayName\": \"Api Gateway Stage\" }\r\n ,\"microsoft.awsconnector/applicationautoscalingscalabletargets\": { \"SingularDisplayName\": \"Application Auto Scaling Scalable Target\" }\r\n ,\"microsoft.awsconnector/appsyncgraphqlapis\": { \"SingularDisplayName\": \"App Sync Graphql Api\" }\r\n ,\"microsoft.awsconnector/autoscalingautoscalinggroups\": { \"SingularDisplayName\": \"Auto Scaling Auto Scaling Group\" }\r\n ,\"microsoft.awsconnector/cloudformationstacks\": { \"SingularDisplayName\": \"Cloud Formation Stack\" }\r\n ,\"microsoft.awsconnector/cloudformationstacksets\": { \"SingularDisplayName\": \"Cloud Formation Stack Set\" }\r\n ,\"microsoft.awsconnector/cloudfrontdistributions\": { \"SingularDisplayName\": \"Cloud Front Distribution\" }\r\n ,\"microsoft.awsconnector/cloudtrailtrails\": { \"SingularDisplayName\": \"Cloud Trail Trail\" }\r\n ,\"microsoft.awsconnector/cloudwatchalarms\": { \"SingularDisplayName\": \"Cloud Watch Alarm\" }\r\n ,\"microsoft.awsconnector/codebuildprojects\": { \"SingularDisplayName\": \"Code Build Project\" }\r\n ,\"microsoft.awsconnector/codebuildsourcecredentialsinfos\": { \"SingularDisplayName\": \"Code Build Source Credentials Info\" }\r\n ,\"microsoft.awsconnector/configserviceconfigurationrecorders\": { \"SingularDisplayName\": \"Config Service Configuration Recorder\" }\r\n ,\"microsoft.awsconnector/configserviceconfigurationrecorderstatuses\": { \"SingularDisplayName\": \"Config Service Configuration Recorder Status\" }\r\n ,\"microsoft.awsconnector/configservicedeliverychannels\": { \"SingularDisplayName\": \"Config Service Delivery Channel\" }\r\n ,\"microsoft.awsconnector/databasemigrationservicereplicationinstances\": { \"SingularDisplayName\": \"Database Migration Service Replication Instance\" }\r\n ,\"microsoft.awsconnector/daxclusters\": { \"SingularDisplayName\": \"DAX Cluster\" }\r\n ,\"microsoft.awsconnector/dynamodbcontinuousbackupsdescriptions\": { \"SingularDisplayName\": \"Dynamo DB Continuous Backups Description\" }\r\n ,\"microsoft.awsconnector/dynamodbtables\": { \"SingularDisplayName\": \"Dynamo DB Table\" }\r\n ,\"microsoft.awsconnector/ec2accountattributes\": { \"SingularDisplayName\": \"EC2 Account Attribute\" }\r\n ,\"microsoft.awsconnector/ec2addresses\": { \"SingularDisplayName\": \"EC2 Address\" }\r\n ,\"microsoft.awsconnector/ec2flowlogs\": { \"SingularDisplayName\": \"EC2 Flow Log\" }\r\n ,\"microsoft.awsconnector/ec2images\": { \"SingularDisplayName\": \"EC2 Image\" }\r\n ,\"microsoft.awsconnector/ec2instances\": { \"SingularDisplayName\": \"Microsoft.AwsConnector ec2 instance\" }\r\n ,\"microsoft.awsconnector/ec2instancestatuses\": { \"SingularDisplayName\": \"EC2 Instance Status\" }\r\n ,\"microsoft.awsconnector/ec2ipams\": { \"SingularDisplayName\": \"EC2 Ipam\" }\r\n ,\"microsoft.awsconnector/ec2keypairs\": { \"SingularDisplayName\": \"EC2 Key Pair\" }\r\n ,\"microsoft.awsconnector/ec2networkacls\": { \"SingularDisplayName\": \"EC2 Network Acl\" }\r\n ,\"microsoft.awsconnector/ec2networkinterfaces\": { \"SingularDisplayName\": \"EC2 Network Interface\" }\r\n ,\"microsoft.awsconnector/ec2routetables\": { \"SingularDisplayName\": \"EC2 Route Table\" }\r\n ,\"microsoft.awsconnector/ec2securitygroups\": { \"SingularDisplayName\": \"EC2 Security Group\" }\r\n ,\"microsoft.awsconnector/ec2snapshots\": { \"SingularDisplayName\": \"EC2 Snapshot\" }\r\n ,\"microsoft.awsconnector/ec2subnets\": { \"SingularDisplayName\": \"EC2 Subnet\" }\r\n ,\"microsoft.awsconnector/ec2volumes\": { \"SingularDisplayName\": \"EC2 Volume\" }\r\n ,\"microsoft.awsconnector/ec2vpcendpoints\": { \"SingularDisplayName\": \"EC2 VPCEndpoint\" }\r\n ,\"microsoft.awsconnector/ec2vpcpeeringconnections\": { \"SingularDisplayName\": \"EC2 VPCPeering Connection\" }\r\n ,\"microsoft.awsconnector/ec2vpcs\": { \"SingularDisplayName\": \"EC2 VPC\" }\r\n ,\"microsoft.awsconnector/ecrimagedetails\": { \"SingularDisplayName\": \"ECR Image Detail\" }\r\n ,\"microsoft.awsconnector/ecrrepositories\": { \"SingularDisplayName\": \"ECR Repository\" }\r\n ,\"microsoft.awsconnector/ecsclusters\": { \"SingularDisplayName\": \"ECS Cluster\" }\r\n ,\"microsoft.awsconnector/ecsservices\": { \"SingularDisplayName\": \"ECS Service\" }\r\n ,\"microsoft.awsconnector/ecstaskdefinitions\": { \"SingularDisplayName\": \"ECS Task Definition\" }\r\n ,\"microsoft.awsconnector/efsfilesystems\": { \"SingularDisplayName\": \"EFS File System\" }\r\n ,\"microsoft.awsconnector/efsmounttargets\": { \"SingularDisplayName\": \"EFS Mount Target\" }\r\n ,\"microsoft.awsconnector/eksnodegroups\": { \"SingularDisplayName\": \"EKS Nodegroup\" }\r\n ,\"microsoft.awsconnector/elasticbeanstalkapplications\": { \"SingularDisplayName\": \"Elastic Beanstalk Application\" }\r\n ,\"microsoft.awsconnector/elasticbeanstalkconfigurationtemplates\": { \"SingularDisplayName\": \"Elastic Beanstalk Configuration Template\" }\r\n ,\"microsoft.awsconnector/elasticbeanstalkenvironments\": { \"SingularDisplayName\": \"Elastic Beanstalk Environment\" }\r\n ,\"microsoft.awsconnector/elasticloadbalancingv2listeners\": { \"SingularDisplayName\": \"Elastic Load Balancing V2 Listener\" }\r\n ,\"microsoft.awsconnector/elasticloadbalancingv2loadbalancers\": { \"SingularDisplayName\": \"Elastic Load Balancing V2 Load Balancer\" }\r\n ,\"microsoft.awsconnector/elasticloadbalancingv2targetgroups\": { \"SingularDisplayName\": \"Elastic Load Balancing V2 Target Group\" }\r\n ,\"microsoft.awsconnector/elasticloadbalancingv2targethealthdescriptions\": { \"SingularDisplayName\": \"Elastic Load Balancing v2 Target Health Description\" }\r\n ,\"microsoft.awsconnector/elasticsearchdomains\": { \"SingularDisplayName\": \"Elasticsearch Domain\" }\r\n ,\"microsoft.awsconnector/emrclusters\": { \"SingularDisplayName\": \"EMR Cluster\" }\r\n ,\"microsoft.awsconnector/guarddutydetectors\": { \"SingularDisplayName\": \"Guard Duty Detector\" }\r\n ,\"microsoft.awsconnector/iamaccesskeylastuseds\": { \"SingularDisplayName\": \"IAM Access Key Last Used\" }\r\n ,\"microsoft.awsconnector/iamaccesskeymetadata\": { \"SingularDisplayName\": \"IAM Access Key Metadata\" }\r\n ,\"microsoft.awsconnector/iamgroups\": { \"SingularDisplayName\": \"IAM Group\" }\r\n ,\"microsoft.awsconnector/iaminstanceprofiles\": { \"SingularDisplayName\": \"IAM Instance Profile\" }\r\n ,\"microsoft.awsconnector/iammanagedpolicies\": { \"SingularDisplayName\": \"IAM Managed Policy\" }\r\n ,\"microsoft.awsconnector/iammfadevices\": { \"SingularDisplayName\": \"IAM MFADevice\" }\r\n ,\"microsoft.awsconnector/iampasswordpolicies\": { \"SingularDisplayName\": \"IAM Password Policy\" }\r\n ,\"microsoft.awsconnector/iampolicyversions\": { \"SingularDisplayName\": \"IAM Policy Version\" }\r\n ,\"microsoft.awsconnector/iamroles\": { \"SingularDisplayName\": \"IAM Role\" }\r\n ,\"microsoft.awsconnector/iamservercertificates\": { \"SingularDisplayName\": \"IAM Server Certificate\" }\r\n ,\"microsoft.awsconnector/iamuserpolicies\": { \"SingularDisplayName\": \"IAM User Policy\" }\r\n ,\"microsoft.awsconnector/iamvirtualmfadevices\": { \"SingularDisplayName\": \"IAM Virtual MFADevice\" }\r\n ,\"microsoft.awsconnector/kmsaliases\": { \"SingularDisplayName\": \"KMS Alias\" }\r\n ,\"microsoft.awsconnector/kmskeys\": { \"SingularDisplayName\": \"KMS Key\" }\r\n ,\"microsoft.awsconnector/lambdafunctioncodelocations\": { \"SingularDisplayName\": \"Lambda Function Code Location\" }\r\n ,\"microsoft.awsconnector/lambdafunctionconfigurations\": { \"SingularDisplayName\": \"Microsoft.AwsConnector lambda function configuration\" }\r\n ,\"microsoft.awsconnector/lambdafunctions\": { \"SingularDisplayName\": \"Lambda Function\" }\r\n ,\"microsoft.awsconnector/licensemanagerlicenses\": { \"SingularDisplayName\": \"License Manager License\" }\r\n ,\"microsoft.awsconnector/lightsailbuckets\": { \"SingularDisplayName\": \"Lightsail Bucket\" }\r\n ,\"microsoft.awsconnector/lightsailinstances\": { \"SingularDisplayName\": \"Lightsail Instance\" }\r\n ,\"microsoft.awsconnector/logsloggroups\": { \"SingularDisplayName\": \"Logs Log Group\" }\r\n ,\"microsoft.awsconnector/logslogstreams\": { \"SingularDisplayName\": \"Logs Log Stream\" }\r\n ,\"microsoft.awsconnector/logsmetricfilters\": { \"SingularDisplayName\": \"Logs Metric Filter\" }\r\n ,\"microsoft.awsconnector/logssubscriptionfilters\": { \"SingularDisplayName\": \"Logs Subscription Filter\" }\r\n ,\"microsoft.awsconnector/macie2jobsummaries\": { \"SingularDisplayName\": \"Macie2 Job Summary\" }\r\n ,\"microsoft.awsconnector/macieallowlists\": { \"SingularDisplayName\": \"Macie Allow List\" }\r\n ,\"microsoft.awsconnector/networkfirewallfirewallpolicies\": { \"SingularDisplayName\": \"Network Firewall Firewall Policy\" }\r\n ,\"microsoft.awsconnector/networkfirewallfirewalls\": { \"SingularDisplayName\": \"Network Firewall Firewall\" }\r\n ,\"microsoft.awsconnector/networkfirewallrulegroups\": { \"SingularDisplayName\": \"Network Firewall Rule Group\" }\r\n ,\"microsoft.awsconnector/opensearchdomainstatuses\": { \"SingularDisplayName\": \"Open Search Domain Status\" }\r\n ,\"microsoft.awsconnector/opensearchservicedomains\": { \"SingularDisplayName\": \"Open Search Service Domain\" }\r\n ,\"microsoft.awsconnector/organizationsaccounts\": { \"SingularDisplayName\": \"Organizations Account\" }\r\n ,\"microsoft.awsconnector/organizationsorganizations\": { \"SingularDisplayName\": \"Organizations Organization\" }\r\n ,\"microsoft.awsconnector/rdsdbclusters\": { \"SingularDisplayName\": \"RDS DBCluster\" }\r\n ,\"microsoft.awsconnector/rdsdbinstances\": { \"SingularDisplayName\": \"RDS DBInstance\" }\r\n ,\"microsoft.awsconnector/rdsdbsnapshotattributesresults\": { \"SingularDisplayName\": \"RDS DBSnapshot Attributes Result\" }\r\n ,\"microsoft.awsconnector/rdsdbsnapshots\": { \"SingularDisplayName\": \"RDS DBSnapshot\" }\r\n ,\"microsoft.awsconnector/rdseventsubscriptions\": { \"SingularDisplayName\": \"RDS Event Subscription\" }\r\n ,\"microsoft.awsconnector/rdsexporttasks\": { \"SingularDisplayName\": \"RDS Export Task\" }\r\n ,\"microsoft.awsconnector/redshiftclusterparametergroups\": { \"SingularDisplayName\": \"Redshift Cluster Parameter Group\" }\r\n ,\"microsoft.awsconnector/redshiftclusters\": { \"SingularDisplayName\": \"Redshift Cluster\" }\r\n ,\"microsoft.awsconnector/route53domainsdomainsummaries\": { \"SingularDisplayName\": \"Route 53 Domains Domain Summary\" }\r\n ,\"microsoft.awsconnector/route53hostedzones\": { \"SingularDisplayName\": \"Route53 Hosted Zone\" }\r\n ,\"microsoft.awsconnector/route53resourcerecordsets\": { \"SingularDisplayName\": \"Route 53 Resource Record Set\" }\r\n ,\"microsoft.awsconnector/s3accesscontrolpolicies\": { \"SingularDisplayName\": \"S3 Access Control Policy\" }\r\n ,\"microsoft.awsconnector/s3accesspoints\": { \"SingularDisplayName\": \"S3 Access Point\" }\r\n ,\"microsoft.awsconnector/s3bucketpolicies\": { \"SingularDisplayName\": \"S3 Bucket Policy\" }\r\n ,\"microsoft.awsconnector/s3buckets\": { \"SingularDisplayName\": \"S3 Bucket\" }\r\n ,\"microsoft.awsconnector/s3controlmultiregionaccesspointpolicydocuments\": { \"SingularDisplayName\": \"S3 Control Multi Region Access Point Policy Document\" }\r\n ,\"microsoft.awsconnector/sagemakerapps\": { \"SingularDisplayName\": \"Sage Maker App\" }\r\n ,\"microsoft.awsconnector/sagemakerdevices\": { \"SingularDisplayName\": \"Sage Maker Device\" }\r\n ,\"microsoft.awsconnector/sagemakerimages\": { \"SingularDisplayName\": \"Sage Maker Image\" }\r\n ,\"microsoft.awsconnector/sagemakernotebookinstancesummaries\": { \"SingularDisplayName\": \"Sage Maker Notebook Instance Summary\" }\r\n ,\"microsoft.awsconnector/secretsmanagerresourcepolicies\": { \"SingularDisplayName\": \"Secrets Manager Resource Policy\" }\r\n ,\"microsoft.awsconnector/secretsmanagersecrets\": { \"SingularDisplayName\": \"Secrets Manager Secret\" }\r\n ,\"microsoft.awsconnector/snssubscriptions\": { \"SingularDisplayName\": \"SNS Subscription\" }\r\n ,\"microsoft.awsconnector/snstopics\": { \"SingularDisplayName\": \"SNS Topic\" }\r\n ,\"microsoft.awsconnector/sqsqueues\": { \"SingularDisplayName\": \"SQS Queue\" }\r\n ,\"microsoft.awsconnector/ssminstanceinformations\": { \"SingularDisplayName\": \"SSM Instance Information\" }\r\n ,\"microsoft.awsconnector/ssmparameters\": { \"SingularDisplayName\": \"SSM Parameter\" }\r\n ,\"microsoft.awsconnector/ssmresourcecompliancesummaryitems\": { \"SingularDisplayName\": \"SSM Resource Compliance Summary Item\" }\r\n ,\"microsoft.awsconnector/wafv2ipsets\": { \"SingularDisplayName\": \"WAFv2 IPSet\" }\r\n ,\"microsoft.awsconnector/wafv2loggingconfigurations\": { \"SingularDisplayName\": \"WAFv2 Logging Configuration\" }\r\n ,\"microsoft.awsconnector/wafv2webaclassociations\": { \"SingularDisplayName\": \"WAFv2 Web ACLAssociation\" }\r\n ,\"microsoft.awsconnector/wafwebaclsummaries\": { \"SingularDisplayName\": \"WAF Web ACLSummary\" }\r\n ,\"microsoft.azureactivedirectory/b2cdirectories\": { \"SingularDisplayName\": \"B2C tenant\" }\r\n ,\"microsoft.azureactivedirectory/ciamdirectories\": { \"SingularDisplayName\": \"External Configuration Tenant\" }\r\n ,\"microsoft.azureactivedirectory/guestusages\": { \"SingularDisplayName\": \"Guest Usage\" }\r\n ,\"microsoft.azurearcdata/datacontrollers\": { \"SingularDisplayName\": \"Azure Arc data controller\" }\r\n ,\"microsoft.azurearcdata/mysqlserver\": { \"SingularDisplayName\": \"MySql Server - Azure Arc\" }\r\n ,\"microsoft.azurearcdata/postgresinstances\": { \"SingularDisplayName\": \"PostgreSQL server ? Azure Arc\" }\r\n ,\"microsoft.azurearcdata/postgressqlserver\": { \"SingularDisplayName\": \"PostgresSql Server - Azure Arc\" }\r\n ,\"microsoft.azurearcdata/sqlmanagedinstances\": { \"SingularDisplayName\": \"SQL managed instance - Azure Arc\" }\r\n ,\"microsoft.azurearcdata/sqlserveresulicenses\": { \"SingularDisplayName\": \"SQL Server ESU license\" }\r\n ,\"microsoft.azurearcdata/sqlserverinstances\": { \"SingularDisplayName\": \"SQL Server - Azure Arc\" }\r\n ,\"microsoft.azurearcdata/sqlserverinstances/databases\": { \"SingularDisplayName\": \"SQL Server database - Azure Arc\" }\r\n ,\"microsoft.azurearcdata/sqlserverlicenses\": { \"SingularDisplayName\": \"SQL Server License\" }\r\n ,\"microsoft.azurebusinesscontinuity/deletedunifiedprotecteditems\": { \"SingularDisplayName\": \"Microsoft.AzureBusinessContinuity deleted unified protected item\" }\r\n ,\"microsoft.azurebusinesscontinuity/unifiedprotecteditems\": { \"SingularDisplayName\": \"Microsoft.AzureBusinessContinuity unified protected item\" }\r\n ,\"microsoft.azurecis/aadapplications\": { \"SingularDisplayName\": \"Microsoft.AzureCis AAD application\" }\r\n ,\"microsoft.azurecis/addressrecords\": { \"SingularDisplayName\": \"Microsoft.AzureCis address record\" }\r\n ,\"microsoft.azurecis/autopilotenvironments\": { \"SingularDisplayName\": \"Microsoft.AzureCis autopilot environment\" }\r\n ,\"microsoft.azurecis/autopilotmachinefunctions\": { \"SingularDisplayName\": \"Microsoft.AzureCis autopilot machine function\" }\r\n ,\"microsoft.azurecis/autopilotsoftwareloadbalancevirtualips\": { \"SingularDisplayName\": \"Microsoft.AzureCis auto pilot software load balance virtual IP\" }\r\n ,\"microsoft.azurecis/azcopies\": { \"SingularDisplayName\": \"Microsoft.AzureCis az copy\" }\r\n ,\"microsoft.azurecis/canonicalnamerecords\": { \"SingularDisplayName\": \"Microsoft.AzureCis canonical name record\" }\r\n ,\"microsoft.azurecis/dsmsallowlists\": { \"SingularDisplayName\": \"Microsoft.AzureCis ds msallowlist\" }\r\n ,\"microsoft.azurecis/dsmscertificates\": { \"SingularDisplayName\": \"Microsoft.AzureCis dsms certificate\" }\r\n ,\"microsoft.azurecis/dsmsrootfolders\": { \"SingularDisplayName\": \"Microsoft.AzureCis dsms root folder\" }\r\n ,\"microsoft.azurecis/dstsapplications\": { \"SingularDisplayName\": \"Microsoft.AzureCis dsts application\" }\r\n ,\"microsoft.azurecis/dstsserviceaccounts\": { \"SingularDisplayName\": \"Microsoft.AzureCis dsts service account\" }\r\n ,\"microsoft.azurecis/dstsserviceclientidentities\": { \"SingularDisplayName\": \"Microsoft.AzureCis dsts service client identity\" }\r\n ,\"microsoft.azurecis/genericgenevaactions\": { \"SingularDisplayName\": \"Microsoft.AzureCis generic geneva action\" }\r\n ,\"microsoft.azurecis/plannedquotas\": { \"SingularDisplayName\": \"Microsoft.AzureCis planned quota\" }\r\n ,\"microsoft.azurecis/pointerrecords\": { \"SingularDisplayName\": \"Microsoft.AzureCis pointer record\" }\r\n ,\"microsoft.azurecis/publishconfigvalues\": { \"SingularDisplayName\": \"Microsoft.AzureCis publish config value\" }\r\n ,\"microsoft.azurecis/pushagentv2accounts\": { \"SingularDisplayName\": \"Microsoft.AzureCis push agent v2 account\" }\r\n ,\"microsoft.azurecis/servicerecords\": { \"SingularDisplayName\": \"Microsoft.AzureCis service record\" }\r\n ,\"microsoft.azurecis/sharedconfigvalues\": { \"SingularDisplayName\": \"Microsoft.AzureCis shared config value\" }\r\n ,\"microsoft.azurecloudmetadata/clouds\": { \"SingularDisplayName\": \"Microsoft.AzureCloudMetadata cloud\" }\r\n ,\"microsoft.azurecloudmetadata/clouds/geographies\": { \"SingularDisplayName\": \"Microsoft.AzureCloudMetadata clouds geography\" }\r\n ,\"microsoft.azurecloudmetadata/clouds/geographies/regions\": { \"SingularDisplayName\": \"Microsoft.AzureCloudMetadata clouds geographies region\" }\r\n ,\"microsoft.azuredatatransfer/connections\": { \"SingularDisplayName\": \"Connection\" }\r\n ,\"microsoft.azuredatatransfer/connections/flows\": { \"SingularDisplayName\": \"Flow\" }\r\n ,\"microsoft.azuredatatransfer/pipelines\": { \"SingularDisplayName\": \"Pipeline\" }\r\n ,\"microsoft.azurefleet/fleets\": { \"SingularDisplayName\": \"Compute Fleet\" }\r\n ,\"microsoft.azurefleet/fleetscomputehub\": { \"SingularDisplayName\": \"Compute Fleet\" }\r\n ,\"microsoft.azureimagetestingforlinux/jobs\": { \"SingularDisplayName\": \"Microsoft.AzureImageTestingForLinux job\" }\r\n ,\"microsoft.azureimagetestingforlinux/jobtemplates\": { \"SingularDisplayName\": \"Microsoft.AzureImageTestingForLinux job template\" }\r\n ,\"microsoft.azurelargeinstance/azurelargeinstances\": { \"SingularDisplayName\": \"Azure Large Instance\" }\r\n ,\"microsoft.azurelargeinstance/azurelargestorageinstances\": { \"SingularDisplayName\": \"Microsoft.AzureLargeInstance Azure large storage instance\" }\r\n ,\"microsoft.azurepercept/accounts\": { \"SingularDisplayName\": \"Microsoft.AzurePercept account\" }\r\n ,\"microsoft.azurepercept/accounts/devices\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts device\" }\r\n ,\"microsoft.azurepercept/accounts/devices/sensors\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts devices sensor\" }\r\n ,\"microsoft.azurepercept/accounts/sensors\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts sensor\" }\r\n ,\"microsoft.azurepercept/accounts/solutioninstances\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts solutioninstance\" }\r\n ,\"microsoft.azurepercept/accounts/solutions\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts solution\" }\r\n ,\"microsoft.azurepercept/accounts/targets\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts target\" }\r\n ,\"microsoft.azureplaywrightservice/accounts\": { \"SingularDisplayName\": \"Playwright Testing\" }\r\n ,\"microsoft.azurescan/scanningaccounts\": { \"SingularDisplayName\": \"ESRP Scan\" }\r\n ,\"microsoft.azuresphere/catalogs\": { \"SingularDisplayName\": \"Azure Sphere Catalog\" }\r\n ,\"microsoft.azurespherev2/catalogs\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalog\" }\r\n ,\"microsoft.azurespherev2/catalogs/artifacts\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs artifact\" }\r\n ,\"microsoft.azurespherev2/catalogs/certificates\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs certificate\" }\r\n ,\"microsoft.azurespherev2/catalogs/deviceregistrations\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs device registration\" }\r\n ,\"microsoft.azurespherev2/catalogs/provisioningpackages\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs provisioning package\" }\r\n ,\"microsoft.azurespherev2/catalogs/syndicationchannels\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs syndication channel\" }\r\n ,\"microsoft.azurespherev2/catalogs/syndicationchannels/deployments\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs syndication channels deployment\" }\r\n ,\"microsoft.azurespherev2/catalogs/updatepackages\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs update package\" }\r\n ,\"microsoft.azurestack/cloudmanifestfiles\": { \"SingularDisplayName\": \"Microsoft.AzureStack cloud manifest file\" }\r\n ,\"microsoft.azurestack/linkedsubscriptions\": { \"SingularDisplayName\": \"Microsoft.AzureStack linked subscription\" }\r\n ,\"microsoft.azurestack/registrations\": { \"SingularDisplayName\": \"Microsoft.AzureStack registration\" }\r\n ,\"microsoft.azurestack/registrations/customersubscriptions\": { \"SingularDisplayName\": \"Microsoft.AzureStack registrations customer subscription\" }\r\n ,\"microsoft.azurestack/registrations/products\": { \"SingularDisplayName\": \"Microsoft.AzureStack registrations product\" }\r\n ,\"microsoft.azurestackhci/clusters\": { \"SingularDisplayName\": \"Azure Local\" }\r\n ,\"microsoft.azurestackhci/clusters/updates/updateruns\": { \"SingularDisplayName\": \"Azure Local\" }\r\n ,\"microsoft.azurestackhci/clusters/updatesummaries\": { \"SingularDisplayName\": \"Azure Local\" }\r\n ,\"microsoft.azurestackhci/devicepools\": { \"SingularDisplayName\": \"Azure Stack\" }\r\n ,\"microsoft.azurestackhci/edgedevices\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI edge device\" }\r\n ,\"microsoft.azurestackhci/edgedevices/jobs\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI edge devices job\" }\r\n ,\"microsoft.azurestackhci/edgemachines\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI edge machine\" }\r\n ,\"microsoft.azurestackhci/edgemachines/jobs\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI edge machines job\" }\r\n ,\"microsoft.azurestackhci/edgenodepools\": { \"SingularDisplayName\": \"Azure Stack\" }\r\n ,\"microsoft.azurestackhci/galleryimages\": { \"SingularDisplayName\": \"Azure Local Gallery image\" }\r\n ,\"microsoft.azurestackhci/logicalnetworks\": { \"SingularDisplayName\": \"Azure Local Logical network\" }\r\n ,\"microsoft.azurestackhci/marketplacegalleryimages\": { \"SingularDisplayName\": \"Azure Local Marketplace Gallery image\" }\r\n ,\"microsoft.azurestackhci/networkinterfaces\": { \"SingularDisplayName\": \"Azure Local VM Network Interface\" }\r\n ,\"microsoft.azurestackhci/networksecuritygroups\": { \"SingularDisplayName\": \"Azure Local Network Security Group\" }\r\n ,\"microsoft.azurestackhci/networksecuritygroups/securityrules\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI network security groups security rule\" }\r\n ,\"microsoft.azurestackhci/storagecontainers\": { \"SingularDisplayName\": \"Azure Local Storage path\" }\r\n ,\"microsoft.azurestackhci/virtualharddisks\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI virtual hard disk\" }\r\n ,\"microsoft.azurestackhci/virtualmachineinstances\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI virtual machine instance\" }\r\n ,\"microsoft.azurestackhci/virtualmachineinstances/guestagents\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI virtual machine instances guest agent\" }\r\n ,\"microsoft.azurestackhci/virtualmachineinstances/hybrididentitymetadata\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI virtual machine instances hybrid identity metadata\" }\r\n ,\"microsoft.azurestackhci/virtualmachines\": { \"SingularDisplayName\": \"Azure Local virtual machine - Azure Arc\" }\r\n ,\"microsoft.azurestackhci/virtualnetworks\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI virtual network\" }\r\n ,\"microsoft.backupsolutions/vmwareapplications\": { \"SingularDisplayName\": \"Microsoft.BackupSolutions vmware application\" }\r\n ,\"microsoft.bakeryhybrid/pies\": { \"SingularDisplayName\": \"Microsoft.BakeryHybrid py\" }\r\n ,\"microsoft.bakeryhybrid/pies/nestedresourcetype\": { \"SingularDisplayName\": \"Microsoft.BakeryHybrid pies nested resource type\" }\r\n ,\"microsoft.baremetal/baremetalconnections\": { \"SingularDisplayName\": \"Microsoft.BareMetal bare metal connection\" }\r\n ,\"microsoft.baremetal/crayservers\": { \"SingularDisplayName\": \"Cray Server\" }\r\n ,\"microsoft.baremetal/monitoringservers\": { \"SingularDisplayName\": \"Monitoring Server\" }\r\n ,\"microsoft.baremetal/peeringsettings\": { \"SingularDisplayName\": \"Microsoft.BareMetal peering setting\" }\r\n ,\"microsoft.baremetalinfrastructure/baremetalinstances\": { \"SingularDisplayName\": \"BareMetal Instance\" }\r\n ,\"microsoft.baremetalinfrastructure/baremetalstorageinstances\": { \"SingularDisplayName\": \"Microsoft.BareMetalInfrastructure bare metal storage instance\" }\r\n ,\"microsoft.batch/batchaccounts\": { \"SingularDisplayName\": \"Batch account\" }\r\n ,\"microsoft.billing/billingaccounts\": { \"SingularDisplayName\": \"Microsoft.Billing billing account\" }\r\n ,\"microsoft.billing/billingaccounts/agreements\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts agreement\" }\r\n ,\"microsoft.billing/billingaccounts/associatedtenants\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts associated tenant\" }\r\n ,\"microsoft.billing/billingaccounts/availablebalance\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts available balance\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profile\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/availablebalance\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles available balance\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/billingsubscriptions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles billing subscription\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/customers\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles customer\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/customers/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles customers billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/customers/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles customers billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/customers/transfers\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles customers transfer\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/instructions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles instruction\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoices\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice section\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice sections billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice sections billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections/billingsubscriptions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice sections billing subscription\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections/products\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice sections product\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections/transfers\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice sections transfer\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/paymentmethodlinks\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles payment method link\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/policies\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles policy\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/transactions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles transaction\" }\r\n ,\"microsoft.billing/billingaccounts/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/billingsubscriptionaliases\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing subscription aliase\" }\r\n ,\"microsoft.billing/billingaccounts/billingsubscriptions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing subscription\" }\r\n ,\"microsoft.billing/billingaccounts/billingsubscriptions/invoices\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing subscriptions invoice\" }\r\n ,\"microsoft.billing/billingaccounts/customers\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts customer\" }\r\n ,\"microsoft.billing/billingaccounts/customers/billingsubscriptions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts customers billing subscription\" }\r\n ,\"microsoft.billing/billingaccounts/customers/policies\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts customers policy\" }\r\n ,\"microsoft.billing/billingaccounts/customers/products\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts customers product\" }\r\n ,\"microsoft.billing/billingaccounts/departments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts department\" }\r\n ,\"microsoft.billing/billingaccounts/departments/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts departments billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/departments/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts departments billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/departments/enrollmentaccounts\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts departments enrollment account\" }\r\n ,\"microsoft.billing/billingaccounts/enrollmentaccounts\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts enrollment account\" }\r\n ,\"microsoft.billing/billingaccounts/enrollmentaccounts/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts enrollment accounts billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/enrollmentaccounts/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts enrollment accounts billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/incentiveschedules\": { \"SingularDisplayName\": \"Incentive Schedule\" }\r\n ,\"microsoft.billing/billingaccounts/incentiveschedules/milestones\": { \"SingularDisplayName\": \"Milestone\" }\r\n ,\"microsoft.billing/billingaccounts/invoices\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts invoice\" }\r\n ,\"microsoft.billing/billingaccounts/invoicesections\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts invoice section\" }\r\n ,\"microsoft.billing/billingaccounts/invoicesections/billingsubscriptions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts invoice sections billing subscription\" }\r\n ,\"microsoft.billing/billingaccounts/invoicesections/products\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts invoice sections product\" }\r\n ,\"microsoft.billing/billingaccounts/invoicesections/transfers\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts invoice sections transfer\" }\r\n ,\"microsoft.billing/billingaccounts/lineofcredit\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts line of credit\" }\r\n ,\"microsoft.billing/billingaccounts/migrations\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts migration\" }\r\n ,\"microsoft.billing/billingaccounts/paymentmethods\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts payment method\" }\r\n ,\"microsoft.billing/billingaccounts/policies\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts policy\" }\r\n ,\"microsoft.billing/billingaccounts/products\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts product\" }\r\n ,\"microsoft.billing/billingaccounts/reservationorders\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts reservation order\" }\r\n ,\"microsoft.billing/billingaccounts/reservationorders/reservations\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts reservation orders reservation\" }\r\n ,\"microsoft.billing/billingaccounts/savingsplanorders\": { \"SingularDisplayName\": \"Savings plan order\" }\r\n ,\"microsoft.billing/billingaccounts/savingsplanorders/savingsplans\": { \"SingularDisplayName\": \"Savings plan\" }\r\n ,\"microsoft.billing/billingperiods\": { \"SingularDisplayName\": \"Microsoft.Billing billing period\" }\r\n ,\"microsoft.billing/billingproperty\": { \"SingularDisplayName\": \"Microsoft.Billing billing property\" }\r\n ,\"microsoft.billing/billingrequests\": { \"SingularDisplayName\": \"Microsoft.Billing billing request\" }\r\n ,\"microsoft.billing/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing role assignment\" }\r\n ,\"microsoft.billing/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing role definition\" }\r\n ,\"microsoft.billing/enrollmentaccounts\": { \"SingularDisplayName\": \"Microsoft.Billing enrollment account\" }\r\n ,\"microsoft.billing/paymentmethods\": { \"SingularDisplayName\": \"Microsoft.Billing payment method\" }\r\n ,\"microsoft.billing/policies\": { \"SingularDisplayName\": \"Microsoft.Billing policy\" }\r\n ,\"microsoft.billing/promotions\": { \"SingularDisplayName\": \"Microsoft.Billing promotion\" }\r\n ,\"microsoft.billing/transfers\": { \"SingularDisplayName\": \"Microsoft.Billing transfer\" }\r\n ,\"microsoft.billingbenefits/credits\": { \"SingularDisplayName\": \"Credit\" }\r\n ,\"microsoft.billingbenefits/discounts\": { \"SingularDisplayName\": \"Discount\" }\r\n ,\"microsoft.billingbenefits/incentiveschedules\": { \"SingularDisplayName\": \"Incentive Schedule\" }\r\n ,\"microsoft.billingbenefits/incentiveschedules/milestones\": { \"SingularDisplayName\": \"Milestone\" }\r\n ,\"microsoft.billingbenefits/maccs\": { \"SingularDisplayName\": \"Microsoft Azure Consumption Commitment\" }\r\n ,\"microsoft.billingbenefits/reservationorderaliases\": { \"SingularDisplayName\": \"Microsoft.BillingBenefits reservation order aliase\" }\r\n ,\"microsoft.billingbenefits/savingsplanorderaliases\": { \"SingularDisplayName\": \"Microsoft.BillingBenefits savings plan order aliase\" }\r\n ,\"microsoft.billingbenefits/savingsplanorders\": { \"SingularDisplayName\": \"Savings plan order\" }\r\n ,\"microsoft.billingbenefits/savingsplanorders/savingsplans\": { \"SingularDisplayName\": \"Savings plan\" }\r\n ,\"microsoft.bing/accounts\": { \"SingularDisplayName\": \"Bing Resource\" }\r\n ,\"microsoft.blockchain/blockchainmembers\": { \"SingularDisplayName\": \"Microsoft.Blockchain blockchain member\" }\r\n ,\"microsoft.blockchain/blockchainmembers/transactionnodes\": { \"SingularDisplayName\": \"Microsoft.Blockchain blockchain members transaction node\" }\r\n ,\"microsoft.blockchaintokens/tokenservices\": { \"SingularDisplayName\": \"Microsoft.BlockchainTokens token service\" }\r\n ,\"microsoft.blockchaintokens/tokenservices/blockchainnetworks\": { \"SingularDisplayName\": \"Microsoft.BlockchainTokens token services blockchain network\" }\r\n ,\"microsoft.blockchaintokens/tokenservices/groups\": { \"SingularDisplayName\": \"Microsoft.BlockchainTokens token services group\" }\r\n ,\"microsoft.blockchaintokens/tokenservices/groups/accounts\": { \"SingularDisplayName\": \"Microsoft.BlockchainTokens token services groups account\" }\r\n ,\"microsoft.blockchaintokens/tokenservices/tokentemplates\": { \"SingularDisplayName\": \"Microsoft.BlockchainTokens token services token template\" }\r\n ,\"microsoft.bluefin/instances\": { \"SingularDisplayName\": \"Microsoft.Bluefin instance\" }\r\n ,\"microsoft.bluefin/instances/datasets\": { \"SingularDisplayName\": \"Microsoft.Bluefin instances dataset\" }\r\n ,\"microsoft.bluefin/instances/pipelines\": { \"SingularDisplayName\": \"Microsoft.Bluefin instances pipeline\" }\r\n ,\"microsoft.blueprint/blueprintassignments\": { \"SingularDisplayName\": \"Microsoft.Blueprint blueprint assignment\" }\r\n ,\"microsoft.blueprint/blueprints\": { \"SingularDisplayName\": \"Microsoft.Blueprint blueprint\" }\r\n ,\"microsoft.blueprint/blueprints/artifacts\": { \"SingularDisplayName\": \"Microsoft.Blueprint blueprints artifact\" }\r\n ,\"microsoft.blueprint/blueprints/versions\": { \"SingularDisplayName\": \"Microsoft.Blueprint blueprints version\" }\r\n ,\"microsoft.botservice/botservices\": { \"SingularDisplayName\": \"Bot Service\" }\r\n ,\"microsoft.cache/redis\": { \"SingularDisplayName\": \"Redis cache\" }\r\n ,\"microsoft.cache/redisenterprise\": { \"SingularDisplayName\": \"Azure Managed Redis\" }\r\n ,\"microsoft.cache/redisenterprise/databases\": { \"SingularDisplayName\": \"Redis Enterprise database\" }\r\n ,\"microsoft.capacity/reservationorders\": { \"SingularDisplayName\": \"Reservation order\" }\r\n ,\"microsoft.capacity/reservationorders/reservations\": { \"SingularDisplayName\": \"Reservation\" }\r\n ,\"microsoft.cascade/sites\": { \"SingularDisplayName\": \"Microsoft.Cascade site\" }\r\n ,\"microsoft.cdn/cdnwebapplicationfirewallpolicies\": { \"SingularDisplayName\": \"Content Delivery Network WAF policy\" }\r\n ,\"microsoft.cdn/edgeactions\": { \"SingularDisplayName\": \"Edge Action\" }\r\n ,\"microsoft.cdn/profiles\": { \"SingularDisplayName\": \"Front Door and CDN profile\" }\r\n ,\"microsoft.cdn/profiles/afdendpoints\": { \"SingularDisplayName\": \"Endpoint\" }\r\n ,\"microsoft.cdn/profiles/afdendpoints/routes\": { \"SingularDisplayName\": \"Route\" }\r\n ,\"microsoft.cdn/profiles/customdomains\": { \"SingularDisplayName\": \"Custom domain\" }\r\n ,\"microsoft.cdn/profiles/endpoints\": { \"SingularDisplayName\": \"CDN endpoint\" }\r\n ,\"microsoft.cdn/profiles/endpoints/customdomains\": { \"SingularDisplayName\": \"CDN custom domain\" }\r\n ,\"microsoft.cdn/profiles/endpoints/origins\": { \"SingularDisplayName\": \"CDN origin\" }\r\n ,\"microsoft.cdn/profiles/origingroups\": { \"SingularDisplayName\": \"Origin group\" }\r\n ,\"microsoft.cdn/profiles/origingroups/origins\": { \"SingularDisplayName\": \"Origin\" }\r\n ,\"microsoft.cdn/profiles/rulesets\": { \"SingularDisplayName\": \"Rule set\" }\r\n ,\"microsoft.cdn/profiles/rulesets/rules\": { \"SingularDisplayName\": \"Rule\" }\r\n ,\"microsoft.cdn/profiles/secrets\": { \"SingularDisplayName\": \"Secret\" }\r\n ,\"microsoft.cdn/profiles/securitypolicies\": { \"SingularDisplayName\": \"Security policy\" }\r\n ,\"microsoft.certificateregistration/certificateorders\": { \"SingularDisplayName\": \"App Service certificate\" }\r\n ,\"microsoft.certify/testsuites\": { \"SingularDisplayName\": \"Microsoft.Certify test suite\" }\r\n ,\"microsoft.certify/validationjobs\": { \"SingularDisplayName\": \"Microsoft.Certify validation job\" }\r\n ,\"microsoft.changeanalysis/profile\": { \"SingularDisplayName\": \"Microsoft.ChangeAnalysis profile\" }\r\n ,\"microsoft.changesafety/changestates\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety change state\" }\r\n ,\"microsoft.changesafety/changestates/stageprogressions\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety change states stage progression\" }\r\n ,\"microsoft.changesafety/stagemaps\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety stage map\" }\r\n ,\"microsoft.changesafety/validations\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety validation\" }\r\n ,\"microsoft.changesafety/validators\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety validator\" }\r\n ,\"microsoft.changesafety/validators/versions\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety validators version\" }\r\n ,\"microsoft.chaos/experiments\": { \"SingularDisplayName\": \"Chaos Experiment\" }\r\n ,\"microsoft.chaos/privateaccesses\": { \"SingularDisplayName\": \"Agent Private Access\" }\r\n ,\"microsoft.chaos/targets\": { \"SingularDisplayName\": \"Microsoft.Chaos target\" }\r\n ,\"microsoft.chaos/targets/capabilities\": { \"SingularDisplayName\": \"Microsoft.Chaos targets capability\" }\r\n ,\"microsoft.classiccompute/domainnames\": { \"SingularDisplayName\": \"Cloud service (classic)\" }\r\n ,\"microsoft.classiccompute/domainnames/slots/roles\": { \"SingularDisplayName\": \"Cloud service role (classic)\" }\r\n ,\"microsoft.classiccompute/virtualmachines\": { \"SingularDisplayName\": \"Virtual machine (classic)\" }\r\n ,\"microsoft.classicnetwork/networksecuritygroups\": { \"SingularDisplayName\": \"Network security group (classic)\" }\r\n ,\"microsoft.classicnetwork/reservedips\": { \"SingularDisplayName\": \"Reserved IP address (classic)\" }\r\n ,\"microsoft.classicnetwork/virtualnetworks\": { \"SingularDisplayName\": \"Virtual network (classic)\" }\r\n })[tolower(id)]\r\n}\r\n", + "$fxv#1": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData/Internal')\r\n_resource_type_2(id: string) {\r\n dynamic({\r\n \"microsoft.classicstorage/storageaccounts\": { \"SingularDisplayName\": \"Storage account (classic)\" }\r\n ,\"microsoft.classicstorage/storageaccounts/disks\": { \"SingularDisplayName\": \"Disk (classic)\" }\r\n ,\"microsoft.classicstorage/storageaccounts/osimages\": { \"SingularDisplayName\": \"OS image (classic)\" }\r\n ,\"microsoft.classicstorage/storageaccounts/vmimages\": { \"SingularDisplayName\": \"VM image (classic)\" }\r\n ,\"microsoft.cleanroom/cleanrooms\": { \"SingularDisplayName\": \"Microsoft.CleanRoom cleanroom\" }\r\n ,\"microsoft.cleanroom/collaborations\": { \"SingularDisplayName\": \"Microsoft.CleanRoom collaboration\" }\r\n ,\"microsoft.cleanroom/collaborations/contracts\": { \"SingularDisplayName\": \"Microsoft.CleanRoom collaborations contract\" }\r\n ,\"microsoft.cleanroom/consortiums\": { \"SingularDisplayName\": \"Microsoft.CleanRoom consortium\" }\r\n ,\"microsoft.cleanroom/microservices\": { \"SingularDisplayName\": \"Microsoft.CleanRoom microservice\" }\r\n ,\"microsoft.cloud/hubs\": { \"SingularDisplayName\": \"FinOps hub\" }\r\n ,\"microsoft.clouddeviceplatform/delegatedidentities\": { \"SingularDisplayName\": \"Microsoft.CloudDevicePlatform delegated identity\" }\r\n ,\"microsoft.cloudhealth/healthmodels\": { \"SingularDisplayName\": \"Health Model\" }\r\n ,\"microsoft.cloudtest/accounts\": { \"SingularDisplayName\": \"CloudTest Account\" }\r\n ,\"microsoft.cloudtest/buildcaches\": { \"SingularDisplayName\": \"1ES Build Cache\" }\r\n ,\"microsoft.cloudtest/hostedpools\": { \"SingularDisplayName\": \"1ES Hosted Pool\" }\r\n ,\"microsoft.cloudtest/images\": { \"SingularDisplayName\": \"1ES Image\" }\r\n ,\"microsoft.cloudtest/pools\": { \"SingularDisplayName\": \"CloudTest Pool\" }\r\n ,\"microsoft.clusterstor/nodes\": { \"SingularDisplayName\": \"ClusterStor\" }\r\n ,\"microsoft.codesigning/codesigningaccounts\": { \"SingularDisplayName\": \"Trusted Signing Account\" }\r\n ,\"microsoft.codespaces/plans\": { \"SingularDisplayName\": \"Microsoft.Codespaces plan\" }\r\n ,\"microsoft.cognitiveservices/accounts\": { \"SingularDisplayName\": \"Azure AI Foundry\" }\r\n ,\"microsoft.cognitiveservices/accounts/projects\": { \"SingularDisplayName\": \"Azure AI Foundry project\" }\r\n ,\"microsoft.cognitiveservices/commitmentplans\": { \"SingularDisplayName\": \"Microsoft.CognitiveServices commitment plan\" }\r\n ,\"microsoft.cognitiveservices/commitmentplans/accountassociations\": { \"SingularDisplayName\": \"Microsoft.CognitiveServices commitment plans account association\" }\r\n ,\"microsoft.communication/communicationservices\": { \"SingularDisplayName\": \"Communication Service\" }\r\n ,\"microsoft.communication/emailservices\": { \"SingularDisplayName\": \"Email Communication Service\" }\r\n ,\"microsoft.communication/emailservices/domains\": { \"SingularDisplayName\": \"Email Communication Services Domain\" }\r\n ,\"microsoft.community/communitytrainings\": { \"SingularDisplayName\": \"Community Training\" }\r\n ,\"microsoft.compositesolutions/compositesolutiondefinitions\": { \"SingularDisplayName\": \"Microsoft.CompositeSolutions composite solution definition\" }\r\n ,\"microsoft.compositesolutions/compositesolutions\": { \"SingularDisplayName\": \"Microsoft.CompositeSolutions composite solution\" }\r\n ,\"microsoft.compute/availabilitysets\": { \"SingularDisplayName\": \"Availability set\" }\r\n ,\"microsoft.compute/capacityreservationgroups\": { \"SingularDisplayName\": \"Capacity Reservation Group\" }\r\n ,\"microsoft.compute/capacityreservationgroups/capacityreservations\": { \"SingularDisplayName\": \"Capacity reservation\" }\r\n ,\"microsoft.compute/capacityreservationgroupscomputehub\": { \"SingularDisplayName\": \"Capacity Reservation Group\" }\r\n ,\"microsoft.compute/cloudservices\": { \"SingularDisplayName\": \"Cloud service (extended support)\" }\r\n ,\"microsoft.compute/computefleetinstances\": { \"SingularDisplayName\": \"Instance\" }\r\n ,\"microsoft.compute/computefleetscalesets\": { \"SingularDisplayName\": \"Virtual machine scale set\" }\r\n ,\"microsoft.compute/diskaccesses\": { \"SingularDisplayName\": \"Disk Access\" }\r\n ,\"microsoft.compute/diskencryptionsets\": { \"SingularDisplayName\": \"Disk Encryption Set\" }\r\n ,\"microsoft.compute/disks\": { \"SingularDisplayName\": \"Disk\" }\r\n ,\"microsoft.compute/galleries\": { \"SingularDisplayName\": \"Azure compute gallery\" }\r\n ,\"microsoft.compute/galleries/applications\": { \"SingularDisplayName\": \"VM application definition\" }\r\n ,\"microsoft.compute/galleries/applications/versions\": { \"SingularDisplayName\": \"VM application version\" }\r\n ,\"microsoft.compute/galleries/images\": { \"SingularDisplayName\": \"VM image definition\" }\r\n ,\"microsoft.compute/galleries/images/versions\": { \"SingularDisplayName\": \"VM image version\" }\r\n ,\"microsoft.compute/galleries/imagescomputehub\": { \"SingularDisplayName\": \"VM image definition\" }\r\n ,\"microsoft.compute/hostgroups\": { \"SingularDisplayName\": \"Host group\" }\r\n ,\"microsoft.compute/hostgroups/hosts\": { \"SingularDisplayName\": \"Host\" }\r\n ,\"microsoft.compute/hostgroupscomputehub\": { \"SingularDisplayName\": \"Host group\" }\r\n ,\"microsoft.compute/images\": { \"SingularDisplayName\": \"Image\" }\r\n ,\"microsoft.compute/imagescomputehub\": { \"SingularDisplayName\": \"Image\" }\r\n ,\"microsoft.compute/locations/communitygalleries/images\": { \"SingularDisplayName\": \"Community image\" }\r\n ,\"microsoft.compute/locations/communitygalleries/imagescomputehub\": { \"SingularDisplayName\": \"Community image\" }\r\n ,\"microsoft.compute/proximityplacementgroups\": { \"SingularDisplayName\": \"Proximity placement group\" }\r\n ,\"microsoft.compute/proximityplacementgroupscomputehub\": { \"SingularDisplayName\": \"Proximity placement group\" }\r\n ,\"microsoft.compute/restorepointcollections\": { \"SingularDisplayName\": \"Restore Point Collection\" }\r\n ,\"microsoft.compute/restorepointcollections/restorepoints\": { \"SingularDisplayName\": \"Restore Point\" }\r\n ,\"microsoft.compute/snapshots\": { \"SingularDisplayName\": \"Snapshot\" }\r\n ,\"microsoft.compute/sshpublickeys\": { \"SingularDisplayName\": \"SSH key\" }\r\n ,\"microsoft.compute/standbypoolinstance\": { \"SingularDisplayName\": \"Standby pool\" }\r\n ,\"microsoft.compute/virtualmachinecomputehub\": { \"SingularDisplayName\": \"Virtual machine\" }\r\n ,\"microsoft.compute/virtualmachineflexinstances\": { \"SingularDisplayName\": \"Instance\" }\r\n ,\"microsoft.compute/virtualmachines\": { \"SingularDisplayName\": \"Virtual machine\" }\r\n ,\"microsoft.compute/virtualmachines/providers/guestconfigurationassignments\": { \"SingularDisplayName\": \"Guest Assignment\" }\r\n ,\"microsoft.compute/virtualmachinescalesets\": { \"SingularDisplayName\": \"Virtual machine scale set\" }\r\n ,\"microsoft.compute/virtualmachinescalesets/providers/guestconfigurationassignments\": { \"SingularDisplayName\": \"Guest Assignment\" }\r\n ,\"microsoft.compute/virtualmachinescalesets/virtualmachines\": { \"SingularDisplayName\": \"Virtual machine scale set instance\" }\r\n ,\"microsoft.compute/virtualmachinescalesets/virtualmachines/networkinterfaces/ipconfigurations/publicipaddresses\": { \"SingularDisplayName\": \"Public IP address\" }\r\n ,\"microsoft.compute/virtualmachinescalesetscomputehub\": { \"SingularDisplayName\": \"Virtual machine scale set\" }\r\n ,\"microsoft.computehub/advisorcost\": { \"SingularDisplayName\": \"Recommendations\" }\r\n ,\"microsoft.computehub/advisoroperationalexcellence\": { \"SingularDisplayName\": \"Recommendations\" }\r\n ,\"microsoft.computehub/advisorperformance\": { \"SingularDisplayName\": \"Recommendations\" }\r\n ,\"microsoft.computehub/advisorreliability\": { \"SingularDisplayName\": \"Recommendations\" }\r\n ,\"microsoft.computehub/advisorsecurity\": { \"SingularDisplayName\": \"Recommendations\" }\r\n ,\"microsoft.computehub/all\": { \"SingularDisplayName\": \"All resources\" }\r\n ,\"microsoft.computehub/backup\": { \"SingularDisplayName\": \"Backup job\" }\r\n ,\"microsoft.computehub/computehubmain\": { \"SingularDisplayName\": \"Compute infrastructure\" }\r\n ,\"microsoft.computehub/healthevents\": { \"SingularDisplayName\": \"Health events\" }\r\n ,\"microsoft.computehub/linuxostype\": { \"SingularDisplayName\": \"Linux OS\" }\r\n ,\"microsoft.computehub/microsoftdefenderfreetrialsubscription\": { \"SingularDisplayName\": \"Microsoft defender\" }\r\n ,\"microsoft.computehub/microsoftdefenderstandardsubscription\": { \"SingularDisplayName\": \"Microsoft defender\" }\r\n ,\"microsoft.computehub/outages\": { \"SingularDisplayName\": \"Outages\" }\r\n ,\"microsoft.computehub/powerstatedeallocated\": { \"SingularDisplayName\": \"Power states\" }\r\n ,\"microsoft.computehub/powerstaterunning\": { \"SingularDisplayName\": \"Power states\" }\r\n ,\"microsoft.computehub/powerstatestopped\": { \"SingularDisplayName\": \"Power states\" }\r\n ,\"microsoft.computehub/provisioningstatefailedresources\": { \"SingularDisplayName\": \"Provisioning states\" }\r\n ,\"microsoft.computehub/provisioningstatesucceededresources\": { \"SingularDisplayName\": \"Provisioning states\" }\r\n ,\"microsoft.computehub/windowsostype\": { \"SingularDisplayName\": \"Windows OS\" }\r\n ,\"microsoft.computeschedule/autoactions\": { \"SingularDisplayName\": \"Automatic Action\" }\r\n ,\"microsoft.computeschedule/autoactions/occurrences\": { \"SingularDisplayName\": \"Microsoft.ComputeSchedule auto actions occurrence\" }\r\n ,\"microsoft.confidentialledger/ledgers\": { \"SingularDisplayName\": \"Confidential Ledger\" }\r\n ,\"microsoft.confidentialledger/managedccfs\": { \"SingularDisplayName\": \"Managed CCF App\" }\r\n ,\"microsoft.confluent/agreements\": { \"SingularDisplayName\": \"Microsoft.Confluent agreement\" }\r\n ,\"microsoft.confluent/organizations\": { \"SingularDisplayName\": \"Confluent organization\" }\r\n ,\"microsoft.connectedcache/cachenodes\": { \"SingularDisplayName\": \"Connected Cache for ISP\" }\r\n ,\"microsoft.connectedcache/enterprisecustomers\": { \"SingularDisplayName\": \"Connected Cache for Enterprise & Education\" }\r\n ,\"microsoft.connectedcache/enterprisemcccustomers\": { \"SingularDisplayName\": \"Connected Cache for Enterprise & Education\" }\r\n ,\"microsoft.connectedcache/enterprisemcccustomers/enterprisemcccachenodes\": { \"SingularDisplayName\": \"MCC CacheNode for Enterprise\" }\r\n ,\"microsoft.connectedcache/ispcustomers\": { \"SingularDisplayName\": \"Connected Cache for ISP\" }\r\n ,\"microsoft.connectedcredentials/credentials\": { \"SingularDisplayName\": \"Microsoft.ConnectedCredentials credential\" }\r\n ,\"microsoft.connectedvehicle/platformaccounts\": { \"SingularDisplayName\": \"Microsoft.ConnectedVehicle platform account\" }\r\n ,\"microsoft.connectedvmwarevsphere/clusters\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere cluster\" }\r\n ,\"microsoft.connectedvmwarevsphere/datastores\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere datastore\" }\r\n ,\"microsoft.connectedvmwarevsphere/hosts\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere host\" }\r\n ,\"microsoft.connectedvmwarevsphere/resourcepools\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere resource pool\" }\r\n ,\"microsoft.connectedvmwarevsphere/vcenters\": { \"SingularDisplayName\": \"VMware vCenter\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachineinstances\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere virtual machine instance\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachineinstances/guestagents\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere virtual machine instances guest agent\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachineinstances/hybrididentitymetadata\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere virtual machine instances hybrid identity metadata\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachines\": { \"SingularDisplayName\": \"VMware + AVS virtual machine\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachines/providers/guestconfigurationassignments\": { \"SingularDisplayName\": \"Guest Assignment\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachinetemplates\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere virtual machine template\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualnetworks\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere virtual network\" }\r\n ,\"microsoft.consumption/budgets\": { \"SingularDisplayName\": \"Microsoft.Consumption budget\" }\r\n ,\"microsoft.consumption/credits\": { \"SingularDisplayName\": \"Microsoft.Consumption credit\" }\r\n ,\"microsoft.consumption/pricesheets\": { \"SingularDisplayName\": \"Microsoft.Consumption pricesheet\" }\r\n ,\"microsoft.containerinstance/containergroupprofiles\": { \"SingularDisplayName\": \"Microsoft.ContainerInstance container group profile\" }\r\n ,\"microsoft.containerinstance/containergroupprofiles/revisions\": { \"SingularDisplayName\": \"Microsoft.ContainerInstance container group profiles revision\" }\r\n ,\"microsoft.containerinstance/containergroups\": { \"SingularDisplayName\": \"Container instances\" }\r\n ,\"microsoft.containerinstance/ngroups\": { \"SingularDisplayName\": \"Microsoft.ContainerInstance ngroup\" }\r\n ,\"microsoft.containerregistry/registries\": { \"SingularDisplayName\": \"Container registry\" }\r\n ,\"microsoft.containerregistry/registries/replications\": { \"SingularDisplayName\": \"Container registry replication\" }\r\n ,\"microsoft.containerregistry/registries/scopemaps\": { \"SingularDisplayName\": \"Container registry scope map\" }\r\n ,\"microsoft.containerregistry/registries/tokens\": { \"SingularDisplayName\": \"Container registry token\" }\r\n ,\"microsoft.containerregistry/registries/webhooks\": { \"SingularDisplayName\": \"Container registry webhook\" }\r\n ,\"microsoft.containerservice/fleets\": { \"SingularDisplayName\": \"Kubernetes fleet manager\" }\r\n ,\"microsoft.containerservice/managedclusters\": { \"SingularDisplayName\": \"Kubernetes service\" }\r\n ,\"microsoft.containerservice/managedclusters/managednamespaces\": { \"SingularDisplayName\": \"Managed namespace\" }\r\n ,\"microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/extensions\": { \"SingularDisplayName\": \"Kubernetes service extension\" }\r\n ,\"microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/fluxconfigurations\": { \"SingularDisplayName\": \"GitOps configuration\" }\r\n ,\"microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/namespaces\": { \"SingularDisplayName\": \"Kubernetes namespace\" }\r\n ,\"microsoft.containerservice/managedclusters/namespaces\": { \"SingularDisplayName\": \"Managed namespace\" }\r\n ,\"microsoft.containerservice/managedclustersnapshots\": { \"SingularDisplayName\": \"Microsoft.ContainerService managedclustersnapshot\" }\r\n ,\"microsoft.containerservice/snapshots\": { \"SingularDisplayName\": \"Microsoft.ContainerService snapshot\" }\r\n ,\"microsoft.containerstorage/pools\": { \"SingularDisplayName\": \"Container storage\" }\r\n ,\"microsoft.costmanagement/alerts\": { \"SingularDisplayName\": \"Microsoft.CostManagement alert\" }\r\n ,\"microsoft.costmanagement/budgets\": { \"SingularDisplayName\": \"Microsoft.CostManagement budget\" }\r\n ,\"microsoft.costmanagement/cloudconnectors\": { \"SingularDisplayName\": \"Microsoft.CostManagement cloud connector\" }\r\n ,\"microsoft.costmanagement/connectors\": { \"SingularDisplayName\": \"Microsoft.CostManagement connector\" }\r\n ,\"microsoft.costmanagement/costallocationrules\": { \"SingularDisplayName\": \"Microsoft.CostManagement cost allocation rule\" }\r\n ,\"microsoft.costmanagement/costdetailsoperationresults\": { \"SingularDisplayName\": \"Microsoft.CostManagement cost details operation result\" }\r\n ,\"microsoft.costmanagement/exports\": { \"SingularDisplayName\": \"Microsoft.CostManagement export\" }\r\n ,\"microsoft.costmanagement/externalbillingaccounts\": { \"SingularDisplayName\": \"Microsoft.CostManagement external billing account\" }\r\n ,\"microsoft.costmanagement/externalsubscriptions\": { \"SingularDisplayName\": \"Microsoft.CostManagement external subscription\" }\r\n ,\"microsoft.costmanagement/markuprules\": { \"SingularDisplayName\": \"Microsoft.CostManagement markup rule\" }\r\n ,\"microsoft.costmanagement/operationstatus\": { \"SingularDisplayName\": \"Microsoft.CostManagement operation statu\" }\r\n ,\"microsoft.costmanagement/reportconfigs\": { \"SingularDisplayName\": \"Microsoft.CostManagement reportconfig\" }\r\n ,\"microsoft.costmanagement/reports\": { \"SingularDisplayName\": \"Microsoft.CostManagement report\" }\r\n ,\"microsoft.costmanagement/scheduledactions\": { \"SingularDisplayName\": \"Microsoft.CostManagement scheduled action\" }\r\n ,\"microsoft.costmanagement/settings\": { \"SingularDisplayName\": \"Microsoft.CostManagement setting\" }\r\n ,\"microsoft.costmanagement/views\": { \"SingularDisplayName\": \"Microsoft.CostManagement view\" }\r\n ,\"microsoft.customerlockbox/requests\": { \"SingularDisplayName\": \"Microsoft.CustomerLockbox request\" }\r\n ,\"microsoft.customerlockbox/tenantoptedin\": { \"SingularDisplayName\": \"Microsoft.CustomerLockbox tenant opted in\" }\r\n ,\"microsoft.customproviders/associations\": { \"SingularDisplayName\": \"Microsoft.CustomProviders association\" }\r\n ,\"microsoft.customproviders/resourceproviders\": { \"SingularDisplayName\": \"Microsoft.CustomProviders resource provider\" }\r\n ,\"microsoft.dashboard/dashboards\": { \"SingularDisplayName\": \"Azure Monitor dashboards with Grafana\" }\r\n ,\"microsoft.dashboard/grafana\": { \"SingularDisplayName\": \"Azure Managed Grafana\" }\r\n ,\"microsoft.dataaccelerator/indexclusters\": { \"SingularDisplayName\": \"Microsoft.DataAccelerator index cluster\" }\r\n ,\"microsoft.databasefleetmanager/fleets\": { \"SingularDisplayName\": \"Database fleet manager\" }\r\n ,\"microsoft.databasefleetmanager/fleets/fleetspaces\": { \"SingularDisplayName\": \"Fleetspaces\" }\r\n ,\"microsoft.databasefleetmanager/fleets/fleetspaces/databases\": { \"SingularDisplayName\": \"Fleet managed database\" }\r\n ,\"microsoft.databasefleetmanager/fleets/tiers\": { \"SingularDisplayName\": \"tier\" }\r\n ,\"microsoft.databasewatcher/watchers\": { \"SingularDisplayName\": \"Database watcher\" }\r\n ,\"microsoft.databox/jobs\": { \"SingularDisplayName\": \"Azure Data Box\" }\r\n ,\"microsoft.databoxedge/databoxedgedevices\": { \"SingularDisplayName\": \"Azure Stack Edge / Data Box Gateway\" }\r\n ,\"microsoft.databricks/accessconnectors\": { \"SingularDisplayName\": \"Access Connector for Azure Databricks\" }\r\n ,\"microsoft.databricks/workspaces\": { \"SingularDisplayName\": \"Azure Databricks Service\" }\r\n ,\"microsoft.datacatalog/catalogs\": { \"SingularDisplayName\": \"Data catalog\" }\r\n ,\"microsoft.datacollaboration/workspaces\": { \"SingularDisplayName\": \"Project CI\" }\r\n ,\"microsoft.datadog/agreements\": { \"SingularDisplayName\": \"Microsoft.Datadog agreement\" }\r\n ,\"microsoft.datadog/monitors\": { \"SingularDisplayName\": \"Datadog\" }\r\n ,\"microsoft.datadog/subscriptionstatuses\": { \"SingularDisplayName\": \"Microsoft.Datadog subscription statuse\" }\r\n ,\"microsoft.datafactory/datafactories\": { \"SingularDisplayName\": \"Data factory\" }\r\n ,\"microsoft.datafactory/factories\": { \"SingularDisplayName\": \"Data factory (V2)\" }\r\n ,\"microsoft.datafactory/factories/pipelines\": { \"SingularDisplayName\": \"Data Factory pipeline\" }\r\n ,\"microsoft.datafactory/factories/triggers\": { \"SingularDisplayName\": \"Data Factory trigger\" }\r\n ,\"microsoft.datalakeanalytics/accounts\": { \"SingularDisplayName\": \"Data Lake Analytics account\" }\r\n ,\"microsoft.datalakestore/accounts\": { \"SingularDisplayName\": \"Data Lake Storage Gen1\" }\r\n ,\"microsoft.datamigration/databasemigrations\": { \"SingularDisplayName\": \"Microsoft.DataMigration database migration\" }\r\n ,\"microsoft.datamigration/migrationservices\": { \"SingularDisplayName\": \"Microsoft.DataMigration migration service\" }\r\n ,\"microsoft.datamigration/services\": { \"SingularDisplayName\": \"Azure Database Migration Service (classic)\" }\r\n ,\"microsoft.datamigration/services/projects\": { \"SingularDisplayName\": \"Azure Database Migration Project\" }\r\n ,\"microsoft.datamigration/sqlmigrationservices\": { \"SingularDisplayName\": \"Azure Database Migration Service\" }\r\n ,\"microsoft.dataprotection/backupvaults\": { \"SingularDisplayName\": \"Backup vault\" }\r\n ,\"microsoft.dataprotection/resourceguards\": { \"SingularDisplayName\": \"Resource Guard\" }\r\n ,\"microsoft.datareplication/replicationfabrics\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication fabric\" }\r\n ,\"microsoft.datareplication/replicationfabrics/fabricagents\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication fabrics fabric agent\" }\r\n ,\"microsoft.datareplication/replicationfabrics/fabricagents/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication fabrics fabric agents operation\" }\r\n ,\"microsoft.datareplication/replicationfabrics/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication fabrics operation\" }\r\n ,\"microsoft.datareplication/replicationvaults\": { \"SingularDisplayName\": \"Data replication vault\" }\r\n ,\"microsoft.datareplication/replicationvaults/alertsettings\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults alert setting\" }\r\n ,\"microsoft.datareplication/replicationvaults/events\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults event\" }\r\n ,\"microsoft.datareplication/replicationvaults/jobs\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults job\" }\r\n ,\"microsoft.datareplication/replicationvaults/jobs/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults jobs operation\" }\r\n ,\"microsoft.datareplication/replicationvaults/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults operation\" }\r\n ,\"microsoft.datareplication/replicationvaults/privateendpointconnectionproxies\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults private endpoint connection proxy\" }\r\n ,\"microsoft.datareplication/replicationvaults/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults private endpoint connection\" }\r\n ,\"microsoft.datareplication/replicationvaults/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults private link resource\" }\r\n ,\"microsoft.datareplication/replicationvaults/protecteditems\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults protected item\" }\r\n ,\"microsoft.datareplication/replicationvaults/protecteditems/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults protected items operation\" }\r\n ,\"microsoft.datareplication/replicationvaults/protecteditems/recoverypoints\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults protected items recovery point\" }\r\n ,\"microsoft.datareplication/replicationvaults/replicationextensions\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults replication extension\" }\r\n ,\"microsoft.datareplication/replicationvaults/replicationextensions/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults replication extensions operation\" }\r\n ,\"microsoft.datareplication/replicationvaults/replicationpolicies\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults replication policy\" }\r\n ,\"microsoft.datareplication/replicationvaults/replicationpolicies/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults replication policies operation\" }\r\n ,\"microsoft.datashare/accounts\": { \"SingularDisplayName\": \"Data Share\" }\r\n ,\"microsoft.dbformariadb/servers\": { \"SingularDisplayName\": \"Azure Database for MariaDB server\" }\r\n ,\"microsoft.dbformysql/flexibleservers\": { \"SingularDisplayName\": \"Azure Database for MySQL flexible server\" }\r\n ,\"microsoft.dbformysql/servers\": { \"SingularDisplayName\": \"MySQL server\" }\r\n ,\"microsoft.dbforpostgresql/flexibleservers\": { \"SingularDisplayName\": \"Azure Database for PostgreSQL flexible server\" }\r\n ,\"microsoft.dbforpostgresql/servergroupsv2\": { \"SingularDisplayName\": \"Azure Cosmos DB for PostgreSQL Cluster\" }\r\n ,\"microsoft.dbforpostgresql/servers\": { \"SingularDisplayName\": \"PostgreSQL server\" }\r\n ,\"microsoft.delegatednetwork/controller\": { \"SingularDisplayName\": \"Microsoft.DelegatedNetwork controller\" }\r\n ,\"microsoft.delegatednetwork/delegatedsubnets\": { \"SingularDisplayName\": \"Microsoft.DelegatedNetwork delegated subnet\" }\r\n ,\"microsoft.delegatednetwork/orchestrators\": { \"SingularDisplayName\": \"Microsoft.DelegatedNetwork orchestrator\" }\r\n ,\"microsoft.dependencymap/maps\": { \"SingularDisplayName\": \"Microsoft.DependencyMap map\" }\r\n ,\"microsoft.dependencymap/maps/discoverysources\": { \"SingularDisplayName\": \"Microsoft.DependencyMap maps discovery source\" }\r\n ,\"microsoft.deploymentmanager/artifactsources\": { \"SingularDisplayName\": \"Microsoft.DeploymentManager artifact source\" }\r\n ,\"microsoft.deploymentmanager/rollouts\": { \"SingularDisplayName\": \"Rollout\" }\r\n ,\"microsoft.deploymentmanager/servicetopologies\": { \"SingularDisplayName\": \"Microsoft.DeploymentManager service topology\" }\r\n ,\"microsoft.deploymentmanager/servicetopologies/services\": { \"SingularDisplayName\": \"Microsoft.DeploymentManager service topologies service\" }\r\n ,\"microsoft.deploymentmanager/servicetopologies/services/serviceunits\": { \"SingularDisplayName\": \"Microsoft.DeploymentManager service topologies services service unit\" }\r\n ,\"microsoft.deploymentmanager/steps\": { \"SingularDisplayName\": \"Microsoft.DeploymentManager step\" }\r\n ,\"microsoft.desktopvirtualization/appattachpackages\": { \"SingularDisplayName\": \"App attach package\" }\r\n ,\"microsoft.desktopvirtualization/applicationgroups\": { \"SingularDisplayName\": \"Application group\" }\r\n ,\"microsoft.desktopvirtualization/hostpools\": { \"SingularDisplayName\": \"Host pool\" }\r\n ,\"microsoft.desktopvirtualization/scalingplans\": { \"SingularDisplayName\": \"Scaling plan\" }\r\n ,\"microsoft.desktopvirtualization/workspaces\": { \"SingularDisplayName\": \"Workspace\" }\r\n ,\"microsoft.devai/instances\": { \"SingularDisplayName\": \"Microsoft.DevAI instance\" }\r\n ,\"microsoft.devai/instances/experiments\": { \"SingularDisplayName\": \"Microsoft.DevAI instances experiment\" }\r\n ,\"microsoft.devai/instances/sandboxes\": { \"SingularDisplayName\": \"Microsoft.DevAI instances sandbox\" }\r\n ,\"microsoft.devai/instances/sandboxes/experiments\": { \"SingularDisplayName\": \"Microsoft.DevAI instances sandboxes experiment\" }\r\n ,\"microsoft.devcenter/devcenters\": { \"SingularDisplayName\": \"Dev center\" }\r\n ,\"microsoft.devcenter/devcenters/devboxdefinitions\": { \"SingularDisplayName\": \"Dev Box definition\" }\r\n ,\"microsoft.devcenter/networkconnections\": { \"SingularDisplayName\": \"Network connection\" }\r\n ,\"microsoft.devcenter/plans\": { \"SingularDisplayName\": \"Dev center plan\" }\r\n ,\"microsoft.devcenter/projects\": { \"SingularDisplayName\": \"Project\" }\r\n ,\"microsoft.devcenter/projects/pools\": { \"SingularDisplayName\": \"Pool\" }\r\n ,\"microsoft.developmentwindows365/developmentcloudpcdelegatedmsis\": { \"SingularDisplayName\": \"Microsoft.DevelopmentWindows365 development cloud pc delegated msi\" }\r\n ,\"microsoft.devhub/iacprofiles\": { \"SingularDisplayName\": \"Infrastructure as Code Automation\" }\r\n ,\"microsoft.devhub/templates\": { \"SingularDisplayName\": \"Microsoft.DevHub template\" }\r\n ,\"microsoft.devhub/templates/versions\": { \"SingularDisplayName\": \"Microsoft.DevHub templates version\" }\r\n ,\"microsoft.devhub/workflows\": { \"SingularDisplayName\": \"Microsoft.DevHub workflow\" }\r\n ,\"microsoft.deviceonboarding/discoveryservices\": { \"SingularDisplayName\": \"Microsoft.DeviceOnboarding discovery service\" }\r\n ,\"microsoft.deviceonboarding/discoveryservices/ownershipvoucherpublickeys\": { \"SingularDisplayName\": \"Microsoft.DeviceOnboarding discovery services ownership voucher public key\" }\r\n ,\"microsoft.deviceonboarding/onboardingservices\": { \"SingularDisplayName\": \"Microsoft.DeviceOnboarding onboarding service\" }\r\n ,\"microsoft.deviceonboarding/onboardingservices/policies\": { \"SingularDisplayName\": \"Microsoft.DeviceOnboarding onboarding services policy\" }\r\n ,\"microsoft.deviceregistry/assetendpointprofiles\": { \"SingularDisplayName\": \"IoT Asset Endpoint Profile\" }\r\n ,\"microsoft.deviceregistry/assets\": { \"SingularDisplayName\": \"IoT Asset\" }\r\n ,\"microsoft.deviceregistry/billingcontainers\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry billing container\" }\r\n ,\"microsoft.deviceregistry/devices\": { \"SingularDisplayName\": \"IoT Device\" }\r\n ,\"microsoft.deviceregistry/discoveredassetendpointprofiles\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry discovered asset endpoint profile\" }\r\n ,\"microsoft.deviceregistry/discoveredassets\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry discovered asset\" }\r\n ,\"microsoft.deviceregistry/namespaces\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespace\" }\r\n ,\"microsoft.deviceregistry/namespaces/assetendpointprofiles\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespaces asset endpoint profile\" }\r\n ,\"microsoft.deviceregistry/namespaces/assets\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespaces asset\" }\r\n ,\"microsoft.deviceregistry/namespaces/devices\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespaces device\" }\r\n ,\"microsoft.deviceregistry/namespaces/discoveredassetendpointprofiles\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespaces discovered asset endpoint profile\" }\r\n ,\"microsoft.deviceregistry/namespaces/discoveredassets\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespaces discovered asset\" }\r\n ,\"microsoft.deviceregistry/schemaregistries\": { \"SingularDisplayName\": \"IoT Schema Registry\" }\r\n ,\"microsoft.deviceregistry/schemaregistries/schemas\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry schema registries schema\" }\r\n ,\"microsoft.deviceregistry/schemaregistries/schemas/schemaversions\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry schema registries schemas schema version\" }\r\n ,\"microsoft.devices/iothubs\": { \"SingularDisplayName\": \"IoT hub\" }\r\n ,\"microsoft.devices/provisioningservices\": { \"SingularDisplayName\": \"Azure IoT Hub Device Provisioning Service (DPS)\" }\r\n ,\"microsoft.deviceupdate/accounts\": { \"SingularDisplayName\": \"Device Update for IoT Hub\" }\r\n ,\"microsoft.deviceupdate/updateaccounts\": { \"SingularDisplayName\": \"Device Update Account\" }\r\n ,\"microsoft.deviceupdate/updateaccounts/activedeployments\": { \"SingularDisplayName\": \"Device Update Active Deployment\" }\r\n ,\"microsoft.deviceupdate/updateaccounts/agents\": { \"SingularDisplayName\": \"Device Update Agent\" }\r\n ,\"microsoft.deviceupdate/updateaccounts/deployments\": { \"SingularDisplayName\": \"Device Update Deployment\" }\r\n ,\"microsoft.deviceupdate/updateaccounts/deviceclasses\": { \"SingularDisplayName\": \"Device Update Device Class\" }\r\n ,\"microsoft.deviceupdate/updateaccounts/updates\": { \"SingularDisplayName\": \"Device Update\" }\r\n ,\"microsoft.devops/pipelines\": { \"SingularDisplayName\": \"Microsoft.DevOps pipeline\" }\r\n ,\"microsoft.devopsinfrastructure/pools\": { \"SingularDisplayName\": \"Managed DevOps Pool\" }\r\n ,\"microsoft.devspaces/controllers\": { \"SingularDisplayName\": \"Microsoft.DevSpaces controller\" }\r\n ,\"microsoft.devtestlab/labs\": { \"SingularDisplayName\": \"DevTest lab\" }\r\n ,\"microsoft.devtestlab/labs/virtualmachines\": { \"SingularDisplayName\": \"DevTest Lab virtual machine\" }\r\n ,\"microsoft.devtestlab/schedules\": { \"SingularDisplayName\": \"Microsoft.DevTestLab schedule\" }\r\n ,\"microsoft.devtunnels/tunnelplans\": { \"SingularDisplayName\": \"Dev Tunnels Domain\" }\r\n ,\"microsoft.diagnostics/apollo\": { \"SingularDisplayName\": \"Microsoft.Diagnostics apollo\" }\r\n ,\"microsoft.digitaltwins/digitaltwinsinstances\": { \"SingularDisplayName\": \"Azure Digital Twins\" }\r\n ,\"microsoft.discovery/agents\": { \"SingularDisplayName\": \"Microsoft Discovery Agent\" }\r\n ,\"microsoft.discovery/bookshelves\": { \"SingularDisplayName\": \"Microsoft Discovery Bookshelf\" }\r\n ,\"microsoft.discovery/datacontainers\": { \"SingularDisplayName\": \"Microsoft Discovery Data Container\" }\r\n ,\"microsoft.discovery/datacontainers/dataassets\": { \"SingularDisplayName\": \"Data asset\" }\r\n ,\"microsoft.discovery/models\": { \"SingularDisplayName\": \"Microsoft Discovery Model\" }\r\n ,\"microsoft.discovery/storages\": { \"SingularDisplayName\": \"Microsoft Discovery Storage\" }\r\n ,\"microsoft.discovery/supercomputers\": { \"SingularDisplayName\": \"Microsoft Discovery Supercomputer\" }\r\n ,\"microsoft.discovery/supercomputers/nodepools\": { \"SingularDisplayName\": \"Nodepool\" }\r\n ,\"microsoft.discovery/tools\": { \"SingularDisplayName\": \"Microsoft Discovery Tool\" }\r\n ,\"microsoft.discovery/workflows\": { \"SingularDisplayName\": \"Microsoft Discovery Workflow\" }\r\n ,\"microsoft.discovery/workspaces\": { \"SingularDisplayName\": \"Microsoft Discovery Workspace\" }\r\n ,\"microsoft.discovery/workspaces/projects\": { \"SingularDisplayName\": \"Microsoft Discovery Project\" }\r\n ,\"microsoft.documentdb/cassandraclusters\": { \"SingularDisplayName\": \"Azure Managed Instance for Apache Cassandra\" }\r\n ,\"microsoft.documentdb/databaseaccounts\": { \"SingularDisplayName\": \"Cosmos DB account\" }\r\n ,\"microsoft.documentdb/fleets\": { \"SingularDisplayName\": \"Azure Cosmos DB Fleet\" }\r\n ,\"microsoft.documentdb/fleetspacepotentialdatabaseaccounts\": { \"SingularDisplayName\": \"Potential Azure Cosmos DB account\" }\r\n ,\"microsoft.documentdb/fleetspacepotentialdatabaseaccountswithlocations\": { \"SingularDisplayName\": \"Potential Azure Cosmos DB account\" }\r\n ,\"microsoft.documentdb/mongoclusters\": { \"SingularDisplayName\": \"Azure Cosmos DB for MongoDB (vCore)\" }\r\n ,\"microsoft.documentdb/throughputpools\": { \"SingularDisplayName\": \"Microsoft.DocumentDB throughput pool\" }\r\n ,\"microsoft.documentdb/throughputpools/throughputpoolaccounts\": { \"SingularDisplayName\": \"Microsoft.DocumentDB throughput pools throughput pool account\" }\r\n ,\"microsoft.domainregistration/domains\": { \"SingularDisplayName\": \"App Service Domain\" }\r\n ,\"microsoft.domainregistration/topleveldomains\": { \"SingularDisplayName\": \"Microsoft.DomainRegistration top level domain\" }\r\n ,\"microsoft.durabletask/namespaces\": { \"SingularDisplayName\": \"Microsoft.DurableTask namespace\" }\r\n ,\"microsoft.durabletask/namespaces/taskhubs\": { \"SingularDisplayName\": \"Task Hub\" }\r\n ,\"microsoft.durabletask/schedulers\": { \"SingularDisplayName\": \"Durable Task Scheduler\" }\r\n ,\"microsoft.durabletask/schedulers/taskhubs\": { \"SingularDisplayName\": \"Task Hub\" }\r\n ,\"microsoft.dynamics365fraudprotection/instances\": { \"SingularDisplayName\": \"Microsoft.Dynamics365FraudProtection instance\" }\r\n ,\"microsoft.easm/workspaces\": { \"SingularDisplayName\": \"Microsoft Defender EASM\" }\r\n ,\"microsoft.edge/configurations\": { \"SingularDisplayName\": \"Site configuration\" }\r\n ,\"microsoft.edge/configurations/arcgatewayconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations arc gateway configuration\" }\r\n ,\"microsoft.edge/configurations/connectivityconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations connectivity configuration\" }\r\n ,\"microsoft.edge/configurations/dynamicconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations dynamic configuration\" }\r\n ,\"microsoft.edge/configurations/dynamicconfigurations/versions\": { \"SingularDisplayName\": \"Microsoft.Edge configurations dynamic configurations version\" }\r\n ,\"microsoft.edge/configurations/networkconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations network configuration\" }\r\n ,\"microsoft.edge/configurations/securityconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations security configuration\" }\r\n ,\"microsoft.edge/configurations/timeserverconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations time server configuration\" }\r\n ,\"microsoft.edge/connectivitystatuses\": { \"SingularDisplayName\": \"Microsoft.Edge connectivity statuse\" }\r\n ,\"microsoft.edge/disconnectedoperations\": { \"SingularDisplayName\": \"Azure Local - disconnected operations\" }\r\n ,\"microsoft.edge/siteawareresourcetypes\": { \"SingularDisplayName\": \"Microsoft.Edge site aware resource type\" }\r\n ,\"microsoft.edge/sites\": { \"SingularDisplayName\": \"Site manager - Azure Arc\" }\r\n ,\"microsoft.edge/updates\": { \"SingularDisplayName\": \"Microsoft.Edge update\" }\r\n ,\"microsoft.edgemarketplace/offers\": { \"SingularDisplayName\": \"Microsoft.EdgeMarketplace offer\" }\r\n ,\"microsoft.edgemarketplace/publishers\": { \"SingularDisplayName\": \"Microsoft.EdgeMarketplace publisher\" }\r\n ,\"microsoft.edgeorder/addresses\": { \"SingularDisplayName\": \"Azure Edge Hardware Center Address\" }\r\n ,\"microsoft.edgeorder/bootstrapconfigurations\": { \"SingularDisplayName\": \"Site Key\" }\r\n ,\"microsoft.edgeorder/orderitems\": { \"SingularDisplayName\": \"Azure Edge Hardware Center\" }\r\n ,\"microsoft.edgeorder/virtual_orderitems\": { \"SingularDisplayName\": \"Device\" }\r\n ,\"microsoft.edgezones/extendedzones\": { \"SingularDisplayName\": \"Microsoft.EdgeZones extended zone\" }\r\n ,\"microsoft.education/grants\": { \"SingularDisplayName\": \"Microsoft.Education grant\" }\r\n ,\"microsoft.education/labs\": { \"SingularDisplayName\": \"Microsoft.Education lab\" }\r\n ,\"microsoft.education/labs/joinrequests\": { \"SingularDisplayName\": \"Microsoft.Education labs join request\" }\r\n ,\"microsoft.education/labs/students\": { \"SingularDisplayName\": \"Microsoft.Education labs student\" }\r\n ,\"microsoft.education/studentlabs\": { \"SingularDisplayName\": \"Microsoft.Education student lab\" }\r\n ,\"microsoft.elastic/monitors\": { \"SingularDisplayName\": \"Elastic Cloud Resource\" }\r\n ,\"microsoft.elasticsan/elasticsans\": { \"SingularDisplayName\": \"Elastic SAN\" }\r\n ,\"microsoft.energydataplatform/energyservices\": { \"SingularDisplayName\": \"Microsoft.EnergyDataPlatform energy service\" }\r\n ,\"microsoft.enterpriseknowledgegraph/services\": { \"SingularDisplayName\": \"Microsoft.EnterpriseKnowledgeGraph service\" }\r\n ,\"microsoft.enterprisesupport/enterprisesupports\": { \"SingularDisplayName\": \"Microsoft.EnterpriseSupport enterprise support\" }\r\n ,\"microsoft.eventgrid/domains\": { \"SingularDisplayName\": \"Event Grid Domain\" }\r\n ,\"microsoft.eventgrid/domains/topics\": { \"SingularDisplayName\": \"Event Grid Domain Topic\" }\r\n ,\"microsoft.eventgrid/eventsubscriptions\": { \"SingularDisplayName\": \"Microsoft.EventGrid event subscription\" }\r\n ,\"microsoft.eventgrid/extensiontopics\": { \"SingularDisplayName\": \"Event Grid extension topic\" }\r\n ,\"microsoft.eventgrid/namespaces\": { \"SingularDisplayName\": \"Event Grid Namespace\" }\r\n ,\"microsoft.eventgrid/namespaces/topics\": { \"SingularDisplayName\": \"Event Grid Namespace Topic\" }\r\n ,\"microsoft.eventgrid/namespaces/topics/eventsubscriptions\": { \"SingularDisplayName\": \"Event Subscription\" }\r\n ,\"microsoft.eventgrid/namespaces/topicspaces\": { \"SingularDisplayName\": \"Event Grid Topic Space\" }\r\n ,\"microsoft.eventgrid/partnerconfigurations\": { \"SingularDisplayName\": \"Event Grid Partner Configuration\" }\r\n ,\"microsoft.eventgrid/partnerdestinations\": { \"SingularDisplayName\": \"Event Grid Partner Destination\" }\r\n ,\"microsoft.eventgrid/partnernamespaces\": { \"SingularDisplayName\": \"Event Grid Partner Namespace\" }\r\n ,\"microsoft.eventgrid/partnernamespaces/channels\": { \"SingularDisplayName\": \"Event Grid Channel\" }\r\n ,\"microsoft.eventgrid/partnerregistrations\": { \"SingularDisplayName\": \"Event Grid Partner Registration\" }\r\n ,\"microsoft.eventgrid/partnertopics\": { \"SingularDisplayName\": \"Event Grid Partner Topic\" }\r\n ,\"microsoft.eventgrid/systemtopics\": { \"SingularDisplayName\": \"Event Grid System Topic\" }\r\n ,\"microsoft.eventgrid/systemtopics/eventsubscriptions\": { \"SingularDisplayName\": \"Event Grid Subscriptions\" }\r\n ,\"microsoft.eventgrid/topics\": { \"SingularDisplayName\": \"Event Grid Topic\" }\r\n ,\"microsoft.eventgrid/topictypes\": { \"SingularDisplayName\": \"Microsoft.EventGrid topic type\" }\r\n ,\"microsoft.eventgrid/verifiedpartners\": { \"SingularDisplayName\": \"Microsoft.EventGrid verified partner\" }\r\n ,\"microsoft.eventhub/clusters\": { \"SingularDisplayName\": \"Event Hubs Cluster\" }\r\n ,\"microsoft.eventhub/namespaces\": { \"SingularDisplayName\": \"Event Hubs namespace\" }\r\n ,\"microsoft.eventhub/namespaces/disasterrecoveryconfigs\": { \"SingularDisplayName\": \"Event Hubs Geo-DR Alias\" }\r\n ,\"microsoft.eventhub/namespaces/eventhubs\": { \"SingularDisplayName\": \"Event Hubs Instance\" }\r\n ,\"microsoft.eventhub/namespaces/providers/diagnosticsettings\": { \"SingularDisplayName\": \"Diagnostic settings\" }\r\n ,\"microsoft.eventhub/namespaces/schemagroups\": { \"SingularDisplayName\": \"Schema Group\" }\r\n ,\"microsoft.experimentation/experimentworkspaces\": { \"SingularDisplayName\": \"Experiment Workspace\" }\r\n ,\"microsoft.extendedlocation/customlocations\": { \"SingularDisplayName\": \"Custom location\" }\r\n ,\"microsoft.fabric/capacities\": { \"SingularDisplayName\": \"Fabric Capacity\" }\r\n ,\"microsoft.fabric/privatelinkservicesforfabric\": { \"SingularDisplayName\": \"Microsoft.Fabric private link services for fabric\" }\r\n ,\"microsoft.fabric/privatelinkservicesforfabric/operationresults\": { \"SingularDisplayName\": \"Microsoft.Fabric private link services for fabric operation result\" }\r\n ,\"microsoft.fabric/privatelinkservicesforfabric/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.Fabric private link services for fabric private endpoint connection\" }\r\n ,\"microsoft.fabric/privatelinkservicesforfabric/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.Fabric private link services for fabric private link resource\" }\r\n ,\"microsoft.fairfieldgardens/deviceprovisioningstates\": { \"SingularDisplayName\": \"Microsoft.FairfieldGardens device provisioning state\" }\r\n ,\"microsoft.fairfieldgardens/provisioningresources\": { \"SingularDisplayName\": \"Fairfield Gardens\" }\r\n ,\"microsoft.fairfieldgardens/provisioningresources/provisioningpolicies\": { \"SingularDisplayName\": \"Provisioning policy\" }\r\n ,\"microsoft.falcon/namespaces\": { \"SingularDisplayName\": \"Microsoft.Falcon namespace\" }\r\n ,\"microsoft.features/featureprovidernamespaces/featureconfigurations\": { \"SingularDisplayName\": \"Preview features\" }\r\n ,\"microsoft.fidalgo/devcenters\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenter\" }\r\n ,\"microsoft.fidalgo/devcenters/attachednetworks\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters attachednetwork\" }\r\n ,\"microsoft.fidalgo/devcenters/catalogs\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters catalog\" }\r\n ,\"microsoft.fidalgo/devcenters/catalogs/items\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters catalogs item\" }\r\n ,\"microsoft.fidalgo/devcenters/devboxdefinitions\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters devboxdefinition\" }\r\n ,\"microsoft.fidalgo/devcenters/environmenttypes\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters environment type\" }\r\n ,\"microsoft.fidalgo/devcenters/galleries\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters gallery\" }\r\n ,\"microsoft.fidalgo/devcenters/galleries/images\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters galleries image\" }\r\n ,\"microsoft.fidalgo/devcenters/galleries/images/versions\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters galleries images version\" }\r\n ,\"microsoft.fidalgo/devcenters/mappings\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters mapping\" }\r\n ,\"microsoft.fidalgo/machinedefinitions\": { \"SingularDisplayName\": \"Microsoft.Fidalgo machinedefinition\" }\r\n ,\"microsoft.fidalgo/networksettings\": { \"SingularDisplayName\": \"Microsoft.Fidalgo networksetting\" }\r\n ,\"microsoft.fidalgo/networksettings/healthchecks\": { \"SingularDisplayName\": \"Microsoft.Fidalgo networksettings healthcheck\" }\r\n ,\"microsoft.fidalgo/projects\": { \"SingularDisplayName\": \"Microsoft.Fidalgo project\" }\r\n ,\"microsoft.fidalgo/projects/attachednetworks\": { \"SingularDisplayName\": \"Microsoft.Fidalgo projects attachednetwork\" }\r\n ,\"microsoft.fidalgo/projects/devboxdefinitions\": { \"SingularDisplayName\": \"Microsoft.Fidalgo projects devboxdefinition\" }\r\n ,\"microsoft.fidalgo/projects/environments\": { \"SingularDisplayName\": \"Microsoft.Fidalgo projects environment\" }\r\n ,\"microsoft.fidalgo/projects/pools\": { \"SingularDisplayName\": \"Microsoft.Fidalgo projects pool\" }\r\n ,\"microsoft.fileshares/fileshares\": { \"SingularDisplayName\": \"File share\" }\r\n ,\"microsoft.fluidrelay/fluidrelayservers\": { \"SingularDisplayName\": \"Fluid Relay\" }\r\n ,\"microsoft.footprintmonitoring/profiles\": { \"SingularDisplayName\": \"Microsoft.FootprintMonitoring profile\" }\r\n ,\"microsoft.footprintmonitoring/profiles/experiments\": { \"SingularDisplayName\": \"Microsoft.FootprintMonitoring profiles experiment\" }\r\n ,\"microsoft.footprintmonitoring/profiles/measurementendpoints\": { \"SingularDisplayName\": \"Microsoft.FootprintMonitoring profiles measurement endpoint\" }\r\n ,\"microsoft.footprintmonitoring/profiles/measurementendpoints/conditions\": { \"SingularDisplayName\": \"Microsoft.FootprintMonitoring profiles measurement endpoints condition\" }\r\n ,\"microsoft.gallery/myareas/galleryitems\": { \"SingularDisplayName\": \"Template\" }\r\n ,\"microsoft.genomics/accounts\": { \"SingularDisplayName\": \"Genomics account\" }\r\n ,\"microsoft.graph/azureadapplication\": { \"SingularDisplayName\": \"Entra application\" }\r\n ,\"microsoft.graph/azureadapplicationprototype\": { \"SingularDisplayName\": \"Microsoft.Graph Azure ad application prototype\" }\r\n ,\"microsoft.graphservices/accounts\": { \"SingularDisplayName\": \"Metered API account\" }\r\n ,\"microsoft.guestconfiguration/guestconfigurationassignments\": { \"SingularDisplayName\": \"Microsoft.GuestConfiguration guest configuration assignment\" }\r\n ,\"microsoft.guestconfiguration/guestconfigurationassignments/reports\": { \"SingularDisplayName\": \"Microsoft.GuestConfiguration guest configuration assignments report\" }\r\n ,\"microsoft.hanaonazure/hanainstances\": { \"SingularDisplayName\": \"SAP HANA on Azure\" }\r\n ,\"microsoft.hanaonazure/sapmonitors\": { \"SingularDisplayName\": \"Azure Monitor for SAP Solutions (classic)\" }\r\n ,\"microsoft.hardware/orders\": { \"SingularDisplayName\": \"Microsoft.Hardware order\" }\r\n ,\"microsoft.hardwaresecuritymodules/cloudhsmclusters\": { \"SingularDisplayName\": \"Azure Cloud HSM\" }\r\n ,\"microsoft.hdinsight/clusterpools\": { \"SingularDisplayName\": \"Azure HDInsight on AKS cluster pool\" }\r\n ,\"microsoft.hdinsight/clusterpools/clusters\": { \"SingularDisplayName\": \"Azure HDInsight on AKS cluster\" }\r\n ,\"microsoft.hdinsight/clusterpools/clusters/instanceviews\": { \"SingularDisplayName\": \"Microsoft.HDInsight clusterpools clusters instance view\" }\r\n ,\"microsoft.hdinsight/clusters\": { \"SingularDisplayName\": \"HDInsight cluster\" }\r\n ,\"microsoft.healthbot/healthbots\": { \"SingularDisplayName\": \"Healthcare agent service\" }\r\n ,\"microsoft.healthcareapis/services\": { \"SingularDisplayName\": \"Azure API for FHIR\" }\r\n ,\"microsoft.healthcareapis/workspaces\": { \"SingularDisplayName\": \"Health Data Services workspace\" }\r\n ,\"microsoft.healthcareapis/workspaces/dicomservices\": { \"SingularDisplayName\": \"DICOM service\" }\r\n ,\"microsoft.healthcareapis/workspaces/fhirservices\": { \"SingularDisplayName\": \"FHIR service\" }\r\n ,\"microsoft.healthcareapis/workspaces/iotconnectors\": { \"SingularDisplayName\": \"MedTech service\" }\r\n ,\"microsoft.healthdataaiservices/deidservices\": { \"SingularDisplayName\": \"De-identification Service\" }\r\n ,\"microsoft.healthmodel/healthmodels\": { \"SingularDisplayName\": \"Health Model\" }\r\n ,\"microsoft.healthplatform/accounts\": { \"SingularDisplayName\": \"Microsoft.HealthPlatform account\" }\r\n ,\"microsoft.help/diagnostics\": { \"SingularDisplayName\": \"Microsoft.Help diagnostic\" }\r\n ,\"microsoft.help/selfhelp\": { \"SingularDisplayName\": \"Microsoft.Help self help\" }\r\n ,\"microsoft.help/simplifiedsolutions\": { \"SingularDisplayName\": \"Microsoft.Help simplified solution\" }\r\n ,\"microsoft.help/solutions\": { \"SingularDisplayName\": \"Microsoft.Help solution\" }\r\n ,\"microsoft.help/troubleshooters\": { \"SingularDisplayName\": \"Microsoft.Help troubleshooter\" }\r\n ,\"microsoft.hpcworkbench/instances\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instance\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chamber\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers/accessprofiles\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chambers access profile\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers/filerequests\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chambers file request\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers/files\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chambers file\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers/storages\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chambers storage\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers/workloads\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chambers workload\" }\r\n ,\"microsoft.hpcworkbench/instances/consortiums\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances consortium\" }\r\n ,\"microsoft.hybridcloud/cloudconnections\": { \"SingularDisplayName\": \"Microsoft.HybridCloud cloud connection\" }\r\n ,\"microsoft.hybridcloud/cloudconnectors\": { \"SingularDisplayName\": \"Microsoft.HybridCloud cloud connector\" }\r\n ,\"microsoft.hybridcompute/arcgatewayassociatedresources\": { \"SingularDisplayName\": \"Arc gateway associated resource\" }\r\n ,\"microsoft.hybridcompute/arcserverwithwac\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/gateways\": { \"SingularDisplayName\": \"Arc gateway\" }\r\n ,\"microsoft.hybridcompute/licenses\": { \"SingularDisplayName\": \"Extended Security Updates - Windows Server 2012/R2\" }\r\n ,\"microsoft.hybridcompute/machines\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/machines/microsoft.awsconnector/ec2instances\": { \"SingularDisplayName\": \"Microsoft.AwsConnector ec2 instance\" }\r\n ,\"microsoft.hybridcompute/machines/microsoft.connectedvmwarevsphere/virtualmachineinstances\": { \"SingularDisplayName\": \"VMware + AVS virtual machine\" }\r\n ,\"microsoft.hybridcompute/machines/providers/guestconfigurationassignments\": { \"SingularDisplayName\": \"Guest Assignment\" }\r\n ,\"microsoft.hybridcompute/machinesesu\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/machinespaygo\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/machinessoftwareassurance\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/machinessovereign\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/privatelinkscopes\": { \"SingularDisplayName\": \"Azure Arc Private Link Scope\" }\r\n ,\"microsoft.hybridcompute/settings\": { \"SingularDisplayName\": \"Microsoft.HybridCompute setting\" }\r\n ,\"microsoft.hybridconnectivity/endpoints\": { \"SingularDisplayName\": \"Microsoft.HybridConnectivity endpoint\" }\r\n ,\"microsoft.hybridconnectivity/endpoints/serviceconfigurations\": { \"SingularDisplayName\": \"Microsoft.HybridConnectivity endpoints service configuration\" }\r\n ,\"microsoft.hybridconnectivity/publiccloudconnectors\": { \"SingularDisplayName\": \"Multicloud connector\" }\r\n ,\"microsoft.hybridconnectivity/solutionconfigurations\": { \"SingularDisplayName\": \"Microsoft.HybridConnectivity solution configuration\" }\r\n ,\"microsoft.hybridconnectivity/solutionconfigurations/inventory\": { \"SingularDisplayName\": \"Microsoft.HybridConnectivity solution configurations inventory\" }\r\n ,\"microsoft.hybridconnectivity/solutiontypes\": { \"SingularDisplayName\": \"Microsoft.HybridConnectivity solution type\" }\r\n ,\"microsoft.hybridcontainerservice/kubernetesversions\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService kubernetes version\" }\r\n ,\"microsoft.hybridcontainerservice/provisionedclusterinstances\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService provisioned cluster instance\" }\r\n ,\"microsoft.hybridcontainerservice/provisionedclusterinstances/agentpools\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService provisioned cluster instances agent pool\" }\r\n ,\"microsoft.hybridcontainerservice/provisionedclusterinstances/hybrididentitymetadata\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService provisioned cluster instances hybrid identity metadata\" }\r\n ,\"microsoft.hybridcontainerservice/provisionedclusterinstances/upgradeprofiles\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService provisioned cluster instances upgrade profile\" }\r\n ,\"microsoft.hybridcontainerservice/provisionedclusters\": { \"SingularDisplayName\": \"Kubernetes hybrid - Azure Arc\" }\r\n ,\"microsoft.hybridcontainerservice/skus\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService SKU\" }\r\n ,\"microsoft.hybridcontainerservice/storagespaces\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService storage space\" }\r\n ,\"microsoft.hybridcontainerservice/virtualnetworks\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService virtual network\" }\r\n ,\"microsoft.hybriddata/datamanagers\": { \"SingularDisplayName\": \"Microsoft.HybridData data manager\" }\r\n ,\"microsoft.hybriddata/datamanagers/dataservices\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers data service\" }\r\n ,\"microsoft.hybriddata/datamanagers/dataservices/jobdefinitions\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers data services job definition\" }\r\n ,\"microsoft.hybriddata/datamanagers/dataservices/jobdefinitions/jobs\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers data services job definitions job\" }\r\n ,\"microsoft.hybriddata/datamanagers/datastores\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers data store\" }\r\n ,\"microsoft.hybriddata/datamanagers/datastoretypes\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers data store type\" }\r\n ,\"microsoft.hybriddata/datamanagers/publickeys\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers public key\" }\r\n ,\"microsoft.hybridnetwork/configurationgroupvalues\": { \"SingularDisplayName\": \"Configuration Group Value\" }\r\n ,\"microsoft.hybridnetwork/devices\": { \"SingularDisplayName\": \"Azure Network Function Manager ? Device\" }\r\n ,\"microsoft.hybridnetwork/networkfunctions\": { \"SingularDisplayName\": \"Azure Network Function Manager ? Network Function\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publisher\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/artifactstores\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers artifact store\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/configurationgroupschemas\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers configuration group schema\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/networkfunctiondefinitiongroups\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers network function definition group\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/networkfunctiondefinitiongroups/networkfunctiondefinitionversions\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers network function definition groups network function definition version\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/networkservicedesigngroups\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers network service design group\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/networkservicedesigngroups/networkservicedesignversions\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers network service design groups network service design version\" }\r\n ,\"microsoft.hybridnetwork/publishers\": { \"SingularDisplayName\": \"Publisher\" }\r\n ,\"microsoft.hybridnetwork/publishers/artifactstores\": { \"SingularDisplayName\": \"Publisher Artifact Store\" }\r\n ,\"microsoft.hybridnetwork/publishers/artifactstores/artifactmanifests\": { \"SingularDisplayName\": \"Publisher Artifact Manifest\" }\r\n ,\"microsoft.hybridnetwork/publishers/configurationgroupschemas\": { \"SingularDisplayName\": \"Configuration Group Schema\" }\r\n ,\"microsoft.hybridnetwork/publishers/networkfunctiondefinitiongroups\": { \"SingularDisplayName\": \"Network Function Definition\" }\r\n ,\"microsoft.hybridnetwork/publishers/networkfunctiondefinitiongroups/networkfunctiondefinitionversions\": { \"SingularDisplayName\": \"Network Function Definition Version\" }\r\n ,\"microsoft.hybridnetwork/publishers/networkservicedesigngroups\": { \"SingularDisplayName\": \"Network Service Design\" }\r\n ,\"microsoft.hybridnetwork/publishers/networkservicedesigngroups/networkservicedesignversions\": { \"SingularDisplayName\": \"Network Service Design Version\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management container\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers/rolloutsequences\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management containers rollout sequence\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers/rollouttiers\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management containers rollout tier\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers/updatespecifications\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management containers update specification\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers/updatespecifications/rollouts\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management containers update specifications rollout\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers/updatespecifications/rollouts/statuses\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management containers update specifications rollouts statuse\" }\r\n ,\"microsoft.hybridnetwork/sitenetworkservices\": { \"SingularDisplayName\": \"Site Network Service\" }\r\n ,\"microsoft.hybridnetwork/sites\": { \"SingularDisplayName\": \"Site\" }\r\n })[tolower(id)]\r\n}\r\n", + "$fxv#10": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Ingestion database\r\n// See known issues @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n//===| Prices |=========================================================================================================\r\n// Supported versions:\r\n// - MS EA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-ea\r\n// - MS MCA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-mca\r\n//======================================================================================================================\r\n\r\n// Prices_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='Transforms Prices_raw into FOCUS 1.2.', folder='Prices')\r\nPrices_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', int(null),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n let prices = materialize(\r\n Prices_raw\r\n | extend PricingCurrency = coalesce(Currency, CurrencyCode) // CurrencyCode last as a fallback only\r\n | extend x_SkuId = coalesce(SkuId, SkuID)\r\n | extend x_SkuMeterId = coalesce(MeterId, MeterID)\r\n | extend x_SkuProductId = coalesce(ProductId, ProductID)\r\n | extend x_SkuTerm = isoMonths(Term)\r\n | project-rename\r\n SkuMeter = MeterName,\r\n x_BaseUnitPrice = BasePrice,\r\n x_EffectivePeriodEnd = EffectiveEndDate,\r\n x_EffectivePeriodStart = EffectiveStartDate,\r\n x_PricingUnitDescription = UnitOfMeasure,\r\n x_SkuIncludedQuantity = IncludedQuantity,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuMeterType = MeterType,\r\n x_SkuOfferId = OfferID,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPriceType = PriceType,\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTier = TierMinimumUnits\r\n | extend ContractedUnitPrice = iff(x_SkuPriceType != 'SavingsPlan', UnitPrice, real(null)) // UnitPrice for savings plan is not the on-demand unit price\r\n | extend ListUnitPrice = iff(x_SkuPriceType != 'SavingsPlan', MarketPrice, real(null)) // MarketPrice for savings plan is not the list price\r\n | extend ChargeCategory = case(\r\n x_SkuPriceType == 'Consumption', 'Usage',\r\n x_SkuPriceType == 'ReservedInstance', 'Purchase',\r\n x_SkuPriceType == 'SavingsPlan', 'Usage', // Savings plan prices are for committed usage, not the purchase\r\n ''\r\n )\r\n | extend SkuPriceIdv2 = strcat(case(x_SkuPriceType == 'Consumption', 'OD', x_SkuPriceType == 'ReservedInstance', 'RI', x_SkuPriceType == 'SavingsPlan', 'SP', 'XX'), substring(ChargeCategory, 0, 1), x_SkuTerm, '_', x_SkuProductId, '_', x_SkuId, '_', x_SkuMeterType, '_', x_SkuTier, x_SkuOfferId)\r\n | extend x_BillingAccountId = iff(BillingAccountId startswith '/', split(BillingAccountId, '/')[-1], coalesce(BillingAccountId, EnrollmentNumber))\r\n | extend x_BillingProfileId = iff(BillingProfileId startswith '/', split(BillingProfileId, '/')[-1], coalesce(BillingProfileId, EnrollmentNumber))\r\n | extend tmp_SavingsPlanKey = strcat(x_SkuMeterId, x_SkuProductId, x_SkuId, x_SkuTier, x_SkuOfferId)\r\n //\r\n // Get latest ingested row based on the unique ID\r\n | extend x_IngestionTime = ingestion_time()\r\n );\r\n //\r\n // Meters for reservations and savings plans to identify commitment eligibility\r\n let riMeters = prices | where x_SkuPriceType == 'ReservedInstance' | distinct x_SkuMeterId;\r\n let spMeters = prices | where x_SkuPriceType == 'SavingsPlan' | distinct x_SkuMeterId;\r\n //\r\n // Copy list/base/contracted prices from on-demand SKUs\r\n prices\r\n | where x_SkuPriceType == 'SavingsPlan'\r\n // If we use join, specify the shuffle key\r\n // TODO: Compare join vs. lookup perf -- | join kind=leftouter hint.strategy=shuffle (prices | where x_SkuPriceType == 'Consumption' | where x_SkuMeterId in (spMeters) | distinct tmp_SavingsPlanKey, ListUnitPrice, ContractedUnitPrice, x_BaseUnitPrice) on tmp_SavingsPlanKey\r\n | lookup kind=leftouter (prices | where x_SkuPriceType == 'Consumption' | where x_SkuMeterId in (spMeters) | distinct tmp_SavingsPlanKey, ListUnitPrice, ContractedUnitPrice, x_BaseUnitPrice) on tmp_SavingsPlanKey\r\n | extend ListUnitPrice = coalesce(ListUnitPrice, ListUnitPrice1)\r\n | extend ContractedUnitPrice = coalesce(ContractedUnitPrice, ContractedUnitPrice1)\r\n | extend x_BaseUnitPrice = coalesce(x_BaseUnitPrice, x_BaseUnitPrice1)\r\n | project-away ListUnitPrice1, ContractedUnitPrice1, x_BaseUnitPrice1, tmp_SavingsPlanKey\r\n | union ((prices | where x_SkuPriceType != 'SavingsPlan'))\r\n //\r\n // Set CommitmentDiscountCategory for reuse\r\n | extend CommitmentDiscountCategory = case(\r\n x_SkuPriceType == 'ReservedInstance', 'Usage',\r\n x_SkuPriceType == 'SavingsPlan', 'Spend',\r\n ''\r\n )\r\n //\r\n // Calculate commitment discount eligibility\r\n // TODO: Would a join be faster?\r\n // TODO: Check this to ensure it's correct\r\n | extend x_CommitmentDiscountSpendEligibility = iff(x_SkuMeterId in (riMeters) and x_SkuPriceType != 'ReservedInstance', 'Eligible', 'Not Eligible')\r\n | extend x_CommitmentDiscountUsageEligibility = iff(x_SkuMeterId in (spMeters), 'Eligible', 'Not Eligible')\r\n //\r\n // TODO: Implement x_CommitmentDiscountNormalizedRatio\r\n | extend x_CommitmentDiscountNormalizedRatio = real(null)\r\n //\r\n // Add PricingUnit and x_PricingBlockSize\r\n // TODO: Compare join vs. lookup perf -- | join kind=leftouter (PricingUnits) on x_PricingUnitDescription | project-away x_PricingUnitDescription1\r\n | lookup kind=leftouter (PricingUnits) on x_PricingUnitDescription\r\n //\r\n | extend x_EffectiveUnitPrice = iff(x_SkuPriceType == 'SavingsPlan', UnitPrice, real(null)) // Savings plan prices are for the effective price, not the contracted price\r\n | extend x_EffectiveUnitPriceDiscount = ContractedUnitPrice - x_EffectiveUnitPrice\r\n | extend x_ContractedUnitPriceDiscount = ListUnitPrice - ContractedUnitPrice\r\n | extend x_TotalUnitPriceDiscount = ListUnitPrice - x_EffectiveUnitPrice\r\n | project\r\n BillingAccountId = tolower(case(\r\n BillingProfileId startswith '/', BillingProfileId,\r\n BillingAccountId startswith '/', BillingAccountId,\r\n strcat('/providers/microsoft.billing/billingaccounts/', x_BillingAccountId, iff(x_BillingProfileId == x_BillingAccountId, '', strcat('/billingprofiles/', x_BillingProfileId)))\r\n )),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName, x_BillingProfileId),\r\n BillingCurrency = coalesce(BillingCurrency, CurrencyCode, Currency), // Currency last as a fallback only\r\n ChargeCategory,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountType = case(\r\n x_SkuPriceType == 'ReservedInstance', 'Reservation',\r\n x_SkuPriceType == 'SavingsPlan', 'Savings plan',\r\n ''\r\n ),\r\n CommitmentDiscountUnit = case(\r\n isempty(CommitmentDiscountCategory), '',\r\n CommitmentDiscountCategory == 'Spend', PricingCurrency,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio == real(1), PricingUnit,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio > real(1), strcat('Normalized ', PricingUnit),\r\n ''\r\n ),\r\n ContractedUnitPrice,\r\n ListUnitPrice,\r\n PricingCategory = case(\r\n x_SkuPriceType == 'Consumption', 'Standard',\r\n x_SkuPriceType == 'ReservedInstance', 'Standard', // Reservation purchases are tracked as \"Standard\"\r\n x_SkuPriceType == 'SavingsPlan', 'Committed',\r\n ''\r\n ),\r\n PricingCurrency,\r\n PricingUnit,\r\n SkuId = coalesce(ProductId, ProductID),\r\n SkuMeter,\r\n SkuPriceId = strcat(x_SkuProductId, '_', x_SkuId, '_', x_SkuMeterType),\r\n SkuPriceIdv2,\r\n x_BaseUnitPrice,\r\n x_BillingAccountAgreement = case(\r\n strlen(x_BillingAccountId) > 32, 'MCA',\r\n strlen(x_BillingAccountId) < 32, 'EA',\r\n 'Unknown'\r\n ),\r\n x_BillingAccountId,\r\n x_BillingProfileId,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_CommitmentDiscountSpendEligibility,\r\n x_CommitmentDiscountUsageEligibility,\r\n x_ContractedUnitPriceDiscount,\r\n x_ContractedUnitPriceDiscountPercent = 1.0 * x_ContractedUnitPriceDiscount / ListUnitPrice * 100,\r\n x_EffectivePeriodEnd = startofmonth(x_EffectivePeriodEnd + 1h),\r\n x_EffectivePeriodStart,\r\n x_EffectiveUnitPrice,\r\n x_EffectiveUnitPriceDiscount,\r\n x_EffectiveUnitPriceDiscountPercent = 1.0 * x_EffectiveUnitPriceDiscount / ContractedUnitPrice * 100,\r\n x_IngestionTime,\r\n x_PricingBlockSize,\r\n x_PricingSubcategory = case(\r\n x_SkuPriceType == 'Consumption' and (x_SkuIncludedQuantity > 0 or x_SkuTier > 0), 'Tiered',\r\n x_SkuPriceType == 'Consumption', 'Standard',\r\n x_SkuPriceType == 'ReservedInstance', 'Standard', // Reservation purchases are tracked as \"Standard\"\r\n x_SkuPriceType == 'SavingsPlan', 'Committed Spend',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_SkuDescription = Product,\r\n x_SkuId,\r\n x_SkuIncludedQuantity,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterSubcategory,\r\n x_SkuMeterType,\r\n x_SkuPriceType,\r\n x_SkuProductId,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuOfferId,\r\n x_SkuPartNumber,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceName = coalesce(x_SourceName, 'Cost Management'),\r\n x_SourceProvider = coalesce(x_SourceProvider, 'Microsoft'),\r\n x_SourceType = coalesce(x_SourceType, 'PriceSheet'),\r\n x_SourceVersion = coalesce(x_SourceVersion, '2023-05-01'),\r\n x_TotalUnitPriceDiscount,\r\n x_TotalUnitPriceDiscountPercent = 1.0 * x_TotalUnitPriceDiscount / ListUnitPrice * 100\r\n}\r\n\r\n// Prices_final_v1_2 table\r\n.create-merge table Prices_final_v1_2 (\r\n BillingAccountId: string,\r\n BillingAccountName: string,\r\n BillingCurrency: string,\r\n ChargeCategory: string,\r\n CommitmentDiscountCategory: string,\r\n CommitmentDiscountType: string,\r\n CommitmentDiscountUnit: string,\r\n ContractedUnitPrice: real,\r\n ListUnitPrice: real,\r\n PricingCategory: string,\r\n PricingCurrency: string, // Azure\r\n PricingUnit: string,\r\n SkuId: string,\r\n SkuMeter: string, // Azure\r\n SkuPriceId: string,\r\n SkuPriceIdv2: string, // Hubs add-on\r\n x_BaseUnitPrice: real, // Azure\r\n x_BillingAccountAgreement: string, // Hubs add-on\r\n x_BillingAccountId: string, // Azure MCA\r\n x_BillingProfileId: string, // Azure MCA\r\n x_CommitmentDiscountNormalizedRatio: real, // Hubs add-on\r\n x_CommitmentDiscountSpendEligibility: string, // Hubs add-on\r\n x_CommitmentDiscountUsageEligibility: string, // Hubs add-on\r\n x_ContractedUnitPriceDiscount: real, // Hubs add-on\r\n x_ContractedUnitPriceDiscountPercent: real, // Hubs add-on\r\n x_EffectivePeriodEnd: datetime, // Azure\r\n x_EffectivePeriodStart: datetime, // Azure\r\n x_EffectiveUnitPrice: real, // Azure\r\n x_EffectiveUnitPriceDiscount: real, // Hubs add-on\r\n x_EffectiveUnitPriceDiscountPercent: real, // Hubs add-on\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_PricingBlockSize: real, // Hubs add-on\r\n x_PricingSubcategory: string, // Hubs add-on\r\n x_PricingUnitDescription: string, // Azure\r\n x_SkuDescription: string, // Azure\r\n x_SkuId: string, // Azure\r\n x_SkuIncludedQuantity: real, // Azure EA\r\n x_SkuMeterCategory: string, // Azure\r\n x_SkuMeterId: string, // Azure\r\n x_SkuMeterSubcategory: string, // Azure\r\n x_SkuMeterType: string, // Azure\r\n x_SkuPriceType: string, // Azure\r\n x_SkuProductId: string, // Azure\r\n x_SkuRegion: string, // Azure\r\n x_SkuServiceFamily: string, // Azure\r\n x_SkuOfferId: string, // Azure EA\r\n x_SkuPartNumber: string, // Azure EA\r\n x_SkuTerm: int, // Azure\r\n x_SkuTier: real, // Azure MCA\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_TotalUnitPriceDiscount: real, // Hubs add-on\r\n x_TotalUnitPriceDiscountPercent: real // Hubs add-on\r\n)\r\n\r\n// Update policy for Prices_raw -> Prices_final_v1_2\r\n.alter table Prices_final_v1_2 policy update\r\n```\r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"Prices_raw\",\r\n \"Query\": \"Prices_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Cost and usage |=================================================================================================\r\n// Supported versions:\r\n// - MS: 1.2-preview, 1.0, 1.0-preview(v1)\r\n// https://aka.ms/costmgmt/exports/focus\r\n// - AWS: 1.0\r\n// https://docs.aws.amazon.com/cur/latest/userguide/table-dictionary-focus-1-0-aws-columns.html\r\n// - GCP: Jan-Jun 2024\r\n// https://cloud.google.com/resources/google-cloud-focus?e=48754805&hl=en\r\n// Links to (Aug 2024): https://services.google.com/fh/files/misc/focus_guide_v1.pdf\r\n// See also:\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/standard-usage\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/detailed-usage\r\n// - OCI: 1.0 \r\n// https://docs.oracle.com/iaas/Content/Billing/Concepts/costusagereportsoverview.htm#costreports__focus-cost-report-schema\r\n//\r\n// Support for non-Azure data is limited to ingestion only. Data is not transformed across versions.\r\n//======================================================================================================================\r\n\r\n// Costs_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='All costs transformed to FOCUS 1.2.', folder='Costs')\r\nCosts_transform_v1_2()\r\n{\r\n let checkString = (column: string, oldValue: string, newValue: string) {\r\n iff(oldValue == newValue, dynamic({}), bag_pack(column, oldValue))\r\n };\r\n let checkInt = (column: string, oldValue: int, newValue: int) {\r\n iff(oldValue == newValue, dynamic({}), bag_pack(column, oldValue))\r\n };\r\n let checkReal = (column: string, oldValue: real, newValue: real) {\r\n iff(oldValue == newValue, dynamic({}), bag_pack(column, oldValue))\r\n };\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n Costs_raw\r\n //\r\n // Dedupe rows\r\n | extend x_IngestionTime = ingestion_time()\r\n | extend x_ChargeId = ''\r\n // TODO: Consider adding a unique charge ID per row\r\n // hash_sha256(strcat(\r\n // // DO NOT CHANGE COLUMNS OR COLUMN ORDER\r\n // // 1. Resource hierarchy (including resource name), highest to lowest\r\n // BillingAccountId,\r\n // x_InvoiceSectionId,\r\n // x_AccountOwnerId,\r\n // SubAccountId,\r\n // x_ResourceGroupName,\r\n // ResourceName,\r\n // // 2. Resource details\r\n // ResourceId,\r\n // RegionId,\r\n // Tags,\r\n // CommitmentDiscountId,\r\n // x_CostCenter,\r\n // // 4. Meter details\r\n // SkuPriceId,\r\n // x_SkuMeterId,\r\n // x_SkuPartNumber,\r\n // x_SkuOfferId,\r\n // x_SkuDetails,\r\n // // 5. Date\r\n // ChargePeriodStart\r\n // ))\r\n //\r\n // Identify data quality issues\r\n // TODO: Remove x_SourceChanges in v1_3 (or later)\r\n | extend x_SourceChanges = trim_end(',', strcat(\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and ChargeFrequency == 'Usage-Based', 'InvalidChargeFrequency,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and EffectiveCost > 0, 'InvalidEffectiveCost,', ''),\r\n iff((isempty(ContractedCost) or ContractedCost == 0) and EffectiveCost != 0, 'MissingContractedCost,', ''),\r\n iff((isempty(ContractedUnitPrice) or ContractedUnitPrice == 0) and x_EffectiveUnitPrice != 0, 'MissingContractedUnitPrice,', ''),\r\n iff(ListCost < ContractedCost, 'ListCostLessThanContractedCost,', ''),\r\n iff(ContractedCost < EffectiveCost, 'ContractedCostLessThanEffectiveCost,', ''),\r\n iff((isempty(ListCost) or ListCost == 0) and (ContractedCost != 0 or EffectiveCost != 0), 'MissingListCost,', ''),\r\n iff((isempty(ListUnitPrice) or ListUnitPrice == 0) and (ContractedUnitPrice != 0 or x_EffectiveUnitPrice != 0), 'MissingListUnitPrice,', ''),\r\n iff((isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(x_BilledUnitPrice - x_EffectiveUnitPrice) < 0.0001)\r\n or (isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(ContractedUnitPrice - x_EffectiveUnitPrice) < 0.0001),\r\n 'XEffectiveUnitPriceRoundingError,', ''),\r\n iff(ConsumedQuantity == 0 and (EffectiveCost != 0 or BilledCost != 0), 'MissingConsumedQuantity,', ''),\r\n iff(PricingQuantity == 0 and (EffectiveCost != 0 or BilledCost != 0), 'MissingPricingQuantity,', ''),\r\n iff(isempty(ProviderName), 'MissingProviderName,', ''),\r\n iff(isempty(PublisherName), 'MissingPublisherName,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceId), 'MissingResourceId,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceName), 'MissingResourceName,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceType), 'MissingResourceType,', ''),\r\n iff(BilledCost > 0 and x_BilledUnitPrice == 0, 'MissingXBilledUnitPrice,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(x_ResourceType), 'MissingXResourceType,', ''),\r\n iff(PricingCategory == 'Standard' and isnotempty(CommitmentDiscountId) and ChargeCategory == 'Usage', 'PricingCategoryShouldBeCommitted,', ''),\r\n iff(x_SkuTerm == '1Year' or x_SkuTerm == '3Years' or x_SkuTerm == '5Years', 'SkuTermShouldBeAnInteger,', '')\r\n ))\r\n //\r\n // Handle provider columns that moved to FOCUS\r\n | extend PricingCurrency = coalesce(PricingCurrency, x_PricingCurrency)\r\n //\r\n // Backup original prices/costs before the merge\r\n | extend old_ContractedCost = ContractedCost\r\n | extend old_ContractedUnitPrice = ContractedUnitPrice\r\n | extend old_ListCost = ListCost\r\n | extend old_ListUnitPrice = ListUnitPrice\r\n | extend old_x_EffectiveUnitPrice = x_EffectiveUnitPrice\r\n //\r\n // Fix columns needed in other changes\r\n | extend old_ProviderName = ProviderName, ProviderName = case(\r\n isnotempty(ProviderName), ProviderName,\r\n isnotempty(coalesce(x_CostCategories, x_Discount, x_Operation, x_ServiceCode, x_UsageType)), 'AWS',\r\n isnotempty(coalesce(tostring(UsageAmount), tostring(x_Cost), x_Credits, x_CostType, tostring(x_CurrencyConversionRate), tostring(x_ExportTime), x_Project, x_ServiceId)), 'GCP',\r\n isnotempty(coalesce(x_BillingProfileId, x_InvoiceSectionId)), 'Microsoft',\r\n ''\r\n )\r\n //\r\n // Identify source\r\n | extend x_SourceName = coalesce(x_SourceName, iff(isnotempty(x_BillingProfileId), 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(isnotempty(x_BillingProfileId), 'FocusCost', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, case(\r\n isnotempty(coalesce(InvoiceId, SkuMeter, PricingCurrency)) and isempty(SkuPriceDetails) and isnotempty(x_SkuDetails), '1.2-preview',\r\n isnotempty(coalesce(InvoiceId, SkuMeter, PricingCurrency)), '1.2',\r\n isnotempty(coalesce(ChargeClass, CommitmentDiscountStatus, tostring(ConsumedQuantity), ConsumedUnit, tostring(ContractedCost), tostring(ContractedUnitPrice), RegionId, RegionName)), '1.0',\r\n isnotempty(coalesce(ChargeSubcategory, Region, tostring(UsageQuantity), UsageUnit)), iff(ProviderName == 'Microsoft', '1.0-preview(v1)', '1.0-preview'),\r\n ''\r\n ))\r\n // Append version check error code\r\n | extend x_SourceChanges = iff(x_SourceVersion == '1.0', x_SourceChanges,\r\n strcat(x_SourceChanges, iff(isempty(x_SourceChanges), '', ','), iff(x_SourceVersion == '', 'UnknownFocusVersion', 'LegacyFocusVersion'))\r\n )\r\n //\r\n // Fix quantities\r\n | extend old_PricingQuantity = PricingQuantity, PricingQuantity = case(\r\n PricingQuantity != 0 or (EffectiveCost == 0 and BilledCost == 0), PricingQuantity,\r\n PricingQuantity == 0 and isnotempty(BilledCost) and BilledCost != 0 and isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, BilledCost / x_BilledUnitPrice,\r\n PricingQuantity == 0 and isnotempty(BilledCost) and BilledCost != 0 and isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, BilledCost / x_BilledUnitPrice,\r\n PricingQuantity == 0 and isnotempty(EffectiveCost) and EffectiveCost != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0, EffectiveCost / x_EffectiveUnitPrice,\r\n PricingQuantity\r\n )\r\n | extend old_ConsumedQuantity = ConsumedQuantity, ConsumedQuantity = case(\r\n isnotempty(ConsumedQuantity) and ConsumedQuantity != 0, ConsumedQuantity,\r\n ChargeCategory == 'Usage', PricingQuantity / coalesce(x_PricingBlockSize, real(1)),\r\n ConsumedQuantity\r\n )\r\n //\r\n // Populate missing prices -- mapping to on-demand prices requires meter ID and offer ID\r\n | extend tmp_MissingPrices = ProviderName == 'Microsoft'\r\n and (isempty(ListUnitPrice) or isempty(ContractedUnitPrice) or ListUnitPrice == 0 or ContractedUnitPrice == 0)\r\n and x_EffectiveUnitPrice != 0\r\n and not(CommitmentDiscountCategory == 'Spend' and CommitmentDiscountStatus == 'Unused')\r\n and isnotempty(strcat(x_SkuMeterId, x_SkuOfferId))\r\n | as allCosts\r\n | where tmp_MissingPrices\r\n | extend tmp_ReservationPriceLookupKey = tolower(strcat(x_BillingProfileId, substring(ChargePeriodStart, 0, 7), x_SkuMeterId, x_SkuOfferId))\r\n | as costsWithMissingPrices\r\n | join kind=leftouter (\r\n Prices_final_v1_2\r\n | extend tmp_ReservationPriceLookupKey = tolower(strcat(x_BillingProfileId, substring(x_EffectivePeriodStart, 0, 7), x_SkuMeterId, x_SkuOfferId))\r\n | where x_SkuPriceType == 'Consumption' and tmp_ReservationPriceLookupKey in ((costsWithMissingPrices | summarize by tmp_ReservationPriceLookupKey))\r\n | summarize ListUnitPrice = min(ListUnitPrice), ContractedUnitPrice = min(ContractedUnitPrice) by tmp_ReservationPriceLookupKey, x_PricingBlockSize, PricingUnit\r\n ) on tmp_ReservationPriceLookupKey\r\n //\r\n // Select the best price to use for each row\r\n | extend x_EffectiveUnitPrice = case(\r\n // If price is a rounding error away from the billed price, use the billed price\r\n isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(x_BilledUnitPrice - x_EffectiveUnitPrice) < 0.0001, x_BilledUnitPrice,\r\n // If price is a rounding error away from the contracted price, use the contracted price\r\n isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(ContractedUnitPrice - x_EffectiveUnitPrice) < 0.0001, ContractedUnitPrice,\r\n x_EffectiveUnitPrice\r\n )\r\n | extend ContractedUnitPrice = case(\r\n // If price is already correct, keep that\r\n (isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0) or (EffectiveCost == 0 and BilledCost == 0), ContractedUnitPrice,\r\n // If both prices use the same scale, use the new one\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize == x_PricingBlockSize1, ContractedUnitPrice1 * x_BillingExchangeRate,\r\n // If prices are the same unit but not the same scale, use the new one but correct the scale\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize != x_PricingBlockSize1 and isnotempty(x_PricingBlockSize) and isnotempty(x_PricingBlockSize1), ContractedUnitPrice1 * x_BillingExchangeRate / x_PricingBlockSize1 * x_PricingBlockSize,\r\n // If billed price is available, assume the billed price is the same as contracted price to support aggregations\r\n isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, x_EffectiveUnitPrice,\r\n // Otherwise, assume the effective price is the same as contracted price to support aggregations\r\n x_EffectiveUnitPrice\r\n )\r\n | extend ListUnitPrice = case(\r\n // If price is already correct, keep that\r\n (isnotempty(ListUnitPrice) and ListUnitPrice != 0) or (EffectiveCost == 0 and BilledCost == 0), ListUnitPrice,\r\n // If both prices use the same scale, use the new one\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize == x_PricingBlockSize1, ListUnitPrice1 * x_BillingExchangeRate,\r\n // If prices are the same unit but not the same scale, use the new one but correct the scale\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize != x_PricingBlockSize1 and isnotempty(x_PricingBlockSize) and isnotempty(x_PricingBlockSize1), ListUnitPrice1 * x_BillingExchangeRate / x_PricingBlockSize1 * x_PricingBlockSize,\r\n // Otherwise, assume the contracted price is the same as list price to support aggregations\r\n ContractedUnitPrice\r\n )\r\n // Calculate missing costs based on new prices -- If cost is already correct, keep that; if not and price is available, recalculate the cost; otherwise, keep the existing cost\r\n | extend ContractedCost = case(\r\n // If not set or there's no cost, keep the original value\r\n (isnotempty(ContractedCost) and ContractedCost != 0) or (EffectiveCost == 0 and BilledCost == 0), ContractedCost,\r\n // ContractedCost is 0 in all other scenarios...\r\n // If 0 and there's a billed cost and prices are the same, use BilledCost\r\n isnotempty(BilledCost) and BilledCost != 0 and ContractedUnitPrice == x_BilledUnitPrice, BilledCost,\r\n // If 0 and there's a billed cost and prices are the same, use EffectiveCost\r\n isnotempty(EffectiveCost) and EffectiveCost != 0 and ContractedUnitPrice == x_EffectiveUnitPrice, EffectiveCost,\r\n // If 0 and there's a price, calculate the cost based on the price\r\n isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0, ContractedUnitPrice * PricingQuantity,\r\n // If 0 and there's no price, assume EffectiveCost\r\n isempty(ContractedUnitPrice) or ContractedUnitPrice == 0, EffectiveCost,\r\n // Fall back to the original value for any unhandled scenarios\r\n ContractedCost\r\n )\r\n | extend ListCost = case(\r\n // If not set or there's no cost, keep the original value\r\n (isnotempty(ListCost) and ListCost != 0) or (EffectiveCost == 0 and BilledCost == 0), ListCost,\r\n // ListCost is 0 in all other scenarios...\r\n // If 0 and there's a contracted cost and prices are the same, use ContractedCost\r\n isnotempty(ContractedCost) and ContractedCost != 0 and ListUnitPrice == ContractedUnitPrice, ContractedCost,\r\n // If 0 and there's a price, calculate the cost based on the price\r\n isnotempty(ListUnitPrice) and ListUnitPrice != 0, ListUnitPrice * PricingQuantity,\r\n // If 0 and there's no price, assume ContractedCost\r\n isempty(ListUnitPrice) or ListUnitPrice == 0, ContractedCost,\r\n // Fall back to the original value for any unhandled scenarios\r\n ListCost\r\n )\r\n // Merge the rest of the unmodified cost records and remove excess columns\r\n | union (allCosts | where not(tmp_MissingPrices))\r\n | project-away x_PricingBlockSize1, PricingUnit1, ListUnitPrice1, ContractedUnitPrice1, tmp_MissingPrices, tmp_ReservationPriceLookupKey, tmp_ReservationPriceLookupKey1\r\n //\r\n | extend SkuPriceDetails = parse_json(SkuPriceDetails)\r\n | extend Tags = parse_json(Tags)\r\n | extend x_SkuDetails = parse_json(x_SkuDetails)\r\n //\r\n // Handle FOCUS 1.0-preview\r\n | extend old_ChargeSubcategory = ChargeSubcategory\r\n | extend old_ChargeCategory = ChargeCategory, ChargeCategory = case(\r\n // Handle FOCUS 1.0-preview ChargeSubcategory\r\n ChargeSubcategory == 'Credit', 'Credit',\r\n ChargeSubcategory == 'Refund', 'Purchase', // We are assuming purchase refunds since we don't have data to indicate usage refunds\r\n ChargeCategory\r\n )\r\n | extend old_ChargeClass = ChargeClass, ChargeClass = case(ChargeSubcategory == 'Refund', 'Correction', ChargeClass)\r\n //\r\n // Populate CapacityReservationId when not specified\r\n | extend CapacityReservationId = coalesce(CapacityReservationId, tostring(coalesce(x_SkuDetails.VMCapacityReservationId, SkuPriceDetails.VMCapacityReservationId, SkuPriceDetails.x_VMCapacityReservationId)))\r\n | extend old_CapacityReservationStatus = CapacityReservationStatus, CapacityReservationStatus = case(\r\n isempty(CapacityReservationId), '',\r\n isnotempty(CapacityReservationStatus), CapacityReservationStatus,\r\n tolower(x_ResourceType) == 'microsoft.compute/capacityreservationgroups/capacityreservations', 'Unused',\r\n 'Used'\r\n )\r\n //\r\n // BUG: ChargeFrequency shows \"Usage-Based\" for monthly recurring savings plan purchases\r\n | extend old_ChargeFrequency = ChargeFrequency, ChargeFrequency = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and ChargeFrequency == 'Usage-Based' and ProviderName == 'Microsoft' and x_SourceVersion startswith '1.0', 'Recurring', ChargeFrequency)\r\n //\r\n // Commitment discounts\r\n | extend x_CommitmentDiscountNormalizedRatio = case(\r\n // Calculate from CommitmentDiscountQuantity, if specified\r\n isnotempty(CommitmentDiscountQuantity) and CommitmentDiscountQuantity != 0, CommitmentDiscountQuantity / PricingQuantity / coalesce(x_PricingBlockSize, real(1)),\r\n // Not applicable\r\n isempty(CommitmentDiscountStatus), real(null),\r\n // Parse from SKU details if not specified explicitly\r\n toreal(coalesce(x_SkuDetails.RINormalizationRatio, SkuPriceDetails.RINormalizationRatio, SkuPriceDetails.x_RINormalizationRatio, dynamic(1)))\r\n )\r\n | extend old_CommitmentDiscountQuantity = CommitmentDiscountQuantity, CommitmentDiscountQuantity = case(\r\n // FOCUS 1.2\r\n isnotempty(CommitmentDiscountQuantity), CommitmentDiscountQuantity,\r\n // FOCUS 1.0-preview, 1.0\r\n isempty(CommitmentDiscountStatus), real(null),\r\n CommitmentDiscountCategory == 'Spend', EffectiveCost / coalesce(x_BillingExchangeRate, real(1)),\r\n CommitmentDiscountCategory == 'Usage' and isnotempty(x_CommitmentDiscountNormalizedRatio), PricingQuantity / coalesce(x_PricingBlockSize, real(1)) * x_CommitmentDiscountNormalizedRatio,\r\n real(null)\r\n )\r\n | extend old_CommitmentDiscountUnit = CommitmentDiscountUnit, CommitmentDiscountUnit = case(\r\n // FOCUS 1.2\r\n isnotempty(CommitmentDiscountUnit), CommitmentDiscountUnit,\r\n // FOCUS 1.0\r\n isempty(CommitmentDiscountQuantity), '',\r\n CommitmentDiscountCategory == 'Spend', PricingCurrency,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio == real(1), ConsumedUnit,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio > real(1), strcat('Normalized ', ConsumedUnit),\r\n ''\r\n )\r\n | extend old_CommitmentDiscountStatus = CommitmentDiscountStatus, CommitmentDiscountStatus = case(\r\n // FOCUS 1.0+\r\n isnotempty(CommitmentDiscountStatus), CommitmentDiscountStatus,\r\n // FOCUS 1.0-preview\r\n ChargeSubcategory == 'Used Commitment', 'Used',\r\n ChargeSubcategory == 'Unused Commitment', 'Unused',\r\n ''\r\n )\r\n | extend x_CommitmentDiscountUtilizationPotential = case(\r\n ChargeCategory == 'Purchase', real(0),\r\n ProviderName == 'Microsoft' and isnotempty(CommitmentDiscountCategory), EffectiveCost,\r\n CommitmentDiscountCategory == 'Usage', ConsumedQuantity,\r\n CommitmentDiscountCategory == 'Spend', EffectiveCost,\r\n real(0)\r\n )\r\n | extend x_CommitmentDiscountUtilizationAmount = iff(CommitmentDiscountStatus == 'Used', x_CommitmentDiscountUtilizationPotential, real(0))\r\n //\r\n // Pricing\r\n | extend old_x_AmortizationClass = x_AmortizationClass, x_AmortizationClass = case(\r\n // FOCUS 1.2\r\n isnotempty(x_AmortizationClass), x_AmortizationClass,\r\n // FOCUS 1.0-preview+\r\n ChargeCategory == 'Purchase' and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeCategory == 'Usage' and isnotempty(CommitmentDiscountId) and isnotempty(CommitmentDiscountStatus), 'Amortized Charge',\r\n ''\r\n )\r\n | extend old_PricingCategory = PricingCategory, PricingCategory = case(\r\n // FOCUS 1.0+\r\n isnotempty(PricingCategory), PricingCategory,\r\n // FOCUS 1.0-preview\r\n PricingCategory == 'On-Demand', 'Standard',\r\n PricingCategory == 'Commitment-Based', 'Committed',\r\n ''\r\n )\r\n //\r\n // Commitment discount utilization\r\n | extend x_CommitmentDiscountUtilizationPotential = case(\r\n ChargeCategory == 'Purchase', real(0),\r\n ProviderName == 'Microsoft' and isnotempty(CommitmentDiscountCategory), EffectiveCost,\r\n CommitmentDiscountCategory == 'Usage', ConsumedQuantity,\r\n CommitmentDiscountCategory == 'Spend', EffectiveCost,\r\n real(0)\r\n )\r\n | extend x_CommitmentDiscountUtilizationAmount = iff(CommitmentDiscountStatus == 'Used', x_CommitmentDiscountUtilizationPotential, real(0))\r\n //\r\n // BUG: Fix ContractedCost that has bad values\r\n | extend ContractedCost = iff(ProviderName == 'Microsoft' and isnotempty(PricingQuantity) and isnotempty(x_PricingBlockSize) and ContractedCost != ContractedUnitPrice * PricingQuantity, ContractedUnitPrice * PricingQuantity, ContractedCost)\r\n //\r\n // Handle FOCUS 1.0-preview UsageQuantity/Unit\r\n | extend ConsumedQuantity = iff(ChargeCategory == 'Usage', coalesce(ConsumedQuantity, UsageQuantity, UsageAmount), real(null))\r\n | extend old_ConsumedUnit = ConsumedUnit, ConsumedUnit = iff(ChargeCategory == 'Usage' and isnotempty(ConsumedQuantity), coalesce(ConsumedUnit, UsageUnit, 'Units'), '')\r\n //\r\n // Convert IDs to lowercase for consistency\r\n | extend BillingAccountId = tolower(BillingAccountId)\r\n | extend CommitmentDiscountId = tolower(CommitmentDiscountId)\r\n //\r\n // BUG: Remove EffectiveCost for commitment discount purchases\r\n | extend old_EffectiveCost = EffectiveCost, EffectiveCost = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), real(0), EffectiveCost)\r\n | extend old_x_EffectiveCostInUsd = x_EffectiveCostInUsd, x_EffectiveCostInUsd = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), real(0), x_EffectiveCostInUsd)\r\n //\r\n // Clean up resource columns\r\n | extend old_ResourceId = ResourceId, ResourceId = case(\r\n isnotempty(ResourceId), ResourceId,\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), CommitmentDiscountId,\r\n ResourceId\r\n )\r\n | extend old_ResourceName = ResourceName, ResourceName = tolower(case(\r\n isnotempty(ResourceName), ResourceName,\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountName), CommitmentDiscountName,\r\n isnotempty(ResourceId), parse_resourceid(ResourceId).ResourceName,\r\n ResourceName\r\n ))\r\n | extend old_x_ResourceType = x_ResourceType, x_ResourceType = case(\r\n isnotempty(x_ResourceType), x_ResourceType,\r\n isnotempty(ResourceId), parse_resourceid(ResourceId).x_ResourceType,\r\n x_ResourceType\r\n )\r\n | extend old_ResourceType = ResourceType, ResourceType = case(\r\n // Use existing resource type display name unless it's an internal resource type ID\r\n isnotempty(ResourceType) and tolower(ResourceType) != tolower(x_ResourceType) and ResourceType !contains '/', ResourceType,\r\n // Use CommitmentDiscountType for commitment discount purchases\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountType), CommitmentDiscountType,\r\n // Look up display name from internal type\r\n isnotempty(x_ResourceType), coalesce(tostring(resource_type(x_ResourceType).SingularDisplayName), ResourceType, x_ResourceType),\r\n ResourceType\r\n )\r\n //\r\n // Handle missing values\r\n | extend old_PublisherName = PublisherName, PublisherName = case(PublisherName == 'Microsoft Corporation', 'Microsoft', isnotempty(PublisherName), PublisherName, x_PublisherCategory == 'Cloud Provider', ProviderName, '')\r\n //\r\n // Handle FOCUS 1.0-preview Region column\r\n | extend old_Region = Region\r\n | extend old_RegionId = RegionId, RegionId = coalesce(RegionId, iff(ProviderName == 'Microsoft', replace_string(tolower(Region), ' ', ''), Region))\r\n | extend RegionName = coalesce(RegionName, Region)\r\n //\r\n // SKU properties\r\n | extend x_SkuCoreCount = toint(coalesce(SkuPriceDetails.CoreCount, SkuPriceDetails.x_VCPUs, x_SkuDetails.VCPUs, SkuPriceDetails.x_VCores, x_SkuDetails.VCores, SkuPriceDetails.x_vCores, x_SkuDetails.vCores))\r\n | extend x_SkuInstanceType = tostring(coalesce(SkuPriceDetails.InstanceType, SkuPriceDetails.x_ServiceType, x_SkuDetails.ServiceType, SkuPriceDetails.x_ServerSku, x_SkuDetails.ServerSku))\r\n | extend x_SkuOperatingSystem = case(\r\n isnotempty(SkuPriceDetails.OperatingSystem), SkuPriceDetails.OperatingSystem,\r\n coalesce(SkuPriceDetails.x_ImageType, x_SkuDetails.ImageType) == 'Canonical', 'Linux',\r\n coalesce(SkuPriceDetails.x_ImageType, x_SkuDetails.ImageType) == 'Windows Server BYOL', 'Windows Server',\r\n x_SkuMeterSubcategory endswith ' Series Windows', 'Windows Server',\r\n coalesce(SkuPriceDetails.x_ImageType, x_SkuDetails.ImageType)\r\n )\r\n | extend x_ConsumedCoreHours = iff(ConsumedUnit == 'Hours' and isnotempty(x_SkuCoreCount), x_SkuCoreCount * ConsumedQuantity, real(null))\r\n | extend SkuPriceDetails = case(\r\n // FOCUS 1.2\r\n isnotempty(SkuPriceDetails), SkuPriceDetails,\r\n // FOCUS 1.0-preview, 1.0\r\n parse_json(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(tostring(x_SkuDetails)\r\n // Prefix all keys with x_ first to avoid double-prefixing\r\n , @'([\\{,])\"', @'\\1\"x_')\r\n // CoreCount for number of CPUs/vCPUs/cores/vCores\r\n , @'\"x_(VCPUs|VCores|vCores)\":', @'\"CoreCount\":')\r\n // TODO: DiskMaxIops for disk I/O operations per second (IOPS)\r\n // TODO: DiskSpace for disk size in GiB\r\n // TODO: DiskType for the kind of disk (e.g., SSD, HDD, NVMe)\r\n // TODO: GpuCount for the number of GPUs\r\n // InstanceType for the resource size/SKU (e.g., ArmSkuName)\r\n , @'\"x_(ServerSku|ServiceType)\":', @'\"InstanceType\":')\r\n // TODO: InstanceSeries for the size family/series\r\n // TODO: MemorySize for the RAM in GiB\r\n // TODO: NetworkMaxIops for network I/O operations per second (IOPS)\r\n // TODO: NetworkMaxThroughput for network max throughput for data transfer in Mbps\r\n // OperatingSystem for the OS name\r\n , @'(\"x_ImageType\":\"Canonical\")', @'\\1,\"OperatingSystem\":\"Linux\"')\r\n , @'(\"x_ImageType\":\"Windows Server( BYOL)?\")', @'\\1,\"OperatingSystem\":\"Windows Server\"')\r\n , @'(\"x_ImageType\":(\"[^\"]+\"))', @'\\1,\"OperatingSystem\":\\2')\r\n // TODO: Redundancy for the level of redundancy (e.g., Local, Zonal, Global)\r\n // TODO: StorageClass for the tier of storage (e.g., Hot, Archive, Nearline)\r\n )\r\n )\r\n | extend SkuPriceDetails = iff(isempty(SkuPriceDetails.OperatingSystem) and isnotempty(x_SkuOperatingSystem),\r\n parse_json(replace_string(tostring(SkuPriceDetails), '}', strcat(@',\"OperatingSystem\":\"', x_SkuOperatingSystem, '\"}'))),\r\n SkuPriceDetails)\r\n //\r\n // Azure Hybrid Benefit\r\n | extend tmp_SqlAhb = tolower(coalesce(x_SkuDetails.AHB, SkuPriceDetails.x_AHB))\r\n | extend x_SkuLicenseType = case(\r\n ChargeCategory != 'Usage', '',\r\n x_SkuMeterCategory in ('Virtual Machines', 'Virtual Machine Licenses') and (x_SkuMeterSubcategory contains 'Windows' or coalesce(SkuPriceDetails.x_ImageType, x_SkuDetails.ImageType) == 'Windows Server BYOL'), 'Windows Server',\r\n isnotempty(tmp_SqlAhb) or x_SkuMeterSubcategory == 'SQL Server Azure Hybrid Benefit', 'SQL Server',\r\n ''\r\n )\r\n | extend x_SkuLicenseStatus = case(\r\n isempty(x_SkuLicenseType), '',\r\n coalesce(SkuPriceDetails.x_ImageType, x_SkuDetails.ImageType) == 'Windows Server BYOL' or tmp_SqlAhb == 'true' or x_SkuMeterSubcategory contains 'Azure Hybrid Benefit', 'Enabled',\r\n (x_SkuMeterSubcategory contains 'Windows') or tmp_SqlAhb == 'false', 'Not Enabled',\r\n ''\r\n )\r\n | extend x_SkuLicenseQuantity = case(\r\n isempty(x_SkuCoreCount) or isempty(x_SkuLicenseType), int(null),\r\n x_SkuCoreCount <= 8, int(8),\r\n x_SkuCoreCount > 8, x_SkuCoreCount,\r\n int(null)\r\n )\r\n | extend x_SkuLicenseUnit = iff(isnotempty(x_SkuLicenseQuantity), 'Cores', '')\r\n //\r\n // Savings\r\n | extend x_CommitmentDiscountSavings = iff(isempty(ContractedCost) or ContractedCost == 0 or ContractedCost - EffectiveCost < 0.0001, real(0), ContractedCost - EffectiveCost)\r\n | extend x_NegotiatedDiscountSavings = iff(isempty(ListCost) or ListCost == 0 or ListCost - ContractedCost < 0.0001, real(0), ListCost - ContractedCost)\r\n | extend x_TotalSavings = iff(isempty(ListCost) or ListCost == 0 or ListCost - EffectiveCost < 0.0001, real(0), ListCost - EffectiveCost)\r\n | extend x_CommitmentDiscountPercent = iff(isempty(ContractedUnitPrice) or ContractedUnitPrice == 0 or ContractedUnitPrice - x_EffectiveUnitPrice < 0.0001, real(0), (ContractedUnitPrice - x_EffectiveUnitPrice) / ContractedUnitPrice)\r\n | extend x_NegotiatedDiscountPercent = iff(isempty(ListUnitPrice) or ListUnitPrice == 0 or ListUnitPrice - ContractedUnitPrice < 0.0001, real(0), (ListUnitPrice - ContractedUnitPrice) / ListUnitPrice)\r\n | extend x_TotalDiscountPercent = iff(isempty(ListUnitPrice) or ListUnitPrice == 0 or ListUnitPrice - x_EffectiveUnitPrice < 0.0001, real(0), (ListUnitPrice - x_EffectiveUnitPrice) / ListUnitPrice)\r\n //\r\n // Minor fixes\r\n | extend old_BillingPeriodEnd = BillingPeriodEnd, BillingPeriodEnd = startofmonth(BillingPeriodEnd)\r\n | extend old_BillingPeriodStart = BillingPeriodStart, BillingPeriodStart = startofmonth(BillingPeriodStart)\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n AvailabilityZone,\r\n BilledCost,\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingAccountType,\r\n BillingCurrency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n CapacityReservationId,\r\n CapacityReservationStatus,\r\n ChargeCategory,\r\n ChargeClass,\r\n ChargeDescription,\r\n ChargeFrequency,\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId,\r\n CommitmentDiscountName,\r\n CommitmentDiscountQuantity,\r\n CommitmentDiscountStatus,\r\n CommitmentDiscountType,\r\n CommitmentDiscountUnit,\r\n ConsumedQuantity,\r\n ConsumedUnit,\r\n ContractedCost = coalesce(ContractedCost, x_OnDemandCost, x_Cost),\r\n ContractedUnitPrice = coalesce(ContractedUnitPrice, x_OnDemandUnitPrice),\r\n EffectiveCost,\r\n InvoiceId = coalesce(InvoiceId, x_InvoiceId),\r\n InvoiceIssuerName,\r\n ListCost,\r\n ListUnitPrice,\r\n PricingCategory,\r\n PricingCurrency = coalesce(PricingCurrency, x_PricingCurrency),\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n PublisherName,\r\n RegionId,\r\n RegionName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory, // TODO: Populate ServiceSubcategory from ServiceName when missing\r\n SkuId,\r\n SkuMeter = coalesce(SkuMeter, x_SkuMeterName),\r\n SkuPriceDetails,\r\n SkuPriceId,\r\n SubAccountId,\r\n SubAccountName = iff(isempty(SubAccountId), '', SubAccountName),\r\n SubAccountType,\r\n Tags,\r\n x_AccountId = iff(x_AccountId == '-2', '', x_AccountId),\r\n x_AccountName = iff(x_AccountId == '-2', '', x_AccountName),\r\n x_AccountOwnerId = iff(x_AccountId == '-2', '', x_AccountOwnerId),\r\n x_AmortizationClass,\r\n x_BilledCostInUsd,\r\n x_BilledUnitPrice,\r\n x_BillingAccountAgreement = case(\r\n ProviderName == 'Microsoft' and x_BillingAccountId == x_BillingProfileId, 'EA',\r\n ProviderName == 'Microsoft' and x_BillingAccountId != x_BillingProfileId, 'MCA',\r\n ProviderName\r\n ),\r\n x_BillingAccountId,\r\n x_BillingAccountName,\r\n x_BillingExchangeRate,\r\n x_BillingExchangeRateDate,\r\n x_BillingItemCode,\r\n x_BillingItemName,\r\n x_BillingProfileId,\r\n x_BillingProfileName,\r\n x_ChargeId,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_CommitmentDiscountPercent,\r\n x_CommitmentDiscountSavings,\r\n x_CommitmentDiscountSpendEligibility = '', // TODO: Add x_CommitmentDiscountSpendEligibility for Costs\r\n x_CommitmentDiscountUsageEligibility = '', // TODO: Add x_CommitmentDiscountUsageEligibility for Costs\r\n x_CommitmentDiscountUtilizationAmount,\r\n x_CommitmentDiscountUtilizationPotential,\r\n x_CommodityCode,\r\n x_CommodityName,\r\n x_ComponentName,\r\n x_ComponentType,\r\n x_ConsumedCoreHours,\r\n x_ContractedCostInUsd = coalesce(x_ContractedCostInUsd, x_OnDemandCostInUsd),\r\n x_CostAllocationRuleName,\r\n x_CostCategories = parse_json(x_CostCategories),\r\n x_CostCenter,\r\n x_CostType,\r\n x_Credits = parse_json(x_Credits),\r\n x_CurrencyConversionRate,\r\n x_CustomerId,\r\n x_CustomerName,\r\n x_Discount = parse_json(x_Discount),\r\n x_EffectiveCostInUsd,\r\n x_EffectiveUnitPrice,\r\n x_ExportTime,\r\n x_IngestionTime,\r\n x_InstanceID,\r\n x_InvoiceIssuerId,\r\n x_InvoiceSectionId = case(\r\n x_InvoiceSectionId == '-2', '',\r\n x_InvoiceSectionId\r\n ),\r\n x_InvoiceSectionName = case(\r\n x_InvoiceSectionName == 'Unassigned', '',\r\n x_InvoiceSectionName\r\n ),\r\n x_ListCostInUsd,\r\n x_Location,\r\n x_NegotiatedDiscountPercent,\r\n x_NegotiatedDiscountSavings,\r\n x_Operation,\r\n x_OwnerAccountID,\r\n x_PartnerCreditApplied,\r\n x_PartnerCreditRate,\r\n x_PricingBlockSize,\r\n x_PricingSubcategory,\r\n x_PricingUnitDescription = iff(x_PricingUnitDescription == 'Unassigned', '', x_PricingUnitDescription),\r\n x_Project,\r\n x_PublisherCategory,\r\n x_PublisherId,\r\n x_ResellerId,\r\n x_ResellerName,\r\n x_ResourceGroupName = tolower(x_ResourceGroupName),\r\n x_ResourceType,\r\n x_ServiceCode,\r\n x_ServiceId,\r\n x_ServiceModel, // TODO: Populate from ServiceName when missing\r\n x_ServicePeriodEnd,\r\n x_ServicePeriodStart,\r\n x_SkuCoreCount,\r\n x_SkuDescription,\r\n x_SkuDetails,\r\n x_SkuInstanceType,\r\n x_SkuIsCreditEligible,\r\n x_SkuLicenseQuantity,\r\n x_SkuLicenseStatus,\r\n x_SkuLicenseType,\r\n x_SkuLicenseUnit,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterSubcategory,\r\n x_SkuOfferId,\r\n x_SkuOperatingSystem,\r\n x_SkuOrderId,\r\n x_SkuOrderName,\r\n x_SkuPartNumber,\r\n x_SkuPlanName,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceChanges,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceValues = bag_merge(\r\n checkString('BillingPeriodEnd', old_BillingPeriodEnd, BillingPeriodEnd),\r\n checkString('BillingPeriodStart', old_BillingPeriodStart, BillingPeriodStart),\r\n checkString('CapacityReservationStatus', old_CapacityReservationStatus, CapacityReservationStatus),\r\n checkString('ChargeCategory', old_ChargeCategory, ChargeCategory),\r\n checkString('ChargeClass', old_ChargeClass, ChargeClass),\r\n checkString('ChargeSubcategory', old_ChargeSubcategory, ''), // Not included in final schema; use empty string\r\n checkString('ChargeFrequency', old_ChargeFrequency, ChargeFrequency),\r\n checkReal('CommitmentDiscountQuantity', old_CommitmentDiscountQuantity, CommitmentDiscountQuantity),\r\n checkString('CommitmentDiscountUnit', old_CommitmentDiscountUnit, CommitmentDiscountUnit),\r\n checkString('CommitmentDiscountStatus', old_CommitmentDiscountStatus, CommitmentDiscountStatus),\r\n checkReal('ConsumedQuantity', old_ConsumedQuantity, ConsumedQuantity),\r\n checkString('ConsumedUnit', old_ConsumedUnit, ConsumedUnit),\r\n checkReal('ContractedCost', old_ContractedCost, ContractedCost),\r\n checkReal('ContractedUnitPrice', old_ContractedUnitPrice, ContractedUnitPrice),\r\n checkReal('EffectiveCost', old_EffectiveCost, EffectiveCost),\r\n checkReal('ListCost', old_ListCost, ListCost),\r\n checkReal('ListUnitPrice', old_ListUnitPrice, ListUnitPrice),\r\n checkString('PricingCategory', old_PricingCategory, PricingCategory),\r\n checkReal('PricingQuantity', old_PricingQuantity, PricingQuantity),\r\n checkString('ProviderName', old_ProviderName, ProviderName),\r\n checkString('PublisherName', old_PublisherName, PublisherName),\r\n checkString('Region', old_Region, ''), // Not included in final schema; use empty string\r\n checkString('RegionId', old_RegionId, RegionId),\r\n checkString('ResourceId', old_ResourceId, ResourceId),\r\n checkString('ResourceName', old_ResourceName, ResourceName),\r\n checkString('ResourceType', old_ResourceType, ResourceType),\r\n checkString('x_AmortizationClass', old_x_AmortizationClass, x_AmortizationClass),\r\n checkReal('x_EffectiveCostInUsd', old_x_EffectiveCostInUsd, x_EffectiveCostInUsd),\r\n checkReal('x_EffectiveUnitPrice', old_x_EffectiveUnitPrice, x_EffectiveUnitPrice),\r\n checkString('x_ResourceType', old_x_ResourceType, x_ResourceType)\r\n ),\r\n x_SourceVersion,\r\n x_SubproductName,\r\n x_TotalDiscountPercent,\r\n x_TotalSavings,\r\n x_UsageType\r\n}\r\n\r\n// Costs_final_v1_2 table\r\n.create-merge table Costs_final_v1_2 (\r\n AvailabilityZone: string,\r\n BilledCost: real,\r\n BillingAccountId: string,\r\n BillingAccountName: string,\r\n BillingAccountType: string,\r\n BillingCurrency: string,\r\n BillingPeriodEnd: datetime,\r\n BillingPeriodStart: datetime,\r\n CapacityReservationId: string,\r\n CapacityReservationStatus: string,\r\n ChargeCategory: string,\r\n ChargeClass: string,\r\n ChargeDescription: string,\r\n ChargeFrequency: string,\r\n ChargePeriodEnd: datetime,\r\n ChargePeriodStart: datetime,\r\n CommitmentDiscountCategory: string,\r\n CommitmentDiscountId: string,\r\n CommitmentDiscountName: string,\r\n CommitmentDiscountQuantity: real,\r\n CommitmentDiscountStatus: string,\r\n CommitmentDiscountType: string,\r\n CommitmentDiscountUnit: string,\r\n ConsumedQuantity: real,\r\n ConsumedUnit: string,\r\n ContractedCost: real,\r\n ContractedUnitPrice: real,\r\n EffectiveCost: real,\r\n InvoiceId: string,\r\n InvoiceIssuerName: string,\r\n ListCost: real,\r\n ListUnitPrice: real,\r\n PricingCategory: string,\r\n PricingCurrency: string,\r\n PricingQuantity: real,\r\n PricingUnit: string,\r\n ProviderName: string,\r\n PublisherName: string,\r\n RegionId: string,\r\n RegionName: string,\r\n ResourceId: string,\r\n ResourceName: string,\r\n ResourceType: string,\r\n ServiceCategory: string,\r\n ServiceName: string,\r\n ServiceSubcategory: string,\r\n SkuId: string,\r\n SkuMeter: string,\r\n SkuPriceDetails: dynamic,\r\n SkuPriceId: string,\r\n SubAccountId: string,\r\n SubAccountName: string,\r\n SubAccountType: string,\r\n Tags: dynamic,\r\n x_AccountId: string, // Azure 1.0-preview(v1)+\r\n x_AccountName: string, // Azure 1.0-preview(v1)+\r\n x_AccountOwnerId: string, // Azure 1.0-preview(v1)+\r\n x_AmortizationClass: string, // Azure 1.2-preview+\r\n x_BilledCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_BilledUnitPrice: real, // Azure 1.0-preview(v1)+\r\n x_BillingAccountAgreement: string, // Hubs add-on\r\n x_BillingAccountId: string, // Azure 1.0-preview(v1)+\r\n x_BillingAccountName: string, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRate: real, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRateDate: datetime, // Azure 1.0-preview(v1)+\r\n x_BillingItemCode: string, // Alibaba 1.0\r\n x_BillingItemName: string, // Alibaba 1.0\r\n x_BillingProfileId: string, // Azure 1.0-preview(v1)+\r\n x_BillingProfileName: string, // Azure 1.0-preview(v1)+\r\n x_ChargeId: string, // Azure 1.0-preview(v1) only\r\n x_CommitmentDiscountNormalizedRatio: real, // Azure 1.2-preview+\r\n x_CommitmentDiscountPercent: real, // Hubs add-on\r\n x_CommitmentDiscountSavings: real, // Hubs add-on\r\n x_CommitmentDiscountSpendEligibility: string, // Hubs add-on\r\n x_CommitmentDiscountUsageEligibility: string, // Hubs add-on\r\n x_CommitmentDiscountUtilizationAmount: real, // Hubs add-on\r\n x_CommitmentDiscountUtilizationPotential: real, // Hubs add-on\r\n x_CommodityCode: string, // Alibaba 1.0\r\n x_CommodityName: string, // Alibaba 1.0\r\n x_ComponentName: string, // Tencent 1.0\r\n x_ComponentType: string, // Tencent 1.0\r\n x_ConsumedCoreHours: real, // Hubs add-on\r\n x_ContractedCostInUsd: real, // Azure 1.0+\r\n x_CostAllocationRuleName: string, // Azure 1.0-preview(v1)+\r\n x_CostCategories: dynamic, // AWS 1.0 (JSON)\r\n x_CostCenter: string, // Azure 1.0-preview(v1)+\r\n x_CostType: string, // GCP Jan 2024\r\n x_Credits: dynamic, // GCP Jan 2024\r\n x_CurrencyConversionRate: real, // GCP Jun 2024\r\n x_CustomerId: string, // Azure 1.0-preview(v1)+\r\n x_CustomerName: string, // Azure 1.0-preview(v1)+\r\n x_Discount: dynamic, // AWS 1.0 (JSON)\r\n x_EffectiveCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_EffectiveUnitPrice: real, // Azure 1.0-preview(v1)+\r\n x_ExportTime: datetime, // GCP Jan 2024 / Tencent 1.0\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_InstanceID: string, // Alibaba 1.0\r\n x_InvoiceIssuerId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionName: string, // Azure 1.0-preview(v1)+\r\n x_ListCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_Location: string, // GCP Jan 2024\r\n x_NegotiatedDiscountPercent:real, // Hubs add-on\r\n x_NegotiatedDiscountSavings:real, // Hubs add-on\r\n x_Operation: string, // AWS 1.0\r\n x_OwnerAccountID: string, // Tencent 1.0\r\n x_PartnerCreditApplied: string, // Azure 1.0-preview(v1)+\r\n x_PartnerCreditRate: string, // Azure 1.0-preview(v1)+\r\n x_PricingBlockSize: real, // Azure 1.0-preview(v1)+\r\n x_PricingSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_PricingUnitDescription: string, // Azure 1.0-preview(v1)+\r\n x_Project: string, // GCP Jan 2024\r\n x_PublisherCategory: string, // Azure 1.0-preview(v1)+\r\n x_PublisherId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceGroupName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceType: string, // Azure 1.0-preview(v1)+\r\n x_ServiceCode: string, // AWS 1.0\r\n x_ServiceId: string, // GCP Jan 2024\r\n x_ServiceModel: string, // Azure 1.2-preview+\r\n x_ServicePeriodEnd: datetime, // Azure 1.0-preview(v1)+\r\n x_ServicePeriodStart: datetime, // Azure 1.0-preview(v1)+\r\n x_SkuCoreCount: int, // Hubs add-on\r\n x_SkuDescription: string, // Azure 1.0-preview(v1)+\r\n x_SkuDetails: dynamic, // Azure 1.0-preview(v1)+\r\n x_SkuInstanceType: string, // Hubs add-on\r\n x_SkuIsCreditEligible: bool, // Azure 1.0-preview(v1)+\r\n x_SkuLicenseQuantity: int, // Hubs add-on\r\n x_SkuLicenseStatus: string, // Hubs add-on\r\n x_SkuLicenseType: string, // Hubs add-on\r\n x_SkuLicenseUnit: string, // Hubs add-on\r\n x_SkuMeterCategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterId: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuOfferId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOperatingSystem: string, // Hubs add-on\r\n x_SkuOrderId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOrderName: string, // Azure 1.0-preview(v1)+\r\n x_SkuPartNumber: string, // Azure 1.0-preview(v1)+\r\n x_SkuPlanName: string, // Azure 1.2-preview+\r\n x_SkuRegion: string, // Azure 1.0-preview(v1)+\r\n x_SkuServiceFamily: string, // Azure 1.0-preview(v1)+\r\n x_SkuTerm: int, // Azure 1.0-preview(v1)+\r\n x_SkuTier: string, // Azure 1.0-preview(v1)+\r\n x_SourceChanges: string, // Hubs add-on\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceValues: dynamic, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_SubproductName: string, // Tencent 1.0\r\n x_TotalDiscountPercent: real, // Hubs add-on\r\n x_TotalSavings: real, // Hubs add-on\r\n x_UsageType: string // AWS 1.0\r\n)\r\n\r\n// Update policy for Costs_raw -> Costs_final_v1_2 table\r\n.alter table Costs_final_v1_2 policy update\r\n```\r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"Costs_raw\",\r\n \"Query\": \"Costs_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Actual costs |===================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// ActualCosts_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='ActualCost exports transformed to FOCUS 1.2.', folder='Costs')\r\nActualCosts_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n ActualCosts_raw\r\n | where ChargeType in ('Purchase', 'Refund') and isnotempty(ReservationId)\r\n //\r\n //\r\n // !!! The rest of the query is reused for both the ActualCosts and AmortizedCosts queries -- Copy all changes in both transform functions !!!\r\n //\r\n //\r\n | extend x_AmortizationClass = case(\r\n ChargeType in ('Purchase', 'Refund') and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', 'Amortized Charge',\r\n ''\r\n )\r\n | extend tmp_ResourceInfo = parse_resourceid(ResourceId)\r\n // TODO: PricingCategory needs to include savings plan usage and spot usage\r\n | extend PricingCategory = case(\r\n x_AmortizationClass == 'Amortized Charge', 'Committed',\r\n ChargeType in ('Usage', 'Purchase'), 'Standard',\r\n ''\r\n )\r\n | extend x_ResourceType = tostring(tmp_ResourceInfo.x_ResourceType)\r\n | project-rename\r\n PricingQuantity = Quantity,\r\n x_PricingUnitDescription = UnitOfMeasure\r\n | join kind=leftouter (PricingUnits) on x_PricingUnitDescription\r\n | join kind=leftouter (Regions) on ResourceLocation\r\n | join kind=leftouter (ResourceTypes | project x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n // TODO: Add the following in 1.2: PublisherName, x_PublisherCategory, x_Environment\r\n | join kind=leftouter (Services | where isnotempty(x_ResourceType) | project x_ResourceType, ServiceName, ServiceCategory, ServiceSubcategory, x_ServiceModel) on x_ResourceType\r\n | join kind=leftouter (Services | where isnotempty(x_ConsumedService) | summarize take_any(ServiceName), take_any(ServiceCategory), take_any(ServiceSubcategory), take_any(x_ServiceModel) by ConsumedService = x_ConsumedService) on ConsumedService\r\n | extend CommitmentDiscountCategory = iff(isnotempty(ReservationId), 'Usage', '') // TODO: CommitmentDiscountCategory needs to handle savings plans\r\n | project\r\n AvailabilityZone = AvailabilityZone,\r\n BilledCost = iff(x_AmortizationClass == 'Amortized Charge', real(0), Cost),\r\n BillingAccountId = strcat('/providers/microsoft.billing/billingaccounts/', BillingAccountId, iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), '', strcat('/billingprofiles/', BillingProfileId))),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName),\r\n BillingAccountType = iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), 'Billing Profile', 'Billing Account'),\r\n BillingCurrency,\r\n BillingPeriodEnd = startofmonth(BillingPeriodEndDate, 1),\r\n BillingPeriodStart = startofmonth(BillingPeriodStartDate),\r\n CapacityReservationId = '',\r\n CapacityReservationStatus = '',\r\n ChargeCategory = case(\r\n ChargeType in ('Usage', 'Purchase', 'Credit', 'Tax'), ChargeType,\r\n ChargeType in ('UnusedReservation', 'UnusedSavingsPlan'), 'Usage',\r\n ChargeType == 'Refund', 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = iff(ChargeType == 'Refund', 'Correction', ''),\r\n ChargeDescription = Product,\r\n ChargeFrequency = case(\r\n Frequency == 'UsageBased', 'Usage-Based',\r\n Frequency == 'OneTime', 'One-Time',\r\n Frequency // \"Recurring\" and any fallback\r\n ),\r\n ChargePeriodEnd = Date + 1d,\r\n ChargePeriodStart = Date,\r\n ChargeSubcategory = '',\r\n // TODO: CommitmentDiscount* columns need to handle savings plans\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId = iff(isnotempty(ReservationId), strcat('/providers/microsoft.capacity/reservationorders/', ProductOrderId, '/reservations/', ReservationId), ''),\r\n CommitmentDiscountName = ReservationName,\r\n CommitmentDiscountQuantity = real(null),\r\n CommitmentDiscountStatus = case(\r\n isempty(ReservationId), '',\r\n isnotempty(ReservationId) and ChargeType == 'Usage', 'Used',\r\n ChargeType startswith 'Unused', 'Unused',\r\n 'Unused'\r\n ),\r\n CommitmentDiscountType = iff(isnotempty(ReservationId), 'Reservation', ''),\r\n CommitmentDiscountUnit = '',\r\n ConsumedQuantity = PricingQuantity * x_PricingBlockSize,\r\n ConsumedUnit = PricingUnit,\r\n ContractedCost = UnitPrice * PricingQuantity,\r\n ContractedUnitPrice = UnitPrice,\r\n EffectiveCost = iff(x_AmortizationClass == 'Principal', real(0), Cost),\r\n InvoiceId = '',\r\n InvoiceIssuerName = 'Microsoft',\r\n ListCost = real(null),\r\n ListUnitPrice = real(null),\r\n PricingCategory,\r\n PricingCurrency = iff(BillingAccountId == BillingProfileId, BillingCurrency, ''),\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName = 'Microsoft',\r\n PublisherName = iff(PublisherType == 'Marketplace', PublisherName, 'Microsoft'),\r\n Region = '',\r\n RegionId,\r\n RegionName,\r\n ResourceId = tostring(tmp_ResourceInfo.ResourceId),\r\n ResourceName = tostring(tmp_ResourceInfo.ResourceName),\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory,\r\n SkuId = '',\r\n SkuMeter = MeterName,\r\n SkuPriceDetails = '',\r\n SkuPriceId = '',\r\n SubAccountId = strcat('/subscriptions/', SubscriptionId),\r\n SubAccountName = iff(isempty(SubscriptionId), '', SubscriptionName),\r\n SubAccountType = iff(isempty(SubscriptionId), '', 'Subscription'),\r\n Tags = iff(Tags startswith '{', Tags, strcat('{', Tags, '}')),\r\n UsageAmount = real(null),\r\n UsageQuantity = real(null),\r\n UsageUnit = '',\r\n x_AccountId = '',\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerId,\r\n x_AmortizationClass = '',\r\n x_BilledCostInUsd = real(null),\r\n x_BilledUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), UnitPrice),\r\n x_BillingAccountId = BillingAccountId,\r\n x_BillingAccountName = BillingAccountName,\r\n x_BillingExchangeRate = real(null),\r\n x_BillingExchangeRateDate = datetime(null),\r\n x_BillingItemCode = '',\r\n x_BillingItemName = '',\r\n x_BillingProfileId = BillingProfileId,\r\n x_BillingProfileName = BillingProfileName,\r\n x_ChargeId = '',\r\n x_CommodityCode = '',\r\n x_CommodityName = '',\r\n x_ComponentType = '',\r\n x_ComponentName = '',\r\n x_ContractedCostInUsd = real(null),\r\n x_Cost = real(null),\r\n x_CostAllocationRuleName = '',\r\n x_CostCategories = '',\r\n x_CostCenter = CostCenter,\r\n x_CostType = '',\r\n x_Credits = '',\r\n x_CurrencyConversionRate = real(null),\r\n x_CustomerId = '',\r\n x_CustomerName = '',\r\n x_Discount = '',\r\n x_EffectiveCostInUsd = real(null),\r\n x_EffectiveUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), EffectivePrice),\r\n x_ExportTime = datetime(null),\r\n x_InstanceID = '',\r\n x_InvoiceId = '',\r\n x_InvoiceIssuerId = '',\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = InvoiceSection,\r\n x_ListCostInUsd = real(null),\r\n x_Location = '',\r\n x_OnDemandCost = real(null),\r\n x_OnDemandCostInUsd = real(null),\r\n x_OnDemandUnitPrice = real(null),\r\n x_Operation = '',\r\n x_OwnerAccountID = '',\r\n x_PartnerCreditApplied = '',\r\n x_PartnerCreditRate = '',\r\n x_PricingBlockSize,\r\n x_PricingCurrency = '',\r\n x_PricingSubcategory = case(\r\n // TODO: Add x_SkuTier when supported by C360 -- PricingCategory == 'Standard' and isnotempty(x_SkuTier), 'Tiered',\r\n PricingCategory == 'Standard', 'Standard',\r\n PricingCategory == 'Committed', strcat('Committed ', CommitmentDiscountCategory),\r\n PricingCategory == 'Dynamic', 'Spot',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_Project = '',\r\n x_PublisherCategory = iff(PublisherType == 'Marketplace', 'Vendor', 'Cloud Provider'),\r\n x_PublisherId = '',\r\n x_ResellerId = '',\r\n x_ResellerName = '',\r\n x_ResourceGroupName = ResourceGroup,\r\n x_ResourceType,\r\n x_ServiceCode = '',\r\n x_ServiceId = '',\r\n x_ServiceModel,\r\n x_ServicePeriodEnd = datetime(null),\r\n x_ServicePeriodStart = datetime(null),\r\n x_SkuDescription = Product,\r\n x_SkuDetails = iff(AdditionalInfo startswith '{', AdditionalInfo, strcat('{', AdditionalInfo, '}')),\r\n x_SkuIsCreditEligible = IsAzureCreditEligible,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterId = MeterId,\r\n x_SkuMeterName = MeterName,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuOfferId = OfferId,\r\n x_SkuOrderId = ProductOrderId,\r\n x_SkuOrderName = ProductOrderName,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPlanName = '',\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTerm = toint(Term),\r\n x_SkuTier = '',\r\n x_SourceName = 'C360',\r\n x_SourceProvider = 'Microsoft',\r\n x_SourceType = 'ActualCost',\r\n x_SourceVersion = 'C360-2025-04',\r\n x_SubproductName = '',\r\n x_UsageType = ''\r\n}\r\n\r\n// Update policy for ActualCosts_raw -> Costs_raw table\r\n.alter table Costs_raw policy update\r\n``` \r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"ActualCosts_raw\",\r\n \"Query\": \"ActualCosts_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Amortized costs |================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// AmortizedCosts_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='ActualCost exports transformed to FOCUS 1.2.', folder='Costs')\r\nAmortizedCosts_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n AmortizedCosts_raw\r\n //\r\n //\r\n // !!! The rest of the query is reused for both the ActualCosts and AmortizedCosts queries -- Copy all changes in both transform functions !!!\r\n //\r\n //\r\n | extend x_AmortizationClass = case(\r\n ChargeType in ('Purchase', 'Refund') and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', 'Amortized Charge',\r\n ''\r\n )\r\n | extend tmp_ResourceInfo = parse_resourceid(ResourceId)\r\n // TODO: PricingCategory needs to include savings plan usage and spot usage\r\n | extend PricingCategory = case(\r\n x_AmortizationClass == 'Amortized Charge', 'Committed',\r\n ChargeType in ('Usage', 'Purchase'), 'Standard',\r\n ''\r\n )\r\n | extend x_ResourceType = tostring(tmp_ResourceInfo.x_ResourceType)\r\n | project-rename\r\n PricingQuantity = Quantity,\r\n x_PricingUnitDescription = UnitOfMeasure\r\n | join kind=leftouter (PricingUnits) on x_PricingUnitDescription\r\n | join kind=leftouter (Regions) on ResourceLocation\r\n | join kind=leftouter (ResourceTypes | project x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n // TODO: Add the following in 1.2: PublisherName, x_PublisherCategory, x_Environment\r\n | join kind=leftouter (Services | where isnotempty(x_ResourceType) | project x_ResourceType, ServiceName, ServiceCategory, ServiceSubcategory, x_ServiceModel) on x_ResourceType\r\n | join kind=leftouter (Services | where isnotempty(x_ConsumedService) | summarize take_any(ServiceName), take_any(ServiceCategory), take_any(ServiceSubcategory), take_any(x_ServiceModel) by ConsumedService = x_ConsumedService) on ConsumedService\r\n | extend CommitmentDiscountCategory = iff(isnotempty(ReservationId), 'Usage', '') // TODO: CommitmentDiscountCategory needs to handle savings plans\r\n | project\r\n AvailabilityZone = AvailabilityZone,\r\n BilledCost = iff(x_AmortizationClass == 'Amortized Charge', real(0), Cost),\r\n BillingAccountId = strcat('/providers/microsoft.billing/billingaccounts/', BillingAccountId, iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), '', strcat('/billingprofiles/', BillingProfileId))),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName),\r\n BillingAccountType = iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), 'Billing Profile', 'Billing Account'),\r\n BillingCurrency,\r\n BillingPeriodEnd = startofmonth(BillingPeriodEndDate, 1),\r\n BillingPeriodStart = startofmonth(BillingPeriodStartDate),\r\n CapacityReservationId = '',\r\n CapacityReservationStatus = '',\r\n ChargeCategory = case(\r\n ChargeType in ('Usage', 'Purchase', 'Credit', 'Tax'), ChargeType,\r\n ChargeType in ('UnusedReservation', 'UnusedSavingsPlan'), 'Usage',\r\n ChargeType == 'Refund', 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = iff(ChargeType == 'Refund', 'Correction', ''),\r\n ChargeDescription = Product,\r\n ChargeFrequency = case(\r\n Frequency == 'UsageBased', 'Usage-Based',\r\n Frequency == 'OneTime', 'One-Time',\r\n Frequency // \"Recurring\" and any fallback\r\n ),\r\n ChargePeriodEnd = Date + 1d,\r\n ChargePeriodStart = Date,\r\n ChargeSubcategory = '',\r\n // TODO: CommitmentDiscount* columns need to handle savings plans\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId = iff(isnotempty(ReservationId), strcat('/providers/microsoft.capacity/reservationorders/', ProductOrderId, '/reservations/', ReservationId), ''),\r\n CommitmentDiscountName = ReservationName,\r\n CommitmentDiscountQuantity = real(null),\r\n CommitmentDiscountStatus = case(\r\n isempty(ReservationId), '',\r\n isnotempty(ReservationId) and ChargeType == 'Usage', 'Used',\r\n ChargeType startswith 'Unused', 'Unused',\r\n 'Unused'\r\n ),\r\n CommitmentDiscountType = iff(isnotempty(ReservationId), 'Reservation', ''),\r\n CommitmentDiscountUnit = '',\r\n ConsumedQuantity = PricingQuantity * x_PricingBlockSize,\r\n ConsumedUnit = PricingUnit,\r\n ContractedCost = UnitPrice * PricingQuantity,\r\n ContractedUnitPrice = UnitPrice,\r\n EffectiveCost = iff(x_AmortizationClass == 'Principal', real(0), Cost),\r\n InvoiceId = '',\r\n InvoiceIssuerName = 'Microsoft',\r\n ListCost = real(null),\r\n ListUnitPrice = real(null),\r\n PricingCategory,\r\n PricingCurrency = iff(BillingAccountId == BillingProfileId, BillingCurrency, ''),\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName = 'Microsoft',\r\n PublisherName = iff(PublisherType == 'Marketplace', PublisherName, 'Microsoft'),\r\n Region = '',\r\n RegionId,\r\n RegionName,\r\n ResourceId = tostring(tmp_ResourceInfo.ResourceId),\r\n ResourceName = tostring(tmp_ResourceInfo.ResourceName),\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory,\r\n SkuId = '',\r\n SkuMeter = MeterName,\r\n SkuPriceDetails = '',\r\n SkuPriceId = '',\r\n SubAccountId = strcat('/subscriptions/', SubscriptionId),\r\n SubAccountName = iff(isempty(SubscriptionId), '', SubscriptionName),\r\n SubAccountType = iff(isempty(SubscriptionId), '', 'Subscription'),\r\n Tags = iff(Tags startswith '{', Tags, strcat('{', Tags, '}')),\r\n UsageAmount = real(null),\r\n UsageQuantity = real(null),\r\n UsageUnit = '',\r\n x_AccountId = '',\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerId,\r\n x_AmortizationClass = '',\r\n x_BilledCostInUsd = real(null),\r\n x_BilledUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), UnitPrice),\r\n x_BillingAccountId = BillingAccountId,\r\n x_BillingAccountName = BillingAccountName,\r\n x_BillingExchangeRate = real(null),\r\n x_BillingExchangeRateDate = datetime(null),\r\n x_BillingItemCode = '',\r\n x_BillingItemName = '',\r\n x_BillingProfileId = BillingProfileId,\r\n x_BillingProfileName = BillingProfileName,\r\n x_ChargeId = '',\r\n x_CommodityCode = '',\r\n x_CommodityName = '',\r\n x_ComponentType = '',\r\n x_ComponentName = '',\r\n x_ContractedCostInUsd = real(null),\r\n x_Cost = real(null),\r\n x_CostAllocationRuleName = '',\r\n x_CostCategories = '',\r\n x_CostCenter = CostCenter,\r\n x_CostType = '',\r\n x_Credits = '',\r\n x_CurrencyConversionRate = real(null),\r\n x_CustomerId = '',\r\n x_CustomerName = '',\r\n x_Discount = '',\r\n x_EffectiveCostInUsd = real(null),\r\n x_EffectiveUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), EffectivePrice),\r\n x_ExportTime = datetime(null),\r\n x_InstanceID = '',\r\n x_InvoiceId = '',\r\n x_InvoiceIssuerId = '',\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = InvoiceSection,\r\n x_ListCostInUsd = real(null),\r\n x_Location = '',\r\n x_OnDemandCost = real(null),\r\n x_OnDemandCostInUsd = real(null),\r\n x_OnDemandUnitPrice = real(null),\r\n x_Operation = '',\r\n x_OwnerAccountID = '',\r\n x_PartnerCreditApplied = '',\r\n x_PartnerCreditRate = '',\r\n x_PricingBlockSize,\r\n x_PricingCurrency = '',\r\n x_PricingSubcategory = case(\r\n // TODO: Add x_SkuTier when supported by C360 -- PricingCategory == 'Standard' and isnotempty(x_SkuTier), 'Tiered',\r\n PricingCategory == 'Standard', 'Standard',\r\n PricingCategory == 'Committed', strcat('Committed ', CommitmentDiscountCategory),\r\n PricingCategory == 'Dynamic', 'Spot',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_Project = '',\r\n x_PublisherCategory = iff(PublisherType == 'Marketplace', 'Vendor', 'Cloud Provider'),\r\n x_PublisherId = '',\r\n x_ResellerId = '',\r\n x_ResellerName = '',\r\n x_ResourceGroupName = ResourceGroup,\r\n x_ResourceType,\r\n x_ServiceCode = '',\r\n x_ServiceId = '',\r\n x_ServiceModel,\r\n x_ServicePeriodEnd = datetime(null),\r\n x_ServicePeriodStart = datetime(null),\r\n x_SkuDescription = Product,\r\n x_SkuDetails = iff(AdditionalInfo startswith '{', AdditionalInfo, strcat('{', AdditionalInfo, '}')),\r\n x_SkuIsCreditEligible = IsAzureCreditEligible,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterId = MeterId,\r\n x_SkuMeterName = MeterName,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuOfferId = OfferId,\r\n x_SkuOrderId = ProductOrderId,\r\n x_SkuOrderName = ProductOrderName,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPlanName = '',\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTerm = toint(Term),\r\n x_SkuTier = '',\r\n x_SourceName = 'C360',\r\n x_SourceProvider = 'Microsoft',\r\n x_SourceType = 'ActualCost',\r\n x_SourceVersion = 'C360-2025-04',\r\n x_SubproductName = '',\r\n x_UsageType = ''\r\n}\r\n\r\n// Update policy for AmortizedCosts_raw -> Costs_raw table\r\n.alter table Costs_raw policy update\r\n``` \r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"AmortizedCosts_raw\",\r\n \"Query\": \"AmortizedCosts_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| CommitmentDiscountUsage |========================================================================================\r\n// Supported versions:\r\n// - MS EA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-ea\r\n// - MS MCA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-mca\r\n//======================================================================================================================\r\n\r\n// CommitmentDiscountUsage_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='All commitment discount usage transformed to FOCUS 1.2. This includes reservationdeatils_raw.', folder='Commitment discounts')\r\nCommitmentDiscountUsage_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n CommitmentDiscountUsage_raw\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Handle resource columns\r\n | extend ResourceId = tolower(InstanceId)\r\n | extend tmp_ResourceDetails = parse_resourceid(ResourceId)\r\n | extend ResourceName = tostring(tmp_ResourceDetails.ResourceName)\r\n | extend SubAccountId = tostring(tmp_ResourceDetails.SubAccountId)\r\n | extend x_ResourceGroupName = tostring(tmp_ResourceDetails.x_ResourceGroupName)\r\n | extend x_ResourceType = tostring(tmp_ResourceDetails.x_ResourceType)\r\n | lookup kind=leftouter (ResourceTypes | distinct x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n | lookup kind=leftouter (Services | distinct x_ResourceType, ServiceName, ServiceCategory, ServiceSubcategory, x_ServiceModel) on x_ResourceType\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n ChargePeriodEnd = UsageDate + 1d,\r\n ChargePeriodStart = UsageDate,\r\n CommitmentDiscountCategory = 'Usage',\r\n CommitmentDiscountId = tolower(strcat('/providers/microsoft.capacity/reservationorders/', ReservationOrderId, '/reservations/', ReservationId)),\r\n CommitmentDiscountQuantity = UsedHours * InstanceFlexibilityRatio,\r\n CommitmentDiscountType = 'Reservation',\r\n CommitmentDiscountUnit = case(\r\n InstanceFlexibilityRatio == 1, 'Hours',\r\n InstanceFlexibilityRatio != 1, 'Normalized Hours',\r\n ''\r\n ),\r\n ConsumedQuantity = UsedHours,\r\n ProviderName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory,\r\n SubAccountId,\r\n x_CommitmentDiscountCommittedCount = TotalReservedQuantity,\r\n x_CommitmentDiscountCommittedAmount = ReservedHours,\r\n // TODO: Is this needed? -- x_CommitmentDiscountKind = Kind,\r\n x_CommitmentDiscountNormalizedGroup = iff(InstanceFlexibilityGroup == 'NA', '', InstanceFlexibilityGroup),\r\n x_CommitmentDiscountNormalizedRatio = InstanceFlexibilityRatio,\r\n x_IngestionTime = ingestion_time(),\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n // x_RowId = hash_sha256(strcat(\r\n // // DO NOT CHANGE COLUMNS OR COLUMN ORDER\r\n // CommitmentDiscountId,\r\n // ResourceId,\r\n // ChargePeriodStart\r\n // )),\r\n x_ServiceModel,\r\n x_SkuOrderId = ReservationOrderId,\r\n x_SkuSize = iff(SkuName == 'NA', '', SkuName),\r\n x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName)),\r\n x_SourceProvider = coalesce(x_SourceProvider, ProviderName),\r\n x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationDetails', '')),\r\n x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2024-03-01', ''))\r\n}\r\n\r\n// CommitmentDiscountUsage_final_v1_2 table\r\n.create-merge table CommitmentDiscountUsage_final_v1_2 (\r\n ChargePeriodEnd: datetime, // Hubs add-on\r\n ChargePeriodStart: datetime, // MS 2023-03-01\r\n CommitmentDiscountCategory: string, // Hubs add-on\r\n CommitmentDiscountId: string, // MS 2023-03-01\r\n CommitmentDiscountQuantity: real, // MS 2023-03-01\r\n CommitmentDiscountType: string, // Hubs add-on\r\n CommitmentDiscountUnit: string, // Hubs add-on\r\n ConsumedQuantity: real, // MS 2023-03-01\r\n ProviderName: string, // Hubs add-on\r\n ResourceId: string, // MS 2023-03-01\r\n ResourceName: string, // Hubs add-on\r\n ResourceType: string, // Hubs add-on\r\n ServiceCategory: string, // Hubs add-on\r\n ServiceName: string, // Hubs add-on\r\n ServiceSubcategory: string, // Hubs add-on\r\n SubAccountId: string, // Hubs add-on\r\n x_CommitmentDiscountCommittedCount: real, // MS 2023-03-01\r\n x_CommitmentDiscountCommittedAmount: real, // MS 2023-03-01\r\n x_CommitmentDiscountNormalizedGroup: string, // MS 2023-03-01\r\n x_CommitmentDiscountNormalizedRatio: real, // MS 2023-03-01\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_ResourceGroupName: string, // Hubs add-on\r\n x_ResourceType: string, // Hubs add-on\r\n x_ServiceModel: string, // Hubs add-on\r\n x_SkuOrderId: string, // MS 2023-03-01\r\n x_SkuSize: string, // MS 2023-03-01\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string // Hubs add-on\r\n)\r\n\r\n// Update policy for CommitmentDiscountUsage_raw -> CommitmentDiscountUsage_final_v1_2 table\r\n.alter table CommitmentDiscountUsage_final_v1_2 policy update\r\n```\r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"CommitmentDiscountUsage_raw\",\r\n \"Query\": \"CommitmentDiscountUsage_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Recommendations |================================================================================================\r\n// Supported datasets/versions:\r\n// - MS CM EA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-ea\r\n// - MS CM MCA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-mca\r\n//======================================================================================================================\r\n\r\n// Recommendations_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='All recommendations transformed to FOCUS 1.2.', folder='Recommendations')\r\nRecommendations_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', int(null),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n Recommendations_raw\r\n | extend x_IngestionTime = ingestion_time()\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Set source columns\r\n | extend x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationRecommendations', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2023-05-01', ''))\r\n //\r\n // Convert JSON cost columns to real\r\n | extend CostWithNoReservedInstances = case(isnotempty(CostWithNoReservedInstances), CostWithNoReservedInstances, isnotempty(CostWithNoReservedInstancesJson), toreal(extract(@'\"value\":([0-9\\.]+)', 1, CostWithNoReservedInstancesJson)), CostWithNoReservedInstances)\r\n | extend NetSavings = case(isnotempty(NetSavings), NetSavings, isnotempty(NetSavingsJson), toreal(extract(@'\"value\":([0-9\\.]+)', 1, NetSavingsJson)), NetSavings)\r\n | extend TotalCostWithReservedInstances = case(isnotempty(TotalCostWithReservedInstances), TotalCostWithReservedInstances, isnotempty(TotalCostWithReservedInstancesJson), toreal(extract(@'\"value\":([0-9\\.]+)', 1, TotalCostWithReservedInstancesJson)), TotalCostWithReservedInstances)\r\n //\r\n // Build recommendation details\r\n | lookup kind=leftouter (Regions | summarize RegionName = make_set(RegionName)[0] by Location = RegionId) on Location\r\n | extend x_RecommendationDetails = case(\r\n // Use incoming x_RecommendationDetails first\r\n isnotempty(x_RecommendationDetails), x_RecommendationDetails,\r\n // Create one for reservation recommendations if needed\r\n x_SourceType == 'ReservationRecommendations', bag_pack(\r\n 'CommitmentDiscountNormalizedGroup', InstanceFlexibilityGroup,\r\n 'CommitmentDiscountNormalizedRatio', InstanceFlexibilityRatio,\r\n 'CommitmentDiscountNormalizedSize', NormalizedSize,\r\n 'CommitmentDiscountResourceType', ResourceType,\r\n 'CommitmentDiscountScope', Scope,\r\n 'LookbackPeriodDuration', case(\r\n LookBackPeriod matches regex @'^Last([0-9]+)Days$', replace_regex(LookBackPeriod, @'^Last([0-9]+)Days$', @'P\\1D'),\r\n LookBackPeriod matches regex @'^[0-9]+$', strcat('P', LookBackPeriod, 'D'),\r\n ''\r\n ),\r\n 'LookbackPeriodStart', FirstUsageDate,\r\n 'RecommendedQuantity', RecommendedQuantity,\r\n 'RecommendedQuantityNormalized', RecommendedQuantityNormalized,\r\n 'RegionId', Location,\r\n 'RegionName', RegionName,\r\n 'SkuMeterId', MeterId,\r\n 'SkuPriceDetails', SkuProperties,\r\n 'SkuSize', coalesce(SKU, SkuName),\r\n 'SkuTerm', isoMonths(Term)\r\n ),\r\n dynamic({})\r\n )\r\n //\r\n // Prefer specified date, then fall back to generating a date based on reservation recommendation lookback period, then validate to ensure it's not in the future\r\n | extend x_RecommendationDate = coalesce(x_RecommendationDate, FirstUsageDate + (toint(extract(@'^P([0-9]+)D$', 1, tostring(x_RecommendationDetails.LookbackPeriodDuration))) * 1d))\r\n | extend x_RecommendationDate = iff(x_RecommendationDate > now(), startofday(now()), x_RecommendationDate)\r\n //\r\n | project\r\n ProviderName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n SubAccountId = coalesce(SubAccountId, iff(isnotempty(SubscriptionId), strcat('/subscriptions/', SubscriptionId), '')),\r\n SubAccountName,\r\n x_EffectiveCostAfter = coalesce(x_EffectiveCostAfter, TotalCostWithReservedInstances),\r\n x_EffectiveCostBefore = coalesce(x_EffectiveCostBefore, CostWithNoReservedInstances),\r\n x_EffectiveCostSavings = coalesce(x_EffectiveCostSavings, NetSavings),\r\n x_IngestionTime,\r\n x_RecommendationCategory, // TODO: Set for reservation recommendations\r\n x_RecommendationDate,\r\n x_RecommendationDescription,\r\n x_RecommendationDetails,\r\n x_RecommendationId, // TODO: Set for reservation recommendations\r\n x_ResourceGroupName,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n// Recommendations_final_v1_2 table\r\n.create-merge table Recommendations_final_v1_2 (\r\n ProviderName: string,\r\n ResourceId: string,\r\n ResourceName: string,\r\n ResourceType: string,\r\n SubAccountId: string,\r\n SubAccountName: string,\r\n x_EffectiveCostAfter: real,\r\n x_EffectiveCostBefore: real,\r\n x_EffectiveCostSavings: real,\r\n x_IngestionTime: datetime,\r\n x_RecommendationCategory: string,\r\n x_RecommendationDate: datetime,\r\n x_RecommendationDescription: string,\r\n x_RecommendationDetails: dynamic,\r\n x_RecommendationId: string,\r\n x_ResourceGroupName: string,\r\n x_SourceName: string,\r\n x_SourceProvider: string,\r\n x_SourceType: string,\r\n x_SourceVersion: string\r\n)\r\n\r\n// Update policy for Recommendations_raw -> Recommendations_final_v1_2 table\r\n.alter table Recommendations_final_v1_2 policy update\r\n```\r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"Recommendations_raw\",\r\n \"Query\": \"Recommendations_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Transactions |===================================================================================================\r\n// Supported versions:\r\n// - MS CM EA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-ea\r\n// - MS CM MCA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-mca\r\n//======================================================================================================================\r\n\r\n// Transactions_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='All transactions transformed to FOCUS 1.2.', folder='Transactions')\r\nTransactions_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', int(null),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n Transactions_raw\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Set source columns\r\n | extend x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationTransactions', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2023-05-01', ''))\r\n //\r\n // Handle BillingPeriodStart/End\r\n | extend BillingMonth = tostring(BillingMonth)\r\n | extend BillingPeriodStart = iff(isempty(BillingMonth), datetime(null), todatetime(strcat(substring(BillingMonth, 0, 4), \"-\", substring(BillingMonth, 4, 2), \"-\", substring(BillingMonth, 6, 2))))\r\n | extend BillingPeriodEnd = iff(isempty(BillingMonth), datetime(null), startofmonth(endofmonth(BillingPeriodStart) + 1d))\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n BilledCost = Amount,\r\n BillingAccountId = case(\r\n BillingProfileId startswith '/', BillingProfileId,\r\n isnotempty(CurrentEnrollmentId), strcat('/providers/Microsoft.Billing/billingAccounts/', CurrentEnrollmentId),\r\n isnotempty(BillingProfileId), strcat('/providers/Microsoft.Billing/billingProfiles/', BillingProfileId),\r\n ''\r\n ),\r\n BillingAccountName = coalesce(BillingProfileName, CurrentEnrollmentId),\r\n BillingCurrency = Currency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n ChargeCategory = case(\r\n EventType in ('Cancel', 'Purchase', 'Refund'), 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = case(\r\n EventType == 'Cancel', 'Cancel', // FOCUS does not handle this scenario\r\n EventType == 'Refund', 'Correction',\r\n ''\r\n ),\r\n ChargeDescription = Description,\r\n ChargeFrequency = case(\r\n BillingFrequency == 'OneTime', 'One-Time',\r\n BillingFrequency == 'Recurring', 'Recurring',\r\n BillingFrequency\r\n ),\r\n ChargePeriodStart = EventDate,\r\n InvoiceId,\r\n PricingQuantity = Quantity,\r\n PricingUnit = 'Reservations',\r\n ProviderName,\r\n RegionId = Region,\r\n RegionName = Region,\r\n SubAccountId = iff(isempty(PurchasingSubscriptionGuid), '', strcat('/subscriptions/', PurchasingSubscriptionGuid)),\r\n SubAccountName = iff(isempty(PurchasingSubscriptionGuid), '', PurchasingSubscriptionName),\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerEmail,\r\n x_CostCenter = CostCenter,\r\n x_InvoiceNumber = Invoice,\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = coalesce(InvoiceSectionName, DepartmentName),\r\n x_IngestionTime = ingestion_time(),\r\n x_MonetaryCommitment = MonetaryCommitment,\r\n x_Overage = Overage,\r\n x_PurchasingBillingAccountId = PurchasingEnrollment,\r\n x_SkuOrderId = ReservationOrderId,\r\n x_SkuOrderName = ReservationOrderName,\r\n x_SkuSize = ArmSkuName,\r\n x_SkuTerm = isoMonths(Term),\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_SubscriptionId = PurchasingSubscriptionGuid,\r\n x_TransactionType = EventType\r\n}\r\n\r\n// Transactions_final_v1_2 table\r\n.create-merge table Transactions_final_v1_2 (\r\n BilledCost: real, // MS CM EA+MCA 2023-05-01\r\n BillingAccountId: string, // MS CM EA+MCA 2023-05-01\r\n BillingAccountName: string, // MS CM EA+MCA 2023-05-01\r\n BillingCurrency: string, // MS CM EA+MCA 2023-05-01\r\n BillingPeriodEnd: datetime, // MS CM EA+MCA 2023-05-01\r\n BillingPeriodStart: datetime, // MS CM EA+MCA 2023-05-01\r\n ChargeCategory: string, // Hubs add-on\r\n ChargeClass: string, // Hubs add-on\r\n ChargeDescription: string, // MS CM EA+MCA 2023-05-01\r\n ChargeFrequency: string, // MS CM EA+MCA 2023-05-01\r\n ChargePeriodStart: datetime, // MS CM EA+MCA 2023-05-01\r\n InvoiceId: string, // MS CM MCA 2023-05-01\r\n PricingQuantity: real, // MS CM EA+MCA 2023-05-01\r\n PricingUnit: string, // Hubs add-on\r\n ProviderName: string, // Hubs add-on\r\n RegionId: string, // MS CM EA+MCA 2023-05-01\r\n RegionName: string, // MS CM EA+MCA 2023-05-01\r\n SubAccountId: string, // MS CM EA+MCA 2023-05-01\r\n SubAccountName: string, // MS CM EA+MCA 2023-05-01\r\n x_AccountName: string, // MS CM EA 2023-05-01\r\n x_AccountOwnerId: string, // MS CM EA 2023-05-01\r\n x_CostCenter: string, // MS CM EA 2023-05-01\r\n x_InvoiceNumber: string, // MS CM MCA 2023-05-01\r\n x_InvoiceSectionId: string, // MS CM MCA 2023-05-01\r\n x_InvoiceSectionName: string, // MS CM MCA 2023-05-01\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_MonetaryCommitment: real, // MS CM EA 2023-05-01\r\n x_Overage: real, // MS CM EA 2023-05-01\r\n x_PurchasingBillingAccountId: string, // MS CM EA 2023-05-01\r\n x_SkuOrderId: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuOrderName: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuSize: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuTerm: int, // MS CM EA+MCA 2023-05-01\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_SubscriptionId: string, // MS CM EA+MCA 2023-05-01\r\n x_TransactionType: string // MS CM EA+MCA 2023-05-01\r\n)\r\n\r\n// Update policy for Transactions_raw -> Transactions_final_v1_2 table\r\n.alter table Transactions_final_v1_2 policy update\r\n```\r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"Transactions_raw\",\r\n \"Query\": \"Transactions_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n", + "$fxv#11": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Common utility functions\r\n//\r\n// TIP: Use Ctrl+K,Ctrl+0 to collapse all regions in VS Code\r\n//======================================================================================================================\r\n\r\n\r\n//===| Date functions |=================================================================================================\r\n\r\n// monthstring\r\n.create-or-alter function \r\nwith (docstring = @'Returns the name of the month for the specified date (e.g. Jan or January)', folder =@'Common') \r\nmonthstring(['date']: datetime, length: int = 9)\r\n{\r\n substring(dynamic(['January','February','March','April','May','June','July','August','September','October','November','December'])[getmonth(['date']) - 1], 0, length)\r\n}\r\n\r\n// datestring\r\n.create-or-alter function \r\nwith (docstring = @'Converts 2 dates into a simple, user-friendly date range (e.g. Jan 1-Jan 3)', folder =@'Common') \r\ndatestring(start: datetime, end: datetime = datetime('0001-01-01'))\r\n{\r\n let month = (d: datetime) { monthstring(d, 3) };\r\n let endDate = iff(end == datetime('0001-01-01'), start, end);\r\n let sameDate = startofday(start) == startofday(endDate);\r\n let sameMonth = startofmonth(start) == startofmonth(endDate);\r\n let sameYear = startofyear(start) == startofyear(endDate);\r\n let fullMonth = startofday(start) == startofmonth(start) and startofday(endDate) == startofday(endofmonth(endDate));\r\n let fullYear = startofday(start) == startofyear(start) and startofday(endDate) == startofday(endofyear(endDate));\r\n let currentYear = sameYear and startofyear(start) == startofyear(now());\r\n case(\r\n // Full year | yyyy (same year) / yyyy-yyyy (diff years)\r\n fullYear,\r\n strcat(getyear(start), iff(sameYear, '', strcat('-', getyear(endDate)))),\r\n // 1 full mo, same year | Mmm yyyy\r\n fullMonth and sameMonth and sameYear,\r\n strcat(month(start), ' ', getyear(start)),\r\n // 2+ full mo, same year | Mmm-Mmm (current year) / Mmm-Mmm yyyy (other year)\r\n fullMonth and sameYear,\r\n strcat(month(start), '-', month(endDate), iff(currentYear, '', strcat(' ', getyear(endDate)))),\r\n // Full mo, diff year | Mmm yyyy-Mmm yyyy\r\n fullMonth and not(sameYear),\r\n strcat(month(start), ' ', getyear(start), '-', month(endDate), ' ', getyear(endDate)),\r\n // Same date | Mmm d (current year) / Mmm d, yyyy (other year)\r\n sameDate,\r\n strcat(month(start), ' ', dayofmonth(start), iff(currentYear, '', strcat(', ', getyear(endDate)))),\r\n // 1 partial M, same Y | Mmm d-d (current year) / Mmm d-d, yyyy (other year)\r\n not(fullMonth) and sameMonth and sameYear,\r\n strcat(month(start), ' ', dayofmonth(start), '-', dayofmonth(endDate), iff(currentYear, '', strcat(' ', getyear(endDate)))),\r\n // 2+ partial M, same Y | Mmm d-Mmm d (current year) / Mmm d-Mmm d, yyyy (other year)\r\n not(fullMonth) and not(sameMonth) and sameYear,\r\n strcat(month(start), ' ', dayofmonth(start), '-', month(endDate), ' ', dayofmonth(endDate), iff(currentYear, '', strcat(', ', getyear(endDate)))),\r\n // All other cases | Mmm d, yyyy-Mmm d, yyyy\r\n strcat(month(start), ' ', dayofmonth(start), ', ', getyear(start), '-', month(endDate), ' ', dayofmonth(endDate), ', ', getyear(endDate))\r\n )\r\n}\r\n\r\n// daterange\r\n.create-or-alter function \r\nwith (docstring = @'DEPRECATED: Please use datestring(); function will be removed on or after the Jan 2026 release', folder =@'Common') \r\ndaterange(start: datetime, end: datetime = datetime('0001-01-01'))\r\n{\r\n datestring(start, end)\r\n}\r\n\r\n// monthsago\r\n.create-or-alter function \r\nwith (docstring = 'DEPRECATED: Please use startofmonth(now(), -<# of months>); function will be removed on or after the Jan 2026 release', folder = 'Common')\r\nmonthsago(months: int)\r\n{\r\n datetime_add('month', -months, startofmonth(now()))\r\n}\r\n\r\n\r\n//===| Number functions |===============================================================================================\r\n// NOTE: Must be defined before string converters\r\n\r\n// delta\r\n.create-or-alter function \r\nwith (docstring = @'Compares 2 values and returns the percentage change from oldval to newval', folder =@'Common') \r\ndelta(oldval: double, newval: double)\r\n{\r\n (newval - todouble(oldval))/oldval\r\n}\r\n\r\n// percentOfTotal\r\n// NOTE: Must be before percent() function\r\n.create-or-alter function \r\nwith (docstring = @'Calculates the percentage of each record based on a required Count column', folder =@'Common') \r\npercentOfTotal(t: (Count: long), tot: long)\r\n{\r\n let total = todouble(tot);\r\n t \r\n | extend Percent = round(Count / total * 100, 3) \r\n | order by Count desc\r\n}\r\n\r\n// percent\r\n.create-or-alter function \r\nwith (docstring = @'Calculates the percentage of each record based on a required Count column', folder =@'Common') \r\npercent(t: (Count: long))\r\n{\r\n let total = todouble(toscalar(t | summarize sum(Count)));\r\n percentOfTotal(t, total)\r\n}\r\n\r\n// plusminus\r\n.create-or-alter function \r\nwith (docstring = 'Shows a +/- sign based on the direction of the number', folder = 'Common')\r\nplusminus(val: string)\r\n{\r\n let neg = substring(val, 0, 1) == '-';\r\n iff(neg, val, strcat('+', val))\r\n}\r\n\r\n// updown\r\n.create-or-alter function \r\nwith (docstring = 'Shows an up/down arrow based on the direction of the number', folder = 'Common')\r\nupdown(val: string)\r\n{\r\n // TODO: Handle 0\r\n let neg = substring(val, 0, 1) == '-';\r\n iff(neg, strcat('↓', substring(val, 1)), strcat('↑', val))\r\n}\r\n\r\n\r\n//===| String functions |===============================================================================================\r\n\r\n// percentstring\r\n// NOTE: Must be defined before deltastring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate a percentage and render as a string', folder = 'Common')\r\npercentstring(num: double, total: double = 1.0, places: int = 9)\r\n{\r\n let value = 1.0 * num / total * 100;\r\n strcat(case(\r\n places != 9, round(value, places),\r\n value < 10, round(value, 2),\r\n round(value, 1)\r\n ), '%')\r\n}\r\n\r\n//----------------------------------------------------------------------------------------------------------------------\r\n\r\n// arraystring\r\n.create-or-alter function \r\nwith (docstring = 'Convert an array to a comma-delimited string', folder = 'Common')\r\narraystring(arr: dynamic)\r\n{\r\n replace_string(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(\r\n tostring(arr)\r\n , @'^\\[\"', '')\r\n , @'\"\\]$', '')\r\n , @'^, ', '')\r\n , @', $', '')\r\n , @'^\\[]$', '')\r\n , '\",\"', ', ')\r\n}\r\n\r\n// deltastring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate a delta percentage and render as a string', folder = 'Common')\r\ndeltastring(oldval: double, newval: double, places: int = 1, useArrows: bool = false)\r\n{\r\n let d = delta(oldval, newval);\r\n strcat(case(useArrows and d > 0, '↑', useArrows and d < 0, '↓', d < 0, '-', ''), percentstring(abs(d), 1, places))\r\n}\r\n\r\n// diffstring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate the difference and render as a string', folder = 'Common')\r\ndiffstring(oldval: double, newval: double, places: int = 1)\r\n{\r\n plusminus(round(newval - oldval, places))\r\n}\r\n\r\n// numberstring\r\n.create-or-alter function \r\nwith (docstring = 'Convert a number to a string', folder = 'Common')\r\nnumberstring(num: double, abbrev: bool = true)\r\n{\r\n replace_regex(case(\r\n num >= 10000000000000, strcat(round(1.0 * num / 1000000000000, 1), 'T'),\r\n num >= 1000000000000, strcat(round(1.0 * num / 1000000000000, 2), 'T'),\r\n num >= 10000000000, strcat(round(1.0 * num / 1000000000, 1), 'B'),\r\n num >= 1000000000, strcat(round(1.0 * num / 1000000000, 2), 'B'),\r\n num >= 10000000, strcat(round(1.0 * num / 1000000, 1), 'M'),\r\n num >= 1000000, strcat(round(1.0 * num / 1000000, 2), 'M'),\r\n num >= 10000, strcat(round(1.0 * num / 1000, 1), 'K'),\r\n // Kusto doesn't support back-refs yet -- num > 1000, replace_regex(tostring(num), @'(\\d)(?=(\\d{3})+\\.)', @'\\1,'), // See https://docs.microsoft.com/azure/data-explorer/kusto/query/re2-library\r\n num > 1000, replace_regex(tostring(num), @'([0-9]{3})$', @',\\1'), //num / 1000, ',', substring(tostring(num), 0) - (num / 1000 * 1000)),\r\n tostring(num)\r\n ), @'\\.0$', '')\r\n}\r\n\r\n\r\n//===| Other |==========================================================================================================\r\n\r\n// ifempty\r\n.create-or-alter function \r\nwith (docstring = 'Replaces an empty value with the specified default value', folder = 'Common')\r\nifempty(val: dynamic, defaultVal: dynamic)\r\n{\r\n iff(isempty(val), defaultVal, val)\r\n}\r\n", + "$fxv#12": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Hub database / Open data functions\r\n// Wrap Ingestion database tables for easy access.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n\r\n// PricingUnits\r\n.create-or-alter function\r\nwith (docstring = 'Gets pricing units from the FinOps toolkit PricingUnits open data.', folder = 'OpenData')\r\nPricingUnits()\r\n{\r\n database('Ingestion').PricingUnits\r\n}\r\n\r\n// Regions\r\n.create-or-alter function\r\nwith (docstring = 'Gets regions from the FinOps toolkit Regions open data.', folder = 'OpenData')\r\nRegion()\r\n{\r\n database('Ingestion').Regions\r\n}\r\n\r\n// ResourceTypes\r\n.create-or-alter function\r\nwith (docstring = 'Gets resource types from the FinOps toolkit ResourceTypes open data.', folder = 'OpenData')\r\nResourceType()\r\n{\r\n database('Ingestion').ResourceTypes\r\n}\r\n\r\n// Services\r\n.create-or-alter function\r\nwith (docstring = 'Gets services from the FinOps toolkit Services open data.', folder = 'OpenData')\r\nServices()\r\n{\r\n database('Ingestion').Services\r\n}\r\n", + "$fxv#13": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Hub database / FOCUS 1.0 functions\r\n// Used for reporting with backward compatibility.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n\r\n// CommitmentDiscountUsage_final_v1_0\r\n.create-or-alter function\r\nwith (docstring = 'Gets all commitment discount usage records aligned to FOCUS 1.0.', folder = 'CommitmentDiscountUsage')\r\nCommitmentDiscountUsage_v1_0()\r\n{\r\n database('Ingestion').CommitmentDiscountUsage_final_v1_0\r\n | union (\r\n database('Ingestion').CommitmentDiscountUsage_final_v1_2\r\n // Convert real to decimal\r\n | extend\r\n CommitmentDiscountQuantity = todecimal(CommitmentDiscountQuantity),\r\n ConsumedQuantity = todecimal(ConsumedQuantity),\r\n x_CommitmentDiscountCommittedCount = todecimal(x_CommitmentDiscountCommittedCount),\r\n x_CommitmentDiscountCommittedAmount = todecimal(x_CommitmentDiscountCommittedAmount),\r\n x_CommitmentDiscountNormalizedRatio = todecimal(x_CommitmentDiscountNormalizedRatio)\r\n )\r\n | project\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId,\r\n CommitmentDiscountType,\r\n ConsumedQuantity,\r\n ProviderName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n SubAccountId,\r\n x_CommitmentDiscountCommittedCount,\r\n x_CommitmentDiscountCommittedAmount,\r\n x_CommitmentDiscountNormalizedGroup,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_CommitmentDiscountQuantity,\r\n x_IngestionTime,\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n x_ServiceModel,\r\n x_SkuOrderId,\r\n x_SkuSize,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n\r\n// Costs_final_v1_0\r\n.create-or-alter function\r\nwith (docstring = 'Gets all cost and usage records aligned to FOCUS 1.0.', folder = 'Costs')\r\nCosts_v1_0()\r\n{\r\n database('Ingestion').Costs_final_v1_0\r\n | union (\r\n database('Ingestion').Costs_final_v1_2\r\n // Convert real to decimal\r\n | extend\r\n BilledCost = todecimal(BilledCost),\r\n CommitmentDiscountQuantity = todecimal(CommitmentDiscountQuantity),\r\n ConsumedQuantity = todecimal(ConsumedQuantity),\r\n ContractedCost = todecimal(ContractedCost),\r\n ContractedUnitPrice = todecimal(ContractedUnitPrice),\r\n EffectiveCost = todecimal(EffectiveCost),\r\n ListCost = todecimal(ListCost),\r\n ListUnitPrice = todecimal(ListUnitPrice),\r\n PricingQuantity = todecimal(PricingQuantity),\r\n x_BilledCostInUsd = todecimal(x_BilledCostInUsd),\r\n x_BilledUnitPrice = todecimal(x_BilledUnitPrice),\r\n x_BillingExchangeRate = todecimal(x_BillingExchangeRate),\r\n x_CommitmentDiscountNormalizedRatio = todecimal(x_CommitmentDiscountNormalizedRatio),\r\n x_ContractedCostInUsd = todecimal(x_ContractedCostInUsd),\r\n x_CurrencyConversionRate = todecimal(x_CurrencyConversionRate),\r\n x_EffectiveCostInUsd = todecimal(x_EffectiveCostInUsd),\r\n x_EffectiveUnitPrice = todecimal(x_EffectiveUnitPrice),\r\n x_ListCostInUsd = todecimal(x_ListCostInUsd),\r\n x_PricingBlockSize = todecimal(x_PricingBlockSize)\r\n // Rename columns\r\n | project-rename\r\n x_InvoiceId = InvoiceId,\r\n x_PricingCurrency = PricingCurrency,\r\n x_SkuMeterName = SkuMeter\r\n // Generate historical x_SkuDetails format from SkuPriceDetails\r\n | extend x_SkuDetails = iff(isnotempty(x_SkuDetails), x_SkuDetails, parse_json(replace_regex(tostring(SkuPriceDetails), @'([\\{,])\"x_', @'\\1\"')))\r\n )\r\n | project\r\n AvailabilityZone,\r\n BilledCost,\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingAccountType,\r\n BillingCurrency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n ChargeCategory,\r\n ChargeClass,\r\n ChargeDescription,\r\n ChargeFrequency,\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId,\r\n CommitmentDiscountName,\r\n CommitmentDiscountStatus,\r\n CommitmentDiscountType,\r\n ConsumedQuantity,\r\n ConsumedUnit,\r\n ContractedCost,\r\n ContractedUnitPrice,\r\n EffectiveCost,\r\n InvoiceIssuerName,\r\n ListCost,\r\n ListUnitPrice,\r\n PricingCategory,\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n PublisherName,\r\n RegionId,\r\n RegionName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n SkuId,\r\n SkuPriceId,\r\n SubAccountId,\r\n SubAccountName,\r\n SubAccountType,\r\n Tags,\r\n x_AccountId,\r\n x_AccountName,\r\n x_AccountOwnerId,\r\n x_BilledCostInUsd,\r\n x_BilledUnitPrice,\r\n x_BillingAccountAgreement,\r\n x_BillingAccountId,\r\n x_BillingAccountName,\r\n x_BillingExchangeRate,\r\n x_BillingExchangeRateDate,\r\n x_BillingProfileId,\r\n x_BillingProfileName,\r\n x_ChargeId,\r\n x_ContractedCostInUsd,\r\n x_CostAllocationRuleName,\r\n x_CostCategories,\r\n x_CostCenter,\r\n x_Credits,\r\n x_CostType,\r\n x_CurrencyConversionRate,\r\n x_CustomerId,\r\n x_CustomerName,\r\n x_Discount,\r\n x_EffectiveCostInUsd,\r\n x_EffectiveUnitPrice,\r\n x_ExportTime,\r\n x_IngestionTime,\r\n x_InvoiceId,\r\n x_InvoiceIssuerId,\r\n x_InvoiceSectionId,\r\n x_InvoiceSectionName,\r\n x_ListCostInUsd,\r\n x_Location,\r\n x_Operation,\r\n x_PartnerCreditApplied,\r\n x_PartnerCreditRate,\r\n x_PricingBlockSize,\r\n x_PricingCurrency,\r\n x_PricingSubcategory,\r\n x_PricingUnitDescription,\r\n x_Project,\r\n x_PublisherCategory,\r\n x_PublisherId,\r\n x_ResellerId,\r\n x_ResellerName,\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n x_ServiceCode,\r\n x_ServiceId,\r\n x_ServicePeriodEnd,\r\n x_ServicePeriodStart,\r\n x_SkuDescription,\r\n x_SkuDetails,\r\n x_SkuIsCreditEligible,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterName,\r\n x_SkuMeterSubcategory,\r\n x_SkuOfferId,\r\n x_SkuOrderId,\r\n x_SkuOrderName,\r\n x_SkuPartNumber,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceChanges,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_UsageType\r\n}\r\n\r\n\r\n// Prices_final_v1_0\r\n.create-or-alter function\r\nwith (docstring = 'Gets all prices aligned to FOCUS 1.0.', folder = 'Prices')\r\nPrices_v1_0()\r\n{\r\n database('Ingestion').Prices_final_v1_0\r\n | union (\r\n database('Ingestion').Prices_final_v1_2\r\n // Convert real to decimal\r\n | extend\r\n ContractedUnitPrice = todecimal(ContractedUnitPrice),\r\n ListUnitPrice = todecimal(ListUnitPrice),\r\n x_BaseUnitPrice = todecimal(x_BaseUnitPrice),\r\n x_CommitmentDiscountNormalizedRatio = todecimal(x_CommitmentDiscountNormalizedRatio),\r\n x_ContractedUnitPriceDiscount = todecimal(x_ContractedUnitPriceDiscount),\r\n x_ContractedUnitPriceDiscountPercent = todecimal(x_ContractedUnitPriceDiscountPercent),\r\n x_EffectiveUnitPrice = todecimal(x_EffectiveUnitPrice),\r\n x_EffectiveUnitPriceDiscount = todecimal(x_EffectiveUnitPriceDiscount),\r\n x_EffectiveUnitPriceDiscountPercent = todecimal(x_EffectiveUnitPriceDiscountPercent),\r\n x_PricingBlockSize = todecimal(x_PricingBlockSize),\r\n x_SkuIncludedQuantity = todecimal(x_SkuIncludedQuantity),\r\n x_SkuTier = todecimal(x_SkuTier),\r\n x_TotalUnitPriceDiscount = todecimal(x_TotalUnitPriceDiscount),\r\n x_TotalUnitPriceDiscountPercent = todecimal(x_TotalUnitPriceDiscountPercent) \r\n // Rename columns\r\n | project-rename\r\n x_PricingCurrency = PricingCurrency,\r\n x_SkuMeterName = SkuMeter\r\n )\r\n | project\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingCurrency,\r\n ChargeCategory,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountType,\r\n ContractedUnitPrice,\r\n ListUnitPrice,\r\n PricingCategory,\r\n PricingUnit,\r\n SkuId,\r\n SkuPriceId,\r\n SkuPriceIdv2,\r\n x_BaseUnitPrice,\r\n x_BillingAccountAgreement,\r\n x_BillingAccountId,\r\n x_BillingProfileId,\r\n x_CommitmentDiscountSpendEligibility,\r\n x_CommitmentDiscountUsageEligibility,\r\n x_ContractedUnitPriceDiscount,\r\n x_ContractedUnitPriceDiscountPercent,\r\n x_EffectivePeriodEnd,\r\n x_EffectivePeriodStart,\r\n x_EffectiveUnitPrice,\r\n x_EffectiveUnitPriceDiscount,\r\n x_EffectiveUnitPriceDiscountPercent,\r\n x_IngestionTime,\r\n x_PricingBlockSize,\r\n x_PricingCurrency,\r\n x_PricingSubcategory,\r\n x_PricingUnitDescription,\r\n x_SkuDescription,\r\n x_SkuId,\r\n x_SkuIncludedQuantity,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterName,\r\n x_SkuMeterSubcategory,\r\n x_SkuMeterType,\r\n x_SkuPriceType,\r\n x_SkuProductId,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuOfferId,\r\n x_SkuPartNumber,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_TotalUnitPriceDiscount,\r\n x_TotalUnitPriceDiscountPercent\r\n}\r\n\r\n\r\n// Recommendations_final_v1_0\r\n.create-or-alter function\r\nwith (docstring = 'Gets all recommendations aligned to FOCUS 1.0.', folder = 'Recommendations')\r\nRecommendations_v1_0()\r\n{\r\n database('Ingestion').Recommendations_final_v1_0\r\n | union (\r\n database('Ingestion').Recommendations_final_v1_2\r\n // Convert real to decimal\r\n | extend\r\n x_EffectiveCostAfter = todecimal(x_EffectiveCostAfter),\r\n x_EffectiveCostBefore = todecimal(x_EffectiveCostBefore),\r\n x_EffectiveCostSavings = todecimal(x_EffectiveCostSavings)\r\n )\r\n | project\r\n ProviderName,\r\n SubAccountId,\r\n x_IngestionTime,\r\n x_EffectiveCostAfter,\r\n x_EffectiveCostBefore,\r\n x_EffectiveCostSavings,\r\n x_RecommendationDate,\r\n x_RecommendationDetails,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n\r\n// Transactions_final_v1_0\r\n.create-or-alter function\r\nwith (docstring = 'Gets all transactions aligned to FOCUS 1.0.', folder = 'Transactions')\r\nTransactions_v1_0()\r\n{\r\n database('Ingestion').Transactions_final_v1_0\r\n | union (\r\n database('Ingestion').Transactions_final_v1_2\r\n // Convert real to decimal\r\n | extend\r\n BilledCost = todecimal(BilledCost),\r\n PricingQuantity = todecimal(PricingQuantity),\r\n x_MonetaryCommitment = todecimal(x_MonetaryCommitment),\r\n x_Overage = todecimal(x_Overage)\r\n // Rename columns\r\n | project-rename\r\n x_InvoiceId = InvoiceId\r\n )\r\n | project\r\n BilledCost,\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingCurrency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n ChargeCategory,\r\n ChargeClass,\r\n ChargeDescription,\r\n ChargeFrequency,\r\n ChargePeriodStart,\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n RegionId,\r\n RegionName,\r\n SubAccountId,\r\n SubAccountName,\r\n x_AccountName,\r\n x_AccountOwnerId,\r\n x_CostCenter,\r\n x_InvoiceId,\r\n x_InvoiceNumber,\r\n x_InvoiceSectionId,\r\n x_InvoiceSectionName,\r\n x_IngestionTime,\r\n x_MonetaryCommitment,\r\n x_Overage,\r\n x_PurchasingBillingAccountId,\r\n x_SkuOrderId,\r\n x_SkuOrderName,\r\n x_SkuSize,\r\n x_SkuTerm,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_SubscriptionId,\r\n x_TransactionType\r\n}\r\n", + "$fxv#14": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Hub database / FOCUS 1.2 functions\r\n// Used for reporting with backward compatibility.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n\r\n// CommitmentDiscountUsage_final_v1_2\r\n.create-or-alter function\r\nwith (docstring = 'Gets all commitment discount usage records aligned to FOCUS 1.2.', folder = 'CommitmentDiscountUsage')\r\nCommitmentDiscountUsage_v1_2()\r\n{\r\n database('Ingestion').CommitmentDiscountUsage_final_v1_2\r\n | union (\r\n database('Ingestion').CommitmentDiscountUsage_final_v1_0\r\n // Convert decimal to real\r\n | extend\r\n ConsumedQuantity = toreal(ConsumedQuantity),\r\n x_CommitmentDiscountCommittedCount = toreal(x_CommitmentDiscountCommittedCount),\r\n x_CommitmentDiscountCommittedAmount = toreal(x_CommitmentDiscountCommittedAmount),\r\n x_CommitmentDiscountNormalizedRatio = toreal(x_CommitmentDiscountNormalizedRatio)\r\n // Add new columns\r\n | lookup kind=leftouter (Services | distinct x_ResourceType, ServiceSubcategory) on x_ResourceType\r\n | extend CommitmentDiscountQuantity = ConsumedQuantity * x_CommitmentDiscountNormalizedRatio\r\n | extend CommitmentDiscountUnit = case(\r\n x_CommitmentDiscountNormalizedRatio == 1, 'Hours',\r\n x_CommitmentDiscountNormalizedRatio > 1, 'Normalized Hours',\r\n ''\r\n )\r\n )\r\n | project\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId,\r\n CommitmentDiscountQuantity,\r\n CommitmentDiscountType,\r\n CommitmentDiscountUnit,\r\n ConsumedQuantity,\r\n ProviderName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory,\r\n SubAccountId,\r\n x_CommitmentDiscountCommittedCount,\r\n x_CommitmentDiscountCommittedAmount,\r\n x_CommitmentDiscountNormalizedGroup,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_IngestionTime,\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n x_ServiceModel,\r\n x_SkuOrderId,\r\n x_SkuSize,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n\r\n// Costs_final_v1_2\r\n.create-or-alter function\r\nwith (docstring = 'Gets all cost and usage records aligned to FOCUS 1.2.', folder = 'Costs')\r\nCosts_v1_2()\r\n{\r\n database('Ingestion').Costs_final_v1_2\r\n | union (\r\n database('Ingestion').Costs_final_v1_0\r\n // Convert decimal to real\r\n | extend\r\n BilledCost = toreal(BilledCost),\r\n ConsumedQuantity = toreal(ConsumedQuantity),\r\n ContractedCost = toreal(ContractedCost),\r\n ContractedUnitPrice = toreal(ContractedUnitPrice),\r\n EffectiveCost = toreal(EffectiveCost),\r\n ListCost = toreal(ListCost),\r\n ListUnitPrice = toreal(ListUnitPrice),\r\n PricingQuantity = toreal(PricingQuantity),\r\n x_BilledCostInUsd = toreal(x_BilledCostInUsd),\r\n x_BilledUnitPrice = toreal(x_BilledUnitPrice),\r\n x_BillingExchangeRate = toreal(x_BillingExchangeRate),\r\n x_ContractedCostInUsd = toreal(x_ContractedCostInUsd),\r\n x_CurrencyConversionRate = toreal(x_CurrencyConversionRate),\r\n x_EffectiveCostInUsd = toreal(x_EffectiveCostInUsd),\r\n x_EffectiveUnitPrice = toreal(x_EffectiveUnitPrice),\r\n x_ListCostInUsd = toreal(x_ListCostInUsd),\r\n x_PricingBlockSize = toreal(x_PricingBlockSize)\r\n // Rename columns\r\n | project-rename\r\n InvoiceId = x_InvoiceId,\r\n PricingCurrency = x_PricingCurrency,\r\n SkuMeter = x_SkuMeterName\r\n // Add new columns\r\n | lookup kind=leftouter (Services | where isnotempty(x_ResourceType) | summarize take_any(ServiceSubcategory), take_any(x_ServiceModel) by x_ResourceType) on x_ResourceType\r\n | extend CapacityReservationId = tostring(x_SkuDetails.VMCapacityReservationId)\r\n | extend CapacityReservationStatus = case(\r\n isempty(CapacityReservationId), '',\r\n tolower(x_ResourceType) == 'microsoft.compute/capacityreservationgroups/capacityreservations', 'Unused',\r\n 'Used'\r\n )\r\n | extend x_CommitmentDiscountNormalizedRatio = case(\r\n // Not applicable\r\n isempty(CommitmentDiscountStatus), real(null),\r\n // Parse from SKU details if not specified explicitly\r\n toreal(coalesce(x_SkuDetails.RINormalizationRatio, dynamic(1)))\r\n )\r\n | extend CommitmentDiscountQuantity = case(\r\n isempty(CommitmentDiscountStatus), real(null),\r\n CommitmentDiscountCategory == 'Spend', EffectiveCost / coalesce(x_BillingExchangeRate, real(1)),\r\n CommitmentDiscountCategory == 'Usage' and isnotempty(x_CommitmentDiscountNormalizedRatio), PricingQuantity / coalesce(x_PricingBlockSize, real(1)) * x_CommitmentDiscountNormalizedRatio,\r\n real(null)\r\n )\r\n | extend CommitmentDiscountUnit = case(\r\n isempty(CommitmentDiscountQuantity), '',\r\n CommitmentDiscountCategory == 'Spend', PricingCurrency,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio == real(1), ConsumedUnit,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio > real(1), strcat('Normalized ', ConsumedUnit),\r\n ''\r\n )\r\n | extend x_AmortizationClass = case(\r\n ChargeCategory == 'Purchase' and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeCategory == 'Usage' and isnotempty(CommitmentDiscountId) and isnotempty(CommitmentDiscountStatus), 'Amortized Charge',\r\n ''\r\n )\r\n // Hubs add-ons\r\n | extend x_CommitmentDiscountUtilizationPotential = case(\r\n ChargeCategory == 'Purchase', real(0),\r\n ProviderName == 'Microsoft' and isnotempty(CommitmentDiscountCategory), EffectiveCost,\r\n CommitmentDiscountCategory == 'Usage', ConsumedQuantity,\r\n CommitmentDiscountCategory == 'Spend', EffectiveCost,\r\n real(0)\r\n )\r\n | extend x_CommitmentDiscountUtilizationAmount = iff(CommitmentDiscountStatus == 'Used', x_CommitmentDiscountUtilizationPotential, real(0))\r\n | extend x_SkuCoreCount = toint(coalesce(x_SkuDetails.VCPUs, x_SkuDetails.VCores, x_SkuDetails.vCores))\r\n | extend x_SkuInstanceType = tostring(coalesce(x_SkuDetails.ServiceType, x_SkuDetails.ServerSku))\r\n | extend x_SkuOperatingSystem = case(\r\n x_SkuDetails.ImageType == 'Canonical', 'Linux',\r\n x_SkuDetails.ImageType == 'Windows Server BYOL', 'Windows Server',\r\n x_SkuMeterSubcategory endswith ' Series Windows', 'Windows Server',\r\n x_SkuDetails.ImageType\r\n )\r\n | extend x_ConsumedCoreHours = iff(ConsumedUnit == 'Hours' and isnotempty(x_SkuCoreCount), x_SkuCoreCount * ConsumedQuantity, real(null))\r\n | extend tmp_SqlAhb = tolower(x_SkuDetails.AHB)\r\n | extend x_SkuLicenseType = case(\r\n x_SkuDetails.ImageType contains 'Windows Server BYOL', 'Windows Server',\r\n x_SkuMeterSubcategory == 'SQL Server Azure Hybrid Benefit', 'SQL Server',\r\n ''\r\n )\r\n | extend x_SkuLicenseStatus = case(\r\n isnotempty(x_SkuLicenseType) or tmp_SqlAhb == 'true' or (x_SkuMeterSubcategory contains 'Azure Hybrid Benefit'), 'Enabled',\r\n (x_SkuMeterSubcategory contains 'Windows') or tmp_SqlAhb == 'false', 'Not enabled',\r\n ''\r\n )\r\n | extend x_SkuLicenseQuantity = case(\r\n isempty(x_SkuCoreCount), int(null),\r\n x_SkuCoreCount <= 8, int(8),\r\n x_SkuCoreCount > 8, x_SkuCoreCount,\r\n int(null)\r\n )\r\n | extend x_SkuLicenseUnit = iff(isnotempty(x_SkuLicenseQuantity), 'Cores', '')\r\n | extend x_CommitmentDiscountSavings = iff(ContractedCost < EffectiveCost, real(0), ContractedCost - EffectiveCost)\r\n | extend x_NegotiatedDiscountSavings = iff(ListCost < ContractedCost, real(0), ListCost - ContractedCost)\r\n | extend x_TotalSavings = iff(ListCost < EffectiveCost, real(0), ListCost - EffectiveCost)\r\n | extend x_CommitmentDiscountPercent = iff(ContractedUnitPrice == 0, real(0), (ContractedUnitPrice - x_EffectiveUnitPrice) / ContractedUnitPrice)\r\n | extend x_NegotiatedDiscountPercent = iff(ListUnitPrice == 0, real(0), (ListUnitPrice - ContractedUnitPrice) / ListUnitPrice)\r\n | extend x_TotalDiscountPercent = iff(ListUnitPrice == 0, real(0), (ListUnitPrice - x_EffectiveUnitPrice) / ListUnitPrice)\r\n // SkuPriceDetails conversion -- Must be after hubs add-ons\r\n | extend SkuPriceDetails = parse_json(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(tostring(x_SkuDetails)\r\n // Prefix all keys with x_ first to avoid double-prefixing\r\n , @'([\\{,])\"', @'\\1\"x_')\r\n // CoreCount for number of CPUs/vCPUs/cores/vCores\r\n , @'\"x_(VCPUs|VCores|vCores)\":', @'\"CoreCount\":')\r\n // TODO: DiskMaxIops for disk I/O operations per second (IOPS)\r\n // TODO: DiskSpace for disk size in GiB\r\n // TODO: DiskType for the kind of disk (e.g., SSD, HDD, NVMe)\r\n // TODO: GpuCount for the number of GPUs\r\n // InstanceType for the resource size/SKU (e.g., ArmSkuName)\r\n , @'\"x_(ServerSku|ServiceType)\":', @'\"InstanceType\":')\r\n // TODO: InstanceSeries for the size family/series\r\n // TODO: MemorySize for the RAM in GiB\r\n // TODO: NetworkMaxIops for network I/O operations per second (IOPS)\r\n // TODO: NetworkMaxThroughput for network max throughput for data transfer in Mbps\r\n // OperatingSystem for the OS name\r\n , @'(\"x_ImageType\":\"Canonical\")', @'\\1,\"OperatingSystem\":\"Linux\"')\r\n , @'(\"x_ImageType\":\"Windows Server( BYOL)?\")', @'\\1,\"OperatingSystem\":\"Windows Server\"')\r\n , @'(\"x_ImageType\":(\"[^\"]+\"))', @'\\1,\"OperatingSystem\":\\2')\r\n // TODO: Redundancy for the level of redundancy (e.g., Local, Zonal, Global)\r\n // TODO: StorageClass for the tier of storage (e.g., Hot, Archive, Nearline)\r\n )\r\n | extend SkuPriceDetails = iff(isempty(SkuPriceDetails.OperatingSystem) and isnotempty(x_SkuOperatingSystem),\r\n parse_json(replace_string(tostring(SkuPriceDetails), '}', strcat(@',\"OperatingSystem\":\"', x_SkuOperatingSystem, '\"}'))),\r\n SkuPriceDetails)\r\n )\r\n | extend SkuPriceDetails = iff(isnotempty(SkuPriceDetails), SkuPriceDetails, parse_json(replace_regex(tostring(x_SkuDetails), @'([\\{,])\"', @'\\1\"x_')))\r\n | project\r\n AvailabilityZone,\r\n BilledCost,\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingAccountType,\r\n BillingCurrency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n CapacityReservationId,\r\n CapacityReservationStatus,\r\n ChargeCategory,\r\n ChargeClass,\r\n ChargeDescription,\r\n ChargeFrequency,\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId,\r\n CommitmentDiscountName,\r\n CommitmentDiscountQuantity,\r\n CommitmentDiscountStatus,\r\n CommitmentDiscountType,\r\n CommitmentDiscountUnit,\r\n ConsumedQuantity,\r\n ConsumedUnit,\r\n ContractedCost,\r\n ContractedUnitPrice,\r\n EffectiveCost,\r\n InvoiceId,\r\n InvoiceIssuerName,\r\n ListCost,\r\n ListUnitPrice,\r\n PricingCategory,\r\n PricingCurrency,\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n PublisherName,\r\n RegionId,\r\n RegionName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory,\r\n SkuId,\r\n SkuMeter,\r\n SkuPriceDetails,\r\n SkuPriceId,\r\n SubAccountId,\r\n SubAccountName,\r\n SubAccountType,\r\n Tags,\r\n x_AccountId,\r\n x_AccountName,\r\n x_AccountOwnerId,\r\n x_AmortizationClass,\r\n x_BilledCostInUsd,\r\n x_BilledUnitPrice,\r\n x_BillingAccountAgreement,\r\n x_BillingAccountId,\r\n x_BillingAccountName,\r\n x_BillingExchangeRate,\r\n x_BillingExchangeRateDate,\r\n x_BillingItemCode,\r\n x_BillingItemName,\r\n x_BillingProfileId,\r\n x_BillingProfileName,\r\n x_ChargeId,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_CommitmentDiscountPercent,\r\n x_CommitmentDiscountSavings,\r\n x_CommitmentDiscountSpendEligibility,\r\n x_CommitmentDiscountUsageEligibility,\r\n x_CommitmentDiscountUtilizationAmount,\r\n x_CommitmentDiscountUtilizationPotential,\r\n x_CommodityCode,\r\n x_CommodityName,\r\n x_ComponentName,\r\n x_ComponentType,\r\n x_ConsumedCoreHours,\r\n x_ContractedCostInUsd,\r\n x_CostAllocationRuleName,\r\n x_CostCategories,\r\n x_CostCenter,\r\n x_CostType,\r\n x_Credits,\r\n x_CurrencyConversionRate,\r\n x_CustomerId,\r\n x_CustomerName,\r\n x_Discount,\r\n x_EffectiveCostInUsd,\r\n x_EffectiveUnitPrice,\r\n x_ExportTime,\r\n x_IngestionTime,\r\n x_InstanceID,\r\n x_InvoiceIssuerId,\r\n x_InvoiceSectionId,\r\n x_InvoiceSectionName,\r\n x_ListCostInUsd,\r\n x_Location,\r\n x_NegotiatedDiscountPercent,\r\n x_NegotiatedDiscountSavings,\r\n x_Operation,\r\n x_OwnerAccountID,\r\n x_PartnerCreditApplied,\r\n x_PartnerCreditRate,\r\n x_PricingBlockSize,\r\n x_PricingSubcategory,\r\n x_PricingUnitDescription,\r\n x_Project,\r\n x_PublisherCategory,\r\n x_PublisherId,\r\n x_ResellerId,\r\n x_ResellerName,\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n x_ServiceCode,\r\n x_ServiceId,\r\n x_ServiceModel,\r\n x_ServicePeriodEnd,\r\n x_ServicePeriodStart,\r\n x_SkuCoreCount,\r\n x_SkuDescription,\r\n x_SkuDetails,\r\n x_SkuInstanceType,\r\n x_SkuIsCreditEligible,\r\n x_SkuLicenseQuantity,\r\n x_SkuLicenseStatus,\r\n x_SkuLicenseType,\r\n x_SkuLicenseUnit,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterSubcategory,\r\n x_SkuOfferId,\r\n x_SkuOperatingSystem,\r\n x_SkuOrderId,\r\n x_SkuOrderName,\r\n x_SkuPartNumber,\r\n x_SkuPlanName,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceChanges,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceValues,\r\n x_SourceVersion,\r\n x_SubproductName,\r\n x_TotalDiscountPercent,\r\n x_TotalSavings,\r\n x_UsageType\r\n}\r\n\r\n\r\n// Prices_final_v1_2\r\n.create-or-alter function\r\nwith (docstring = 'Gets all prices aligned to FOCUS 1.2.', folder = 'Prices')\r\nPrices_v1_2()\r\n{\r\n database('Ingestion').Prices_final_v1_2\r\n | union (\r\n database('Ingestion').Prices_final_v1_0\r\n // Convert decimal to real\r\n | extend\r\n ContractedUnitPrice = toreal(ContractedUnitPrice),\r\n ListUnitPrice = toreal(ListUnitPrice),\r\n x_BaseUnitPrice = toreal(x_BaseUnitPrice),\r\n x_ContractedUnitPriceDiscount = toreal(x_ContractedUnitPriceDiscount),\r\n x_ContractedUnitPriceDiscountPercent = toreal(x_ContractedUnitPriceDiscountPercent),\r\n x_EffectiveUnitPrice = toreal(x_EffectiveUnitPrice),\r\n x_EffectiveUnitPriceDiscount = toreal(x_EffectiveUnitPriceDiscount),\r\n x_EffectiveUnitPriceDiscountPercent = toreal(x_EffectiveUnitPriceDiscountPercent),\r\n x_PricingBlockSize = toreal(x_PricingBlockSize),\r\n x_SkuIncludedQuantity = toreal(x_SkuIncludedQuantity),\r\n x_SkuTier = toreal(x_SkuTier),\r\n x_TotalUnitPriceDiscount = toreal(x_TotalUnitPriceDiscount),\r\n x_TotalUnitPriceDiscountPercent = toreal(x_TotalUnitPriceDiscountPercent) \r\n // Rename columns\r\n | project-rename\r\n PricingCurrency = x_PricingCurrency,\r\n SkuMeter = x_SkuMeterName\r\n )\r\n | project\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingCurrency,\r\n ChargeCategory,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountType,\r\n CommitmentDiscountUnit,\r\n ContractedUnitPrice,\r\n ListUnitPrice,\r\n PricingCategory,\r\n PricingCurrency,\r\n PricingUnit,\r\n SkuId,\r\n SkuMeter,\r\n SkuPriceId,\r\n SkuPriceIdv2,\r\n x_BaseUnitPrice,\r\n x_BillingAccountAgreement,\r\n x_BillingAccountId,\r\n x_BillingProfileId,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_CommitmentDiscountSpendEligibility,\r\n x_CommitmentDiscountUsageEligibility,\r\n x_ContractedUnitPriceDiscount,\r\n x_ContractedUnitPriceDiscountPercent,\r\n x_EffectivePeriodEnd,\r\n x_EffectivePeriodStart,\r\n x_EffectiveUnitPrice,\r\n x_EffectiveUnitPriceDiscount,\r\n x_EffectiveUnitPriceDiscountPercent,\r\n x_IngestionTime,\r\n x_PricingBlockSize,\r\n x_PricingSubcategory,\r\n x_PricingUnitDescription,\r\n x_SkuDescription,\r\n x_SkuId,\r\n x_SkuIncludedQuantity,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterSubcategory,\r\n x_SkuMeterType,\r\n x_SkuPriceType,\r\n x_SkuProductId,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuOfferId,\r\n x_SkuPartNumber,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_TotalUnitPriceDiscount,\r\n x_TotalUnitPriceDiscountPercent\r\n}\r\n\r\n\r\n// Recommendations_final_v1_2\r\n.create-or-alter function\r\nwith (docstring = 'Gets all recommendations aligned to FOCUS 1.2.', folder = 'Recommendations')\r\nRecommendations_v1_2()\r\n{\r\n database('Ingestion').Recommendations_final_v1_2\r\n | union (\r\n database('Ingestion').Recommendations_final_v1_0\r\n // Convert decimal to real\r\n | extend\r\n x_EffectiveCostAfter = toreal(x_EffectiveCostAfter),\r\n x_EffectiveCostBefore = toreal(x_EffectiveCostBefore),\r\n x_EffectiveCostSavings = toreal(x_EffectiveCostSavings)\r\n )\r\n | project\r\n ProviderName,\r\n SubAccountId,\r\n x_IngestionTime,\r\n x_EffectiveCostAfter,\r\n x_EffectiveCostBefore,\r\n x_EffectiveCostSavings,\r\n x_RecommendationDate,\r\n x_RecommendationDetails,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n\r\n// Transactions_final_v1_2\r\n.create-or-alter function\r\nwith (docstring = 'Gets all transactions aligned to FOCUS 1.2.', folder = 'Transactions')\r\nTransactions_v1_2()\r\n{\r\n database('Ingestion').Transactions_final_v1_2\r\n | union (\r\n database('Ingestion').Transactions_final_v1_0\r\n // Convert decimal to real\r\n | extend\r\n BilledCost = toreal(BilledCost),\r\n PricingQuantity = toreal(PricingQuantity),\r\n x_MonetaryCommitment = toreal(x_MonetaryCommitment),\r\n x_Overage = toreal(x_Overage)\r\n // Rename columns\r\n | project-rename\r\n InvoiceId = x_InvoiceId\r\n )\r\n | project\r\n BilledCost,\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingCurrency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n ChargeCategory,\r\n ChargeClass,\r\n ChargeDescription,\r\n ChargeFrequency,\r\n ChargePeriodStart,\r\n InvoiceId,\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n RegionId,\r\n RegionName,\r\n SubAccountId,\r\n SubAccountName,\r\n x_AccountName,\r\n x_AccountOwnerId,\r\n x_CostCenter,\r\n x_InvoiceNumber,\r\n x_InvoiceSectionId,\r\n x_InvoiceSectionName,\r\n x_IngestionTime,\r\n x_MonetaryCommitment,\r\n x_Overage,\r\n x_PurchasingBillingAccountId,\r\n x_SkuOrderId,\r\n x_SkuOrderName,\r\n x_SkuSize,\r\n x_SkuTerm,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_SubscriptionId,\r\n x_TransactionType\r\n}\r\n\r\n\r\n//======================================================================================================================\r\n// Latest FOCUS version\r\n//======================================================================================================================\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all commitment discount usage records with the latest supported version of the FOCUS schema.', folder = 'CommitmentDiscountUsage')\r\nCommitmentDiscountUsage()\r\n{\r\n CommitmentDiscountUsage_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all cost and usage records with the latest supported version of the FOCUS schema.', folder = 'Costs')\r\nCosts()\r\n{\r\n Costs_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all prices with the latest supported version of the FOCUS schema.', folder = 'Prices')\r\nPrices()\r\n{\r\n Prices_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all recommendations with the latest supported version of the FOCUS schema.', folder = 'Recommendations')\r\nRecommendations()\r\n{\r\n Recommendations_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all transactions with the latest supported version of the FOCUS schema.', folder = 'Transactions')\r\nTransactions()\r\n{\r\n Transactions_v1_2()\r\n}\r\n", + "$fxv#15": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Hub database / Latest FOCUS version functions\r\n// Used for ad hoc queries.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all commitment discount usage records with the latest supported version of the FOCUS schema.', folder = 'CommitmentDiscountUsage')\r\nCommitmentDiscountUsage()\r\n{\r\n CommitmentDiscountUsage_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all cost and usage records with the latest supported version of the FOCUS schema.', folder = 'Costs')\r\nCosts()\r\n{\r\n Costs_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all prices with the latest supported version of the FOCUS schema.', folder = 'Prices')\r\nPrices()\r\n{\r\n Prices_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all recommendations with the latest supported version of the FOCUS schema.', folder = 'Recommendations')\r\nRecommendations()\r\n{\r\n Recommendations_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all transactions with the latest supported version of the FOCUS schema.', folder = 'Transactions')\r\nTransactions()\r\n{\r\n Transactions_v1_2()\r\n}\r\n", + "$fxv#2": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData/Internal')\r\n_resource_type_3(id: string) {\r\n dynamic({\r\n \"microsoft.hybridnetwork/vendors\": { \"SingularDisplayName\": \"Azure Network Function Manager ? vendor\" }\r\n ,\"microsoft.hybridonboarding/extensionmanagers\": { \"SingularDisplayName\": \"Microsoft.HybridOnboarding extension manager\" }\r\n ,\"microsoft.impact/connectors\": { \"SingularDisplayName\": \"Impact Reporting Connector\" }\r\n ,\"microsoft.impact/impactcategories\": { \"SingularDisplayName\": \"Microsoft.Impact impact category\" }\r\n ,\"microsoft.impact/topologyimpacts\": { \"SingularDisplayName\": \"Microsoft.Impact topology impact\" }\r\n ,\"microsoft.impact/workloadimpacts\": { \"SingularDisplayName\": \"Microsoft.Impact workload impact\" }\r\n ,\"microsoft.impact/workloadimpacts/insights\": { \"SingularDisplayName\": \"Microsoft.Impact workload impacts insight\" }\r\n ,\"microsoft.importexport/jobs\": { \"SingularDisplayName\": \"Microsoft.ImportExport job\" }\r\n ,\"microsoft.insights/actiongroups\": { \"SingularDisplayName\": \"Action group\" }\r\n ,\"microsoft.insights/activitylogalerts\": { \"SingularDisplayName\": \"Activity log alert rule\" }\r\n ,\"microsoft.insights/alertrules\": { \"SingularDisplayName\": \"Microsoft.Insights alertrule\" }\r\n ,\"microsoft.insights/alertrules/incidents\": { \"SingularDisplayName\": \"Microsoft.insights alertrules incident\" }\r\n ,\"microsoft.insights/autoscalesettings\": { \"SingularDisplayName\": \"Microsoft.Insights autoscalesetting\" }\r\n ,\"microsoft.insights/components\": { \"SingularDisplayName\": \"Application Insights app\" }\r\n ,\"microsoft.insights/datacollectionendpoints\": { \"SingularDisplayName\": \"Data collection endpoint\" }\r\n ,\"microsoft.insights/datacollectionruleassociations\": { \"SingularDisplayName\": \"Microsoft.Insights data collection rule association\" }\r\n ,\"microsoft.insights/datacollectionrules\": { \"SingularDisplayName\": \"Data collection rule\" }\r\n ,\"microsoft.insights/datacollectionrulesresources\": { \"SingularDisplayName\": \"Data collection rule associated resource\" }\r\n ,\"microsoft.insights/diagnosticsettings\": { \"SingularDisplayName\": \"Diagnostic settings\" }\r\n ,\"microsoft.insights/diagnosticsettingscategories\": { \"SingularDisplayName\": \"Microsoft.Insights diagnostic settings category\" }\r\n ,\"microsoft.insights/guestdiagnosticsettings\": { \"SingularDisplayName\": \"Microsoft.insights guest diagnostic setting\" }\r\n ,\"microsoft.insights/guestdiagnosticsettingsassociation\": { \"SingularDisplayName\": \"Microsoft.insights guest diagnostic settings association\" }\r\n ,\"microsoft.insights/logprofiles\": { \"SingularDisplayName\": \"Microsoft.Insights logprofile\" }\r\n ,\"microsoft.insights/metricalerts\": { \"SingularDisplayName\": \"Metric alert rule\" }\r\n ,\"microsoft.insights/notificationstatus\": { \"SingularDisplayName\": \"Microsoft.Insights notification statu\" }\r\n ,\"microsoft.insights/privatelinkscopeoperationstatuses\": { \"SingularDisplayName\": \"Microsoft.insights private link scope operation statuse\" }\r\n ,\"microsoft.insights/privatelinkscopes\": { \"SingularDisplayName\": \"Azure Monitor Private Link Scope\" }\r\n ,\"microsoft.insights/scheduledqueryrules\": { \"SingularDisplayName\": \"Log search alert rule\" }\r\n ,\"microsoft.insights/tenantactiongroups\": { \"SingularDisplayName\": \"Microsoft.Insights tenant action group\" }\r\n ,\"microsoft.insights/tenantactiongroups/notificationstatus\": { \"SingularDisplayName\": \"Microsoft.Insights tenant action groups notification statu\" }\r\n ,\"microsoft.insights/vminsightsonboardingstatuses\": { \"SingularDisplayName\": \"Microsoft.Insights VM insights onboarding statuse\" }\r\n ,\"microsoft.insights/webtests\": { \"SingularDisplayName\": \"Application Insights availability test\" }\r\n ,\"microsoft.insights/workbooks\": { \"SingularDisplayName\": \"Azure Workbook\" }\r\n ,\"microsoft.insights/workbooktemplates\": { \"SingularDisplayName\": \"Azure Workbook Template\" }\r\n ,\"microsoft.integrationspaces/spaces\": { \"SingularDisplayName\": \"Integration Environment\" }\r\n ,\"microsoft.intelligentitdigitaltwin/digitaltwins\": { \"SingularDisplayName\": \"Microsoft.IntelligentITDigitalTwin digital twin\" }\r\n ,\"microsoft.intelligentitdigitaltwin/digitaltwins/assets\": { \"SingularDisplayName\": \"Microsoft.IntelligentITDigitalTwin digital twins asset\" }\r\n ,\"microsoft.intelligentitdigitaltwin/digitaltwins/executionplans\": { \"SingularDisplayName\": \"Microsoft.IntelligentITDigitalTwin digital twins execution plan\" }\r\n ,\"microsoft.intelligentitdigitaltwin/digitaltwins/testplans\": { \"SingularDisplayName\": \"Microsoft.IntelligentITDigitalTwin digital twins test plan\" }\r\n ,\"microsoft.intelligentitdigitaltwin/digitaltwins/tests\": { \"SingularDisplayName\": \"Microsoft.IntelligentITDigitalTwin digital twins test\" }\r\n ,\"microsoft.inventory/subscriptioninternalproperties\": { \"SingularDisplayName\": \"Microsoft.Inventory subscription internal property\" }\r\n ,\"microsoft.iotcentral/iotapps\": { \"SingularDisplayName\": \"IoT Central Application\" }\r\n ,\"microsoft.iotfirmwaredefense/workspaces\": { \"SingularDisplayName\": \"Firmware analysis workspace\" }\r\n ,\"microsoft.iotfirmwaredefense/workspaces/firmwares\": { \"SingularDisplayName\": \"Microsoft.IoTFirmwareDefense workspaces firmware\" }\r\n ,\"microsoft.iotfirmwaredefense/workspaces/firmwares/summaries\": { \"SingularDisplayName\": \"Microsoft.IoTFirmwareDefense workspaces firmwares summary\" }\r\n ,\"microsoft.iotoperations/instances\": { \"SingularDisplayName\": \"Azure IoT Operations\" }\r\n ,\"microsoft.iotoperations/instances/brokers\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances broker\" }\r\n ,\"microsoft.iotoperations/instances/brokers/authentications\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances brokers authentication\" }\r\n ,\"microsoft.iotoperations/instances/brokers/authorizations\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances brokers authorization\" }\r\n ,\"microsoft.iotoperations/instances/brokers/listeners\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances brokers listener\" }\r\n ,\"microsoft.iotoperations/instances/dataflowendpoints\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances dataflow endpoint\" }\r\n ,\"microsoft.iotoperations/instances/dataflowprofiles\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances dataflow profile\" }\r\n ,\"microsoft.iotoperations/instances/dataflowprofiles/dataflows\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances dataflow profiles dataflow\" }\r\n ,\"microsoft.iotoperationsdataprocessor/instances\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsDataProcessor instance\" }\r\n ,\"microsoft.iotoperationsdataprocessor/instances/datasets\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsDataProcessor instances dataset\" }\r\n ,\"microsoft.iotoperationsdataprocessor/instances/pipelines\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsDataProcessor instances pipeline\" }\r\n ,\"microsoft.iotoperationsmq/mq\": { \"SingularDisplayName\": \"IoT Operations Ops MQ\" }\r\n ,\"microsoft.iotoperationsmq/mq/broker\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq broker\" }\r\n ,\"microsoft.iotoperationsmq/mq/broker/authentication\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq broker authentication\" }\r\n ,\"microsoft.iotoperationsmq/mq/broker/authorization\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq broker authorization\" }\r\n ,\"microsoft.iotoperationsmq/mq/broker/listener\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq broker listener\" }\r\n ,\"microsoft.iotoperationsmq/mq/datalakeconnector\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq data lake connector\" }\r\n ,\"microsoft.iotoperationsmq/mq/datalakeconnector/topicmap\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq data lake connector topic map\" }\r\n ,\"microsoft.iotoperationsmq/mq/diagnosticservice\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq diagnostic service\" }\r\n ,\"microsoft.iotoperationsmq/mq/kafkaconnector\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq kafka connector\" }\r\n ,\"microsoft.iotoperationsmq/mq/kafkaconnector/topicmap\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq kafka connector topic map\" }\r\n ,\"microsoft.iotoperationsmq/mq/mqttbridgeconnector\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq mqtt bridge connector\" }\r\n ,\"microsoft.iotoperationsmq/mq/mqttbridgeconnector/topicmap\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq mqtt bridge connector topic map\" }\r\n ,\"microsoft.iotoperationsorchestrator/instances\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsOrchestrator instance\" }\r\n ,\"microsoft.iotoperationsorchestrator/solutions\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsOrchestrator solution\" }\r\n ,\"microsoft.iotoperationsorchestrator/targets\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsOrchestrator target\" }\r\n ,\"microsoft.iotsecurity/alerttypes\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity alert type\" }\r\n ,\"microsoft.iotsecurity/defendersettings\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity defender setting\" }\r\n ,\"microsoft.iotsecurity/onpremisesensors\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity on premise sensor\" }\r\n ,\"microsoft.iotsecurity/recommendationtypes\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity recommendation type\" }\r\n ,\"microsoft.iotsecurity/sensors\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity sensor\" }\r\n ,\"microsoft.iotsecurity/sites\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity site\" }\r\n ,\"microsoft.keyvault/managedhsms\": { \"SingularDisplayName\": \"Azure Key Vault Managed HSM\" }\r\n ,\"microsoft.keyvault/vaults\": { \"SingularDisplayName\": \"Key vault\" }\r\n ,\"microsoft.kubernetes/connectedclusters\": { \"SingularDisplayName\": \"Kubernetes - Azure Arc\" }\r\n ,\"microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/extensions\": { \"SingularDisplayName\": \"Kubernetes - Azure Arc extension\" }\r\n ,\"microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/fluxconfigurations\": { \"SingularDisplayName\": \"GitOps configuration\" }\r\n ,\"microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/namespaces\": { \"SingularDisplayName\": \"Kubernetes - Azure Arc namespace\" }\r\n ,\"microsoft.kubernetesconfiguration/extensions\": { \"SingularDisplayName\": \"Kubernetes service extension\" }\r\n ,\"microsoft.kubernetesconfiguration/extensiontypes\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration extension type\" }\r\n ,\"microsoft.kubernetesconfiguration/extensiontypes/versions\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration extension types version\" }\r\n ,\"microsoft.kubernetesconfiguration/fluxconfigurations\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration flux configuration\" }\r\n ,\"microsoft.kubernetesconfiguration/fluxconfigurations/operations\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration flux configurations operation\" }\r\n ,\"microsoft.kubernetesconfiguration/privatelinkscopes\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration private link scope\" }\r\n ,\"microsoft.kubernetesconfiguration/privatelinkscopes/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration private link scopes private endpoint connection\" }\r\n ,\"microsoft.kubernetesconfiguration/privatelinkscopes/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration private link scopes private link resource\" }\r\n ,\"microsoft.kubernetesconfiguration/sourcecontrolconfigurations\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration source control configuration\" }\r\n ,\"microsoft.kubernetesruntime/bgppeers\": { \"SingularDisplayName\": \"Microsoft.KubernetesRuntime bgp peer\" }\r\n ,\"microsoft.kubernetesruntime/loadbalancers\": { \"SingularDisplayName\": \"Arc Load Balancer\" }\r\n ,\"microsoft.kubernetesruntime/services\": { \"SingularDisplayName\": \"Microsoft.KubernetesRuntime service\" }\r\n ,\"microsoft.kubernetesruntime/storageclasses\": { \"SingularDisplayName\": \"Microsoft.KubernetesRuntime storage class\" }\r\n ,\"microsoft.kusto/clusters\": { \"SingularDisplayName\": \"Azure Data Explorer Cluster\" }\r\n ,\"microsoft.kusto/clusters/databases\": { \"SingularDisplayName\": \"Azure Data Explorer Database\" }\r\n ,\"microsoft.labservices/labaccounts\": { \"SingularDisplayName\": \"Lab account\" }\r\n ,\"microsoft.labservices/labaccounts/labs\": { \"SingularDisplayName\": \"Lab\" }\r\n ,\"microsoft.labservices/labplans\": { \"SingularDisplayName\": \"Lab plan\" }\r\n ,\"microsoft.labservices/labs\": { \"SingularDisplayName\": \"Lab\" }\r\n ,\"microsoft.liftrpilot/organizations\": { \"SingularDisplayName\": \"Azure Pilot\" }\r\n ,\"microsoft.loadtestservice/loadtestmappings\": { \"SingularDisplayName\": \"Microsoft.LoadTestService load test mapping\" }\r\n ,\"microsoft.loadtestservice/loadtestprofilemappings\": { \"SingularDisplayName\": \"Microsoft.LoadTestService load test profile mapping\" }\r\n ,\"microsoft.loadtestservice/loadtests\": { \"SingularDisplayName\": \"Azure Load Testing\" }\r\n ,\"microsoft.loadtestservice/playwrightworkspaces\": { \"SingularDisplayName\": \"Playwright Workspace\" }\r\n ,\"microsoft.logic/businessprocesses\": { \"SingularDisplayName\": \"Business Process\" }\r\n ,\"microsoft.logic/integrationaccounts\": { \"SingularDisplayName\": \"Logic app integration account\" }\r\n ,\"microsoft.logic/integrationserviceenvironments\": { \"SingularDisplayName\": \"Integration Service Environment\" }\r\n ,\"microsoft.logic/integrationserviceenvironments/health\": { \"SingularDisplayName\": \"Microsoft.Logic integration service environments health\" }\r\n ,\"microsoft.logic/integrationserviceenvironments/managedapis\": { \"SingularDisplayName\": \"Managed Connector\" }\r\n ,\"microsoft.logic/templates\": { \"SingularDisplayName\": \"Logic Apps Template\" }\r\n ,\"microsoft.logic/workflows\": { \"SingularDisplayName\": \"Logic app\" }\r\n ,\"microsoft.logz/monitors\": { \"SingularDisplayName\": \"Logz.io\" }\r\n ,\"microsoft.logz/monitors/accounts\": { \"SingularDisplayName\": \"Logz sub account\" }\r\n ,\"microsoft.m365/m365resources\": { \"SingularDisplayName\": \"Microsoft.M365 m365 resource\" }\r\n ,\"microsoft.m365consumptionservices/services\": { \"SingularDisplayName\": \"Microsoft.M365ConsumptionServices service\" }\r\n ,\"microsoft.machinelearning/commitmentplans\": { \"SingularDisplayName\": \"Microsoft.MachineLearning commitment plan\" }\r\n ,\"microsoft.machinelearning/commitmentplans/commitmentassociations\": { \"SingularDisplayName\": \"Microsoft.MachineLearning commitment plans commitment association\" }\r\n ,\"microsoft.machinelearning/webservices\": { \"SingularDisplayName\": \"Machine Learning Studio (classic) web service\" }\r\n ,\"microsoft.machinelearning/workspaces\": { \"SingularDisplayName\": \"Machine Learning Studio (classic) workspace\" }\r\n ,\"microsoft.machinelearningexperimentation/accounts\": { \"SingularDisplayName\": \"Microsoft.MachineLearningExperimentation account\" }\r\n ,\"microsoft.machinelearningexperimentation/accounts/workspaces\": { \"SingularDisplayName\": \"Microsoft.MachineLearningExperimentation accounts workspace\" }\r\n ,\"microsoft.machinelearningexperimentation/accounts/workspaces/projects\": { \"SingularDisplayName\": \"Microsoft.MachineLearningExperimentation accounts workspaces project\" }\r\n ,\"microsoft.machinelearningservices/aistudio\": { \"SingularDisplayName\": \"Azure AI Foundry\" }\r\n ,\"microsoft.machinelearningservices/aistudiocreate\": { \"SingularDisplayName\": \"Azure AI Foundry\" }\r\n ,\"microsoft.machinelearningservices/registries\": { \"SingularDisplayName\": \"Azure Machine Learning registry\" }\r\n ,\"microsoft.machinelearningservices/workspaces\": { \"SingularDisplayName\": \"Azure Machine Learning workspace\" }\r\n ,\"microsoft.machinelearningservices/workspaces/onlineendpoints\": { \"SingularDisplayName\": \"Machine learning online endpoint\" }\r\n ,\"microsoft.machinelearningservices/workspaces/onlineendpoints/deployments\": { \"SingularDisplayName\": \"Machine learning online deployment\" }\r\n ,\"microsoft.machinelearningservices/workspacescreate\": { \"SingularDisplayName\": \"Azure Machine Learning workspace\" }\r\n ,\"microsoft.maintenance/configurationassignments\": { \"SingularDisplayName\": \"Microsoft.Maintenance configuration assignment\" }\r\n ,\"microsoft.maintenance/maintenanceconfigurations\": { \"SingularDisplayName\": \"Maintenance Configuration\" }\r\n ,\"microsoft.maintenance/maintenanceconfigurationsaumbladeresource\": { \"SingularDisplayName\": \"Maintenance configuration\" }\r\n ,\"microsoft.maintenance/maintenanceconfigurationsbladeresource\": { \"SingularDisplayName\": \"Maintenance configuration\" }\r\n ,\"microsoft.maintenance/publicmaintenanceconfigurations\": { \"SingularDisplayName\": \"Microsoft.Maintenance public maintenance configuration\" }\r\n ,\"microsoft.managedidentity/identities\": { \"SingularDisplayName\": \"Microsoft.ManagedIdentity identity\" }\r\n ,\"microsoft.managedidentity/userassignedidentities\": { \"SingularDisplayName\": \"Managed Identity\" }\r\n ,\"microsoft.managednetwork/managednetworks\": { \"SingularDisplayName\": \"Microsoft.ManagedNetwork managed network\" }\r\n ,\"microsoft.managednetwork/managednetworks/managednetworkgroups\": { \"SingularDisplayName\": \"Microsoft.ManagedNetwork managed networks managed network group\" }\r\n ,\"microsoft.managednetwork/managednetworks/managednetworkpeeringpolicies\": { \"SingularDisplayName\": \"Microsoft.ManagedNetwork managed networks managed network peering policy\" }\r\n ,\"microsoft.managednetworkfabric/accesscontrollists\": { \"SingularDisplayName\": \"Access Control List (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/internetgatewayrules\": { \"SingularDisplayName\": \"Internet Gateway Rule (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/internetgateways\": { \"SingularDisplayName\": \"Internet Gateway (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/ipcommunities\": { \"SingularDisplayName\": \"IP Community (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/ipextendedcommunities\": { \"SingularDisplayName\": \"IP Extended Community (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/ipprefixes\": { \"SingularDisplayName\": \"IP Prefix (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/l2isolationdomains\": { \"SingularDisplayName\": \"Layer 2 Isolation Domain (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/l3isolationdomains\": { \"SingularDisplayName\": \"Layer 3 Isolation Domain (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/l3isolationdomains/externalnetworks\": { \"SingularDisplayName\": \"External Network (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/l3isolationdomains/internalnetworks\": { \"SingularDisplayName\": \"Internal Network (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/neighborgroups\": { \"SingularDisplayName\": \"Neighbor Group (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkdevices\": { \"SingularDisplayName\": \"Network Device (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkdevices/networkinterfaces\": { \"SingularDisplayName\": \"Network Interface (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkfabriccontrollers\": { \"SingularDisplayName\": \"Network Fabric Controller (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkfabrics\": { \"SingularDisplayName\": \"Network Fabric (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkfabrics/networktonetworkinterconnects\": { \"SingularDisplayName\": \"Network to Network Interconnect (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkfabricskus\": { \"SingularDisplayName\": \"Network Fabric SKU (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkmonitors\": { \"SingularDisplayName\": \"Microsoft.ManagedNetworkFabric network monitor\" }\r\n ,\"microsoft.managednetworkfabric/networkpacketbrokers\": { \"SingularDisplayName\": \"Network Packet Broker (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkracks\": { \"SingularDisplayName\": \"Network Rack (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networktaprules\": { \"SingularDisplayName\": \"Network Tap Rule (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networktaps\": { \"SingularDisplayName\": \"Network Tap (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/routepolicies\": { \"SingularDisplayName\": \"Route Policy (Operator Nexus)\" }\r\n ,\"microsoft.managedservices/marketplaceregistrationdefinitions\": { \"SingularDisplayName\": \"Microsoft.ManagedServices marketplace registration definition\" }\r\n ,\"microsoft.managedservices/registrationassignments\": { \"SingularDisplayName\": \"Microsoft.ManagedServices registration assignment\" }\r\n ,\"microsoft.managedservices/registrationdefinitions\": { \"SingularDisplayName\": \"Azure Lighthouse\" }\r\n ,\"microsoft.management/managementgroups\": { \"SingularDisplayName\": \"Microsoft.Management management group\" }\r\n ,\"microsoft.management/managementgroups/microsoft.resources/deploymentstacks\": { \"SingularDisplayName\": \"Deployment stack\" }\r\n ,\"microsoft.management/managementgroups/providers/privatelinkassociations\": { \"SingularDisplayName\": \"Application Gateway\" }\r\n ,\"microsoft.management/managementgroups/providers/templatespecs\": { \"SingularDisplayName\": \"Template spec\" }\r\n ,\"microsoft.management/managementgroups/settings\": { \"SingularDisplayName\": \"Microsoft.Management management groups setting\" }\r\n ,\"microsoft.management/managementgroups/subscriptions\": { \"SingularDisplayName\": \"Microsoft.Management management groups subscription\" }\r\n ,\"microsoft.management/servicegroups\": { \"SingularDisplayName\": \"Service group\" }\r\n ,\"microsoft.managementpartner/partners\": { \"SingularDisplayName\": \"Microsoft.ManagementPartner partner\" }\r\n ,\"microsoft.manufacturingplatform/manufacturingdataservices\": { \"SingularDisplayName\": \"Factory Operations Agent in Azure AI Foundry\" }\r\n ,\"microsoft.maps/accounts\": { \"SingularDisplayName\": \"Azure Maps Account\" }\r\n ,\"microsoft.maps/accounts/creators\": { \"SingularDisplayName\": \"Azure Maps Creator Resource\" }\r\n ,\"microsoft.marketplace/privatestores\": { \"SingularDisplayName\": \"Microsoft.Marketplace private store\" }\r\n ,\"microsoft.marketplace/privatestores/adminrequestapprovals\": { \"SingularDisplayName\": \"Microsoft.Marketplace private stores admin request approval\" }\r\n ,\"microsoft.marketplace/privatestores/collections\": { \"SingularDisplayName\": \"Microsoft.Marketplace private stores collection\" }\r\n ,\"microsoft.marketplace/privatestores/collections/offers\": { \"SingularDisplayName\": \"Microsoft.Marketplace private stores collections offer\" }\r\n ,\"microsoft.marketplace/privatestores/offers\": { \"SingularDisplayName\": \"Microsoft.Marketplace private stores offer\" }\r\n ,\"microsoft.marketplace/privatestores/requestapprovals\": { \"SingularDisplayName\": \"Microsoft.Marketplace private stores request approval\" }\r\n ,\"microsoft.media/mediaservices\": { \"SingularDisplayName\": \"Media service\" }\r\n ,\"microsoft.media/mediaservices/accountfilters\": { \"SingularDisplayName\": \"Microsoft.Media media services account filter\" }\r\n ,\"microsoft.media/mediaservices/assets\": { \"SingularDisplayName\": \"Microsoft.Media media services asset\" }\r\n ,\"microsoft.media/mediaservices/assets/assetfilters\": { \"SingularDisplayName\": \"Microsoft.Media media services assets asset filter\" }\r\n ,\"microsoft.media/mediaservices/assets/tracks\": { \"SingularDisplayName\": \"Microsoft.Media media services assets track\" }\r\n ,\"microsoft.media/mediaservices/assets/tracks/operationresults\": { \"SingularDisplayName\": \"Microsoft.Media media services assets tracks operation result\" }\r\n ,\"microsoft.media/mediaservices/assets/tracks/operationstatuses\": { \"SingularDisplayName\": \"Microsoft.Media media services assets tracks operation statuse\" }\r\n ,\"microsoft.media/mediaservices/contentkeypolicies\": { \"SingularDisplayName\": \"Microsoft.Media media services content key policy\" }\r\n ,\"microsoft.media/mediaservices/liveevents\": { \"SingularDisplayName\": \"Live event\" }\r\n ,\"microsoft.media/mediaservices/liveevents/liveoutputs\": { \"SingularDisplayName\": \"Microsoft.Media mediaservices live events live output\" }\r\n ,\"microsoft.media/mediaservices/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.Media mediaservices private endpoint connection\" }\r\n ,\"microsoft.media/mediaservices/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.Media mediaservices private link resource\" }\r\n ,\"microsoft.media/mediaservices/streamingendpoints\": { \"SingularDisplayName\": \"Streaming Endpoint\" }\r\n ,\"microsoft.media/mediaservices/streaminglocators\": { \"SingularDisplayName\": \"Microsoft.Media media services streaming locator\" }\r\n ,\"microsoft.media/mediaservices/streamingpolicies\": { \"SingularDisplayName\": \"Microsoft.Media media services streaming policy\" }\r\n ,\"microsoft.media/mediaservices/transforms\": { \"SingularDisplayName\": \"Microsoft.Media media services transform\" }\r\n ,\"microsoft.media/mediaservices/transforms/jobs\": { \"SingularDisplayName\": \"Microsoft.Media media services transforms job\" }\r\n ,\"microsoft.mesh/worlds\": { \"SingularDisplayName\": \"Microsoft.Mesh world\" }\r\n ,\"microsoft.mesh/worlds/events\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds event\" }\r\n ,\"microsoft.mesh/worlds/events/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds events access policy\" }\r\n ,\"microsoft.mesh/worlds/spaces\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds space\" }\r\n ,\"microsoft.mesh/worlds/spaces/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds spaces access policy\" }\r\n ,\"microsoft.mesh/worlds/templates\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds template\" }\r\n ,\"microsoft.mesh/worlds/templates/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds templates access policy\" }\r\n ,\"microsoft.messagingcatalog/catalogs\": { \"SingularDisplayName\": \"Microsoft.MessagingCatalog catalog\" }\r\n ,\"microsoft.messagingconnectors/connectors\": { \"SingularDisplayName\": \"Microsoft.MessagingConnectors connector\" }\r\n ,\"microsoft.metaverse/metaverses\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverse\" }\r\n ,\"microsoft.metaverse/metaverses/events\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses event\" }\r\n ,\"microsoft.metaverse/metaverses/events/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses events access policy\" }\r\n ,\"microsoft.metaverse/metaverses/spaces\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses space\" }\r\n ,\"microsoft.metaverse/metaverses/spaces/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses spaces access policy\" }\r\n ,\"microsoft.metaverse/metaverses/templates\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses template\" }\r\n ,\"microsoft.metaverse/metaverses/templates/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses templates access policy\" }\r\n ,\"microsoft.migrate/assessmentprojects\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment project\" }\r\n ,\"microsoft.migrate/assessmentprojects/aksassessmentoptions\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects aks assessment option\" }\r\n ,\"microsoft.migrate/assessmentprojects/aksassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects aks assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/aksassessments/assessedwebapps\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects aks assessments assessed web app\" }\r\n ,\"microsoft.migrate/assessmentprojects/aksassessments/clusters\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects aks assessments cluster\" }\r\n ,\"microsoft.migrate/assessmentprojects/aksassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects aks assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/assessmentoptions\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects assessment option\" }\r\n ,\"microsoft.migrate/assessmentprojects/assessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/assessments/assessedmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects assessments assessed machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/assessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/avsassessmentoptions\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects avs assessment option\" }\r\n ,\"microsoft.migrate/assessmentprojects/avsassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects avs assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/avsassessments/avsassessedmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects avs assessments avs assessed machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/avsassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects avs assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business case\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/avssummaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases avs summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/evaluatedavsmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases evaluated avs machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/evaluatedmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases evaluated machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/evaluatedsqlentities\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases evaluated sql entity\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/evaluatedwebapps\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases evaluated web app\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/iaassummaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases iaas summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/overviewsummaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases overview summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/paassummaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases paas summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects group\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/assessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/assessments/assessedmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups assessments assessed machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/avsassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups avs assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/avsassessments/avsassessedmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups avs assessments avs assessed machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments/assessedsqldatabases\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessments assessed sql database\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments/assessedsqlinstances\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessments assessed sql instance\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments/assessedsqlmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessments assessed sql machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments/recommendedassessedentities\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessments recommended assessed entity\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/webappassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups web app assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/webappassessments/assessedwebapps\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups web app assessments assessed web app\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/webappassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups web app assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/webappassessments/webappserviceplans\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups web app assessments web app service plan\" }\r\n ,\"microsoft.migrate/assessmentprojects/heterogeneousassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects heterogeneous assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/heterogeneousassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects heterogeneous assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/hypervcollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects hypervcollector\" }\r\n ,\"microsoft.migrate/assessmentprojects/importcollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects importcollector\" }\r\n ,\"microsoft.migrate/assessmentprojects/importsqlcollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects import sql collector\" }\r\n ,\"microsoft.migrate/assessmentprojects/machines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects private endpoint connection\" }\r\n ,\"microsoft.migrate/assessmentprojects/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects private link resource\" }\r\n ,\"microsoft.migrate/assessmentprojects/projectsummary\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects project summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/servercollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects servercollector\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessmentoptions\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessment option\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessments/assessedsqldatabases\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessments assessed sql database\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessments/assessedsqlinstances\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessments assessed sql instance\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessments/assessedsqlmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessments assessed sql machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlcollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sqlcollector\" }\r\n ,\"microsoft.migrate/assessmentprojects/vmwarecollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects vmwarecollector\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappassessmentoptions\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app assessment option\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappassessments/assessedwebapps\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app assessments assessed web app\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappassessments/webappserviceplans\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app assessments web app service plan\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappcollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app collector\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappcompoundassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app compound assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappcompoundassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app compound assessments summary\" }\r\n ,\"microsoft.migrate/migrateprojects\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate project\" }\r\n ,\"microsoft.migrate/migrateprojects/databaseinstances\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate projects database instance\" }\r\n ,\"microsoft.migrate/migrateprojects/databases\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate projects database\" }\r\n ,\"microsoft.migrate/migrateprojects/machines\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate projects machine\" }\r\n ,\"microsoft.migrate/migrateprojects/migrateevents\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate projects migrate event\" }\r\n ,\"microsoft.migrate/migrateprojects/solutions\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate projects solution\" }\r\n ,\"microsoft.migrate/modernizeprojects\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize project\" }\r\n ,\"microsoft.migrate/modernizeprojects/deployedresources\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects deployed resource\" }\r\n ,\"microsoft.migrate/modernizeprojects/jobs\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects job\" }\r\n ,\"microsoft.migrate/modernizeprojects/jobs/operations\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects jobs operation\" }\r\n ,\"microsoft.migrate/modernizeprojects/migrateagents\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects migrate agent\" }\r\n ,\"microsoft.migrate/modernizeprojects/migrateagents/operations\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects migrate agents operation\" }\r\n ,\"microsoft.migrate/modernizeprojects/operations\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects operation\" }\r\n ,\"microsoft.migrate/modernizeprojects/workloaddeployments\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects workload deployment\" }\r\n ,\"microsoft.migrate/modernizeprojects/workloaddeployments/operations\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects workload deployments operation\" }\r\n ,\"microsoft.migrate/modernizeprojects/workloadinstances\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects workload instance\" }\r\n ,\"microsoft.migrate/modernizeprojects/workloadinstances/operations\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects workload instances operation\" }\r\n ,\"microsoft.migrate/movecollections\": { \"SingularDisplayName\": \"Microsoft.Migrate move collection\" }\r\n ,\"microsoft.migrate/movecollections/moveresources\": { \"SingularDisplayName\": \"Microsoft.Migrate move collections move resource\" }\r\n ,\"microsoft.migrate/projects\": { \"SingularDisplayName\": \"Migration project\" }\r\n ,\"microsoft.mission/approvals\": { \"SingularDisplayName\": \"Approval\" }\r\n ,\"microsoft.mission/catalogs\": { \"SingularDisplayName\": \"Catalog\" }\r\n ,\"microsoft.mission/communities\": { \"SingularDisplayName\": \"Community\" }\r\n ,\"microsoft.mission/communities/communityendpoints\": { \"SingularDisplayName\": \"Community endpoint\" }\r\n ,\"microsoft.mission/communities/transithubs\": { \"SingularDisplayName\": \"Transit hub\" }\r\n ,\"microsoft.mission/enclaveconnections\": { \"SingularDisplayName\": \"Enclave connection\" }\r\n ,\"microsoft.mission/externalconnections\": { \"SingularDisplayName\": \"Microsoft.Mission external connection\" }\r\n ,\"microsoft.mission/internalconnections\": { \"SingularDisplayName\": \"Microsoft.Mission internal connection\" }\r\n ,\"microsoft.mission/virtualenclaves\": { \"SingularDisplayName\": \"Enclave\" }\r\n ,\"microsoft.mission/virtualenclaves/enclaveendpoints\": { \"SingularDisplayName\": \"Enclave endpoint\" }\r\n ,\"microsoft.mission/virtualenclaves/endpoints\": { \"SingularDisplayName\": \"Endpoint\" }\r\n ,\"microsoft.mission/virtualenclaves/workloads\": { \"SingularDisplayName\": \"Workload\" }\r\n ,\"microsoft.mixedreality/objectanchorsaccounts\": { \"SingularDisplayName\": \"Object Anchors Account\" }\r\n ,\"microsoft.mixedreality/objectunderstandingaccounts\": { \"SingularDisplayName\": \"Object Understanding Account\" }\r\n ,\"microsoft.mixedreality/remoterenderingaccounts\": { \"SingularDisplayName\": \"Remote Rendering Account\" }\r\n ,\"microsoft.mixedreality/spatialanchorsaccounts\": { \"SingularDisplayName\": \"Spatial Anchors Account\" }\r\n ,\"microsoft.mixedreality/spatialmapsaccounts\": { \"SingularDisplayName\": \"Microsoft.MixedReality spatial maps account\" }\r\n ,\"microsoft.mobilenetwork/amfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork amf deployment\" }\r\n ,\"microsoft.mobilenetwork/clusterservices\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork cluster service\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks\": { \"SingularDisplayName\": \"Mobile Network\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks/datanetworks\": { \"SingularDisplayName\": \"Data Network\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks/services\": { \"SingularDisplayName\": \"Service\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks/simpolicies\": { \"SingularDisplayName\": \"SIM Policy\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks/sites\": { \"SingularDisplayName\": \"Mobile Network Site\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks/slices\": { \"SingularDisplayName\": \"Slice\" }\r\n ,\"microsoft.mobilenetwork/nrfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork nrf deployment\" }\r\n ,\"microsoft.mobilenetwork/nssfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork nssf deployment\" }\r\n ,\"microsoft.mobilenetwork/observabilityservices\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork observability service\" }\r\n ,\"microsoft.mobilenetwork/packetcorecontrolplanes\": { \"SingularDisplayName\": \"Packet Core Control Plane\" }\r\n ,\"microsoft.mobilenetwork/packetcorecontrolplanes/packetcoredataplanes\": { \"SingularDisplayName\": \"Packet Core Data Plane\" }\r\n ,\"microsoft.mobilenetwork/packetcorecontrolplanes/packetcoredataplanes/attacheddatanetworks\": { \"SingularDisplayName\": \"Attached Data Network\" }\r\n ,\"microsoft.mobilenetwork/packetcorecontrolplaneversions\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork packet core control plane version\" }\r\n ,\"microsoft.mobilenetwork/radioaccessnetworks\": { \"SingularDisplayName\": \"Radio Access Network Insights\" }\r\n ,\"microsoft.mobilenetwork/sdmdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork sdm deployment\" }\r\n ,\"microsoft.mobilenetwork/simgroups\": { \"SingularDisplayName\": \"SIM Group\" }\r\n ,\"microsoft.mobilenetwork/simgroups/sims\": { \"SingularDisplayName\": \"SIM\" }\r\n ,\"microsoft.mobilenetwork/sims\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork sim\" }\r\n ,\"microsoft.mobilenetwork/smfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork smf deployment\" }\r\n ,\"microsoft.mobilenetwork/upfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork upf deployment\" }\r\n ,\"microsoft.mobilenetwork/virtualizedmmedeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork virtualized mme deployment\" }\r\n ,\"microsoft.mobilenetwork/vnfagentdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork vnf agent deployment\" }\r\n ,\"microsoft.mobilepacketcore/amfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore amf deployment\" }\r\n ,\"microsoft.mobilepacketcore/clusterservices\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore cluster service\" }\r\n ,\"microsoft.mobilepacketcore/networkfunctions\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore network function\" }\r\n ,\"microsoft.mobilepacketcore/nrfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore nrf deployment\" }\r\n ,\"microsoft.mobilepacketcore/nssfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore nssf deployment\" }\r\n ,\"microsoft.mobilepacketcore/observabilityservices\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore observability service\" }\r\n ,\"microsoft.mobilepacketcore/smfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore smf deployment\" }\r\n ,\"microsoft.mobilepacketcore/upfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore upf deployment\" }\r\n ,\"microsoft.modsimworkbench/workbenches\": { \"SingularDisplayName\": \"Modeling and Simulation Workbench\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers\": { \"SingularDisplayName\": \"Chamber\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/connectors\": { \"SingularDisplayName\": \"Chamber Connector\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/filerequests\": { \"SingularDisplayName\": \"Chamber Data Pipeline File Request\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/files\": { \"SingularDisplayName\": \"Chamber Data Pipeline File\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/licenses\": { \"SingularDisplayName\": \"Chamber License\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/storages\": { \"SingularDisplayName\": \"Chamber Storage\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/workloads\": { \"SingularDisplayName\": \"Chamber VM\" }\r\n ,\"microsoft.modsimworkbench/workbenches/sharedstorages\": { \"SingularDisplayName\": \"Shared Storage\" }\r\n ,\"microsoft.monitor/accounts\": { \"SingularDisplayName\": \"Azure Monitor workspace\" }\r\n ,\"microsoft.monitor/investigations\": { \"SingularDisplayName\": \"Microsoft.Monitor investigation\" }\r\n ,\"microsoft.monitor/pipelinegroups\": { \"SingularDisplayName\": \"Azure Monitor pipeline\" }\r\n ,\"microsoft.mysqldiscovery/mysqlsites\": { \"SingularDisplayName\": \"Microsoft.MySQLDiscovery my sqlsite\" }\r\n ,\"microsoft.mysqldiscovery/mysqlsites/agents\": { \"SingularDisplayName\": \"Microsoft.MySQLDiscovery my sqlsites agent\" }\r\n ,\"microsoft.mysqldiscovery/mysqlsites/errorsummaries\": { \"SingularDisplayName\": \"Microsoft.MySQLDiscovery my sqlsites error summary\" }\r\n ,\"microsoft.mysqldiscovery/mysqlsites/mysqlservers\": { \"SingularDisplayName\": \"Microsoft.MySQLDiscovery my sqlsites my sqlserver\" }\r\n ,\"microsoft.mysqldiscovery/mysqlsites/summaries\": { \"SingularDisplayName\": \"Microsoft.MySQLDiscovery my sqlsites summary\" }\r\n ,\"microsoft.netapp/netappaccounts\": { \"SingularDisplayName\": \"NetApp account\" }\r\n ,\"microsoft.netapp/netappaccounts/backuppolicies\": { \"SingularDisplayName\": \"Backup Policy\" }\r\n ,\"microsoft.netapp/netappaccounts/backupvaults\": { \"SingularDisplayName\": \"Backup vault\" }\r\n ,\"microsoft.netapp/netappaccounts/capacitypools\": { \"SingularDisplayName\": \"Capacity pool\" }\r\n ,\"microsoft.netapp/netappaccounts/capacitypools/volumes\": { \"SingularDisplayName\": \"Volume\" }\r\n ,\"microsoft.netapp/netappaccounts/capacitypools/volumes/snapshots\": { \"SingularDisplayName\": \"Snapshot\" }\r\n ,\"microsoft.netapp/netappaccounts/capacitypools/volumes/volumequotarules\": { \"SingularDisplayName\": \"User and group quota\" }\r\n ,\"microsoft.netapp/netappaccounts/snapshotpolicies\": { \"SingularDisplayName\": \"Snapshot policy\" }\r\n ,\"microsoft.netapp/netappaccounts/volumegroups\": { \"SingularDisplayName\": \"VolumeGroup\" }\r\n ,\"microsoft.network/applicationgatewayavailablessloptions\": { \"SingularDisplayName\": \"Microsoft.Network application gateway available ssl option\" }\r\n ,\"microsoft.network/applicationgatewayavailablessloptions/predefinedpolicies\": { \"SingularDisplayName\": \"Microsoft.Network application gateway available ssl options predefined policy\" }\r\n ,\"microsoft.network/applicationgateways\": { \"SingularDisplayName\": \"Application gateway\" }\r\n ,\"microsoft.network/applicationgatewaywebapplicationfirewallpolicies\": { \"SingularDisplayName\": \"Application Gateway WAF policy\" }\r\n ,\"microsoft.network/applicationsecuritygroups\": { \"SingularDisplayName\": \"Application security group\" }\r\n ,\"microsoft.network/azurefirewalls\": { \"SingularDisplayName\": \"Firewall\" }\r\n ,\"microsoft.network/azurewebcategories\": { \"SingularDisplayName\": \"Microsoft.Network Azure web category\" }\r\n ,\"microsoft.network/bastionhosts\": { \"SingularDisplayName\": \"Bastion\" }\r\n ,\"microsoft.network/cloudserviceslots\": { \"SingularDisplayName\": \"Microsoft.Network cloud service slot\" }\r\n ,\"microsoft.network/connections\": { \"SingularDisplayName\": \"Connection\" }\r\n ,\"microsoft.network/customipprefixes\": { \"SingularDisplayName\": \"Custom IP Prefix\" }\r\n ,\"microsoft.network/ddoscustompolicies\": { \"SingularDisplayName\": \"Microsoft.Network DDoS custom policy\" }\r\n ,\"microsoft.network/ddosprotectionplans\": { \"SingularDisplayName\": \"DDoS protection plan\" }\r\n ,\"microsoft.network/dnsforwardingrulesets\": { \"SingularDisplayName\": \"DNS forwarding ruleset\" }\r\n ,\"microsoft.network/dnsresolverdomainlists\": { \"SingularDisplayName\": \"DNS Domain List\" }\r\n ,\"microsoft.network/dnsresolverpolicies\": { \"SingularDisplayName\": \"DNS Security Policy\" }\r\n ,\"microsoft.network/dnsresolvers\": { \"SingularDisplayName\": \"DNS private resolver\" }\r\n ,\"microsoft.network/dnszones\": { \"SingularDisplayName\": \"DNS zone\" }\r\n ,\"microsoft.network/dscpconfigurations\": { \"SingularDisplayName\": \"Microsoft.Network DSCP configuration\" }\r\n ,\"microsoft.network/expressroutecircuits\": { \"SingularDisplayName\": \"ExpressRoute circuit\" }\r\n ,\"microsoft.network/expressroutecrossconnections\": { \"SingularDisplayName\": \"Microsoft.Network express route cross connection\" }\r\n ,\"microsoft.network/expressroutecrossconnections/peerings\": { \"SingularDisplayName\": \"Microsoft.Network express route cross connections peering\" }\r\n ,\"microsoft.network/expressroutegateways\": { \"SingularDisplayName\": \"ExpressRoute Gateway\" }\r\n ,\"microsoft.network/expressroutegateways/expressrouteconnections\": { \"SingularDisplayName\": \"Microsoft.Network express route gateways express route connection\" }\r\n ,\"microsoft.network/expressrouteports\": { \"SingularDisplayName\": \"ExpressRoute Direct\" }\r\n ,\"microsoft.network/expressrouteportslocations\": { \"SingularDisplayName\": \"Microsoft.Network express route ports location\" }\r\n ,\"microsoft.network/firewallpolicies\": { \"SingularDisplayName\": \"Firewall Policy\" }\r\n ,\"microsoft.network/frontdoors\": { \"SingularDisplayName\": \"Front Door and CDN profiles\" }\r\n ,\"microsoft.network/frontdoorwebapplicationfirewallpolicies\": { \"SingularDisplayName\": \"Front Door WAF policy\" }\r\n ,\"microsoft.network/ipallocations\": { \"SingularDisplayName\": \"Microsoft.Network IP allocation\" }\r\n ,\"microsoft.network/ipgroups\": { \"SingularDisplayName\": \"IP Group\" }\r\n ,\"microsoft.network/loadbalancers\": { \"SingularDisplayName\": \"Load balancer\" }\r\n ,\"microsoft.network/localnetworkgateways\": { \"SingularDisplayName\": \"Local network gateway\" }\r\n ,\"microsoft.network/natgateways\": { \"SingularDisplayName\": \"NAT gateway\" }\r\n ,\"microsoft.network/networkexperimentprofiles\": { \"SingularDisplayName\": \"Microsoft.Network network experiment profile\" }\r\n ,\"microsoft.network/networkexperimentprofiles/experiments\": { \"SingularDisplayName\": \"Microsoft.Network network experiment profiles experiment\" }\r\n ,\"microsoft.network/networkinterfaces\": { \"SingularDisplayName\": \"Network interface\" }\r\n ,\"microsoft.network/networkmanagerconnections\": { \"SingularDisplayName\": \"Microsoft.Network network manager connection\" }\r\n ,\"microsoft.network/networkmanagers\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/connectivityconfigurations\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/ipampools\": { \"SingularDisplayName\": \"IP address pool\" }\r\n ,\"microsoft.network/networkmanagers/networkgroups\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/routingconfigurations\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/securityadminconfigurations\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/securityuserconfigurations\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/verifierworkspaces\": { \"SingularDisplayName\": \"Verifier Workspace\" }\r\n ,\"microsoft.network/networkprofiles\": { \"SingularDisplayName\": \"Microsoft.Network network profile\" }\r\n ,\"microsoft.network/networksecuritygroups\": { \"SingularDisplayName\": \"Network security group\" }\r\n ,\"microsoft.network/networksecurityperimeters\": { \"SingularDisplayName\": \"Network Security Perimeter\" }\r\n ,\"microsoft.network/networksecurityperimeters/profiles\": { \"SingularDisplayName\": \"Network Security Perimeter Profile\" }\r\n ,\"microsoft.network/networkverifiers\": { \"SingularDisplayName\": \"Virtual Network Verifier\" }\r\n ,\"microsoft.network/networkvirtualappliances\": { \"SingularDisplayName\": \"Microsoft.Network network virtual appliance\" }\r\n ,\"microsoft.network/networkwatchers\": { \"SingularDisplayName\": \"Network Watcher\" }\r\n ,\"microsoft.network/networkwatchers/flowlogs\": { \"SingularDisplayName\": \"Flow log\" }\r\n ,\"microsoft.network/p2svpngateways\": { \"SingularDisplayName\": \"VPN Gateway (Point to Site)\" }\r\n ,\"microsoft.network/privatednszones\": { \"SingularDisplayName\": \"Private DNS zone\" }\r\n ,\"microsoft.network/privatednszones/virtualnetworklinks\": { \"SingularDisplayName\": \"Virtual network link\" }\r\n ,\"microsoft.network/privateendpoints\": { \"SingularDisplayName\": \"Private endpoint\" }\r\n ,\"microsoft.network/privatelinkservices\": { \"SingularDisplayName\": \"Private link service\" }\r\n ,\"microsoft.network/publicipaddresses\": { \"SingularDisplayName\": \"Public IP address\" }\r\n ,\"microsoft.network/publicipprefixes\": { \"SingularDisplayName\": \"Public IP Prefix\" }\r\n ,\"microsoft.network/routefilters\": { \"SingularDisplayName\": \"Route filter\" }\r\n ,\"microsoft.network/routetables\": { \"SingularDisplayName\": \"Route table\" }\r\n ,\"microsoft.network/securitypartnerproviders\": { \"SingularDisplayName\": \"Microsoft.Network security partner provider\" }\r\n ,\"microsoft.network/serviceendpointpolicies\": { \"SingularDisplayName\": \"Service endpoint policy\" }\r\n ,\"microsoft.network/trafficmanagergeographichierarchies\": { \"SingularDisplayName\": \"Microsoft.Network traffic manager geographic hierarchy\" }\r\n ,\"microsoft.network/trafficmanagerprofiles\": { \"SingularDisplayName\": \"Traffic Manager profile\" }\r\n ,\"microsoft.network/trafficmanagerusermetricskeys\": { \"SingularDisplayName\": \"Microsoft.Network traffic manager user metrics key\" }\r\n ,\"microsoft.network/virtualhubs\": { \"SingularDisplayName\": \"Microsoft.Network/virtualHub\" }\r\n ,\"microsoft.network/virtualnetworkgateways\": { \"SingularDisplayName\": \"Virtual network gateway\" }\r\n ,\"microsoft.network/virtualnetworks\": { \"SingularDisplayName\": \"Virtual network\" }\r\n ,\"microsoft.network/virtualnetworktaps\": { \"SingularDisplayName\": \"Virtual network terminal access point\" }\r\n ,\"microsoft.network/virtualrouters\": { \"SingularDisplayName\": \"Microsoft.Network virtual router\" }\r\n ,\"microsoft.network/virtualrouters/peerings\": { \"SingularDisplayName\": \"Microsoft.Network virtual routers peering\" }\r\n ,\"microsoft.network/virtualwans\": { \"SingularDisplayName\": \"Virtual WAN\" }\r\n ,\"microsoft.network/vpngateways\": { \"SingularDisplayName\": \"VPN Gateway (Site to Site)\" }\r\n ,\"microsoft.network/vpngateways/vpnconnections\": { \"SingularDisplayName\": \"Microsoft.Network VPN gateways VPN connection\" }\r\n ,\"microsoft.network/vpngateways/vpnconnections/vpnlinkconnections\": { \"SingularDisplayName\": \"Microsoft.Network VPN gateways VPN connections VPN link connection\" }\r\n ,\"microsoft.network/vpnserverconfigurations\": { \"SingularDisplayName\": \"Microsoft.Network VPN server configuration\" }\r\n ,\"microsoft.network/vpnsites\": { \"SingularDisplayName\": \"Microsoft.Network VPN site\" }\r\n ,\"microsoft.network/vpnsites/vpnsitelinks\": { \"SingularDisplayName\": \"Microsoft.Network VPN sites VPN site link\" }\r\n ,\"microsoft.networkanalytics/dataconnectors\": { \"SingularDisplayName\": \"AIOps - Data Connector\" }\r\n ,\"microsoft.networkanalytics/datalakehouses\": { \"SingularDisplayName\": \"AIOps - Data LakeHouse\" }\r\n ,\"microsoft.networkanalytics/dataproducts\": { \"SingularDisplayName\": \"Azure Operator Insights ? Data Product\" }\r\n ,\"microsoft.networkanalytics/dataproducts/datatypes\": { \"SingularDisplayName\": \"Data Type\" }\r\n ,\"microsoft.networkanalytics/dataproductscatalogs\": { \"SingularDisplayName\": \"Microsoft.NetworkAnalytics data products catalog\" }\r\n ,\"microsoft.networkanalytics/metricsingestionendpoints\": { \"SingularDisplayName\": \"Microsoft.NetworkAnalytics metrics ingestion endpoint\" }\r\n ,\"microsoft.networkanalytics/networkanalyticsproducts\": { \"SingularDisplayName\": \"Microsoft.NetworkAnalytics network analytics product\" }\r\n ,\"microsoft.networkcloud/baremetalmachines\": { \"SingularDisplayName\": \"Bare Metal Machine (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/cloudservicesnetworks\": { \"SingularDisplayName\": \"Cloud Services Network (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/clustermanagers\": { \"SingularDisplayName\": \"Cluster Manager (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/clusters\": { \"SingularDisplayName\": \"Cluster (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/clusters/baremetalmachinekeysets\": { \"SingularDisplayName\": \"Cluster Bare Metal Machine Key Set (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/clusters/bmckeysets\": { \"SingularDisplayName\": \"Cluster Baseboard Management Controller Key Set (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/clusters/metricsconfigurations\": { \"SingularDisplayName\": \"Cluster Metrics Configuration (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/edgeclustermachineskus\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud edge cluster machine SKU\" }\r\n ,\"microsoft.networkcloud/edgeclusterruntimeversions\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud edge cluster runtime version\" }\r\n ,\"microsoft.networkcloud/edgeclusters\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud edge cluster\" }\r\n ,\"microsoft.networkcloud/edgeclusters/nodes\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud edge clusters node\" }\r\n ,\"microsoft.networkcloud/edgeclusterskus\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud edge cluster SKU\" }\r\n ,\"microsoft.networkcloud/kubernetesclusters\": { \"SingularDisplayName\": \"Kubernetes Cluster (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/kubernetesclusters/agentpools\": { \"SingularDisplayName\": \"Agent Pool (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/kubernetesclusters/features\": { \"SingularDisplayName\": \"Kubernetes Cluster Feature (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/l2networks\": { \"SingularDisplayName\": \"Layer 2 Network (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/l3networks\": { \"SingularDisplayName\": \"Layer 3 Network (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/racks\": { \"SingularDisplayName\": \"Compute Rack (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/rackskus\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud rack SKU\" }\r\n ,\"microsoft.networkcloud/registrationhubs\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud registration hub\" }\r\n ,\"microsoft.networkcloud/registrationhubs/images\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud registration hubs image\" }\r\n ,\"microsoft.networkcloud/registrationhubs/machines\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud registration hubs machine\" }\r\n ,\"microsoft.networkcloud/storageappliances\": { \"SingularDisplayName\": \"Storage Appliance (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/trunkednetworks\": { \"SingularDisplayName\": \"Trunked Network (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/virtualmachines\": { \"SingularDisplayName\": \"Virtual Machine (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/virtualmachines/consoles\": { \"SingularDisplayName\": \"Virtual Machine Console (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/volumes\": { \"SingularDisplayName\": \"Volume (Operator Nexus)\" }\r\n ,\"microsoft.networkfunction/azuretrafficcollectors\": { \"SingularDisplayName\": \"ExpressRoute traffic collector\" }\r\n ,\"microsoft.networkfunction/meshvpns\": { \"SingularDisplayName\": \"Mesh VPN\" }\r\n ,\"microsoft.nexusidentity/identitycontrollers\": { \"SingularDisplayName\": \"Microsoft.NexusIdentity identity controller\" }\r\n ,\"microsoft.nexusidentity/identitysets\": { \"SingularDisplayName\": \"Microsoft.NexusIdentity identity set\" }\r\n ,\"microsoft.notebooks/notebookproxies\": { \"SingularDisplayName\": \"Microsoft.Notebooks notebook proxy\" }\r\n ,\"microsoft.notificationhubs/namespaces\": { \"SingularDisplayName\": \"Notification Hub Namespace\" }\r\n ,\"microsoft.notificationhubs/namespaces/notificationhubs\": { \"SingularDisplayName\": \"Notification Hub\" }\r\n ,\"microsoft.objectstore/osnamespaces\": { \"SingularDisplayName\": \"Microsoft.ObjectStore os namespace\" }\r\n })[tolower(id)]\r\n}\r\n", + "$fxv#3": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData/Internal')\r\n_resource_type_4(id: string) {\r\n dynamic({\r\n \"microsoft.offazure/hypervsites\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv site\" }\r\n ,\"microsoft.offazure/hypervsites/clusters\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites cluster\" }\r\n ,\"microsoft.offazure/hypervsites/hosts\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites host\" }\r\n ,\"microsoft.offazure/hypervsites/jobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites job\" }\r\n ,\"microsoft.offazure/hypervsites/machines\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites machine\" }\r\n ,\"microsoft.offazure/hypervsites/machines/softwareinventories\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites machines software inventory\" }\r\n ,\"microsoft.offazure/hypervsites/operationsstatus\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites operations statu\" }\r\n ,\"microsoft.offazure/hypervsites/runasaccounts\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites run as account\" }\r\n ,\"microsoft.offazure/importsites\": { \"SingularDisplayName\": \"Microsoft.OffAzure import site\" }\r\n ,\"microsoft.offazure/importsites/deletejobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure import sites delete job\" }\r\n ,\"microsoft.offazure/importsites/exportjobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure import sites export job\" }\r\n ,\"microsoft.offazure/importsites/importjobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure import sites import job\" }\r\n ,\"microsoft.offazure/importsites/jobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure import sites job\" }\r\n ,\"microsoft.offazure/importsites/machines\": { \"SingularDisplayName\": \"Microsoft.OffAzure import sites machine\" }\r\n ,\"microsoft.offazure/mastersites\": { \"SingularDisplayName\": \"Microsoft.OffAzure master site\" }\r\n ,\"microsoft.offazure/mastersites/operationsstatus\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites operations statu\" }\r\n ,\"microsoft.offazure/mastersites/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites private endpoint connection\" }\r\n ,\"microsoft.offazure/mastersites/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites private link resource\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql site\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/discoverysitedatasources\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites discovery site data source\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/jobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites job\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/operationsstatus\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites operations statu\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/runasaccounts\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites run as account\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/sqlavailabilitygroups\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites sql availability group\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/sqldatabases\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites sql database\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/sqlservers\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites sql server\" }\r\n ,\"microsoft.offazure/mastersites/webappsites\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app site\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/discoverysitedatasources\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites discovery site data source\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/extendedmachines\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites extended machine\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/iiswebapplications\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites iis web application\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/iiswebservers\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites iis web server\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/runasaccounts\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites runasaccount\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/tomcatwebapplications\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites tomcat web application\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/tomcatwebservers\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites tomcat web server\" }\r\n ,\"microsoft.offazure/serversites\": { \"SingularDisplayName\": \"Microsoft.OffAzure server site\" }\r\n ,\"microsoft.offazure/serversites/jobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure server sites job\" }\r\n ,\"microsoft.offazure/serversites/machines\": { \"SingularDisplayName\": \"Microsoft.OffAzure server sites machine\" }\r\n ,\"microsoft.offazure/serversites/machines/softwareinventories\": { \"SingularDisplayName\": \"Microsoft.OffAzure server sites machines software inventory\" }\r\n ,\"microsoft.offazure/serversites/operationsstatus\": { \"SingularDisplayName\": \"Microsoft.OffAzure server sites operations statu\" }\r\n ,\"microsoft.offazure/serversites/runasaccounts\": { \"SingularDisplayName\": \"Microsoft.OffAzure server sites run as account\" }\r\n ,\"microsoft.offazure/vmwaresites\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware site\" }\r\n ,\"microsoft.offazure/vmwaresites/hosts\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites host\" }\r\n ,\"microsoft.offazure/vmwaresites/jobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites job\" }\r\n ,\"microsoft.offazure/vmwaresites/machines\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites machine\" }\r\n ,\"microsoft.offazure/vmwaresites/machines/softwareinventories\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites machines software inventory\" }\r\n ,\"microsoft.offazure/vmwaresites/operationsstatus\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites operations statu\" }\r\n ,\"microsoft.offazure/vmwaresites/runasaccounts\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites run as account\" }\r\n ,\"microsoft.offazure/vmwaresites/vcenters\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites vcenter\" }\r\n ,\"microsoft.offazurespringboot/springbootsites\": { \"SingularDisplayName\": \"Microsoft.OffAzureSpringBoot springbootsite\" }\r\n ,\"microsoft.offazurespringboot/springbootsites/errorsummaries\": { \"SingularDisplayName\": \"Microsoft.OffAzureSpringBoot springbootsites error summary\" }\r\n ,\"microsoft.offazurespringboot/springbootsites/springbootapps\": { \"SingularDisplayName\": \"Microsoft.OffAzureSpringBoot springbootsites springbootapp\" }\r\n ,\"microsoft.offazurespringboot/springbootsites/springbootservers\": { \"SingularDisplayName\": \"Microsoft.OffAzureSpringBoot springbootsites springbootserver\" }\r\n ,\"microsoft.offazurespringboot/springbootsites/summaries\": { \"SingularDisplayName\": \"Microsoft.OffAzureSpringBoot springbootsites summary\" }\r\n ,\"microsoft.onlineexperimentation/workspaces\": { \"SingularDisplayName\": \"Online Experimentation Workspace\" }\r\n ,\"microsoft.openenergyplatform/energyservices\": { \"SingularDisplayName\": \"Azure Data Manager for Energy\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspace\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces/applicationregistrations\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspaces application registration\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces/applications\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspaces application\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces/eventgridfilters\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspaces event grid filter\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces/shares\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspaces share\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces/sharesubscriptions\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspaces share subscription\" }\r\n ,\"microsoft.operationalinsights/clusters\": { \"SingularDisplayName\": \"Log Analytics dedicated cluster\" }\r\n ,\"microsoft.operationalinsights/querypacks\": { \"SingularDisplayName\": \"Log Analytics query pack\" }\r\n ,\"microsoft.operationalinsights/workspaces\": { \"SingularDisplayName\": \"Log Analytics workspace\" }\r\n ,\"microsoft.operationsmanagement/managementassociations\": { \"SingularDisplayName\": \"Microsoft.OperationsManagement management association\" }\r\n ,\"microsoft.operationsmanagement/solutions\": { \"SingularDisplayName\": \"Solution\" }\r\n ,\"microsoft.operatorvoicemail/operatorvoicemailinstances\": { \"SingularDisplayName\": \"Microsoft.OperatorVoicemail operator voicemail instance\" }\r\n ,\"microsoft.oraclediscovery/oraclesites\": { \"SingularDisplayName\": \"Microsoft.OracleDiscovery oracle site\" }\r\n ,\"microsoft.oraclediscovery/oraclesites/errorsummaries\": { \"SingularDisplayName\": \"Microsoft.OracleDiscovery oracle sites error summary\" }\r\n ,\"microsoft.oraclediscovery/oraclesites/oracledatabases\": { \"SingularDisplayName\": \"Microsoft.OracleDiscovery oracle sites oracle database\" }\r\n ,\"microsoft.oraclediscovery/oraclesites/oracleservers\": { \"SingularDisplayName\": \"Microsoft.OracleDiscovery oracle sites oracle server\" }\r\n ,\"microsoft.oraclediscovery/oraclesites/summaries\": { \"SingularDisplayName\": \"Microsoft.OracleDiscovery oracle sites summary\" }\r\n ,\"microsoft.orbital/cloudaccessrouters\": { \"SingularDisplayName\": \"Cloud Access Router\" }\r\n ,\"microsoft.orbital/contactprofiles\": { \"SingularDisplayName\": \"Contact Profile\" }\r\n ,\"microsoft.orbital/edgesites\": { \"SingularDisplayName\": \"Edge Site\" }\r\n ,\"microsoft.orbital/geocatalogs\": { \"SingularDisplayName\": \"GeoCatalog\" }\r\n ,\"microsoft.orbital/globalcommunicationssites\": { \"SingularDisplayName\": \"Microsoft.Orbital global communications site\" }\r\n ,\"microsoft.orbital/groundstations\": { \"SingularDisplayName\": \"Ground Station\" }\r\n ,\"microsoft.orbital/l2connections\": { \"SingularDisplayName\": \"L2 Connection\" }\r\n ,\"microsoft.orbital/sdwancontrollers\": { \"SingularDisplayName\": \"SDWAN Controller\" }\r\n ,\"microsoft.orbital/spacecrafts\": { \"SingularDisplayName\": \"Spacecraft\" }\r\n ,\"microsoft.orbital/spacecrafts/contacts\": { \"SingularDisplayName\": \"Contact\" }\r\n ,\"microsoft.orbital/terminals\": { \"SingularDisplayName\": \"Cloud Access Terminal\" }\r\n ,\"microsoft.partnermanagedconsumerrecurrence/recurrences\": { \"SingularDisplayName\": \"Microsoft.PartnerManagedConsumerRecurrence recurrence\" }\r\n ,\"microsoft.partnermanagedconsumerrecurrence/recurrences/operationresult\": { \"SingularDisplayName\": \"Microsoft.PartnerManagedConsumerRecurrence recurrences operation result\" }\r\n ,\"microsoft.peering/peerasns\": { \"SingularDisplayName\": \"Microsoft.Peering peer asn\" }\r\n ,\"microsoft.peering/peerings\": { \"SingularDisplayName\": \"Peering\" }\r\n ,\"microsoft.peering/peerings/registeredasns\": { \"SingularDisplayName\": \"Registered ASN\" }\r\n ,\"microsoft.peering/peerings/registeredprefixes\": { \"SingularDisplayName\": \"Registered prefix\" }\r\n ,\"microsoft.peering/peeringservices\": { \"SingularDisplayName\": \"Peering Service\" }\r\n ,\"microsoft.peering/peeringservices/prefixes\": { \"SingularDisplayName\": \"Peering Service Prefix\" }\r\n ,\"microsoft.pki/pkis\": { \"SingularDisplayName\": \"Microsoft.Pki PKI\" }\r\n ,\"microsoft.pki/pkis/certificateauthorities\": { \"SingularDisplayName\": \"Microsoft.Pki pkis certificate authority\" }\r\n ,\"microsoft.pki/pkis/enrollmentpolicies\": { \"SingularDisplayName\": \"Microsoft.Pki pkis enrollment policy\" }\r\n ,\"microsoft.policyinsights/attestations\": { \"SingularDisplayName\": \"Microsoft.PolicyInsights attestation\" }\r\n ,\"microsoft.policyinsights/policymetadata\": { \"SingularDisplayName\": \"Microsoft.PolicyInsights policy metadata\" }\r\n ,\"microsoft.policyinsights/remediations\": { \"SingularDisplayName\": \"Microsoft.PolicyInsights remediation\" }\r\n ,\"microsoft.portal/consoles\": { \"SingularDisplayName\": \"Microsoft.Portal console\" }\r\n ,\"microsoft.portal/dashboards\": { \"SingularDisplayName\": \"Shared dashboard\" }\r\n ,\"microsoft.portal/tenantconfigurations\": { \"SingularDisplayName\": \"Microsoft.Portal tenant configuration\" }\r\n ,\"microsoft.portal/usersettings\": { \"SingularDisplayName\": \"Microsoft.Portal user setting\" }\r\n ,\"microsoft.portal/virtual-privatedashboards\": { \"SingularDisplayName\": \"Private dashboard\" }\r\n ,\"microsoft.portalservices/copilotsettings\": { \"SingularDisplayName\": \"Microsoft.PortalServices copilot setting\" }\r\n ,\"microsoft.portalservices/dashboards\": { \"SingularDisplayName\": \"Shared dashboard\" }\r\n ,\"microsoft.portalservices/extensions\": { \"SingularDisplayName\": \"Portal Extension\" }\r\n ,\"microsoft.portalservices/extensions/deployments\": { \"SingularDisplayName\": \"Extension Deployment\" }\r\n ,\"microsoft.portalservices/extensions/slots\": { \"SingularDisplayName\": \"Extension Slot\" }\r\n ,\"microsoft.portalservices/extensions/versions\": { \"SingularDisplayName\": \"Extension Version\" }\r\n ,\"microsoft.portalservices/settings\": { \"SingularDisplayName\": \"Microsoft.PortalServices setting\" }\r\n ,\"microsoft.powerbi/privatelinkservicesforpowerbi\": { \"SingularDisplayName\": \"Microsoft.PowerBI private link services for power bi\" }\r\n ,\"microsoft.powerbi/privatelinkservicesforpowerbi/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.PowerBI private link services for power bi private endpoint connection\" }\r\n ,\"microsoft.powerbi/privatelinkservicesforpowerbi/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.PowerBI private link services for power bi private link resource\" }\r\n ,\"microsoft.powerbi/workspacecollections\": { \"SingularDisplayName\": \"Microsoft.PowerBI workspace collection\" }\r\n ,\"microsoft.powerbidedicated/autoscalevcores\": { \"SingularDisplayName\": \"Microsoft.PowerBIDedicated auto scale vcore\" }\r\n ,\"microsoft.powerbidedicated/capacities\": { \"SingularDisplayName\": \"Power BI Embedded\" }\r\n ,\"microsoft.powerplatform/accounts\": { \"SingularDisplayName\": \"Microsoft.PowerPlatform account\" }\r\n ,\"microsoft.premonition/libraries\": { \"SingularDisplayName\": \"Microsoft.Premonition library\" }\r\n ,\"microsoft.premonition/libraries/analyses\": { \"SingularDisplayName\": \"Microsoft.Premonition libraries analyse\" }\r\n ,\"microsoft.premonition/libraries/samples\": { \"SingularDisplayName\": \"Microsoft.Premonition libraries sample\" }\r\n ,\"microsoft.professionalservice/resources\": { \"SingularDisplayName\": \"Professional Service\" }\r\n ,\"microsoft.programmableconnectivity/gateways\": { \"SingularDisplayName\": \"APC Gateway\" }\r\n ,\"microsoft.programmableconnectivity/operatorapiconnections\": { \"SingularDisplayName\": \"APC Operator API Connection\" }\r\n ,\"microsoft.programmableconnectivity/operatorapiplans\": { \"SingularDisplayName\": \"APC Operator API Plan\" }\r\n ,\"microsoft.proposal/proposals\": { \"SingularDisplayName\": \"Microsoft.Proposal proposal\" }\r\n ,\"microsoft.providerhub/providerregistrations\": { \"SingularDisplayName\": \"Resource Provider as a Service\" }\r\n ,\"microsoft.providerhub/providerregistrations/customrollouts\": { \"SingularDisplayName\": \"Rollout\" }\r\n ,\"microsoft.providerhub/providerregistrations/defaultrollouts\": { \"SingularDisplayName\": \"Rollout\" }\r\n ,\"microsoft.providerhub/providerregistrations/resourcetyperegistrations\": { \"SingularDisplayName\": \"Resource Type\" }\r\n ,\"microsoft.providerhub/providerregistrations/resourcetyperegistrations/resourcetyperegistrations\": { \"SingularDisplayName\": \"Resource Type\" }\r\n ,\"microsoft.providerhubdevtest/regionalstresstests\": { \"SingularDisplayName\": \"Microsoft.ProviderHubDevTest regional stresstest\" }\r\n ,\"microsoft.providerhubdevtest/stresstests\": { \"SingularDisplayName\": \"Microsoft.ProviderHubDevTest stresstest\" }\r\n ,\"microsoft.purview/accounts\": { \"SingularDisplayName\": \"Microsoft Purview account\" }\r\n ,\"microsoft.quantum/provideraccounts\": { \"SingularDisplayName\": \"Microsoft.Quantum provider account\" }\r\n ,\"microsoft.quantum/workspaces\": { \"SingularDisplayName\": \"Quantum Workspace\" }\r\n ,\"microsoft.quota/groupquotas\": { \"SingularDisplayName\": \"Microsoft.Quota group quota\" }\r\n ,\"microsoft.quota/groupquotas/groupquotarequests\": { \"SingularDisplayName\": \"Microsoft.Quota group quotas group quota request\" }\r\n ,\"microsoft.quota/groupquotas/quotaallocationrequests\": { \"SingularDisplayName\": \"Microsoft.Quota group quotas quota allocation request\" }\r\n ,\"microsoft.quota/groupquotas/quotaallocations\": { \"SingularDisplayName\": \"Microsoft.Quota group quotas quota allocation\" }\r\n ,\"microsoft.quota/groupquotas/subscriptionrequests\": { \"SingularDisplayName\": \"Microsoft.Quota group quotas subscription request\" }\r\n ,\"microsoft.quota/groupquotas/subscriptions\": { \"SingularDisplayName\": \"Microsoft.Quota group quotas subscription\" }\r\n ,\"microsoft.quota/quotarequests\": { \"SingularDisplayName\": \"Microsoft.Quota quota request\" }\r\n ,\"microsoft.quota/quotas\": { \"SingularDisplayName\": \"Microsoft.Quota quota\" }\r\n ,\"microsoft.quota/usages\": { \"SingularDisplayName\": \"Microsoft.Quota usage\" }\r\n ,\"microsoft.recommendationsservice/accounts\": { \"SingularDisplayName\": \"Intelligent Recommendations Account\" }\r\n ,\"microsoft.recommendationsservice/accounts/modeling\": { \"SingularDisplayName\": \"Modeling\" }\r\n ,\"microsoft.recommendationsservice/accounts/serviceendpoints\": { \"SingularDisplayName\": \"Service Endpoint\" }\r\n ,\"microsoft.recoveryservices/replicationeligibilityresults\": { \"SingularDisplayName\": \"Microsoft.RecoveryServices replication eligibility result\" }\r\n ,\"microsoft.recoveryservices/vaults\": { \"SingularDisplayName\": \"Recovery Services vault\" }\r\n ,\"microsoft.recoveryservices/vaults/backupfabrics/protectioncontainers/protecteditems\": { \"SingularDisplayName\": \"Backup Item\" }\r\n ,\"microsoft.recoveryservicesbvtd/vaults\": { \"SingularDisplayName\": \"Recovery Services BVTD\" }\r\n ,\"microsoft.recoveryservicesbvtd2/vaults\": { \"SingularDisplayName\": \"Recovery Services BVTD2\" }\r\n ,\"microsoft.recoveryservicesintd/vaults\": { \"SingularDisplayName\": \"Recovery Services INTD\" }\r\n ,\"microsoft.recoveryservicesintd2/vaults\": { \"SingularDisplayName\": \"Recovery Services INTD2\" }\r\n ,\"microsoft.redhatopenshift/openshiftclusters\": { \"SingularDisplayName\": \"Azure Red Hat OpenShift cluster\" }\r\n ,\"microsoft.relationships/dependencyof\": { \"SingularDisplayName\": \"Dependency Relationship\" }\r\n ,\"microsoft.relationships/servicegroupmember\": { \"SingularDisplayName\": \"Service group member relationship\" }\r\n ,\"microsoft.relationships/servicegrouprelationships\": { \"SingularDisplayName\": \"Connected Resource\" }\r\n ,\"microsoft.relay/namespaces\": { \"SingularDisplayName\": \"Relay\" }\r\n ,\"microsoft.relay/namespaces/hybridconnections\": { \"SingularDisplayName\": \"Hybrid connection\" }\r\n ,\"microsoft.relay/namespaces/wcfrelays\": { \"SingularDisplayName\": \"WCF relay\" }\r\n ,\"microsoft.resilience/resiliencestates\": { \"SingularDisplayName\": \"Microsoft.Resilience resilience state\" }\r\n ,\"microsoft.resourceconnector/appliances\": { \"SingularDisplayName\": \"Resource bridge\" }\r\n ,\"microsoft.resourcegraph/queries\": { \"SingularDisplayName\": \"Resource Graph query\" }\r\n ,\"microsoft.resourcehealth/availabilitystatuses\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth availability statuse\" }\r\n ,\"microsoft.resourcehealth/childavailabilitystatuses\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth child availability statuse\" }\r\n ,\"microsoft.resourcehealth/emergingissues\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth emerging issue\" }\r\n ,\"microsoft.resourcehealth/events\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth event\" }\r\n ,\"microsoft.resourcehealth/events/impactedresources\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth events impacted resource\" }\r\n ,\"microsoft.resourcehealth/metadata\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth metadata\" }\r\n ,\"microsoft.resources/builtintemplatespecs\": { \"SingularDisplayName\": \"Built-in template spec\" }\r\n ,\"microsoft.resources/changes\": { \"SingularDisplayName\": \"Microsoft.Resources change\" }\r\n ,\"microsoft.resources/databoundaries\": { \"SingularDisplayName\": \"Microsoft.Resources data boundary\" }\r\n ,\"microsoft.resources/deletedresources\": { \"SingularDisplayName\": \"Recycle Bin\" }\r\n ,\"microsoft.resources/deployments\": { \"SingularDisplayName\": \"Microsoft.Resources deployment\" }\r\n ,\"microsoft.resources/deployments/operations\": { \"SingularDisplayName\": \"Microsoft.Resources deployments operation\" }\r\n ,\"microsoft.resources/deploymentscripts\": { \"SingularDisplayName\": \"Deployment Script\" }\r\n ,\"microsoft.resources/deploymentstacks\": { \"SingularDisplayName\": \"Deployment stack\" }\r\n ,\"microsoft.resources/mobobrokers\": { \"SingularDisplayName\": \"Microsoft.Resources mobo broker\" }\r\n ,\"microsoft.resources/resourcechange\": { \"SingularDisplayName\": \"Change Analysis\" }\r\n ,\"microsoft.resources/resourcechanges\": { \"SingularDisplayName\": \"Resource change\" }\r\n ,\"microsoft.resources/resourcegraphvisualizer\": { \"SingularDisplayName\": \"Resource Graph Visualizer\" }\r\n ,\"microsoft.resources/resourcegroups\": { \"SingularDisplayName\": \"Microsoft.Resources resource group\" }\r\n ,\"microsoft.resources/resources\": { \"SingularDisplayName\": \"Resource\" }\r\n ,\"microsoft.resources/snapshots\": { \"SingularDisplayName\": \"Microsoft.Resources snapshot\" }\r\n ,\"microsoft.resources/subscriptions\": { \"SingularDisplayName\": \"Subscription\" }\r\n ,\"microsoft.resources/subscriptions/resourcegroups\": { \"SingularDisplayName\": \"Resource group\" }\r\n ,\"microsoft.resources/tags\": { \"SingularDisplayName\": \"Microsoft.Resources tag\" }\r\n ,\"microsoft.resources/templatespecs\": { \"SingularDisplayName\": \"Template spec\" }\r\n ,\"microsoft.resources/virtualsubscriptionsforresourcepicker\": { \"SingularDisplayName\": \"Subscription\" }\r\n ,\"microsoft.saas/applications\": { \"SingularDisplayName\": \"Software as a Service (classic)\" }\r\n ,\"microsoft.saas/resources\": { \"SingularDisplayName\": \"SaaS\" }\r\n ,\"microsoft.saas/saasresources\": { \"SingularDisplayName\": \"SaaS (classic)\" }\r\n ,\"microsoft.saashub/cloudservices\": { \"SingularDisplayName\": \"Microsoft.SaaSHub cloud service\" }\r\n ,\"microsoft.saashub/cloudservices/hidden\": { \"SingularDisplayName\": \"Microsoft SaaS\" }\r\n ,\"microsoft.saashub/saasresources\": { \"SingularDisplayName\": \"Microsoft.SaaSHub saas resource\" }\r\n ,\"microsoft.salescopilot/conversationintelligencerecordingaccounts\": { \"SingularDisplayName\": \"Microsoft.SalesCopilot conversation intelligence recording account\" }\r\n ,\"microsoft.scheduler/jobcollections\": { \"SingularDisplayName\": \"Scheduler job collection\" }\r\n ,\"microsoft.scheduler/jobcollections/jobs\": { \"SingularDisplayName\": \"Scheduler job\" }\r\n ,\"microsoft.scom/managedinstances\": { \"SingularDisplayName\": \"SCOM managed instance\" }\r\n ,\"microsoft.scvmm/availabilitysets\": { \"SingularDisplayName\": \"Microsoft.ScVmm availability set\" }\r\n ,\"microsoft.scvmm/clouds\": { \"SingularDisplayName\": \"Microsoft.ScVmm cloud\" }\r\n ,\"microsoft.scvmm/virtualmachineinstances\": { \"SingularDisplayName\": \"Microsoft.ScVmm virtual machine instance\" }\r\n ,\"microsoft.scvmm/virtualmachineinstances/guestagents\": { \"SingularDisplayName\": \"Microsoft.ScVmm virtual machine instances guest agent\" }\r\n ,\"microsoft.scvmm/virtualmachineinstances/hybrididentitymetadata\": { \"SingularDisplayName\": \"Microsoft.ScVmm virtual machine instances hybrid identity metadata\" }\r\n ,\"microsoft.scvmm/virtualmachines\": { \"SingularDisplayName\": \"SCVMM virtual machine - Azure Arc\" }\r\n ,\"microsoft.scvmm/virtualmachinetemplates\": { \"SingularDisplayName\": \"Microsoft.ScVmm virtual machine template\" }\r\n ,\"microsoft.scvmm/virtualnetworks\": { \"SingularDisplayName\": \"Microsoft.ScVmm virtual network\" }\r\n ,\"microsoft.scvmm/vmmservers\": { \"SingularDisplayName\": \"SCVMM management server\" }\r\n ,\"microsoft.search/searchservices\": { \"SingularDisplayName\": \"Search service\" }\r\n ,\"microsoft.secretmanagementsampleprovider/forecasts\": { \"SingularDisplayName\": \"Microsoft.SecretManagementSampleProvider forecast\" }\r\n ,\"microsoft.secretsynccontroller/azurekeyvaultsecretproviderclasses\": { \"SingularDisplayName\": \"Microsoft.SecretSyncController Azure key vault secret provider class\" }\r\n ,\"microsoft.secretsynccontroller/secretsyncs\": { \"SingularDisplayName\": \"Microsoft.SecretSyncController secret sync\" }\r\n ,\"microsoft.security/adaptivenetworkhardenings\": { \"SingularDisplayName\": \"Microsoft.Security adaptive network hardening\" }\r\n ,\"microsoft.security/advancedthreatprotectionsettings\": { \"SingularDisplayName\": \"Microsoft.Security advanced threat protection setting\" }\r\n ,\"microsoft.security/alertssuppressionrules\": { \"SingularDisplayName\": \"Microsoft.Security alerts suppression rule\" }\r\n ,\"microsoft.security/apicollections\": { \"SingularDisplayName\": \"Microsoft.Security API collection\" }\r\n ,\"microsoft.security/applications\": { \"SingularDisplayName\": \"Microsoft.Security application\" }\r\n ,\"microsoft.security/assessmentmetadata\": { \"SingularDisplayName\": \"Microsoft.Security assessment metadata\" }\r\n ,\"microsoft.security/assessments\": { \"SingularDisplayName\": \"Microsoft.Security assessment\" }\r\n ,\"microsoft.security/assessments/governanceassignments\": { \"SingularDisplayName\": \"Microsoft.Security assessments governance assignment\" }\r\n ,\"microsoft.security/assessments/subassessments\": { \"SingularDisplayName\": \"Microsoft.Security assessments sub assessment\" }\r\n ,\"microsoft.security/assignments\": { \"SingularDisplayName\": \"Microsoft.Security assignment\" }\r\n ,\"microsoft.security/automations\": { \"SingularDisplayName\": \"Microsoft.Security automation\" }\r\n ,\"microsoft.security/autoprovisioningsettings\": { \"SingularDisplayName\": \"Microsoft.Security auto provisioning setting\" }\r\n ,\"microsoft.security/complianceresults\": { \"SingularDisplayName\": \"Microsoft.Security compliance result\" }\r\n ,\"microsoft.security/compliances\": { \"SingularDisplayName\": \"Microsoft.Security compliance\" }\r\n ,\"microsoft.security/connectors\": { \"SingularDisplayName\": \"Microsoft.Security connector\" }\r\n ,\"microsoft.security/customassessmentautomations\": { \"SingularDisplayName\": \"Microsoft.Security custom assessment automation\" }\r\n ,\"microsoft.security/defenderforstoragesettings\": { \"SingularDisplayName\": \"Microsoft.Security defender for storage setting\" }\r\n ,\"microsoft.security/defenderforstoragesettings/malwarescans\": { \"SingularDisplayName\": \"Microsoft.Security defender for storage settings malware scan\" }\r\n ,\"microsoft.security/devicesecuritygroups\": { \"SingularDisplayName\": \"Microsoft.Security device security group\" }\r\n ,\"microsoft.security/governancerules\": { \"SingularDisplayName\": \"Microsoft.Security governance rule\" }\r\n ,\"microsoft.security/governancerules/operationresults\": { \"SingularDisplayName\": \"Microsoft.Security governance rules operation result\" }\r\n ,\"microsoft.security/healthreports\": { \"SingularDisplayName\": \"Microsoft.Security health report\" }\r\n ,\"microsoft.security/informationprotectionpolicies\": { \"SingularDisplayName\": \"Microsoft.Security information protection policy\" }\r\n ,\"microsoft.security/iotsecuritysolutions\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solution\" }\r\n ,\"microsoft.security/iotsecuritysolutions/analyticsmodels\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions analytics model\" }\r\n ,\"microsoft.security/iotsecuritysolutions/analyticsmodels/aggregatedalerts\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions analytics models aggregated alert\" }\r\n ,\"microsoft.security/iotsecuritysolutions/analyticsmodels/aggregatedrecommendations\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions analytics models aggregated recommendation\" }\r\n ,\"microsoft.security/iotsecuritysolutions/iotalerts\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions IoT alert\" }\r\n ,\"microsoft.security/iotsecuritysolutions/iotalerttypes\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions IoT alert type\" }\r\n ,\"microsoft.security/iotsecuritysolutions/iotrecommendations\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions IoT recommendation\" }\r\n ,\"microsoft.security/iotsecuritysolutions/iotrecommendationtypes\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions IoT recommendation type\" }\r\n ,\"microsoft.security/locations/alerts\": { \"SingularDisplayName\": \"Security Alert\" }\r\n ,\"microsoft.security/mdeonboardings\": { \"SingularDisplayName\": \"Microsoft.Security mde onboarding\" }\r\n ,\"microsoft.security/pricings\": { \"SingularDisplayName\": \"Defender for Cloud\" }\r\n ,\"microsoft.security/pricings/securityoperators\": { \"SingularDisplayName\": \"Microsoft.Security pricings security operator\" }\r\n ,\"microsoft.security/regulatorycompliancestandards\": { \"SingularDisplayName\": \"Microsoft.Security regulatory compliance standard\" }\r\n ,\"microsoft.security/regulatorycompliancestandards/regulatorycompliancecontrols\": { \"SingularDisplayName\": \"Microsoft.Security regulatory compliance standards regulatory compliance control\" }\r\n ,\"microsoft.security/regulatorycompliancestandards/regulatorycompliancecontrols/regulatorycomplianceassessments\": { \"SingularDisplayName\": \"Microsoft.Security regulatory compliance standards regulatory compliance controls regulatory compliance assessment\" }\r\n ,\"microsoft.security/securescores\": { \"SingularDisplayName\": \"Microsoft.Security secure score\" }\r\n ,\"microsoft.security/securityconnectors\": { \"SingularDisplayName\": \"Microsoft.Security security connector\" }\r\n ,\"microsoft.security/securityconnectors/devops\": { \"SingularDisplayName\": \"Microsoft.Security security connectors devop\" }\r\n ,\"microsoft.security/securitycontacts\": { \"SingularDisplayName\": \"Microsoft.Security security contact\" }\r\n ,\"microsoft.security/sensitivitysettings\": { \"SingularDisplayName\": \"Microsoft.Security sensitivity setting\" }\r\n ,\"microsoft.security/servervulnerabilityassessments\": { \"SingularDisplayName\": \"Microsoft.Security server vulnerability assessment\" }\r\n ,\"microsoft.security/servervulnerabilityassessmentssettings\": { \"SingularDisplayName\": \"Microsoft.Security server vulnerability assessments setting\" }\r\n ,\"microsoft.security/settings\": { \"SingularDisplayName\": \"Microsoft.Security setting\" }\r\n ,\"microsoft.security/standards\": { \"SingularDisplayName\": \"Microsoft.Security standard\" }\r\n ,\"microsoft.security/workspacesettings\": { \"SingularDisplayName\": \"Microsoft.Security workspace setting\" }\r\n ,\"microsoft.securitycopilot/capacities\": { \"SingularDisplayName\": \"Microsoft Security compute capacity\" }\r\n ,\"microsoft.securitydetonation/chambers\": { \"SingularDisplayName\": \"Security Detonation Chamber\" }\r\n ,\"microsoft.securityinsightsarg/sentinel\": { \"SingularDisplayName\": \"Microsoft Sentinel\" }\r\n ,\"microsoft.sentinelplatformservices/sentinelplatformservices\": { \"SingularDisplayName\": \"Microsoft.SentinelPlatformServices sentinel platform service\" }\r\n ,\"microsoft.serialconsole/consoleservices\": { \"SingularDisplayName\": \"Microsoft.SerialConsole console service\" }\r\n ,\"microsoft.serialconsole/serialports\": { \"SingularDisplayName\": \"Microsoft.SerialConsole serial port\" }\r\n ,\"microsoft.servicebus/namespaces\": { \"SingularDisplayName\": \"Service Bus namespace\" }\r\n ,\"microsoft.servicebus/namespaces/disasterrecoveryconfigs\": { \"SingularDisplayName\": \"Service Bus Geo-DR Alias\" }\r\n ,\"microsoft.servicebus/namespaces/queues\": { \"SingularDisplayName\": \"Service Bus queue\" }\r\n ,\"microsoft.servicebus/namespaces/topics\": { \"SingularDisplayName\": \"Service Bus topic\" }\r\n ,\"microsoft.servicebus/namespaces/topics/subscriptions\": { \"SingularDisplayName\": \"Service Bus Subscription\" }\r\n ,\"microsoft.servicefabric/clusters\": { \"SingularDisplayName\": \"Service Fabric cluster\" }\r\n ,\"microsoft.servicefabric/managedclusters\": { \"SingularDisplayName\": \"Service Fabric managed cluster\" }\r\n ,\"microsoft.servicefabricmesh/applications\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh application\" }\r\n ,\"microsoft.servicefabricmesh/applications/services\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh applications service\" }\r\n ,\"microsoft.servicefabricmesh/applications/services/replicas\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh applications services replica\" }\r\n ,\"microsoft.servicefabricmesh/gateways\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh gateway\" }\r\n ,\"microsoft.servicefabricmesh/networks\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh network\" }\r\n ,\"microsoft.servicefabricmesh/secrets\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh secret\" }\r\n ,\"microsoft.servicefabricmesh/secrets/values\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh secrets value\" }\r\n ,\"microsoft.servicefabricmesh/volumes\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh volume\" }\r\n ,\"microsoft.servicelinker/dryruns\": { \"SingularDisplayName\": \"Microsoft.ServiceLinker dryrun\" }\r\n ,\"microsoft.servicelinker/linkers\": { \"SingularDisplayName\": \"Microsoft.ServiceLinker linker\" }\r\n ,\"microsoft.servicenetworking/trafficcontrollers\": { \"SingularDisplayName\": \"Application Gateway for Containers\" }\r\n ,\"microsoft.serviceshub/connectors\": { \"SingularDisplayName\": \"Services Hub Connector\" }\r\n ,\"microsoft.signalrservice/signalr\": { \"SingularDisplayName\": \"SignalR\" }\r\n ,\"microsoft.signalrservice/signalr/replicas\": { \"SingularDisplayName\": \"SignalR Replica\" }\r\n ,\"microsoft.signalrservice/webpubsub\": { \"SingularDisplayName\": \"Web PubSub Service\" }\r\n ,\"microsoft.signalrservice/webpubsub/replicas\": { \"SingularDisplayName\": \"Web PubSub Service Replica\" }\r\n ,\"microsoft.skytap/billingnodes\": { \"SingularDisplayName\": \"Microsoft.Skytap billing node\" }\r\n ,\"microsoft.skytap/interfaces\": { \"SingularDisplayName\": \"Microsoft.Skytap interface\" }\r\n ,\"microsoft.skytap/nodes\": { \"SingularDisplayName\": \"Microsoft.Skytap node\" }\r\n ,\"microsoft.softwareplan/hybridusebenefits\": { \"SingularDisplayName\": \"Microsoft.SoftwarePlan hybrid use benefit\" }\r\n ,\"microsoft.solutions/applicationdefinitions\": { \"SingularDisplayName\": \"Service catalog managed application definition\" }\r\n ,\"microsoft.solutions/applications\": { \"SingularDisplayName\": \"Managed application\" }\r\n ,\"microsoft.solutions/jitrequests\": { \"SingularDisplayName\": \"Microsoft.Solutions JIT request\" }\r\n ,\"microsoft.sovereign/landingzoneaccounts\": { \"SingularDisplayName\": \"Landing zone account\" }\r\n ,\"microsoft.sovereign/landingzoneaccounts/landingzoneconfigurations\": { \"SingularDisplayName\": \"Landing Zone Configuration\" }\r\n ,\"microsoft.sovereign/landingzoneaccounts/landingzoneregistrations\": { \"SingularDisplayName\": \"Landing Zone Registration\" }\r\n ,\"microsoft.sovereign/landingzoneconfigurations\": { \"SingularDisplayName\": \"Landing Zone Configuration\" }\r\n ,\"microsoft.sovereign/landingzoneregistrations\": { \"SingularDisplayName\": \"Landing Zone Registration\" }\r\n ,\"microsoft.sovereign/transparencylogs\": { \"SingularDisplayName\": \"Transparency log\" }\r\n ,\"microsoft.sql/azuresql\": { \"SingularDisplayName\": \"Azure SQL resource\" }\r\n ,\"microsoft.sql/instancepools\": { \"SingularDisplayName\": \"Instance pool\" }\r\n ,\"microsoft.sql/managedinstances\": { \"SingularDisplayName\": \"SQL managed instance\" }\r\n ,\"microsoft.sql/managedinstances/databases\": { \"SingularDisplayName\": \"Managed database\" }\r\n ,\"microsoft.sql/servers\": { \"SingularDisplayName\": \"SQL server\" }\r\n ,\"microsoft.sql/servers/databases\": { \"SingularDisplayName\": \"SQL database\" }\r\n ,\"microsoft.sql/servers/elasticpools\": { \"SingularDisplayName\": \"SQL elastic pool\" }\r\n ,\"microsoft.sql/servers/jobagents\": { \"SingularDisplayName\": \"Elastic Job agent\" }\r\n ,\"microsoft.sql/virtualclusters\": { \"SingularDisplayName\": \"Virtual cluster\" }\r\n ,\"microsoft.sqlvirtualmachine/sqlvirtualmachinegroups\": { \"SingularDisplayName\": \"Microsoft.SqlVirtualMachine sql virtual machine group\" }\r\n ,\"microsoft.sqlvirtualmachine/sqlvirtualmachinegroups/availabilitygrouplisteners\": { \"SingularDisplayName\": \"Microsoft.SqlVirtualMachine sql virtual machine groups availability group listener\" }\r\n ,\"microsoft.sqlvirtualmachine/sqlvirtualmachines\": { \"SingularDisplayName\": \"SQL virtual machine\" }\r\n ,\"microsoft.standbypool/standbycontainergrouppools\": { \"SingularDisplayName\": \"Microsoft.StandbyPool standby container group pool\" }\r\n ,\"microsoft.standbypool/standbycontainergrouppools/runtimeviews\": { \"SingularDisplayName\": \"Microsoft.StandbyPool standby container group pools runtime view\" }\r\n ,\"microsoft.standbypool/standbyvirtualmachinepools\": { \"SingularDisplayName\": \"Microsoft.StandbyPool standby virtual machine pool\" }\r\n ,\"microsoft.standbypool/standbyvirtualmachinepools/runtimeviews\": { \"SingularDisplayName\": \"Microsoft.StandbyPool standby virtual machine pools runtime view\" }\r\n ,\"microsoft.standbypool/standbyvirtualmachinepools/standbyvirtualmachines\": { \"SingularDisplayName\": \"Microsoft.StandbyPool standby virtual machine pools standby virtual machine\" }\r\n ,\"microsoft.storage/storageaccounts\": { \"SingularDisplayName\": \"Storage account\" }\r\n ,\"microsoft.storageactions/storagetasks\": { \"SingularDisplayName\": \"Storage task - Azure Storage Actions\" }\r\n ,\"microsoft.storagecache/amlfilesystems\": { \"SingularDisplayName\": \"Azure Managed Lustre\" }\r\n ,\"microsoft.storagecache/caches\": { \"SingularDisplayName\": \"HPC cache\" }\r\n ,\"microsoft.storagediscovery/storagediscoveryworkspaces\": { \"SingularDisplayName\": \"Storage Discovery workspace\" }\r\n ,\"microsoft.storagehub/all\": { \"SingularDisplayName\": \"All resources\" }\r\n ,\"microsoft.storagehub/policycomplianceresources\": { \"SingularDisplayName\": \"Policy compliance\" }\r\n ,\"microsoft.storageinsights/storagecollectionrules\": { \"SingularDisplayName\": \"Microsoft.StorageInsights storage collection rule\" }\r\n ,\"microsoft.storagemover/storagemovers\": { \"SingularDisplayName\": \"Storage mover\" }\r\n ,\"microsoft.storagepool/diskpools\": { \"SingularDisplayName\": \"Microsoft.StoragePool disk pool\" }\r\n ,\"microsoft.storagepool/diskpools/iscsitargets\": { \"SingularDisplayName\": \"Microsoft.StoragePool disk pools iscsi target\" }\r\n ,\"microsoft.storagesync/storagesyncservices\": { \"SingularDisplayName\": \"Storage Sync Service\" }\r\n ,\"microsoft.storagetasks/storagetasks\": { \"SingularDisplayName\": \"Microsoft.StorageTasks storage task\" }\r\n ,\"microsoft.storsimple/managers\": { \"SingularDisplayName\": \"StorSimple device manager\" }\r\n ,\"microsoft.storsimple/managers/accesscontrolrecords\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers access control record\" }\r\n ,\"microsoft.storsimple/managers/bandwidthsettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers bandwidth setting\" }\r\n ,\"microsoft.storsimple/managers/certificates\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers certificate\" }\r\n ,\"microsoft.storsimple/managers/devices\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers device\" }\r\n ,\"microsoft.storsimple/managers/devices/alertsettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices alert setting\" }\r\n ,\"microsoft.storsimple/managers/devices/backuppolicies\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices backup policy\" }\r\n ,\"microsoft.storsimple/managers/devices/backuppolicies/schedules\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices backup policies schedule\" }\r\n ,\"microsoft.storsimple/managers/devices/backupschedulegroups\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices backup schedule group\" }\r\n ,\"microsoft.storsimple/managers/devices/chapsettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices chap setting\" }\r\n ,\"microsoft.storsimple/managers/devices/fileservers\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices fileserver\" }\r\n ,\"microsoft.storsimple/managers/devices/fileservers/shares\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices fileservers share\" }\r\n ,\"microsoft.storsimple/managers/devices/iscsiservers\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices iscsiserver\" }\r\n ,\"microsoft.storsimple/managers/devices/iscsiservers/disks\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices iscsiservers disk\" }\r\n ,\"microsoft.storsimple/managers/devices/jobs\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices job\" }\r\n ,\"microsoft.storsimple/managers/devices/networksettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices network setting\" }\r\n ,\"microsoft.storsimple/managers/devices/securitysettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices security setting\" }\r\n ,\"microsoft.storsimple/managers/devices/timesettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices time setting\" }\r\n ,\"microsoft.storsimple/managers/devices/updatesummary\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices update summary\" }\r\n ,\"microsoft.storsimple/managers/devices/volumecontainers\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices volume container\" }\r\n ,\"microsoft.storsimple/managers/devices/volumecontainers/volumes\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices volume containers volume\" }\r\n ,\"microsoft.storsimple/managers/encryptionsettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers encryption setting\" }\r\n ,\"microsoft.storsimple/managers/extendedinformation\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers extended information\" }\r\n ,\"microsoft.storsimple/managers/storageaccountcredentials\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers storage account credential\" }\r\n ,\"microsoft.storsimple/managers/storagedomains\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers storage domain\" }\r\n ,\"microsoft.streamanalytics/clusters\": { \"SingularDisplayName\": \"Stream Analytics cluster\" }\r\n ,\"microsoft.streamanalytics/streamingjobs\": { \"SingularDisplayName\": \"Stream Analytics job\" }\r\n ,\"microsoft.subscription/aliases\": { \"SingularDisplayName\": \"Microsoft.Subscription aliase\" }\r\n ,\"microsoft.subscription/changetenantrequest\": { \"SingularDisplayName\": \"Microsoft.Subscription change tenant request\" }\r\n ,\"microsoft.subscription/policies\": { \"SingularDisplayName\": \"Microsoft.Subscription policy\" }\r\n ,\"microsoft.subscription/subscriptiondefinitions\": { \"SingularDisplayName\": \"Microsoft.Subscription subscription definition\" }\r\n ,\"microsoft.subscription/subscriptionoperations\": { \"SingularDisplayName\": \"Microsoft.Subscription subscription operation\" }\r\n ,\"microsoft.support/fileworkspaces\": { \"SingularDisplayName\": \"Microsoft.Support file workspace\" }\r\n ,\"microsoft.support/fileworkspaces/files\": { \"SingularDisplayName\": \"Microsoft.Support file workspaces file\" }\r\n ,\"microsoft.support/services\": { \"SingularDisplayName\": \"Microsoft.Support service\" }\r\n ,\"microsoft.support/services/problemclassifications\": { \"SingularDisplayName\": \"Microsoft.Support services problem classification\" }\r\n ,\"microsoft.support/supporttickets\": { \"SingularDisplayName\": \"Support Request\" }\r\n ,\"microsoft.sustainabilityservices/calculations\": { \"SingularDisplayName\": \"Project Sustainability Calculator\" }\r\n ,\"microsoft.symphony/instances\": { \"SingularDisplayName\": \"Microsoft.Symphony instance\" }\r\n ,\"microsoft.symphony/solutions\": { \"SingularDisplayName\": \"Microsoft.Symphony solution\" }\r\n ,\"microsoft.symphony/targets\": { \"SingularDisplayName\": \"Microsoft.Symphony target\" }\r\n ,\"microsoft.synapse/privatelinkhubs\": { \"SingularDisplayName\": \"Synapse private link hub\" }\r\n ,\"microsoft.synapse/workspaces\": { \"SingularDisplayName\": \"Synapse workspace\" }\r\n ,\"microsoft.synapse/workspaces/bigdatapools\": { \"SingularDisplayName\": \"Apache Spark pool\" }\r\n ,\"microsoft.synapse/workspaces/kustopools\": { \"SingularDisplayName\": \"Data Explorer pool\" }\r\n ,\"microsoft.synapse/workspaces/kustopools/databases\": { \"SingularDisplayName\": \"Data Explorer Database\" }\r\n ,\"microsoft.synapse/workspaces/scopepools\": { \"SingularDisplayName\": \"SCOPE pool\" }\r\n ,\"microsoft.synapse/workspaces/sqlpools\": { \"SingularDisplayName\": \"Dedicated SQL pool\" }\r\n ,\"microsoft.syntex/accounts\": { \"SingularDisplayName\": \"Microsoft.Syntex account\" }\r\n ,\"microsoft.syntex/documentprocessors\": { \"SingularDisplayName\": \"Microsoft.Syntex document processor\" }\r\n ,\"microsoft.test/healthdataaiservices\": { \"SingularDisplayName\": \"Azure Health Data and AI Services\" }\r\n ,\"microsoft.timeseriesinsights/environments\": { \"SingularDisplayName\": \"Microsoft.TimeSeriesInsights environment\" }\r\n ,\"microsoft.timeseriesinsights/environments/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.TimeSeriesInsights environments access policy\" }\r\n ,\"microsoft.timeseriesinsights/environments/eventsources\": { \"SingularDisplayName\": \"Microsoft.TimeSeriesInsights environments event source\" }\r\n ,\"microsoft.timeseriesinsights/environments/referencedatasets\": { \"SingularDisplayName\": \"Microsoft.TimeSeriesInsights environments reference data set\" }\r\n ,\"microsoft.toolchainorchestrator/activations\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator activation\" }\r\n ,\"microsoft.toolchainorchestrator/campaigns\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator campaign\" }\r\n ,\"microsoft.toolchainorchestrator/campaigns/versions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator campaigns version\" }\r\n ,\"microsoft.toolchainorchestrator/catalogs\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator catalog\" }\r\n ,\"microsoft.toolchainorchestrator/catalogs/versions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator catalogs version\" }\r\n ,\"microsoft.toolchainorchestrator/diagnostics\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator diagnostic\" }\r\n ,\"microsoft.toolchainorchestrator/instances\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator instance\" }\r\n ,\"microsoft.toolchainorchestrator/instances/versions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator instances version\" }\r\n ,\"microsoft.toolchainorchestrator/solutions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator solution\" }\r\n ,\"microsoft.toolchainorchestrator/solutions/versions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator solutions version\" }\r\n ,\"microsoft.toolchainorchestrator/targets\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator target\" }\r\n ,\"microsoft.toolchainorchestrator/targets/versions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator targets version\" }\r\n ,\"microsoft.updatemanager/updaterules\": { \"SingularDisplayName\": \"Update Rule\" }\r\n ,\"microsoft.usagebilling/accounts\": { \"SingularDisplayName\": \"Microsoft.UsageBilling account\" }\r\n ,\"microsoft.usagebilling/accounts/dataexports\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts data export\" }\r\n ,\"microsoft.usagebilling/accounts/inputs\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts input\" }\r\n ,\"microsoft.usagebilling/accounts/metricexports\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts metric export\" }\r\n ,\"microsoft.usagebilling/accounts/pav2outputs\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts pav2output\" }\r\n ,\"microsoft.usagebilling/accounts/pipelines\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts pipeline\" }\r\n ,\"microsoft.usagebilling/accounts/pipelines/outputselectors\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts pipelines output selector\" }\r\n ,\"microsoft.verifiedid/authorities\": { \"SingularDisplayName\": \"Microsoft.VerifiedId authority\" }\r\n ,\"microsoft.videoindexer/accounts\": { \"SingularDisplayName\": \"Azure AI Video Indexer\" }\r\n ,\"microsoft.virtualmachineimages/imagetemplates\": { \"SingularDisplayName\": \"Image template\" }\r\n ,\"microsoft.visualstudio/account\": { \"SingularDisplayName\": \"Azure DevOps organization\" }\r\n ,\"microsoft.vmware/resourcepools\": { \"SingularDisplayName\": \"Microsoft.VMware resource pool\" }\r\n ,\"microsoft.vmware/vcenters\": { \"SingularDisplayName\": \"Microsoft.VMware vcenter\" }\r\n ,\"microsoft.vmware/vcenters/inventoryitems\": { \"SingularDisplayName\": \"Microsoft.VMware vcenters inventory item\" }\r\n ,\"microsoft.vmware/virtualmachines\": { \"SingularDisplayName\": \"Microsoft.VMware virtual machine\" }\r\n ,\"microsoft.vmware/virtualmachinetemplates\": { \"SingularDisplayName\": \"Microsoft.VMware virtual machine template\" }\r\n ,\"microsoft.vmware/virtualnetworks\": { \"SingularDisplayName\": \"Microsoft.VMware virtual network\" }\r\n ,\"microsoft.vmwarecloudsimple/dedicatedcloudnodes\": { \"SingularDisplayName\": \"Microsoft.VMwareCloudSimple dedicated cloud node\" }\r\n ,\"microsoft.vmwarecloudsimple/dedicatedcloudservices\": { \"SingularDisplayName\": \"Microsoft.VMwareCloudSimple dedicated cloud service\" }\r\n ,\"microsoft.vmwarecloudsimple/virtualmachines\": { \"SingularDisplayName\": \"Microsoft.VMwareCloudSimple virtual machine\" }\r\n ,\"microsoft.vnfmanager/devices\": { \"SingularDisplayName\": \"Microsoft.VnfManager device\" }\r\n ,\"microsoft.vnfmanager/vendors\": { \"SingularDisplayName\": \"Microsoft.VnfManager vendor\" }\r\n ,\"microsoft.vnfmanager/vendors/skus\": { \"SingularDisplayName\": \"Microsoft.VnfManager vendors SKU\" }\r\n ,\"microsoft.vnfmanager/vnfs\": { \"SingularDisplayName\": \"Microsoft.VnfManager vnf\" }\r\n ,\"microsoft.voiceservices/communicationsgateways\": { \"SingularDisplayName\": \"Communications Gateway\" }\r\n ,\"microsoft.voiceservices/communicationsgateways/testlines\": { \"SingularDisplayName\": \"Communications Gateway Test Line\" }\r\n ,\"microsoft.vsonline/accounts\": { \"SingularDisplayName\": \"Microsoft.VSOnline account\" }\r\n ,\"microsoft.vsonline/plans\": { \"SingularDisplayName\": \"Visual Studio Online Plan\" }\r\n ,\"microsoft.web/certificates\": { \"SingularDisplayName\": \"Microsoft.Web certificate\" }\r\n ,\"microsoft.web/connectiongateways\": { \"SingularDisplayName\": \"App Service on-premises data gateway\" }\r\n ,\"microsoft.web/connections\": { \"SingularDisplayName\": \"App Service API connection\" }\r\n ,\"microsoft.web/containerapps\": { \"SingularDisplayName\": \"Microsoft.Web container app\" }\r\n ,\"microsoft.web/containerapps/revisions\": { \"SingularDisplayName\": \"Microsoft.Web container apps revision\" }\r\n ,\"microsoft.web/customapis\": { \"SingularDisplayName\": \"Logic apps custom connector\" }\r\n ,\"microsoft.web/deletedsites\": { \"SingularDisplayName\": \"Microsoft.Web deleted site\" }\r\n ,\"microsoft.web/hostingenvironments\": { \"SingularDisplayName\": \"App Service Environment\" }\r\n ,\"microsoft.web/ishostingenvironmentnameavailable\": { \"SingularDisplayName\": \"Microsoft.Web ishostingenvironmentnameavailable\" }\r\n ,\"microsoft.web/kubeenvironments\": { \"SingularDisplayName\": \"App Service Kubernetes Environment\" }\r\n ,\"microsoft.web/logicappstemplate\": { \"SingularDisplayName\": \"Logic Apps Template\" }\r\n ,\"microsoft.web/publishingusers\": { \"SingularDisplayName\": \"Microsoft.Web publishing user\" }\r\n ,\"microsoft.web/serverfarms\": { \"SingularDisplayName\": \"App Service plan\" }\r\n ,\"microsoft.web/sites\": { \"SingularDisplayName\": \"App Service web app\" }\r\n ,\"microsoft.web/sites/slots\": { \"SingularDisplayName\": \"App Service deployment slot\" }\r\n ,\"microsoft.web/sourcecontrols\": { \"SingularDisplayName\": \"Microsoft.Web sourcecontrol\" }\r\n ,\"microsoft.web/staticsites\": { \"SingularDisplayName\": \"Static Web App\" }\r\n ,\"microsoft.weightsandbiases/instances\": { \"SingularDisplayName\": \"Azure Native Weights & Biases Cloud Service\" }\r\n ,\"microsoft.whiteboxcadlprovider/whiteboxresources\": { \"SingularDisplayName\": \"Microsoft.WhiteBoxCadlProvider white box resource\" }\r\n ,\"microsoft.windows365/cloudpcdelegatedmsis\": { \"SingularDisplayName\": \"Microsoft.Windows365 cloud pc delegated msi\" }\r\n ,\"microsoft.windowsesu/multipleactivationkeys\": { \"SingularDisplayName\": \"Microsoft.WindowsESU multiple activation key\" }\r\n ,\"microsoft.windowsiot/deviceservices\": { \"SingularDisplayName\": \"Microsoft.WindowsIoT device service\" }\r\n ,\"microsoft.windowspushnotificationservices/registrations\": { \"SingularDisplayName\": \"Windows Push Notification Service\" }\r\n ,\"microsoft.workloadmonitor/monitors\": { \"SingularDisplayName\": \"Microsoft.WorkloadMonitor monitor\" }\r\n ,\"microsoft.workloadmonitor/monitors/history\": { \"SingularDisplayName\": \"Microsoft.WorkloadMonitor monitors history\" }\r\n ,\"microsoft.workloads/configurationvalidationresults\": { \"SingularDisplayName\": \"Microsoft.Workloads configuration validation result\" }\r\n ,\"microsoft.workloads/connectors\": { \"SingularDisplayName\": \"Microsoft.Workloads connector\" }\r\n ,\"microsoft.workloads/connectors/acssbackups\": { \"SingularDisplayName\": \"Microsoft.Workloads connectors acss backup\" }\r\n ,\"microsoft.workloads/connectors/amsinsights\": { \"SingularDisplayName\": \"Microsoft.Workloads connectors ams insight\" }\r\n ,\"microsoft.workloads/connectors/sapvirtualinstancemonitors\": { \"SingularDisplayName\": \"Microsoft.Workloads connectors sap virtual instance monitor\" }\r\n ,\"microsoft.workloads/epicvirtualinstances\": { \"SingularDisplayName\": \"Virtual Instance for Epic solution\" }\r\n ,\"microsoft.workloads/insights\": { \"SingularDisplayName\": \"Microsoft.Workloads insight\" }\r\n ,\"microsoft.workloads/instancegroupmonitors\": { \"SingularDisplayName\": \"Microsoft.Workloads instance group monitor\" }\r\n ,\"microsoft.workloads/instancehealthdefinitions\": { \"SingularDisplayName\": \"Microsoft.Workloads instance health definition\" }\r\n ,\"microsoft.workloads/instancehealthdefinitions/signaldefinitions\": { \"SingularDisplayName\": \"Microsoft.Workloads instance health definitions signal definition\" }\r\n ,\"microsoft.workloads/instancemonitors\": { \"SingularDisplayName\": \"Microsoft.Workloads instance monitor\" }\r\n ,\"microsoft.workloads/monitors\": { \"SingularDisplayName\": \"Azure Monitor for SAP solutions\" }\r\n ,\"microsoft.workloads/oraclevirtualinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads oracle virtual instance\" }\r\n ,\"microsoft.workloads/oraclevirtualinstances/databaseinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads oracle virtual instances database instance\" }\r\n ,\"microsoft.workloads/phpworkloads\": { \"SingularDisplayName\": \"Microsoft.Workloads php workload\" }\r\n ,\"microsoft.workloads/phpworkloads/wordpressinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads php workloads wordpress instance\" }\r\n ,\"microsoft.workloads/sapdiscoverysites\": { \"SingularDisplayName\": \"Microsoft.Workloads sap discovery site\" }\r\n ,\"microsoft.workloads/sapdiscoverysites/sapinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads sap discovery sites sap instance\" }\r\n ,\"microsoft.workloads/sapdiscoverysites/sapinstances/serverinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads sap discovery sites sap instances server instance\" }\r\n ,\"microsoft.workloads/sapvirtualinstances\": { \"SingularDisplayName\": \"Virtual Instance for SAP solutions\" }\r\n ,\"microsoft.workloads/sapvirtualinstances/applicationinstances\": { \"SingularDisplayName\": \"App server instance for SAP solutions\" }\r\n ,\"microsoft.workloads/sapvirtualinstances/centralinstances\": { \"SingularDisplayName\": \"Central service instance for SAP solutions\" }\r\n ,\"microsoft.workloads/sapvirtualinstances/databaseinstances\": { \"SingularDisplayName\": \"Database for SAP solutions\" }\r\n ,\"microsoft.workloads/virtualinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads virtual instance\" }\r\n ,\"microsoft.workloads/virtualinstances/components\": { \"SingularDisplayName\": \"Microsoft.Workloads virtual instances component\" }\r\n ,\"microsoft.workloads/workloadinstance\": { \"SingularDisplayName\": \"My Resource\" }\r\n ,\"microsoft.zerotrustsegmentation/segmentationmanagers\": { \"SingularDisplayName\": \"Segmentation Manager\" }\r\n ,\"mongodb.atlas/organizations\": { \"SingularDisplayName\": \"MongoDB Atlas Organization\" }\r\n ,\"neon.postgres/organizations\": { \"SingularDisplayName\": \"Neon Serverless Postgres Organization\" }\r\n ,\"newrelic.observability/monitors\": { \"SingularDisplayName\": \"New Relic\" }\r\n ,\"nginx.nginxplus/nginxdeployments\": { \"SingularDisplayName\": \"NGINXaaS\" }\r\n ,\"oracle.database/autonomousdatabases\": { \"SingularDisplayName\": \"Autonomous Database\" }\r\n ,\"oracle.database/basedb\": { \"SingularDisplayName\": \"Autonomous Database\" }\r\n ,\"oracle.database/cloudexadatainfrastructures\": { \"SingularDisplayName\": \"Oracle Exadata Infrastructure\" }\r\n ,\"oracle.database/cloudvmclusters\": { \"SingularDisplayName\": \"Oracle Exadata VM Cluster\" }\r\n ,\"oracle.database/exadbvmclusters\": { \"SingularDisplayName\": \"Oracle Exascale VM Cluster\" }\r\n ,\"oracle.database/exascaledbstoragevaults\": { \"SingularDisplayName\": \"Oracle Exascale DB Storage Vault\" }\r\n ,\"oracle.database/networkanchors\": { \"SingularDisplayName\": \"Network Anchor\" }\r\n ,\"oracle.database/oraclesubscriptions\": { \"SingularDisplayName\": \"OracleSubscription\" }\r\n ,\"oracle.database/resourceanchors\": { \"SingularDisplayName\": \"Resource Anchor\" }\r\n ,\"paloaltonetworks.cloudngfw/firewalls\": { \"SingularDisplayName\": \"Cloud NGFW by Palo Alto Networks\" }\r\n ,\"paloaltonetworks.cloudngfw/globalrulestacks\": { \"SingularDisplayName\": \"Global Rulestack\" }\r\n ,\"paloaltonetworks.cloudngfw/localrulestacks\": { \"SingularDisplayName\": \"Local Rulestack for Cloud NGFW by Palo Alto Networks\" }\r\n ,\"pinecone.vectordb/organizations\": { \"SingularDisplayName\": \"Azure Native Pinecone Cloud Service\" }\r\n ,\"purestorage.block/reservations\": { \"SingularDisplayName\": \"Azure Native Pure Storage Cloud Service\" }\r\n ,\"purestorage.block/storagepools\": { \"SingularDisplayName\": \"Storage pool\" }\r\n ,\"purestorage.block/storagepools/avsstoragecontainers\": { \"SingularDisplayName\": \"PureStorage.Block storage pools avs storage container\" }\r\n })[tolower(id)]\r\n}\r\n", + "$fxv#4": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData/Internal')\r\n_resource_type_5(id: string) {\r\n dynamic({\r\n \"qumulo.qaas/storages\": { \"SingularDisplayName\": \"Qumulo.QaaS storage\" }\r\n ,\"qumulo.storage/filesystems\": { \"SingularDisplayName\": \"Azure Native Qumulo Scalable File Service\" }\r\n ,\"solarwinds.observability/organizations\": { \"SingularDisplayName\": \"SolarWinds Observability\" }\r\n ,\"splitio.experimentation/experimentationworkspaces\": { \"SingularDisplayName\": \"Split Experimentation Workspace\" }\r\n ,\"wandisco.fusion/migrators\": { \"SingularDisplayName\": \"LiveData Migrator\" }\r\n ,\"wandisco.fusion/migrators/datatransferagents\": { \"SingularDisplayName\": \"Data Transfer Agent\" }\r\n ,\"wandisco.fusion/migrators/exclusiontemplates\": { \"SingularDisplayName\": \"Exclusion\" }\r\n ,\"wandisco.fusion/migrators/livedatamigrations\": { \"SingularDisplayName\": \"Migration\" }\r\n ,\"wandisco.fusion/migrators/metadatamigrations\": { \"SingularDisplayName\": \"Metadata Migration\" }\r\n ,\"wandisco.fusion/migrators/metadatatargets\": { \"SingularDisplayName\": \"Metadata Target\" }\r\n ,\"wandisco.fusion/migrators/pathmappings\": { \"SingularDisplayName\": \"Path Mapping\" }\r\n ,\"wandisco.fusion/migrators/targets\": { \"SingularDisplayName\": \"Target\" }\r\n ,\"wandisco.fusion/migrators/verifications\": { \"SingularDisplayName\": \"Verification\" }\r\n })[tolower(id)]\r\n}\r\n", + "$fxv#5": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n// resource_type\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData')\r\nresource_type(id: string) {\r\n coalesce(_resource_type_1(id), _resource_type_2(id), _resource_type_3(id), _resource_type_4(id), _resource_type_5(id))\r\n}\r\n", + "$fxv#6": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Common utility functions\r\n//\r\n// TIP: Use Ctrl+K,Ctrl+0 to collapse all regions in VS Code\r\n//======================================================================================================================\r\n\r\n\r\n//===| Date functions |=================================================================================================\r\n\r\n// monthstring\r\n.create-or-alter function \r\nwith (docstring = @'Returns the name of the month for the specified date (e.g. Jan or January)', folder =@'Common') \r\nmonthstring(['date']: datetime, length: int = 9)\r\n{\r\n substring(dynamic(['January','February','March','April','May','June','July','August','September','October','November','December'])[getmonth(['date']) - 1], 0, length)\r\n}\r\n\r\n// datestring\r\n.create-or-alter function \r\nwith (docstring = @'Converts 2 dates into a simple, user-friendly date range (e.g. Jan 1-Jan 3)', folder =@'Common') \r\ndatestring(start: datetime, end: datetime = datetime('0001-01-01'))\r\n{\r\n let month = (d: datetime) { monthstring(d, 3) };\r\n let endDate = iff(end == datetime('0001-01-01'), start, end);\r\n let sameDate = startofday(start) == startofday(endDate);\r\n let sameMonth = startofmonth(start) == startofmonth(endDate);\r\n let sameYear = startofyear(start) == startofyear(endDate);\r\n let fullMonth = startofday(start) == startofmonth(start) and startofday(endDate) == startofday(endofmonth(endDate));\r\n let fullYear = startofday(start) == startofyear(start) and startofday(endDate) == startofday(endofyear(endDate));\r\n let currentYear = sameYear and startofyear(start) == startofyear(now());\r\n case(\r\n // Full year | yyyy (same year) / yyyy-yyyy (diff years)\r\n fullYear,\r\n strcat(getyear(start), iff(sameYear, '', strcat('-', getyear(endDate)))),\r\n // 1 full mo, same year | Mmm yyyy\r\n fullMonth and sameMonth and sameYear,\r\n strcat(month(start), ' ', getyear(start)),\r\n // 2+ full mo, same year | Mmm-Mmm (current year) / Mmm-Mmm yyyy (other year)\r\n fullMonth and sameYear,\r\n strcat(month(start), '-', month(endDate), iff(currentYear, '', strcat(' ', getyear(endDate)))),\r\n // Full mo, diff year | Mmm yyyy-Mmm yyyy\r\n fullMonth and not(sameYear),\r\n strcat(month(start), ' ', getyear(start), '-', month(endDate), ' ', getyear(endDate)),\r\n // Same date | Mmm d (current year) / Mmm d, yyyy (other year)\r\n sameDate,\r\n strcat(month(start), ' ', dayofmonth(start), iff(currentYear, '', strcat(', ', getyear(endDate)))),\r\n // 1 partial M, same Y | Mmm d-d (current year) / Mmm d-d, yyyy (other year)\r\n not(fullMonth) and sameMonth and sameYear,\r\n strcat(month(start), ' ', dayofmonth(start), '-', dayofmonth(endDate), iff(currentYear, '', strcat(' ', getyear(endDate)))),\r\n // 2+ partial M, same Y | Mmm d-Mmm d (current year) / Mmm d-Mmm d, yyyy (other year)\r\n not(fullMonth) and not(sameMonth) and sameYear,\r\n strcat(month(start), ' ', dayofmonth(start), '-', month(endDate), ' ', dayofmonth(endDate), iff(currentYear, '', strcat(', ', getyear(endDate)))),\r\n // All other cases | Mmm d, yyyy-Mmm d, yyyy\r\n strcat(month(start), ' ', dayofmonth(start), ', ', getyear(start), '-', month(endDate), ' ', dayofmonth(endDate), ', ', getyear(endDate))\r\n )\r\n}\r\n\r\n// daterange\r\n.create-or-alter function \r\nwith (docstring = @'DEPRECATED: Please use datestring(); function will be removed on or after the Jan 2026 release', folder =@'Common') \r\ndaterange(start: datetime, end: datetime = datetime('0001-01-01'))\r\n{\r\n datestring(start, end)\r\n}\r\n\r\n// monthsago\r\n.create-or-alter function \r\nwith (docstring = 'DEPRECATED: Please use startofmonth(now(), -<# of months>); function will be removed on or after the Jan 2026 release', folder = 'Common')\r\nmonthsago(months: int)\r\n{\r\n datetime_add('month', -months, startofmonth(now()))\r\n}\r\n\r\n\r\n//===| Number functions |===============================================================================================\r\n// NOTE: Must be defined before string converters\r\n\r\n// delta\r\n.create-or-alter function \r\nwith (docstring = @'Compares 2 values and returns the percentage change from oldval to newval', folder =@'Common') \r\ndelta(oldval: double, newval: double)\r\n{\r\n (newval - todouble(oldval))/oldval\r\n}\r\n\r\n// percentOfTotal\r\n// NOTE: Must be before percent() function\r\n.create-or-alter function \r\nwith (docstring = @'Calculates the percentage of each record based on a required Count column', folder =@'Common') \r\npercentOfTotal(t: (Count: long), tot: long)\r\n{\r\n let total = todouble(tot);\r\n t \r\n | extend Percent = round(Count / total * 100, 3) \r\n | order by Count desc\r\n}\r\n\r\n// percent\r\n.create-or-alter function \r\nwith (docstring = @'Calculates the percentage of each record based on a required Count column', folder =@'Common') \r\npercent(t: (Count: long))\r\n{\r\n let total = todouble(toscalar(t | summarize sum(Count)));\r\n percentOfTotal(t, total)\r\n}\r\n\r\n// plusminus\r\n.create-or-alter function \r\nwith (docstring = 'Shows a +/- sign based on the direction of the number', folder = 'Common')\r\nplusminus(val: string)\r\n{\r\n let neg = substring(val, 0, 1) == '-';\r\n iff(neg, val, strcat('+', val))\r\n}\r\n\r\n// updown\r\n.create-or-alter function \r\nwith (docstring = 'Shows an up/down arrow based on the direction of the number', folder = 'Common')\r\nupdown(val: string)\r\n{\r\n // TODO: Handle 0\r\n let neg = substring(val, 0, 1) == '-';\r\n iff(neg, strcat('↓', substring(val, 1)), strcat('↑', val))\r\n}\r\n\r\n\r\n//===| String functions |===============================================================================================\r\n\r\n// percentstring\r\n// NOTE: Must be defined before deltastring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate a percentage and render as a string', folder = 'Common')\r\npercentstring(num: double, total: double = 1.0, places: int = 9)\r\n{\r\n let value = 1.0 * num / total * 100;\r\n strcat(case(\r\n places != 9, round(value, places),\r\n value < 10, round(value, 2),\r\n round(value, 1)\r\n ), '%')\r\n}\r\n\r\n//----------------------------------------------------------------------------------------------------------------------\r\n\r\n// arraystring\r\n.create-or-alter function \r\nwith (docstring = 'Convert an array to a comma-delimited string', folder = 'Common')\r\narraystring(arr: dynamic)\r\n{\r\n replace_string(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(\r\n tostring(arr)\r\n , @'^\\[\"', '')\r\n , @'\"\\]$', '')\r\n , @'^, ', '')\r\n , @', $', '')\r\n , @'^\\[]$', '')\r\n , '\",\"', ', ')\r\n}\r\n\r\n// deltastring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate a delta percentage and render as a string', folder = 'Common')\r\ndeltastring(oldval: double, newval: double, places: int = 1, useArrows: bool = false)\r\n{\r\n let d = delta(oldval, newval);\r\n strcat(case(useArrows and d > 0, '↑', useArrows and d < 0, '↓', d < 0, '-', ''), percentstring(abs(d), 1, places))\r\n}\r\n\r\n// diffstring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate the difference and render as a string', folder = 'Common')\r\ndiffstring(oldval: double, newval: double, places: int = 1)\r\n{\r\n plusminus(round(newval - oldval, places))\r\n}\r\n\r\n// numberstring\r\n.create-or-alter function \r\nwith (docstring = 'Convert a number to a string', folder = 'Common')\r\nnumberstring(num: double, abbrev: bool = true)\r\n{\r\n replace_regex(case(\r\n num >= 10000000000000, strcat(round(1.0 * num / 1000000000000, 1), 'T'),\r\n num >= 1000000000000, strcat(round(1.0 * num / 1000000000000, 2), 'T'),\r\n num >= 10000000000, strcat(round(1.0 * num / 1000000000, 1), 'B'),\r\n num >= 1000000000, strcat(round(1.0 * num / 1000000000, 2), 'B'),\r\n num >= 10000000, strcat(round(1.0 * num / 1000000, 1), 'M'),\r\n num >= 1000000, strcat(round(1.0 * num / 1000000, 2), 'M'),\r\n num >= 10000, strcat(round(1.0 * num / 1000, 1), 'K'),\r\n // Kusto doesn't support back-refs yet -- num > 1000, replace_regex(tostring(num), @'(\\d)(?=(\\d{3})+\\.)', @'\\1,'), // See https://docs.microsoft.com/azure/data-explorer/kusto/query/re2-library\r\n num > 1000, replace_regex(tostring(num), @'([0-9]{3})$', @',\\1'), //num / 1000, ',', substring(tostring(num), 0) - (num / 1000 * 1000)),\r\n tostring(num)\r\n ), @'\\.0$', '')\r\n}\r\n\r\n\r\n//===| Other |==========================================================================================================\r\n\r\n// ifempty\r\n.create-or-alter function \r\nwith (docstring = 'Replaces an empty value with the specified default value', folder = 'Common')\r\nifempty(val: dynamic, defaultVal: dynamic)\r\n{\r\n iff(isempty(val), defaultVal, val)\r\n}\r\n", + "$fxv#7": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Ingestion database\r\n// Used for data ingestion, normalization, and cleansing.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n//===| Settings |=======================================================================================================\r\n\r\n.create-merge table HubSettingsLog (\r\n version: string,\r\n scopes: dynamic,\r\n retention: dynamic\r\n)\r\n\r\n//----------------------------------------------------------------------------------------------------------------------\r\n\r\n// HubSettings function\r\n.create-or-alter function\r\nwith (docstring='Gets the latest version of hub settings.', folder='Settings')\r\nHubSettings()\r\n{\r\n HubSettingsLog\r\n | extend timestamp = ingestion_time()\r\n | summarize arg_max(timestamp, *)\r\n}\r\n\r\n//----------------------------------------------------------------------------------------------------------------------\r\n\r\n// HubScopes function\r\n.create-or-alter function\r\nwith (docstring='Gets the currently configured scopes.', folder='Settings')\r\nHubScopes()\r\n{\r\n HubSettings\r\n | project scopes\r\n | mv-expand scopes\r\n}\r\n\r\n\r\n//===| Open data |======================================================================================================\r\n\r\n// PricingUnits -- Create table if it doesn't exist\r\n.create-merge table PricingUnits ( ignore: string )\r\n\r\n// PricingUnits -- Remove all columns\r\n.alter table PricingUnits ( ignore: string )\r\n\r\n// PricingUnits -- Redefine all columns to change types\r\n.alter table PricingUnits (\r\n x_PricingUnitDescription: string,\r\n x_PricingBlockSize: real,\r\n PricingUnit: string\r\n)\r\n\r\n// Regions\r\n.create-merge table Regions(\r\n ResourceLocation: string,\r\n RegionId: string,\r\n RegionName: string\r\n)\r\n\r\n// ResourceTypes\r\n.create-merge table ResourceTypes(\r\n x_ResourceType: string,\r\n SingularDisplayName: string,\r\n PluralDisplayName: string,\r\n LowerSingularDisplayName: string,\r\n LowerPluralDisplayName: string,\r\n IsPreview: bool,\r\n Description: string,\r\n IconUri: string\r\n)\r\n\r\n// Services\r\n.create-merge table Services(\r\n x_ConsumedService: string,\r\n x_ResourceType: string,\r\n ServiceName: string,\r\n ServiceCategory: string,\r\n ServiceSubcategory: string,\r\n PublisherName: string,\r\n x_PublisherCategory: string,\r\n x_Environment: string,\r\n x_ServiceModel: string\r\n)\r\n\r\n//----------------------------------------------------------------------------------------------------------------------\r\n\r\n// parse_resourceid\r\n.create-or-alter function\r\nwith (docstring = 'Parses an Azure resource ID to extract resource attributes like the name, type, resource group, and subaccount ID.', folder = 'Common')\r\nparse_resourceid(resourceId: string) {\r\n let ResourceId = tolower(resourceId);\r\n // let ResourceId = tolower('/providers/Microsoft.BillingBenefits/savingsPlanOrders/2d2e284b-0638-427e-b8c6-1b874d4f17c8/sp/xxx');\r\n let SubAccountId = tostring(extract('/subscriptions/[^/]+', 1, ResourceId));\r\n let x_ResourceGroupName = tostring(extract('/resourcegroups/[^/]+', 1, ResourceId));\r\n let providerPath = iff(ResourceId !contains '/providers/', '', split(iff(ResourceId startswith '/subscriptions/', strcat('/providers/microsoft.resources/', ResourceId), ResourceId), '/providers/')[-1]);\r\n let x_ResourceProvider = iff(isempty(providerPath), '', split(providerPath, '/')[0]);\r\n let tmp_ResourceProviderPath = iff(isempty(providerPath), '', substring(providerPath, strlen(x_ResourceProvider) + 1));\r\n let segments = split(tmp_ResourceProviderPath, '/');\r\n let ResourceName = trim(@'/+', replace_string(strcat_array(array_iff(\r\n dynamic([false, true, false, true, false, true, false, true, false, true, false, true, false, true, false, true, false, true]),\r\n segments, dynamic([])), '/'), '//', '/'));\r\n let x_ResourceTypePath = trim(@'/+', replace_string(strcat_array(array_iff(\r\n dynamic([true, false, true, false, true, false, true, false, true, false, true, false, true, false, true, false, true, false]),\r\n segments, dynamic([])), '/'), '//', '/'));\r\n let xRT = iff(isempty(x_ResourceProvider) or isempty(x_ResourceTypePath), '', strcat(x_ResourceProvider, '/', x_ResourceTypePath));\r\n // TODO: Remove ResourceType in 0.9\r\n bag_pack('ResourceId', ResourceId, 'ResourceName', ResourceName, 'ResourceType', xRT, 'SubAccountId', SubAccountId, 'x_ResourceGroupName', x_ResourceGroupName, 'x_ResourceProvider', x_ResourceProvider, 'x_ResourceType', xRT)\r\n}\r\n", + "$fxv#8": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Ingestion database\r\n// Used for data ingestion, normalization, and cleansing.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n//===| ActualCosts |====================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// ActualCosts_raw table -- Create the table if it doesn't exist\r\n.create-merge table ActualCosts_raw ( ignore: string )\r\n\r\n// ActualCosts_raw table -- Remove all columns to allow changing column types\r\n.alter table ActualCosts_raw ( ignore: string )\r\n\r\n// ActualCosts_raw table -- Redefine all columns\r\n.alter table ActualCosts_raw (\r\n AccountName: string,\r\n AccountOwnerId: string,\r\n AdditionalInfo: string,\r\n AvailabilityZone: string,\r\n BillingAccountId: string, \r\n BillingAccountName: string,\r\n BillingCurrency: string,\r\n BillingPeriodEndDate: datetime,\r\n BillingPeriodStartDate: datetime,\r\n BillingProfileId: string,\r\n BillingProfileName: string,\r\n ChargeType: string,\r\n ConsumedService: string,\r\n CostCenter: string,\r\n Cost: real,\r\n Date: datetime,\r\n EffectivePrice: real,\r\n Frequency: string,\r\n InvoiceSection: string,\r\n InvoiceSectionId: string,\r\n IsAzureCreditEligible: bool,\r\n MeterCategory: string,\r\n MeterId: string,\r\n MeterName: string,\r\n MeterRegion: string,\r\n MeterSubCategory: string,\r\n OfferId: string,\r\n PartNumber: string,\r\n PlanName: string,\r\n Product: string,\r\n ProductOrderId: string,\r\n ProductOrderName: string,\r\n PublisherName: string,\r\n PublisherType: string,\r\n Quantity: real,\r\n ReservationId: string,\r\n ReservationName: string,\r\n ResourceGroup: string,\r\n ResourceId: string,\r\n ResourceLocation: string,\r\n ResourceName: string,\r\n ServiceFamily: string,\r\n ServiceInfo1: string,\r\n ServiceInfo2: string,\r\n SubscriptionId: string,\r\n SubscriptionName: string,\r\n Tags: string,\r\n Term: string,\r\n UnitOfMeasure: string,\r\n UnitPrice: real\r\n)\r\n\r\n// ActualCosts_raw ingestion mapping\r\n.create-or-alter table ActualCosts_raw ingestion parquet mapping \"ActualCosts_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"AccountName\", \"Properties\": { \"Field\": \"AccountName\" } },\r\n { \"Column\": \"AccountOwnerId\", \"Properties\": { \"Field\": \"AccountOwnerId\" } },\r\n { \"Column\": \"AdditionalInfo\", \"Properties\": { \"Field\": \"AdditionalInfo\" } },\r\n { \"Column\": \"AvailabilityZone\", \"Properties\": { \"Field\": \"AvailabilityZone\" } },\r\n { \"Column\": \"BillingAccountId\", \"Properties\": { \"Field\": \"BillingAccountId\" } },\r\n { \"Column\": \"BillingAccountName\", \"Properties\": { \"Field\": \"BillingAccountName\" } },\r\n { \"Column\": \"BillingCurrency\", \"Properties\": { \"Field\": \"BillingCurrency\" } },\r\n { \"Column\": \"BillingPeriodEndDate\", \"Properties\": { \"Field\": \"BillingPeriodEndDate\" } },\r\n { \"Column\": \"BillingPeriodStartDate\", \"Properties\": { \"Field\": \"BillingPeriodStartDate\" } },\r\n { \"Column\": \"BillingProfileId\", \"Properties\": { \"Field\": \"BillingProfileId\" } },\r\n { \"Column\": \"BillingProfileName\", \"Properties\": { \"Field\": \"BillingProfileName\" } },\r\n { \"Column\": \"ChargeType\", \"Properties\": { \"Field\": \"ChargeType\" } },\r\n { \"Column\": \"ConsumedService\", \"Properties\": { \"Field\": \"ConsumedService\" } },\r\n { \"Column\": \"CostCenter\", \"Properties\": { \"Field\": \"CostCenter\" } },\r\n { \"Column\": \"Cost\", \"Properties\": { \"Field\": \"Cost\" } },\r\n { \"Column\": \"Date\", \"Properties\": { \"Field\": \"Date\" } },\r\n { \"Column\": \"EffectivePrice\", \"Properties\": { \"Field\": \"EffectivePrice\" } },\r\n { \"Column\": \"Frequency\", \"Properties\": { \"Field\": \"Frequency\" } },\r\n { \"Column\": \"InvoiceSection\", \"Properties\": { \"Field\": \"InvoiceSection\" } },\r\n { \"Column\": \"InvoiceSectionId\", \"Properties\": { \"Field\": \"InvoiceSectionId\" } },\r\n { \"Column\": \"IsAzureCreditEligible\", \"Properties\": { \"Field\": \"IsAzureCreditEligible\" } },\r\n { \"Column\": \"MeterCategory\", \"Properties\": { \"Field\": \"MeterCategory\" } },\r\n { \"Column\": \"MeterId\", \"Properties\": { \"Field\": \"MeterId\" } },\r\n { \"Column\": \"MeterName\", \"Properties\": { \"Field\": \"MeterName\" } },\r\n { \"Column\": \"MeterRegion\", \"Properties\": { \"Field\": \"MeterRegion\" } },\r\n { \"Column\": \"MeterSubCategory\", \"Properties\": { \"Field\": \"MeterSubCategory\" } },\r\n { \"Column\": \"OfferId\", \"Properties\": { \"Field\": \"OfferId\" } },\r\n { \"Column\": \"PartNumber\", \"Properties\": { \"Field\": \"PartNumber\" } },\r\n { \"Column\": \"PlanName\", \"Properties\": { \"Field\": \"PlanName\" } },\r\n { \"Column\": \"Product\", \"Properties\": { \"Field\": \"Product\" } },\r\n { \"Column\": \"ProductOrderId\", \"Properties\": { \"Field\": \"ProductOrderId\" } },\r\n { \"Column\": \"ProductOrderName\", \"Properties\": { \"Field\": \"ProductOrderName\" } },\r\n { \"Column\": \"PublisherName\", \"Properties\": { \"Field\": \"PublisherName\" } },\r\n { \"Column\": \"PublisherType\", \"Properties\": { \"Field\": \"PublisherType\" } },\r\n { \"Column\": \"Quantity\", \"Properties\": { \"Field\": \"Quantity\" } },\r\n { \"Column\": \"ReservationId\", \"Properties\": { \"Field\": \"ReservationId\" } },\r\n { \"Column\": \"ReservationName\", \"Properties\": { \"Field\": \"ReservationName\" } },\r\n { \"Column\": \"ResourceGroup\", \"Properties\": { \"Field\": \"ResourceGroup\" } },\r\n { \"Column\": \"ResourceId\", \"Properties\": { \"Field\": \"ResourceId\" } },\r\n { \"Column\": \"ResourceLocation\", \"Properties\": { \"Field\": \"ResourceLocation\" } },\r\n { \"Column\": \"ResourceName\", \"Properties\": { \"Field\": \"ResourceName\" } },\r\n { \"Column\": \"ServiceFamily\", \"Properties\": { \"Field\": \"ServiceFamily\" } },\r\n { \"Column\": \"ServiceInfo1\", \"Properties\": { \"Field\": \"ServiceInfo1\" } },\r\n { \"Column\": \"ServiceInfo2\", \"Properties\": { \"Field\": \"ServiceInfo2\" } },\r\n { \"Column\": \"SubscriptionId\", \"Properties\": { \"Field\": \"SubscriptionId\" } },\r\n { \"Column\": \"SubscriptionName\", \"Properties\": { \"Field\": \"SubscriptionName\" } },\r\n { \"Column\": \"Tags\", \"Properties\": { \"Field\": \"Tags\" } },\r\n { \"Column\": \"Term\", \"Properties\": { \"Field\": \"Term\" } },\r\n { \"Column\": \"UnitOfMeasure\", \"Properties\": { \"Field\": \"UnitOfMeasure\" } },\r\n { \"Column\": \"UnitPrice\", \"Properties\": { \"Field\": \"UnitPrice\" } }\r\n]\r\n```\r\n\r\n// ActualCosts_raw retention policy (clear historical data)\r\n.alter-merge table ActualCosts_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// ActualCosts_raw retention policy (set the user-defined retention period)\r\n.alter-merge table ActualCosts_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable ActualCosts_raw streaming ingestion (required for Fabric)\r\n.alter table ActualCosts_raw policy streamingingestion disable\r\n\r\n\r\n//===| AmortizedCosts |=================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// AmortizedCosts_raw table -- Create the table if it doesn't exist\r\n.create-merge table AmortizedCosts_raw ( ignore: string )\r\n\r\n// AmortizedCosts_raw table -- Remove all columns to allow changing column types\r\n.alter table AmortizedCosts_raw ( ignore: string )\r\n\r\n// AmortizedCosts_raw table -- Redefine all columns\r\n.alter table AmortizedCosts_raw (\r\n AccountName: string,\r\n AccountOwnerId: string,\r\n AdditionalInfo: string,\r\n AvailabilityZone: string,\r\n BillingAccountId: string, \r\n BillingAccountName: string,\r\n BillingCurrency: string,\r\n BillingPeriodEndDate: datetime,\r\n BillingPeriodStartDate: datetime,\r\n BillingProfileId: string,\r\n BillingProfileName: string,\r\n ChargeType: string,\r\n ConsumedService: string,\r\n CostCenter: string,\r\n Cost: real,\r\n Date: datetime,\r\n EffectivePrice: real,\r\n Frequency: string,\r\n InvoiceSection: string,\r\n InvoiceSectionId: string,\r\n IsAzureCreditEligible: bool,\r\n MeterCategory: string,\r\n MeterId: string,\r\n MeterName: string,\r\n MeterRegion: string,\r\n MeterSubCategory: string,\r\n OfferId: string,\r\n PartNumber: string,\r\n PlanName: string,\r\n Product: string,\r\n ProductOrderId: string,\r\n ProductOrderName: string,\r\n PublisherName: string,\r\n PublisherType: string,\r\n Quantity: real,\r\n ReservationId: string,\r\n ReservationName: string,\r\n ResourceGroup: string,\r\n ResourceId: string,\r\n ResourceLocation: string,\r\n ResourceName: string,\r\n ServiceFamily: string,\r\n ServiceInfo1: string,\r\n ServiceInfo2: string,\r\n SubscriptionId: string,\r\n SubscriptionName: string,\r\n Tags: string,\r\n Term: string,\r\n UnitOfMeasure: string,\r\n UnitPrice: real\r\n)\r\n\r\n// AmortizedCosts_raw ingestion mapping\r\n.create-or-alter table AmortizedCosts_raw ingestion parquet mapping \"AmortizedCosts_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"AccountName\", \"Properties\": { \"Field\": \"AccountName\" } },\r\n { \"Column\": \"AccountOwnerId\", \"Properties\": { \"Field\": \"AccountOwnerId\" } },\r\n { \"Column\": \"AdditionalInfo\", \"Properties\": { \"Field\": \"AdditionalInfo\" } },\r\n { \"Column\": \"AvailabilityZone\", \"Properties\": { \"Field\": \"AvailabilityZone\" } },\r\n { \"Column\": \"BillingAccountId\", \"Properties\": { \"Field\": \"BillingAccountId\" } },\r\n { \"Column\": \"BillingAccountName\", \"Properties\": { \"Field\": \"BillingAccountName\" } },\r\n { \"Column\": \"BillingCurrency\", \"Properties\": { \"Field\": \"BillingCurrency\" } },\r\n { \"Column\": \"BillingPeriodEndDate\", \"Properties\": { \"Field\": \"BillingPeriodEndDate\" } },\r\n { \"Column\": \"BillingPeriodStartDate\", \"Properties\": { \"Field\": \"BillingPeriodStartDate\" } },\r\n { \"Column\": \"BillingProfileId\", \"Properties\": { \"Field\": \"BillingProfileId\" } },\r\n { \"Column\": \"BillingProfileName\", \"Properties\": { \"Field\": \"BillingProfileName\" } },\r\n { \"Column\": \"ChargeType\", \"Properties\": { \"Field\": \"ChargeType\" } },\r\n { \"Column\": \"ConsumedService\", \"Properties\": { \"Field\": \"ConsumedService\" } },\r\n { \"Column\": \"CostCenter\", \"Properties\": { \"Field\": \"CostCenter\" } },\r\n { \"Column\": \"Cost\", \"Properties\": { \"Field\": \"Cost\" } },\r\n { \"Column\": \"Date\", \"Properties\": { \"Field\": \"Date\" } },\r\n { \"Column\": \"EffectivePrice\", \"Properties\": { \"Field\": \"EffectivePrice\" } },\r\n { \"Column\": \"Frequency\", \"Properties\": { \"Field\": \"Frequency\" } },\r\n { \"Column\": \"InvoiceSection\", \"Properties\": { \"Field\": \"InvoiceSection\" } },\r\n { \"Column\": \"InvoiceSectionId\", \"Properties\": { \"Field\": \"InvoiceSectionId\" } },\r\n { \"Column\": \"IsAzureCreditEligible\", \"Properties\": { \"Field\": \"IsAzureCreditEligible\" } },\r\n { \"Column\": \"MeterCategory\", \"Properties\": { \"Field\": \"MeterCategory\" } },\r\n { \"Column\": \"MeterId\", \"Properties\": { \"Field\": \"MeterId\" } },\r\n { \"Column\": \"MeterName\", \"Properties\": { \"Field\": \"MeterName\" } },\r\n { \"Column\": \"MeterRegion\", \"Properties\": { \"Field\": \"MeterRegion\" } },\r\n { \"Column\": \"MeterSubCategory\", \"Properties\": { \"Field\": \"MeterSubCategory\" } },\r\n { \"Column\": \"OfferId\", \"Properties\": { \"Field\": \"OfferId\" } },\r\n { \"Column\": \"PartNumber\", \"Properties\": { \"Field\": \"PartNumber\" } },\r\n { \"Column\": \"PlanName\", \"Properties\": { \"Field\": \"PlanName\" } },\r\n { \"Column\": \"Product\", \"Properties\": { \"Field\": \"Product\" } },\r\n { \"Column\": \"ProductOrderId\", \"Properties\": { \"Field\": \"ProductOrderId\" } },\r\n { \"Column\": \"ProductOrderName\", \"Properties\": { \"Field\": \"ProductOrderName\" } },\r\n { \"Column\": \"PublisherName\", \"Properties\": { \"Field\": \"PublisherName\" } },\r\n { \"Column\": \"PublisherType\", \"Properties\": { \"Field\": \"PublisherType\" } },\r\n { \"Column\": \"Quantity\", \"Properties\": { \"Field\": \"Quantity\" } },\r\n { \"Column\": \"ReservationId\", \"Properties\": { \"Field\": \"ReservationId\" } },\r\n { \"Column\": \"ReservationName\", \"Properties\": { \"Field\": \"ReservationName\" } },\r\n { \"Column\": \"ResourceGroup\", \"Properties\": { \"Field\": \"ResourceGroup\" } },\r\n { \"Column\": \"ResourceId\", \"Properties\": { \"Field\": \"ResourceId\" } },\r\n { \"Column\": \"ResourceLocation\", \"Properties\": { \"Field\": \"ResourceLocation\" } },\r\n { \"Column\": \"ResourceName\", \"Properties\": { \"Field\": \"ResourceName\" } },\r\n { \"Column\": \"ServiceFamily\", \"Properties\": { \"Field\": \"ServiceFamily\" } },\r\n { \"Column\": \"ServiceInfo1\", \"Properties\": { \"Field\": \"ServiceInfo1\" } },\r\n { \"Column\": \"ServiceInfo2\", \"Properties\": { \"Field\": \"ServiceInfo2\" } },\r\n { \"Column\": \"SubscriptionId\", \"Properties\": { \"Field\": \"SubscriptionId\" } },\r\n { \"Column\": \"SubscriptionName\", \"Properties\": { \"Field\": \"SubscriptionName\" } },\r\n { \"Column\": \"Tags\", \"Properties\": { \"Field\": \"Tags\" } },\r\n { \"Column\": \"Term\", \"Properties\": { \"Field\": \"Term\" } },\r\n { \"Column\": \"UnitOfMeasure\", \"Properties\": { \"Field\": \"UnitOfMeasure\" } },\r\n { \"Column\": \"UnitPrice\", \"Properties\": { \"Field\": \"UnitPrice\" } }\r\n]\r\n```\r\n\r\n// AmortizedCosts_raw retention policy (clear historical data)\r\n.alter-merge table AmortizedCosts_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// AmortizedCosts_raw retention policy (set the user-defined retention period)\r\n.alter-merge table AmortizedCosts_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable AmortizedCosts_raw streaming ingestion (required for Fabric)\r\n.alter table AmortizedCosts_raw policy streamingingestion disable\r\n\r\n\r\n//===| CommitmentDiscountUsage |========================================================================================\r\n// Supported versions:\r\n// - MS EA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-ea\r\n// - MS MCA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-mca\r\n//======================================================================================================================\r\n\r\n// CommitmentDiscountUsage_raw table -- Create the table if it doesn't exist\r\n.create-merge table CommitmentDiscountUsage_raw ( ignore: string )\r\n\r\n// CommitmentDiscountUsage_raw table -- Remove all columns to allow changing column types\r\n.alter table CommitmentDiscountUsage_raw ( ignore: string )\r\n\r\n// CommitmentDiscountUsage_raw table -- Redefine all columns\r\n.alter table CommitmentDiscountUsage_raw (\r\n InstanceFlexibilityGroup: string,\r\n InstanceFlexibilityRatio: real,\r\n InstanceId: string,\r\n Kind: string,\r\n ReservationId: string,\r\n ReservationOrderId: string,\r\n ReservedHours: real,\r\n SkuName: string,\r\n TotalReservedQuantity: real,\r\n UsageDate: datetime,\r\n UsedHours: real,\r\n x_SourceName: string, // Hubs v1_0+\r\n x_SourceProvider: string, // Hubs v1_0+\r\n x_SourceType: string, // Hubs v1_0+\r\n x_SourceVersion: string // Hubs v1_0+\r\n)\r\n\r\n// CommitmentDiscountUsage_raw ingestion mapping\r\n.create-or-alter table CommitmentDiscountUsage_raw ingestion parquet mapping \"CommitmentDiscountUsage_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"InstanceFlexibilityGroup\", \"Properties\": { \"Field\": \"InstanceFlexibilityGroup\" } },\r\n { \"Column\": \"InstanceFlexibilityRatio\", \"Properties\": { \"Field\": \"InstanceFlexibilityRatio\" } },\r\n { \"Column\": \"InstanceId\", \"Properties\": { \"Field\": \"InstanceId\" } },\r\n { \"Column\": \"Kind\", \"Properties\": { \"Field\": \"Kind\" } },\r\n { \"Column\": \"ReservationId\", \"Properties\": { \"Field\": \"ReservationId\" } },\r\n { \"Column\": \"ReservationOrderId\", \"Properties\": { \"Field\": \"ReservationOrderId\" } },\r\n { \"Column\": \"ReservedHours\", \"Properties\": { \"Field\": \"ReservedHours\" } },\r\n { \"Column\": \"SkuName\", \"Properties\": { \"Field\": \"SkuName\" } },\r\n { \"Column\": \"TotalReservedQuantity\", \"Properties\": { \"Field\": \"TotalReservedQuantity\" } },\r\n { \"Column\": \"UsageDate\", \"Properties\": { \"Field\": \"UsageDate\" } },\r\n { \"Column\": \"UsedHours\", \"Properties\": { \"Field\": \"UsedHours\" } },\r\n { \"Column\": \"x_SourceName\", \"Properties\": { \"Field\": \"x_SourceName\" } },\r\n { \"Column\": \"x_SourceProvider\", \"Properties\": { \"Field\": \"x_SourceProvider\" } },\r\n { \"Column\": \"x_SourceType\", \"Properties\": { \"Field\": \"x_SourceType\" } },\r\n { \"Column\": \"x_SourceVersion\", \"Properties\": { \"Field\": \"x_SourceVersion\" } }\r\n]\r\n```\r\n\r\n// CommitmentDiscountUsage_raw retention policy (clear historical data)\r\n.alter-merge table CommitmentDiscountUsage_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// CommitmentDiscountUsage_raw retention policy (set the user-defined retention period)\r\n.alter-merge table CommitmentDiscountUsage_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable CommitmentDiscountUsage_raw streaming ingestion (required for Fabric)\r\n.alter table CommitmentDiscountUsage_raw policy streamingingestion disable\r\n\r\n\r\n//===| Costs |==========================================================================================================\r\n// Supported versions:\r\n// - MS: 1.0, 1.0-preview(v1) -- See https://aka.ms/costmgmt/exports/focus\r\n// - AWS: 1.0 -- See https://docs.aws.amazon.com/cur/latest/userguide/table-dictionary-focus-1-0-aws-columns.html\r\n// - GCP: Jan-Jun 2024 -- See https://cloud.google.com/resources/google-cloud-focus?e=48754805&hl=en\r\n// Links to (Aug 2024): https://services.google.com/fh/files/misc/focus_guide_v1.pdf\r\n// See also:\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/standard-usage\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/detailed-usage\r\n// - OCI: 1.0 -- See https://docs.oracle.com/iaas/Content/Billing/Concepts/costusagereportsoverview.htm#costreports__focus-cost-report-schema\r\n// - Tencent: 1.0 -- See https://www.tencentcloud.com/document/product/555/67495 / https://www.tencentcloud.com/document/product/555/67496\r\n// - Alibaba: 1.0 -- See https://www.alibabacloud.com/help/en/user-center/user-guide/export-alibaba-cloud-standard-billing-focus\r\n//\r\n// Support for non-Azure data is limited to ingestion only. Data is not transformed across versions.\r\n//======================================================================================================================\r\n\r\n// Costs_raw table -- Create the table if it doesn't exist\r\n.create-merge table Costs_raw ( ignore: string )\r\n\r\n// Costs_raw table -- Remove all columns to allow changing column types\r\n.alter table Costs_raw ( ignore: string )\r\n\r\n// Costs_raw table -- Redefine all columns\r\n.alter table Costs_raw (\r\n AvailabilityZone: string, // FOCUS 0.5+\r\n BilledCost: real, // FOCUS 0.5+\r\n BillingAccountId: string, // FOCUS 0.5+\r\n BillingAccountName: string, // FOCUS 0.5+\r\n BillingAccountType: string, // Azure 1.0-preview(v1)+\r\n BillingCurrency: string, // FOCUS 0.5+\r\n BillingPeriodEnd: datetime, // FOCUS 0.5+\r\n BillingPeriodStart: datetime, // FOCUS 0.5+\r\n CapacityReservationId: string, // FOCUS 1.1+\r\n CapacityReservationStatus: string, // FOCUS 1.1+\r\n ChargeCategory: string, // FOCUS 1.0-preview+\r\n ChargeClass: string, // FOCUS 1.0+\r\n ChargeDescription: string, // FOCUS 1.0+\r\n ChargeFrequency: string, // FOCUS 1.0+\r\n ChargePeriodEnd: datetime, // FOCUS 0.5+\r\n ChargePeriodStart: datetime, // FOCUS 0.5+\r\n ChargeSubcategory: string, // FOCUS 1.0-preview only\r\n CommitmentDiscountCategory: string, // FOCUS 1.0-preview+\r\n CommitmentDiscountId: string, // FOCUS 1.0-preview+\r\n CommitmentDiscountName: string, // FOCUS 1.0-preview+\r\n CommitmentDiscountQuantity: real, // FOCUS 1.1+\r\n CommitmentDiscountStatus: string, // FOCUS 1.0+\r\n CommitmentDiscountType: string, // FOCUS 1.0-preview+\r\n CommitmentDiscountUnit: string, // FOCUS 1.1+\r\n ConsumedQuantity: real, // FOCUS 1.0+\r\n ConsumedUnit: string, // FOCUS 1.0+\r\n ContractedCost: real, // FOCUS 1.0+\r\n ContractedUnitPrice: real, // FOCUS 1.0+\r\n EffectiveCost: real, // FOCUS 1.0-preview+\r\n InvoiceId: string, // FOCUS 1.2+\r\n InvoiceIssuerName: string, // FOCUS 0.5+\r\n ListCost: real, // FOCUS 1.0-preview+\r\n ListUnitPrice: real, // FOCUS 1.0-preview+\r\n PricingCategory: string, // FOCUS 1.0-preview+\r\n PricingCurrency: string, // FOCUS 1.2+\r\n PricingQuantity: real, // FOCUS 1.0-preview+\r\n PricingUnit: string, // FOCUS 1.0-preview+\r\n ProviderName: string, // FOCUS 0.5+\r\n PublisherName: string, // FOCUS 0.5+\r\n Region: string, // FOCUS 0.5-1.0-preview (deprecated)\r\n RegionId: string, // FOCUS 1.0+\r\n RegionName: string, // FOCUS 1.0+\r\n ResourceId: string, // FOCUS 0.5+\r\n ResourceName: string, // FOCUS 0.5+\r\n ResourceType: string, // FOCUS 1.0-preview+\r\n ServiceCategory: string, // FOCUS 0.5+\r\n ServiceName: string, // FOCUS 0.5+\r\n ServiceSubcategory: string, // FOCUS 1.1+\r\n SkuId: string, // FOCUS 1.0-preview+\r\n SkuMeter: string, // FOCUS 1.1+\r\n SkuPriceDetails: string, // FOCUS 1.1+\r\n SkuPriceId: string, // FOCUS 1.0-preview+\r\n SubAccountId: string, // FOCUS 0.5+\r\n SubAccountName: string, // FOCUS 0.5+\r\n SubAccountType: string, // Azure 1.0-preview(v1)+\r\n Tags: string, // FOCUS 1.0-preview+\r\n UsageAmount: real, // GCP Jan 2024 -- Removed Mar 2024 (UsageQuantity)\r\n UsageQuantity: real, // FOCUS 1.0-preview only\r\n UsageUnit: string, // FOCUS 1.0-preview only\r\n x_AccountId: string, // Azure 1.0-preview(v1)+\r\n x_AccountName: string, // Azure 1.0-preview(v1)+\r\n x_AccountOwnerId: string, // Azure 1.0-preview(v1)+\r\n x_AmortizationClass: string, // Azure 1.2-preview+\r\n x_BilledCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_BilledUnitPrice: real, // Azure 1.0-preview(v1)+\r\n x_BillingAccountId: string, // Azure 1.0-preview(v1)+\r\n x_BillingAccountName: string, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRate: real, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRateDate: datetime, // Azure 1.0-preview(v1)+\r\n x_BillingItemCode: string, // Alibaba 1.0+\r\n x_BillingItemName: string, // Alibaba 1.0+\r\n x_BillingProfileId: string, // Azure 1.0-preview(v1)+\r\n x_BillingProfileName: string, // Azure 1.0-preview(v1)+\r\n x_ChargeId: string, // Azure 1.0-preview(v1) only\r\n x_CommodityCode: string, // Alibaba 1.0+\r\n x_CommodityName: string, // Alibaba 1.0+\r\n x_ComponentName: string, // Tencent 1.0+\r\n x_ComponentType: string, // Tencent 1.0+\r\n x_ContractedCostInUsd: real, // Azure 1.0+\r\n x_Cost: real, // GCP Jan 2024 -- Removed Jun 2024 (ContractedCost)\r\n x_CostAllocationRuleName: string, // Azure 1.0-preview(v1)+\r\n x_CostCategories: string, // AWS 1.0 (JSON)\r\n x_CostCenter: string, // Azure 1.0-preview(v1)+\r\n x_CostType: string, // GCP Jan 2024\r\n x_Credits: string, // GCP Jan 2024\r\n x_CurrencyConversionRate: real, // GCP Jun 2024\r\n x_CustomerId: string, // Azure 1.0-preview(v1)+\r\n x_CustomerName: string, // Azure 1.0-preview(v1)+\r\n x_Discount: string, // AWS 1.0 (JSON)\r\n x_EffectiveCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_EffectiveUnitPrice: real, // Azure 1.0-preview(v1)+\r\n x_ExportTime: datetime, // GCP Jan 2024 / Tencent 1.0+\r\n x_InstanceID: string, // Alibaba 1.0+\r\n x_InvoiceId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceIssuerId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionName: string, // Azure 1.0-preview(v1)+\r\n x_ListCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_Location: string, // GCP Jan 2024\r\n x_OnDemandCost: real, // Azure 1.0-preview(v1) only\r\n x_OnDemandCostInUsd: real, // Azure 1.0-preview(v1) only\r\n x_OnDemandUnitPrice: real, // Azure 1.0-preview(v1) only\r\n x_Operation: string, // AWS 1.0\r\n x_OwnerAccountID: string, // Tencent 1.0+\r\n x_PartnerCreditApplied: string, // Azure 1.0-preview(v1)+\r\n x_PartnerCreditRate: string, // Azure 1.0-preview(v1)+\r\n x_PricingBlockSize: real, // Azure 1.0-preview(v1)+\r\n x_PricingCurrency: string, // Azure 1.0-preview(v1)-1.0r2\r\n x_PricingSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_PricingUnitDescription: string, // Azure 1.0-preview(v1)+\r\n x_Project: string, // GCP Jan 2024\r\n x_PublisherCategory: string, // Azure 1.0-preview(v1)+\r\n x_PublisherId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceGroupName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceType: string, // Azure 1.0-preview(v1)+\r\n x_ServiceCode: string, // AWS 1.0\r\n x_ServiceId: string, // GCP Jan 2024\r\n x_ServiceModel: string, // Azure 1.2-preview+\r\n x_ServicePeriodEnd: datetime, // Azure 1.0-preview(v1)+\r\n x_ServicePeriodStart: datetime, // Azure 1.0-preview(v1)+\r\n x_SkuDescription: string, // Azure 1.0-preview(v1)+\r\n x_SkuDetails: string, // Azure 1.0-preview(v1)-1.2-preview\r\n x_SkuIsCreditEligible: bool, // Azure 1.0-preview(v1)+\r\n x_SkuMeterCategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterId: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterName: string, // Azure 1.0-preview(v1)-1.0r2\r\n x_SkuMeterSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuOfferId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOrderId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOrderName: string, // Azure 1.0-preview(v1)+\r\n x_SkuPartNumber: string, // Azure 1.0-preview(v1)+\r\n x_SkuPlanName: string, // Azure 1.2-preview+\r\n x_SkuRegion: string, // Azure 1.0-preview(v1)+\r\n x_SkuServiceFamily: string, // Azure 1.0-preview(v1)+\r\n x_SkuTerm: int, // Azure 1.0-preview(v1)+\r\n x_SkuTier: string, // Azure 1.0-preview(v1)+\r\n x_SourceName: string, // Hubs v1_0+\r\n x_SourceProvider: string, // Hubs v1_0+\r\n x_SourceType: string, // Hubs v1_0+\r\n x_SourceVersion: string, // Hubs v1_0+\r\n x_SubproductName: string, // Tencent 1.0+ // cSpell:ignore Subproduct\r\n x_UsageType: string // AWS 1.0\r\n)\r\n\r\n// Costs_raw ingestion mapping\r\n.create-or-alter table Costs_raw ingestion parquet mapping \"Costs_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"AvailabilityZone\", \"Properties\": { \"Field\": \"AvailabilityZone\" } },\r\n { \"Column\": \"BilledCost\", \"Properties\": { \"Field\": \"BilledCost\" } },\r\n { \"Column\": \"BillingAccountId\", \"Properties\": { \"Field\": \"BillingAccountId\" } },\r\n { \"Column\": \"BillingAccountName\", \"Properties\": { \"Field\": \"BillingAccountName\" } },\r\n { \"Column\": \"BillingAccountType\", \"Properties\": { \"Field\": \"BillingAccountType\" } },\r\n { \"Column\": \"BillingCurrency\", \"Properties\": { \"Field\": \"BillingCurrency\" } },\r\n { \"Column\": \"BillingPeriodEnd\", \"Properties\": { \"Field\": \"BillingPeriodEnd\" } },\r\n { \"Column\": \"BillingPeriodStart\", \"Properties\": { \"Field\": \"BillingPeriodStart\" } },\r\n { \"Column\": \"CapacityReservationId\", \"Properties\": { \"Field\": \"CapacityReservationId\" } },\r\n { \"Column\": \"CapacityReservationStatus\", \"Properties\": { \"Field\": \"CapacityReservationStatus\" } },\r\n { \"Column\": \"ChargeCategory\", \"Properties\": { \"Field\": \"ChargeCategory\" } },\r\n { \"Column\": \"ChargeClass\", \"Properties\": { \"Field\": \"ChargeClass\" } },\r\n { \"Column\": \"ChargeDescription\", \"Properties\": { \"Field\": \"ChargeDescription\" } },\r\n { \"Column\": \"ChargeFrequency\", \"Properties\": { \"Field\": \"ChargeFrequency\" } },\r\n { \"Column\": \"ChargePeriodEnd\", \"Properties\": { \"Field\": \"ChargePeriodEnd\" } },\r\n { \"Column\": \"ChargePeriodStart\", \"Properties\": { \"Field\": \"ChargePeriodStart\" } },\r\n { \"Column\": \"ChargeSubcategory\", \"Properties\": { \"Field\": \"ChargeSubcategory\" } },\r\n { \"Column\": \"CommitmentDiscountCategory\", \"Properties\": { \"Field\": \"CommitmentDiscountCategory\" } },\r\n { \"Column\": \"CommitmentDiscountId\", \"Properties\": { \"Field\": \"CommitmentDiscountId\" } },\r\n { \"Column\": \"CommitmentDiscountName\", \"Properties\": { \"Field\": \"CommitmentDiscountName\" } },\r\n { \"Column\": \"CommitmentDiscountQuantity\", \"Properties\": { \"Field\": \"CommitmentDiscountQuantity\" } },\r\n { \"Column\": \"CommitmentDiscountStatus\", \"Properties\": { \"Field\": \"CommitmentDiscountStatus\" } },\r\n { \"Column\": \"CommitmentDiscountType\", \"Properties\": { \"Field\": \"CommitmentDiscountType\" } },\r\n { \"Column\": \"CommitmentDiscountUnit\", \"Properties\": { \"Field\": \"CommitmentDiscountUnit\" } },\r\n { \"Column\": \"ConsumedQuantity\", \"Properties\": { \"Field\": \"ConsumedQuantity\" } },\r\n { \"Column\": \"ConsumedUnit\", \"Properties\": { \"Field\": \"ConsumedUnit\" } },\r\n { \"Column\": \"ContractedCost\", \"Properties\": { \"Field\": \"ContractedCost\" } },\r\n { \"Column\": \"ContractedUnitPrice\", \"Properties\": { \"Field\": \"ContractedUnitPrice\" } },\r\n { \"Column\": \"EffectiveCost\", \"Properties\": { \"Field\": \"EffectiveCost\" } },\r\n { \"Column\": \"InvoiceId\", \"Properties\": { \"Field\": \"InvoiceId\" } },\r\n { \"Column\": \"InvoiceIssuerName\", \"Properties\": { \"Field\": \"InvoiceIssuerName\" } },\r\n { \"Column\": \"ListCost\", \"Properties\": { \"Field\": \"ListCost\" } },\r\n { \"Column\": \"ListUnitPrice\", \"Properties\": { \"Field\": \"ListUnitPrice\" } },\r\n { \"Column\": \"PricingCategory\", \"Properties\": { \"Field\": \"PricingCategory\" } },\r\n { \"Column\": \"PricingCurrency\", \"Properties\": { \"Field\": \"PricingCurrency\" } },\r\n { \"Column\": \"PricingQuantity\", \"Properties\": { \"Field\": \"PricingQuantity\" } },\r\n { \"Column\": \"PricingUnit\", \"Properties\": { \"Field\": \"PricingUnit\" } },\r\n { \"Column\": \"ProviderName\", \"Properties\": { \"Field\": \"ProviderName\" } },\r\n { \"Column\": \"PublisherName\", \"Properties\": { \"Field\": \"PublisherName\" } },\r\n { \"Column\": \"Region\", \"Properties\": { \"Field\": \"Region\" } },\r\n { \"Column\": \"RegionId\", \"Properties\": { \"Field\": \"RegionId\" } },\r\n { \"Column\": \"RegionName\", \"Properties\": { \"Field\": \"RegionName\" } },\r\n { \"Column\": \"ResourceId\", \"Properties\": { \"Field\": \"ResourceId\" } },\r\n { \"Column\": \"ResourceName\", \"Properties\": { \"Field\": \"ResourceName\" } },\r\n { \"Column\": \"ResourceType\", \"Properties\": { \"Field\": \"ResourceType\" } },\r\n { \"Column\": \"ServiceCategory\", \"Properties\": { \"Field\": \"ServiceCategory\" } },\r\n { \"Column\": \"ServiceName\", \"Properties\": { \"Field\": \"ServiceName\" } },\r\n { \"Column\": \"ServiceSubcategory\", \"Properties\": { \"Field\": \"ServiceSubcategory\" } },\r\n { \"Column\": \"SkuId\", \"Properties\": { \"Field\": \"SkuId\" } },\r\n { \"Column\": \"SkuMeter\", \"Properties\": { \"Field\": \"SkuMeter\" } },\r\n { \"Column\": \"SkuPriceDetails\", \"Properties\": { \"Field\": \"SkuPriceDetails\" } },\r\n { \"Column\": \"SkuPriceId\", \"Properties\": { \"Field\": \"SkuPriceId\" } },\r\n { \"Column\": \"SubAccountId\", \"Properties\": { \"Field\": \"SubAccountId\" } },\r\n { \"Column\": \"SubAccountName\", \"Properties\": { \"Field\": \"SubAccountName\" } },\r\n { \"Column\": \"SubAccountType\", \"Properties\": { \"Field\": \"SubAccountType\" } },\r\n { \"Column\": \"Tags\", \"Properties\": { \"Field\": \"Tags\" } },\r\n { \"Column\": \"UsageAmount\", \"Properties\": { \"Field\": \"UsageAmount\" } },\r\n { \"Column\": \"UsageQuantity\", \"Properties\": { \"Field\": \"UsageQuantity\" } },\r\n { \"Column\": \"UsageUnit\", \"Properties\": { \"Field\": \"UsageUnit\" } },\r\n { \"Column\": \"x_AccountId\", \"Properties\": { \"Field\": \"x_AccountId\" } },\r\n { \"Column\": \"x_AccountName\", \"Properties\": { \"Field\": \"x_AccountName\" } },\r\n { \"Column\": \"x_AccountOwnerId\", \"Properties\": { \"Field\": \"x_AccountOwnerId\" } },\r\n { \"Column\": \"x_AmortizationClass\", \"Properties\": { \"Field\": \"x_AmortizationClass\" } },\r\n { \"Column\": \"x_BilledCostInUsd\", \"Properties\": { \"Field\": \"x_BilledCostInUsd\" } },\r\n { \"Column\": \"x_BilledUnitPrice\", \"Properties\": { \"Field\": \"x_BilledUnitPrice\" } },\r\n { \"Column\": \"x_BillingAccountId\", \"Properties\": { \"Field\": \"x_BillingAccountId\" } },\r\n { \"Column\": \"x_BillingAccountName\", \"Properties\": { \"Field\": \"x_BillingAccountName\" } },\r\n { \"Column\": \"x_BillingExchangeRate\", \"Properties\": { \"Field\": \"x_BillingExchangeRate\" } },\r\n { \"Column\": \"x_BillingExchangeRateDate\", \"Properties\": { \"Field\": \"x_BillingExchangeRateDate\" } },\r\n { \"Column\": \"x_BillingItemCode\", \"Properties\": { \"Field\": \"x_BillingItemCode\" } },\r\n { \"Column\": \"x_BillingItemName\", \"Properties\": { \"Field\": \"x_BillingItemName\" } },\r\n { \"Column\": \"x_BillingProfileId\", \"Properties\": { \"Field\": \"x_BillingProfileId\" } },\r\n { \"Column\": \"x_BillingProfileName\", \"Properties\": { \"Field\": \"x_BillingProfileName\" } },\r\n { \"Column\": \"x_ChargeId\", \"Properties\": { \"Field\": \"x_ChargeId\" } },\r\n { \"Column\": \"x_ContractedCostInUsd\", \"Properties\": { \"Field\": \"x_ContractedCostInUsd\" } },\r\n { \"Column\": \"x_CommodityCode\", \"Properties\": { \"Field\": \"x_CommodityCode\" } },\r\n { \"Column\": \"x_CommodityName\", \"Properties\": { \"Field\": \"x_CommodityName\" } },\r\n { \"Column\": \"x_ComponentName\", \"Properties\": { \"Field\": \"x_ComponentName\" } },\r\n { \"Column\": \"x_ComponentType\", \"Properties\": { \"Field\": \"x_ComponentType\" } },\r\n { \"Column\": \"x_Cost\", \"Properties\": { \"Field\": \"x_Cost\" } },\r\n { \"Column\": \"x_CostAllocationRuleName\", \"Properties\": { \"Field\": \"x_CostAllocationRuleName\" } },\r\n { \"Column\": \"x_CostCategories\", \"Properties\": { \"Field\": \"x_CostCategories\" } },\r\n { \"Column\": \"x_CostCenter\", \"Properties\": { \"Field\": \"x_CostCenter\" } },\r\n { \"Column\": \"x_Credits\", \"Properties\": { \"Field\": \"x_Credits\" } },\r\n { \"Column\": \"x_CostType\", \"Properties\": { \"Field\": \"x_CostType\" } },\r\n { \"Column\": \"x_CurrencyConversionRate\", \"Properties\": { \"Field\": \"x_CurrencyConversionRate\" } },\r\n { \"Column\": \"x_CustomerId\", \"Properties\": { \"Field\": \"x_CustomerId\" } },\r\n { \"Column\": \"x_CustomerName\", \"Properties\": { \"Field\": \"x_CustomerName\" } },\r\n { \"Column\": \"x_Discount\", \"Properties\": { \"Field\": \"x_Discount\" } },\r\n { \"Column\": \"x_EffectiveCostInUsd\", \"Properties\": { \"Field\": \"x_EffectiveCostInUsd\" } },\r\n { \"Column\": \"x_EffectiveUnitPrice\", \"Properties\": { \"Field\": \"x_EffectiveUnitPrice\" } },\r\n { \"Column\": \"x_ExportTime\", \"Properties\": { \"Field\": \"x_ExportTime\" } },\r\n { \"Column\": \"x_InstanceID\", \"Properties\": { \"Field\": \"x_InstanceID\" } },\r\n { \"Column\": \"x_InvoiceId\", \"Properties\": { \"Field\": \"x_InvoiceId\" } },\r\n { \"Column\": \"x_InvoiceIssuerId\", \"Properties\": { \"Field\": \"x_InvoiceIssuerId\" } },\r\n { \"Column\": \"x_InvoiceSectionId\", \"Properties\": { \"Field\": \"x_InvoiceSectionId\" } },\r\n { \"Column\": \"x_InvoiceSectionName\", \"Properties\": { \"Field\": \"x_InvoiceSectionName\" } },\r\n { \"Column\": \"x_ListCostInUsd\", \"Properties\": { \"Field\": \"x_ListCostInUsd\" } },\r\n { \"Column\": \"x_Location\", \"Properties\": { \"Field\": \"x_Location\" } },\r\n { \"Column\": \"x_OnDemandCost\", \"Properties\": { \"Field\": \"x_OnDemandCost\" } },\r\n { \"Column\": \"x_OnDemandCostInUsd\", \"Properties\": { \"Field\": \"x_OnDemandCostInUsd\" } },\r\n { \"Column\": \"x_OnDemandUnitPrice\", \"Properties\": { \"Field\": \"x_OnDemandUnitPrice\" } },\r\n { \"Column\": \"x_Operation\", \"Properties\": { \"Field\": \"x_Operation\" } },\r\n { \"Column\": \"x_OwnerAccountID\", \"Properties\": { \"Field\": \"x_OwnerAccountID\" } },\r\n { \"Column\": \"x_PartnerCreditApplied\", \"Properties\": { \"Field\": \"x_PartnerCreditApplied\" } },\r\n { \"Column\": \"x_PartnerCreditRate\", \"Properties\": { \"Field\": \"x_PartnerCreditRate\" } },\r\n { \"Column\": \"x_PricingBlockSize\", \"Properties\": { \"Field\": \"x_PricingBlockSize\" } },\r\n { \"Column\": \"x_PricingCurrency\", \"Properties\": { \"Field\": \"x_PricingCurrency\" } },\r\n { \"Column\": \"x_PricingSubcategory\", \"Properties\": { \"Field\": \"x_PricingSubcategory\" } },\r\n { \"Column\": \"x_PricingUnitDescription\", \"Properties\": { \"Field\": \"x_PricingUnitDescription\" } },\r\n { \"Column\": \"x_Project\", \"Properties\": { \"Field\": \"x_Project\" } },\r\n { \"Column\": \"x_PublisherCategory\", \"Properties\": { \"Field\": \"x_PublisherCategory\" } },\r\n { \"Column\": \"x_PublisherId\", \"Properties\": { \"Field\": \"x_PublisherId\" } },\r\n { \"Column\": \"x_ResellerId\", \"Properties\": { \"Field\": \"x_ResellerId\" } },\r\n { \"Column\": \"x_ResellerName\", \"Properties\": { \"Field\": \"x_ResellerName\" } },\r\n { \"Column\": \"x_ResourceGroupName\", \"Properties\": { \"Field\": \"x_ResourceGroupName\" } },\r\n { \"Column\": \"x_ResourceType\", \"Properties\": { \"Field\": \"x_ResourceType\" } },\r\n { \"Column\": \"x_ServiceCode\", \"Properties\": { \"Field\": \"x_ServiceCode\" } },\r\n { \"Column\": \"x_ServiceId\", \"Properties\": { \"Field\": \"x_ServiceId\" } },\r\n { \"Column\": \"x_ServiceModel\", \"Properties\": { \"Field\": \"x_ServiceModel\" } },\r\n { \"Column\": \"x_ServicePeriodEnd\", \"Properties\": { \"Field\": \"x_ServicePeriodEnd\" } },\r\n { \"Column\": \"x_ServicePeriodStart\", \"Properties\": { \"Field\": \"x_ServicePeriodStart\" } },\r\n { \"Column\": \"x_SkuDescription\", \"Properties\": { \"Field\": \"x_SkuDescription\" } },\r\n { \"Column\": \"x_SkuDetails\", \"Properties\": { \"Field\": \"x_SkuDetails\" } },\r\n { \"Column\": \"x_SkuIsCreditEligible\", \"Properties\": { \"Field\": \"x_SkuIsCreditEligible\" } },\r\n { \"Column\": \"x_SkuMeterCategory\", \"Properties\": { \"Field\": \"x_SkuMeterCategory\" } },\r\n { \"Column\": \"x_SkuMeterId\", \"Properties\": { \"Field\": \"x_SkuMeterId\" } },\r\n { \"Column\": \"x_SkuMeterName\", \"Properties\": { \"Field\": \"x_SkuMeterName\" } },\r\n { \"Column\": \"x_SkuMeterSubcategory\", \"Properties\": { \"Field\": \"x_SkuMeterSubcategory\" } },\r\n { \"Column\": \"x_SkuOfferId\", \"Properties\": { \"Field\": \"x_SkuOfferId\" } },\r\n { \"Column\": \"x_SkuOrderId\", \"Properties\": { \"Field\": \"x_SkuOrderId\" } },\r\n { \"Column\": \"x_SkuOrderName\", \"Properties\": { \"Field\": \"x_SkuOrderName\" } },\r\n { \"Column\": \"x_SkuPartNumber\", \"Properties\": { \"Field\": \"x_SkuPartNumber\" } },\r\n { \"Column\": \"x_SkuPlanName\", \"Properties\": { \"Field\": \"x_SkuPlanName\" } },\r\n { \"Column\": \"x_SkuRegion\", \"Properties\": { \"Field\": \"x_SkuRegion\" } },\r\n { \"Column\": \"x_SkuServiceFamily\", \"Properties\": { \"Field\": \"x_SkuServiceFamily\" } },\r\n { \"Column\": \"x_SkuTerm\", \"Properties\": { \"Field\": \"x_SkuTerm\" } },\r\n { \"Column\": \"x_SkuTier\", \"Properties\": { \"Field\": \"x_SkuTier\" } },\r\n { \"Column\": \"x_SourceName\", \"Properties\": { \"Field\": \"x_SourceName\" } },\r\n { \"Column\": \"x_SourceProvider\", \"Properties\": { \"Field\": \"x_SourceProvider\" } },\r\n { \"Column\": \"x_SourceType\", \"Properties\": { \"Field\": \"x_SourceType\" } },\r\n { \"Column\": \"x_SourceVersion\", \"Properties\": { \"Field\": \"x_SourceVersion\" } },\r\n { \"Column\": \"x_SubproductName\", \"Properties\": { \"Field\": \"x_SubproductName\" } },\r\n { \"Column\": \"x_UsageType\", \"Properties\": { \"Field\": \"x_UsageType\" } }\r\n]\r\n```\r\n\r\n// Costs_raw retention policy (clear historical data)\r\n.alter-merge table Costs_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// Costs_raw retention policy (set the user-defined retention period)\r\n.alter-merge table Costs_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable Costs_raw streaming ingestion (required for Fabric)\r\n.alter table Costs_raw policy streamingingestion disable\r\n\r\n\r\n//===| Prices |=========================================================================================================\r\n// NOTE: Must be before cost details.\r\n//\r\n// Supported versions:\r\n// - MS EA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-ea\r\n// - MS MCA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-mca\r\n//======================================================================================================================\r\n\r\n// Prices_raw table -- Create the table if it doesn't exist\r\n.create-merge table Prices_raw ( ignore: string )\r\n\r\n// Prices_raw table -- Remove all columns to allow changing column types\r\n.alter table Prices_raw ( ignore: string )\r\n\r\n// Prices_raw table -- Redefine all columns\r\n.alter table Prices_raw (\r\n BasePrice: real, // Azure EA + MCA\r\n BillingAccountId: string, // Azure MCA\r\n BillingAccountName: string, // Azure MCA\r\n BillingCurrency: string, // Azure MCA\r\n BillingProfileId: string, // Azure MCA\r\n BillingProfileName: string, // Azure MCA\r\n Currency: string, // Azure MCA\r\n CurrencyCode: string, // Azure EA\r\n EffectiveEndDate: datetime, // Azure MCA\r\n EffectiveStartDate: datetime, // Azure EA + MCA\r\n EnrollmentNumber: string, // Azure EA\r\n IncludedQuantity: real, // Azure EA\r\n MarketPrice: real, // Azure EA + MCA\r\n MeterCategory: string, // Azure EA + MCA\r\n MeterId: string, // Azure MCA\r\n MeterID: string, // Azure EA\r\n MeterName: string, // Azure EA + MCA\r\n MeterRegion: string, // Azure EA + MCA\r\n MeterSubCategory: string, // Azure EA + MCA\r\n MeterType: string, // Azure EA + MCA\r\n OfferID: string, // Azure EA\r\n PartNumber: string, // Azure EA\r\n PriceType: string, // Azure EA + MCA\r\n Product: string, // Azure EA + MCA\r\n ProductId: string, // Azure MCA\r\n ProductID: string, // Azure EA\r\n ServiceFamily: string, // Azure EA + MCA\r\n SkuId: string, // Azure MCA\r\n SkuID: string, // Azure EA\r\n Term: string, // Azure EA + MCA\r\n TierMinimumUnits: real, // Azure MCA\r\n UnitOfMeasure: string, // Azure EA + MCA\r\n UnitPrice: real, // Azure EA + MCA\r\n x_SourceName: string, // Hubs v1_0+\r\n x_SourceProvider: string, // Hubs v1_0+\r\n x_SourceType: string, // Hubs v1_0+\r\n x_SourceVersion: string // Hubs v1_0+\r\n)\r\n\r\n// Prices_raw ingestion mapping\r\n.create-or-alter table Prices_raw ingestion parquet mapping \"Prices_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"BasePrice\", \"Properties\": { \"Field\": \"BasePrice\" } },\r\n { \"Column\": \"BillingAccountId\", \"Properties\": { \"Field\": \"BillingAccountId\" } },\r\n { \"Column\": \"BillingAccountName\", \"Properties\": { \"Field\": \"BillingAccountName\" } },\r\n { \"Column\": \"BillingCurrency\", \"Properties\": { \"Field\": \"BillingCurrency\" } },\r\n { \"Column\": \"BillingProfileId\", \"Properties\": { \"Field\": \"BillingProfileId\" } },\r\n { \"Column\": \"BillingProfileName\", \"Properties\": { \"Field\": \"BillingProfileName\" } },\r\n { \"Column\": \"Currency\", \"Properties\": { \"Field\": \"Currency\" } },\r\n { \"Column\": \"CurrencyCode\", \"Properties\": { \"Field\": \"CurrencyCode\" } },\r\n { \"Column\": \"EffectiveEndDate\", \"Properties\": { \"Field\": \"EffectiveEndDate\" } },\r\n { \"Column\": \"EffectiveStartDate\", \"Properties\": { \"Field\": \"EffectiveStartDate\" } },\r\n { \"Column\": \"EnrollmentNumber\", \"Properties\": { \"Field\": \"EnrollmentNumber\" } },\r\n { \"Column\": \"IncludedQuantity\", \"Properties\": { \"Field\": \"IncludedQuantity\" } },\r\n { \"Column\": \"MarketPrice\", \"Properties\": { \"Field\": \"MarketPrice\" } },\r\n { \"Column\": \"MeterCategory\", \"Properties\": { \"Field\": \"MeterCategory\" } },\r\n { \"Column\": \"MeterId\", \"Properties\": { \"Field\": \"MeterId\" } },\r\n { \"Column\": \"MeterID\", \"Properties\": { \"Field\": \"MeterID\" } },\r\n { \"Column\": \"MeterName\", \"Properties\": { \"Field\": \"MeterName\" } },\r\n { \"Column\": \"MeterRegion\", \"Properties\": { \"Field\": \"MeterRegion\" } },\r\n { \"Column\": \"MeterSubCategory\", \"Properties\": { \"Field\": \"MeterSubCategory\" } },\r\n { \"Column\": \"MeterType\", \"Properties\": { \"Field\": \"MeterType\" } },\r\n { \"Column\": \"OfferID\", \"Properties\": { \"Field\": \"OfferID\" } },\r\n { \"Column\": \"PartNumber\", \"Properties\": { \"Field\": \"PartNumber\" } },\r\n { \"Column\": \"PriceType\", \"Properties\": { \"Field\": \"PriceType\" } },\r\n { \"Column\": \"Product\", \"Properties\": { \"Field\": \"Product\" } },\r\n { \"Column\": \"ProductId\", \"Properties\": { \"Field\": \"ProductId\" } },\r\n { \"Column\": \"ProductID\", \"Properties\": { \"Field\": \"ProductID\" } },\r\n { \"Column\": \"ServiceFamily\", \"Properties\": { \"Field\": \"ServiceFamily\" } },\r\n { \"Column\": \"SkuId\", \"Properties\": { \"Field\": \"SkuId\" } },\r\n { \"Column\": \"SkuID\", \"Properties\": { \"Field\": \"SkuID\" } },\r\n { \"Column\": \"Term\", \"Properties\": { \"Field\": \"Term\" } },\r\n { \"Column\": \"TierMinimumUnits\", \"Properties\": { \"Field\": \"TierMinimumUnits\" } },\r\n { \"Column\": \"UnitOfMeasure\", \"Properties\": { \"Field\": \"UnitOfMeasure\" } },\r\n { \"Column\": \"UnitPrice\", \"Properties\": { \"Field\": \"UnitPrice\" } },\r\n { \"Column\": \"x_SourceName\", \"Properties\": { \"Field\": \"x_SourceName\" } },\r\n { \"Column\": \"x_SourceProvider\", \"Properties\": { \"Field\": \"x_SourceProvider\" } },\r\n { \"Column\": \"x_SourceType\", \"Properties\": { \"Field\": \"x_SourceType\" } },\r\n { \"Column\": \"x_SourceVersion\", \"Properties\": { \"Field\": \"x_SourceVersion\" } }\r\n]\r\n```\r\n\r\n// Prices_raw retention policy (clear historical data)\r\n.alter-merge table Prices_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// Prices_raw retention policy (set the user-defined retention period)\r\n.alter-merge table Prices_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable Prices_raw streaming ingestion (required for Fabric)\r\n.alter table Prices_raw policy streamingingestion disable\r\n\r\n\r\n//===| Recommendations |================================================================================================\r\n// Supported datasets/versions:\r\n// - MS CM EA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-ea\r\n// - MS CM MCA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-mca\r\n//======================================================================================================================\r\n\r\n// Recommendations_raw table -- Create the table if it doesn't exist\r\n.create-merge table Recommendations_raw ( ignore: string )\r\n\r\n// Recommendations_raw table -- Remove all columns to allow changing column types\r\n.alter table Recommendations_raw ( ignore: string )\r\n\r\n// Recommendations_raw table -- Redefine all columns\r\n.alter table Recommendations_raw (\r\n CostWithNoReservedInstances: real, // MS CM EA resv reco 2024-05-01\r\n CostWithNoReservedInstancesJson: string, // MS CM MCA resv reco 2024-05-01 -- Renamed to remove spaces and flag as JSON\r\n FirstUsageDate: datetime, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n InstanceFlexibilityGroup: string, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n InstanceFlexibilityRatio: real, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n Location: string, // MS CM EA+MCA resv reco 2024-05-01\r\n LookBackPeriod: string, // MS CM EA+MCA resv reco 2024-05-01\r\n MeterId: string, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n NetSavings: real, // MS CM EA resv reco 2024-05-01\r\n NetSavingsJson: string, // MS CM MCA resv reco 2024-05-01 -- Renamed to remove spaces and flag as JSON\r\n NormalizedSize: string, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n ProviderName: string, // Hubs v1_2\r\n RecommendedQuantity: real, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n RecommendedQuantityNormalized: real, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n ResourceId: string, // Hubs v1_2\r\n ResourceName: string, // Hubs v1_2\r\n ResourceType: string, // Hubs v1_2, MS CM EA+MCA resv reco 2024-05-01\r\n Scope: string, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n SKU: string, // MS CM EA resv reco 2024-05-01\r\n SkuName: string, // MS CM MCA resv reco 2024-05-01 -- Renamed to remove spaces\r\n SkuProperties: string, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n SubAccountId: string, // Hubs v1_2\r\n SubAccountName: string, // Hubs v1_2\r\n SubscriptionId: string, // MS CM EA+MCA resv reco 2024-05-01\r\n Term: string, // MS CM EA+MCA resv reco 2024-05-01\r\n TotalCostWithReservedInstances: real, // MS CM EA resv reco 2024-05-01\r\n TotalCostWithReservedInstancesJson: string, // MS CM MCA resv reco 2024-05-01 -- Renamed to remove spaces and flag as JSON\r\n x_EffectiveCostAfter: real, // Hubs v1_2\r\n x_EffectiveCostBefore: real, // Hubs v1_2\r\n x_EffectiveCostSavings: real, // Hubs v1_2\r\n x_RecommendationCategory: string, // Hubs v1_2\r\n x_RecommendationDate: datetime, // Hubs v1_2\r\n x_RecommendationDescription: string, // Hubs v1_2\r\n x_RecommendationDetails: dynamic, // Hubs v1_2\r\n x_RecommendationId: string, // Hubs v1_2\r\n x_ResourceGroupName: string, // Hubs v1_2\r\n x_SourceName: string, // Hubs v1_0+\r\n x_SourceProvider: string, // Hubs v1_0+\r\n x_SourceType: string, // Hubs v1_0+\r\n x_SourceVersion: string // Hubs v1_0+\r\n)\r\n\r\n// Recommendations_raw ingestion mapping\r\n.create-or-alter table Recommendations_raw ingestion parquet mapping \"Recommendations_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"CostWithNoReservedInstances\", \"Properties\": { \"Field\": \"CostWithNoReservedInstances\" } },\r\n { \"Column\": \"CostWithNoReservedInstancesJson\", \"Properties\": { \"Field\": \"CostWithNoReservedInstancesJson\" } },\r\n { \"Column\": \"FirstUsageDate\", \"Properties\": { \"Field\": \"FirstUsageDate\" } },\r\n { \"Column\": \"InstanceFlexibilityGroup\", \"Properties\": { \"Field\": \"InstanceFlexibilityGroup\" } },\r\n { \"Column\": \"InstanceFlexibilityRatio\", \"Properties\": { \"Field\": \"InstanceFlexibilityRatio\" } },\r\n { \"Column\": \"Location\", \"Properties\": { \"Field\": \"Location\" } },\r\n { \"Column\": \"LookBackPeriod\", \"Properties\": { \"Field\": \"LookBackPeriod\" } },\r\n { \"Column\": \"MeterId\", \"Properties\": { \"Field\": \"MeterId\" } },\r\n { \"Column\": \"NetSavings\", \"Properties\": { \"Field\": \"NetSavings\" } },\r\n { \"Column\": \"NetSavingsJson\", \"Properties\": { \"Field\": \"NetSavingsJson\" } },\r\n { \"Column\": \"NormalizedSize\", \"Properties\": { \"Field\": \"NormalizedSize\" } },\r\n { \"Column\": \"ProviderName\", \"Properties\": { \"Field\": \"ProviderName\" } },\r\n { \"Column\": \"RecommendedQuantity\", \"Properties\": { \"Field\": \"RecommendedQuantity\" } },\r\n { \"Column\": \"RecommendedQuantityNormalized\", \"Properties\": { \"Field\": \"RecommendedQuantityNormalized\" } },\r\n { \"Column\": \"ResourceId\", \"Properties\": { \"Field\": \"ResourceId\" } },\r\n { \"Column\": \"ResourceName\", \"Properties\": { \"Field\": \"ResourceName\" } },\r\n { \"Column\": \"ResourceType\", \"Properties\": { \"Field\": \"ResourceType\" } },\r\n { \"Column\": \"Scope\", \"Properties\": { \"Field\": \"Scope\" } },\r\n { \"Column\": \"SKU\", \"Properties\": { \"Field\": \"SKU\" } },\r\n { \"Column\": \"SkuName\", \"Properties\": { \"Field\": \"SkuName\" } },\r\n { \"Column\": \"SkuProperties\", \"Properties\": { \"Field\": \"SkuProperties\" } },\r\n { \"Column\": \"SubAccountId\", \"Properties\": { \"Field\": \"SubAccountId\" } },\r\n { \"Column\": \"SubAccountName\", \"Properties\": { \"Field\": \"SubAccountName\" } },\r\n { \"Column\": \"SubscriptionId\", \"Properties\": { \"Field\": \"SubscriptionId\" } },\r\n { \"Column\": \"Term\", \"Properties\": { \"Field\": \"Term\" } },\r\n { \"Column\": \"TotalCostWithReservedInstances\", \"Properties\": { \"Field\": \"TotalCostWithReservedInstances\" } },\r\n { \"Column\": \"TotalCostWithReservedInstancesJson\", \"Properties\": { \"Field\": \"TotalCostWithReservedInstancesJson\" } },\r\n { \"Column\": \"x_EffectiveCostAfter\", \"Properties\": { \"Field\": \"x_EffectiveCostAfter\" } },\r\n { \"Column\": \"x_EffectiveCostBefore\", \"Properties\": { \"Field\": \"x_EffectiveCostBefore\" } },\r\n { \"Column\": \"x_EffectiveCostSavings\", \"Properties\": { \"Field\": \"x_EffectiveCostSavings\" } },\r\n { \"Column\": \"x_RecommendationCategory\", \"Properties\": { \"Field\": \"x_RecommendationCategory\" } },\r\n { \"Column\": \"x_RecommendationDate\", \"Properties\": { \"Field\": \"x_RecommendationDate\" } },\r\n { \"Column\": \"x_RecommendationDescription\", \"Properties\": { \"Field\": \"x_RecommendationDescription\" } },\r\n { \"Column\": \"x_RecommendationDetails\", \"Properties\": { \"Field\": \"x_RecommendationDetails\" } },\r\n { \"Column\": \"x_RecommendationId\", \"Properties\": { \"Field\": \"x_RecommendationId\" } },\r\n { \"Column\": \"x_ResourceGroupName\", \"Properties\": { \"Field\": \"x_ResourceGroupName\" } },\r\n { \"Column\": \"x_SourceName\", \"Properties\": { \"Field\": \"x_SourceName\" } },\r\n { \"Column\": \"x_SourceProvider\", \"Properties\": { \"Field\": \"x_SourceProvider\" } },\r\n { \"Column\": \"x_SourceType\", \"Properties\": { \"Field\": \"x_SourceType\" } },\r\n { \"Column\": \"x_SourceVersion\", \"Properties\": { \"Field\": \"x_SourceVersion\" } }\r\n]\r\n```\r\n\r\n// Recommendations_raw retention policy (clear historical data)\r\n.alter-merge table Recommendations_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// Recommendations_raw retention policy (set the user-defined retention period)\r\n.alter-merge table Recommendations_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable Recommendations_raw streaming ingestion (required for Fabric)\r\n.alter table Recommendations_raw policy streamingingestion disable\r\n\r\n\r\n//===| Transactions |===================================================================================================\r\n// Supported versions:\r\n// - MS CM EA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-ea\r\n// - MS CM MCA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-mca\r\n//======================================================================================================================\r\n\r\n// Transactions_raw table -- Create the table if it doesn't exist\r\n.create-merge table Transactions_raw ( ignore: string )\r\n\r\n// Transactions_raw table -- Remove all columns to allow changing column types\r\n.alter table Transactions_raw ( ignore: string )\r\n\r\n// Transactions_raw table -- Redefine all columns\r\n.alter table Transactions_raw (\r\n AccountName: string, // MS CM EA resv trans 2023-05-01\r\n AccountOwnerEmail: string, // MS CM EA resv trans 2023-05-01\r\n Amount: real, // MS CM EA+MCA resv trans 2023-05-01\r\n ArmSkuName: string, // MS CM EA+MCA resv trans 2023-05-01\r\n BillingFrequency: string, // MS CM EA+MCA resv trans 2023-05-01\r\n BillingMonth: string, // MS CM EA resv trans 2023-05-01\r\n BillingProfileId: string, // MS CM MCA resv trans 2023-05-01\r\n BillingProfileName: string, // MS CM MCA resv trans 2023-05-01\r\n CostCenter: string, // MS CM EA resv trans 2023-05-01\r\n Currency: string, // MS CM EA+MCA resv trans 2023-05-01\r\n CurrentEnrollmentId: string, // MS CM EA resv trans 2023-05-01\r\n DepartmentName: string, // MS CM EA resv trans 2023-05-01\r\n Description: string, // MS CM EA+MCA resv trans 2023-05-01\r\n EventDate: datetime, // MS CM EA+MCA resv trans 2023-05-01\r\n EventType: string, // MS CM EA+MCA resv trans 2023-05-01\r\n Invoice: string, // MS CM EA+MCA resv trans 2023-05-01\r\n InvoiceId: string, // MS CM EA+MCA resv trans 2023-05-01\r\n InvoiceSectionId: string, // MS CM MCA resv trans 2023-05-01\r\n InvoiceSectionName: string, // MS CM MCA resv trans 2023-05-01\r\n MonetaryCommitment: real, // MS CM EA resv trans 2023-05-01\r\n Overage: real, // MS CM EA resv trans 2023-05-01\r\n PurchasingEnrollment: string, // MS CM EA resv trans 2023-05-01\r\n PurchasingSubscriptionGuid: string, // MS CM EA+MCA resv trans 2023-05-01\r\n PurchasingSubscriptionName: string, // MS CM EA+MCA resv trans 2023-05-01\r\n Quantity: real, // MS CM EA+MCA resv trans 2023-05-01\r\n Region: string, // MS CM EA+MCA resv trans 2023-05-01\r\n ReservationOrderId: string, // MS CM EA+MCA resv trans 2023-05-01\r\n ReservationOrderName: string, // MS CM EA+MCA resv trans 2023-05-01\r\n Term: string, // MS CM EA+MCA resv trans 2023-05-01\r\n x_SourceName: string, // Hubs v1_0+\r\n x_SourceProvider: string, // Hubs v1_0+\r\n x_SourceType: string, // Hubs v1_0+\r\n x_SourceVersion: string // Hubs v1_0+\r\n)\r\n\r\n// Transactions_raw ingestion mapping\r\n.create-or-alter table Transactions_raw ingestion parquet mapping \"Transactions_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"AccountName\", \"Properties\": { \"Field\": \"AccountName\" } },\r\n { \"Column\": \"AccountOwnerEmail\", \"Properties\": { \"Field\": \"AccountOwnerEmail\" } },\r\n { \"Column\": \"Amount\", \"Properties\": { \"Field\": \"Amount\" } },\r\n { \"Column\": \"ArmSkuName\", \"Properties\": { \"Field\": \"ArmSkuName\" } },\r\n { \"Column\": \"BillingFrequency\", \"Properties\": { \"Field\": \"BillingFrequency\" } },\r\n { \"Column\": \"BillingMonth\", \"Properties\": { \"Field\": \"BillingMonth\" } },\r\n { \"Column\": \"BillingProfileId\", \"Properties\": { \"Field\": \"BillingProfileId\" } },\r\n { \"Column\": \"BillingProfileName\", \"Properties\": { \"Field\": \"BillingProfileName\" } },\r\n { \"Column\": \"CostCenter\", \"Properties\": { \"Field\": \"CostCenter\" } },\r\n { \"Column\": \"Currency\", \"Properties\": { \"Field\": \"Currency\" } },\r\n { \"Column\": \"CurrentEnrollmentId\", \"Properties\": { \"Field\": \"CurrentEnrollmentId\" } },\r\n { \"Column\": \"DepartmentName\", \"Properties\": { \"Field\": \"DepartmentName\" } },\r\n { \"Column\": \"Description\", \"Properties\": { \"Field\": \"Description\" } },\r\n { \"Column\": \"EventDate\", \"Properties\": { \"Field\": \"EventDate\" } },\r\n { \"Column\": \"EventType\", \"Properties\": { \"Field\": \"EventType\" } },\r\n { \"Column\": \"Invoice\", \"Properties\": { \"Field\": \"Invoice\" } },\r\n { \"Column\": \"InvoiceId\", \"Properties\": { \"Field\": \"InvoiceId\" } },\r\n { \"Column\": \"InvoiceSectionId\", \"Properties\": { \"Field\": \"InvoiceSectionId\" } },\r\n { \"Column\": \"InvoiceSectionName\", \"Properties\": { \"Field\": \"InvoiceSectionName\" } },\r\n { \"Column\": \"MonetaryCommitment\", \"Properties\": { \"Field\": \"MonetaryCommitment\" } },\r\n { \"Column\": \"Overage\", \"Properties\": { \"Field\": \"Overage\" } },\r\n { \"Column\": \"PurchasingEnrollment\", \"Properties\": { \"Field\": \"PurchasingEnrollment\" } },\r\n { \"Column\": \"PurchasingSubscriptionGuid\", \"Properties\": { \"Field\": \"PurchasingSubscriptionGuid\" } },\r\n { \"Column\": \"PurchasingSubscriptionName\", \"Properties\": { \"Field\": \"PurchasingSubscriptionName\" } },\r\n { \"Column\": \"Quantity\", \"Properties\": { \"Field\": \"Quantity\" } },\r\n { \"Column\": \"Region\", \"Properties\": { \"Field\": \"Region\" } },\r\n { \"Column\": \"ReservationOrderId\", \"Properties\": { \"Field\": \"ReservationOrderId\" } },\r\n { \"Column\": \"ReservationOrderName\", \"Properties\": { \"Field\": \"ReservationOrderName\" } },\r\n { \"Column\": \"Term\", \"Properties\": { \"Field\": \"Term\" } },\r\n { \"Column\": \"x_SourceName\", \"Properties\": { \"Field\": \"x_SourceName\" } },\r\n { \"Column\": \"x_SourceProvider\", \"Properties\": { \"Field\": \"x_SourceProvider\" } },\r\n { \"Column\": \"x_SourceType\", \"Properties\": { \"Field\": \"x_SourceType\" } },\r\n { \"Column\": \"x_SourceVersion\", \"Properties\": { \"Field\": \"x_SourceVersion\" } }\r\n]\r\n```\r\n\r\n// Transactions_raw retention policy (clear historical data)\r\n.alter-merge table Transactions_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// Transactions_raw retention policy (set the user-defined retention period)\r\n.alter-merge table Transactions_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable Transactions_raw streaming ingestion (required for Fabric)\r\n.alter table Transactions_raw policy streamingingestion disable\r\n\r\n", + "$fxv#9": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Ingestion database\r\n// Used for data ingestion, normalization, and cleansing.\r\n// See known issues @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n//===| Prices |=========================================================================================================\r\n// Supported versions:\r\n// - MS EA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-ea\r\n// - MS MCA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-mca\r\n//======================================================================================================================\r\n\r\n// Prices_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: All prices transformed to FOCUS 1.0. Use Prices_transform_v1_2() instead.', folder='Prices')\r\nPrices_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', toint(''),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n let prices = materialize(\r\n Prices_raw\r\n //\r\n // Change real to decimal\r\n | extend\r\n BasePrice = todecimal(BasePrice),\r\n IncludedQuantity = todecimal(IncludedQuantity),\r\n MarketPrice = todecimal(MarketPrice),\r\n TierMinimumUnits = todecimal(TierMinimumUnits),\r\n UnitPrice = todecimal(UnitPrice)\r\n //\r\n | extend x_SkuId = coalesce(SkuId, SkuID)\r\n | extend x_SkuMeterId = coalesce(MeterId, MeterID)\r\n | extend x_SkuProductId = coalesce(ProductId, ProductID)\r\n | extend x_SkuTerm = isoMonths(Term)\r\n | project-rename\r\n x_BaseUnitPrice = BasePrice,\r\n x_EffectivePeriodEnd = EffectiveEndDate,\r\n x_EffectivePeriodStart = EffectiveStartDate,\r\n x_PricingUnitDescription = UnitOfMeasure,\r\n x_SkuIncludedQuantity = IncludedQuantity,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterName = MeterName,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuMeterType = MeterType,\r\n x_SkuOfferId = OfferID,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPriceType = PriceType,\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTier = TierMinimumUnits\r\n | extend ContractedUnitPrice = iff(x_SkuPriceType != 'SavingsPlan', UnitPrice, todecimal('')) // UnitPrice for savings plan is not the on-demand unit price\r\n | extend ListUnitPrice = iff(x_SkuPriceType != 'SavingsPlan', MarketPrice, todecimal('')) // MarketPrice for savings plan is not the list price\r\n | extend ChargeCategory = case(\r\n x_SkuPriceType == 'Consumption', 'Usage',\r\n x_SkuPriceType == 'ReservedInstance', 'Purchase',\r\n x_SkuPriceType == 'SavingsPlan', 'Usage', // Savings plan prices are for committed usage, not the purchase\r\n ''\r\n )\r\n | extend SkuPriceIdv2 = strcat(case(x_SkuPriceType == 'Consumption', 'OD', x_SkuPriceType == 'ReservedInstance', 'RI', x_SkuPriceType == 'SavingsPlan', 'SP', 'XX'), substring(ChargeCategory, 0, 1), x_SkuTerm, '_', x_SkuProductId, '_', x_SkuId, '_', x_SkuMeterType, '_', x_SkuTier, x_SkuOfferId)\r\n | extend x_BillingAccountId = iff(BillingAccountId startswith '/', split(BillingAccountId, '/')[-1], coalesce(BillingAccountId, EnrollmentNumber))\r\n | extend x_BillingProfileId = iff(BillingProfileId startswith '/', split(BillingProfileId, '/')[-1], coalesce(BillingProfileId, EnrollmentNumber))\r\n | extend tmp_SavingsPlanKey = strcat(x_SkuMeterId, x_SkuProductId, x_SkuId, x_SkuTier, x_SkuOfferId)\r\n //\r\n // Get latest ingested row based on the unique ID\r\n | extend x_IngestionTime = ingestion_time()\r\n );\r\n //\r\n // Meters for reservations and savings plans to identify commitment eligibility\r\n let riMeters = prices | where x_SkuPriceType == 'ReservedInstance' | distinct x_SkuMeterId;\r\n let spMeters = prices | where x_SkuPriceType == 'SavingsPlan' | distinct x_SkuMeterId;\r\n //\r\n // Copy list/base/contracted prices from on-demand SKUs\r\n prices\r\n | where x_SkuPriceType == 'SavingsPlan'\r\n // If we use join, specify the shuffle key\r\n // TODO: Compare join vs. lookup perf -- | join kind=leftouter hint.strategy=shuffle (prices | where x_SkuPriceType == 'Consumption' | where x_SkuMeterId in (spMeters) | distinct tmp_SavingsPlanKey, ListUnitPrice, ContractedUnitPrice, x_BaseUnitPrice) on tmp_SavingsPlanKey\r\n | lookup kind=leftouter (prices | where x_SkuPriceType == 'Consumption' | where x_SkuMeterId in (spMeters) | distinct tmp_SavingsPlanKey, ListUnitPrice, ContractedUnitPrice, x_BaseUnitPrice) on tmp_SavingsPlanKey\r\n | extend ListUnitPrice = coalesce(ListUnitPrice, ListUnitPrice1)\r\n | extend ContractedUnitPrice = coalesce(ContractedUnitPrice, ContractedUnitPrice1)\r\n | extend x_BaseUnitPrice = coalesce(x_BaseUnitPrice, x_BaseUnitPrice1)\r\n | project-away ListUnitPrice1, ContractedUnitPrice1, x_BaseUnitPrice1, tmp_SavingsPlanKey\r\n | union ((prices | where x_SkuPriceType != 'SavingsPlan'))\r\n //\r\n // Calculate commitment discount elgibility\r\n // TODO: Would a join be faster?\r\n | extend x_CommitmentDiscountSpendEligibility = iff(x_SkuMeterId in (riMeters) and x_SkuPriceType != 'ReservedInstance', 'Eligible', 'Not Eligible')\r\n | extend x_CommitmentDiscountUsageEligibility = iff(x_SkuMeterId in (spMeters), 'Eligible', 'Not Eligible')\r\n //\r\n // Add PricingUnit and x_PricingBlockSize\r\n // TODO: Compare join vs. lookup perf -- | join kind=leftouter (PricingUnits) on x_PricingUnitDescription | project-away x_PricingUnitDescription1\r\n | lookup kind=leftouter (PricingUnits | extend x_PricingBlockSize = todecimal(x_PricingBlockSize)) on x_PricingUnitDescription\r\n //\r\n | extend x_EffectiveUnitPrice = iff(x_SkuPriceType == 'SavingsPlan', UnitPrice, todecimal('')) // Savings plan prices are for the effective price, not the contracted price\r\n | extend x_EffectiveUnitPriceDiscount = ContractedUnitPrice - x_EffectiveUnitPrice\r\n | extend x_ContractedUnitPriceDiscount = ListUnitPrice - ContractedUnitPrice\r\n | extend x_TotalUnitPriceDiscount = ListUnitPrice - x_EffectiveUnitPrice\r\n | project\r\n BillingAccountId = tolower(case(\r\n BillingProfileId startswith '/', BillingProfileId,\r\n BillingAccountId startswith '/', BillingAccountId,\r\n strcat('/providers/microsoft.billing/billingaccounts/', x_BillingAccountId, iff(x_BillingProfileId == x_BillingAccountId, '', strcat('/billingprofiles/', x_BillingProfileId)))\r\n )),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName, x_BillingProfileId),\r\n BillingCurrency = coalesce(BillingCurrency, CurrencyCode, Currency), // Currency last as a fallback only\r\n ChargeCategory,\r\n CommitmentDiscountCategory = case(\r\n x_SkuPriceType == 'ReservedInstance', 'Usage',\r\n x_SkuPriceType == 'SavingsPlan', 'Spend',\r\n ''\r\n ),\r\n CommitmentDiscountType = case(\r\n x_SkuPriceType == 'ReservedInstance', 'Reservation',\r\n x_SkuPriceType == 'SavingsPlan', 'Savings plan',\r\n ''\r\n ),\r\n ContractedUnitPrice,\r\n ListUnitPrice,\r\n PricingCategory = case(\r\n x_SkuPriceType == 'Consumption', 'Standard',\r\n x_SkuPriceType == 'ReservedInstance', 'Standard', // Reservation purchases are tracked as \"Standard\"\r\n x_SkuPriceType == 'SavingsPlan', 'Committed',\r\n ''\r\n ),\r\n PricingUnit,\r\n SkuId = coalesce(ProductId, ProductID),\r\n SkuPriceId = strcat(x_SkuProductId, '_', x_SkuId, '_', x_SkuMeterType),\r\n SkuPriceIdv2,\r\n x_BaseUnitPrice,\r\n x_BillingAccountAgreement = case(\r\n strlen(x_BillingAccountId) > 32, 'MCA',\r\n strlen(x_BillingAccountId) < 32, 'EA',\r\n 'Unknown'\r\n ),\r\n x_BillingAccountId,\r\n x_BillingProfileId,\r\n x_CommitmentDiscountSpendEligibility,\r\n x_CommitmentDiscountUsageEligibility,\r\n x_ContractedUnitPriceDiscount,\r\n x_ContractedUnitPriceDiscountPercent = 1.0 * x_ContractedUnitPriceDiscount / ListUnitPrice * 100,\r\n x_EffectivePeriodEnd = startofmonth(x_EffectivePeriodEnd + 1h),\r\n x_EffectivePeriodStart,\r\n x_EffectiveUnitPrice,\r\n x_EffectiveUnitPriceDiscount,\r\n x_EffectiveUnitPriceDiscountPercent = 1.0 * x_EffectiveUnitPriceDiscount / ContractedUnitPrice * 100,\r\n x_IngestionTime,\r\n x_PricingBlockSize,\r\n x_PricingCurrency = coalesce(Currency, CurrencyCode), // CurrencyCode last as a fallback only\r\n x_PricingSubcategory = case(\r\n x_SkuPriceType == 'Consumption' and (x_SkuIncludedQuantity > 0 or x_SkuTier > 0), 'Tiered',\r\n x_SkuPriceType == 'Consumption', 'Standard',\r\n x_SkuPriceType == 'ReservedInstance', 'Standard', // Reservation purchases are tracked as \"Standard\"\r\n x_SkuPriceType == 'SavingsPlan', 'Committed Spend',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_SkuDescription = Product,\r\n x_SkuId,\r\n x_SkuIncludedQuantity,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterName,\r\n x_SkuMeterSubcategory,\r\n x_SkuMeterType,\r\n x_SkuPriceType,\r\n x_SkuProductId,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuOfferId,\r\n x_SkuPartNumber,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceName = coalesce(x_SourceName, 'Cost Management'),\r\n x_SourceProvider = coalesce(x_SourceProvider, 'Microsoft'),\r\n x_SourceType = coalesce(x_SourceType, 'PriceSheet'),\r\n x_SourceVersion = coalesce(x_SourceVersion, '2023-05-01'),\r\n x_TotalUnitPriceDiscount,\r\n x_TotalUnitPriceDiscountPercent = 1.0 * x_TotalUnitPriceDiscount / ListUnitPrice * 100\r\n}\r\n\r\n// Prices_final_v1_0 table\r\n.create-merge table Prices_final_v1_0 (\r\n BillingAccountId: string,\r\n BillingAccountName: string,\r\n BillingCurrency: string,\r\n ChargeCategory: string,\r\n CommitmentDiscountCategory: string,\r\n CommitmentDiscountType: string,\r\n ContractedUnitPrice: decimal,\r\n ListUnitPrice: decimal,\r\n PricingCategory: string,\r\n PricingUnit: string,\r\n SkuId: string,\r\n SkuPriceId: string,\r\n SkuPriceIdv2: string, // Hubs add-on\r\n x_BaseUnitPrice: decimal, // Azure\r\n x_BillingAccountAgreement: string, // Hubs add-on\r\n x_BillingAccountId: string, // Azure MCA\r\n x_BillingProfileId: string, // Azure MCA\r\n x_CommitmentDiscountSpendEligibility: string, // Hubs add-on\r\n x_CommitmentDiscountUsageEligibility: string, // Hubs add-on\r\n x_ContractedUnitPriceDiscount: decimal, // Hubs add-on\r\n x_ContractedUnitPriceDiscountPercent: decimal, // Hubs add-on\r\n x_EffectivePeriodEnd: datetime, // Azure\r\n x_EffectivePeriodStart: datetime, // Azure\r\n x_EffectiveUnitPrice: decimal, // Azure\r\n x_EffectiveUnitPriceDiscount: decimal, // Hubs add-on\r\n x_EffectiveUnitPriceDiscountPercent: decimal, // Hubs add-on\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_PricingBlockSize: decimal, // Hubs add-on\r\n x_PricingCurrency: string, // Azure\r\n x_PricingSubcategory: string, // Hubs add-on\r\n x_PricingUnitDescription: string, // Azure\r\n x_SkuDescription: string, // Azure\r\n x_SkuId: string, // Azure\r\n x_SkuIncludedQuantity: decimal, // Azure EA\r\n x_SkuMeterCategory: string, // Azure\r\n x_SkuMeterId: string, // Azure\r\n x_SkuMeterName: string, // Azure\r\n x_SkuMeterSubcategory: string, // Azure\r\n x_SkuMeterType: string, // Azure\r\n x_SkuPriceType: string, // Azure\r\n x_SkuProductId: string, // Azure\r\n x_SkuRegion: string, // Azure\r\n x_SkuServiceFamily: string, // Azure\r\n x_SkuOfferId: string, // Azure EA\r\n x_SkuPartNumber: string, // Azure EA\r\n x_SkuTerm: int, // Azure\r\n x_SkuTier: decimal, // Azure MCA\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_TotalUnitPriceDiscount: decimal, // Hubs add-on\r\n x_TotalUnitPriceDiscountPercent: decimal // Hubs add-on\r\n)\r\n\r\n// Update policy for Prices_raw -> Prices_final_v1_0\r\n.alter table Prices_final_v1_0 policy update\r\n```\r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"Prices_raw\",\r\n \"Query\": \"Prices_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Cost and usage |=================================================================================================\r\n// Supported versions:\r\n// - MS: 1.0, 1.0-preview(v1) -- See https://aka.ms/costmgmt/exports/focus\r\n// - AWS: 1.0 -- See https://docs.aws.amazon.com/cur/latest/userguide/table-dictionary-focus-1-0-aws-columns.html\r\n// - GCP: Jan-Jun 2024 -- See https://cloud.google.com/resources/google-cloud-focus?e=48754805&hl=en\r\n// Links to (Aug 2024): https://services.google.com/fh/files/misc/focus_guide_v1.pdf\r\n// See also:\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/standard-usage\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/detailed-usage\r\n// - OCI: 1.0 -- See https://docs.oracle.com/iaas/Content/Billing/Concepts/costusagereportsoverview.htm#costreports__focus-cost-report-schema\r\n//\r\n// Support for non-Azure data is limited to ingestion only. Data is not transformed across versions.\r\n//======================================================================================================================\r\n\r\n// Costs_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: All costs transformed to FOCUS 1.0. Use Costs_transform_v1_2() instead.', folder='Costs')\r\nCosts_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n Costs_raw\r\n //\r\n // Change real to decimal\r\n | extend\r\n BilledCost = todecimal(BilledCost),\r\n CommitmentDiscountQuantity = todecimal(CommitmentDiscountQuantity),\r\n ConsumedQuantity = todecimal(ConsumedQuantity),\r\n ContractedCost = todecimal(ContractedCost),\r\n ContractedUnitPrice = todecimal(ContractedUnitPrice),\r\n EffectiveCost = todecimal(EffectiveCost),\r\n ListCost = todecimal(ListCost),\r\n ListUnitPrice = todecimal(ListUnitPrice),\r\n PricingQuantity = todecimal(PricingQuantity),\r\n UsageAmount = todecimal(UsageAmount),\r\n UsageQuantity = todecimal(UsageQuantity),\r\n x_BilledCostInUsd = todecimal(x_BilledCostInUsd),\r\n x_BilledUnitPrice = todecimal(x_BilledUnitPrice),\r\n x_BillingExchangeRate = todecimal(x_BillingExchangeRate),\r\n x_ContractedCostInUsd = todecimal(x_ContractedCostInUsd),\r\n x_Cost = todecimal(x_Cost),\r\n x_CurrencyConversionRate = todecimal(x_CurrencyConversionRate),\r\n x_EffectiveCostInUsd = todecimal(x_EffectiveCostInUsd),\r\n x_EffectiveUnitPrice = todecimal(x_EffectiveUnitPrice),\r\n x_ListCostInUsd = todecimal(x_ListCostInUsd),\r\n x_OnDemandCost = todecimal(x_OnDemandCost),\r\n x_OnDemandCostInUsd = todecimal(x_OnDemandCostInUsd),\r\n x_OnDemandUnitPrice = todecimal(x_OnDemandUnitPrice),\r\n x_PricingBlockSize = todecimal(x_PricingBlockSize)\r\n //\r\n // Dedupe rows\r\n | extend x_IngestionTime = ingestion_time()\r\n | extend x_ChargeId = ''\r\n // TODO: Consider adding a unique charge ID per row\r\n // hash_sha256(strcat(\r\n // // DO NOT CHANGE COLUMNS OR COLUMN ORDER\r\n // // 1. Resource hierarchy (including resource name), highest to lowest\r\n // BillingAccountId,\r\n // x_InvoiceSectionId,\r\n // x_AccountOwnerId,\r\n // SubAccountId,\r\n // x_ResourceGroupName,\r\n // ResourceName,\r\n // // 2. Resource details\r\n // ResourceId,\r\n // RegionId,\r\n // Tags,\r\n // CommitmentDiscountId,\r\n // x_CostCenter,\r\n // // 4. Meter details\r\n // SkuPriceId,\r\n // x_SkuMeterId,\r\n // x_SkuPartNumber,\r\n // x_SkuOfferId,\r\n // x_SkuDetails,\r\n // // 5. Date\r\n // ChargePeriodStart\r\n // ))\r\n //\r\n // Identify data quality issues\r\n | extend x_SourceChanges = trim_end(',', strcat(\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and ChargeFrequency == 'Usage-Based', 'InvalidChargeFrequency,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and EffectiveCost > 0, 'InvalidEffectiveCost,', ''),\r\n iff((isempty(ContractedCost) or ContractedCost == 0) and EffectiveCost != 0, 'MissingContractedCost,', ''),\r\n iff((isempty(ContractedUnitPrice) or ContractedUnitPrice == 0) and x_EffectiveUnitPrice != 0, 'MissingContractedUnitPrice,', ''),\r\n iff(ListCost < ContractedCost, 'ListCostLessThanContractedCost,', ''),\r\n iff(ContractedCost < EffectiveCost, 'ContractedCostLessThanEffectiveCost,', ''),\r\n iff((isempty(ListCost) or ListCost == 0) and (ContractedCost != 0 or EffectiveCost != 0), 'MissingListCost,', ''),\r\n iff((isempty(ListUnitPrice) or ListUnitPrice == 0) and (ContractedUnitPrice != 0 or x_EffectiveUnitPrice != 0), 'MissingListUnitPrice,', ''),\r\n iff((isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(x_BilledUnitPrice - x_EffectiveUnitPrice) < 0.0001)\r\n or (isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(ContractedUnitPrice - x_EffectiveUnitPrice) < 0.0001),\r\n 'XEffectiveUnitPriceRoundingError,', ''),\r\n iff(ConsumedQuantity == 0 and (EffectiveCost != 0 or BilledCost != 0), 'MissingConsumedQuantity,', ''),\r\n iff(PricingQuantity == 0 and (EffectiveCost != 0 or BilledCost != 0), 'MissingPricingQuantity,', ''),\r\n iff(isempty(ProviderName), 'MissingProviderName,', ''),\r\n iff(isempty(PublisherName), 'MissingPublisherName,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceId), 'MissingResourceId,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceName), 'MissingResourceName,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceType), 'MissingResourceType,', ''),\r\n iff(BilledCost > 0 and x_BilledUnitPrice == 0, 'MissingXBilledUnitPrice,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(x_ResourceType), 'MissingXResourceType,', ''),\r\n iff(PricingCategory == 'Standard' and isnotempty(CommitmentDiscountId) and ChargeCategory == 'Usage', 'PricingCategoryShouldBeCommitted,', ''),\r\n iff(x_SkuTerm == '1Year' or x_SkuTerm == '3Years' or x_SkuTerm == '5Years', 'SkuTermShouldBeAnInteger,', '')\r\n ))\r\n //\r\n // Fix columns needed in other changes\r\n | extend ProviderName = case(\r\n isnotempty(ProviderName), ProviderName,\r\n isnotempty(coalesce(x_CostCategories, x_Discount, x_Operation, x_ServiceCode, x_UsageType)), 'AWS',\r\n isnotempty(coalesce(tostring(UsageAmount), tostring(x_Cost), x_Credits, x_CostType, tostring(x_CurrencyConversionRate), tostring(x_ExportTime), x_Project, x_ServiceId)), 'GCP',\r\n isnotempty(coalesce(x_BillingProfileId, x_InvoiceSectionId)), 'Microsoft',\r\n ''\r\n )\r\n //\r\n // Identify source\r\n | extend x_SourceName = coalesce(x_SourceName, iff(isnotempty(x_BillingProfileId), 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(isnotempty(x_BillingProfileId), 'FocusCost', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, case(\r\n isnotempty(coalesce(InvoiceId, SkuMeter, PricingCurrency)) and isempty(SkuPriceDetails) and isnotempty(x_SkuDetails), '1.2-preview',\r\n isnotempty(coalesce(InvoiceId, SkuMeter, PricingCurrency)), '1.2',\r\n isnotempty(coalesce(ChargeClass, CommitmentDiscountStatus, tostring(ConsumedQuantity), ConsumedUnit, tostring(ContractedCost), tostring(ContractedUnitPrice), RegionId, RegionName)), '1.0',\r\n isnotempty(coalesce(ChargeSubcategory, Region, tostring(UsageQuantity), UsageUnit)), iff(ProviderName == 'Microsoft', '1.0-preview(v1)', '1.0-preview'),\r\n ''\r\n ))\r\n // Append version check error code\r\n | extend x_SourceChanges = iff(x_SourceVersion == '1.0', x_SourceChanges,\r\n strcat(x_SourceChanges, iff(isempty(x_SourceChanges), '', ','), iff(x_SourceVersion == '', 'UnknownFocusVersion', 'LegacyFocusVersion'))\r\n )\r\n //\r\n // Fix quantities\r\n | extend PricingQuantity = case(\r\n PricingQuantity != 0 or (EffectiveCost == 0 and BilledCost == 0), PricingQuantity,\r\n PricingQuantity == 0 and isnotempty(BilledCost) and BilledCost != 0 and isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, BilledCost / x_BilledUnitPrice,\r\n PricingQuantity == 0 and isnotempty(BilledCost) and BilledCost != 0 and isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, BilledCost / x_BilledUnitPrice,\r\n PricingQuantity == 0 and isnotempty(EffectiveCost) and EffectiveCost != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0, EffectiveCost / x_EffectiveUnitPrice,\r\n PricingQuantity\r\n )\r\n | extend ConsumedQuantity = case(\r\n isnotempty(ConsumedQuantity) and ConsumedQuantity != 0, ConsumedQuantity,\r\n ChargeCategory == 'Usage', PricingQuantity / coalesce(x_PricingBlockSize, decimal(1)),\r\n ConsumedQuantity\r\n )\r\n //\r\n // Populate missing prices -- mapping to on-demand prices requires meter ID and offer ID\r\n | extend tmp_MissingPrices = ProviderName == 'Microsoft'\r\n and (ListUnitPrice == 0 or ContractedUnitPrice == 0)\r\n and x_EffectiveUnitPrice != 0\r\n and not(CommitmentDiscountCategory == 'Spend' and CommitmentDiscountStatus == 'Unused')\r\n and isnotempty(strcat(x_SkuMeterId, x_SkuOfferId))\r\n | as allCosts\r\n | where tmp_MissingPrices\r\n | extend tmp_ReservationPriceLookupKey = tolower(strcat(x_BillingProfileId, substring(ChargePeriodStart, 0, 7), x_SkuMeterId, x_SkuOfferId))\r\n | as costsWithMissingPrices\r\n | join kind=leftouter (\r\n Prices_final_v1_0\r\n | extend tmp_ReservationPriceLookupKey = tolower(strcat(x_BillingProfileId, substring(x_EffectivePeriodStart, 0, 7), x_SkuMeterId, x_SkuOfferId))\r\n | where x_SkuPriceType == 'Consumption' and tmp_ReservationPriceLookupKey in ((costsWithMissingPrices | summarize by tmp_ReservationPriceLookupKey))\r\n | summarize ListUnitPrice = min(ListUnitPrice), ContractedUnitPrice = min(ContractedUnitPrice) by tmp_ReservationPriceLookupKey, x_PricingBlockSize, PricingUnit\r\n ) on tmp_ReservationPriceLookupKey\r\n //\r\n // Select the best price to use for each row\r\n // TODO: Save values before changing -- | extend x_old_ContractedUnitPrice = ContractedUnitPrice, x_old_EffectiveUnitPrice = x_EffectiveUnitPrice, x_old_ListUnitPrice = ListUnitPrice, x_old_ListCost = ListCost, x_old_ContractedCost = ContractedCost\r\n | extend x_EffectiveUnitPrice = case(\r\n // If price is a rounding error away from the billed price, use the billed price\r\n isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(x_BilledUnitPrice - x_EffectiveUnitPrice) < 0.0001, x_BilledUnitPrice,\r\n // If price is a rounding error away from the contracted price, use the contracted price\r\n isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(ContractedUnitPrice - x_EffectiveUnitPrice) < 0.0001, ContractedUnitPrice,\r\n x_EffectiveUnitPrice\r\n )\r\n | extend ContractedUnitPrice = case(\r\n // If price is already correct, keep that\r\n (isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0) or (EffectiveCost == 0 and BilledCost == 0), ContractedUnitPrice,\r\n // If both prices use the same scale, use the new one\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize == x_PricingBlockSize1, ContractedUnitPrice1 * x_BillingExchangeRate,\r\n // If prices are the same unit but not the same scale, use the new one but correct the scale\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize != x_PricingBlockSize1 and isnotempty(x_PricingBlockSize) and isnotempty(x_PricingBlockSize1), ContractedUnitPrice1 * x_BillingExchangeRate / x_PricingBlockSize1 * x_PricingBlockSize,\r\n // If billed price is available, assume the billed price is the same as contracted price to support aggregations\r\n isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, x_EffectiveUnitPrice,\r\n // Otherwise, assume the effective price is the same as contracted price to support aggregations\r\n x_EffectiveUnitPrice\r\n )\r\n | extend ListUnitPrice = case(\r\n // If price is already correct, keep that\r\n (isnotempty(ListUnitPrice) and ListUnitPrice != 0) or (EffectiveCost == 0 and BilledCost == 0), ListUnitPrice,\r\n // If both prices use the same scale, use the new one\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize == x_PricingBlockSize1, ListUnitPrice1 * x_BillingExchangeRate,\r\n // If prices are the same unit but not the same scale, use the new one but correct the scale\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize != x_PricingBlockSize1 and isnotempty(x_PricingBlockSize) and isnotempty(x_PricingBlockSize1), ListUnitPrice1 * x_BillingExchangeRate / x_PricingBlockSize1 * x_PricingBlockSize,\r\n // Otherwise, assume the contracted price is the same as list price to support aggregations\r\n ContractedUnitPrice\r\n )\r\n // Calculate missing costs based on new prices -- If cost is already correct, keep that; if not and price is available, recalculate the cost; otherwise, keep the existing cost\r\n | extend ContractedCost = case(\r\n // If not set or there's no cost, keep the original value\r\n (isnotempty(ContractedCost) and ContractedCost != 0) or (EffectiveCost == 0 and BilledCost == 0), ContractedCost,\r\n // ContractedCost is 0 in all other scenarios...\r\n // If 0 and there's a billed cost and prices are the same, use BilledCost\r\n isnotempty(BilledCost) and BilledCost != 0 and ContractedUnitPrice == x_BilledUnitPrice, BilledCost,\r\n // If 0 and there's a billed cost and prices are the same, use EffectiveCost\r\n isnotempty(EffectiveCost) and EffectiveCost != 0 and ContractedUnitPrice == x_EffectiveUnitPrice, EffectiveCost,\r\n // If 0 and there's a price, calculate the cost based on the price\r\n isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0, ContractedUnitPrice * PricingQuantity,\r\n // If 0 and there's no price, assume EffectiveCost\r\n isempty(ContractedUnitPrice) or ContractedUnitPrice == 0, EffectiveCost,\r\n // Fall back to the original value for any unhandled scenarios\r\n ContractedCost\r\n )\r\n | extend ListCost = case(\r\n // If not set or there's no cost, keep the original value\r\n (isnotempty(ListCost) and ListCost != 0) or (EffectiveCost == 0 and BilledCost == 0), ListCost,\r\n // ListCost is 0 in all other scenarios...\r\n // If 0 and there's a contracted cost and prices are the same, use ContractedCost\r\n isnotempty(ContractedCost) and ContractedCost != 0 and ListUnitPrice == ContractedUnitPrice, ContractedCost,\r\n // If 0 and there's a price, calculate the cost based on the price\r\n isnotempty(ListUnitPrice) and ListUnitPrice != 0, ListUnitPrice * PricingQuantity,\r\n // If 0 and there's no price, assume ContractedCost\r\n isempty(ListUnitPrice) or ListUnitPrice == 0, ContractedCost,\r\n // Fall back to the original value for any unhandled scenarios\r\n ListCost\r\n )\r\n // Merge the rest of the unmodified cost records and remove excess columns\r\n | union (allCosts | where not(tmp_MissingPrices))\r\n | project-away x_PricingBlockSize1, PricingUnit1, ListUnitPrice1, ContractedUnitPrice1, tmp_MissingPrices, tmp_ReservationPriceLookupKey, tmp_ReservationPriceLookupKey1\r\n //\r\n // BUG: Fix ContractedCost that has bad values\r\n | extend ContractedCost = iff(ProviderName == 'Microsoft' and isnotempty(PricingQuantity) and isnotempty(x_PricingBlockSize) and ContractedCost != ContractedUnitPrice * PricingQuantity, ContractedUnitPrice * PricingQuantity, ContractedCost)\r\n //\r\n // Handle FOCUS 1.0-preview UsageQuantity/Unit\r\n | extend ConsumedQuantity = iff(ChargeCategory == 'Usage', coalesce(ConsumedQuantity, UsageQuantity, UsageAmount), todecimal(''))\r\n | extend ConsumedUnit = iff(ChargeCategory == 'Usage' and isnotempty(ConsumedQuantity), coalesce(ConsumedUnit, UsageUnit, 'Units'), '')\r\n //\r\n // Convert IDs to lowercase for consistency\r\n | extend CommitmentDiscountId = tolower(CommitmentDiscountId)\r\n //\r\n // BUG: Remove EffectiveCost for commitment discount purchases\r\n | extend EffectiveCost = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), decimal(0), EffectiveCost)\r\n | extend x_EffectiveCostInUsd = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), decimal(0), x_EffectiveCostInUsd)\r\n //\r\n // Clean up resource columns\r\n | extend ResourceId = case(\r\n isnotempty(ResourceId), ResourceId,\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), CommitmentDiscountId,\r\n ResourceId)\r\n | extend ResourceName = tolower(case(\r\n isnotempty(ResourceName), ResourceName,\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountName), CommitmentDiscountName,\r\n isnotempty(ResourceId), parse_resourceid(ResourceId).ResourceName,\r\n ResourceName))\r\n | extend x_ResourceType = case(\r\n isnotempty(x_ResourceType), x_ResourceType,\r\n isnotempty(ResourceId), parse_resourceid(ResourceId).x_ResourceType,\r\n x_ResourceType)\r\n | extend ResourceType = case(\r\n // Use existing resource type display name unless it's an internal resource type ID\r\n isnotempty(ResourceType) and tolower(ResourceType) != tolower(x_ResourceType) and ResourceType !contains '/', ResourceType,\r\n // Use CommitmentDiscountType for commitment discount purchases\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountType), CommitmentDiscountType,\r\n // Look up display name from internal type\r\n isnotempty(x_ResourceType), coalesce(resource_type(x_ResourceType).SingularDisplayName, ResourceType, x_ResourceType),\r\n ResourceType)\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n AvailabilityZone,\r\n BilledCost,\r\n BillingAccountId = tolower(BillingAccountId),\r\n BillingAccountName,\r\n BillingAccountType,\r\n BillingCurrency,\r\n BillingPeriodEnd = startofmonth(BillingPeriodEnd),\r\n BillingPeriodStart = startofmonth(BillingPeriodStart),\r\n ChargeCategory = case(\r\n // Handle FOCUS 1.0-preview ChargeSubcategory\r\n ChargeSubcategory == 'Credit', 'Credit',\r\n ChargeSubcategory == 'Refund', 'Purchase', // We are assuming purchase refunds since we don't have data to indicate usage refunds\r\n ChargeCategory\r\n ),\r\n ChargeClass = case(ChargeSubcategory == 'Refund', 'Correction', ChargeClass),\r\n ChargeDescription,\r\n // BUG: ChargeFrequency shows \"Usage-Based\" for monthly recurring savings plan purchases\r\n ChargeFrequency = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and ChargeFrequency == 'Usage-Based' and ProviderName == 'Microsoft' and x_SourceVersion startswith '1.0', 'Recurring', ChargeFrequency),\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId = tolower(CommitmentDiscountId),\r\n CommitmentDiscountName,\r\n CommitmentDiscountStatus = case(\r\n // Handle FOCUS 1.0-preview ChargeSubcategory\r\n ChargeSubcategory == 'Used Commitment', 'Used',\r\n ChargeSubcategory == 'Unused Commitment', 'Unused',\r\n CommitmentDiscountStatus\r\n ),\r\n CommitmentDiscountType,\r\n ConsumedQuantity,\r\n ConsumedUnit,\r\n ContractedCost = coalesce(ContractedCost, x_OnDemandCost, x_Cost),\r\n ContractedUnitPrice = coalesce(ContractedUnitPrice, x_OnDemandUnitPrice),\r\n EffectiveCost,\r\n InvoiceIssuerName,\r\n ListCost,\r\n ListUnitPrice,\r\n PricingCategory = case(\r\n // Handle FOCUS 1.0-preview PricingCategory values\r\n PricingCategory == 'On-Demand', 'Standard',\r\n PricingCategory == 'Commitment-Based', 'Committed',\r\n PricingCategory\r\n ),\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n // Handle missing PublisherName values\r\n PublisherName = case(PublisherName == 'Microsoft Corporation', 'Microsoft', isnotempty(PublisherName), PublisherName, x_PublisherCategory == 'Cloud Provider', ProviderName, ''),\r\n // Handle FOCUS 1.0-preview Region column\r\n RegionId = coalesce(RegionId, iff(ProviderName == 'Microsoft', replace_string(tolower(Region), ' ', ''), Region)),\r\n RegionName = coalesce(RegionName, Region),\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n SkuId,\r\n SkuPriceId,\r\n SubAccountId,\r\n SubAccountName,\r\n SubAccountType, // Azure 1.0-preview(v1)+\r\n Tags = parse_json(Tags),\r\n x_AccountId, // Azure 1.0-preview(v1)+\r\n x_AccountName, // Azure 1.0-preview(v1)+\r\n x_AccountOwnerId, // Azure 1.0-preview(v1)+\r\n x_BilledCostInUsd, // Azure 1.0-preview(v1)+\r\n x_BilledUnitPrice, // Azure 1.0-preview(v1)+\r\n x_BillingAccountAgreement = case(\r\n ProviderName == 'Microsoft' and x_BillingAccountId == x_BillingProfileId, 'EA',\r\n ProviderName == 'Microsoft' and x_BillingAccountId != x_BillingProfileId, 'MCA',\r\n ProviderName\r\n ), // Hubs add-on\r\n x_BillingAccountId, // Azure 1.0-preview(v1)+\r\n x_BillingAccountName, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRate, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRateDate, // Azure 1.0-preview(v1)+\r\n x_BillingProfileId, // Azure 1.0-preview(v1)+\r\n x_BillingProfileName, // Azure 1.0-preview(v1)+\r\n x_ChargeId, // Azure 1.0-preview(v1) only\r\n x_ContractedCostInUsd = coalesce(x_ContractedCostInUsd, x_OnDemandCostInUsd), // Azure 1.0+\r\n x_CostAllocationRuleName, // Azure 1.0-preview(v1)+\r\n x_CostCategories = parse_json(x_CostCategories), // AWS 1.0 (JSON)\r\n x_CostCenter, // Azure 1.0-preview(v1)+\r\n x_Credits = parse_json(x_Credits), // GCP Jan 2024\r\n x_CostType, // GCP Jan 2024\r\n x_CurrencyConversionRate, // GCP Jun 2024\r\n x_CustomerId, // Azure 1.0-preview(v1)+\r\n x_CustomerName, // Azure 1.0-preview(v1)+\r\n x_Discount = parse_json(x_Discount), // AWS 1.0 (JSON)\r\n x_EffectiveCostInUsd, // Azure 1.0-preview(v1)+\r\n x_EffectiveUnitPrice, // Azure 1.0-preview(v1)+\r\n x_ExportTime, // GCP Jan 2024\r\n x_IngestionTime, // Hubs add-on\r\n x_InvoiceId = coalesce(InvoiceId, x_InvoiceId), // Azure 1.0-preview(v1)+\r\n x_InvoiceIssuerId, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionId = case( // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionId == '-2', '',\r\n x_InvoiceSectionId\r\n ),\r\n x_InvoiceSectionName = case( // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionName == 'Unassigned', '',\r\n x_InvoiceSectionName\r\n ),\r\n x_ListCostInUsd, // Azure 1.0-preview(v1)+\r\n x_Location, // GCP Jan 2024\r\n x_Operation, // AWS 1.0\r\n x_PartnerCreditApplied, // Azure 1.0-preview(v1)+\r\n x_PartnerCreditRate, // Azure 1.0-preview(v1)+\r\n x_PricingBlockSize, // Azure 1.0-preview(v1)+\r\n x_PricingCurrency = coalesce(PricingCurrency, x_PricingCurrency), // Azure 1.0-preview(v1)+\r\n x_PricingSubcategory, // Azure 1.0-preview(v1)+\r\n x_PricingUnitDescription, // Azure 1.0-preview(v1)+\r\n x_Project, // GCP Jan 2024\r\n x_PublisherCategory, // Azure 1.0-preview(v1)+\r\n x_PublisherId, // Azure 1.0-preview(v1)+\r\n x_ResellerId, // Azure 1.0-preview(v1)+\r\n x_ResellerName, // Azure 1.0-preview(v1)+\r\n x_ResourceGroupName = tolower(x_ResourceGroupName), // Azure 1.0-preview(v1)+\r\n x_ResourceType, // Azure 1.0-preview(v1)+\r\n x_ServiceCode, // AWS 1.0\r\n x_ServiceId, // GCP Jan 2024\r\n x_ServicePeriodEnd, // Azure 1.0-preview(v1)+\r\n x_ServicePeriodStart, // Azure 1.0-preview(v1)+\r\n x_SkuDescription, // Azure 1.0-preview(v1)+\r\n x_SkuDetails = parse_json(x_SkuDetails), // Azure 1.0-preview(v1)+\r\n x_SkuIsCreditEligible, // Azure 1.0-preview(v1)+\r\n x_SkuMeterCategory, // Azure 1.0-preview(v1)+\r\n x_SkuMeterId, // Azure 1.0-preview(v1)+\r\n x_SkuMeterName = coalesce(SkuMeter, x_SkuMeterName), // Azure 1.0-preview(v1)+\r\n x_SkuMeterSubcategory, // Azure 1.0-preview(v1)+\r\n x_SkuOfferId, // Azure 1.0-preview(v1)+\r\n x_SkuOrderId, // Azure 1.0-preview(v1)+\r\n x_SkuOrderName, // Azure 1.0-preview(v1)+\r\n x_SkuPartNumber, // Azure 1.0-preview(v1)+\r\n x_SkuRegion, // Azure 1.0-preview(v1)+\r\n x_SkuServiceFamily, // Azure 1.0-preview(v1)+\r\n x_SkuTerm, // Azure 1.0-preview(v1)+\r\n x_SkuTier, // Azure 1.0-preview(v1)+\r\n x_SourceChanges, // Hubs add-on\r\n x_SourceName, // Hubs add-on\r\n x_SourceProvider, // Hubs add-on\r\n x_SourceType, // Hubs add-on\r\n x_SourceVersion, // Hubs add-on\r\n x_UsageType // AWS 1.0\r\n}\r\n\r\n// Costs_final_v1_0 table\r\n.create-merge table Costs_final_v1_0 (\r\n AvailabilityZone: string,\r\n BilledCost: decimal,\r\n BillingAccountId: string,\r\n BillingAccountName: string,\r\n BillingAccountType: string, // Azure 1.0-preview(v1)+\r\n BillingCurrency: string,\r\n BillingPeriodEnd: datetime,\r\n BillingPeriodStart: datetime,\r\n ChargeCategory: string,\r\n ChargeClass: string,\r\n ChargeDescription: string,\r\n ChargeFrequency: string,\r\n ChargePeriodEnd: datetime,\r\n ChargePeriodStart: datetime,\r\n CommitmentDiscountCategory: string, // FOCUS 1.0-preview only\r\n CommitmentDiscountId: string,\r\n CommitmentDiscountName: string,\r\n CommitmentDiscountStatus: string,\r\n CommitmentDiscountType: string,\r\n ConsumedQuantity: decimal,\r\n ConsumedUnit: string,\r\n ContractedCost: decimal,\r\n ContractedUnitPrice: decimal,\r\n EffectiveCost: decimal,\r\n InvoiceIssuerName: string,\r\n ListCost: decimal,\r\n ListUnitPrice: decimal,\r\n PricingCategory: string,\r\n PricingQuantity: decimal,\r\n PricingUnit: string,\r\n ProviderName: string,\r\n PublisherName: string,\r\n RegionId: string,\r\n RegionName: string,\r\n ResourceId: string,\r\n ResourceName: string,\r\n ResourceType: string,\r\n ServiceCategory: string,\r\n ServiceName: string,\r\n SkuId: string,\r\n SkuPriceId: string,\r\n SubAccountId: string,\r\n SubAccountName: string,\r\n SubAccountType: string,\r\n Tags: dynamic,\r\n x_AccountId: string, // Azure 1.0-preview(v1)+\r\n x_AccountName: string, // Azure 1.0-preview(v1)+\r\n x_AccountOwnerId: string, // Azure 1.0-preview(v1)+\r\n x_BilledCostInUsd: decimal, // Azure 1.0-preview(v1)+\r\n x_BilledUnitPrice: decimal, // Azure 1.0-preview(v1)+\r\n x_BillingAccountAgreement: string, // Hubs add-on\r\n x_BillingAccountId: string, // Azure 1.0-preview(v1)+\r\n x_BillingAccountName: string, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRate: decimal, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRateDate: datetime, // Azure 1.0-preview(v1)+\r\n x_BillingProfileId: string, // Azure 1.0-preview(v1)+\r\n x_BillingProfileName: string, // Azure 1.0-preview(v1)+\r\n x_ChargeId: string, // Azure 1.0-preview(v1) only\r\n x_ContractedCostInUsd: decimal, // Azure 1.0+\r\n x_CostAllocationRuleName: string, // Azure 1.0-preview(v1)+\r\n x_CostCategories: dynamic, // AWS 1.0 (JSON)\r\n x_CostCenter: string, // Azure 1.0-preview(v1)+\r\n x_Credits: dynamic, // GCP Jan 2024\r\n x_CostType: string, // GCP Jan 2024\r\n x_CurrencyConversionRate: decimal, // GCP Jun 2024\r\n x_CustomerId: string, // Azure 1.0-preview(v1)+\r\n x_CustomerName: string, // Azure 1.0-preview(v1)+\r\n x_Discount: dynamic, // AWS 1.0 (JSON)\r\n x_EffectiveCostInUsd: decimal, // Azure 1.0-preview(v1)+\r\n x_EffectiveUnitPrice: decimal, // Azure 1.0-preview(v1)+\r\n x_ExportTime: datetime, // GCP Jan 2024\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_InvoiceId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceIssuerId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionName: string, // Azure 1.0-preview(v1)+\r\n x_ListCostInUsd: decimal, // Azure 1.0-preview(v1)+\r\n x_Location: string, // GCP Jan 2024\r\n x_Operation: string, // AWS 1.0\r\n x_PartnerCreditApplied: string, // Azure 1.0-preview(v1)+\r\n x_PartnerCreditRate: string, // Azure 1.0-preview(v1)+\r\n x_PricingBlockSize: decimal, // Azure 1.0-preview(v1)+\r\n x_PricingCurrency: string, // Azure 1.0-preview(v1)+\r\n x_PricingSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_PricingUnitDescription: string, // Azure 1.0-preview(v1)+\r\n x_Project: string, // GCP Jan 2024\r\n x_PublisherCategory: string, // Azure 1.0-preview(v1)+\r\n x_PublisherId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceGroupName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceType: string, // Azure 1.0-preview(v1)+\r\n x_ServiceCode: string, // AWS 1.0\r\n x_ServiceId: string, // GCP Jan 2024\r\n x_ServicePeriodEnd: datetime, // Azure 1.0-preview(v1)+\r\n x_ServicePeriodStart: datetime, // Azure 1.0-preview(v1)+\r\n x_SkuDescription: string, // Azure 1.0-preview(v1)+\r\n x_SkuDetails: dynamic, // Azure 1.0-preview(v1)+\r\n x_SkuIsCreditEligible: bool, // Azure 1.0-preview(v1)+\r\n x_SkuMeterCategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterId: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterName: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuOfferId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOrderId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOrderName: string, // Azure 1.0-preview(v1)+\r\n x_SkuPartNumber: string, // Azure 1.0-preview(v1)+\r\n x_SkuRegion: string, // Azure 1.0-preview(v1)+\r\n x_SkuServiceFamily: string, // Azure 1.0-preview(v1)+\r\n x_SkuTerm: int, // Azure 1.0-preview(v1)+\r\n x_SkuTier: string, // Azure 1.0-preview(v1)+\r\n x_SourceChanges: string, // Hubs add-on\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_UsageType: string // AWS 1.0\r\n)\r\n\r\n// Update policy for Costs_raw -> Costs_final_v1_0 table\r\n.alter table Costs_final_v1_0 policy update\r\n```\r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"Costs_raw\",\r\n \"Query\": \"Costs_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Actual costs |===================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// ActualCosts_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: ActualCost exports transformed to FOCUS 1.0. Use ActualCosts_transform_v1_2() instead.', folder='Costs')\r\nActualCosts_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n // TODO: Transform actual costs to FOCUS 1.0 format\r\n ActualCosts_raw\r\n | where ChargeType in ('Purchase', 'Refund') and isnotempty(ReservationId)\r\n //\r\n //\r\n // !!! The rest of the query is reused for both the ActualCosts and AmortizedCosts queries -- Copy all changes in both transform functions !!!\r\n //\r\n //\r\n | extend x_AmortizationClass = case(\r\n ChargeType in ('Purchase', 'Refund') and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', 'Amortized Charge',\r\n ''\r\n )\r\n | extend tmp_ResourceInfo = parse_resourceid(ResourceId)\r\n // TODO: PricingCategory needs to include savings plan usage and spot usage\r\n | extend PricingCategory = case(\r\n x_AmortizationClass == 'Amortized Charge', 'Committed',\r\n ChargeType in ('Usage', 'Purchase'), 'Standard',\r\n ''\r\n )\r\n | extend x_ResourceType = tostring(tmp_ResourceInfo.x_ResourceType)\r\n | project-rename\r\n PricingQuantity = Quantity,\r\n x_PricingUnitDescription = UnitOfMeasure\r\n | join kind=leftouter (PricingUnits) on x_PricingUnitDescription\r\n | join kind=leftouter (Regions) on ResourceLocation\r\n | join kind=leftouter (ResourceTypes | project x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n // TODO: Add the following in 1.2: ServiceSubcategory, PublisherName, x_PublisherCategory, x_Environment, x_ServiceModel\r\n | join kind=leftouter (Services | where isnotempty(x_ResourceType) | project x_ResourceType, ServiceName, ServiceCategory) on x_ResourceType\r\n | join kind=leftouter (Services | where isnotempty(x_ConsumedService) | summarize take_any(ServiceName), take_any(ServiceCategory) by ConsumedService = x_ConsumedService) on ConsumedService\r\n | extend CommitmentDiscountCategory = iff(isnotempty(ReservationId), 'Usage', '') // TODO: CommitmentDiscountCategory needs to handle savings plans\r\n | project\r\n AvailabilityZone = AvailabilityZone,\r\n BilledCost = iff(x_AmortizationClass == 'Amortized Charge', real(0), Cost),\r\n BillingAccountId = strcat('/providers/microsoft.billing/billingaccounts/', BillingAccountId, iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), '', strcat('/billingprofiles/', BillingProfileId))),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName),\r\n BillingAccountType = iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), 'Billing Profile', 'Billing Account'),\r\n BillingCurrency,\r\n BillingPeriodEnd = startofmonth(BillingPeriodEndDate, 1),\r\n BillingPeriodStart = startofmonth(BillingPeriodStartDate),\r\n CapacityReservationId = '',\r\n CapacityReservationStatus = '',\r\n ChargeCategory = case(\r\n ChargeType in ('Usage', 'Purchase', 'Credit', 'Tax'), ChargeType,\r\n ChargeType in ('UnusedReservation', 'UnusedSavingsPlan'), 'Usage',\r\n ChargeType == 'Refund', 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = iff(ChargeType == 'Refund', 'Correction', ''),\r\n ChargeDescription = Product,\r\n ChargeFrequency = case(\r\n Frequency == 'UsageBased', 'Usage-Based',\r\n Frequency == 'OneTime', 'One-Time',\r\n Frequency // \"Recurring\" and any fallback\r\n ),\r\n ChargePeriodStart = Date,\r\n ChargePeriodEnd = Date + 1d,\r\n ChargeSubcategory = '',\r\n // TODO: CommitmentDiscount* columns need to handle savings plans\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId = iff(isnotempty(ReservationId), strcat('/providers/microsoft.capacity/reservationorders/', ProductOrderId, '/reservations/', ReservationId), ''),\r\n CommitmentDiscountName = ReservationName,\r\n CommitmentDiscountQuantity = real(null),\r\n CommitmentDiscountStatus = case(\r\n isempty(ReservationId), '',\r\n isnotempty(ReservationId) and ChargeType == 'Usage', 'Used',\r\n ChargeType startswith 'Unused', 'Unused',\r\n 'Unused'\r\n ),\r\n CommitmentDiscountType = iff(isnotempty(ReservationId), 'Reservation', ''),\r\n CommitmentDiscountUnit = '',\r\n ConsumedQuantity = PricingQuantity * x_PricingBlockSize,\r\n ConsumedUnit = PricingUnit,\r\n ContractedCost = UnitPrice * PricingQuantity,\r\n ContractedUnitPrice = UnitPrice,\r\n EffectiveCost = iff(x_AmortizationClass == 'Principal', real(0), Cost),\r\n InvoiceId = '',\r\n InvoiceIssuerName = 'Microsoft',\r\n ListCost = real(null),\r\n ListUnitPrice = real(null),\r\n PricingCategory,\r\n PricingCurrency = '',\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName = 'Microsoft',\r\n PublisherName = iff(PublisherType == 'Marketplace', PublisherName, 'Microsoft'),\r\n Region = '',\r\n RegionId,\r\n RegionName,\r\n ResourceId = tostring(tmp_ResourceInfo.ResourceId),\r\n ResourceName = tostring(tmp_ResourceInfo.ResourceName),\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory = '',\r\n SkuId = '',\r\n SkuMeter = '',\r\n SkuPriceDetails = '',\r\n SkuPriceId = '',\r\n SubAccountId = strcat('/subscriptions/', SubscriptionId),\r\n SubAccountName = SubscriptionName,\r\n SubAccountType = iff(isempty(SubscriptionId), '', 'Subscription'),\r\n Tags = iff(Tags startswith '{', Tags, strcat('{', Tags, '}')),\r\n UsageAmount = real(null),\r\n UsageQuantity = real(null),\r\n UsageUnit = '',\r\n x_AccountId = '',\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerId,\r\n x_AmortizationClass = '',\r\n x_BilledCostInUsd = real(null),\r\n x_BilledUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), UnitPrice),\r\n x_BillingAccountId = BillingAccountId,\r\n x_BillingAccountName = BillingAccountName,\r\n x_BillingExchangeRate = real(null),\r\n x_BillingExchangeRateDate = datetime(null),\r\n x_BillingProfileId = BillingProfileId,\r\n x_BillingProfileName = BillingProfileName,\r\n x_BillingItemCode = '',\r\n x_BillingItemName = '',\r\n x_ChargeId = '',\r\n x_CommodityCode = '',\r\n x_CommodityName = '',\r\n x_ComponentName = '',\r\n x_ComponentType = '',\r\n x_ContractedCostInUsd = real(null),\r\n x_Cost = real(null),\r\n x_CostAllocationRuleName = '',\r\n x_CostCategories = '',\r\n x_CostCenter = CostCenter,\r\n x_CostType = '',\r\n x_Credits = '',\r\n x_CurrencyConversionRate = real(null),\r\n x_CustomerId = '',\r\n x_CustomerName = '',\r\n x_Discount = '',\r\n x_EffectiveCostInUsd = real(null),\r\n x_EffectiveUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), EffectivePrice),\r\n x_ExportTime = datetime(null),\r\n x_InstanceID = '',\r\n x_InvoiceId = '',\r\n x_InvoiceIssuerId = '',\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = InvoiceSection,\r\n x_ListCostInUsd = real(null),\r\n x_Location = '',\r\n x_OnDemandCost = real(null),\r\n x_OnDemandCostInUsd = real(null),\r\n x_OnDemandUnitPrice = real(null),\r\n x_Operation = '',\r\n x_OwnerAccountID = '',\r\n x_PartnerCreditApplied = '',\r\n x_PartnerCreditRate = '',\r\n x_PricingBlockSize,\r\n x_PricingCurrency = iff(BillingAccountId == BillingProfileId, BillingCurrency, ''),\r\n x_PricingSubcategory = case(\r\n // TODO: Add x_SkuTier when supported by C360 -- PricingCategory == 'Standard' and isnotempty(x_SkuTier), 'Tiered',\r\n PricingCategory == 'Standard', 'Standard',\r\n PricingCategory == 'Committed', strcat('Committed ', CommitmentDiscountCategory),\r\n PricingCategory == 'Dynamic', 'Spot',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_Project = '',\r\n x_PublisherCategory = iff(PublisherType == 'Marketplace', 'Vendor', 'Cloud Provider'),\r\n x_PublisherId = '',\r\n x_ResellerId = '',\r\n x_ResellerName = '',\r\n x_ResourceGroupName = ResourceGroup,\r\n x_ResourceType,\r\n x_ServiceCode = '',\r\n x_ServiceId = '',\r\n x_ServiceModel = '',\r\n x_ServicePeriodEnd = datetime(null),\r\n x_ServicePeriodStart = datetime(null),\r\n x_SkuDescription = Product,\r\n x_SkuDetails = iff(AdditionalInfo startswith '{', AdditionalInfo, strcat('{', AdditionalInfo, '}')),\r\n x_SkuIsCreditEligible = IsAzureCreditEligible,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterId = MeterId,\r\n x_SkuMeterName = MeterName,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuOfferId = OfferId,\r\n x_SkuOrderId = ProductOrderId,\r\n x_SkuOrderName = ProductOrderName,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPlanName = '',\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTerm = toint(Term),\r\n x_SkuTier = '',\r\n x_SourceName = 'C360',\r\n x_SourceProvider = 'Microsoft',\r\n x_SourceType = 'ActualCost',\r\n x_SourceVersion = 'C360-2025-04',\r\n x_SubproductName = '',\r\n x_UsageType = ''\r\n}\r\n\r\n// Update policy for ActualCosts_raw -> Costs_raw table\r\n.alter table Costs_raw policy update\r\n``` \r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"ActualCosts_raw\",\r\n \"Query\": \"ActualCosts_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Amortized costs |================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// AmortizedCosts_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: ActualCost exports transformed to FOCUS 1.0. Use AmortizedCosts_transform_v1_2() instead.', folder='Costs')\r\nAmortizedCosts_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n // TODO: Transform actual costs to FOCUS 1.0 format\r\n AmortizedCosts_raw\r\n //\r\n //\r\n // !!! The rest of the query is reused for both the ActualCosts and AmortizedCosts queries -- Copy all changes in both transform functions !!!\r\n //\r\n //\r\n | extend x_AmortizationClass = case(\r\n ChargeType in ('Purchase', 'Refund') and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', 'Amortized Charge',\r\n ''\r\n )\r\n | extend tmp_ResourceInfo = parse_resourceid(ResourceId)\r\n // TODO: PricingCategory needs to include savings plan usage and spot usage\r\n | extend PricingCategory = case(\r\n x_AmortizationClass == 'Amortized Charge', 'Committed',\r\n ChargeType in ('Usage', 'Purchase'), 'Standard',\r\n ''\r\n )\r\n | extend x_ResourceType = tostring(tmp_ResourceInfo.x_ResourceType)\r\n | project-rename\r\n PricingQuantity = Quantity,\r\n x_PricingUnitDescription = UnitOfMeasure\r\n | join kind=leftouter (PricingUnits) on x_PricingUnitDescription\r\n | join kind=leftouter (Regions) on ResourceLocation\r\n | join kind=leftouter (ResourceTypes | project x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n // TODO: Add the following in 1.2: ServiceSubcategory, PublisherName, x_PublisherCategory, x_Environment, x_ServiceModel\r\n | join kind=leftouter (Services | where isnotempty(x_ResourceType) | project x_ResourceType, ServiceName, ServiceCategory) on x_ResourceType\r\n | join kind=leftouter (Services | where isnotempty(x_ConsumedService) | summarize take_any(ServiceName), take_any(ServiceCategory) by ConsumedService = x_ConsumedService) on ConsumedService\r\n | extend CommitmentDiscountCategory = iff(isnotempty(ReservationId), 'Usage', '') // TODO: CommitmentDiscountCategory needs to handle savings plans\r\n | project\r\n AvailabilityZone = AvailabilityZone,\r\n BilledCost = iff(x_AmortizationClass == 'Amortized Charge', real(0), Cost),\r\n BillingAccountId = strcat('/providers/microsoft.billing/billingaccounts/', BillingAccountId, iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), '', strcat('/billingprofiles/', BillingProfileId))),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName),\r\n BillingAccountType = iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), 'Billing Profile', 'Billing Account'),\r\n BillingCurrency,\r\n BillingPeriodEnd = startofmonth(BillingPeriodEndDate, 1),\r\n BillingPeriodStart = startofmonth(BillingPeriodStartDate),\r\n CapacityReservationId = '',\r\n CapacityReservationStatus = '',\r\n ChargeCategory = case(\r\n ChargeType in ('Usage', 'Purchase', 'Credit', 'Tax'), ChargeType,\r\n ChargeType in ('UnusedReservation', 'UnusedSavingsPlan'), 'Usage',\r\n ChargeType == 'Refund', 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = iff(ChargeType == 'Refund', 'Correction', ''),\r\n ChargeDescription = Product,\r\n ChargeFrequency = case(\r\n Frequency == 'UsageBased', 'Usage-Based',\r\n Frequency == 'OneTime', 'One-Time',\r\n Frequency // \"Recurring\" and any fallback\r\n ),\r\n ChargePeriodStart = Date,\r\n ChargePeriodEnd = Date + 1d,\r\n ChargeSubcategory = '',\r\n // TODO: CommitmentDiscount* columns need to handle savings plans\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId = iff(isnotempty(ReservationId), strcat('/providers/microsoft.capacity/reservationorders/', ProductOrderId, '/reservations/', ReservationId), ''),\r\n CommitmentDiscountName = ReservationName,\r\n CommitmentDiscountQuantity = real(null),\r\n CommitmentDiscountStatus = case(\r\n isempty(ReservationId), '',\r\n isnotempty(ReservationId) and ChargeType == 'Usage', 'Used',\r\n ChargeType startswith 'Unused', 'Unused',\r\n 'Unused'\r\n ),\r\n CommitmentDiscountType = iff(isnotempty(ReservationId), 'Reservation', ''),\r\n CommitmentDiscountUnit = '',\r\n ConsumedQuantity = PricingQuantity * x_PricingBlockSize,\r\n ConsumedUnit = PricingUnit,\r\n ContractedCost = UnitPrice * PricingQuantity,\r\n ContractedUnitPrice = UnitPrice,\r\n EffectiveCost = iff(x_AmortizationClass == 'Principal', real(0), Cost),\r\n InvoiceId = '',\r\n InvoiceIssuerName = 'Microsoft',\r\n ListCost = real(null),\r\n ListUnitPrice = real(null),\r\n PricingCategory,\r\n PricingCurrency = '',\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName = 'Microsoft',\r\n PublisherName = iff(PublisherType == 'Marketplace', PublisherName, 'Microsoft'),\r\n Region = '',\r\n RegionId,\r\n RegionName,\r\n ResourceId = tostring(tmp_ResourceInfo.ResourceId),\r\n ResourceName = tostring(tmp_ResourceInfo.ResourceName),\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory = '',\r\n SkuId = '',\r\n SkuMeter = '',\r\n SkuPriceDetails = '',\r\n SkuPriceId = '',\r\n SubAccountId = strcat('/subscriptions/', SubscriptionId),\r\n SubAccountName = SubscriptionName,\r\n SubAccountType = iff(isempty(SubscriptionId), '', 'Subscription'),\r\n Tags = iff(Tags startswith '{', Tags, strcat('{', Tags, '}')),\r\n UsageAmount = real(null),\r\n UsageQuantity = real(null),\r\n UsageUnit = '',\r\n x_AccountId = '',\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerId,\r\n x_AmortizationClass = '',\r\n x_BilledCostInUsd = real(null),\r\n x_BilledUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), UnitPrice),\r\n x_BillingAccountId = BillingAccountId,\r\n x_BillingAccountName = BillingAccountName,\r\n x_BillingExchangeRate = real(null),\r\n x_BillingExchangeRateDate = datetime(null),\r\n x_BillingProfileId = BillingProfileId,\r\n x_BillingProfileName = BillingProfileName,\r\n x_BillingItemCode = '',\r\n x_BillingItemName = '',\r\n x_ChargeId = '',\r\n x_CommodityCode = '',\r\n x_CommodityName = '',\r\n x_ComponentName = '',\r\n x_ComponentType = '',\r\n x_ContractedCostInUsd = real(null),\r\n x_Cost = real(null),\r\n x_CostAllocationRuleName = '',\r\n x_CostCategories = '',\r\n x_CostCenter = CostCenter,\r\n x_CostType = '',\r\n x_Credits = '',\r\n x_CurrencyConversionRate = real(null),\r\n x_CustomerId = '',\r\n x_CustomerName = '',\r\n x_Discount = '',\r\n x_EffectiveCostInUsd = real(null),\r\n x_EffectiveUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), EffectivePrice),\r\n x_ExportTime = datetime(null),\r\n x_InstanceID = '',\r\n x_InvoiceId = '',\r\n x_InvoiceIssuerId = '',\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = InvoiceSection,\r\n x_ListCostInUsd = real(null),\r\n x_Location = '',\r\n x_OnDemandCost = real(null),\r\n x_OnDemandCostInUsd = real(null),\r\n x_OnDemandUnitPrice = real(null),\r\n x_Operation = '',\r\n x_OwnerAccountID = '',\r\n x_PartnerCreditApplied = '',\r\n x_PartnerCreditRate = '',\r\n x_PricingBlockSize,\r\n x_PricingCurrency = iff(BillingAccountId == BillingProfileId, BillingCurrency, ''),\r\n x_PricingSubcategory = case(\r\n // TODO: Add x_SkuTier when supported by C360 -- PricingCategory == 'Standard' and isnotempty(x_SkuTier), 'Tiered',\r\n PricingCategory == 'Standard', 'Standard',\r\n PricingCategory == 'Committed', strcat('Committed ', CommitmentDiscountCategory),\r\n PricingCategory == 'Dynamic', 'Spot',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_Project = '',\r\n x_PublisherCategory = iff(PublisherType == 'Marketplace', 'Vendor', 'Cloud Provider'),\r\n x_PublisherId = '',\r\n x_ResellerId = '',\r\n x_ResellerName = '',\r\n x_ResourceGroupName = ResourceGroup,\r\n x_ResourceType,\r\n x_ServiceCode = '',\r\n x_ServiceId = '',\r\n x_ServiceModel = '',\r\n x_ServicePeriodEnd = datetime(null),\r\n x_ServicePeriodStart = datetime(null),\r\n x_SkuDescription = Product,\r\n x_SkuDetails = iff(AdditionalInfo startswith '{', AdditionalInfo, strcat('{', AdditionalInfo, '}')),\r\n x_SkuIsCreditEligible = IsAzureCreditEligible,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterId = MeterId,\r\n x_SkuMeterName = MeterName,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuOfferId = OfferId,\r\n x_SkuOrderId = ProductOrderId,\r\n x_SkuOrderName = ProductOrderName,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPlanName = '',\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTerm = toint(Term),\r\n x_SkuTier = '',\r\n x_SourceName = 'C360',\r\n x_SourceProvider = 'Microsoft',\r\n x_SourceType = 'ActualCost',\r\n x_SourceVersion = 'C360-2025-04',\r\n x_SubproductName = '',\r\n x_UsageType = ''\r\n}\r\n\r\n// Update policy for AmortizedCosts_raw -> Costs_raw table\r\n.alter table Costs_raw policy update\r\n``` \r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"AmortizedCosts_raw\",\r\n \"Query\": \"AmortizedCosts_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| CommitmentDiscountUsage |========================================================================================\r\n// Supported versions:\r\n// - MS EA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-ea\r\n// - MS MCA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-mca\r\n//======================================================================================================================\r\n\r\n// CommitmentDiscountUsage_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: All commitment discount usage transformed to FOCUS 1.0. This includes reservationdeatils_raw. Use CommitmentDiscountUsage_transform_v1_2() instead.', folder='Commitment discounts')\r\nCommitmentDiscountUsage_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n CommitmentDiscountUsage_raw\r\n //\r\n // Change real to decimal\r\n | extend\r\n InstanceFlexibilityRatio = todecimal(InstanceFlexibilityRatio),\r\n ReservedHours = todecimal(ReservedHours),\r\n TotalReservedQuantity = todecimal(TotalReservedQuantity),\r\n UsedHours = todecimal(UsedHours)\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Handle resource columns\r\n | extend ResourceId = tolower(InstanceId)\r\n | extend tmp_ResourceDetails = parse_resourceid(ResourceId)\r\n | extend ResourceName = tostring(tmp_ResourceDetails.ResourceName)\r\n | extend SubAccountId = tostring(tmp_ResourceDetails.SubAccountId)\r\n | extend x_ResourceGroupName = tostring(tmp_ResourceDetails.x_ResourceGroupName)\r\n | extend x_ResourceType = tostring(tmp_ResourceDetails.x_ResourceType)\r\n | lookup kind=leftouter (ResourceTypes | distinct x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n | lookup kind=leftouter (Services | distinct x_ResourceType, ServiceName, ServiceCategory, x_ServiceModel) on x_ResourceType\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n ChargePeriodEnd = UsageDate + 1d,\r\n ChargePeriodStart = UsageDate,\r\n CommitmentDiscountCategory = 'Usage',\r\n CommitmentDiscountId = tolower(strcat('/providers/microsoft.capacity/reservationorders/', ReservationOrderId, '/reservations/', ReservationId)),\r\n CommitmentDiscountType = 'Reservation',\r\n ConsumedQuantity = UsedHours,\r\n ProviderName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n SubAccountId,\r\n x_CommitmentDiscountCommittedCount = TotalReservedQuantity,\r\n x_CommitmentDiscountCommittedAmount = ReservedHours,\r\n // TODO: Is this needed? -- x_CommitmentDiscountKind = Kind,\r\n x_CommitmentDiscountNormalizedGroup = iff(InstanceFlexibilityGroup == 'NA', '', InstanceFlexibilityGroup),\r\n x_CommitmentDiscountNormalizedRatio = InstanceFlexibilityRatio,\r\n x_CommitmentDiscountQuantity = UsedHours * InstanceFlexibilityRatio,\r\n x_IngestionTime = ingestion_time(),\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n // x_RowId = hash_sha256(strcat(\r\n // // DO NOT CHANGE COLUMNS OR COLUMN ORDER\r\n // CommitmentDiscountId,\r\n // ResourceId,\r\n // ChargePeriodStart\r\n // )),\r\n x_ServiceModel,\r\n x_SkuOrderId = ReservationOrderId,\r\n x_SkuSize = iff(SkuName == 'NA', '', SkuName),\r\n x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName)),\r\n x_SourceProvider = coalesce(x_SourceProvider, ProviderName),\r\n x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationDetails', '')),\r\n x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2024-03-01', ''))\r\n}\r\n\r\n// CommitmentDiscountUsage_final_v1_0 table\r\n.create-merge table CommitmentDiscountUsage_final_v1_0 (\r\n ChargePeriodEnd: datetime, // Hubs add-on\r\n ChargePeriodStart: datetime, // MS 2023-03-01\r\n CommitmentDiscountCategory: string, // Hubs add-on\r\n CommitmentDiscountId: string, // MS 2023-03-01\r\n CommitmentDiscountType: string, // Hubs add-on\r\n ConsumedQuantity: decimal, // MS 2023-03-01\r\n ProviderName: string, // Hubs add-on\r\n ResourceId: string, // MS 2023-03-01\r\n ResourceName: string, // Hubs add-on\r\n ResourceType: string, // Hubs add-on\r\n ServiceCategory: string, // Hubs add-on\r\n ServiceName: string, // Hubs add-on\r\n SubAccountId: string, // Hubs add-on\r\n x_CommitmentDiscountCommittedCount: decimal, // MS 2023-03-01\r\n x_CommitmentDiscountCommittedAmount: decimal, // MS 2023-03-01\r\n x_CommitmentDiscountNormalizedGroup: string, // MS 2023-03-01\r\n x_CommitmentDiscountNormalizedRatio: decimal, // MS 2023-03-01\r\n x_CommitmentDiscountQuantity: decimal, // MS 2023-03-01\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_ResourceGroupName: string, // Hubs add-on\r\n x_ResourceType: string, // Hubs add-on\r\n x_ServiceModel: string, // Hubs add-on\r\n x_SkuOrderId: string, // MS 2023-03-01\r\n x_SkuSize: string, // MS 2023-03-01\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string // Hubs add-on\r\n)\r\n\r\n// Update policy for CommitmentDiscountUsage_raw -> CommitmentDiscountUsage_final_v1_0 table\r\n.alter table CommitmentDiscountUsage_final_v1_0 policy update\r\n```\r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"CommitmentDiscountUsage_raw\",\r\n \"Query\": \"CommitmentDiscountUsage_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Recommendations |================================================================================================\r\n// Supported datasets/versions:\r\n// - MS CM EA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-ea\r\n// - MS CM MCA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-mca\r\n//======================================================================================================================\r\n\r\n// Recommendations_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: All recommendations transformed to FOCUS 1.0. Use Recommendations_transform_v1_2() instead.', folder='Recommendations')\r\nRecommendations_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', toint(''),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n Recommendations_raw\r\n //\r\n // Change real to decimal\r\n | extend\r\n CostWithNoReservedInstances = todecimal(CostWithNoReservedInstances),\r\n InstanceFlexibilityRatio = todecimal(InstanceFlexibilityRatio),\r\n NetSavings = todecimal(NetSavings),\r\n RecommendedQuantity = todecimal(RecommendedQuantity),\r\n RecommendedQuantityNormalized = todecimal(RecommendedQuantityNormalized),\r\n TotalCostWithReservedInstances = todecimal(TotalCostWithReservedInstances)\r\n //\r\n | extend x_IngestionTime = ingestion_time()\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Set source columns\r\n | extend x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationRecommendations', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2023-05-01', ''))\r\n //\r\n // Convert JSON cost columns to decimal\r\n | extend CostWithNoReservedInstances = case(isnotempty(CostWithNoReservedInstances), CostWithNoReservedInstances, isnotempty(CostWithNoReservedInstancesJson), todecimal(extract(@'\"value\":([0-9\\.]+)', 1, CostWithNoReservedInstancesJson)), CostWithNoReservedInstances)\r\n | extend NetSavings = case(isnotempty(NetSavings), NetSavings, isnotempty(NetSavingsJson), todecimal(extract(@'\"value\":([0-9\\.]+)', 1, NetSavingsJson)), NetSavings)\r\n | extend TotalCostWithReservedInstances = case(isnotempty(TotalCostWithReservedInstances), TotalCostWithReservedInstances, isnotempty(TotalCostWithReservedInstancesJson), todecimal(extract(@'\"value\":([0-9\\.]+)', 1, TotalCostWithReservedInstancesJson)), TotalCostWithReservedInstances)\r\n //\r\n // Build recommendation details\r\n | lookup kind=leftouter (Regions | summarize RegionName = make_set(RegionName)[0] by Location = RegionId) on Location\r\n | extend x_RecommendationDetails = case(\r\n x_SourceType == 'ReservationRecommendations', bag_pack(\r\n 'CommitmentDiscountNormalizedGroup', InstanceFlexibilityGroup,\r\n 'CommitmentDiscountNormalizedRatio', InstanceFlexibilityRatio,\r\n 'CommitmentDiscountNormalizedSize', NormalizedSize,\r\n 'CommitmentDiscountResourceType', ResourceType,\r\n 'CommitmentDiscountScope', Scope,\r\n 'LookbackPeriodDuration', case(\r\n LookBackPeriod matches regex @'^Last([0-9]+)Days$', replace_regex(LookBackPeriod, @'^Last([0-9]+)Days$', @'P\\1D'),\r\n LookBackPeriod matches regex @'^[0-9]+$', strcat('P', LookBackPeriod, 'D'),\r\n ''\r\n ),\r\n 'LookbackPeriodStart', FirstUsageDate,\r\n 'RecommendedQuantity', RecommendedQuantity,\r\n 'RecommendedQuantityNormalized', RecommendedQuantityNormalized,\r\n 'RegionId', Location,\r\n 'RegionName', RegionName,\r\n 'SkuMeterId', MeterId,\r\n 'SkuPriceDetails', SkuProperties,\r\n 'SkuSize', coalesce(SKU, SkuName),\r\n 'SkuTerm', isoMonths(Term)\r\n ),\r\n dynamic({})\r\n )\r\n //\r\n // Sort columns and apply final transforms\r\n | extend x_RecommendationDate = FirstUsageDate + (toint(extract(@'^P([0-9]+)D$', 1, tostring(x_RecommendationDetails.LookbackPeriodDuration))) * 1d)\r\n | extend x_RecommendationDate = iff(x_RecommendationDate > now(), startofday(now()), x_RecommendationDate)\r\n | project\r\n ProviderName,\r\n SubAccountId = iff(isnotempty(SubscriptionId), strcat('/subscriptions/', SubscriptionId), ''),\r\n x_IngestionTime,\r\n x_EffectiveCostAfter = TotalCostWithReservedInstances,\r\n x_EffectiveCostBefore = CostWithNoReservedInstances,\r\n x_EffectiveCostSavings = NetSavings,\r\n x_RecommendationDate = FirstUsageDate + (toint(extract(@'^P([0-9]+)D$', 1, tostring(x_RecommendationDetails.LookbackPeriodDuration))) * 1d),\r\n x_RecommendationDetails,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n// Recommendations_final_v1_0 table\r\n.create-merge table Recommendations_final_v1_0 (\r\n ProviderName: string,\r\n SubAccountId: string,\r\n x_IngestionTime: datetime,\r\n x_EffectiveCostAfter: decimal,\r\n x_EffectiveCostBefore: decimal,\r\n x_EffectiveCostSavings: decimal,\r\n x_RecommendationDate: datetime,\r\n x_RecommendationDetails: dynamic,\r\n x_SourceName: string,\r\n x_SourceProvider: string,\r\n x_SourceType: string,\r\n x_SourceVersion: string\r\n)\r\n\r\n// Update policy for Recommendations_raw -> Recommendations_final_v1_0 table\r\n.alter table Recommendations_final_v1_0 policy update\r\n```\r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"Recommendations_raw\",\r\n \"Query\": \"Recommendations_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Transactions |===================================================================================================\r\n// Supported versions:\r\n// - MS CM EA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-ea\r\n// - MS CM MCA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-mca\r\n//======================================================================================================================\r\n\r\n// Transactions_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: All transactions transformed to FOCUS 1.0. Use Transactions_transform_v1_2() instead.', folder='Transactions')\r\nTransactions_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', toint(''),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n Transactions_raw\r\n //\r\n // Change real to decimal\r\n | extend\r\n Amount = todecimal(Amount),\r\n MonetaryCommitment = todecimal(MonetaryCommitment),\r\n Overage = todecimal(Overage),\r\n Quantity = todecimal(Quantity)\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Set source columns\r\n | extend x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationTransactions', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2023-05-01', ''))\r\n //\r\n // Handle BillingPeriodStart/End\r\n | extend BillingMonth = tostring(BillingMonth)\r\n | extend BillingPeriodStart = iff(isempty(BillingMonth), datetime(null), todatetime(strcat(substring(BillingMonth, 0, 4), \"-\", substring(BillingMonth, 4, 2), \"-\", substring(BillingMonth, 6, 2))))\r\n | extend BillingPeriodEnd = iff(isempty(BillingMonth), datetime(null), startofmonth(endofmonth(BillingPeriodStart) + 1d))\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n BilledCost = Amount,\r\n BillingAccountId = case(\r\n BillingProfileId startswith '/', BillingProfileId,\r\n isnotempty(CurrentEnrollmentId), strcat('/providers/Microsoft.Billing/billingAccounts/', CurrentEnrollmentId),\r\n isnotempty(BillingProfileId), strcat('/providers/Microsoft.Billing/billingProfiles/', BillingProfileId),\r\n ''\r\n ),\r\n BillingAccountName = coalesce(BillingProfileName, CurrentEnrollmentId),\r\n BillingCurrency = Currency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n ChargeCategory = case(\r\n EventType in ('Cancel', 'Purchase', 'Refund'), 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = case(\r\n EventType == 'Cancel', 'Cancel', // FOCUS does not handle this scenario\r\n EventType == 'Refund', 'Correction',\r\n ''\r\n ),\r\n ChargeDescription = Description,\r\n ChargeFrequency = case(\r\n BillingFrequency == 'OneTime', 'One-Time',\r\n BillingFrequency == 'Recurring', 'Recurring',\r\n BillingFrequency\r\n ),\r\n ChargePeriodStart = EventDate,\r\n PricingQuantity = Quantity,\r\n PricingUnit = 'Reservations',\r\n ProviderName,\r\n RegionId = Region,\r\n RegionName = Region,\r\n SubAccountId = iff(isempty(PurchasingSubscriptionGuid), '', strcat('/subscriptions/', PurchasingSubscriptionGuid)),\r\n SubAccountName = iff(isempty(PurchasingSubscriptionGuid), '', PurchasingSubscriptionName),\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerEmail,\r\n x_CostCenter = CostCenter,\r\n x_InvoiceId = InvoiceId,\r\n x_InvoiceNumber = Invoice,\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = coalesce(InvoiceSectionName, DepartmentName),\r\n x_IngestionTime = ingestion_time(),\r\n x_MonetaryCommitment = MonetaryCommitment,\r\n x_Overage = Overage,\r\n x_PurchasingBillingAccountId = PurchasingEnrollment,\r\n x_SkuOrderId = ReservationOrderId,\r\n x_SkuOrderName = ReservationOrderName,\r\n x_SkuSize = ArmSkuName,\r\n x_SkuTerm = isoMonths(Term),\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_SubscriptionId = PurchasingSubscriptionGuid,\r\n x_TransactionType = EventType\r\n}\r\n\r\n// Transactions_final_v1_0 table\r\n.create-merge table Transactions_final_v1_0 (\r\n BilledCost: decimal, // MS CM EA+MCA 2023-05-01\r\n BillingAccountId: string, // MS CM EA+MCA 2023-05-01\r\n BillingAccountName: string, // MS CM EA+MCA 2023-05-01\r\n BillingCurrency: string, // MS CM EA+MCA 2023-05-01\r\n BillingPeriodEnd: datetime, // MS CM EA+MCA 2023-05-01\r\n BillingPeriodStart: datetime, // MS CM EA+MCA 2023-05-01\r\n ChargeCategory: string, // Hubs add-on\r\n ChargeClass: string, // Hubs add-on\r\n ChargeDescription: string, // MS CM EA+MCA 2023-05-01\r\n ChargeFrequency: string, // MS CM EA+MCA 2023-05-01\r\n ChargePeriodStart: datetime, // MS CM EA+MCA 2023-05-01\r\n PricingQuantity: decimal, // MS CM EA+MCA 2023-05-01\r\n PricingUnit: string, // Hubs add-on\r\n ProviderName: string, // Hubs add-on\r\n RegionId: string, // MS CM EA+MCA 2023-05-01\r\n RegionName: string, // MS CM EA+MCA 2023-05-01\r\n SubAccountId: string, // MS CM EA+MCA 2023-05-01\r\n SubAccountName: string, // MS CM EA+MCA 2023-05-01\r\n x_AccountName: string, // MS CM EA 2023-05-01\r\n x_AccountOwnerId: string, // MS CM EA 2023-05-01\r\n x_CostCenter: string, // MS CM EA 2023-05-01\r\n x_InvoiceId: string, // MS CM MCA 2023-05-01\r\n x_InvoiceNumber: string, // MS CM MCA 2023-05-01\r\n x_InvoiceSectionId: string, // MS CM MCA 2023-05-01\r\n x_InvoiceSectionName: string, // MS CM MCA 2023-05-01\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_MonetaryCommitment: decimal, // MS CM EA 2023-05-01\r\n x_Overage: decimal, // MS CM EA 2023-05-01\r\n x_PurchasingBillingAccountId: string, // MS CM EA 2023-05-01\r\n x_SkuOrderId: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuOrderName: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuSize: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuTerm: int, // MS CM EA+MCA 2023-05-01\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_SubscriptionId: string, // MS CM EA+MCA 2023-05-01\r\n x_TransactionType: string // MS CM EA+MCA 2023-05-01\r\n)\r\n\r\n// Update policy for Transactions_raw -> Transactions_final_v1_0 table\r\n.alter table Transactions_final_v1_0 policy update\r\n```\r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"Transactions_raw\",\r\n \"Query\": \"Transactions_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n", + "CONFIG": "config", + "HUB_DATA_EXPLORER": "hubDataExplorer", + "HUB_DB": "Hub", + "INGESTION": "ingestion", + "INGESTION_DB": "Ingestion", + "INGESTION_ID_SEPARATOR": "__", + "ftkGitTag": "13", + "ftkReleaseUri": "[if(endsWith(variables('finOpsToolkitVersion'), '-dev'), 'https://raw.githubusercontent.com/microsoft/finops-toolkit/refs/heads/dev/src/open-data', format('https://raw.githubusercontent.com/microsoft/finops-toolkit/refs/tags/v{0}/src/open-data', variables('ftkGitTag')))]", + "useFabric": "[not(empty(parameters('fabricQueryUri')))]", + "useAzure": "[and(not(variables('useFabric')), not(empty(parameters('clusterName'))))]", + "dataExplorerPrivateDnsZoneName": "[replace(format('privatelink.{0}.{1}', parameters('app').hub.location, replace(environment().suffixes.storage, 'core', 'kusto')), '..', '.')]", + "ingestionCapacity": { + "Dev(No SLA)_Standard_E2a_v4": 1, + "Dev(No SLA)_Standard_D11_v2": 1, + "Standard_D11_v2": 2, + "Standard_D12_v2": 4, + "Standard_D13_v2": 8, + "Standard_D14_v2": 16, + "Standard_D16d_v5": 16, + "Standard_D32d_v4": 32, + "Standard_D32d_v5": 32, + "Standard_DS13_v2+1TB_PS": 8, + "Standard_DS13_v2+2TB_PS": 8, + "Standard_DS14_v2+3TB_PS": 16, + "Standard_DS14_v2+4TB_PS": 16, + "Standard_E2a_v4": 2, + "Standard_E2ads_v5": 2, + "Standard_E2d_v4": 2, + "Standard_E2d_v5": 2, + "Standard_E4a_v4": 4, + "Standard_E4ads_v5": 4, + "Standard_E4d_v4": 4, + "Standard_E4d_v5": 4, + "Standard_E8a_v4": 8, + "Standard_E8ads_v5": 8, + "Standard_E8as_v4+1TB_PS": 8, + "Standard_E8as_v4+2TB_PS": 8, + "Standard_E8as_v5+1TB_PS": 8, + "Standard_E8as_v5+2TB_PS": 8, + "Standard_E8d_v4": 8, + "Standard_E8d_v5": 8, + "Standard_E8s_v4+1TB_PS": 8, + "Standard_E8s_v4+2TB_PS": 8, + "Standard_E8s_v5+1TB_PS": 8, + "Standard_E8s_v5+2TB_PS": 8, + "Standard_E16a_v4": 16, + "Standard_E16ads_v5": 16, + "Standard_E16as_v4+3TB_PS": 16, + "Standard_E16as_v4+4TB_PS": 16, + "Standard_E16as_v5+3TB_PS": 16, + "Standard_E16as_v5+4TB_PS": 16, + "Standard_E16d_v4": 16, + "Standard_E16d_v5": 16, + "Standard_E16s_v4+3TB_PS": 16, + "Standard_E16s_v4+4TB_PS": 16, + "Standard_E16s_v5+3TB_PS": 16, + "Standard_E16s_v5+4TB_PS": 16, + "Standard_E64i_v3": 64, + "Standard_E80ids_v4": 80, + "Standard_EC8ads_v5": 8, + "Standard_EC8as_v5+1TB_PS": 8, + "Standard_EC8as_v5+2TB_PS": 8, + "Standard_EC16ads_v5": 16, + "Standard_EC16as_v5+3TB_PS": 16, + "Standard_EC16as_v5+4TB_PS": 16, + "Standard_L4s": 4, + "Standard_L8as_v3": 8, + "Standard_L8s": 8, + "Standard_L8s_v2": 8, + "Standard_L8s_v3": 8, + "Standard_L16as_v3": 16, + "Standard_L16s": 16, + "Standard_L16s_v2": 16, + "Standard_L16s_v3": 16, + "Standard_L32as_v3": 32, + "Standard_L32s_v3": 32 + }, + "dataExplorerIngestionCapacity": "[if(variables('useFabric'), parameters('fabricCapacityUnits'), if(not(variables('useAzure')), 1, coalesce(tryGet(variables('ingestionCapacity'), parameters('clusterSku')), 1)))]", + "dataExplorerUri": "[if(variables('useFabric'), parameters('fabricQueryUri'), format('https://{0}.{1}.kusto.windows.net', replace(parameters('clusterName'), '_', '-'), parameters('app').hub.location))]", + "finOpsToolkitVersion": "13.0" + }, + "resources": { + "cluster::adfClusterAdmin": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Kusto/clusters/principalAssignments", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}', replace(parameters('clusterName'), '_', '-'), 'adf-mi-cluster-admin')]", + "properties": { + "principalType": "App", + "principalId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "tenantId": "[reference('dataFactory', '2018-06-01', 'full').identity.tenantId]", + "role": "AllDatabasesAdmin" + }, + "dependsOn": [ + "cluster", + "dataFactory" + ] + }, + "cluster::ingestionDb": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Kusto/clusters/databases", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}', replace(parameters('clusterName'), '_', '-'), variables('INGESTION_DB'))]", + "location": "[parameters('app').hub.location]", + "kind": "ReadWrite", + "dependsOn": [ + "cluster" + ] + }, + "cluster::hubDb": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Kusto/clusters/databases", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}', replace(parameters('clusterName'), '_', '-'), variables('HUB_DB'))]", + "location": "[parameters('app').hub.location]", + "kind": "ReadWrite", + "dependsOn": [ + "cluster" + ] + }, + "dataFactoryVNet::dataExplorerManagedPrivateEndpoint": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', variables('HUB_DATA_EXPLORER'))]", + "properties": { + "name": "[variables('HUB_DATA_EXPLORER')]", + "groupId": "cluster", + "privateLinkResourceId": "[resourceId('Microsoft.Kusto/clusters', replace(parameters('clusterName'), '_', '-'))]", + "fqdns": [ + "[format('https://{0}.{1}.kusto.windows.net', replace(parameters('clusterName'), '_', '-'), parameters('app').hub.location)]" + ] + }, + "dependsOn": [ + "appRegistration", + "cluster" + ] + }, + "dataFactory": { + "existing": true, + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "dependsOn": [ + "appRegistration" + ] + }, + "blobPrivateDnsZone": { + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]", + "dependsOn": [ + "appRegistration" + ] + }, + "queuePrivateDnsZone": { + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.queue.{0}', environment().suffixes.storage)]", + "dependsOn": [ + "appRegistration" + ] + }, + "tablePrivateDnsZone": { + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.table.{0}', environment().suffixes.storage)]", + "dependsOn": [ + "appRegistration" + ] + }, + "storage": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]", + "dependsOn": [ + "appRegistration" + ] + }, + "cluster": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Kusto/clusters", + "apiVersion": "2023-08-15", + "name": "[replace(parameters('clusterName'), '_', '-')]", + "location": "[parameters('app').hub.location]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Kusto/clusters'), createObject()))]", + "sku": { + "name": "[parameters('clusterSku')]", + "tier": "[if(startsWith(parameters('clusterSku'), 'Dev(No SLA)_'), 'Basic', 'Standard')]", + "capacity": "[if(startsWith(parameters('clusterSku'), 'Dev(No SLA)_'), 1, if(equals(parameters('clusterCapacity'), 1), 2, parameters('clusterCapacity')))]" + }, + "identity": { + "type": "SystemAssigned" + }, + "properties": { + "enableStreamingIngest": true, + "enableAutoStop": false, + "publicNetworkAccess": "[if(parameters('app').hub.options.privateRouting, 'Disabled', 'Enabled')]" + }, + "dependsOn": [ + "appRegistration" + ] + }, + "clusterStorageAccess": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "name": "[guid(replace(parameters('clusterName'), '_', '-'), subscription().id, 'Storage Blob Data Contributor')]", + "properties": { + "description": "Give \"Storage Blob Data Contributor\" to the cluster", + "principalId": "[reference('cluster', '2023-08-15', 'full').identity.principalId]", + "principalType": "ServicePrincipal", + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe')]" + }, + "dependsOn": [ + "appRegistration", + "cluster" + ] + }, + "dataExplorerPrivateDnsZone": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[variables('dataExplorerPrivateDnsZoneName')]", + "location": "global", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Network/privateDnsZones'), createObject()))]", + "properties": {} + }, + "dataExplorerPrivateDnsZoneLink": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', variables('dataExplorerPrivateDnsZoneName'), format('{0}-link', replace(variables('dataExplorerPrivateDnsZoneName'), '.', '-')))]", + "location": "global", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Network/privateDnsZones/virtualNetworkLinks'), createObject()))]", + "properties": { + "virtualNetwork": { + "id": "[parameters('app').hub.routing.networkId]" + }, + "registrationEnabled": false + }, + "dependsOn": [ + "dataExplorerPrivateDnsZone" + ] + }, + "dataExplorerEndpoint": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-ep', replace(parameters('clusterName'), '_', '-'))]", + "location": "[parameters('app').hub.location]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Network/privateEndpoints'), createObject()))]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.dataExplorer]" + }, + "privateLinkServiceConnections": [ + { + "name": "dataExplorerLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Kusto/clusters', replace(parameters('clusterName'), '_', '-'))]", + "groupIds": [ + "cluster" + ] + } + } + ] + }, + "dependsOn": [ + "cluster" + ] + }, + "dataExplorerPrivateDnsZoneGroup": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-ep', replace(parameters('clusterName'), '_', '-')), 'dataExplorer-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "privatelink-westus-kusto-net", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', variables('dataExplorerPrivateDnsZoneName'))]" + } + }, + { + "name": "privatelink-blob-core-windows-net", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" + } + }, + { + "name": "privatelink-table-core-windows-net", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.table.{0}', environment().suffixes.storage))]" + } + }, + { + "name": "privatelink-queue-core-windows-net", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.queue.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "appRegistration", + "dataExplorerEndpoint", + "dataExplorerPrivateDnsZone" + ] + }, + "dataFactoryVNet": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'default')]", + "dependsOn": [ + "appRegistration" + ] + }, + "linkedService_dataExplorer": { + "condition": "[or(variables('useAzure'), variables('useFabric'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, variables('HUB_DATA_EXPLORER'))]", + "properties": "[shallowMerge(createArray(createObject('type', 'AzureDataExplorer', 'parameters', createObject('database', createObject('type', 'String', 'defaultValue', variables('INGESTION_DB'))), 'typeProperties', createObject('endpoint', variables('dataExplorerUri'), 'database', '@{linkedService().database}', 'tenant', reference('dataFactory', '2018-06-01', 'full').identity.tenantId, 'servicePrincipalId', reference('dataFactory', '2018-06-01', 'full').identity.principalId)), __bicep.privateRoutingForLinkedServices(parameters('app').hub)))]", + "dependsOn": [ + "appRegistration", + "cluster", + "dataFactory" + ] + }, + "linkedService_ftkRepo": { + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ftkRepo')]", + "properties": "[shallowMerge(createArray(createObject('type', 'HttpServer', 'parameters', createObject('filePath', createObject('type', 'string')), 'typeProperties', createObject('url', '@concat(''https://gitapp.hub.com/microsoft/finops-toolkit/'', linkedService().filePath)', 'enableServerCertificateValidation', true(), 'authenticationType', 'Anonymous')), __bicep.privateRoutingForLinkedServices(parameters('app').hub)))]", + "dependsOn": [ + "appRegistration" + ] + }, + "dataset_dataExplorer": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, variables('HUB_DATA_EXPLORER'))]", + "properties": { + "type": "AzureDataExplorerTable", + "linkedServiceName": { + "parameters": { + "database": "@dataset().database" + }, + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference" + }, + "parameters": { + "database": { + "type": "String", + "defaultValue": "[variables('INGESTION_DB')]" + }, + "table": { + "type": "String" + } + }, + "typeProperties": { + "table": { + "value": "@dataset().table", + "type": "Expression" + } + } + }, + "dependsOn": [ + "appRegistration", + "linkedService_dataExplorer" + ] + }, + "dataset_ftkReleaseFile": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ftkReleaseFile')]", + "properties": { + "linkedServiceName": { + "referenceName": "ftkRepo", + "type": "LinkedServiceReference" + }, + "parameters": { + "fileName": { + "type": "string" + }, + "version": { + "type": "string", + "defaultValue": "[variables('ftkGitTag')]" + } + }, + "annotations": [], + "type": "DelimitedText", + "typeProperties": { + "location": { + "type": "HttpServerLocation", + "relativeUrl": { + "value": "@concat('releases/download/v', dataset().version, '/', dataset().fileName)", + "type": "Expression" + } + }, + "columnDelimiter": ",", + "escapeChar": "\\", + "firstRowAsHeader": true, + "quoteChar": "\"" + }, + "schema": [] + }, + "dependsOn": [ + "appRegistration", + "linkedService_ftkRepo" + ] + }, + "pipeline_InitializeHub": { + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_InitializeHub', variables('CONFIG')))]", + "properties": { + "activities": [ + { + "name": "Get Config", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference" + } + } + }, + { + "name": "Set Version", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Get Config", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "variableName": "version", + "value": { + "value": "@activity('Get Config').output.firstRow.version", + "type": "Expression" + } + } + }, + { + "name": "Set Scopes", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Get Config", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "variableName": "scopes", + "value": { + "value": "@string(activity('Get Config').output.firstRow.scopes)", + "type": "Expression" + } + } + }, + { + "name": "Set Retention", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Get Config", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "variableName": "retention", + "value": { + "value": "@string(activity('Get Config').output.firstRow.retention)", + "type": "Expression" + } + } + }, + { + "name": "Until Capacity Is Available", + "type": "Until", + "dependsOn": [ + { + "activity": "Set Version", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set Scopes", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set Retention", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "expression": { + "value": "@equals(variables('tryAgain'), false)", + "type": "Expression" + }, + "activities": [ + { + "name": "Confirm Ingestion Capacity", + "type": "AzureDataExplorerCommand", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": ".show capacity | where Resource == 'Ingestions' | project Remaining", + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } + }, + { + "name": "If Has Capacity", + "type": "IfCondition", + "dependsOn": [ + { + "activity": "Confirm Ingestion Capacity", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "expression": { + "value": "@or(equals(activity('Confirm Ingestion Capacity').output.count, 0), greater(activity('Confirm Ingestion Capacity').output.value[0].Remaining, 0))", + "type": "Expression" + }, + "ifFalseActivities": [ + { + "name": "Wait for Ingestion", + "type": "Wait", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "waitTimeInSeconds": 15 + } + }, + { + "name": "Try Again", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Wait for Ingestion", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "tryAgain", + "value": true + } + } + ], + "ifTrueActivities": [ + { + "name": "Set ingestion policy in ADX", + "type": "AzureDataExplorerCommand", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": { + "value": "[if(variables('useFabric'), format('.show database {0} policy managed_identity', variables('INGESTION_DB')), format('.alter-merge database {0} policy managed_identity \"[ {{ ''ObjectId'' : ''{1}'', ''AllowedUsages'' : ''NativeIngestion'' }}]\"', variables('INGESTION_DB'), reference('cluster', '2023-08-15', 'full').identity.principalId))]", + "type": "Expression" + }, + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } + }, + { + "name": "Save Hub Settings in ADX", + "type": "AzureDataExplorerCommand", + "dependsOn": [ + { + "activity": "Set ingestion policy in ADX", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": { + "value": "@concat('.append HubSettingsLog <| print version=\"', variables('version'), '\",scopes=dynamic(', variables('scopes'), '),retention=dynamic(', variables('retention'), ') | extend scopes = iff(isnull(scopes[0]), pack_array(scopes), scopes) | mv-apply scopeObj = scopes on (where isnotempty(scopeObj.scope) | summarize scopes = make_set(scopeObj.scope))')", + "type": "Expression" + }, + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } + }, + { + "name": "Update PricingUnits in ADX", + "type": "AzureDataExplorerCommand", + "dependsOn": [ + { + "activity": "Save Hub Settings in ADX", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": "[format('.set-or-replace PricingUnits <| externaldata(x_PricingUnitDescription: string, AccountTypes: string, x_PricingBlockSize: real, PricingUnit: string)[@\"{0}/PricingUnits.csv\"] with (format=\"csv\", ignoreFirstRecord=true) | project-away AccountTypes', variables('ftkReleaseUri'))]", + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } + }, + { + "name": "Update Regions in ADX", + "type": "AzureDataExplorerCommand", + "dependsOn": [ + { + "activity": "Update PricingUnits in ADX", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": "[format('.set-or-replace Regions <| externaldata(ResourceLocation: string, RegionId: string, RegionName: string)[@\"{0}/Regions.csv\"] with (format=\"csv\", ignoreFirstRecord=true)', variables('ftkReleaseUri'))]", + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } + }, + { + "name": "Update ResourceTypes in ADX", + "type": "AzureDataExplorerCommand", + "dependsOn": [ + { + "activity": "Update Regions in ADX", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": "[format('.set-or-replace ResourceTypes <| externaldata(x_ResourceType: string, SingularDisplayName: string, PluralDisplayName: string, LowerSingularDisplayName: string, LowerPluralDisplayName: string, IsPreview: bool, Description: string, IconUri: string, Links: string)[@\"{0}/ResourceTypes.csv\"] with (format=\"csv\", ignoreFirstRecord=true) | project-away Links', variables('ftkReleaseUri'))]", + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } + }, + { + "name": "Update Services in ADX", + "type": "AzureDataExplorerCommand", + "dependsOn": [ + { + "activity": "Update ResourceTypes in ADX", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": "[format('.set-or-replace Services <| externaldata(x_ConsumedService: string, x_ResourceType: string, ServiceName: string, ServiceCategory: string, ServiceSubcategory: string, PublisherName: string, x_PublisherCategory: string, x_Environment: string, x_ServiceModel: string)[@\"{0}/Services.csv\"] with (format=\"csv\", ignoreFirstRecord=true)', variables('ftkReleaseUri'))]", + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } + }, + { + "name": "Ingestion Complete", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Update Services in ADX", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "tryAgain", + "value": false + } + } + ] + } + }, + { + "name": "Abort On Error", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "If Has Capacity", + "dependencyConditions": [ + "Failed" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "tryAgain", + "value": false + } + } + ], + "timeout": "0.02:00:00" + } + }, + { + "name": "Timeout Error", + "type": "Fail", + "dependsOn": [ + { + "activity": "Until Capacity Is Available", + "dependencyConditions": [ + "Failed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "message": "Data Explorer ingestion timed out after 2 hours while waiting for available capacity. Please re-run this pipeline to re-attempt ingestion. If you continue to see this error, please report an issue at https://aka.ms/ftk/ideas.", + "errorCode": "DataExplorerIngestionTimeout" + } + } + ], + "concurrency": 1, + "variables": { + "version": { + "type": "String" + }, + "scopes": { + "type": "String" + }, + "retention": { + "type": "String" + }, + "tryAgain": { + "type": "Boolean", + "defaultValue": true + } + } + }, + "dependsOn": [ + "appRegistration", + "cluster", + "linkedService_dataExplorer" + ], + "metadata": { + "description": "Initializes the hub instance based on the configuration settings." + } + }, + "pipeline_ToDataExplorer": { + "condition": "[or(variables('useAzure'), variables('useFabric'))]", + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_ETL_dataExplorer', variables('INGESTION')))]", + "properties": { + "activities": [ + { + "name": "Read Hub Config", + "description": "Read the hub config to determine how long data should be retained.", + "type": "Lookup", + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": false, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference", + "parameters": { + "fileName": "settings.json", + "folderPath": "[variables('CONFIG')]" + } + } + } + }, + { + "name": "Set Final Retention Months", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Read Hub Config", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "finalRetentionMonths", + "value": { + "value": "@coalesce(activity('Read Hub Config').output.firstRow.retention.final.months, 999)", + "type": "Expression" + } + } + }, + { + "name": "Until Capacity Is Available", + "type": "Until", + "dependsOn": [ + { + "activity": "Set Final Retention Months", + "dependencyConditions": [ + "Completed", + "Skipped" + ] + } + ], + "userProperties": [], + "typeProperties": { + "expression": { + "value": "@equals(variables('tryAgain'), false)", + "type": "Expression" + }, + "activities": [ + { + "name": "Confirm Ingestion Capacity", + "type": "AzureDataExplorerCommand", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": ".show capacity | where Resource == 'Ingestions' | project Remaining", + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference" + } + }, + { + "name": "If Has Capacity", + "type": "IfCondition", + "dependsOn": [ + { + "activity": "Confirm Ingestion Capacity", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "expression": { + "value": "@or(equals(activity('Confirm Ingestion Capacity').output.count, 0), greater(activity('Confirm Ingestion Capacity').output.value[0].Remaining, 0))", + "type": "Expression" + }, + "ifFalseActivities": [ + { + "name": "Wait for Ingestion", + "type": "Wait", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "waitTimeInSeconds": 15 + } + }, + { + "name": "Try Again", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Wait for Ingestion", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "tryAgain", + "value": true + } + } + ], + "ifTrueActivities": [ + { + "name": "Pre-Ingest Cleanup", + "description": "Cost Management exports include all month-to-date data from the previous export run. To ensure data is not double-reported, it must be dropped from the raw table before ingestion completes. Remove previous ingestions into the raw table for the month and any previous runs of the current ingestion month file in any table.", + "type": "AzureDataExplorerCommand", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "typeProperties": { + "command": { + "value": "@concat('.drop extents <| .show extents | where (TableName == \"', pipeline().parameters.table, '\" and Tags !has \"drop-by:', pipeline().parameters.ingestionId, '\" and Tags has \"drop-by:', pipeline().parameters.folderPath, '\") or (Tags has \"drop-by:', pipeline().parameters.ingestionId, '\" and Tags has \"drop-by:', pipeline().parameters.folderPath, '/', pipeline().parameters.originalFileName, '\")')", + "type": "Expression" + }, + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } + }, + { + "name": "Ingest Data", + "type": "AzureDataExplorerCommand", + "dependsOn": [ + { + "activity": "Pre-Ingest Cleanup", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 3, + "retryIntervalInSeconds": 120, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": { + "value": "[format('@concat(''.ingest into table '', pipeline().parameters.table, '' (\"abfss://{0}@{1}.dfs.{2}/'', pipeline().parameters.folderPath, ''/'', pipeline().parameters.fileName, '';{3}\") with (format=\"parquet\", ingestionMappingReference=\"'', pipeline().parameters.table, ''_mapping\", tags=\"[\\\"drop-by:'', pipeline().parameters.ingestionId, ''\\\", \\\"drop-by:'', pipeline().parameters.folderPath, ''/'', pipeline().parameters.originalFileName, ''\\\", \\\"drop-by:ftk-version-{4}\\\"]\"); print Success = assert(iff(toscalar($command_results | project-keep HasErrors) == false, true, false), \"Ingestion Failed\")'')', variables('INGESTION'), parameters('app').storage, environment().suffixes.storage, if(variables('useFabric'), 'impersonate', 'managed_identity=system'), variables('finOpsToolkitVersion'))]", + "type": "Expression" + }, + "commandTimeout": "01:00:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } + }, + { + "name": "Post-Ingest Cleanup", + "description": "Cost Management exports include all month-to-date data from the previous export run. To ensure data is not double-reported, it must be dropped after ingestion completes. Remove the current ingestion month file from raw and any old ingestions for the month from the final table.", + "type": "AzureDataExplorerCommand", + "dependsOn": [ + { + "activity": "Ingest Data", + "dependencyConditions": [ + "Completed" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "typeProperties": { + "command": { + "value": "@concat('.drop extents <| .show extents | extend isOldFinalData = (TableName startswith \"', replace(pipeline().parameters.table, '_raw', '_final_v'), '\" and Tags !has \"drop-by:', pipeline().parameters.ingestionId, '\" and Tags has \"drop-by:', pipeline().parameters.folderPath, '\") | extend isPastFinalRetention = (TableName startswith \"', replace(pipeline().parameters.table, '_raw', '_final_v'), '\" and todatetime(substring(strcat(replace_string(extract(\"drop-by:[A-Za-z]+/(\\\\d{4}/\\\\d{2}(/\\\\d{2})?)\", 1, Tags), \"/\", \"-\"), \"-01\"), 0, 10)) < datetime_add(\"month\", -', if(lessOrEquals(variables('finalRetentionMonths'), 0), 0, variables('finalRetentionMonths')), ', startofmonth(now()))) | where isOldFinalData or isPastFinalRetention')", + "type": "Expression" + }, + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } + }, + { + "name": "Ingestion Complete", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Post-Ingest Cleanup", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "tryAgain", + "value": false + } + }, + { + "name": "Abort On Ingestion Error", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Ingest Data", + "dependencyConditions": [ + "Failed" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "tryAgain", + "value": false + } + }, + { + "name": "Ingestion Failed Error", + "type": "Fail", + "dependsOn": [ + { + "activity": "Abort On Ingestion Error", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Data Explorer ingestion into the ', pipeline().parameters.table, ' table failed. Please fix the error and rerun ingestion for the following folder path: \"', pipeline().parameters.folderPath, '\". File: ', pipeline().parameters.originalFileName, '. Error: ', if(greater(length(activity('Ingest Data').output.errors), 0), activity('Ingest Data').output.errors[0].Message, 'Unknown'), ' (Code: ', if(greater(length(activity('Ingest Data').output.errors), 0), activity('Ingest Data').output.errors[0].Code, 'None'), ')')", + "type": "Expression" + }, + "errorCode": "DataExplorerIngestionFailed" + } + }, + { + "name": "Abort On Pre-Ingest Drop Error", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Pre-Ingest Cleanup", + "dependencyConditions": [ + "Failed" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "tryAgain", + "value": false + } + }, + { + "name": "Pre-Ingest Drop Failed Error", + "type": "Fail", + "dependsOn": [ + { + "activity": "Abort On Pre-Ingest Drop Error", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Data Explorer pre-ingestion cleanup (drop extents from raw table) for the ', pipeline().parameters.table, ' table failed. Ingestion was not completed. Please fix the error and rerun ingestion for the following folder path: \"', pipeline().parameters.folderPath, '\". File: ', pipeline().parameters.originalFileName, '. Error: ', if(greater(length(activity('Pre-Ingest Cleanup').output.errors), 0), activity('Pre-Ingest Cleanup').output.errors[0].Message, 'Unknown'), ' (Code: ', if(greater(length(activity('Pre-Ingest Cleanup').output.errors), 0), activity('Pre-Ingest Cleanup').output.errors[0].Code, 'None'), ')')", + "type": "Expression" + }, + "errorCode": "DataExplorerPreIngestionDropFailed" + } + }, + { + "name": "Abort On Post-Ingest Drop Error", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Post-Ingest Cleanup", + "dependencyConditions": [ + "Failed" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "tryAgain", + "value": false + } + }, + { + "name": "Post-Ingest Drop Failed Error", + "type": "Fail", + "dependsOn": [ + { + "activity": "Abort On Post-Ingest Drop Error", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Data Explorer post-ingestion cleanup (drop extents from final tables) for the ', replace(pipeline().parameters.table, '_raw', '_final_*'), ' table failed. Please fix the error and rerun ingestion for the following folder path: \"', pipeline().parameters.folderPath, '\". File: ', pipeline().parameters.originalFileName, '. Error: ', if(greater(length(activity('Post-Ingest Cleanup').output.errors), 0), activity('Post-Ingest Cleanup').output.errors[0].Message, 'Unknown'), ' (Code: ', if(greater(length(activity('Post-Ingest Cleanup').output.errors), 0), activity('Post-Ingest Cleanup').output.errors[0].Code, 'None'), ')')", + "type": "Expression" + }, + "errorCode": "DataExplorerPostIngestionDropFailed" + } + } + ] + } + } + ], + "timeout": "0.02:00:00" + } + } + ], + "parameters": { + "folderPath": { + "type": "string" + }, + "fileName": { + "type": "string" + }, + "originalFileName": { + "type": "string" + }, + "ingestionId": { + "type": "string" + }, + "table": { + "type": "string" + } + }, + "variables": { + "tryAgain": { + "type": "Boolean", + "defaultValue": true + }, + "logRetentionDays": { + "type": "Integer", + "defaultValue": 0 + }, + "finalRetentionMonths": { + "type": "Integer", + "defaultValue": 999 + } + }, + "annotations": [] + }, + "dependsOn": [ + "appRegistration", + "linkedService_dataExplorer" + ], + "metadata": { + "description": "Ingests parquet data into an Azure Data Explorer cluster." + } + }, + "pipeline_ExecuteIngestionETL": { + "condition": "[or(variables('useAzure'), variables('useFabric'))]", + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_ExecuteETL', variables('INGESTION')))]", + "properties": { + "concurrency": 1, + "activities": [ + { + "name": "Wait", + "description": "Files may not be available immediately after being created.", + "type": "Wait", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "waitTimeInSeconds": 60 + } + }, + { + "name": "Set Container Folder Path", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Wait", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "containerFolderPath", + "value": { + "value": "@join(skip(array(split(pipeline().parameters.folderPath, '/')), 1), '/')", + "type": "Expression" + } + } + }, + { + "name": "Get Existing Parquet Files", + "description": "Get the previously ingested files so we can get file paths.", + "type": "GetMetadata", + "dependsOn": [ + { + "activity": "Set Container Folder Path", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "dataset": { + "referenceName": "ingestion_files", + "type": "DatasetReference", + "parameters": { + "folderPath": "@variables('containerFolderPath')" + } + }, + "fieldList": [ + "childItems" + ], + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "ParquetReadSettings" + } + } + }, + { + "name": "Filter Out Folders", + "description": "Remove any folders or manifest files.", + "type": "Filter", + "dependsOn": [ + { + "activity": "Get Existing Parquet Files", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@if(contains(activity('Get Existing Parquet Files').output, 'childItems'), activity('Get Existing Parquet Files').output.childItems, json('[]'))", + "type": "Expression" + }, + "condition": { + "value": "@and(equals(item().type, 'File'), not(contains(toLower(item().name), 'manifest.json')))", + "type": "Expression" + } + } + }, + { + "name": "Set Ingestion Timestamp", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Wait", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "timestamp", + "value": { + "value": "@utcNow()", + "type": "Expression" + } + } + }, + { + "name": "For Each Old File", + "description": "Loop thru each of the existing files.", + "type": "ForEach", + "dependsOn": [ + { + "activity": "Filter Out Folders", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set Ingestion Timestamp", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "batchCount": "[variables('dataExplorerIngestionCapacity')]", + "items": { + "value": "@activity('Filter Out Folders').output.Value", + "type": "Expression" + }, + "activities": [ + { + "name": "Execute", + "description": "Run the ADX ETL pipeline.", + "type": "ExecutePipeline", + "dependsOn": [], + "policy": { + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "pipeline": { + "referenceName": "[format('{0}_ETL_dataExplorer', variables('INGESTION'))]", + "type": "PipelineReference" + }, + "waitOnCompletion": true, + "parameters": { + "folderPath": { + "value": "@variables('containerFolderPath')", + "type": "Expression" + }, + "fileName": { + "value": "@item().name", + "type": "Expression" + }, + "originalFileName": { + "value": "[format('@last(array(split(item().name, ''{0}'')))', variables('INGESTION_ID_SEPARATOR'))]", + "type": "Expression" + }, + "ingestionId": { + "value": "[format('@concat(first(array(split(item().name, ''{0}''))), ''_'', variables(''timestamp''))', variables('INGESTION_ID_SEPARATOR'))]", + "type": "Expression" + }, + "table": { + "value": "@concat(first(array(split(variables('containerFolderPath'), '/'))), '_raw')", + "type": "Expression" + } + } + } + } + ] + } + }, + { + "name": "If No Files", + "description": "If there are no files found, fail the pipeline.", + "type": "IfCondition", + "dependsOn": [ + { + "activity": "Filter Out Folders", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "expression": { + "value": "@equals(length(activity('Filter Out Folders').output.Value), 0)", + "type": "Expression" + }, + "ifTrueActivities": [ + { + "name": "Files Not Found", + "type": "Fail", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Unable to locate parquet files to ingest from the ', pipeline().parameters.folderPath, ' path. Please confirm the folder path is the full path, including the \"ingestion\" container and not starting with or ending with a slash (\"/\").')", + "type": "Expression" + }, + "errorCode": "IngestionFilesNotFound" + } + } + ] + } + } + ], + "parameters": { + "folderPath": { + "type": "string" + } + }, + "variables": { + "containerFolderPath": { + "type": "string" + }, + "timestamp": { + "type": "string" + } + }, + "annotations": [ + "New ingestion" + ] + }, + "dependsOn": [ + "appRegistration", + "pipeline_ToDataExplorer" + ], + "metadata": { + "description": "Queues the ingestion_ETL_dataExplorer pipeline to account for Data Factory pipeline trigger limits." + } + }, + "appRegistration": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_Register", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "version": { + "value": "[variables('finOpsToolkitVersion')]" + }, + "features": { + "value": [ + "DataFactory", + "Storage" + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "8267413868206701537" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppFeature": { + "type": "string", + "allowedValues": [ + "DataFactory", + "KeyVault", + "Storage" + ], + "metadata": { + "description": "FinOps hub app features.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "version": { + "type": "string", + "metadata": { + "description": "Required. Version number of the FinOps hub app." + } + }, + "features": { + "type": "array", + "items": { + "$ref": "#/definitions/HubAppFeature" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which features the app requires. Allowed values: \"DataFactory\", \"KeyVault\", \"Storage\". Default: [] (none)." + } + }, + "storageRoles": { + "type": "array", + "items": { + "type": "string" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which RBAC roles the Data Factory identity needs on the storage account, if created. This is in addition to Storage Blob Data Contributor for reading and managing content. Default: [] (none)." + } + }, + "telemetryString": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n<#\r\n.SYNOPSIS\r\nManages Azure Data Factory triggers and pipelines during FinOps hub deployment.\r\n\r\n.DESCRIPTION\r\nThis script is called by Bicep deployment scripts to start/stop Data Factory triggers\r\nand optionally run pipelines. It handles two types of triggers:\r\n\r\n1. Schedule triggers - Can be started/stopped directly\r\n2. BlobEventsTriggers - Require Event Grid subscription management before start/stop\r\n\r\nBlobEventsTriggers use Event Grid to listen for storage blob events. Before stopping,\r\nthe Event Grid subscription must be removed (unsubscribed). Before starting, the\r\nsubscription must be added and fully provisioned. This script handles this automatically\r\nby detecting BlobEventsTriggers via the BlobPathBeginsWith property.\r\n\r\nThe script uses retry logic with linear backoff to handle transient API failures and\r\nwait for Event Grid subscription provisioning/deprovisioning to complete.\r\n\r\n.PARAMETER DataFactoryResourceGroup\r\nThe resource group containing the Data Factory.\r\n\r\n.PARAMETER DataFactoryName\r\nThe name of the Data Factory instance.\r\n\r\n.PARAMETER Pipelines\r\nPipe-delimited list of pipeline names to run (e.g., \"pipeline1|pipeline2\").\r\n\r\n.PARAMETER StartTriggers\r\nSwitch to start all stopped triggers (with Event Grid subscription for blob triggers).\r\n\r\n.PARAMETER StopTriggers\r\nSwitch to stop all running triggers (with Event Grid unsubscription for blob triggers).\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StopTriggers\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StartTriggers\r\n#>\r\n\r\nparam(\r\n [string] $DataFactoryResourceGroup,\r\n [string] $DataFactoryName,\r\n [string] $Pipelines = \"\",\r\n [switch] $StartTriggers,\r\n [switch] $StopTriggers\r\n)\r\n\r\n$MAX_RETRIES = 20\r\n$DeploymentScriptOutputs = @{}\r\n\r\nfunction Write-Log($Message)\r\n{\r\n Write-Output \"$(Get-Date -Format 'HH:mm:ss') $Message\"\r\n}\r\n\r\nfunction Invoke-WithRetry([scriptblock]$Action, [string]$Name, [int]$Delay = 5)\r\n{\r\n for ($i = 1; $i -le $MAX_RETRIES; $i++)\r\n {\r\n try { return & $Action }\r\n catch\r\n {\r\n Write-Log \"$Name failed (attempt $i/${MAX_RETRIES}): $($_.Exception.Message)\"\r\n if ($i -eq $MAX_RETRIES) { throw }\r\n Start-Sleep -Seconds ($Delay * $i)\r\n }\r\n }\r\n}\r\n\r\nfunction Set-BlobTriggerSubscription([string]$TriggerName, [switch]$Subscribe)\r\n{\r\n $targetStatus = if ($Subscribe) { 'Enabled' } else { 'Disabled' }\r\n $action = if ($Subscribe) { 'Subscribing' } else { 'Unsubscribing' }\r\n\r\n Write-Log \"$action $TriggerName to events...\"\r\n Invoke-WithRetry -Name \"$action $TriggerName\" -Delay 5 -Action {\r\n if ($Subscribe)\r\n {\r\n Add-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n else\r\n {\r\n Remove-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n\r\n $status = Get-AzDataFactoryV2TriggerSubscriptionStatus `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName\r\n if ($status.Status -ne $targetStatus)\r\n {\r\n throw \"Subscription status is $($status.Status), expected $targetStatus\"\r\n }\r\n }\r\n}\r\n\r\nif ($StartTriggers -or $StopTriggers)\r\n{\r\n $triggers = Invoke-WithRetry -Name \"Get triggers\" -Action {\r\n Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n | Where-Object {\r\n ($StartTriggers -and $_.Properties.RuntimeState -ne \"Started\") `\r\n -or ($StopTriggers -and $_.Properties.RuntimeState -ne \"Stopped\")\r\n }\r\n }\r\n\r\n Write-Log \"Found $($triggers.Count) trigger(s) to $(if ($StartTriggers) { 'start' } else { 'stop' })\"\r\n\r\n $triggers | ForEach-Object {\r\n $triggerName = $_.Name\r\n $isBlobTrigger = $null -ne $_.Properties.BlobPathBeginsWith\r\n\r\n if ($StopTriggers)\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName }\r\n Write-Log \"Stopping trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Stop $triggerName\" -Action {\r\n Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n else\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName -Subscribe }\r\n Write-Log \"Starting trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Start $triggerName\" -Action {\r\n Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n\r\n Invoke-WithRetry -Name \"Wait for $triggerName\" -Action {\r\n $state = (Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName).Properties.RuntimeState\r\n $expected = if ($StartTriggers) { 'Started' } else { 'Stopped' }\r\n if ($state -ne $expected) { throw \"Trigger is $state, expected $expected\" }\r\n }\r\n\r\n Write-Log \"...done\"\r\n $DeploymentScriptOutputs[$triggerName] = $true\r\n }\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($Pipelines))\r\n{\r\n $Pipelines.Split('|') | ForEach-Object {\r\n Write-Log \"Running pipeline $_...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", + "usesDataFactory": "[contains(parameters('features'), 'DataFactory')]", + "usesKeyVault": "[contains(parameters('features'), 'KeyVault')]", + "usesStorage": "[contains(parameters('features'), 'Storage')]", + "telemetryId": "[format('ftk-hubapp-{0}{1}{2}', parameters('app').id, if(empty(parameters('telemetryString')), '', '_'), parameters('telemetryString'))]", + "telemetryProps": { + "mode": "Incremental", + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "[format('FTK: {0}', parameters('app').id)]", + "version": "[parameters('version')]" + } + }, + "resources": [] + } + }, + "autoStartRbacRoles": [ + "673868aa-7521-48a0-acc6-0f60742d39f5" + ], + "factoryStorageRoles": "[union(parameters('storageRoles'), createArray('17d1049b-9a84-46fb-8f53-869881c3d3ab', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe', 'acdd72a7-3385-48ef-bd42-f606fba81ae7'))]", + "storageInfrastructureEncryptionProperties": "[if(not(parameters('app').hub.options.storageInfrastructureEncryption), createObject(), createObject('encryption', createObject('keySource', 'Microsoft.Storage', 'requireInfrastructureEncryption', parameters('app').hub.options.storageInfrastructureEncryption)))]" + }, + "resources": { + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').storage)]", + "properties": { + "name": "[parameters('app').storage]", + "groupId": "dfs", + "privateLinkResourceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "fqdns": [ + "[reference('storageAccount').primaryEndpoints.dfs]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "storageAccount" + ] + }, + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').keyVault)]", + "properties": { + "name": "[parameters('app').keyVault]", + "groupId": "vault", + "privateLinkResourceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "fqdns": [ + "[reference('keyVault').vaultUri]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "keyVault" + ] + }, + "dataFactory::managedVirtualNetwork": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'default')]", + "properties": {}, + "dependsOn": [ + "dataFactory" + ] + }, + "dataFactory::managedIntegrationRuntime": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/integrationRuntimes", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ManagedIntegrationRuntime')]", + "properties": { + "type": "Managed", + "managedVirtualNetwork": { + "referenceName": "default", + "type": "ManagedVirtualNetworkReference" + }, + "typeProperties": { + "computeProperties": { + "location": "[parameters('app').hub.location]", + "dataFlowProperties": { + "computeType": "General", + "coreCount": 8, + "timeToLive": 10, + "cleanup": false, + "customProperties": [] + }, + "copyComputeScaleProperties": { + "dataIntegrationUnit": 16, + "timeToLive": 30 + }, + "pipelineExternalComputeScaleProperties": { + "timeToLive": 30, + "numberOfPipelineNodes": 1, + "numberOfExternalNodes": 1 + } + } + } + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedVirtualNetwork" + ] + }, + "dataFactory::linkedService_keyVault": { + "condition": "[and(variables('usesDataFactory'), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').keyVault)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureKeyVault", + "typeProperties": { + "baseUrl": "[reference(format('Microsoft.KeyVault/vaults/{0}', parameters('app').keyVault), '2023-02-01').vaultUri]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "keyVault" + ] + }, + "dataFactory::linkedService_storageAccount": { + "condition": "[and(variables('usesDataFactory'), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').storage)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureBlobFS", + "typeProperties": { + "url": "[reference(format('Microsoft.Storage/storageAccounts/{0}', parameters('app').storage), '2021-08-01').primaryEndpoints.dfs]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "storageAccount" + ] + }, + "storageAccount::blobService": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]", + "dependsOn": [ + "storageAccount" + ] + }, + "blobEndpoint::blobPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "blobEndpoint" + ] + }, + "dfsEndpoint::dfsPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.dfs.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "dfsEndpoint" + ] + }, + "keyVaultPrivateDnsZone::keyVaultPrivateDnsZoneLink": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), format('{0}-link', replace(format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "properties": { + "virtualNetwork": { + "id": "[parameters('app').hub.routing.networkId]" + }, + "registrationEnabled": false + }, + "dependsOn": [ + "keyVaultPrivateDnsZone" + ] + }, + "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')))]" + } + } + ] + }, + "dependsOn": [ + "keyVaultEndpoint", + "keyVaultPrivateDnsZone" + ] + }, + "appTelemetry": { + "condition": "[parameters('app').hub.options.enableTelemetry]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "[if(lessOrEquals(length(variables('telemetryId')), 64), variables('telemetryId'), substring(variables('telemetryId'), 0, 64))]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Resources/deployments')]", + "properties": "[variables('telemetryProps')]" + }, + "dataFactory": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.DataFactory/factories')]", + "identity": { + "type": "SystemAssigned" + }, + "properties": { + "globalConfigurations": { + "PipelineBillingEnabled": "true" + } + } + }, + "storageRoleAssignments": { + "copy": { + "name": "storageRoleAssignments", + "count": "[length(variables('factoryStorageRoles'))]" + }, + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "name": "[guid(resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage), variables('factoryStorageRoles')[copyIndex()], resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('factoryStorageRoles')[copyIndex()])]", + "principalId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "storageAccount" + ] + }, + "triggerManagerIdentity": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[format('{0}_triggerManager', parameters('app').dataFactory)]", + "location": "[parameters('app').hub.location]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "dependsOn": [ + "dataFactory" + ] + }, + "triggerManagerRoleAssignments": { + "copy": { + "name": "triggerManagerRoleAssignments", + "count": "[length(variables('autoStartRbacRoles'))]" + }, + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]", + "name": "[guid(resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory), variables('autoStartRbacRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', format('{0}_triggerManager', parameters('app').dataFactory)))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('autoStartRbacRoles')[copyIndex()])]", + "principalId": "[reference('triggerManagerIdentity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "triggerManagerIdentity" + ] + }, + "storageAccount": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]", + "location": "[parameters('app').hub.location]", + "sku": { + "name": "[parameters('app').hub.options.storageSku]" + }, + "kind": "BlockBlobStorage", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Storage/storageAccounts')]", + "properties": "[shallowMerge(createArray(variables('storageInfrastructureEncryptionProperties'), createObject('supportsHttpsTrafficOnly', true(), 'allowSharedKeyAccess', true(), 'isHnsEnabled', true(), 'minimumTlsVersion', 'TLS1_2', 'allowBlobPublicAccess', false(), 'publicNetworkAccess', 'Enabled', 'networkAcls', createObject('bypass', 'AzureServices', 'defaultAction', if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')))))]" + }, + "blobPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]" + }, + "blobEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-blob-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "blobLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "blob" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "dfsPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]" + }, + "dfsEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-dfs-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "dfsLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "dfs" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "keyVault": { + "condition": "[variables('usesKeyVault')]", + "type": "Microsoft.KeyVault/vaults", + "apiVersion": "2023-02-01", + "name": "[parameters('app').keyVault]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.KeyVault/vaults')]", + "properties": { + "sku": { + "name": "[parameters('app').hub.options.keyVaultSku]", + "family": "A" + }, + "enabledForDeployment": true, + "enabledForTemplateDeployment": true, + "enabledForDiskEncryption": true, + "enableSoftDelete": true, + "softDeleteRetentionInDays": 90, + "enablePurgeProtection": "[if(parameters('app').hub.options.keyVaultEnablePurgeProtection, true(), null())]", + "enableRbacAuthorization": false, + "createMode": "default", + "tenantId": "[subscription().tenantId]", + "accessPolicies": [ + { + "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "tenantId": "[subscription().tenantId]", + "permissions": { + "secrets": [ + "get" + ] + } + } + ], + "networkAcls": { + "bypass": "AzureServices", + "defaultAction": "[if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')]" + } + }, + "dependsOn": [ + "dataFactory" + ] + }, + "keyVaultPrivateDnsZone": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones')]", + "properties": {} + }, + "keyVaultEndpoint": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-ep', parameters('app').keyVault)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.keyVault]" + }, + "privateLinkServiceConnections": [ + { + "name": "keyVaultLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "groupIds": [ + "vault" + ] + } + } + ] + }, + "dependsOn": [ + "keyVault" + ] + }, + "getKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint", + "getStoragePrivateEndpointConnections", + "keyVault" + ] + }, + "approveKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + }, + "privateEndpointConnections": { + "value": "[reference('getKeyVaultPrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getKeyVaultPrivateEndpointConnections", + "keyVault" + ] + }, + "getStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint", + "stopTriggers", + "storageAccount" + ] + }, + "approveStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + }, + "privateEndpointConnections": { + "value": "[reference('getStoragePrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getStoragePrivateEndpointConnections", + "storageAccount" + ] + }, + "stopTriggers": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.{1}_ADF.StopTriggers', parameters('app').publisher, parameters('app').name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "arguments": { + "value": "[join(createArray(format('-DataFactoryResourceGroup \"{0}\"', resourceGroup().name), format('-DataFactoryName \"{0}\"', parameters('app').dataFactory), '-StopTriggers'), ' ')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + }, + "dependsOn": [ + "appTelemetry", + "dataFactory", + "triggerManagerIdentity", + "triggerManagerRoleAssignments" + ] + } + }, + "outputs": { + "dataFactoryId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Data Factory instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]" + }, + "keyVaultId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Key Vault instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]" + }, + "storageAccountId": { + "type": "string", + "metadata": { + "description": "Resource ID of the storage account instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Principal ID for the managed identity used by Data Factory." + }, + "value": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" + }, + "triggerManagerIdentityName": { + "type": "string", + "metadata": { + "description": "Name of the managed identity used to create and stop ADF triggers." + }, + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + } + } + } + } + }, + "ingestion_OpenDataInternalScripts": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_ADX.IngestionOpenDataInternal", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "clusterName": { + "value": "[replace(parameters('clusterName'), '_', '-')]" + }, + "databaseName": { + "value": "[variables('INGESTION_DB')]" + }, + "scripts": { + "value": { + "OpenDataFunctions_resource_type_1": "[variables('$fxv#0')]", + "OpenDataFunctions_resource_type_2": "[variables('$fxv#1')]", + "OpenDataFunctions_resource_type_3": "[variables('$fxv#2')]", + "OpenDataFunctions_resource_type_4": "[variables('$fxv#3')]", + "OpenDataFunctions_resource_type_5": "[variables('$fxv#4')]" + } + }, + "continueOnErrors": { + "value": "[parameters('continueOnErrors')]" + }, + "forceUpdateTag": { + "value": "[parameters('forceUpdateTag')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "18008676165167943669" + } + }, + "parameters": { + "clusterName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer instance." + } + }, + "databaseName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer database to create or update." + } + }, + "scripts": { + "type": "object", + "metadata": { + "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." + } + }, + "continueOnErrors": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + } + }, + "forceUpdateTag": { + "type": "string", + "defaultValue": "[utcNow()]", + "metadata": { + "description": "Optional. Forces the table to be updated if different from the last time it was deployed." + } + } + }, + "resources": [ + { + "copy": { + "name": "cluster::database::script", + "count": "[length(items(parameters('scripts')))]" + }, + "type": "Microsoft.Kusto/clusters/databases/scripts", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", + "properties": { + "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", + "continueOnErrors": "[parameters('continueOnErrors')]", + "forceUpdateTag": "[parameters('forceUpdateTag')]" + } + } + ] + } + }, + "dependsOn": [ + "cluster", + "cluster::ingestionDb" + ] + }, + "ingestion_InitScripts": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_ADX.IngestionInit", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "clusterName": { + "value": "[replace(parameters('clusterName'), '_', '-')]" + }, + "databaseName": { + "value": "[variables('INGESTION_DB')]" + }, + "scripts": { + "value": { + "openData": "[variables('$fxv#5')]", + "common": "[variables('$fxv#6')]", + "infra": "[variables('$fxv#7')]", + "rawTables": "[replace(variables('$fxv#8'), '$$rawRetentionInDays$$', string(parameters('rawRetentionInDays')))]" + } + }, + "continueOnErrors": { + "value": "[parameters('continueOnErrors')]" + }, + "forceUpdateTag": { + "value": "[parameters('forceUpdateTag')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "18008676165167943669" + } + }, + "parameters": { + "clusterName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer instance." + } + }, + "databaseName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer database to create or update." + } + }, + "scripts": { + "type": "object", + "metadata": { + "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." + } + }, + "continueOnErrors": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + } + }, + "forceUpdateTag": { + "type": "string", + "defaultValue": "[utcNow()]", + "metadata": { + "description": "Optional. Forces the table to be updated if different from the last time it was deployed." + } + } + }, + "resources": [ + { + "copy": { + "name": "cluster::database::script", + "count": "[length(items(parameters('scripts')))]" + }, + "type": "Microsoft.Kusto/clusters/databases/scripts", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", + "properties": { + "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", + "continueOnErrors": "[parameters('continueOnErrors')]", + "forceUpdateTag": "[parameters('forceUpdateTag')]" + } + } + ] + } + }, + "dependsOn": [ + "cluster", + "cluster::ingestionDb", + "ingestion_OpenDataInternalScripts" + ] + }, + "ingestion_VersionedScripts": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_ADX.IngestionVersioned", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "clusterName": { + "value": "[replace(parameters('clusterName'), '_', '-')]" + }, + "databaseName": { + "value": "[variables('INGESTION_DB')]" + }, + "scripts": { + "value": { + "v1_0": "[variables('$fxv#9')]", + "v1_2": "[variables('$fxv#10')]" + } + }, + "continueOnErrors": { + "value": "[parameters('continueOnErrors')]" + }, + "forceUpdateTag": { + "value": "[parameters('forceUpdateTag')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "18008676165167943669" + } + }, + "parameters": { + "clusterName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer instance." + } + }, + "databaseName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer database to create or update." + } + }, + "scripts": { + "type": "object", + "metadata": { + "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." + } + }, + "continueOnErrors": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + } + }, + "forceUpdateTag": { + "type": "string", + "defaultValue": "[utcNow()]", + "metadata": { + "description": "Optional. Forces the table to be updated if different from the last time it was deployed." + } + } + }, + "resources": [ + { + "copy": { + "name": "cluster::database::script", + "count": "[length(items(parameters('scripts')))]" + }, + "type": "Microsoft.Kusto/clusters/databases/scripts", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", + "properties": { + "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", + "continueOnErrors": "[parameters('continueOnErrors')]", + "forceUpdateTag": "[parameters('forceUpdateTag')]" + } + } + ] + } + }, + "dependsOn": [ + "cluster", + "cluster::ingestionDb", + "ingestion_InitScripts" + ] + }, + "hub_InitScripts": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_ADX.HubInit", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "clusterName": { + "value": "[replace(parameters('clusterName'), '_', '-')]" + }, + "databaseName": { + "value": "[variables('HUB_DB')]" + }, + "scripts": { + "value": { + "common": "[variables('$fxv#11')]", + "openData": "[variables('$fxv#12')]" + } + }, + "continueOnErrors": { + "value": "[parameters('continueOnErrors')]" + }, + "forceUpdateTag": { + "value": "[parameters('forceUpdateTag')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "18008676165167943669" + } + }, + "parameters": { + "clusterName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer instance." + } + }, + "databaseName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer database to create or update." + } + }, + "scripts": { + "type": "object", + "metadata": { + "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." + } + }, + "continueOnErrors": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + } + }, + "forceUpdateTag": { + "type": "string", + "defaultValue": "[utcNow()]", + "metadata": { + "description": "Optional. Forces the table to be updated if different from the last time it was deployed." + } + } + }, + "resources": [ + { + "copy": { + "name": "cluster::database::script", + "count": "[length(items(parameters('scripts')))]" + }, + "type": "Microsoft.Kusto/clusters/databases/scripts", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", + "properties": { + "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", + "continueOnErrors": "[parameters('continueOnErrors')]", + "forceUpdateTag": "[parameters('forceUpdateTag')]" + } + } + ] + } + }, + "dependsOn": [ + "cluster", + "cluster::hubDb", + "ingestion_InitScripts" + ] + }, + "hub_VersionedScripts": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_ADX.HubVersioned", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "clusterName": { + "value": "[replace(parameters('clusterName'), '_', '-')]" + }, + "databaseName": { + "value": "[variables('HUB_DB')]" + }, + "scripts": { + "value": { + "v1_0": "[variables('$fxv#13')]", + "v1_2": "[variables('$fxv#14')]" + } + }, + "continueOnErrors": { + "value": "[parameters('continueOnErrors')]" + }, + "forceUpdateTag": { + "value": "[parameters('forceUpdateTag')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "18008676165167943669" + } + }, + "parameters": { + "clusterName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer instance." + } + }, + "databaseName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer database to create or update." + } + }, + "scripts": { + "type": "object", + "metadata": { + "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." + } + }, + "continueOnErrors": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + } + }, + "forceUpdateTag": { + "type": "string", + "defaultValue": "[utcNow()]", + "metadata": { + "description": "Optional. Forces the table to be updated if different from the last time it was deployed." + } + } + }, + "resources": [ + { + "copy": { + "name": "cluster::database::script", + "count": "[length(items(parameters('scripts')))]" + }, + "type": "Microsoft.Kusto/clusters/databases/scripts", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", + "properties": { + "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", + "continueOnErrors": "[parameters('continueOnErrors')]", + "forceUpdateTag": "[parameters('forceUpdateTag')]" + } + } + ] + } + }, + "dependsOn": [ + "cluster", + "cluster::hubDb", + "hub_InitScripts", + "ingestion_VersionedScripts" + ] + }, + "hub_LatestScripts": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_ADX.HubLatest", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "clusterName": { + "value": "[replace(parameters('clusterName'), '_', '-')]" + }, + "databaseName": { + "value": "[variables('HUB_DB')]" + }, + "scripts": { + "value": { + "latest": "[variables('$fxv#15')]" + } + }, + "continueOnErrors": { + "value": "[parameters('continueOnErrors')]" + }, + "forceUpdateTag": { + "value": "[parameters('forceUpdateTag')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "18008676165167943669" + } + }, + "parameters": { + "clusterName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer instance." + } + }, + "databaseName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer database to create or update." + } + }, + "scripts": { + "type": "object", + "metadata": { + "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." + } + }, + "continueOnErrors": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + } + }, + "forceUpdateTag": { + "type": "string", + "defaultValue": "[utcNow()]", + "metadata": { + "description": "Optional. Forces the table to be updated if different from the last time it was deployed." + } + } + }, + "resources": [ + { + "copy": { + "name": "cluster::database::script", + "count": "[length(items(parameters('scripts')))]" + }, + "type": "Microsoft.Kusto/clusters/databases/scripts", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", + "properties": { + "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", + "continueOnErrors": "[parameters('continueOnErrors')]", + "forceUpdateTag": "[parameters('forceUpdateTag')]" + } + } + ] + } + }, + "dependsOn": [ + "cluster", + "cluster::hubDb", + "hub_VersionedScripts" + ] + }, + "getDataExplorerPrivateEndpointConnections": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetDataExplorerPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "dataExplorerName": { + "value": "[replace(parameters('clusterName'), '_', '-')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "17977949049147119573" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "dataExplorerName": { + "type": "string", + "metadata": { + "description": "Required. Name of the ADX cluster." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Kusto/clusters/privateEndpointConnections", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}', parameters('dataExplorerName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Kusto/clusters', parameters('dataExplorerName')), '2023-08-15').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "cluster", + "dataFactoryVNet::dataExplorerManagedPrivateEndpoint" + ] + }, + "approveDataExplorerPrivateEndpointConnections": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveDataExplorerPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "dataExplorerName": { + "value": "[replace(parameters('clusterName'), '_', '-')]" + }, + "privateEndpointConnections": { + "value": "[reference('getDataExplorerPrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "17977949049147119573" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "dataExplorerName": { + "type": "string", + "metadata": { + "description": "Required. Name of the ADX cluster." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Kusto/clusters/privateEndpointConnections", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}', parameters('dataExplorerName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Kusto/clusters', parameters('dataExplorerName')), '2023-08-15').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "cluster", + "getDataExplorerPrivateEndpointConnections" + ] + }, + "trigger_IngestionManifestAdded": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core_IngestionManifestAddedTrigger", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "dataFactoryName": { + "value": "[parameters('app').dataFactory]" + }, + "triggerName": { + "value": "[format('{0}_ManifestAdded', variables('INGESTION'))]" + }, + "pipelineName": { + "value": "[format('{0}_ExecuteETL', variables('INGESTION'))]" + }, + "pipelineParameters": { + "value": { + "folderPath": "@triggerBody().folderPath" + } + }, + "storageAccountName": { + "value": "[parameters('app').storage]" + }, + "storageContainer": { + "value": "[variables('INGESTION')]" + }, + "storagePathEndsWith": { + "value": "manifest.json" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "12667288354191065454" + } + }, + "parameters": { + "dataFactoryName": { + "type": "string", + "metadata": { + "description": "Required. Name of the publisher-specific Data Factory instance." + } + }, + "triggerName": { + "type": "string", + "metadata": { + "description": "Required. Name of the Data Factory trigger to create or update." + } + }, + "storageAccountName": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Azure storage container to monitor for updates and trigger events for." + } + }, + "storageContainer": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Azure storage container to monitor for updates and trigger events for." + } + }, + "storagePathStartsWith": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Beginning of the storage path within the specified storageContainer to monitor for updates and trigger events for." + } + }, + "storagePathEndsWith": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. End of the storage path to monitor for updates and trigger events for." + } + }, + "pipelineName": { + "type": "string", + "metadata": { + "description": "Required. Name of the Data Factory pipeline to execute when the trigger is executed." + } + }, + "pipelineParameters": { + "type": "object", + "metadata": { + "description": "Required. Parameters to pass to the pipeline when the trigger is executed." + } + } + }, + "resources": [ + { + "condition": "[not(empty(parameters('storageAccountName')))]", + "type": "Microsoft.DataFactory/factories/triggers", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('dataFactoryName'), parameters('triggerName'))]", + "properties": { + "annotations": [], + "pipelines": [ + { + "pipelineReference": { + "referenceName": "[parameters('pipelineName')]", + "type": "PipelineReference" + }, + "parameters": "[parameters('pipelineParameters')]" + } + ], + "type": "BlobEventsTrigger", + "typeProperties": { + "blobPathBeginsWith": "[format('/{0}/blobs/{1}', parameters('storageContainer'), parameters('storagePathStartsWith'))]", + "blobPathEndsWith": "[parameters('storagePathEndsWith')]", + "ignoreEmptyBlobs": true, + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]", + "events": [ + "Microsoft.Storage.BlobCreated" + ] + } + } + } + ] + } + }, + "dependsOn": [ + "appRegistration", + "pipeline_ExecuteIngestionETL" + ] + }, + "runInitializationPipeline": { + "condition": "[or(variables('useAzure'), variables('useFabric'))]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_InitializeHub", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "dataFactoryInstances": { + "value": [ + "[parameters('app').dataFactory]" + ] + }, + "identityName": { + "value": "[reference('appRegistration').outputs.triggerManagerIdentityName.value]" + }, + "startPipelines": { + "value": [ + "[format('{0}_InitializeHub', variables('CONFIG'))]" + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "3919636936819908918" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "dataFactoryInstances": { + "type": "array", + "items": { + "type": "string" + }, + "metadata": { + "description": "Required. List of Azure Data Factory instances to start triggers for. Can be up to 1 per publisher." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to use when starting the triggers." + } + }, + "startAllTriggers": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Start all triggers for the Data Factory instances. Default: false." + } + }, + "startPipelines": { + "type": "array", + "items": { + "type": "string" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. List of pipelines to run. Default: [] (no pipelines)." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n<#\r\n.SYNOPSIS\r\nManages Azure Data Factory triggers and pipelines during FinOps hub deployment.\r\n\r\n.DESCRIPTION\r\nThis script is called by Bicep deployment scripts to start/stop Data Factory triggers\r\nand optionally run pipelines. It handles two types of triggers:\r\n\r\n1. Schedule triggers - Can be started/stopped directly\r\n2. BlobEventsTriggers - Require Event Grid subscription management before start/stop\r\n\r\nBlobEventsTriggers use Event Grid to listen for storage blob events. Before stopping,\r\nthe Event Grid subscription must be removed (unsubscribed). Before starting, the\r\nsubscription must be added and fully provisioned. This script handles this automatically\r\nby detecting BlobEventsTriggers via the BlobPathBeginsWith property.\r\n\r\nThe script uses retry logic with linear backoff to handle transient API failures and\r\nwait for Event Grid subscription provisioning/deprovisioning to complete.\r\n\r\n.PARAMETER DataFactoryResourceGroup\r\nThe resource group containing the Data Factory.\r\n\r\n.PARAMETER DataFactoryName\r\nThe name of the Data Factory instance.\r\n\r\n.PARAMETER Pipelines\r\nPipe-delimited list of pipeline names to run (e.g., \"pipeline1|pipeline2\").\r\n\r\n.PARAMETER StartTriggers\r\nSwitch to start all stopped triggers (with Event Grid subscription for blob triggers).\r\n\r\n.PARAMETER StopTriggers\r\nSwitch to stop all running triggers (with Event Grid unsubscription for blob triggers).\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StopTriggers\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StartTriggers\r\n#>\r\n\r\nparam(\r\n [string] $DataFactoryResourceGroup,\r\n [string] $DataFactoryName,\r\n [string] $Pipelines = \"\",\r\n [switch] $StartTriggers,\r\n [switch] $StopTriggers\r\n)\r\n\r\n$MAX_RETRIES = 20\r\n$DeploymentScriptOutputs = @{}\r\n\r\nfunction Write-Log($Message)\r\n{\r\n Write-Output \"$(Get-Date -Format 'HH:mm:ss') $Message\"\r\n}\r\n\r\nfunction Invoke-WithRetry([scriptblock]$Action, [string]$Name, [int]$Delay = 5)\r\n{\r\n for ($i = 1; $i -le $MAX_RETRIES; $i++)\r\n {\r\n try { return & $Action }\r\n catch\r\n {\r\n Write-Log \"$Name failed (attempt $i/${MAX_RETRIES}): $($_.Exception.Message)\"\r\n if ($i -eq $MAX_RETRIES) { throw }\r\n Start-Sleep -Seconds ($Delay * $i)\r\n }\r\n }\r\n}\r\n\r\nfunction Set-BlobTriggerSubscription([string]$TriggerName, [switch]$Subscribe)\r\n{\r\n $targetStatus = if ($Subscribe) { 'Enabled' } else { 'Disabled' }\r\n $action = if ($Subscribe) { 'Subscribing' } else { 'Unsubscribing' }\r\n\r\n Write-Log \"$action $TriggerName to events...\"\r\n Invoke-WithRetry -Name \"$action $TriggerName\" -Delay 5 -Action {\r\n if ($Subscribe)\r\n {\r\n Add-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n else\r\n {\r\n Remove-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n\r\n $status = Get-AzDataFactoryV2TriggerSubscriptionStatus `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName\r\n if ($status.Status -ne $targetStatus)\r\n {\r\n throw \"Subscription status is $($status.Status), expected $targetStatus\"\r\n }\r\n }\r\n}\r\n\r\nif ($StartTriggers -or $StopTriggers)\r\n{\r\n $triggers = Invoke-WithRetry -Name \"Get triggers\" -Action {\r\n Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n | Where-Object {\r\n ($StartTriggers -and $_.Properties.RuntimeState -ne \"Started\") `\r\n -or ($StopTriggers -and $_.Properties.RuntimeState -ne \"Stopped\")\r\n }\r\n }\r\n\r\n Write-Log \"Found $($triggers.Count) trigger(s) to $(if ($StartTriggers) { 'start' } else { 'stop' })\"\r\n\r\n $triggers | ForEach-Object {\r\n $triggerName = $_.Name\r\n $isBlobTrigger = $null -ne $_.Properties.BlobPathBeginsWith\r\n\r\n if ($StopTriggers)\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName }\r\n Write-Log \"Stopping trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Stop $triggerName\" -Action {\r\n Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n else\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName -Subscribe }\r\n Write-Log \"Starting trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Start $triggerName\" -Action {\r\n Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n\r\n Invoke-WithRetry -Name \"Wait for $triggerName\" -Action {\r\n $state = (Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName).Properties.RuntimeState\r\n $expected = if ($StartTriggers) { 'Started' } else { 'Stopped' }\r\n if ($state -ne $expected) { throw \"Trigger is $state, expected $expected\" }\r\n }\r\n\r\n Write-Log \"...done\"\r\n $DeploymentScriptOutputs[$triggerName] = $true\r\n }\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($Pipelines))\r\n{\r\n $Pipelines.Split('|') | ForEach-Object {\r\n Write-Log \"Running pipeline $_...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", + "uniqueInstances": "[union(filter(parameters('dataFactoryInstances'), lambda('adf', not(empty(lambdaVariables('adf'))))), createArray())]" + }, + "resources": { + "initialize": { + "copy": { + "name": "initialize", + "count": "[length(variables('uniqueInstances'))]" + }, + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[if(lessOrEquals(length(format('Microsoft.FinOpsHubs.Init_{0}', variables('uniqueInstances')[copyIndex()])), 64), format('Microsoft.FinOpsHubs.Init_{0}', variables('uniqueInstances')[copyIndex()]), substring(format('Microsoft.FinOpsHubs.Init_{0}', variables('uniqueInstances')[copyIndex()]), 0, 64))]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[parameters('identityName')]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "arguments": { + "value": "[join(filter(createArray(format('-DataFactoryResourceGroup \"{0}\"', resourceGroup().name), format('-DataFactoryName \"{0}\"', variables('uniqueInstances')[copyIndex()]), if(not(empty(parameters('startPipelines'))), format('-Pipelines \"{0}\"', join(parameters('startPipelines'), '|')), ''), if(parameters('startAllTriggers'), '-StartTriggers', '')), lambda('arg', not(empty(lambdaVariables('arg'))))), ' ')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + } + } + } + } + }, + "dependsOn": [ + "appRegistration", + "ingestion_InitScripts", + "ingestion_OpenDataInternalScripts", + "ingestion_VersionedScripts", + "pipeline_InitializeHub" + ] + } + }, + "outputs": { + "clusterId": { + "type": "string", + "metadata": { + "description": "The resource ID of the cluster." + }, + "value": "[if(variables('useFabric'), '', resourceId('Microsoft.Kusto/clusters', replace(parameters('clusterName'), '_', '-')))]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "The ID of the cluster system assigned managed identity." + }, + "value": "[if(variables('useFabric'), '', reference('cluster', '2023-08-15', 'full').identity.principalId)]" + }, + "clusterName": { + "type": "string", + "metadata": { + "description": "The name of the cluster." + }, + "value": "[if(variables('useFabric'), '', replace(parameters('clusterName'), '_', '-'))]" + }, + "clusterUri": { + "type": "string", + "metadata": { + "description": "The URI of the cluster." + }, + "value": "[variables('dataExplorerUri')]" + }, + "ingestionDbName": { + "type": "string", + "metadata": { + "description": "The name of the database for data ingestion." + }, + "value": "[variables('INGESTION_DB')]" + }, + "hubDbName": { + "type": "string", + "metadata": { + "description": "The name of the database for queries." + }, + "value": "[variables('HUB_DB')]" + }, + "clusterIngestionCapacity": { + "type": "int", + "metadata": { + "description": "Max ingestion capacity of the cluster." + }, + "value": "[variables('dataExplorerIngestionCapacity')]" + } + } + } + }, + "dependsOn": [ + "cmExports", + "core", + "deleteOldResources" + ] + }, + "remoteHub": { + "condition": "[not(empty(parameters('remoteHubStorageKey')))]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.RemoteHub", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[__bicep.newApp(variables('hub'), 'Microsoft.FinOpsHubs', 'RemoteHub')]" + }, + "remoteStorageKey": { + "value": "[parameters('remoteHubStorageKey')]" + }, + "remoteHubStorageUri": { + "value": "[parameters('remoteHubStorageUri')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6617373141576044697" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "privateRoutingForLinkedServices": { + "parameters": [ + { + "$ref": "#/definitions/_1.HubProperties", + "name": "hub" + } + ], + "output": { + "type": "object", + "value": "[if(parameters('hub').options.privateRouting, createObject('connectVia', createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference')), createObject())]" + }, + "metadata": { + "description": "Returns an object that represents the properties needed to enable private routing for linked services. Use property expansion (`...value`) to apply to a linkedServices resource.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "remoteStorageKey": { + "type": "securestring", + "metadata": { + "description": "Required. Create and store a key for a remote storage account." + } + }, + "remoteHubStorageUri": { + "type": "string", + "metadata": { + "description": "Required. Remote storage account for ingestion dataset." + } + }, + "ingestionContainerName": { + "type": "string", + "defaultValue": "ingestion", + "metadata": { + "description": "Optional. Name of the ingestion container. Default: ingestion." + } + } + }, + "variables": { + "storageKeySecretName": "[format('{0}-storage-key', toLower(parameters('app').hub.name))]", + "finOpsToolkitVersion": "13.0" + }, + "resources": { + "dataFactory::linkedService_remoteHubStorage": { + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'remoteHubStorage')]", + "properties": "[shallowMerge(createArray(createObject('annotations', createArray(), 'parameters', createObject(), 'type', 'AzureBlobFS', 'typeProperties', createObject('url', parameters('remoteHubStorageUri'), 'accountKey', createObject('type', 'AzureKeyVaultSecret', 'store', createObject('referenceName', parameters('app').keyVault, 'type', 'LinkedServiceReference'), 'secretName', variables('storageKeySecretName')))), __bicep.privateRoutingForLinkedServices(parameters('app').hub)))]", + "dependsOn": [ + "appRegistration" + ] + }, + "dataFactory::dataset_ingestion": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('ingestionContainerName'))]", + "properties": { + "annotations": [], + "parameters": { + "blobPath": { + "type": "String" + } + }, + "type": "Parquet", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().blobPath}", + "type": "Expression" + }, + "fileSystem": "[parameters('ingestionContainerName')]" + } + }, + "linkedServiceName": { + "parameters": {}, + "referenceName": "remoteHubStorage", + "type": "LinkedServiceReference" + } + }, + "dependsOn": [ + "appRegistration", + "dataFactory::linkedService_remoteHubStorage" + ] + }, + "dataFactory::dataset_ingestion_files": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_files', parameters('ingestionContainerName')))]", + "properties": { + "annotations": [], + "parameters": { + "folderPath": { + "type": "String" + } + }, + "type": "Parquet", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileSystem": "[parameters('ingestionContainerName')]", + "folderPath": { + "value": "@dataset().folderPath", + "type": "Expression" + } + } + }, + "linkedServiceName": { + "parameters": {}, + "referenceName": "remoteHubStorage", + "type": "LinkedServiceReference" + } + }, + "dependsOn": [ + "appRegistration", + "dataFactory::linkedService_remoteHubStorage" + ] + }, + "dataFactory::dataset_ingestion_manifest": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ingestion_manifest')]", + "properties": { + "annotations": [], + "parameters": { + "fileName": { + "type": "String", + "defaultValue": "manifest.json" + }, + "folderPath": { + "type": "String", + "defaultValue": "[parameters('ingestionContainerName')]" + } + }, + "type": "Json", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().fileName}", + "type": "Expression" + }, + "folderPath": { + "value": "@{dataset().folderPath}", + "type": "Expression" + } + } + }, + "linkedServiceName": { + "parameters": {}, + "referenceName": "remoteHubStorage", + "type": "LinkedServiceReference" + } + }, + "dependsOn": [ + "appRegistration", + "dataFactory::linkedService_remoteHubStorage" + ] + }, + "keyVault": { + "existing": true, + "type": "Microsoft.KeyVault/vaults", + "apiVersion": "2023-02-01", + "name": "[parameters('app').keyVault]", + "dependsOn": [ + "appRegistration" + ] + }, + "dataFactory": { + "existing": true, + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "dependsOn": [ + "appRegistration" + ] + }, + "appRegistration": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.RemoteHub_Register", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "version": { + "value": "[variables('finOpsToolkitVersion')]" + }, + "features": { + "value": [ + "DataFactory", + "KeyVault", + "Storage" + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "8267413868206701537" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppFeature": { + "type": "string", + "allowedValues": [ + "DataFactory", + "KeyVault", + "Storage" + ], + "metadata": { + "description": "FinOps hub app features.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "version": { + "type": "string", + "metadata": { + "description": "Required. Version number of the FinOps hub app." + } + }, + "features": { + "type": "array", + "items": { + "$ref": "#/definitions/HubAppFeature" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which features the app requires. Allowed values: \"DataFactory\", \"KeyVault\", \"Storage\". Default: [] (none)." + } + }, + "storageRoles": { + "type": "array", + "items": { + "type": "string" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which RBAC roles the Data Factory identity needs on the storage account, if created. This is in addition to Storage Blob Data Contributor for reading and managing content. Default: [] (none)." + } + }, + "telemetryString": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n<#\r\n.SYNOPSIS\r\nManages Azure Data Factory triggers and pipelines during FinOps hub deployment.\r\n\r\n.DESCRIPTION\r\nThis script is called by Bicep deployment scripts to start/stop Data Factory triggers\r\nand optionally run pipelines. It handles two types of triggers:\r\n\r\n1. Schedule triggers - Can be started/stopped directly\r\n2. BlobEventsTriggers - Require Event Grid subscription management before start/stop\r\n\r\nBlobEventsTriggers use Event Grid to listen for storage blob events. Before stopping,\r\nthe Event Grid subscription must be removed (unsubscribed). Before starting, the\r\nsubscription must be added and fully provisioned. This script handles this automatically\r\nby detecting BlobEventsTriggers via the BlobPathBeginsWith property.\r\n\r\nThe script uses retry logic with linear backoff to handle transient API failures and\r\nwait for Event Grid subscription provisioning/deprovisioning to complete.\r\n\r\n.PARAMETER DataFactoryResourceGroup\r\nThe resource group containing the Data Factory.\r\n\r\n.PARAMETER DataFactoryName\r\nThe name of the Data Factory instance.\r\n\r\n.PARAMETER Pipelines\r\nPipe-delimited list of pipeline names to run (e.g., \"pipeline1|pipeline2\").\r\n\r\n.PARAMETER StartTriggers\r\nSwitch to start all stopped triggers (with Event Grid subscription for blob triggers).\r\n\r\n.PARAMETER StopTriggers\r\nSwitch to stop all running triggers (with Event Grid unsubscription for blob triggers).\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StopTriggers\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StartTriggers\r\n#>\r\n\r\nparam(\r\n [string] $DataFactoryResourceGroup,\r\n [string] $DataFactoryName,\r\n [string] $Pipelines = \"\",\r\n [switch] $StartTriggers,\r\n [switch] $StopTriggers\r\n)\r\n\r\n$MAX_RETRIES = 20\r\n$DeploymentScriptOutputs = @{}\r\n\r\nfunction Write-Log($Message)\r\n{\r\n Write-Output \"$(Get-Date -Format 'HH:mm:ss') $Message\"\r\n}\r\n\r\nfunction Invoke-WithRetry([scriptblock]$Action, [string]$Name, [int]$Delay = 5)\r\n{\r\n for ($i = 1; $i -le $MAX_RETRIES; $i++)\r\n {\r\n try { return & $Action }\r\n catch\r\n {\r\n Write-Log \"$Name failed (attempt $i/${MAX_RETRIES}): $($_.Exception.Message)\"\r\n if ($i -eq $MAX_RETRIES) { throw }\r\n Start-Sleep -Seconds ($Delay * $i)\r\n }\r\n }\r\n}\r\n\r\nfunction Set-BlobTriggerSubscription([string]$TriggerName, [switch]$Subscribe)\r\n{\r\n $targetStatus = if ($Subscribe) { 'Enabled' } else { 'Disabled' }\r\n $action = if ($Subscribe) { 'Subscribing' } else { 'Unsubscribing' }\r\n\r\n Write-Log \"$action $TriggerName to events...\"\r\n Invoke-WithRetry -Name \"$action $TriggerName\" -Delay 5 -Action {\r\n if ($Subscribe)\r\n {\r\n Add-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n else\r\n {\r\n Remove-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n\r\n $status = Get-AzDataFactoryV2TriggerSubscriptionStatus `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName\r\n if ($status.Status -ne $targetStatus)\r\n {\r\n throw \"Subscription status is $($status.Status), expected $targetStatus\"\r\n }\r\n }\r\n}\r\n\r\nif ($StartTriggers -or $StopTriggers)\r\n{\r\n $triggers = Invoke-WithRetry -Name \"Get triggers\" -Action {\r\n Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n | Where-Object {\r\n ($StartTriggers -and $_.Properties.RuntimeState -ne \"Started\") `\r\n -or ($StopTriggers -and $_.Properties.RuntimeState -ne \"Stopped\")\r\n }\r\n }\r\n\r\n Write-Log \"Found $($triggers.Count) trigger(s) to $(if ($StartTriggers) { 'start' } else { 'stop' })\"\r\n\r\n $triggers | ForEach-Object {\r\n $triggerName = $_.Name\r\n $isBlobTrigger = $null -ne $_.Properties.BlobPathBeginsWith\r\n\r\n if ($StopTriggers)\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName }\r\n Write-Log \"Stopping trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Stop $triggerName\" -Action {\r\n Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n else\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName -Subscribe }\r\n Write-Log \"Starting trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Start $triggerName\" -Action {\r\n Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n\r\n Invoke-WithRetry -Name \"Wait for $triggerName\" -Action {\r\n $state = (Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName).Properties.RuntimeState\r\n $expected = if ($StartTriggers) { 'Started' } else { 'Stopped' }\r\n if ($state -ne $expected) { throw \"Trigger is $state, expected $expected\" }\r\n }\r\n\r\n Write-Log \"...done\"\r\n $DeploymentScriptOutputs[$triggerName] = $true\r\n }\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($Pipelines))\r\n{\r\n $Pipelines.Split('|') | ForEach-Object {\r\n Write-Log \"Running pipeline $_...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", + "usesDataFactory": "[contains(parameters('features'), 'DataFactory')]", + "usesKeyVault": "[contains(parameters('features'), 'KeyVault')]", + "usesStorage": "[contains(parameters('features'), 'Storage')]", + "telemetryId": "[format('ftk-hubapp-{0}{1}{2}', parameters('app').id, if(empty(parameters('telemetryString')), '', '_'), parameters('telemetryString'))]", + "telemetryProps": { + "mode": "Incremental", + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "[format('FTK: {0}', parameters('app').id)]", + "version": "[parameters('version')]" + } + }, + "resources": [] + } + }, + "autoStartRbacRoles": [ + "673868aa-7521-48a0-acc6-0f60742d39f5" + ], + "factoryStorageRoles": "[union(parameters('storageRoles'), createArray('17d1049b-9a84-46fb-8f53-869881c3d3ab', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe', 'acdd72a7-3385-48ef-bd42-f606fba81ae7'))]", + "storageInfrastructureEncryptionProperties": "[if(not(parameters('app').hub.options.storageInfrastructureEncryption), createObject(), createObject('encryption', createObject('keySource', 'Microsoft.Storage', 'requireInfrastructureEncryption', parameters('app').hub.options.storageInfrastructureEncryption)))]" + }, + "resources": { + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').storage)]", + "properties": { + "name": "[parameters('app').storage]", + "groupId": "dfs", + "privateLinkResourceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "fqdns": [ + "[reference('storageAccount').primaryEndpoints.dfs]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "storageAccount" + ] + }, + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').keyVault)]", + "properties": { + "name": "[parameters('app').keyVault]", + "groupId": "vault", + "privateLinkResourceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "fqdns": [ + "[reference('keyVault').vaultUri]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "keyVault" + ] + }, + "dataFactory::managedVirtualNetwork": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'default')]", + "properties": {}, + "dependsOn": [ + "dataFactory" + ] + }, + "dataFactory::managedIntegrationRuntime": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/integrationRuntimes", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ManagedIntegrationRuntime')]", + "properties": { + "type": "Managed", + "managedVirtualNetwork": { + "referenceName": "default", + "type": "ManagedVirtualNetworkReference" + }, + "typeProperties": { + "computeProperties": { + "location": "[parameters('app').hub.location]", + "dataFlowProperties": { + "computeType": "General", + "coreCount": 8, + "timeToLive": 10, + "cleanup": false, + "customProperties": [] + }, + "copyComputeScaleProperties": { + "dataIntegrationUnit": 16, + "timeToLive": 30 + }, + "pipelineExternalComputeScaleProperties": { + "timeToLive": 30, + "numberOfPipelineNodes": 1, + "numberOfExternalNodes": 1 + } + } + } + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedVirtualNetwork" + ] + }, + "dataFactory::linkedService_keyVault": { + "condition": "[and(variables('usesDataFactory'), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').keyVault)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureKeyVault", + "typeProperties": { + "baseUrl": "[reference(format('Microsoft.KeyVault/vaults/{0}', parameters('app').keyVault), '2023-02-01').vaultUri]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "keyVault" + ] + }, + "dataFactory::linkedService_storageAccount": { + "condition": "[and(variables('usesDataFactory'), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').storage)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureBlobFS", + "typeProperties": { + "url": "[reference(format('Microsoft.Storage/storageAccounts/{0}', parameters('app').storage), '2021-08-01').primaryEndpoints.dfs]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "storageAccount" + ] + }, + "storageAccount::blobService": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]", + "dependsOn": [ + "storageAccount" + ] + }, + "blobEndpoint::blobPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "blobEndpoint" + ] + }, + "dfsEndpoint::dfsPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.dfs.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "dfsEndpoint" + ] + }, + "keyVaultPrivateDnsZone::keyVaultPrivateDnsZoneLink": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), format('{0}-link', replace(format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "properties": { + "virtualNetwork": { + "id": "[parameters('app').hub.routing.networkId]" + }, + "registrationEnabled": false + }, + "dependsOn": [ + "keyVaultPrivateDnsZone" + ] + }, + "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')))]" + } + } + ] + }, + "dependsOn": [ + "keyVaultEndpoint", + "keyVaultPrivateDnsZone" + ] + }, + "appTelemetry": { + "condition": "[parameters('app').hub.options.enableTelemetry]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "[if(lessOrEquals(length(variables('telemetryId')), 64), variables('telemetryId'), substring(variables('telemetryId'), 0, 64))]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Resources/deployments')]", + "properties": "[variables('telemetryProps')]" + }, + "dataFactory": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.DataFactory/factories')]", + "identity": { + "type": "SystemAssigned" + }, + "properties": { + "globalConfigurations": { + "PipelineBillingEnabled": "true" + } + } + }, + "storageRoleAssignments": { + "copy": { + "name": "storageRoleAssignments", + "count": "[length(variables('factoryStorageRoles'))]" + }, + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "name": "[guid(resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage), variables('factoryStorageRoles')[copyIndex()], resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('factoryStorageRoles')[copyIndex()])]", + "principalId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "storageAccount" + ] + }, + "triggerManagerIdentity": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[format('{0}_triggerManager', parameters('app').dataFactory)]", + "location": "[parameters('app').hub.location]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "dependsOn": [ + "dataFactory" + ] + }, + "triggerManagerRoleAssignments": { + "copy": { + "name": "triggerManagerRoleAssignments", + "count": "[length(variables('autoStartRbacRoles'))]" + }, + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]", + "name": "[guid(resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory), variables('autoStartRbacRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', format('{0}_triggerManager', parameters('app').dataFactory)))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('autoStartRbacRoles')[copyIndex()])]", + "principalId": "[reference('triggerManagerIdentity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "triggerManagerIdentity" + ] + }, + "storageAccount": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]", + "location": "[parameters('app').hub.location]", + "sku": { + "name": "[parameters('app').hub.options.storageSku]" + }, + "kind": "BlockBlobStorage", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Storage/storageAccounts')]", + "properties": "[shallowMerge(createArray(variables('storageInfrastructureEncryptionProperties'), createObject('supportsHttpsTrafficOnly', true(), 'allowSharedKeyAccess', true(), 'isHnsEnabled', true(), 'minimumTlsVersion', 'TLS1_2', 'allowBlobPublicAccess', false(), 'publicNetworkAccess', 'Enabled', 'networkAcls', createObject('bypass', 'AzureServices', 'defaultAction', if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')))))]" + }, + "blobPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]" + }, + "blobEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-blob-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "blobLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "blob" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "dfsPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]" + }, + "dfsEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-dfs-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "dfsLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "dfs" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "keyVault": { + "condition": "[variables('usesKeyVault')]", + "type": "Microsoft.KeyVault/vaults", + "apiVersion": "2023-02-01", + "name": "[parameters('app').keyVault]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.KeyVault/vaults')]", + "properties": { + "sku": { + "name": "[parameters('app').hub.options.keyVaultSku]", + "family": "A" + }, + "enabledForDeployment": true, + "enabledForTemplateDeployment": true, + "enabledForDiskEncryption": true, + "enableSoftDelete": true, + "softDeleteRetentionInDays": 90, + "enablePurgeProtection": "[if(parameters('app').hub.options.keyVaultEnablePurgeProtection, true(), null())]", + "enableRbacAuthorization": false, + "createMode": "default", + "tenantId": "[subscription().tenantId]", + "accessPolicies": [ + { + "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "tenantId": "[subscription().tenantId]", + "permissions": { + "secrets": [ + "get" + ] + } + } + ], + "networkAcls": { + "bypass": "AzureServices", + "defaultAction": "[if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')]" + } + }, + "dependsOn": [ + "dataFactory" + ] + }, + "keyVaultPrivateDnsZone": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones')]", + "properties": {} + }, + "keyVaultEndpoint": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-ep', parameters('app').keyVault)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.keyVault]" + }, + "privateLinkServiceConnections": [ + { + "name": "keyVaultLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "groupIds": [ + "vault" + ] + } + } + ] + }, + "dependsOn": [ + "keyVault" + ] + }, + "getKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint", + "getStoragePrivateEndpointConnections", + "keyVault" + ] + }, + "approveKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + }, + "privateEndpointConnections": { + "value": "[reference('getKeyVaultPrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getKeyVaultPrivateEndpointConnections", + "keyVault" + ] + }, + "getStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint", + "stopTriggers", + "storageAccount" + ] + }, + "approveStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + }, + "privateEndpointConnections": { + "value": "[reference('getStoragePrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getStoragePrivateEndpointConnections", + "storageAccount" + ] + }, + "stopTriggers": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.{1}_ADF.StopTriggers', parameters('app').publisher, parameters('app').name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "arguments": { + "value": "[join(createArray(format('-DataFactoryResourceGroup \"{0}\"', resourceGroup().name), format('-DataFactoryName \"{0}\"', parameters('app').dataFactory), '-StopTriggers'), ' ')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + }, + "dependsOn": [ + "appTelemetry", + "dataFactory", + "triggerManagerIdentity", + "triggerManagerRoleAssignments" + ] + } + }, + "outputs": { + "dataFactoryId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Data Factory instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]" + }, + "keyVaultId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Key Vault instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]" + }, + "storageAccountId": { + "type": "string", + "metadata": { + "description": "Resource ID of the storage account instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Principal ID for the managed identity used by Data Factory." + }, + "value": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" + }, + "triggerManagerIdentityName": { + "type": "string", + "metadata": { + "description": "Name of the managed identity used to create and stop ADF triggers." + }, + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + } + } + } + } + }, + "keyVault_secret": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "keyVault_secret", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "vaultName": { + "value": "[parameters('app').keyVault]" + }, + "secretName": { + "value": "[variables('storageKeySecretName')]" + }, + "secretValue": { + "value": "[parameters('remoteStorageKey')]" + }, + "secretExpirationInSeconds": { + "value": 1702648632 + }, + "secretNotBeforeInSeconds": { + "value": 10000 + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "11304809191141616403" + } + }, + "parameters": { + "vaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the publisher-specific Key Vault instance." + } + }, + "secretName": { + "type": "string", + "metadata": { + "description": "Required. Name of the Key Vault secret to create or update." + } + }, + "secretValue": { + "type": "securestring", + "metadata": { + "description": "Required. Value of the Key Vault secret." + } + }, + "secretExpirationInSeconds": { + "type": "int", + "defaultValue": -1, + "metadata": { + "description": "Optional. Value of the Key Vault secret expiration date (exp) property. This is represented as seconds since Jan 1, 1970." + } + }, + "secretNotBeforeInSeconds": { + "type": "int", + "defaultValue": -1, + "metadata": { + "description": "Optional. Value of the Key Vault secret not before date (nbf) property. This is represented as seconds since Jan 1, 1970." + } + } + }, + "resources": [ + { + "type": "Microsoft.KeyVault/vaults/secrets", + "apiVersion": "2023-02-01", + "name": "[format('{0}/{1}', parameters('vaultName'), parameters('secretName'))]", + "properties": { + "attributes": "[union(createObject('enabled', true()), if(lessOrEquals(parameters('secretExpirationInSeconds'), 0), createObject(), createObject('exp', parameters('secretExpirationInSeconds'))), if(lessOrEquals(parameters('secretNotBeforeInSeconds'), 0), createObject(), createObject('nbf', parameters('secretNotBeforeInSeconds'))))]", + "value": "[parameters('secretValue')]" + } + } + ], + "outputs": { + "secretName": { + "type": "string", + "metadata": { + "description": "Name of the Key Vault secret." + }, + "value": "[parameters('secretName')]" + } + } + } + }, + "dependsOn": [ + "appRegistration" + ] + } + }, + "outputs": { + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Name of the Key Vault instance." + }, + "value": "[parameters('app').keyVault]" + } + } + } + }, + "dependsOn": [ + "core" + ] + }, + "deleteOldResources": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.DeleteOldResources", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[reference('core').outputs.app.value]" + }, + "identityName": { + "value": "[reference('core').outputs.triggerManagerIdentityName.value]" + }, + "scriptContent": { + "value": "[variables('$fxv#1')]" + }, + "environmentVariables": { + "value": [ + { + "name": "DataFactorySubscriptionId", + "value": "[subscription().id]" + }, + { + "name": "DataFactoryResourceGroup", + "value": "[resourceGroup().name]" + }, + { + "name": "DataFactoryName", + "value": "[reference('core').outputs.app.value.dataFactory]" + } + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + }, + "dependsOn": [ + "core" + ] + }, + "startTriggers": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.StartTriggers", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[reference('core').outputs.app.value]" + }, + "dataFactoryInstances": { + "value": [ + "[reference('core').outputs.app.value.dataFactory]", + "[reference('cmExports').outputs.app.value.dataFactory]" + ] + }, + "identityName": { + "value": "[reference('core').outputs.triggerManagerIdentityName.value]" + }, + "startAllTriggers": { + "value": true + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "3919636936819908918" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "dataFactoryInstances": { + "type": "array", + "items": { + "type": "string" + }, + "metadata": { + "description": "Required. List of Azure Data Factory instances to start triggers for. Can be up to 1 per publisher." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to use when starting the triggers." + } + }, + "startAllTriggers": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Start all triggers for the Data Factory instances. Default: false." + } + }, + "startPipelines": { + "type": "array", + "items": { + "type": "string" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. List of pipelines to run. Default: [] (no pipelines)." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n<#\r\n.SYNOPSIS\r\nManages Azure Data Factory triggers and pipelines during FinOps hub deployment.\r\n\r\n.DESCRIPTION\r\nThis script is called by Bicep deployment scripts to start/stop Data Factory triggers\r\nand optionally run pipelines. It handles two types of triggers:\r\n\r\n1. Schedule triggers - Can be started/stopped directly\r\n2. BlobEventsTriggers - Require Event Grid subscription management before start/stop\r\n\r\nBlobEventsTriggers use Event Grid to listen for storage blob events. Before stopping,\r\nthe Event Grid subscription must be removed (unsubscribed). Before starting, the\r\nsubscription must be added and fully provisioned. This script handles this automatically\r\nby detecting BlobEventsTriggers via the BlobPathBeginsWith property.\r\n\r\nThe script uses retry logic with linear backoff to handle transient API failures and\r\nwait for Event Grid subscription provisioning/deprovisioning to complete.\r\n\r\n.PARAMETER DataFactoryResourceGroup\r\nThe resource group containing the Data Factory.\r\n\r\n.PARAMETER DataFactoryName\r\nThe name of the Data Factory instance.\r\n\r\n.PARAMETER Pipelines\r\nPipe-delimited list of pipeline names to run (e.g., \"pipeline1|pipeline2\").\r\n\r\n.PARAMETER StartTriggers\r\nSwitch to start all stopped triggers (with Event Grid subscription for blob triggers).\r\n\r\n.PARAMETER StopTriggers\r\nSwitch to stop all running triggers (with Event Grid unsubscription for blob triggers).\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StopTriggers\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StartTriggers\r\n#>\r\n\r\nparam(\r\n [string] $DataFactoryResourceGroup,\r\n [string] $DataFactoryName,\r\n [string] $Pipelines = \"\",\r\n [switch] $StartTriggers,\r\n [switch] $StopTriggers\r\n)\r\n\r\n$MAX_RETRIES = 20\r\n$DeploymentScriptOutputs = @{}\r\n\r\nfunction Write-Log($Message)\r\n{\r\n Write-Output \"$(Get-Date -Format 'HH:mm:ss') $Message\"\r\n}\r\n\r\nfunction Invoke-WithRetry([scriptblock]$Action, [string]$Name, [int]$Delay = 5)\r\n{\r\n for ($i = 1; $i -le $MAX_RETRIES; $i++)\r\n {\r\n try { return & $Action }\r\n catch\r\n {\r\n Write-Log \"$Name failed (attempt $i/${MAX_RETRIES}): $($_.Exception.Message)\"\r\n if ($i -eq $MAX_RETRIES) { throw }\r\n Start-Sleep -Seconds ($Delay * $i)\r\n }\r\n }\r\n}\r\n\r\nfunction Set-BlobTriggerSubscription([string]$TriggerName, [switch]$Subscribe)\r\n{\r\n $targetStatus = if ($Subscribe) { 'Enabled' } else { 'Disabled' }\r\n $action = if ($Subscribe) { 'Subscribing' } else { 'Unsubscribing' }\r\n\r\n Write-Log \"$action $TriggerName to events...\"\r\n Invoke-WithRetry -Name \"$action $TriggerName\" -Delay 5 -Action {\r\n if ($Subscribe)\r\n {\r\n Add-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n else\r\n {\r\n Remove-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n\r\n $status = Get-AzDataFactoryV2TriggerSubscriptionStatus `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName\r\n if ($status.Status -ne $targetStatus)\r\n {\r\n throw \"Subscription status is $($status.Status), expected $targetStatus\"\r\n }\r\n }\r\n}\r\n\r\nif ($StartTriggers -or $StopTriggers)\r\n{\r\n $triggers = Invoke-WithRetry -Name \"Get triggers\" -Action {\r\n Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n | Where-Object {\r\n ($StartTriggers -and $_.Properties.RuntimeState -ne \"Started\") `\r\n -or ($StopTriggers -and $_.Properties.RuntimeState -ne \"Stopped\")\r\n }\r\n }\r\n\r\n Write-Log \"Found $($triggers.Count) trigger(s) to $(if ($StartTriggers) { 'start' } else { 'stop' })\"\r\n\r\n $triggers | ForEach-Object {\r\n $triggerName = $_.Name\r\n $isBlobTrigger = $null -ne $_.Properties.BlobPathBeginsWith\r\n\r\n if ($StopTriggers)\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName }\r\n Write-Log \"Stopping trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Stop $triggerName\" -Action {\r\n Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n else\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName -Subscribe }\r\n Write-Log \"Starting trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Start $triggerName\" -Action {\r\n Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n\r\n Invoke-WithRetry -Name \"Wait for $triggerName\" -Action {\r\n $state = (Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName).Properties.RuntimeState\r\n $expected = if ($StartTriggers) { 'Started' } else { 'Stopped' }\r\n if ($state -ne $expected) { throw \"Trigger is $state, expected $expected\" }\r\n }\r\n\r\n Write-Log \"...done\"\r\n $DeploymentScriptOutputs[$triggerName] = $true\r\n }\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($Pipelines))\r\n{\r\n $Pipelines.Split('|') | ForEach-Object {\r\n Write-Log \"Running pipeline $_...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", + "uniqueInstances": "[union(filter(parameters('dataFactoryInstances'), lambda('adf', not(empty(lambdaVariables('adf'))))), createArray())]" + }, + "resources": { + "initialize": { + "copy": { + "name": "initialize", + "count": "[length(variables('uniqueInstances'))]" + }, + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[if(lessOrEquals(length(format('Microsoft.FinOpsHubs.Init_{0}', variables('uniqueInstances')[copyIndex()])), 64), format('Microsoft.FinOpsHubs.Init_{0}', variables('uniqueInstances')[copyIndex()]), substring(format('Microsoft.FinOpsHubs.Init_{0}', variables('uniqueInstances')[copyIndex()]), 0, 64))]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[parameters('identityName')]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "arguments": { + "value": "[join(filter(createArray(format('-DataFactoryResourceGroup \"{0}\"', resourceGroup().name), format('-DataFactoryName \"{0}\"', variables('uniqueInstances')[copyIndex()]), if(not(empty(parameters('startPipelines'))), format('-Pipelines \"{0}\"', join(parameters('startPipelines'), '|')), ''), if(parameters('startAllTriggers'), '-StartTriggers', '')), lambda('arg', not(empty(lambdaVariables('arg'))))), ' ')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + } + } + } + } + }, + "dependsOn": [ + "analytics", + "cmExports", + "cmManagedExports", + "core", + "deleteOldResources", + "remoteHub" + ] + } + }, + "outputs": { + "name": { + "type": "string", + "metadata": { + "description": "Name of the deployed hub instance." + }, + "value": "[parameters('hubName')]" + }, + "location": { + "type": "string", + "metadata": { + "description": "Azure resource location resources were deployed to." + }, + "value": "[parameters('location')]" + }, + "dataFactoryName": { + "type": "string", + "metadata": { + "description": "Name of the Data Factory." + }, + "value": "[reference('core').outputs.dataFactoryName.value]" + }, + "storageAccountId": { + "type": "string", + "metadata": { + "description": "Resource ID of the storage account created for the hub instance. This must be used when creating the Cost Management export." + }, + "value": "[resourceId('Microsoft.Storage/storageAccounts', reference('core').outputs.storageAccountName.value)]" + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Name of the storage account created for the hub instance. This must be used when connecting FinOps toolkit Power BI reports to your data." + }, + "value": "[reference('core').outputs.storageAccountName.value]" + }, + "storageUrlForPowerBI": { + "type": "string", + "metadata": { + "description": "URL to use when connecting custom Power BI reports to your data." + }, + "value": "[reference('core').outputs.storageUrlForPowerBI.value]" + }, + "clusterId": { + "type": "string", + "metadata": { + "description": "The resource ID of the Data Explorer cluster." + }, + "value": "[if(not(variables('useAzureDataExplorer')), '', reference('analytics').outputs.clusterId.value)]" + }, + "clusterUri": { + "type": "string", + "metadata": { + "description": "The URI of the Data Explorer cluster." + }, + "value": "[if(variables('useFabric'), parameters('fabricQueryUri'), if(not(variables('useAzureDataExplorer')), '', reference('analytics').outputs.clusterUri.value))]" + }, + "ingestionDbName": { + "type": "string", + "metadata": { + "description": "The name of the Data Explorer database used for ingesting data." + }, + "value": "[if(or(variables('useFabric'), variables('useAzureDataExplorer')), reference('analytics').outputs.ingestionDbName.value, '')]" + }, + "hubDbName": { + "type": "string", + "metadata": { + "description": "The name of the Data Explorer database used for querying data." + }, + "value": "[if(or(variables('useFabric'), variables('useAzureDataExplorer')), reference('analytics').outputs.hubDbName.value, '')]" + }, + "managedIdentityId": { + "type": "string", + "metadata": { + "description": "Object ID of the Data Factory managed identity. This will be needed when configuring managed exports." + }, + "value": "[reference('core').outputs.principalId.value]" + }, + "managedIdentityTenantId": { + "type": "string", + "metadata": { + "description": "Azure AD tenant ID. This will be needed when configuring managed exports." + }, + "value": "[tenant().tenantId]" + } + } + } + } + } + ], + "outputs": { + "name": { + "type": "string", + "metadata": { + "description": "Name of the resource group." + }, + "value": "[parameters('hubName')]" + }, + "location": { + "type": "string", + "metadata": { + "description": "Azure resource location resources were deployed to." + }, + "value": "[parameters('location')]" + }, + "dataFactoryName": { + "type": "string", + "metadata": { + "description": "Name of the Data Factory instance." + }, + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.dataFactoryName.value]" + }, + "storageAccountId": { + "type": "string", + "metadata": { + "description": "Resource ID of the deployed storage account." + }, + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.storageAccountId.value]" + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Name of the storage account created for the hub instance. This must be used when connecting FinOps toolkit Power BI reports to your data." + }, + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.storageAccountName.value]" + }, + "storageUrlForPowerBI": { + "type": "string", + "metadata": { + "description": "URL to use when connecting custom Power BI reports to your data." + }, + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.storageUrlForPowerBI.value]" + }, + "clusterId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Data Explorer cluster." + }, + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.clusterId.value]" + }, + "clusterUri": { + "type": "string", + "metadata": { + "description": "URI of the Data Explorer cluster." + }, + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.clusterUri.value]" + }, + "ingestionDbName": { + "type": "string", + "metadata": { + "description": "Name of the Data Explorer database used for ingesting data." + }, + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.ingestionDbName.value]" + }, + "hubDbName": { + "type": "string", + "metadata": { + "description": "Name of the Data Explorer database used for querying data." + }, + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.hubDbName.value]" + }, + "managedIdentityId": { + "type": "string", + "metadata": { + "description": "Object ID of the Data Factory managed identity. This will be needed when configuring managed exports." + }, + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.managedIdentityId.value]" + }, + "managedIdentityTenantId": { + "type": "string", + "metadata": { + "description": "Azure AD tenant ID. This will be needed when configuring managed exports." + }, + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.managedIdentityTenantId.value]" + } + } +} \ No newline at end of file diff --git a/docs/deploy/finops-hub-13.0.ui.json b/docs/deploy/finops-hub-13.0.ui.json new file mode 100644 index 000000000..201aefd47 --- /dev/null +++ b/docs/deploy/finops-hub-13.0.ui.json @@ -0,0 +1,792 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/0.1.2-preview/CreateUIDefinition.MultiVm.json#", + "handler": "Microsoft.Azure.CreateUIDef", + "version": "0.1.2-preview", + "parameters": { + "config": { + "basics": { + "description": "FinOps hubs are a reliable, trustworthy platform for cost analytics, insights, and optimization. Connect your hub to one or more billing accounts and subscriptions and build custom reports in Power BI or other tools. [Learn more](https://aka.ms/finops/hubs)", + "location": { + "label": "Location", + "resourceTypes": [ + "Microsoft.DataFactory/factories", + "Microsoft.KeyVault/vaults", + "Microsoft.Kusto/clusters", + "Microsoft.ManagedIdentity/userAssignedIdentities", + "Microsoft.Network/privateDnsZones", + "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "Microsoft.Network/privateEndpoints", + "Microsoft.Resources/deploymentScripts", + "Microsoft.Storage/storageAccounts" + ] + } + } + }, + "resourceTypes": [ + "Microsoft.DataFactory/factories", + "Microsoft.KeyVault/vaults", + "Microsoft.Kusto/clusters", + "Microsoft.ManagedIdentity/userAssignedIdentities", + "Microsoft.Network/privateDnsZones", + "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "Microsoft.Network/privateEndpoints", + "Microsoft.Resources/deploymentScripts", + "Microsoft.Storage/storageAccounts" + ], + "basics": [ + { + "name": "hubName", + "type": "Microsoft.Common.TextBox", + "label": "Hub name", + "defaultValue": "finops-hub", + "toolTip": "Name of the FinOps hub instance. Used to ensure unique resource names.", + "constraints": { + "required": true, + "regex": "^[a-zA-Z0-9][a-zA-Z0-9\\-]{0,61}[a-zA-Z0-9]$", + "validationMessage": "Name must be between 3 and 63 characters long and can contain only lowercase letters, numbers, and hyphens. The first and last characters in the name must be alphanumeric." + }, + "visible": true + }, + { + "name": "dataExplorer", + "type": "Microsoft.Common.Section", + "label": "Azure Data Explorer (optional)", + "elements": [ + { + "name": "dataExplorerIntro", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "Azure Data Explorer is a fast, scalable service for advanced, big data analytics. Data Explorer is optional but recommended when monitoring more than $100K in spend. FinOps hubs with Data Explorer starts at $120/mo for a single node cluster plus $10/mo per million in monitored spend (<0.02% of annual spend)." + } + }, + { + "name": "dataExplorerName", + "type": "Microsoft.Common.TextBox", + "label": "Cluster name", + "toolTip": "Name of the Azure Data Explorer cluster, if desired. If not specified, Data Explorer will not be deployed.", + "constraints": { + "required": false, + "regex": "^[a-zA-Z0-9][a-zA-Z0-9\\-]{0,20}[a-z0-9]$", + "validationMessage": "Name must be between 4 and 22 characters long and can contain only lowercase letters, numbers, and hyphens. The first and last characters in the name must be alphanumeric." + }, + "visible": true + } + ], + "visible": true + }, + { + "name": "fabric", + "type": "Microsoft.Common.Section", + "label": "Microsoft Fabric (optional)", + "elements": [ + { + "name": "fabricIntro", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "Microsoft Fabric is a unified data platform for data ingestion, processing, and analytics. FinOps hubs with Fabric starts at $300/mo for an F2 SKU plus $10/mo per million in monitored spend." + } + }, + { + "name": "fabricQueryUri", + "type": "Microsoft.Common.TextBox", + "label": "Fabric eventhouse query URI", + "toolTip": "Query URI for the Microsoft Fabric eventhouse, if applicable.", + "constraints": { + "required": false, + "regex": "^https://.*", + "validationMessage": "Query URI must be a full Microsoft Fabric eventhouse URL." + }, + "visible": true + }, + { + "name": "fabricCapacity", + "type": "Microsoft.Common.DropDown", + "label": "Fabric capacity", + "defaultValue": "F2", + "toolTip": "Fabric capacity configured for this workspace. This must be set at deployment time to tune data ingestion performance. If you update the capacity, redeploy this template.", + "constraints": { + "required": false, + "allowedValues": [ + { "label": "Trial", "value": "1" }, + { "label": "F2", "value": "2" }, + { "label": "F4", "value": "4" }, + { "label": "F8", "value": "8" }, + { "label": "F16", "value": "16" }, + { "label": "F32", "value": "32" }, + { "label": "F64", "value": "64" }, + { "label": "F128", "value": "128" }, + { "label": "F256", "value": "256" }, + { "label": "F512", "value": "512" }, + { "label": "F1024", "value": "1024" }, + { "label": "F2048", "value": "2048" } + ] + }, + "visible": true + } + ], + "visible": true + } + ], + "steps": [ + { + "name": "pricing", + "label": "Pricing", + "elements": [ + { + "name": "pricingIntro", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "Estimated costs below are based on list prices for the default settings. Refer to the Azure pricing calculator for the most accurate pricing based on your discounts. Learn more @ https://aka.ms/finops/hubs/calculator" + } + }, + { + "name": "storage", + "type": "Microsoft.Common.Section", + "label": "Data Factory + storage", + "elements": [ + { + "name": "storagePricingIntro", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "FinOps hubs utilize Azure Data Factory for data processing and Azure Data Lake Storage Gen2 for staging during data ingestion." + } + }, + { + "name": "storageEstimate", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "Estimated cost: Starts at ~$5/mo per million in monitored spend*." + } + }, + { + "name": "storageSku", + "type": "Microsoft.Common.DropDown", + "label": "Storage redundancy", + "defaultValue": "Locally-redundant (LRS) - Lowest cost", + "toolTip": "The data in your storage account is always replicated to ensure durability and high availability. Choose a replication strategy that matches your durability requirements. [Learn more](https://go.microsoft.com/fwlink/?linkid=2163103)", + "constraints": { + "required": false, + "allowedValues": [ + { + "label": "Locally-redundant (LRS) - Lowest cost", + "value": "Premium_LRS" + }, + { + "label": "Zone-redundant (ZRS) - High availability", + "value": "Premium_ZRS" + } + ] + }, + "visible": true + } + ], + "visible": true + }, + { + "name": "dataExplorer", + "type": "Microsoft.Common.Section", + "label": "Data Explorer (optional)", + "elements": [ + { + "name": "dataExplorerPricingIntro", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "Azure Data Explorer is optional. If a cluster name is not specified on the Basics tab, this setting is ignored." + } + }, + { + "name": "dataExplorerEstimate", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "Added cost: Starts at $120/mo plus another ~$5/mo per million in monitored spend* (<0.02% of annual spend)." + } + }, + { + "name": "dataExplorerSku", + "type": "Microsoft.Common.DropDown", + "label": "Data Explorer SKU", + "defaultValue": "Dev/test: D11_v2 (no SLA) - Extra small, 78GB cache (~$121/mo)", + "toolTip": "Select an Azure Data Explorer SKU based on your needs. Consider how many accounts and datasets that are required for your needs. We recommend starting small and scaling up based on performance. [Learn more](https://learn.microsoft.com/azure/data-explorer/manage-cluster-choose-sku)", + "constraints": { + "required": false, + "allowedValues": [ + { + "label": "Dev/test: E2a_v4 (no SLA) - Extra small, 24GB cache (~$110/mo)", + "value": "Dev(No SLA)_Standard_E2a_v4" + }, + { + "label": "Dev/test: D11_v2 (no SLA) - Extra small, 78GB cache (~$121/mo)", + "value": "Dev(No SLA)_Standard_D11_v2" + }, + { + "label": "D11_v2 - Extra small, 78GB cache (~$245/mo)", + "value": "Standard_D11_v2" + }, + { + "label": "D12_v2 - Small, 162GB cache", + "value": "Standard_D12_v2" + }, + { + "label": "D13_v2 - Medium, 335GB cache", + "value": "Standard_D13_v2" + }, + { + "label": "D14_v2 - Large, 680GB cache", + "value": "Standard_D14_v2" + }, + { + "label": "D16d_v5 - Large, 485GB cache", + "value": "Standard_D16d_v5" + }, + { + "label": "D32d_v4 - Extra large, 976GB cache", + "value": "Standard_D32d_v4" + }, + { + "label": "D32d_v5 - Extra large, 976GB cache", + "value": "Standard_D32d_v5" + }, + { + "label": "DS13_v2+1TB_PS - Medium", + "value": "Standard_DS13_v2+1TB_PS" + }, + { + "label": "DS13_v2+2TB_PS - Medium", + "value": "Standard_DS13_v2+2TB_PS" + }, + { + "label": "DS14_v2+3TB_PS - Large", + "value": "Standard_DS14_v2+3TB_PS" + }, + { + "label": "DS14_v2+4TB_PS - Large", + "value": "Standard_DS14_v2+4TB_PS" + }, + { + "label": "E2a_v4 - Extra small, 30GB cache (~$220/mo)", + "value": "Standard_E2a_v4" + }, + { + "label": "E2ads_v5 - Extra small", + "value": "Standard_E2ads_v5" + }, + { + "label": "E2d_v4 - Extra small", + "value": "Standard_E2d_v4" + }, + { + "label": "E2d_v5 - Extra small", + "value": "Standard_E2d_v5" + }, + { + "label": "E4a_v4 - Small", + "value": "Standard_E4a_v4" + }, + { + "label": "E4ads_v5 - Small", + "value": "Standard_E4ads_v5" + }, + { + "label": "E4d_v4 - Small", + "value": "Standard_E4d_v4" + }, + { + "label": "E4d_v5 - Small", + "value": "Standard_E4d_v5" + }, + { + "label": "E8a_v4 - Medium", + "value": "Standard_E8a_v4" + }, + { + "label": "E8ads_v5 - Medium", + "value": "Standard_E8ads_v5" + }, + { + "label": "E8as_v4+1TB_PS - Medium", + "value": "Standard_E8as_v4+1TB_PS" + }, + { + "label": "E8as_v4+2TB_PS - Medium", + "value": "Standard_E8as_v4+2TB_PS" + }, + { + "label": "E8as_v5+1TB_PS - Medium", + "value": "Standard_E8as_v5+1TB_PS" + }, + { + "label": "E8as_v5+2TB_PS - Medium", + "value": "Standard_E8as_v5+2TB_PS" + }, + { + "label": "E8d_v4 - Medium", + "value": "Standard_E8d_v4" + }, + { + "label": "E8d_v5 - Medium", + "value": "Standard_E8d_v5" + }, + { + "label": "E8s_v4+1TB_PS - Medium", + "value": "Standard_E8s_v4+1TB_PS" + }, + { + "label": "E8s_v4+2TB_PS - Medium", + "value": "Standard_E8s_v4+2TB_PS" + }, + { + "label": "E8s_v5+1TB_PS - Medium", + "value": "Standard_E8s_v5+1TB_PS" + }, + { + "label": "E8s_v5+2TB_PS - Medium", + "value": "Standard_E8s_v5+2TB_PS" + }, + { + "label": "E16a_v4 - Large", + "value": "Standard_E16a_v4" + }, + { + "label": "E16ads_v5 - Large", + "value": "Standard_E16ads_v5" + }, + { + "label": "E16as_v4+3TB_PS - Large", + "value": "Standard_E16as_v4+3TB_PS" + }, + { + "label": "E16as_v4+4TB_PS - Large", + "value": "Standard_E16as_v4+4TB_PS" + }, + { + "label": "E16as_v5+3TB_PS - Large", + "value": "Standard_E16as_v5+3TB_PS" + }, + { + "label": "E16as_v5+4TB_PS - Large", + "value": "Standard_E16as_v5+4TB_PS" + }, + { + "label": "E16d_v4 - Large", + "value": "Standard_E16d_v4" + }, + { + "label": "E16d_v5 - Large", + "value": "Standard_E16d_v5" + }, + { + "label": "E16s_v4+3TB_PS - Large", + "value": "Standard_E16s_v4+3TB_PS" + }, + { + "label": "E16s_v4+4TB_PS - Large", + "value": "Standard_E16s_v4+4TB_PS" + }, + { + "label": "E16s_v5+3TB_PS - Large", + "value": "Standard_E16s_v5+3TB_PS" + }, + { + "label": "E16s_v5+4TB_PS - Large", + "value": "Standard_E16s_v5+4TB_PS" + }, + { + "label": "E64i_v3 - Extra large", + "value": "Standard_E64i_v3" + }, + { + "label": "E80ids_v4 - Extra large", + "value": "Standard_E80ids_v4" + }, + { + "label": "EC8ads_v5 - Medium", + "value": "Standard_EC8ads_v5" + }, + { + "label": "EC8as_v5+1TB_PS - Medium", + "value": "Standard_EC8as_v5+1TB_PS" + }, + { + "label": "EC8as_v5+2TB_PS - Medium", + "value": "Standard_EC8as_v5+2TB_PS" + }, + { + "label": "EC16ads_v5 - Large", + "value": "Standard_EC16ads_v5" + }, + { + "label": "EC16as_v5+3TB_PS - Large", + "value": "Standard_EC16as_v5+3TB_PS" + }, + { + "label": "EC16as_v5+4TB_PS - Large", + "value": "Standard_EC16as_v5+4TB_PS" + }, + { + "label": "L4s - Small", + "value": "Standard_L4s" + }, + { + "label": "L8as_v3 - Medium", + "value": "Standard_L8as_v3" + }, + { + "label": "L8s - Medium", + "value": "Standard_L8s" + }, + { + "label": "L8s_v2 - Medium", + "value": "Standard_L8s_v2" + }, + { + "label": "L8s_v3 - Medium", + "value": "Standard_L8s_v3" + }, + { + "label": "L16as_v3 - Large", + "value": "Standard_L16as_v3" + }, + { + "label": "L16s - Large", + "value": "Standard_L16s" + }, + { + "label": "L16s_v2 - Large", + "value": "Standard_L16s_v2" + }, + { + "label": "L16s_v3 - Large", + "value": "Standard_L16s_v3" + }, + { + "label": "L32as_v3 - Extra large", + "value": "Standard_L32as_v3" + }, + { + "label": "L32s_v3 - Extra large", + "value": "Standard_L32s_v3" + } + ] + }, + "visible": true + } + ], + "visible": true + }, + { + "name": "spacer", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "" + } + }, + { + "name": "monitoredSpendNote", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "* Monitored spend refers to how much cost data is stored based on desired retention. For instance, $1 million per month in spend for 13 months is $13 million in monitored spend. The basic deployment with Data Explorer would be $250/mo - $120/mo for a single node cluster plus $10 times 13 for Data Factory and storage costs." + } + } + ] + }, + { + "name": "retention", + "label": "Data retention", + "elements": [ + { + "name": "retentionIntro", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "Data retention settings indicate how long to keep data available for reporting or additional processing. Retained data contributes to monitored spend and the total storage costs for the hub instance." + } + }, + { + "name": "storage", + "type": "Microsoft.Common.Section", + "label": "Storage", + "visible": false, + "elements": [ + { + "name": "storageIntro", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "Storage retention is not implemented yet. You can set the retention period but files will not be automatically removed." + } + }, + { + "name": "msexportsDays", + "type": "Microsoft.Common.TextBox", + "label": "Export retention (days)", + "defaultValue": "0", + "toolTip": "Indicates how many days Cost Management exports are kept in the msexports container. If 0, exported data will be deleted after ingestion. Manifest files are always retained for troubleshooting. If set to any higher value, exported data will be retained indefinitely. This retention setting has not been implemented yet.", + "constraints": { + "required": false, + "regex": "^[0-9]{1,4}$", + "validationMessage": "Number of days must be between 0 and 9999." + }, + "visible": true + }, + { + "name": "ingestionMonths", + "type": "Microsoft.Common.TextBox", + "label": "Ingestion retention (months)", + "defaultValue": "13", + "toolTip": "Indicates how many months exported data is kept in the ingestion container. This retention setting has not been implemented yet.", + "constraints": { + "required": false, + "regex": "^[0-9]{1,2}$", + "validationMessage": "Number of months must be between 0 and 99." + }, + "visible": true + } + ] + }, + { + "name": "dataExplorer", + "type": "Microsoft.Common.Section", + "label": "Data Explorer (optional)", + "visible": true, + "elements": [ + { + "name": "dataExplorerIntro", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "Azure Data Explorer is optional. If a cluster name is not specified on the Basics tab, this setting is ignored." + } + }, + { + "name": "rawDays", + "type": "Microsoft.Common.TextBox", + "label": "Raw data retention (days)", + "defaultValue": "0", + "toolTip": "Indicates how many days ingested data should be retained in 'raw' tables. If 0, ingested data is deleted immediately after ingested into 'final' (normalized) tables.", + "constraints": { + "required": false, + "regex": "^[0-9]{1,4}$", + "validationMessage": "Number of days must be between 0 and 9999." + }, + "visible": true + }, + { + "name": "finalMonths", + "type": "Microsoft.Common.TextBox", + "label": "Normalized data retention (months)", + "defaultValue": "13", + "toolTip": "Indicates how many closed (complete) months data should be retained in 'final' (normalized) tables. If 0, ingested data is deleted on the first of the next month. This retention setting is only enforced when new data is ingested. If data ingestion stops, historical data will be retained indefinitely.", + "constraints": { + "required": false, + "regex": "^[0-9]{1,2}$", + "validationMessage": "Number of months must be between 0 and 99." + }, + "visible": true + } + ] + } + ] + }, + { + "name": "advanced", + "label": "Advanced", + "elements": [ + { + "name": "storage", + "type": "Microsoft.Common.Section", + "label": "Infrastructure encryption", + "elements": [ + { + "name": "infraEncryptionIntro", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "Infrastructure encryption can be enabled for the entire storage account. To enable infrastructure encryption for this storage account, you must check this box at the time that you deploy this template. Learn more. To learn about pricing for encryption scopes, see Blob Storage pricing." + } + }, + { + "name": "enableInfrastructureEncryption", + "type": "Microsoft.Common.CheckBox", + "label": "Enable Infrastructure Encryption", + "toolTip": "Infrastructure encryption is recommended for scenarios where doubly encrypting data is necessary for compliance requirements. For most other scenarios, Azure Storage encryption provides a sufficiently powerful encryption algorithm, and there is unlikely to be a benefit to using infrastructure encryption unless for compliance requirements." + } + ], + "visible": true + }, + { + "name": "managedExports", + "type": "Microsoft.Common.Section", + "label": "Managed exports", + "elements": [ + { + "name": "managedExportsIntro", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "Managed exports allow your FinOps hub instance to create and run Cost Management exports on your behalf using a schedule you define. This feature is not supported for Microsoft Customer Agreement (MCA) billing profiles." + } + }, + { + "name": "enableManagedExports", + "type": "Microsoft.Common.CheckBox", + "label": "Enable managed exports", + "toolTip": "Creating exports in Cost Management requires the User Access Administrator role. You must have access to grant User Access Administrator to your FinOps hub to enable managed exports." + } + ], + "visible": true + }, + { + "name": "networking", + "type": "Microsoft.Common.Section", + "label": "Networking", + "elements": [ + { + "name": "infraEncryptionIntro", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "For enhanced security, consider blocking external access to your FinOps hub instance to enable secure, private connectivity to your cloud resources by eliminating exposure to the public internet. Private endpoints can incur significant additional cost. Please understand the cost impact before selecting this option." + } + }, + { + "name": "enablePublicAccess", + "type": "Microsoft.Common.DropDown", + "label": "Access", + "toolTip": "Indicate whether FinOps hubs resources should be accessible by the internet or not. If set to private, all resources will be hidden behind a virtual network private endpoint. This will incur additional costs for data crossing virtual network boundaries and for virtual network peering, if configured.", + "defaultValue": "Public", + "multiLine": true, + "constraints": { + "allowedValues": [ + { + "label": "Public", + "description": "Allow external access to storage and Data Explorer.", + "value": true + }, + { + "label": "Private", + "description": "Block all external access. This will incur additional charges.", + "value": false + } + ] + } + }, + { + "name": "virtualNetworkAddressPrefix", + "type": "Microsoft.Common.TextBox", + "label": "Address prefix", + "toolTip": "Address space for the workload. Minimum /26 subnet size is required for the workload.", + "defaultValue": "10.20.30.0/26", + "constraints": { + "validations": [ + { + "regex": "^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\/([8-9]|1[0-9]|2[0-6])$", + "message": "Address prefix must be a valid IPv4 address in the format 'XXX.XXX.XXX.XXX/YY' where YY is between 8 and 26 (minimum /26 subnet size required)." + } + ] + } + } + ], + "visible": true + }, + { + "name": "remoteHub", + "type": "Microsoft.Common.Section", + "label": "Remote hub configuration", + "elements": [ + { + "name": "remoteHubIntro", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "Configure this hub to send data to a remote FinOps hub in another tenant or subscription. This enables cross-tenant cost management scenarios where a central tenant collects cost data from multiple tenants. Leave these fields empty if this is not a remote hub setup." + } + }, + { + "name": "remoteHubStorageUri", + "type": "Microsoft.Common.TextBox", + "label": "Remote hub storage URI", + "toolTip": "Data Lake storage endpoint from the remote hub storage account. Copy from the storage account Settings > Endpoints > Data Lake storage. Example: https://myremotehub.dfs.core.windows.net/", + "constraints": { + "required": false, + "regex": "^$|^https://.*\\.dfs\\.core\\.windows\\.net/?$", + "validationMessage": "Must be a valid Data Lake storage endpoint URL in the format: https://storageaccount.dfs.core.windows.net/" + }, + "visible": true + }, + { + "name": "remoteHubStorageKey", + "type": "Microsoft.Common.PasswordBox", + "label": { + "password": "Remote hub storage key" + }, + "toolTip": "Storage account access key for the remote hub. Copy from the remote hub storage account Security + networking > Access keys > key1/2 > Key.", + "constraints": { + "required": false, + "regex": "^$|^[A-Za-z0-9+/]{86}==$", + "validationMessage": "Must be a valid storage account access key (base64 encoded, ending with ==)" + }, + "options": { + "hideConfirmation": true + }, + "visible": true + } + ], + "visible": true + } + ] + }, + { + "name": "tags", + "label": "Tags", + "elements": [ + { + "name": "tagsByResource", + "label": "Tags", + "toolTip": "Tags to apply to resources.", + "type": "Microsoft.Common.TagsByResource", + "resources": [ + "Microsoft.DataFactory/factories", + "Microsoft.KeyVault/vaults", + "Microsoft.Kusto/clusters", + "Microsoft.ManagedIdentity/userAssignedIdentities", + "Microsoft.Network/privateDnsZones", + "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "Microsoft.Network/privateEndpoints", + "Microsoft.Resources/deploymentScripts", + "Microsoft.Storage/storageAccounts" + ] + } + ] + } + ], + "outputs": { + "hubName": "[basics('hubName')]", + "location": "[location()]", + "storageSku": "[steps('pricing').storage.storageSku]", + "dataExplorerName": "[basics('dataExplorer').dataExplorerName]", + "fabricQueryUri": "[basics('fabric').fabricQueryUri]", + "fabricCapacity": "[basics('fabric').fabricCapacity]", + "enableInfrastructureEncryption": "[steps('advanced').storage.enableInfrastructureEncryption]", + "enableManagedExports": "[steps('advanced').managedExports.enableManagedExports]", + "enablePublicAccess": "[steps('advanced').networking.enablePublicAccess]", + "virtualNetworkAddressPrefix": "[steps('advanced').networking.virtualNetworkAddressPrefix]", + "dataExplorerSku": "[steps('pricing').dataExplorer.dataExplorerSku]", + "exportRetentionInDays": "[steps('retention').storage.msexportsDays]", + "ingestionRetentionInMonths": "[steps('retention').storage.ingestionMonths]", + "dataExplorerRawRetentionInDays": "[steps('retention').dataExplorer.rawDays]", + "dataExplorerFinalRetentionInMonths": "[steps('retention').dataExplorer.finalMonths]", + "remoteHubStorageUri": "[steps('advanced').remoteHub.remoteHubStorageUri]", + "remoteHubStorageKey": "[steps('advanced').remoteHub.remoteHubStorageKey]", + "tagsByResource": "[steps('tags').tagsByResource]" + } + } +} diff --git a/docs/deploy/finops-hub-latest.json b/docs/deploy/finops-hub-latest.json index 4dd0a7f94..56ae568cd 100644 --- a/docs/deploy/finops-hub-latest.json +++ b/docs/deploy/finops-hub-latest.json @@ -4,8 +4,8 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "1039455044893847676" + "version": "0.40.2.10011", + "templateHash": "15508602991110868852" } }, "parameters": { @@ -40,6 +40,13 @@ "description": "Optional. Enable infrastructure encryption on the storage account. Default = false." } }, + "enablePurgeProtection": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Enable purge protection for the Key Vault. Default: false." + } + }, "remoteHubStorageUri": { "type": "string", "defaultValue": "", @@ -233,7 +240,7 @@ "resources": [ { "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", + "apiVersion": "2025-04-01", "name": "hub", "properties": { "expressionEvaluationOptions": { @@ -253,6 +260,9 @@ "enableInfrastructureEncryption": { "value": "[parameters('enableInfrastructureEncryption')]" }, + "enablePurgeProtection": { + "value": "[parameters('enablePurgeProtection')]" + }, "enableManagedExports": { "value": "[parameters('enableManagedExports')]" }, @@ -312,43 +322,29 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "7621563314037220493" + "version": "0.40.2.10011", + "templateHash": "2881649287387479903" } }, "definitions": { "_1.HubAppProperties": { "type": "object", "properties": { + "id": { + "type": "string" + }, "name": { "type": "string" }, - "displayName": { + "publisher": { + "type": "string" + }, + "suffix": { "type": "string" }, "tags": { "type": "object" }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, "dataFactory": { "type": "string" }, @@ -357,25 +353,24 @@ }, "storage": { "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" } }, "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", "dataFactory": "Name of the Data Factory instance for this publisher.", "keyVault": "Name of the KeyVault instance for this publisher.", "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", "description": "FinOps hub app configuration settings.", "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "fx/hub-types.bicep" } } }, @@ -409,6 +404,9 @@ "keyVaultSku": { "type": "string" }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, "networkAddressPrefix": { "type": "string" }, @@ -448,6 +446,7 @@ "options": { "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", "privateRouting": "Indicates whether private network routing is enabled.", "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", @@ -461,7 +460,7 @@ "apps": {}, "description": "FinOps hub instance properties.", "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "fx/hub-types.bicep" } } }, @@ -497,6 +496,9 @@ "subnets": { "type": "object", "properties": { + "dataExplorer": { + "type": "string" + }, "dataFactory": { "type": "string" }, @@ -523,6 +525,7 @@ "table": "Resource ID and name for the table storage DNS zone." }, "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", "dataFactory": "Resource ID of the subnet for Data Factory instances.", "keyVault": "Resource ID of the subnet for Key Vault instances.", "scripts": "Resource ID of the subnet for deployment script storage.", @@ -530,7 +533,7 @@ }, "description": "FinOps hub private network routing properties.", "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "fx/hub-types.bicep" } } }, @@ -549,7 +552,7 @@ "name": "Resource name.", "description": "Resource ID and name.", "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "fx/hub-types.bicep" } } } @@ -571,7 +574,7 @@ }, "metadata": { "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "fx/hub-types.bicep" } } }, @@ -595,7 +598,7 @@ }, "metadata": { "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "fx/hub-types.bicep" } } }, @@ -607,54 +610,38 @@ }, { "type": "string", - "name": "publisherName" - }, - { - "type": "string", - "name": "publisherDisplayName" - }, - { - "type": "string", - "name": "publisherSuffix" - }, - { - "type": "object", - "name": "publisherTags" + "name": "id" }, { "type": "string", - "name": "appName" + "name": "name" }, { "type": "string", - "name": "appDisplayName" + "name": "publisher" }, { "type": "string", - "name": "version" + "name": "suffix" } ], "output": { "$ref": "#/definitions/_1.HubAppProperties", "value": { - "name": "[parameters('appName')]", - "displayName": "[parameters('appDisplayName')]", - "tags": "[union(parameters('hub').tags, parameters('publisherTags'), createObject('ftk-hubapp', parameters('appName'), 'ftk-hubapp-version', parameters('version')))]", - "publisher": { - "name": "[parameters('publisherName')]", - "displayName": "[parameters('publisherDisplayName')]", - "suffix": "[parameters('publisherSuffix')]", - "tags": "[union(parameters('hub').tags, parameters('publisherTags'))]" - }, + "id": "[parameters('id')]", + "name": "[parameters('name')]", + "publisher": "[parameters('publisher')]", + "suffix": "[parameters('suffix')]", + "tags": "[union(parameters('hub').tags, createObject('ftk-hubapp-publisher', parameters('publisher')))]", "hub": "[parameters('hub')]", - "dataFactory": "[replace(format('{0}-{1}', take(format('{0}-engine', replace(parameters('hub').name, '_', '-')), sub(sub(63, length(parameters('publisherSuffix'))), 1)), parameters('publisherSuffix')), '--', '-')]", - "keyVault": "[replace(format('{0}-{1}', take(format('{0}-vault', replace(parameters('hub').name, '_', '-')), sub(sub(24, length(parameters('publisherSuffix'))), 1)), parameters('publisherSuffix')), '--', '-')]", - "storage": "[format('{0}{1}', take(_1.safeStorageName(parameters('hub').name), sub(24, length(parameters('publisherSuffix')))), parameters('publisherSuffix'))]" + "dataFactory": "[replace(format('{0}-{1}', take(format('{0}-engine', replace(parameters('hub').name, '_', '-')), sub(sub(63, length(parameters('suffix'))), 1)), parameters('suffix')), '--', '-')]", + "keyVault": "[replace(format('{0}-{1}', take(format('{0}-vault', replace(parameters('hub').name, '_', '-')), sub(sub(24, length(parameters('suffix'))), 1)), parameters('suffix')), '--', '-')]", + "storage": "[format('{0}{1}', take(_1.safeStorageName(parameters('hub').name), sub(24, length(parameters('suffix')))), parameters('suffix'))]" } }, "metadata": { "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "fx/hub-types.bicep" } } }, @@ -692,6 +679,10 @@ "type": "string", "name": "keyVaultSku" }, + { + "type": "bool", + "name": "keyVaultEnablePurgeProtection" + }, { "type": "bool", "name": "enableInfrastructureEncryption" @@ -725,6 +716,7 @@ "options": { "enableTelemetry": "[coalesce(parameters('isTelemetryEnabled'), true())]", "keyVaultSku": "[parameters('keyVaultSku')]", + "keyVaultEnablePurgeProtection": "[parameters('keyVaultEnablePurgeProtection')]", "networkAddressPrefix": "[parameters('networkAddressPrefix')]", "privateRouting": "[not(parameters('enablePublicAccess'))]", "publisherIsolation": false, @@ -742,6 +734,7 @@ "table": "[if(parameters('enablePublicAccess'), createObject('id', '', 'name', ''), _1.dnsZoneIdName('table'))]" }, "subnets": { + "dataExplorer": "[if(parameters('enablePublicAccess'), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('networkName'), 'dataExplorer-subnet'))]", "dataFactory": "[if(parameters('enablePublicAccess'), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('networkName'), 'private-endpoint-subnet'))]", "keyVault": "[if(parameters('enablePublicAccess'), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('networkName'), 'private-endpoint-subnet'))]", "scripts": "[if(parameters('enablePublicAccess'), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('networkName'), 'script-subnet'))]", @@ -755,7 +748,7 @@ }, "metadata": { "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "fx/hub-types.bicep" } } }, @@ -772,7 +765,7 @@ }, "metadata": { "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "fx/hub-types.bicep" } } } @@ -799,7 +792,7 @@ "metadata": { "description": "Returns a tags dictionary that includes tags for the FinOps hub instance.", "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "fx/hub-types.bicep" } } }, @@ -811,33 +804,21 @@ }, { "type": "string", - "name": "publisherDisplayName" - }, - { - "type": "string", - "name": "publisherName" - }, - { - "type": "string", - "name": "appPartialName" - }, - { - "type": "string", - "name": "appDisplayName" + "name": "publisher" }, { "type": "string", - "name": "version" + "name": "app" } ], "output": { "$ref": "#/definitions/_1.HubAppProperties", - "value": "[_1.newAppInternal(parameters('hub'), parameters('publisherName'), parameters('publisherDisplayName'), if(or(not(parameters('hub').options.publisherIsolation), equals(parameters('publisherName'), 'Microsoft.FinOpsHubs')), parameters('hub').core.suffix, uniqueString(parameters('publisherName'))), createObject('ftk-hubapp-publisher', parameters('publisherName')), format('{0}.{1}', parameters('publisherName'), parameters('appPartialName')), parameters('appDisplayName'), parameters('version'))]" + "value": "[_1.newAppInternal(parameters('hub'), format('{0}.{1}', parameters('publisher'), parameters('app')), parameters('app'), parameters('publisher'), if(or(not(parameters('hub').options.publisherIsolation), equals(parameters('publisher'), 'Microsoft.FinOpsHubs')), parameters('hub').core.suffix, uniqueString(parameters('publisher'))))]" }, "metadata": { "description": "Creates a new FinOps hub app configuration object.", "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "fx/hub-types.bicep" } } }, @@ -867,6 +848,10 @@ "type": "string", "name": "keyVaultSku" }, + { + "type": "bool", + "name": "keyVaultEnablePurgeProtection" + }, { "type": "bool", "name": "enableInfrastructureEncryption" @@ -886,12 +871,12 @@ ], "output": { "$ref": "#/definitions/_1.HubProperties", - "value": "[_1.newHubInternal(format('{0}/providers/Microsoft.Cloud/hubs/{1}', resourceGroup().id, parameters('name')), parameters('name'), uniqueString(parameters('name'), resourceGroup().id), parameters('location'), parameters('tags'), parameters('tagsByResource'), parameters('storageSku'), parameters('keyVaultSku'), parameters('enableInfrastructureEncryption'), parameters('enablePublicAccess'), format('{0}-vnet-{1}', _1.safeStorageName(parameters('name')), parameters('location')), parameters('networkAddressPrefix'), coalesce(parameters('isTelemetryEnabled'), true()))]" + "value": "[_1.newHubInternal(format('{0}/providers/Microsoft.Cloud/hubs/{1}', resourceGroup().id, parameters('name')), parameters('name'), uniqueString(parameters('name'), resourceGroup().id), parameters('location'), parameters('tags'), parameters('tagsByResource'), parameters('storageSku'), parameters('keyVaultSku'), parameters('keyVaultEnablePurgeProtection'), parameters('enableInfrastructureEncryption'), parameters('enablePublicAccess'), format('{0}-vnet-{1}', _1.safeStorageName(parameters('name')), parameters('location')), parameters('networkAddressPrefix'), coalesce(parameters('isTelemetryEnabled'), true()))]" }, "metadata": { "description": "Creates a new FinOps hub configuration object.", "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "fx/hub-types.bicep" } } } @@ -941,6 +926,13 @@ "description": "Optional. SKU to use for the KeyVault instance, if enabled. Allowed values: \"standard\", \"premium\". Default: \"premium\"." } }, + "enablePurgeProtection": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Enable purge protection for the Key Vault. Default: false." + } + }, "remoteHubStorageUri": { "type": "string", "defaultValue": "", @@ -1139,14 +1131,14 @@ } }, "variables": { - "$fxv#0": "12.0", - "$fxv#1": "12.0", - "hub": "[__bicep.newHub(parameters('hubName'), parameters('location'), parameters('tags'), parameters('tagsByResource'), parameters('storageSku'), parameters('keyVaultSku'), parameters('enableInfrastructureEncryption'), parameters('enablePublicAccess'), parameters('virtualNetworkAddressPrefix'), parameters('enableDefaultTelemetry'))]", + "$fxv#0": "13.0", + "$fxv#1": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n# Init outputs\r\n$DeploymentScriptOutputs = @{}\r\n\r\n#\r\n$adfParams = @{\r\n ResourceGroupName = $env:DataFactoryResourceGroup\r\n DataFactoryName = $env:DataFactoryName\r\n}\r\n\r\n# Delete old triggers\r\n$triggers = Get-AzDataFactoryV2Trigger @adfParams -ErrorAction SilentlyContinue `\r\n| Where-Object { $_.Name -match '^msexports(_(setup|daily|monthly|extract|FileAdded))?$' }\r\n$DeploymentScriptOutputs[\"stopTriggers\"] = $triggers | Stop-AzDataFactoryV2Trigger -Force -ErrorAction SilentlyContinue\r\n$DeploymentScriptOutputs[\"deleteTriggers\"] = $triggers | Remove-AzDataFactoryV2Trigger -Force -ErrorAction SilentlyContinue\r\n\r\n# Delete old pipelines\r\n$DeploymentScriptOutputs[\"pipelines\"] = Get-AzDataFactoryV2Pipeline @adfParams -ErrorAction SilentlyContinue `\r\n| Where-Object { $_.Name -match '^(msexports_(backfill|extract|fill|get|run|setup|transform)|config_(BackfillData|ExportData|RunBackfill|RunExports))$' } `\r\n| Remove-AzDataFactoryV2Pipeline -Force -ErrorAction SilentlyContinue\r\n\r\n# Delete old datasets\r\n$DeploymentScriptOutputs[\"datasets\"] = Get-AzDataFactoryV2Dataset @adfParams -ErrorAction SilentlyContinue `\r\n| Where-Object { $_.Name -eq 'manifest' } `\r\n| Remove-AzDataFactoryV2Dataset -Force -ErrorAction SilentlyContinue\r\n", + "hub": "[__bicep.newHub(parameters('hubName'), parameters('location'), parameters('tags'), parameters('tagsByResource'), parameters('storageSku'), parameters('keyVaultSku'), parameters('enablePurgeProtection'), parameters('enableInfrastructureEncryption'), parameters('enablePublicAccess'), parameters('virtualNetworkAddressPrefix'), parameters('enableDefaultTelemetry'))]", "useFabric": "[not(empty(parameters('fabricQueryUri')))]", - "deployDataExplorer": "[and(not(variables('useFabric')), not(empty(parameters('dataExplorerName'))))]", + "useAzureDataExplorer": "[and(not(variables('useFabric')), not(empty(parameters('dataExplorerName'))))]", "telemetryId": "00f120b5-2007-6120-0000-40b000000000", - "telemetryString": "[join(createArray(if(or(empty(parameters('remoteHubStorageUri')), empty(parameters('remoteHubStorageKey'))), '', 'R'), substring(split(parameters('storageSku'), '_')[1], 0, 1), if(not(variables('useFabric')), '', format('F{0}', parameters('fabricCapacityUnits'))), if(not(variables('deployDataExplorer')), '', format('X{0}', substring(parameters('dataExplorerSku'), 0, 1))), if(not(variables('deployDataExplorer')), '', replace(replace(replace(replace(replace(replace(replace(replace(split(split(parameters('dataExplorerSku'), 'Standard_')[1], '_')[0], 'C', ''), 'D', ''), 'E', ''), 'L', ''), 'a', ''), 'd', ''), 'i', ''), 's', '')), if(or(not(variables('deployDataExplorer')), equals(parameters('dataExplorerCapacity'), 1)), '', format('x{0}', parameters('dataExplorerCapacity'))), if(parameters('enablePublicAccess'), '', 'P')), '')]", - "_1.finOpsToolkitVersion": "12.0" + "telemetryString": "[join(createArray(if(or(empty(parameters('remoteHubStorageUri')), empty(parameters('remoteHubStorageKey'))), '', 'R'), substring(split(parameters('storageSku'), '_')[1], 0, 1), if(not(variables('useFabric')), '', format('F{0}', parameters('fabricCapacityUnits'))), if(not(variables('useAzureDataExplorer')), '', format('X{0}', substring(parameters('dataExplorerSku'), 0, 1))), if(not(variables('useAzureDataExplorer')), '', replace(replace(replace(replace(replace(replace(replace(replace(split(split(parameters('dataExplorerSku'), 'Standard_')[1], '_')[0], 'C', ''), 'D', ''), 'E', ''), 'L', ''), 'a', ''), 'd', ''), 'i', ''), 's', '')), if(or(not(variables('useAzureDataExplorer')), equals(parameters('dataExplorerCapacity'), 1)), '', format('x{0}', parameters('dataExplorerCapacity'))), if(parameters('enablePublicAccess'), '', 'P')), '')]", + "_1.finOpsToolkitVersion": "13.0" }, "resources": { "telemetry": { @@ -1170,18 +1162,33 @@ } } }, - "infrastructure": { + "core": { "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.FinOpsHubs.Infrastructure", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core", "properties": { "expressionEvaluationOptions": { "scope": "inner" }, "mode": "Incremental", "parameters": { - "hub": { - "value": "[variables('hub')]" + "app": { + "value": "[__bicep.newApp(variables('hub'), 'Microsoft.FinOpsHubs', 'Core')]" + }, + "scopesToMonitor": { + "value": "[parameters('scopesToMonitor')]" + }, + "msexportRetentionInDays": { + "value": "[parameters('exportRetentionInDays')]" + }, + "ingestionRetentionInMonths": { + "value": "[parameters('ingestionRetentionInMonths')]" + }, + "rawRetentionInDays": { + "value": "[parameters('dataExplorerRawRetentionInDays')]" + }, + "finalRetentionInMonths": { + "value": "[parameters('dataExplorerFinalRetentionInMonths')]" } }, "template": { @@ -1191,11 +1198,101 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "8095489755767003461" + "version": "0.40.2.10011", + "templateHash": "9360754644605054925" } }, "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, "_1.HubRoutingProperties": { "type": "object", "properties": { @@ -1228,6 +1325,9 @@ "subnets": { "type": "object", "properties": { + "dataExplorer": { + "type": "string" + }, "dataFactory": { "type": "string" }, @@ -1254,6 +1354,7 @@ "table": "Resource ID and name for the table storage DNS zone." }, "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", "dataFactory": "Resource ID of the subnet for Data Factory instances.", "keyVault": "Resource ID of the subnet for Key Vault instances.", "scripts": "Resource ID of the subnet for deployment script storage.", @@ -1261,7 +1362,7 @@ }, "description": "FinOps hub private network routing properties.", "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "../../fx/hub-types.bicep" } } }, @@ -1280,11 +1381,11 @@ "name": "Resource name.", "description": "Resource ID and name.", "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "../../fx/hub-types.bicep" } } }, - "HubProperties": { + "HubAppProperties": { "type": "object", "properties": { "id": { @@ -1293,1039 +1394,400 @@ "name": { "type": "string" }, - "location": { + "publisher": { "type": "string" }, - "tags": { - "type": "object" + "suffix": { + "type": "string" }, - "tagsByResource": { + "tags": { "type": "object" }, - "version": { + "dataFactory": { "type": "string" }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" - }, - "keyVaultSku": { - "type": "string" - }, - "networkAddressPrefix": { - "type": "string" - }, - "privateRouting": { - "type": "bool" - }, - "publisherIsolation": { - "type": "bool" - }, - "storageInfrastructureEncryption": { - "type": "bool" - }, - "storageSku": { - "type": "string" - } - } + "keyVault": { + "type": "string" }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" + "storage": { + "type": "string" }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" - } - } + "hub": { + "$ref": "#/definitions/_1.HubProperties" } }, "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", - "location": "Azure resource location of the FinOps hub instance.", - "tags": "Tags to apply to all FinOps hub resources.", - "tagsByResource": "Tags to apply to resources based on their resource type.", - "version": "FinOps hub version number.", - "options": { - "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", - "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", - "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", - "privateRouting": "Indicates whether private network routing is enabled.", - "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", - "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", - "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." - }, - "routing": "FinOps hub private network routing properties, if enabled.", - "core": { - "suffix": "Unique suffix used for shared resources." - }, - "apps": {}, - "description": "FinOps hub instance properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" } } } }, - "functions": [ - { - "namespace": "__bicep", - "members": { - "getHubTags": { - "parameters": [ - { - "$ref": "#/definitions/HubProperties", - "name": "hub" - }, - { - "type": "string", - "name": "resourceType" - } - ], - "output": { - "type": "object", - "value": "[union(parameters('hub').tags, coalesce(tryGet(parameters('hub').tagsByResource, parameters('resourceType')), createObject()))]" - }, - "metadata": { - "description": "Returns a tags dictionary that includes tags for the FinOps hub instance.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - } - } - ], "parameters": { - "hub": { - "$ref": "#/definitions/HubProperties", + "app": { + "$ref": "#/definitions/HubAppProperties", "metadata": { - "description": "Required. FinOps hub instance properties." + "description": "Required. FinOps hub app getting deployed." } - } - }, - "variables": { - "nsgName": "[format('{0}-nsg', parameters('hub').routing.networkName)]", - "finopsHubSubnetName": "private-endpoint-subnet", - "scriptSubnetName": "script-subnet", - "dataExplorerSubnetName": "dataExplorer-subnet", - "subnets": "[if(not(parameters('hub').options.privateRouting), createArray(), createArray(createObject('name', variables('finopsHubSubnetName'), 'properties', createObject('addressPrefix', cidrSubnet(parameters('hub').options.networkAddressPrefix, 28, 0), 'networkSecurityGroup', createObject('id', resourceId('Microsoft.Network/networkSecurityGroups', variables('nsgName'))), 'serviceEndpoints', createArray(createObject('service', 'Microsoft.Storage')))), createObject('name', variables('scriptSubnetName'), 'properties', createObject('addressPrefix', cidrSubnet(parameters('hub').options.networkAddressPrefix, 28, 1), 'networkSecurityGroup', createObject('id', resourceId('Microsoft.Network/networkSecurityGroups', variables('nsgName'))), 'delegations', createArray(createObject('name', 'Microsoft.ContainerInstance/containerGroups', 'properties', createObject('serviceName', 'Microsoft.ContainerInstance/containerGroups'))), 'serviceEndpoints', createArray(createObject('service', 'Microsoft.Storage')))), createObject('name', variables('dataExplorerSubnetName'), 'properties', createObject('addressPrefix', cidrSubnet(parameters('hub').options.networkAddressPrefix, 27, 1), 'networkSecurityGroup', createObject('id', resourceId('Microsoft.Network/networkSecurityGroups', variables('nsgName')))))))]" - }, - "resources": { - "vNet::finopsHubSubnet": { - "condition": "[parameters('hub').options.privateRouting]", - "existing": true, - "type": "Microsoft.Network/virtualNetworks/subnets", - "apiVersion": "2023-11-01", - "name": "[format('{0}/{1}', parameters('hub').routing.networkName, variables('finopsHubSubnetName'))]", - "dependsOn": [ - "vNet" - ] }, - "vNet::scriptSubnet": { - "condition": "[parameters('hub').options.privateRouting]", - "existing": true, - "type": "Microsoft.Network/virtualNetworks/subnets", - "apiVersion": "2023-11-01", - "name": "[format('{0}/{1}', parameters('hub').routing.networkName, variables('scriptSubnetName'))]", - "dependsOn": [ - "vNet" - ] + "scopesToMonitor": { + "type": "array", + "metadata": { + "description": "Optional. List of scope IDs to monitor and ingest cost for." + } }, - "vNet::dataExplorerSubnet": { - "condition": "[parameters('hub').options.privateRouting]", - "existing": true, - "type": "Microsoft.Network/virtualNetworks/subnets", - "apiVersion": "2023-11-01", - "name": "[format('{0}/{1}', parameters('hub').routing.networkName, variables('dataExplorerSubnetName'))]", - "dependsOn": [ - "vNet" - ] + "msexportRetentionInDays": { + "type": "int", + "defaultValue": 0, + "metadata": { + "description": "Optional. Number of days of data to retain in the msexports container. Default: 0." + } }, - "blobPrivateDnsZone::blobPrivateDnsZoneLink": { - "condition": "[parameters('hub').options.privateRouting]", - "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", - "apiVersion": "2024-06-01", - "name": "[format('{0}/{1}', string(parameters('hub').routing.dnsZones.blob.name), format('{0}-link', replace(string(parameters('hub').routing.dnsZones.blob.name), '.', '-')))]", - "location": "global", - "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", - "properties": { - "registrationEnabled": false, - "virtualNetwork": { - "id": "[parameters('hub').routing.networkId]" - } - }, - "dependsOn": [ - "blobPrivateDnsZone" - ] + "ingestionRetentionInMonths": { + "type": "int", + "defaultValue": 13, + "metadata": { + "description": "Optional. Number of months of data to retain in the ingestion container. Default: 13." + } }, - "dfsPrivateDnsZone::dfsPrivateDnsZoneLink": { - "condition": "[parameters('hub').options.privateRouting]", - "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", - "apiVersion": "2024-06-01", - "name": "[format('{0}/{1}', string(parameters('hub').routing.dnsZones.dfs.name), format('{0}-link', replace(string(parameters('hub').routing.dnsZones.dfs.name), '.', '-')))]", - "location": "global", - "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", - "properties": { - "registrationEnabled": false, - "virtualNetwork": { - "id": "[parameters('hub').routing.networkId]" - } - }, - "dependsOn": [ - "dfsPrivateDnsZone" - ] + "rawRetentionInDays": { + "type": "int", + "defaultValue": 0, + "metadata": { + "description": "Optional. Number of days of data to retain in the Data Explorer *_raw tables. Default: 0." + } }, - "queuePrivateDnsZone::queuePrivateDnsZoneLink": { - "condition": "[parameters('hub').options.privateRouting]", - "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", - "apiVersion": "2024-06-01", - "name": "[format('{0}/{1}', string(parameters('hub').routing.dnsZones.queue.name), format('{0}-link', replace(string(parameters('hub').routing.dnsZones.queue.name), '.', '-')))]", - "location": "global", - "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "finalRetentionInMonths": { + "type": "int", + "defaultValue": 13, + "metadata": { + "description": "Optional. Number of months of data to retain in the Data Explorer *_final_v* tables. Default: 13." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\nWrite-Output \"Updating settings.json file...\"\r\nWrite-Output \" Storage account: $env:storageAccountName\"\r\nWrite-Output \" Container: $env:containerName\"\r\n\r\n$validateScopes = { $_.scope.Length -gt 45 }\r\n\r\n# Initialize variables\r\n$fileName = 'settings.json'\r\n$filePath = Join-Path -Path . -ChildPath $fileName\r\n$newScopes = $env:scopes.Split('|') | ForEach-Object { [PSCustomObject]@{ scope = $_ } } | Where-Object $validateScopes\r\n\r\n# Get storage context\r\n$storageContext = @{\r\n Context = New-AzStorageContext -StorageAccountName $env:storageAccountName -UseConnectedAccount\r\n Container = $env:containerName\r\n}\r\n\r\n# Download existing settings, if they exist\r\n$blob = Get-AzStorageBlobContent @storageContext -Blob $fileName -Destination $filePath -Force\r\nif ($blob)\r\n{\r\n $text = Get-Content $filePath -Raw\r\n Write-Output \"---------\"\r\n Write-Output $text\r\n Write-Output \"---------\"\r\n $json = $text | ConvertFrom-Json\r\n Write-Output \"Existing settings.json file found. Updating...\"\r\n\r\n # Rename exportScopes to scopes + convert to object array\r\n if ($json.exportScopes)\r\n {\r\n Write-Output \" Updating exportScopes...\"\r\n if ($json.exportScopes[0] -is [string])\r\n {\r\n Write-Output \" Converting string array to object array...\"\r\n $json.exportScopes = @($json.exportScopes | Where-Object $validateScopes | ForEach-Object { @{ scope = $_ } })\r\n if (-not ($json.exportScopes -is [array]))\r\n {\r\n Write-Output \" Converting single object to object array...\"\r\n $json.exportScopes = @($json.exportScopes)\r\n }\r\n }\r\n\r\n Write-Output \" Renaming to 'scopes'...\"\r\n $json | Add-Member -MemberType NoteProperty -Name scopes -Value $json.exportScopes\r\n $json.PSObject.Properties.Remove('exportScopes')\r\n }\r\n\r\n # Force string array to object array with unique values\r\n if ($json.scopes)\r\n {\r\n Write-Output \" Converting string array to object array...\"\r\n $scopeArray = @()\r\n $json.scopes | Where-Object $validateScopes | ForEach-Object { $scopeArray += $_ } | Select-Object -Unique\r\n $json.scopes = @() + $scopeArray\r\n }\r\n}\r\n\r\n# Set default if not found\r\nif (!$json)\r\n{\r\n Write-Output \"No existing settings.json file found. Creating new file...\"\r\n $json = [ordered]@{\r\n '$schema' = 'https://aka.ms/finops/hubs/settings-schema'\r\n type = 'HubInstance'\r\n version = ''\r\n learnMore = 'https://aka.ms/finops/hubs'\r\n scopes = @()\r\n retention = @{\r\n 'msexports' = @{\r\n days = 0\r\n }\r\n 'ingestion' = @{\r\n months = 13\r\n }\r\n 'raw' = @{\r\n days = 0\r\n }\r\n 'final' = @{\r\n months = 13\r\n }\r\n }\r\n }\r\n\r\n $text = $json | ConvertTo-Json\r\n Write-Output \"---------\"\r\n Write-Output $text\r\n Write-Output \"---------\"\r\n}\r\n\r\n# Set default retention\r\nif (!($json.retention))\r\n{\r\n # In case the retention object is not present in the settings.json file (versions before 0.4), add it with default values\r\n $retention = @\"\r\n {\r\n \"msexports\": {\r\n \"days\": 0\r\n },\r\n \"ingestion\": {\r\n \"months\": 13\r\n },\r\n \"raw\": {\r\n \"days\": 0\r\n },\r\n \"final\": {\r\n \"months\": 13\r\n }\r\n }\r\n\"@\r\n $json | Add-Member -Name retention -Value (ConvertFrom-Json $retention) -MemberType NoteProperty\r\n}\r\n\r\n# Set or update msexports retention\r\nif (!($json.retention.msexports))\r\n{\r\n $json.retention | Add-Member -Name msexports -Value (ConvertFrom-Json \"{\"\"days\"\":$($env:msexportRetentionInDays)}\") -MemberType NoteProperty\r\n}\r\nelse\r\n{\r\n $json.retention.msexports.days = [Int32]::Parse($env:msexportRetentionInDays)\r\n}\r\n\r\n# Set or update ingestion retention\r\nif (!($json.retention.ingestion))\r\n{\r\n $json.retention | Add-Member -Name ingestion -Value (ConvertFrom-Json \"{\"\"months\"\":$($env:ingestionRetentionInMonths)}\") -MemberType NoteProperty\r\n}\r\nelse\r\n{\r\n $json.retention.ingestion.months = [Int32]::Parse($env:ingestionRetentionInMonths)\r\n}\r\n\r\n# Set or update raw retention\r\nif (!($json.retention.raw))\r\n{\r\n $json.retention | Add-Member -Name raw -Value (ConvertFrom-Json \"{\"\"days\"\":$($env:rawRetentionInDays)}\") -MemberType NoteProperty\r\n}\r\nelse\r\n{\r\n $json.retention.raw.days = [Int32]::Parse($env:rawRetentionInDays)\r\n}\r\n\r\n# Set or update final retention\r\nif (!($json.retention.final))\r\n{\r\n $json.retention | Add-Member -Name final -Value (ConvertFrom-Json \"{\"\"months\"\":$($env:finalRetentionInMonths)}\") -MemberType NoteProperty\r\n}\r\nelse\r\n{\r\n $json.retention.final.months = [Int32]::Parse($env:finalRetentionInMonths)\r\n}\r\n\r\n# Updating settings\r\nWrite-Output \"Updating version to $env:ftkVersion...\"\r\n$json.version = $env:ftkVersion\r\n$json.scopes = ($json.scopes + $newScopes) | Sort-Object scope -Unique\r\nif ($null -eq $json.scopes) { $json.scopes = @() }\r\n$text = $json | ConvertTo-Json\r\nWrite-Output \"---------\"\r\nWrite-Output $text\r\nWrite-Output \"---------\"\r\n$text | Out-File $filePath\r\n\r\n# Upload new/updated settings\r\nWrite-Output \"Uploading settings.json file...\"\r\nSet-AzStorageBlobContent @storageContext -File $filePath -Force | Out-Null\r\n", + "CONFIG": "config", + "INGESTION": "ingestion", + "finOpsToolkitVersion": "13.0" + }, + "resources": { + "dataFactory::dataset_config": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, variables('CONFIG'))]", "properties": { - "registrationEnabled": false, - "virtualNetwork": { - "id": "[parameters('hub').routing.networkId]" + "linkedServiceName": { + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" + }, + "type": "Json", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().fileName}", + "type": "Expression" + }, + "folderPath": { + "value": "@{dataset().folderPath}", + "type": "Expression" + } + } + }, + "parameters": { + "fileName": { + "type": "String", + "defaultValue": "settings.json" + }, + "folderPath": { + "type": "String", + "defaultValue": "[reference('configContainer').outputs.containerName.value]" + } } }, "dependsOn": [ - "queuePrivateDnsZone" + "appRegistration", + "configContainer" ] }, - "tablePrivateDnsZone::tablePrivateDnsZoneLink": { - "condition": "[parameters('hub').options.privateRouting]", - "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", - "apiVersion": "2024-06-01", - "name": "[format('{0}/{1}', string(parameters('hub').routing.dnsZones.table.name), format('{0}-link', replace(string(parameters('hub').routing.dnsZones.table.name), '.', '-')))]", - "location": "global", - "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "dataFactory::dataset_ingestion": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, variables('INGESTION'))]", "properties": { - "registrationEnabled": false, - "virtualNetwork": { - "id": "[parameters('hub').routing.networkId]" + "annotations": [], + "parameters": { + "blobPath": { + "type": "String" + } + }, + "type": "Parquet", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().blobPath}", + "type": "Expression" + }, + "fileSystem": "[reference('ingestionContainer').outputs.containerName.value]" + } + }, + "linkedServiceName": { + "parameters": {}, + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" } }, "dependsOn": [ - "tablePrivateDnsZone" + "appRegistration", + "ingestionContainer" ] }, - "scriptEndpoint::scriptPrivateDnsZoneGroup": { - "condition": "[parameters('hub').options.privateRouting]", - "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", - "apiVersion": "2023-11-01", - "name": "[format('{0}/{1}', format('{0}-blob-ep', string(parameters('hub').routing.scriptStorage)), 'blob-endpoint-zone')]", + "dataFactory::dataset_ingestion_files": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_files', variables('INGESTION')))]", "properties": { - "privateDnsZoneConfigs": [ - { - "name": "[string(parameters('hub').routing.dnsZones.blob.name)]", - "properties": { - "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', string(parameters('hub').routing.dnsZones.blob.name))]" + "annotations": [], + "parameters": { + "folderPath": { + "type": "String" + } + }, + "type": "Parquet", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileSystem": "[reference('ingestionContainer').outputs.containerName.value]", + "folderPath": { + "value": "@dataset().folderPath", + "type": "Expression" } } - ] + }, + "linkedServiceName": { + "parameters": {}, + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" + } }, "dependsOn": [ - "blobPrivateDnsZone", - "scriptEndpoint" + "appRegistration", + "ingestionContainer" ] }, - "nsg": { - "condition": "[parameters('hub').options.privateRouting]", - "type": "Microsoft.Network/networkSecurityGroups", - "apiVersion": "2023-11-01", - "name": "[variables('nsgName')]", - "location": "[parameters('hub').location]", - "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/networkSecurityGroups')]", + "dataFactory::dataset_ingestion_manifest": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ingestion_manifest')]", "properties": { - "securityRules": [ - { - "name": "AllowVnetInBound", - "properties": { - "priority": 100, - "direction": "Inbound", - "access": "Allow", - "protocol": "*", - "sourcePortRange": "*", - "destinationPortRange": "*", - "sourceAddressPrefix": "VirtualNetwork", - "destinationAddressPrefix": "VirtualNetwork" - } - }, - { - "name": "AllowAzureLoadBalancerInBound", - "properties": { - "priority": 200, - "direction": "Inbound", - "access": "Allow", - "protocol": "*", - "sourcePortRange": "*", - "destinationPortRange": "*", - "sourceAddressPrefix": "AzureLoadBalancer", - "destinationAddressPrefix": "*" - } - }, - { - "name": "DenyAllInBound", - "properties": { - "priority": 4096, - "direction": "Inbound", - "access": "Deny", - "protocol": "*", - "sourcePortRange": "*", - "destinationPortRange": "*", - "sourceAddressPrefix": "*", - "destinationAddressPrefix": "*" - } - }, - { - "name": "AllowVnetOutBound", - "properties": { - "priority": 100, - "direction": "Outbound", - "access": "Allow", - "protocol": "*", - "sourcePortRange": "*", - "destinationPortRange": "*", - "sourceAddressPrefix": "VirtualNetwork", - "destinationAddressPrefix": "VirtualNetwork" - } - }, - { - "name": "AllowInternetOutBound", - "properties": { - "priority": 200, - "direction": "Outbound", - "access": "Allow", - "protocol": "*", - "sourcePortRange": "*", - "destinationPortRange": "*", - "sourceAddressPrefix": "*", - "destinationAddressPrefix": "Internet" - } + "annotations": [], + "parameters": { + "fileName": { + "type": "String", + "defaultValue": "manifest.json" }, - { - "name": "DenyAllOutBound", - "properties": { - "priority": 4096, - "direction": "Outbound", - "access": "Deny", - "protocol": "*", - "sourcePortRange": "*", - "destinationPortRange": "*", - "sourceAddressPrefix": "*", - "destinationAddressPrefix": "*" + "folderPath": { + "type": "String", + "defaultValue": "[variables('INGESTION')]" + } + }, + "type": "Json", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().fileName}", + "type": "Expression" + }, + "folderPath": { + "value": "@{dataset().folderPath}", + "type": "Expression" } } - ] - } - }, - "vNet": { - "condition": "[parameters('hub').options.privateRouting]", - "type": "Microsoft.Network/virtualNetworks", - "apiVersion": "2023-11-01", - "name": "[parameters('hub').routing.networkName]", - "location": "[parameters('hub').location]", - "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/virtualNetworks')]", - "properties": { - "addressSpace": { - "addressPrefixes": [ - "[parameters('hub').options.networkAddressPrefix]" - ] }, - "subnets": "[variables('subnets')]" + "linkedServiceName": { + "parameters": {}, + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" + } }, "dependsOn": [ - "nsg" - ] - }, - "blobPrivateDnsZone": { - "condition": "[parameters('hub').options.privateRouting]", - "type": "Microsoft.Network/privateDnsZones", - "apiVersion": "2024-06-01", - "name": "[string(parameters('hub').routing.dnsZones.blob.name)]", - "location": "global", - "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/privateDnsZones')]", - "properties": {}, - "dependsOn": [ - "vNet" + "appRegistration" ] }, - "dfsPrivateDnsZone": { - "condition": "[parameters('hub').options.privateRouting]", - "type": "Microsoft.Network/privateDnsZones", - "apiVersion": "2024-06-01", - "name": "[string(parameters('hub').routing.dnsZones.dfs.name)]", - "location": "global", - "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/privateDnsZones')]", - "properties": {}, + "dataFactory": { + "existing": true, + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", "dependsOn": [ - "vNet" + "appRegistration" ] }, - "queuePrivateDnsZone": { - "condition": "[parameters('hub').options.privateRouting]", - "type": "Microsoft.Network/privateDnsZones", - "apiVersion": "2024-06-01", - "name": "[string(parameters('hub').routing.dnsZones.queue.name)]", - "location": "global", - "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/privateDnsZones')]", - "properties": {}, - "dependsOn": [ - "vNet" - ] - }, - "tablePrivateDnsZone": { - "condition": "[parameters('hub').options.privateRouting]", - "type": "Microsoft.Network/privateDnsZones", - "apiVersion": "2024-06-01", - "name": "[string(parameters('hub').routing.dnsZones.table.name)]", - "location": "global", - "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/privateDnsZones')]", - "properties": {}, - "dependsOn": [ - "vNet" - ] - }, - "scriptStorageAccount": { - "condition": "[parameters('hub').options.privateRouting]", - "type": "Microsoft.Storage/storageAccounts", - "apiVersion": "2022-09-01", - "name": "[string(parameters('hub').routing.scriptStorage)]", - "location": "[parameters('hub').location]", - "sku": { - "name": "Standard_LRS" - }, - "kind": "StorageV2", - "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/storageAccounts')]", - "properties": { - "supportsHttpsTrafficOnly": true, - "allowSharedKeyAccess": true, - "isHnsEnabled": false, - "minimumTlsVersion": "TLS1_2", - "allowBlobPublicAccess": false, - "publicNetworkAccess": "Enabled", - "networkAcls": { - "bypass": "AzureServices", - "defaultAction": "Deny", - "virtualNetworkRules": [ - { - "id": "[parameters('hub').routing.subnets.scripts]", - "action": "Allow" - } - ] - } - }, - "dependsOn": [ - "vNet::scriptSubnet" - ] - }, - "scriptEndpoint": { - "condition": "[parameters('hub').options.privateRouting]", - "type": "Microsoft.Network/privateEndpoints", - "apiVersion": "2023-11-01", - "name": "[format('{0}-blob-ep', string(parameters('hub').routing.scriptStorage))]", - "location": "[parameters('hub').location]", - "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Network/privateEndpoints')]", + "infrastructure": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core_Infrastructure", "properties": { - "subnet": { - "id": "[parameters('hub').routing.subnets.storage]" + "expressionEvaluationOptions": { + "scope": "inner" }, - "privateLinkServiceConnections": [ - { - "name": "scriptLink", - "properties": { - "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', string(parameters('hub').routing.scriptStorage))]", - "groupIds": [ - "blob" - ] - } + "mode": "Incremental", + "parameters": { + "hub": { + "value": "[parameters('app').hub]" } - ] - }, - "dependsOn": [ - "scriptStorageAccount", - "vNet::scriptSubnet" - ] - } - }, - "outputs": { - "config": { - "$ref": "#/definitions/HubProperties", - "metadata": { - "description": "FinOps hub configuration settings." - }, - "value": "[parameters('hub')]" - }, - "vNetId": { - "type": "string", - "metadata": { - "description": "Resource ID of the virtual network." - }, - "value": "[if(not(parameters('hub').options.privateRouting), '', resourceId('Microsoft.Network/virtualNetworks', parameters('hub').routing.networkName))]" - }, - "vNetAddressSpace": { - "type": "array", - "metadata": { - "description": "Virtual network address prefixes." - }, - "value": "[if(not(parameters('hub').options.privateRouting), createArray(), reference('vNet').addressSpace.addressPrefixes)]" - }, - "vNetSubnets": { - "type": "array", - "metadata": { - "description": "Virtual network subnets." - }, - "value": "[if(not(parameters('hub').options.privateRouting), createArray(), reference('vNet').subnets)]" - }, - "finopsHubSubnetId": { - "type": "string", - "metadata": { - "description": "Resource ID of the FinOps hub network subnet." - }, - "value": "[if(not(parameters('hub').options.privateRouting), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('hub').routing.networkName, variables('finopsHubSubnetName')))]" - }, - "scriptSubnetId": { - "type": "string", - "metadata": { - "description": "Resource ID of the script storage account network subnet." - }, - "value": "[if(not(parameters('hub').options.privateRouting), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('hub').routing.networkName, variables('scriptSubnetName')))]" - }, - "dataExplorerSubnetId": { - "type": "string", - "metadata": { - "description": "Resource ID of the Data Explorer network subnet." - }, - "value": "[if(not(parameters('hub').options.privateRouting), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('hub').routing.networkName, variables('dataExplorerSubnetName')))]" - } - } - } - } - }, - "core": { - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.FinOpsHubs.Core", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "hub": { - "value": "[variables('hub')]" - }, - "telemetryString": { - "value": "[variables('telemetryString')]" - }, - "scopesToMonitor": { - "value": "[parameters('scopesToMonitor')]" - }, - "msexportRetentionInDays": { - "value": "[parameters('exportRetentionInDays')]" - }, - "ingestionRetentionInMonths": { - "value": "[parameters('ingestionRetentionInMonths')]" - }, - "rawRetentionInDays": { - "value": "[parameters('dataExplorerRawRetentionInDays')]" - }, - "finalRetentionInMonths": { - "value": "[parameters('dataExplorerFinalRetentionInMonths')]" - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "languageVersion": "2.0", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "15396896523851766061" - } - }, - "definitions": { - "_1.HubRoutingProperties": { - "type": "object", - "properties": { - "networkId": { - "type": "string" - }, - "networkName": { - "type": "string" - }, - "scriptStorage": { - "type": "string" }, - "dnsZones": { - "type": "object", - "properties": { - "blob": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "dfs": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "queue": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "table": { - "$ref": "#/definitions/_1.IdNameObject" + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "14104368317935185659" } - } - }, - "subnets": { - "type": "object", - "properties": { - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" + }, + "definitions": { + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } }, - "scripts": { - "type": "string" + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } }, - "storage": { - "type": "string" - } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubProperties": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - }, - "location": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "tagsByResource": { - "type": "object" - }, - "version": { - "type": "string" - }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" - }, - "keyVaultSku": { - "type": "string" - }, - "networkAddressPrefix": { - "type": "string" - }, - "privateRouting": { - "type": "bool" - }, - "publisherIsolation": { - "type": "bool" - }, - "storageInfrastructureEncryption": { - "type": "bool" - }, - "storageSku": { - "type": "string" - } - } - }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" - }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" - } - } - } - }, - "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", - "location": "Azure resource location of the FinOps hub instance.", - "tags": "Tags to apply to all FinOps hub resources.", - "tagsByResource": "Tags to apply to resources based on their resource type.", - "version": "FinOps hub version number.", - "options": { - "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", - "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", - "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", - "privateRouting": "Indicates whether private network routing is enabled.", - "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", - "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", - "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." - }, - "routing": "FinOps hub private network routing properties, if enabled.", - "core": { - "suffix": "Unique suffix used for shared resources." - }, - "apps": {}, - "description": "FinOps hub instance properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - }, - "parameters": { - "hub": { - "$ref": "#/definitions/HubProperties", - "metadata": { - "description": "Required. FinOps hub instance to deploy the app to." - } - }, - "scopesToMonitor": { - "type": "array", - "metadata": { - "description": "Optional. List of scope IDs to monitor and ingest cost for." - } - }, - "msexportRetentionInDays": { - "type": "int", - "defaultValue": 0, - "metadata": { - "description": "Optional. Number of days of data to retain in the msexports container. Default: 0." - } - }, - "ingestionRetentionInMonths": { - "type": "int", - "defaultValue": 13, - "metadata": { - "description": "Optional. Number of months of data to retain in the ingestion container. Default: 13." - } - }, - "rawRetentionInDays": { - "type": "int", - "defaultValue": 0, - "metadata": { - "description": "Optional. Number of days of data to retain in the Data Explorer *_raw tables. Default: 0." - } - }, - "finalRetentionInMonths": { - "type": "int", - "defaultValue": 13, - "metadata": { - "description": "Optional. Number of months of data to retain in the Data Explorer *_final_v* tables. Default: 13." - } - }, - "telemetryString": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." - } - } - }, - "variables": { - "$fxv#0": "12.0", - "$fxv#1": "12.0", - "$fxv#2": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\nWrite-Output \"Updating settings.json file...\"\r\nWrite-Output \" Storage account: $env:storageAccountName\"\r\nWrite-Output \" Container: $env:containerName\"\r\n\r\n$validateScopes = { $_.Length -gt 45 }\r\n\r\n# Initialize variables\r\n$fileName = 'settings.json'\r\n$filePath = Join-Path -Path . -ChildPath $fileName\r\n$newScopes = $env:scopes.Split('|') | Where-Object $validateScopes | ForEach-Object { @{ scope = $_ } }\r\n\r\n# Get storage context\r\n$storageContext = @{\r\n Context = New-AzStorageContext -StorageAccountName $env:storageAccountName -UseConnectedAccount\r\n Container = $env:containerName\r\n}\r\n\r\n# Download existing settings, if they exist\r\n$blob = Get-AzStorageBlobContent @storageContext -Blob $fileName -Destination $filePath -Force\r\nif ($blob)\r\n{\r\n $text = Get-Content $filePath -Raw\r\n Write-Output \"---------\"\r\n Write-Output $text\r\n Write-Output \"---------\"\r\n $json = $text | ConvertFrom-Json\r\n Write-Output \"Existing settings.json file found. Updating...\"\r\n\r\n # Rename exportScopes to scopes + convert to object array\r\n if ($json.exportScopes)\r\n {\r\n Write-Output \" Updating exportScopes...\"\r\n if ($json.exportScopes[0] -is [string])\r\n {\r\n Write-Output \" Converting string array to object array...\"\r\n $json.exportScopes = @($json.exportScopes | Where-Object $validateScopes | ForEach-Object { @{ scope = $_ } })\r\n if (-not ($json.exportScopes -is [array]))\r\n {\r\n Write-Output \" Converting single object to object array...\"\r\n $json.exportScopes = @($json.exportScopes)\r\n }\r\n }\r\n\r\n Write-Output \" Renaming to 'scopes'...\"\r\n $json | Add-Member -MemberType NoteProperty -Name scopes -Value $json.exportScopes\r\n $json.PSObject.Properties.Remove('exportScopes')\r\n }\r\n\r\n # Force string array to object array with unique values\r\n if ($json.scopes)\r\n {\r\n Write-Output \" Converting string array to object array...\"\r\n $scopeArray = @()\r\n $json.scopes | Where-Object $validateScopes | ForEach-Object { $scopeArray += $_ } | Select-Object -Unique\r\n $json.scopes = @() + $scopeArray\r\n }\r\n}\r\n\r\n# Set default if not found\r\nif (!$json)\r\n{\r\n Write-Output \"No existing settings.json file found. Creating new file...\"\r\n $json = [ordered]@{\r\n '$schema' = 'https://aka.ms/finops/hubs/settings-schema'\r\n type = 'HubInstance'\r\n version = ''\r\n learnMore = 'https://aka.ms/finops/hubs'\r\n scopes = @()\r\n retention = @{\r\n 'msexports' = @{\r\n days = 0\r\n }\r\n 'ingestion' = @{\r\n months = 13\r\n }\r\n 'raw' = @{\r\n days = 0\r\n }\r\n 'final' = @{\r\n months = 13\r\n }\r\n }\r\n }\r\n\r\n $text = $json | ConvertTo-Json\r\n Write-Output \"---------\"\r\n Write-Output $text\r\n Write-Output \"---------\"\r\n}\r\n\r\n# Set default retention\r\nif (!($json.retention))\r\n{\r\n # In case the retention object is not present in the settings.json file (versions before 0.4), add it with default values\r\n $retention = @\"\r\n {\r\n \"msexports\": {\r\n \"days\": 0\r\n },\r\n \"ingestion\": {\r\n \"months\": 13\r\n },\r\n \"raw\": {\r\n \"days\": 0\r\n },\r\n \"final\": {\r\n \"months\": 13\r\n }\r\n }\r\n\"@\r\n $json | Add-Member -Name retention -Value (ConvertFrom-Json $retention) -MemberType NoteProperty\r\n}\r\n\r\n# Set or update msexports retention\r\nif (!($json.retention.msexports))\r\n{\r\n $json.retention | Add-Member -Name msexports -Value (ConvertFrom-Json \"{\"\"days\"\":$($env:msexportRetentionInDays)}\") -MemberType NoteProperty\r\n}\r\nelse\r\n{\r\n $json.retention.msexports.days = [Int32]::Parse($env:msexportRetentionInDays)\r\n}\r\n\r\n# Set or update ingestion retention\r\nif (!($json.retention.ingestion))\r\n{\r\n $json.retention | Add-Member -Name ingestion -Value (ConvertFrom-Json \"{\"\"months\"\":$($env:ingestionRetentionInMonths)}\") -MemberType NoteProperty\r\n}\r\nelse\r\n{\r\n $json.retention.ingestion.months = [Int32]::Parse($env:ingestionRetentionInMonths)\r\n}\r\n\r\n# Set or update raw retention\r\nif (!($json.retention.raw))\r\n{\r\n $json.retention | Add-Member -Name raw -Value (ConvertFrom-Json \"{\"\"days\"\":$($env:rawRetentionInDays)}\") -MemberType NoteProperty\r\n}\r\nelse\r\n{\r\n $json.retention.raw.days = [Int32]::Parse($env:rawRetentionInDays)\r\n}\r\n\r\n# Set or update final retention\r\nif (!($json.retention.final))\r\n{\r\n $json.retention | Add-Member -Name final -Value (ConvertFrom-Json \"{\"\"months\"\":$($env:finalRetentionInMonths)}\") -MemberType NoteProperty\r\n}\r\nelse\r\n{\r\n $json.retention.final.months = [Int32]::Parse($env:finalRetentionInMonths)\r\n}\r\n\r\n# Updating settings\r\nWrite-Output \"Updating version to $env:ftkVersion...\"\r\n$json.version = $env:ftkVersion\r\n$json.scopes = (@() + $json.scopes + $newScopes) | Select-Object -Unique\r\nif ($null -eq $json.scopes) { $json.scopes = @() }\r\n$text = $json | ConvertTo-Json\r\nWrite-Output \"---------\"\r\nWrite-Output $text\r\nWrite-Output \"---------\"\r\n$text | Out-File $filePath\r\n\r\n# Upload new/updated settings\r\nWrite-Output \"Uploading settings.json file...\"\r\nSet-AzStorageBlobContent @storageContext -File $filePath -Force | Out-Null\r\n" - }, - "resources": { - "appRegistration": { - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.FinOpsHubs.Core_Register", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "hub": { - "value": "[parameters('hub')]" - }, - "publisher": { - "value": "Microsoft FinOps hubs" - }, - "namespace": { - "value": "Microsoft.FinOpsHubs" - }, - "appName": { - "value": "Core" - }, - "displayName": { - "value": "FinOps hub core" - }, - "appVersion": { - "value": "[variables('$fxv#0')]" - }, - "features": { - "value": [ - "DataFactory", - "Storage" - ] - }, - "telemetryString": { - "value": "[parameters('telemetryString')]" - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "languageVersion": "2.0", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "15179190433979236138" - } - }, - "definitions": { - "_1.HubRoutingProperties": { + "HubProperties": { "type": "object", "properties": { - "networkId": { + "id": { "type": "string" }, - "networkName": { + "name": { "type": "string" }, - "scriptStorage": { + "location": { "type": "string" }, - "dnsZones": { - "type": "object", - "properties": { - "blob": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "dfs": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "queue": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "table": { - "$ref": "#/definitions/_1.IdNameObject" - } - } + "tags": { + "type": "object" }, - "subnets": { + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { "type": "object", "properties": { - "dataFactory": { - "type": "string" + "enableTelemetry": { + "type": "bool" }, - "keyVault": { + "keyVaultSku": { "type": "string" }, - "scripts": { - "type": "string" + "keyVaultEnablePurgeProtection": { + "type": "bool" }, - "storage": { - "type": "string" - } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubAppFeature": { - "type": "string", - "allowedValues": [ - "DataFactory", - "KeyVault", - "Storage" - ], - "metadata": { - "description": "FinOps hub app features.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubAppProperties": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/HubProperties" - }, - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "storage": { - "type": "string" - } - }, - "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", - "dataFactory": "Name of the Data Factory instance for this publisher.", - "keyVault": "Name of the KeyVault instance for this publisher.", - "storage": "Name of the storage account for this publisher.", - "description": "FinOps hub app configuration settings.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubProperties": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - }, - "location": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "tagsByResource": { - "type": "object" - }, - "version": { - "type": "string" - }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" - }, - "keyVaultSku": { - "type": "string" - }, - "networkAddressPrefix": { + "networkAddressPrefix": { "type": "string" }, "privateRouting": { @@ -2364,6 +1826,7 @@ "options": { "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", "privateRouting": "Indicates whether private network routing is enabled.", "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", @@ -2377,7 +1840,7 @@ "apps": {}, "description": "FinOps hub instance properties.", "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "../../fx/hub-types.bicep" } } } @@ -2386,99 +1849,7 @@ { "namespace": "__bicep", "members": { - "getAppTags": { - "parameters": [ - { - "$ref": "#/definitions/HubAppProperties", - "name": "app" - }, - { - "type": "string", - "name": "resourceType" - }, - { - "type": "bool", - "nullable": true, - "name": "forceAppTags" - } - ], - "output": { - "type": "object", - "value": "[union(if(or(parameters('app').hub.options.publisherIsolation, coalesce(parameters('forceAppTags'), false())), parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" - }, - "metadata": { - "description": "Returns a tags dictionary that includes tags for the FinOps hub app.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "getPublisherTags": { - "parameters": [ - { - "$ref": "#/definitions/HubAppProperties", - "name": "app" - }, - { - "type": "string", - "name": "resourceType" - } - ], - "output": { - "type": "object", - "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').publisher.tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" - }, - "metadata": { - "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "newApp": { - "parameters": [ - { - "$ref": "#/definitions/HubProperties", - "name": "hub" - }, - { - "type": "string", - "name": "publisherDisplayName" - }, - { - "type": "string", - "name": "publisherName" - }, - { - "type": "string", - "name": "appPartialName" - }, - { - "type": "string", - "name": "appDisplayName" - }, - { - "type": "string", - "name": "version" - } - ], - "output": { - "$ref": "#/definitions/HubAppProperties", - "value": "[_1.newAppInternal(parameters('hub'), parameters('publisherName'), parameters('publisherDisplayName'), if(or(not(parameters('hub').options.publisherIsolation), equals(parameters('publisherName'), 'Microsoft.FinOpsHubs')), parameters('hub').core.suffix, uniqueString(parameters('publisherName'))), createObject('ftk-hubapp-publisher', parameters('publisherName')), format('{0}.{1}', parameters('publisherName'), parameters('appPartialName')), parameters('appDisplayName'), parameters('version'))]" - }, - "metadata": { - "description": "Creates a new FinOps hub app configuration object.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - } - }, - { - "namespace": "_1", - "members": { - "newAppInternal": { + "getHubTags": { "parameters": [ { "$ref": "#/definitions/HubProperties", @@ -2486,71 +1857,17 @@ }, { "type": "string", - "name": "publisherName" - }, - { - "type": "string", - "name": "publisherDisplayName" - }, - { - "type": "string", - "name": "publisherSuffix" - }, - { - "type": "object", - "name": "publisherTags" - }, - { - "type": "string", - "name": "appName" - }, - { - "type": "string", - "name": "appDisplayName" - }, - { - "type": "string", - "name": "version" - } - ], - "output": { - "$ref": "#/definitions/HubAppProperties", - "value": { - "name": "[parameters('appName')]", - "displayName": "[parameters('appDisplayName')]", - "tags": "[union(parameters('hub').tags, parameters('publisherTags'), createObject('ftk-hubapp', parameters('appName'), 'ftk-hubapp-version', parameters('version')))]", - "publisher": { - "name": "[parameters('publisherName')]", - "displayName": "[parameters('publisherDisplayName')]", - "suffix": "[parameters('publisherSuffix')]", - "tags": "[union(parameters('hub').tags, parameters('publisherTags'))]" - }, - "hub": "[parameters('hub')]", - "dataFactory": "[replace(format('{0}-{1}', take(format('{0}-engine', replace(parameters('hub').name, '_', '-')), sub(sub(63, length(parameters('publisherSuffix'))), 1)), parameters('publisherSuffix')), '--', '-')]", - "keyVault": "[replace(format('{0}-{1}', take(format('{0}-vault', replace(parameters('hub').name, '_', '-')), sub(sub(24, length(parameters('publisherSuffix'))), 1)), parameters('publisherSuffix')), '--', '-')]", - "storage": "[format('{0}{1}', take(_1.safeStorageName(parameters('hub').name), sub(24, length(parameters('publisherSuffix')))), parameters('publisherSuffix'))]" - } - }, - "metadata": { - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "safeStorageName": { - "parameters": [ - { - "type": "string", - "name": "name" + "name": "resourceType" } ], "output": { - "type": "string", - "value": "[replace(replace(toLower(parameters('name')), '-', ''), '_', '')]" + "type": "object", + "value": "[union(parameters('hub').tags, coalesce(tryGet(parameters('hub').tagsByResource, parameters('resourceType')), createObject()))]" }, "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub instance.", "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "sourceTemplate": "../../fx/hub-types.bicep" } } } @@ -2563,391 +1880,411 @@ "metadata": { "description": "Required. FinOps hub instance properties." } - }, - "publisher": { - "type": "string", - "metadata": { - "description": "Required. Display name of the FinOps hub app publisher." - } - }, - "namespace": { - "type": "string", - "metadata": { - "description": "Required. Namespace to use for the FinOps hub app publisher. Will be combined with appName to form a fully-qualified identifier. Must be an alphanumeric string without spaces or special characters except for periods. This value should never change and will be used to uniquely identify the publisher. A change would require migrating content to the new publisher. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." - } - }, - "appName": { - "type": "string", - "metadata": { - "description": "Required. Unique identifier of the FinOps hub app within the publisher namespace. Must be an alphanumeric string without spaces or special characters. This name should never change and will be used with the namespace to fully qualify the app. A change would require migrating content to the new app. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." - } - }, - "displayName": { - "type": "string", - "metadata": { - "description": "Required. Display name of the FinOps hub app." - } - }, - "appVersion": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Version number of the FinOps hub app." - } - }, - "features": { - "type": "array", - "items": { - "$ref": "#/definitions/HubAppFeature" - }, - "defaultValue": [], - "metadata": { - "description": "Optional. Indicate which features the app requires. Allowed values: \"Storage\". Default: [] (none)." - } - }, - "telemetryString": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." - } } }, "variables": { - "app": "[__bicep.newApp(parameters('hub'), parameters('publisher'), parameters('namespace'), parameters('appName'), parameters('displayName'), parameters('appVersion'))]", - "usesDataFactory": "[contains(parameters('features'), 'DataFactory')]", - "usesKeyVault": "[contains(parameters('features'), 'KeyVault')]", - "usesStorage": "[contains(parameters('features'), 'Storage')]", - "telemetryId": "[format('ftk-hubapp-{0}{1}{2}', variables('app').name, if(empty(parameters('telemetryString')), '', '_'), parameters('telemetryString'))]", - "telemetryProps": { - "mode": "Incremental", - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "[format('FTK: {0} - {1} {2}', parameters('publisher'), parameters('displayName'), variables('telemetryId'))]", - "version": "[parameters('appVersion')]" - } - }, - "resources": [] - } - }, - "storageInfrastructureEncryptionProperties": "[if(not(parameters('hub').options.storageInfrastructureEncryption), createObject(), createObject('encryption', createObject('keySource', 'Microsoft.Storage', 'requireInfrastructureEncryption', parameters('hub').options.storageInfrastructureEncryption)))]" + "nsgName": "[format('{0}-nsg', parameters('hub').routing.networkName)]", + "finopsHubSubnetName": "private-endpoint-subnet", + "scriptSubnetName": "script-subnet", + "dataExplorerSubnetName": "dataExplorer-subnet", + "subnets": "[if(not(parameters('hub').options.privateRouting), createArray(), createArray(createObject('name', variables('finopsHubSubnetName'), 'properties', createObject('addressPrefix', cidrSubnet(parameters('hub').options.networkAddressPrefix, 28, 0), 'networkSecurityGroup', createObject('id', resourceId('Microsoft.Network/networkSecurityGroups', variables('nsgName'))), 'serviceEndpoints', createArray(createObject('service', 'Microsoft.Storage')))), createObject('name', variables('scriptSubnetName'), 'properties', createObject('addressPrefix', cidrSubnet(parameters('hub').options.networkAddressPrefix, 28, 1), 'networkSecurityGroup', createObject('id', resourceId('Microsoft.Network/networkSecurityGroups', variables('nsgName'))), 'delegations', createArray(createObject('name', 'Microsoft.ContainerInstance/containerGroups', 'properties', createObject('serviceName', 'Microsoft.ContainerInstance/containerGroups'))), 'serviceEndpoints', createArray(createObject('service', 'Microsoft.Storage')))), createObject('name', variables('dataExplorerSubnetName'), 'properties', createObject('addressPrefix', cidrSubnet(parameters('hub').options.networkAddressPrefix, 27, 1), 'networkSecurityGroup', createObject('id', resourceId('Microsoft.Network/networkSecurityGroups', variables('nsgName')))))))]" }, "resources": { - "storageAccount::blobService": { - "condition": "[variables('usesStorage')]", - "type": "Microsoft.Storage/storageAccounts/blobServices", - "apiVersion": "2022-09-01", - "name": "[format('{0}/{1}', variables('app').storage, 'default')]", + "vNet::finopsHubSubnet": { + "condition": "[parameters('hub').options.privateRouting]", + "existing": true, + "type": "Microsoft.Network/virtualNetworks/subnets", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', parameters('hub').routing.networkName, variables('finopsHubSubnetName'))]", "dependsOn": [ - "storageAccount" + "vNet" ] }, - "blobEndpoint::blobPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "vNet::scriptSubnet": { + "condition": "[parameters('hub').options.privateRouting]", + "existing": true, + "type": "Microsoft.Network/virtualNetworks/subnets", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', parameters('hub').routing.networkName, variables('scriptSubnetName'))]", + "dependsOn": [ + "vNet" + ] + }, + "vNet::dataExplorerSubnet": { + "condition": "[parameters('hub').options.privateRouting]", + "existing": true, + "type": "Microsoft.Network/virtualNetworks/subnets", "apiVersion": "2023-11-01", - "name": "[format('{0}/{1}', format('{0}-blob-ep', variables('app').storage), 'storage-endpoint-zone')]", + "name": "[format('{0}/{1}', parameters('hub').routing.networkName, variables('dataExplorerSubnetName'))]", + "dependsOn": [ + "vNet" + ] + }, + "blobPrivateDnsZone::blobPrivateDnsZoneLink": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', string(parameters('hub').routing.dnsZones.blob.name), format('{0}-link', replace(string(parameters('hub').routing.dnsZones.blob.name), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", "properties": { - "privateDnsZoneConfigs": [ - { - "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]", - "properties": { - "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" - } - } - ] + "registrationEnabled": false, + "virtualNetwork": { + "id": "[parameters('hub').routing.networkId]" + } }, "dependsOn": [ - "blobEndpoint" + "blobPrivateDnsZone" ] }, - "dfsEndpoint::dfsPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", - "apiVersion": "2023-11-01", - "name": "[format('{0}/{1}', format('{0}-dfs-ep', variables('app').storage), 'dfs-endpoint-zone')]", + "dfsPrivateDnsZone::dfsPrivateDnsZoneLink": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', string(parameters('hub').routing.dnsZones.dfs.name), format('{0}-link', replace(string(parameters('hub').routing.dnsZones.dfs.name), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", "properties": { - "privateDnsZoneConfigs": [ - { - "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]", - "properties": { - "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.dfs.{0}', environment().suffixes.storage))]" - } - } - ] + "registrationEnabled": false, + "virtualNetwork": { + "id": "[parameters('hub').routing.networkId]" + } }, "dependsOn": [ - "dfsEndpoint" + "dfsPrivateDnsZone" ] }, - "keyVault::keyVault_accessPolicies": { - "condition": "[variables('usesKeyVault')]", - "type": "Microsoft.KeyVault/vaults/accessPolicies", - "apiVersion": "2023-02-01", - "name": "[format('{0}/{1}', variables('app').keyVault, 'add')]", + "queuePrivateDnsZone::queuePrivateDnsZoneLink": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', string(parameters('hub').routing.dnsZones.queue.name), format('{0}-link', replace(string(parameters('hub').routing.dnsZones.queue.name), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", "properties": { - "accessPolicies": [ - { - "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", - "tenantId": "[subscription().tenantId]", - "permissions": { - "secrets": [ - "get" - ] - } - } - ] + "registrationEnabled": false, + "virtualNetwork": { + "id": "[parameters('hub').routing.networkId]" + } }, "dependsOn": [ - "dataFactory", - "keyVault" + "queuePrivateDnsZone" ] }, - "keyVaultPrivateDnsZone::keyVaultPrivateDnsZoneLink": { - "condition": "[and(variables('usesKeyVault'), parameters('hub').options.privateRouting)]", + "tablePrivateDnsZone::tablePrivateDnsZoneLink": { + "condition": "[parameters('hub').options.privateRouting]", "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", "apiVersion": "2024-06-01", - "name": "[format('{0}/{1}', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), format('{0}-link', replace(format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), '.', '-')))]", + "name": "[format('{0}/{1}', string(parameters('hub').routing.dnsZones.table.name), format('{0}-link', replace(string(parameters('hub').routing.dnsZones.table.name), '.', '-')))]", "location": "global", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", "properties": { + "registrationEnabled": false, "virtualNetwork": { "id": "[parameters('hub').routing.networkId]" - }, - "registrationEnabled": false + } }, "dependsOn": [ - "keyVaultPrivateDnsZone" + "tablePrivateDnsZone" ] }, - "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { - "condition": "[and(variables('usesKeyVault'), parameters('hub').options.privateRouting)]", + "scriptEndpoint::scriptPrivateDnsZoneGroup": { + "condition": "[parameters('hub').options.privateRouting]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", - "name": "[format('{0}/{1}', format('{0}-ep', variables('app').keyVault), 'keyvault-endpoint-zone')]", + "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('hub').routing.scriptStorage), 'blob-endpoint-zone')]", "properties": { "privateDnsZoneConfigs": [ { - "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "name": "[string(parameters('hub').routing.dnsZones.blob.name)]", "properties": { - "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')))]" + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', string(parameters('hub').routing.dnsZones.blob.name))]" } } ] }, "dependsOn": [ - "keyVaultEndpoint", - "keyVaultPrivateDnsZone" + "blobPrivateDnsZone", + "scriptEndpoint" ] }, - "appTelemetry": { - "condition": "[parameters('hub').options.enableTelemetry]", - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "[if(lessOrEquals(length(variables('telemetryId')), 64), variables('telemetryId'), substring(variables('telemetryId'), 0, 64))]", - "tags": "[__bicep.getAppTags(variables('app'), 'Microsoft.Resources/deployments', true())]", - "properties": "[variables('telemetryProps')]" - }, - "dataFactory": { - "condition": "[variables('usesDataFactory')]", - "type": "Microsoft.DataFactory/factories", - "apiVersion": "2018-06-01", - "name": "[variables('app').dataFactory]", - "location": "[variables('app').hub.location]", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.DataFactory/factories')]", - "identity": { - "type": "SystemAssigned" - }, + "nsg": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/networkSecurityGroups", + "apiVersion": "2023-11-01", + "name": "[variables('nsgName')]", + "location": "[parameters('hub').location]", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/networkSecurityGroups')]", "properties": { - "globalConfigurations": { - "PipelineBillingEnabled": "true" - } + "securityRules": [ + { + "name": "AllowVnetInBound", + "properties": { + "priority": 100, + "direction": "Inbound", + "access": "Allow", + "protocol": "*", + "sourcePortRange": "*", + "destinationPortRange": "*", + "sourceAddressPrefix": "VirtualNetwork", + "destinationAddressPrefix": "VirtualNetwork" + } + }, + { + "name": "AllowAzureLoadBalancerInBound", + "properties": { + "priority": 200, + "direction": "Inbound", + "access": "Allow", + "protocol": "*", + "sourcePortRange": "*", + "destinationPortRange": "*", + "sourceAddressPrefix": "AzureLoadBalancer", + "destinationAddressPrefix": "*" + } + }, + { + "name": "DenyAllInBound", + "properties": { + "priority": 4096, + "direction": "Inbound", + "access": "Deny", + "protocol": "*", + "sourcePortRange": "*", + "destinationPortRange": "*", + "sourceAddressPrefix": "*", + "destinationAddressPrefix": "*" + } + }, + { + "name": "AllowVnetOutBound", + "properties": { + "priority": 100, + "direction": "Outbound", + "access": "Allow", + "protocol": "*", + "sourcePortRange": "*", + "destinationPortRange": "*", + "sourceAddressPrefix": "VirtualNetwork", + "destinationAddressPrefix": "VirtualNetwork" + } + }, + { + "name": "AllowInternetOutBound", + "properties": { + "priority": 200, + "direction": "Outbound", + "access": "Allow", + "protocol": "*", + "sourcePortRange": "*", + "destinationPortRange": "*", + "sourceAddressPrefix": "*", + "destinationAddressPrefix": "Internet" + } + }, + { + "name": "DenyAllOutBound", + "properties": { + "priority": 4096, + "direction": "Outbound", + "access": "Deny", + "protocol": "*", + "sourcePortRange": "*", + "destinationPortRange": "*", + "sourceAddressPrefix": "*", + "destinationAddressPrefix": "*" + } + } + ] } }, - "storageAccount": { - "condition": "[variables('usesStorage')]", - "type": "Microsoft.Storage/storageAccounts", - "apiVersion": "2022-09-01", - "name": "[variables('app').storage]", + "vNet": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/virtualNetworks", + "apiVersion": "2023-11-01", + "name": "[parameters('hub').routing.networkName]", "location": "[parameters('hub').location]", - "sku": { - "name": "[parameters('hub').options.storageSku]" + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/virtualNetworks')]", + "properties": { + "addressSpace": { + "addressPrefixes": [ + "[parameters('hub').options.networkAddressPrefix]" + ] + }, + "subnets": "[variables('subnets')]" }, - "kind": "BlockBlobStorage", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Storage/storageAccounts')]", - "properties": "[shallowMerge(createArray(variables('storageInfrastructureEncryptionProperties'), createObject('supportsHttpsTrafficOnly', true(), 'allowSharedKeyAccess', true(), 'isHnsEnabled', true(), 'minimumTlsVersion', 'TLS1_2', 'allowBlobPublicAccess', false(), 'publicNetworkAccess', 'Enabled', 'networkAcls', createObject('bypass', 'AzureServices', 'defaultAction', if(parameters('hub').options.privateRouting, 'Deny', 'Allow')))))]" + "dependsOn": [ + "nsg" + ] }, "blobPrivateDnsZone": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "existing": true, + "condition": "[parameters('hub').options.privateRouting]", "type": "Microsoft.Network/privateDnsZones", "apiVersion": "2024-06-01", - "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]" + "name": "[string(parameters('hub').routing.dnsZones.blob.name)]", + "location": "global", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/privateDnsZones')]", + "properties": {}, + "dependsOn": [ + "vNet" + ] }, - "blobEndpoint": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints", - "apiVersion": "2023-11-01", - "name": "[format('{0}-blob-ep', variables('app').storage)]", + "dfsPrivateDnsZone": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[string(parameters('hub').routing.dnsZones.dfs.name)]", + "location": "global", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/privateDnsZones')]", + "properties": {}, + "dependsOn": [ + "vNet" + ] + }, + "queuePrivateDnsZone": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[string(parameters('hub').routing.dnsZones.queue.name)]", + "location": "global", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/privateDnsZones')]", + "properties": {}, + "dependsOn": [ + "vNet" + ] + }, + "tablePrivateDnsZone": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[string(parameters('hub').routing.dnsZones.table.name)]", + "location": "global", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/privateDnsZones')]", + "properties": {}, + "dependsOn": [ + "vNet" + ] + }, + "scriptStorageAccount": { + "condition": "[parameters('hub').options.privateRouting]", + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('hub').routing.scriptStorage]", "location": "[parameters('hub').location]", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Network/privateEndpoints')]", + "sku": { + "name": "Standard_LRS" + }, + "kind": "StorageV2", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Storage/storageAccounts')]", "properties": { - "subnet": { - "id": "[parameters('hub').routing.subnets.storage]" - }, - "privateLinkServiceConnections": [ - { - "name": "blobLink", - "properties": { - "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', variables('app').storage)]", - "groupIds": [ - "blob" - ] + "supportsHttpsTrafficOnly": true, + "allowSharedKeyAccess": true, + "isHnsEnabled": false, + "minimumTlsVersion": "TLS1_2", + "allowBlobPublicAccess": false, + "publicNetworkAccess": "Enabled", + "networkAcls": { + "bypass": "AzureServices", + "defaultAction": "Deny", + "virtualNetworkRules": [ + { + "id": "[parameters('hub').routing.subnets.scripts]", + "action": "Allow" } - } - ] + ] + } }, "dependsOn": [ - "storageAccount" + "vNet::scriptSubnet" ] }, - "dfsPrivateDnsZone": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "existing": true, - "type": "Microsoft.Network/privateDnsZones", - "apiVersion": "2024-06-01", - "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]" - }, - "dfsEndpoint": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", + "scriptEndpoint": { + "condition": "[parameters('hub').options.privateRouting]", "type": "Microsoft.Network/privateEndpoints", "apiVersion": "2023-11-01", - "name": "[format('{0}-dfs-ep', variables('app').storage)]", + "name": "[format('{0}-blob-ep', parameters('hub').routing.scriptStorage)]", "location": "[parameters('hub').location]", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Network/privateEndpoints')]", + "tags": "[__bicep.getHubTags(parameters('hub'), 'Microsoft.Network/privateEndpoints')]", "properties": { "subnet": { "id": "[parameters('hub').routing.subnets.storage]" }, "privateLinkServiceConnections": [ { - "name": "dfsLink", + "name": "scriptLink", "properties": { - "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', variables('app').storage)]", + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('hub').routing.scriptStorage)]", "groupIds": [ - "dfs" + "blob" ] } } ] }, "dependsOn": [ - "storageAccount" + "scriptStorageAccount", + "vNet::scriptSubnet" ] + } + }, + "outputs": { + "config": { + "$ref": "#/definitions/HubProperties", + "metadata": { + "description": "FinOps hub configuration settings." + }, + "value": "[parameters('hub')]" }, - "keyVault": { - "condition": "[variables('usesKeyVault')]", - "type": "Microsoft.KeyVault/vaults", - "apiVersion": "2023-02-01", - "name": "[variables('app').keyVault]", - "location": "[parameters('hub').location]", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.KeyVault/vaults')]", - "properties": { - "sku": { - "name": "[parameters('hub').options.keyVaultSku]", - "family": "A" - }, - "enabledForDeployment": true, - "enabledForTemplateDeployment": true, - "enabledForDiskEncryption": true, - "enableSoftDelete": true, - "softDeleteRetentionInDays": 90, - "enableRbacAuthorization": false, - "createMode": "default", - "tenantId": "[subscription().tenantId]", - "accessPolicies": [ - { - "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", - "tenantId": "[subscription().tenantId]", - "permissions": { - "secrets": [ - "get" - ] - } - } - ], - "networkAcls": { - "bypass": "AzureServices", - "defaultAction": "[if(parameters('hub').options.privateRouting, 'Deny', 'Allow')]" - } + "vNetId": { + "type": "string", + "metadata": { + "description": "Resource ID of the virtual network." }, - "dependsOn": [ - "dataFactory" - ] + "value": "[if(not(parameters('hub').options.privateRouting), '', resourceId('Microsoft.Network/virtualNetworks', parameters('hub').routing.networkName))]" }, - "keyVaultPrivateDnsZone": { - "condition": "[and(variables('usesKeyVault'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateDnsZones", - "apiVersion": "2024-06-01", - "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", - "location": "global", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Network/privateDnsZones')]", - "properties": {} + "vNetAddressSpace": { + "type": "array", + "metadata": { + "description": "Virtual network address prefixes." + }, + "value": "[if(not(parameters('hub').options.privateRouting), createArray(), reference('vNet').addressSpace.addressPrefixes)]" }, - "keyVaultEndpoint": { - "condition": "[and(variables('usesKeyVault'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints", - "apiVersion": "2023-11-01", - "name": "[format('{0}-ep', variables('app').keyVault)]", - "location": "[parameters('hub').location]", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Network/privateEndpoints')]", - "properties": { - "subnet": { - "id": "[parameters('hub').routing.subnets.keyVault]" - }, - "privateLinkServiceConnections": [ - { - "name": "keyVaultLink", - "properties": { - "privateLinkServiceId": "[resourceId('Microsoft.KeyVault/vaults', variables('app').keyVault)]", - "groupIds": [ - "vault" - ] - } - } - ] + "vNetSubnets": { + "type": "array", + "metadata": { + "description": "Virtual network subnets." }, - "dependsOn": [ - "keyVault" - ] - } - }, - "outputs": { - "app": { - "$ref": "#/definitions/HubAppProperties", + "value": "[if(not(parameters('hub').options.privateRouting), createArray(), reference('vNet').subnets)]" + }, + "finopsHubSubnetId": { + "type": "string", "metadata": { - "description": "FinOps hub app configuration." + "description": "Resource ID of the FinOps hub network subnet." }, - "value": "[variables('app')]" + "value": "[if(not(parameters('hub').options.privateRouting), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('hub').routing.networkName, variables('finopsHubSubnetName')))]" }, - "principalId": { + "scriptSubnetId": { "type": "string", "metadata": { - "description": "Principal ID for the managed identity used by Data Factory." + "description": "Resource ID of the script storage account network subnet." }, - "value": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" + "value": "[if(not(parameters('hub').options.privateRouting), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('hub').routing.networkName, variables('scriptSubnetName')))]" + }, + "dataExplorerSubnetId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Data Explorer network subnet." + }, + "value": "[if(not(parameters('hub').options.privateRouting), '', resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('hub').routing.networkName, variables('dataExplorerSubnetName')))]" } } } } }, - "configContainer": { + "appRegistration": { "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.FinOpsHubs.Core_Storage.ConfigContainer", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core_Register", "properties": { "expressionEvaluationOptions": { "scope": "inner" @@ -2955,13 +2292,16 @@ "mode": "Incremental", "parameters": { "app": { - "value": "[reference('appRegistration').outputs.app.value]" + "value": "[parameters('app')]" }, - "container": { - "value": "config" + "version": { + "value": "[variables('finOpsToolkitVersion')]" }, - "forceCreateBlobManagerIdentity": { - "value": true + "features": { + "value": [ + "DataFactory", + "Storage" + ] } }, "template": { @@ -2971,8 +2311,8 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "13960345490822271084" + "version": "0.40.2.10011", + "templateHash": "8267413868206701537" } }, "definitions": { @@ -3006,6 +2346,9 @@ "keyVaultSku": { "type": "string" }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, "networkAddressPrefix": { "type": "string" }, @@ -3045,6 +2388,7 @@ "options": { "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", "privateRouting": "Indicates whether private network routing is enabled.", "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", @@ -3094,6 +2438,9 @@ "subnets": { "type": "object", "properties": { + "dataExplorer": { + "type": "string" + }, "dataFactory": { "type": "string" }, @@ -3120,6 +2467,7 @@ "table": "Resource ID and name for the table storage DNS zone." }, "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", "dataFactory": "Resource ID of the subnet for Data Factory instances.", "keyVault": "Resource ID of the subnet for Key Vault instances.", "scripts": "Resource ID of the subnet for deployment script storage.", @@ -3150,38 +2498,38 @@ } } }, + "HubAppFeature": { + "type": "string", + "allowedValues": [ + "DataFactory", + "KeyVault", + "Storage" + ], + "metadata": { + "description": "FinOps hub app features.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, "HubAppProperties": { "type": "object", "properties": { + "id": { + "type": "string" + }, "name": { "type": "string" }, - "displayName": { + "publisher": { + "type": "string" + }, + "suffix": { "type": "string" }, "tags": { "type": "object" }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, "dataFactory": { "type": "string" }, @@ -3190,22 +2538,21 @@ }, "storage": { "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" } }, "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", "dataFactory": "Name of the Data Factory instance for this publisher.", "keyVault": "Name of the KeyVault instance for this publisher.", "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", "description": "FinOps hub app configuration settings.", "__bicep_imported_from!": { "sourceTemplate": "hub-types.bicep" @@ -3213,569 +2560,949 @@ } } }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } + } + ], "parameters": { "app": { "$ref": "#/definitions/HubAppProperties", "metadata": { - "description": "Required. FinOps hub app that storage is getting updated for." + "description": "Required. FinOps hub app getting deployed." } }, - "container": { + "version": { "type": "string", "metadata": { - "description": "Required. Name of the storage container to create or update." + "description": "Required. Version number of the FinOps hub app." } }, - "files": { - "type": "object", - "defaultValue": {}, + "features": { + "type": "array", + "items": { + "$ref": "#/definitions/HubAppFeature" + }, + "defaultValue": [], "metadata": { - "description": "Optional. Dictionary of key/value pairs for the files to upload to the specified container. The key is the target path under the container and the value is the contents of the file. Default: {} (no files to upload)." + "description": "Optional. Indicate which features the app requires. Allowed values: \"DataFactory\", \"KeyVault\", \"Storage\". Default: [] (none)." } }, - "forceCreateBlobManagerIdentity": { - "type": "bool", - "defaultValue": false, + "storageRoles": { + "type": "array", + "items": { + "type": "string" + }, + "defaultValue": [], "metadata": { - "description": "Optional. Indicates whether to create the blob manager user assigned identity even if files are not being uploaded. Default: false." + "description": "Optional. Indicate which RBAC roles the Data Factory identity needs on the storage account, if created. This is in addition to Storage Blob Data Contributor for reading and managing content. Default: [] (none)." + } + }, + "telemetryString": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." } } }, "variables": { - "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n# Get storage context\r\n$storageContext = @{\r\n Context = New-AzStorageContext -StorageAccountName $env:storageAccountName -UseConnectedAccount\r\n Container = $env:containerName\r\n}\r\n\r\n# Uploading files\r\n$files = $env:files | ConvertFrom-Json -Depth 10\r\nWrite-Output \"Uploading ${$files.PSObject.Properties.Count} files...\"\r\n$files.PSObject.Properties | ForEach-Object {\r\n $filePath = $_.Name\r\n $tempPath = \"./$($filePath -replace \"/\", \"_\")\"\r\n Write-Output \" Uploading $filePath...\"\r\n $_.Value | Out-File $tempPath\r\n Set-AzStorageBlobContent @storageContext -File $tempPath -Blob $filePath -Force | Out-Null\r\n}\r\n", - "fileCount": "[length(items(parameters('files')))]", - "hasFiles": "[greater(variables('fileCount'), 0)]" + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n<#\r\n.SYNOPSIS\r\nManages Azure Data Factory triggers and pipelines during FinOps hub deployment.\r\n\r\n.DESCRIPTION\r\nThis script is called by Bicep deployment scripts to start/stop Data Factory triggers\r\nand optionally run pipelines. It handles two types of triggers:\r\n\r\n1. Schedule triggers - Can be started/stopped directly\r\n2. BlobEventsTriggers - Require Event Grid subscription management before start/stop\r\n\r\nBlobEventsTriggers use Event Grid to listen for storage blob events. Before stopping,\r\nthe Event Grid subscription must be removed (unsubscribed). Before starting, the\r\nsubscription must be added and fully provisioned. This script handles this automatically\r\nby detecting BlobEventsTriggers via the BlobPathBeginsWith property.\r\n\r\nThe script uses retry logic with linear backoff to handle transient API failures and\r\nwait for Event Grid subscription provisioning/deprovisioning to complete.\r\n\r\n.PARAMETER DataFactoryResourceGroup\r\nThe resource group containing the Data Factory.\r\n\r\n.PARAMETER DataFactoryName\r\nThe name of the Data Factory instance.\r\n\r\n.PARAMETER Pipelines\r\nPipe-delimited list of pipeline names to run (e.g., \"pipeline1|pipeline2\").\r\n\r\n.PARAMETER StartTriggers\r\nSwitch to start all stopped triggers (with Event Grid subscription for blob triggers).\r\n\r\n.PARAMETER StopTriggers\r\nSwitch to stop all running triggers (with Event Grid unsubscription for blob triggers).\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StopTriggers\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StartTriggers\r\n#>\r\n\r\nparam(\r\n [string] $DataFactoryResourceGroup,\r\n [string] $DataFactoryName,\r\n [string] $Pipelines = \"\",\r\n [switch] $StartTriggers,\r\n [switch] $StopTriggers\r\n)\r\n\r\n$MAX_RETRIES = 20\r\n$DeploymentScriptOutputs = @{}\r\n\r\nfunction Write-Log($Message)\r\n{\r\n Write-Output \"$(Get-Date -Format 'HH:mm:ss') $Message\"\r\n}\r\n\r\nfunction Invoke-WithRetry([scriptblock]$Action, [string]$Name, [int]$Delay = 5)\r\n{\r\n for ($i = 1; $i -le $MAX_RETRIES; $i++)\r\n {\r\n try { return & $Action }\r\n catch\r\n {\r\n Write-Log \"$Name failed (attempt $i/${MAX_RETRIES}): $($_.Exception.Message)\"\r\n if ($i -eq $MAX_RETRIES) { throw }\r\n Start-Sleep -Seconds ($Delay * $i)\r\n }\r\n }\r\n}\r\n\r\nfunction Set-BlobTriggerSubscription([string]$TriggerName, [switch]$Subscribe)\r\n{\r\n $targetStatus = if ($Subscribe) { 'Enabled' } else { 'Disabled' }\r\n $action = if ($Subscribe) { 'Subscribing' } else { 'Unsubscribing' }\r\n\r\n Write-Log \"$action $TriggerName to events...\"\r\n Invoke-WithRetry -Name \"$action $TriggerName\" -Delay 5 -Action {\r\n if ($Subscribe)\r\n {\r\n Add-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n else\r\n {\r\n Remove-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n\r\n $status = Get-AzDataFactoryV2TriggerSubscriptionStatus `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName\r\n if ($status.Status -ne $targetStatus)\r\n {\r\n throw \"Subscription status is $($status.Status), expected $targetStatus\"\r\n }\r\n }\r\n}\r\n\r\nif ($StartTriggers -or $StopTriggers)\r\n{\r\n $triggers = Invoke-WithRetry -Name \"Get triggers\" -Action {\r\n Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n | Where-Object {\r\n ($StartTriggers -and $_.Properties.RuntimeState -ne \"Started\") `\r\n -or ($StopTriggers -and $_.Properties.RuntimeState -ne \"Stopped\")\r\n }\r\n }\r\n\r\n Write-Log \"Found $($triggers.Count) trigger(s) to $(if ($StartTriggers) { 'start' } else { 'stop' })\"\r\n\r\n $triggers | ForEach-Object {\r\n $triggerName = $_.Name\r\n $isBlobTrigger = $null -ne $_.Properties.BlobPathBeginsWith\r\n\r\n if ($StopTriggers)\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName }\r\n Write-Log \"Stopping trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Stop $triggerName\" -Action {\r\n Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n else\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName -Subscribe }\r\n Write-Log \"Starting trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Start $triggerName\" -Action {\r\n Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n\r\n Invoke-WithRetry -Name \"Wait for $triggerName\" -Action {\r\n $state = (Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName).Properties.RuntimeState\r\n $expected = if ($StartTriggers) { 'Started' } else { 'Stopped' }\r\n if ($state -ne $expected) { throw \"Trigger is $state, expected $expected\" }\r\n }\r\n\r\n Write-Log \"...done\"\r\n $DeploymentScriptOutputs[$triggerName] = $true\r\n }\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($Pipelines))\r\n{\r\n $Pipelines.Split('|') | ForEach-Object {\r\n Write-Log \"Running pipeline $_...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", + "usesDataFactory": "[contains(parameters('features'), 'DataFactory')]", + "usesKeyVault": "[contains(parameters('features'), 'KeyVault')]", + "usesStorage": "[contains(parameters('features'), 'Storage')]", + "telemetryId": "[format('ftk-hubapp-{0}{1}{2}', parameters('app').id, if(empty(parameters('telemetryString')), '', '_'), parameters('telemetryString'))]", + "telemetryProps": { + "mode": "Incremental", + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "[format('FTK: {0}', parameters('app').id)]", + "version": "[parameters('version')]" + } + }, + "resources": [] + } + }, + "autoStartRbacRoles": [ + "673868aa-7521-48a0-acc6-0f60742d39f5" + ], + "factoryStorageRoles": "[union(parameters('storageRoles'), createArray('17d1049b-9a84-46fb-8f53-869881c3d3ab', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe', 'acdd72a7-3385-48ef-bd42-f606fba81ae7'))]", + "storageInfrastructureEncryptionProperties": "[if(not(parameters('app').hub.options.storageInfrastructureEncryption), createObject(), createObject('encryption', createObject('keySource', 'Microsoft.Storage', 'requireInfrastructureEncryption', parameters('app').hub.options.storageInfrastructureEncryption)))]" }, "resources": { - "storageAccount::blobService::targetContainer": { - "type": "Microsoft.Storage/storageAccounts/blobServices/containers", - "apiVersion": "2022-09-01", - "name": "[format('{0}/{1}/{2}', parameters('app').storage, 'default', parameters('container'))]", + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').storage)]", "properties": { - "publicAccess": "None", - "metadata": {} - } + "name": "[parameters('app').storage]", + "groupId": "dfs", + "privateLinkResourceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "fqdns": [ + "[reference('storageAccount').primaryEndpoints.dfs]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "storageAccount" + ] }, - "storageAccount::blobService": { - "existing": true, - "type": "Microsoft.Storage/storageAccounts/blobServices", - "apiVersion": "2022-09-01", - "name": "[format('{0}/{1}', parameters('app').storage, 'default')]" + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').keyVault)]", + "properties": { + "name": "[parameters('app').keyVault]", + "groupId": "vault", + "privateLinkResourceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "fqdns": [ + "[reference('keyVault').vaultUri]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "keyVault" + ] }, - "storageAccount": { - "existing": true, - "type": "Microsoft.Storage/storageAccounts", - "apiVersion": "2022-09-01", - "name": "[parameters('app').storage]" + "dataFactory::managedVirtualNetwork": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'default')]", + "properties": {}, + "dependsOn": [ + "dataFactory" + ] }, - "identity": { - "condition": "[or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity'))]", - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "[format('{0}.Identity', deployment().name)]", + "dataFactory::managedIntegrationRuntime": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/integrationRuntimes", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ManagedIntegrationRuntime')]", "properties": { - "expressionEvaluationOptions": { - "scope": "inner" + "type": "Managed", + "managedVirtualNetwork": { + "referenceName": "default", + "type": "ManagedVirtualNetworkReference" }, - "mode": "Incremental", - "parameters": { - "app": { - "value": "[parameters('app')]" - }, - "identityName": { - "value": "[format('{0}_blobManager', parameters('app').storage)]" - }, - "roleAssignmentResourceId": { - "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" - }, - "roles": { - "value": [ - "ba92f5b4-2d11-453d-a403-e96b0029c9fe", - "69566ab7-960f-475b-8e7c-b3118f30c6bd" - ] + "typeProperties": { + "computeProperties": { + "location": "[parameters('app').hub.location]", + "dataFlowProperties": { + "computeType": "General", + "coreCount": 8, + "timeToLive": 10, + "cleanup": false, + "customProperties": [] + }, + "copyComputeScaleProperties": { + "dataIntegrationUnit": 16, + "timeToLive": 30 + }, + "pipelineExternalComputeScaleProperties": { + "timeToLive": 30, + "numberOfPipelineNodes": 1, + "numberOfExternalNodes": 1 + } } + } + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedVirtualNetwork" + ] + }, + "dataFactory::linkedService_keyVault": { + "condition": "[and(variables('usesDataFactory'), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').keyVault)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureKeyVault", + "typeProperties": { + "baseUrl": "[reference(format('Microsoft.KeyVault/vaults/{0}', parameters('app').keyVault), '2023-02-01').vaultUri]" }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "languageVersion": "2.0", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "4534337491931150093" + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "keyVault" + ] + }, + "dataFactory::linkedService_storageAccount": { + "condition": "[and(variables('usesDataFactory'), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').storage)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureBlobFS", + "typeProperties": { + "url": "[reference(format('Microsoft.Storage/storageAccounts/{0}', parameters('app').storage), '2021-08-01').primaryEndpoints.dfs]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "storageAccount" + ] + }, + "storageAccount::blobService": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]", + "dependsOn": [ + "storageAccount" + ] + }, + "blobEndpoint::blobPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" } - }, - "definitions": { - "_1.HubProperties": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - }, - "location": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "tagsByResource": { - "type": "object" - }, - "version": { - "type": "string" - }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" - }, - "keyVaultSku": { - "type": "string" - }, - "networkAddressPrefix": { - "type": "string" - }, - "privateRouting": { - "type": "bool" - }, - "publisherIsolation": { - "type": "bool" - }, - "storageInfrastructureEncryption": { - "type": "bool" - }, - "storageSku": { - "type": "string" - } - } - }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" - }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" - } - } - } - }, - "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", - "location": "Azure resource location of the FinOps hub instance.", - "tags": "Tags to apply to all FinOps hub resources.", - "tagsByResource": "Tags to apply to resources based on their resource type.", - "version": "FinOps hub version number.", - "options": { - "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", - "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", - "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", - "privateRouting": "Indicates whether private network routing is enabled.", - "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", - "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", - "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." - }, - "routing": "FinOps hub private network routing properties, if enabled.", - "core": { - "suffix": "Unique suffix used for shared resources." - }, - "apps": {}, - "description": "FinOps hub instance properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.HubRoutingProperties": { - "type": "object", - "properties": { - "networkId": { - "type": "string" - }, - "networkName": { - "type": "string" - }, - "scriptStorage": { - "type": "string" - }, - "dnsZones": { - "type": "object", - "properties": { - "blob": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "dfs": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "queue": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "table": { - "$ref": "#/definitions/_1.IdNameObject" - } - } - }, - "subnets": { - "type": "object", - "properties": { - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "scripts": { - "type": "string" - }, - "storage": { - "type": "string" - } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubAppProperties": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "storage": { - "type": "string" - } - }, - "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", - "dataFactory": "Name of the Data Factory instance for this publisher.", - "keyVault": "Name of the KeyVault instance for this publisher.", - "storage": "Name of the storage account for this publisher.", - "description": "FinOps hub app configuration settings.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - }, - "functions": [ - { - "namespace": "__bicep", - "members": { - "getPublisherTags": { - "parameters": [ - { - "$ref": "#/definitions/HubAppProperties", - "name": "app" - }, - { - "type": "string", - "name": "resourceType" - } - ], - "output": { - "type": "object", - "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').publisher.tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" - }, - "metadata": { - "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - } - } - ], - "parameters": { - "app": { - "$ref": "#/definitions/HubAppProperties", - "metadata": { - "description": "Required. FinOps hub app the identity is associated with." - } - }, - "identityName": { - "type": "string", - "metadata": { - "description": "Required. Name of the user assigned identity." - } - }, - "roleAssignmentResourceId": { - "type": "string", - "metadata": { - "description": "Required. Resource ID of the resource access is being granted for." - } - }, - "roles": { - "type": "array", - "items": { - "type": "string" - }, - "metadata": { - "description": "Required. List of RBAC role assignment GUIDs." - } - } - }, - "resources": { - "identity": { - "type": "Microsoft.ManagedIdentity/userAssignedIdentities", - "apiVersion": "2023-01-31", - "name": "[parameters('identityName')]", - "tags": "[__bicep.getPublisherTags(parameters('app'), 'Microsoft.ManagedIdentity/userAssignedIdentities')]", - "location": "[parameters('app').hub.location]" - }, - "identityRoleAssignments": { - "copy": { - "name": "identityRoleAssignments", - "count": "[length(parameters('roles'))]" - }, - "type": "Microsoft.Authorization/roleAssignments", - "apiVersion": "2022-04-01", - "name": "[guid(parameters('roleAssignmentResourceId'), parameters('roles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", - "properties": { - "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', parameters('roles')[copyIndex()])]", - "principalId": "[reference('identity').principalId]", - "principalType": "ServicePrincipal" - }, - "dependsOn": [ - "identity" - ] + } + ] + }, + "dependsOn": [ + "blobEndpoint" + ] + }, + "dfsEndpoint::dfsPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.dfs.{0}', environment().suffixes.storage))]" } - }, - "outputs": { - "id": { - "type": "string", - "metadata": { - "description": "Resource ID of the user assigned identity." - }, - "value": "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName'))]" - }, - "name": { - "type": "string", - "metadata": { - "description": "Name of the user assigned identity." - }, - "value": "[parameters('identityName')]" - }, - "principalId": { - "type": "string", - "metadata": { - "description": "Principal ID of the user assigned identity." - }, - "value": "[reference('identity').principalId]" + } + ] + }, + "dependsOn": [ + "dfsEndpoint" + ] + }, + "keyVaultPrivateDnsZone::keyVaultPrivateDnsZoneLink": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), format('{0}-link', replace(format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "properties": { + "virtualNetwork": { + "id": "[parameters('app').hub.routing.networkId]" + }, + "registrationEnabled": false + }, + "dependsOn": [ + "keyVaultPrivateDnsZone" + ] + }, + "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')))]" } } - } - } + ] + }, + "dependsOn": [ + "keyVaultEndpoint", + "keyVaultPrivateDnsZone" + ] }, - "uploadFiles": { - "condition": "[variables('hasFiles')]", + "appTelemetry": { + "condition": "[parameters('app').hub.options.enableTelemetry]", "type": "Microsoft.Resources/deployments", "apiVersion": "2022-09-01", - "name": "[format('{0}.Upload', deployment().name)]", + "name": "[if(lessOrEquals(length(variables('telemetryId')), 64), variables('telemetryId'), substring(variables('telemetryId'), 0, 64))]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Resources/deployments')]", + "properties": "[variables('telemetryProps')]" + }, + "dataFactory": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.DataFactory/factories')]", + "identity": { + "type": "SystemAssigned" + }, "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "app": { - "value": "[parameters('app')]" - }, - "identityName": { - "value": "[reference('identity').outputs.name.value]" - }, - "environmentVariables": { - "value": [ - { - "name": "storageAccountName", - "value": "[parameters('app').storage]" - }, - { - "name": "containerName", - "value": "[parameters('container')]" - }, - { - "name": "files", - "value": "[string(parameters('files'))]" - } - ] - }, - "scriptContent": { - "value": "[variables('$fxv#0')]" - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "languageVersion": "2.0", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "588615643779078900" - } - }, - "definitions": { - "EnvironmentVariable": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "value": { - "type": "string" - } + "globalConfigurations": { + "PipelineBillingEnabled": "true" + } + } + }, + "storageRoleAssignments": { + "copy": { + "name": "storageRoleAssignments", + "count": "[length(variables('factoryStorageRoles'))]" + }, + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "name": "[guid(resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage), variables('factoryStorageRoles')[copyIndex()], resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('factoryStorageRoles')[copyIndex()])]", + "principalId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "storageAccount" + ] + }, + "triggerManagerIdentity": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[format('{0}_triggerManager', parameters('app').dataFactory)]", + "location": "[parameters('app').hub.location]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "dependsOn": [ + "dataFactory" + ] + }, + "triggerManagerRoleAssignments": { + "copy": { + "name": "triggerManagerRoleAssignments", + "count": "[length(variables('autoStartRbacRoles'))]" + }, + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]", + "name": "[guid(resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory), variables('autoStartRbacRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', format('{0}_triggerManager', parameters('app').dataFactory)))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('autoStartRbacRoles')[copyIndex()])]", + "principalId": "[reference('triggerManagerIdentity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "triggerManagerIdentity" + ] + }, + "storageAccount": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]", + "location": "[parameters('app').hub.location]", + "sku": { + "name": "[parameters('app').hub.options.storageSku]" + }, + "kind": "BlockBlobStorage", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Storage/storageAccounts')]", + "properties": "[shallowMerge(createArray(variables('storageInfrastructureEncryptionProperties'), createObject('supportsHttpsTrafficOnly', true(), 'allowSharedKeyAccess', true(), 'isHnsEnabled', true(), 'minimumTlsVersion', 'TLS1_2', 'allowBlobPublicAccess', false(), 'publicNetworkAccess', 'Enabled', 'networkAcls', createObject('bypass', 'AzureServices', 'defaultAction', if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')))))]" + }, + "blobPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]" + }, + "blobEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-blob-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "blobLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "blob" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "dfsPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]" + }, + "dfsEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-dfs-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "dfsLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "dfs" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "keyVault": { + "condition": "[variables('usesKeyVault')]", + "type": "Microsoft.KeyVault/vaults", + "apiVersion": "2023-02-01", + "name": "[parameters('app').keyVault]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.KeyVault/vaults')]", + "properties": { + "sku": { + "name": "[parameters('app').hub.options.keyVaultSku]", + "family": "A" + }, + "enabledForDeployment": true, + "enabledForTemplateDeployment": true, + "enabledForDiskEncryption": true, + "enableSoftDelete": true, + "softDeleteRetentionInDays": 90, + "enablePurgeProtection": "[if(parameters('app').hub.options.keyVaultEnablePurgeProtection, true(), null())]", + "enableRbacAuthorization": false, + "createMode": "default", + "tenantId": "[subscription().tenantId]", + "accessPolicies": [ + { + "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "tenantId": "[subscription().tenantId]", + "permissions": { + "secrets": [ + "get" + ] + } + } + ], + "networkAcls": { + "bypass": "AzureServices", + "defaultAction": "[if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')]" + } + }, + "dependsOn": [ + "dataFactory" + ] + }, + "keyVaultPrivateDnsZone": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones')]", + "properties": {} + }, + "keyVaultEndpoint": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-ep', parameters('app').keyVault)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.keyVault]" + }, + "privateLinkServiceConnections": [ + { + "name": "keyVaultLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "groupIds": [ + "vault" + ] + } + } + ] + }, + "dependsOn": [ + "keyVault" + ] + }, + "getKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." } }, - "_1.HubProperties": { - "type": "object", + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - }, - "location": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "tagsByResource": { - "type": "object" - }, - "version": { - "type": "string" - }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" - }, - "keyVaultSku": { - "type": "string" - }, - "networkAddressPrefix": { - "type": "string" - }, - "privateRouting": { - "type": "bool" - }, - "publisherIsolation": { - "type": "bool" - }, - "storageInfrastructureEncryption": { - "type": "bool" - }, - "storageSku": { - "type": "string" - } - } - }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" - }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" - } - } + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" } - }, - "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint", + "getStoragePrivateEndpointConnections", + "keyVault" + ] + }, + "approveKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + }, + "privateEndpointConnections": { + "value": "[reference('getKeyVaultPrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getKeyVaultPrivateEndpointConnections", + "keyVault" + ] + }, + "getStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint", + "stopTriggers", + "storageAccount" + ] + }, + "approveStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + }, + "privateEndpointConnections": { + "value": "[reference('getStoragePrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getStoragePrivateEndpointConnections", + "storageAccount" + ] + }, + "stopTriggers": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.{1}_ADF.StopTriggers', parameters('app').publisher, parameters('app').name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "arguments": { + "value": "[join(createArray(format('-DataFactoryResourceGroup \"{0}\"', resourceGroup().name), format('-DataFactoryName \"{0}\"', parameters('app').dataFactory), '-StopTriggers'), ' ')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", "location": "Azure resource location of the FinOps hub instance.", "tags": "Tags to apply to all FinOps hub resources.", "tagsByResource": "Tags to apply to resources based on their resource type.", @@ -3783,6 +3510,7 @@ "options": { "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", "privateRouting": "Indicates whether private network routing is enabled.", "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", @@ -3832,6 +3560,9 @@ "subnets": { "type": "object", "properties": { + "dataExplorer": { + "type": "string" + }, "dataFactory": { "type": "string" }, @@ -3858,6 +3589,7 @@ "table": "Resource ID and name for the table storage DNS zone." }, "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", "dataFactory": "Resource ID of the subnet for Data Factory instances.", "keyVault": "Resource ID of the subnet for Key Vault instances.", "scripts": "Resource ID of the subnet for deployment script storage.", @@ -3891,35 +3623,21 @@ "HubAppProperties": { "type": "object", "properties": { + "id": { + "type": "string" + }, "name": { "type": "string" }, - "displayName": { + "publisher": { + "type": "string" + }, + "suffix": { "type": "string" }, "tags": { "type": "object" }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, "dataFactory": { "type": "string" }, @@ -3928,22 +3646,21 @@ }, "storage": { "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" } }, "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", "dataFactory": "Name of the Data Factory instance for this publisher.", "keyVault": "Name of the KeyVault instance for this publisher.", "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", "description": "FinOps hub app configuration settings.", "__bicep_imported_from!": { "sourceTemplate": "hub-types.bicep" @@ -3997,7 +3714,8 @@ }, "variables": { "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", - "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', format('{0}cg', parameters('app').hub.routing.scriptStorage), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" }, "resources": { "identity": { @@ -4022,7 +3740,7 @@ "condition": "[parameters('app').hub.options.privateRouting]", "type": "Microsoft.Authorization/roleAssignments", "apiVersion": "2022-04-01", - "scope": "[format('Microsoft.Storage/storageAccounts/{0}', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", "properties": { "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", @@ -4046,7 +3764,7 @@ "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} } }, - "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '9.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", "dependsOn": [ "identity", "identityRoleAssignments" @@ -4056,57 +3774,60 @@ } }, "dependsOn": [ - "identity" + "appTelemetry", + "dataFactory", + "triggerManagerIdentity", + "triggerManagerRoleAssignments" ] } }, "outputs": { - "containerName": { + "dataFactoryId": { "type": "string", "metadata": { - "description": "The name of the storage container." + "description": "Resource ID of the Data Factory instance used by the FinOps hub app." }, - "value": "[parameters('container')]" + "value": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]" }, - "filesUploaded": { - "type": "int", + "keyVaultId": { + "type": "string", "metadata": { - "description": "The number of files uploaded to the storage container." + "description": "Resource ID of the Key Vault instance used by the FinOps hub app." }, - "value": "[variables('fileCount')]" + "value": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]" }, - "identityId": { + "storageAccountId": { "type": "string", "metadata": { - "description": "Resource ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + "description": "Resource ID of the storage account instance used by the FinOps hub app." }, - "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.id.value, '')]" + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" }, - "identityName": { + "principalId": { "type": "string", "metadata": { - "description": "Name of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + "description": "Principal ID for the managed identity used by Data Factory." }, - "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.name.value, '')]" + "value": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" }, - "identityPrincipalId": { + "triggerManagerIdentityName": { "type": "string", "metadata": { - "description": "Principal ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + "description": "Name of the managed identity used to create and stop ADF triggers." }, - "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.principalId.value, '')]" + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" } } } }, "dependsOn": [ - "appRegistration" + "infrastructure" ] }, - "ingestionContainer": { + "configContainer": { "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.FinOpsHubs.Core_Storage.IngestionContainer", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core_Storage.ConfigContainer", "properties": { "expressionEvaluationOptions": { "scope": "inner" @@ -4114,10 +3835,13 @@ "mode": "Incremental", "parameters": { "app": { - "value": "[reference('appRegistration').outputs.app.value]" + "value": "[parameters('app')]" }, "container": { - "value": "ingestion" + "value": "[variables('CONFIG')]" + }, + "forceCreateBlobManagerIdentity": { + "value": true } }, "template": { @@ -4127,8 +3851,8 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "13960345490822271084" + "version": "0.40.2.10011", + "templateHash": "6646280599480816180" } }, "definitions": { @@ -4162,6 +3886,9 @@ "keyVaultSku": { "type": "string" }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, "networkAddressPrefix": { "type": "string" }, @@ -4201,6 +3928,7 @@ "options": { "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", "privateRouting": "Indicates whether private network routing is enabled.", "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", @@ -4250,6 +3978,9 @@ "subnets": { "type": "object", "properties": { + "dataExplorer": { + "type": "string" + }, "dataFactory": { "type": "string" }, @@ -4276,6 +4007,7 @@ "table": "Resource ID and name for the table storage DNS zone." }, "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", "dataFactory": "Resource ID of the subnet for Data Factory instances.", "keyVault": "Resource ID of the subnet for Key Vault instances.", "scripts": "Resource ID of the subnet for deployment script storage.", @@ -4309,35 +4041,21 @@ "HubAppProperties": { "type": "object", "properties": { + "id": { + "type": "string" + }, "name": { "type": "string" }, - "displayName": { + "publisher": { + "type": "string" + }, + "suffix": { "type": "string" }, "tags": { "type": "object" }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, "dataFactory": { "type": "string" }, @@ -4346,22 +4064,21 @@ }, "storage": { "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" } }, "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", "dataFactory": "Name of the Data Factory instance for this publisher.", "keyVault": "Name of the KeyVault instance for this publisher.", "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", "description": "FinOps hub app configuration settings.", "__bicep_imported_from!": { "sourceTemplate": "hub-types.bicep" @@ -4427,7 +4144,7 @@ "identity": { "condition": "[or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity'))]", "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", + "apiVersion": "2025-04-01", "name": "[format('{0}.Identity', deployment().name)]", "properties": { "expressionEvaluationOptions": { @@ -4458,8 +4175,8 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "4534337491931150093" + "version": "0.40.2.10011", + "templateHash": "18138592202204453545" } }, "definitions": { @@ -4493,6 +4210,9 @@ "keyVaultSku": { "type": "string" }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, "networkAddressPrefix": { "type": "string" }, @@ -4532,6 +4252,7 @@ "options": { "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", "privateRouting": "Indicates whether private network routing is enabled.", "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", @@ -4581,6 +4302,9 @@ "subnets": { "type": "object", "properties": { + "dataExplorer": { + "type": "string" + }, "dataFactory": { "type": "string" }, @@ -4607,6 +4331,7 @@ "table": "Resource ID and name for the table storage DNS zone." }, "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", "dataFactory": "Resource ID of the subnet for Data Factory instances.", "keyVault": "Resource ID of the subnet for Key Vault instances.", "scripts": "Resource ID of the subnet for deployment script storage.", @@ -4640,35 +4365,21 @@ "HubAppProperties": { "type": "object", "properties": { + "id": { + "type": "string" + }, "name": { "type": "string" }, - "displayName": { + "publisher": { + "type": "string" + }, + "suffix": { "type": "string" }, "tags": { "type": "object" }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, "dataFactory": { "type": "string" }, @@ -4677,22 +4388,21 @@ }, "storage": { "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" } }, "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", "dataFactory": "Name of the Data Factory instance for this publisher.", "keyVault": "Name of the KeyVault instance for this publisher.", "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", "description": "FinOps hub app configuration settings.", "__bicep_imported_from!": { "sourceTemplate": "hub-types.bicep" @@ -4704,7 +4414,7 @@ { "namespace": "__bicep", "members": { - "getPublisherTags": { + "getAppPublisherTags": { "parameters": [ { "$ref": "#/definitions/HubAppProperties", @@ -4717,7 +4427,7 @@ ], "output": { "type": "object", - "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').publisher.tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" }, "metadata": { "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", @@ -4763,7 +4473,7 @@ "type": "Microsoft.ManagedIdentity/userAssignedIdentities", "apiVersion": "2023-01-31", "name": "[parameters('identityName')]", - "tags": "[__bicep.getPublisherTags(parameters('app'), 'Microsoft.ManagedIdentity/userAssignedIdentities')]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.ManagedIdentity/userAssignedIdentities')]", "location": "[parameters('app').hub.location]" }, "identityRoleAssignments": { @@ -4813,7 +4523,7 @@ "uploadFiles": { "condition": "[variables('hasFiles')]", "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", + "apiVersion": "2025-04-01", "name": "[format('{0}.Upload', deployment().name)]", "properties": { "expressionEvaluationOptions": { @@ -4854,8 +4564,8 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "588615643779078900" + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" } }, "definitions": { @@ -4900,6 +4610,9 @@ "keyVaultSku": { "type": "string" }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, "networkAddressPrefix": { "type": "string" }, @@ -4939,6 +4652,7 @@ "options": { "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", "privateRouting": "Indicates whether private network routing is enabled.", "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", @@ -4988,6 +4702,9 @@ "subnets": { "type": "object", "properties": { + "dataExplorer": { + "type": "string" + }, "dataFactory": { "type": "string" }, @@ -5014,6 +4731,7 @@ "table": "Resource ID and name for the table storage DNS zone." }, "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", "dataFactory": "Resource ID of the subnet for Data Factory instances.", "keyVault": "Resource ID of the subnet for Key Vault instances.", "scripts": "Resource ID of the subnet for deployment script storage.", @@ -5047,35 +4765,21 @@ "HubAppProperties": { "type": "object", "properties": { + "id": { + "type": "string" + }, "name": { "type": "string" }, - "displayName": { + "publisher": { + "type": "string" + }, + "suffix": { "type": "string" }, "tags": { "type": "object" }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, "dataFactory": { "type": "string" }, @@ -5084,22 +4788,21 @@ }, "storage": { "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" } }, "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", "dataFactory": "Name of the Data Factory instance for this publisher.", "keyVault": "Name of the KeyVault instance for this publisher.", "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", "description": "FinOps hub app configuration settings.", "__bicep_imported_from!": { "sourceTemplate": "hub-types.bicep" @@ -5153,7 +4856,8 @@ }, "variables": { "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", - "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', format('{0}cg', parameters('app').hub.routing.scriptStorage), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" }, "resources": { "identity": { @@ -5178,7 +4882,7 @@ "condition": "[parameters('app').hub.options.privateRouting]", "type": "Microsoft.Authorization/roleAssignments", "apiVersion": "2022-04-01", - "scope": "[format('Microsoft.Storage/storageAccounts/{0}', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", "properties": { "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", @@ -5202,7 +4906,7 @@ "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} } }, - "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '9.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", "dependsOn": [ "identity", "identityRoleAssignments" @@ -5259,10 +4963,10 @@ "appRegistration" ] }, - "uploadSettings": { + "ingestionContainer": { "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.FinOpsHubs.Core_Storage.UpdateSettings", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core_Storage.IngestionContainer", "properties": { "expressionEvaluationOptions": { "scope": "inner" @@ -5270,52 +4974,10 @@ "mode": "Incremental", "parameters": { "app": { - "value": "[reference('appRegistration').outputs.app.value]" - }, - "identityName": { - "value": "[reference('configContainer').outputs.identityName.value]" - }, - "scriptName": { - "value": "[format('{0}_uploadSettings', reference('appRegistration').outputs.app.value.storage)]" - }, - "environmentVariables": { - "value": [ - { - "name": "ftkVersion", - "value": "[variables('$fxv#1')]" - }, - { - "name": "scopes", - "value": "[join(parameters('scopesToMonitor'), '|')]" - }, - { - "name": "msexportRetentionInDays", - "value": "[string(parameters('msexportRetentionInDays'))]" - }, - { - "name": "ingestionRetentionInMonths", - "value": "[string(parameters('ingestionRetentionInMonths'))]" - }, - { - "name": "rawRetentionInDays", - "value": "[string(parameters('rawRetentionInDays'))]" - }, - { - "name": "finalRetentionInMonths", - "value": "[string(parameters('finalRetentionInMonths'))]" - }, - { - "name": "storageAccountName", - "value": "[reference('appRegistration').outputs.app.value.storage]" - }, - { - "name": "containerName", - "value": "config" - } - ] + "value": "[parameters('app')]" }, - "scriptContent": { - "value": "[variables('$fxv#2')]" + "container": { + "value": "[variables('INGESTION')]" } }, "template": { @@ -5325,22 +4987,11 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "588615643779078900" + "version": "0.40.2.10011", + "templateHash": "6646280599480816180" } }, "definitions": { - "EnvironmentVariable": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "value": { - "type": "string" - } - } - }, "_1.HubProperties": { "type": "object", "properties": { @@ -5371,6 +5022,9 @@ "keyVaultSku": { "type": "string" }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, "networkAddressPrefix": { "type": "string" }, @@ -5410,6 +5064,7 @@ "options": { "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", "privateRouting": "Indicates whether private network routing is enabled.", "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", @@ -5459,6 +5114,9 @@ "subnets": { "type": "object", "properties": { + "dataExplorer": { + "type": "string" + }, "dataFactory": { "type": "string" }, @@ -5485,6 +5143,7 @@ "table": "Resource ID and name for the table storage DNS zone." }, "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", "dataFactory": "Resource ID of the subnet for Data Factory instances.", "keyVault": "Resource ID of the subnet for Key Vault instances.", "scripts": "Resource ID of the subnet for deployment script storage.", @@ -5518,35 +5177,21 @@ "HubAppProperties": { "type": "object", "properties": { + "id": { + "type": "string" + }, "name": { "type": "string" }, - "displayName": { + "publisher": { + "type": "string" + }, + "suffix": { "type": "string" }, "tags": { "type": "object" }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, "dataFactory": { "type": "string" }, @@ -5555,22 +5200,21 @@ }, "storage": { "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" } }, "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", "dataFactory": "Name of the Data Factory instance for this publisher.", "keyVault": "Name of the KeyVault instance for this publisher.", "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", "description": "FinOps hub app configuration settings.", "__bicep_imported_from!": { "sourceTemplate": "hub-types.bicep" @@ -5582,1635 +5226,679 @@ "app": { "$ref": "#/definitions/HubAppProperties", "metadata": { - "description": "Required. FinOps hub app the deployment script is being run for." - } - }, - "identityName": { - "type": "string", - "metadata": { - "description": "Required. Name of the managed identity to create." - } - }, - "scriptName": { - "type": "string", - "defaultValue": "[deployment().name]", - "metadata": { - "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + "description": "Required. FinOps hub app that storage is getting updated for." } }, - "scriptContent": { + "container": { "type": "string", "metadata": { - "description": "Required. Name of the deployment script to create." + "description": "Required. Name of the storage container to create or update." } }, - "arguments": { - "type": "string", - "defaultValue": "", + "files": { + "type": "object", + "defaultValue": {}, "metadata": { - "description": "Optional. Additional arguments to pass into the deployment script." + "description": "Optional. Dictionary of key/value pairs for the files to upload to the specified container. The key is the target path under the container and the value is the contents of the file. Default: {} (no files to upload)." } }, - "environmentVariables": { - "type": "array", - "items": { - "$ref": "#/definitions/EnvironmentVariable" - }, - "defaultValue": [], + "forceCreateBlobManagerIdentity": { + "type": "bool", + "defaultValue": false, "metadata": { - "description": "Optional. Environment variables to use for the deployment script." + "description": "Optional. Indicates whether to create the blob manager user assigned identity even if files are not being uploaded. Default: false." } } }, "variables": { - "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", - "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', format('{0}cg', parameters('app').hub.routing.scriptStorage), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n# Get storage context\r\n$storageContext = @{\r\n Context = New-AzStorageContext -StorageAccountName $env:storageAccountName -UseConnectedAccount\r\n Container = $env:containerName\r\n}\r\n\r\n# Uploading files\r\n$files = $env:files | ConvertFrom-Json -Depth 10\r\nWrite-Output \"Uploading ${$files.PSObject.Properties.Count} files...\"\r\n$files.PSObject.Properties | ForEach-Object {\r\n $filePath = $_.Name\r\n $tempPath = \"./$($filePath -replace \"/\", \"_\")\"\r\n Write-Output \" Uploading $filePath...\"\r\n $_.Value | Out-File $tempPath\r\n Set-AzStorageBlobContent @storageContext -File $tempPath -Blob $filePath -Force | Out-Null\r\n}\r\n", + "fileCount": "[length(items(parameters('files')))]", + "hasFiles": "[greater(variables('fileCount'), 0)]" }, "resources": { - "identity": { - "type": "Microsoft.ManagedIdentity/userAssignedIdentities", - "apiVersion": "2023-01-31", - "name": "[parameters('identityName')]", - "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", - "location": "[parameters('app').hub.location]" + "storageAccount::blobService::targetContainer": { + "type": "Microsoft.Storage/storageAccounts/blobServices/containers", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}/{2}', parameters('app').storage, 'default', parameters('container'))]", + "properties": { + "publicAccess": "None", + "metadata": {} + } }, - "scriptStorageAccount": { - "condition": "[parameters('app').hub.options.privateRouting]", + "storageAccount::blobService": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]" + }, + "storageAccount": { "existing": true, "type": "Microsoft.Storage/storageAccounts", "apiVersion": "2022-09-01", - "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + "name": "[parameters('app').storage]" }, - "identityRoleAssignments": { - "copy": { - "name": "identityRoleAssignments", - "count": "[length(variables('privateEndpointDeploymentRoles'))]" - }, - "condition": "[parameters('app').hub.options.privateRouting]", - "type": "Microsoft.Authorization/roleAssignments", - "apiVersion": "2022-04-01", - "scope": "[format('Microsoft.Storage/storageAccounts/{0}', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", - "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "identity": { + "condition": "[or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity'))]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.Identity', deployment().name)]", "properties": { - "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", - "principalId": "[reference('identity').principalId]", - "principalType": "ServicePrincipal" - }, - "dependsOn": [ - "identity" - ] - }, - "script": { - "type": "Microsoft.Resources/deploymentScripts", - "apiVersion": "2023-08-01", - "name": "[parameters('scriptName')]", - "kind": "AzurePowerShell", - "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", - "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", - "identity": { - "type": "UserAssigned", - "userAssignedIdentities": { - "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} - } - }, - "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '9.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", - "dependsOn": [ - "identity", - "identityRoleAssignments" - ] - } - } - } - }, - "dependsOn": [ - "appRegistration", - "configContainer" - ] - } - }, - "outputs": { - "dataFactoryName": { - "type": "string", - "metadata": { - "description": "Name of the Data Factory." - }, - "value": "[reference('appRegistration').outputs.app.value.dataFactory]" - }, - "storageAccountName": { - "type": "string", - "metadata": { - "description": "Name of the storage account created for the hub instance. This must be used when connecting FinOps toolkit Power BI reports to your data." - }, - "value": "[reference('appRegistration').outputs.app.value.storage]" - }, - "configContainer": { - "type": "string", - "metadata": { - "description": "The name of the container used for configuration settings." - }, - "value": "[reference('configContainer').outputs.containerName.value]" - }, - "ingestionContainer": { - "type": "string", - "metadata": { - "description": "The name of the container used for normalized data ingestion." - }, - "value": "[reference('ingestionContainer').outputs.containerName.value]" - }, - "storageUrlForPowerBI": { - "type": "string", - "metadata": { - "description": "URL to use when connecting custom Power BI reports to your data." - }, - "value": "[format('https://{0}.dfs.{1}/{2}', reference('appRegistration').outputs.app.value.storage, environment().suffixes.storage, reference('ingestionContainer').outputs.containerName.value)]" - }, - "principalId": { - "type": "string", - "metadata": { - "description": "Object ID of the Data Factory managed identity. This will be needed when configuring managed exports." - }, - "value": "[reference('appRegistration').outputs.principalId.value]" - }, - "publisherTags": { - "type": "object", - "metadata": { - "description": "Tags for the FinOps hub publisher." - }, - "value": "[reference('appRegistration').outputs.app.value.publisher.tags]" - } - } - } - }, - "dependsOn": [ - "infrastructure" - ] - }, - "cmExports": { - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.CostManagement.Exports", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "hub": { - "value": "[variables('hub')]" - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "languageVersion": "2.0", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "12652260421176486151" - } - }, - "definitions": { - "_1.HubRoutingProperties": { - "type": "object", - "properties": { - "networkId": { - "type": "string" - }, - "networkName": { - "type": "string" - }, - "scriptStorage": { - "type": "string" - }, - "dnsZones": { - "type": "object", - "properties": { - "blob": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "dfs": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "queue": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "table": { - "$ref": "#/definitions/_1.IdNameObject" - } - } - }, - "subnets": { - "type": "object", - "properties": { - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "scripts": { - "type": "string" - }, - "storage": { - "type": "string" - } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubProperties": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - }, - "location": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "tagsByResource": { - "type": "object" - }, - "version": { - "type": "string" - }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" - }, - "keyVaultSku": { - "type": "string" - }, - "networkAddressPrefix": { - "type": "string" - }, - "privateRouting": { - "type": "bool" - }, - "publisherIsolation": { - "type": "bool" - }, - "storageInfrastructureEncryption": { - "type": "bool" - }, - "storageSku": { - "type": "string" - } - } - }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" - }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" - } - } - } - }, - "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", - "location": "Azure resource location of the FinOps hub instance.", - "tags": "Tags to apply to all FinOps hub resources.", - "tagsByResource": "Tags to apply to resources based on their resource type.", - "version": "FinOps hub version number.", - "options": { - "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", - "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", - "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", - "privateRouting": "Indicates whether private network routing is enabled.", - "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", - "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", - "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." - }, - "routing": "FinOps hub private network routing properties, if enabled.", - "core": { - "suffix": "Unique suffix used for shared resources." - }, - "apps": {}, - "description": "FinOps hub instance properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - }, - "parameters": { - "hub": { - "$ref": "#/definitions/HubProperties", - "metadata": { - "description": "Required. FinOps hub instance to deploy the app to." - } - } - }, - "variables": { - "$fxv#0": "12.0", - "$fxv#1": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\":{\"name\":\"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingPeriodStartDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStartDate\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingPeriodEndDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEndDate\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountOwnerId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"SubscriptionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubscriptionId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"SubscriptionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubscriptionName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Date\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"Date\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Product\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Product\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PartNumber\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceFamily\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterCategory\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterSubCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterSubCategory\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterRegion\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Quantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Quantity\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"EffectivePrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectivePrice\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Cost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Cost\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"UnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UnitPrice\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceLocation\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceLocation\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AvailabilityZone\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AvailabilityZone\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ConsumedService\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedService\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceInfo1\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceInfo1\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceInfo2\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceInfo2\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AdditionalInfo\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AdditionalInfo\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSectionId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"InvoiceSection\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSection\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CostCenter\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"UnitOfMeasure\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"UnitOfMeasure\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceGroup\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceGroup\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ReservationId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ReservationName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ProductOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductOrderId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ProductOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductOrderName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"OfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"OfferId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"IsAzureCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"IsAzureCreditEligible\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PlanName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PlanName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ChargeType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeType\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Frequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Frequency\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PublisherType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherType\"}\r\n }\r\n ]\r\n }\r\n}\r\n", - "$fxv#10": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"InstanceFlexibilityGroup\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityGroup\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InstanceFlexibilityRatio\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityRatio\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InstanceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InstanceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Kind\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Kind\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservedHours\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ReservedHours\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"TotalReservedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"TotalReservedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UsageDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"UsageDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UsedHours\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UsedHours\" }\r\n }\r\n ]\r\n }\r\n}\r\n", - "$fxv#11": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"SKU\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SKU\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Location\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Location\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CostWithNoReservedInstances\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CostWithNoReservedInstances\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"FirstUsageDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"FirstUsageDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InstanceFlexibilityRatio\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityRatio\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InstanceFlexibilityGroup\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityGroup\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"LookBackPeriod\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"LookBackPeriod\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"NetSavings\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"NetSavings\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"NormalizedSize\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"NormalizedSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RecommendedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"RecommendedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RecommendedQuantityNormalized\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"RecommendedQuantityNormalized\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Scope\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Scope\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuProperties\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuProperties\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"TotalCostWithReservedInstances\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"TotalCostWithReservedInstances\" }\r\n }\r\n ]\r\n }\r\n}\r\n", - "$fxv#12": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"Cost With No ReservedInstances\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CostWithNoReservedInstancesJson\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"First UsageDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"FirstUsageDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Instance Flexibility Ratio\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityRatio\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Instance Flexibility Group\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityGroup\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Location\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Location\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"LookBackPeriod\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"LookBackPeriod\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterID\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Net Savings\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"NetSavingsJson\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Normalized Size\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"NormalizedSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Recommended Quantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"RecommendedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Recommended Quantity Normalized\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"RecommendedQuantityNormalized\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"scope\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Scope\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Sku Properties\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuProperties\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Total Cost With ReservedInstances\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"TotalCostWithReservedInstancesJson\" }\r\n }\r\n ]\r\n }\r\n}\r\n", - "$fxv#13": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"AccountOwnerEmail\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountOwnerEmail\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Amount\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Amount\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ArmSkuName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ArmSkuName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingMonth\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingMonth\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Currency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Currency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CurrentEnrollmentId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CurrentEnrollmentId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"DepartmentName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"DepartmentName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Description\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Description\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EventDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EventDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EventType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"EventType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MonetaryCommitment\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"MonetaryCommitment\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Overage\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Overage\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PurchasingSubscriptionGuid\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PurchasingSubscriptionGuid\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PurchasingSubscriptionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PurchasingSubscriptionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PurchasingEnrollment\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PurchasingEnrollment\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Quantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Quantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Region\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Region\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n }\r\n ]\r\n }\r\n}\r\n", - "$fxv#14": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"Amount\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Amount\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ArmSkuName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ArmSkuName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Currency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Currency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Description\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Description\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EventDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EventDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EventType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"EventType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Invoice\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Invoice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PurchasingSubscriptionGuid\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PurchasingSubscriptionGuid\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PurchasingSubscriptionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PurchasingSubscriptionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Quantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Quantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Region\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Region\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n }\r\n ]\r\n }\r\n}\r\n", - "$fxv#2": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\":{\"name\":\"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingPeriodStartDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStartDate\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingPeriodEndDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEndDate\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountOwnerId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"SubscriptionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubscriptionId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"SubscriptionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubscriptionName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Date\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"Date\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Product\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Product\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PartNumber\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceFamily\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterCategory\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterSubCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterSubCategory\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterRegion\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Quantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Quantity\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"EffectivePrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectivePrice\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Cost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Cost\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"UnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UnitPrice\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceLocation\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceLocation\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AvailabilityZone\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AvailabilityZone\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ConsumedService\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedService\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceInfo1\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceInfo1\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceInfo2\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceInfo2\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AdditionalInfo\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AdditionalInfo\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSectionId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"InvoiceSection\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSection\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CostCenter\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"UnitOfMeasure\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"UnitOfMeasure\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceGroup\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceGroup\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ReservationId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ReservationName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ProductOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductOrderId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ProductOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductOrderName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"OfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"OfferId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"IsAzureCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"IsAzureCreditEligible\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PlanName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PlanName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ChargeType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeType\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Frequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Frequency\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PublisherType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherType\"}\r\n }\r\n ]\r\n }\r\n}\r\n", - "$fxv#3": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"BilledCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BilledCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CapacityReservationId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CapacityReservationId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CapacityReservationStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CapacityReservationStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ConsumedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectiveCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceIssuerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceIssuerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"PricingQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProviderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProviderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuMeter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuMeter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountOwnerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AmortizationClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AmortizationClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRateDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRateDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ContractedCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ContractedCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostAllocationRuleName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostAllocationRuleName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceIssuerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceIssuerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ListCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ListCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditApplied\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditApplied\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditRate\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingBlockSize\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PricingBlockSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingUnitDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingUnitDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceGroupName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceGroupName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServiceModel\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ServiceModel\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuIsCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_SkuIsCreditEligible\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOfferId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPlanName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPlanName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTerm\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTerm\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTier\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTier\" }\r\n }\r\n ]\r\n }\r\n}\r\n", - "$fxv#4": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"BilledCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BilledCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CapacityReservationId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CapacityReservationId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CapacityReservationStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CapacityReservationStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ConsumedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectiveCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceIssuerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceIssuerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"PricingQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProviderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProviderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuMeter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuMeter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountOwnerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AmortizationClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AmortizationClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRateDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRateDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ContractedCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ContractedCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostAllocationRuleName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostAllocationRuleName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceIssuerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceIssuerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ListCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ListCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditApplied\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditApplied\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditRate\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingBlockSize\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PricingBlockSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingUnitDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingUnitDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceGroupName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceGroupName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServiceModel\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ServiceModel\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuIsCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_SkuIsCreditEligible\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOfferId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPlanName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPlanName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTerm\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTerm\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTier\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTier\" }\r\n }\r\n ]\r\n }\r\n}\r\n", - "$fxv#5": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"BilledCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BilledCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ConsumedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectiveCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceIssuerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceIssuerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"PricingQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProviderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProviderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountOwnerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRateDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRateDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ContractedCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ContractedCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostAllocationRuleName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostAllocationRuleName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceIssuerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceIssuerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ListCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ListCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditApplied\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditApplied\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditRate\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingBlockSize\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PricingBlockSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingUnitDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingUnitDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceGroupName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceGroupName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuIsCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_SkuIsCreditEligible\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOfferId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTerm\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTerm\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTier\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTier\" }\r\n }\r\n ]\r\n }\r\n}\r\n", - "$fxv#6": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"BilledCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BilledCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ConsumedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectiveCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceIssuerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceIssuerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"PricingQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProviderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProviderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountOwnerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRateDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRateDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ContractedCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ContractedCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostAllocationRuleName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostAllocationRuleName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceIssuerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceIssuerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ListCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ListCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditApplied\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditApplied\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditRate\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingBlockSize\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PricingBlockSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingUnitDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingUnitDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceGroupName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceGroupName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuIsCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_SkuIsCreditEligible\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOfferId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTerm\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTerm\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTier\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTier\" }\r\n }\r\n ]\r\n }\r\n}\r\n", - "$fxv#7": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"AvailabilityZone\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AvailabilityZone\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BilledCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BilledCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectiveCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceIssuerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceIssuerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"PricingQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProviderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProviderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Region\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Region\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UsageQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UsageQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UsageUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"UsageUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountOwnerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRateDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRateDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ChargeId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ChargeId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostAllocationRuleName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostAllocationRuleName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceIssuerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceIssuerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_OnDemandCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_OnDemandCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_OnDemandCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_OnDemandCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_OnDemandUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_OnDemandUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditApplied\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditApplied\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingBlockSize\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PricingBlockSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingUnitDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingUnitDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceGroupName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceGroupName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuIsCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_SkuIsCreditEligible\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOfferId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTerm\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTerm\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTier\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTier\" }\r\n }\r\n ]\r\n }\r\n}\r\n", - "$fxv#8": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"EnrollmentNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"EnrollmentNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterID\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterID\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterSubCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterSubCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Product\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Product\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuID\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuID\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProductID\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductID\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UnitOfMeasure\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"UnitOfMeasure\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveStartDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EffectiveStartDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveEndDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EffectiveEndDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BasePrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BasePrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MarketPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"MarketPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CurrencyCode\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CurrencyCode\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"IncludedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"IncludedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"OfferID\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"OfferID\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PriceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PriceType\" }\r\n }\r\n ]\r\n }\r\n}\r\n", - "$fxv#9": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Product\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Product\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProductId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UnitOfMeasure\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"UnitOfMeasure\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterSubCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterSubCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"TierMinimumUnits\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"TierMinimumUnits\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveStartDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EffectiveStartDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveEndDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EffectiveEndDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BasePrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BasePrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MarketPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"MarketPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Currency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Currency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PriceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PriceType\" }\r\n }\r\n ]\r\n }\r\n}\r\n" - }, - "resources": { - "appRegistration": { - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.CostManagement.Exports_Register", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "hub": { - "value": "[parameters('hub')]" - }, - "publisher": { - "value": "Microsoft FinOps hubs" - }, - "namespace": { - "value": "Microsoft.FinOpsHubs" - }, - "appName": { - "value": "Core" - }, - "displayName": { - "value": "FinOps hub core" - }, - "appVersion": { - "value": "[variables('$fxv#0')]" - }, - "features": { - "value": [ - "DataFactory", - "Storage" - ] - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "languageVersion": "2.0", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "15179190433979236138" - } - }, - "definitions": { - "_1.HubRoutingProperties": { - "type": "object", - "properties": { - "networkId": { - "type": "string" - }, - "networkName": { - "type": "string" - }, - "scriptStorage": { - "type": "string" + "expressionEvaluationOptions": { + "scope": "inner" }, - "dnsZones": { - "type": "object", - "properties": { - "blob": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "dfs": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "queue": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "table": { - "$ref": "#/definitions/_1.IdNameObject" - } + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_blobManager', parameters('app').storage)]" + }, + "roleAssignmentResourceId": { + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "roles": { + "value": [ + "ba92f5b4-2d11-453d-a403-e96b0029c9fe", + "69566ab7-960f-475b-8e7c-b3118f30c6bd" + ] } }, - "subnets": { - "type": "object", - "properties": { - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "scripts": { - "type": "string" - }, - "storage": { - "type": "string" + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "18138592202204453545" } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubAppFeature": { - "type": "string", - "allowedValues": [ - "DataFactory", - "KeyVault", - "Storage" - ], - "metadata": { - "description": "FinOps hub app features.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubAppProperties": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } }, - "displayName": { - "type": "string" + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } }, - "suffix": { - "type": "string" + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } }, - "tags": { - "type": "object" + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } } - } - }, - "hub": { - "$ref": "#/definitions/HubProperties" - }, - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "storage": { - "type": "string" - } - }, - "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", - "dataFactory": "Name of the Data Factory instance for this publisher.", - "keyVault": "Name of the KeyVault instance for this publisher.", - "storage": "Name of the storage account for this publisher.", - "description": "FinOps hub app configuration settings.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubProperties": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - }, - "location": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "tagsByResource": { - "type": "object" - }, - "version": { - "type": "string" - }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" - }, - "keyVaultSku": { - "type": "string" - }, - "networkAddressPrefix": { - "type": "string" - }, - "privateRouting": { - "type": "bool" - }, - "publisherIsolation": { - "type": "bool" - }, - "storageInfrastructureEncryption": { - "type": "bool" - }, - "storageSku": { - "type": "string" - } - } - }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" - }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" - } - } - } - }, - "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", - "location": "Azure resource location of the FinOps hub instance.", - "tags": "Tags to apply to all FinOps hub resources.", - "tagsByResource": "Tags to apply to resources based on their resource type.", - "version": "FinOps hub version number.", - "options": { - "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", - "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", - "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", - "privateRouting": "Indicates whether private network routing is enabled.", - "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", - "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", - "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." - }, - "routing": "FinOps hub private network routing properties, if enabled.", - "core": { - "suffix": "Unique suffix used for shared resources." - }, - "apps": {}, - "description": "FinOps hub instance properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - }, - "functions": [ - { - "namespace": "__bicep", - "members": { - "getAppTags": { - "parameters": [ - { - "$ref": "#/definitions/HubAppProperties", - "name": "app" - }, - { - "type": "string", - "name": "resourceType" - }, + }, + "functions": [ { - "type": "bool", - "nullable": true, - "name": "forceAppTags" + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } } ], - "output": { - "type": "object", - "value": "[union(if(or(parameters('app').hub.options.publisherIsolation, coalesce(parameters('forceAppTags'), false())), parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" - }, - "metadata": { - "description": "Returns a tags dictionary that includes tags for the FinOps hub app.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "getPublisherTags": { - "parameters": [ - { + "parameters": { + "app": { "$ref": "#/definitions/HubAppProperties", - "name": "app" + "metadata": { + "description": "Required. FinOps hub app the identity is associated with." + } }, - { + "identityName": { "type": "string", - "name": "resourceType" - } - ], - "output": { - "type": "object", - "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').publisher.tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" - }, - "metadata": { - "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "newApp": { - "parameters": [ - { - "$ref": "#/definitions/HubProperties", - "name": "hub" + "metadata": { + "description": "Required. Name of the user assigned identity." + } }, - { + "roleAssignmentResourceId": { "type": "string", - "name": "publisherDisplayName" + "metadata": { + "description": "Required. Resource ID of the resource access is being granted for." + } }, - { - "type": "string", - "name": "publisherName" + "roles": { + "type": "array", + "items": { + "type": "string" + }, + "metadata": { + "description": "Required. List of RBAC role assignment GUIDs." + } + } + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.ManagedIdentity/userAssignedIdentities')]", + "location": "[parameters('app').hub.location]" }, - { + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(parameters('roles'))]" + }, + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "name": "[guid(parameters('roleAssignmentResourceId'), parameters('roles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', parameters('roles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + } + }, + "outputs": { + "id": { "type": "string", - "name": "appPartialName" + "metadata": { + "description": "Resource ID of the user assigned identity." + }, + "value": "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName'))]" }, - { + "name": { "type": "string", - "name": "appDisplayName" + "metadata": { + "description": "Name of the user assigned identity." + }, + "value": "[parameters('identityName')]" }, - { + "principalId": { "type": "string", - "name": "version" - } - ], - "output": { - "$ref": "#/definitions/HubAppProperties", - "value": "[_1.newAppInternal(parameters('hub'), parameters('publisherName'), parameters('publisherDisplayName'), if(or(not(parameters('hub').options.publisherIsolation), equals(parameters('publisherName'), 'Microsoft.FinOpsHubs')), parameters('hub').core.suffix, uniqueString(parameters('publisherName'))), createObject('ftk-hubapp-publisher', parameters('publisherName')), format('{0}.{1}', parameters('publisherName'), parameters('appPartialName')), parameters('appDisplayName'), parameters('version'))]" - }, - "metadata": { - "description": "Creates a new FinOps hub app configuration object.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" + "metadata": { + "description": "Principal ID of the user assigned identity." + }, + "value": "[reference('identity').principalId]" } } } } }, - { - "namespace": "_1", - "members": { - "newAppInternal": { - "parameters": [ - { - "$ref": "#/definitions/HubProperties", - "name": "hub" - }, - { - "type": "string", - "name": "publisherName" - }, - { - "type": "string", - "name": "publisherDisplayName" - }, - { - "type": "string", - "name": "publisherSuffix" - }, - { - "type": "object", - "name": "publisherTags" - }, - { - "type": "string", - "name": "appName" - }, - { - "type": "string", - "name": "appDisplayName" - }, - { - "type": "string", - "name": "version" - } - ], - "output": { - "$ref": "#/definitions/HubAppProperties", - "value": { - "name": "[parameters('appName')]", - "displayName": "[parameters('appDisplayName')]", - "tags": "[union(parameters('hub').tags, parameters('publisherTags'), createObject('ftk-hubapp', parameters('appName'), 'ftk-hubapp-version', parameters('version')))]", - "publisher": { - "name": "[parameters('publisherName')]", - "displayName": "[parameters('publisherDisplayName')]", - "suffix": "[parameters('publisherSuffix')]", - "tags": "[union(parameters('hub').tags, parameters('publisherTags'))]" + "uploadFiles": { + "condition": "[variables('hasFiles')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.Upload', deployment().name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[reference('identity').outputs.name.value]" + }, + "environmentVariables": { + "value": [ + { + "name": "storageAccountName", + "value": "[parameters('app').storage]" }, - "hub": "[parameters('hub')]", - "dataFactory": "[replace(format('{0}-{1}', take(format('{0}-engine', replace(parameters('hub').name, '_', '-')), sub(sub(63, length(parameters('publisherSuffix'))), 1)), parameters('publisherSuffix')), '--', '-')]", - "keyVault": "[replace(format('{0}-{1}', take(format('{0}-vault', replace(parameters('hub').name, '_', '-')), sub(sub(24, length(parameters('publisherSuffix'))), 1)), parameters('publisherSuffix')), '--', '-')]", - "storage": "[format('{0}{1}', take(_1.safeStorageName(parameters('hub').name), sub(24, length(parameters('publisherSuffix')))), parameters('publisherSuffix'))]" - } + { + "name": "containerName", + "value": "[parameters('container')]" + }, + { + "name": "files", + "value": "[string(parameters('files'))]" + } + ] }, - "metadata": { - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } + "scriptContent": { + "value": "[variables('$fxv#0')]" } }, - "safeStorageName": { - "parameters": [ - { - "type": "string", - "name": "name" - } - ], - "output": { - "type": "string", - "value": "[replace(replace(toLower(parameters('name')), '-', ''), '_', '')]" - }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", "metadata": { - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - } - } - ], - "parameters": { - "hub": { - "$ref": "#/definitions/HubProperties", - "metadata": { - "description": "Required. FinOps hub instance properties." - } - }, - "publisher": { - "type": "string", - "metadata": { - "description": "Required. Display name of the FinOps hub app publisher." - } - }, - "namespace": { - "type": "string", - "metadata": { - "description": "Required. Namespace to use for the FinOps hub app publisher. Will be combined with appName to form a fully-qualified identifier. Must be an alphanumeric string without spaces or special characters except for periods. This value should never change and will be used to uniquely identify the publisher. A change would require migrating content to the new publisher. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." - } - }, - "appName": { - "type": "string", - "metadata": { - "description": "Required. Unique identifier of the FinOps hub app within the publisher namespace. Must be an alphanumeric string without spaces or special characters. This name should never change and will be used with the namespace to fully qualify the app. A change would require migrating content to the new app. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." - } - }, - "displayName": { - "type": "string", - "metadata": { - "description": "Required. Display name of the FinOps hub app." - } - }, - "appVersion": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Version number of the FinOps hub app." - } - }, - "features": { - "type": "array", - "items": { - "$ref": "#/definitions/HubAppFeature" - }, - "defaultValue": [], - "metadata": { - "description": "Optional. Indicate which features the app requires. Allowed values: \"Storage\". Default: [] (none)." - } - }, - "telemetryString": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." - } - } - }, - "variables": { - "app": "[__bicep.newApp(parameters('hub'), parameters('publisher'), parameters('namespace'), parameters('appName'), parameters('displayName'), parameters('appVersion'))]", - "usesDataFactory": "[contains(parameters('features'), 'DataFactory')]", - "usesKeyVault": "[contains(parameters('features'), 'KeyVault')]", - "usesStorage": "[contains(parameters('features'), 'Storage')]", - "telemetryId": "[format('ftk-hubapp-{0}{1}{2}', variables('app').name, if(empty(parameters('telemetryString')), '', '_'), parameters('telemetryString'))]", - "telemetryProps": { - "mode": "Incremental", - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "[format('FTK: {0} - {1} {2}', parameters('publisher'), parameters('displayName'), variables('telemetryId'))]", - "version": "[parameters('appVersion')]" - } - }, - "resources": [] - } - }, - "storageInfrastructureEncryptionProperties": "[if(not(parameters('hub').options.storageInfrastructureEncryption), createObject(), createObject('encryption', createObject('keySource', 'Microsoft.Storage', 'requireInfrastructureEncryption', parameters('hub').options.storageInfrastructureEncryption)))]" - }, - "resources": { - "storageAccount::blobService": { - "condition": "[variables('usesStorage')]", - "type": "Microsoft.Storage/storageAccounts/blobServices", - "apiVersion": "2022-09-01", - "name": "[format('{0}/{1}', variables('app').storage, 'default')]", - "dependsOn": [ - "storageAccount" - ] - }, - "blobEndpoint::blobPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", - "apiVersion": "2023-11-01", - "name": "[format('{0}/{1}', format('{0}-blob-ep', variables('app').storage), 'storage-endpoint-zone')]", - "properties": { - "privateDnsZoneConfigs": [ - { - "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]", - "properties": { - "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" - } - } - ] - }, - "dependsOn": [ - "blobEndpoint" - ] - }, - "dfsEndpoint::dfsPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", - "apiVersion": "2023-11-01", - "name": "[format('{0}/{1}', format('{0}-dfs-ep', variables('app').storage), 'dfs-endpoint-zone')]", - "properties": { - "privateDnsZoneConfigs": [ - { - "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]", - "properties": { - "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.dfs.{0}', environment().suffixes.storage))]" - } - } - ] - }, - "dependsOn": [ - "dfsEndpoint" - ] - }, - "keyVault::keyVault_accessPolicies": { - "condition": "[variables('usesKeyVault')]", - "type": "Microsoft.KeyVault/vaults/accessPolicies", - "apiVersion": "2023-02-01", - "name": "[format('{0}/{1}', variables('app').keyVault, 'add')]", - "properties": { - "accessPolicies": [ - { - "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", - "tenantId": "[subscription().tenantId]", - "permissions": { - "secrets": [ - "get" - ] - } - } - ] - }, - "dependsOn": [ - "dataFactory", - "keyVault" - ] - }, - "keyVaultPrivateDnsZone::keyVaultPrivateDnsZoneLink": { - "condition": "[and(variables('usesKeyVault'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", - "apiVersion": "2024-06-01", - "name": "[format('{0}/{1}', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), format('{0}-link', replace(format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), '.', '-')))]", - "location": "global", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", - "properties": { - "virtualNetwork": { - "id": "[parameters('hub').routing.networkId]" - }, - "registrationEnabled": false - }, - "dependsOn": [ - "keyVaultPrivateDnsZone" - ] - }, - "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { - "condition": "[and(variables('usesKeyVault'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", - "apiVersion": "2023-11-01", - "name": "[format('{0}/{1}', format('{0}-ep', variables('app').keyVault), 'keyvault-endpoint-zone')]", - "properties": { - "privateDnsZoneConfigs": [ - { - "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", - "properties": { - "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')))]" + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" } - } - ] - }, - "dependsOn": [ - "keyVaultEndpoint", - "keyVaultPrivateDnsZone" - ] - }, - "appTelemetry": { - "condition": "[parameters('hub').options.enableTelemetry]", - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "[if(lessOrEquals(length(variables('telemetryId')), 64), variables('telemetryId'), substring(variables('telemetryId'), 0, 64))]", - "tags": "[__bicep.getAppTags(variables('app'), 'Microsoft.Resources/deployments', true())]", - "properties": "[variables('telemetryProps')]" - }, - "dataFactory": { - "condition": "[variables('usesDataFactory')]", - "type": "Microsoft.DataFactory/factories", - "apiVersion": "2018-06-01", - "name": "[variables('app').dataFactory]", - "location": "[variables('app').hub.location]", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.DataFactory/factories')]", - "identity": { - "type": "SystemAssigned" - }, - "properties": { - "globalConfigurations": { - "PipelineBillingEnabled": "true" - } - } - }, - "storageAccount": { - "condition": "[variables('usesStorage')]", - "type": "Microsoft.Storage/storageAccounts", - "apiVersion": "2022-09-01", - "name": "[variables('app').storage]", - "location": "[parameters('hub').location]", - "sku": { - "name": "[parameters('hub').options.storageSku]" - }, - "kind": "BlockBlobStorage", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Storage/storageAccounts')]", - "properties": "[shallowMerge(createArray(variables('storageInfrastructureEncryptionProperties'), createObject('supportsHttpsTrafficOnly', true(), 'allowSharedKeyAccess', true(), 'isHnsEnabled', true(), 'minimumTlsVersion', 'TLS1_2', 'allowBlobPublicAccess', false(), 'publicNetworkAccess', 'Enabled', 'networkAcls', createObject('bypass', 'AzureServices', 'defaultAction', if(parameters('hub').options.privateRouting, 'Deny', 'Allow')))))]" - }, - "blobPrivateDnsZone": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "existing": true, - "type": "Microsoft.Network/privateDnsZones", - "apiVersion": "2024-06-01", - "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]" - }, - "blobEndpoint": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints", - "apiVersion": "2023-11-01", - "name": "[format('{0}-blob-ep', variables('app').storage)]", - "location": "[parameters('hub').location]", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Network/privateEndpoints')]", - "properties": { - "subnet": { - "id": "[parameters('hub').routing.subnets.storage]" - }, - "privateLinkServiceConnections": [ - { - "name": "blobLink", - "properties": { - "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', variables('app').storage)]", - "groupIds": [ - "blob" - ] - } - } - ] - }, - "dependsOn": [ - "storageAccount" - ] - }, - "dfsPrivateDnsZone": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "existing": true, - "type": "Microsoft.Network/privateDnsZones", - "apiVersion": "2024-06-01", - "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]" - }, - "dfsEndpoint": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints", - "apiVersion": "2023-11-01", - "name": "[format('{0}-dfs-ep', variables('app').storage)]", - "location": "[parameters('hub').location]", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Network/privateEndpoints')]", - "properties": { - "subnet": { - "id": "[parameters('hub').routing.subnets.storage]" - }, - "privateLinkServiceConnections": [ - { - "name": "dfsLink", - "properties": { - "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', variables('app').storage)]", - "groupIds": [ - "dfs" - ] - } - } - ] - }, - "dependsOn": [ - "storageAccount" - ] - }, - "keyVault": { - "condition": "[variables('usesKeyVault')]", - "type": "Microsoft.KeyVault/vaults", - "apiVersion": "2023-02-01", - "name": "[variables('app').keyVault]", - "location": "[parameters('hub').location]", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.KeyVault/vaults')]", - "properties": { - "sku": { - "name": "[parameters('hub').options.keyVaultSku]", - "family": "A" - }, - "enabledForDeployment": true, - "enabledForTemplateDeployment": true, - "enabledForDiskEncryption": true, - "enableSoftDelete": true, - "softDeleteRetentionInDays": 90, - "enableRbacAuthorization": false, - "createMode": "default", - "tenantId": "[subscription().tenantId]", - "accessPolicies": [ - { - "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", - "tenantId": "[subscription().tenantId]", - "permissions": { - "secrets": [ - "get" - ] - } - } - ], - "networkAcls": { - "bypass": "AzureServices", - "defaultAction": "[if(parameters('hub').options.privateRouting, 'Deny', 'Allow')]" - } - }, - "dependsOn": [ - "dataFactory" - ] - }, - "keyVaultPrivateDnsZone": { - "condition": "[and(variables('usesKeyVault'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateDnsZones", - "apiVersion": "2024-06-01", - "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", - "location": "global", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Network/privateDnsZones')]", - "properties": {} - }, - "keyVaultEndpoint": { - "condition": "[and(variables('usesKeyVault'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints", - "apiVersion": "2023-11-01", - "name": "[format('{0}-ep', variables('app').keyVault)]", - "location": "[parameters('hub').location]", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Network/privateEndpoints')]", - "properties": { - "subnet": { - "id": "[parameters('hub').routing.subnets.keyVault]" - }, - "privateLinkServiceConnections": [ - { - "name": "keyVaultLink", - "properties": { - "privateLinkServiceId": "[resourceId('Microsoft.KeyVault/vaults', variables('app').keyVault)]", - "groupIds": [ - "vault" - ] - } - } - ] - }, - "dependsOn": [ - "keyVault" - ] - } - }, - "outputs": { - "app": { - "$ref": "#/definitions/HubAppProperties", - "metadata": { - "description": "FinOps hub app configuration." - }, - "value": "[variables('app')]" - }, - "principalId": { - "type": "string", - "metadata": { - "description": "Principal ID for the managed identity used by Data Factory." - }, - "value": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" - } - } - } - } - }, - "schemaFiles": { - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.CostManagement.Exports_Storage.SchemaFiles", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "app": { - "value": "[reference('appRegistration').outputs.app.value]" - }, - "container": { - "value": "config" - }, - "files": { - "value": { - "schemas/actualcost_c360-2025-04.json": "[variables('$fxv#1')]", - "schemas/amortizedcost_c360-2025-04.json": "[variables('$fxv#2')]", - "schemas/focuscost_1.2.json": "[variables('$fxv#3')]", - "schemas/focuscost_1.2-preview.json": "[variables('$fxv#4')]", - "schemas/focuscost_1.0r2.json": "[variables('$fxv#5')]", - "schemas/focuscost_1.0.json": "[variables('$fxv#6')]", - "schemas/focuscost_1.0-preview(v1).json": "[variables('$fxv#7')]", - "schemas/pricesheet_2023-05-01_ea.json": "[variables('$fxv#8')]", - "schemas/pricesheet_2023-05-01_mca.json": "[variables('$fxv#9')]", - "schemas/reservationdetails_2023-03-01.json": "[variables('$fxv#10')]", - "schemas/reservationrecommendations_2023-05-01_ea.json": "[variables('$fxv#11')]", - "schemas/reservationrecommendations_2023-05-01_mca.json": "[variables('$fxv#12')]", - "schemas/reservationtransactions_2023-05-01_ea.json": "[variables('$fxv#13')]", - "schemas/reservationtransactions_2023-05-01_mca.json": "[variables('$fxv#14')]" - } - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "languageVersion": "2.0", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "13960345490822271084" - } - }, - "definitions": { - "_1.HubProperties": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - }, - "location": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "tagsByResource": { - "type": "object" - }, - "version": { - "type": "string" - }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" - }, - "keyVaultSku": { - "type": "string" - }, - "networkAddressPrefix": { - "type": "string" - }, - "privateRouting": { - "type": "bool" - }, - "publisherIsolation": { - "type": "bool" - }, - "storageInfrastructureEncryption": { - "type": "bool" + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } }, - "storageSku": { - "type": "string" - } - } - }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" - }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" - } - } - } - }, - "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", - "location": "Azure resource location of the FinOps hub instance.", - "tags": "Tags to apply to all FinOps hub resources.", - "tagsByResource": "Tags to apply to resources based on their resource type.", - "version": "FinOps hub version number.", - "options": { - "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", - "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", - "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", - "privateRouting": "Indicates whether private network routing is enabled.", - "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", - "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", - "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." - }, - "routing": "FinOps hub private network routing properties, if enabled.", - "core": { - "suffix": "Unique suffix used for shared resources." - }, - "apps": {}, - "description": "FinOps hub instance properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.HubRoutingProperties": { - "type": "object", - "properties": { - "networkId": { - "type": "string" - }, - "networkName": { - "type": "string" - }, - "scriptStorage": { - "type": "string" - }, - "dnsZones": { - "type": "object", - "properties": { - "blob": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "dfs": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "queue": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "table": { - "$ref": "#/definitions/_1.IdNameObject" - } - } - }, - "subnets": { - "type": "object", - "properties": { - "dataFactory": { - "type": "string" + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } }, - "keyVault": { - "type": "string" + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } }, - "scripts": { - "type": "string" + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } }, - "storage": { - "type": "string" - } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubAppProperties": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "storage": { - "type": "string" - } - }, - "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", - "dataFactory": "Name of the Data Factory instance for this publisher.", - "keyVault": "Name of the KeyVault instance for this publisher.", - "storage": "Name of the storage account for this publisher.", - "description": "FinOps hub app configuration settings.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - }, - "parameters": { - "app": { - "$ref": "#/definitions/HubAppProperties", - "metadata": { - "description": "Required. FinOps hub app that storage is getting updated for." - } - }, - "container": { - "type": "string", - "metadata": { - "description": "Required. Name of the storage container to create or update." - } - }, - "files": { - "type": "object", - "defaultValue": {}, - "metadata": { - "description": "Optional. Dictionary of key/value pairs for the files to upload to the specified container. The key is the target path under the container and the value is the contents of the file. Default: {} (no files to upload)." - } - }, - "forceCreateBlobManagerIdentity": { - "type": "bool", - "defaultValue": false, - "metadata": { - "description": "Optional. Indicates whether to create the blob manager user assigned identity even if files are not being uploaded. Default: false." - } - } - }, - "variables": { - "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n# Get storage context\r\n$storageContext = @{\r\n Context = New-AzStorageContext -StorageAccountName $env:storageAccountName -UseConnectedAccount\r\n Container = $env:containerName\r\n}\r\n\r\n# Uploading files\r\n$files = $env:files | ConvertFrom-Json -Depth 10\r\nWrite-Output \"Uploading ${$files.PSObject.Properties.Count} files...\"\r\n$files.PSObject.Properties | ForEach-Object {\r\n $filePath = $_.Name\r\n $tempPath = \"./$($filePath -replace \"/\", \"_\")\"\r\n Write-Output \" Uploading $filePath...\"\r\n $_.Value | Out-File $tempPath\r\n Set-AzStorageBlobContent @storageContext -File $tempPath -Blob $filePath -Force | Out-Null\r\n}\r\n", - "fileCount": "[length(items(parameters('files')))]", - "hasFiles": "[greater(variables('fileCount'), 0)]" - }, - "resources": { - "storageAccount::blobService::targetContainer": { - "type": "Microsoft.Storage/storageAccounts/blobServices/containers", - "apiVersion": "2022-09-01", - "name": "[format('{0}/{1}/{2}', parameters('app').storage, 'default', parameters('container'))]", - "properties": { - "publicAccess": "None", - "metadata": {} - } - }, - "storageAccount::blobService": { - "existing": true, - "type": "Microsoft.Storage/storageAccounts/blobServices", - "apiVersion": "2022-09-01", - "name": "[format('{0}/{1}', parameters('app').storage, 'default')]" - }, - "storageAccount": { - "existing": true, - "type": "Microsoft.Storage/storageAccounts", - "apiVersion": "2022-09-01", - "name": "[parameters('app').storage]" - }, - "identity": { - "condition": "[or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity'))]", - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "[format('{0}.Identity', deployment().name)]", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "app": { - "value": "[parameters('app')]" - }, - "identityName": { - "value": "[format('{0}_blobManager', parameters('app').storage)]" - }, - "roleAssignmentResourceId": { - "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" - }, - "roles": { - "value": [ - "ba92f5b4-2d11-453d-a403-e96b0029c9fe", - "69566ab7-960f-475b-8e7c-b3118f30c6bd" - ] - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "languageVersion": "2.0", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "4534337491931150093" - } - }, - "definitions": { - "_1.HubProperties": { + "HubAppProperties": { "type": "object", "properties": { "id": { @@ -7219,823 +5907,463 @@ "name": { "type": "string" }, - "location": { + "publisher": { "type": "string" }, - "tags": { - "type": "object" + "suffix": { + "type": "string" }, - "tagsByResource": { + "tags": { "type": "object" }, - "version": { + "dataFactory": { "type": "string" }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" - }, - "keyVaultSku": { - "type": "string" - }, - "networkAddressPrefix": { - "type": "string" - }, - "privateRouting": { - "type": "bool" - }, - "publisherIsolation": { - "type": "bool" - }, - "storageInfrastructureEncryption": { - "type": "bool" - }, - "storageSku": { - "type": "string" - } - } + "keyVault": { + "type": "string" }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" + "storage": { + "type": "string" }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" - } - } + "hub": { + "$ref": "#/definitions/_1.HubProperties" } }, "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", - "location": "Azure resource location of the FinOps hub instance.", - "tags": "Tags to apply to all FinOps hub resources.", - "tagsByResource": "Tags to apply to resources based on their resource type.", - "version": "FinOps hub version number.", - "options": { - "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", - "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", - "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", - "privateRouting": "Indicates whether private network routing is enabled.", - "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", - "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", - "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." - }, - "routing": "FinOps hub private network routing properties, if enabled.", - "core": { - "suffix": "Unique suffix used for shared resources." - }, - "apps": {}, - "description": "FinOps hub instance properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.HubRoutingProperties": { - "type": "object", - "properties": { - "networkId": { - "type": "string" - }, - "networkName": { - "type": "string" - }, - "scriptStorage": { - "type": "string" - }, - "dnsZones": { - "type": "object", - "properties": { - "blob": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "dfs": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "queue": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "table": { - "$ref": "#/definitions/_1.IdNameObject" - } - } - }, - "subnets": { - "type": "object", - "properties": { - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "scripts": { - "type": "string" - }, - "storage": { - "type": "string" - } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubAppProperties": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "storage": { - "type": "string" - } - }, - "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", - "dataFactory": "Name of the Data Factory instance for this publisher.", - "keyVault": "Name of the KeyVault instance for this publisher.", - "storage": "Name of the storage account for this publisher.", - "description": "FinOps hub app configuration settings.", + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", "__bicep_imported_from!": { "sourceTemplate": "hub-types.bicep" } } } }, - "functions": [ - { - "namespace": "__bicep", - "members": { - "getPublisherTags": { - "parameters": [ - { - "$ref": "#/definitions/HubAppProperties", - "name": "app" - }, - { - "type": "string", - "name": "resourceType" - } - ], - "output": { - "type": "object", - "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').publisher.tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" - }, - "metadata": { - "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - } - } - ], "parameters": { "app": { "$ref": "#/definitions/HubAppProperties", "metadata": { - "description": "Required. FinOps hub app the identity is associated with." + "description": "Required. FinOps hub app the deployment script is being run for." } }, "identityName": { "type": "string", "metadata": { - "description": "Required. Name of the user assigned identity." + "description": "Required. Name of the managed identity to create." } }, - "roleAssignmentResourceId": { + "scriptName": { "type": "string", + "defaultValue": "[deployment().name]", "metadata": { - "description": "Required. Resource ID of the resource access is being granted for." + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." } }, - "roles": { + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { "type": "array", "items": { - "type": "string" + "$ref": "#/definitions/EnvironmentVariable" }, + "defaultValue": [], "metadata": { - "description": "Required. List of RBAC role assignment GUIDs." + "description": "Optional. Environment variables to use for the deployment script." } } }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, "resources": { "identity": { "type": "Microsoft.ManagedIdentity/userAssignedIdentities", "apiVersion": "2023-01-31", "name": "[parameters('identityName')]", - "tags": "[__bicep.getPublisherTags(parameters('app'), 'Microsoft.ManagedIdentity/userAssignedIdentities')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", "location": "[parameters('app').hub.location]" }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, "identityRoleAssignments": { "copy": { "name": "identityRoleAssignments", - "count": "[length(parameters('roles'))]" + "count": "[length(variables('privateEndpointDeploymentRoles'))]" }, + "condition": "[parameters('app').hub.options.privateRouting]", "type": "Microsoft.Authorization/roleAssignments", "apiVersion": "2022-04-01", - "name": "[guid(parameters('roleAssignmentResourceId'), parameters('roles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", "properties": { - "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', parameters('roles')[copyIndex()])]", + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", "principalId": "[reference('identity').principalId]", "principalType": "ServicePrincipal" }, "dependsOn": [ "identity" ] - } - }, - "outputs": { - "id": { - "type": "string", - "metadata": { - "description": "Resource ID of the user assigned identity." - }, - "value": "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName'))]" - }, - "name": { - "type": "string", - "metadata": { - "description": "Name of the user assigned identity." - }, - "value": "[parameters('identityName')]" }, - "principalId": { - "type": "string", - "metadata": { - "description": "Principal ID of the user assigned identity." + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } }, - "value": "[reference('identity').principalId]" + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] } } } - } - }, - "uploadFiles": { - "condition": "[variables('hasFiles')]", - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "[format('{0}.Upload', deployment().name)]", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "app": { - "value": "[parameters('app')]" - }, - "identityName": { - "value": "[reference('identity').outputs.name.value]" - }, - "environmentVariables": { - "value": [ - { - "name": "storageAccountName", - "value": "[parameters('app').storage]" - }, - { - "name": "containerName", - "value": "[parameters('container')]" - }, - { - "name": "files", - "value": "[string(parameters('files'))]" - } - ] - }, - "scriptContent": { - "value": "[variables('$fxv#0')]" + }, + "dependsOn": [ + "identity" + ] + } + }, + "outputs": { + "containerName": { + "type": "string", + "metadata": { + "description": "The name of the storage container." + }, + "value": "[parameters('container')]" + }, + "filesUploaded": { + "type": "int", + "metadata": { + "description": "The number of files uploaded to the storage container." + }, + "value": "[variables('fileCount')]" + }, + "identityId": { + "type": "string", + "metadata": { + "description": "Resource ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.id.value, '')]" + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Name of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.name.value, '')]" + }, + "identityPrincipalId": { + "type": "string", + "metadata": { + "description": "Principal ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.principalId.value, '')]" + } + } + } + }, + "dependsOn": [ + "appRegistration" + ] + }, + "uploadSettings": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core_Storage.UpdateSettings", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[reference('configContainer').outputs.identityName.value]" + }, + "scriptName": { + "value": "[format('{0}_uploadSettings', parameters('app').storage)]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "environmentVariables": { + "value": [ + { + "name": "ftkVersion", + "value": "[variables('finOpsToolkitVersion')]" + }, + { + "name": "scopes", + "value": "[join(parameters('scopesToMonitor'), '|')]" + }, + { + "name": "msexportRetentionInDays", + "value": "[string(parameters('msexportRetentionInDays'))]" + }, + { + "name": "ingestionRetentionInMonths", + "value": "[string(parameters('ingestionRetentionInMonths'))]" + }, + { + "name": "rawRetentionInDays", + "value": "[string(parameters('rawRetentionInDays'))]" + }, + { + "name": "finalRetentionInMonths", + "value": "[string(parameters('finalRetentionInMonths'))]" + }, + { + "name": "storageAccountName", + "value": "[parameters('app').storage]" + }, + { + "name": "containerName", + "value": "config" + } + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } } }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "languageVersion": "2.0", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "588615643779078900" + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" } - }, - "definitions": { - "EnvironmentVariable": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "value": { - "type": "string" - } - } - }, - "_1.HubProperties": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - }, - "location": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "tagsByResource": { - "type": "object" - }, - "version": { - "type": "string" - }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" - }, - "keyVaultSku": { - "type": "string" - }, - "networkAddressPrefix": { - "type": "string" - }, - "privateRouting": { - "type": "bool" - }, - "publisherIsolation": { - "type": "bool" - }, - "storageInfrastructureEncryption": { - "type": "bool" - }, - "storageSku": { - "type": "string" - } - } - }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" - }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" - } - } - } - }, - "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", - "location": "Azure resource location of the FinOps hub instance.", - "tags": "Tags to apply to all FinOps hub resources.", - "tagsByResource": "Tags to apply to resources based on their resource type.", - "version": "FinOps hub version number.", - "options": { - "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", - "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", - "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", - "privateRouting": "Indicates whether private network routing is enabled.", - "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", - "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", - "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." - }, - "routing": "FinOps hub private network routing properties, if enabled.", - "core": { - "suffix": "Unique suffix used for shared resources." - }, - "apps": {}, - "description": "FinOps hub instance properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" }, - "_1.HubRoutingProperties": { - "type": "object", - "properties": { - "networkId": { - "type": "string" - }, - "networkName": { - "type": "string" - }, - "scriptStorage": { - "type": "string" - }, - "dnsZones": { - "type": "object", - "properties": { - "blob": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "dfs": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "queue": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "table": { - "$ref": "#/definitions/_1.IdNameObject" - } - } - }, - "subnets": { - "type": "object", - "properties": { - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "scripts": { - "type": "string" - }, - "storage": { - "type": "string" - } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } + "queue": { + "$ref": "#/definitions/_1.IdNameObject" }, - "HubAppProperties": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "storage": { - "type": "string" - } - }, - "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", - "dataFactory": "Name of the Data Factory instance for this publisher.", - "keyVault": "Name of the KeyVault instance for this publisher.", - "storage": "Name of the storage account for this publisher.", - "description": "FinOps hub app configuration settings.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } + "table": { + "$ref": "#/definitions/_1.IdNameObject" } - }, - "parameters": { - "app": { - "$ref": "#/definitions/HubAppProperties", - "metadata": { - "description": "Required. FinOps hub app the deployment script is being run for." - } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" }, - "identityName": { - "type": "string", - "metadata": { - "description": "Required. Name of the managed identity to create." - } - }, - "scriptName": { - "type": "string", - "defaultValue": "[deployment().name]", - "metadata": { - "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." - } - }, - "scriptContent": { - "type": "string", - "metadata": { - "description": "Required. Name of the deployment script to create." - } - }, - "arguments": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Additional arguments to pass into the deployment script." - } - }, - "environmentVariables": { - "type": "array", - "items": { - "$ref": "#/definitions/EnvironmentVariable" - }, - "defaultValue": [], - "metadata": { - "description": "Optional. Environment variables to use for the deployment script." - } - } - }, - "variables": { - "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", - "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', format('{0}cg', parameters('app').hub.routing.scriptStorage), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" - }, - "resources": { - "identity": { - "type": "Microsoft.ManagedIdentity/userAssignedIdentities", - "apiVersion": "2023-01-31", - "name": "[parameters('identityName')]", - "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", - "location": "[parameters('app').hub.location]" + "dataFactory": { + "type": "string" }, - "scriptStorageAccount": { - "condition": "[parameters('app').hub.options.privateRouting]", - "existing": true, - "type": "Microsoft.Storage/storageAccounts", - "apiVersion": "2022-09-01", - "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + "keyVault": { + "type": "string" }, - "identityRoleAssignments": { - "copy": { - "name": "identityRoleAssignments", - "count": "[length(variables('privateEndpointDeploymentRoles'))]" - }, - "condition": "[parameters('app').hub.options.privateRouting]", - "type": "Microsoft.Authorization/roleAssignments", - "apiVersion": "2022-04-01", - "scope": "[format('Microsoft.Storage/storageAccounts/{0}', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", - "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", - "properties": { - "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", - "principalId": "[reference('identity').principalId]", - "principalType": "ServicePrincipal" - }, - "dependsOn": [ - "identity" - ] + "scripts": { + "type": "string" }, - "script": { - "type": "Microsoft.Resources/deploymentScripts", - "apiVersion": "2023-08-01", - "name": "[parameters('scriptName')]", - "kind": "AzurePowerShell", - "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", - "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", - "identity": { - "type": "UserAssigned", - "userAssignedIdentities": { - "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} - } - }, - "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '9.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", - "dependsOn": [ - "identity", - "identityRoleAssignments" - ] + "storage": { + "type": "string" } } } }, - "dependsOn": [ - "identity" - ] - } - }, - "outputs": { - "containerName": { - "type": "string", - "metadata": { - "description": "The name of the storage container." - }, - "value": "[parameters('container')]" - }, - "filesUploaded": { - "type": "int", "metadata": { - "description": "The number of files uploaded to the storage container." - }, - "value": "[variables('fileCount')]" + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } }, - "identityId": { - "type": "string", - "metadata": { - "description": "Resource ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } }, - "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.id.value, '')]" - }, - "identityName": { - "type": "string", "metadata": { - "description": "Name of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." - }, - "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.name.value, '')]" + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } }, - "identityPrincipalId": { - "type": "string", - "metadata": { - "description": "Principal ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." - }, - "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.principalId.value, '')]" - } - } - } - }, - "dependsOn": [ - "appRegistration" - ] - }, - "exportContainer": { - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.CostManagement.Exports_Storage.ExportContainer", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "app": { - "value": "[reference('appRegistration').outputs.app.value]" - }, - "container": { - "value": "msexports" - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "languageVersion": "2.0", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "13960345490822271084" - } - }, - "definitions": { - "_1.HubProperties": { + "HubAppProperties": { "type": "object", "properties": { "id": { @@ -8044,227 +6372,38 @@ "name": { "type": "string" }, - "location": { + "publisher": { "type": "string" }, - "tags": { - "type": "object" + "suffix": { + "type": "string" }, - "tagsByResource": { + "tags": { "type": "object" }, - "version": { + "dataFactory": { "type": "string" }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" - }, - "keyVaultSku": { - "type": "string" - }, - "networkAddressPrefix": { - "type": "string" - }, - "privateRouting": { - "type": "bool" - }, - "publisherIsolation": { - "type": "bool" - }, - "storageInfrastructureEncryption": { - "type": "bool" - }, - "storageSku": { - "type": "string" - } - } + "keyVault": { + "type": "string" }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" + "storage": { + "type": "string" }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" - } - } + "hub": { + "$ref": "#/definitions/_1.HubProperties" } }, "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", - "location": "Azure resource location of the FinOps hub instance.", - "tags": "Tags to apply to all FinOps hub resources.", - "tagsByResource": "Tags to apply to resources based on their resource type.", - "version": "FinOps hub version number.", - "options": { - "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", - "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", - "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", - "privateRouting": "Indicates whether private network routing is enabled.", - "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", - "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", - "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." - }, - "routing": "FinOps hub private network routing properties, if enabled.", - "core": { - "suffix": "Unique suffix used for shared resources." - }, - "apps": {}, - "description": "FinOps hub instance properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.HubRoutingProperties": { - "type": "object", - "properties": { - "networkId": { - "type": "string" - }, - "networkName": { - "type": "string" - }, - "scriptStorage": { - "type": "string" - }, - "dnsZones": { - "type": "object", - "properties": { - "blob": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "dfs": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "queue": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "table": { - "$ref": "#/definitions/_1.IdNameObject" - } - } - }, - "subnets": { - "type": "object", - "properties": { - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "scripts": { - "type": "string" - }, - "storage": { - "type": "string" - } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubAppProperties": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "storage": { - "type": "string" - } - }, - "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", "dataFactory": "Name of the Data Factory instance for this publisher.", "keyVault": "Name of the KeyVault instance for this publisher.", "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", "description": "FinOps hub app configuration settings.", "__bicep_imported_from!": { "sourceTemplate": "hub-types.bicep" @@ -8276,1797 +6415,5970 @@ "app": { "$ref": "#/definitions/HubAppProperties", "metadata": { - "description": "Required. FinOps hub app that storage is getting updated for." + "description": "Required. FinOps hub app the deployment script is being run for." } }, - "container": { + "identityName": { "type": "string", "metadata": { - "description": "Required. Name of the storage container to create or update." + "description": "Required. Name of the managed identity to create." } }, - "files": { - "type": "object", - "defaultValue": {}, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", "metadata": { - "description": "Optional. Dictionary of key/value pairs for the files to upload to the specified container. The key is the target path under the container and the value is the contents of the file. Default: {} (no files to upload)." + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." } }, - "forceCreateBlobManagerIdentity": { - "type": "bool", - "defaultValue": false, + "scriptContent": { + "type": "string", "metadata": { - "description": "Optional. Indicates whether to create the blob manager user assigned identity even if files are not being uploaded. Default: false." + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." } } }, "variables": { - "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n# Get storage context\r\n$storageContext = @{\r\n Context = New-AzStorageContext -StorageAccountName $env:storageAccountName -UseConnectedAccount\r\n Container = $env:containerName\r\n}\r\n\r\n# Uploading files\r\n$files = $env:files | ConvertFrom-Json -Depth 10\r\nWrite-Output \"Uploading ${$files.PSObject.Properties.Count} files...\"\r\n$files.PSObject.Properties | ForEach-Object {\r\n $filePath = $_.Name\r\n $tempPath = \"./$($filePath -replace \"/\", \"_\")\"\r\n Write-Output \" Uploading $filePath...\"\r\n $_.Value | Out-File $tempPath\r\n Set-AzStorageBlobContent @storageContext -File $tempPath -Blob $filePath -Force | Out-Null\r\n}\r\n", - "fileCount": "[length(items(parameters('files')))]", - "hasFiles": "[greater(variables('fileCount'), 0)]" + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" }, "resources": { - "storageAccount::blobService::targetContainer": { - "type": "Microsoft.Storage/storageAccounts/blobServices/containers", - "apiVersion": "2022-09-01", - "name": "[format('{0}/{1}/{2}', parameters('app').storage, 'default', parameters('container'))]", - "properties": { - "publicAccess": "None", - "metadata": {} - } - }, - "storageAccount::blobService": { - "existing": true, - "type": "Microsoft.Storage/storageAccounts/blobServices", - "apiVersion": "2022-09-01", - "name": "[format('{0}/{1}', parameters('app').storage, 'default')]" + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" }, - "storageAccount": { + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", "existing": true, "type": "Microsoft.Storage/storageAccounts", "apiVersion": "2022-09-01", - "name": "[parameters('app').storage]" + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" }, - "identity": { - "condition": "[or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity'))]", - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "[format('{0}.Identity', deployment().name)]", + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "app": { - "value": "[parameters('app')]" - }, - "identityName": { - "value": "[format('{0}_blobManager', parameters('app').storage)]" - }, - "roleAssignmentResourceId": { - "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" - }, - "roles": { - "value": [ - "ba92f5b4-2d11-453d-a403-e96b0029c9fe", - "69566ab7-960f-475b-8e7c-b3118f30c6bd" - ] - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "languageVersion": "2.0", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "4534337491931150093" - } - }, - "definitions": { - "_1.HubProperties": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - }, - "location": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "tagsByResource": { - "type": "object" - }, - "version": { - "type": "string" - }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" - }, - "keyVaultSku": { - "type": "string" - }, - "networkAddressPrefix": { - "type": "string" - }, - "privateRouting": { - "type": "bool" - }, - "publisherIsolation": { - "type": "bool" - }, - "storageInfrastructureEncryption": { - "type": "bool" - }, - "storageSku": { - "type": "string" - } - } - }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" - }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" - } - } - } - }, - "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", - "location": "Azure resource location of the FinOps hub instance.", - "tags": "Tags to apply to all FinOps hub resources.", - "tagsByResource": "Tags to apply to resources based on their resource type.", - "version": "FinOps hub version number.", - "options": { - "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", - "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", - "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", - "privateRouting": "Indicates whether private network routing is enabled.", - "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", - "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", - "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." - }, - "routing": "FinOps hub private network routing properties, if enabled.", - "core": { - "suffix": "Unique suffix used for shared resources." - }, - "apps": {}, - "description": "FinOps hub instance properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.HubRoutingProperties": { - "type": "object", - "properties": { - "networkId": { - "type": "string" - }, - "networkName": { - "type": "string" - }, - "scriptStorage": { - "type": "string" - }, - "dnsZones": { - "type": "object", - "properties": { - "blob": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "dfs": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "queue": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "table": { - "$ref": "#/definitions/_1.IdNameObject" - } - } - }, - "subnets": { - "type": "object", - "properties": { - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "scripts": { - "type": "string" - }, - "storage": { - "type": "string" - } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubAppProperties": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "storage": { - "type": "string" - } - }, - "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", - "dataFactory": "Name of the Data Factory instance for this publisher.", - "keyVault": "Name of the KeyVault instance for this publisher.", - "storage": "Name of the storage account for this publisher.", - "description": "FinOps hub app configuration settings.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - }, - "functions": [ - { - "namespace": "__bicep", - "members": { - "getPublisherTags": { - "parameters": [ - { - "$ref": "#/definitions/HubAppProperties", - "name": "app" - }, - { - "type": "string", - "name": "resourceType" - } - ], - "output": { - "type": "object", - "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').publisher.tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" - }, - "metadata": { - "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - } - } - ], - "parameters": { - "app": { - "$ref": "#/definitions/HubAppProperties", - "metadata": { - "description": "Required. FinOps hub app the identity is associated with." - } - }, - "identityName": { - "type": "string", - "metadata": { - "description": "Required. Name of the user assigned identity." - } - }, - "roleAssignmentResourceId": { - "type": "string", - "metadata": { - "description": "Required. Resource ID of the resource access is being granted for." - } - }, - "roles": { - "type": "array", - "items": { - "type": "string" - }, - "metadata": { - "description": "Required. List of RBAC role assignment GUIDs." - } - } - }, - "resources": { - "identity": { - "type": "Microsoft.ManagedIdentity/userAssignedIdentities", - "apiVersion": "2023-01-31", - "name": "[parameters('identityName')]", - "tags": "[__bicep.getPublisherTags(parameters('app'), 'Microsoft.ManagedIdentity/userAssignedIdentities')]", - "location": "[parameters('app').hub.location]" - }, - "identityRoleAssignments": { - "copy": { - "name": "identityRoleAssignments", - "count": "[length(parameters('roles'))]" - }, - "type": "Microsoft.Authorization/roleAssignments", - "apiVersion": "2022-04-01", - "name": "[guid(parameters('roleAssignmentResourceId'), parameters('roles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", - "properties": { - "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', parameters('roles')[copyIndex()])]", - "principalId": "[reference('identity').principalId]", - "principalType": "ServicePrincipal" - }, - "dependsOn": [ - "identity" - ] - } - }, - "outputs": { - "id": { - "type": "string", - "metadata": { - "description": "Resource ID of the user assigned identity." - }, - "value": "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName'))]" - }, - "name": { - "type": "string", - "metadata": { - "description": "Name of the user assigned identity." - }, - "value": "[parameters('identityName')]" - }, - "principalId": { - "type": "string", - "metadata": { - "description": "Principal ID of the user assigned identity." - }, - "value": "[reference('identity').principalId]" - } - } - } - } - }, - "uploadFiles": { - "condition": "[variables('hasFiles')]", - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "[format('{0}.Upload', deployment().name)]", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "app": { - "value": "[parameters('app')]" - }, - "identityName": { - "value": "[reference('identity').outputs.name.value]" - }, - "environmentVariables": { - "value": [ - { - "name": "storageAccountName", - "value": "[parameters('app').storage]" - }, - { - "name": "containerName", - "value": "[parameters('container')]" - }, - { - "name": "files", - "value": "[string(parameters('files'))]" - } - ] - }, - "scriptContent": { - "value": "[variables('$fxv#0')]" - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "languageVersion": "2.0", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "588615643779078900" - } - }, - "definitions": { - "EnvironmentVariable": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "value": { - "type": "string" - } - } - }, - "_1.HubProperties": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - }, - "location": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "tagsByResource": { - "type": "object" - }, - "version": { - "type": "string" - }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" - }, - "keyVaultSku": { - "type": "string" - }, - "networkAddressPrefix": { - "type": "string" - }, - "privateRouting": { - "type": "bool" - }, - "publisherIsolation": { - "type": "bool" - }, - "storageInfrastructureEncryption": { - "type": "bool" - }, - "storageSku": { - "type": "string" - } - } - }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" - }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" - } - } - } - }, - "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", - "location": "Azure resource location of the FinOps hub instance.", - "tags": "Tags to apply to all FinOps hub resources.", - "tagsByResource": "Tags to apply to resources based on their resource type.", - "version": "FinOps hub version number.", - "options": { - "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", - "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", - "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", - "privateRouting": "Indicates whether private network routing is enabled.", - "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", - "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", - "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." - }, - "routing": "FinOps hub private network routing properties, if enabled.", - "core": { - "suffix": "Unique suffix used for shared resources." - }, - "apps": {}, - "description": "FinOps hub instance properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.HubRoutingProperties": { - "type": "object", - "properties": { - "networkId": { - "type": "string" - }, - "networkName": { - "type": "string" - }, - "scriptStorage": { - "type": "string" - }, - "dnsZones": { - "type": "object", - "properties": { - "blob": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "dfs": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "queue": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "table": { - "$ref": "#/definitions/_1.IdNameObject" - } - } - }, - "subnets": { - "type": "object", - "properties": { - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "scripts": { - "type": "string" - }, - "storage": { - "type": "string" - } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubAppProperties": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "storage": { - "type": "string" - } - }, - "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", - "dataFactory": "Name of the Data Factory instance for this publisher.", - "keyVault": "Name of the KeyVault instance for this publisher.", - "storage": "Name of the storage account for this publisher.", - "description": "FinOps hub app configuration settings.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - }, - "parameters": { - "app": { - "$ref": "#/definitions/HubAppProperties", - "metadata": { - "description": "Required. FinOps hub app the deployment script is being run for." - } - }, - "identityName": { - "type": "string", - "metadata": { - "description": "Required. Name of the managed identity to create." - } - }, - "scriptName": { - "type": "string", - "defaultValue": "[deployment().name]", - "metadata": { - "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." - } - }, - "scriptContent": { - "type": "string", - "metadata": { - "description": "Required. Name of the deployment script to create." - } - }, - "arguments": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Additional arguments to pass into the deployment script." - } - }, - "environmentVariables": { - "type": "array", - "items": { - "$ref": "#/definitions/EnvironmentVariable" - }, - "defaultValue": [], - "metadata": { - "description": "Optional. Environment variables to use for the deployment script." - } - } - }, - "variables": { - "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", - "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', format('{0}cg', parameters('app').hub.routing.scriptStorage), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" - }, - "resources": { - "identity": { - "type": "Microsoft.ManagedIdentity/userAssignedIdentities", - "apiVersion": "2023-01-31", - "name": "[parameters('identityName')]", - "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", - "location": "[parameters('app').hub.location]" - }, - "scriptStorageAccount": { - "condition": "[parameters('app').hub.options.privateRouting]", - "existing": true, - "type": "Microsoft.Storage/storageAccounts", - "apiVersion": "2022-09-01", - "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" - }, - "identityRoleAssignments": { - "copy": { - "name": "identityRoleAssignments", - "count": "[length(variables('privateEndpointDeploymentRoles'))]" - }, - "condition": "[parameters('app').hub.options.privateRouting]", - "type": "Microsoft.Authorization/roleAssignments", - "apiVersion": "2022-04-01", - "scope": "[format('Microsoft.Storage/storageAccounts/{0}', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", - "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", - "properties": { - "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", - "principalId": "[reference('identity').principalId]", - "principalType": "ServicePrincipal" - }, - "dependsOn": [ - "identity" - ] - }, - "script": { - "type": "Microsoft.Resources/deploymentScripts", - "apiVersion": "2023-08-01", - "name": "[parameters('scriptName')]", - "kind": "AzurePowerShell", - "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", - "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", - "identity": { - "type": "UserAssigned", - "userAssignedIdentities": { - "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} - } - }, - "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '9.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", - "dependsOn": [ - "identity", - "identityRoleAssignments" - ] - } - } - } + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" }, "dependsOn": [ "identity" ] - } - }, - "outputs": { - "containerName": { - "type": "string", - "metadata": { - "description": "The name of the storage container." - }, - "value": "[parameters('container')]" - }, - "filesUploaded": { - "type": "int", - "metadata": { - "description": "The number of files uploaded to the storage container." - }, - "value": "[variables('fileCount')]" - }, - "identityId": { - "type": "string", - "metadata": { - "description": "Resource ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." - }, - "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.id.value, '')]" - }, - "identityName": { - "type": "string", - "metadata": { - "description": "Name of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." - }, - "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.name.value, '')]" }, - "identityPrincipalId": { - "type": "string", - "metadata": { - "description": "Principal ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } }, - "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.principalId.value, '')]" + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] } } } }, "dependsOn": [ - "appRegistration" + "appRegistration", + "configContainer" ] } }, "outputs": { - "exportContainer": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Properties of the hub app." + }, + "value": "[parameters('app')]" + }, + "dataFactoryName": { "type": "string", "metadata": { - "description": "Name of the container used for Cost Management exports." + "description": "Name of the Data Factory." }, - "value": "[reference('exportContainer').outputs.containerName.value]" + "value": "[parameters('app').dataFactory]" }, - "schemaFilesUploaded": { - "type": "int", + "storageAccountName": { + "type": "string", "metadata": { - "description": "Number of schema files uploaded." + "description": "Name of the storage account created for the hub instance. This must be used when connecting FinOps toolkit Power BI reports to your data." }, - "value": "[reference('schemaFiles').outputs.filesUploaded.value]" + "value": "[parameters('app').storage]" + }, + "storageUrlForPowerBI": { + "type": "string", + "metadata": { + "description": "URL to use when connecting custom Power BI reports to your data." + }, + "value": "[format('https://{0}.dfs.{1}/{2}', parameters('app').storage, environment().suffixes.storage, variables('INGESTION'))]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Object ID of the Data Factory managed identity. This will be needed when configuring managed exports." + }, + "value": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" + }, + "triggerManagerIdentityName": { + "type": "string", + "metadata": { + "description": "Name of the managed identity used to create and stop ADF triggers." + }, + "value": "[reference('appRegistration').outputs.triggerManagerIdentityName.value]" } } } - }, - "dependsOn": [ - "core" - ] + } }, - "dataExplorer": { - "condition": "[variables('deployDataExplorer')]", + "cmExports": { "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "dataExplorer", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.Exports", "properties": { "expressionEvaluationOptions": { "scope": "inner" }, "mode": "Incremental", "parameters": { - "clusterName": { - "value": "[parameters('dataExplorerName')]" - }, - "clusterSku": { - "value": "[parameters('dataExplorerSku')]" - }, - "clusterCapacity": { - "value": "[parameters('dataExplorerCapacity')]" - }, - "location": { - "value": "[parameters('location')]" - }, - "tags": { - "value": "[variables('hub').tags]" - }, - "tagsByResource": { - "value": "[parameters('tagsByResource')]" - }, - "dataFactoryName": { - "value": "[reference('core').outputs.dataFactoryName.value]" - }, - "rawRetentionInDays": { - "value": "[parameters('dataExplorerRawRetentionInDays')]" - }, - "virtualNetworkId": "[if(parameters('enablePublicAccess'), createObject('value', ''), createObject('value', reference('infrastructure').outputs.vNetId.value))]", - "privateEndpointSubnetId": "[if(parameters('enablePublicAccess'), createObject('value', ''), createObject('value', reference('infrastructure').outputs.dataExplorerSubnetId.value))]", - "enablePublicAccess": { - "value": "[parameters('enablePublicAccess')]" - }, - "storageAccountName": { - "value": "[reference('core').outputs.storageAccountName.value]" + "app": { + "value": "[__bicep.newApp(variables('hub'), 'Microsoft.CostManagement', 'Exports')]" } }, "template": { "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", "contentVersion": "1.0.0.0", "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "12711851392414163333" + "version": "0.40.2.10011", + "templateHash": "13738069470076156306" } }, - "parameters": { - "clusterName": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Name of the Azure Data Explorer cluster to use for advanced analytics. If empty, Azure Data Explorer will not be deployed. Required to use with Power BI if you have more than $2-5M/mo in costs being monitored. Default: \"\" (do not use)." - } - }, - "clusterSku": { - "type": "string", - "defaultValue": "Dev(No SLA)_Standard_E2a_v4", - "allowedValues": [ - "Dev(No SLA)_Standard_E2a_v4", - "Dev(No SLA)_Standard_D11_v2", - "Standard_D11_v2", - "Standard_D12_v2", - "Standard_D13_v2", - "Standard_D14_v2", - "Standard_D16d_v5", - "Standard_D32d_v4", - "Standard_D32d_v5", - "Standard_DS13_v2+1TB_PS", - "Standard_DS13_v2+2TB_PS", - "Standard_DS14_v2+3TB_PS", - "Standard_DS14_v2+4TB_PS", - "Standard_E2a_v4", - "Standard_E2ads_v5", - "Standard_E2d_v4", - "Standard_E2d_v5", - "Standard_E4a_v4", - "Standard_E4ads_v5", - "Standard_E4d_v4", - "Standard_E4d_v5", - "Standard_E8a_v4", - "Standard_E8ads_v5", - "Standard_E8as_v4+1TB_PS", - "Standard_E8as_v4+2TB_PS", - "Standard_E8as_v5+1TB_PS", - "Standard_E8as_v5+2TB_PS", - "Standard_E8d_v4", - "Standard_E8d_v5", - "Standard_E8s_v4+1TB_PS", - "Standard_E8s_v4+2TB_PS", - "Standard_E8s_v5+1TB_PS", - "Standard_E8s_v5+2TB_PS", - "Standard_E16a_v4", - "Standard_E16ads_v5", - "Standard_E16as_v4+3TB_PS", - "Standard_E16as_v4+4TB_PS", - "Standard_E16as_v5+3TB_PS", - "Standard_E16as_v5+4TB_PS", - "Standard_E16d_v4", - "Standard_E16d_v5", - "Standard_E16s_v4+3TB_PS", - "Standard_E16s_v4+4TB_PS", - "Standard_E16s_v5+3TB_PS", - "Standard_E16s_v5+4TB_PS", - "Standard_E64i_v3", - "Standard_E80ids_v4", - "Standard_EC8ads_v5", - "Standard_EC8as_v5+1TB_PS", - "Standard_EC8as_v5+2TB_PS", - "Standard_EC16ads_v5", - "Standard_EC16as_v5+3TB_PS", - "Standard_EC16as_v5+4TB_PS", - "Standard_L4s", - "Standard_L8as_v3", - "Standard_L8s", - "Standard_L8s_v2", - "Standard_L8s_v3", - "Standard_L16as_v3", - "Standard_L16s", - "Standard_L16s_v2", - "Standard_L16s_v3", - "Standard_L32as_v3", - "Standard_L32s_v3" - ], + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, "metadata": { - "description": "Optional. Name of the Azure Data Explorer SKU. Default: \"Dev(No SLA)_Standard_E2a_v4\"." + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } } }, - "clusterCapacity": { - "type": "int", - "defaultValue": 1, - "minValue": 1, - "maxValue": 1000, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, "metadata": { - "description": "Optional. Number of nodes to use in the cluster. Allowed values: 1 for the Basic SKU tier and 2-1000 for Standard. Default: 1 for dev/test SKUs, 2 for standard SKUs." + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } } }, - "forceUpdateTag": { - "type": "string", - "defaultValue": "[utcNow()]", + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, "metadata": { - "description": "Optional. Forces the table to be updated if different from the last time it was deployed." + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } } }, - "continueOnErrors": { - "type": "bool", - "defaultValue": false, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, "metadata": { - "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } } - }, - "location": { - "type": "string", - "defaultValue": "[resourceGroup().location]", + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", "metadata": { - "description": "Optional. Azure location to use for the managed identity and deployment script to auto-start triggers. Default: (resource group location)." + "description": "Required. FinOps hub app getting deployed." } + } + }, + "variables": { + "$fxv#0": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\":{\"name\":\"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingPeriodStartDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStartDate\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingPeriodEndDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEndDate\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountOwnerId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"SubscriptionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubscriptionId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"SubscriptionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubscriptionName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Date\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"Date\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Product\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Product\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PartNumber\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceFamily\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterCategory\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterSubCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterSubCategory\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterRegion\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Quantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Quantity\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"EffectivePrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectivePrice\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Cost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Cost\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"UnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UnitPrice\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceLocation\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceLocation\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AvailabilityZone\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AvailabilityZone\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ConsumedService\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedService\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceInfo1\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceInfo1\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceInfo2\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceInfo2\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AdditionalInfo\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AdditionalInfo\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSectionId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"InvoiceSection\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSection\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CostCenter\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"UnitOfMeasure\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"UnitOfMeasure\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceGroup\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceGroup\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ReservationId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ReservationName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ProductOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductOrderId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ProductOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductOrderName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"OfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"OfferId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"IsAzureCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"IsAzureCreditEligible\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PlanName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PlanName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ChargeType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeType\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Frequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Frequency\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PublisherType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherType\"}\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#1": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\":{\"name\":\"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingPeriodStartDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStartDate\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingPeriodEndDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEndDate\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountOwnerId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"SubscriptionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubscriptionId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"SubscriptionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubscriptionName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Date\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"Date\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Product\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Product\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PartNumber\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceFamily\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterCategory\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterSubCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterSubCategory\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterRegion\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"MeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Quantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Quantity\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"EffectivePrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectivePrice\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Cost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Cost\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"UnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UnitPrice\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceLocation\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceLocation\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AvailabilityZone\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AvailabilityZone\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ConsumedService\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedService\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceInfo1\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceInfo1\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ServiceInfo2\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceInfo2\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"AdditionalInfo\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AdditionalInfo\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSectionId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"InvoiceSection\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSection\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CostCenter\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"UnitOfMeasure\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"UnitOfMeasure\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ResourceGroup\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceGroup\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ReservationId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ReservationName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ProductOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductOrderId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ProductOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductOrderName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"OfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"OfferId\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"IsAzureCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"IsAzureCreditEligible\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PlanName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PlanName\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"ChargeType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeType\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"Frequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Frequency\"}\r\n },\r\n {\r\n \"source\":{\"name\":\"PublisherType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherType\"}\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#10": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"SKU\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SKU\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Location\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Location\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CostWithNoReservedInstances\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CostWithNoReservedInstances\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"FirstUsageDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"FirstUsageDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InstanceFlexibilityRatio\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityRatio\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InstanceFlexibilityGroup\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityGroup\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"LookBackPeriod\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"LookBackPeriod\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"NetSavings\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"NetSavings\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"NormalizedSize\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"NormalizedSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RecommendedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"RecommendedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RecommendedQuantityNormalized\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"RecommendedQuantityNormalized\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Scope\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Scope\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuProperties\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuProperties\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"TotalCostWithReservedInstances\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"TotalCostWithReservedInstances\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#11": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"Cost With No ReservedInstances\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CostWithNoReservedInstancesJson\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"First UsageDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"FirstUsageDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Instance Flexibility Ratio\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityRatio\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Instance Flexibility Group\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityGroup\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Location\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Location\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"LookBackPeriod\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"LookBackPeriod\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterID\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Net Savings\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"NetSavingsJson\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Normalized Size\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"NormalizedSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Recommended Quantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"RecommendedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Recommended Quantity Normalized\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"RecommendedQuantityNormalized\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"scope\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Scope\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Sku Properties\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuProperties\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Total Cost With ReservedInstances\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"TotalCostWithReservedInstancesJson\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#12": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"AccountOwnerEmail\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AccountOwnerEmail\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Amount\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Amount\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ArmSkuName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ArmSkuName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingMonth\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingMonth\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Currency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Currency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CurrentEnrollmentId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CurrentEnrollmentId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"DepartmentName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"DepartmentName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Description\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Description\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EventDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EventDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EventType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"EventType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MonetaryCommitment\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"MonetaryCommitment\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Overage\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Overage\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PurchasingSubscriptionGuid\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PurchasingSubscriptionGuid\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PurchasingSubscriptionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PurchasingSubscriptionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PurchasingEnrollment\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PurchasingEnrollment\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Quantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Quantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Region\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Region\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#13": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"Amount\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Amount\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ArmSkuName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ArmSkuName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Currency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Currency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Description\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Description\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EventDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EventDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EventType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"EventType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Invoice\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Invoice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PurchasingSubscriptionGuid\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PurchasingSubscriptionGuid\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PurchasingSubscriptionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PurchasingSubscriptionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Quantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"Quantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Region\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Region\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#2": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"BilledCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BilledCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CapacityReservationId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CapacityReservationId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CapacityReservationStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CapacityReservationStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ConsumedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectiveCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceIssuerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceIssuerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"PricingQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProviderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProviderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuMeter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuMeter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountOwnerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AmortizationClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AmortizationClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRateDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRateDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ContractedCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ContractedCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostAllocationRuleName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostAllocationRuleName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceIssuerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceIssuerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ListCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ListCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditApplied\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditApplied\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditRate\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingBlockSize\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PricingBlockSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingUnitDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingUnitDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceGroupName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceGroupName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServiceModel\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ServiceModel\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuIsCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_SkuIsCreditEligible\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOfferId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPlanName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPlanName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTerm\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTerm\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTier\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTier\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#3": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"BilledCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BilledCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CapacityReservationId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CapacityReservationId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CapacityReservationStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CapacityReservationStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ConsumedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectiveCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceIssuerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceIssuerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"PricingQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProviderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProviderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuMeter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuMeter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountOwnerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AmortizationClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AmortizationClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRateDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRateDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ContractedCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ContractedCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostAllocationRuleName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostAllocationRuleName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceIssuerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceIssuerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ListCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ListCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditApplied\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditApplied\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditRate\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingBlockSize\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PricingBlockSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingUnitDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingUnitDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceGroupName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceGroupName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServiceModel\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ServiceModel\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuIsCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_SkuIsCreditEligible\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOfferId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPlanName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPlanName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTerm\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTerm\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTier\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTier\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#4": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"BilledCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BilledCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ConsumedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectiveCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceIssuerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceIssuerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"PricingQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProviderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProviderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountOwnerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRateDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRateDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ContractedCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ContractedCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostAllocationRuleName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostAllocationRuleName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceIssuerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceIssuerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ListCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ListCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditApplied\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditApplied\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditRate\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingBlockSize\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PricingBlockSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingUnitDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingUnitDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceGroupName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceGroupName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuIsCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_SkuIsCreditEligible\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOfferId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTerm\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTerm\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTier\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTier\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#5": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"BilledCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BilledCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeClass\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeClass\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountStatus\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountStatus\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ConsumedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ConsumedUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ConsumedUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ContractedUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ContractedUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectiveCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceIssuerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceIssuerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"PricingQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProviderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProviderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"RegionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"RegionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountOwnerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRateDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRateDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ContractedCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ContractedCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostAllocationRuleName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostAllocationRuleName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceIssuerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceIssuerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ListCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_ListCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditApplied\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditApplied\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditRate\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingBlockSize\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PricingBlockSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingUnitDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingUnitDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceGroupName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceGroupName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuIsCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_SkuIsCreditEligible\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOfferId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTerm\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTerm\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTier\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTier\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#6": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"AvailabilityZone\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"AvailabilityZone\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BilledCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BilledCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingPeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"BillingPeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeFrequency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeFrequency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"ChargePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ChargeSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ChargeSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CommitmentDiscountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CommitmentDiscountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"EffectiveCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InvoiceIssuerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InvoiceIssuerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ListUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ListUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"PricingQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PricingUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PricingUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProviderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProviderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PublisherName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PublisherName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Region\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Region\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuPriceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuPriceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SubAccountType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SubAccountType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Tags\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Tags\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UsageQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UsageQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UsageUnit\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"UsageUnit\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_AccountOwnerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_AccountOwnerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BilledUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BilledUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingExchangeRateDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_BillingExchangeRateDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ChargeId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ChargeId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostAllocationRuleName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostAllocationRuleName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CostCenter\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CostCenter\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_CustomerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_CustomerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_EffectiveUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_EffectiveUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceIssuerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceIssuerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_InvoiceSectionName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_InvoiceSectionName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_OnDemandCost\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_OnDemandCost\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_OnDemandCostInUsd\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_OnDemandCostInUsd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_OnDemandUnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_OnDemandUnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditApplied\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditApplied\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PartnerCreditRate\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PartnerCreditRate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingBlockSize\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"x_PricingBlockSize\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PricingUnitDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PricingUnitDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_PublisherId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_PublisherId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResellerName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResellerName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceGroupName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceGroupName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ResourceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_ResourceType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodEnd\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodEnd\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_ServicePeriodStart\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"x_ServicePeriodStart\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDescription\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDescription\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuDetails\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuDetails\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuIsCreditEligible\", \"type\": \"Boolean\" },\r\n \"sink\": { \"name\": \"x_SkuIsCreditEligible\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuMeterSubcategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuMeterSubcategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOfferId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOfferId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuOrderName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuOrderName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuPartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuPartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTerm\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTerm\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"x_SkuTier\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"x_SkuTier\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#7": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"EnrollmentNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"EnrollmentNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterID\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterID\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterSubCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterSubCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Product\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Product\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuID\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuID\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProductID\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductID\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UnitOfMeasure\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"UnitOfMeasure\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PartNumber\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PartNumber\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveStartDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EffectiveStartDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveEndDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EffectiveEndDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BasePrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BasePrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MarketPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"MarketPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"CurrencyCode\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"CurrencyCode\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"IncludedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"IncludedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"OfferID\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"OfferID\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PriceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PriceType\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#8": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"BillingAccountId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingAccountName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingAccountName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingProfileId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingProfileName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingProfileName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ServiceFamily\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ServiceFamily\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Product\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Product\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ProductId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ProductId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UnitOfMeasure\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"UnitOfMeasure\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterType\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterSubCategory\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterSubCategory\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MeterRegion\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"MeterRegion\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"TierMinimumUnits\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"TierMinimumUnits\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveStartDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EffectiveStartDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"EffectiveEndDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"EffectiveEndDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UnitPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UnitPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BasePrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"BasePrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"MarketPrice\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"MarketPrice\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Currency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Currency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"BillingCurrency\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"BillingCurrency\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Term\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Term\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"PriceType\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"PriceType\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "$fxv#9": "{\r\n \"additionalColumns\": [],\r\n \"translator\": {\r\n \"type\": \"TabularTranslator\",\r\n \"mappings\": [\r\n {\r\n \"source\": { \"name\": \"InstanceFlexibilityGroup\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityGroup\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InstanceFlexibilityRatio\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"InstanceFlexibilityRatio\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"InstanceId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"InstanceId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"Kind\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"Kind\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationOrderId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationOrderId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservationId\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"ReservationId\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"ReservedHours\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"ReservedHours\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"SkuName\", \"type\": \"String\" },\r\n \"sink\": { \"name\": \"SkuName\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"TotalReservedQuantity\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"TotalReservedQuantity\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UsageDate\", \"type\": \"DateTimeOffset\" },\r\n \"sink\": { \"name\": \"UsageDate\" }\r\n },\r\n {\r\n \"source\": { \"name\": \"UsedHours\", \"type\": \"Decimal\" },\r\n \"sink\": { \"name\": \"UsedHours\" }\r\n }\r\n ]\r\n }\r\n}\r\n", + "CONFIG": "config", + "INGESTION": "ingestion", + "MSEXPORTS": "msexports", + "ingestionIdFileNameSeparator": "__", + "finOpsToolkitVersion": "13.0" + }, + "resources": { + "dataFactory::linkedService_storageAccount": { + "existing": true, + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').storage)]", + "dependsOn": [ + "appRegistration" + ] }, - "tags": { - "type": "object", - "defaultValue": {}, - "metadata": { - "description": "Optional. Tags to apply to all resources." - } + "dataFactory::dataset_config": { + "existing": true, + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, variables('CONFIG'))]", + "dependsOn": [ + "appRegistration" + ] }, - "tagsByResource": { - "type": "object", - "defaultValue": {}, - "metadata": { - "description": "Optional. Tags to apply to resources based on their resource type. Resource type specific tags will be merged with tags for all resources." - } + "dataFactory::dataset_ingestion": { + "existing": true, + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, variables('INGESTION'))]", + "dependsOn": [ + "appRegistration" + ] + }, + "dataFactory::dataset_ingestion_files": { + "existing": true, + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_files', variables('INGESTION')))]", + "dependsOn": [ + "appRegistration" + ] }, - "dataFactoryName": { - "type": "string", - "metadata": { - "description": "Required. Name of the Data Factory instance." - } + "dataFactory::dataset_ingestion_manifest": { + "existing": true, + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ingestion_manifest')]", + "dependsOn": [ + "appRegistration" + ] }, - "rawRetentionInDays": { - "type": "int", - "defaultValue": 0, - "metadata": { - "description": "Optional. Number of days of data to retain in the Data Explorer *_raw tables. Default: 0." - } + "dataFactory::dataset_msexports_manifest": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'msexports_manifest')]", + "properties": { + "parameters": { + "fileName": { + "type": "String", + "defaultValue": "manifest.json" + }, + "folderPath": { + "type": "String", + "defaultValue": "[variables('MSEXPORTS')]" + } + }, + "type": "Json", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().fileName}", + "type": "Expression" + }, + "folderPath": { + "value": "@{dataset().folderPath}", + "type": "Expression" + } + } + }, + "linkedServiceName": { + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" + } + }, + "dependsOn": [ + "appRegistration" + ] }, - "storageAccountName": { - "type": "string", - "metadata": { - "description": "Required. Name of the storage account to use for data ingestion." - } + "dataFactory::dataset_msexports": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, replace(format('{0}', variables('MSEXPORTS')), '-', '_'))]", + "properties": { + "parameters": { + "blobPath": { + "type": "String" + } + }, + "type": "DelimitedText", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().blobPath}", + "type": "Expression" + }, + "fileSystem": "[reference('exportContainer').outputs.containerName.value]" + }, + "columnDelimiter": ",", + "escapeChar": "\"", + "quoteChar": "\"", + "firstRowAsHeader": true + }, + "linkedServiceName": { + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" + } + }, + "dependsOn": [ + "appRegistration", + "exportContainer" + ] }, - "virtualNetworkId": { - "type": "string", - "metadata": { - "description": "Required. Resource ID of the virtual network for private endpoints." - } + "dataFactory::dataset_msexports_gzip": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_gzip', variables('MSEXPORTS')))]", + "properties": { + "parameters": { + "blobPath": { + "type": "String" + } + }, + "type": "DelimitedText", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().blobPath}", + "type": "Expression" + }, + "fileSystem": "[variables('MSEXPORTS')]" + }, + "columnDelimiter": ",", + "escapeChar": "\"", + "quoteChar": "\"", + "firstRowAsHeader": true, + "compressionCodec": "Gzip" + }, + "linkedServiceName": { + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" + } + }, + "dependsOn": [ + "appRegistration" + ] }, - "privateEndpointSubnetId": { - "type": "string", - "metadata": { - "description": "Required. Resource ID of the subnet for private endpoints." - } + "dataFactory::dataset_msexports_parquet": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_parquet', variables('MSEXPORTS')))]", + "properties": { + "parameters": { + "blobPath": { + "type": "String" + } + }, + "type": "Parquet", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().blobPath}", + "type": "Expression" + }, + "fileSystem": "[variables('MSEXPORTS')]" + } + }, + "linkedServiceName": { + "referenceName": "[parameters('app').storage]", + "type": "LinkedServiceReference" + } + }, + "dependsOn": [ + "appRegistration" + ] }, - "enablePublicAccess": { - "type": "bool", - "metadata": { - "description": "Optional. Enable public access." - } - } - }, - "variables": { - "$fxv#0": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData/Internal')\r\n_resource_type_1(id: string) {\r\n dynamic({\r\n \"arizeai.observabilityeval/organizations\": { \"SingularDisplayName\": \"Azure Native Arize AI Cloud Service\" }\r\n ,\"astronomer.astro/organizations\": { \"SingularDisplayName\": \"Astro Organization\" }\r\n ,\"citrix.services/xenappessentials\": { \"SingularDisplayName\": \"Citrix Virtual Apps Essentials\" }\r\n ,\"citrix.services/xendesktopessentials\": { \"SingularDisplayName\": \"Citrix Virtual Desktops Essentials\" }\r\n ,\"commvault.contentstore/cloudaccounts\": { \"SingularDisplayName\": \"Commvault Cloud Account\" }\r\n ,\"commvault.contentstore/cloudaccounts/plans\": { \"SingularDisplayName\": \"Commvault.ContentStore cloud accounts plan\" }\r\n ,\"commvault.contentstore/cloudaccounts/protectiongroups\": { \"SingularDisplayName\": \"Commvault.ContentStore cloud accounts protection group\" }\r\n ,\"commvault.contentstore/cloudaccounts/protectiongroups/protecteditems\": { \"SingularDisplayName\": \"Commvault.ContentStore cloud accounts protection groups protected item\" }\r\n ,\"commvault.contentstore/cloudaccounts/storages\": { \"SingularDisplayName\": \"Commvault.ContentStore cloud accounts storage\" }\r\n ,\"dell.storage/filesystems\": { \"SingularDisplayName\": \"Dell PowerScale\" }\r\n ,\"dynatrace.observability/monitors\": { \"SingularDisplayName\": \"Dynatrace\" }\r\n ,\"github.network/networksettings\": { \"SingularDisplayName\": \"GitHub.Network network setting\" }\r\n ,\"informatica.datamanagement/organizations\": { \"SingularDisplayName\": \"Informatica Organization\" }\r\n ,\"lambdatest.hyperexecute/organizations\": { \"SingularDisplayName\": \"Azure Native LambdaTest - HyperExecute Cloud Service\" }\r\n ,\"microsoft.aad/domainservices\": { \"SingularDisplayName\": \"Microsoft Entra Domain Services\" }\r\n ,\"microsoft.aadiam/diagnosticsettings\": { \"SingularDisplayName\": \"Microsoft.aadiam diagnostic setting\" }\r\n ,\"microsoft.aadiam/privatelinkforazuread\": { \"SingularDisplayName\": \"Private Link for Microsoft Entra ID\" }\r\n ,\"microsoft.advisor/advisorscore\": { \"SingularDisplayName\": \"Microsoft.Advisor advisor score\" }\r\n ,\"microsoft.advisor/assessments\": { \"SingularDisplayName\": \"Microsoft.Advisor assessment\" }\r\n ,\"microsoft.advisor/configurations\": { \"SingularDisplayName\": \"Microsoft.Advisor configuration\" }\r\n ,\"microsoft.advisor/generaterecommendations\": { \"SingularDisplayName\": \"Microsoft.Advisor generate recommendation\" }\r\n ,\"microsoft.advisor/metadata\": { \"SingularDisplayName\": \"Microsoft.Advisor metadata\" }\r\n ,\"microsoft.advisor/recommendations\": { \"SingularDisplayName\": \"Microsoft.Advisor recommendation\" }\r\n ,\"microsoft.advisor/recommendations/suppressions\": { \"SingularDisplayName\": \"Microsoft.Advisor recommendations suppression\" }\r\n ,\"microsoft.advisor/resiliencyreviews\": { \"SingularDisplayName\": \"Microsoft.Advisor resiliency review\" }\r\n ,\"microsoft.agfoodplatform/farmbeats\": { \"SingularDisplayName\": \"Azure Data Manager for Agriculture\" }\r\n ,\"microsoft.agfoodplatform/farmbeatsextensiondefinitions\": { \"SingularDisplayName\": \"Microsoft.AgFoodPlatform farm beats extension definition\" }\r\n ,\"microsoft.agfoodplatform/farmbeatssolutiondefinitions\": { \"SingularDisplayName\": \"Microsoft.AgFoodPlatform farm beats solution definition\" }\r\n ,\"microsoft.agricultureplatform/agriservices\": { \"SingularDisplayName\": \"Agriculture data solutions\" }\r\n ,\"microsoft.akshybrid/agentpools\": { \"SingularDisplayName\": \"Microsoft.AksHybrid agent pool\" }\r\n ,\"microsoft.akshybrid/provisionedclusters\": { \"SingularDisplayName\": \"Microsoft.AksHybrid provisioned cluster\" }\r\n ,\"microsoft.akshybrid/upgradeprofiles\": { \"SingularDisplayName\": \"Microsoft.AksHybrid upgrade profile\" }\r\n ,\"microsoft.alertsmanagement/actionrules\": { \"SingularDisplayName\": \"Alert processing rule\" }\r\n ,\"microsoft.alertsmanagement/alerts\": { \"SingularDisplayName\": \"Microsoft.AlertsManagement alert\" }\r\n ,\"microsoft.alertsmanagement/alerts/enrichments\": { \"SingularDisplayName\": \"Microsoft.AlertsManagement alerts enrichment\" }\r\n ,\"microsoft.alertsmanagement/prometheusrulegroups\": { \"SingularDisplayName\": \"Prometheus rule group\" }\r\n ,\"microsoft.alertsmanagement/smartdetectoralertrules\": { \"SingularDisplayName\": \"Smart detector alert rule\" }\r\n ,\"microsoft.alertsmanagement/smartgroups\": { \"SingularDisplayName\": \"Microsoft.AlertsManagement smart group\" }\r\n ,\"microsoft.alertsmanagement/tenantactivitylogalerts\": { \"SingularDisplayName\": \"Microsoft.AlertsManagement tenant activity log alert\" }\r\n ,\"microsoft.all/arcvirtualmachines\": { \"SingularDisplayName\": \"Azure Arc virtual machine\" }\r\n ,\"microsoft.all/hcivirtualmachines\": { \"SingularDisplayName\": \"Azure Local Virtual Machine - Azure Arc\" }\r\n ,\"microsoft.all/virtualmachines\": { \"SingularDisplayName\": \"Virtual machine\" }\r\n ,\"microsoft.analysisservices/servers\": { \"SingularDisplayName\": \"Analysis Services server\" }\r\n ,\"microsoft.anybuild/clusters\": { \"SingularDisplayName\": \"AnyBuild cluster\" }\r\n ,\"microsoft.apicenter/deletedservices\": { \"SingularDisplayName\": \"Microsoft.ApiCenter deleted service\" }\r\n ,\"microsoft.apicenter/services\": { \"SingularDisplayName\": \"API Center\" }\r\n ,\"microsoft.apicenter/services/workspaces\": { \"SingularDisplayName\": \"Workspace\" }\r\n ,\"microsoft.apimanagement/gateways\": { \"SingularDisplayName\": \"API Management gateway\" }\r\n ,\"microsoft.apimanagement/gateways/configconnections\": { \"SingularDisplayName\": \"Microsoft.ApiManagement gateways config connection\" }\r\n ,\"microsoft.apimanagement/service\": { \"SingularDisplayName\": \"API Management service\" }\r\n ,\"microsoft.apimanagement/service/workspaces\": { \"SingularDisplayName\": \"Workspace\" }\r\n ,\"microsoft.apisecurity/defendersettings\": { \"SingularDisplayName\": \"Microsoft.ApiSecurity defender setting\" }\r\n ,\"microsoft.app/agents\": { \"SingularDisplayName\": \"SRE Agent\" }\r\n ,\"microsoft.app/builders\": { \"SingularDisplayName\": \"Microsoft.App builder\" }\r\n ,\"microsoft.app/builders/builds\": { \"SingularDisplayName\": \"Microsoft.App builders build\" }\r\n ,\"microsoft.app/connectedenvironments\": { \"SingularDisplayName\": \"Container Apps Connected Environment\" }\r\n ,\"microsoft.app/containerapps\": { \"SingularDisplayName\": \"Container App\" }\r\n ,\"microsoft.app/jobs\": { \"SingularDisplayName\": \"Container App Job\" }\r\n ,\"microsoft.app/logicapps\": { \"SingularDisplayName\": \"Logic app\" }\r\n ,\"microsoft.app/logicapps/workflows\": { \"SingularDisplayName\": \"Logic app workflow\" }\r\n ,\"microsoft.app/managedenvironments\": { \"SingularDisplayName\": \"Container Apps Environment\" }\r\n ,\"microsoft.app/sessionpools\": { \"SingularDisplayName\": \"Container App Session Pool\" }\r\n ,\"microsoft.app/spaces\": { \"SingularDisplayName\": \"App Space\" }\r\n ,\"microsoft.appassessment/migrateprojects\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate project\" }\r\n ,\"microsoft.appassessment/migrateprojects/assessments\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects assessment\" }\r\n ,\"microsoft.appassessment/migrateprojects/assessments/assessedapplications\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects assessments assessed application\" }\r\n ,\"microsoft.appassessment/migrateprojects/assessments/assessedmachines\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects assessments assessed machine\" }\r\n ,\"microsoft.appassessment/migrateprojects/assessments/machinestoassess\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects assessments machines to asses\" }\r\n ,\"microsoft.appassessment/migrateprojects/sites\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects site\" }\r\n ,\"microsoft.appassessment/migrateprojects/sites/applianceconfigurations\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects sites appliance configuration\" }\r\n ,\"microsoft.appcomplianceautomation/reports\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation report\" }\r\n ,\"microsoft.appcomplianceautomation/reports/evidences\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation reports evidence\" }\r\n ,\"microsoft.appcomplianceautomation/reports/scopingconfigurations\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation reports scoping configuration\" }\r\n ,\"microsoft.appcomplianceautomation/reports/snapshots\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation reports snapshot\" }\r\n ,\"microsoft.appcomplianceautomation/reports/snapshots/controls\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation reports snapshots control\" }\r\n ,\"microsoft.appcomplianceautomation/reports/webhooks\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation reports webhook\" }\r\n ,\"microsoft.appconfiguration/configurationstores\": { \"SingularDisplayName\": \"App Configuration\" }\r\n ,\"microsoft.applicationmigration/discoveryhubs\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration discovery hub\" }\r\n ,\"microsoft.applicationmigration/discoveryhubs/applications\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration discovery hubs application\" }\r\n ,\"microsoft.applicationmigration/discoveryhubs/applications/members\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration discovery hubs applications member\" }\r\n ,\"microsoft.applicationmigration/pgsqlsites\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration pgsqlsite\" }\r\n ,\"microsoft.applicationmigration/pgsqlsites/agents\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration pgsqlsites agent\" }\r\n ,\"microsoft.applicationmigration/pgsqlsites/pgsqldatabases\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration pgsqlsites pgsqldatabase\" }\r\n ,\"microsoft.applicationmigration/pgsqlsites/pgsqlinstances\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration pgsqlsites pgsqlinstance\" }\r\n ,\"microsoft.appplatform/spring\": { \"SingularDisplayName\": \"Azure Spring Apps\" }\r\n ,\"microsoft.appsecurity/appprotectmanagedrulesetmanifests\": { \"SingularDisplayName\": \"Microsoft.AppSecurity app protect managed rule set manifest\" }\r\n ,\"microsoft.appsecurity/policies\": { \"SingularDisplayName\": \"App Protect Policy\" }\r\n ,\"microsoft.arc/all\": { \"SingularDisplayName\": \"Azure Arc enabled resource\" }\r\n ,\"microsoft.arc/allfairfax\": { \"SingularDisplayName\": \"Azure Arc enabled resource\" }\r\n ,\"microsoft.arc/kubernetesresources\": { \"SingularDisplayName\": \"Azure Arc Kubernetes cluster\" }\r\n ,\"microsoft.arc/kubernetesresourcesfairfax\": { \"SingularDisplayName\": \"Azure Arc Kubernetes cluster\" }\r\n ,\"microsoft.arcnetworking/arcnwloadbalancers\": { \"SingularDisplayName\": \"Microsoft.ArcNetworking arc nw load balancer\" }\r\n ,\"microsoft.aszlabhardware/labservers\": { \"SingularDisplayName\": \"Microsoft.AszLabHardware labserver\" }\r\n ,\"microsoft.aszlabhardware/reservations\": { \"SingularDisplayName\": \"Microsoft.AszLabHardware reservation\" }\r\n ,\"microsoft.aszlabhardware/reservations/servers\": { \"SingularDisplayName\": \"Microsoft.AszLabHardware reservations server\" }\r\n ,\"microsoft.aszlabhardware/servers\": { \"SingularDisplayName\": \"Microsoft.AszLabHardware server\" }\r\n ,\"microsoft.attestation/attestationproviders\": { \"SingularDisplayName\": \"Attestation provider\" }\r\n ,\"microsoft.authorization/accessreviewhistorydefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization access review history definition\" }\r\n ,\"microsoft.authorization/accessreviewscheduledefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization access review schedule definition\" }\r\n ,\"microsoft.authorization/accessreviewscheduledefinitions/instances\": { \"SingularDisplayName\": \"Microsoft.Authorization access review schedule definitions instance\" }\r\n ,\"microsoft.authorization/accessreviewscheduledefinitions/instances/decisions\": { \"SingularDisplayName\": \"Microsoft.Authorization access review schedule definitions instances decision\" }\r\n ,\"microsoft.authorization/accessreviewschedulesettings\": { \"SingularDisplayName\": \"Microsoft.Authorization access review schedule setting\" }\r\n ,\"microsoft.authorization/datapolicymanifests\": { \"SingularDisplayName\": \"Microsoft.Authorization data policy manifest\" }\r\n ,\"microsoft.authorization/denyassignments\": { \"SingularDisplayName\": \"Microsoft.Authorization deny assignment\" }\r\n ,\"microsoft.authorization/locks\": { \"SingularDisplayName\": \"Microsoft.Authorization lock\" }\r\n ,\"microsoft.authorization/policyassignments\": { \"SingularDisplayName\": \"Microsoft.Authorization policy assignment\" }\r\n ,\"microsoft.authorization/policydefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization policy definition\" }\r\n ,\"microsoft.authorization/policydefinitions/versions\": { \"SingularDisplayName\": \"Microsoft.Authorization policy definitions version\" }\r\n ,\"microsoft.authorization/policyexemptions\": { \"SingularDisplayName\": \"Microsoft.Authorization policy exemption\" }\r\n ,\"microsoft.authorization/policysetdefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization policy set definition\" }\r\n ,\"microsoft.authorization/policysetdefinitions/versions\": { \"SingularDisplayName\": \"Microsoft.Authorization policy set definitions version\" }\r\n ,\"microsoft.authorization/privatelinkassociations\": { \"SingularDisplayName\": \"Microsoft.Authorization private link association\" }\r\n ,\"microsoft.authorization/provideroperations\": { \"SingularDisplayName\": \"Microsoft.Authorization provider operation\" }\r\n ,\"microsoft.authorization/resourcemanagementprivatelinks\": { \"SingularDisplayName\": \"Resource management private link\" }\r\n ,\"microsoft.authorization/roleassignmentapprovals\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment approval\" }\r\n ,\"microsoft.authorization/roleassignmentapprovals/stages\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment approvals stage\" }\r\n ,\"microsoft.authorization/roleassignments\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment\" }\r\n ,\"microsoft.authorization/roleassignmentscheduleinstances\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment schedule instance\" }\r\n ,\"microsoft.authorization/roleassignmentschedulerequests\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment schedule request\" }\r\n ,\"microsoft.authorization/roleassignmentschedules\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment schedule\" }\r\n ,\"microsoft.authorization/roledefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization role definition\" }\r\n ,\"microsoft.authorization/roleeligibilityscheduleinstances\": { \"SingularDisplayName\": \"Microsoft.Authorization role eligibility schedule instance\" }\r\n ,\"microsoft.authorization/roleeligibilityschedulerequests\": { \"SingularDisplayName\": \"Microsoft.Authorization role eligibility schedule request\" }\r\n ,\"microsoft.authorization/roleeligibilityschedules\": { \"SingularDisplayName\": \"Microsoft.Authorization role eligibility schedule\" }\r\n ,\"microsoft.authorization/rolemanagementalertconfigurations\": { \"SingularDisplayName\": \"Microsoft.Authorization role management alert configuration\" }\r\n ,\"microsoft.authorization/rolemanagementalertdefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization role management alert definition\" }\r\n ,\"microsoft.authorization/rolemanagementalertoperations\": { \"SingularDisplayName\": \"Microsoft.Authorization role management alert operation\" }\r\n ,\"microsoft.authorization/rolemanagementalerts\": { \"SingularDisplayName\": \"Microsoft.Authorization role management alert\" }\r\n ,\"microsoft.authorization/rolemanagementalerts/alertincidents\": { \"SingularDisplayName\": \"Microsoft.Authorization role management alerts alert incident\" }\r\n ,\"microsoft.authorization/rolemanagementpolicies\": { \"SingularDisplayName\": \"Microsoft.Authorization role management policy\" }\r\n ,\"microsoft.authorization/rolemanagementpolicyassignments\": { \"SingularDisplayName\": \"Microsoft.Authorization role management policy assignment\" }\r\n ,\"microsoft.automanage/bestpractices\": { \"SingularDisplayName\": \"Microsoft.Automanage best practice\" }\r\n ,\"microsoft.automanage/bestpractices/versions\": { \"SingularDisplayName\": \"Microsoft.Automanage best practices version\" }\r\n ,\"microsoft.automanage/configurationprofileassignments\": { \"SingularDisplayName\": \"Microsoft.Automanage configuration profile assignment\" }\r\n ,\"microsoft.automanage/configurationprofileassignments/reports\": { \"SingularDisplayName\": \"Microsoft.Automanage configuration profile assignments report\" }\r\n ,\"microsoft.automanage/configurationprofiles\": { \"SingularDisplayName\": \"Microsoft.Automanage configuration profile\" }\r\n ,\"microsoft.automanage/configurationprofiles/versions\": { \"SingularDisplayName\": \"Microsoft.Automanage configuration profiles version\" }\r\n ,\"microsoft.automanage/serviceprincipals\": { \"SingularDisplayName\": \"ServicePrincipals\" }\r\n ,\"microsoft.automation/automationaccounts\": { \"SingularDisplayName\": \"Automation account\" }\r\n ,\"microsoft.automation/automationaccounts/hybridrunbookworkergroups\": { \"SingularDisplayName\": \"Automation hybrid worker group\" }\r\n ,\"microsoft.automation/automationaccounts/runbooks\": { \"SingularDisplayName\": \"Automation runbook\" }\r\n ,\"microsoft.autonomousdevelopmentplatform/accounts\": { \"SingularDisplayName\": \"Microsoft.AutonomousDevelopmentPlatform account\" }\r\n ,\"microsoft.autonomousdevelopmentplatform/accounts/datapools\": { \"SingularDisplayName\": \"Microsoft.AutonomousDevelopmentPlatform accounts data pool\" }\r\n ,\"microsoft.autonomousdevelopmentplatform/workspaces\": { \"SingularDisplayName\": \"Microsoft.AutonomousDevelopmentPlatform workspace\" }\r\n ,\"microsoft.avs/privateclouds\": { \"SingularDisplayName\": \"Azure VMware Solution private cloud\" }\r\n ,\"microsoft.awsconnector/accessanalyzeranalyzers\": { \"SingularDisplayName\": \"Access Analyzer Analyzer\" }\r\n ,\"microsoft.awsconnector/acmcertificatesummaries\": { \"SingularDisplayName\": \"ACM Certificate Summary\" }\r\n ,\"microsoft.awsconnector/apigatewayrestapis\": { \"SingularDisplayName\": \"Api Gateway Rest Api\" }\r\n ,\"microsoft.awsconnector/apigatewaystages\": { \"SingularDisplayName\": \"Api Gateway Stage\" }\r\n ,\"microsoft.awsconnector/applicationautoscalingscalabletargets\": { \"SingularDisplayName\": \"Application Auto Scaling Scalable Target\" }\r\n ,\"microsoft.awsconnector/appsyncgraphqlapis\": { \"SingularDisplayName\": \"App Sync Graphql Api\" }\r\n ,\"microsoft.awsconnector/autoscalingautoscalinggroups\": { \"SingularDisplayName\": \"Auto Scaling Auto Scaling Group\" }\r\n ,\"microsoft.awsconnector/cloudformationstacks\": { \"SingularDisplayName\": \"Cloud Formation Stack\" }\r\n ,\"microsoft.awsconnector/cloudformationstacksets\": { \"SingularDisplayName\": \"Cloud Formation Stack Set\" }\r\n ,\"microsoft.awsconnector/cloudfrontdistributions\": { \"SingularDisplayName\": \"Cloud Front Distribution\" }\r\n ,\"microsoft.awsconnector/cloudtrailtrails\": { \"SingularDisplayName\": \"Cloud Trail Trail\" }\r\n ,\"microsoft.awsconnector/cloudwatchalarms\": { \"SingularDisplayName\": \"Cloud Watch Alarm\" }\r\n ,\"microsoft.awsconnector/codebuildprojects\": { \"SingularDisplayName\": \"Code Build Project\" }\r\n ,\"microsoft.awsconnector/codebuildsourcecredentialsinfos\": { \"SingularDisplayName\": \"Code Build Source Credentials Info\" }\r\n ,\"microsoft.awsconnector/configserviceconfigurationrecorders\": { \"SingularDisplayName\": \"Config Service Configuration Recorder\" }\r\n ,\"microsoft.awsconnector/configserviceconfigurationrecorderstatuses\": { \"SingularDisplayName\": \"Config Service Configuration Recorder Status\" }\r\n ,\"microsoft.awsconnector/configservicedeliverychannels\": { \"SingularDisplayName\": \"Config Service Delivery Channel\" }\r\n ,\"microsoft.awsconnector/databasemigrationservicereplicationinstances\": { \"SingularDisplayName\": \"Database Migration Service Replication Instance\" }\r\n ,\"microsoft.awsconnector/daxclusters\": { \"SingularDisplayName\": \"DAX Cluster\" }\r\n ,\"microsoft.awsconnector/dynamodbcontinuousbackupsdescriptions\": { \"SingularDisplayName\": \"Dynamo DB Continuous Backups Description\" }\r\n ,\"microsoft.awsconnector/dynamodbtables\": { \"SingularDisplayName\": \"Dynamo DB Table\" }\r\n ,\"microsoft.awsconnector/ec2accountattributes\": { \"SingularDisplayName\": \"EC2 Account Attribute\" }\r\n ,\"microsoft.awsconnector/ec2addresses\": { \"SingularDisplayName\": \"EC2 Address\" }\r\n ,\"microsoft.awsconnector/ec2flowlogs\": { \"SingularDisplayName\": \"EC2 Flow Log\" }\r\n ,\"microsoft.awsconnector/ec2images\": { \"SingularDisplayName\": \"EC2 Image\" }\r\n ,\"microsoft.awsconnector/ec2instances\": { \"SingularDisplayName\": \"Microsoft.AwsConnector ec2 instance\" }\r\n ,\"microsoft.awsconnector/ec2instancestatuses\": { \"SingularDisplayName\": \"EC2 Instance Status\" }\r\n ,\"microsoft.awsconnector/ec2ipams\": { \"SingularDisplayName\": \"EC2 Ipam\" }\r\n ,\"microsoft.awsconnector/ec2keypairs\": { \"SingularDisplayName\": \"EC2 Key Pair\" }\r\n ,\"microsoft.awsconnector/ec2networkacls\": { \"SingularDisplayName\": \"EC2 Network Acl\" }\r\n ,\"microsoft.awsconnector/ec2networkinterfaces\": { \"SingularDisplayName\": \"EC2 Network Interface\" }\r\n ,\"microsoft.awsconnector/ec2routetables\": { \"SingularDisplayName\": \"EC2 Route Table\" }\r\n ,\"microsoft.awsconnector/ec2securitygroups\": { \"SingularDisplayName\": \"EC2 Security Group\" }\r\n ,\"microsoft.awsconnector/ec2snapshots\": { \"SingularDisplayName\": \"EC2 Snapshot\" }\r\n ,\"microsoft.awsconnector/ec2subnets\": { \"SingularDisplayName\": \"EC2 Subnet\" }\r\n ,\"microsoft.awsconnector/ec2volumes\": { \"SingularDisplayName\": \"EC2 Volume\" }\r\n ,\"microsoft.awsconnector/ec2vpcendpoints\": { \"SingularDisplayName\": \"EC2 VPCEndpoint\" }\r\n ,\"microsoft.awsconnector/ec2vpcpeeringconnections\": { \"SingularDisplayName\": \"EC2 VPCPeering Connection\" }\r\n ,\"microsoft.awsconnector/ec2vpcs\": { \"SingularDisplayName\": \"EC2 VPC\" }\r\n ,\"microsoft.awsconnector/ecrimagedetails\": { \"SingularDisplayName\": \"ECR Image Detail\" }\r\n ,\"microsoft.awsconnector/ecrrepositories\": { \"SingularDisplayName\": \"ECR Repository\" }\r\n ,\"microsoft.awsconnector/ecsclusters\": { \"SingularDisplayName\": \"ECS Cluster\" }\r\n ,\"microsoft.awsconnector/ecsservices\": { \"SingularDisplayName\": \"ECS Service\" }\r\n ,\"microsoft.awsconnector/ecstaskdefinitions\": { \"SingularDisplayName\": \"ECS Task Definition\" }\r\n ,\"microsoft.awsconnector/efsfilesystems\": { \"SingularDisplayName\": \"EFS File System\" }\r\n ,\"microsoft.awsconnector/efsmounttargets\": { \"SingularDisplayName\": \"EFS Mount Target\" }\r\n ,\"microsoft.awsconnector/eksnodegroups\": { \"SingularDisplayName\": \"EKS Nodegroup\" }\r\n ,\"microsoft.awsconnector/elasticbeanstalkapplications\": { \"SingularDisplayName\": \"Elastic Beanstalk Application\" }\r\n ,\"microsoft.awsconnector/elasticbeanstalkconfigurationtemplates\": { \"SingularDisplayName\": \"Elastic Beanstalk Configuration Template\" }\r\n ,\"microsoft.awsconnector/elasticbeanstalkenvironments\": { \"SingularDisplayName\": \"Elastic Beanstalk Environment\" }\r\n ,\"microsoft.awsconnector/elasticloadbalancingv2listeners\": { \"SingularDisplayName\": \"Elastic Load Balancing V2 Listener\" }\r\n ,\"microsoft.awsconnector/elasticloadbalancingv2loadbalancers\": { \"SingularDisplayName\": \"Elastic Load Balancing V2 Load Balancer\" }\r\n ,\"microsoft.awsconnector/elasticloadbalancingv2targetgroups\": { \"SingularDisplayName\": \"Elastic Load Balancing V2 Target Group\" }\r\n ,\"microsoft.awsconnector/elasticloadbalancingv2targethealthdescriptions\": { \"SingularDisplayName\": \"Elastic Load Balancing v2 Target Health Description\" }\r\n ,\"microsoft.awsconnector/elasticsearchdomains\": { \"SingularDisplayName\": \"Elasticsearch Domain\" }\r\n ,\"microsoft.awsconnector/emrclusters\": { \"SingularDisplayName\": \"EMR Cluster\" }\r\n ,\"microsoft.awsconnector/guarddutydetectors\": { \"SingularDisplayName\": \"Guard Duty Detector\" }\r\n ,\"microsoft.awsconnector/iamaccesskeylastuseds\": { \"SingularDisplayName\": \"IAM Access Key Last Used\" }\r\n ,\"microsoft.awsconnector/iamaccesskeymetadata\": { \"SingularDisplayName\": \"IAM Access Key Metadata\" }\r\n ,\"microsoft.awsconnector/iamgroups\": { \"SingularDisplayName\": \"IAM Group\" }\r\n ,\"microsoft.awsconnector/iaminstanceprofiles\": { \"SingularDisplayName\": \"IAM Instance Profile\" }\r\n ,\"microsoft.awsconnector/iammanagedpolicies\": { \"SingularDisplayName\": \"IAM Managed Policy\" }\r\n ,\"microsoft.awsconnector/iammfadevices\": { \"SingularDisplayName\": \"IAM MFADevice\" }\r\n ,\"microsoft.awsconnector/iampasswordpolicies\": { \"SingularDisplayName\": \"IAM Password Policy\" }\r\n ,\"microsoft.awsconnector/iampolicyversions\": { \"SingularDisplayName\": \"IAM Policy Version\" }\r\n ,\"microsoft.awsconnector/iamroles\": { \"SingularDisplayName\": \"IAM Role\" }\r\n ,\"microsoft.awsconnector/iamservercertificates\": { \"SingularDisplayName\": \"IAM Server Certificate\" }\r\n ,\"microsoft.awsconnector/iamuserpolicies\": { \"SingularDisplayName\": \"IAM User Policy\" }\r\n ,\"microsoft.awsconnector/iamvirtualmfadevices\": { \"SingularDisplayName\": \"IAM Virtual MFADevice\" }\r\n ,\"microsoft.awsconnector/kmsaliases\": { \"SingularDisplayName\": \"KMS Alias\" }\r\n ,\"microsoft.awsconnector/kmskeys\": { \"SingularDisplayName\": \"KMS Key\" }\r\n ,\"microsoft.awsconnector/lambdafunctioncodelocations\": { \"SingularDisplayName\": \"Lambda Function Code Location\" }\r\n ,\"microsoft.awsconnector/lambdafunctionconfigurations\": { \"SingularDisplayName\": \"Microsoft.AwsConnector lambda function configuration\" }\r\n ,\"microsoft.awsconnector/lambdafunctions\": { \"SingularDisplayName\": \"Lambda Function\" }\r\n ,\"microsoft.awsconnector/licensemanagerlicenses\": { \"SingularDisplayName\": \"License Manager License\" }\r\n ,\"microsoft.awsconnector/lightsailbuckets\": { \"SingularDisplayName\": \"Lightsail Bucket\" }\r\n ,\"microsoft.awsconnector/lightsailinstances\": { \"SingularDisplayName\": \"Lightsail Instance\" }\r\n ,\"microsoft.awsconnector/logsloggroups\": { \"SingularDisplayName\": \"Logs Log Group\" }\r\n ,\"microsoft.awsconnector/logslogstreams\": { \"SingularDisplayName\": \"Logs Log Stream\" }\r\n ,\"microsoft.awsconnector/logsmetricfilters\": { \"SingularDisplayName\": \"Logs Metric Filter\" }\r\n ,\"microsoft.awsconnector/logssubscriptionfilters\": { \"SingularDisplayName\": \"Logs Subscription Filter\" }\r\n ,\"microsoft.awsconnector/macie2jobsummaries\": { \"SingularDisplayName\": \"Macie2 Job Summary\" }\r\n ,\"microsoft.awsconnector/macieallowlists\": { \"SingularDisplayName\": \"Macie Allow List\" }\r\n ,\"microsoft.awsconnector/networkfirewallfirewallpolicies\": { \"SingularDisplayName\": \"Network Firewall Firewall Policy\" }\r\n ,\"microsoft.awsconnector/networkfirewallfirewalls\": { \"SingularDisplayName\": \"Network Firewall Firewall\" }\r\n ,\"microsoft.awsconnector/networkfirewallrulegroups\": { \"SingularDisplayName\": \"Network Firewall Rule Group\" }\r\n ,\"microsoft.awsconnector/opensearchdomainstatuses\": { \"SingularDisplayName\": \"Open Search Domain Status\" }\r\n ,\"microsoft.awsconnector/opensearchservicedomains\": { \"SingularDisplayName\": \"Open Search Service Domain\" }\r\n ,\"microsoft.awsconnector/organizationsaccounts\": { \"SingularDisplayName\": \"Organizations Account\" }\r\n ,\"microsoft.awsconnector/organizationsorganizations\": { \"SingularDisplayName\": \"Organizations Organization\" }\r\n ,\"microsoft.awsconnector/rdsdbclusters\": { \"SingularDisplayName\": \"RDS DBCluster\" }\r\n ,\"microsoft.awsconnector/rdsdbinstances\": { \"SingularDisplayName\": \"RDS DBInstance\" }\r\n ,\"microsoft.awsconnector/rdsdbsnapshotattributesresults\": { \"SingularDisplayName\": \"RDS DBSnapshot Attributes Result\" }\r\n ,\"microsoft.awsconnector/rdsdbsnapshots\": { \"SingularDisplayName\": \"RDS DBSnapshot\" }\r\n ,\"microsoft.awsconnector/rdseventsubscriptions\": { \"SingularDisplayName\": \"RDS Event Subscription\" }\r\n ,\"microsoft.awsconnector/rdsexporttasks\": { \"SingularDisplayName\": \"RDS Export Task\" }\r\n ,\"microsoft.awsconnector/redshiftclusterparametergroups\": { \"SingularDisplayName\": \"Redshift Cluster Parameter Group\" }\r\n ,\"microsoft.awsconnector/redshiftclusters\": { \"SingularDisplayName\": \"Redshift Cluster\" }\r\n ,\"microsoft.awsconnector/route53domainsdomainsummaries\": { \"SingularDisplayName\": \"Route 53 Domains Domain Summary\" }\r\n ,\"microsoft.awsconnector/route53hostedzones\": { \"SingularDisplayName\": \"Route53 Hosted Zone\" }\r\n ,\"microsoft.awsconnector/route53resourcerecordsets\": { \"SingularDisplayName\": \"Route 53 Resource Record Set\" }\r\n ,\"microsoft.awsconnector/s3accesscontrolpolicies\": { \"SingularDisplayName\": \"S3 Access Control Policy\" }\r\n ,\"microsoft.awsconnector/s3accesspoints\": { \"SingularDisplayName\": \"S3 Access Point\" }\r\n ,\"microsoft.awsconnector/s3bucketpolicies\": { \"SingularDisplayName\": \"S3 Bucket Policy\" }\r\n ,\"microsoft.awsconnector/s3buckets\": { \"SingularDisplayName\": \"S3 Bucket\" }\r\n ,\"microsoft.awsconnector/s3controlmultiregionaccesspointpolicydocuments\": { \"SingularDisplayName\": \"S3 Control Multi Region Access Point Policy Document\" }\r\n ,\"microsoft.awsconnector/sagemakerapps\": { \"SingularDisplayName\": \"Sage Maker App\" }\r\n ,\"microsoft.awsconnector/sagemakerdevices\": { \"SingularDisplayName\": \"Sage Maker Device\" }\r\n ,\"microsoft.awsconnector/sagemakerimages\": { \"SingularDisplayName\": \"Sage Maker Image\" }\r\n ,\"microsoft.awsconnector/sagemakernotebookinstancesummaries\": { \"SingularDisplayName\": \"Sage Maker Notebook Instance Summary\" }\r\n ,\"microsoft.awsconnector/secretsmanagerresourcepolicies\": { \"SingularDisplayName\": \"Secrets Manager Resource Policy\" }\r\n ,\"microsoft.awsconnector/secretsmanagersecrets\": { \"SingularDisplayName\": \"Secrets Manager Secret\" }\r\n ,\"microsoft.awsconnector/snssubscriptions\": { \"SingularDisplayName\": \"SNS Subscription\" }\r\n ,\"microsoft.awsconnector/snstopics\": { \"SingularDisplayName\": \"SNS Topic\" }\r\n ,\"microsoft.awsconnector/sqsqueues\": { \"SingularDisplayName\": \"SQS Queue\" }\r\n ,\"microsoft.awsconnector/ssminstanceinformations\": { \"SingularDisplayName\": \"SSM Instance Information\" }\r\n ,\"microsoft.awsconnector/ssmparameters\": { \"SingularDisplayName\": \"SSM Parameter\" }\r\n ,\"microsoft.awsconnector/ssmresourcecompliancesummaryitems\": { \"SingularDisplayName\": \"SSM Resource Compliance Summary Item\" }\r\n ,\"microsoft.awsconnector/wafv2ipsets\": { \"SingularDisplayName\": \"WAFv2 IPSet\" }\r\n ,\"microsoft.awsconnector/wafv2loggingconfigurations\": { \"SingularDisplayName\": \"WAFv2 Logging Configuration\" }\r\n ,\"microsoft.awsconnector/wafv2webaclassociations\": { \"SingularDisplayName\": \"WAFv2 Web ACLAssociation\" }\r\n ,\"microsoft.awsconnector/wafwebaclsummaries\": { \"SingularDisplayName\": \"WAF Web ACLSummary\" }\r\n ,\"microsoft.azureactivedirectory/b2cdirectories\": { \"SingularDisplayName\": \"B2C tenant\" }\r\n ,\"microsoft.azureactivedirectory/ciamdirectories\": { \"SingularDisplayName\": \"External Configuration Tenant\" }\r\n ,\"microsoft.azureactivedirectory/guestusages\": { \"SingularDisplayName\": \"Guest Usage\" }\r\n ,\"microsoft.azurearcdata/datacontrollers\": { \"SingularDisplayName\": \"Azure Arc data controller\" }\r\n ,\"microsoft.azurearcdata/mysqlserver\": { \"SingularDisplayName\": \"MySql Server - Azure Arc\" }\r\n ,\"microsoft.azurearcdata/postgresinstances\": { \"SingularDisplayName\": \"PostgreSQL server ? Azure Arc\" }\r\n ,\"microsoft.azurearcdata/postgressqlserver\": { \"SingularDisplayName\": \"PostgresSql Server - Azure Arc\" }\r\n ,\"microsoft.azurearcdata/sqlmanagedinstances\": { \"SingularDisplayName\": \"SQL managed instance - Azure Arc\" }\r\n ,\"microsoft.azurearcdata/sqlserveresulicenses\": { \"SingularDisplayName\": \"SQL Server ESU license\" }\r\n ,\"microsoft.azurearcdata/sqlserverinstances\": { \"SingularDisplayName\": \"SQL Server - Azure Arc\" }\r\n ,\"microsoft.azurearcdata/sqlserverinstances/databases\": { \"SingularDisplayName\": \"SQL Server database - Azure Arc\" }\r\n ,\"microsoft.azurearcdata/sqlserverlicenses\": { \"SingularDisplayName\": \"SQL Server License\" }\r\n ,\"microsoft.azurebusinesscontinuity/deletedunifiedprotecteditems\": { \"SingularDisplayName\": \"Microsoft.AzureBusinessContinuity deleted unified protected item\" }\r\n ,\"microsoft.azurebusinesscontinuity/unifiedprotecteditems\": { \"SingularDisplayName\": \"Microsoft.AzureBusinessContinuity unified protected item\" }\r\n ,\"microsoft.azurecis/aadapplications\": { \"SingularDisplayName\": \"Microsoft.AzureCis AAD application\" }\r\n ,\"microsoft.azurecis/addressrecords\": { \"SingularDisplayName\": \"Microsoft.AzureCis address record\" }\r\n ,\"microsoft.azurecis/autopilotenvironments\": { \"SingularDisplayName\": \"Microsoft.AzureCis autopilot environment\" }\r\n ,\"microsoft.azurecis/autopilotmachinefunctions\": { \"SingularDisplayName\": \"Microsoft.AzureCis autopilot machine function\" }\r\n ,\"microsoft.azurecis/autopilotsoftwareloadbalancevirtualips\": { \"SingularDisplayName\": \"Microsoft.AzureCis auto pilot software load balance virtual IP\" }\r\n ,\"microsoft.azurecis/azcopies\": { \"SingularDisplayName\": \"Microsoft.AzureCis az copy\" }\r\n ,\"microsoft.azurecis/canonicalnamerecords\": { \"SingularDisplayName\": \"Microsoft.AzureCis canonical name record\" }\r\n ,\"microsoft.azurecis/dsmsallowlists\": { \"SingularDisplayName\": \"Microsoft.AzureCis ds msallowlist\" }\r\n ,\"microsoft.azurecis/dsmscertificates\": { \"SingularDisplayName\": \"Microsoft.AzureCis dsms certificate\" }\r\n ,\"microsoft.azurecis/dsmsrootfolders\": { \"SingularDisplayName\": \"Microsoft.AzureCis dsms root folder\" }\r\n ,\"microsoft.azurecis/dstsapplications\": { \"SingularDisplayName\": \"Microsoft.AzureCis dsts application\" }\r\n ,\"microsoft.azurecis/dstsserviceaccounts\": { \"SingularDisplayName\": \"Microsoft.AzureCis dsts service account\" }\r\n ,\"microsoft.azurecis/dstsserviceclientidentities\": { \"SingularDisplayName\": \"Microsoft.AzureCis dsts service client identity\" }\r\n ,\"microsoft.azurecis/genericgenevaactions\": { \"SingularDisplayName\": \"Microsoft.AzureCis generic geneva action\" }\r\n ,\"microsoft.azurecis/plannedquotas\": { \"SingularDisplayName\": \"Microsoft.AzureCis planned quota\" }\r\n ,\"microsoft.azurecis/pointerrecords\": { \"SingularDisplayName\": \"Microsoft.AzureCis pointer record\" }\r\n ,\"microsoft.azurecis/publishconfigvalues\": { \"SingularDisplayName\": \"Microsoft.AzureCis publish config value\" }\r\n ,\"microsoft.azurecis/pushagentv2accounts\": { \"SingularDisplayName\": \"Microsoft.AzureCis push agent v2 account\" }\r\n ,\"microsoft.azurecis/servicerecords\": { \"SingularDisplayName\": \"Microsoft.AzureCis service record\" }\r\n ,\"microsoft.azurecis/sharedconfigvalues\": { \"SingularDisplayName\": \"Microsoft.AzureCis shared config value\" }\r\n ,\"microsoft.azurecloudmetadata/clouds\": { \"SingularDisplayName\": \"Microsoft.AzureCloudMetadata cloud\" }\r\n ,\"microsoft.azurecloudmetadata/clouds/geographies\": { \"SingularDisplayName\": \"Microsoft.AzureCloudMetadata clouds geography\" }\r\n ,\"microsoft.azurecloudmetadata/clouds/geographies/regions\": { \"SingularDisplayName\": \"Microsoft.AzureCloudMetadata clouds geographies region\" }\r\n ,\"microsoft.azuredatatransfer/connections\": { \"SingularDisplayName\": \"Connection\" }\r\n ,\"microsoft.azuredatatransfer/connections/flows\": { \"SingularDisplayName\": \"Flow\" }\r\n ,\"microsoft.azuredatatransfer/pipelines\": { \"SingularDisplayName\": \"Pipeline\" }\r\n ,\"microsoft.azurefleet/fleets\": { \"SingularDisplayName\": \"Compute Fleet\" }\r\n ,\"microsoft.azurefleet/fleetscomputehub\": { \"SingularDisplayName\": \"Compute Fleet\" }\r\n ,\"microsoft.azureimagetestingforlinux/jobs\": { \"SingularDisplayName\": \"Microsoft.AzureImageTestingForLinux job\" }\r\n ,\"microsoft.azureimagetestingforlinux/jobtemplates\": { \"SingularDisplayName\": \"Microsoft.AzureImageTestingForLinux job template\" }\r\n ,\"microsoft.azurelargeinstance/azurelargeinstances\": { \"SingularDisplayName\": \"Azure Large Instance\" }\r\n ,\"microsoft.azurelargeinstance/azurelargestorageinstances\": { \"SingularDisplayName\": \"Microsoft.AzureLargeInstance Azure large storage instance\" }\r\n ,\"microsoft.azurepercept/accounts\": { \"SingularDisplayName\": \"Microsoft.AzurePercept account\" }\r\n ,\"microsoft.azurepercept/accounts/devices\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts device\" }\r\n ,\"microsoft.azurepercept/accounts/devices/sensors\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts devices sensor\" }\r\n ,\"microsoft.azurepercept/accounts/sensors\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts sensor\" }\r\n ,\"microsoft.azurepercept/accounts/solutioninstances\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts solutioninstance\" }\r\n ,\"microsoft.azurepercept/accounts/solutions\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts solution\" }\r\n ,\"microsoft.azurepercept/accounts/targets\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts target\" }\r\n ,\"microsoft.azureplaywrightservice/accounts\": { \"SingularDisplayName\": \"Playwright Testing\" }\r\n ,\"microsoft.azurescan/scanningaccounts\": { \"SingularDisplayName\": \"ESRP Scan\" }\r\n ,\"microsoft.azuresphere/catalogs\": { \"SingularDisplayName\": \"Azure Sphere Catalog\" }\r\n ,\"microsoft.azurespherev2/catalogs\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalog\" }\r\n ,\"microsoft.azurespherev2/catalogs/artifacts\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs artifact\" }\r\n ,\"microsoft.azurespherev2/catalogs/certificates\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs certificate\" }\r\n ,\"microsoft.azurespherev2/catalogs/deviceregistrations\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs device registration\" }\r\n ,\"microsoft.azurespherev2/catalogs/provisioningpackages\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs provisioning package\" }\r\n ,\"microsoft.azurespherev2/catalogs/syndicationchannels\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs syndication channel\" }\r\n ,\"microsoft.azurespherev2/catalogs/syndicationchannels/deployments\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs syndication channels deployment\" }\r\n ,\"microsoft.azurespherev2/catalogs/updatepackages\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs update package\" }\r\n ,\"microsoft.azurestack/cloudmanifestfiles\": { \"SingularDisplayName\": \"Microsoft.AzureStack cloud manifest file\" }\r\n ,\"microsoft.azurestack/linkedsubscriptions\": { \"SingularDisplayName\": \"Microsoft.AzureStack linked subscription\" }\r\n ,\"microsoft.azurestack/registrations\": { \"SingularDisplayName\": \"Microsoft.AzureStack registration\" }\r\n ,\"microsoft.azurestack/registrations/customersubscriptions\": { \"SingularDisplayName\": \"Microsoft.AzureStack registrations customer subscription\" }\r\n ,\"microsoft.azurestack/registrations/products\": { \"SingularDisplayName\": \"Microsoft.AzureStack registrations product\" }\r\n ,\"microsoft.azurestackhci/clusters\": { \"SingularDisplayName\": \"Azure Local\" }\r\n ,\"microsoft.azurestackhci/clusters/updates/updateruns\": { \"SingularDisplayName\": \"Azure Local\" }\r\n ,\"microsoft.azurestackhci/clusters/updatesummaries\": { \"SingularDisplayName\": \"Azure Local\" }\r\n ,\"microsoft.azurestackhci/devicepools\": { \"SingularDisplayName\": \"Azure Stack\" }\r\n ,\"microsoft.azurestackhci/edgedevices\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI edge device\" }\r\n ,\"microsoft.azurestackhci/edgedevices/jobs\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI edge devices job\" }\r\n ,\"microsoft.azurestackhci/edgemachines\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI edge machine\" }\r\n ,\"microsoft.azurestackhci/edgemachines/jobs\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI edge machines job\" }\r\n ,\"microsoft.azurestackhci/edgenodepools\": { \"SingularDisplayName\": \"Azure Stack\" }\r\n ,\"microsoft.azurestackhci/galleryimages\": { \"SingularDisplayName\": \"Azure Local Gallery image\" }\r\n ,\"microsoft.azurestackhci/logicalnetworks\": { \"SingularDisplayName\": \"Azure Local Logical network\" }\r\n ,\"microsoft.azurestackhci/marketplacegalleryimages\": { \"SingularDisplayName\": \"Azure Local Marketplace Gallery image\" }\r\n ,\"microsoft.azurestackhci/networkinterfaces\": { \"SingularDisplayName\": \"Azure Local VM Network Interface\" }\r\n ,\"microsoft.azurestackhci/networksecuritygroups\": { \"SingularDisplayName\": \"Azure Local Network Security Group\" }\r\n ,\"microsoft.azurestackhci/networksecuritygroups/securityrules\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI network security groups security rule\" }\r\n ,\"microsoft.azurestackhci/storagecontainers\": { \"SingularDisplayName\": \"Azure Local Storage path\" }\r\n ,\"microsoft.azurestackhci/virtualharddisks\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI virtual hard disk\" }\r\n ,\"microsoft.azurestackhci/virtualmachineinstances\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI virtual machine instance\" }\r\n ,\"microsoft.azurestackhci/virtualmachineinstances/guestagents\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI virtual machine instances guest agent\" }\r\n ,\"microsoft.azurestackhci/virtualmachineinstances/hybrididentitymetadata\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI virtual machine instances hybrid identity metadata\" }\r\n ,\"microsoft.azurestackhci/virtualmachines\": { \"SingularDisplayName\": \"Azure Local virtual machine - Azure Arc\" }\r\n ,\"microsoft.azurestackhci/virtualnetworks\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI virtual network\" }\r\n ,\"microsoft.backupsolutions/vmwareapplications\": { \"SingularDisplayName\": \"Microsoft.BackupSolutions vmware application\" }\r\n ,\"microsoft.bakeryhybrid/pies\": { \"SingularDisplayName\": \"Microsoft.BakeryHybrid py\" }\r\n ,\"microsoft.bakeryhybrid/pies/nestedresourcetype\": { \"SingularDisplayName\": \"Microsoft.BakeryHybrid pies nested resource type\" }\r\n ,\"microsoft.baremetal/baremetalconnections\": { \"SingularDisplayName\": \"Microsoft.BareMetal bare metal connection\" }\r\n ,\"microsoft.baremetal/crayservers\": { \"SingularDisplayName\": \"Cray Server\" }\r\n ,\"microsoft.baremetal/monitoringservers\": { \"SingularDisplayName\": \"Monitoring Server\" }\r\n ,\"microsoft.baremetal/peeringsettings\": { \"SingularDisplayName\": \"Microsoft.BareMetal peering setting\" }\r\n ,\"microsoft.baremetalinfrastructure/baremetalinstances\": { \"SingularDisplayName\": \"BareMetal Instance\" }\r\n ,\"microsoft.baremetalinfrastructure/baremetalstorageinstances\": { \"SingularDisplayName\": \"Microsoft.BareMetalInfrastructure bare metal storage instance\" }\r\n ,\"microsoft.batch/batchaccounts\": { \"SingularDisplayName\": \"Batch account\" }\r\n ,\"microsoft.billing/billingaccounts\": { \"SingularDisplayName\": \"Microsoft.Billing billing account\" }\r\n ,\"microsoft.billing/billingaccounts/agreements\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts agreement\" }\r\n ,\"microsoft.billing/billingaccounts/associatedtenants\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts associated tenant\" }\r\n ,\"microsoft.billing/billingaccounts/availablebalance\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts available balance\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profile\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/availablebalance\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles available balance\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/billingsubscriptions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles billing subscription\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/customers\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles customer\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/customers/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles customers billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/customers/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles customers billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/customers/transfers\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles customers transfer\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/instructions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles instruction\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoices\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice section\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice sections billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice sections billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections/billingsubscriptions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice sections billing subscription\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections/products\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice sections product\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections/transfers\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice sections transfer\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/paymentmethodlinks\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles payment method link\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/policies\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles policy\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/transactions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles transaction\" }\r\n ,\"microsoft.billing/billingaccounts/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/billingsubscriptionaliases\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing subscription aliase\" }\r\n ,\"microsoft.billing/billingaccounts/billingsubscriptions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing subscription\" }\r\n ,\"microsoft.billing/billingaccounts/billingsubscriptions/invoices\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing subscriptions invoice\" }\r\n ,\"microsoft.billing/billingaccounts/customers\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts customer\" }\r\n ,\"microsoft.billing/billingaccounts/customers/billingsubscriptions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts customers billing subscription\" }\r\n ,\"microsoft.billing/billingaccounts/customers/policies\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts customers policy\" }\r\n ,\"microsoft.billing/billingaccounts/customers/products\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts customers product\" }\r\n ,\"microsoft.billing/billingaccounts/departments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts department\" }\r\n ,\"microsoft.billing/billingaccounts/departments/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts departments billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/departments/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts departments billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/departments/enrollmentaccounts\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts departments enrollment account\" }\r\n ,\"microsoft.billing/billingaccounts/enrollmentaccounts\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts enrollment account\" }\r\n ,\"microsoft.billing/billingaccounts/enrollmentaccounts/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts enrollment accounts billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/enrollmentaccounts/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts enrollment accounts billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/incentiveschedules\": { \"SingularDisplayName\": \"Incentive Schedule\" }\r\n ,\"microsoft.billing/billingaccounts/incentiveschedules/milestones\": { \"SingularDisplayName\": \"Milestone\" }\r\n ,\"microsoft.billing/billingaccounts/invoices\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts invoice\" }\r\n ,\"microsoft.billing/billingaccounts/invoicesections\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts invoice section\" }\r\n ,\"microsoft.billing/billingaccounts/invoicesections/billingsubscriptions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts invoice sections billing subscription\" }\r\n ,\"microsoft.billing/billingaccounts/invoicesections/products\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts invoice sections product\" }\r\n ,\"microsoft.billing/billingaccounts/invoicesections/transfers\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts invoice sections transfer\" }\r\n ,\"microsoft.billing/billingaccounts/lineofcredit\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts line of credit\" }\r\n ,\"microsoft.billing/billingaccounts/migrations\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts migration\" }\r\n ,\"microsoft.billing/billingaccounts/paymentmethods\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts payment method\" }\r\n ,\"microsoft.billing/billingaccounts/policies\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts policy\" }\r\n ,\"microsoft.billing/billingaccounts/products\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts product\" }\r\n ,\"microsoft.billing/billingaccounts/reservationorders\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts reservation order\" }\r\n ,\"microsoft.billing/billingaccounts/reservationorders/reservations\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts reservation orders reservation\" }\r\n ,\"microsoft.billing/billingaccounts/savingsplanorders\": { \"SingularDisplayName\": \"Savings plan order\" }\r\n ,\"microsoft.billing/billingaccounts/savingsplanorders/savingsplans\": { \"SingularDisplayName\": \"Savings plan\" }\r\n ,\"microsoft.billing/billingperiods\": { \"SingularDisplayName\": \"Microsoft.Billing billing period\" }\r\n ,\"microsoft.billing/billingproperty\": { \"SingularDisplayName\": \"Microsoft.Billing billing property\" }\r\n ,\"microsoft.billing/billingrequests\": { \"SingularDisplayName\": \"Microsoft.Billing billing request\" }\r\n ,\"microsoft.billing/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing role assignment\" }\r\n ,\"microsoft.billing/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing role definition\" }\r\n ,\"microsoft.billing/enrollmentaccounts\": { \"SingularDisplayName\": \"Microsoft.Billing enrollment account\" }\r\n ,\"microsoft.billing/paymentmethods\": { \"SingularDisplayName\": \"Microsoft.Billing payment method\" }\r\n ,\"microsoft.billing/policies\": { \"SingularDisplayName\": \"Microsoft.Billing policy\" }\r\n ,\"microsoft.billing/promotions\": { \"SingularDisplayName\": \"Microsoft.Billing promotion\" }\r\n ,\"microsoft.billing/transfers\": { \"SingularDisplayName\": \"Microsoft.Billing transfer\" }\r\n ,\"microsoft.billingbenefits/credits\": { \"SingularDisplayName\": \"Credit\" }\r\n ,\"microsoft.billingbenefits/discounts\": { \"SingularDisplayName\": \"Discount\" }\r\n ,\"microsoft.billingbenefits/incentiveschedules\": { \"SingularDisplayName\": \"Incentive Schedule\" }\r\n ,\"microsoft.billingbenefits/incentiveschedules/milestones\": { \"SingularDisplayName\": \"Milestone\" }\r\n ,\"microsoft.billingbenefits/maccs\": { \"SingularDisplayName\": \"Microsoft Azure Consumption Commitment\" }\r\n ,\"microsoft.billingbenefits/reservationorderaliases\": { \"SingularDisplayName\": \"Microsoft.BillingBenefits reservation order aliase\" }\r\n ,\"microsoft.billingbenefits/savingsplanorderaliases\": { \"SingularDisplayName\": \"Microsoft.BillingBenefits savings plan order aliase\" }\r\n ,\"microsoft.billingbenefits/savingsplanorders\": { \"SingularDisplayName\": \"Savings plan order\" }\r\n ,\"microsoft.billingbenefits/savingsplanorders/savingsplans\": { \"SingularDisplayName\": \"Savings plan\" }\r\n ,\"microsoft.bing/accounts\": { \"SingularDisplayName\": \"Bing Resource\" }\r\n ,\"microsoft.blockchain/blockchainmembers\": { \"SingularDisplayName\": \"Microsoft.Blockchain blockchain member\" }\r\n ,\"microsoft.blockchain/blockchainmembers/transactionnodes\": { \"SingularDisplayName\": \"Microsoft.Blockchain blockchain members transaction node\" }\r\n ,\"microsoft.blockchaintokens/tokenservices\": { \"SingularDisplayName\": \"Microsoft.BlockchainTokens token service\" }\r\n ,\"microsoft.blockchaintokens/tokenservices/blockchainnetworks\": { \"SingularDisplayName\": \"Microsoft.BlockchainTokens token services blockchain network\" }\r\n ,\"microsoft.blockchaintokens/tokenservices/groups\": { \"SingularDisplayName\": \"Microsoft.BlockchainTokens token services group\" }\r\n ,\"microsoft.blockchaintokens/tokenservices/groups/accounts\": { \"SingularDisplayName\": \"Microsoft.BlockchainTokens token services groups account\" }\r\n ,\"microsoft.blockchaintokens/tokenservices/tokentemplates\": { \"SingularDisplayName\": \"Microsoft.BlockchainTokens token services token template\" }\r\n ,\"microsoft.bluefin/instances\": { \"SingularDisplayName\": \"Microsoft.Bluefin instance\" }\r\n ,\"microsoft.bluefin/instances/datasets\": { \"SingularDisplayName\": \"Microsoft.Bluefin instances dataset\" }\r\n ,\"microsoft.bluefin/instances/pipelines\": { \"SingularDisplayName\": \"Microsoft.Bluefin instances pipeline\" }\r\n ,\"microsoft.blueprint/blueprintassignments\": { \"SingularDisplayName\": \"Microsoft.Blueprint blueprint assignment\" }\r\n ,\"microsoft.blueprint/blueprints\": { \"SingularDisplayName\": \"Microsoft.Blueprint blueprint\" }\r\n ,\"microsoft.blueprint/blueprints/artifacts\": { \"SingularDisplayName\": \"Microsoft.Blueprint blueprints artifact\" }\r\n ,\"microsoft.blueprint/blueprints/versions\": { \"SingularDisplayName\": \"Microsoft.Blueprint blueprints version\" }\r\n ,\"microsoft.botservice/botservices\": { \"SingularDisplayName\": \"Bot Service\" }\r\n ,\"microsoft.cache/redis\": { \"SingularDisplayName\": \"Redis cache\" }\r\n ,\"microsoft.cache/redisenterprise\": { \"SingularDisplayName\": \"Azure Managed Redis\" }\r\n ,\"microsoft.cache/redisenterprise/databases\": { \"SingularDisplayName\": \"Redis Enterprise database\" }\r\n ,\"microsoft.capacity/reservationorders\": { \"SingularDisplayName\": \"Reservation order\" }\r\n ,\"microsoft.capacity/reservationorders/reservations\": { \"SingularDisplayName\": \"Reservation\" }\r\n ,\"microsoft.cascade/sites\": { \"SingularDisplayName\": \"Microsoft.Cascade site\" }\r\n ,\"microsoft.cdn/cdnwebapplicationfirewallpolicies\": { \"SingularDisplayName\": \"Content Delivery Network WAF policy\" }\r\n ,\"microsoft.cdn/edgeactions\": { \"SingularDisplayName\": \"Edge Action\" }\r\n ,\"microsoft.cdn/profiles\": { \"SingularDisplayName\": \"Front Door and CDN profile\" }\r\n ,\"microsoft.cdn/profiles/afdendpoints\": { \"SingularDisplayName\": \"Endpoint\" }\r\n ,\"microsoft.cdn/profiles/afdendpoints/routes\": { \"SingularDisplayName\": \"Route\" }\r\n ,\"microsoft.cdn/profiles/customdomains\": { \"SingularDisplayName\": \"Custom domain\" }\r\n ,\"microsoft.cdn/profiles/endpoints\": { \"SingularDisplayName\": \"CDN endpoint\" }\r\n ,\"microsoft.cdn/profiles/endpoints/customdomains\": { \"SingularDisplayName\": \"CDN custom domain\" }\r\n ,\"microsoft.cdn/profiles/endpoints/origins\": { \"SingularDisplayName\": \"CDN origin\" }\r\n ,\"microsoft.cdn/profiles/origingroups\": { \"SingularDisplayName\": \"Origin group\" }\r\n ,\"microsoft.cdn/profiles/origingroups/origins\": { \"SingularDisplayName\": \"Origin\" }\r\n ,\"microsoft.cdn/profiles/rulesets\": { \"SingularDisplayName\": \"Rule set\" }\r\n ,\"microsoft.cdn/profiles/rulesets/rules\": { \"SingularDisplayName\": \"Rule\" }\r\n ,\"microsoft.cdn/profiles/secrets\": { \"SingularDisplayName\": \"Secret\" }\r\n ,\"microsoft.cdn/profiles/securitypolicies\": { \"SingularDisplayName\": \"Security policy\" }\r\n ,\"microsoft.certificateregistration/certificateorders\": { \"SingularDisplayName\": \"App Service certificate\" }\r\n ,\"microsoft.certify/testsuites\": { \"SingularDisplayName\": \"Microsoft.Certify test suite\" }\r\n ,\"microsoft.certify/validationjobs\": { \"SingularDisplayName\": \"Microsoft.Certify validation job\" }\r\n ,\"microsoft.changeanalysis/profile\": { \"SingularDisplayName\": \"Microsoft.ChangeAnalysis profile\" }\r\n ,\"microsoft.changesafety/changestates\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety change state\" }\r\n ,\"microsoft.changesafety/changestates/stageprogressions\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety change states stage progression\" }\r\n ,\"microsoft.changesafety/stagemaps\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety stage map\" }\r\n ,\"microsoft.changesafety/validations\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety validation\" }\r\n ,\"microsoft.changesafety/validators\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety validator\" }\r\n ,\"microsoft.changesafety/validators/versions\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety validators version\" }\r\n ,\"microsoft.chaos/experiments\": { \"SingularDisplayName\": \"Chaos Experiment\" }\r\n ,\"microsoft.chaos/privateaccesses\": { \"SingularDisplayName\": \"Agent Private Access\" }\r\n ,\"microsoft.chaos/targets\": { \"SingularDisplayName\": \"Microsoft.Chaos target\" }\r\n ,\"microsoft.chaos/targets/capabilities\": { \"SingularDisplayName\": \"Microsoft.Chaos targets capability\" }\r\n ,\"microsoft.classiccompute/domainnames\": { \"SingularDisplayName\": \"Cloud service (classic)\" }\r\n ,\"microsoft.classiccompute/domainnames/slots/roles\": { \"SingularDisplayName\": \"Cloud service role (classic)\" }\r\n ,\"microsoft.classiccompute/virtualmachines\": { \"SingularDisplayName\": \"Virtual machine (classic)\" }\r\n ,\"microsoft.classicnetwork/networksecuritygroups\": { \"SingularDisplayName\": \"Network security group (classic)\" }\r\n ,\"microsoft.classicnetwork/reservedips\": { \"SingularDisplayName\": \"Reserved IP address (classic)\" }\r\n ,\"microsoft.classicnetwork/virtualnetworks\": { \"SingularDisplayName\": \"Virtual network (classic)\" }\r\n })[tolower(id)]\r\n}\r\n", - "$fxv#1": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData/Internal')\r\n_resource_type_2(id: string) {\r\n dynamic({\r\n \"microsoft.classicstorage/storageaccounts\": { \"SingularDisplayName\": \"Storage account (classic)\" }\r\n ,\"microsoft.classicstorage/storageaccounts/disks\": { \"SingularDisplayName\": \"Disk (classic)\" }\r\n ,\"microsoft.classicstorage/storageaccounts/osimages\": { \"SingularDisplayName\": \"OS image (classic)\" }\r\n ,\"microsoft.classicstorage/storageaccounts/vmimages\": { \"SingularDisplayName\": \"VM image (classic)\" }\r\n ,\"microsoft.cleanroom/cleanrooms\": { \"SingularDisplayName\": \"Microsoft.CleanRoom cleanroom\" }\r\n ,\"microsoft.cleanroom/collaborations\": { \"SingularDisplayName\": \"Microsoft.CleanRoom collaboration\" }\r\n ,\"microsoft.cleanroom/collaborations/contracts\": { \"SingularDisplayName\": \"Microsoft.CleanRoom collaborations contract\" }\r\n ,\"microsoft.cleanroom/consortiums\": { \"SingularDisplayName\": \"Microsoft.CleanRoom consortium\" }\r\n ,\"microsoft.cleanroom/microservices\": { \"SingularDisplayName\": \"Microsoft.CleanRoom microservice\" }\r\n ,\"microsoft.cloud/hubs\": { \"SingularDisplayName\": \"FinOps hub\" }\r\n ,\"microsoft.clouddeviceplatform/delegatedidentities\": { \"SingularDisplayName\": \"Microsoft.CloudDevicePlatform delegated identity\" }\r\n ,\"microsoft.cloudhealth/healthmodels\": { \"SingularDisplayName\": \"Health Model\" }\r\n ,\"microsoft.cloudtest/accounts\": { \"SingularDisplayName\": \"CloudTest Account\" }\r\n ,\"microsoft.cloudtest/buildcaches\": { \"SingularDisplayName\": \"1ES Build Cache\" }\r\n ,\"microsoft.cloudtest/hostedpools\": { \"SingularDisplayName\": \"1ES Hosted Pool\" }\r\n ,\"microsoft.cloudtest/images\": { \"SingularDisplayName\": \"1ES Image\" }\r\n ,\"microsoft.cloudtest/pools\": { \"SingularDisplayName\": \"CloudTest Pool\" }\r\n ,\"microsoft.clusterstor/nodes\": { \"SingularDisplayName\": \"ClusterStor\" }\r\n ,\"microsoft.codesigning/codesigningaccounts\": { \"SingularDisplayName\": \"Trusted Signing Account\" }\r\n ,\"microsoft.codespaces/plans\": { \"SingularDisplayName\": \"Microsoft.Codespaces plan\" }\r\n ,\"microsoft.cognitiveservices/accounts\": { \"SingularDisplayName\": \"Azure AI Foundry\" }\r\n ,\"microsoft.cognitiveservices/accounts/projects\": { \"SingularDisplayName\": \"Azure AI Foundry project\" }\r\n ,\"microsoft.cognitiveservices/commitmentplans\": { \"SingularDisplayName\": \"Microsoft.CognitiveServices commitment plan\" }\r\n ,\"microsoft.cognitiveservices/commitmentplans/accountassociations\": { \"SingularDisplayName\": \"Microsoft.CognitiveServices commitment plans account association\" }\r\n ,\"microsoft.communication/communicationservices\": { \"SingularDisplayName\": \"Communication Service\" }\r\n ,\"microsoft.communication/emailservices\": { \"SingularDisplayName\": \"Email Communication Service\" }\r\n ,\"microsoft.communication/emailservices/domains\": { \"SingularDisplayName\": \"Email Communication Services Domain\" }\r\n ,\"microsoft.community/communitytrainings\": { \"SingularDisplayName\": \"Community Training\" }\r\n ,\"microsoft.compositesolutions/compositesolutiondefinitions\": { \"SingularDisplayName\": \"Microsoft.CompositeSolutions composite solution definition\" }\r\n ,\"microsoft.compositesolutions/compositesolutions\": { \"SingularDisplayName\": \"Microsoft.CompositeSolutions composite solution\" }\r\n ,\"microsoft.compute/availabilitysets\": { \"SingularDisplayName\": \"Availability set\" }\r\n ,\"microsoft.compute/capacityreservationgroups\": { \"SingularDisplayName\": \"Capacity Reservation Group\" }\r\n ,\"microsoft.compute/capacityreservationgroups/capacityreservations\": { \"SingularDisplayName\": \"Capacity reservation\" }\r\n ,\"microsoft.compute/capacityreservationgroupscomputehub\": { \"SingularDisplayName\": \"Capacity Reservation Group\" }\r\n ,\"microsoft.compute/cloudservices\": { \"SingularDisplayName\": \"Cloud service (extended support)\" }\r\n ,\"microsoft.compute/computefleetinstances\": { \"SingularDisplayName\": \"Instance\" }\r\n ,\"microsoft.compute/computefleetscalesets\": { \"SingularDisplayName\": \"Virtual machine scale set\" }\r\n ,\"microsoft.compute/diskaccesses\": { \"SingularDisplayName\": \"Disk Access\" }\r\n ,\"microsoft.compute/diskencryptionsets\": { \"SingularDisplayName\": \"Disk Encryption Set\" }\r\n ,\"microsoft.compute/disks\": { \"SingularDisplayName\": \"Disk\" }\r\n ,\"microsoft.compute/galleries\": { \"SingularDisplayName\": \"Azure compute gallery\" }\r\n ,\"microsoft.compute/galleries/applications\": { \"SingularDisplayName\": \"VM application definition\" }\r\n ,\"microsoft.compute/galleries/applications/versions\": { \"SingularDisplayName\": \"VM application version\" }\r\n ,\"microsoft.compute/galleries/images\": { \"SingularDisplayName\": \"VM image definition\" }\r\n ,\"microsoft.compute/galleries/images/versions\": { \"SingularDisplayName\": \"VM image version\" }\r\n ,\"microsoft.compute/galleries/imagescomputehub\": { \"SingularDisplayName\": \"VM image definition\" }\r\n ,\"microsoft.compute/hostgroups\": { \"SingularDisplayName\": \"Host group\" }\r\n ,\"microsoft.compute/hostgroups/hosts\": { \"SingularDisplayName\": \"Host\" }\r\n ,\"microsoft.compute/hostgroupscomputehub\": { \"SingularDisplayName\": \"Host group\" }\r\n ,\"microsoft.compute/images\": { \"SingularDisplayName\": \"Image\" }\r\n ,\"microsoft.compute/imagescomputehub\": { \"SingularDisplayName\": \"Image\" }\r\n ,\"microsoft.compute/locations/communitygalleries/images\": { \"SingularDisplayName\": \"Community image\" }\r\n ,\"microsoft.compute/locations/communitygalleries/imagescomputehub\": { \"SingularDisplayName\": \"Community image\" }\r\n ,\"microsoft.compute/proximityplacementgroups\": { \"SingularDisplayName\": \"Proximity placement group\" }\r\n ,\"microsoft.compute/proximityplacementgroupscomputehub\": { \"SingularDisplayName\": \"Proximity placement group\" }\r\n ,\"microsoft.compute/restorepointcollections\": { \"SingularDisplayName\": \"Restore Point Collection\" }\r\n ,\"microsoft.compute/restorepointcollections/restorepoints\": { \"SingularDisplayName\": \"Restore Point\" }\r\n ,\"microsoft.compute/snapshots\": { \"SingularDisplayName\": \"Snapshot\" }\r\n ,\"microsoft.compute/sshpublickeys\": { \"SingularDisplayName\": \"SSH key\" }\r\n ,\"microsoft.compute/standbypoolinstance\": { \"SingularDisplayName\": \"Standby pool\" }\r\n ,\"microsoft.compute/virtualmachinecomputehub\": { \"SingularDisplayName\": \"Virtual machine\" }\r\n ,\"microsoft.compute/virtualmachineflexinstances\": { \"SingularDisplayName\": \"Instance\" }\r\n ,\"microsoft.compute/virtualmachines\": { \"SingularDisplayName\": \"Virtual machine\" }\r\n ,\"microsoft.compute/virtualmachines/providers/guestconfigurationassignments\": { \"SingularDisplayName\": \"Guest Assignment\" }\r\n ,\"microsoft.compute/virtualmachinescalesets\": { \"SingularDisplayName\": \"Virtual machine scale set\" }\r\n ,\"microsoft.compute/virtualmachinescalesets/providers/guestconfigurationassignments\": { \"SingularDisplayName\": \"Guest Assignment\" }\r\n ,\"microsoft.compute/virtualmachinescalesets/virtualmachines\": { \"SingularDisplayName\": \"Virtual machine scale set instance\" }\r\n ,\"microsoft.compute/virtualmachinescalesets/virtualmachines/networkinterfaces/ipconfigurations/publicipaddresses\": { \"SingularDisplayName\": \"Public IP address\" }\r\n ,\"microsoft.compute/virtualmachinescalesetscomputehub\": { \"SingularDisplayName\": \"Virtual machine scale set\" }\r\n ,\"microsoft.computehub/advisorcost\": { \"SingularDisplayName\": \"Recommendations\" }\r\n ,\"microsoft.computehub/advisoroperationalexcellence\": { \"SingularDisplayName\": \"Recommendations\" }\r\n ,\"microsoft.computehub/advisorperformance\": { \"SingularDisplayName\": \"Recommendations\" }\r\n ,\"microsoft.computehub/advisorreliability\": { \"SingularDisplayName\": \"Recommendations\" }\r\n ,\"microsoft.computehub/advisorsecurity\": { \"SingularDisplayName\": \"Recommendations\" }\r\n ,\"microsoft.computehub/all\": { \"SingularDisplayName\": \"All resources\" }\r\n ,\"microsoft.computehub/backup\": { \"SingularDisplayName\": \"Backup job\" }\r\n ,\"microsoft.computehub/computehubmain\": { \"SingularDisplayName\": \"Compute infrastructure\" }\r\n ,\"microsoft.computehub/healthevents\": { \"SingularDisplayName\": \"Health events\" }\r\n ,\"microsoft.computehub/linuxostype\": { \"SingularDisplayName\": \"Linux OS\" }\r\n ,\"microsoft.computehub/microsoftdefenderfreetrialsubscription\": { \"SingularDisplayName\": \"Microsoft defender\" }\r\n ,\"microsoft.computehub/microsoftdefenderstandardsubscription\": { \"SingularDisplayName\": \"Microsoft defender\" }\r\n ,\"microsoft.computehub/outages\": { \"SingularDisplayName\": \"Outages\" }\r\n ,\"microsoft.computehub/powerstatedeallocated\": { \"SingularDisplayName\": \"Power states\" }\r\n ,\"microsoft.computehub/powerstaterunning\": { \"SingularDisplayName\": \"Power states\" }\r\n ,\"microsoft.computehub/powerstatestopped\": { \"SingularDisplayName\": \"Power states\" }\r\n ,\"microsoft.computehub/provisioningstatefailedresources\": { \"SingularDisplayName\": \"Provisioning states\" }\r\n ,\"microsoft.computehub/provisioningstatesucceededresources\": { \"SingularDisplayName\": \"Provisioning states\" }\r\n ,\"microsoft.computehub/windowsostype\": { \"SingularDisplayName\": \"Windows OS\" }\r\n ,\"microsoft.computeschedule/autoactions\": { \"SingularDisplayName\": \"Automatic Action\" }\r\n ,\"microsoft.computeschedule/autoactions/occurrences\": { \"SingularDisplayName\": \"Microsoft.ComputeSchedule auto actions occurrence\" }\r\n ,\"microsoft.confidentialledger/ledgers\": { \"SingularDisplayName\": \"Confidential Ledger\" }\r\n ,\"microsoft.confidentialledger/managedccfs\": { \"SingularDisplayName\": \"Managed CCF App\" }\r\n ,\"microsoft.confluent/agreements\": { \"SingularDisplayName\": \"Microsoft.Confluent agreement\" }\r\n ,\"microsoft.confluent/organizations\": { \"SingularDisplayName\": \"Confluent organization\" }\r\n ,\"microsoft.connectedcache/cachenodes\": { \"SingularDisplayName\": \"Connected Cache for ISP\" }\r\n ,\"microsoft.connectedcache/enterprisecustomers\": { \"SingularDisplayName\": \"Connected Cache for Enterprise & Education\" }\r\n ,\"microsoft.connectedcache/enterprisemcccustomers\": { \"SingularDisplayName\": \"Connected Cache for Enterprise & Education\" }\r\n ,\"microsoft.connectedcache/enterprisemcccustomers/enterprisemcccachenodes\": { \"SingularDisplayName\": \"MCC CacheNode for Enterprise\" }\r\n ,\"microsoft.connectedcache/ispcustomers\": { \"SingularDisplayName\": \"Connected Cache for ISP\" }\r\n ,\"microsoft.connectedcredentials/credentials\": { \"SingularDisplayName\": \"Microsoft.ConnectedCredentials credential\" }\r\n ,\"microsoft.connectedvehicle/platformaccounts\": { \"SingularDisplayName\": \"Microsoft.ConnectedVehicle platform account\" }\r\n ,\"microsoft.connectedvmwarevsphere/clusters\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere cluster\" }\r\n ,\"microsoft.connectedvmwarevsphere/datastores\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere datastore\" }\r\n ,\"microsoft.connectedvmwarevsphere/hosts\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere host\" }\r\n ,\"microsoft.connectedvmwarevsphere/resourcepools\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere resource pool\" }\r\n ,\"microsoft.connectedvmwarevsphere/vcenters\": { \"SingularDisplayName\": \"VMware vCenter\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachineinstances\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere virtual machine instance\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachineinstances/guestagents\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere virtual machine instances guest agent\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachineinstances/hybrididentitymetadata\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere virtual machine instances hybrid identity metadata\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachines\": { \"SingularDisplayName\": \"VMware + AVS virtual machine\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachines/providers/guestconfigurationassignments\": { \"SingularDisplayName\": \"Guest Assignment\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachinetemplates\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere virtual machine template\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualnetworks\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere virtual network\" }\r\n ,\"microsoft.consumption/budgets\": { \"SingularDisplayName\": \"Microsoft.Consumption budget\" }\r\n ,\"microsoft.consumption/credits\": { \"SingularDisplayName\": \"Microsoft.Consumption credit\" }\r\n ,\"microsoft.consumption/pricesheets\": { \"SingularDisplayName\": \"Microsoft.Consumption pricesheet\" }\r\n ,\"microsoft.containerinstance/containergroupprofiles\": { \"SingularDisplayName\": \"Microsoft.ContainerInstance container group profile\" }\r\n ,\"microsoft.containerinstance/containergroupprofiles/revisions\": { \"SingularDisplayName\": \"Microsoft.ContainerInstance container group profiles revision\" }\r\n ,\"microsoft.containerinstance/containergroups\": { \"SingularDisplayName\": \"Container instances\" }\r\n ,\"microsoft.containerinstance/ngroups\": { \"SingularDisplayName\": \"Microsoft.ContainerInstance ngroup\" }\r\n ,\"microsoft.containerregistry/registries\": { \"SingularDisplayName\": \"Container registry\" }\r\n ,\"microsoft.containerregistry/registries/replications\": { \"SingularDisplayName\": \"Container registry replication\" }\r\n ,\"microsoft.containerregistry/registries/scopemaps\": { \"SingularDisplayName\": \"Container registry scope map\" }\r\n ,\"microsoft.containerregistry/registries/tokens\": { \"SingularDisplayName\": \"Container registry token\" }\r\n ,\"microsoft.containerregistry/registries/webhooks\": { \"SingularDisplayName\": \"Container registry webhook\" }\r\n ,\"microsoft.containerservice/fleets\": { \"SingularDisplayName\": \"Kubernetes fleet manager\" }\r\n ,\"microsoft.containerservice/managedclusters\": { \"SingularDisplayName\": \"Kubernetes service\" }\r\n ,\"microsoft.containerservice/managedclusters/managednamespaces\": { \"SingularDisplayName\": \"Managed namespace\" }\r\n ,\"microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/extensions\": { \"SingularDisplayName\": \"Kubernetes service extension\" }\r\n ,\"microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/fluxconfigurations\": { \"SingularDisplayName\": \"GitOps configuration\" }\r\n ,\"microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/namespaces\": { \"SingularDisplayName\": \"Kubernetes namespace\" }\r\n ,\"microsoft.containerservice/managedclusters/namespaces\": { \"SingularDisplayName\": \"Managed namespace\" }\r\n ,\"microsoft.containerservice/managedclustersnapshots\": { \"SingularDisplayName\": \"Microsoft.ContainerService managedclustersnapshot\" }\r\n ,\"microsoft.containerservice/snapshots\": { \"SingularDisplayName\": \"Microsoft.ContainerService snapshot\" }\r\n ,\"microsoft.containerstorage/pools\": { \"SingularDisplayName\": \"Container storage\" }\r\n ,\"microsoft.costmanagement/alerts\": { \"SingularDisplayName\": \"Microsoft.CostManagement alert\" }\r\n ,\"microsoft.costmanagement/budgets\": { \"SingularDisplayName\": \"Microsoft.CostManagement budget\" }\r\n ,\"microsoft.costmanagement/cloudconnectors\": { \"SingularDisplayName\": \"Microsoft.CostManagement cloud connector\" }\r\n ,\"microsoft.costmanagement/connectors\": { \"SingularDisplayName\": \"Microsoft.CostManagement connector\" }\r\n ,\"microsoft.costmanagement/costallocationrules\": { \"SingularDisplayName\": \"Microsoft.CostManagement cost allocation rule\" }\r\n ,\"microsoft.costmanagement/costdetailsoperationresults\": { \"SingularDisplayName\": \"Microsoft.CostManagement cost details operation result\" }\r\n ,\"microsoft.costmanagement/exports\": { \"SingularDisplayName\": \"Microsoft.CostManagement export\" }\r\n ,\"microsoft.costmanagement/externalbillingaccounts\": { \"SingularDisplayName\": \"Microsoft.CostManagement external billing account\" }\r\n ,\"microsoft.costmanagement/externalsubscriptions\": { \"SingularDisplayName\": \"Microsoft.CostManagement external subscription\" }\r\n ,\"microsoft.costmanagement/markuprules\": { \"SingularDisplayName\": \"Microsoft.CostManagement markup rule\" }\r\n ,\"microsoft.costmanagement/operationstatus\": { \"SingularDisplayName\": \"Microsoft.CostManagement operation statu\" }\r\n ,\"microsoft.costmanagement/reportconfigs\": { \"SingularDisplayName\": \"Microsoft.CostManagement reportconfig\" }\r\n ,\"microsoft.costmanagement/reports\": { \"SingularDisplayName\": \"Microsoft.CostManagement report\" }\r\n ,\"microsoft.costmanagement/scheduledactions\": { \"SingularDisplayName\": \"Microsoft.CostManagement scheduled action\" }\r\n ,\"microsoft.costmanagement/settings\": { \"SingularDisplayName\": \"Microsoft.CostManagement setting\" }\r\n ,\"microsoft.costmanagement/views\": { \"SingularDisplayName\": \"Microsoft.CostManagement view\" }\r\n ,\"microsoft.customerlockbox/requests\": { \"SingularDisplayName\": \"Microsoft.CustomerLockbox request\" }\r\n ,\"microsoft.customerlockbox/tenantoptedin\": { \"SingularDisplayName\": \"Microsoft.CustomerLockbox tenant opted in\" }\r\n ,\"microsoft.customproviders/associations\": { \"SingularDisplayName\": \"Microsoft.CustomProviders association\" }\r\n ,\"microsoft.customproviders/resourceproviders\": { \"SingularDisplayName\": \"Microsoft.CustomProviders resource provider\" }\r\n ,\"microsoft.dashboard/dashboards\": { \"SingularDisplayName\": \"Azure Monitor dashboards with Grafana\" }\r\n ,\"microsoft.dashboard/grafana\": { \"SingularDisplayName\": \"Azure Managed Grafana\" }\r\n ,\"microsoft.dataaccelerator/indexclusters\": { \"SingularDisplayName\": \"Microsoft.DataAccelerator index cluster\" }\r\n ,\"microsoft.databasefleetmanager/fleets\": { \"SingularDisplayName\": \"Database fleet manager\" }\r\n ,\"microsoft.databasefleetmanager/fleets/fleetspaces\": { \"SingularDisplayName\": \"Fleetspaces\" }\r\n ,\"microsoft.databasefleetmanager/fleets/fleetspaces/databases\": { \"SingularDisplayName\": \"Fleet managed database\" }\r\n ,\"microsoft.databasefleetmanager/fleets/tiers\": { \"SingularDisplayName\": \"tier\" }\r\n ,\"microsoft.databasewatcher/watchers\": { \"SingularDisplayName\": \"Database watcher\" }\r\n ,\"microsoft.databox/jobs\": { \"SingularDisplayName\": \"Azure Data Box\" }\r\n ,\"microsoft.databoxedge/databoxedgedevices\": { \"SingularDisplayName\": \"Azure Stack Edge / Data Box Gateway\" }\r\n ,\"microsoft.databricks/accessconnectors\": { \"SingularDisplayName\": \"Access Connector for Azure Databricks\" }\r\n ,\"microsoft.databricks/workspaces\": { \"SingularDisplayName\": \"Azure Databricks Service\" }\r\n ,\"microsoft.datacatalog/catalogs\": { \"SingularDisplayName\": \"Data catalog\" }\r\n ,\"microsoft.datacollaboration/workspaces\": { \"SingularDisplayName\": \"Project CI\" }\r\n ,\"microsoft.datadog/agreements\": { \"SingularDisplayName\": \"Microsoft.Datadog agreement\" }\r\n ,\"microsoft.datadog/monitors\": { \"SingularDisplayName\": \"Datadog\" }\r\n ,\"microsoft.datadog/subscriptionstatuses\": { \"SingularDisplayName\": \"Microsoft.Datadog subscription statuse\" }\r\n ,\"microsoft.datafactory/datafactories\": { \"SingularDisplayName\": \"Data factory\" }\r\n ,\"microsoft.datafactory/factories\": { \"SingularDisplayName\": \"Data factory (V2)\" }\r\n ,\"microsoft.datafactory/factories/pipelines\": { \"SingularDisplayName\": \"Data Factory pipeline\" }\r\n ,\"microsoft.datafactory/factories/triggers\": { \"SingularDisplayName\": \"Data Factory trigger\" }\r\n ,\"microsoft.datalakeanalytics/accounts\": { \"SingularDisplayName\": \"Data Lake Analytics account\" }\r\n ,\"microsoft.datalakestore/accounts\": { \"SingularDisplayName\": \"Data Lake Storage Gen1\" }\r\n ,\"microsoft.datamigration/databasemigrations\": { \"SingularDisplayName\": \"Microsoft.DataMigration database migration\" }\r\n ,\"microsoft.datamigration/migrationservices\": { \"SingularDisplayName\": \"Microsoft.DataMigration migration service\" }\r\n ,\"microsoft.datamigration/services\": { \"SingularDisplayName\": \"Azure Database Migration Service (classic)\" }\r\n ,\"microsoft.datamigration/services/projects\": { \"SingularDisplayName\": \"Azure Database Migration Project\" }\r\n ,\"microsoft.datamigration/sqlmigrationservices\": { \"SingularDisplayName\": \"Azure Database Migration Service\" }\r\n ,\"microsoft.dataprotection/backupvaults\": { \"SingularDisplayName\": \"Backup vault\" }\r\n ,\"microsoft.dataprotection/resourceguards\": { \"SingularDisplayName\": \"Resource Guard\" }\r\n ,\"microsoft.datareplication/replicationfabrics\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication fabric\" }\r\n ,\"microsoft.datareplication/replicationfabrics/fabricagents\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication fabrics fabric agent\" }\r\n ,\"microsoft.datareplication/replicationfabrics/fabricagents/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication fabrics fabric agents operation\" }\r\n ,\"microsoft.datareplication/replicationfabrics/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication fabrics operation\" }\r\n ,\"microsoft.datareplication/replicationvaults\": { \"SingularDisplayName\": \"Data replication vault\" }\r\n ,\"microsoft.datareplication/replicationvaults/alertsettings\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults alert setting\" }\r\n ,\"microsoft.datareplication/replicationvaults/events\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults event\" }\r\n ,\"microsoft.datareplication/replicationvaults/jobs\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults job\" }\r\n ,\"microsoft.datareplication/replicationvaults/jobs/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults jobs operation\" }\r\n ,\"microsoft.datareplication/replicationvaults/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults operation\" }\r\n ,\"microsoft.datareplication/replicationvaults/privateendpointconnectionproxies\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults private endpoint connection proxy\" }\r\n ,\"microsoft.datareplication/replicationvaults/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults private endpoint connection\" }\r\n ,\"microsoft.datareplication/replicationvaults/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults private link resource\" }\r\n ,\"microsoft.datareplication/replicationvaults/protecteditems\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults protected item\" }\r\n ,\"microsoft.datareplication/replicationvaults/protecteditems/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults protected items operation\" }\r\n ,\"microsoft.datareplication/replicationvaults/protecteditems/recoverypoints\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults protected items recovery point\" }\r\n ,\"microsoft.datareplication/replicationvaults/replicationextensions\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults replication extension\" }\r\n ,\"microsoft.datareplication/replicationvaults/replicationextensions/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults replication extensions operation\" }\r\n ,\"microsoft.datareplication/replicationvaults/replicationpolicies\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults replication policy\" }\r\n ,\"microsoft.datareplication/replicationvaults/replicationpolicies/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults replication policies operation\" }\r\n ,\"microsoft.datashare/accounts\": { \"SingularDisplayName\": \"Data Share\" }\r\n ,\"microsoft.dbformariadb/servers\": { \"SingularDisplayName\": \"Azure Database for MariaDB server\" }\r\n ,\"microsoft.dbformysql/flexibleservers\": { \"SingularDisplayName\": \"Azure Database for MySQL flexible server\" }\r\n ,\"microsoft.dbformysql/servers\": { \"SingularDisplayName\": \"MySQL server\" }\r\n ,\"microsoft.dbforpostgresql/flexibleservers\": { \"SingularDisplayName\": \"Azure Database for PostgreSQL flexible server\" }\r\n ,\"microsoft.dbforpostgresql/servergroupsv2\": { \"SingularDisplayName\": \"Azure Cosmos DB for PostgreSQL Cluster\" }\r\n ,\"microsoft.dbforpostgresql/servers\": { \"SingularDisplayName\": \"PostgreSQL server\" }\r\n ,\"microsoft.delegatednetwork/controller\": { \"SingularDisplayName\": \"Microsoft.DelegatedNetwork controller\" }\r\n ,\"microsoft.delegatednetwork/delegatedsubnets\": { \"SingularDisplayName\": \"Microsoft.DelegatedNetwork delegated subnet\" }\r\n ,\"microsoft.delegatednetwork/orchestrators\": { \"SingularDisplayName\": \"Microsoft.DelegatedNetwork orchestrator\" }\r\n ,\"microsoft.dependencymap/maps\": { \"SingularDisplayName\": \"Microsoft.DependencyMap map\" }\r\n ,\"microsoft.dependencymap/maps/discoverysources\": { \"SingularDisplayName\": \"Microsoft.DependencyMap maps discovery source\" }\r\n ,\"microsoft.deploymentmanager/artifactsources\": { \"SingularDisplayName\": \"Microsoft.DeploymentManager artifact source\" }\r\n ,\"microsoft.deploymentmanager/rollouts\": { \"SingularDisplayName\": \"Rollout\" }\r\n ,\"microsoft.deploymentmanager/servicetopologies\": { \"SingularDisplayName\": \"Microsoft.DeploymentManager service topology\" }\r\n ,\"microsoft.deploymentmanager/servicetopologies/services\": { \"SingularDisplayName\": \"Microsoft.DeploymentManager service topologies service\" }\r\n ,\"microsoft.deploymentmanager/servicetopologies/services/serviceunits\": { \"SingularDisplayName\": \"Microsoft.DeploymentManager service topologies services service unit\" }\r\n ,\"microsoft.deploymentmanager/steps\": { \"SingularDisplayName\": \"Microsoft.DeploymentManager step\" }\r\n ,\"microsoft.desktopvirtualization/appattachpackages\": { \"SingularDisplayName\": \"App attach package\" }\r\n ,\"microsoft.desktopvirtualization/applicationgroups\": { \"SingularDisplayName\": \"Application group\" }\r\n ,\"microsoft.desktopvirtualization/hostpools\": { \"SingularDisplayName\": \"Host pool\" }\r\n ,\"microsoft.desktopvirtualization/scalingplans\": { \"SingularDisplayName\": \"Scaling plan\" }\r\n ,\"microsoft.desktopvirtualization/workspaces\": { \"SingularDisplayName\": \"Workspace\" }\r\n ,\"microsoft.devai/instances\": { \"SingularDisplayName\": \"Microsoft.DevAI instance\" }\r\n ,\"microsoft.devai/instances/experiments\": { \"SingularDisplayName\": \"Microsoft.DevAI instances experiment\" }\r\n ,\"microsoft.devai/instances/sandboxes\": { \"SingularDisplayName\": \"Microsoft.DevAI instances sandbox\" }\r\n ,\"microsoft.devai/instances/sandboxes/experiments\": { \"SingularDisplayName\": \"Microsoft.DevAI instances sandboxes experiment\" }\r\n ,\"microsoft.devcenter/devcenters\": { \"SingularDisplayName\": \"Dev center\" }\r\n ,\"microsoft.devcenter/devcenters/devboxdefinitions\": { \"SingularDisplayName\": \"Dev Box definition\" }\r\n ,\"microsoft.devcenter/networkconnections\": { \"SingularDisplayName\": \"Network connection\" }\r\n ,\"microsoft.devcenter/plans\": { \"SingularDisplayName\": \"Dev center plan\" }\r\n ,\"microsoft.devcenter/projects\": { \"SingularDisplayName\": \"Project\" }\r\n ,\"microsoft.devcenter/projects/pools\": { \"SingularDisplayName\": \"Pool\" }\r\n ,\"microsoft.developmentwindows365/developmentcloudpcdelegatedmsis\": { \"SingularDisplayName\": \"Microsoft.DevelopmentWindows365 development cloud pc delegated msi\" }\r\n ,\"microsoft.devhub/iacprofiles\": { \"SingularDisplayName\": \"Infrastructure as Code Automation\" }\r\n ,\"microsoft.devhub/templates\": { \"SingularDisplayName\": \"Microsoft.DevHub template\" }\r\n ,\"microsoft.devhub/templates/versions\": { \"SingularDisplayName\": \"Microsoft.DevHub templates version\" }\r\n ,\"microsoft.devhub/workflows\": { \"SingularDisplayName\": \"Microsoft.DevHub workflow\" }\r\n ,\"microsoft.deviceonboarding/discoveryservices\": { \"SingularDisplayName\": \"Microsoft.DeviceOnboarding discovery service\" }\r\n ,\"microsoft.deviceonboarding/discoveryservices/ownershipvoucherpublickeys\": { \"SingularDisplayName\": \"Microsoft.DeviceOnboarding discovery services ownership voucher public key\" }\r\n ,\"microsoft.deviceonboarding/onboardingservices\": { \"SingularDisplayName\": \"Microsoft.DeviceOnboarding onboarding service\" }\r\n ,\"microsoft.deviceonboarding/onboardingservices/policies\": { \"SingularDisplayName\": \"Microsoft.DeviceOnboarding onboarding services policy\" }\r\n ,\"microsoft.deviceregistry/assetendpointprofiles\": { \"SingularDisplayName\": \"IoT Asset Endpoint Profile\" }\r\n ,\"microsoft.deviceregistry/assets\": { \"SingularDisplayName\": \"IoT Asset\" }\r\n ,\"microsoft.deviceregistry/billingcontainers\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry billing container\" }\r\n ,\"microsoft.deviceregistry/devices\": { \"SingularDisplayName\": \"IoT Device\" }\r\n ,\"microsoft.deviceregistry/discoveredassetendpointprofiles\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry discovered asset endpoint profile\" }\r\n ,\"microsoft.deviceregistry/discoveredassets\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry discovered asset\" }\r\n ,\"microsoft.deviceregistry/namespaces\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespace\" }\r\n ,\"microsoft.deviceregistry/namespaces/assetendpointprofiles\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespaces asset endpoint profile\" }\r\n ,\"microsoft.deviceregistry/namespaces/assets\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespaces asset\" }\r\n ,\"microsoft.deviceregistry/namespaces/devices\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespaces device\" }\r\n ,\"microsoft.deviceregistry/namespaces/discoveredassetendpointprofiles\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespaces discovered asset endpoint profile\" }\r\n ,\"microsoft.deviceregistry/namespaces/discoveredassets\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespaces discovered asset\" }\r\n ,\"microsoft.deviceregistry/schemaregistries\": { \"SingularDisplayName\": \"IoT Schema Registry\" }\r\n ,\"microsoft.deviceregistry/schemaregistries/schemas\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry schema registries schema\" }\r\n ,\"microsoft.deviceregistry/schemaregistries/schemas/schemaversions\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry schema registries schemas schema version\" }\r\n ,\"microsoft.devices/iothubs\": { \"SingularDisplayName\": \"IoT hub\" }\r\n ,\"microsoft.devices/provisioningservices\": { \"SingularDisplayName\": \"Azure IoT Hub Device Provisioning Service (DPS)\" }\r\n ,\"microsoft.deviceupdate/accounts\": { \"SingularDisplayName\": \"Device Update for IoT Hub\" }\r\n ,\"microsoft.deviceupdate/updateaccounts\": { \"SingularDisplayName\": \"Device Update Account\" }\r\n ,\"microsoft.deviceupdate/updateaccounts/activedeployments\": { \"SingularDisplayName\": \"Device Update Active Deployment\" }\r\n ,\"microsoft.deviceupdate/updateaccounts/agents\": { \"SingularDisplayName\": \"Device Update Agent\" }\r\n ,\"microsoft.deviceupdate/updateaccounts/deployments\": { \"SingularDisplayName\": \"Device Update Deployment\" }\r\n ,\"microsoft.deviceupdate/updateaccounts/deviceclasses\": { \"SingularDisplayName\": \"Device Update Device Class\" }\r\n ,\"microsoft.deviceupdate/updateaccounts/updates\": { \"SingularDisplayName\": \"Device Update\" }\r\n ,\"microsoft.devops/pipelines\": { \"SingularDisplayName\": \"Microsoft.DevOps pipeline\" }\r\n ,\"microsoft.devopsinfrastructure/pools\": { \"SingularDisplayName\": \"Managed DevOps Pool\" }\r\n ,\"microsoft.devspaces/controllers\": { \"SingularDisplayName\": \"Microsoft.DevSpaces controller\" }\r\n ,\"microsoft.devtestlab/labs\": { \"SingularDisplayName\": \"DevTest lab\" }\r\n ,\"microsoft.devtestlab/labs/virtualmachines\": { \"SingularDisplayName\": \"DevTest Lab virtual machine\" }\r\n ,\"microsoft.devtestlab/schedules\": { \"SingularDisplayName\": \"Microsoft.DevTestLab schedule\" }\r\n ,\"microsoft.devtunnels/tunnelplans\": { \"SingularDisplayName\": \"Dev Tunnels Domain\" }\r\n ,\"microsoft.diagnostics/apollo\": { \"SingularDisplayName\": \"Microsoft.Diagnostics apollo\" }\r\n ,\"microsoft.digitaltwins/digitaltwinsinstances\": { \"SingularDisplayName\": \"Azure Digital Twins\" }\r\n ,\"microsoft.discovery/agents\": { \"SingularDisplayName\": \"Microsoft Discovery Agent\" }\r\n ,\"microsoft.discovery/bookshelves\": { \"SingularDisplayName\": \"Microsoft Discovery Bookshelf\" }\r\n ,\"microsoft.discovery/datacontainers\": { \"SingularDisplayName\": \"Microsoft Discovery Data Container\" }\r\n ,\"microsoft.discovery/datacontainers/dataassets\": { \"SingularDisplayName\": \"Data asset\" }\r\n ,\"microsoft.discovery/models\": { \"SingularDisplayName\": \"Microsoft Discovery Model\" }\r\n ,\"microsoft.discovery/storages\": { \"SingularDisplayName\": \"Microsoft Discovery Storage\" }\r\n ,\"microsoft.discovery/supercomputers\": { \"SingularDisplayName\": \"Microsoft Discovery Supercomputer\" }\r\n ,\"microsoft.discovery/supercomputers/nodepools\": { \"SingularDisplayName\": \"Nodepool\" }\r\n ,\"microsoft.discovery/tools\": { \"SingularDisplayName\": \"Microsoft Discovery Tool\" }\r\n ,\"microsoft.discovery/workflows\": { \"SingularDisplayName\": \"Microsoft Discovery Workflow\" }\r\n ,\"microsoft.discovery/workspaces\": { \"SingularDisplayName\": \"Microsoft Discovery Workspace\" }\r\n ,\"microsoft.discovery/workspaces/projects\": { \"SingularDisplayName\": \"Microsoft Discovery Project\" }\r\n ,\"microsoft.documentdb/cassandraclusters\": { \"SingularDisplayName\": \"Azure Managed Instance for Apache Cassandra\" }\r\n ,\"microsoft.documentdb/databaseaccounts\": { \"SingularDisplayName\": \"Cosmos DB account\" }\r\n ,\"microsoft.documentdb/fleets\": { \"SingularDisplayName\": \"Azure Cosmos DB Fleet\" }\r\n ,\"microsoft.documentdb/fleetspacepotentialdatabaseaccounts\": { \"SingularDisplayName\": \"Potential Azure Cosmos DB account\" }\r\n ,\"microsoft.documentdb/fleetspacepotentialdatabaseaccountswithlocations\": { \"SingularDisplayName\": \"Potential Azure Cosmos DB account\" }\r\n ,\"microsoft.documentdb/mongoclusters\": { \"SingularDisplayName\": \"Azure Cosmos DB for MongoDB (vCore)\" }\r\n ,\"microsoft.documentdb/throughputpools\": { \"SingularDisplayName\": \"Microsoft.DocumentDB throughput pool\" }\r\n ,\"microsoft.documentdb/throughputpools/throughputpoolaccounts\": { \"SingularDisplayName\": \"Microsoft.DocumentDB throughput pools throughput pool account\" }\r\n ,\"microsoft.domainregistration/domains\": { \"SingularDisplayName\": \"App Service Domain\" }\r\n ,\"microsoft.domainregistration/topleveldomains\": { \"SingularDisplayName\": \"Microsoft.DomainRegistration top level domain\" }\r\n ,\"microsoft.durabletask/namespaces\": { \"SingularDisplayName\": \"Microsoft.DurableTask namespace\" }\r\n ,\"microsoft.durabletask/namespaces/taskhubs\": { \"SingularDisplayName\": \"Task Hub\" }\r\n ,\"microsoft.durabletask/schedulers\": { \"SingularDisplayName\": \"Durable Task Scheduler\" }\r\n ,\"microsoft.durabletask/schedulers/taskhubs\": { \"SingularDisplayName\": \"Task Hub\" }\r\n ,\"microsoft.dynamics365fraudprotection/instances\": { \"SingularDisplayName\": \"Microsoft.Dynamics365FraudProtection instance\" }\r\n ,\"microsoft.easm/workspaces\": { \"SingularDisplayName\": \"Microsoft Defender EASM\" }\r\n ,\"microsoft.edge/configurations\": { \"SingularDisplayName\": \"Site configuration\" }\r\n ,\"microsoft.edge/configurations/arcgatewayconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations arc gateway configuration\" }\r\n ,\"microsoft.edge/configurations/connectivityconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations connectivity configuration\" }\r\n ,\"microsoft.edge/configurations/dynamicconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations dynamic configuration\" }\r\n ,\"microsoft.edge/configurations/dynamicconfigurations/versions\": { \"SingularDisplayName\": \"Microsoft.Edge configurations dynamic configurations version\" }\r\n ,\"microsoft.edge/configurations/networkconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations network configuration\" }\r\n ,\"microsoft.edge/configurations/securityconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations security configuration\" }\r\n ,\"microsoft.edge/configurations/timeserverconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations time server configuration\" }\r\n ,\"microsoft.edge/connectivitystatuses\": { \"SingularDisplayName\": \"Microsoft.Edge connectivity statuse\" }\r\n ,\"microsoft.edge/disconnectedoperations\": { \"SingularDisplayName\": \"Azure Local - disconnected operations\" }\r\n ,\"microsoft.edge/siteawareresourcetypes\": { \"SingularDisplayName\": \"Microsoft.Edge site aware resource type\" }\r\n ,\"microsoft.edge/sites\": { \"SingularDisplayName\": \"Site manager - Azure Arc\" }\r\n ,\"microsoft.edge/updates\": { \"SingularDisplayName\": \"Microsoft.Edge update\" }\r\n ,\"microsoft.edgemarketplace/offers\": { \"SingularDisplayName\": \"Microsoft.EdgeMarketplace offer\" }\r\n ,\"microsoft.edgemarketplace/publishers\": { \"SingularDisplayName\": \"Microsoft.EdgeMarketplace publisher\" }\r\n ,\"microsoft.edgeorder/addresses\": { \"SingularDisplayName\": \"Azure Edge Hardware Center Address\" }\r\n ,\"microsoft.edgeorder/bootstrapconfigurations\": { \"SingularDisplayName\": \"Site Key\" }\r\n ,\"microsoft.edgeorder/orderitems\": { \"SingularDisplayName\": \"Azure Edge Hardware Center\" }\r\n ,\"microsoft.edgeorder/virtual_orderitems\": { \"SingularDisplayName\": \"Device\" }\r\n ,\"microsoft.edgezones/extendedzones\": { \"SingularDisplayName\": \"Microsoft.EdgeZones extended zone\" }\r\n ,\"microsoft.education/grants\": { \"SingularDisplayName\": \"Microsoft.Education grant\" }\r\n ,\"microsoft.education/labs\": { \"SingularDisplayName\": \"Microsoft.Education lab\" }\r\n ,\"microsoft.education/labs/joinrequests\": { \"SingularDisplayName\": \"Microsoft.Education labs join request\" }\r\n ,\"microsoft.education/labs/students\": { \"SingularDisplayName\": \"Microsoft.Education labs student\" }\r\n ,\"microsoft.education/studentlabs\": { \"SingularDisplayName\": \"Microsoft.Education student lab\" }\r\n ,\"microsoft.elastic/monitors\": { \"SingularDisplayName\": \"Elastic Cloud Resource\" }\r\n ,\"microsoft.elasticsan/elasticsans\": { \"SingularDisplayName\": \"Elastic SAN\" }\r\n ,\"microsoft.energydataplatform/energyservices\": { \"SingularDisplayName\": \"Microsoft.EnergyDataPlatform energy service\" }\r\n ,\"microsoft.enterpriseknowledgegraph/services\": { \"SingularDisplayName\": \"Microsoft.EnterpriseKnowledgeGraph service\" }\r\n ,\"microsoft.enterprisesupport/enterprisesupports\": { \"SingularDisplayName\": \"Microsoft.EnterpriseSupport enterprise support\" }\r\n ,\"microsoft.eventgrid/domains\": { \"SingularDisplayName\": \"Event Grid Domain\" }\r\n ,\"microsoft.eventgrid/domains/topics\": { \"SingularDisplayName\": \"Event Grid Domain Topic\" }\r\n ,\"microsoft.eventgrid/eventsubscriptions\": { \"SingularDisplayName\": \"Microsoft.EventGrid event subscription\" }\r\n ,\"microsoft.eventgrid/extensiontopics\": { \"SingularDisplayName\": \"Event Grid extension topic\" }\r\n ,\"microsoft.eventgrid/namespaces\": { \"SingularDisplayName\": \"Event Grid Namespace\" }\r\n ,\"microsoft.eventgrid/namespaces/topics\": { \"SingularDisplayName\": \"Event Grid Namespace Topic\" }\r\n ,\"microsoft.eventgrid/namespaces/topics/eventsubscriptions\": { \"SingularDisplayName\": \"Event Subscription\" }\r\n ,\"microsoft.eventgrid/namespaces/topicspaces\": { \"SingularDisplayName\": \"Event Grid Topic Space\" }\r\n ,\"microsoft.eventgrid/partnerconfigurations\": { \"SingularDisplayName\": \"Event Grid Partner Configuration\" }\r\n ,\"microsoft.eventgrid/partnerdestinations\": { \"SingularDisplayName\": \"Event Grid Partner Destination\" }\r\n ,\"microsoft.eventgrid/partnernamespaces\": { \"SingularDisplayName\": \"Event Grid Partner Namespace\" }\r\n ,\"microsoft.eventgrid/partnernamespaces/channels\": { \"SingularDisplayName\": \"Event Grid Channel\" }\r\n ,\"microsoft.eventgrid/partnerregistrations\": { \"SingularDisplayName\": \"Event Grid Partner Registration\" }\r\n ,\"microsoft.eventgrid/partnertopics\": { \"SingularDisplayName\": \"Event Grid Partner Topic\" }\r\n ,\"microsoft.eventgrid/systemtopics\": { \"SingularDisplayName\": \"Event Grid System Topic\" }\r\n ,\"microsoft.eventgrid/systemtopics/eventsubscriptions\": { \"SingularDisplayName\": \"Event Grid Subscriptions\" }\r\n ,\"microsoft.eventgrid/topics\": { \"SingularDisplayName\": \"Event Grid Topic\" }\r\n ,\"microsoft.eventgrid/topictypes\": { \"SingularDisplayName\": \"Microsoft.EventGrid topic type\" }\r\n ,\"microsoft.eventgrid/verifiedpartners\": { \"SingularDisplayName\": \"Microsoft.EventGrid verified partner\" }\r\n ,\"microsoft.eventhub/clusters\": { \"SingularDisplayName\": \"Event Hubs Cluster\" }\r\n ,\"microsoft.eventhub/namespaces\": { \"SingularDisplayName\": \"Event Hubs namespace\" }\r\n ,\"microsoft.eventhub/namespaces/disasterrecoveryconfigs\": { \"SingularDisplayName\": \"Event Hubs Geo-DR Alias\" }\r\n ,\"microsoft.eventhub/namespaces/eventhubs\": { \"SingularDisplayName\": \"Event Hubs Instance\" }\r\n ,\"microsoft.eventhub/namespaces/providers/diagnosticsettings\": { \"SingularDisplayName\": \"Diagnostic settings\" }\r\n ,\"microsoft.eventhub/namespaces/schemagroups\": { \"SingularDisplayName\": \"Schema Group\" }\r\n ,\"microsoft.experimentation/experimentworkspaces\": { \"SingularDisplayName\": \"Experiment Workspace\" }\r\n ,\"microsoft.extendedlocation/customlocations\": { \"SingularDisplayName\": \"Custom location\" }\r\n ,\"microsoft.fabric/capacities\": { \"SingularDisplayName\": \"Fabric Capacity\" }\r\n ,\"microsoft.fabric/privatelinkservicesforfabric\": { \"SingularDisplayName\": \"Microsoft.Fabric private link services for fabric\" }\r\n ,\"microsoft.fabric/privatelinkservicesforfabric/operationresults\": { \"SingularDisplayName\": \"Microsoft.Fabric private link services for fabric operation result\" }\r\n ,\"microsoft.fabric/privatelinkservicesforfabric/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.Fabric private link services for fabric private endpoint connection\" }\r\n ,\"microsoft.fabric/privatelinkservicesforfabric/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.Fabric private link services for fabric private link resource\" }\r\n ,\"microsoft.fairfieldgardens/deviceprovisioningstates\": { \"SingularDisplayName\": \"Microsoft.FairfieldGardens device provisioning state\" }\r\n ,\"microsoft.fairfieldgardens/provisioningresources\": { \"SingularDisplayName\": \"Fairfield Gardens\" }\r\n ,\"microsoft.fairfieldgardens/provisioningresources/provisioningpolicies\": { \"SingularDisplayName\": \"Provisioning policy\" }\r\n ,\"microsoft.falcon/namespaces\": { \"SingularDisplayName\": \"Microsoft.Falcon namespace\" }\r\n ,\"microsoft.features/featureprovidernamespaces/featureconfigurations\": { \"SingularDisplayName\": \"Preview features\" }\r\n ,\"microsoft.fidalgo/devcenters\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenter\" }\r\n ,\"microsoft.fidalgo/devcenters/attachednetworks\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters attachednetwork\" }\r\n ,\"microsoft.fidalgo/devcenters/catalogs\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters catalog\" }\r\n ,\"microsoft.fidalgo/devcenters/catalogs/items\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters catalogs item\" }\r\n ,\"microsoft.fidalgo/devcenters/devboxdefinitions\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters devboxdefinition\" }\r\n ,\"microsoft.fidalgo/devcenters/environmenttypes\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters environment type\" }\r\n ,\"microsoft.fidalgo/devcenters/galleries\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters gallery\" }\r\n ,\"microsoft.fidalgo/devcenters/galleries/images\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters galleries image\" }\r\n ,\"microsoft.fidalgo/devcenters/galleries/images/versions\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters galleries images version\" }\r\n ,\"microsoft.fidalgo/devcenters/mappings\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters mapping\" }\r\n ,\"microsoft.fidalgo/machinedefinitions\": { \"SingularDisplayName\": \"Microsoft.Fidalgo machinedefinition\" }\r\n ,\"microsoft.fidalgo/networksettings\": { \"SingularDisplayName\": \"Microsoft.Fidalgo networksetting\" }\r\n ,\"microsoft.fidalgo/networksettings/healthchecks\": { \"SingularDisplayName\": \"Microsoft.Fidalgo networksettings healthcheck\" }\r\n ,\"microsoft.fidalgo/projects\": { \"SingularDisplayName\": \"Microsoft.Fidalgo project\" }\r\n ,\"microsoft.fidalgo/projects/attachednetworks\": { \"SingularDisplayName\": \"Microsoft.Fidalgo projects attachednetwork\" }\r\n ,\"microsoft.fidalgo/projects/devboxdefinitions\": { \"SingularDisplayName\": \"Microsoft.Fidalgo projects devboxdefinition\" }\r\n ,\"microsoft.fidalgo/projects/environments\": { \"SingularDisplayName\": \"Microsoft.Fidalgo projects environment\" }\r\n ,\"microsoft.fidalgo/projects/pools\": { \"SingularDisplayName\": \"Microsoft.Fidalgo projects pool\" }\r\n ,\"microsoft.fileshares/fileshares\": { \"SingularDisplayName\": \"File share\" }\r\n ,\"microsoft.fluidrelay/fluidrelayservers\": { \"SingularDisplayName\": \"Fluid Relay\" }\r\n ,\"microsoft.footprintmonitoring/profiles\": { \"SingularDisplayName\": \"Microsoft.FootprintMonitoring profile\" }\r\n ,\"microsoft.footprintmonitoring/profiles/experiments\": { \"SingularDisplayName\": \"Microsoft.FootprintMonitoring profiles experiment\" }\r\n ,\"microsoft.footprintmonitoring/profiles/measurementendpoints\": { \"SingularDisplayName\": \"Microsoft.FootprintMonitoring profiles measurement endpoint\" }\r\n ,\"microsoft.footprintmonitoring/profiles/measurementendpoints/conditions\": { \"SingularDisplayName\": \"Microsoft.FootprintMonitoring profiles measurement endpoints condition\" }\r\n ,\"microsoft.gallery/myareas/galleryitems\": { \"SingularDisplayName\": \"Template\" }\r\n ,\"microsoft.genomics/accounts\": { \"SingularDisplayName\": \"Genomics account\" }\r\n ,\"microsoft.graph/azureadapplication\": { \"SingularDisplayName\": \"Entra application\" }\r\n ,\"microsoft.graph/azureadapplicationprototype\": { \"SingularDisplayName\": \"Microsoft.Graph Azure ad application prototype\" }\r\n ,\"microsoft.graphservices/accounts\": { \"SingularDisplayName\": \"Metered API account\" }\r\n ,\"microsoft.guestconfiguration/guestconfigurationassignments\": { \"SingularDisplayName\": \"Microsoft.GuestConfiguration guest configuration assignment\" }\r\n ,\"microsoft.guestconfiguration/guestconfigurationassignments/reports\": { \"SingularDisplayName\": \"Microsoft.GuestConfiguration guest configuration assignments report\" }\r\n ,\"microsoft.hanaonazure/hanainstances\": { \"SingularDisplayName\": \"SAP HANA on Azure\" }\r\n ,\"microsoft.hanaonazure/sapmonitors\": { \"SingularDisplayName\": \"Azure Monitor for SAP Solutions (classic)\" }\r\n ,\"microsoft.hardware/orders\": { \"SingularDisplayName\": \"Microsoft.Hardware order\" }\r\n ,\"microsoft.hardwaresecuritymodules/cloudhsmclusters\": { \"SingularDisplayName\": \"Azure Cloud HSM\" }\r\n ,\"microsoft.hdinsight/clusterpools\": { \"SingularDisplayName\": \"Azure HDInsight on AKS cluster pool\" }\r\n ,\"microsoft.hdinsight/clusterpools/clusters\": { \"SingularDisplayName\": \"Azure HDInsight on AKS cluster\" }\r\n ,\"microsoft.hdinsight/clusterpools/clusters/instanceviews\": { \"SingularDisplayName\": \"Microsoft.HDInsight clusterpools clusters instance view\" }\r\n ,\"microsoft.hdinsight/clusters\": { \"SingularDisplayName\": \"HDInsight cluster\" }\r\n ,\"microsoft.healthbot/healthbots\": { \"SingularDisplayName\": \"Healthcare agent service\" }\r\n ,\"microsoft.healthcareapis/services\": { \"SingularDisplayName\": \"Azure API for FHIR\" }\r\n ,\"microsoft.healthcareapis/workspaces\": { \"SingularDisplayName\": \"Health Data Services workspace\" }\r\n ,\"microsoft.healthcareapis/workspaces/dicomservices\": { \"SingularDisplayName\": \"DICOM service\" }\r\n ,\"microsoft.healthcareapis/workspaces/fhirservices\": { \"SingularDisplayName\": \"FHIR service\" }\r\n ,\"microsoft.healthcareapis/workspaces/iotconnectors\": { \"SingularDisplayName\": \"MedTech service\" }\r\n ,\"microsoft.healthdataaiservices/deidservices\": { \"SingularDisplayName\": \"De-identification Service\" }\r\n ,\"microsoft.healthmodel/healthmodels\": { \"SingularDisplayName\": \"Health Model\" }\r\n ,\"microsoft.healthplatform/accounts\": { \"SingularDisplayName\": \"Microsoft.HealthPlatform account\" }\r\n ,\"microsoft.help/diagnostics\": { \"SingularDisplayName\": \"Microsoft.Help diagnostic\" }\r\n ,\"microsoft.help/selfhelp\": { \"SingularDisplayName\": \"Microsoft.Help self help\" }\r\n ,\"microsoft.help/simplifiedsolutions\": { \"SingularDisplayName\": \"Microsoft.Help simplified solution\" }\r\n ,\"microsoft.help/solutions\": { \"SingularDisplayName\": \"Microsoft.Help solution\" }\r\n ,\"microsoft.help/troubleshooters\": { \"SingularDisplayName\": \"Microsoft.Help troubleshooter\" }\r\n ,\"microsoft.hpcworkbench/instances\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instance\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chamber\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers/accessprofiles\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chambers access profile\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers/filerequests\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chambers file request\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers/files\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chambers file\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers/storages\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chambers storage\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers/workloads\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chambers workload\" }\r\n ,\"microsoft.hpcworkbench/instances/consortiums\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances consortium\" }\r\n ,\"microsoft.hybridcloud/cloudconnections\": { \"SingularDisplayName\": \"Microsoft.HybridCloud cloud connection\" }\r\n ,\"microsoft.hybridcloud/cloudconnectors\": { \"SingularDisplayName\": \"Microsoft.HybridCloud cloud connector\" }\r\n ,\"microsoft.hybridcompute/arcgatewayassociatedresources\": { \"SingularDisplayName\": \"Arc gateway associated resource\" }\r\n ,\"microsoft.hybridcompute/arcserverwithwac\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/gateways\": { \"SingularDisplayName\": \"Arc gateway\" }\r\n ,\"microsoft.hybridcompute/licenses\": { \"SingularDisplayName\": \"Extended Security Updates - Windows Server 2012/R2\" }\r\n ,\"microsoft.hybridcompute/machines\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/machines/microsoft.awsconnector/ec2instances\": { \"SingularDisplayName\": \"Microsoft.AwsConnector ec2 instance\" }\r\n ,\"microsoft.hybridcompute/machines/microsoft.connectedvmwarevsphere/virtualmachineinstances\": { \"SingularDisplayName\": \"VMware + AVS virtual machine\" }\r\n ,\"microsoft.hybridcompute/machines/providers/guestconfigurationassignments\": { \"SingularDisplayName\": \"Guest Assignment\" }\r\n ,\"microsoft.hybridcompute/machinesesu\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/machinespaygo\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/machinessoftwareassurance\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/machinessovereign\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/privatelinkscopes\": { \"SingularDisplayName\": \"Azure Arc Private Link Scope\" }\r\n ,\"microsoft.hybridcompute/settings\": { \"SingularDisplayName\": \"Microsoft.HybridCompute setting\" }\r\n ,\"microsoft.hybridconnectivity/endpoints\": { \"SingularDisplayName\": \"Microsoft.HybridConnectivity endpoint\" }\r\n ,\"microsoft.hybridconnectivity/endpoints/serviceconfigurations\": { \"SingularDisplayName\": \"Microsoft.HybridConnectivity endpoints service configuration\" }\r\n ,\"microsoft.hybridconnectivity/publiccloudconnectors\": { \"SingularDisplayName\": \"Multicloud connector\" }\r\n ,\"microsoft.hybridconnectivity/solutionconfigurations\": { \"SingularDisplayName\": \"Microsoft.HybridConnectivity solution configuration\" }\r\n ,\"microsoft.hybridconnectivity/solutionconfigurations/inventory\": { \"SingularDisplayName\": \"Microsoft.HybridConnectivity solution configurations inventory\" }\r\n ,\"microsoft.hybridconnectivity/solutiontypes\": { \"SingularDisplayName\": \"Microsoft.HybridConnectivity solution type\" }\r\n ,\"microsoft.hybridcontainerservice/kubernetesversions\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService kubernetes version\" }\r\n ,\"microsoft.hybridcontainerservice/provisionedclusterinstances\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService provisioned cluster instance\" }\r\n ,\"microsoft.hybridcontainerservice/provisionedclusterinstances/agentpools\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService provisioned cluster instances agent pool\" }\r\n ,\"microsoft.hybridcontainerservice/provisionedclusterinstances/hybrididentitymetadata\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService provisioned cluster instances hybrid identity metadata\" }\r\n ,\"microsoft.hybridcontainerservice/provisionedclusterinstances/upgradeprofiles\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService provisioned cluster instances upgrade profile\" }\r\n ,\"microsoft.hybridcontainerservice/provisionedclusters\": { \"SingularDisplayName\": \"Kubernetes hybrid - Azure Arc\" }\r\n ,\"microsoft.hybridcontainerservice/skus\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService SKU\" }\r\n ,\"microsoft.hybridcontainerservice/storagespaces\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService storage space\" }\r\n ,\"microsoft.hybridcontainerservice/virtualnetworks\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService virtual network\" }\r\n ,\"microsoft.hybriddata/datamanagers\": { \"SingularDisplayName\": \"Microsoft.HybridData data manager\" }\r\n ,\"microsoft.hybriddata/datamanagers/dataservices\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers data service\" }\r\n ,\"microsoft.hybriddata/datamanagers/dataservices/jobdefinitions\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers data services job definition\" }\r\n ,\"microsoft.hybriddata/datamanagers/dataservices/jobdefinitions/jobs\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers data services job definitions job\" }\r\n ,\"microsoft.hybriddata/datamanagers/datastores\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers data store\" }\r\n ,\"microsoft.hybriddata/datamanagers/datastoretypes\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers data store type\" }\r\n ,\"microsoft.hybriddata/datamanagers/publickeys\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers public key\" }\r\n ,\"microsoft.hybridnetwork/configurationgroupvalues\": { \"SingularDisplayName\": \"Configuration Group Value\" }\r\n ,\"microsoft.hybridnetwork/devices\": { \"SingularDisplayName\": \"Azure Network Function Manager ? Device\" }\r\n ,\"microsoft.hybridnetwork/networkfunctions\": { \"SingularDisplayName\": \"Azure Network Function Manager ? Network Function\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publisher\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/artifactstores\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers artifact store\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/configurationgroupschemas\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers configuration group schema\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/networkfunctiondefinitiongroups\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers network function definition group\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/networkfunctiondefinitiongroups/networkfunctiondefinitionversions\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers network function definition groups network function definition version\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/networkservicedesigngroups\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers network service design group\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/networkservicedesigngroups/networkservicedesignversions\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers network service design groups network service design version\" }\r\n ,\"microsoft.hybridnetwork/publishers\": { \"SingularDisplayName\": \"Publisher\" }\r\n ,\"microsoft.hybridnetwork/publishers/artifactstores\": { \"SingularDisplayName\": \"Publisher Artifact Store\" }\r\n ,\"microsoft.hybridnetwork/publishers/artifactstores/artifactmanifests\": { \"SingularDisplayName\": \"Publisher Artifact Manifest\" }\r\n ,\"microsoft.hybridnetwork/publishers/configurationgroupschemas\": { \"SingularDisplayName\": \"Configuration Group Schema\" }\r\n ,\"microsoft.hybridnetwork/publishers/networkfunctiondefinitiongroups\": { \"SingularDisplayName\": \"Network Function Definition\" }\r\n ,\"microsoft.hybridnetwork/publishers/networkfunctiondefinitiongroups/networkfunctiondefinitionversions\": { \"SingularDisplayName\": \"Network Function Definition Version\" }\r\n ,\"microsoft.hybridnetwork/publishers/networkservicedesigngroups\": { \"SingularDisplayName\": \"Network Service Design\" }\r\n ,\"microsoft.hybridnetwork/publishers/networkservicedesigngroups/networkservicedesignversions\": { \"SingularDisplayName\": \"Network Service Design Version\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management container\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers/rolloutsequences\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management containers rollout sequence\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers/rollouttiers\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management containers rollout tier\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers/updatespecifications\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management containers update specification\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers/updatespecifications/rollouts\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management containers update specifications rollout\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers/updatespecifications/rollouts/statuses\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management containers update specifications rollouts statuse\" }\r\n ,\"microsoft.hybridnetwork/sitenetworkservices\": { \"SingularDisplayName\": \"Site Network Service\" }\r\n ,\"microsoft.hybridnetwork/sites\": { \"SingularDisplayName\": \"Site\" }\r\n })[tolower(id)]\r\n}\r\n", - "$fxv#10": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Ingestion database\r\n// See known issues @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n//===| Prices |=========================================================================================================\r\n// Supported versions:\r\n// - MS EA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-ea\r\n// - MS MCA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-mca\r\n//======================================================================================================================\r\n\r\n// Prices_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='Transforms Prices_raw into FOCUS 1.2.', folder='Prices')\r\nPrices_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', int(null),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n let prices = materialize(\r\n Prices_raw\r\n | extend PricingCurrency = coalesce(Currency, CurrencyCode) // CurrencyCode last as a fallback only\r\n | extend x_SkuId = coalesce(SkuId, SkuID)\r\n | extend x_SkuMeterId = coalesce(MeterId, MeterID)\r\n | extend x_SkuProductId = coalesce(ProductId, ProductID)\r\n | extend x_SkuTerm = isoMonths(Term)\r\n | project-rename\r\n SkuMeter = MeterName,\r\n x_BaseUnitPrice = BasePrice,\r\n x_EffectivePeriodEnd = EffectiveEndDate,\r\n x_EffectivePeriodStart = EffectiveStartDate,\r\n x_PricingUnitDescription = UnitOfMeasure,\r\n x_SkuIncludedQuantity = IncludedQuantity,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuMeterType = MeterType,\r\n x_SkuOfferId = OfferID,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPriceType = PriceType,\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTier = TierMinimumUnits\r\n | extend ContractedUnitPrice = iff(x_SkuPriceType != 'SavingsPlan', UnitPrice, real(null)) // UnitPrice for savings plan is not the on-demand unit price\r\n | extend ListUnitPrice = iff(x_SkuPriceType != 'SavingsPlan', MarketPrice, real(null)) // MarketPrice for savings plan is not the list price\r\n | extend ChargeCategory = case(\r\n x_SkuPriceType == 'Consumption', 'Usage',\r\n x_SkuPriceType == 'ReservedInstance', 'Purchase',\r\n x_SkuPriceType == 'SavingsPlan', 'Usage', // Savings plan prices are for committed usage, not the purchase\r\n ''\r\n )\r\n | extend SkuPriceIdv2 = strcat(case(x_SkuPriceType == 'Consumption', 'OD', x_SkuPriceType == 'ReservedInstance', 'RI', x_SkuPriceType == 'SavingsPlan', 'SP', 'XX'), substring(ChargeCategory, 0, 1), x_SkuTerm, '_', x_SkuProductId, '_', x_SkuId, '_', x_SkuMeterType, '_', x_SkuTier, x_SkuOfferId)\r\n | extend x_BillingAccountId = iff(BillingAccountId startswith '/', split(BillingAccountId, '/')[-1], coalesce(BillingAccountId, EnrollmentNumber))\r\n | extend x_BillingProfileId = iff(BillingProfileId startswith '/', split(BillingProfileId, '/')[-1], coalesce(BillingProfileId, EnrollmentNumber))\r\n | extend tmp_SavingsPlanKey = strcat(x_SkuMeterId, x_SkuProductId, x_SkuId, x_SkuTier, x_SkuOfferId)\r\n //\r\n // Get latest ingested row based on the unique ID\r\n | extend x_IngestionTime = ingestion_time()\r\n );\r\n //\r\n // Meters for reservations and savings plans to identify commitment eligibility\r\n let riMeters = prices | where x_SkuPriceType == 'ReservedInstance' | distinct x_SkuMeterId;\r\n let spMeters = prices | where x_SkuPriceType == 'SavingsPlan' | distinct x_SkuMeterId;\r\n //\r\n // Copy list/base/contracted prices from on-demand SKUs\r\n prices\r\n | where x_SkuPriceType == 'SavingsPlan'\r\n // If we use join, specify the shuffle key\r\n // TODO: Compare join vs. lookup perf -- | join kind=leftouter hint.strategy=shuffle (prices | where x_SkuPriceType == 'Consumption' | where x_SkuMeterId in (spMeters) | distinct tmp_SavingsPlanKey, ListUnitPrice, ContractedUnitPrice, x_BaseUnitPrice) on tmp_SavingsPlanKey\r\n | lookup kind=leftouter (prices | where x_SkuPriceType == 'Consumption' | where x_SkuMeterId in (spMeters) | distinct tmp_SavingsPlanKey, ListUnitPrice, ContractedUnitPrice, x_BaseUnitPrice) on tmp_SavingsPlanKey\r\n | extend ListUnitPrice = coalesce(ListUnitPrice, ListUnitPrice1)\r\n | extend ContractedUnitPrice = coalesce(ContractedUnitPrice, ContractedUnitPrice1)\r\n | extend x_BaseUnitPrice = coalesce(x_BaseUnitPrice, x_BaseUnitPrice1)\r\n | project-away ListUnitPrice1, ContractedUnitPrice1, x_BaseUnitPrice1, tmp_SavingsPlanKey\r\n | union ((prices | where x_SkuPriceType != 'SavingsPlan'))\r\n //\r\n // Set CommitmentDiscountCategory for reuse\r\n | extend CommitmentDiscountCategory = case(\r\n x_SkuPriceType == 'ReservedInstance', 'Usage',\r\n x_SkuPriceType == 'SavingsPlan', 'Spend',\r\n ''\r\n )\r\n //\r\n // Calculate commitment discount eligibility\r\n // TODO: Would a join be faster?\r\n // TODO: Check this to ensure it's correct\r\n | extend x_CommitmentDiscountSpendEligibility = iff(x_SkuMeterId in (riMeters) and x_SkuPriceType != 'ReservedInstance', 'Eligible', 'Not Eligible')\r\n | extend x_CommitmentDiscountUsageEligibility = iff(x_SkuMeterId in (spMeters), 'Eligible', 'Not Eligible')\r\n //\r\n // TODO: Implement x_CommitmentDiscountNormalizedRatio\r\n | extend x_CommitmentDiscountNormalizedRatio = real(null)\r\n //\r\n // Add PricingUnit and x_PricingBlockSize\r\n // TODO: Compare join vs. lookup perf -- | join kind=leftouter (PricingUnits) on x_PricingUnitDescription | project-away x_PricingUnitDescription1\r\n | lookup kind=leftouter (PricingUnits) on x_PricingUnitDescription\r\n //\r\n | extend x_EffectiveUnitPrice = iff(x_SkuPriceType == 'SavingsPlan', UnitPrice, real(null)) // Savings plan prices are for the effective price, not the contracted price\r\n | extend x_EffectiveUnitPriceDiscount = ContractedUnitPrice - x_EffectiveUnitPrice\r\n | extend x_ContractedUnitPriceDiscount = ListUnitPrice - ContractedUnitPrice\r\n | extend x_TotalUnitPriceDiscount = ListUnitPrice - x_EffectiveUnitPrice\r\n | project\r\n BillingAccountId = tolower(case(\r\n BillingProfileId startswith '/', BillingProfileId,\r\n BillingAccountId startswith '/', BillingAccountId,\r\n strcat('/providers/microsoft.billing/billingaccounts/', x_BillingAccountId, iff(x_BillingProfileId == x_BillingAccountId, '', strcat('/billingprofiles/', x_BillingProfileId)))\r\n )),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName, x_BillingProfileId),\r\n BillingCurrency = coalesce(BillingCurrency, CurrencyCode, Currency), // Currency last as a fallback only\r\n ChargeCategory,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountType = case(\r\n x_SkuPriceType == 'ReservedInstance', 'Reservation',\r\n x_SkuPriceType == 'SavingsPlan', 'Savings plan',\r\n ''\r\n ),\r\n CommitmentDiscountUnit = case(\r\n isempty(CommitmentDiscountCategory), '',\r\n CommitmentDiscountCategory == 'Spend', PricingCurrency,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio == real(1), PricingUnit,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio > real(1), strcat('Normalized ', PricingUnit),\r\n ''\r\n ),\r\n ContractedUnitPrice,\r\n ListUnitPrice,\r\n PricingCategory = case(\r\n x_SkuPriceType == 'Consumption', 'Standard',\r\n x_SkuPriceType == 'ReservedInstance', 'Standard', // Reservation purchases are tracked as \"Standard\"\r\n x_SkuPriceType == 'SavingsPlan', 'Committed',\r\n ''\r\n ),\r\n PricingCurrency,\r\n PricingUnit,\r\n SkuId = coalesce(ProductId, ProductID),\r\n SkuMeter,\r\n SkuPriceId = strcat(x_SkuProductId, '_', x_SkuId, '_', x_SkuMeterType),\r\n SkuPriceIdv2,\r\n x_BaseUnitPrice,\r\n x_BillingAccountAgreement = case(\r\n strlen(x_BillingAccountId) > 32, 'MCA',\r\n strlen(x_BillingAccountId) < 32, 'EA',\r\n 'Unknown'\r\n ),\r\n x_BillingAccountId,\r\n x_BillingProfileId,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_CommitmentDiscountSpendEligibility,\r\n x_CommitmentDiscountUsageEligibility,\r\n x_ContractedUnitPriceDiscount,\r\n x_ContractedUnitPriceDiscountPercent = 1.0 * x_ContractedUnitPriceDiscount / ListUnitPrice * 100,\r\n x_EffectivePeriodEnd = startofmonth(x_EffectivePeriodEnd + 1h),\r\n x_EffectivePeriodStart,\r\n x_EffectiveUnitPrice,\r\n x_EffectiveUnitPriceDiscount,\r\n x_EffectiveUnitPriceDiscountPercent = 1.0 * x_EffectiveUnitPriceDiscount / ContractedUnitPrice * 100,\r\n x_IngestionTime,\r\n x_PricingBlockSize,\r\n x_PricingSubcategory = case(\r\n x_SkuPriceType == 'Consumption' and (x_SkuIncludedQuantity > 0 or x_SkuTier > 0), 'Tiered',\r\n x_SkuPriceType == 'Consumption', 'Standard',\r\n x_SkuPriceType == 'ReservedInstance', 'Standard', // Reservation purchases are tracked as \"Standard\"\r\n x_SkuPriceType == 'SavingsPlan', 'Committed Spend',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_SkuDescription = Product,\r\n x_SkuId,\r\n x_SkuIncludedQuantity,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterSubcategory,\r\n x_SkuMeterType,\r\n x_SkuPriceType,\r\n x_SkuProductId,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuOfferId,\r\n x_SkuPartNumber,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceName = coalesce(x_SourceName, 'Cost Management'),\r\n x_SourceProvider = coalesce(x_SourceProvider, 'Microsoft'),\r\n x_SourceType = coalesce(x_SourceType, 'PriceSheet'),\r\n x_SourceVersion = coalesce(x_SourceVersion, '2023-05-01'),\r\n x_TotalUnitPriceDiscount,\r\n x_TotalUnitPriceDiscountPercent = 1.0 * x_TotalUnitPriceDiscount / ListUnitPrice * 100\r\n}\r\n\r\n// Prices_final_v1_2 table\r\n.create-merge table Prices_final_v1_2 (\r\n BillingAccountId: string,\r\n BillingAccountName: string,\r\n BillingCurrency: string,\r\n ChargeCategory: string,\r\n CommitmentDiscountCategory: string,\r\n CommitmentDiscountType: string,\r\n CommitmentDiscountUnit: string,\r\n ContractedUnitPrice: real,\r\n ListUnitPrice: real,\r\n PricingCategory: string,\r\n PricingCurrency: string, // Azure\r\n PricingUnit: string,\r\n SkuId: string,\r\n SkuMeter: string, // Azure\r\n SkuPriceId: string,\r\n SkuPriceIdv2: string, // Hubs add-on\r\n x_BaseUnitPrice: real, // Azure\r\n x_BillingAccountAgreement: string, // Hubs add-on\r\n x_BillingAccountId: string, // Azure MCA\r\n x_BillingProfileId: string, // Azure MCA\r\n x_CommitmentDiscountNormalizedRatio: real, // Hubs add-on\r\n x_CommitmentDiscountSpendEligibility: string, // Hubs add-on\r\n x_CommitmentDiscountUsageEligibility: string, // Hubs add-on\r\n x_ContractedUnitPriceDiscount: real, // Hubs add-on\r\n x_ContractedUnitPriceDiscountPercent: real, // Hubs add-on\r\n x_EffectivePeriodEnd: datetime, // Azure\r\n x_EffectivePeriodStart: datetime, // Azure\r\n x_EffectiveUnitPrice: real, // Azure\r\n x_EffectiveUnitPriceDiscount: real, // Hubs add-on\r\n x_EffectiveUnitPriceDiscountPercent: real, // Hubs add-on\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_PricingBlockSize: real, // Hubs add-on\r\n x_PricingSubcategory: string, // Hubs add-on\r\n x_PricingUnitDescription: string, // Azure\r\n x_SkuDescription: string, // Azure\r\n x_SkuId: string, // Azure\r\n x_SkuIncludedQuantity: real, // Azure EA\r\n x_SkuMeterCategory: string, // Azure\r\n x_SkuMeterId: string, // Azure\r\n x_SkuMeterSubcategory: string, // Azure\r\n x_SkuMeterType: string, // Azure\r\n x_SkuPriceType: string, // Azure\r\n x_SkuProductId: string, // Azure\r\n x_SkuRegion: string, // Azure\r\n x_SkuServiceFamily: string, // Azure\r\n x_SkuOfferId: string, // Azure EA\r\n x_SkuPartNumber: string, // Azure EA\r\n x_SkuTerm: int, // Azure\r\n x_SkuTier: real, // Azure MCA\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_TotalUnitPriceDiscount: real, // Hubs add-on\r\n x_TotalUnitPriceDiscountPercent: real // Hubs add-on\r\n)\r\n\r\n// Update policy for Prices_raw -> Prices_final_v1_2\r\n.alter table Prices_final_v1_2 policy update\r\n```\r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"Prices_raw\",\r\n \"Query\": \"Prices_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Cost and usage |=================================================================================================\r\n// Supported versions:\r\n// - MS: 1.2-preview, 1.0, 1.0-preview(v1)\r\n// https://aka.ms/costmgmt/exports/focus\r\n// - AWS: 1.0\r\n// https://docs.aws.amazon.com/cur/latest/userguide/table-dictionary-focus-1-0-aws-columns.html\r\n// - GCP: Jan-Jun 2024\r\n// https://cloud.google.com/resources/google-cloud-focus?e=48754805&hl=en\r\n// Links to (Aug 2024): https://services.google.com/fh/files/misc/focus_guide_v1.pdf\r\n// See also:\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/standard-usage\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/detailed-usage\r\n// - OCI: 1.0 \r\n// https://docs.oracle.com/iaas/Content/Billing/Concepts/costusagereportsoverview.htm#costreports__focus-cost-report-schema\r\n//\r\n// Support for non-Azure data is limited to ingestion only. Data is not transformed across versions.\r\n//======================================================================================================================\r\n\r\n// Costs_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='All costs transformed to FOCUS 1.2.', folder='Costs')\r\nCosts_transform_v1_2()\r\n{\r\n let checkString = (column: string, oldValue: string, newValue: string) {\r\n iff(oldValue == newValue, dynamic({}), bag_pack(column, oldValue))\r\n };\r\n let checkInt = (column: string, oldValue: int, newValue: int) {\r\n iff(oldValue == newValue, dynamic({}), bag_pack(column, oldValue))\r\n };\r\n let checkReal = (column: string, oldValue: real, newValue: real) {\r\n iff(oldValue == newValue, dynamic({}), bag_pack(column, oldValue))\r\n };\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n Costs_raw\r\n //\r\n // Dedupe rows\r\n | extend x_IngestionTime = ingestion_time()\r\n | extend x_ChargeId = ''\r\n // TODO: Consider adding a unique charge ID per row\r\n // hash_sha256(strcat(\r\n // // DO NOT CHANGE COLUMNS OR COLUMN ORDER\r\n // // 1. Resource hierarchy (including resource name), highest to lowest\r\n // BillingAccountId,\r\n // x_InvoiceSectionId,\r\n // x_AccountOwnerId,\r\n // SubAccountId,\r\n // x_ResourceGroupName,\r\n // ResourceName,\r\n // // 2. Resource details\r\n // ResourceId,\r\n // RegionId,\r\n // Tags,\r\n // CommitmentDiscountId,\r\n // x_CostCenter,\r\n // // 4. Meter details\r\n // SkuPriceId,\r\n // x_SkuMeterId,\r\n // x_SkuPartNumber,\r\n // x_SkuOfferId,\r\n // x_SkuDetails,\r\n // // 5. Date\r\n // ChargePeriodStart\r\n // ))\r\n //\r\n // Identify data quality issues\r\n // TODO: Remove x_SourceChanges in v1_3 (or later)\r\n | extend x_SourceChanges = trim_end(',', strcat(\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and ChargeFrequency == 'Usage-Based', 'InvalidChargeFrequency,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and EffectiveCost > 0, 'InvalidEffectiveCost,', ''),\r\n iff((isempty(ContractedCost) or ContractedCost == 0) and EffectiveCost != 0, 'MissingContractedCost,', ''),\r\n iff((isempty(ContractedUnitPrice) or ContractedUnitPrice == 0) and x_EffectiveUnitPrice != 0, 'MissingContractedUnitPrice,', ''),\r\n iff(ListCost < ContractedCost, 'ListCostLessThanContractedCost,', ''),\r\n iff(ContractedCost < EffectiveCost, 'ContractedCostLessThanEffectiveCost,', ''),\r\n iff((isempty(ListCost) or ListCost == 0) and (ContractedCost != 0 or EffectiveCost != 0), 'MissingListCost,', ''),\r\n iff((isempty(ListUnitPrice) or ListUnitPrice == 0) and (ContractedUnitPrice != 0 or x_EffectiveUnitPrice != 0), 'MissingListUnitPrice,', ''),\r\n iff((isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(x_BilledUnitPrice - x_EffectiveUnitPrice) < 0.0001)\r\n or (isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(ContractedUnitPrice - x_EffectiveUnitPrice) < 0.0001),\r\n 'XEffectiveUnitPriceRoundingError,', ''),\r\n iff(ConsumedQuantity == 0 and (EffectiveCost != 0 or BilledCost != 0), 'MissingConsumedQuantity,', ''),\r\n iff(PricingQuantity == 0 and (EffectiveCost != 0 or BilledCost != 0), 'MissingPricingQuantity,', ''),\r\n iff(isempty(ProviderName), 'MissingProviderName,', ''),\r\n iff(isempty(PublisherName), 'MissingPublisherName,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceId), 'MissingResourceId,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceName), 'MissingResourceName,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceType), 'MissingResourceType,', ''),\r\n iff(BilledCost > 0 and x_BilledUnitPrice == 0, 'MissingXBilledUnitPrice,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(x_ResourceType), 'MissingXResourceType,', ''),\r\n iff(PricingCategory == 'Standard' and isnotempty(CommitmentDiscountId) and ChargeCategory == 'Usage', 'PricingCategoryShouldBeCommitted,', ''),\r\n iff(x_SkuTerm == '1Year' or x_SkuTerm == '3Years' or x_SkuTerm == '5Years', 'SkuTermShouldBeAnInteger,', '')\r\n ))\r\n //\r\n // Handle provider columns that moved to FOCUS\r\n | extend PricingCurrency = coalesce(PricingCurrency, x_PricingCurrency)\r\n //\r\n // Backup original prices/costs before the merge\r\n | extend old_ContractedCost = ContractedCost\r\n | extend old_ContractedUnitPrice = ContractedUnitPrice\r\n | extend old_ListCost = ListCost\r\n | extend old_ListUnitPrice = ListUnitPrice\r\n | extend old_x_EffectiveUnitPrice = x_EffectiveUnitPrice\r\n //\r\n // Fix columns needed in other changes\r\n | extend old_ProviderName = ProviderName, ProviderName = case(\r\n isnotempty(ProviderName), ProviderName,\r\n isnotempty(coalesce(x_CostCategories, x_Discount, x_Operation, x_ServiceCode, x_UsageType)), 'AWS',\r\n isnotempty(coalesce(tostring(UsageAmount), tostring(x_Cost), x_Credits, x_CostType, tostring(x_CurrencyConversionRate), tostring(x_ExportTime), x_Project, x_ServiceId)), 'GCP',\r\n isnotempty(coalesce(x_BillingProfileId, x_InvoiceSectionId)), 'Microsoft',\r\n ''\r\n )\r\n //\r\n // Identify source\r\n | extend x_SourceName = coalesce(x_SourceName, iff(isnotempty(x_BillingProfileId), 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(isnotempty(x_BillingProfileId), 'FocusCost', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, case(\r\n isnotempty(coalesce(InvoiceId, SkuMeter, PricingCurrency)) and isempty(SkuPriceDetails) and isnotempty(x_SkuDetails), '1.2-preview',\r\n isnotempty(coalesce(InvoiceId, SkuMeter, PricingCurrency)), '1.2',\r\n isnotempty(coalesce(ChargeClass, CommitmentDiscountStatus, tostring(ConsumedQuantity), ConsumedUnit, tostring(ContractedCost), tostring(ContractedUnitPrice), RegionId, RegionName)), '1.0',\r\n isnotempty(coalesce(ChargeSubcategory, Region, tostring(UsageQuantity), UsageUnit)), iff(ProviderName == 'Microsoft', '1.0-preview(v1)', '1.0-preview'),\r\n ''\r\n ))\r\n // Append version check error code\r\n | extend x_SourceChanges = iff(x_SourceVersion == '1.0', x_SourceChanges,\r\n strcat(x_SourceChanges, iff(isempty(x_SourceChanges), '', ','), iff(x_SourceVersion == '', 'UnknownFocusVersion', 'LegacyFocusVersion'))\r\n )\r\n //\r\n // Fix quantities\r\n | extend old_PricingQuantity = PricingQuantity, PricingQuantity = case(\r\n PricingQuantity != 0 or (EffectiveCost == 0 and BilledCost == 0), PricingQuantity,\r\n PricingQuantity == 0 and isnotempty(BilledCost) and BilledCost != 0 and isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, BilledCost / x_BilledUnitPrice,\r\n PricingQuantity == 0 and isnotempty(BilledCost) and BilledCost != 0 and isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, BilledCost / x_BilledUnitPrice,\r\n PricingQuantity == 0 and isnotempty(EffectiveCost) and EffectiveCost != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0, EffectiveCost / x_EffectiveUnitPrice,\r\n PricingQuantity\r\n )\r\n | extend old_ConsumedQuantity = ConsumedQuantity, ConsumedQuantity = case(\r\n isnotempty(ConsumedQuantity) and ConsumedQuantity != 0, ConsumedQuantity,\r\n ChargeCategory == 'Usage', PricingQuantity / coalesce(x_PricingBlockSize, real(1)),\r\n ConsumedQuantity\r\n )\r\n //\r\n // Populate missing prices -- mapping to on-demand prices requires meter ID and offer ID\r\n | extend tmp_MissingPrices = ProviderName == 'Microsoft'\r\n and (isempty(ListUnitPrice) or isempty(ContractedUnitPrice) or ListUnitPrice == 0 or ContractedUnitPrice == 0)\r\n and x_EffectiveUnitPrice != 0\r\n and not(CommitmentDiscountCategory == 'Spend' and CommitmentDiscountStatus == 'Unused')\r\n and isnotempty(strcat(x_SkuMeterId, x_SkuOfferId))\r\n | as allCosts\r\n | where tmp_MissingPrices\r\n | extend tmp_ReservationPriceLookupKey = tolower(strcat(x_BillingProfileId, substring(ChargePeriodStart, 0, 7), x_SkuMeterId, x_SkuOfferId))\r\n | as costsWithMissingPrices\r\n | join kind=leftouter (\r\n Prices_final_v1_2\r\n | extend tmp_ReservationPriceLookupKey = tolower(strcat(x_BillingProfileId, substring(x_EffectivePeriodStart, 0, 7), x_SkuMeterId, x_SkuOfferId))\r\n | where x_SkuPriceType == 'Consumption' and tmp_ReservationPriceLookupKey in ((costsWithMissingPrices | summarize by tmp_ReservationPriceLookupKey))\r\n | summarize ListUnitPrice = min(ListUnitPrice), ContractedUnitPrice = min(ContractedUnitPrice) by tmp_ReservationPriceLookupKey, x_PricingBlockSize, PricingUnit\r\n ) on tmp_ReservationPriceLookupKey\r\n //\r\n // Select the best price to use for each row\r\n | extend x_EffectiveUnitPrice = case(\r\n // If price is a rounding error away from the billed price, use the billed price\r\n isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(x_BilledUnitPrice - x_EffectiveUnitPrice) < 0.0001, x_BilledUnitPrice,\r\n // If price is a rounding error away from the contracted price, use the contracted price\r\n isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(ContractedUnitPrice - x_EffectiveUnitPrice) < 0.0001, ContractedUnitPrice,\r\n x_EffectiveUnitPrice\r\n )\r\n | extend ContractedUnitPrice = case(\r\n // If price is already correct, keep that\r\n (isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0) or (EffectiveCost == 0 and BilledCost == 0), ContractedUnitPrice,\r\n // If both prices use the same scale, use the new one\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize == x_PricingBlockSize1, ContractedUnitPrice1 * x_BillingExchangeRate,\r\n // If prices are the same unit but not the same scale, use the new one but correct the scale\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize != x_PricingBlockSize1 and isnotempty(x_PricingBlockSize) and isnotempty(x_PricingBlockSize1), ContractedUnitPrice1 * x_BillingExchangeRate / x_PricingBlockSize1 * x_PricingBlockSize,\r\n // If billed price is available, assume the billed price is the same as contracted price to support aggregations\r\n isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, x_EffectiveUnitPrice,\r\n // Otherwise, assume the effective price is the same as contracted price to support aggregations\r\n x_EffectiveUnitPrice\r\n )\r\n | extend ListUnitPrice = case(\r\n // If price is already correct, keep that\r\n (isnotempty(ListUnitPrice) and ListUnitPrice != 0) or (EffectiveCost == 0 and BilledCost == 0), ListUnitPrice,\r\n // If both prices use the same scale, use the new one\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize == x_PricingBlockSize1, ListUnitPrice1 * x_BillingExchangeRate,\r\n // If prices are the same unit but not the same scale, use the new one but correct the scale\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize != x_PricingBlockSize1 and isnotempty(x_PricingBlockSize) and isnotempty(x_PricingBlockSize1), ListUnitPrice1 * x_BillingExchangeRate / x_PricingBlockSize1 * x_PricingBlockSize,\r\n // Otherwise, assume the contracted price is the same as list price to support aggregations\r\n ContractedUnitPrice\r\n )\r\n // Calculate missing costs based on new prices -- If cost is already correct, keep that; if not and price is available, recalculate the cost; otherwise, keep the existing cost\r\n | extend ContractedCost = case(\r\n // If not set or there's no cost, keep the original value\r\n (isnotempty(ContractedCost) and ContractedCost != 0) or (EffectiveCost == 0 and BilledCost == 0), ContractedCost,\r\n // ContractedCost is 0 in all other scenarios...\r\n // If 0 and there's a billed cost and prices are the same, use BilledCost\r\n isnotempty(BilledCost) and BilledCost != 0 and ContractedUnitPrice == x_BilledUnitPrice, BilledCost,\r\n // If 0 and there's a billed cost and prices are the same, use EffectiveCost\r\n isnotempty(EffectiveCost) and EffectiveCost != 0 and ContractedUnitPrice == x_EffectiveUnitPrice, EffectiveCost,\r\n // If 0 and there's a price, calculate the cost based on the price\r\n isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0, ContractedUnitPrice * PricingQuantity,\r\n // If 0 and there's no price, assume EffectiveCost\r\n isempty(ContractedUnitPrice) or ContractedUnitPrice == 0, EffectiveCost,\r\n // Fall back to the original value for any unhandled scenarios\r\n ContractedCost\r\n )\r\n | extend ListCost = case(\r\n // If not set or there's no cost, keep the original value\r\n (isnotempty(ListCost) and ListCost != 0) or (EffectiveCost == 0 and BilledCost == 0), ListCost,\r\n // ListCost is 0 in all other scenarios...\r\n // If 0 and there's a contracted cost and prices are the same, use ContractedCost\r\n isnotempty(ContractedCost) and ContractedCost != 0 and ListUnitPrice == ContractedUnitPrice, ContractedCost,\r\n // If 0 and there's a price, calculate the cost based on the price\r\n isnotempty(ListUnitPrice) and ListUnitPrice != 0, ListUnitPrice * PricingQuantity,\r\n // If 0 and there's no price, assume ContractedCost\r\n isempty(ListUnitPrice) or ListUnitPrice == 0, ContractedCost,\r\n // Fall back to the original value for any unhandled scenarios\r\n ListCost\r\n )\r\n // Merge the rest of the unmodified cost records and remove excess columns\r\n | union (allCosts | where not(tmp_MissingPrices))\r\n | project-away x_PricingBlockSize1, PricingUnit1, ListUnitPrice1, ContractedUnitPrice1, tmp_MissingPrices, tmp_ReservationPriceLookupKey, tmp_ReservationPriceLookupKey1\r\n //\r\n | extend SkuPriceDetails = parse_json(SkuPriceDetails)\r\n | extend Tags = parse_json(Tags)\r\n | extend x_SkuDetails = parse_json(x_SkuDetails)\r\n //\r\n // Handle FOCUS 1.0-preview\r\n | extend old_ChargeSubcategory = ChargeSubcategory\r\n | extend old_ChargeCategory = ChargeCategory, ChargeCategory = case(\r\n // Handle FOCUS 1.0-preview ChargeSubcategory\r\n ChargeSubcategory == 'Credit', 'Credit',\r\n ChargeSubcategory == 'Refund', 'Purchase', // We are assuming purchase refunds since we don't have data to indicate usage refunds\r\n ChargeCategory\r\n )\r\n | extend old_ChargeClass = ChargeClass, ChargeClass = case(ChargeSubcategory == 'Refund', 'Correction', ChargeClass)\r\n //\r\n // Populate CapacityReservationId when not specified\r\n | extend CapacityReservationId = coalesce(CapacityReservationId, tostring(coalesce(x_SkuDetails.VMCapacityReservationId, SkuPriceDetails.VMCapacityReservationId, SkuPriceDetails.x_VMCapacityReservationId)))\r\n | extend old_CapacityReservationStatus = CapacityReservationStatus, CapacityReservationStatus = case(\r\n isempty(CapacityReservationId), '',\r\n isnotempty(CapacityReservationStatus), CapacityReservationStatus,\r\n tolower(x_ResourceType) == 'microsoft.compute/capacityreservationgroups/capacityreservations', 'Unused',\r\n 'Used'\r\n )\r\n //\r\n // BUG: ChargeFrequency shows \"Usage-Based\" for monthly recurring savings plan purchases\r\n | extend old_ChargeFrequency = ChargeFrequency, ChargeFrequency = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and ChargeFrequency == 'Usage-Based' and ProviderName == 'Microsoft' and x_SourceVersion startswith '1.0', 'Recurring', ChargeFrequency)\r\n //\r\n // Commitment discounts\r\n | extend x_CommitmentDiscountNormalizedRatio = case(\r\n // Calculate from CommitmentDiscountQuantity, if specified\r\n isnotempty(CommitmentDiscountQuantity) and CommitmentDiscountQuantity != 0, CommitmentDiscountQuantity / PricingQuantity / coalesce(x_PricingBlockSize, real(1)),\r\n // Not applicable\r\n isempty(CommitmentDiscountStatus), real(null),\r\n // Parse from SKU details if not specified explicitly\r\n toreal(coalesce(x_SkuDetails.RINormalizationRatio, SkuPriceDetails.RINormalizationRatio, SkuPriceDetails.x_RINormalizationRatio, dynamic(1)))\r\n )\r\n | extend old_CommitmentDiscountQuantity = CommitmentDiscountQuantity, CommitmentDiscountQuantity = case(\r\n // FOCUS 1.2\r\n isnotempty(CommitmentDiscountQuantity), CommitmentDiscountQuantity,\r\n // FOCUS 1.0-preview, 1.0\r\n isempty(CommitmentDiscountStatus), real(null),\r\n CommitmentDiscountCategory == 'Spend', EffectiveCost / coalesce(x_BillingExchangeRate, real(1)),\r\n CommitmentDiscountCategory == 'Usage' and isnotempty(x_CommitmentDiscountNormalizedRatio), PricingQuantity / coalesce(x_PricingBlockSize, real(1)) * x_CommitmentDiscountNormalizedRatio,\r\n real(null)\r\n )\r\n | extend old_CommitmentDiscountUnit = CommitmentDiscountUnit, CommitmentDiscountUnit = case(\r\n // FOCUS 1.2\r\n isnotempty(CommitmentDiscountUnit), CommitmentDiscountUnit,\r\n // FOCUS 1.0\r\n isempty(CommitmentDiscountQuantity), '',\r\n CommitmentDiscountCategory == 'Spend', PricingCurrency,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio == real(1), ConsumedUnit,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio > real(1), strcat('Normalized ', ConsumedUnit),\r\n ''\r\n )\r\n | extend old_CommitmentDiscountStatus = CommitmentDiscountStatus, CommitmentDiscountStatus = case(\r\n // FOCUS 1.0+\r\n isnotempty(CommitmentDiscountStatus), CommitmentDiscountStatus,\r\n // FOCUS 1.0-preview\r\n ChargeSubcategory == 'Used Commitment', 'Used',\r\n ChargeSubcategory == 'Unused Commitment', 'Unused',\r\n ''\r\n )\r\n | extend x_CommitmentDiscountUtilizationPotential = case(\r\n ChargeCategory == 'Purchase', real(0),\r\n ProviderName == 'Microsoft' and isnotempty(CommitmentDiscountCategory), EffectiveCost,\r\n CommitmentDiscountCategory == 'Usage', ConsumedQuantity,\r\n CommitmentDiscountCategory == 'Spend', EffectiveCost,\r\n real(0)\r\n )\r\n | extend x_CommitmentDiscountUtilizationAmount = iff(CommitmentDiscountStatus == 'Used', x_CommitmentDiscountUtilizationPotential, real(0))\r\n //\r\n // Pricing\r\n | extend old_x_AmortizationClass = x_AmortizationClass, x_AmortizationClass = case(\r\n // FOCUS 1.2\r\n isnotempty(x_AmortizationClass), x_AmortizationClass,\r\n // FOCUS 1.0-preview+\r\n ChargeCategory == 'Purchase' and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeCategory == 'Usage' and isnotempty(CommitmentDiscountId) and isnotempty(CommitmentDiscountStatus), 'Amortized Charge',\r\n ''\r\n )\r\n | extend old_PricingCategory = PricingCategory, PricingCategory = case(\r\n // FOCUS 1.0+\r\n isnotempty(PricingCategory), PricingCategory,\r\n // FOCUS 1.0-preview\r\n PricingCategory == 'On-Demand', 'Standard',\r\n PricingCategory == 'Commitment-Based', 'Committed',\r\n ''\r\n )\r\n //\r\n // Commitment discount utilization\r\n | extend x_CommitmentDiscountUtilizationPotential = case(\r\n ChargeCategory == 'Purchase', real(0),\r\n ProviderName == 'Microsoft' and isnotempty(CommitmentDiscountCategory), EffectiveCost,\r\n CommitmentDiscountCategory == 'Usage', ConsumedQuantity,\r\n CommitmentDiscountCategory == 'Spend', EffectiveCost,\r\n real(0)\r\n )\r\n | extend x_CommitmentDiscountUtilizationAmount = iff(CommitmentDiscountStatus == 'Used', x_CommitmentDiscountUtilizationPotential, real(0))\r\n //\r\n // BUG: Fix ContractedCost that has bad values\r\n | extend ContractedCost = iff(ProviderName == 'Microsoft' and isnotempty(PricingQuantity) and isnotempty(x_PricingBlockSize) and ContractedCost != ContractedUnitPrice * PricingQuantity, ContractedUnitPrice * PricingQuantity, ContractedCost)\r\n //\r\n // Handle FOCUS 1.0-preview UsageQuantity/Unit\r\n | extend ConsumedQuantity = iff(ChargeCategory == 'Usage', coalesce(ConsumedQuantity, UsageQuantity, UsageAmount), real(null))\r\n | extend old_ConsumedUnit = ConsumedUnit, ConsumedUnit = iff(ChargeCategory == 'Usage' and isnotempty(ConsumedQuantity), coalesce(ConsumedUnit, UsageUnit, 'Units'), '')\r\n //\r\n // Convert IDs to lowercase for consistency\r\n | extend BillingAccountId = tolower(BillingAccountId)\r\n | extend CommitmentDiscountId = tolower(CommitmentDiscountId)\r\n //\r\n // BUG: Remove EffectiveCost for commitment discount purchases\r\n | extend old_EffectiveCost = EffectiveCost, EffectiveCost = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), real(0), EffectiveCost)\r\n | extend old_x_EffectiveCostInUsd = x_EffectiveCostInUsd, x_EffectiveCostInUsd = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), real(0), x_EffectiveCostInUsd)\r\n //\r\n // Clean up resource columns\r\n | extend old_ResourceId = ResourceId, ResourceId = case(\r\n isnotempty(ResourceId), ResourceId,\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), CommitmentDiscountId,\r\n ResourceId\r\n )\r\n | extend old_ResourceName = ResourceName, ResourceName = tolower(case(\r\n isnotempty(ResourceName), ResourceName,\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountName), CommitmentDiscountName,\r\n isnotempty(ResourceId), parse_resourceid(ResourceId).ResourceName,\r\n ResourceName\r\n ))\r\n | extend old_x_ResourceType = x_ResourceType, x_ResourceType = case(\r\n isnotempty(x_ResourceType), x_ResourceType,\r\n isnotempty(ResourceId), parse_resourceid(ResourceId).x_ResourceType,\r\n x_ResourceType\r\n )\r\n | extend old_ResourceType = ResourceType, ResourceType = case(\r\n // Use existing resource type display name unless it's an internal resource type ID\r\n isnotempty(ResourceType) and tolower(ResourceType) != tolower(x_ResourceType) and ResourceType !contains '/', ResourceType,\r\n // Use CommitmentDiscountType for commitment discount purchases\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountType), CommitmentDiscountType,\r\n // Look up display name from internal type\r\n isnotempty(x_ResourceType), coalesce(tostring(resource_type(x_ResourceType).SingularDisplayName), ResourceType, x_ResourceType),\r\n ResourceType\r\n )\r\n //\r\n // Handle missing values\r\n | extend old_PublisherName = PublisherName, PublisherName = case(PublisherName == 'Microsoft Corporation', 'Microsoft', isnotempty(PublisherName), PublisherName, x_PublisherCategory == 'Cloud Provider', ProviderName, '')\r\n //\r\n // Handle FOCUS 1.0-preview Region column\r\n | extend old_Region = Region\r\n | extend old_RegionId = RegionId, RegionId = coalesce(RegionId, iff(ProviderName == 'Microsoft', replace_string(tolower(Region), ' ', ''), Region))\r\n | extend RegionName = coalesce(RegionName, Region)\r\n //\r\n // SKU properties\r\n | extend x_SkuCoreCount = toint(coalesce(SkuPriceDetails.CoreCount, SkuPriceDetails.x_VCPUs, x_SkuDetails.VCPUs, SkuPriceDetails.x_VCores, x_SkuDetails.VCores, SkuPriceDetails.x_vCores, x_SkuDetails.vCores))\r\n | extend x_SkuInstanceType = tostring(coalesce(SkuPriceDetails.InstanceType, SkuPriceDetails.x_ServiceType, x_SkuDetails.ServiceType, SkuPriceDetails.x_ServerSku, x_SkuDetails.ServerSku))\r\n | extend x_SkuOperatingSystem = case(\r\n isnotempty(SkuPriceDetails.OperatingSystem), SkuPriceDetails.OperatingSystem,\r\n coalesce(SkuPriceDetails.x_ImageType, x_SkuDetails.ImageType) == 'Canonical', 'Linux',\r\n coalesce(SkuPriceDetails.x_ImageType, x_SkuDetails.ImageType) == 'Windows Server BYOL', 'Windows Server',\r\n x_SkuMeterSubcategory endswith ' Series Windows', 'Windows Server',\r\n coalesce(SkuPriceDetails.x_ImageType, x_SkuDetails.ImageType)\r\n )\r\n | extend x_ConsumedCoreHours = iff(ConsumedUnit == 'Hours' and isnotempty(x_SkuCoreCount), x_SkuCoreCount * ConsumedQuantity, real(null))\r\n | extend SkuPriceDetails = case(\r\n // FOCUS 1.2\r\n isnotempty(SkuPriceDetails), SkuPriceDetails,\r\n // FOCUS 1.0-preview, 1.0\r\n parse_json(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(tostring(x_SkuDetails)\r\n // Prefix all keys with x_ first to avoid double-prefixing\r\n , @'([\\{,])\"', @'\\1\"x_')\r\n // CoreCount for number of CPUs/vCPUs/cores/vCores\r\n , @'\"x_(VCPUs|VCores|vCores)\":', @'\"CoreCount\":')\r\n // TODO: DiskMaxIops for disk I/O operations per second (IOPS)\r\n // TODO: DiskSpace for disk size in GiB\r\n // TODO: DiskType for the kind of disk (e.g., SSD, HDD, NVMe)\r\n // TODO: GpuCount for the number of GPUs\r\n // InstanceType for the resource size/SKU (e.g., ArmSkuName)\r\n , @'\"x_(ServerSku|ServiceType)\":', @'\"InstanceType\":')\r\n // TODO: InstanceSeries for the size family/series\r\n // TODO: MemorySize for the RAM in GiB\r\n // TODO: NetworkMaxIops for network I/O operations per second (IOPS)\r\n // TODO: NetworkMaxThroughput for network max throughput for data transfer in Mbps\r\n // OperatingSystem for the OS name\r\n , @'(\"x_ImageType\":\"Canonical\")', @'\\1,\"OperatingSystem\":\"Linux\"')\r\n , @'(\"x_ImageType\":\"Windows Server( BYOL)?\")', @'\\1,\"OperatingSystem\":\"Windows Server\"')\r\n , @'(\"x_ImageType\":(\"[^\"]+\"))', @'\\1,\"OperatingSystem\":\\2')\r\n // TODO: Redundancy for the level of redundancy (e.g., Local, Zonal, Global)\r\n // TODO: StorageClass for the tier of storage (e.g., Hot, Archive, Nearline)\r\n )\r\n )\r\n | extend SkuPriceDetails = iff(isempty(SkuPriceDetails.OperatingSystem) and isnotempty(x_SkuOperatingSystem),\r\n parse_json(replace_string(tostring(SkuPriceDetails), '}', strcat(@',\"OperatingSystem\":\"', x_SkuOperatingSystem, '\"}'))),\r\n SkuPriceDetails)\r\n //\r\n // Azure Hybrid Benefit\r\n | extend tmp_SqlAhb = tolower(coalesce(x_SkuDetails.AHB, SkuPriceDetails.x_AHB))\r\n | extend x_SkuLicenseType = case(\r\n ChargeCategory != 'Usage', '',\r\n x_SkuMeterCategory in ('Virtual Machines', 'Virtual Machine Licenses') and (x_SkuMeterSubcategory contains 'Windows' or coalesce(SkuPriceDetails.x_ImageType, x_SkuDetails.ImageType) == 'Windows Server BYOL'), 'Windows Server',\r\n isnotempty(tmp_SqlAhb) or x_SkuMeterSubcategory == 'SQL Server Azure Hybrid Benefit', 'SQL Server',\r\n ''\r\n )\r\n | extend x_SkuLicenseStatus = case(\r\n isempty(x_SkuLicenseType), '',\r\n coalesce(SkuPriceDetails.x_ImageType, x_SkuDetails.ImageType) == 'Windows Server BYOL' or tmp_SqlAhb == 'true' or x_SkuMeterSubcategory contains 'Azure Hybrid Benefit', 'Enabled',\r\n (x_SkuMeterSubcategory contains 'Windows') or tmp_SqlAhb == 'false', 'Not Enabled',\r\n ''\r\n )\r\n | extend x_SkuLicenseQuantity = case(\r\n isempty(x_SkuCoreCount) or isempty(x_SkuLicenseType), int(null),\r\n x_SkuCoreCount <= 8, int(8),\r\n x_SkuCoreCount > 8, x_SkuCoreCount,\r\n int(null)\r\n )\r\n | extend x_SkuLicenseUnit = iff(isnotempty(x_SkuLicenseQuantity), 'Cores', '')\r\n //\r\n // Savings\r\n | extend x_CommitmentDiscountSavings = iff(isempty(ContractedCost) or ContractedCost == 0 or ContractedCost - EffectiveCost < 0.0001, real(0), ContractedCost - EffectiveCost)\r\n | extend x_NegotiatedDiscountSavings = iff(isempty(ListCost) or ListCost == 0 or ListCost - ContractedCost < 0.0001, real(0), ListCost - ContractedCost)\r\n | extend x_TotalSavings = iff(isempty(ListCost) or ListCost == 0 or ListCost - EffectiveCost < 0.0001, real(0), ListCost - EffectiveCost)\r\n | extend x_CommitmentDiscountPercent = iff(isempty(ContractedUnitPrice) or ContractedUnitPrice == 0 or ContractedUnitPrice - x_EffectiveUnitPrice < 0.0001, real(0), (ContractedUnitPrice - x_EffectiveUnitPrice) / ContractedUnitPrice)\r\n | extend x_NegotiatedDiscountPercent = iff(isempty(ListUnitPrice) or ListUnitPrice == 0 or ListUnitPrice - ContractedUnitPrice < 0.0001, real(0), (ListUnitPrice - ContractedUnitPrice) / ListUnitPrice)\r\n | extend x_TotalDiscountPercent = iff(isempty(ListUnitPrice) or ListUnitPrice == 0 or ListUnitPrice - x_EffectiveUnitPrice < 0.0001, real(0), (ListUnitPrice - x_EffectiveUnitPrice) / ListUnitPrice)\r\n //\r\n // Minor fixes\r\n | extend old_BillingPeriodEnd = BillingPeriodEnd, BillingPeriodEnd = startofmonth(BillingPeriodEnd)\r\n | extend old_BillingPeriodStart = BillingPeriodStart, BillingPeriodStart = startofmonth(BillingPeriodStart)\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n AvailabilityZone,\r\n BilledCost,\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingAccountType,\r\n BillingCurrency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n CapacityReservationId,\r\n CapacityReservationStatus,\r\n ChargeCategory,\r\n ChargeClass,\r\n ChargeDescription,\r\n ChargeFrequency,\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId,\r\n CommitmentDiscountName,\r\n CommitmentDiscountQuantity,\r\n CommitmentDiscountStatus,\r\n CommitmentDiscountType,\r\n CommitmentDiscountUnit,\r\n ConsumedQuantity,\r\n ConsumedUnit,\r\n ContractedCost = coalesce(ContractedCost, x_OnDemandCost, x_Cost),\r\n ContractedUnitPrice = coalesce(ContractedUnitPrice, x_OnDemandUnitPrice),\r\n EffectiveCost,\r\n InvoiceId = coalesce(InvoiceId, x_InvoiceId),\r\n InvoiceIssuerName,\r\n ListCost,\r\n ListUnitPrice,\r\n PricingCategory,\r\n PricingCurrency,\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n PublisherName,\r\n RegionId,\r\n RegionName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory, // TODO: Populate ServiceSubcategory from ServiceName when missing\r\n SkuId,\r\n SkuMeter,\r\n SkuPriceDetails,\r\n SkuPriceId,\r\n SubAccountId,\r\n SubAccountName = iff(isempty(SubAccountId), '', SubAccountName),\r\n SubAccountType,\r\n Tags,\r\n x_AccountId = iff(x_AccountId == '-2', '', x_AccountId),\r\n x_AccountName = iff(x_AccountId == '-2', '', x_AccountName),\r\n x_AccountOwnerId = iff(x_AccountId == '-2', '', x_AccountOwnerId),\r\n x_AmortizationClass,\r\n x_BilledCostInUsd,\r\n x_BilledUnitPrice,\r\n x_BillingAccountAgreement = case(\r\n ProviderName == 'Microsoft' and x_BillingAccountId == x_BillingProfileId, 'EA',\r\n ProviderName == 'Microsoft' and x_BillingAccountId != x_BillingProfileId, 'MCA',\r\n ProviderName\r\n ),\r\n x_BillingAccountId,\r\n x_BillingAccountName,\r\n x_BillingExchangeRate,\r\n x_BillingExchangeRateDate,\r\n x_BillingItemCode,\r\n x_BillingItemName,\r\n x_BillingProfileId,\r\n x_BillingProfileName,\r\n x_ChargeId,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_CommitmentDiscountPercent,\r\n x_CommitmentDiscountSavings,\r\n x_CommitmentDiscountSpendEligibility = '', // TODO: Add x_CommitmentDiscountSpendEligibility for Costs\r\n x_CommitmentDiscountUsageEligibility = '', // TODO: Add x_CommitmentDiscountUsageEligibility for Costs\r\n x_CommitmentDiscountUtilizationAmount,\r\n x_CommitmentDiscountUtilizationPotential,\r\n x_CommodityCode,\r\n x_CommodityName,\r\n x_ComponentName,\r\n x_ComponentType,\r\n x_ConsumedCoreHours,\r\n x_ContractedCostInUsd = coalesce(x_ContractedCostInUsd, x_OnDemandCostInUsd),\r\n x_CostAllocationRuleName,\r\n x_CostCategories = parse_json(x_CostCategories),\r\n x_CostCenter,\r\n x_CostType,\r\n x_Credits = parse_json(x_Credits),\r\n x_CurrencyConversionRate,\r\n x_CustomerId,\r\n x_CustomerName,\r\n x_Discount = parse_json(x_Discount),\r\n x_EffectiveCostInUsd,\r\n x_EffectiveUnitPrice,\r\n x_ExportTime,\r\n x_IngestionTime,\r\n x_InstanceID,\r\n x_InvoiceIssuerId,\r\n x_InvoiceSectionId = case(\r\n x_InvoiceSectionId == '-2', '',\r\n x_InvoiceSectionId\r\n ),\r\n x_InvoiceSectionName = case(\r\n x_InvoiceSectionName == 'Unassigned', '',\r\n x_InvoiceSectionName\r\n ),\r\n x_ListCostInUsd,\r\n x_Location,\r\n x_NegotiatedDiscountPercent,\r\n x_NegotiatedDiscountSavings,\r\n x_Operation,\r\n x_OwnerAccountID,\r\n x_PartnerCreditApplied,\r\n x_PartnerCreditRate,\r\n x_PricingBlockSize,\r\n x_PricingSubcategory,\r\n x_PricingUnitDescription = iff(x_PricingUnitDescription == 'Unassigned', '', x_PricingUnitDescription),\r\n x_Project,\r\n x_PublisherCategory,\r\n x_PublisherId,\r\n x_ResellerId,\r\n x_ResellerName,\r\n x_ResourceGroupName = tolower(x_ResourceGroupName),\r\n x_ResourceType,\r\n x_ServiceCode,\r\n x_ServiceId,\r\n x_ServiceModel, // TODO: Populate from ServiceName when missing\r\n x_ServicePeriodEnd,\r\n x_ServicePeriodStart,\r\n x_SkuCoreCount,\r\n x_SkuDescription,\r\n x_SkuDetails,\r\n x_SkuInstanceType,\r\n x_SkuIsCreditEligible,\r\n x_SkuLicenseQuantity,\r\n x_SkuLicenseStatus,\r\n x_SkuLicenseType,\r\n x_SkuLicenseUnit,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterSubcategory,\r\n x_SkuOfferId,\r\n x_SkuOperatingSystem,\r\n x_SkuOrderId,\r\n x_SkuOrderName,\r\n x_SkuPartNumber,\r\n x_SkuPlanName,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceChanges,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceValues = bag_merge(\r\n checkString('BillingPeriodEnd', old_BillingPeriodEnd, BillingPeriodEnd),\r\n checkString('BillingPeriodStart', old_BillingPeriodStart, BillingPeriodStart),\r\n checkString('CapacityReservationStatus', old_CapacityReservationStatus, CapacityReservationStatus),\r\n checkString('ChargeCategory', old_ChargeCategory, ChargeCategory),\r\n checkString('ChargeClass', old_ChargeClass, ChargeClass),\r\n checkString('ChargeSubcategory', old_ChargeSubcategory, ''), // Not included in final schema; use empty string\r\n checkString('ChargeFrequency', old_ChargeFrequency, ChargeFrequency),\r\n checkReal('CommitmentDiscountQuantity', old_CommitmentDiscountQuantity, CommitmentDiscountQuantity),\r\n checkString('CommitmentDiscountUnit', old_CommitmentDiscountUnit, CommitmentDiscountUnit),\r\n checkString('CommitmentDiscountStatus', old_CommitmentDiscountStatus, CommitmentDiscountStatus),\r\n checkReal('ConsumedQuantity', old_ConsumedQuantity, ConsumedQuantity),\r\n checkString('ConsumedUnit', old_ConsumedUnit, ConsumedUnit),\r\n checkReal('ContractedCost', old_ContractedCost, ContractedCost),\r\n checkReal('ContractedUnitPrice', old_ContractedUnitPrice, ContractedUnitPrice),\r\n checkReal('EffectiveCost', old_EffectiveCost, EffectiveCost),\r\n checkReal('ListCost', old_ListCost, ListCost),\r\n checkReal('ListUnitPrice', old_ListUnitPrice, ListUnitPrice),\r\n checkString('PricingCategory', old_PricingCategory, PricingCategory),\r\n checkReal('PricingQuantity', old_PricingQuantity, PricingQuantity),\r\n checkString('ProviderName', old_ProviderName, ProviderName),\r\n checkString('PublisherName', old_PublisherName, PublisherName),\r\n checkString('Region', old_Region, ''), // Not included in final schema; use empty string\r\n checkString('RegionId', old_RegionId, RegionId),\r\n checkString('ResourceId', old_ResourceId, ResourceId),\r\n checkString('ResourceName', old_ResourceName, ResourceName),\r\n checkString('ResourceType', old_ResourceType, ResourceType),\r\n checkString('x_AmortizationClass', old_x_AmortizationClass, x_AmortizationClass),\r\n checkReal('x_EffectiveCostInUsd', old_x_EffectiveCostInUsd, x_EffectiveCostInUsd),\r\n checkReal('x_EffectiveUnitPrice', old_x_EffectiveUnitPrice, x_EffectiveUnitPrice),\r\n checkString('x_ResourceType', old_x_ResourceType, x_ResourceType)\r\n ),\r\n x_SourceVersion,\r\n x_SubproductName,\r\n x_TotalDiscountPercent,\r\n x_TotalSavings,\r\n x_UsageType\r\n}\r\n\r\n// Costs_final_v1_2 table\r\n.create-merge table Costs_final_v1_2 (\r\n AvailabilityZone: string,\r\n BilledCost: real,\r\n BillingAccountId: string,\r\n BillingAccountName: string,\r\n BillingAccountType: string,\r\n BillingCurrency: string,\r\n BillingPeriodEnd: datetime,\r\n BillingPeriodStart: datetime,\r\n CapacityReservationId: string,\r\n CapacityReservationStatus: string,\r\n ChargeCategory: string,\r\n ChargeClass: string,\r\n ChargeDescription: string,\r\n ChargeFrequency: string,\r\n ChargePeriodEnd: datetime,\r\n ChargePeriodStart: datetime,\r\n CommitmentDiscountCategory: string,\r\n CommitmentDiscountId: string,\r\n CommitmentDiscountName: string,\r\n CommitmentDiscountQuantity: real,\r\n CommitmentDiscountStatus: string,\r\n CommitmentDiscountType: string,\r\n CommitmentDiscountUnit: string,\r\n ConsumedQuantity: real,\r\n ConsumedUnit: string,\r\n ContractedCost: real,\r\n ContractedUnitPrice: real,\r\n EffectiveCost: real,\r\n InvoiceId: string,\r\n InvoiceIssuerName: string,\r\n ListCost: real,\r\n ListUnitPrice: real,\r\n PricingCategory: string,\r\n PricingCurrency: string,\r\n PricingQuantity: real,\r\n PricingUnit: string,\r\n ProviderName: string,\r\n PublisherName: string,\r\n RegionId: string,\r\n RegionName: string,\r\n ResourceId: string,\r\n ResourceName: string,\r\n ResourceType: string,\r\n ServiceCategory: string,\r\n ServiceName: string,\r\n ServiceSubcategory: string,\r\n SkuId: string,\r\n SkuMeter: string,\r\n SkuPriceDetails: dynamic,\r\n SkuPriceId: string,\r\n SubAccountId: string,\r\n SubAccountName: string,\r\n SubAccountType: string,\r\n Tags: dynamic,\r\n x_AccountId: string, // Azure 1.0-preview(v1)+\r\n x_AccountName: string, // Azure 1.0-preview(v1)+\r\n x_AccountOwnerId: string, // Azure 1.0-preview(v1)+\r\n x_AmortizationClass: string, // Azure 1.2-preview+\r\n x_BilledCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_BilledUnitPrice: real, // Azure 1.0-preview(v1)+\r\n x_BillingAccountAgreement: string, // Hubs add-on\r\n x_BillingAccountId: string, // Azure 1.0-preview(v1)+\r\n x_BillingAccountName: string, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRate: real, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRateDate: datetime, // Azure 1.0-preview(v1)+\r\n x_BillingItemCode: string, // Alibaba 1.0\r\n x_BillingItemName: string, // Alibaba 1.0\r\n x_BillingProfileId: string, // Azure 1.0-preview(v1)+\r\n x_BillingProfileName: string, // Azure 1.0-preview(v1)+\r\n x_ChargeId: string, // Azure 1.0-preview(v1) only\r\n x_CommitmentDiscountNormalizedRatio: real, // Azure 1.2-preview+\r\n x_CommitmentDiscountPercent: real, // Hubs add-on\r\n x_CommitmentDiscountSavings: real, // Hubs add-on\r\n x_CommitmentDiscountSpendEligibility: string, // Hubs add-on\r\n x_CommitmentDiscountUsageEligibility: string, // Hubs add-on\r\n x_CommitmentDiscountUtilizationAmount: real, // Hubs add-on\r\n x_CommitmentDiscountUtilizationPotential: real, // Hubs add-on\r\n x_CommodityCode: string, // Alibaba 1.0\r\n x_CommodityName: string, // Alibaba 1.0\r\n x_ComponentName: string, // Tencent 1.0\r\n x_ComponentType: string, // Tencent 1.0\r\n x_ConsumedCoreHours: real, // Hubs add-on\r\n x_ContractedCostInUsd: real, // Azure 1.0+\r\n x_CostAllocationRuleName: string, // Azure 1.0-preview(v1)+\r\n x_CostCategories: dynamic, // AWS 1.0 (JSON)\r\n x_CostCenter: string, // Azure 1.0-preview(v1)+\r\n x_CostType: string, // GCP Jan 2024\r\n x_Credits: dynamic, // GCP Jan 2024\r\n x_CurrencyConversionRate: real, // GCP Jun 2024\r\n x_CustomerId: string, // Azure 1.0-preview(v1)+\r\n x_CustomerName: string, // Azure 1.0-preview(v1)+\r\n x_Discount: dynamic, // AWS 1.0 (JSON)\r\n x_EffectiveCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_EffectiveUnitPrice: real, // Azure 1.0-preview(v1)+\r\n x_ExportTime: datetime, // GCP Jan 2024 / Tencent 1.0\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_InstanceID: string, // Alibaba 1.0\r\n x_InvoiceIssuerId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionName: string, // Azure 1.0-preview(v1)+\r\n x_ListCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_Location: string, // GCP Jan 2024\r\n x_NegotiatedDiscountPercent:real, // Hubs add-on\r\n x_NegotiatedDiscountSavings:real, // Hubs add-on\r\n x_Operation: string, // AWS 1.0\r\n x_OwnerAccountID: string, // Tencent 1.0\r\n x_PartnerCreditApplied: string, // Azure 1.0-preview(v1)+\r\n x_PartnerCreditRate: string, // Azure 1.0-preview(v1)+\r\n x_PricingBlockSize: real, // Azure 1.0-preview(v1)+\r\n x_PricingSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_PricingUnitDescription: string, // Azure 1.0-preview(v1)+\r\n x_Project: string, // GCP Jan 2024\r\n x_PublisherCategory: string, // Azure 1.0-preview(v1)+\r\n x_PublisherId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceGroupName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceType: string, // Azure 1.0-preview(v1)+\r\n x_ServiceCode: string, // AWS 1.0\r\n x_ServiceId: string, // GCP Jan 2024\r\n x_ServiceModel: string, // Azure 1.2-preview+\r\n x_ServicePeriodEnd: datetime, // Azure 1.0-preview(v1)+\r\n x_ServicePeriodStart: datetime, // Azure 1.0-preview(v1)+\r\n x_SkuCoreCount: int, // Hubs add-on\r\n x_SkuDescription: string, // Azure 1.0-preview(v1)+\r\n x_SkuDetails: dynamic, // Azure 1.0-preview(v1)+\r\n x_SkuInstanceType: string, // Hubs add-on\r\n x_SkuIsCreditEligible: bool, // Azure 1.0-preview(v1)+\r\n x_SkuLicenseQuantity: int, // Hubs add-on\r\n x_SkuLicenseStatus: string, // Hubs add-on\r\n x_SkuLicenseType: string, // Hubs add-on\r\n x_SkuLicenseUnit: string, // Hubs add-on\r\n x_SkuMeterCategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterId: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuOfferId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOperatingSystem: string, // Hubs add-on\r\n x_SkuOrderId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOrderName: string, // Azure 1.0-preview(v1)+\r\n x_SkuPartNumber: string, // Azure 1.0-preview(v1)+\r\n x_SkuPlanName: string, // Azure 1.2-preview+\r\n x_SkuRegion: string, // Azure 1.0-preview(v1)+\r\n x_SkuServiceFamily: string, // Azure 1.0-preview(v1)+\r\n x_SkuTerm: int, // Azure 1.0-preview(v1)+\r\n x_SkuTier: string, // Azure 1.0-preview(v1)+\r\n x_SourceChanges: string, // Hubs add-on\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceValues: dynamic, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_SubproductName: string, // Tencent 1.0\r\n x_TotalDiscountPercent: real, // Hubs add-on\r\n x_TotalSavings: real, // Hubs add-on\r\n x_UsageType: string // AWS 1.0\r\n)\r\n\r\n// Update policy for Costs_raw -> Costs_final_v1_2 table\r\n.alter table Costs_final_v1_2 policy update\r\n```\r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"Costs_raw\",\r\n \"Query\": \"Costs_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Actual costs |===================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// ActualCosts_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='ActualCost exports transformed to FOCUS 1.2.', folder='Costs')\r\nActualCosts_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n ActualCosts_raw\r\n | where ChargeType in ('Purchase', 'Refund') and isnotempty(ReservationId)\r\n //\r\n //\r\n // !!! The rest of the query is reused for both the ActualCosts and AmortizedCosts queries -- Copy all changes in both transform functions !!!\r\n //\r\n //\r\n | extend x_AmortizationClass = case(\r\n ChargeType in ('Purchase', 'Refund') and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', 'Amortized Charge',\r\n ''\r\n )\r\n | extend tmp_ResourceInfo = parse_resourceid(ResourceId)\r\n // TODO: PricingCategory needs to include savings plan usage and spot usage\r\n | extend PricingCategory = case(\r\n x_AmortizationClass == 'Amortized Charge', 'Committed',\r\n ChargeType in ('Usage', 'Purchase'), 'Standard',\r\n ''\r\n )\r\n | extend x_ResourceType = tostring(tmp_ResourceInfo.x_ResourceType)\r\n | project-rename\r\n PricingQuantity = Quantity,\r\n x_PricingUnitDescription = UnitOfMeasure\r\n | join kind=leftouter (PricingUnits) on x_PricingUnitDescription\r\n | join kind=leftouter (Regions) on ResourceLocation\r\n | join kind=leftouter (ResourceTypes | project x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n // TODO: Add the following in 1.2: PublisherName, x_PublisherCategory, x_Environment\r\n | join kind=leftouter (Services | where isnotempty(x_ResourceType) | project x_ResourceType, ServiceName, ServiceCategory, ServiceSubcategory, x_ServiceModel) on x_ResourceType\r\n | join kind=leftouter (Services | where isnotempty(x_ConsumedService) | summarize take_any(ServiceName), take_any(ServiceCategory), take_any(ServiceSubcategory), take_any(x_ServiceModel) by ConsumedService = x_ConsumedService) on ConsumedService\r\n | extend CommitmentDiscountCategory = iff(isnotempty(ReservationId), 'Usage', '') // TODO: CommitmentDiscountCategory needs to handle savings plans\r\n | project\r\n AvailabilityZone = AvailabilityZone,\r\n BilledCost = iff(x_AmortizationClass == 'Amortized Charge', real(0), Cost),\r\n BillingAccountId = strcat('/providers/microsoft.billing/billingaccounts/', BillingAccountId, iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), '', strcat('/billingprofiles/', BillingProfileId))),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName),\r\n BillingAccountType = iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), 'Billing Profile', 'Billing Account'),\r\n BillingCurrency,\r\n BillingPeriodEnd = startofmonth(BillingPeriodEndDate, 1),\r\n BillingPeriodStart = startofmonth(BillingPeriodStartDate),\r\n CapacityReservationId = '',\r\n CapacityReservationStatus = '',\r\n ChargeCategory = case(\r\n ChargeType in ('Usage', 'Purchase', 'Credit', 'Tax'), ChargeType,\r\n ChargeType in ('UnusedReservation', 'UnusedSavingsPlan'), 'Usage',\r\n ChargeType == 'Refund', 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = iff(ChargeType == 'Refund', 'Correction', ''),\r\n ChargeDescription = Product,\r\n ChargeFrequency = case(\r\n Frequency == 'UsageBased', 'Usage-Based',\r\n Frequency == 'OneTime', 'One-Time',\r\n Frequency // \"Recurring\" and any fallback\r\n ),\r\n ChargePeriodEnd = Date + 1d,\r\n ChargePeriodStart = Date,\r\n ChargeSubcategory = '',\r\n // TODO: CommitmentDiscount* columns need to handle savings plans\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId = iff(isnotempty(ReservationId), strcat('/providers/microsoft.capacity/reservationorders/', ProductOrderId, '/reservations/', ReservationId), ''),\r\n CommitmentDiscountName = ReservationName,\r\n CommitmentDiscountQuantity = real(null),\r\n CommitmentDiscountStatus = case(\r\n isempty(ReservationId), '',\r\n isnotempty(ReservationId) and ChargeType == 'Usage', 'Used',\r\n ChargeType startswith 'Unused', 'Unused',\r\n 'Unused'\r\n ),\r\n CommitmentDiscountType = iff(isnotempty(ReservationId), 'Reservation', ''),\r\n CommitmentDiscountUnit = '',\r\n ConsumedQuantity = PricingQuantity * x_PricingBlockSize,\r\n ConsumedUnit = PricingUnit,\r\n ContractedCost = UnitPrice * PricingQuantity,\r\n ContractedUnitPrice = UnitPrice,\r\n EffectiveCost = iff(x_AmortizationClass == 'Principal', real(0), Cost),\r\n InvoiceId = '',\r\n InvoiceIssuerName = 'Microsoft',\r\n ListCost = real(null),\r\n ListUnitPrice = real(null),\r\n PricingCategory,\r\n PricingCurrency = iff(BillingAccountId == BillingProfileId, BillingCurrency, ''),\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName = 'Microsoft',\r\n PublisherName = iff(PublisherType == 'Marketplace', PublisherName, 'Microsoft'),\r\n Region = '',\r\n RegionId,\r\n RegionName,\r\n ResourceId = tostring(tmp_ResourceInfo.ResourceId),\r\n ResourceName = tostring(tmp_ResourceInfo.ResourceName),\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory,\r\n SkuId = '',\r\n SkuMeter = MeterName,\r\n SkuPriceDetails = '',\r\n SkuPriceId = '',\r\n SubAccountId = strcat('/subscriptions/', SubscriptionId),\r\n SubAccountName = iff(isempty(SubscriptionId), '', SubscriptionName),\r\n SubAccountType = iff(isempty(SubscriptionId), '', 'Subscription'),\r\n Tags = iff(Tags startswith '{', Tags, strcat('{', Tags, '}')),\r\n UsageAmount = real(null),\r\n UsageQuantity = real(null),\r\n UsageUnit = '',\r\n x_AccountId = '',\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerId,\r\n x_AmortizationClass = '',\r\n x_BilledCostInUsd = real(null),\r\n x_BilledUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), UnitPrice),\r\n x_BillingAccountId = BillingAccountId,\r\n x_BillingAccountName = BillingAccountName,\r\n x_BillingExchangeRate = real(null),\r\n x_BillingExchangeRateDate = datetime(null),\r\n x_BillingItemCode = '',\r\n x_BillingItemName = '',\r\n x_BillingProfileId = BillingProfileId,\r\n x_BillingProfileName = BillingProfileName,\r\n x_ChargeId = '',\r\n x_CommodityCode = '',\r\n x_CommodityName = '',\r\n x_ComponentType = '',\r\n x_ComponentName = '',\r\n x_ContractedCostInUsd = real(null),\r\n x_Cost = real(null),\r\n x_CostAllocationRuleName = '',\r\n x_CostCategories = '',\r\n x_CostCenter = CostCenter,\r\n x_CostType = '',\r\n x_Credits = '',\r\n x_CurrencyConversionRate = real(null),\r\n x_CustomerId = '',\r\n x_CustomerName = '',\r\n x_Discount = '',\r\n x_EffectiveCostInUsd = real(null),\r\n x_EffectiveUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), EffectivePrice),\r\n x_ExportTime = datetime(null),\r\n x_InstanceID = '',\r\n x_InvoiceId = '',\r\n x_InvoiceIssuerId = '',\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = InvoiceSection,\r\n x_ListCostInUsd = real(null),\r\n x_Location = '',\r\n x_OnDemandCost = real(null),\r\n x_OnDemandCostInUsd = real(null),\r\n x_OnDemandUnitPrice = real(null),\r\n x_Operation = '',\r\n x_OwnerAccountID = '',\r\n x_PartnerCreditApplied = '',\r\n x_PartnerCreditRate = '',\r\n x_PricingBlockSize,\r\n x_PricingCurrency = '',\r\n x_PricingSubcategory = case(\r\n // TODO: Add x_SkuTier when supported by C360 -- PricingCategory == 'Standard' and isnotempty(x_SkuTier), 'Tiered',\r\n PricingCategory == 'Standard', 'Standard',\r\n PricingCategory == 'Committed', strcat('Committed ', CommitmentDiscountCategory),\r\n PricingCategory == 'Dynamic', 'Spot',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_Project = '',\r\n x_PublisherCategory = iff(PublisherType == 'Marketplace', 'Vendor', 'Cloud Provider'),\r\n x_PublisherId = '',\r\n x_ResellerId = '',\r\n x_ResellerName = '',\r\n x_ResourceGroupName = ResourceGroup,\r\n x_ResourceType,\r\n x_ServiceCode = '',\r\n x_ServiceId = '',\r\n x_ServiceModel,\r\n x_ServicePeriodEnd = datetime(null),\r\n x_ServicePeriodStart = datetime(null),\r\n x_SkuDescription = Product,\r\n x_SkuDetails = iff(AdditionalInfo startswith '{', AdditionalInfo, strcat('{', AdditionalInfo, '}')),\r\n x_SkuIsCreditEligible = IsAzureCreditEligible,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterId = MeterId,\r\n x_SkuMeterName = MeterName,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuOfferId = OfferId,\r\n x_SkuOrderId = ProductOrderId,\r\n x_SkuOrderName = ProductOrderName,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPlanName = '',\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTerm = toint(Term),\r\n x_SkuTier = '',\r\n x_SourceName = 'C360',\r\n x_SourceProvider = 'Microsoft',\r\n x_SourceType = 'ActualCost',\r\n x_SourceVersion = 'C360-2025-04',\r\n x_SubproductName = '',\r\n x_UsageType = ''\r\n}\r\n\r\n// Update policy for ActualCosts_raw -> Costs_raw table\r\n.alter table Costs_raw policy update\r\n``` \r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"ActualCosts_raw\",\r\n \"Query\": \"ActualCosts_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Amortized costs |================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// AmortizedCosts_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='ActualCost exports transformed to FOCUS 1.2.', folder='Costs')\r\nAmortizedCosts_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n AmortizedCosts_raw\r\n //\r\n //\r\n // !!! The rest of the query is reused for both the ActualCosts and AmortizedCosts queries -- Copy all changes in both transform functions !!!\r\n //\r\n //\r\n | extend x_AmortizationClass = case(\r\n ChargeType in ('Purchase', 'Refund') and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', 'Amortized Charge',\r\n ''\r\n )\r\n | extend tmp_ResourceInfo = parse_resourceid(ResourceId)\r\n // TODO: PricingCategory needs to include savings plan usage and spot usage\r\n | extend PricingCategory = case(\r\n x_AmortizationClass == 'Amortized Charge', 'Committed',\r\n ChargeType in ('Usage', 'Purchase'), 'Standard',\r\n ''\r\n )\r\n | extend x_ResourceType = tostring(tmp_ResourceInfo.x_ResourceType)\r\n | project-rename\r\n PricingQuantity = Quantity,\r\n x_PricingUnitDescription = UnitOfMeasure\r\n | join kind=leftouter (PricingUnits) on x_PricingUnitDescription\r\n | join kind=leftouter (Regions) on ResourceLocation\r\n | join kind=leftouter (ResourceTypes | project x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n // TODO: Add the following in 1.2: PublisherName, x_PublisherCategory, x_Environment\r\n | join kind=leftouter (Services | where isnotempty(x_ResourceType) | project x_ResourceType, ServiceName, ServiceCategory, ServiceSubcategory, x_ServiceModel) on x_ResourceType\r\n | join kind=leftouter (Services | where isnotempty(x_ConsumedService) | summarize take_any(ServiceName), take_any(ServiceCategory), take_any(ServiceSubcategory), take_any(x_ServiceModel) by ConsumedService = x_ConsumedService) on ConsumedService\r\n | extend CommitmentDiscountCategory = iff(isnotempty(ReservationId), 'Usage', '') // TODO: CommitmentDiscountCategory needs to handle savings plans\r\n | project\r\n AvailabilityZone = AvailabilityZone,\r\n BilledCost = iff(x_AmortizationClass == 'Amortized Charge', real(0), Cost),\r\n BillingAccountId = strcat('/providers/microsoft.billing/billingaccounts/', BillingAccountId, iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), '', strcat('/billingprofiles/', BillingProfileId))),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName),\r\n BillingAccountType = iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), 'Billing Profile', 'Billing Account'),\r\n BillingCurrency,\r\n BillingPeriodEnd = startofmonth(BillingPeriodEndDate, 1),\r\n BillingPeriodStart = startofmonth(BillingPeriodStartDate),\r\n CapacityReservationId = '',\r\n CapacityReservationStatus = '',\r\n ChargeCategory = case(\r\n ChargeType in ('Usage', 'Purchase', 'Credit', 'Tax'), ChargeType,\r\n ChargeType in ('UnusedReservation', 'UnusedSavingsPlan'), 'Usage',\r\n ChargeType == 'Refund', 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = iff(ChargeType == 'Refund', 'Correction', ''),\r\n ChargeDescription = Product,\r\n ChargeFrequency = case(\r\n Frequency == 'UsageBased', 'Usage-Based',\r\n Frequency == 'OneTime', 'One-Time',\r\n Frequency // \"Recurring\" and any fallback\r\n ),\r\n ChargePeriodEnd = Date + 1d,\r\n ChargePeriodStart = Date,\r\n ChargeSubcategory = '',\r\n // TODO: CommitmentDiscount* columns need to handle savings plans\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId = iff(isnotempty(ReservationId), strcat('/providers/microsoft.capacity/reservationorders/', ProductOrderId, '/reservations/', ReservationId), ''),\r\n CommitmentDiscountName = ReservationName,\r\n CommitmentDiscountQuantity = real(null),\r\n CommitmentDiscountStatus = case(\r\n isempty(ReservationId), '',\r\n isnotempty(ReservationId) and ChargeType == 'Usage', 'Used',\r\n ChargeType startswith 'Unused', 'Unused',\r\n 'Unused'\r\n ),\r\n CommitmentDiscountType = iff(isnotempty(ReservationId), 'Reservation', ''),\r\n CommitmentDiscountUnit = '',\r\n ConsumedQuantity = PricingQuantity * x_PricingBlockSize,\r\n ConsumedUnit = PricingUnit,\r\n ContractedCost = UnitPrice * PricingQuantity,\r\n ContractedUnitPrice = UnitPrice,\r\n EffectiveCost = iff(x_AmortizationClass == 'Principal', real(0), Cost),\r\n InvoiceId = '',\r\n InvoiceIssuerName = 'Microsoft',\r\n ListCost = real(null),\r\n ListUnitPrice = real(null),\r\n PricingCategory,\r\n PricingCurrency = iff(BillingAccountId == BillingProfileId, BillingCurrency, ''),\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName = 'Microsoft',\r\n PublisherName = iff(PublisherType == 'Marketplace', PublisherName, 'Microsoft'),\r\n Region = '',\r\n RegionId,\r\n RegionName,\r\n ResourceId = tostring(tmp_ResourceInfo.ResourceId),\r\n ResourceName = tostring(tmp_ResourceInfo.ResourceName),\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory,\r\n SkuId = '',\r\n SkuMeter = MeterName,\r\n SkuPriceDetails = '',\r\n SkuPriceId = '',\r\n SubAccountId = strcat('/subscriptions/', SubscriptionId),\r\n SubAccountName = iff(isempty(SubscriptionId), '', SubscriptionName),\r\n SubAccountType = iff(isempty(SubscriptionId), '', 'Subscription'),\r\n Tags = iff(Tags startswith '{', Tags, strcat('{', Tags, '}')),\r\n UsageAmount = real(null),\r\n UsageQuantity = real(null),\r\n UsageUnit = '',\r\n x_AccountId = '',\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerId,\r\n x_AmortizationClass = '',\r\n x_BilledCostInUsd = real(null),\r\n x_BilledUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), UnitPrice),\r\n x_BillingAccountId = BillingAccountId,\r\n x_BillingAccountName = BillingAccountName,\r\n x_BillingExchangeRate = real(null),\r\n x_BillingExchangeRateDate = datetime(null),\r\n x_BillingItemCode = '',\r\n x_BillingItemName = '',\r\n x_BillingProfileId = BillingProfileId,\r\n x_BillingProfileName = BillingProfileName,\r\n x_ChargeId = '',\r\n x_CommodityCode = '',\r\n x_CommodityName = '',\r\n x_ComponentType = '',\r\n x_ComponentName = '',\r\n x_ContractedCostInUsd = real(null),\r\n x_Cost = real(null),\r\n x_CostAllocationRuleName = '',\r\n x_CostCategories = '',\r\n x_CostCenter = CostCenter,\r\n x_CostType = '',\r\n x_Credits = '',\r\n x_CurrencyConversionRate = real(null),\r\n x_CustomerId = '',\r\n x_CustomerName = '',\r\n x_Discount = '',\r\n x_EffectiveCostInUsd = real(null),\r\n x_EffectiveUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), EffectivePrice),\r\n x_ExportTime = datetime(null),\r\n x_InstanceID = '',\r\n x_InvoiceId = '',\r\n x_InvoiceIssuerId = '',\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = InvoiceSection,\r\n x_ListCostInUsd = real(null),\r\n x_Location = '',\r\n x_OnDemandCost = real(null),\r\n x_OnDemandCostInUsd = real(null),\r\n x_OnDemandUnitPrice = real(null),\r\n x_Operation = '',\r\n x_OwnerAccountID = '',\r\n x_PartnerCreditApplied = '',\r\n x_PartnerCreditRate = '',\r\n x_PricingBlockSize,\r\n x_PricingCurrency = '',\r\n x_PricingSubcategory = case(\r\n // TODO: Add x_SkuTier when supported by C360 -- PricingCategory == 'Standard' and isnotempty(x_SkuTier), 'Tiered',\r\n PricingCategory == 'Standard', 'Standard',\r\n PricingCategory == 'Committed', strcat('Committed ', CommitmentDiscountCategory),\r\n PricingCategory == 'Dynamic', 'Spot',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_Project = '',\r\n x_PublisherCategory = iff(PublisherType == 'Marketplace', 'Vendor', 'Cloud Provider'),\r\n x_PublisherId = '',\r\n x_ResellerId = '',\r\n x_ResellerName = '',\r\n x_ResourceGroupName = ResourceGroup,\r\n x_ResourceType,\r\n x_ServiceCode = '',\r\n x_ServiceId = '',\r\n x_ServiceModel,\r\n x_ServicePeriodEnd = datetime(null),\r\n x_ServicePeriodStart = datetime(null),\r\n x_SkuDescription = Product,\r\n x_SkuDetails = iff(AdditionalInfo startswith '{', AdditionalInfo, strcat('{', AdditionalInfo, '}')),\r\n x_SkuIsCreditEligible = IsAzureCreditEligible,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterId = MeterId,\r\n x_SkuMeterName = MeterName,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuOfferId = OfferId,\r\n x_SkuOrderId = ProductOrderId,\r\n x_SkuOrderName = ProductOrderName,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPlanName = '',\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTerm = toint(Term),\r\n x_SkuTier = '',\r\n x_SourceName = 'C360',\r\n x_SourceProvider = 'Microsoft',\r\n x_SourceType = 'ActualCost',\r\n x_SourceVersion = 'C360-2025-04',\r\n x_SubproductName = '',\r\n x_UsageType = ''\r\n}\r\n\r\n// Update policy for AmortizedCosts_raw -> Costs_raw table\r\n.alter table Costs_raw policy update\r\n``` \r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"AmortizedCosts_raw\",\r\n \"Query\": \"AmortizedCosts_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| CommitmentDiscountUsage |========================================================================================\r\n// Supported versions:\r\n// - MS EA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-ea\r\n// - MS MCA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-mca\r\n//======================================================================================================================\r\n\r\n// CommitmentDiscountUsage_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='All commitment discount usage transformed to FOCUS 1.2. This includes reservationdeatils_raw.', folder='Commitment discounts')\r\nCommitmentDiscountUsage_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n CommitmentDiscountUsage_raw\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Handle resource columns\r\n | extend ResourceId = tolower(InstanceId)\r\n | extend tmp_ResourceDetails = parse_resourceid(ResourceId)\r\n | extend ResourceName = tostring(tmp_ResourceDetails.ResourceName)\r\n | extend SubAccountId = tostring(tmp_ResourceDetails.SubAccountId)\r\n | extend x_ResourceGroupName = tostring(tmp_ResourceDetails.x_ResourceGroupName)\r\n | extend x_ResourceType = tostring(tmp_ResourceDetails.x_ResourceType)\r\n | lookup kind=leftouter (ResourceTypes | distinct x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n | lookup kind=leftouter (Services | distinct x_ResourceType, ServiceName, ServiceCategory, ServiceSubcategory, x_ServiceModel) on x_ResourceType\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n ChargePeriodEnd = UsageDate + 1d,\r\n ChargePeriodStart = UsageDate,\r\n CommitmentDiscountCategory = 'Usage',\r\n CommitmentDiscountId = tolower(strcat('/providers/microsoft.capacity/reservationorders/', ReservationOrderId, '/reservations/', ReservationId)),\r\n CommitmentDiscountQuantity = UsedHours * InstanceFlexibilityRatio,\r\n CommitmentDiscountType = 'Reservation',\r\n CommitmentDiscountUnit = case(\r\n InstanceFlexibilityRatio == 1, 'Hours',\r\n InstanceFlexibilityRatio != 1, 'Normalized Hours',\r\n ''\r\n ),\r\n ConsumedQuantity = UsedHours,\r\n ProviderName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory,\r\n SubAccountId,\r\n x_CommitmentDiscountCommittedCount = TotalReservedQuantity,\r\n x_CommitmentDiscountCommittedAmount = ReservedHours,\r\n // TODO: Is this needed? -- x_CommitmentDiscountKind = Kind,\r\n x_CommitmentDiscountNormalizedGroup = iff(InstanceFlexibilityGroup == 'NA', '', InstanceFlexibilityGroup),\r\n x_CommitmentDiscountNormalizedRatio = InstanceFlexibilityRatio,\r\n x_IngestionTime = ingestion_time(),\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n // x_RowId = hash_sha256(strcat(\r\n // // DO NOT CHANGE COLUMNS OR COLUMN ORDER\r\n // CommitmentDiscountId,\r\n // ResourceId,\r\n // ChargePeriodStart\r\n // )),\r\n x_ServiceModel,\r\n x_SkuOrderId = ReservationOrderId,\r\n x_SkuSize = iff(SkuName == 'NA', '', SkuName),\r\n x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName)),\r\n x_SourceProvider = coalesce(x_SourceProvider, ProviderName),\r\n x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationDetails', '')),\r\n x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2024-03-01', ''))\r\n}\r\n\r\n// CommitmentDiscountUsage_final_v1_2 table\r\n.create-merge table CommitmentDiscountUsage_final_v1_2 (\r\n ChargePeriodEnd: datetime, // Hubs add-on\r\n ChargePeriodStart: datetime, // MS 2023-03-01\r\n CommitmentDiscountCategory: string, // Hubs add-on\r\n CommitmentDiscountId: string, // MS 2023-03-01\r\n CommitmentDiscountQuantity: real, // MS 2023-03-01\r\n CommitmentDiscountType: string, // Hubs add-on\r\n CommitmentDiscountUnit: string, // Hubs add-on\r\n ConsumedQuantity: real, // MS 2023-03-01\r\n ProviderName: string, // Hubs add-on\r\n ResourceId: string, // MS 2023-03-01\r\n ResourceName: string, // Hubs add-on\r\n ResourceType: string, // Hubs add-on\r\n ServiceCategory: string, // Hubs add-on\r\n ServiceName: string, // Hubs add-on\r\n ServiceSubcategory: string, // Hubs add-on\r\n SubAccountId: string, // Hubs add-on\r\n x_CommitmentDiscountCommittedCount: real, // MS 2023-03-01\r\n x_CommitmentDiscountCommittedAmount: real, // MS 2023-03-01\r\n x_CommitmentDiscountNormalizedGroup: string, // MS 2023-03-01\r\n x_CommitmentDiscountNormalizedRatio: real, // MS 2023-03-01\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_ResourceGroupName: string, // Hubs add-on\r\n x_ResourceType: string, // Hubs add-on\r\n x_ServiceModel: string, // Hubs add-on\r\n x_SkuOrderId: string, // MS 2023-03-01\r\n x_SkuSize: string, // MS 2023-03-01\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string // Hubs add-on\r\n)\r\n\r\n// Update policy for CommitmentDiscountUsage_raw -> CommitmentDiscountUsage_final_v1_2 table\r\n.alter table CommitmentDiscountUsage_final_v1_2 policy update\r\n```\r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"CommitmentDiscountUsage_raw\",\r\n \"Query\": \"CommitmentDiscountUsage_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Recommendations |================================================================================================\r\n// Supported datasets/versions:\r\n// - MS CM EA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-ea\r\n// - MS CM MCA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-mca\r\n//======================================================================================================================\r\n\r\n// Recommendations_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='All recommendations transformed to FOCUS 1.2.', folder='Recommendations')\r\nRecommendations_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', int(null),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n Recommendations_raw\r\n | extend x_IngestionTime = ingestion_time()\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Set source columns\r\n | extend x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationRecommendations', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2023-05-01', ''))\r\n //\r\n // Convert JSON cost columns to real\r\n | extend CostWithNoReservedInstances = case(isnotempty(CostWithNoReservedInstances), CostWithNoReservedInstances, isnotempty(CostWithNoReservedInstancesJson), toreal(extract(@'\"value\":([0-9\\.]+)', 1, CostWithNoReservedInstancesJson)), CostWithNoReservedInstances)\r\n | extend NetSavings = case(isnotempty(NetSavings), NetSavings, isnotempty(NetSavingsJson), toreal(extract(@'\"value\":([0-9\\.]+)', 1, NetSavingsJson)), NetSavings)\r\n | extend TotalCostWithReservedInstances = case(isnotempty(TotalCostWithReservedInstances), TotalCostWithReservedInstances, isnotempty(TotalCostWithReservedInstancesJson), toreal(extract(@'\"value\":([0-9\\.]+)', 1, TotalCostWithReservedInstancesJson)), TotalCostWithReservedInstances)\r\n //\r\n // Build recommendation details\r\n | lookup kind=leftouter (Regions | summarize RegionName = make_set(RegionName)[0] by Location = RegionId) on Location\r\n | extend x_RecommendationDetails = case(\r\n // Use incoming x_RecommendationDetails first\r\n isnotempty(x_RecommendationDetails), x_RecommendationDetails,\r\n // Create one for reservation recommendations if needed\r\n x_SourceType == 'ReservationRecommendations', bag_pack(\r\n 'CommitmentDiscountNormalizedGroup', InstanceFlexibilityGroup,\r\n 'CommitmentDiscountNormalizedRatio', InstanceFlexibilityRatio,\r\n 'CommitmentDiscountNormalizedSize', NormalizedSize,\r\n 'CommitmentDiscountResourceType', ResourceType,\r\n 'CommitmentDiscountScope', Scope,\r\n 'LookbackPeriodDuration', case(\r\n LookBackPeriod matches regex @'^Last([0-9]+)Days$', replace_regex(LookBackPeriod, @'^Last([0-9]+)Days$', @'P\\1D'),\r\n LookBackPeriod matches regex @'^[0-9]+$', strcat('P', LookBackPeriod, 'D'),\r\n ''\r\n ),\r\n 'LookbackPeriodStart', FirstUsageDate,\r\n 'RecommendedQuantity', RecommendedQuantity,\r\n 'RecommendedQuantityNormalized', RecommendedQuantityNormalized,\r\n 'RegionId', Location,\r\n 'RegionName', RegionName,\r\n 'SkuMeterId', MeterId,\r\n 'SkuPriceDetails', SkuProperties,\r\n 'SkuSize', coalesce(SKU, SkuName),\r\n 'SkuTerm', isoMonths(Term)\r\n ),\r\n dynamic({})\r\n )\r\n //\r\n // Prefer specified date, then fall back to generating a date based on reservation recommendation lookback period, then validate to ensure it's not in the future\r\n | extend x_RecommendationDate = coalesce(x_RecommendationDate, FirstUsageDate + (toint(extract(@'^P([0-9]+)D$', 1, tostring(x_RecommendationDetails.LookbackPeriodDuration))) * 1d))\r\n | extend x_RecommendationDate = iff(x_RecommendationDate > now(), startofday(now()), x_RecommendationDate)\r\n //\r\n | project\r\n ProviderName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n SubAccountId = coalesce(SubAccountId, iff(isnotempty(SubscriptionId), strcat('/subscriptions/', SubscriptionId), '')),\r\n SubAccountName,\r\n x_EffectiveCostAfter = coalesce(x_EffectiveCostAfter, TotalCostWithReservedInstances),\r\n x_EffectiveCostBefore = coalesce(x_EffectiveCostBefore, CostWithNoReservedInstances),\r\n x_EffectiveCostSavings = coalesce(x_EffectiveCostSavings, NetSavings),\r\n x_IngestionTime,\r\n x_RecommendationCategory, // TODO: Set for reservation recommendations\r\n x_RecommendationDate,\r\n x_RecommendationDescription,\r\n x_RecommendationDetails,\r\n x_RecommendationId, // TODO: Set for reservation recommendations\r\n x_ResourceGroupName,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n// Recommendations_final_v1_2 table\r\n.create-merge table Recommendations_final_v1_2 (\r\n ProviderName: string,\r\n ResourceId: string,\r\n ResourceName: string,\r\n ResourceType: string,\r\n SubAccountId: string,\r\n SubAccountName: string,\r\n x_EffectiveCostAfter: real,\r\n x_EffectiveCostBefore: real,\r\n x_EffectiveCostSavings: real,\r\n x_IngestionTime: datetime,\r\n x_RecommendationCategory: string,\r\n x_RecommendationDate: datetime,\r\n x_RecommendationDescription: string,\r\n x_RecommendationDetails: dynamic,\r\n x_RecommendationId: string,\r\n x_ResourceGroupName: string,\r\n x_SourceName: string,\r\n x_SourceProvider: string,\r\n x_SourceType: string,\r\n x_SourceVersion: string\r\n)\r\n\r\n// Update policy for Recommendations_raw -> Recommendations_final_v1_2 table\r\n.alter table Recommendations_final_v1_2 policy update\r\n```\r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"Recommendations_raw\",\r\n \"Query\": \"Recommendations_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Transactions |===================================================================================================\r\n// Supported versions:\r\n// - MS CM EA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-ea\r\n// - MS CM MCA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-mca\r\n//======================================================================================================================\r\n\r\n// Transactions_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='All transactions transformed to FOCUS 1.2.', folder='Transactions')\r\nTransactions_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', int(null),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n Transactions_raw\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Set source columns\r\n | extend x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationTransactions', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2023-05-01', ''))\r\n //\r\n // Handle BillingPeriodStart/End\r\n | extend BillingMonth = tostring(BillingMonth)\r\n | extend BillingPeriodStart = iff(isempty(BillingMonth), datetime(null), todatetime(strcat(substring(BillingMonth, 0, 4), \"-\", substring(BillingMonth, 4, 2), \"-\", substring(BillingMonth, 6, 2))))\r\n | extend BillingPeriodEnd = iff(isempty(BillingMonth), datetime(null), startofmonth(endofmonth(BillingPeriodStart) + 1d))\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n BilledCost = Amount,\r\n BillingAccountId = case(\r\n BillingProfileId startswith '/', BillingProfileId,\r\n isnotempty(CurrentEnrollmentId), strcat('/providers/Microsoft.Billing/billingAccounts/', CurrentEnrollmentId),\r\n isnotempty(BillingProfileId), strcat('/providers/Microsoft.Billing/billingProfiles/', BillingProfileId),\r\n ''\r\n ),\r\n BillingAccountName = coalesce(BillingProfileName, CurrentEnrollmentId),\r\n BillingCurrency = Currency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n ChargeCategory = case(\r\n EventType in ('Cancel', 'Purchase', 'Refund'), 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = case(\r\n EventType == 'Cancel', 'Cancel', // FOCUS does not handle this scenario\r\n EventType == 'Refund', 'Correction',\r\n ''\r\n ),\r\n ChargeDescription = Description,\r\n ChargeFrequency = case(\r\n BillingFrequency == 'OneTime', 'One-Time',\r\n BillingFrequency == 'Recurring', 'Recurring',\r\n BillingFrequency\r\n ),\r\n ChargePeriodStart = EventDate,\r\n InvoiceId,\r\n PricingQuantity = Quantity,\r\n PricingUnit = 'Reservations',\r\n ProviderName,\r\n RegionId = Region,\r\n RegionName = Region,\r\n SubAccountId = iff(isempty(PurchasingSubscriptionGuid), '', strcat('/subscriptions/', PurchasingSubscriptionGuid)),\r\n SubAccountName = iff(isempty(PurchasingSubscriptionGuid), '', PurchasingSubscriptionName),\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerEmail,\r\n x_CostCenter = CostCenter,\r\n x_InvoiceNumber = Invoice,\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = coalesce(InvoiceSectionName, DepartmentName),\r\n x_IngestionTime = ingestion_time(),\r\n x_MonetaryCommitment = MonetaryCommitment,\r\n x_Overage = Overage,\r\n x_PurchasingBillingAccountId = PurchasingEnrollment,\r\n x_SkuOrderId = ReservationOrderId,\r\n x_SkuOrderName = ReservationOrderName,\r\n x_SkuSize = ArmSkuName,\r\n x_SkuTerm = isoMonths(Term),\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_SubscriptionId = PurchasingSubscriptionGuid,\r\n x_TransactionType = EventType\r\n}\r\n\r\n// Transactions_final_v1_2 table\r\n.create-merge table Transactions_final_v1_2 (\r\n BilledCost: real, // MS CM EA+MCA 2023-05-01\r\n BillingAccountId: string, // MS CM EA+MCA 2023-05-01\r\n BillingAccountName: string, // MS CM EA+MCA 2023-05-01\r\n BillingCurrency: string, // MS CM EA+MCA 2023-05-01\r\n BillingPeriodEnd: datetime, // MS CM EA+MCA 2023-05-01\r\n BillingPeriodStart: datetime, // MS CM EA+MCA 2023-05-01\r\n ChargeCategory: string, // Hubs add-on\r\n ChargeClass: string, // Hubs add-on\r\n ChargeDescription: string, // MS CM EA+MCA 2023-05-01\r\n ChargeFrequency: string, // MS CM EA+MCA 2023-05-01\r\n ChargePeriodStart: datetime, // MS CM EA+MCA 2023-05-01\r\n InvoiceId: string, // MS CM MCA 2023-05-01\r\n PricingQuantity: real, // MS CM EA+MCA 2023-05-01\r\n PricingUnit: string, // Hubs add-on\r\n ProviderName: string, // Hubs add-on\r\n RegionId: string, // MS CM EA+MCA 2023-05-01\r\n RegionName: string, // MS CM EA+MCA 2023-05-01\r\n SubAccountId: string, // MS CM EA+MCA 2023-05-01\r\n SubAccountName: string, // MS CM EA+MCA 2023-05-01\r\n x_AccountName: string, // MS CM EA 2023-05-01\r\n x_AccountOwnerId: string, // MS CM EA 2023-05-01\r\n x_CostCenter: string, // MS CM EA 2023-05-01\r\n x_InvoiceNumber: string, // MS CM MCA 2023-05-01\r\n x_InvoiceSectionId: string, // MS CM MCA 2023-05-01\r\n x_InvoiceSectionName: string, // MS CM MCA 2023-05-01\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_MonetaryCommitment: real, // MS CM EA 2023-05-01\r\n x_Overage: real, // MS CM EA 2023-05-01\r\n x_PurchasingBillingAccountId: string, // MS CM EA 2023-05-01\r\n x_SkuOrderId: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuOrderName: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuSize: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuTerm: int, // MS CM EA+MCA 2023-05-01\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_SubscriptionId: string, // MS CM EA+MCA 2023-05-01\r\n x_TransactionType: string // MS CM EA+MCA 2023-05-01\r\n)\r\n\r\n// Update policy for Transactions_raw -> Transactions_final_v1_2 table\r\n.alter table Transactions_final_v1_2 policy update\r\n```\r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"Transactions_raw\",\r\n \"Query\": \"Transactions_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n", - "$fxv#11": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Common utility functions\r\n//\r\n// TIP: Use Ctrl+K,Ctrl+0 to collapse all regions in VS Code\r\n//======================================================================================================================\r\n\r\n\r\n//===| Date functions |=================================================================================================\r\n\r\n// monthstring\r\n.create-or-alter function \r\nwith (docstring = @'Returns the name of the month for the specified date (e.g. Jan or January)', folder =@'Common') \r\nmonthstring(['date']: datetime, length: int = 9)\r\n{\r\n substring(dynamic(['January','February','March','April','May','June','July','August','September','October','November','December'])[getmonth(['date']) - 1], 0, length)\r\n}\r\n\r\n// datestring\r\n.create-or-alter function \r\nwith (docstring = @'Converts 2 dates into a simple, user-friendly date range (e.g. Jan 1-Jan 3)', folder =@'Common') \r\ndatestring(start: datetime, end: datetime = datetime('0001-01-01'))\r\n{\r\n let month = (d: datetime) { monthstring(d, 3) };\r\n let endDate = iff(end == datetime('0001-01-01'), start, end);\r\n let sameDate = startofday(start) == startofday(endDate);\r\n let sameMonth = startofmonth(start) == startofmonth(endDate);\r\n let sameYear = startofyear(start) == startofyear(endDate);\r\n let fullMonth = startofday(start) == startofmonth(start) and startofday(endDate) == startofday(endofmonth(endDate));\r\n let fullYear = startofday(start) == startofyear(start) and startofday(endDate) == startofday(endofyear(endDate));\r\n let currentYear = sameYear and startofyear(start) == startofyear(now());\r\n case(\r\n // Full year | yyyy (same year) / yyyy-yyyy (diff years)\r\n fullYear,\r\n strcat(getyear(start), iff(sameYear, '', strcat('-', getyear(endDate)))),\r\n // 1 full mo, same year | Mmm yyyy\r\n fullMonth and sameMonth and sameYear,\r\n strcat(month(start), ' ', getyear(start)),\r\n // 2+ full mo, same year | Mmm-Mmm (current year) / Mmm-Mmm yyyy (other year)\r\n fullMonth and sameYear,\r\n strcat(month(start), '-', month(endDate), iff(currentYear, '', strcat(' ', getyear(endDate)))),\r\n // Full mo, diff year | Mmm yyyy-Mmm yyyy\r\n fullMonth and not(sameYear),\r\n strcat(month(start), ' ', getyear(start), '-', month(endDate), ' ', getyear(endDate)),\r\n // Same date | Mmm d (current year) / Mmm d, yyyy (other year)\r\n sameDate,\r\n strcat(month(start), ' ', dayofmonth(start), iff(currentYear, '', strcat(', ', getyear(endDate)))),\r\n // 1 partial M, same Y | Mmm d-d (current year) / Mmm d-d, yyyy (other year)\r\n not(fullMonth) and sameMonth and sameYear,\r\n strcat(month(start), ' ', dayofmonth(start), '-', dayofmonth(endDate), iff(currentYear, '', strcat(' ', getyear(endDate)))),\r\n // 2+ partial M, same Y | Mmm d-Mmm d (current year) / Mmm d-Mmm d, yyyy (other year)\r\n not(fullMonth) and not(sameMonth) and sameYear,\r\n strcat(month(start), ' ', dayofmonth(start), '-', month(endDate), ' ', dayofmonth(endDate), iff(currentYear, '', strcat(', ', getyear(endDate)))),\r\n // All other cases | Mmm d, yyyy-Mmm d, yyyy\r\n strcat(month(start), ' ', dayofmonth(start), ', ', getyear(start), '-', month(endDate), ' ', dayofmonth(endDate), ', ', getyear(endDate))\r\n )\r\n}\r\n\r\n// daterange\r\n.create-or-alter function \r\nwith (docstring = @'DEPRECATED: Please use datestring(); function will be removed on or after the Jan 2026 release', folder =@'Common') \r\ndaterange(start: datetime, end: datetime = datetime('0001-01-01'))\r\n{\r\n datestring(start, end)\r\n}\r\n\r\n// monthsago\r\n.create-or-alter function \r\nwith (docstring = 'DEPRECATED: Please use startofmonth(now(), -<# of months>); function will be removed on or after the Jan 2026 release', folder = 'Common')\r\nmonthsago(months: int)\r\n{\r\n datetime_add('month', -months, startofmonth(now()))\r\n}\r\n\r\n\r\n//===| Number functions |===============================================================================================\r\n// NOTE: Must be defined before string converters\r\n\r\n// delta\r\n.create-or-alter function \r\nwith (docstring = @'Compares 2 values and returns the percentage change from oldval to newval', folder =@'Common') \r\ndelta(oldval: double, newval: double)\r\n{\r\n (newval - todouble(oldval))/oldval\r\n}\r\n\r\n// percentOfTotal\r\n// NOTE: Must be before percent() function\r\n.create-or-alter function \r\nwith (docstring = @'Calculates the percentage of each record based on a required Count column', folder =@'Common') \r\npercentOfTotal(t: (Count: long), tot: long)\r\n{\r\n let total = todouble(tot);\r\n t \r\n | extend Percent = round(Count / total * 100, 3) \r\n | order by Count desc\r\n}\r\n\r\n// percent\r\n.create-or-alter function \r\nwith (docstring = @'Calculates the percentage of each record based on a required Count column', folder =@'Common') \r\npercent(t: (Count: long))\r\n{\r\n let total = todouble(toscalar(t | summarize sum(Count)));\r\n percentOfTotal(t, total)\r\n}\r\n\r\n// plusminus\r\n.create-or-alter function \r\nwith (docstring = 'Shows a +/- sign based on the direction of the number', folder = 'Common')\r\nplusminus(val: string)\r\n{\r\n let neg = substring(val, 0, 1) == '-';\r\n iff(neg, val, strcat('+', val))\r\n}\r\n\r\n// updown\r\n.create-or-alter function \r\nwith (docstring = 'Shows an up/down arrow based on the direction of the number', folder = 'Common')\r\nupdown(val: string)\r\n{\r\n // TODO: Handle 0\r\n let neg = substring(val, 0, 1) == '-';\r\n iff(neg, strcat('↓', substring(val, 1)), strcat('↑', val))\r\n}\r\n\r\n\r\n//===| String functions |===============================================================================================\r\n\r\n// percentstring\r\n// NOTE: Must be defined before deltastring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate a percentage and render as a string', folder = 'Common')\r\npercentstring(num: double, total: double = 1.0, places: int = 9)\r\n{\r\n let value = 1.0 * num / total * 100;\r\n strcat(case(\r\n places != 9, round(value, places),\r\n value < 10, round(value, 2),\r\n round(value, 1)\r\n ), '%')\r\n}\r\n\r\n//----------------------------------------------------------------------------------------------------------------------\r\n\r\n// arraystring\r\n.create-or-alter function \r\nwith (docstring = 'Convert an array to a comma-delimited string', folder = 'Common')\r\narraystring(arr: dynamic)\r\n{\r\n replace_string(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(\r\n tostring(arr)\r\n , @'^\\[\"', '')\r\n , @'\"\\]$', '')\r\n , @'^, ', '')\r\n , @', $', '')\r\n , @'^\\[]$', '')\r\n , '\",\"', ', ')\r\n}\r\n\r\n// deltastring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate a delta percentage and render as a string', folder = 'Common')\r\ndeltastring(oldval: double, newval: double, places: int = 1, useArrows: bool = false)\r\n{\r\n let d = delta(oldval, newval);\r\n strcat(case(useArrows and d > 0, '↑', useArrows and d < 0, '↓', d < 0, '-', ''), percentstring(abs(d), 1, places))\r\n}\r\n\r\n// diffstring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate the difference and render as a string', folder = 'Common')\r\ndiffstring(oldval: double, newval: double, places: int = 1)\r\n{\r\n plusminus(round(newval - oldval, places))\r\n}\r\n\r\n// numberstring\r\n.create-or-alter function \r\nwith (docstring = 'Convert a number to a string', folder = 'Common')\r\nnumberstring(num: double, abbrev: bool = true)\r\n{\r\n replace_regex(case(\r\n num >= 10000000000000, strcat(round(1.0 * num / 1000000000000, 1), 'T'),\r\n num >= 1000000000000, strcat(round(1.0 * num / 1000000000000, 2), 'T'),\r\n num >= 10000000000, strcat(round(1.0 * num / 1000000000, 1), 'B'),\r\n num >= 1000000000, strcat(round(1.0 * num / 1000000000, 2), 'B'),\r\n num >= 10000000, strcat(round(1.0 * num / 1000000, 1), 'M'),\r\n num >= 1000000, strcat(round(1.0 * num / 1000000, 2), 'M'),\r\n num >= 10000, strcat(round(1.0 * num / 1000, 1), 'K'),\r\n // Kusto doesn't support back-refs yet -- num > 1000, replace_regex(tostring(num), @'(\\d)(?=(\\d{3})+\\.)', @'\\1,'), // See https://docs.microsoft.com/azure/data-explorer/kusto/query/re2-library\r\n num > 1000, replace_regex(tostring(num), @'([0-9]{3})$', @',\\1'), //num / 1000, ',', substring(tostring(num), 0) - (num / 1000 * 1000)),\r\n tostring(num)\r\n ), @'\\.0$', '')\r\n}\r\n\r\n\r\n//===| Other |==========================================================================================================\r\n\r\n// ifempty\r\n.create-or-alter function \r\nwith (docstring = 'Replaces an empty value with the specified default value', folder = 'Common')\r\nifempty(val: dynamic, defaultVal: dynamic)\r\n{\r\n iff(isempty(val), defaultVal, val)\r\n}\r\n", - "$fxv#12": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Hub database / Open data functions\r\n// Wrap Ingestion database tables for easy access.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n\r\n// PricingUnits\r\n.create-or-alter function\r\nwith (docstring = 'Gets pricing units from the FinOps toolkit PricingUnits open data.', folder = 'OpenData')\r\nPricingUnits()\r\n{\r\n database('Ingestion').PricingUnits\r\n}\r\n\r\n// Regions\r\n.create-or-alter function\r\nwith (docstring = 'Gets regions from the FinOps toolkit Regions open data.', folder = 'OpenData')\r\nRegion()\r\n{\r\n database('Ingestion').Regions\r\n}\r\n\r\n// ResourceTypes\r\n.create-or-alter function\r\nwith (docstring = 'Gets resource types from the FinOps toolkit ResourceTypes open data.', folder = 'OpenData')\r\nResourceType()\r\n{\r\n database('Ingestion').ResourceTypes\r\n}\r\n\r\n// Services\r\n.create-or-alter function\r\nwith (docstring = 'Gets services from the FinOps toolkit Services open data.', folder = 'OpenData')\r\nServices()\r\n{\r\n database('Ingestion').Services\r\n}\r\n", - "$fxv#13": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Hub database / FOCUS 1.0 functions\r\n// Used for reporting with backward compatibility.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n\r\n// CommitmentDiscountUsage_final_v1_0\r\n.create-or-alter function\r\nwith (docstring = 'Gets all commitment discount usage records aligned to FOCUS 1.0.', folder = 'CommitmentDiscountUsage')\r\nCommitmentDiscountUsage_v1_0()\r\n{\r\n database('Ingestion').CommitmentDiscountUsage_final_v1_0\r\n | union (\r\n database('Ingestion').CommitmentDiscountUsage_final_v1_2\r\n // Convert real to decimal\r\n | extend\r\n CommitmentDiscountQuantity = todecimal(CommitmentDiscountQuantity),\r\n ConsumedQuantity = todecimal(ConsumedQuantity),\r\n x_CommitmentDiscountCommittedCount = todecimal(x_CommitmentDiscountCommittedCount),\r\n x_CommitmentDiscountCommittedAmount = todecimal(x_CommitmentDiscountCommittedAmount),\r\n x_CommitmentDiscountNormalizedRatio = todecimal(x_CommitmentDiscountNormalizedRatio)\r\n )\r\n | project\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId,\r\n CommitmentDiscountType,\r\n ConsumedQuantity,\r\n ProviderName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n SubAccountId,\r\n x_CommitmentDiscountCommittedCount,\r\n x_CommitmentDiscountCommittedAmount,\r\n x_CommitmentDiscountNormalizedGroup,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_CommitmentDiscountQuantity,\r\n x_IngestionTime,\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n x_ServiceModel,\r\n x_SkuOrderId,\r\n x_SkuSize,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n\r\n// Costs_final_v1_0\r\n.create-or-alter function\r\nwith (docstring = 'Gets all cost and usage records aligned to FOCUS 1.0.', folder = 'Costs')\r\nCosts_v1_0()\r\n{\r\n database('Ingestion').Costs_final_v1_0\r\n | union (\r\n database('Ingestion').Costs_final_v1_2\r\n // Convert real to decimal\r\n | extend\r\n BilledCost = todecimal(BilledCost),\r\n CommitmentDiscountQuantity = todecimal(CommitmentDiscountQuantity),\r\n ConsumedQuantity = todecimal(ConsumedQuantity),\r\n ContractedCost = todecimal(ContractedCost),\r\n ContractedUnitPrice = todecimal(ContractedUnitPrice),\r\n EffectiveCost = todecimal(EffectiveCost),\r\n ListCost = todecimal(ListCost),\r\n ListUnitPrice = todecimal(ListUnitPrice),\r\n PricingQuantity = todecimal(PricingQuantity),\r\n x_BilledCostInUsd = todecimal(x_BilledCostInUsd),\r\n x_BilledUnitPrice = todecimal(x_BilledUnitPrice),\r\n x_BillingExchangeRate = todecimal(x_BillingExchangeRate),\r\n x_CommitmentDiscountNormalizedRatio = todecimal(x_CommitmentDiscountNormalizedRatio),\r\n x_ContractedCostInUsd = todecimal(x_ContractedCostInUsd),\r\n x_CurrencyConversionRate = todecimal(x_CurrencyConversionRate),\r\n x_EffectiveCostInUsd = todecimal(x_EffectiveCostInUsd),\r\n x_EffectiveUnitPrice = todecimal(x_EffectiveUnitPrice),\r\n x_ListCostInUsd = todecimal(x_ListCostInUsd),\r\n x_PricingBlockSize = todecimal(x_PricingBlockSize)\r\n // Rename columns\r\n | project-rename\r\n x_InvoiceId = InvoiceId,\r\n x_PricingCurrency = PricingCurrency,\r\n x_SkuMeterName = SkuMeter\r\n // Generate historical x_SkuDetails format from SkuPriceDetails\r\n | extend x_SkuDetails = iff(isnotempty(x_SkuDetails), x_SkuDetails, parse_json(replace_regex(tostring(SkuPriceDetails), @'([\\{,])\"x_', @'\\1\"')))\r\n )\r\n | project\r\n AvailabilityZone,\r\n BilledCost,\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingAccountType,\r\n BillingCurrency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n ChargeCategory,\r\n ChargeClass,\r\n ChargeDescription,\r\n ChargeFrequency,\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId,\r\n CommitmentDiscountName,\r\n CommitmentDiscountStatus,\r\n CommitmentDiscountType,\r\n ConsumedQuantity,\r\n ConsumedUnit,\r\n ContractedCost,\r\n ContractedUnitPrice,\r\n EffectiveCost,\r\n InvoiceIssuerName,\r\n ListCost,\r\n ListUnitPrice,\r\n PricingCategory,\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n PublisherName,\r\n RegionId,\r\n RegionName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n SkuId,\r\n SkuPriceId,\r\n SubAccountId,\r\n SubAccountName,\r\n SubAccountType,\r\n Tags,\r\n x_AccountId,\r\n x_AccountName,\r\n x_AccountOwnerId,\r\n x_BilledCostInUsd,\r\n x_BilledUnitPrice,\r\n x_BillingAccountAgreement,\r\n x_BillingAccountId,\r\n x_BillingAccountName,\r\n x_BillingExchangeRate,\r\n x_BillingExchangeRateDate,\r\n x_BillingProfileId,\r\n x_BillingProfileName,\r\n x_ChargeId,\r\n x_ContractedCostInUsd,\r\n x_CostAllocationRuleName,\r\n x_CostCategories,\r\n x_CostCenter,\r\n x_Credits,\r\n x_CostType,\r\n x_CurrencyConversionRate,\r\n x_CustomerId,\r\n x_CustomerName,\r\n x_Discount,\r\n x_EffectiveCostInUsd,\r\n x_EffectiveUnitPrice,\r\n x_ExportTime,\r\n x_IngestionTime,\r\n x_InvoiceId,\r\n x_InvoiceIssuerId,\r\n x_InvoiceSectionId,\r\n x_InvoiceSectionName,\r\n x_ListCostInUsd,\r\n x_Location,\r\n x_Operation,\r\n x_PartnerCreditApplied,\r\n x_PartnerCreditRate,\r\n x_PricingBlockSize,\r\n x_PricingCurrency,\r\n x_PricingSubcategory,\r\n x_PricingUnitDescription,\r\n x_Project,\r\n x_PublisherCategory,\r\n x_PublisherId,\r\n x_ResellerId,\r\n x_ResellerName,\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n x_ServiceCode,\r\n x_ServiceId,\r\n x_ServicePeriodEnd,\r\n x_ServicePeriodStart,\r\n x_SkuDescription,\r\n x_SkuDetails,\r\n x_SkuIsCreditEligible,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterName,\r\n x_SkuMeterSubcategory,\r\n x_SkuOfferId,\r\n x_SkuOrderId,\r\n x_SkuOrderName,\r\n x_SkuPartNumber,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceChanges,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_UsageType\r\n}\r\n\r\n\r\n// Prices_final_v1_0\r\n.create-or-alter function\r\nwith (docstring = 'Gets all prices aligned to FOCUS 1.0.', folder = 'Prices')\r\nPrices_v1_0()\r\n{\r\n database('Ingestion').Prices_final_v1_0\r\n | union (\r\n database('Ingestion').Prices_final_v1_2\r\n // Convert real to decimal\r\n | extend\r\n ContractedUnitPrice = todecimal(ContractedUnitPrice),\r\n ListUnitPrice = todecimal(ListUnitPrice),\r\n x_BaseUnitPrice = todecimal(x_BaseUnitPrice),\r\n x_CommitmentDiscountNormalizedRatio = todecimal(x_CommitmentDiscountNormalizedRatio),\r\n x_ContractedUnitPriceDiscount = todecimal(x_ContractedUnitPriceDiscount),\r\n x_ContractedUnitPriceDiscountPercent = todecimal(x_ContractedUnitPriceDiscountPercent),\r\n x_EffectiveUnitPrice = todecimal(x_EffectiveUnitPrice),\r\n x_EffectiveUnitPriceDiscount = todecimal(x_EffectiveUnitPriceDiscount),\r\n x_EffectiveUnitPriceDiscountPercent = todecimal(x_EffectiveUnitPriceDiscountPercent),\r\n x_PricingBlockSize = todecimal(x_PricingBlockSize),\r\n x_SkuIncludedQuantity = todecimal(x_SkuIncludedQuantity),\r\n x_SkuTier = todecimal(x_SkuTier),\r\n x_TotalUnitPriceDiscount = todecimal(x_TotalUnitPriceDiscount),\r\n x_TotalUnitPriceDiscountPercent = todecimal(x_TotalUnitPriceDiscountPercent) \r\n // Rename columns\r\n | project-rename\r\n x_PricingCurrency = PricingCurrency,\r\n x_SkuMeterName = SkuMeter\r\n )\r\n | project\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingCurrency,\r\n ChargeCategory,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountType,\r\n ContractedUnitPrice,\r\n ListUnitPrice,\r\n PricingCategory,\r\n PricingUnit,\r\n SkuId,\r\n SkuPriceId,\r\n SkuPriceIdv2,\r\n x_BaseUnitPrice,\r\n x_BillingAccountAgreement,\r\n x_BillingAccountId,\r\n x_BillingProfileId,\r\n x_CommitmentDiscountSpendEligibility,\r\n x_CommitmentDiscountUsageEligibility,\r\n x_ContractedUnitPriceDiscount,\r\n x_ContractedUnitPriceDiscountPercent,\r\n x_EffectivePeriodEnd,\r\n x_EffectivePeriodStart,\r\n x_EffectiveUnitPrice,\r\n x_EffectiveUnitPriceDiscount,\r\n x_EffectiveUnitPriceDiscountPercent,\r\n x_IngestionTime,\r\n x_PricingBlockSize,\r\n x_PricingCurrency,\r\n x_PricingSubcategory,\r\n x_PricingUnitDescription,\r\n x_SkuDescription,\r\n x_SkuId,\r\n x_SkuIncludedQuantity,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterName,\r\n x_SkuMeterSubcategory,\r\n x_SkuMeterType,\r\n x_SkuPriceType,\r\n x_SkuProductId,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuOfferId,\r\n x_SkuPartNumber,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_TotalUnitPriceDiscount,\r\n x_TotalUnitPriceDiscountPercent\r\n}\r\n\r\n\r\n// Recommendations_final_v1_0\r\n.create-or-alter function\r\nwith (docstring = 'Gets all recommendations aligned to FOCUS 1.0.', folder = 'Recommendations')\r\nRecommendations_v1_0()\r\n{\r\n database('Ingestion').Recommendations_final_v1_0\r\n | union (\r\n database('Ingestion').Recommendations_final_v1_2\r\n // Convert real to decimal\r\n | extend\r\n x_EffectiveCostAfter = todecimal(x_EffectiveCostAfter),\r\n x_EffectiveCostBefore = todecimal(x_EffectiveCostBefore),\r\n x_EffectiveCostSavings = todecimal(x_EffectiveCostSavings)\r\n )\r\n | project\r\n ProviderName,\r\n SubAccountId,\r\n x_IngestionTime,\r\n x_EffectiveCostAfter,\r\n x_EffectiveCostBefore,\r\n x_EffectiveCostSavings,\r\n x_RecommendationDate,\r\n x_RecommendationDetails,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n\r\n// Transactions_final_v1_0\r\n.create-or-alter function\r\nwith (docstring = 'Gets all transactions aligned to FOCUS 1.0.', folder = 'Transactions')\r\nTransactions_v1_0()\r\n{\r\n database('Ingestion').Transactions_final_v1_0\r\n | union (\r\n database('Ingestion').Transactions_final_v1_2\r\n // Convert real to decimal\r\n | extend\r\n BilledCost = todecimal(BilledCost),\r\n PricingQuantity = todecimal(PricingQuantity),\r\n x_MonetaryCommitment = todecimal(x_MonetaryCommitment),\r\n x_Overage = todecimal(x_Overage)\r\n // Rename columns\r\n | project-rename\r\n x_InvoiceId = InvoiceId\r\n )\r\n | project\r\n BilledCost,\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingCurrency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n ChargeCategory,\r\n ChargeClass,\r\n ChargeDescription,\r\n ChargeFrequency,\r\n ChargePeriodStart,\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n RegionId,\r\n RegionName,\r\n SubAccountId,\r\n SubAccountName,\r\n x_AccountName,\r\n x_AccountOwnerId,\r\n x_CostCenter,\r\n x_InvoiceId,\r\n x_InvoiceNumber,\r\n x_InvoiceSectionId,\r\n x_InvoiceSectionName,\r\n x_IngestionTime,\r\n x_MonetaryCommitment,\r\n x_Overage,\r\n x_PurchasingBillingAccountId,\r\n x_SkuOrderId,\r\n x_SkuOrderName,\r\n x_SkuSize,\r\n x_SkuTerm,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_SubscriptionId,\r\n x_TransactionType\r\n}\r\n", - "$fxv#14": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Hub database / FOCUS 1.2 functions\r\n// Used for reporting with backward compatibility.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n\r\n// CommitmentDiscountUsage_final_v1_2\r\n.create-or-alter function\r\nwith (docstring = 'Gets all commitment discount usage records aligned to FOCUS 1.2.', folder = 'CommitmentDiscountUsage')\r\nCommitmentDiscountUsage_v1_2()\r\n{\r\n database('Ingestion').CommitmentDiscountUsage_final_v1_2\r\n | union (\r\n database('Ingestion').CommitmentDiscountUsage_final_v1_0\r\n // Convert decimal to real\r\n | extend\r\n ConsumedQuantity = toreal(ConsumedQuantity),\r\n x_CommitmentDiscountCommittedCount = toreal(x_CommitmentDiscountCommittedCount),\r\n x_CommitmentDiscountCommittedAmount = toreal(x_CommitmentDiscountCommittedAmount),\r\n x_CommitmentDiscountNormalizedRatio = toreal(x_CommitmentDiscountNormalizedRatio)\r\n // Add new columns\r\n | lookup kind=leftouter (Services | distinct x_ResourceType, ServiceSubcategory) on x_ResourceType\r\n | extend CommitmentDiscountQuantity = ConsumedQuantity * x_CommitmentDiscountNormalizedRatio\r\n | extend CommitmentDiscountUnit = case(\r\n x_CommitmentDiscountNormalizedRatio == 1, 'Hours',\r\n x_CommitmentDiscountNormalizedRatio > 1, 'Normalized Hours',\r\n ''\r\n )\r\n )\r\n | project\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId,\r\n CommitmentDiscountQuantity,\r\n CommitmentDiscountType,\r\n CommitmentDiscountUnit,\r\n ConsumedQuantity,\r\n ProviderName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory,\r\n SubAccountId,\r\n x_CommitmentDiscountCommittedCount,\r\n x_CommitmentDiscountCommittedAmount,\r\n x_CommitmentDiscountNormalizedGroup,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_IngestionTime,\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n x_ServiceModel,\r\n x_SkuOrderId,\r\n x_SkuSize,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n\r\n// Costs_final_v1_2\r\n.create-or-alter function\r\nwith (docstring = 'Gets all cost and usage records aligned to FOCUS 1.2.', folder = 'Costs')\r\nCosts_v1_2()\r\n{\r\n database('Ingestion').Costs_final_v1_2\r\n | union (\r\n database('Ingestion').Costs_final_v1_0\r\n // Convert decimal to real\r\n | extend\r\n BilledCost = toreal(BilledCost),\r\n ConsumedQuantity = toreal(ConsumedQuantity),\r\n ContractedCost = toreal(ContractedCost),\r\n ContractedUnitPrice = toreal(ContractedUnitPrice),\r\n EffectiveCost = toreal(EffectiveCost),\r\n ListCost = toreal(ListCost),\r\n ListUnitPrice = toreal(ListUnitPrice),\r\n PricingQuantity = toreal(PricingQuantity),\r\n x_BilledCostInUsd = toreal(x_BilledCostInUsd),\r\n x_BilledUnitPrice = toreal(x_BilledUnitPrice),\r\n x_BillingExchangeRate = toreal(x_BillingExchangeRate),\r\n x_ContractedCostInUsd = toreal(x_ContractedCostInUsd),\r\n x_CurrencyConversionRate = toreal(x_CurrencyConversionRate),\r\n x_EffectiveCostInUsd = toreal(x_EffectiveCostInUsd),\r\n x_EffectiveUnitPrice = toreal(x_EffectiveUnitPrice),\r\n x_ListCostInUsd = toreal(x_ListCostInUsd),\r\n x_PricingBlockSize = toreal(x_PricingBlockSize)\r\n // Rename columns\r\n | project-rename\r\n InvoiceId = x_InvoiceId,\r\n PricingCurrency = x_PricingCurrency,\r\n SkuMeter = x_SkuMeterName\r\n // Add new columns\r\n | join kind=leftouter (Services | where isnotempty(x_ResourceType) | project x_ResourceType, ServiceSubcategory, x_ServiceModel) on x_ResourceType\r\n | extend CapacityReservationId = tostring(x_SkuDetails.VMCapacityReservationId)\r\n | extend CapacityReservationStatus = case(\r\n isempty(CapacityReservationId), '',\r\n tolower(x_ResourceType) == 'microsoft.compute/capacityreservationgroups/capacityreservations', 'Unused',\r\n 'Used'\r\n )\r\n | extend x_CommitmentDiscountNormalizedRatio = case(\r\n // Not applicable\r\n isempty(CommitmentDiscountStatus), real(null),\r\n // Parse from SKU details if not specified explicitly\r\n toreal(coalesce(x_SkuDetails.RINormalizationRatio, dynamic(1)))\r\n )\r\n | extend CommitmentDiscountQuantity = case(\r\n isempty(CommitmentDiscountStatus), real(null),\r\n CommitmentDiscountCategory == 'Spend', EffectiveCost / coalesce(x_BillingExchangeRate, real(1)),\r\n CommitmentDiscountCategory == 'Usage' and isnotempty(x_CommitmentDiscountNormalizedRatio), PricingQuantity / coalesce(x_PricingBlockSize, real(1)) * x_CommitmentDiscountNormalizedRatio,\r\n real(null)\r\n )\r\n | extend CommitmentDiscountUnit = case(\r\n isempty(CommitmentDiscountQuantity), '',\r\n CommitmentDiscountCategory == 'Spend', PricingCurrency,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio == real(1), ConsumedUnit,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio > real(1), strcat('Normalized ', ConsumedUnit),\r\n ''\r\n )\r\n | extend x_AmortizationClass = case(\r\n ChargeCategory == 'Purchase' and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeCategory == 'Usage' and isnotempty(CommitmentDiscountId) and isnotempty(CommitmentDiscountStatus), 'Amortized Charge',\r\n ''\r\n )\r\n // Hubs add-ons\r\n | extend x_CommitmentDiscountUtilizationPotential = case(\r\n ChargeCategory == 'Purchase', real(0),\r\n ProviderName == 'Microsoft' and isnotempty(CommitmentDiscountCategory), EffectiveCost,\r\n CommitmentDiscountCategory == 'Usage', ConsumedQuantity,\r\n CommitmentDiscountCategory == 'Spend', EffectiveCost,\r\n real(0)\r\n )\r\n | extend x_CommitmentDiscountUtilizationAmount = iff(CommitmentDiscountStatus == 'Used', x_CommitmentDiscountUtilizationPotential, real(0))\r\n | extend x_SkuCoreCount = toint(coalesce(x_SkuDetails.VCPUs, x_SkuDetails.VCores, x_SkuDetails.vCores))\r\n | extend x_SkuInstanceType = tostring(coalesce(x_SkuDetails.ServiceType, x_SkuDetails.ServerSku))\r\n | extend x_SkuOperatingSystem = case(\r\n x_SkuDetails.ImageType == 'Canonical', 'Linux',\r\n x_SkuDetails.ImageType == 'Windows Server BYOL', 'Windows Server',\r\n x_SkuMeterSubcategory endswith ' Series Windows', 'Windows Server',\r\n x_SkuDetails.ImageType\r\n )\r\n | extend x_ConsumedCoreHours = iff(ConsumedUnit == 'Hours' and isnotempty(x_SkuCoreCount), x_SkuCoreCount * ConsumedQuantity, real(null))\r\n | extend tmp_SqlAhb = tolower(x_SkuDetails.AHB)\r\n | extend x_SkuLicenseType = case(\r\n x_SkuDetails.ImageType contains 'Windows Server BYOL', 'Windows Server',\r\n x_SkuMeterSubcategory == 'SQL Server Azure Hybrid Benefit', 'SQL Server',\r\n ''\r\n )\r\n | extend x_SkuLicenseStatus = case(\r\n isnotempty(x_SkuLicenseType) or tmp_SqlAhb == 'true' or (x_SkuMeterSubcategory contains 'Azure Hybrid Benefit'), 'Enabled',\r\n (x_SkuMeterSubcategory contains 'Windows') or tmp_SqlAhb == 'false', 'Not enabled',\r\n ''\r\n )\r\n | extend x_SkuLicenseQuantity = case(\r\n isempty(x_SkuCoreCount), int(null),\r\n x_SkuCoreCount <= 8, int(8),\r\n x_SkuCoreCount > 8, x_SkuCoreCount,\r\n int(null)\r\n )\r\n | extend x_SkuLicenseUnit = iff(isnotempty(x_SkuLicenseQuantity), 'Cores', '')\r\n | extend x_CommitmentDiscountSavings = iff(ContractedCost < EffectiveCost, real(0), ContractedCost - EffectiveCost)\r\n | extend x_NegotiatedDiscountSavings = iff(ListCost < ContractedCost, real(0), ListCost - ContractedCost)\r\n | extend x_TotalSavings = iff(ListCost < EffectiveCost, real(0), ListCost - EffectiveCost)\r\n | extend x_CommitmentDiscountPercent = iff(ContractedUnitPrice == 0, real(0), (ContractedUnitPrice - x_EffectiveUnitPrice) / ContractedUnitPrice)\r\n | extend x_NegotiatedDiscountPercent = iff(ListUnitPrice == 0, real(0), (ListUnitPrice - ContractedUnitPrice) / ListUnitPrice)\r\n | extend x_TotalDiscountPercent = iff(ListUnitPrice == 0, real(0), (ListUnitPrice - x_EffectiveUnitPrice) / ListUnitPrice)\r\n // SkuPriceDetails conversion -- Must be after hubs add-ons\r\n | extend SkuPriceDetails = parse_json(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(tostring(x_SkuDetails)\r\n // Prefix all keys with x_ first to avoid double-prefixing\r\n , @'([\\{,])\"', @'\\1\"x_')\r\n // CoreCount for number of CPUs/vCPUs/cores/vCores\r\n , @'\"x_(VCPUs|VCores|vCores)\":', @'\"CoreCount\":')\r\n // TODO: DiskMaxIops for disk I/O operations per second (IOPS)\r\n // TODO: DiskSpace for disk size in GiB\r\n // TODO: DiskType for the kind of disk (e.g., SSD, HDD, NVMe)\r\n // TODO: GpuCount for the number of GPUs\r\n // InstanceType for the resource size/SKU (e.g., ArmSkuName)\r\n , @'\"x_(ServerSku|ServiceType)\":', @'\"InstanceType\":')\r\n // TODO: InstanceSeries for the size family/series\r\n // TODO: MemorySize for the RAM in GiB\r\n // TODO: NetworkMaxIops for network I/O operations per second (IOPS)\r\n // TODO: NetworkMaxThroughput for network max throughput for data transfer in Mbps\r\n // OperatingSystem for the OS name\r\n , @'(\"x_ImageType\":\"Canonical\")', @'\\1,\"OperatingSystem\":\"Linux\"')\r\n , @'(\"x_ImageType\":\"Windows Server( BYOL)?\")', @'\\1,\"OperatingSystem\":\"Windows Server\"')\r\n , @'(\"x_ImageType\":(\"[^\"]+\"))', @'\\1,\"OperatingSystem\":\\2')\r\n // TODO: Redundancy for the level of redundancy (e.g., Local, Zonal, Global)\r\n // TODO: StorageClass for the tier of storage (e.g., Hot, Archive, Nearline)\r\n )\r\n | extend SkuPriceDetails = iff(isempty(SkuPriceDetails.OperatingSystem) and isnotempty(x_SkuOperatingSystem),\r\n parse_json(replace_string(tostring(SkuPriceDetails), '}', strcat(@',\"OperatingSystem\":\"', x_SkuOperatingSystem, '\"}'))),\r\n SkuPriceDetails)\r\n )\r\n | extend SkuPriceDetails = iff(isnotempty(SkuPriceDetails), SkuPriceDetails, parse_json(replace_regex(tostring(x_SkuDetails), @'([\\{,])\"', @'\\1\"x_')))\r\n | project\r\n AvailabilityZone,\r\n BilledCost,\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingAccountType,\r\n BillingCurrency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n CapacityReservationId,\r\n CapacityReservationStatus,\r\n ChargeCategory,\r\n ChargeClass,\r\n ChargeDescription,\r\n ChargeFrequency,\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId,\r\n CommitmentDiscountName,\r\n CommitmentDiscountQuantity,\r\n CommitmentDiscountStatus,\r\n CommitmentDiscountType,\r\n CommitmentDiscountUnit,\r\n ConsumedQuantity,\r\n ConsumedUnit,\r\n ContractedCost,\r\n ContractedUnitPrice,\r\n EffectiveCost,\r\n InvoiceId,\r\n InvoiceIssuerName,\r\n ListCost,\r\n ListUnitPrice,\r\n PricingCategory,\r\n PricingCurrency,\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n PublisherName,\r\n RegionId,\r\n RegionName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory,\r\n SkuId,\r\n SkuMeter,\r\n SkuPriceDetails,\r\n SkuPriceId,\r\n SubAccountId,\r\n SubAccountName,\r\n SubAccountType,\r\n Tags,\r\n x_AccountId,\r\n x_AccountName,\r\n x_AccountOwnerId,\r\n x_AmortizationClass,\r\n x_BilledCostInUsd,\r\n x_BilledUnitPrice,\r\n x_BillingAccountAgreement,\r\n x_BillingAccountId,\r\n x_BillingAccountName,\r\n x_BillingExchangeRate,\r\n x_BillingExchangeRateDate,\r\n x_BillingItemCode,\r\n x_BillingItemName,\r\n x_BillingProfileId,\r\n x_BillingProfileName,\r\n x_ChargeId,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_CommitmentDiscountPercent,\r\n x_CommitmentDiscountSavings,\r\n x_CommitmentDiscountSpendEligibility,\r\n x_CommitmentDiscountUsageEligibility,\r\n x_CommitmentDiscountUtilizationAmount,\r\n x_CommitmentDiscountUtilizationPotential,\r\n x_CommodityCode,\r\n x_CommodityName,\r\n x_ComponentName,\r\n x_ComponentType,\r\n x_ConsumedCoreHours,\r\n x_ContractedCostInUsd,\r\n x_CostAllocationRuleName,\r\n x_CostCategories,\r\n x_CostCenter,\r\n x_CostType,\r\n x_Credits,\r\n x_CurrencyConversionRate,\r\n x_CustomerId,\r\n x_CustomerName,\r\n x_Discount,\r\n x_EffectiveCostInUsd,\r\n x_EffectiveUnitPrice,\r\n x_ExportTime,\r\n x_IngestionTime,\r\n x_InstanceID,\r\n x_InvoiceIssuerId,\r\n x_InvoiceSectionId,\r\n x_InvoiceSectionName,\r\n x_ListCostInUsd,\r\n x_Location,\r\n x_NegotiatedDiscountPercent,\r\n x_NegotiatedDiscountSavings,\r\n x_Operation,\r\n x_OwnerAccountID,\r\n x_PartnerCreditApplied,\r\n x_PartnerCreditRate,\r\n x_PricingBlockSize,\r\n x_PricingSubcategory,\r\n x_PricingUnitDescription,\r\n x_Project,\r\n x_PublisherCategory,\r\n x_PublisherId,\r\n x_ResellerId,\r\n x_ResellerName,\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n x_ServiceCode,\r\n x_ServiceId,\r\n x_ServiceModel,\r\n x_ServicePeriodEnd,\r\n x_ServicePeriodStart,\r\n x_SkuCoreCount,\r\n x_SkuDescription,\r\n x_SkuDetails,\r\n x_SkuInstanceType,\r\n x_SkuIsCreditEligible,\r\n x_SkuLicenseQuantity,\r\n x_SkuLicenseStatus,\r\n x_SkuLicenseType,\r\n x_SkuLicenseUnit,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterSubcategory,\r\n x_SkuOfferId,\r\n x_SkuOperatingSystem,\r\n x_SkuOrderId,\r\n x_SkuOrderName,\r\n x_SkuPartNumber,\r\n x_SkuPlanName,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceChanges,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceValues,\r\n x_SourceVersion,\r\n x_SubproductName,\r\n x_TotalDiscountPercent,\r\n x_TotalSavings,\r\n x_UsageType\r\n}\r\n\r\n\r\n// Prices_final_v1_2\r\n.create-or-alter function\r\nwith (docstring = 'Gets all prices aligned to FOCUS 1.2.', folder = 'Prices')\r\nPrices_v1_2()\r\n{\r\n database('Ingestion').Prices_final_v1_2\r\n | union (\r\n database('Ingestion').Prices_final_v1_0\r\n // Convert decimal to real\r\n | extend\r\n ContractedUnitPrice = toreal(ContractedUnitPrice),\r\n ListUnitPrice = toreal(ListUnitPrice),\r\n x_BaseUnitPrice = toreal(x_BaseUnitPrice),\r\n x_ContractedUnitPriceDiscount = toreal(x_ContractedUnitPriceDiscount),\r\n x_ContractedUnitPriceDiscountPercent = toreal(x_ContractedUnitPriceDiscountPercent),\r\n x_EffectiveUnitPrice = toreal(x_EffectiveUnitPrice),\r\n x_EffectiveUnitPriceDiscount = toreal(x_EffectiveUnitPriceDiscount),\r\n x_EffectiveUnitPriceDiscountPercent = toreal(x_EffectiveUnitPriceDiscountPercent),\r\n x_PricingBlockSize = toreal(x_PricingBlockSize),\r\n x_SkuIncludedQuantity = toreal(x_SkuIncludedQuantity),\r\n x_SkuTier = toreal(x_SkuTier),\r\n x_TotalUnitPriceDiscount = toreal(x_TotalUnitPriceDiscount),\r\n x_TotalUnitPriceDiscountPercent = toreal(x_TotalUnitPriceDiscountPercent) \r\n // Rename columns\r\n | project-rename\r\n PricingCurrency = x_PricingCurrency,\r\n SkuMeter = x_SkuMeterName\r\n )\r\n | project\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingCurrency,\r\n ChargeCategory,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountType,\r\n CommitmentDiscountUnit,\r\n ContractedUnitPrice,\r\n ListUnitPrice,\r\n PricingCategory,\r\n PricingCurrency,\r\n PricingUnit,\r\n SkuId,\r\n SkuMeter,\r\n SkuPriceId,\r\n SkuPriceIdv2,\r\n x_BaseUnitPrice,\r\n x_BillingAccountAgreement,\r\n x_BillingAccountId,\r\n x_BillingProfileId,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_CommitmentDiscountSpendEligibility,\r\n x_CommitmentDiscountUsageEligibility,\r\n x_ContractedUnitPriceDiscount,\r\n x_ContractedUnitPriceDiscountPercent,\r\n x_EffectivePeriodEnd,\r\n x_EffectivePeriodStart,\r\n x_EffectiveUnitPrice,\r\n x_EffectiveUnitPriceDiscount,\r\n x_EffectiveUnitPriceDiscountPercent,\r\n x_IngestionTime,\r\n x_PricingBlockSize,\r\n x_PricingSubcategory,\r\n x_PricingUnitDescription,\r\n x_SkuDescription,\r\n x_SkuId,\r\n x_SkuIncludedQuantity,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterSubcategory,\r\n x_SkuMeterType,\r\n x_SkuPriceType,\r\n x_SkuProductId,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuOfferId,\r\n x_SkuPartNumber,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_TotalUnitPriceDiscount,\r\n x_TotalUnitPriceDiscountPercent\r\n}\r\n\r\n\r\n// Recommendations_final_v1_2\r\n.create-or-alter function\r\nwith (docstring = 'Gets all recommendations aligned to FOCUS 1.2.', folder = 'Recommendations')\r\nRecommendations_v1_2()\r\n{\r\n database('Ingestion').Recommendations_final_v1_2\r\n | union (\r\n database('Ingestion').Recommendations_final_v1_0\r\n // Convert decimal to real\r\n | extend\r\n x_EffectiveCostAfter = toreal(x_EffectiveCostAfter),\r\n x_EffectiveCostBefore = toreal(x_EffectiveCostBefore),\r\n x_EffectiveCostSavings = toreal(x_EffectiveCostSavings)\r\n )\r\n | project\r\n ProviderName,\r\n SubAccountId,\r\n x_IngestionTime,\r\n x_EffectiveCostAfter,\r\n x_EffectiveCostBefore,\r\n x_EffectiveCostSavings,\r\n x_RecommendationDate,\r\n x_RecommendationDetails,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n\r\n// Transactions_final_v1_2\r\n.create-or-alter function\r\nwith (docstring = 'Gets all transactions aligned to FOCUS 1.2.', folder = 'Transactions')\r\nTransactions_v1_2()\r\n{\r\n database('Ingestion').Transactions_final_v1_2\r\n | union (\r\n database('Ingestion').Transactions_final_v1_0\r\n // Convert decimal to real\r\n | extend\r\n BilledCost = toreal(BilledCost),\r\n PricingQuantity = toreal(PricingQuantity),\r\n x_MonetaryCommitment = toreal(x_MonetaryCommitment),\r\n x_Overage = toreal(x_Overage)\r\n // Rename columns\r\n | project-rename\r\n InvoiceId = x_InvoiceId\r\n )\r\n | project\r\n BilledCost,\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingCurrency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n ChargeCategory,\r\n ChargeClass,\r\n ChargeDescription,\r\n ChargeFrequency,\r\n ChargePeriodStart,\r\n InvoiceId,\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n RegionId,\r\n RegionName,\r\n SubAccountId,\r\n SubAccountName,\r\n x_AccountName,\r\n x_AccountOwnerId,\r\n x_CostCenter,\r\n x_InvoiceNumber,\r\n x_InvoiceSectionId,\r\n x_InvoiceSectionName,\r\n x_IngestionTime,\r\n x_MonetaryCommitment,\r\n x_Overage,\r\n x_PurchasingBillingAccountId,\r\n x_SkuOrderId,\r\n x_SkuOrderName,\r\n x_SkuSize,\r\n x_SkuTerm,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_SubscriptionId,\r\n x_TransactionType\r\n}\r\n\r\n\r\n//======================================================================================================================\r\n// Latest FOCUS version\r\n//======================================================================================================================\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all commitment discount usage records with the latest supported version of the FOCUS schema.', folder = 'CommitmentDiscountUsage')\r\nCommitmentDiscountUsage()\r\n{\r\n CommitmentDiscountUsage_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all cost and usage records with the latest supported version of the FOCUS schema.', folder = 'Costs')\r\nCosts()\r\n{\r\n Costs_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all prices with the latest supported version of the FOCUS schema.', folder = 'Prices')\r\nPrices()\r\n{\r\n Prices_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all recommendations with the latest supported version of the FOCUS schema.', folder = 'Recommendations')\r\nRecommendations()\r\n{\r\n Recommendations_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all transactions with the latest supported version of the FOCUS schema.', folder = 'Transactions')\r\nTransactions()\r\n{\r\n Transactions_v1_2()\r\n}\r\n", - "$fxv#15": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Hub database / Latest FOCUS version functions\r\n// Used for ad hoc queries.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all commitment discount usage records with the latest supported version of the FOCUS schema.', folder = 'CommitmentDiscountUsage')\r\nCommitmentDiscountUsage()\r\n{\r\n CommitmentDiscountUsage_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all cost and usage records with the latest supported version of the FOCUS schema.', folder = 'Costs')\r\nCosts()\r\n{\r\n Costs_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all prices with the latest supported version of the FOCUS schema.', folder = 'Prices')\r\nPrices()\r\n{\r\n Prices_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all recommendations with the latest supported version of the FOCUS schema.', folder = 'Recommendations')\r\nRecommendations()\r\n{\r\n Recommendations_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all transactions with the latest supported version of the FOCUS schema.', folder = 'Transactions')\r\nTransactions()\r\n{\r\n Transactions_v1_2()\r\n}\r\n", - "$fxv#2": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData/Internal')\r\n_resource_type_3(id: string) {\r\n dynamic({\r\n \"microsoft.hybridnetwork/vendors\": { \"SingularDisplayName\": \"Azure Network Function Manager ? vendor\" }\r\n ,\"microsoft.hybridonboarding/extensionmanagers\": { \"SingularDisplayName\": \"Microsoft.HybridOnboarding extension manager\" }\r\n ,\"microsoft.impact/connectors\": { \"SingularDisplayName\": \"Impact Reporting Connector\" }\r\n ,\"microsoft.impact/impactcategories\": { \"SingularDisplayName\": \"Microsoft.Impact impact category\" }\r\n ,\"microsoft.impact/topologyimpacts\": { \"SingularDisplayName\": \"Microsoft.Impact topology impact\" }\r\n ,\"microsoft.impact/workloadimpacts\": { \"SingularDisplayName\": \"Microsoft.Impact workload impact\" }\r\n ,\"microsoft.impact/workloadimpacts/insights\": { \"SingularDisplayName\": \"Microsoft.Impact workload impacts insight\" }\r\n ,\"microsoft.importexport/jobs\": { \"SingularDisplayName\": \"Microsoft.ImportExport job\" }\r\n ,\"microsoft.insights/actiongroups\": { \"SingularDisplayName\": \"Action group\" }\r\n ,\"microsoft.insights/activitylogalerts\": { \"SingularDisplayName\": \"Activity log alert rule\" }\r\n ,\"microsoft.insights/alertrules\": { \"SingularDisplayName\": \"Microsoft.Insights alertrule\" }\r\n ,\"microsoft.insights/alertrules/incidents\": { \"SingularDisplayName\": \"Microsoft.insights alertrules incident\" }\r\n ,\"microsoft.insights/autoscalesettings\": { \"SingularDisplayName\": \"Microsoft.Insights autoscalesetting\" }\r\n ,\"microsoft.insights/components\": { \"SingularDisplayName\": \"Application Insights app\" }\r\n ,\"microsoft.insights/datacollectionendpoints\": { \"SingularDisplayName\": \"Data collection endpoint\" }\r\n ,\"microsoft.insights/datacollectionruleassociations\": { \"SingularDisplayName\": \"Microsoft.Insights data collection rule association\" }\r\n ,\"microsoft.insights/datacollectionrules\": { \"SingularDisplayName\": \"Data collection rule\" }\r\n ,\"microsoft.insights/datacollectionrulesresources\": { \"SingularDisplayName\": \"Data collection rule associated resource\" }\r\n ,\"microsoft.insights/diagnosticsettings\": { \"SingularDisplayName\": \"Diagnostic settings\" }\r\n ,\"microsoft.insights/diagnosticsettingscategories\": { \"SingularDisplayName\": \"Microsoft.Insights diagnostic settings category\" }\r\n ,\"microsoft.insights/guestdiagnosticsettings\": { \"SingularDisplayName\": \"Microsoft.insights guest diagnostic setting\" }\r\n ,\"microsoft.insights/guestdiagnosticsettingsassociation\": { \"SingularDisplayName\": \"Microsoft.insights guest diagnostic settings association\" }\r\n ,\"microsoft.insights/logprofiles\": { \"SingularDisplayName\": \"Microsoft.Insights logprofile\" }\r\n ,\"microsoft.insights/metricalerts\": { \"SingularDisplayName\": \"Metric alert rule\" }\r\n ,\"microsoft.insights/notificationstatus\": { \"SingularDisplayName\": \"Microsoft.Insights notification statu\" }\r\n ,\"microsoft.insights/privatelinkscopeoperationstatuses\": { \"SingularDisplayName\": \"Microsoft.insights private link scope operation statuse\" }\r\n ,\"microsoft.insights/privatelinkscopes\": { \"SingularDisplayName\": \"Azure Monitor Private Link Scope\" }\r\n ,\"microsoft.insights/scheduledqueryrules\": { \"SingularDisplayName\": \"Log search alert rule\" }\r\n ,\"microsoft.insights/tenantactiongroups\": { \"SingularDisplayName\": \"Microsoft.Insights tenant action group\" }\r\n ,\"microsoft.insights/tenantactiongroups/notificationstatus\": { \"SingularDisplayName\": \"Microsoft.Insights tenant action groups notification statu\" }\r\n ,\"microsoft.insights/vminsightsonboardingstatuses\": { \"SingularDisplayName\": \"Microsoft.Insights VM insights onboarding statuse\" }\r\n ,\"microsoft.insights/webtests\": { \"SingularDisplayName\": \"Application Insights availability test\" }\r\n ,\"microsoft.insights/workbooks\": { \"SingularDisplayName\": \"Azure Workbook\" }\r\n ,\"microsoft.insights/workbooktemplates\": { \"SingularDisplayName\": \"Azure Workbook Template\" }\r\n ,\"microsoft.integrationspaces/spaces\": { \"SingularDisplayName\": \"Integration Environment\" }\r\n ,\"microsoft.intelligentitdigitaltwin/digitaltwins\": { \"SingularDisplayName\": \"Microsoft.IntelligentITDigitalTwin digital twin\" }\r\n ,\"microsoft.intelligentitdigitaltwin/digitaltwins/assets\": { \"SingularDisplayName\": \"Microsoft.IntelligentITDigitalTwin digital twins asset\" }\r\n ,\"microsoft.intelligentitdigitaltwin/digitaltwins/executionplans\": { \"SingularDisplayName\": \"Microsoft.IntelligentITDigitalTwin digital twins execution plan\" }\r\n ,\"microsoft.intelligentitdigitaltwin/digitaltwins/testplans\": { \"SingularDisplayName\": \"Microsoft.IntelligentITDigitalTwin digital twins test plan\" }\r\n ,\"microsoft.intelligentitdigitaltwin/digitaltwins/tests\": { \"SingularDisplayName\": \"Microsoft.IntelligentITDigitalTwin digital twins test\" }\r\n ,\"microsoft.inventory/subscriptioninternalproperties\": { \"SingularDisplayName\": \"Microsoft.Inventory subscription internal property\" }\r\n ,\"microsoft.iotcentral/iotapps\": { \"SingularDisplayName\": \"IoT Central Application\" }\r\n ,\"microsoft.iotfirmwaredefense/workspaces\": { \"SingularDisplayName\": \"Firmware analysis workspace\" }\r\n ,\"microsoft.iotfirmwaredefense/workspaces/firmwares\": { \"SingularDisplayName\": \"Microsoft.IoTFirmwareDefense workspaces firmware\" }\r\n ,\"microsoft.iotfirmwaredefense/workspaces/firmwares/summaries\": { \"SingularDisplayName\": \"Microsoft.IoTFirmwareDefense workspaces firmwares summary\" }\r\n ,\"microsoft.iotoperations/instances\": { \"SingularDisplayName\": \"Azure IoT Operations\" }\r\n ,\"microsoft.iotoperations/instances/brokers\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances broker\" }\r\n ,\"microsoft.iotoperations/instances/brokers/authentications\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances brokers authentication\" }\r\n ,\"microsoft.iotoperations/instances/brokers/authorizations\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances brokers authorization\" }\r\n ,\"microsoft.iotoperations/instances/brokers/listeners\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances brokers listener\" }\r\n ,\"microsoft.iotoperations/instances/dataflowendpoints\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances dataflow endpoint\" }\r\n ,\"microsoft.iotoperations/instances/dataflowprofiles\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances dataflow profile\" }\r\n ,\"microsoft.iotoperations/instances/dataflowprofiles/dataflows\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances dataflow profiles dataflow\" }\r\n ,\"microsoft.iotoperationsdataprocessor/instances\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsDataProcessor instance\" }\r\n ,\"microsoft.iotoperationsdataprocessor/instances/datasets\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsDataProcessor instances dataset\" }\r\n ,\"microsoft.iotoperationsdataprocessor/instances/pipelines\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsDataProcessor instances pipeline\" }\r\n ,\"microsoft.iotoperationsmq/mq\": { \"SingularDisplayName\": \"IoT Operations Ops MQ\" }\r\n ,\"microsoft.iotoperationsmq/mq/broker\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq broker\" }\r\n ,\"microsoft.iotoperationsmq/mq/broker/authentication\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq broker authentication\" }\r\n ,\"microsoft.iotoperationsmq/mq/broker/authorization\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq broker authorization\" }\r\n ,\"microsoft.iotoperationsmq/mq/broker/listener\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq broker listener\" }\r\n ,\"microsoft.iotoperationsmq/mq/datalakeconnector\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq data lake connector\" }\r\n ,\"microsoft.iotoperationsmq/mq/datalakeconnector/topicmap\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq data lake connector topic map\" }\r\n ,\"microsoft.iotoperationsmq/mq/diagnosticservice\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq diagnostic service\" }\r\n ,\"microsoft.iotoperationsmq/mq/kafkaconnector\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq kafka connector\" }\r\n ,\"microsoft.iotoperationsmq/mq/kafkaconnector/topicmap\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq kafka connector topic map\" }\r\n ,\"microsoft.iotoperationsmq/mq/mqttbridgeconnector\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq mqtt bridge connector\" }\r\n ,\"microsoft.iotoperationsmq/mq/mqttbridgeconnector/topicmap\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq mqtt bridge connector topic map\" }\r\n ,\"microsoft.iotoperationsorchestrator/instances\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsOrchestrator instance\" }\r\n ,\"microsoft.iotoperationsorchestrator/solutions\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsOrchestrator solution\" }\r\n ,\"microsoft.iotoperationsorchestrator/targets\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsOrchestrator target\" }\r\n ,\"microsoft.iotsecurity/alerttypes\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity alert type\" }\r\n ,\"microsoft.iotsecurity/defendersettings\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity defender setting\" }\r\n ,\"microsoft.iotsecurity/onpremisesensors\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity on premise sensor\" }\r\n ,\"microsoft.iotsecurity/recommendationtypes\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity recommendation type\" }\r\n ,\"microsoft.iotsecurity/sensors\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity sensor\" }\r\n ,\"microsoft.iotsecurity/sites\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity site\" }\r\n ,\"microsoft.keyvault/managedhsms\": { \"SingularDisplayName\": \"Azure Key Vault Managed HSM\" }\r\n ,\"microsoft.keyvault/vaults\": { \"SingularDisplayName\": \"Key vault\" }\r\n ,\"microsoft.kubernetes/connectedclusters\": { \"SingularDisplayName\": \"Kubernetes - Azure Arc\" }\r\n ,\"microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/extensions\": { \"SingularDisplayName\": \"Kubernetes - Azure Arc extension\" }\r\n ,\"microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/fluxconfigurations\": { \"SingularDisplayName\": \"GitOps configuration\" }\r\n ,\"microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/namespaces\": { \"SingularDisplayName\": \"Kubernetes - Azure Arc namespace\" }\r\n ,\"microsoft.kubernetesconfiguration/extensions\": { \"SingularDisplayName\": \"Kubernetes service extension\" }\r\n ,\"microsoft.kubernetesconfiguration/extensiontypes\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration extension type\" }\r\n ,\"microsoft.kubernetesconfiguration/extensiontypes/versions\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration extension types version\" }\r\n ,\"microsoft.kubernetesconfiguration/fluxconfigurations\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration flux configuration\" }\r\n ,\"microsoft.kubernetesconfiguration/fluxconfigurations/operations\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration flux configurations operation\" }\r\n ,\"microsoft.kubernetesconfiguration/privatelinkscopes\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration private link scope\" }\r\n ,\"microsoft.kubernetesconfiguration/privatelinkscopes/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration private link scopes private endpoint connection\" }\r\n ,\"microsoft.kubernetesconfiguration/privatelinkscopes/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration private link scopes private link resource\" }\r\n ,\"microsoft.kubernetesconfiguration/sourcecontrolconfigurations\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration source control configuration\" }\r\n ,\"microsoft.kubernetesruntime/bgppeers\": { \"SingularDisplayName\": \"Microsoft.KubernetesRuntime bgp peer\" }\r\n ,\"microsoft.kubernetesruntime/loadbalancers\": { \"SingularDisplayName\": \"Arc Load Balancer\" }\r\n ,\"microsoft.kubernetesruntime/services\": { \"SingularDisplayName\": \"Microsoft.KubernetesRuntime service\" }\r\n ,\"microsoft.kubernetesruntime/storageclasses\": { \"SingularDisplayName\": \"Microsoft.KubernetesRuntime storage class\" }\r\n ,\"microsoft.kusto/clusters\": { \"SingularDisplayName\": \"Azure Data Explorer Cluster\" }\r\n ,\"microsoft.kusto/clusters/databases\": { \"SingularDisplayName\": \"Azure Data Explorer Database\" }\r\n ,\"microsoft.labservices/labaccounts\": { \"SingularDisplayName\": \"Lab account\" }\r\n ,\"microsoft.labservices/labaccounts/labs\": { \"SingularDisplayName\": \"Lab\" }\r\n ,\"microsoft.labservices/labplans\": { \"SingularDisplayName\": \"Lab plan\" }\r\n ,\"microsoft.labservices/labs\": { \"SingularDisplayName\": \"Lab\" }\r\n ,\"microsoft.liftrpilot/organizations\": { \"SingularDisplayName\": \"Azure Pilot\" }\r\n ,\"microsoft.loadtestservice/loadtestmappings\": { \"SingularDisplayName\": \"Microsoft.LoadTestService load test mapping\" }\r\n ,\"microsoft.loadtestservice/loadtestprofilemappings\": { \"SingularDisplayName\": \"Microsoft.LoadTestService load test profile mapping\" }\r\n ,\"microsoft.loadtestservice/loadtests\": { \"SingularDisplayName\": \"Azure Load Testing\" }\r\n ,\"microsoft.loadtestservice/playwrightworkspaces\": { \"SingularDisplayName\": \"Playwright Workspace\" }\r\n ,\"microsoft.logic/businessprocesses\": { \"SingularDisplayName\": \"Business Process\" }\r\n ,\"microsoft.logic/integrationaccounts\": { \"SingularDisplayName\": \"Logic app integration account\" }\r\n ,\"microsoft.logic/integrationserviceenvironments\": { \"SingularDisplayName\": \"Integration Service Environment\" }\r\n ,\"microsoft.logic/integrationserviceenvironments/health\": { \"SingularDisplayName\": \"Microsoft.Logic integration service environments health\" }\r\n ,\"microsoft.logic/integrationserviceenvironments/managedapis\": { \"SingularDisplayName\": \"Managed Connector\" }\r\n ,\"microsoft.logic/templates\": { \"SingularDisplayName\": \"Logic Apps Template\" }\r\n ,\"microsoft.logic/workflows\": { \"SingularDisplayName\": \"Logic app\" }\r\n ,\"microsoft.logz/monitors\": { \"SingularDisplayName\": \"Logz.io\" }\r\n ,\"microsoft.logz/monitors/accounts\": { \"SingularDisplayName\": \"Logz sub account\" }\r\n ,\"microsoft.m365/m365resources\": { \"SingularDisplayName\": \"Microsoft.M365 m365 resource\" }\r\n ,\"microsoft.m365consumptionservices/services\": { \"SingularDisplayName\": \"Microsoft.M365ConsumptionServices service\" }\r\n ,\"microsoft.machinelearning/commitmentplans\": { \"SingularDisplayName\": \"Microsoft.MachineLearning commitment plan\" }\r\n ,\"microsoft.machinelearning/commitmentplans/commitmentassociations\": { \"SingularDisplayName\": \"Microsoft.MachineLearning commitment plans commitment association\" }\r\n ,\"microsoft.machinelearning/webservices\": { \"SingularDisplayName\": \"Machine Learning Studio (classic) web service\" }\r\n ,\"microsoft.machinelearning/workspaces\": { \"SingularDisplayName\": \"Machine Learning Studio (classic) workspace\" }\r\n ,\"microsoft.machinelearningexperimentation/accounts\": { \"SingularDisplayName\": \"Microsoft.MachineLearningExperimentation account\" }\r\n ,\"microsoft.machinelearningexperimentation/accounts/workspaces\": { \"SingularDisplayName\": \"Microsoft.MachineLearningExperimentation accounts workspace\" }\r\n ,\"microsoft.machinelearningexperimentation/accounts/workspaces/projects\": { \"SingularDisplayName\": \"Microsoft.MachineLearningExperimentation accounts workspaces project\" }\r\n ,\"microsoft.machinelearningservices/aistudio\": { \"SingularDisplayName\": \"Azure AI Foundry\" }\r\n ,\"microsoft.machinelearningservices/aistudiocreate\": { \"SingularDisplayName\": \"Azure AI Foundry\" }\r\n ,\"microsoft.machinelearningservices/registries\": { \"SingularDisplayName\": \"Azure Machine Learning registry\" }\r\n ,\"microsoft.machinelearningservices/workspaces\": { \"SingularDisplayName\": \"Azure Machine Learning workspace\" }\r\n ,\"microsoft.machinelearningservices/workspaces/onlineendpoints\": { \"SingularDisplayName\": \"Machine learning online endpoint\" }\r\n ,\"microsoft.machinelearningservices/workspaces/onlineendpoints/deployments\": { \"SingularDisplayName\": \"Machine learning online deployment\" }\r\n ,\"microsoft.machinelearningservices/workspacescreate\": { \"SingularDisplayName\": \"Azure Machine Learning workspace\" }\r\n ,\"microsoft.maintenance/configurationassignments\": { \"SingularDisplayName\": \"Microsoft.Maintenance configuration assignment\" }\r\n ,\"microsoft.maintenance/maintenanceconfigurations\": { \"SingularDisplayName\": \"Maintenance Configuration\" }\r\n ,\"microsoft.maintenance/maintenanceconfigurationsaumbladeresource\": { \"SingularDisplayName\": \"Maintenance configuration\" }\r\n ,\"microsoft.maintenance/maintenanceconfigurationsbladeresource\": { \"SingularDisplayName\": \"Maintenance configuration\" }\r\n ,\"microsoft.maintenance/publicmaintenanceconfigurations\": { \"SingularDisplayName\": \"Microsoft.Maintenance public maintenance configuration\" }\r\n ,\"microsoft.managedidentity/identities\": { \"SingularDisplayName\": \"Microsoft.ManagedIdentity identity\" }\r\n ,\"microsoft.managedidentity/userassignedidentities\": { \"SingularDisplayName\": \"Managed Identity\" }\r\n ,\"microsoft.managednetwork/managednetworks\": { \"SingularDisplayName\": \"Microsoft.ManagedNetwork managed network\" }\r\n ,\"microsoft.managednetwork/managednetworks/managednetworkgroups\": { \"SingularDisplayName\": \"Microsoft.ManagedNetwork managed networks managed network group\" }\r\n ,\"microsoft.managednetwork/managednetworks/managednetworkpeeringpolicies\": { \"SingularDisplayName\": \"Microsoft.ManagedNetwork managed networks managed network peering policy\" }\r\n ,\"microsoft.managednetworkfabric/accesscontrollists\": { \"SingularDisplayName\": \"Access Control List (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/internetgatewayrules\": { \"SingularDisplayName\": \"Internet Gateway Rule (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/internetgateways\": { \"SingularDisplayName\": \"Internet Gateway (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/ipcommunities\": { \"SingularDisplayName\": \"IP Community (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/ipextendedcommunities\": { \"SingularDisplayName\": \"IP Extended Community (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/ipprefixes\": { \"SingularDisplayName\": \"IP Prefix (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/l2isolationdomains\": { \"SingularDisplayName\": \"Layer 2 Isolation Domain (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/l3isolationdomains\": { \"SingularDisplayName\": \"Layer 3 Isolation Domain (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/l3isolationdomains/externalnetworks\": { \"SingularDisplayName\": \"External Network (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/l3isolationdomains/internalnetworks\": { \"SingularDisplayName\": \"Internal Network (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/neighborgroups\": { \"SingularDisplayName\": \"Neighbor Group (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkdevices\": { \"SingularDisplayName\": \"Network Device (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkdevices/networkinterfaces\": { \"SingularDisplayName\": \"Network Interface (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkfabriccontrollers\": { \"SingularDisplayName\": \"Network Fabric Controller (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkfabrics\": { \"SingularDisplayName\": \"Network Fabric (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkfabrics/networktonetworkinterconnects\": { \"SingularDisplayName\": \"Network to Network Interconnect (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkfabricskus\": { \"SingularDisplayName\": \"Network Fabric SKU (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkmonitors\": { \"SingularDisplayName\": \"Microsoft.ManagedNetworkFabric network monitor\" }\r\n ,\"microsoft.managednetworkfabric/networkpacketbrokers\": { \"SingularDisplayName\": \"Network Packet Broker (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkracks\": { \"SingularDisplayName\": \"Network Rack (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networktaprules\": { \"SingularDisplayName\": \"Network Tap Rule (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networktaps\": { \"SingularDisplayName\": \"Network Tap (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/routepolicies\": { \"SingularDisplayName\": \"Route Policy (Operator Nexus)\" }\r\n ,\"microsoft.managedservices/marketplaceregistrationdefinitions\": { \"SingularDisplayName\": \"Microsoft.ManagedServices marketplace registration definition\" }\r\n ,\"microsoft.managedservices/registrationassignments\": { \"SingularDisplayName\": \"Microsoft.ManagedServices registration assignment\" }\r\n ,\"microsoft.managedservices/registrationdefinitions\": { \"SingularDisplayName\": \"Azure Lighthouse\" }\r\n ,\"microsoft.management/managementgroups\": { \"SingularDisplayName\": \"Microsoft.Management management group\" }\r\n ,\"microsoft.management/managementgroups/microsoft.resources/deploymentstacks\": { \"SingularDisplayName\": \"Deployment stack\" }\r\n ,\"microsoft.management/managementgroups/providers/privatelinkassociations\": { \"SingularDisplayName\": \"Application Gateway\" }\r\n ,\"microsoft.management/managementgroups/providers/templatespecs\": { \"SingularDisplayName\": \"Template spec\" }\r\n ,\"microsoft.management/managementgroups/settings\": { \"SingularDisplayName\": \"Microsoft.Management management groups setting\" }\r\n ,\"microsoft.management/managementgroups/subscriptions\": { \"SingularDisplayName\": \"Microsoft.Management management groups subscription\" }\r\n ,\"microsoft.management/servicegroups\": { \"SingularDisplayName\": \"Service group\" }\r\n ,\"microsoft.managementpartner/partners\": { \"SingularDisplayName\": \"Microsoft.ManagementPartner partner\" }\r\n ,\"microsoft.manufacturingplatform/manufacturingdataservices\": { \"SingularDisplayName\": \"Factory Operations Agent in Azure AI Foundry\" }\r\n ,\"microsoft.maps/accounts\": { \"SingularDisplayName\": \"Azure Maps Account\" }\r\n ,\"microsoft.maps/accounts/creators\": { \"SingularDisplayName\": \"Azure Maps Creator Resource\" }\r\n ,\"microsoft.marketplace/privatestores\": { \"SingularDisplayName\": \"Microsoft.Marketplace private store\" }\r\n ,\"microsoft.marketplace/privatestores/adminrequestapprovals\": { \"SingularDisplayName\": \"Microsoft.Marketplace private stores admin request approval\" }\r\n ,\"microsoft.marketplace/privatestores/collections\": { \"SingularDisplayName\": \"Microsoft.Marketplace private stores collection\" }\r\n ,\"microsoft.marketplace/privatestores/collections/offers\": { \"SingularDisplayName\": \"Microsoft.Marketplace private stores collections offer\" }\r\n ,\"microsoft.marketplace/privatestores/offers\": { \"SingularDisplayName\": \"Microsoft.Marketplace private stores offer\" }\r\n ,\"microsoft.marketplace/privatestores/requestapprovals\": { \"SingularDisplayName\": \"Microsoft.Marketplace private stores request approval\" }\r\n ,\"microsoft.media/mediaservices\": { \"SingularDisplayName\": \"Media service\" }\r\n ,\"microsoft.media/mediaservices/accountfilters\": { \"SingularDisplayName\": \"Microsoft.Media media services account filter\" }\r\n ,\"microsoft.media/mediaservices/assets\": { \"SingularDisplayName\": \"Microsoft.Media media services asset\" }\r\n ,\"microsoft.media/mediaservices/assets/assetfilters\": { \"SingularDisplayName\": \"Microsoft.Media media services assets asset filter\" }\r\n ,\"microsoft.media/mediaservices/assets/tracks\": { \"SingularDisplayName\": \"Microsoft.Media media services assets track\" }\r\n ,\"microsoft.media/mediaservices/assets/tracks/operationresults\": { \"SingularDisplayName\": \"Microsoft.Media media services assets tracks operation result\" }\r\n ,\"microsoft.media/mediaservices/assets/tracks/operationstatuses\": { \"SingularDisplayName\": \"Microsoft.Media media services assets tracks operation statuse\" }\r\n ,\"microsoft.media/mediaservices/contentkeypolicies\": { \"SingularDisplayName\": \"Microsoft.Media media services content key policy\" }\r\n ,\"microsoft.media/mediaservices/liveevents\": { \"SingularDisplayName\": \"Live event\" }\r\n ,\"microsoft.media/mediaservices/liveevents/liveoutputs\": { \"SingularDisplayName\": \"Microsoft.Media mediaservices live events live output\" }\r\n ,\"microsoft.media/mediaservices/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.Media mediaservices private endpoint connection\" }\r\n ,\"microsoft.media/mediaservices/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.Media mediaservices private link resource\" }\r\n ,\"microsoft.media/mediaservices/streamingendpoints\": { \"SingularDisplayName\": \"Streaming Endpoint\" }\r\n ,\"microsoft.media/mediaservices/streaminglocators\": { \"SingularDisplayName\": \"Microsoft.Media media services streaming locator\" }\r\n ,\"microsoft.media/mediaservices/streamingpolicies\": { \"SingularDisplayName\": \"Microsoft.Media media services streaming policy\" }\r\n ,\"microsoft.media/mediaservices/transforms\": { \"SingularDisplayName\": \"Microsoft.Media media services transform\" }\r\n ,\"microsoft.media/mediaservices/transforms/jobs\": { \"SingularDisplayName\": \"Microsoft.Media media services transforms job\" }\r\n ,\"microsoft.mesh/worlds\": { \"SingularDisplayName\": \"Microsoft.Mesh world\" }\r\n ,\"microsoft.mesh/worlds/events\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds event\" }\r\n ,\"microsoft.mesh/worlds/events/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds events access policy\" }\r\n ,\"microsoft.mesh/worlds/spaces\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds space\" }\r\n ,\"microsoft.mesh/worlds/spaces/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds spaces access policy\" }\r\n ,\"microsoft.mesh/worlds/templates\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds template\" }\r\n ,\"microsoft.mesh/worlds/templates/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds templates access policy\" }\r\n ,\"microsoft.messagingcatalog/catalogs\": { \"SingularDisplayName\": \"Microsoft.MessagingCatalog catalog\" }\r\n ,\"microsoft.messagingconnectors/connectors\": { \"SingularDisplayName\": \"Microsoft.MessagingConnectors connector\" }\r\n ,\"microsoft.metaverse/metaverses\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverse\" }\r\n ,\"microsoft.metaverse/metaverses/events\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses event\" }\r\n ,\"microsoft.metaverse/metaverses/events/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses events access policy\" }\r\n ,\"microsoft.metaverse/metaverses/spaces\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses space\" }\r\n ,\"microsoft.metaverse/metaverses/spaces/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses spaces access policy\" }\r\n ,\"microsoft.metaverse/metaverses/templates\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses template\" }\r\n ,\"microsoft.metaverse/metaverses/templates/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses templates access policy\" }\r\n ,\"microsoft.migrate/assessmentprojects\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment project\" }\r\n ,\"microsoft.migrate/assessmentprojects/aksassessmentoptions\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects aks assessment option\" }\r\n ,\"microsoft.migrate/assessmentprojects/aksassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects aks assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/aksassessments/assessedwebapps\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects aks assessments assessed web app\" }\r\n ,\"microsoft.migrate/assessmentprojects/aksassessments/clusters\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects aks assessments cluster\" }\r\n ,\"microsoft.migrate/assessmentprojects/aksassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects aks assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/assessmentoptions\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects assessment option\" }\r\n ,\"microsoft.migrate/assessmentprojects/assessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/assessments/assessedmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects assessments assessed machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/assessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/avsassessmentoptions\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects avs assessment option\" }\r\n ,\"microsoft.migrate/assessmentprojects/avsassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects avs assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/avsassessments/avsassessedmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects avs assessments avs assessed machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/avsassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects avs assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business case\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/avssummaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases avs summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/evaluatedavsmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases evaluated avs machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/evaluatedmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases evaluated machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/evaluatedsqlentities\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases evaluated sql entity\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/evaluatedwebapps\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases evaluated web app\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/iaassummaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases iaas summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/overviewsummaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases overview summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/paassummaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases paas summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects group\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/assessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/assessments/assessedmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups assessments assessed machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/avsassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups avs assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/avsassessments/avsassessedmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups avs assessments avs assessed machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments/assessedsqldatabases\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessments assessed sql database\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments/assessedsqlinstances\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessments assessed sql instance\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments/assessedsqlmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessments assessed sql machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments/recommendedassessedentities\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessments recommended assessed entity\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/webappassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups web app assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/webappassessments/assessedwebapps\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups web app assessments assessed web app\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/webappassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups web app assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/webappassessments/webappserviceplans\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups web app assessments web app service plan\" }\r\n ,\"microsoft.migrate/assessmentprojects/heterogeneousassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects heterogeneous assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/heterogeneousassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects heterogeneous assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/hypervcollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects hypervcollector\" }\r\n ,\"microsoft.migrate/assessmentprojects/importcollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects importcollector\" }\r\n ,\"microsoft.migrate/assessmentprojects/importsqlcollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects import sql collector\" }\r\n ,\"microsoft.migrate/assessmentprojects/machines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects private endpoint connection\" }\r\n ,\"microsoft.migrate/assessmentprojects/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects private link resource\" }\r\n ,\"microsoft.migrate/assessmentprojects/projectsummary\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects project summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/servercollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects servercollector\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessmentoptions\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessment option\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessments/assessedsqldatabases\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessments assessed sql database\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessments/assessedsqlinstances\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessments assessed sql instance\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessments/assessedsqlmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessments assessed sql machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlcollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sqlcollector\" }\r\n ,\"microsoft.migrate/assessmentprojects/vmwarecollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects vmwarecollector\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappassessmentoptions\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app assessment option\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappassessments/assessedwebapps\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app assessments assessed web app\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappassessments/webappserviceplans\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app assessments web app service plan\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappcollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app collector\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappcompoundassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app compound assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappcompoundassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app compound assessments summary\" }\r\n ,\"microsoft.migrate/migrateprojects\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate project\" }\r\n ,\"microsoft.migrate/migrateprojects/databaseinstances\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate projects database instance\" }\r\n ,\"microsoft.migrate/migrateprojects/databases\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate projects database\" }\r\n ,\"microsoft.migrate/migrateprojects/machines\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate projects machine\" }\r\n ,\"microsoft.migrate/migrateprojects/migrateevents\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate projects migrate event\" }\r\n ,\"microsoft.migrate/migrateprojects/solutions\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate projects solution\" }\r\n ,\"microsoft.migrate/modernizeprojects\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize project\" }\r\n ,\"microsoft.migrate/modernizeprojects/deployedresources\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects deployed resource\" }\r\n ,\"microsoft.migrate/modernizeprojects/jobs\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects job\" }\r\n ,\"microsoft.migrate/modernizeprojects/jobs/operations\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects jobs operation\" }\r\n ,\"microsoft.migrate/modernizeprojects/migrateagents\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects migrate agent\" }\r\n ,\"microsoft.migrate/modernizeprojects/migrateagents/operations\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects migrate agents operation\" }\r\n ,\"microsoft.migrate/modernizeprojects/operations\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects operation\" }\r\n ,\"microsoft.migrate/modernizeprojects/workloaddeployments\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects workload deployment\" }\r\n ,\"microsoft.migrate/modernizeprojects/workloaddeployments/operations\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects workload deployments operation\" }\r\n ,\"microsoft.migrate/modernizeprojects/workloadinstances\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects workload instance\" }\r\n ,\"microsoft.migrate/modernizeprojects/workloadinstances/operations\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects workload instances operation\" }\r\n ,\"microsoft.migrate/movecollections\": { \"SingularDisplayName\": \"Microsoft.Migrate move collection\" }\r\n ,\"microsoft.migrate/movecollections/moveresources\": { \"SingularDisplayName\": \"Microsoft.Migrate move collections move resource\" }\r\n ,\"microsoft.migrate/projects\": { \"SingularDisplayName\": \"Migration project\" }\r\n ,\"microsoft.mission/approvals\": { \"SingularDisplayName\": \"Approval\" }\r\n ,\"microsoft.mission/catalogs\": { \"SingularDisplayName\": \"Catalog\" }\r\n ,\"microsoft.mission/communities\": { \"SingularDisplayName\": \"Community\" }\r\n ,\"microsoft.mission/communities/communityendpoints\": { \"SingularDisplayName\": \"Community endpoint\" }\r\n ,\"microsoft.mission/communities/transithubs\": { \"SingularDisplayName\": \"Transit hub\" }\r\n ,\"microsoft.mission/enclaveconnections\": { \"SingularDisplayName\": \"Enclave connection\" }\r\n ,\"microsoft.mission/externalconnections\": { \"SingularDisplayName\": \"Microsoft.Mission external connection\" }\r\n ,\"microsoft.mission/internalconnections\": { \"SingularDisplayName\": \"Microsoft.Mission internal connection\" }\r\n ,\"microsoft.mission/virtualenclaves\": { \"SingularDisplayName\": \"Enclave\" }\r\n ,\"microsoft.mission/virtualenclaves/enclaveendpoints\": { \"SingularDisplayName\": \"Enclave endpoint\" }\r\n ,\"microsoft.mission/virtualenclaves/endpoints\": { \"SingularDisplayName\": \"Endpoint\" }\r\n ,\"microsoft.mission/virtualenclaves/workloads\": { \"SingularDisplayName\": \"Workload\" }\r\n ,\"microsoft.mixedreality/objectanchorsaccounts\": { \"SingularDisplayName\": \"Object Anchors Account\" }\r\n ,\"microsoft.mixedreality/objectunderstandingaccounts\": { \"SingularDisplayName\": \"Object Understanding Account\" }\r\n ,\"microsoft.mixedreality/remoterenderingaccounts\": { \"SingularDisplayName\": \"Remote Rendering Account\" }\r\n ,\"microsoft.mixedreality/spatialanchorsaccounts\": { \"SingularDisplayName\": \"Spatial Anchors Account\" }\r\n ,\"microsoft.mixedreality/spatialmapsaccounts\": { \"SingularDisplayName\": \"Microsoft.MixedReality spatial maps account\" }\r\n ,\"microsoft.mobilenetwork/amfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork amf deployment\" }\r\n ,\"microsoft.mobilenetwork/clusterservices\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork cluster service\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks\": { \"SingularDisplayName\": \"Mobile Network\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks/datanetworks\": { \"SingularDisplayName\": \"Data Network\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks/services\": { \"SingularDisplayName\": \"Service\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks/simpolicies\": { \"SingularDisplayName\": \"SIM Policy\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks/sites\": { \"SingularDisplayName\": \"Mobile Network Site\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks/slices\": { \"SingularDisplayName\": \"Slice\" }\r\n ,\"microsoft.mobilenetwork/nrfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork nrf deployment\" }\r\n ,\"microsoft.mobilenetwork/nssfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork nssf deployment\" }\r\n ,\"microsoft.mobilenetwork/observabilityservices\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork observability service\" }\r\n ,\"microsoft.mobilenetwork/packetcorecontrolplanes\": { \"SingularDisplayName\": \"Packet Core Control Plane\" }\r\n ,\"microsoft.mobilenetwork/packetcorecontrolplanes/packetcoredataplanes\": { \"SingularDisplayName\": \"Packet Core Data Plane\" }\r\n ,\"microsoft.mobilenetwork/packetcorecontrolplanes/packetcoredataplanes/attacheddatanetworks\": { \"SingularDisplayName\": \"Attached Data Network\" }\r\n ,\"microsoft.mobilenetwork/packetcorecontrolplaneversions\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork packet core control plane version\" }\r\n ,\"microsoft.mobilenetwork/radioaccessnetworks\": { \"SingularDisplayName\": \"Radio Access Network Insights\" }\r\n ,\"microsoft.mobilenetwork/sdmdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork sdm deployment\" }\r\n ,\"microsoft.mobilenetwork/simgroups\": { \"SingularDisplayName\": \"SIM Group\" }\r\n ,\"microsoft.mobilenetwork/simgroups/sims\": { \"SingularDisplayName\": \"SIM\" }\r\n ,\"microsoft.mobilenetwork/sims\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork sim\" }\r\n ,\"microsoft.mobilenetwork/smfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork smf deployment\" }\r\n ,\"microsoft.mobilenetwork/upfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork upf deployment\" }\r\n ,\"microsoft.mobilenetwork/virtualizedmmedeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork virtualized mme deployment\" }\r\n ,\"microsoft.mobilenetwork/vnfagentdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork vnf agent deployment\" }\r\n ,\"microsoft.mobilepacketcore/amfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore amf deployment\" }\r\n ,\"microsoft.mobilepacketcore/clusterservices\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore cluster service\" }\r\n ,\"microsoft.mobilepacketcore/networkfunctions\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore network function\" }\r\n ,\"microsoft.mobilepacketcore/nrfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore nrf deployment\" }\r\n ,\"microsoft.mobilepacketcore/nssfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore nssf deployment\" }\r\n ,\"microsoft.mobilepacketcore/observabilityservices\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore observability service\" }\r\n ,\"microsoft.mobilepacketcore/smfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore smf deployment\" }\r\n ,\"microsoft.mobilepacketcore/upfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore upf deployment\" }\r\n ,\"microsoft.modsimworkbench/workbenches\": { \"SingularDisplayName\": \"Modeling and Simulation Workbench\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers\": { \"SingularDisplayName\": \"Chamber\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/connectors\": { \"SingularDisplayName\": \"Chamber Connector\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/filerequests\": { \"SingularDisplayName\": \"Chamber Data Pipeline File Request\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/files\": { \"SingularDisplayName\": \"Chamber Data Pipeline File\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/licenses\": { \"SingularDisplayName\": \"Chamber License\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/storages\": { \"SingularDisplayName\": \"Chamber Storage\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/workloads\": { \"SingularDisplayName\": \"Chamber VM\" }\r\n ,\"microsoft.modsimworkbench/workbenches/sharedstorages\": { \"SingularDisplayName\": \"Shared Storage\" }\r\n ,\"microsoft.monitor/accounts\": { \"SingularDisplayName\": \"Azure Monitor workspace\" }\r\n ,\"microsoft.monitor/investigations\": { \"SingularDisplayName\": \"Microsoft.Monitor investigation\" }\r\n ,\"microsoft.monitor/pipelinegroups\": { \"SingularDisplayName\": \"Azure Monitor pipeline\" }\r\n ,\"microsoft.mysqldiscovery/mysqlsites\": { \"SingularDisplayName\": \"Microsoft.MySQLDiscovery my sqlsite\" }\r\n ,\"microsoft.mysqldiscovery/mysqlsites/agents\": { \"SingularDisplayName\": \"Microsoft.MySQLDiscovery my sqlsites agent\" }\r\n ,\"microsoft.mysqldiscovery/mysqlsites/errorsummaries\": { \"SingularDisplayName\": \"Microsoft.MySQLDiscovery my sqlsites error summary\" }\r\n ,\"microsoft.mysqldiscovery/mysqlsites/mysqlservers\": { \"SingularDisplayName\": \"Microsoft.MySQLDiscovery my sqlsites my sqlserver\" }\r\n ,\"microsoft.mysqldiscovery/mysqlsites/summaries\": { \"SingularDisplayName\": \"Microsoft.MySQLDiscovery my sqlsites summary\" }\r\n ,\"microsoft.netapp/netappaccounts\": { \"SingularDisplayName\": \"NetApp account\" }\r\n ,\"microsoft.netapp/netappaccounts/backuppolicies\": { \"SingularDisplayName\": \"Backup Policy\" }\r\n ,\"microsoft.netapp/netappaccounts/backupvaults\": { \"SingularDisplayName\": \"Backup vault\" }\r\n ,\"microsoft.netapp/netappaccounts/capacitypools\": { \"SingularDisplayName\": \"Capacity pool\" }\r\n ,\"microsoft.netapp/netappaccounts/capacitypools/volumes\": { \"SingularDisplayName\": \"Volume\" }\r\n ,\"microsoft.netapp/netappaccounts/capacitypools/volumes/snapshots\": { \"SingularDisplayName\": \"Snapshot\" }\r\n ,\"microsoft.netapp/netappaccounts/capacitypools/volumes/volumequotarules\": { \"SingularDisplayName\": \"User and group quota\" }\r\n ,\"microsoft.netapp/netappaccounts/snapshotpolicies\": { \"SingularDisplayName\": \"Snapshot policy\" }\r\n ,\"microsoft.netapp/netappaccounts/volumegroups\": { \"SingularDisplayName\": \"VolumeGroup\" }\r\n ,\"microsoft.network/applicationgatewayavailablessloptions\": { \"SingularDisplayName\": \"Microsoft.Network application gateway available ssl option\" }\r\n ,\"microsoft.network/applicationgatewayavailablessloptions/predefinedpolicies\": { \"SingularDisplayName\": \"Microsoft.Network application gateway available ssl options predefined policy\" }\r\n ,\"microsoft.network/applicationgateways\": { \"SingularDisplayName\": \"Application gateway\" }\r\n ,\"microsoft.network/applicationgatewaywebapplicationfirewallpolicies\": { \"SingularDisplayName\": \"Application Gateway WAF policy\" }\r\n ,\"microsoft.network/applicationsecuritygroups\": { \"SingularDisplayName\": \"Application security group\" }\r\n ,\"microsoft.network/azurefirewalls\": { \"SingularDisplayName\": \"Firewall\" }\r\n ,\"microsoft.network/azurewebcategories\": { \"SingularDisplayName\": \"Microsoft.Network Azure web category\" }\r\n ,\"microsoft.network/bastionhosts\": { \"SingularDisplayName\": \"Bastion\" }\r\n ,\"microsoft.network/cloudserviceslots\": { \"SingularDisplayName\": \"Microsoft.Network cloud service slot\" }\r\n ,\"microsoft.network/connections\": { \"SingularDisplayName\": \"Connection\" }\r\n ,\"microsoft.network/customipprefixes\": { \"SingularDisplayName\": \"Custom IP Prefix\" }\r\n ,\"microsoft.network/ddoscustompolicies\": { \"SingularDisplayName\": \"Microsoft.Network DDoS custom policy\" }\r\n ,\"microsoft.network/ddosprotectionplans\": { \"SingularDisplayName\": \"DDoS protection plan\" }\r\n ,\"microsoft.network/dnsforwardingrulesets\": { \"SingularDisplayName\": \"DNS forwarding ruleset\" }\r\n ,\"microsoft.network/dnsresolverdomainlists\": { \"SingularDisplayName\": \"DNS Domain List\" }\r\n ,\"microsoft.network/dnsresolverpolicies\": { \"SingularDisplayName\": \"DNS Security Policy\" }\r\n ,\"microsoft.network/dnsresolvers\": { \"SingularDisplayName\": \"DNS private resolver\" }\r\n ,\"microsoft.network/dnszones\": { \"SingularDisplayName\": \"DNS zone\" }\r\n ,\"microsoft.network/dscpconfigurations\": { \"SingularDisplayName\": \"Microsoft.Network DSCP configuration\" }\r\n ,\"microsoft.network/expressroutecircuits\": { \"SingularDisplayName\": \"ExpressRoute circuit\" }\r\n ,\"microsoft.network/expressroutecrossconnections\": { \"SingularDisplayName\": \"Microsoft.Network express route cross connection\" }\r\n ,\"microsoft.network/expressroutecrossconnections/peerings\": { \"SingularDisplayName\": \"Microsoft.Network express route cross connections peering\" }\r\n ,\"microsoft.network/expressroutegateways\": { \"SingularDisplayName\": \"ExpressRoute Gateway\" }\r\n ,\"microsoft.network/expressroutegateways/expressrouteconnections\": { \"SingularDisplayName\": \"Microsoft.Network express route gateways express route connection\" }\r\n ,\"microsoft.network/expressrouteports\": { \"SingularDisplayName\": \"ExpressRoute Direct\" }\r\n ,\"microsoft.network/expressrouteportslocations\": { \"SingularDisplayName\": \"Microsoft.Network express route ports location\" }\r\n ,\"microsoft.network/firewallpolicies\": { \"SingularDisplayName\": \"Firewall Policy\" }\r\n ,\"microsoft.network/frontdoors\": { \"SingularDisplayName\": \"Front Door and CDN profiles\" }\r\n ,\"microsoft.network/frontdoorwebapplicationfirewallpolicies\": { \"SingularDisplayName\": \"Front Door WAF policy\" }\r\n ,\"microsoft.network/ipallocations\": { \"SingularDisplayName\": \"Microsoft.Network IP allocation\" }\r\n ,\"microsoft.network/ipgroups\": { \"SingularDisplayName\": \"IP Group\" }\r\n ,\"microsoft.network/loadbalancers\": { \"SingularDisplayName\": \"Load balancer\" }\r\n ,\"microsoft.network/localnetworkgateways\": { \"SingularDisplayName\": \"Local network gateway\" }\r\n ,\"microsoft.network/natgateways\": { \"SingularDisplayName\": \"NAT gateway\" }\r\n ,\"microsoft.network/networkexperimentprofiles\": { \"SingularDisplayName\": \"Microsoft.Network network experiment profile\" }\r\n ,\"microsoft.network/networkexperimentprofiles/experiments\": { \"SingularDisplayName\": \"Microsoft.Network network experiment profiles experiment\" }\r\n ,\"microsoft.network/networkinterfaces\": { \"SingularDisplayName\": \"Network interface\" }\r\n ,\"microsoft.network/networkmanagerconnections\": { \"SingularDisplayName\": \"Microsoft.Network network manager connection\" }\r\n ,\"microsoft.network/networkmanagers\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/connectivityconfigurations\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/ipampools\": { \"SingularDisplayName\": \"IP address pool\" }\r\n ,\"microsoft.network/networkmanagers/networkgroups\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/routingconfigurations\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/securityadminconfigurations\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/securityuserconfigurations\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/verifierworkspaces\": { \"SingularDisplayName\": \"Verifier Workspace\" }\r\n ,\"microsoft.network/networkprofiles\": { \"SingularDisplayName\": \"Microsoft.Network network profile\" }\r\n ,\"microsoft.network/networksecuritygroups\": { \"SingularDisplayName\": \"Network security group\" }\r\n ,\"microsoft.network/networksecurityperimeters\": { \"SingularDisplayName\": \"Network Security Perimeter\" }\r\n ,\"microsoft.network/networksecurityperimeters/profiles\": { \"SingularDisplayName\": \"Network Security Perimeter Profile\" }\r\n ,\"microsoft.network/networkverifiers\": { \"SingularDisplayName\": \"Virtual Network Verifier\" }\r\n ,\"microsoft.network/networkvirtualappliances\": { \"SingularDisplayName\": \"Microsoft.Network network virtual appliance\" }\r\n ,\"microsoft.network/networkwatchers\": { \"SingularDisplayName\": \"Network Watcher\" }\r\n ,\"microsoft.network/networkwatchers/flowlogs\": { \"SingularDisplayName\": \"Flow log\" }\r\n ,\"microsoft.network/p2svpngateways\": { \"SingularDisplayName\": \"VPN Gateway (Point to Site)\" }\r\n ,\"microsoft.network/privatednszones\": { \"SingularDisplayName\": \"Private DNS zone\" }\r\n ,\"microsoft.network/privatednszones/virtualnetworklinks\": { \"SingularDisplayName\": \"Virtual network link\" }\r\n ,\"microsoft.network/privateendpoints\": { \"SingularDisplayName\": \"Private endpoint\" }\r\n ,\"microsoft.network/privatelinkservices\": { \"SingularDisplayName\": \"Private link service\" }\r\n ,\"microsoft.network/publicipaddresses\": { \"SingularDisplayName\": \"Public IP address\" }\r\n ,\"microsoft.network/publicipprefixes\": { \"SingularDisplayName\": \"Public IP Prefix\" }\r\n ,\"microsoft.network/routefilters\": { \"SingularDisplayName\": \"Route filter\" }\r\n ,\"microsoft.network/routetables\": { \"SingularDisplayName\": \"Route table\" }\r\n ,\"microsoft.network/securitypartnerproviders\": { \"SingularDisplayName\": \"Microsoft.Network security partner provider\" }\r\n ,\"microsoft.network/serviceendpointpolicies\": { \"SingularDisplayName\": \"Service endpoint policy\" }\r\n ,\"microsoft.network/trafficmanagergeographichierarchies\": { \"SingularDisplayName\": \"Microsoft.Network traffic manager geographic hierarchy\" }\r\n ,\"microsoft.network/trafficmanagerprofiles\": { \"SingularDisplayName\": \"Traffic Manager profile\" }\r\n ,\"microsoft.network/trafficmanagerusermetricskeys\": { \"SingularDisplayName\": \"Microsoft.Network traffic manager user metrics key\" }\r\n ,\"microsoft.network/virtualhubs\": { \"SingularDisplayName\": \"Microsoft.Network/virtualHub\" }\r\n ,\"microsoft.network/virtualnetworkgateways\": { \"SingularDisplayName\": \"Virtual network gateway\" }\r\n ,\"microsoft.network/virtualnetworks\": { \"SingularDisplayName\": \"Virtual network\" }\r\n ,\"microsoft.network/virtualnetworktaps\": { \"SingularDisplayName\": \"Virtual network terminal access point\" }\r\n ,\"microsoft.network/virtualrouters\": { \"SingularDisplayName\": \"Microsoft.Network virtual router\" }\r\n ,\"microsoft.network/virtualrouters/peerings\": { \"SingularDisplayName\": \"Microsoft.Network virtual routers peering\" }\r\n ,\"microsoft.network/virtualwans\": { \"SingularDisplayName\": \"Virtual WAN\" }\r\n ,\"microsoft.network/vpngateways\": { \"SingularDisplayName\": \"VPN Gateway (Site to Site)\" }\r\n ,\"microsoft.network/vpngateways/vpnconnections\": { \"SingularDisplayName\": \"Microsoft.Network VPN gateways VPN connection\" }\r\n ,\"microsoft.network/vpngateways/vpnconnections/vpnlinkconnections\": { \"SingularDisplayName\": \"Microsoft.Network VPN gateways VPN connections VPN link connection\" }\r\n ,\"microsoft.network/vpnserverconfigurations\": { \"SingularDisplayName\": \"Microsoft.Network VPN server configuration\" }\r\n ,\"microsoft.network/vpnsites\": { \"SingularDisplayName\": \"Microsoft.Network VPN site\" }\r\n ,\"microsoft.network/vpnsites/vpnsitelinks\": { \"SingularDisplayName\": \"Microsoft.Network VPN sites VPN site link\" }\r\n ,\"microsoft.networkanalytics/dataconnectors\": { \"SingularDisplayName\": \"AIOps - Data Connector\" }\r\n ,\"microsoft.networkanalytics/datalakehouses\": { \"SingularDisplayName\": \"AIOps - Data LakeHouse\" }\r\n ,\"microsoft.networkanalytics/dataproducts\": { \"SingularDisplayName\": \"Azure Operator Insights ? Data Product\" }\r\n ,\"microsoft.networkanalytics/dataproducts/datatypes\": { \"SingularDisplayName\": \"Data Type\" }\r\n ,\"microsoft.networkanalytics/dataproductscatalogs\": { \"SingularDisplayName\": \"Microsoft.NetworkAnalytics data products catalog\" }\r\n ,\"microsoft.networkanalytics/metricsingestionendpoints\": { \"SingularDisplayName\": \"Microsoft.NetworkAnalytics metrics ingestion endpoint\" }\r\n ,\"microsoft.networkanalytics/networkanalyticsproducts\": { \"SingularDisplayName\": \"Microsoft.NetworkAnalytics network analytics product\" }\r\n ,\"microsoft.networkcloud/baremetalmachines\": { \"SingularDisplayName\": \"Bare Metal Machine (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/cloudservicesnetworks\": { \"SingularDisplayName\": \"Cloud Services Network (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/clustermanagers\": { \"SingularDisplayName\": \"Cluster Manager (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/clusters\": { \"SingularDisplayName\": \"Cluster (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/clusters/baremetalmachinekeysets\": { \"SingularDisplayName\": \"Cluster Bare Metal Machine Key Set (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/clusters/bmckeysets\": { \"SingularDisplayName\": \"Cluster Baseboard Management Controller Key Set (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/clusters/metricsconfigurations\": { \"SingularDisplayName\": \"Cluster Metrics Configuration (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/edgeclustermachineskus\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud edge cluster machine SKU\" }\r\n ,\"microsoft.networkcloud/edgeclusterruntimeversions\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud edge cluster runtime version\" }\r\n ,\"microsoft.networkcloud/edgeclusters\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud edge cluster\" }\r\n ,\"microsoft.networkcloud/edgeclusters/nodes\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud edge clusters node\" }\r\n ,\"microsoft.networkcloud/edgeclusterskus\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud edge cluster SKU\" }\r\n ,\"microsoft.networkcloud/kubernetesclusters\": { \"SingularDisplayName\": \"Kubernetes Cluster (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/kubernetesclusters/agentpools\": { \"SingularDisplayName\": \"Agent Pool (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/kubernetesclusters/features\": { \"SingularDisplayName\": \"Kubernetes Cluster Feature (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/l2networks\": { \"SingularDisplayName\": \"Layer 2 Network (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/l3networks\": { \"SingularDisplayName\": \"Layer 3 Network (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/racks\": { \"SingularDisplayName\": \"Compute Rack (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/rackskus\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud rack SKU\" }\r\n ,\"microsoft.networkcloud/registrationhubs\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud registration hub\" }\r\n ,\"microsoft.networkcloud/registrationhubs/images\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud registration hubs image\" }\r\n ,\"microsoft.networkcloud/registrationhubs/machines\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud registration hubs machine\" }\r\n ,\"microsoft.networkcloud/storageappliances\": { \"SingularDisplayName\": \"Storage Appliance (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/trunkednetworks\": { \"SingularDisplayName\": \"Trunked Network (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/virtualmachines\": { \"SingularDisplayName\": \"Virtual Machine (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/virtualmachines/consoles\": { \"SingularDisplayName\": \"Virtual Machine Console (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/volumes\": { \"SingularDisplayName\": \"Volume (Operator Nexus)\" }\r\n ,\"microsoft.networkfunction/azuretrafficcollectors\": { \"SingularDisplayName\": \"ExpressRoute traffic collector\" }\r\n ,\"microsoft.networkfunction/meshvpns\": { \"SingularDisplayName\": \"Mesh VPN\" }\r\n ,\"microsoft.nexusidentity/identitycontrollers\": { \"SingularDisplayName\": \"Microsoft.NexusIdentity identity controller\" }\r\n ,\"microsoft.nexusidentity/identitysets\": { \"SingularDisplayName\": \"Microsoft.NexusIdentity identity set\" }\r\n ,\"microsoft.notebooks/notebookproxies\": { \"SingularDisplayName\": \"Microsoft.Notebooks notebook proxy\" }\r\n ,\"microsoft.notificationhubs/namespaces\": { \"SingularDisplayName\": \"Notification Hub Namespace\" }\r\n ,\"microsoft.notificationhubs/namespaces/notificationhubs\": { \"SingularDisplayName\": \"Notification Hub\" }\r\n ,\"microsoft.objectstore/osnamespaces\": { \"SingularDisplayName\": \"Microsoft.ObjectStore os namespace\" }\r\n })[tolower(id)]\r\n}\r\n", - "$fxv#3": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData/Internal')\r\n_resource_type_4(id: string) {\r\n dynamic({\r\n \"microsoft.offazure/hypervsites\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv site\" }\r\n ,\"microsoft.offazure/hypervsites/clusters\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites cluster\" }\r\n ,\"microsoft.offazure/hypervsites/hosts\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites host\" }\r\n ,\"microsoft.offazure/hypervsites/jobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites job\" }\r\n ,\"microsoft.offazure/hypervsites/machines\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites machine\" }\r\n ,\"microsoft.offazure/hypervsites/machines/softwareinventories\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites machines software inventory\" }\r\n ,\"microsoft.offazure/hypervsites/operationsstatus\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites operations statu\" }\r\n ,\"microsoft.offazure/hypervsites/runasaccounts\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites run as account\" }\r\n ,\"microsoft.offazure/importsites\": { \"SingularDisplayName\": \"Microsoft.OffAzure import site\" }\r\n ,\"microsoft.offazure/importsites/deletejobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure import sites delete job\" }\r\n ,\"microsoft.offazure/importsites/exportjobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure import sites export job\" }\r\n ,\"microsoft.offazure/importsites/importjobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure import sites import job\" }\r\n ,\"microsoft.offazure/importsites/jobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure import sites job\" }\r\n ,\"microsoft.offazure/importsites/machines\": { \"SingularDisplayName\": \"Microsoft.OffAzure import sites machine\" }\r\n ,\"microsoft.offazure/mastersites\": { \"SingularDisplayName\": \"Microsoft.OffAzure master site\" }\r\n ,\"microsoft.offazure/mastersites/operationsstatus\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites operations statu\" }\r\n ,\"microsoft.offazure/mastersites/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites private endpoint connection\" }\r\n ,\"microsoft.offazure/mastersites/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites private link resource\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql site\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/discoverysitedatasources\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites discovery site data source\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/jobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites job\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/operationsstatus\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites operations statu\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/runasaccounts\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites run as account\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/sqlavailabilitygroups\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites sql availability group\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/sqldatabases\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites sql database\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/sqlservers\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites sql server\" }\r\n ,\"microsoft.offazure/mastersites/webappsites\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app site\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/discoverysitedatasources\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites discovery site data source\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/extendedmachines\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites extended machine\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/iiswebapplications\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites iis web application\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/iiswebservers\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites iis web server\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/runasaccounts\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites runasaccount\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/tomcatwebapplications\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites tomcat web application\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/tomcatwebservers\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites tomcat web server\" }\r\n ,\"microsoft.offazure/serversites\": { \"SingularDisplayName\": \"Microsoft.OffAzure server site\" }\r\n ,\"microsoft.offazure/serversites/jobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure server sites job\" }\r\n ,\"microsoft.offazure/serversites/machines\": { \"SingularDisplayName\": \"Microsoft.OffAzure server sites machine\" }\r\n ,\"microsoft.offazure/serversites/machines/softwareinventories\": { \"SingularDisplayName\": \"Microsoft.OffAzure server sites machines software inventory\" }\r\n ,\"microsoft.offazure/serversites/operationsstatus\": { \"SingularDisplayName\": \"Microsoft.OffAzure server sites operations statu\" }\r\n ,\"microsoft.offazure/serversites/runasaccounts\": { \"SingularDisplayName\": \"Microsoft.OffAzure server sites run as account\" }\r\n ,\"microsoft.offazure/vmwaresites\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware site\" }\r\n ,\"microsoft.offazure/vmwaresites/hosts\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites host\" }\r\n ,\"microsoft.offazure/vmwaresites/jobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites job\" }\r\n ,\"microsoft.offazure/vmwaresites/machines\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites machine\" }\r\n ,\"microsoft.offazure/vmwaresites/machines/softwareinventories\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites machines software inventory\" }\r\n ,\"microsoft.offazure/vmwaresites/operationsstatus\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites operations statu\" }\r\n ,\"microsoft.offazure/vmwaresites/runasaccounts\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites run as account\" }\r\n ,\"microsoft.offazure/vmwaresites/vcenters\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites vcenter\" }\r\n ,\"microsoft.offazurespringboot/springbootsites\": { \"SingularDisplayName\": \"Microsoft.OffAzureSpringBoot springbootsite\" }\r\n ,\"microsoft.offazurespringboot/springbootsites/errorsummaries\": { \"SingularDisplayName\": \"Microsoft.OffAzureSpringBoot springbootsites error summary\" }\r\n ,\"microsoft.offazurespringboot/springbootsites/springbootapps\": { \"SingularDisplayName\": \"Microsoft.OffAzureSpringBoot springbootsites springbootapp\" }\r\n ,\"microsoft.offazurespringboot/springbootsites/springbootservers\": { \"SingularDisplayName\": \"Microsoft.OffAzureSpringBoot springbootsites springbootserver\" }\r\n ,\"microsoft.offazurespringboot/springbootsites/summaries\": { \"SingularDisplayName\": \"Microsoft.OffAzureSpringBoot springbootsites summary\" }\r\n ,\"microsoft.onlineexperimentation/workspaces\": { \"SingularDisplayName\": \"Online Experimentation Workspace\" }\r\n ,\"microsoft.openenergyplatform/energyservices\": { \"SingularDisplayName\": \"Azure Data Manager for Energy\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspace\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces/applicationregistrations\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspaces application registration\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces/applications\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspaces application\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces/eventgridfilters\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspaces event grid filter\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces/shares\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspaces share\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces/sharesubscriptions\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspaces share subscription\" }\r\n ,\"microsoft.operationalinsights/clusters\": { \"SingularDisplayName\": \"Log Analytics dedicated cluster\" }\r\n ,\"microsoft.operationalinsights/querypacks\": { \"SingularDisplayName\": \"Log Analytics query pack\" }\r\n ,\"microsoft.operationalinsights/workspaces\": { \"SingularDisplayName\": \"Log Analytics workspace\" }\r\n ,\"microsoft.operationsmanagement/managementassociations\": { \"SingularDisplayName\": \"Microsoft.OperationsManagement management association\" }\r\n ,\"microsoft.operationsmanagement/solutions\": { \"SingularDisplayName\": \"Solution\" }\r\n ,\"microsoft.operatorvoicemail/operatorvoicemailinstances\": { \"SingularDisplayName\": \"Microsoft.OperatorVoicemail operator voicemail instance\" }\r\n ,\"microsoft.oraclediscovery/oraclesites\": { \"SingularDisplayName\": \"Microsoft.OracleDiscovery oracle site\" }\r\n ,\"microsoft.oraclediscovery/oraclesites/errorsummaries\": { \"SingularDisplayName\": \"Microsoft.OracleDiscovery oracle sites error summary\" }\r\n ,\"microsoft.oraclediscovery/oraclesites/oracledatabases\": { \"SingularDisplayName\": \"Microsoft.OracleDiscovery oracle sites oracle database\" }\r\n ,\"microsoft.oraclediscovery/oraclesites/oracleservers\": { \"SingularDisplayName\": \"Microsoft.OracleDiscovery oracle sites oracle server\" }\r\n ,\"microsoft.oraclediscovery/oraclesites/summaries\": { \"SingularDisplayName\": \"Microsoft.OracleDiscovery oracle sites summary\" }\r\n ,\"microsoft.orbital/cloudaccessrouters\": { \"SingularDisplayName\": \"Cloud Access Router\" }\r\n ,\"microsoft.orbital/contactprofiles\": { \"SingularDisplayName\": \"Contact Profile\" }\r\n ,\"microsoft.orbital/edgesites\": { \"SingularDisplayName\": \"Edge Site\" }\r\n ,\"microsoft.orbital/geocatalogs\": { \"SingularDisplayName\": \"GeoCatalog\" }\r\n ,\"microsoft.orbital/globalcommunicationssites\": { \"SingularDisplayName\": \"Microsoft.Orbital global communications site\" }\r\n ,\"microsoft.orbital/groundstations\": { \"SingularDisplayName\": \"Ground Station\" }\r\n ,\"microsoft.orbital/l2connections\": { \"SingularDisplayName\": \"L2 Connection\" }\r\n ,\"microsoft.orbital/sdwancontrollers\": { \"SingularDisplayName\": \"SDWAN Controller\" }\r\n ,\"microsoft.orbital/spacecrafts\": { \"SingularDisplayName\": \"Spacecraft\" }\r\n ,\"microsoft.orbital/spacecrafts/contacts\": { \"SingularDisplayName\": \"Contact\" }\r\n ,\"microsoft.orbital/terminals\": { \"SingularDisplayName\": \"Cloud Access Terminal\" }\r\n ,\"microsoft.partnermanagedconsumerrecurrence/recurrences\": { \"SingularDisplayName\": \"Microsoft.PartnerManagedConsumerRecurrence recurrence\" }\r\n ,\"microsoft.partnermanagedconsumerrecurrence/recurrences/operationresult\": { \"SingularDisplayName\": \"Microsoft.PartnerManagedConsumerRecurrence recurrences operation result\" }\r\n ,\"microsoft.peering/peerasns\": { \"SingularDisplayName\": \"Microsoft.Peering peer asn\" }\r\n ,\"microsoft.peering/peerings\": { \"SingularDisplayName\": \"Peering\" }\r\n ,\"microsoft.peering/peerings/registeredasns\": { \"SingularDisplayName\": \"Registered ASN\" }\r\n ,\"microsoft.peering/peerings/registeredprefixes\": { \"SingularDisplayName\": \"Registered prefix\" }\r\n ,\"microsoft.peering/peeringservices\": { \"SingularDisplayName\": \"Peering Service\" }\r\n ,\"microsoft.peering/peeringservices/prefixes\": { \"SingularDisplayName\": \"Peering Service Prefix\" }\r\n ,\"microsoft.pki/pkis\": { \"SingularDisplayName\": \"Microsoft.Pki PKI\" }\r\n ,\"microsoft.pki/pkis/certificateauthorities\": { \"SingularDisplayName\": \"Microsoft.Pki pkis certificate authority\" }\r\n ,\"microsoft.pki/pkis/enrollmentpolicies\": { \"SingularDisplayName\": \"Microsoft.Pki pkis enrollment policy\" }\r\n ,\"microsoft.policyinsights/attestations\": { \"SingularDisplayName\": \"Microsoft.PolicyInsights attestation\" }\r\n ,\"microsoft.policyinsights/policymetadata\": { \"SingularDisplayName\": \"Microsoft.PolicyInsights policy metadata\" }\r\n ,\"microsoft.policyinsights/remediations\": { \"SingularDisplayName\": \"Microsoft.PolicyInsights remediation\" }\r\n ,\"microsoft.portal/consoles\": { \"SingularDisplayName\": \"Microsoft.Portal console\" }\r\n ,\"microsoft.portal/dashboards\": { \"SingularDisplayName\": \"Shared dashboard\" }\r\n ,\"microsoft.portal/tenantconfigurations\": { \"SingularDisplayName\": \"Microsoft.Portal tenant configuration\" }\r\n ,\"microsoft.portal/usersettings\": { \"SingularDisplayName\": \"Microsoft.Portal user setting\" }\r\n ,\"microsoft.portal/virtual-privatedashboards\": { \"SingularDisplayName\": \"Private dashboard\" }\r\n ,\"microsoft.portalservices/copilotsettings\": { \"SingularDisplayName\": \"Microsoft.PortalServices copilot setting\" }\r\n ,\"microsoft.portalservices/dashboards\": { \"SingularDisplayName\": \"Shared dashboard\" }\r\n ,\"microsoft.portalservices/extensions\": { \"SingularDisplayName\": \"Portal Extension\" }\r\n ,\"microsoft.portalservices/extensions/deployments\": { \"SingularDisplayName\": \"Extension Deployment\" }\r\n ,\"microsoft.portalservices/extensions/slots\": { \"SingularDisplayName\": \"Extension Slot\" }\r\n ,\"microsoft.portalservices/extensions/versions\": { \"SingularDisplayName\": \"Extension Version\" }\r\n ,\"microsoft.portalservices/settings\": { \"SingularDisplayName\": \"Microsoft.PortalServices setting\" }\r\n ,\"microsoft.powerbi/privatelinkservicesforpowerbi\": { \"SingularDisplayName\": \"Microsoft.PowerBI private link services for power bi\" }\r\n ,\"microsoft.powerbi/privatelinkservicesforpowerbi/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.PowerBI private link services for power bi private endpoint connection\" }\r\n ,\"microsoft.powerbi/privatelinkservicesforpowerbi/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.PowerBI private link services for power bi private link resource\" }\r\n ,\"microsoft.powerbi/workspacecollections\": { \"SingularDisplayName\": \"Microsoft.PowerBI workspace collection\" }\r\n ,\"microsoft.powerbidedicated/autoscalevcores\": { \"SingularDisplayName\": \"Microsoft.PowerBIDedicated auto scale vcore\" }\r\n ,\"microsoft.powerbidedicated/capacities\": { \"SingularDisplayName\": \"Power BI Embedded\" }\r\n ,\"microsoft.powerplatform/accounts\": { \"SingularDisplayName\": \"Microsoft.PowerPlatform account\" }\r\n ,\"microsoft.premonition/libraries\": { \"SingularDisplayName\": \"Microsoft.Premonition library\" }\r\n ,\"microsoft.premonition/libraries/analyses\": { \"SingularDisplayName\": \"Microsoft.Premonition libraries analyse\" }\r\n ,\"microsoft.premonition/libraries/samples\": { \"SingularDisplayName\": \"Microsoft.Premonition libraries sample\" }\r\n ,\"microsoft.professionalservice/resources\": { \"SingularDisplayName\": \"Professional Service\" }\r\n ,\"microsoft.programmableconnectivity/gateways\": { \"SingularDisplayName\": \"APC Gateway\" }\r\n ,\"microsoft.programmableconnectivity/operatorapiconnections\": { \"SingularDisplayName\": \"APC Operator API Connection\" }\r\n ,\"microsoft.programmableconnectivity/operatorapiplans\": { \"SingularDisplayName\": \"APC Operator API Plan\" }\r\n ,\"microsoft.proposal/proposals\": { \"SingularDisplayName\": \"Microsoft.Proposal proposal\" }\r\n ,\"microsoft.providerhub/providerregistrations\": { \"SingularDisplayName\": \"Resource Provider as a Service\" }\r\n ,\"microsoft.providerhub/providerregistrations/customrollouts\": { \"SingularDisplayName\": \"Rollout\" }\r\n ,\"microsoft.providerhub/providerregistrations/defaultrollouts\": { \"SingularDisplayName\": \"Rollout\" }\r\n ,\"microsoft.providerhub/providerregistrations/resourcetyperegistrations\": { \"SingularDisplayName\": \"Resource Type\" }\r\n ,\"microsoft.providerhub/providerregistrations/resourcetyperegistrations/resourcetyperegistrations\": { \"SingularDisplayName\": \"Resource Type\" }\r\n ,\"microsoft.providerhubdevtest/regionalstresstests\": { \"SingularDisplayName\": \"Microsoft.ProviderHubDevTest regional stresstest\" }\r\n ,\"microsoft.providerhubdevtest/stresstests\": { \"SingularDisplayName\": \"Microsoft.ProviderHubDevTest stresstest\" }\r\n ,\"microsoft.purview/accounts\": { \"SingularDisplayName\": \"Microsoft Purview account\" }\r\n ,\"microsoft.quantum/provideraccounts\": { \"SingularDisplayName\": \"Microsoft.Quantum provider account\" }\r\n ,\"microsoft.quantum/workspaces\": { \"SingularDisplayName\": \"Quantum Workspace\" }\r\n ,\"microsoft.quota/groupquotas\": { \"SingularDisplayName\": \"Microsoft.Quota group quota\" }\r\n ,\"microsoft.quota/groupquotas/groupquotarequests\": { \"SingularDisplayName\": \"Microsoft.Quota group quotas group quota request\" }\r\n ,\"microsoft.quota/groupquotas/quotaallocationrequests\": { \"SingularDisplayName\": \"Microsoft.Quota group quotas quota allocation request\" }\r\n ,\"microsoft.quota/groupquotas/quotaallocations\": { \"SingularDisplayName\": \"Microsoft.Quota group quotas quota allocation\" }\r\n ,\"microsoft.quota/groupquotas/subscriptionrequests\": { \"SingularDisplayName\": \"Microsoft.Quota group quotas subscription request\" }\r\n ,\"microsoft.quota/groupquotas/subscriptions\": { \"SingularDisplayName\": \"Microsoft.Quota group quotas subscription\" }\r\n ,\"microsoft.quota/quotarequests\": { \"SingularDisplayName\": \"Microsoft.Quota quota request\" }\r\n ,\"microsoft.quota/quotas\": { \"SingularDisplayName\": \"Microsoft.Quota quota\" }\r\n ,\"microsoft.quota/usages\": { \"SingularDisplayName\": \"Microsoft.Quota usage\" }\r\n ,\"microsoft.recommendationsservice/accounts\": { \"SingularDisplayName\": \"Intelligent Recommendations Account\" }\r\n ,\"microsoft.recommendationsservice/accounts/modeling\": { \"SingularDisplayName\": \"Modeling\" }\r\n ,\"microsoft.recommendationsservice/accounts/serviceendpoints\": { \"SingularDisplayName\": \"Service Endpoint\" }\r\n ,\"microsoft.recoveryservices/replicationeligibilityresults\": { \"SingularDisplayName\": \"Microsoft.RecoveryServices replication eligibility result\" }\r\n ,\"microsoft.recoveryservices/vaults\": { \"SingularDisplayName\": \"Recovery Services vault\" }\r\n ,\"microsoft.recoveryservices/vaults/backupfabrics/protectioncontainers/protecteditems\": { \"SingularDisplayName\": \"Backup Item\" }\r\n ,\"microsoft.recoveryservicesbvtd/vaults\": { \"SingularDisplayName\": \"Recovery Services BVTD\" }\r\n ,\"microsoft.recoveryservicesbvtd2/vaults\": { \"SingularDisplayName\": \"Recovery Services BVTD2\" }\r\n ,\"microsoft.recoveryservicesintd/vaults\": { \"SingularDisplayName\": \"Recovery Services INTD\" }\r\n ,\"microsoft.recoveryservicesintd2/vaults\": { \"SingularDisplayName\": \"Recovery Services INTD2\" }\r\n ,\"microsoft.redhatopenshift/openshiftclusters\": { \"SingularDisplayName\": \"Azure Red Hat OpenShift cluster\" }\r\n ,\"microsoft.relationships/dependencyof\": { \"SingularDisplayName\": \"Dependency Relationship\" }\r\n ,\"microsoft.relationships/servicegroupmember\": { \"SingularDisplayName\": \"Service group member relationship\" }\r\n ,\"microsoft.relationships/servicegrouprelationships\": { \"SingularDisplayName\": \"Connected Resource\" }\r\n ,\"microsoft.relay/namespaces\": { \"SingularDisplayName\": \"Relay\" }\r\n ,\"microsoft.relay/namespaces/hybridconnections\": { \"SingularDisplayName\": \"Hybrid connection\" }\r\n ,\"microsoft.relay/namespaces/wcfrelays\": { \"SingularDisplayName\": \"WCF relay\" }\r\n ,\"microsoft.resilience/resiliencestates\": { \"SingularDisplayName\": \"Microsoft.Resilience resilience state\" }\r\n ,\"microsoft.resourceconnector/appliances\": { \"SingularDisplayName\": \"Resource bridge\" }\r\n ,\"microsoft.resourcegraph/queries\": { \"SingularDisplayName\": \"Resource Graph query\" }\r\n ,\"microsoft.resourcehealth/availabilitystatuses\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth availability statuse\" }\r\n ,\"microsoft.resourcehealth/childavailabilitystatuses\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth child availability statuse\" }\r\n ,\"microsoft.resourcehealth/emergingissues\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth emerging issue\" }\r\n ,\"microsoft.resourcehealth/events\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth event\" }\r\n ,\"microsoft.resourcehealth/events/impactedresources\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth events impacted resource\" }\r\n ,\"microsoft.resourcehealth/metadata\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth metadata\" }\r\n ,\"microsoft.resources/builtintemplatespecs\": { \"SingularDisplayName\": \"Built-in template spec\" }\r\n ,\"microsoft.resources/changes\": { \"SingularDisplayName\": \"Microsoft.Resources change\" }\r\n ,\"microsoft.resources/databoundaries\": { \"SingularDisplayName\": \"Microsoft.Resources data boundary\" }\r\n ,\"microsoft.resources/deletedresources\": { \"SingularDisplayName\": \"Recycle Bin\" }\r\n ,\"microsoft.resources/deployments\": { \"SingularDisplayName\": \"Microsoft.Resources deployment\" }\r\n ,\"microsoft.resources/deployments/operations\": { \"SingularDisplayName\": \"Microsoft.Resources deployments operation\" }\r\n ,\"microsoft.resources/deploymentscripts\": { \"SingularDisplayName\": \"Deployment Script\" }\r\n ,\"microsoft.resources/deploymentstacks\": { \"SingularDisplayName\": \"Deployment stack\" }\r\n ,\"microsoft.resources/mobobrokers\": { \"SingularDisplayName\": \"Microsoft.Resources mobo broker\" }\r\n ,\"microsoft.resources/resourcechange\": { \"SingularDisplayName\": \"Change Analysis\" }\r\n ,\"microsoft.resources/resourcechanges\": { \"SingularDisplayName\": \"Resource change\" }\r\n ,\"microsoft.resources/resourcegraphvisualizer\": { \"SingularDisplayName\": \"Resource Graph Visualizer\" }\r\n ,\"microsoft.resources/resourcegroups\": { \"SingularDisplayName\": \"Microsoft.Resources resource group\" }\r\n ,\"microsoft.resources/resources\": { \"SingularDisplayName\": \"Resource\" }\r\n ,\"microsoft.resources/snapshots\": { \"SingularDisplayName\": \"Microsoft.Resources snapshot\" }\r\n ,\"microsoft.resources/subscriptions\": { \"SingularDisplayName\": \"Subscription\" }\r\n ,\"microsoft.resources/subscriptions/resourcegroups\": { \"SingularDisplayName\": \"Resource group\" }\r\n ,\"microsoft.resources/tags\": { \"SingularDisplayName\": \"Microsoft.Resources tag\" }\r\n ,\"microsoft.resources/templatespecs\": { \"SingularDisplayName\": \"Template spec\" }\r\n ,\"microsoft.resources/virtualsubscriptionsforresourcepicker\": { \"SingularDisplayName\": \"Subscription\" }\r\n ,\"microsoft.saas/applications\": { \"SingularDisplayName\": \"Software as a Service (classic)\" }\r\n ,\"microsoft.saas/resources\": { \"SingularDisplayName\": \"SaaS\" }\r\n ,\"microsoft.saas/saasresources\": { \"SingularDisplayName\": \"SaaS (classic)\" }\r\n ,\"microsoft.saashub/cloudservices\": { \"SingularDisplayName\": \"Microsoft.SaaSHub cloud service\" }\r\n ,\"microsoft.saashub/cloudservices/hidden\": { \"SingularDisplayName\": \"Microsoft SaaS\" }\r\n ,\"microsoft.saashub/saasresources\": { \"SingularDisplayName\": \"Microsoft.SaaSHub saas resource\" }\r\n ,\"microsoft.salescopilot/conversationintelligencerecordingaccounts\": { \"SingularDisplayName\": \"Microsoft.SalesCopilot conversation intelligence recording account\" }\r\n ,\"microsoft.scheduler/jobcollections\": { \"SingularDisplayName\": \"Scheduler job collection\" }\r\n ,\"microsoft.scheduler/jobcollections/jobs\": { \"SingularDisplayName\": \"Scheduler job\" }\r\n ,\"microsoft.scom/managedinstances\": { \"SingularDisplayName\": \"SCOM managed instance\" }\r\n ,\"microsoft.scvmm/availabilitysets\": { \"SingularDisplayName\": \"Microsoft.ScVmm availability set\" }\r\n ,\"microsoft.scvmm/clouds\": { \"SingularDisplayName\": \"Microsoft.ScVmm cloud\" }\r\n ,\"microsoft.scvmm/virtualmachineinstances\": { \"SingularDisplayName\": \"Microsoft.ScVmm virtual machine instance\" }\r\n ,\"microsoft.scvmm/virtualmachineinstances/guestagents\": { \"SingularDisplayName\": \"Microsoft.ScVmm virtual machine instances guest agent\" }\r\n ,\"microsoft.scvmm/virtualmachineinstances/hybrididentitymetadata\": { \"SingularDisplayName\": \"Microsoft.ScVmm virtual machine instances hybrid identity metadata\" }\r\n ,\"microsoft.scvmm/virtualmachines\": { \"SingularDisplayName\": \"SCVMM virtual machine - Azure Arc\" }\r\n ,\"microsoft.scvmm/virtualmachinetemplates\": { \"SingularDisplayName\": \"Microsoft.ScVmm virtual machine template\" }\r\n ,\"microsoft.scvmm/virtualnetworks\": { \"SingularDisplayName\": \"Microsoft.ScVmm virtual network\" }\r\n ,\"microsoft.scvmm/vmmservers\": { \"SingularDisplayName\": \"SCVMM management server\" }\r\n ,\"microsoft.search/searchservices\": { \"SingularDisplayName\": \"Search service\" }\r\n ,\"microsoft.secretmanagementsampleprovider/forecasts\": { \"SingularDisplayName\": \"Microsoft.SecretManagementSampleProvider forecast\" }\r\n ,\"microsoft.secretsynccontroller/azurekeyvaultsecretproviderclasses\": { \"SingularDisplayName\": \"Microsoft.SecretSyncController Azure key vault secret provider class\" }\r\n ,\"microsoft.secretsynccontroller/secretsyncs\": { \"SingularDisplayName\": \"Microsoft.SecretSyncController secret sync\" }\r\n ,\"microsoft.security/adaptivenetworkhardenings\": { \"SingularDisplayName\": \"Microsoft.Security adaptive network hardening\" }\r\n ,\"microsoft.security/advancedthreatprotectionsettings\": { \"SingularDisplayName\": \"Microsoft.Security advanced threat protection setting\" }\r\n ,\"microsoft.security/alertssuppressionrules\": { \"SingularDisplayName\": \"Microsoft.Security alerts suppression rule\" }\r\n ,\"microsoft.security/apicollections\": { \"SingularDisplayName\": \"Microsoft.Security API collection\" }\r\n ,\"microsoft.security/applications\": { \"SingularDisplayName\": \"Microsoft.Security application\" }\r\n ,\"microsoft.security/assessmentmetadata\": { \"SingularDisplayName\": \"Microsoft.Security assessment metadata\" }\r\n ,\"microsoft.security/assessments\": { \"SingularDisplayName\": \"Microsoft.Security assessment\" }\r\n ,\"microsoft.security/assessments/governanceassignments\": { \"SingularDisplayName\": \"Microsoft.Security assessments governance assignment\" }\r\n ,\"microsoft.security/assessments/subassessments\": { \"SingularDisplayName\": \"Microsoft.Security assessments sub assessment\" }\r\n ,\"microsoft.security/assignments\": { \"SingularDisplayName\": \"Microsoft.Security assignment\" }\r\n ,\"microsoft.security/automations\": { \"SingularDisplayName\": \"Microsoft.Security automation\" }\r\n ,\"microsoft.security/autoprovisioningsettings\": { \"SingularDisplayName\": \"Microsoft.Security auto provisioning setting\" }\r\n ,\"microsoft.security/complianceresults\": { \"SingularDisplayName\": \"Microsoft.Security compliance result\" }\r\n ,\"microsoft.security/compliances\": { \"SingularDisplayName\": \"Microsoft.Security compliance\" }\r\n ,\"microsoft.security/connectors\": { \"SingularDisplayName\": \"Microsoft.Security connector\" }\r\n ,\"microsoft.security/customassessmentautomations\": { \"SingularDisplayName\": \"Microsoft.Security custom assessment automation\" }\r\n ,\"microsoft.security/defenderforstoragesettings\": { \"SingularDisplayName\": \"Microsoft.Security defender for storage setting\" }\r\n ,\"microsoft.security/defenderforstoragesettings/malwarescans\": { \"SingularDisplayName\": \"Microsoft.Security defender for storage settings malware scan\" }\r\n ,\"microsoft.security/devicesecuritygroups\": { \"SingularDisplayName\": \"Microsoft.Security device security group\" }\r\n ,\"microsoft.security/governancerules\": { \"SingularDisplayName\": \"Microsoft.Security governance rule\" }\r\n ,\"microsoft.security/governancerules/operationresults\": { \"SingularDisplayName\": \"Microsoft.Security governance rules operation result\" }\r\n ,\"microsoft.security/healthreports\": { \"SingularDisplayName\": \"Microsoft.Security health report\" }\r\n ,\"microsoft.security/informationprotectionpolicies\": { \"SingularDisplayName\": \"Microsoft.Security information protection policy\" }\r\n ,\"microsoft.security/iotsecuritysolutions\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solution\" }\r\n ,\"microsoft.security/iotsecuritysolutions/analyticsmodels\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions analytics model\" }\r\n ,\"microsoft.security/iotsecuritysolutions/analyticsmodels/aggregatedalerts\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions analytics models aggregated alert\" }\r\n ,\"microsoft.security/iotsecuritysolutions/analyticsmodels/aggregatedrecommendations\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions analytics models aggregated recommendation\" }\r\n ,\"microsoft.security/iotsecuritysolutions/iotalerts\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions IoT alert\" }\r\n ,\"microsoft.security/iotsecuritysolutions/iotalerttypes\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions IoT alert type\" }\r\n ,\"microsoft.security/iotsecuritysolutions/iotrecommendations\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions IoT recommendation\" }\r\n ,\"microsoft.security/iotsecuritysolutions/iotrecommendationtypes\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions IoT recommendation type\" }\r\n ,\"microsoft.security/locations/alerts\": { \"SingularDisplayName\": \"Security Alert\" }\r\n ,\"microsoft.security/mdeonboardings\": { \"SingularDisplayName\": \"Microsoft.Security mde onboarding\" }\r\n ,\"microsoft.security/pricings\": { \"SingularDisplayName\": \"Defender for Cloud\" }\r\n ,\"microsoft.security/pricings/securityoperators\": { \"SingularDisplayName\": \"Microsoft.Security pricings security operator\" }\r\n ,\"microsoft.security/regulatorycompliancestandards\": { \"SingularDisplayName\": \"Microsoft.Security regulatory compliance standard\" }\r\n ,\"microsoft.security/regulatorycompliancestandards/regulatorycompliancecontrols\": { \"SingularDisplayName\": \"Microsoft.Security regulatory compliance standards regulatory compliance control\" }\r\n ,\"microsoft.security/regulatorycompliancestandards/regulatorycompliancecontrols/regulatorycomplianceassessments\": { \"SingularDisplayName\": \"Microsoft.Security regulatory compliance standards regulatory compliance controls regulatory compliance assessment\" }\r\n ,\"microsoft.security/securescores\": { \"SingularDisplayName\": \"Microsoft.Security secure score\" }\r\n ,\"microsoft.security/securityconnectors\": { \"SingularDisplayName\": \"Microsoft.Security security connector\" }\r\n ,\"microsoft.security/securityconnectors/devops\": { \"SingularDisplayName\": \"Microsoft.Security security connectors devop\" }\r\n ,\"microsoft.security/securitycontacts\": { \"SingularDisplayName\": \"Microsoft.Security security contact\" }\r\n ,\"microsoft.security/sensitivitysettings\": { \"SingularDisplayName\": \"Microsoft.Security sensitivity setting\" }\r\n ,\"microsoft.security/servervulnerabilityassessments\": { \"SingularDisplayName\": \"Microsoft.Security server vulnerability assessment\" }\r\n ,\"microsoft.security/servervulnerabilityassessmentssettings\": { \"SingularDisplayName\": \"Microsoft.Security server vulnerability assessments setting\" }\r\n ,\"microsoft.security/settings\": { \"SingularDisplayName\": \"Microsoft.Security setting\" }\r\n ,\"microsoft.security/standards\": { \"SingularDisplayName\": \"Microsoft.Security standard\" }\r\n ,\"microsoft.security/workspacesettings\": { \"SingularDisplayName\": \"Microsoft.Security workspace setting\" }\r\n ,\"microsoft.securitycopilot/capacities\": { \"SingularDisplayName\": \"Microsoft Security compute capacity\" }\r\n ,\"microsoft.securitydetonation/chambers\": { \"SingularDisplayName\": \"Security Detonation Chamber\" }\r\n ,\"microsoft.securityinsightsarg/sentinel\": { \"SingularDisplayName\": \"Microsoft Sentinel\" }\r\n ,\"microsoft.sentinelplatformservices/sentinelplatformservices\": { \"SingularDisplayName\": \"Microsoft.SentinelPlatformServices sentinel platform service\" }\r\n ,\"microsoft.serialconsole/consoleservices\": { \"SingularDisplayName\": \"Microsoft.SerialConsole console service\" }\r\n ,\"microsoft.serialconsole/serialports\": { \"SingularDisplayName\": \"Microsoft.SerialConsole serial port\" }\r\n ,\"microsoft.servicebus/namespaces\": { \"SingularDisplayName\": \"Service Bus namespace\" }\r\n ,\"microsoft.servicebus/namespaces/disasterrecoveryconfigs\": { \"SingularDisplayName\": \"Service Bus Geo-DR Alias\" }\r\n ,\"microsoft.servicebus/namespaces/queues\": { \"SingularDisplayName\": \"Service Bus queue\" }\r\n ,\"microsoft.servicebus/namespaces/topics\": { \"SingularDisplayName\": \"Service Bus topic\" }\r\n ,\"microsoft.servicebus/namespaces/topics/subscriptions\": { \"SingularDisplayName\": \"Service Bus Subscription\" }\r\n ,\"microsoft.servicefabric/clusters\": { \"SingularDisplayName\": \"Service Fabric cluster\" }\r\n ,\"microsoft.servicefabric/managedclusters\": { \"SingularDisplayName\": \"Service Fabric managed cluster\" }\r\n ,\"microsoft.servicefabricmesh/applications\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh application\" }\r\n ,\"microsoft.servicefabricmesh/applications/services\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh applications service\" }\r\n ,\"microsoft.servicefabricmesh/applications/services/replicas\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh applications services replica\" }\r\n ,\"microsoft.servicefabricmesh/gateways\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh gateway\" }\r\n ,\"microsoft.servicefabricmesh/networks\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh network\" }\r\n ,\"microsoft.servicefabricmesh/secrets\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh secret\" }\r\n ,\"microsoft.servicefabricmesh/secrets/values\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh secrets value\" }\r\n ,\"microsoft.servicefabricmesh/volumes\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh volume\" }\r\n ,\"microsoft.servicelinker/dryruns\": { \"SingularDisplayName\": \"Microsoft.ServiceLinker dryrun\" }\r\n ,\"microsoft.servicelinker/linkers\": { \"SingularDisplayName\": \"Microsoft.ServiceLinker linker\" }\r\n ,\"microsoft.servicenetworking/trafficcontrollers\": { \"SingularDisplayName\": \"Application Gateway for Containers\" }\r\n ,\"microsoft.serviceshub/connectors\": { \"SingularDisplayName\": \"Services Hub Connector\" }\r\n ,\"microsoft.signalrservice/signalr\": { \"SingularDisplayName\": \"SignalR\" }\r\n ,\"microsoft.signalrservice/signalr/replicas\": { \"SingularDisplayName\": \"SignalR Replica\" }\r\n ,\"microsoft.signalrservice/webpubsub\": { \"SingularDisplayName\": \"Web PubSub Service\" }\r\n ,\"microsoft.signalrservice/webpubsub/replicas\": { \"SingularDisplayName\": \"Web PubSub Service Replica\" }\r\n ,\"microsoft.skytap/billingnodes\": { \"SingularDisplayName\": \"Microsoft.Skytap billing node\" }\r\n ,\"microsoft.skytap/interfaces\": { \"SingularDisplayName\": \"Microsoft.Skytap interface\" }\r\n ,\"microsoft.skytap/nodes\": { \"SingularDisplayName\": \"Microsoft.Skytap node\" }\r\n ,\"microsoft.softwareplan/hybridusebenefits\": { \"SingularDisplayName\": \"Microsoft.SoftwarePlan hybrid use benefit\" }\r\n ,\"microsoft.solutions/applicationdefinitions\": { \"SingularDisplayName\": \"Service catalog managed application definition\" }\r\n ,\"microsoft.solutions/applications\": { \"SingularDisplayName\": \"Managed application\" }\r\n ,\"microsoft.solutions/jitrequests\": { \"SingularDisplayName\": \"Microsoft.Solutions JIT request\" }\r\n ,\"microsoft.sovereign/landingzoneaccounts\": { \"SingularDisplayName\": \"Landing zone account\" }\r\n ,\"microsoft.sovereign/landingzoneaccounts/landingzoneconfigurations\": { \"SingularDisplayName\": \"Landing Zone Configuration\" }\r\n ,\"microsoft.sovereign/landingzoneaccounts/landingzoneregistrations\": { \"SingularDisplayName\": \"Landing Zone Registration\" }\r\n ,\"microsoft.sovereign/landingzoneconfigurations\": { \"SingularDisplayName\": \"Landing Zone Configuration\" }\r\n ,\"microsoft.sovereign/landingzoneregistrations\": { \"SingularDisplayName\": \"Landing Zone Registration\" }\r\n ,\"microsoft.sovereign/transparencylogs\": { \"SingularDisplayName\": \"Transparency log\" }\r\n ,\"microsoft.sql/azuresql\": { \"SingularDisplayName\": \"Azure SQL resource\" }\r\n ,\"microsoft.sql/instancepools\": { \"SingularDisplayName\": \"Instance pool\" }\r\n ,\"microsoft.sql/managedinstances\": { \"SingularDisplayName\": \"SQL managed instance\" }\r\n ,\"microsoft.sql/managedinstances/databases\": { \"SingularDisplayName\": \"Managed database\" }\r\n ,\"microsoft.sql/servers\": { \"SingularDisplayName\": \"SQL server\" }\r\n ,\"microsoft.sql/servers/databases\": { \"SingularDisplayName\": \"SQL database\" }\r\n ,\"microsoft.sql/servers/elasticpools\": { \"SingularDisplayName\": \"SQL elastic pool\" }\r\n ,\"microsoft.sql/servers/jobagents\": { \"SingularDisplayName\": \"Elastic Job agent\" }\r\n ,\"microsoft.sql/virtualclusters\": { \"SingularDisplayName\": \"Virtual cluster\" }\r\n ,\"microsoft.sqlvirtualmachine/sqlvirtualmachinegroups\": { \"SingularDisplayName\": \"Microsoft.SqlVirtualMachine sql virtual machine group\" }\r\n ,\"microsoft.sqlvirtualmachine/sqlvirtualmachinegroups/availabilitygrouplisteners\": { \"SingularDisplayName\": \"Microsoft.SqlVirtualMachine sql virtual machine groups availability group listener\" }\r\n ,\"microsoft.sqlvirtualmachine/sqlvirtualmachines\": { \"SingularDisplayName\": \"SQL virtual machine\" }\r\n ,\"microsoft.standbypool/standbycontainergrouppools\": { \"SingularDisplayName\": \"Microsoft.StandbyPool standby container group pool\" }\r\n ,\"microsoft.standbypool/standbycontainergrouppools/runtimeviews\": { \"SingularDisplayName\": \"Microsoft.StandbyPool standby container group pools runtime view\" }\r\n ,\"microsoft.standbypool/standbyvirtualmachinepools\": { \"SingularDisplayName\": \"Microsoft.StandbyPool standby virtual machine pool\" }\r\n ,\"microsoft.standbypool/standbyvirtualmachinepools/runtimeviews\": { \"SingularDisplayName\": \"Microsoft.StandbyPool standby virtual machine pools runtime view\" }\r\n ,\"microsoft.standbypool/standbyvirtualmachinepools/standbyvirtualmachines\": { \"SingularDisplayName\": \"Microsoft.StandbyPool standby virtual machine pools standby virtual machine\" }\r\n ,\"microsoft.storage/storageaccounts\": { \"SingularDisplayName\": \"Storage account\" }\r\n ,\"microsoft.storageactions/storagetasks\": { \"SingularDisplayName\": \"Storage task - Azure Storage Actions\" }\r\n ,\"microsoft.storagecache/amlfilesystems\": { \"SingularDisplayName\": \"Azure Managed Lustre\" }\r\n ,\"microsoft.storagecache/caches\": { \"SingularDisplayName\": \"HPC cache\" }\r\n ,\"microsoft.storagediscovery/storagediscoveryworkspaces\": { \"SingularDisplayName\": \"Storage Discovery workspace\" }\r\n ,\"microsoft.storagehub/all\": { \"SingularDisplayName\": \"All resources\" }\r\n ,\"microsoft.storagehub/policycomplianceresources\": { \"SingularDisplayName\": \"Policy compliance\" }\r\n ,\"microsoft.storageinsights/storagecollectionrules\": { \"SingularDisplayName\": \"Microsoft.StorageInsights storage collection rule\" }\r\n ,\"microsoft.storagemover/storagemovers\": { \"SingularDisplayName\": \"Storage mover\" }\r\n ,\"microsoft.storagepool/diskpools\": { \"SingularDisplayName\": \"Microsoft.StoragePool disk pool\" }\r\n ,\"microsoft.storagepool/diskpools/iscsitargets\": { \"SingularDisplayName\": \"Microsoft.StoragePool disk pools iscsi target\" }\r\n ,\"microsoft.storagesync/storagesyncservices\": { \"SingularDisplayName\": \"Storage Sync Service\" }\r\n ,\"microsoft.storagetasks/storagetasks\": { \"SingularDisplayName\": \"Microsoft.StorageTasks storage task\" }\r\n ,\"microsoft.storsimple/managers\": { \"SingularDisplayName\": \"StorSimple device manager\" }\r\n ,\"microsoft.storsimple/managers/accesscontrolrecords\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers access control record\" }\r\n ,\"microsoft.storsimple/managers/bandwidthsettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers bandwidth setting\" }\r\n ,\"microsoft.storsimple/managers/certificates\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers certificate\" }\r\n ,\"microsoft.storsimple/managers/devices\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers device\" }\r\n ,\"microsoft.storsimple/managers/devices/alertsettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices alert setting\" }\r\n ,\"microsoft.storsimple/managers/devices/backuppolicies\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices backup policy\" }\r\n ,\"microsoft.storsimple/managers/devices/backuppolicies/schedules\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices backup policies schedule\" }\r\n ,\"microsoft.storsimple/managers/devices/backupschedulegroups\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices backup schedule group\" }\r\n ,\"microsoft.storsimple/managers/devices/chapsettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices chap setting\" }\r\n ,\"microsoft.storsimple/managers/devices/fileservers\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices fileserver\" }\r\n ,\"microsoft.storsimple/managers/devices/fileservers/shares\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices fileservers share\" }\r\n ,\"microsoft.storsimple/managers/devices/iscsiservers\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices iscsiserver\" }\r\n ,\"microsoft.storsimple/managers/devices/iscsiservers/disks\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices iscsiservers disk\" }\r\n ,\"microsoft.storsimple/managers/devices/jobs\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices job\" }\r\n ,\"microsoft.storsimple/managers/devices/networksettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices network setting\" }\r\n ,\"microsoft.storsimple/managers/devices/securitysettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices security setting\" }\r\n ,\"microsoft.storsimple/managers/devices/timesettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices time setting\" }\r\n ,\"microsoft.storsimple/managers/devices/updatesummary\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices update summary\" }\r\n ,\"microsoft.storsimple/managers/devices/volumecontainers\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices volume container\" }\r\n ,\"microsoft.storsimple/managers/devices/volumecontainers/volumes\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices volume containers volume\" }\r\n ,\"microsoft.storsimple/managers/encryptionsettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers encryption setting\" }\r\n ,\"microsoft.storsimple/managers/extendedinformation\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers extended information\" }\r\n ,\"microsoft.storsimple/managers/storageaccountcredentials\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers storage account credential\" }\r\n ,\"microsoft.storsimple/managers/storagedomains\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers storage domain\" }\r\n ,\"microsoft.streamanalytics/clusters\": { \"SingularDisplayName\": \"Stream Analytics cluster\" }\r\n ,\"microsoft.streamanalytics/streamingjobs\": { \"SingularDisplayName\": \"Stream Analytics job\" }\r\n ,\"microsoft.subscription/aliases\": { \"SingularDisplayName\": \"Microsoft.Subscription aliase\" }\r\n ,\"microsoft.subscription/changetenantrequest\": { \"SingularDisplayName\": \"Microsoft.Subscription change tenant request\" }\r\n ,\"microsoft.subscription/policies\": { \"SingularDisplayName\": \"Microsoft.Subscription policy\" }\r\n ,\"microsoft.subscription/subscriptiondefinitions\": { \"SingularDisplayName\": \"Microsoft.Subscription subscription definition\" }\r\n ,\"microsoft.subscription/subscriptionoperations\": { \"SingularDisplayName\": \"Microsoft.Subscription subscription operation\" }\r\n ,\"microsoft.support/fileworkspaces\": { \"SingularDisplayName\": \"Microsoft.Support file workspace\" }\r\n ,\"microsoft.support/fileworkspaces/files\": { \"SingularDisplayName\": \"Microsoft.Support file workspaces file\" }\r\n ,\"microsoft.support/services\": { \"SingularDisplayName\": \"Microsoft.Support service\" }\r\n ,\"microsoft.support/services/problemclassifications\": { \"SingularDisplayName\": \"Microsoft.Support services problem classification\" }\r\n ,\"microsoft.support/supporttickets\": { \"SingularDisplayName\": \"Support Request\" }\r\n ,\"microsoft.sustainabilityservices/calculations\": { \"SingularDisplayName\": \"Project Sustainability Calculator\" }\r\n ,\"microsoft.symphony/instances\": { \"SingularDisplayName\": \"Microsoft.Symphony instance\" }\r\n ,\"microsoft.symphony/solutions\": { \"SingularDisplayName\": \"Microsoft.Symphony solution\" }\r\n ,\"microsoft.symphony/targets\": { \"SingularDisplayName\": \"Microsoft.Symphony target\" }\r\n ,\"microsoft.synapse/privatelinkhubs\": { \"SingularDisplayName\": \"Synapse private link hub\" }\r\n ,\"microsoft.synapse/workspaces\": { \"SingularDisplayName\": \"Synapse workspace\" }\r\n ,\"microsoft.synapse/workspaces/bigdatapools\": { \"SingularDisplayName\": \"Apache Spark pool\" }\r\n ,\"microsoft.synapse/workspaces/kustopools\": { \"SingularDisplayName\": \"Data Explorer pool\" }\r\n ,\"microsoft.synapse/workspaces/kustopools/databases\": { \"SingularDisplayName\": \"Data Explorer Database\" }\r\n ,\"microsoft.synapse/workspaces/scopepools\": { \"SingularDisplayName\": \"SCOPE pool\" }\r\n ,\"microsoft.synapse/workspaces/sqlpools\": { \"SingularDisplayName\": \"Dedicated SQL pool\" }\r\n ,\"microsoft.syntex/accounts\": { \"SingularDisplayName\": \"Microsoft.Syntex account\" }\r\n ,\"microsoft.syntex/documentprocessors\": { \"SingularDisplayName\": \"Microsoft.Syntex document processor\" }\r\n ,\"microsoft.test/healthdataaiservices\": { \"SingularDisplayName\": \"Azure Health Data and AI Services\" }\r\n ,\"microsoft.timeseriesinsights/environments\": { \"SingularDisplayName\": \"Microsoft.TimeSeriesInsights environment\" }\r\n ,\"microsoft.timeseriesinsights/environments/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.TimeSeriesInsights environments access policy\" }\r\n ,\"microsoft.timeseriesinsights/environments/eventsources\": { \"SingularDisplayName\": \"Microsoft.TimeSeriesInsights environments event source\" }\r\n ,\"microsoft.timeseriesinsights/environments/referencedatasets\": { \"SingularDisplayName\": \"Microsoft.TimeSeriesInsights environments reference data set\" }\r\n ,\"microsoft.toolchainorchestrator/activations\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator activation\" }\r\n ,\"microsoft.toolchainorchestrator/campaigns\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator campaign\" }\r\n ,\"microsoft.toolchainorchestrator/campaigns/versions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator campaigns version\" }\r\n ,\"microsoft.toolchainorchestrator/catalogs\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator catalog\" }\r\n ,\"microsoft.toolchainorchestrator/catalogs/versions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator catalogs version\" }\r\n ,\"microsoft.toolchainorchestrator/diagnostics\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator diagnostic\" }\r\n ,\"microsoft.toolchainorchestrator/instances\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator instance\" }\r\n ,\"microsoft.toolchainorchestrator/instances/versions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator instances version\" }\r\n ,\"microsoft.toolchainorchestrator/solutions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator solution\" }\r\n ,\"microsoft.toolchainorchestrator/solutions/versions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator solutions version\" }\r\n ,\"microsoft.toolchainorchestrator/targets\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator target\" }\r\n ,\"microsoft.toolchainorchestrator/targets/versions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator targets version\" }\r\n ,\"microsoft.updatemanager/updaterules\": { \"SingularDisplayName\": \"Update Rule\" }\r\n ,\"microsoft.usagebilling/accounts\": { \"SingularDisplayName\": \"Microsoft.UsageBilling account\" }\r\n ,\"microsoft.usagebilling/accounts/dataexports\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts data export\" }\r\n ,\"microsoft.usagebilling/accounts/inputs\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts input\" }\r\n ,\"microsoft.usagebilling/accounts/metricexports\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts metric export\" }\r\n ,\"microsoft.usagebilling/accounts/pav2outputs\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts pav2output\" }\r\n ,\"microsoft.usagebilling/accounts/pipelines\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts pipeline\" }\r\n ,\"microsoft.usagebilling/accounts/pipelines/outputselectors\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts pipelines output selector\" }\r\n ,\"microsoft.verifiedid/authorities\": { \"SingularDisplayName\": \"Microsoft.VerifiedId authority\" }\r\n ,\"microsoft.videoindexer/accounts\": { \"SingularDisplayName\": \"Azure AI Video Indexer\" }\r\n ,\"microsoft.virtualmachineimages/imagetemplates\": { \"SingularDisplayName\": \"Image template\" }\r\n ,\"microsoft.visualstudio/account\": { \"SingularDisplayName\": \"Azure DevOps organization\" }\r\n ,\"microsoft.vmware/resourcepools\": { \"SingularDisplayName\": \"Microsoft.VMware resource pool\" }\r\n ,\"microsoft.vmware/vcenters\": { \"SingularDisplayName\": \"Microsoft.VMware vcenter\" }\r\n ,\"microsoft.vmware/vcenters/inventoryitems\": { \"SingularDisplayName\": \"Microsoft.VMware vcenters inventory item\" }\r\n ,\"microsoft.vmware/virtualmachines\": { \"SingularDisplayName\": \"Microsoft.VMware virtual machine\" }\r\n ,\"microsoft.vmware/virtualmachinetemplates\": { \"SingularDisplayName\": \"Microsoft.VMware virtual machine template\" }\r\n ,\"microsoft.vmware/virtualnetworks\": { \"SingularDisplayName\": \"Microsoft.VMware virtual network\" }\r\n ,\"microsoft.vmwarecloudsimple/dedicatedcloudnodes\": { \"SingularDisplayName\": \"Microsoft.VMwareCloudSimple dedicated cloud node\" }\r\n ,\"microsoft.vmwarecloudsimple/dedicatedcloudservices\": { \"SingularDisplayName\": \"Microsoft.VMwareCloudSimple dedicated cloud service\" }\r\n ,\"microsoft.vmwarecloudsimple/virtualmachines\": { \"SingularDisplayName\": \"Microsoft.VMwareCloudSimple virtual machine\" }\r\n ,\"microsoft.vnfmanager/devices\": { \"SingularDisplayName\": \"Microsoft.VnfManager device\" }\r\n ,\"microsoft.vnfmanager/vendors\": { \"SingularDisplayName\": \"Microsoft.VnfManager vendor\" }\r\n ,\"microsoft.vnfmanager/vendors/skus\": { \"SingularDisplayName\": \"Microsoft.VnfManager vendors SKU\" }\r\n ,\"microsoft.vnfmanager/vnfs\": { \"SingularDisplayName\": \"Microsoft.VnfManager vnf\" }\r\n ,\"microsoft.voiceservices/communicationsgateways\": { \"SingularDisplayName\": \"Communications Gateway\" }\r\n ,\"microsoft.voiceservices/communicationsgateways/testlines\": { \"SingularDisplayName\": \"Communications Gateway Test Line\" }\r\n ,\"microsoft.vsonline/accounts\": { \"SingularDisplayName\": \"Microsoft.VSOnline account\" }\r\n ,\"microsoft.vsonline/plans\": { \"SingularDisplayName\": \"Visual Studio Online Plan\" }\r\n ,\"microsoft.web/certificates\": { \"SingularDisplayName\": \"Microsoft.Web certificate\" }\r\n ,\"microsoft.web/connectiongateways\": { \"SingularDisplayName\": \"App Service on-premises data gateway\" }\r\n ,\"microsoft.web/connections\": { \"SingularDisplayName\": \"App Service API connection\" }\r\n ,\"microsoft.web/containerapps\": { \"SingularDisplayName\": \"Microsoft.Web container app\" }\r\n ,\"microsoft.web/containerapps/revisions\": { \"SingularDisplayName\": \"Microsoft.Web container apps revision\" }\r\n ,\"microsoft.web/customapis\": { \"SingularDisplayName\": \"Logic apps custom connector\" }\r\n ,\"microsoft.web/deletedsites\": { \"SingularDisplayName\": \"Microsoft.Web deleted site\" }\r\n ,\"microsoft.web/hostingenvironments\": { \"SingularDisplayName\": \"App Service Environment\" }\r\n ,\"microsoft.web/ishostingenvironmentnameavailable\": { \"SingularDisplayName\": \"Microsoft.Web ishostingenvironmentnameavailable\" }\r\n ,\"microsoft.web/kubeenvironments\": { \"SingularDisplayName\": \"App Service Kubernetes Environment\" }\r\n ,\"microsoft.web/logicappstemplate\": { \"SingularDisplayName\": \"Logic Apps Template\" }\r\n ,\"microsoft.web/publishingusers\": { \"SingularDisplayName\": \"Microsoft.Web publishing user\" }\r\n ,\"microsoft.web/serverfarms\": { \"SingularDisplayName\": \"App Service plan\" }\r\n ,\"microsoft.web/sites\": { \"SingularDisplayName\": \"App Service web app\" }\r\n ,\"microsoft.web/sites/slots\": { \"SingularDisplayName\": \"App Service deployment slot\" }\r\n ,\"microsoft.web/sourcecontrols\": { \"SingularDisplayName\": \"Microsoft.Web sourcecontrol\" }\r\n ,\"microsoft.web/staticsites\": { \"SingularDisplayName\": \"Static Web App\" }\r\n ,\"microsoft.weightsandbiases/instances\": { \"SingularDisplayName\": \"Azure Native Weights & Biases Cloud Service\" }\r\n ,\"microsoft.whiteboxcadlprovider/whiteboxresources\": { \"SingularDisplayName\": \"Microsoft.WhiteBoxCadlProvider white box resource\" }\r\n ,\"microsoft.windows365/cloudpcdelegatedmsis\": { \"SingularDisplayName\": \"Microsoft.Windows365 cloud pc delegated msi\" }\r\n ,\"microsoft.windowsesu/multipleactivationkeys\": { \"SingularDisplayName\": \"Microsoft.WindowsESU multiple activation key\" }\r\n ,\"microsoft.windowsiot/deviceservices\": { \"SingularDisplayName\": \"Microsoft.WindowsIoT device service\" }\r\n ,\"microsoft.windowspushnotificationservices/registrations\": { \"SingularDisplayName\": \"Windows Push Notification Service\" }\r\n ,\"microsoft.workloadmonitor/monitors\": { \"SingularDisplayName\": \"Microsoft.WorkloadMonitor monitor\" }\r\n ,\"microsoft.workloadmonitor/monitors/history\": { \"SingularDisplayName\": \"Microsoft.WorkloadMonitor monitors history\" }\r\n ,\"microsoft.workloads/configurationvalidationresults\": { \"SingularDisplayName\": \"Microsoft.Workloads configuration validation result\" }\r\n ,\"microsoft.workloads/connectors\": { \"SingularDisplayName\": \"Microsoft.Workloads connector\" }\r\n ,\"microsoft.workloads/connectors/acssbackups\": { \"SingularDisplayName\": \"Microsoft.Workloads connectors acss backup\" }\r\n ,\"microsoft.workloads/connectors/amsinsights\": { \"SingularDisplayName\": \"Microsoft.Workloads connectors ams insight\" }\r\n ,\"microsoft.workloads/connectors/sapvirtualinstancemonitors\": { \"SingularDisplayName\": \"Microsoft.Workloads connectors sap virtual instance monitor\" }\r\n ,\"microsoft.workloads/epicvirtualinstances\": { \"SingularDisplayName\": \"Virtual Instance for Epic solution\" }\r\n ,\"microsoft.workloads/insights\": { \"SingularDisplayName\": \"Microsoft.Workloads insight\" }\r\n ,\"microsoft.workloads/instancegroupmonitors\": { \"SingularDisplayName\": \"Microsoft.Workloads instance group monitor\" }\r\n ,\"microsoft.workloads/instancehealthdefinitions\": { \"SingularDisplayName\": \"Microsoft.Workloads instance health definition\" }\r\n ,\"microsoft.workloads/instancehealthdefinitions/signaldefinitions\": { \"SingularDisplayName\": \"Microsoft.Workloads instance health definitions signal definition\" }\r\n ,\"microsoft.workloads/instancemonitors\": { \"SingularDisplayName\": \"Microsoft.Workloads instance monitor\" }\r\n ,\"microsoft.workloads/monitors\": { \"SingularDisplayName\": \"Azure Monitor for SAP solutions\" }\r\n ,\"microsoft.workloads/oraclevirtualinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads oracle virtual instance\" }\r\n ,\"microsoft.workloads/oraclevirtualinstances/databaseinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads oracle virtual instances database instance\" }\r\n ,\"microsoft.workloads/phpworkloads\": { \"SingularDisplayName\": \"Microsoft.Workloads php workload\" }\r\n ,\"microsoft.workloads/phpworkloads/wordpressinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads php workloads wordpress instance\" }\r\n ,\"microsoft.workloads/sapdiscoverysites\": { \"SingularDisplayName\": \"Microsoft.Workloads sap discovery site\" }\r\n ,\"microsoft.workloads/sapdiscoverysites/sapinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads sap discovery sites sap instance\" }\r\n ,\"microsoft.workloads/sapdiscoverysites/sapinstances/serverinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads sap discovery sites sap instances server instance\" }\r\n ,\"microsoft.workloads/sapvirtualinstances\": { \"SingularDisplayName\": \"Virtual Instance for SAP solutions\" }\r\n ,\"microsoft.workloads/sapvirtualinstances/applicationinstances\": { \"SingularDisplayName\": \"App server instance for SAP solutions\" }\r\n ,\"microsoft.workloads/sapvirtualinstances/centralinstances\": { \"SingularDisplayName\": \"Central service instance for SAP solutions\" }\r\n ,\"microsoft.workloads/sapvirtualinstances/databaseinstances\": { \"SingularDisplayName\": \"Database for SAP solutions\" }\r\n ,\"microsoft.workloads/virtualinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads virtual instance\" }\r\n ,\"microsoft.workloads/virtualinstances/components\": { \"SingularDisplayName\": \"Microsoft.Workloads virtual instances component\" }\r\n ,\"microsoft.workloads/workloadinstance\": { \"SingularDisplayName\": \"My Resource\" }\r\n ,\"microsoft.zerotrustsegmentation/segmentationmanagers\": { \"SingularDisplayName\": \"Segmentation Manager\" }\r\n ,\"mongodb.atlas/organizations\": { \"SingularDisplayName\": \"MongoDB Atlas Organization\" }\r\n ,\"neon.postgres/organizations\": { \"SingularDisplayName\": \"Neon Serverless Postgres Organization\" }\r\n ,\"newrelic.observability/monitors\": { \"SingularDisplayName\": \"New Relic\" }\r\n ,\"nginx.nginxplus/nginxdeployments\": { \"SingularDisplayName\": \"NGINXaaS\" }\r\n ,\"oracle.database/autonomousdatabases\": { \"SingularDisplayName\": \"Autonomous Database\" }\r\n ,\"oracle.database/basedb\": { \"SingularDisplayName\": \"Autonomous Database\" }\r\n ,\"oracle.database/cloudexadatainfrastructures\": { \"SingularDisplayName\": \"Oracle Exadata Infrastructure\" }\r\n ,\"oracle.database/cloudvmclusters\": { \"SingularDisplayName\": \"Oracle Exadata VM Cluster\" }\r\n ,\"oracle.database/exadbvmclusters\": { \"SingularDisplayName\": \"Oracle Exascale VM Cluster\" }\r\n ,\"oracle.database/exascaledbstoragevaults\": { \"SingularDisplayName\": \"Oracle Exascale DB Storage Vault\" }\r\n ,\"oracle.database/networkanchors\": { \"SingularDisplayName\": \"Network Anchor\" }\r\n ,\"oracle.database/oraclesubscriptions\": { \"SingularDisplayName\": \"OracleSubscription\" }\r\n ,\"oracle.database/resourceanchors\": { \"SingularDisplayName\": \"Resource Anchor\" }\r\n ,\"paloaltonetworks.cloudngfw/firewalls\": { \"SingularDisplayName\": \"Cloud NGFW by Palo Alto Networks\" }\r\n ,\"paloaltonetworks.cloudngfw/globalrulestacks\": { \"SingularDisplayName\": \"Global Rulestack\" }\r\n ,\"paloaltonetworks.cloudngfw/localrulestacks\": { \"SingularDisplayName\": \"Local Rulestack for Cloud NGFW by Palo Alto Networks\" }\r\n ,\"pinecone.vectordb/organizations\": { \"SingularDisplayName\": \"Azure Native Pinecone Cloud Service\" }\r\n ,\"purestorage.block/reservations\": { \"SingularDisplayName\": \"Azure Native Pure Storage Cloud Service\" }\r\n ,\"purestorage.block/storagepools\": { \"SingularDisplayName\": \"Storage pool\" }\r\n ,\"purestorage.block/storagepools/avsstoragecontainers\": { \"SingularDisplayName\": \"PureStorage.Block storage pools avs storage container\" }\r\n })[tolower(id)]\r\n}\r\n", - "$fxv#4": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData/Internal')\r\n_resource_type_5(id: string) {\r\n dynamic({\r\n \"qumulo.qaas/storages\": { \"SingularDisplayName\": \"Qumulo.QaaS storage\" }\r\n ,\"qumulo.storage/filesystems\": { \"SingularDisplayName\": \"Azure Native Qumulo Scalable File Service\" }\r\n ,\"solarwinds.observability/organizations\": { \"SingularDisplayName\": \"SolarWinds Observability\" }\r\n ,\"splitio.experimentation/experimentationworkspaces\": { \"SingularDisplayName\": \"Split Experimentation Workspace\" }\r\n ,\"wandisco.fusion/migrators\": { \"SingularDisplayName\": \"LiveData Migrator\" }\r\n ,\"wandisco.fusion/migrators/datatransferagents\": { \"SingularDisplayName\": \"Data Transfer Agent\" }\r\n ,\"wandisco.fusion/migrators/exclusiontemplates\": { \"SingularDisplayName\": \"Exclusion\" }\r\n ,\"wandisco.fusion/migrators/livedatamigrations\": { \"SingularDisplayName\": \"Migration\" }\r\n ,\"wandisco.fusion/migrators/metadatamigrations\": { \"SingularDisplayName\": \"Metadata Migration\" }\r\n ,\"wandisco.fusion/migrators/metadatatargets\": { \"SingularDisplayName\": \"Metadata Target\" }\r\n ,\"wandisco.fusion/migrators/pathmappings\": { \"SingularDisplayName\": \"Path Mapping\" }\r\n ,\"wandisco.fusion/migrators/targets\": { \"SingularDisplayName\": \"Target\" }\r\n ,\"wandisco.fusion/migrators/verifications\": { \"SingularDisplayName\": \"Verification\" }\r\n })[tolower(id)]\r\n}\r\n", - "$fxv#5": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n// resource_type\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData')\r\nresource_type(id: string) {\r\n coalesce(_resource_type_1(id), _resource_type_2(id), _resource_type_3(id), _resource_type_4(id), _resource_type_5(id))\r\n}\r\n", - "$fxv#6": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Common utility functions\r\n//\r\n// TIP: Use Ctrl+K,Ctrl+0 to collapse all regions in VS Code\r\n//======================================================================================================================\r\n\r\n\r\n//===| Date functions |=================================================================================================\r\n\r\n// monthstring\r\n.create-or-alter function \r\nwith (docstring = @'Returns the name of the month for the specified date (e.g. Jan or January)', folder =@'Common') \r\nmonthstring(['date']: datetime, length: int = 9)\r\n{\r\n substring(dynamic(['January','February','March','April','May','June','July','August','September','October','November','December'])[getmonth(['date']) - 1], 0, length)\r\n}\r\n\r\n// datestring\r\n.create-or-alter function \r\nwith (docstring = @'Converts 2 dates into a simple, user-friendly date range (e.g. Jan 1-Jan 3)', folder =@'Common') \r\ndatestring(start: datetime, end: datetime = datetime('0001-01-01'))\r\n{\r\n let month = (d: datetime) { monthstring(d, 3) };\r\n let endDate = iff(end == datetime('0001-01-01'), start, end);\r\n let sameDate = startofday(start) == startofday(endDate);\r\n let sameMonth = startofmonth(start) == startofmonth(endDate);\r\n let sameYear = startofyear(start) == startofyear(endDate);\r\n let fullMonth = startofday(start) == startofmonth(start) and startofday(endDate) == startofday(endofmonth(endDate));\r\n let fullYear = startofday(start) == startofyear(start) and startofday(endDate) == startofday(endofyear(endDate));\r\n let currentYear = sameYear and startofyear(start) == startofyear(now());\r\n case(\r\n // Full year | yyyy (same year) / yyyy-yyyy (diff years)\r\n fullYear,\r\n strcat(getyear(start), iff(sameYear, '', strcat('-', getyear(endDate)))),\r\n // 1 full mo, same year | Mmm yyyy\r\n fullMonth and sameMonth and sameYear,\r\n strcat(month(start), ' ', getyear(start)),\r\n // 2+ full mo, same year | Mmm-Mmm (current year) / Mmm-Mmm yyyy (other year)\r\n fullMonth and sameYear,\r\n strcat(month(start), '-', month(endDate), iff(currentYear, '', strcat(' ', getyear(endDate)))),\r\n // Full mo, diff year | Mmm yyyy-Mmm yyyy\r\n fullMonth and not(sameYear),\r\n strcat(month(start), ' ', getyear(start), '-', month(endDate), ' ', getyear(endDate)),\r\n // Same date | Mmm d (current year) / Mmm d, yyyy (other year)\r\n sameDate,\r\n strcat(month(start), ' ', dayofmonth(start), iff(currentYear, '', strcat(', ', getyear(endDate)))),\r\n // 1 partial M, same Y | Mmm d-d (current year) / Mmm d-d, yyyy (other year)\r\n not(fullMonth) and sameMonth and sameYear,\r\n strcat(month(start), ' ', dayofmonth(start), '-', dayofmonth(endDate), iff(currentYear, '', strcat(' ', getyear(endDate)))),\r\n // 2+ partial M, same Y | Mmm d-Mmm d (current year) / Mmm d-Mmm d, yyyy (other year)\r\n not(fullMonth) and not(sameMonth) and sameYear,\r\n strcat(month(start), ' ', dayofmonth(start), '-', month(endDate), ' ', dayofmonth(endDate), iff(currentYear, '', strcat(', ', getyear(endDate)))),\r\n // All other cases | Mmm d, yyyy-Mmm d, yyyy\r\n strcat(month(start), ' ', dayofmonth(start), ', ', getyear(start), '-', month(endDate), ' ', dayofmonth(endDate), ', ', getyear(endDate))\r\n )\r\n}\r\n\r\n// daterange\r\n.create-or-alter function \r\nwith (docstring = @'DEPRECATED: Please use datestring(); function will be removed on or after the Jan 2026 release', folder =@'Common') \r\ndaterange(start: datetime, end: datetime = datetime('0001-01-01'))\r\n{\r\n datestring(start, end)\r\n}\r\n\r\n// monthsago\r\n.create-or-alter function \r\nwith (docstring = 'DEPRECATED: Please use startofmonth(now(), -<# of months>); function will be removed on or after the Jan 2026 release', folder = 'Common')\r\nmonthsago(months: int)\r\n{\r\n datetime_add('month', -months, startofmonth(now()))\r\n}\r\n\r\n\r\n//===| Number functions |===============================================================================================\r\n// NOTE: Must be defined before string converters\r\n\r\n// delta\r\n.create-or-alter function \r\nwith (docstring = @'Compares 2 values and returns the percentage change from oldval to newval', folder =@'Common') \r\ndelta(oldval: double, newval: double)\r\n{\r\n (newval - todouble(oldval))/oldval\r\n}\r\n\r\n// percentOfTotal\r\n// NOTE: Must be before percent() function\r\n.create-or-alter function \r\nwith (docstring = @'Calculates the percentage of each record based on a required Count column', folder =@'Common') \r\npercentOfTotal(t: (Count: long), tot: long)\r\n{\r\n let total = todouble(tot);\r\n t \r\n | extend Percent = round(Count / total * 100, 3) \r\n | order by Count desc\r\n}\r\n\r\n// percent\r\n.create-or-alter function \r\nwith (docstring = @'Calculates the percentage of each record based on a required Count column', folder =@'Common') \r\npercent(t: (Count: long))\r\n{\r\n let total = todouble(toscalar(t | summarize sum(Count)));\r\n percentOfTotal(t, total)\r\n}\r\n\r\n// plusminus\r\n.create-or-alter function \r\nwith (docstring = 'Shows a +/- sign based on the direction of the number', folder = 'Common')\r\nplusminus(val: string)\r\n{\r\n let neg = substring(val, 0, 1) == '-';\r\n iff(neg, val, strcat('+', val))\r\n}\r\n\r\n// updown\r\n.create-or-alter function \r\nwith (docstring = 'Shows an up/down arrow based on the direction of the number', folder = 'Common')\r\nupdown(val: string)\r\n{\r\n // TODO: Handle 0\r\n let neg = substring(val, 0, 1) == '-';\r\n iff(neg, strcat('↓', substring(val, 1)), strcat('↑', val))\r\n}\r\n\r\n\r\n//===| String functions |===============================================================================================\r\n\r\n// percentstring\r\n// NOTE: Must be defined before deltastring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate a percentage and render as a string', folder = 'Common')\r\npercentstring(num: double, total: double = 1.0, places: int = 9)\r\n{\r\n let value = 1.0 * num / total * 100;\r\n strcat(case(\r\n places != 9, round(value, places),\r\n value < 10, round(value, 2),\r\n round(value, 1)\r\n ), '%')\r\n}\r\n\r\n//----------------------------------------------------------------------------------------------------------------------\r\n\r\n// arraystring\r\n.create-or-alter function \r\nwith (docstring = 'Convert an array to a comma-delimited string', folder = 'Common')\r\narraystring(arr: dynamic)\r\n{\r\n replace_string(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(\r\n tostring(arr)\r\n , @'^\\[\"', '')\r\n , @'\"\\]$', '')\r\n , @'^, ', '')\r\n , @', $', '')\r\n , @'^\\[]$', '')\r\n , '\",\"', ', ')\r\n}\r\n\r\n// deltastring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate a delta percentage and render as a string', folder = 'Common')\r\ndeltastring(oldval: double, newval: double, places: int = 1, useArrows: bool = false)\r\n{\r\n let d = delta(oldval, newval);\r\n strcat(case(useArrows and d > 0, '↑', useArrows and d < 0, '↓', d < 0, '-', ''), percentstring(abs(d), 1, places))\r\n}\r\n\r\n// diffstring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate the difference and render as a string', folder = 'Common')\r\ndiffstring(oldval: double, newval: double, places: int = 1)\r\n{\r\n plusminus(round(newval - oldval, places))\r\n}\r\n\r\n// numberstring\r\n.create-or-alter function \r\nwith (docstring = 'Convert a number to a string', folder = 'Common')\r\nnumberstring(num: double, abbrev: bool = true)\r\n{\r\n replace_regex(case(\r\n num >= 10000000000000, strcat(round(1.0 * num / 1000000000000, 1), 'T'),\r\n num >= 1000000000000, strcat(round(1.0 * num / 1000000000000, 2), 'T'),\r\n num >= 10000000000, strcat(round(1.0 * num / 1000000000, 1), 'B'),\r\n num >= 1000000000, strcat(round(1.0 * num / 1000000000, 2), 'B'),\r\n num >= 10000000, strcat(round(1.0 * num / 1000000, 1), 'M'),\r\n num >= 1000000, strcat(round(1.0 * num / 1000000, 2), 'M'),\r\n num >= 10000, strcat(round(1.0 * num / 1000, 1), 'K'),\r\n // Kusto doesn't support back-refs yet -- num > 1000, replace_regex(tostring(num), @'(\\d)(?=(\\d{3})+\\.)', @'\\1,'), // See https://docs.microsoft.com/azure/data-explorer/kusto/query/re2-library\r\n num > 1000, replace_regex(tostring(num), @'([0-9]{3})$', @',\\1'), //num / 1000, ',', substring(tostring(num), 0) - (num / 1000 * 1000)),\r\n tostring(num)\r\n ), @'\\.0$', '')\r\n}\r\n\r\n\r\n//===| Other |==========================================================================================================\r\n\r\n// ifempty\r\n.create-or-alter function \r\nwith (docstring = 'Replaces an empty value with the specified default value', folder = 'Common')\r\nifempty(val: dynamic, defaultVal: dynamic)\r\n{\r\n iff(isempty(val), defaultVal, val)\r\n}\r\n", - "$fxv#7": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Ingestion database\r\n// Used for data ingestion, normalization, and cleansing.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n//===| Settings |=======================================================================================================\r\n\r\n.create-merge table HubSettingsLog (\r\n version: string,\r\n scopes: dynamic,\r\n retention: dynamic\r\n)\r\n\r\n//----------------------------------------------------------------------------------------------------------------------\r\n\r\n// HubSettings function\r\n.create-or-alter function\r\nwith (docstring='Gets the latest version of hub settings.', folder='Settings')\r\nHubSettings()\r\n{\r\n HubSettingsLog\r\n | extend timestamp = ingestion_time()\r\n | summarize arg_max(timestamp, *)\r\n}\r\n\r\n//----------------------------------------------------------------------------------------------------------------------\r\n\r\n// HubScopes function\r\n.create-or-alter function\r\nwith (docstring='Gets the currently configured scopes.', folder='Settings')\r\nHubScopes()\r\n{\r\n HubSettings\r\n | project scopes\r\n | mv-expand scopes\r\n}\r\n\r\n\r\n//===| Open data |======================================================================================================\r\n\r\n// PricingUnits -- Create table if it doesn't exist\r\n.create-merge table PricingUnits ( ignore: string )\r\n\r\n// PricingUnits -- Remove all columns\r\n.alter table PricingUnits ( ignore: string )\r\n\r\n// PricingUnits -- Redefine all columns to change types\r\n.alter table PricingUnits (\r\n x_PricingUnitDescription: string,\r\n x_PricingBlockSize: real,\r\n PricingUnit: string\r\n)\r\n\r\n// Regions\r\n.create-merge table Regions(\r\n ResourceLocation: string,\r\n RegionId: string,\r\n RegionName: string\r\n)\r\n\r\n// ResourceTypes\r\n.create-merge table ResourceTypes(\r\n x_ResourceType: string,\r\n SingularDisplayName: string,\r\n PluralDisplayName: string,\r\n LowerSingularDisplayName: string,\r\n LowerPluralDisplayName: string,\r\n IsPreview: bool,\r\n Description: string,\r\n IconUri: string\r\n)\r\n\r\n// Services\r\n.create-merge table Services(\r\n x_ConsumedService: string,\r\n x_ResourceType: string,\r\n ServiceName: string,\r\n ServiceCategory: string,\r\n ServiceSubcategory: string,\r\n PublisherName: string,\r\n x_PublisherCategory: string,\r\n x_Environment: string,\r\n x_ServiceModel: string\r\n)\r\n\r\n//----------------------------------------------------------------------------------------------------------------------\r\n\r\n// parse_resourceid\r\n.create-or-alter function\r\nwith (docstring = 'Parses an Azure resource ID to extract resource attributes like the name, type, resource group, and subaccount ID.', folder = 'Common')\r\nparse_resourceid(resourceId: string) {\r\n let ResourceId = tolower(resourceId);\r\n // let ResourceId = tolower('/providers/Microsoft.BillingBenefits/savingsPlanOrders/2d2e284b-0638-427e-b8c6-1b874d4f17c8/sp/xxx');\r\n let SubAccountId = tostring(extract('/subscriptions/[^/]+', 1, ResourceId));\r\n let x_ResourceGroupName = tostring(extract('/resourcegroups/[^/]+', 1, ResourceId));\r\n let providerPath = iff(ResourceId !contains '/providers/', '', split(iff(ResourceId startswith '/subscriptions/', strcat('/providers/microsoft.resources/', ResourceId), ResourceId), '/providers/')[-1]);\r\n let x_ResourceProvider = iff(isempty(providerPath), '', split(providerPath, '/')[0]);\r\n let tmp_ResourceProviderPath = iff(isempty(providerPath), '', substring(providerPath, strlen(x_ResourceProvider) + 1));\r\n let segments = split(tmp_ResourceProviderPath, '/');\r\n let ResourceName = trim(@'/+', replace_string(strcat_array(array_iff(\r\n dynamic([false, true, false, true, false, true, false, true, false, true, false, true, false, true, false, true, false, true]),\r\n segments, dynamic([])), '/'), '//', '/'));\r\n let x_ResourceTypePath = trim(@'/+', replace_string(strcat_array(array_iff(\r\n dynamic([true, false, true, false, true, false, true, false, true, false, true, false, true, false, true, false, true, false]),\r\n segments, dynamic([])), '/'), '//', '/'));\r\n let xRT = iff(isempty(x_ResourceProvider) or isempty(x_ResourceTypePath), '', strcat(x_ResourceProvider, '/', x_ResourceTypePath));\r\n // TODO: Remove ResourceType in 0.9\r\n bag_pack('ResourceId', ResourceId, 'ResourceName', ResourceName, 'ResourceType', xRT, 'SubAccountId', SubAccountId, 'x_ResourceGroupName', x_ResourceGroupName, 'x_ResourceProvider', x_ResourceProvider, 'x_ResourceType', xRT)\r\n}\r\n", - "$fxv#8": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Ingestion database\r\n// Used for data ingestion, normalization, and cleansing.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n//===| ActualCosts |====================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// ActualCosts_raw table -- Create the table if it doesn't exist\r\n.create-merge table ActualCosts_raw ( ignore: string )\r\n\r\n// ActualCosts_raw table -- Remove all columns to allow changing column types\r\n.alter table ActualCosts_raw ( ignore: string )\r\n\r\n// ActualCosts_raw table -- Redefine all columns\r\n.alter table ActualCosts_raw (\r\n AccountName: string,\r\n AccountOwnerId: string,\r\n AdditionalInfo: string,\r\n AvailabilityZone: string,\r\n BillingAccountId: string, \r\n BillingAccountName: string,\r\n BillingCurrency: string,\r\n BillingPeriodEndDate: datetime,\r\n BillingPeriodStartDate: datetime,\r\n BillingProfileId: string,\r\n BillingProfileName: string,\r\n ChargeType: string,\r\n ConsumedService: string,\r\n CostCenter: string,\r\n Cost: real,\r\n Date: datetime,\r\n EffectivePrice: real,\r\n Frequency: string,\r\n InvoiceSection: string,\r\n InvoiceSectionId: string,\r\n IsAzureCreditEligible: bool,\r\n MeterCategory: string,\r\n MeterId: string,\r\n MeterName: string,\r\n MeterRegion: string,\r\n MeterSubCategory: string,\r\n OfferId: string,\r\n PartNumber: string,\r\n PlanName: string,\r\n Product: string,\r\n ProductOrderId: string,\r\n ProductOrderName: string,\r\n PublisherName: string,\r\n PublisherType: string,\r\n Quantity: real,\r\n ReservationId: string,\r\n ReservationName: string,\r\n ResourceGroup: string,\r\n ResourceId: string,\r\n ResourceLocation: string,\r\n ResourceName: string,\r\n ServiceFamily: string,\r\n ServiceInfo1: string,\r\n ServiceInfo2: string,\r\n SubscriptionId: string,\r\n SubscriptionName: string,\r\n Tags: string,\r\n Term: string,\r\n UnitOfMeasure: string,\r\n UnitPrice: real\r\n)\r\n\r\n// ActualCosts_raw ingestion mapping\r\n.create-or-alter table ActualCosts_raw ingestion parquet mapping \"ActualCosts_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"AccountName\", \"Properties\": { \"Field\": \"AccountName\" } },\r\n { \"Column\": \"AccountOwnerId\", \"Properties\": { \"Field\": \"AccountOwnerId\" } },\r\n { \"Column\": \"AdditionalInfo\", \"Properties\": { \"Field\": \"AdditionalInfo\" } },\r\n { \"Column\": \"AvailabilityZone\", \"Properties\": { \"Field\": \"AvailabilityZone\" } },\r\n { \"Column\": \"BillingAccountId\", \"Properties\": { \"Field\": \"BillingAccountId\" } },\r\n { \"Column\": \"BillingAccountName\", \"Properties\": { \"Field\": \"BillingAccountName\" } },\r\n { \"Column\": \"BillingCurrency\", \"Properties\": { \"Field\": \"BillingCurrency\" } },\r\n { \"Column\": \"BillingPeriodEndDate\", \"Properties\": { \"Field\": \"BillingPeriodEndDate\" } },\r\n { \"Column\": \"BillingPeriodStartDate\", \"Properties\": { \"Field\": \"BillingPeriodStartDate\" } },\r\n { \"Column\": \"BillingProfileId\", \"Properties\": { \"Field\": \"BillingProfileId\" } },\r\n { \"Column\": \"BillingProfileName\", \"Properties\": { \"Field\": \"BillingProfileName\" } },\r\n { \"Column\": \"ChargeType\", \"Properties\": { \"Field\": \"ChargeType\" } },\r\n { \"Column\": \"ConsumedService\", \"Properties\": { \"Field\": \"ConsumedService\" } },\r\n { \"Column\": \"CostCenter\", \"Properties\": { \"Field\": \"CostCenter\" } },\r\n { \"Column\": \"Cost\", \"Properties\": { \"Field\": \"Cost\" } },\r\n { \"Column\": \"Date\", \"Properties\": { \"Field\": \"Date\" } },\r\n { \"Column\": \"EffectivePrice\", \"Properties\": { \"Field\": \"EffectivePrice\" } },\r\n { \"Column\": \"Frequency\", \"Properties\": { \"Field\": \"Frequency\" } },\r\n { \"Column\": \"InvoiceSection\", \"Properties\": { \"Field\": \"InvoiceSection\" } },\r\n { \"Column\": \"InvoiceSectionId\", \"Properties\": { \"Field\": \"InvoiceSectionId\" } },\r\n { \"Column\": \"IsAzureCreditEligible\", \"Properties\": { \"Field\": \"IsAzureCreditEligible\" } },\r\n { \"Column\": \"MeterCategory\", \"Properties\": { \"Field\": \"MeterCategory\" } },\r\n { \"Column\": \"MeterId\", \"Properties\": { \"Field\": \"MeterId\" } },\r\n { \"Column\": \"MeterName\", \"Properties\": { \"Field\": \"MeterName\" } },\r\n { \"Column\": \"MeterRegion\", \"Properties\": { \"Field\": \"MeterRegion\" } },\r\n { \"Column\": \"MeterSubCategory\", \"Properties\": { \"Field\": \"MeterSubCategory\" } },\r\n { \"Column\": \"OfferId\", \"Properties\": { \"Field\": \"OfferId\" } },\r\n { \"Column\": \"PartNumber\", \"Properties\": { \"Field\": \"PartNumber\" } },\r\n { \"Column\": \"PlanName\", \"Properties\": { \"Field\": \"PlanName\" } },\r\n { \"Column\": \"Product\", \"Properties\": { \"Field\": \"Product\" } },\r\n { \"Column\": \"ProductOrderId\", \"Properties\": { \"Field\": \"ProductOrderId\" } },\r\n { \"Column\": \"ProductOrderName\", \"Properties\": { \"Field\": \"ProductOrderName\" } },\r\n { \"Column\": \"PublisherName\", \"Properties\": { \"Field\": \"PublisherName\" } },\r\n { \"Column\": \"PublisherType\", \"Properties\": { \"Field\": \"PublisherType\" } },\r\n { \"Column\": \"Quantity\", \"Properties\": { \"Field\": \"Quantity\" } },\r\n { \"Column\": \"ReservationId\", \"Properties\": { \"Field\": \"ReservationId\" } },\r\n { \"Column\": \"ReservationName\", \"Properties\": { \"Field\": \"ReservationName\" } },\r\n { \"Column\": \"ResourceGroup\", \"Properties\": { \"Field\": \"ResourceGroup\" } },\r\n { \"Column\": \"ResourceId\", \"Properties\": { \"Field\": \"ResourceId\" } },\r\n { \"Column\": \"ResourceLocation\", \"Properties\": { \"Field\": \"ResourceLocation\" } },\r\n { \"Column\": \"ResourceName\", \"Properties\": { \"Field\": \"ResourceName\" } },\r\n { \"Column\": \"ServiceFamily\", \"Properties\": { \"Field\": \"ServiceFamily\" } },\r\n { \"Column\": \"ServiceInfo1\", \"Properties\": { \"Field\": \"ServiceInfo1\" } },\r\n { \"Column\": \"ServiceInfo2\", \"Properties\": { \"Field\": \"ServiceInfo2\" } },\r\n { \"Column\": \"SubscriptionId\", \"Properties\": { \"Field\": \"SubscriptionId\" } },\r\n { \"Column\": \"SubscriptionName\", \"Properties\": { \"Field\": \"SubscriptionName\" } },\r\n { \"Column\": \"Tags\", \"Properties\": { \"Field\": \"Tags\" } },\r\n { \"Column\": \"Term\", \"Properties\": { \"Field\": \"Term\" } },\r\n { \"Column\": \"UnitOfMeasure\", \"Properties\": { \"Field\": \"UnitOfMeasure\" } },\r\n { \"Column\": \"UnitPrice\", \"Properties\": { \"Field\": \"UnitPrice\" } }\r\n]\r\n```\r\n\r\n// ActualCosts_raw retention policy (clear historical data)\r\n.alter-merge table ActualCosts_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// ActualCosts_raw retention policy (set the user-defined retention period)\r\n.alter-merge table ActualCosts_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable ActualCosts_raw streaming ingestion (required for Fabric)\r\n.alter table ActualCosts_raw policy streamingingestion disable\r\n\r\n\r\n//===| AmortizedCosts |=================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// AmortizedCosts_raw table -- Create the table if it doesn't exist\r\n.create-merge table AmortizedCosts_raw ( ignore: string )\r\n\r\n// AmortizedCosts_raw table -- Remove all columns to allow changing column types\r\n.alter table AmortizedCosts_raw ( ignore: string )\r\n\r\n// AmortizedCosts_raw table -- Redefine all columns\r\n.alter table AmortizedCosts_raw (\r\n AccountName: string,\r\n AccountOwnerId: string,\r\n AdditionalInfo: string,\r\n AvailabilityZone: string,\r\n BillingAccountId: string, \r\n BillingAccountName: string,\r\n BillingCurrency: string,\r\n BillingPeriodEndDate: datetime,\r\n BillingPeriodStartDate: datetime,\r\n BillingProfileId: string,\r\n BillingProfileName: string,\r\n ChargeType: string,\r\n ConsumedService: string,\r\n CostCenter: string,\r\n Cost: real,\r\n Date: datetime,\r\n EffectivePrice: real,\r\n Frequency: string,\r\n InvoiceSection: string,\r\n InvoiceSectionId: string,\r\n IsAzureCreditEligible: bool,\r\n MeterCategory: string,\r\n MeterId: string,\r\n MeterName: string,\r\n MeterRegion: string,\r\n MeterSubCategory: string,\r\n OfferId: string,\r\n PartNumber: string,\r\n PlanName: string,\r\n Product: string,\r\n ProductOrderId: string,\r\n ProductOrderName: string,\r\n PublisherName: string,\r\n PublisherType: string,\r\n Quantity: real,\r\n ReservationId: string,\r\n ReservationName: string,\r\n ResourceGroup: string,\r\n ResourceId: string,\r\n ResourceLocation: string,\r\n ResourceName: string,\r\n ServiceFamily: string,\r\n ServiceInfo1: string,\r\n ServiceInfo2: string,\r\n SubscriptionId: string,\r\n SubscriptionName: string,\r\n Tags: string,\r\n Term: string,\r\n UnitOfMeasure: string,\r\n UnitPrice: real\r\n)\r\n\r\n// AmortizedCosts_raw ingestion mapping\r\n.create-or-alter table AmortizedCosts_raw ingestion parquet mapping \"AmortizedCosts_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"AccountName\", \"Properties\": { \"Field\": \"AccountName\" } },\r\n { \"Column\": \"AccountOwnerId\", \"Properties\": { \"Field\": \"AccountOwnerId\" } },\r\n { \"Column\": \"AdditionalInfo\", \"Properties\": { \"Field\": \"AdditionalInfo\" } },\r\n { \"Column\": \"AvailabilityZone\", \"Properties\": { \"Field\": \"AvailabilityZone\" } },\r\n { \"Column\": \"BillingAccountId\", \"Properties\": { \"Field\": \"BillingAccountId\" } },\r\n { \"Column\": \"BillingAccountName\", \"Properties\": { \"Field\": \"BillingAccountName\" } },\r\n { \"Column\": \"BillingCurrency\", \"Properties\": { \"Field\": \"BillingCurrency\" } },\r\n { \"Column\": \"BillingPeriodEndDate\", \"Properties\": { \"Field\": \"BillingPeriodEndDate\" } },\r\n { \"Column\": \"BillingPeriodStartDate\", \"Properties\": { \"Field\": \"BillingPeriodStartDate\" } },\r\n { \"Column\": \"BillingProfileId\", \"Properties\": { \"Field\": \"BillingProfileId\" } },\r\n { \"Column\": \"BillingProfileName\", \"Properties\": { \"Field\": \"BillingProfileName\" } },\r\n { \"Column\": \"ChargeType\", \"Properties\": { \"Field\": \"ChargeType\" } },\r\n { \"Column\": \"ConsumedService\", \"Properties\": { \"Field\": \"ConsumedService\" } },\r\n { \"Column\": \"CostCenter\", \"Properties\": { \"Field\": \"CostCenter\" } },\r\n { \"Column\": \"Cost\", \"Properties\": { \"Field\": \"Cost\" } },\r\n { \"Column\": \"Date\", \"Properties\": { \"Field\": \"Date\" } },\r\n { \"Column\": \"EffectivePrice\", \"Properties\": { \"Field\": \"EffectivePrice\" } },\r\n { \"Column\": \"Frequency\", \"Properties\": { \"Field\": \"Frequency\" } },\r\n { \"Column\": \"InvoiceSection\", \"Properties\": { \"Field\": \"InvoiceSection\" } },\r\n { \"Column\": \"InvoiceSectionId\", \"Properties\": { \"Field\": \"InvoiceSectionId\" } },\r\n { \"Column\": \"IsAzureCreditEligible\", \"Properties\": { \"Field\": \"IsAzureCreditEligible\" } },\r\n { \"Column\": \"MeterCategory\", \"Properties\": { \"Field\": \"MeterCategory\" } },\r\n { \"Column\": \"MeterId\", \"Properties\": { \"Field\": \"MeterId\" } },\r\n { \"Column\": \"MeterName\", \"Properties\": { \"Field\": \"MeterName\" } },\r\n { \"Column\": \"MeterRegion\", \"Properties\": { \"Field\": \"MeterRegion\" } },\r\n { \"Column\": \"MeterSubCategory\", \"Properties\": { \"Field\": \"MeterSubCategory\" } },\r\n { \"Column\": \"OfferId\", \"Properties\": { \"Field\": \"OfferId\" } },\r\n { \"Column\": \"PartNumber\", \"Properties\": { \"Field\": \"PartNumber\" } },\r\n { \"Column\": \"PlanName\", \"Properties\": { \"Field\": \"PlanName\" } },\r\n { \"Column\": \"Product\", \"Properties\": { \"Field\": \"Product\" } },\r\n { \"Column\": \"ProductOrderId\", \"Properties\": { \"Field\": \"ProductOrderId\" } },\r\n { \"Column\": \"ProductOrderName\", \"Properties\": { \"Field\": \"ProductOrderName\" } },\r\n { \"Column\": \"PublisherName\", \"Properties\": { \"Field\": \"PublisherName\" } },\r\n { \"Column\": \"PublisherType\", \"Properties\": { \"Field\": \"PublisherType\" } },\r\n { \"Column\": \"Quantity\", \"Properties\": { \"Field\": \"Quantity\" } },\r\n { \"Column\": \"ReservationId\", \"Properties\": { \"Field\": \"ReservationId\" } },\r\n { \"Column\": \"ReservationName\", \"Properties\": { \"Field\": \"ReservationName\" } },\r\n { \"Column\": \"ResourceGroup\", \"Properties\": { \"Field\": \"ResourceGroup\" } },\r\n { \"Column\": \"ResourceId\", \"Properties\": { \"Field\": \"ResourceId\" } },\r\n { \"Column\": \"ResourceLocation\", \"Properties\": { \"Field\": \"ResourceLocation\" } },\r\n { \"Column\": \"ResourceName\", \"Properties\": { \"Field\": \"ResourceName\" } },\r\n { \"Column\": \"ServiceFamily\", \"Properties\": { \"Field\": \"ServiceFamily\" } },\r\n { \"Column\": \"ServiceInfo1\", \"Properties\": { \"Field\": \"ServiceInfo1\" } },\r\n { \"Column\": \"ServiceInfo2\", \"Properties\": { \"Field\": \"ServiceInfo2\" } },\r\n { \"Column\": \"SubscriptionId\", \"Properties\": { \"Field\": \"SubscriptionId\" } },\r\n { \"Column\": \"SubscriptionName\", \"Properties\": { \"Field\": \"SubscriptionName\" } },\r\n { \"Column\": \"Tags\", \"Properties\": { \"Field\": \"Tags\" } },\r\n { \"Column\": \"Term\", \"Properties\": { \"Field\": \"Term\" } },\r\n { \"Column\": \"UnitOfMeasure\", \"Properties\": { \"Field\": \"UnitOfMeasure\" } },\r\n { \"Column\": \"UnitPrice\", \"Properties\": { \"Field\": \"UnitPrice\" } }\r\n]\r\n```\r\n\r\n// AmortizedCosts_raw retention policy (clear historical data)\r\n.alter-merge table AmortizedCosts_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// AmortizedCosts_raw retention policy (set the user-defined retention period)\r\n.alter-merge table AmortizedCosts_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable AmortizedCosts_raw streaming ingestion (required for Fabric)\r\n.alter table AmortizedCosts_raw policy streamingingestion disable\r\n\r\n\r\n//===| CommitmentDiscountUsage |========================================================================================\r\n// Supported versions:\r\n// - MS EA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-ea\r\n// - MS MCA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-mca\r\n//======================================================================================================================\r\n\r\n// CommitmentDiscountUsage_raw table -- Create the table if it doesn't exist\r\n.create-merge table CommitmentDiscountUsage_raw ( ignore: string )\r\n\r\n// CommitmentDiscountUsage_raw table -- Remove all columns to allow changing column types\r\n.alter table CommitmentDiscountUsage_raw ( ignore: string )\r\n\r\n// CommitmentDiscountUsage_raw table -- Redefine all columns\r\n.alter table CommitmentDiscountUsage_raw (\r\n InstanceFlexibilityGroup: string,\r\n InstanceFlexibilityRatio: real,\r\n InstanceId: string,\r\n Kind: string,\r\n ReservationId: string,\r\n ReservationOrderId: string,\r\n ReservedHours: real,\r\n SkuName: string,\r\n TotalReservedQuantity: real,\r\n UsageDate: datetime,\r\n UsedHours: real,\r\n x_SourceName: string, // Hubs v1_0+\r\n x_SourceProvider: string, // Hubs v1_0+\r\n x_SourceType: string, // Hubs v1_0+\r\n x_SourceVersion: string // Hubs v1_0+\r\n)\r\n\r\n// CommitmentDiscountUsage_raw ingestion mapping\r\n.create-or-alter table CommitmentDiscountUsage_raw ingestion parquet mapping \"CommitmentDiscountUsage_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"InstanceFlexibilityGroup\", \"Properties\": { \"Field\": \"InstanceFlexibilityGroup\" } },\r\n { \"Column\": \"InstanceFlexibilityRatio\", \"Properties\": { \"Field\": \"InstanceFlexibilityRatio\" } },\r\n { \"Column\": \"InstanceId\", \"Properties\": { \"Field\": \"InstanceId\" } },\r\n { \"Column\": \"Kind\", \"Properties\": { \"Field\": \"Kind\" } },\r\n { \"Column\": \"ReservationId\", \"Properties\": { \"Field\": \"ReservationId\" } },\r\n { \"Column\": \"ReservationOrderId\", \"Properties\": { \"Field\": \"ReservationOrderId\" } },\r\n { \"Column\": \"ReservedHours\", \"Properties\": { \"Field\": \"ReservedHours\" } },\r\n { \"Column\": \"SkuName\", \"Properties\": { \"Field\": \"SkuName\" } },\r\n { \"Column\": \"TotalReservedQuantity\", \"Properties\": { \"Field\": \"TotalReservedQuantity\" } },\r\n { \"Column\": \"UsageDate\", \"Properties\": { \"Field\": \"UsageDate\" } },\r\n { \"Column\": \"UsedHours\", \"Properties\": { \"Field\": \"UsedHours\" } },\r\n { \"Column\": \"x_SourceName\", \"Properties\": { \"Field\": \"x_SourceName\" } },\r\n { \"Column\": \"x_SourceProvider\", \"Properties\": { \"Field\": \"x_SourceProvider\" } },\r\n { \"Column\": \"x_SourceType\", \"Properties\": { \"Field\": \"x_SourceType\" } },\r\n { \"Column\": \"x_SourceVersion\", \"Properties\": { \"Field\": \"x_SourceVersion\" } }\r\n]\r\n```\r\n\r\n// CommitmentDiscountUsage_raw retention policy (clear historical data)\r\n.alter-merge table CommitmentDiscountUsage_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// CommitmentDiscountUsage_raw retention policy (set the user-defined retention period)\r\n.alter-merge table CommitmentDiscountUsage_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable CommitmentDiscountUsage_raw streaming ingestion (required for Fabric)\r\n.alter table CommitmentDiscountUsage_raw policy streamingingestion disable\r\n\r\n\r\n//===| Costs |==========================================================================================================\r\n// Supported versions:\r\n// - MS: 1.0, 1.0-preview(v1) -- See https://aka.ms/costmgmt/exports/focus\r\n// - AWS: 1.0 -- See https://docs.aws.amazon.com/cur/latest/userguide/table-dictionary-focus-1-0-aws-columns.html\r\n// - GCP: Jan-Jun 2024 -- See https://cloud.google.com/resources/google-cloud-focus?e=48754805&hl=en\r\n// Links to (Aug 2024): https://services.google.com/fh/files/misc/focus_guide_v1.pdf\r\n// See also:\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/standard-usage\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/detailed-usage\r\n// - OCI: 1.0 -- See https://docs.oracle.com/iaas/Content/Billing/Concepts/costusagereportsoverview.htm#costreports__focus-cost-report-schema\r\n// - Tencent: 1.0 -- See https://www.tencentcloud.com/document/product/555/67495 / https://www.tencentcloud.com/document/product/555/67496\r\n// - Alibaba: 1.0 -- See https://www.alibabacloud.com/help/en/user-center/user-guide/export-alibaba-cloud-standard-billing-focus\r\n//\r\n// Support for non-Azure data is limited to ingestion only. Data is not transformed across versions.\r\n//======================================================================================================================\r\n\r\n// Costs_raw table -- Create the table if it doesn't exist\r\n.create-merge table Costs_raw ( ignore: string )\r\n\r\n// Costs_raw table -- Remove all columns to allow changing column types\r\n.alter table Costs_raw ( ignore: string )\r\n\r\n// Costs_raw table -- Redefine all columns\r\n.alter table Costs_raw (\r\n AvailabilityZone: string, // FOCUS 0.5+\r\n BilledCost: real, // FOCUS 0.5+\r\n BillingAccountId: string, // FOCUS 0.5+\r\n BillingAccountName: string, // FOCUS 0.5+\r\n BillingAccountType: string, // Azure 1.0-preview(v1)+\r\n BillingCurrency: string, // FOCUS 0.5+\r\n BillingPeriodEnd: datetime, // FOCUS 0.5+\r\n BillingPeriodStart: datetime, // FOCUS 0.5+\r\n CapacityReservationId: string, // FOCUS 1.1+\r\n CapacityReservationStatus: string, // FOCUS 1.1+\r\n ChargeCategory: string, // FOCUS 1.0-preview+\r\n ChargeClass: string, // FOCUS 1.0+\r\n ChargeDescription: string, // FOCUS 1.0+\r\n ChargeFrequency: string, // FOCUS 1.0+\r\n ChargePeriodEnd: datetime, // FOCUS 0.5+\r\n ChargePeriodStart: datetime, // FOCUS 0.5+\r\n ChargeSubcategory: string, // FOCUS 1.0-preview only\r\n CommitmentDiscountCategory: string, // FOCUS 1.0-preview+\r\n CommitmentDiscountId: string, // FOCUS 1.0-preview+\r\n CommitmentDiscountName: string, // FOCUS 1.0-preview+\r\n CommitmentDiscountQuantity: real, // FOCUS 1.1+\r\n CommitmentDiscountStatus: string, // FOCUS 1.0+\r\n CommitmentDiscountType: string, // FOCUS 1.0-preview+\r\n CommitmentDiscountUnit: string, // FOCUS 1.1+\r\n ConsumedQuantity: real, // FOCUS 1.0+\r\n ConsumedUnit: string, // FOCUS 1.0+\r\n ContractedCost: real, // FOCUS 1.0+\r\n ContractedUnitPrice: real, // FOCUS 1.0+\r\n EffectiveCost: real, // FOCUS 1.0-preview+\r\n InvoiceId: string, // FOCUS 1.2+\r\n InvoiceIssuerName: string, // FOCUS 0.5+\r\n ListCost: real, // FOCUS 1.0-preview+\r\n ListUnitPrice: real, // FOCUS 1.0-preview+\r\n PricingCategory: string, // FOCUS 1.0-preview+\r\n PricingCurrency: string, // FOCUS 1.2+\r\n PricingQuantity: real, // FOCUS 1.0-preview+\r\n PricingUnit: string, // FOCUS 1.0-preview+\r\n ProviderName: string, // FOCUS 0.5+\r\n PublisherName: string, // FOCUS 0.5+\r\n Region: string, // FOCUS 0.5-1.0-preview (deprecated)\r\n RegionId: string, // FOCUS 1.0+\r\n RegionName: string, // FOCUS 1.0+\r\n ResourceId: string, // FOCUS 0.5+\r\n ResourceName: string, // FOCUS 0.5+\r\n ResourceType: string, // FOCUS 1.0-preview+\r\n ServiceCategory: string, // FOCUS 0.5+\r\n ServiceName: string, // FOCUS 0.5+\r\n ServiceSubcategory: string, // FOCUS 1.1+\r\n SkuId: string, // FOCUS 1.0-preview+\r\n SkuMeter: string, // FOCUS 1.1+\r\n SkuPriceDetails: string, // FOCUS 1.1+\r\n SkuPriceId: string, // FOCUS 1.0-preview+\r\n SubAccountId: string, // FOCUS 0.5+\r\n SubAccountName: string, // FOCUS 0.5+\r\n SubAccountType: string, // Azure 1.0-preview(v1)+\r\n Tags: string, // FOCUS 1.0-preview+\r\n UsageAmount: real, // GCP Jan 2024 -- Removed Mar 2024 (UsageQuantity)\r\n UsageQuantity: real, // FOCUS 1.0-preview only\r\n UsageUnit: string, // FOCUS 1.0-preview only\r\n x_AccountId: string, // Azure 1.0-preview(v1)+\r\n x_AccountName: string, // Azure 1.0-preview(v1)+\r\n x_AccountOwnerId: string, // Azure 1.0-preview(v1)+\r\n x_AmortizationClass: string, // Azure 1.2-preview+\r\n x_BilledCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_BilledUnitPrice: real, // Azure 1.0-preview(v1)+\r\n x_BillingAccountId: string, // Azure 1.0-preview(v1)+\r\n x_BillingAccountName: string, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRate: real, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRateDate: datetime, // Azure 1.0-preview(v1)+\r\n x_BillingItemCode: string, // Alibaba 1.0+\r\n x_BillingItemName: string, // Alibaba 1.0+\r\n x_BillingProfileId: string, // Azure 1.0-preview(v1)+\r\n x_BillingProfileName: string, // Azure 1.0-preview(v1)+\r\n x_ChargeId: string, // Azure 1.0-preview(v1) only\r\n x_CommodityCode: string, // Alibaba 1.0+\r\n x_CommodityName: string, // Alibaba 1.0+\r\n x_ComponentName: string, // Tencent 1.0+\r\n x_ComponentType: string, // Tencent 1.0+\r\n x_ContractedCostInUsd: real, // Azure 1.0+\r\n x_Cost: real, // GCP Jan 2024 -- Removed Jun 2024 (ContractedCost)\r\n x_CostAllocationRuleName: string, // Azure 1.0-preview(v1)+\r\n x_CostCategories: string, // AWS 1.0 (JSON)\r\n x_CostCenter: string, // Azure 1.0-preview(v1)+\r\n x_CostType: string, // GCP Jan 2024\r\n x_Credits: string, // GCP Jan 2024\r\n x_CurrencyConversionRate: real, // GCP Jun 2024\r\n x_CustomerId: string, // Azure 1.0-preview(v1)+\r\n x_CustomerName: string, // Azure 1.0-preview(v1)+\r\n x_Discount: string, // AWS 1.0 (JSON)\r\n x_EffectiveCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_EffectiveUnitPrice: real, // Azure 1.0-preview(v1)+\r\n x_ExportTime: datetime, // GCP Jan 2024 / Tencent 1.0+\r\n x_InstanceID: string, // Alibaba 1.0+\r\n x_InvoiceId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceIssuerId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionName: string, // Azure 1.0-preview(v1)+\r\n x_ListCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_Location: string, // GCP Jan 2024\r\n x_OnDemandCost: real, // Azure 1.0-preview(v1) only\r\n x_OnDemandCostInUsd: real, // Azure 1.0-preview(v1) only\r\n x_OnDemandUnitPrice: real, // Azure 1.0-preview(v1) only\r\n x_Operation: string, // AWS 1.0\r\n x_OwnerAccountID: string, // Tencent 1.0+\r\n x_PartnerCreditApplied: string, // Azure 1.0-preview(v1)+\r\n x_PartnerCreditRate: string, // Azure 1.0-preview(v1)+\r\n x_PricingBlockSize: real, // Azure 1.0-preview(v1)+\r\n x_PricingCurrency: string, // Azure 1.0-preview(v1)-1.0r2\r\n x_PricingSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_PricingUnitDescription: string, // Azure 1.0-preview(v1)+\r\n x_Project: string, // GCP Jan 2024\r\n x_PublisherCategory: string, // Azure 1.0-preview(v1)+\r\n x_PublisherId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceGroupName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceType: string, // Azure 1.0-preview(v1)+\r\n x_ServiceCode: string, // AWS 1.0\r\n x_ServiceId: string, // GCP Jan 2024\r\n x_ServiceModel: string, // Azure 1.2-preview+\r\n x_ServicePeriodEnd: datetime, // Azure 1.0-preview(v1)+\r\n x_ServicePeriodStart: datetime, // Azure 1.0-preview(v1)+\r\n x_SkuDescription: string, // Azure 1.0-preview(v1)+\r\n x_SkuDetails: string, // Azure 1.0-preview(v1)-1.2-preview\r\n x_SkuIsCreditEligible: bool, // Azure 1.0-preview(v1)+\r\n x_SkuMeterCategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterId: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterName: string, // Azure 1.0-preview(v1)-1.0r2\r\n x_SkuMeterSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuOfferId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOrderId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOrderName: string, // Azure 1.0-preview(v1)+\r\n x_SkuPartNumber: string, // Azure 1.0-preview(v1)+\r\n x_SkuPlanName: string, // Azure 1.2-preview+\r\n x_SkuRegion: string, // Azure 1.0-preview(v1)+\r\n x_SkuServiceFamily: string, // Azure 1.0-preview(v1)+\r\n x_SkuTerm: int, // Azure 1.0-preview(v1)+\r\n x_SkuTier: string, // Azure 1.0-preview(v1)+\r\n x_SourceName: string, // Hubs v1_0+\r\n x_SourceProvider: string, // Hubs v1_0+\r\n x_SourceType: string, // Hubs v1_0+\r\n x_SourceVersion: string, // Hubs v1_0+\r\n x_SubproductName: string, // Tencent 1.0+ // cSpell:ignore Subproduct\r\n x_UsageType: string // AWS 1.0\r\n)\r\n\r\n// Costs_raw ingestion mapping\r\n.create-or-alter table Costs_raw ingestion parquet mapping \"Costs_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"AvailabilityZone\", \"Properties\": { \"Field\": \"AvailabilityZone\" } },\r\n { \"Column\": \"BilledCost\", \"Properties\": { \"Field\": \"BilledCost\" } },\r\n { \"Column\": \"BillingAccountId\", \"Properties\": { \"Field\": \"BillingAccountId\" } },\r\n { \"Column\": \"BillingAccountName\", \"Properties\": { \"Field\": \"BillingAccountName\" } },\r\n { \"Column\": \"BillingAccountType\", \"Properties\": { \"Field\": \"BillingAccountType\" } },\r\n { \"Column\": \"BillingCurrency\", \"Properties\": { \"Field\": \"BillingCurrency\" } },\r\n { \"Column\": \"BillingPeriodEnd\", \"Properties\": { \"Field\": \"BillingPeriodEnd\" } },\r\n { \"Column\": \"BillingPeriodStart\", \"Properties\": { \"Field\": \"BillingPeriodStart\" } },\r\n { \"Column\": \"CapacityReservationId\", \"Properties\": { \"Field\": \"CapacityReservationId\" } },\r\n { \"Column\": \"CapacityReservationStatus\", \"Properties\": { \"Field\": \"CapacityReservationStatus\" } },\r\n { \"Column\": \"ChargeCategory\", \"Properties\": { \"Field\": \"ChargeCategory\" } },\r\n { \"Column\": \"ChargeClass\", \"Properties\": { \"Field\": \"ChargeClass\" } },\r\n { \"Column\": \"ChargeDescription\", \"Properties\": { \"Field\": \"ChargeDescription\" } },\r\n { \"Column\": \"ChargeFrequency\", \"Properties\": { \"Field\": \"ChargeFrequency\" } },\r\n { \"Column\": \"ChargePeriodEnd\", \"Properties\": { \"Field\": \"ChargePeriodEnd\" } },\r\n { \"Column\": \"ChargePeriodStart\", \"Properties\": { \"Field\": \"ChargePeriodStart\" } },\r\n { \"Column\": \"ChargeSubcategory\", \"Properties\": { \"Field\": \"ChargeSubcategory\" } },\r\n { \"Column\": \"CommitmentDiscountCategory\", \"Properties\": { \"Field\": \"CommitmentDiscountCategory\" } },\r\n { \"Column\": \"CommitmentDiscountId\", \"Properties\": { \"Field\": \"CommitmentDiscountId\" } },\r\n { \"Column\": \"CommitmentDiscountName\", \"Properties\": { \"Field\": \"CommitmentDiscountName\" } },\r\n { \"Column\": \"CommitmentDiscountQuantity\", \"Properties\": { \"Field\": \"CommitmentDiscountQuantity\" } },\r\n { \"Column\": \"CommitmentDiscountStatus\", \"Properties\": { \"Field\": \"CommitmentDiscountStatus\" } },\r\n { \"Column\": \"CommitmentDiscountType\", \"Properties\": { \"Field\": \"CommitmentDiscountType\" } },\r\n { \"Column\": \"CommitmentDiscountUnit\", \"Properties\": { \"Field\": \"CommitmentDiscountUnit\" } },\r\n { \"Column\": \"ConsumedQuantity\", \"Properties\": { \"Field\": \"ConsumedQuantity\" } },\r\n { \"Column\": \"ConsumedUnit\", \"Properties\": { \"Field\": \"ConsumedUnit\" } },\r\n { \"Column\": \"ContractedCost\", \"Properties\": { \"Field\": \"ContractedCost\" } },\r\n { \"Column\": \"ContractedUnitPrice\", \"Properties\": { \"Field\": \"ContractedUnitPrice\" } },\r\n { \"Column\": \"EffectiveCost\", \"Properties\": { \"Field\": \"EffectiveCost\" } },\r\n { \"Column\": \"InvoiceId\", \"Properties\": { \"Field\": \"InvoiceId\" } },\r\n { \"Column\": \"InvoiceIssuerName\", \"Properties\": { \"Field\": \"InvoiceIssuerName\" } },\r\n { \"Column\": \"ListCost\", \"Properties\": { \"Field\": \"ListCost\" } },\r\n { \"Column\": \"ListUnitPrice\", \"Properties\": { \"Field\": \"ListUnitPrice\" } },\r\n { \"Column\": \"PricingCategory\", \"Properties\": { \"Field\": \"PricingCategory\" } },\r\n { \"Column\": \"PricingCurrency\", \"Properties\": { \"Field\": \"PricingCurrency\" } },\r\n { \"Column\": \"PricingQuantity\", \"Properties\": { \"Field\": \"PricingQuantity\" } },\r\n { \"Column\": \"PricingUnit\", \"Properties\": { \"Field\": \"PricingUnit\" } },\r\n { \"Column\": \"ProviderName\", \"Properties\": { \"Field\": \"ProviderName\" } },\r\n { \"Column\": \"PublisherName\", \"Properties\": { \"Field\": \"PublisherName\" } },\r\n { \"Column\": \"Region\", \"Properties\": { \"Field\": \"Region\" } },\r\n { \"Column\": \"RegionId\", \"Properties\": { \"Field\": \"RegionId\" } },\r\n { \"Column\": \"RegionName\", \"Properties\": { \"Field\": \"RegionName\" } },\r\n { \"Column\": \"ResourceId\", \"Properties\": { \"Field\": \"ResourceId\" } },\r\n { \"Column\": \"ResourceName\", \"Properties\": { \"Field\": \"ResourceName\" } },\r\n { \"Column\": \"ResourceType\", \"Properties\": { \"Field\": \"ResourceType\" } },\r\n { \"Column\": \"ServiceCategory\", \"Properties\": { \"Field\": \"ServiceCategory\" } },\r\n { \"Column\": \"ServiceName\", \"Properties\": { \"Field\": \"ServiceName\" } },\r\n { \"Column\": \"ServiceSubcategory\", \"Properties\": { \"Field\": \"ServiceSubcategory\" } },\r\n { \"Column\": \"SkuId\", \"Properties\": { \"Field\": \"SkuId\" } },\r\n { \"Column\": \"SkuMeter\", \"Properties\": { \"Field\": \"SkuMeter\" } },\r\n { \"Column\": \"SkuPriceDetails\", \"Properties\": { \"Field\": \"SkuPriceDetails\" } },\r\n { \"Column\": \"SkuPriceId\", \"Properties\": { \"Field\": \"SkuPriceId\" } },\r\n { \"Column\": \"SubAccountId\", \"Properties\": { \"Field\": \"SubAccountId\" } },\r\n { \"Column\": \"SubAccountName\", \"Properties\": { \"Field\": \"SubAccountName\" } },\r\n { \"Column\": \"SubAccountType\", \"Properties\": { \"Field\": \"SubAccountType\" } },\r\n { \"Column\": \"Tags\", \"Properties\": { \"Field\": \"Tags\" } },\r\n { \"Column\": \"UsageAmount\", \"Properties\": { \"Field\": \"UsageAmount\" } },\r\n { \"Column\": \"UsageQuantity\", \"Properties\": { \"Field\": \"UsageQuantity\" } },\r\n { \"Column\": \"UsageUnit\", \"Properties\": { \"Field\": \"UsageUnit\" } },\r\n { \"Column\": \"x_AccountId\", \"Properties\": { \"Field\": \"x_AccountId\" } },\r\n { \"Column\": \"x_AccountName\", \"Properties\": { \"Field\": \"x_AccountName\" } },\r\n { \"Column\": \"x_AccountOwnerId\", \"Properties\": { \"Field\": \"x_AccountOwnerId\" } },\r\n { \"Column\": \"x_AmortizationClass\", \"Properties\": { \"Field\": \"x_AmortizationClass\" } },\r\n { \"Column\": \"x_BilledCostInUsd\", \"Properties\": { \"Field\": \"x_BilledCostInUsd\" } },\r\n { \"Column\": \"x_BilledUnitPrice\", \"Properties\": { \"Field\": \"x_BilledUnitPrice\" } },\r\n { \"Column\": \"x_BillingAccountId\", \"Properties\": { \"Field\": \"x_BillingAccountId\" } },\r\n { \"Column\": \"x_BillingAccountName\", \"Properties\": { \"Field\": \"x_BillingAccountName\" } },\r\n { \"Column\": \"x_BillingExchangeRate\", \"Properties\": { \"Field\": \"x_BillingExchangeRate\" } },\r\n { \"Column\": \"x_BillingExchangeRateDate\", \"Properties\": { \"Field\": \"x_BillingExchangeRateDate\" } },\r\n { \"Column\": \"x_BillingItemCode\", \"Properties\": { \"Field\": \"x_BillingItemCode\" } },\r\n { \"Column\": \"x_BillingItemName\", \"Properties\": { \"Field\": \"x_BillingItemName\" } },\r\n { \"Column\": \"x_BillingProfileId\", \"Properties\": { \"Field\": \"x_BillingProfileId\" } },\r\n { \"Column\": \"x_BillingProfileName\", \"Properties\": { \"Field\": \"x_BillingProfileName\" } },\r\n { \"Column\": \"x_ChargeId\", \"Properties\": { \"Field\": \"x_ChargeId\" } },\r\n { \"Column\": \"x_ContractedCostInUsd\", \"Properties\": { \"Field\": \"x_ContractedCostInUsd\" } },\r\n { \"Column\": \"x_CommodityCode\", \"Properties\": { \"Field\": \"x_CommodityCode\" } },\r\n { \"Column\": \"x_CommodityName\", \"Properties\": { \"Field\": \"x_CommodityName\" } },\r\n { \"Column\": \"x_ComponentName\", \"Properties\": { \"Field\": \"x_ComponentName\" } },\r\n { \"Column\": \"x_ComponentType\", \"Properties\": { \"Field\": \"x_ComponentType\" } },\r\n { \"Column\": \"x_Cost\", \"Properties\": { \"Field\": \"x_Cost\" } },\r\n { \"Column\": \"x_CostAllocationRuleName\", \"Properties\": { \"Field\": \"x_CostAllocationRuleName\" } },\r\n { \"Column\": \"x_CostCategories\", \"Properties\": { \"Field\": \"x_CostCategories\" } },\r\n { \"Column\": \"x_CostCenter\", \"Properties\": { \"Field\": \"x_CostCenter\" } },\r\n { \"Column\": \"x_Credits\", \"Properties\": { \"Field\": \"x_Credits\" } },\r\n { \"Column\": \"x_CostType\", \"Properties\": { \"Field\": \"x_CostType\" } },\r\n { \"Column\": \"x_CurrencyConversionRate\", \"Properties\": { \"Field\": \"x_CurrencyConversionRate\" } },\r\n { \"Column\": \"x_CustomerId\", \"Properties\": { \"Field\": \"x_CustomerId\" } },\r\n { \"Column\": \"x_CustomerName\", \"Properties\": { \"Field\": \"x_CustomerName\" } },\r\n { \"Column\": \"x_Discount\", \"Properties\": { \"Field\": \"x_Discount\" } },\r\n { \"Column\": \"x_EffectiveCostInUsd\", \"Properties\": { \"Field\": \"x_EffectiveCostInUsd\" } },\r\n { \"Column\": \"x_EffectiveUnitPrice\", \"Properties\": { \"Field\": \"x_EffectiveUnitPrice\" } },\r\n { \"Column\": \"x_ExportTime\", \"Properties\": { \"Field\": \"x_ExportTime\" } },\r\n { \"Column\": \"x_InstanceID\", \"Properties\": { \"Field\": \"x_InstanceID\" } },\r\n { \"Column\": \"x_InvoiceId\", \"Properties\": { \"Field\": \"x_InvoiceId\" } },\r\n { \"Column\": \"x_InvoiceIssuerId\", \"Properties\": { \"Field\": \"x_InvoiceIssuerId\" } },\r\n { \"Column\": \"x_InvoiceSectionId\", \"Properties\": { \"Field\": \"x_InvoiceSectionId\" } },\r\n { \"Column\": \"x_InvoiceSectionName\", \"Properties\": { \"Field\": \"x_InvoiceSectionName\" } },\r\n { \"Column\": \"x_ListCostInUsd\", \"Properties\": { \"Field\": \"x_ListCostInUsd\" } },\r\n { \"Column\": \"x_Location\", \"Properties\": { \"Field\": \"x_Location\" } },\r\n { \"Column\": \"x_OnDemandCost\", \"Properties\": { \"Field\": \"x_OnDemandCost\" } },\r\n { \"Column\": \"x_OnDemandCostInUsd\", \"Properties\": { \"Field\": \"x_OnDemandCostInUsd\" } },\r\n { \"Column\": \"x_OnDemandUnitPrice\", \"Properties\": { \"Field\": \"x_OnDemandUnitPrice\" } },\r\n { \"Column\": \"x_Operation\", \"Properties\": { \"Field\": \"x_Operation\" } },\r\n { \"Column\": \"x_OwnerAccountID\", \"Properties\": { \"Field\": \"x_OwnerAccountID\" } },\r\n { \"Column\": \"x_PartnerCreditApplied\", \"Properties\": { \"Field\": \"x_PartnerCreditApplied\" } },\r\n { \"Column\": \"x_PartnerCreditRate\", \"Properties\": { \"Field\": \"x_PartnerCreditRate\" } },\r\n { \"Column\": \"x_PricingBlockSize\", \"Properties\": { \"Field\": \"x_PricingBlockSize\" } },\r\n { \"Column\": \"x_PricingCurrency\", \"Properties\": { \"Field\": \"x_PricingCurrency\" } },\r\n { \"Column\": \"x_PricingSubcategory\", \"Properties\": { \"Field\": \"x_PricingSubcategory\" } },\r\n { \"Column\": \"x_PricingUnitDescription\", \"Properties\": { \"Field\": \"x_PricingUnitDescription\" } },\r\n { \"Column\": \"x_Project\", \"Properties\": { \"Field\": \"x_Project\" } },\r\n { \"Column\": \"x_PublisherCategory\", \"Properties\": { \"Field\": \"x_PublisherCategory\" } },\r\n { \"Column\": \"x_PublisherId\", \"Properties\": { \"Field\": \"x_PublisherId\" } },\r\n { \"Column\": \"x_ResellerId\", \"Properties\": { \"Field\": \"x_ResellerId\" } },\r\n { \"Column\": \"x_ResellerName\", \"Properties\": { \"Field\": \"x_ResellerName\" } },\r\n { \"Column\": \"x_ResourceGroupName\", \"Properties\": { \"Field\": \"x_ResourceGroupName\" } },\r\n { \"Column\": \"x_ResourceType\", \"Properties\": { \"Field\": \"x_ResourceType\" } },\r\n { \"Column\": \"x_ServiceCode\", \"Properties\": { \"Field\": \"x_ServiceCode\" } },\r\n { \"Column\": \"x_ServiceId\", \"Properties\": { \"Field\": \"x_ServiceId\" } },\r\n { \"Column\": \"x_ServiceModel\", \"Properties\": { \"Field\": \"x_ServiceModel\" } },\r\n { \"Column\": \"x_ServicePeriodEnd\", \"Properties\": { \"Field\": \"x_ServicePeriodEnd\" } },\r\n { \"Column\": \"x_ServicePeriodStart\", \"Properties\": { \"Field\": \"x_ServicePeriodStart\" } },\r\n { \"Column\": \"x_SkuDescription\", \"Properties\": { \"Field\": \"x_SkuDescription\" } },\r\n { \"Column\": \"x_SkuDetails\", \"Properties\": { \"Field\": \"x_SkuDetails\" } },\r\n { \"Column\": \"x_SkuIsCreditEligible\", \"Properties\": { \"Field\": \"x_SkuIsCreditEligible\" } },\r\n { \"Column\": \"x_SkuMeterCategory\", \"Properties\": { \"Field\": \"x_SkuMeterCategory\" } },\r\n { \"Column\": \"x_SkuMeterId\", \"Properties\": { \"Field\": \"x_SkuMeterId\" } },\r\n { \"Column\": \"x_SkuMeterName\", \"Properties\": { \"Field\": \"x_SkuMeterName\" } },\r\n { \"Column\": \"x_SkuMeterSubcategory\", \"Properties\": { \"Field\": \"x_SkuMeterSubcategory\" } },\r\n { \"Column\": \"x_SkuOfferId\", \"Properties\": { \"Field\": \"x_SkuOfferId\" } },\r\n { \"Column\": \"x_SkuOrderId\", \"Properties\": { \"Field\": \"x_SkuOrderId\" } },\r\n { \"Column\": \"x_SkuOrderName\", \"Properties\": { \"Field\": \"x_SkuOrderName\" } },\r\n { \"Column\": \"x_SkuPartNumber\", \"Properties\": { \"Field\": \"x_SkuPartNumber\" } },\r\n { \"Column\": \"x_SkuPlanName\", \"Properties\": { \"Field\": \"x_SkuPlanName\" } },\r\n { \"Column\": \"x_SkuRegion\", \"Properties\": { \"Field\": \"x_SkuRegion\" } },\r\n { \"Column\": \"x_SkuServiceFamily\", \"Properties\": { \"Field\": \"x_SkuServiceFamily\" } },\r\n { \"Column\": \"x_SkuTerm\", \"Properties\": { \"Field\": \"x_SkuTerm\" } },\r\n { \"Column\": \"x_SkuTier\", \"Properties\": { \"Field\": \"x_SkuTier\" } },\r\n { \"Column\": \"x_SourceName\", \"Properties\": { \"Field\": \"x_SourceName\" } },\r\n { \"Column\": \"x_SourceProvider\", \"Properties\": { \"Field\": \"x_SourceProvider\" } },\r\n { \"Column\": \"x_SourceType\", \"Properties\": { \"Field\": \"x_SourceType\" } },\r\n { \"Column\": \"x_SourceVersion\", \"Properties\": { \"Field\": \"x_SourceVersion\" } },\r\n { \"Column\": \"x_SubproductName\", \"Properties\": { \"Field\": \"x_SubproductName\" } },\r\n { \"Column\": \"x_UsageType\", \"Properties\": { \"Field\": \"x_UsageType\" } }\r\n]\r\n```\r\n\r\n// Costs_raw retention policy (clear historical data)\r\n.alter-merge table Costs_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// Costs_raw retention policy (set the user-defined retention period)\r\n.alter-merge table Costs_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable Costs_raw streaming ingestion (required for Fabric)\r\n.alter table Costs_raw policy streamingingestion disable\r\n\r\n\r\n//===| Prices |=========================================================================================================\r\n// NOTE: Must be before cost details.\r\n//\r\n// Supported versions:\r\n// - MS EA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-ea\r\n// - MS MCA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-mca\r\n//======================================================================================================================\r\n\r\n// Prices_raw table -- Create the table if it doesn't exist\r\n.create-merge table Prices_raw ( ignore: string )\r\n\r\n// Prices_raw table -- Remove all columns to allow changing column types\r\n.alter table Prices_raw ( ignore: string )\r\n\r\n// Prices_raw table -- Redefine all columns\r\n.alter table Prices_raw (\r\n BasePrice: real, // Azure EA + MCA\r\n BillingAccountId: string, // Azure MCA\r\n BillingAccountName: string, // Azure MCA\r\n BillingCurrency: string, // Azure MCA\r\n BillingProfileId: string, // Azure MCA\r\n BillingProfileName: string, // Azure MCA\r\n Currency: string, // Azure MCA\r\n CurrencyCode: string, // Azure EA\r\n EffectiveEndDate: datetime, // Azure MCA\r\n EffectiveStartDate: datetime, // Azure EA + MCA\r\n EnrollmentNumber: string, // Azure EA\r\n IncludedQuantity: real, // Azure EA\r\n MarketPrice: real, // Azure EA + MCA\r\n MeterCategory: string, // Azure EA + MCA\r\n MeterId: string, // Azure MCA\r\n MeterID: string, // Azure EA\r\n MeterName: string, // Azure EA + MCA\r\n MeterRegion: string, // Azure EA + MCA\r\n MeterSubCategory: string, // Azure EA + MCA\r\n MeterType: string, // Azure EA + MCA\r\n OfferID: string, // Azure EA\r\n PartNumber: string, // Azure EA\r\n PriceType: string, // Azure EA + MCA\r\n Product: string, // Azure EA + MCA\r\n ProductId: string, // Azure MCA\r\n ProductID: string, // Azure EA\r\n ServiceFamily: string, // Azure EA + MCA\r\n SkuId: string, // Azure MCA\r\n SkuID: string, // Azure EA\r\n Term: string, // Azure EA + MCA\r\n TierMinimumUnits: real, // Azure MCA\r\n UnitOfMeasure: string, // Azure EA + MCA\r\n UnitPrice: real, // Azure EA + MCA\r\n x_SourceName: string, // Hubs v1_0+\r\n x_SourceProvider: string, // Hubs v1_0+\r\n x_SourceType: string, // Hubs v1_0+\r\n x_SourceVersion: string // Hubs v1_0+\r\n)\r\n\r\n// Prices_raw ingestion mapping\r\n.create-or-alter table Prices_raw ingestion parquet mapping \"Prices_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"BasePrice\", \"Properties\": { \"Field\": \"BasePrice\" } },\r\n { \"Column\": \"BillingAccountId\", \"Properties\": { \"Field\": \"BillingAccountId\" } },\r\n { \"Column\": \"BillingAccountName\", \"Properties\": { \"Field\": \"BillingAccountName\" } },\r\n { \"Column\": \"BillingCurrency\", \"Properties\": { \"Field\": \"BillingCurrency\" } },\r\n { \"Column\": \"BillingProfileId\", \"Properties\": { \"Field\": \"BillingProfileId\" } },\r\n { \"Column\": \"BillingProfileName\", \"Properties\": { \"Field\": \"BillingProfileName\" } },\r\n { \"Column\": \"Currency\", \"Properties\": { \"Field\": \"Currency\" } },\r\n { \"Column\": \"CurrencyCode\", \"Properties\": { \"Field\": \"CurrencyCode\" } },\r\n { \"Column\": \"EffectiveEndDate\", \"Properties\": { \"Field\": \"EffectiveEndDate\" } },\r\n { \"Column\": \"EffectiveStartDate\", \"Properties\": { \"Field\": \"EffectiveStartDate\" } },\r\n { \"Column\": \"EnrollmentNumber\", \"Properties\": { \"Field\": \"EnrollmentNumber\" } },\r\n { \"Column\": \"IncludedQuantity\", \"Properties\": { \"Field\": \"IncludedQuantity\" } },\r\n { \"Column\": \"MarketPrice\", \"Properties\": { \"Field\": \"MarketPrice\" } },\r\n { \"Column\": \"MeterCategory\", \"Properties\": { \"Field\": \"MeterCategory\" } },\r\n { \"Column\": \"MeterId\", \"Properties\": { \"Field\": \"MeterId\" } },\r\n { \"Column\": \"MeterID\", \"Properties\": { \"Field\": \"MeterID\" } },\r\n { \"Column\": \"MeterName\", \"Properties\": { \"Field\": \"MeterName\" } },\r\n { \"Column\": \"MeterRegion\", \"Properties\": { \"Field\": \"MeterRegion\" } },\r\n { \"Column\": \"MeterSubCategory\", \"Properties\": { \"Field\": \"MeterSubCategory\" } },\r\n { \"Column\": \"MeterType\", \"Properties\": { \"Field\": \"MeterType\" } },\r\n { \"Column\": \"OfferID\", \"Properties\": { \"Field\": \"OfferID\" } },\r\n { \"Column\": \"PartNumber\", \"Properties\": { \"Field\": \"PartNumber\" } },\r\n { \"Column\": \"PriceType\", \"Properties\": { \"Field\": \"PriceType\" } },\r\n { \"Column\": \"Product\", \"Properties\": { \"Field\": \"Product\" } },\r\n { \"Column\": \"ProductId\", \"Properties\": { \"Field\": \"ProductId\" } },\r\n { \"Column\": \"ProductID\", \"Properties\": { \"Field\": \"ProductID\" } },\r\n { \"Column\": \"ServiceFamily\", \"Properties\": { \"Field\": \"ServiceFamily\" } },\r\n { \"Column\": \"SkuId\", \"Properties\": { \"Field\": \"SkuId\" } },\r\n { \"Column\": \"SkuID\", \"Properties\": { \"Field\": \"SkuID\" } },\r\n { \"Column\": \"Term\", \"Properties\": { \"Field\": \"Term\" } },\r\n { \"Column\": \"TierMinimumUnits\", \"Properties\": { \"Field\": \"TierMinimumUnits\" } },\r\n { \"Column\": \"UnitOfMeasure\", \"Properties\": { \"Field\": \"UnitOfMeasure\" } },\r\n { \"Column\": \"UnitPrice\", \"Properties\": { \"Field\": \"UnitPrice\" } },\r\n { \"Column\": \"x_SourceName\", \"Properties\": { \"Field\": \"x_SourceName\" } },\r\n { \"Column\": \"x_SourceProvider\", \"Properties\": { \"Field\": \"x_SourceProvider\" } },\r\n { \"Column\": \"x_SourceType\", \"Properties\": { \"Field\": \"x_SourceType\" } },\r\n { \"Column\": \"x_SourceVersion\", \"Properties\": { \"Field\": \"x_SourceVersion\" } }\r\n]\r\n```\r\n\r\n// Prices_raw retention policy (clear historical data)\r\n.alter-merge table Prices_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// Prices_raw retention policy (set the user-defined retention period)\r\n.alter-merge table Prices_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable Prices_raw streaming ingestion (required for Fabric)\r\n.alter table Prices_raw policy streamingingestion disable\r\n\r\n\r\n//===| Recommendations |================================================================================================\r\n// Supported datasets/versions:\r\n// - MS CM EA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-ea\r\n// - MS CM MCA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-mca\r\n//======================================================================================================================\r\n\r\n// Recommendations_raw table -- Create the table if it doesn't exist\r\n.create-merge table Recommendations_raw ( ignore: string )\r\n\r\n// Recommendations_raw table -- Remove all columns to allow changing column types\r\n.alter table Recommendations_raw ( ignore: string )\r\n\r\n// Recommendations_raw table -- Redefine all columns\r\n.alter table Recommendations_raw (\r\n CostWithNoReservedInstances: real, // MS CM EA resv reco 2024-05-01\r\n CostWithNoReservedInstancesJson: string, // MS CM MCA resv reco 2024-05-01 -- Renamed to remove spaces and flag as JSON\r\n FirstUsageDate: datetime, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n InstanceFlexibilityGroup: string, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n InstanceFlexibilityRatio: real, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n Location: string, // MS CM EA+MCA resv reco 2024-05-01\r\n LookBackPeriod: string, // MS CM EA+MCA resv reco 2024-05-01\r\n MeterId: string, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n NetSavings: real, // MS CM EA resv reco 2024-05-01\r\n NetSavingsJson: string, // MS CM MCA resv reco 2024-05-01 -- Renamed to remove spaces and flag as JSON\r\n NormalizedSize: string, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n ProviderName: string, // Hubs v1_2\r\n RecommendedQuantity: real, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n RecommendedQuantityNormalized: real, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n ResourceId: string, // Hubs v1_2\r\n ResourceName: string, // Hubs v1_2\r\n ResourceType: string, // Hubs v1_2, MS CM EA+MCA resv reco 2024-05-01\r\n Scope: string, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n SKU: string, // MS CM EA resv reco 2024-05-01\r\n SkuName: string, // MS CM MCA resv reco 2024-05-01 -- Renamed to remove spaces\r\n SkuProperties: string, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n SubAccountId: string, // Hubs v1_2\r\n SubAccountName: string, // Hubs v1_2\r\n SubscriptionId: string, // MS CM EA+MCA resv reco 2024-05-01\r\n Term: string, // MS CM EA+MCA resv reco 2024-05-01\r\n TotalCostWithReservedInstances: real, // MS CM EA resv reco 2024-05-01\r\n TotalCostWithReservedInstancesJson: string, // MS CM MCA resv reco 2024-05-01 -- Renamed to remove spaces and flag as JSON\r\n x_EffectiveCostAfter: real, // Hubs v1_2\r\n x_EffectiveCostBefore: real, // Hubs v1_2\r\n x_EffectiveCostSavings: real, // Hubs v1_2\r\n x_RecommendationCategory: string, // Hubs v1_2\r\n x_RecommendationDate: datetime, // Hubs v1_2\r\n x_RecommendationDescription: string, // Hubs v1_2\r\n x_RecommendationDetails: dynamic, // Hubs v1_2\r\n x_RecommendationId: string, // Hubs v1_2\r\n x_ResourceGroupName: string, // Hubs v1_2\r\n x_SourceName: string, // Hubs v1_0+\r\n x_SourceProvider: string, // Hubs v1_0+\r\n x_SourceType: string, // Hubs v1_0+\r\n x_SourceVersion: string // Hubs v1_0+\r\n)\r\n\r\n// Recommendations_raw ingestion mapping\r\n.create-or-alter table Recommendations_raw ingestion parquet mapping \"Recommendations_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"CostWithNoReservedInstances\", \"Properties\": { \"Field\": \"CostWithNoReservedInstances\" } },\r\n { \"Column\": \"CostWithNoReservedInstancesJson\", \"Properties\": { \"Field\": \"CostWithNoReservedInstancesJson\" } },\r\n { \"Column\": \"FirstUsageDate\", \"Properties\": { \"Field\": \"FirstUsageDate\" } },\r\n { \"Column\": \"InstanceFlexibilityGroup\", \"Properties\": { \"Field\": \"InstanceFlexibilityGroup\" } },\r\n { \"Column\": \"InstanceFlexibilityRatio\", \"Properties\": { \"Field\": \"InstanceFlexibilityRatio\" } },\r\n { \"Column\": \"Location\", \"Properties\": { \"Field\": \"Location\" } },\r\n { \"Column\": \"LookBackPeriod\", \"Properties\": { \"Field\": \"LookBackPeriod\" } },\r\n { \"Column\": \"MeterId\", \"Properties\": { \"Field\": \"MeterId\" } },\r\n { \"Column\": \"NetSavings\", \"Properties\": { \"Field\": \"NetSavings\" } },\r\n { \"Column\": \"NetSavingsJson\", \"Properties\": { \"Field\": \"NetSavingsJson\" } },\r\n { \"Column\": \"NormalizedSize\", \"Properties\": { \"Field\": \"NormalizedSize\" } },\r\n { \"Column\": \"ProviderName\", \"Properties\": { \"Field\": \"ProviderName\" } },\r\n { \"Column\": \"RecommendedQuantity\", \"Properties\": { \"Field\": \"RecommendedQuantity\" } },\r\n { \"Column\": \"RecommendedQuantityNormalized\", \"Properties\": { \"Field\": \"RecommendedQuantityNormalized\" } },\r\n { \"Column\": \"ResourceId\", \"Properties\": { \"Field\": \"ResourceId\" } },\r\n { \"Column\": \"ResourceName\", \"Properties\": { \"Field\": \"ResourceName\" } },\r\n { \"Column\": \"ResourceType\", \"Properties\": { \"Field\": \"ResourceType\" } },\r\n { \"Column\": \"Scope\", \"Properties\": { \"Field\": \"Scope\" } },\r\n { \"Column\": \"SKU\", \"Properties\": { \"Field\": \"SKU\" } },\r\n { \"Column\": \"SkuName\", \"Properties\": { \"Field\": \"SkuName\" } },\r\n { \"Column\": \"SkuProperties\", \"Properties\": { \"Field\": \"SkuProperties\" } },\r\n { \"Column\": \"SubAccountId\", \"Properties\": { \"Field\": \"SubAccountId\" } },\r\n { \"Column\": \"SubAccountName\", \"Properties\": { \"Field\": \"SubAccountName\" } },\r\n { \"Column\": \"SubscriptionId\", \"Properties\": { \"Field\": \"SubscriptionId\" } },\r\n { \"Column\": \"Term\", \"Properties\": { \"Field\": \"Term\" } },\r\n { \"Column\": \"TotalCostWithReservedInstances\", \"Properties\": { \"Field\": \"TotalCostWithReservedInstances\" } },\r\n { \"Column\": \"TotalCostWithReservedInstancesJson\", \"Properties\": { \"Field\": \"TotalCostWithReservedInstancesJson\" } },\r\n { \"Column\": \"x_EffectiveCostAfter\", \"Properties\": { \"Field\": \"x_EffectiveCostAfter\" } },\r\n { \"Column\": \"x_EffectiveCostBefore\", \"Properties\": { \"Field\": \"x_EffectiveCostBefore\" } },\r\n { \"Column\": \"x_EffectiveCostSavings\", \"Properties\": { \"Field\": \"x_EffectiveCostSavings\" } },\r\n { \"Column\": \"x_RecommendationCategory\", \"Properties\": { \"Field\": \"x_RecommendationCategory\" } },\r\n { \"Column\": \"x_RecommendationDate\", \"Properties\": { \"Field\": \"x_RecommendationDate\" } },\r\n { \"Column\": \"x_RecommendationDescription\", \"Properties\": { \"Field\": \"x_RecommendationDescription\" } },\r\n { \"Column\": \"x_RecommendationDetails\", \"Properties\": { \"Field\": \"x_RecommendationDetails\" } },\r\n { \"Column\": \"x_RecommendationId\", \"Properties\": { \"Field\": \"x_RecommendationId\" } },\r\n { \"Column\": \"x_ResourceGroupName\", \"Properties\": { \"Field\": \"x_ResourceGroupName\" } },\r\n { \"Column\": \"x_SourceName\", \"Properties\": { \"Field\": \"x_SourceName\" } },\r\n { \"Column\": \"x_SourceProvider\", \"Properties\": { \"Field\": \"x_SourceProvider\" } },\r\n { \"Column\": \"x_SourceType\", \"Properties\": { \"Field\": \"x_SourceType\" } },\r\n { \"Column\": \"x_SourceVersion\", \"Properties\": { \"Field\": \"x_SourceVersion\" } }\r\n]\r\n```\r\n\r\n// Recommendations_raw retention policy (clear historical data)\r\n.alter-merge table Recommendations_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// Recommendations_raw retention policy (set the user-defined retention period)\r\n.alter-merge table Recommendations_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable Recommendations_raw streaming ingestion (required for Fabric)\r\n.alter table Recommendations_raw policy streamingingestion disable\r\n\r\n\r\n//===| Transactions |===================================================================================================\r\n// Supported versions:\r\n// - MS CM EA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-ea\r\n// - MS CM MCA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-mca\r\n//======================================================================================================================\r\n\r\n// Transactions_raw table -- Create the table if it doesn't exist\r\n.create-merge table Transactions_raw ( ignore: string )\r\n\r\n// Transactions_raw table -- Remove all columns to allow changing column types\r\n.alter table Transactions_raw ( ignore: string )\r\n\r\n// Transactions_raw table -- Redefine all columns\r\n.alter table Transactions_raw (\r\n AccountName: string, // MS CM EA resv trans 2023-05-01\r\n AccountOwnerEmail: string, // MS CM EA resv trans 2023-05-01\r\n Amount: real, // MS CM EA+MCA resv trans 2023-05-01\r\n ArmSkuName: string, // MS CM EA+MCA resv trans 2023-05-01\r\n BillingFrequency: string, // MS CM EA+MCA resv trans 2023-05-01\r\n BillingMonth: string, // MS CM EA resv trans 2023-05-01\r\n BillingProfileId: string, // MS CM MCA resv trans 2023-05-01\r\n BillingProfileName: string, // MS CM MCA resv trans 2023-05-01\r\n CostCenter: string, // MS CM EA resv trans 2023-05-01\r\n Currency: string, // MS CM EA+MCA resv trans 2023-05-01\r\n CurrentEnrollmentId: string, // MS CM EA resv trans 2023-05-01\r\n DepartmentName: string, // MS CM EA resv trans 2023-05-01\r\n Description: string, // MS CM EA+MCA resv trans 2023-05-01\r\n EventDate: datetime, // MS CM EA+MCA resv trans 2023-05-01\r\n EventType: string, // MS CM EA+MCA resv trans 2023-05-01\r\n Invoice: string, // MS CM EA+MCA resv trans 2023-05-01\r\n InvoiceId: string, // MS CM EA+MCA resv trans 2023-05-01\r\n InvoiceSectionId: string, // MS CM MCA resv trans 2023-05-01\r\n InvoiceSectionName: string, // MS CM MCA resv trans 2023-05-01\r\n MonetaryCommitment: real, // MS CM EA resv trans 2023-05-01\r\n Overage: real, // MS CM EA resv trans 2023-05-01\r\n PurchasingEnrollment: string, // MS CM EA resv trans 2023-05-01\r\n PurchasingSubscriptionGuid: string, // MS CM EA+MCA resv trans 2023-05-01\r\n PurchasingSubscriptionName: string, // MS CM EA+MCA resv trans 2023-05-01\r\n Quantity: real, // MS CM EA+MCA resv trans 2023-05-01\r\n Region: string, // MS CM EA+MCA resv trans 2023-05-01\r\n ReservationOrderId: string, // MS CM EA+MCA resv trans 2023-05-01\r\n ReservationOrderName: string, // MS CM EA+MCA resv trans 2023-05-01\r\n Term: string, // MS CM EA+MCA resv trans 2023-05-01\r\n x_SourceName: string, // Hubs v1_0+\r\n x_SourceProvider: string, // Hubs v1_0+\r\n x_SourceType: string, // Hubs v1_0+\r\n x_SourceVersion: string // Hubs v1_0+\r\n)\r\n\r\n// Transactions_raw ingestion mapping\r\n.create-or-alter table Transactions_raw ingestion parquet mapping \"Transactions_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"AccountName\", \"Properties\": { \"Field\": \"AccountName\" } },\r\n { \"Column\": \"AccountOwnerEmail\", \"Properties\": { \"Field\": \"AccountOwnerEmail\" } },\r\n { \"Column\": \"Amount\", \"Properties\": { \"Field\": \"Amount\" } },\r\n { \"Column\": \"ArmSkuName\", \"Properties\": { \"Field\": \"ArmSkuName\" } },\r\n { \"Column\": \"BillingFrequency\", \"Properties\": { \"Field\": \"BillingFrequency\" } },\r\n { \"Column\": \"BillingMonth\", \"Properties\": { \"Field\": \"BillingMonth\" } },\r\n { \"Column\": \"BillingProfileId\", \"Properties\": { \"Field\": \"BillingProfileId\" } },\r\n { \"Column\": \"BillingProfileName\", \"Properties\": { \"Field\": \"BillingProfileName\" } },\r\n { \"Column\": \"CostCenter\", \"Properties\": { \"Field\": \"CostCenter\" } },\r\n { \"Column\": \"Currency\", \"Properties\": { \"Field\": \"Currency\" } },\r\n { \"Column\": \"CurrentEnrollmentId\", \"Properties\": { \"Field\": \"CurrentEnrollmentId\" } },\r\n { \"Column\": \"DepartmentName\", \"Properties\": { \"Field\": \"DepartmentName\" } },\r\n { \"Column\": \"Description\", \"Properties\": { \"Field\": \"Description\" } },\r\n { \"Column\": \"EventDate\", \"Properties\": { \"Field\": \"EventDate\" } },\r\n { \"Column\": \"EventType\", \"Properties\": { \"Field\": \"EventType\" } },\r\n { \"Column\": \"Invoice\", \"Properties\": { \"Field\": \"Invoice\" } },\r\n { \"Column\": \"InvoiceId\", \"Properties\": { \"Field\": \"InvoiceId\" } },\r\n { \"Column\": \"InvoiceSectionId\", \"Properties\": { \"Field\": \"InvoiceSectionId\" } },\r\n { \"Column\": \"InvoiceSectionName\", \"Properties\": { \"Field\": \"InvoiceSectionName\" } },\r\n { \"Column\": \"MonetaryCommitment\", \"Properties\": { \"Field\": \"MonetaryCommitment\" } },\r\n { \"Column\": \"Overage\", \"Properties\": { \"Field\": \"Overage\" } },\r\n { \"Column\": \"PurchasingEnrollment\", \"Properties\": { \"Field\": \"PurchasingEnrollment\" } },\r\n { \"Column\": \"PurchasingSubscriptionGuid\", \"Properties\": { \"Field\": \"PurchasingSubscriptionGuid\" } },\r\n { \"Column\": \"PurchasingSubscriptionName\", \"Properties\": { \"Field\": \"PurchasingSubscriptionName\" } },\r\n { \"Column\": \"Quantity\", \"Properties\": { \"Field\": \"Quantity\" } },\r\n { \"Column\": \"Region\", \"Properties\": { \"Field\": \"Region\" } },\r\n { \"Column\": \"ReservationOrderId\", \"Properties\": { \"Field\": \"ReservationOrderId\" } },\r\n { \"Column\": \"ReservationOrderName\", \"Properties\": { \"Field\": \"ReservationOrderName\" } },\r\n { \"Column\": \"Term\", \"Properties\": { \"Field\": \"Term\" } },\r\n { \"Column\": \"x_SourceName\", \"Properties\": { \"Field\": \"x_SourceName\" } },\r\n { \"Column\": \"x_SourceProvider\", \"Properties\": { \"Field\": \"x_SourceProvider\" } },\r\n { \"Column\": \"x_SourceType\", \"Properties\": { \"Field\": \"x_SourceType\" } },\r\n { \"Column\": \"x_SourceVersion\", \"Properties\": { \"Field\": \"x_SourceVersion\" } }\r\n]\r\n```\r\n\r\n// Transactions_raw retention policy (clear historical data)\r\n.alter-merge table Transactions_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// Transactions_raw retention policy (set the user-defined retention period)\r\n.alter-merge table Transactions_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable Transactions_raw streaming ingestion (required for Fabric)\r\n.alter table Transactions_raw policy streamingingestion disable\r\n\r\n", - "$fxv#9": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Ingestion database\r\n// Used for data ingestion, normalization, and cleansing.\r\n// See known issues @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n//===| Prices |=========================================================================================================\r\n// Supported versions:\r\n// - MS EA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-ea\r\n// - MS MCA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-mca\r\n//======================================================================================================================\r\n\r\n// Prices_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: All prices transformed to FOCUS 1.0. Use Prices_transform_v1_2() instead.', folder='Prices')\r\nPrices_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', toint(''),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n let prices = materialize(\r\n Prices_raw\r\n //\r\n // Change real to decimal\r\n | extend\r\n BasePrice = todecimal(BasePrice),\r\n IncludedQuantity = todecimal(IncludedQuantity),\r\n MarketPrice = todecimal(MarketPrice),\r\n TierMinimumUnits = todecimal(TierMinimumUnits),\r\n UnitPrice = todecimal(UnitPrice)\r\n //\r\n | extend x_SkuId = coalesce(SkuId, SkuID)\r\n | extend x_SkuMeterId = coalesce(MeterId, MeterID)\r\n | extend x_SkuProductId = coalesce(ProductId, ProductID)\r\n | extend x_SkuTerm = isoMonths(Term)\r\n | project-rename\r\n x_BaseUnitPrice = BasePrice,\r\n x_EffectivePeriodEnd = EffectiveEndDate,\r\n x_EffectivePeriodStart = EffectiveStartDate,\r\n x_PricingUnitDescription = UnitOfMeasure,\r\n x_SkuIncludedQuantity = IncludedQuantity,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterName = MeterName,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuMeterType = MeterType,\r\n x_SkuOfferId = OfferID,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPriceType = PriceType,\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTier = TierMinimumUnits\r\n | extend ContractedUnitPrice = iff(x_SkuPriceType != 'SavingsPlan', UnitPrice, todecimal('')) // UnitPrice for savings plan is not the on-demand unit price\r\n | extend ListUnitPrice = iff(x_SkuPriceType != 'SavingsPlan', MarketPrice, todecimal('')) // MarketPrice for savings plan is not the list price\r\n | extend ChargeCategory = case(\r\n x_SkuPriceType == 'Consumption', 'Usage',\r\n x_SkuPriceType == 'ReservedInstance', 'Purchase',\r\n x_SkuPriceType == 'SavingsPlan', 'Usage', // Savings plan prices are for committed usage, not the purchase\r\n ''\r\n )\r\n | extend SkuPriceIdv2 = strcat(case(x_SkuPriceType == 'Consumption', 'OD', x_SkuPriceType == 'ReservedInstance', 'RI', x_SkuPriceType == 'SavingsPlan', 'SP', 'XX'), substring(ChargeCategory, 0, 1), x_SkuTerm, '_', x_SkuProductId, '_', x_SkuId, '_', x_SkuMeterType, '_', x_SkuTier, x_SkuOfferId)\r\n | extend x_BillingAccountId = iff(BillingAccountId startswith '/', split(BillingAccountId, '/')[-1], coalesce(BillingAccountId, EnrollmentNumber))\r\n | extend x_BillingProfileId = iff(BillingProfileId startswith '/', split(BillingProfileId, '/')[-1], coalesce(BillingProfileId, EnrollmentNumber))\r\n | extend tmp_SavingsPlanKey = strcat(x_SkuMeterId, x_SkuProductId, x_SkuId, x_SkuTier, x_SkuOfferId)\r\n //\r\n // Get latest ingested row based on the unique ID\r\n | extend x_IngestionTime = ingestion_time()\r\n );\r\n //\r\n // Meters for reservations and savings plans to identify commitment eligibility\r\n let riMeters = prices | where x_SkuPriceType == 'ReservedInstance' | distinct x_SkuMeterId;\r\n let spMeters = prices | where x_SkuPriceType == 'SavingsPlan' | distinct x_SkuMeterId;\r\n //\r\n // Copy list/base/contracted prices from on-demand SKUs\r\n prices\r\n | where x_SkuPriceType == 'SavingsPlan'\r\n // If we use join, specify the shuffle key\r\n // TODO: Compare join vs. lookup perf -- | join kind=leftouter hint.strategy=shuffle (prices | where x_SkuPriceType == 'Consumption' | where x_SkuMeterId in (spMeters) | distinct tmp_SavingsPlanKey, ListUnitPrice, ContractedUnitPrice, x_BaseUnitPrice) on tmp_SavingsPlanKey\r\n | lookup kind=leftouter (prices | where x_SkuPriceType == 'Consumption' | where x_SkuMeterId in (spMeters) | distinct tmp_SavingsPlanKey, ListUnitPrice, ContractedUnitPrice, x_BaseUnitPrice) on tmp_SavingsPlanKey\r\n | extend ListUnitPrice = coalesce(ListUnitPrice, ListUnitPrice1)\r\n | extend ContractedUnitPrice = coalesce(ContractedUnitPrice, ContractedUnitPrice1)\r\n | extend x_BaseUnitPrice = coalesce(x_BaseUnitPrice, x_BaseUnitPrice1)\r\n | project-away ListUnitPrice1, ContractedUnitPrice1, x_BaseUnitPrice1, tmp_SavingsPlanKey\r\n | union ((prices | where x_SkuPriceType != 'SavingsPlan'))\r\n //\r\n // Calculate commitment discount elgibility\r\n // TODO: Would a join be faster?\r\n | extend x_CommitmentDiscountSpendEligibility = iff(x_SkuMeterId in (riMeters) and x_SkuPriceType != 'ReservedInstance', 'Eligible', 'Not Eligible')\r\n | extend x_CommitmentDiscountUsageEligibility = iff(x_SkuMeterId in (spMeters), 'Eligible', 'Not Eligible')\r\n //\r\n // Add PricingUnit and x_PricingBlockSize\r\n // TODO: Compare join vs. lookup perf -- | join kind=leftouter (PricingUnits) on x_PricingUnitDescription | project-away x_PricingUnitDescription1\r\n | lookup kind=leftouter (PricingUnits | extend x_PricingBlockSize = todecimal(x_PricingBlockSize)) on x_PricingUnitDescription\r\n //\r\n | extend x_EffectiveUnitPrice = iff(x_SkuPriceType == 'SavingsPlan', UnitPrice, todecimal('')) // Savings plan prices are for the effective price, not the contracted price\r\n | extend x_EffectiveUnitPriceDiscount = ContractedUnitPrice - x_EffectiveUnitPrice\r\n | extend x_ContractedUnitPriceDiscount = ListUnitPrice - ContractedUnitPrice\r\n | extend x_TotalUnitPriceDiscount = ListUnitPrice - x_EffectiveUnitPrice\r\n | project\r\n BillingAccountId = tolower(case(\r\n BillingProfileId startswith '/', BillingProfileId,\r\n BillingAccountId startswith '/', BillingAccountId,\r\n strcat('/providers/microsoft.billing/billingaccounts/', x_BillingAccountId, iff(x_BillingProfileId == x_BillingAccountId, '', strcat('/billingprofiles/', x_BillingProfileId)))\r\n )),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName, x_BillingProfileId),\r\n BillingCurrency = coalesce(BillingCurrency, CurrencyCode, Currency), // Currency last as a fallback only\r\n ChargeCategory,\r\n CommitmentDiscountCategory = case(\r\n x_SkuPriceType == 'ReservedInstance', 'Usage',\r\n x_SkuPriceType == 'SavingsPlan', 'Spend',\r\n ''\r\n ),\r\n CommitmentDiscountType = case(\r\n x_SkuPriceType == 'ReservedInstance', 'Reservation',\r\n x_SkuPriceType == 'SavingsPlan', 'Savings plan',\r\n ''\r\n ),\r\n ContractedUnitPrice,\r\n ListUnitPrice,\r\n PricingCategory = case(\r\n x_SkuPriceType == 'Consumption', 'Standard',\r\n x_SkuPriceType == 'ReservedInstance', 'Standard', // Reservation purchases are tracked as \"Standard\"\r\n x_SkuPriceType == 'SavingsPlan', 'Committed',\r\n ''\r\n ),\r\n PricingUnit,\r\n SkuId = coalesce(ProductId, ProductID),\r\n SkuPriceId = strcat(x_SkuProductId, '_', x_SkuId, '_', x_SkuMeterType),\r\n SkuPriceIdv2,\r\n x_BaseUnitPrice,\r\n x_BillingAccountAgreement = case(\r\n strlen(x_BillingAccountId) > 32, 'MCA',\r\n strlen(x_BillingAccountId) < 32, 'EA',\r\n 'Unknown'\r\n ),\r\n x_BillingAccountId,\r\n x_BillingProfileId,\r\n x_CommitmentDiscountSpendEligibility,\r\n x_CommitmentDiscountUsageEligibility,\r\n x_ContractedUnitPriceDiscount,\r\n x_ContractedUnitPriceDiscountPercent = 1.0 * x_ContractedUnitPriceDiscount / ListUnitPrice * 100,\r\n x_EffectivePeriodEnd = startofmonth(x_EffectivePeriodEnd + 1h),\r\n x_EffectivePeriodStart,\r\n x_EffectiveUnitPrice,\r\n x_EffectiveUnitPriceDiscount,\r\n x_EffectiveUnitPriceDiscountPercent = 1.0 * x_EffectiveUnitPriceDiscount / ContractedUnitPrice * 100,\r\n x_IngestionTime,\r\n x_PricingBlockSize,\r\n x_PricingCurrency = coalesce(Currency, CurrencyCode), // CurrencyCode last as a fallback only\r\n x_PricingSubcategory = case(\r\n x_SkuPriceType == 'Consumption' and (x_SkuIncludedQuantity > 0 or x_SkuTier > 0), 'Tiered',\r\n x_SkuPriceType == 'Consumption', 'Standard',\r\n x_SkuPriceType == 'ReservedInstance', 'Standard', // Reservation purchases are tracked as \"Standard\"\r\n x_SkuPriceType == 'SavingsPlan', 'Committed Spend',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_SkuDescription = Product,\r\n x_SkuId,\r\n x_SkuIncludedQuantity,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterName,\r\n x_SkuMeterSubcategory,\r\n x_SkuMeterType,\r\n x_SkuPriceType,\r\n x_SkuProductId,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuOfferId,\r\n x_SkuPartNumber,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceName = coalesce(x_SourceName, 'Cost Management'),\r\n x_SourceProvider = coalesce(x_SourceProvider, 'Microsoft'),\r\n x_SourceType = coalesce(x_SourceType, 'PriceSheet'),\r\n x_SourceVersion = coalesce(x_SourceVersion, '2023-05-01'),\r\n x_TotalUnitPriceDiscount,\r\n x_TotalUnitPriceDiscountPercent = 1.0 * x_TotalUnitPriceDiscount / ListUnitPrice * 100\r\n}\r\n\r\n// Prices_final_v1_0 table\r\n.create-merge table Prices_final_v1_0 (\r\n BillingAccountId: string,\r\n BillingAccountName: string,\r\n BillingCurrency: string,\r\n ChargeCategory: string,\r\n CommitmentDiscountCategory: string,\r\n CommitmentDiscountType: string,\r\n ContractedUnitPrice: decimal,\r\n ListUnitPrice: decimal,\r\n PricingCategory: string,\r\n PricingUnit: string,\r\n SkuId: string,\r\n SkuPriceId: string,\r\n SkuPriceIdv2: string, // Hubs add-on\r\n x_BaseUnitPrice: decimal, // Azure\r\n x_BillingAccountAgreement: string, // Hubs add-on\r\n x_BillingAccountId: string, // Azure MCA\r\n x_BillingProfileId: string, // Azure MCA\r\n x_CommitmentDiscountSpendEligibility: string, // Hubs add-on\r\n x_CommitmentDiscountUsageEligibility: string, // Hubs add-on\r\n x_ContractedUnitPriceDiscount: decimal, // Hubs add-on\r\n x_ContractedUnitPriceDiscountPercent: decimal, // Hubs add-on\r\n x_EffectivePeriodEnd: datetime, // Azure\r\n x_EffectivePeriodStart: datetime, // Azure\r\n x_EffectiveUnitPrice: decimal, // Azure\r\n x_EffectiveUnitPriceDiscount: decimal, // Hubs add-on\r\n x_EffectiveUnitPriceDiscountPercent: decimal, // Hubs add-on\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_PricingBlockSize: decimal, // Hubs add-on\r\n x_PricingCurrency: string, // Azure\r\n x_PricingSubcategory: string, // Hubs add-on\r\n x_PricingUnitDescription: string, // Azure\r\n x_SkuDescription: string, // Azure\r\n x_SkuId: string, // Azure\r\n x_SkuIncludedQuantity: decimal, // Azure EA\r\n x_SkuMeterCategory: string, // Azure\r\n x_SkuMeterId: string, // Azure\r\n x_SkuMeterName: string, // Azure\r\n x_SkuMeterSubcategory: string, // Azure\r\n x_SkuMeterType: string, // Azure\r\n x_SkuPriceType: string, // Azure\r\n x_SkuProductId: string, // Azure\r\n x_SkuRegion: string, // Azure\r\n x_SkuServiceFamily: string, // Azure\r\n x_SkuOfferId: string, // Azure EA\r\n x_SkuPartNumber: string, // Azure EA\r\n x_SkuTerm: int, // Azure\r\n x_SkuTier: decimal, // Azure MCA\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_TotalUnitPriceDiscount: decimal, // Hubs add-on\r\n x_TotalUnitPriceDiscountPercent: decimal // Hubs add-on\r\n)\r\n\r\n// Update policy for Prices_raw -> Prices_final_v1_0\r\n.alter table Prices_final_v1_0 policy update\r\n```\r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"Prices_raw\",\r\n \"Query\": \"Prices_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Cost and usage |=================================================================================================\r\n// Supported versions:\r\n// - MS: 1.0, 1.0-preview(v1) -- See https://aka.ms/costmgmt/exports/focus\r\n// - AWS: 1.0 -- See https://docs.aws.amazon.com/cur/latest/userguide/table-dictionary-focus-1-0-aws-columns.html\r\n// - GCP: Jan-Jun 2024 -- See https://cloud.google.com/resources/google-cloud-focus?e=48754805&hl=en\r\n// Links to (Aug 2024): https://services.google.com/fh/files/misc/focus_guide_v1.pdf\r\n// See also:\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/standard-usage\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/detailed-usage\r\n// - OCI: 1.0 -- See https://docs.oracle.com/iaas/Content/Billing/Concepts/costusagereportsoverview.htm#costreports__focus-cost-report-schema\r\n//\r\n// Support for non-Azure data is limited to ingestion only. Data is not transformed across versions.\r\n//======================================================================================================================\r\n\r\n// Costs_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: All costs transformed to FOCUS 1.0. Use Costs_transform_v1_2() instead.', folder='Costs')\r\nCosts_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n Costs_raw\r\n //\r\n // Change real to decimal\r\n | extend\r\n BilledCost = todecimal(BilledCost),\r\n CommitmentDiscountQuantity = todecimal(CommitmentDiscountQuantity),\r\n ConsumedQuantity = todecimal(ConsumedQuantity),\r\n ContractedCost = todecimal(ContractedCost),\r\n ContractedUnitPrice = todecimal(ContractedUnitPrice),\r\n EffectiveCost = todecimal(EffectiveCost),\r\n ListCost = todecimal(ListCost),\r\n ListUnitPrice = todecimal(ListUnitPrice),\r\n PricingQuantity = todecimal(PricingQuantity),\r\n UsageAmount = todecimal(UsageAmount),\r\n UsageQuantity = todecimal(UsageQuantity),\r\n x_BilledCostInUsd = todecimal(x_BilledCostInUsd),\r\n x_BilledUnitPrice = todecimal(x_BilledUnitPrice),\r\n x_BillingExchangeRate = todecimal(x_BillingExchangeRate),\r\n x_ContractedCostInUsd = todecimal(x_ContractedCostInUsd),\r\n x_Cost = todecimal(x_Cost),\r\n x_CurrencyConversionRate = todecimal(x_CurrencyConversionRate),\r\n x_EffectiveCostInUsd = todecimal(x_EffectiveCostInUsd),\r\n x_EffectiveUnitPrice = todecimal(x_EffectiveUnitPrice),\r\n x_ListCostInUsd = todecimal(x_ListCostInUsd),\r\n x_OnDemandCost = todecimal(x_OnDemandCost),\r\n x_OnDemandCostInUsd = todecimal(x_OnDemandCostInUsd),\r\n x_OnDemandUnitPrice = todecimal(x_OnDemandUnitPrice),\r\n x_PricingBlockSize = todecimal(x_PricingBlockSize)\r\n //\r\n // Dedupe rows\r\n | extend x_IngestionTime = ingestion_time()\r\n | extend x_ChargeId = ''\r\n // TODO: Consider adding a unique charge ID per row\r\n // hash_sha256(strcat(\r\n // // DO NOT CHANGE COLUMNS OR COLUMN ORDER\r\n // // 1. Resource hierarchy (including resource name), highest to lowest\r\n // BillingAccountId,\r\n // x_InvoiceSectionId,\r\n // x_AccountOwnerId,\r\n // SubAccountId,\r\n // x_ResourceGroupName,\r\n // ResourceName,\r\n // // 2. Resource details\r\n // ResourceId,\r\n // RegionId,\r\n // Tags,\r\n // CommitmentDiscountId,\r\n // x_CostCenter,\r\n // // 4. Meter details\r\n // SkuPriceId,\r\n // x_SkuMeterId,\r\n // x_SkuPartNumber,\r\n // x_SkuOfferId,\r\n // x_SkuDetails,\r\n // // 5. Date\r\n // ChargePeriodStart\r\n // ))\r\n //\r\n // Identify data quality issues\r\n | extend x_SourceChanges = trim_end(',', strcat(\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and ChargeFrequency == 'Usage-Based', 'InvalidChargeFrequency,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and EffectiveCost > 0, 'InvalidEffectiveCost,', ''),\r\n iff((isempty(ContractedCost) or ContractedCost == 0) and EffectiveCost != 0, 'MissingContractedCost,', ''),\r\n iff((isempty(ContractedUnitPrice) or ContractedUnitPrice == 0) and x_EffectiveUnitPrice != 0, 'MissingContractedUnitPrice,', ''),\r\n iff(ListCost < ContractedCost, 'ListCostLessThanContractedCost,', ''),\r\n iff(ContractedCost < EffectiveCost, 'ContractedCostLessThanEffectiveCost,', ''),\r\n iff((isempty(ListCost) or ListCost == 0) and (ContractedCost != 0 or EffectiveCost != 0), 'MissingListCost,', ''),\r\n iff((isempty(ListUnitPrice) or ListUnitPrice == 0) and (ContractedUnitPrice != 0 or x_EffectiveUnitPrice != 0), 'MissingListUnitPrice,', ''),\r\n iff((isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(x_BilledUnitPrice - x_EffectiveUnitPrice) < 0.0001)\r\n or (isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(ContractedUnitPrice - x_EffectiveUnitPrice) < 0.0001),\r\n 'XEffectiveUnitPriceRoundingError,', ''),\r\n iff(ConsumedQuantity == 0 and (EffectiveCost != 0 or BilledCost != 0), 'MissingConsumedQuantity,', ''),\r\n iff(PricingQuantity == 0 and (EffectiveCost != 0 or BilledCost != 0), 'MissingPricingQuantity,', ''),\r\n iff(isempty(ProviderName), 'MissingProviderName,', ''),\r\n iff(isempty(PublisherName), 'MissingPublisherName,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceId), 'MissingResourceId,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceName), 'MissingResourceName,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceType), 'MissingResourceType,', ''),\r\n iff(BilledCost > 0 and x_BilledUnitPrice == 0, 'MissingXBilledUnitPrice,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(x_ResourceType), 'MissingXResourceType,', ''),\r\n iff(PricingCategory == 'Standard' and isnotempty(CommitmentDiscountId) and ChargeCategory == 'Usage', 'PricingCategoryShouldBeCommitted,', ''),\r\n iff(x_SkuTerm == '1Year' or x_SkuTerm == '3Years' or x_SkuTerm == '5Years', 'SkuTermShouldBeAnInteger,', '')\r\n ))\r\n //\r\n // Fix columns needed in other changes\r\n | extend ProviderName = case(\r\n isnotempty(ProviderName), ProviderName,\r\n isnotempty(coalesce(x_CostCategories, x_Discount, x_Operation, x_ServiceCode, x_UsageType)), 'AWS',\r\n isnotempty(coalesce(tostring(UsageAmount), tostring(x_Cost), x_Credits, x_CostType, tostring(x_CurrencyConversionRate), tostring(x_ExportTime), x_Project, x_ServiceId)), 'GCP',\r\n isnotempty(coalesce(x_BillingProfileId, x_InvoiceSectionId)), 'Microsoft',\r\n ''\r\n )\r\n //\r\n // Identify source\r\n | extend x_SourceName = coalesce(x_SourceName, iff(isnotempty(x_BillingProfileId), 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(isnotempty(x_BillingProfileId), 'FocusCost', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, case(\r\n isnotempty(coalesce(InvoiceId, SkuMeter, PricingCurrency)) and isempty(SkuPriceDetails) and isnotempty(x_SkuDetails), '1.2-preview',\r\n isnotempty(coalesce(InvoiceId, SkuMeter, PricingCurrency)), '1.2',\r\n isnotempty(coalesce(ChargeClass, CommitmentDiscountStatus, tostring(ConsumedQuantity), ConsumedUnit, tostring(ContractedCost), tostring(ContractedUnitPrice), RegionId, RegionName)), '1.0',\r\n isnotempty(coalesce(ChargeSubcategory, Region, tostring(UsageQuantity), UsageUnit)), iff(ProviderName == 'Microsoft', '1.0-preview(v1)', '1.0-preview'),\r\n ''\r\n ))\r\n // Append version check error code\r\n | extend x_SourceChanges = iff(x_SourceVersion == '1.0', x_SourceChanges,\r\n strcat(x_SourceChanges, iff(isempty(x_SourceChanges), '', ','), iff(x_SourceVersion == '', 'UnknownFocusVersion', 'LegacyFocusVersion'))\r\n )\r\n //\r\n // Fix quantities\r\n | extend PricingQuantity = case(\r\n PricingQuantity != 0 or (EffectiveCost == 0 and BilledCost == 0), PricingQuantity,\r\n PricingQuantity == 0 and isnotempty(BilledCost) and BilledCost != 0 and isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, BilledCost / x_BilledUnitPrice,\r\n PricingQuantity == 0 and isnotempty(BilledCost) and BilledCost != 0 and isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, BilledCost / x_BilledUnitPrice,\r\n PricingQuantity == 0 and isnotempty(EffectiveCost) and EffectiveCost != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0, EffectiveCost / x_EffectiveUnitPrice,\r\n PricingQuantity\r\n )\r\n | extend ConsumedQuantity = case(\r\n isnotempty(ConsumedQuantity) and ConsumedQuantity != 0, ConsumedQuantity,\r\n ChargeCategory == 'Usage', PricingQuantity / coalesce(x_PricingBlockSize, decimal(1)),\r\n ConsumedQuantity\r\n )\r\n //\r\n // Populate missing prices -- mapping to on-demand prices requires meter ID and offer ID\r\n | extend tmp_MissingPrices = ProviderName == 'Microsoft'\r\n and (ListUnitPrice == 0 or ContractedUnitPrice == 0)\r\n and x_EffectiveUnitPrice != 0\r\n and not(CommitmentDiscountCategory == 'Spend' and CommitmentDiscountStatus == 'Unused')\r\n and isnotempty(strcat(x_SkuMeterId, x_SkuOfferId))\r\n | as allCosts\r\n | where tmp_MissingPrices\r\n | extend tmp_ReservationPriceLookupKey = tolower(strcat(x_BillingProfileId, substring(ChargePeriodStart, 0, 7), x_SkuMeterId, x_SkuOfferId))\r\n | as costsWithMissingPrices\r\n | join kind=leftouter (\r\n Prices_final_v1_0\r\n | extend tmp_ReservationPriceLookupKey = tolower(strcat(x_BillingProfileId, substring(x_EffectivePeriodStart, 0, 7), x_SkuMeterId, x_SkuOfferId))\r\n | where x_SkuPriceType == 'Consumption' and tmp_ReservationPriceLookupKey in ((costsWithMissingPrices | summarize by tmp_ReservationPriceLookupKey))\r\n | summarize ListUnitPrice = min(ListUnitPrice), ContractedUnitPrice = min(ContractedUnitPrice) by tmp_ReservationPriceLookupKey, x_PricingBlockSize, PricingUnit\r\n ) on tmp_ReservationPriceLookupKey\r\n //\r\n // Select the best price to use for each row\r\n // TODO: Save values before changing -- | extend x_old_ContractedUnitPrice = ContractedUnitPrice, x_old_EffectiveUnitPrice = x_EffectiveUnitPrice, x_old_ListUnitPrice = ListUnitPrice, x_old_ListCost = ListCost, x_old_ContractedCost = ContractedCost\r\n | extend x_EffectiveUnitPrice = case(\r\n // If price is a rounding error away from the billed price, use the billed price\r\n isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(x_BilledUnitPrice - x_EffectiveUnitPrice) < 0.0001, x_BilledUnitPrice,\r\n // If price is a rounding error away from the contracted price, use the contracted price\r\n isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(ContractedUnitPrice - x_EffectiveUnitPrice) < 0.0001, ContractedUnitPrice,\r\n x_EffectiveUnitPrice\r\n )\r\n | extend ContractedUnitPrice = case(\r\n // If price is already correct, keep that\r\n (isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0) or (EffectiveCost == 0 and BilledCost == 0), ContractedUnitPrice,\r\n // If both prices use the same scale, use the new one\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize == x_PricingBlockSize1, ContractedUnitPrice1 * x_BillingExchangeRate,\r\n // If prices are the same unit but not the same scale, use the new one but correct the scale\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize != x_PricingBlockSize1 and isnotempty(x_PricingBlockSize) and isnotempty(x_PricingBlockSize1), ContractedUnitPrice1 * x_BillingExchangeRate / x_PricingBlockSize1 * x_PricingBlockSize,\r\n // If billed price is available, assume the billed price is the same as contracted price to support aggregations\r\n isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, x_EffectiveUnitPrice,\r\n // Otherwise, assume the effective price is the same as contracted price to support aggregations\r\n x_EffectiveUnitPrice\r\n )\r\n | extend ListUnitPrice = case(\r\n // If price is already correct, keep that\r\n (isnotempty(ListUnitPrice) and ListUnitPrice != 0) or (EffectiveCost == 0 and BilledCost == 0), ListUnitPrice,\r\n // If both prices use the same scale, use the new one\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize == x_PricingBlockSize1, ListUnitPrice1 * x_BillingExchangeRate,\r\n // If prices are the same unit but not the same scale, use the new one but correct the scale\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize != x_PricingBlockSize1 and isnotempty(x_PricingBlockSize) and isnotempty(x_PricingBlockSize1), ListUnitPrice1 * x_BillingExchangeRate / x_PricingBlockSize1 * x_PricingBlockSize,\r\n // Otherwise, assume the contracted price is the same as list price to support aggregations\r\n ContractedUnitPrice\r\n )\r\n // Calculate missing costs based on new prices -- If cost is already correct, keep that; if not and price is available, recalculate the cost; otherwise, keep the existing cost\r\n | extend ContractedCost = case(\r\n // If not set or there's no cost, keep the original value\r\n (isnotempty(ContractedCost) and ContractedCost != 0) or (EffectiveCost == 0 and BilledCost == 0), ContractedCost,\r\n // ContractedCost is 0 in all other scenarios...\r\n // If 0 and there's a billed cost and prices are the same, use BilledCost\r\n isnotempty(BilledCost) and BilledCost != 0 and ContractedUnitPrice == x_BilledUnitPrice, BilledCost,\r\n // If 0 and there's a billed cost and prices are the same, use EffectiveCost\r\n isnotempty(EffectiveCost) and EffectiveCost != 0 and ContractedUnitPrice == x_EffectiveUnitPrice, EffectiveCost,\r\n // If 0 and there's a price, calculate the cost based on the price\r\n isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0, ContractedUnitPrice * PricingQuantity,\r\n // If 0 and there's no price, assume EffectiveCost\r\n isempty(ContractedUnitPrice) or ContractedUnitPrice == 0, EffectiveCost,\r\n // Fall back to the original value for any unhandled scenarios\r\n ContractedCost\r\n )\r\n | extend ListCost = case(\r\n // If not set or there's no cost, keep the original value\r\n (isnotempty(ListCost) and ListCost != 0) or (EffectiveCost == 0 and BilledCost == 0), ListCost,\r\n // ListCost is 0 in all other scenarios...\r\n // If 0 and there's a contracted cost and prices are the same, use ContractedCost\r\n isnotempty(ContractedCost) and ContractedCost != 0 and ListUnitPrice == ContractedUnitPrice, ContractedCost,\r\n // If 0 and there's a price, calculate the cost based on the price\r\n isnotempty(ListUnitPrice) and ListUnitPrice != 0, ListUnitPrice * PricingQuantity,\r\n // If 0 and there's no price, assume ContractedCost\r\n isempty(ListUnitPrice) or ListUnitPrice == 0, ContractedCost,\r\n // Fall back to the original value for any unhandled scenarios\r\n ListCost\r\n )\r\n // Merge the rest of the unmodified cost records and remove excess columns\r\n | union (allCosts | where not(tmp_MissingPrices))\r\n | project-away x_PricingBlockSize1, PricingUnit1, ListUnitPrice1, ContractedUnitPrice1, tmp_MissingPrices, tmp_ReservationPriceLookupKey, tmp_ReservationPriceLookupKey1\r\n //\r\n // BUG: Fix ContractedCost that has bad values\r\n | extend ContractedCost = iff(ProviderName == 'Microsoft' and isnotempty(PricingQuantity) and isnotempty(x_PricingBlockSize) and ContractedCost != ContractedUnitPrice * PricingQuantity, ContractedUnitPrice * PricingQuantity, ContractedCost)\r\n //\r\n // Handle FOCUS 1.0-preview UsageQuantity/Unit\r\n | extend ConsumedQuantity = iff(ChargeCategory == 'Usage', coalesce(ConsumedQuantity, UsageQuantity, UsageAmount), todecimal(''))\r\n | extend ConsumedUnit = iff(ChargeCategory == 'Usage' and isnotempty(ConsumedQuantity), coalesce(ConsumedUnit, UsageUnit, 'Units'), '')\r\n //\r\n // Convert IDs to lowercase for consistency\r\n | extend CommitmentDiscountId = tolower(CommitmentDiscountId)\r\n //\r\n // BUG: Remove EffectiveCost for commitment discount purchases\r\n | extend EffectiveCost = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), decimal(0), EffectiveCost)\r\n | extend x_EffectiveCostInUsd = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), decimal(0), x_EffectiveCostInUsd)\r\n //\r\n // Clean up resource columns\r\n | extend ResourceId = case(\r\n isnotempty(ResourceId), ResourceId,\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), CommitmentDiscountId,\r\n ResourceId)\r\n | extend ResourceName = tolower(case(\r\n isnotempty(ResourceName), ResourceName,\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountName), CommitmentDiscountName,\r\n isnotempty(ResourceId), parse_resourceid(ResourceId).ResourceName,\r\n ResourceName))\r\n | extend x_ResourceType = case(\r\n isnotempty(x_ResourceType), x_ResourceType,\r\n isnotempty(ResourceId), parse_resourceid(ResourceId).x_ResourceType,\r\n x_ResourceType)\r\n | extend ResourceType = case(\r\n // Use existing resource type display name unless it's an internal resource type ID\r\n isnotempty(ResourceType) and tolower(ResourceType) != tolower(x_ResourceType) and ResourceType !contains '/', ResourceType,\r\n // Use CommitmentDiscountType for commitment discount purchases\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountType), CommitmentDiscountType,\r\n // Look up display name from internal type\r\n isnotempty(x_ResourceType), coalesce(resource_type(x_ResourceType).SingularDisplayName, ResourceType, x_ResourceType),\r\n ResourceType)\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n AvailabilityZone,\r\n BilledCost,\r\n BillingAccountId = tolower(BillingAccountId),\r\n BillingAccountName,\r\n BillingAccountType,\r\n BillingCurrency,\r\n BillingPeriodEnd = startofmonth(BillingPeriodEnd),\r\n BillingPeriodStart = startofmonth(BillingPeriodStart),\r\n ChargeCategory = case(\r\n // Handle FOCUS 1.0-preview ChargeSubcategory\r\n ChargeSubcategory == 'Credit', 'Credit',\r\n ChargeSubcategory == 'Refund', 'Purchase', // We are assuming purchase refunds since we don't have data to indicate usage refunds\r\n ChargeCategory\r\n ),\r\n ChargeClass = case(ChargeSubcategory == 'Refund', 'Correction', ChargeClass),\r\n ChargeDescription,\r\n // BUG: ChargeFrequency shows \"Usage-Based\" for monthly recurring savings plan purchases\r\n ChargeFrequency = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and ChargeFrequency == 'Usage-Based' and ProviderName == 'Microsoft' and x_SourceVersion startswith '1.0', 'Recurring', ChargeFrequency),\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId = tolower(CommitmentDiscountId),\r\n CommitmentDiscountName,\r\n CommitmentDiscountStatus = case(\r\n // Handle FOCUS 1.0-preview ChargeSubcategory\r\n ChargeSubcategory == 'Used Commitment', 'Used',\r\n ChargeSubcategory == 'Unused Commitment', 'Unused',\r\n CommitmentDiscountStatus\r\n ),\r\n CommitmentDiscountType,\r\n ConsumedQuantity,\r\n ConsumedUnit,\r\n ContractedCost = coalesce(ContractedCost, x_OnDemandCost, x_Cost),\r\n ContractedUnitPrice = coalesce(ContractedUnitPrice, x_OnDemandUnitPrice),\r\n EffectiveCost,\r\n InvoiceIssuerName,\r\n ListCost,\r\n ListUnitPrice,\r\n PricingCategory = case(\r\n // Handle FOCUS 1.0-preview PricingCategory values\r\n PricingCategory == 'On-Demand', 'Standard',\r\n PricingCategory == 'Commitment-Based', 'Committed',\r\n PricingCategory\r\n ),\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n // Handle missing PublisherName values\r\n PublisherName = case(PublisherName == 'Microsoft Corporation', 'Microsoft', isnotempty(PublisherName), PublisherName, x_PublisherCategory == 'Cloud Provider', ProviderName, ''),\r\n // Handle FOCUS 1.0-preview Region column\r\n RegionId = coalesce(RegionId, iff(ProviderName == 'Microsoft', replace_string(tolower(Region), ' ', ''), Region)),\r\n RegionName = coalesce(RegionName, Region),\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n SkuId,\r\n SkuPriceId,\r\n SubAccountId,\r\n SubAccountName,\r\n SubAccountType, // Azure 1.0-preview(v1)+\r\n Tags = parse_json(Tags),\r\n x_AccountId, // Azure 1.0-preview(v1)+\r\n x_AccountName, // Azure 1.0-preview(v1)+\r\n x_AccountOwnerId, // Azure 1.0-preview(v1)+\r\n x_BilledCostInUsd, // Azure 1.0-preview(v1)+\r\n x_BilledUnitPrice, // Azure 1.0-preview(v1)+\r\n x_BillingAccountAgreement = case(\r\n ProviderName == 'Microsoft' and x_BillingAccountId == x_BillingProfileId, 'EA',\r\n ProviderName == 'Microsoft' and x_BillingAccountId != x_BillingProfileId, 'MCA',\r\n ProviderName\r\n ), // Hubs add-on\r\n x_BillingAccountId, // Azure 1.0-preview(v1)+\r\n x_BillingAccountName, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRate, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRateDate, // Azure 1.0-preview(v1)+\r\n x_BillingProfileId, // Azure 1.0-preview(v1)+\r\n x_BillingProfileName, // Azure 1.0-preview(v1)+\r\n x_ChargeId, // Azure 1.0-preview(v1) only\r\n x_ContractedCostInUsd = coalesce(x_ContractedCostInUsd, x_OnDemandCostInUsd), // Azure 1.0+\r\n x_CostAllocationRuleName, // Azure 1.0-preview(v1)+\r\n x_CostCategories = parse_json(x_CostCategories), // AWS 1.0 (JSON)\r\n x_CostCenter, // Azure 1.0-preview(v1)+\r\n x_Credits = parse_json(x_Credits), // GCP Jan 2024\r\n x_CostType, // GCP Jan 2024\r\n x_CurrencyConversionRate, // GCP Jun 2024\r\n x_CustomerId, // Azure 1.0-preview(v1)+\r\n x_CustomerName, // Azure 1.0-preview(v1)+\r\n x_Discount = parse_json(x_Discount), // AWS 1.0 (JSON)\r\n x_EffectiveCostInUsd, // Azure 1.0-preview(v1)+\r\n x_EffectiveUnitPrice, // Azure 1.0-preview(v1)+\r\n x_ExportTime, // GCP Jan 2024\r\n x_IngestionTime, // Hubs add-on\r\n x_InvoiceId = coalesce(InvoiceId, x_InvoiceId), // Azure 1.0-preview(v1)+\r\n x_InvoiceIssuerId, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionId = case( // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionId == '-2', '',\r\n x_InvoiceSectionId\r\n ),\r\n x_InvoiceSectionName = case( // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionName == 'Unassigned', '',\r\n x_InvoiceSectionName\r\n ),\r\n x_ListCostInUsd, // Azure 1.0-preview(v1)+\r\n x_Location, // GCP Jan 2024\r\n x_Operation, // AWS 1.0\r\n x_PartnerCreditApplied, // Azure 1.0-preview(v1)+\r\n x_PartnerCreditRate, // Azure 1.0-preview(v1)+\r\n x_PricingBlockSize, // Azure 1.0-preview(v1)+\r\n x_PricingCurrency = coalesce(PricingCurrency, x_PricingCurrency), // Azure 1.0-preview(v1)+\r\n x_PricingSubcategory, // Azure 1.0-preview(v1)+\r\n x_PricingUnitDescription, // Azure 1.0-preview(v1)+\r\n x_Project, // GCP Jan 2024\r\n x_PublisherCategory, // Azure 1.0-preview(v1)+\r\n x_PublisherId, // Azure 1.0-preview(v1)+\r\n x_ResellerId, // Azure 1.0-preview(v1)+\r\n x_ResellerName, // Azure 1.0-preview(v1)+\r\n x_ResourceGroupName = tolower(x_ResourceGroupName), // Azure 1.0-preview(v1)+\r\n x_ResourceType, // Azure 1.0-preview(v1)+\r\n x_ServiceCode, // AWS 1.0\r\n x_ServiceId, // GCP Jan 2024\r\n x_ServicePeriodEnd, // Azure 1.0-preview(v1)+\r\n x_ServicePeriodStart, // Azure 1.0-preview(v1)+\r\n x_SkuDescription, // Azure 1.0-preview(v1)+\r\n x_SkuDetails = parse_json(x_SkuDetails), // Azure 1.0-preview(v1)+\r\n x_SkuIsCreditEligible, // Azure 1.0-preview(v1)+\r\n x_SkuMeterCategory, // Azure 1.0-preview(v1)+\r\n x_SkuMeterId, // Azure 1.0-preview(v1)+\r\n x_SkuMeterName = coalesce(SkuMeter, x_SkuMeterName), // Azure 1.0-preview(v1)+\r\n x_SkuMeterSubcategory, // Azure 1.0-preview(v1)+\r\n x_SkuOfferId, // Azure 1.0-preview(v1)+\r\n x_SkuOrderId, // Azure 1.0-preview(v1)+\r\n x_SkuOrderName, // Azure 1.0-preview(v1)+\r\n x_SkuPartNumber, // Azure 1.0-preview(v1)+\r\n x_SkuRegion, // Azure 1.0-preview(v1)+\r\n x_SkuServiceFamily, // Azure 1.0-preview(v1)+\r\n x_SkuTerm, // Azure 1.0-preview(v1)+\r\n x_SkuTier, // Azure 1.0-preview(v1)+\r\n x_SourceChanges, // Hubs add-on\r\n x_SourceName, // Hubs add-on\r\n x_SourceProvider, // Hubs add-on\r\n x_SourceType, // Hubs add-on\r\n x_SourceVersion, // Hubs add-on\r\n x_UsageType // AWS 1.0\r\n}\r\n\r\n// Costs_final_v1_0 table\r\n.create-merge table Costs_final_v1_0 (\r\n AvailabilityZone: string,\r\n BilledCost: decimal,\r\n BillingAccountId: string,\r\n BillingAccountName: string,\r\n BillingAccountType: string, // Azure 1.0-preview(v1)+\r\n BillingCurrency: string,\r\n BillingPeriodEnd: datetime,\r\n BillingPeriodStart: datetime,\r\n ChargeCategory: string,\r\n ChargeClass: string,\r\n ChargeDescription: string,\r\n ChargeFrequency: string,\r\n ChargePeriodEnd: datetime,\r\n ChargePeriodStart: datetime,\r\n CommitmentDiscountCategory: string, // FOCUS 1.0-preview only\r\n CommitmentDiscountId: string,\r\n CommitmentDiscountName: string,\r\n CommitmentDiscountStatus: string,\r\n CommitmentDiscountType: string,\r\n ConsumedQuantity: decimal,\r\n ConsumedUnit: string,\r\n ContractedCost: decimal,\r\n ContractedUnitPrice: decimal,\r\n EffectiveCost: decimal,\r\n InvoiceIssuerName: string,\r\n ListCost: decimal,\r\n ListUnitPrice: decimal,\r\n PricingCategory: string,\r\n PricingQuantity: decimal,\r\n PricingUnit: string,\r\n ProviderName: string,\r\n PublisherName: string,\r\n RegionId: string,\r\n RegionName: string,\r\n ResourceId: string,\r\n ResourceName: string,\r\n ResourceType: string,\r\n ServiceCategory: string,\r\n ServiceName: string,\r\n SkuId: string,\r\n SkuPriceId: string,\r\n SubAccountId: string,\r\n SubAccountName: string,\r\n SubAccountType: string,\r\n Tags: dynamic,\r\n x_AccountId: string, // Azure 1.0-preview(v1)+\r\n x_AccountName: string, // Azure 1.0-preview(v1)+\r\n x_AccountOwnerId: string, // Azure 1.0-preview(v1)+\r\n x_BilledCostInUsd: decimal, // Azure 1.0-preview(v1)+\r\n x_BilledUnitPrice: decimal, // Azure 1.0-preview(v1)+\r\n x_BillingAccountAgreement: string, // Hubs add-on\r\n x_BillingAccountId: string, // Azure 1.0-preview(v1)+\r\n x_BillingAccountName: string, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRate: decimal, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRateDate: datetime, // Azure 1.0-preview(v1)+\r\n x_BillingProfileId: string, // Azure 1.0-preview(v1)+\r\n x_BillingProfileName: string, // Azure 1.0-preview(v1)+\r\n x_ChargeId: string, // Azure 1.0-preview(v1) only\r\n x_ContractedCostInUsd: decimal, // Azure 1.0+\r\n x_CostAllocationRuleName: string, // Azure 1.0-preview(v1)+\r\n x_CostCategories: dynamic, // AWS 1.0 (JSON)\r\n x_CostCenter: string, // Azure 1.0-preview(v1)+\r\n x_Credits: dynamic, // GCP Jan 2024\r\n x_CostType: string, // GCP Jan 2024\r\n x_CurrencyConversionRate: decimal, // GCP Jun 2024\r\n x_CustomerId: string, // Azure 1.0-preview(v1)+\r\n x_CustomerName: string, // Azure 1.0-preview(v1)+\r\n x_Discount: dynamic, // AWS 1.0 (JSON)\r\n x_EffectiveCostInUsd: decimal, // Azure 1.0-preview(v1)+\r\n x_EffectiveUnitPrice: decimal, // Azure 1.0-preview(v1)+\r\n x_ExportTime: datetime, // GCP Jan 2024\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_InvoiceId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceIssuerId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionName: string, // Azure 1.0-preview(v1)+\r\n x_ListCostInUsd: decimal, // Azure 1.0-preview(v1)+\r\n x_Location: string, // GCP Jan 2024\r\n x_Operation: string, // AWS 1.0\r\n x_PartnerCreditApplied: string, // Azure 1.0-preview(v1)+\r\n x_PartnerCreditRate: string, // Azure 1.0-preview(v1)+\r\n x_PricingBlockSize: decimal, // Azure 1.0-preview(v1)+\r\n x_PricingCurrency: string, // Azure 1.0-preview(v1)+\r\n x_PricingSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_PricingUnitDescription: string, // Azure 1.0-preview(v1)+\r\n x_Project: string, // GCP Jan 2024\r\n x_PublisherCategory: string, // Azure 1.0-preview(v1)+\r\n x_PublisherId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceGroupName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceType: string, // Azure 1.0-preview(v1)+\r\n x_ServiceCode: string, // AWS 1.0\r\n x_ServiceId: string, // GCP Jan 2024\r\n x_ServicePeriodEnd: datetime, // Azure 1.0-preview(v1)+\r\n x_ServicePeriodStart: datetime, // Azure 1.0-preview(v1)+\r\n x_SkuDescription: string, // Azure 1.0-preview(v1)+\r\n x_SkuDetails: dynamic, // Azure 1.0-preview(v1)+\r\n x_SkuIsCreditEligible: bool, // Azure 1.0-preview(v1)+\r\n x_SkuMeterCategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterId: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterName: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuOfferId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOrderId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOrderName: string, // Azure 1.0-preview(v1)+\r\n x_SkuPartNumber: string, // Azure 1.0-preview(v1)+\r\n x_SkuRegion: string, // Azure 1.0-preview(v1)+\r\n x_SkuServiceFamily: string, // Azure 1.0-preview(v1)+\r\n x_SkuTerm: int, // Azure 1.0-preview(v1)+\r\n x_SkuTier: string, // Azure 1.0-preview(v1)+\r\n x_SourceChanges: string, // Hubs add-on\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_UsageType: string // AWS 1.0\r\n)\r\n\r\n// Update policy for Costs_raw -> Costs_final_v1_0 table\r\n.alter table Costs_final_v1_0 policy update\r\n```\r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"Costs_raw\",\r\n \"Query\": \"Costs_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Actual costs |===================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// ActualCosts_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: ActualCost exports transformed to FOCUS 1.0. Use ActualCosts_transform_v1_2() instead.', folder='Costs')\r\nActualCosts_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n // TODO: Transform actual costs to FOCUS 1.0 format\r\n ActualCosts_raw\r\n | where ChargeType in ('Purchase', 'Refund') and isnotempty(ReservationId)\r\n //\r\n //\r\n // !!! The rest of the query is reused for both the ActualCosts and AmortizedCosts queries -- Copy all changes in both transform functions !!!\r\n //\r\n //\r\n | extend x_AmortizationClass = case(\r\n ChargeType in ('Purchase', 'Refund') and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', 'Amortized Charge',\r\n ''\r\n )\r\n | extend tmp_ResourceInfo = parse_resourceid(ResourceId)\r\n // TODO: PricingCategory needs to include savings plan usage and spot usage\r\n | extend PricingCategory = case(\r\n x_AmortizationClass == 'Amortized Charge', 'Committed',\r\n ChargeType in ('Usage', 'Purchase'), 'Standard',\r\n ''\r\n )\r\n | extend x_ResourceType = tostring(tmp_ResourceInfo.x_ResourceType)\r\n | project-rename\r\n PricingQuantity = Quantity,\r\n x_PricingUnitDescription = UnitOfMeasure\r\n | join kind=leftouter (PricingUnits) on x_PricingUnitDescription\r\n | join kind=leftouter (Regions) on ResourceLocation\r\n | join kind=leftouter (ResourceTypes | project x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n // TODO: Add the following in 1.2: ServiceSubcategory, PublisherName, x_PublisherCategory, x_Environment, x_ServiceModel\r\n | join kind=leftouter (Services | where isnotempty(x_ResourceType) | project x_ResourceType, ServiceName, ServiceCategory) on x_ResourceType\r\n | join kind=leftouter (Services | where isnotempty(x_ConsumedService) | summarize take_any(ServiceName), take_any(ServiceCategory) by ConsumedService = x_ConsumedService) on ConsumedService\r\n | extend CommitmentDiscountCategory = iff(isnotempty(ReservationId), 'Usage', '') // TODO: CommitmentDiscountCategory needs to handle savings plans\r\n | project\r\n AvailabilityZone = AvailabilityZone,\r\n BilledCost = iff(x_AmortizationClass == 'Amortized Charge', real(0), Cost),\r\n BillingAccountId = strcat('/providers/microsoft.billing/billingaccounts/', BillingAccountId, iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), '', strcat('/billingprofiles/', BillingProfileId))),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName),\r\n BillingAccountType = iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), 'Billing Profile', 'Billing Account'),\r\n BillingCurrency,\r\n BillingPeriodEnd = startofmonth(BillingPeriodEndDate, 1),\r\n BillingPeriodStart = startofmonth(BillingPeriodStartDate),\r\n CapacityReservationId = '',\r\n CapacityReservationStatus = '',\r\n ChargeCategory = case(\r\n ChargeType in ('Usage', 'Purchase', 'Credit', 'Tax'), ChargeType,\r\n ChargeType in ('UnusedReservation', 'UnusedSavingsPlan'), 'Usage',\r\n ChargeType == 'Refund', 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = iff(ChargeType == 'Refund', 'Correction', ''),\r\n ChargeDescription = Product,\r\n ChargeFrequency = case(\r\n Frequency == 'UsageBased', 'Usage-Based',\r\n Frequency == 'OneTime', 'One-Time',\r\n Frequency // \"Recurring\" and any fallback\r\n ),\r\n ChargePeriodStart = Date,\r\n ChargePeriodEnd = Date + 1d,\r\n ChargeSubcategory = '',\r\n // TODO: CommitmentDiscount* columns need to handle savings plans\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId = iff(isnotempty(ReservationId), strcat('/providers/microsoft.capacity/reservationorders/', ProductOrderId, '/reservations/', ReservationId), ''),\r\n CommitmentDiscountName = ReservationName,\r\n CommitmentDiscountQuantity = real(null),\r\n CommitmentDiscountStatus = case(\r\n isempty(ReservationId), '',\r\n isnotempty(ReservationId) and ChargeType == 'Usage', 'Used',\r\n ChargeType startswith 'Unused', 'Unused',\r\n 'Unused'\r\n ),\r\n CommitmentDiscountType = iff(isnotempty(ReservationId), 'Reservation', ''),\r\n CommitmentDiscountUnit = '',\r\n ConsumedQuantity = PricingQuantity * x_PricingBlockSize,\r\n ConsumedUnit = PricingUnit,\r\n ContractedCost = UnitPrice * PricingQuantity,\r\n ContractedUnitPrice = UnitPrice,\r\n EffectiveCost = iff(x_AmortizationClass == 'Principal', real(0), Cost),\r\n InvoiceId = '',\r\n InvoiceIssuerName = 'Microsoft',\r\n ListCost = real(null),\r\n ListUnitPrice = real(null),\r\n PricingCategory,\r\n PricingCurrency = '',\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName = 'Microsoft',\r\n PublisherName = iff(PublisherType == 'Marketplace', PublisherName, 'Microsoft'),\r\n Region = '',\r\n RegionId,\r\n RegionName,\r\n ResourceId = tostring(tmp_ResourceInfo.ResourceId),\r\n ResourceName = tostring(tmp_ResourceInfo.ResourceName),\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory = '',\r\n SkuId = '',\r\n SkuMeter = '',\r\n SkuPriceDetails = '',\r\n SkuPriceId = '',\r\n SubAccountId = strcat('/subscriptions/', SubscriptionId),\r\n SubAccountName = SubscriptionName,\r\n SubAccountType = iff(isempty(SubscriptionId), '', 'Subscription'),\r\n Tags = iff(Tags startswith '{', Tags, strcat('{', Tags, '}')),\r\n UsageAmount = real(null),\r\n UsageQuantity = real(null),\r\n UsageUnit = '',\r\n x_AccountId = '',\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerId,\r\n x_AmortizationClass = '',\r\n x_BilledCostInUsd = real(null),\r\n x_BilledUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), UnitPrice),\r\n x_BillingAccountId = BillingAccountId,\r\n x_BillingAccountName = BillingAccountName,\r\n x_BillingExchangeRate = real(null),\r\n x_BillingExchangeRateDate = datetime(null),\r\n x_BillingProfileId = BillingProfileId,\r\n x_BillingProfileName = BillingProfileName,\r\n x_BillingItemCode = '',\r\n x_BillingItemName = '',\r\n x_ChargeId = '',\r\n x_CommodityCode = '',\r\n x_CommodityName = '',\r\n x_ComponentName = '',\r\n x_ComponentType = '',\r\n x_ContractedCostInUsd = real(null),\r\n x_Cost = real(null),\r\n x_CostAllocationRuleName = '',\r\n x_CostCategories = '',\r\n x_CostCenter = CostCenter,\r\n x_CostType = '',\r\n x_Credits = '',\r\n x_CurrencyConversionRate = real(null),\r\n x_CustomerId = '',\r\n x_CustomerName = '',\r\n x_Discount = '',\r\n x_EffectiveCostInUsd = real(null),\r\n x_EffectiveUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), EffectivePrice),\r\n x_ExportTime = datetime(null),\r\n x_InstanceID = '',\r\n x_InvoiceId = '',\r\n x_InvoiceIssuerId = '',\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = InvoiceSection,\r\n x_ListCostInUsd = real(null),\r\n x_Location = '',\r\n x_OnDemandCost = real(null),\r\n x_OnDemandCostInUsd = real(null),\r\n x_OnDemandUnitPrice = real(null),\r\n x_Operation = '',\r\n x_OwnerAccountID = '',\r\n x_PartnerCreditApplied = '',\r\n x_PartnerCreditRate = '',\r\n x_PricingBlockSize,\r\n x_PricingCurrency = iff(BillingAccountId == BillingProfileId, BillingCurrency, ''),\r\n x_PricingSubcategory = case(\r\n // TODO: Add x_SkuTier when supported by C360 -- PricingCategory == 'Standard' and isnotempty(x_SkuTier), 'Tiered',\r\n PricingCategory == 'Standard', 'Standard',\r\n PricingCategory == 'Committed', strcat('Committed ', CommitmentDiscountCategory),\r\n PricingCategory == 'Dynamic', 'Spot',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_Project = '',\r\n x_PublisherCategory = iff(PublisherType == 'Marketplace', 'Vendor', 'Cloud Provider'),\r\n x_PublisherId = '',\r\n x_ResellerId = '',\r\n x_ResellerName = '',\r\n x_ResourceGroupName = ResourceGroup,\r\n x_ResourceType,\r\n x_ServiceCode = '',\r\n x_ServiceId = '',\r\n x_ServiceModel = '',\r\n x_ServicePeriodEnd = datetime(null),\r\n x_ServicePeriodStart = datetime(null),\r\n x_SkuDescription = Product,\r\n x_SkuDetails = iff(AdditionalInfo startswith '{', AdditionalInfo, strcat('{', AdditionalInfo, '}')),\r\n x_SkuIsCreditEligible = IsAzureCreditEligible,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterId = MeterId,\r\n x_SkuMeterName = MeterName,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuOfferId = OfferId,\r\n x_SkuOrderId = ProductOrderId,\r\n x_SkuOrderName = ProductOrderName,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPlanName = '',\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTerm = toint(Term),\r\n x_SkuTier = '',\r\n x_SourceName = 'C360',\r\n x_SourceProvider = 'Microsoft',\r\n x_SourceType = 'ActualCost',\r\n x_SourceVersion = 'C360-2025-04',\r\n x_SubproductName = '',\r\n x_UsageType = ''\r\n}\r\n\r\n// Update policy for ActualCosts_raw -> Costs_raw table\r\n.alter table Costs_raw policy update\r\n``` \r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"ActualCosts_raw\",\r\n \"Query\": \"ActualCosts_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Amortized costs |================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// AmortizedCosts_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: ActualCost exports transformed to FOCUS 1.0. Use AmortizedCosts_transform_v1_2() instead.', folder='Costs')\r\nAmortizedCosts_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n // TODO: Transform actual costs to FOCUS 1.0 format\r\n AmortizedCosts_raw\r\n //\r\n //\r\n // !!! The rest of the query is reused for both the ActualCosts and AmortizedCosts queries -- Copy all changes in both transform functions !!!\r\n //\r\n //\r\n | extend x_AmortizationClass = case(\r\n ChargeType in ('Purchase', 'Refund') and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', 'Amortized Charge',\r\n ''\r\n )\r\n | extend tmp_ResourceInfo = parse_resourceid(ResourceId)\r\n // TODO: PricingCategory needs to include savings plan usage and spot usage\r\n | extend PricingCategory = case(\r\n x_AmortizationClass == 'Amortized Charge', 'Committed',\r\n ChargeType in ('Usage', 'Purchase'), 'Standard',\r\n ''\r\n )\r\n | extend x_ResourceType = tostring(tmp_ResourceInfo.x_ResourceType)\r\n | project-rename\r\n PricingQuantity = Quantity,\r\n x_PricingUnitDescription = UnitOfMeasure\r\n | join kind=leftouter (PricingUnits) on x_PricingUnitDescription\r\n | join kind=leftouter (Regions) on ResourceLocation\r\n | join kind=leftouter (ResourceTypes | project x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n // TODO: Add the following in 1.2: ServiceSubcategory, PublisherName, x_PublisherCategory, x_Environment, x_ServiceModel\r\n | join kind=leftouter (Services | where isnotempty(x_ResourceType) | project x_ResourceType, ServiceName, ServiceCategory) on x_ResourceType\r\n | join kind=leftouter (Services | where isnotempty(x_ConsumedService) | summarize take_any(ServiceName), take_any(ServiceCategory) by ConsumedService = x_ConsumedService) on ConsumedService\r\n | extend CommitmentDiscountCategory = iff(isnotempty(ReservationId), 'Usage', '') // TODO: CommitmentDiscountCategory needs to handle savings plans\r\n | project\r\n AvailabilityZone = AvailabilityZone,\r\n BilledCost = iff(x_AmortizationClass == 'Amortized Charge', real(0), Cost),\r\n BillingAccountId = strcat('/providers/microsoft.billing/billingaccounts/', BillingAccountId, iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), '', strcat('/billingprofiles/', BillingProfileId))),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName),\r\n BillingAccountType = iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), 'Billing Profile', 'Billing Account'),\r\n BillingCurrency,\r\n BillingPeriodEnd = startofmonth(BillingPeriodEndDate, 1),\r\n BillingPeriodStart = startofmonth(BillingPeriodStartDate),\r\n CapacityReservationId = '',\r\n CapacityReservationStatus = '',\r\n ChargeCategory = case(\r\n ChargeType in ('Usage', 'Purchase', 'Credit', 'Tax'), ChargeType,\r\n ChargeType in ('UnusedReservation', 'UnusedSavingsPlan'), 'Usage',\r\n ChargeType == 'Refund', 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = iff(ChargeType == 'Refund', 'Correction', ''),\r\n ChargeDescription = Product,\r\n ChargeFrequency = case(\r\n Frequency == 'UsageBased', 'Usage-Based',\r\n Frequency == 'OneTime', 'One-Time',\r\n Frequency // \"Recurring\" and any fallback\r\n ),\r\n ChargePeriodStart = Date,\r\n ChargePeriodEnd = Date + 1d,\r\n ChargeSubcategory = '',\r\n // TODO: CommitmentDiscount* columns need to handle savings plans\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId = iff(isnotempty(ReservationId), strcat('/providers/microsoft.capacity/reservationorders/', ProductOrderId, '/reservations/', ReservationId), ''),\r\n CommitmentDiscountName = ReservationName,\r\n CommitmentDiscountQuantity = real(null),\r\n CommitmentDiscountStatus = case(\r\n isempty(ReservationId), '',\r\n isnotempty(ReservationId) and ChargeType == 'Usage', 'Used',\r\n ChargeType startswith 'Unused', 'Unused',\r\n 'Unused'\r\n ),\r\n CommitmentDiscountType = iff(isnotempty(ReservationId), 'Reservation', ''),\r\n CommitmentDiscountUnit = '',\r\n ConsumedQuantity = PricingQuantity * x_PricingBlockSize,\r\n ConsumedUnit = PricingUnit,\r\n ContractedCost = UnitPrice * PricingQuantity,\r\n ContractedUnitPrice = UnitPrice,\r\n EffectiveCost = iff(x_AmortizationClass == 'Principal', real(0), Cost),\r\n InvoiceId = '',\r\n InvoiceIssuerName = 'Microsoft',\r\n ListCost = real(null),\r\n ListUnitPrice = real(null),\r\n PricingCategory,\r\n PricingCurrency = '',\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName = 'Microsoft',\r\n PublisherName = iff(PublisherType == 'Marketplace', PublisherName, 'Microsoft'),\r\n Region = '',\r\n RegionId,\r\n RegionName,\r\n ResourceId = tostring(tmp_ResourceInfo.ResourceId),\r\n ResourceName = tostring(tmp_ResourceInfo.ResourceName),\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory = '',\r\n SkuId = '',\r\n SkuMeter = '',\r\n SkuPriceDetails = '',\r\n SkuPriceId = '',\r\n SubAccountId = strcat('/subscriptions/', SubscriptionId),\r\n SubAccountName = SubscriptionName,\r\n SubAccountType = iff(isempty(SubscriptionId), '', 'Subscription'),\r\n Tags = iff(Tags startswith '{', Tags, strcat('{', Tags, '}')),\r\n UsageAmount = real(null),\r\n UsageQuantity = real(null),\r\n UsageUnit = '',\r\n x_AccountId = '',\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerId,\r\n x_AmortizationClass = '',\r\n x_BilledCostInUsd = real(null),\r\n x_BilledUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), UnitPrice),\r\n x_BillingAccountId = BillingAccountId,\r\n x_BillingAccountName = BillingAccountName,\r\n x_BillingExchangeRate = real(null),\r\n x_BillingExchangeRateDate = datetime(null),\r\n x_BillingProfileId = BillingProfileId,\r\n x_BillingProfileName = BillingProfileName,\r\n x_BillingItemCode = '',\r\n x_BillingItemName = '',\r\n x_ChargeId = '',\r\n x_CommodityCode = '',\r\n x_CommodityName = '',\r\n x_ComponentName = '',\r\n x_ComponentType = '',\r\n x_ContractedCostInUsd = real(null),\r\n x_Cost = real(null),\r\n x_CostAllocationRuleName = '',\r\n x_CostCategories = '',\r\n x_CostCenter = CostCenter,\r\n x_CostType = '',\r\n x_Credits = '',\r\n x_CurrencyConversionRate = real(null),\r\n x_CustomerId = '',\r\n x_CustomerName = '',\r\n x_Discount = '',\r\n x_EffectiveCostInUsd = real(null),\r\n x_EffectiveUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), EffectivePrice),\r\n x_ExportTime = datetime(null),\r\n x_InstanceID = '',\r\n x_InvoiceId = '',\r\n x_InvoiceIssuerId = '',\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = InvoiceSection,\r\n x_ListCostInUsd = real(null),\r\n x_Location = '',\r\n x_OnDemandCost = real(null),\r\n x_OnDemandCostInUsd = real(null),\r\n x_OnDemandUnitPrice = real(null),\r\n x_Operation = '',\r\n x_OwnerAccountID = '',\r\n x_PartnerCreditApplied = '',\r\n x_PartnerCreditRate = '',\r\n x_PricingBlockSize,\r\n x_PricingCurrency = iff(BillingAccountId == BillingProfileId, BillingCurrency, ''),\r\n x_PricingSubcategory = case(\r\n // TODO: Add x_SkuTier when supported by C360 -- PricingCategory == 'Standard' and isnotempty(x_SkuTier), 'Tiered',\r\n PricingCategory == 'Standard', 'Standard',\r\n PricingCategory == 'Committed', strcat('Committed ', CommitmentDiscountCategory),\r\n PricingCategory == 'Dynamic', 'Spot',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_Project = '',\r\n x_PublisherCategory = iff(PublisherType == 'Marketplace', 'Vendor', 'Cloud Provider'),\r\n x_PublisherId = '',\r\n x_ResellerId = '',\r\n x_ResellerName = '',\r\n x_ResourceGroupName = ResourceGroup,\r\n x_ResourceType,\r\n x_ServiceCode = '',\r\n x_ServiceId = '',\r\n x_ServiceModel = '',\r\n x_ServicePeriodEnd = datetime(null),\r\n x_ServicePeriodStart = datetime(null),\r\n x_SkuDescription = Product,\r\n x_SkuDetails = iff(AdditionalInfo startswith '{', AdditionalInfo, strcat('{', AdditionalInfo, '}')),\r\n x_SkuIsCreditEligible = IsAzureCreditEligible,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterId = MeterId,\r\n x_SkuMeterName = MeterName,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuOfferId = OfferId,\r\n x_SkuOrderId = ProductOrderId,\r\n x_SkuOrderName = ProductOrderName,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPlanName = '',\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTerm = toint(Term),\r\n x_SkuTier = '',\r\n x_SourceName = 'C360',\r\n x_SourceProvider = 'Microsoft',\r\n x_SourceType = 'ActualCost',\r\n x_SourceVersion = 'C360-2025-04',\r\n x_SubproductName = '',\r\n x_UsageType = ''\r\n}\r\n\r\n// Update policy for AmortizedCosts_raw -> Costs_raw table\r\n.alter table Costs_raw policy update\r\n``` \r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"AmortizedCosts_raw\",\r\n \"Query\": \"AmortizedCosts_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| CommitmentDiscountUsage |========================================================================================\r\n// Supported versions:\r\n// - MS EA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-ea\r\n// - MS MCA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-mca\r\n//======================================================================================================================\r\n\r\n// CommitmentDiscountUsage_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: All commitment discount usage transformed to FOCUS 1.0. This includes reservationdeatils_raw. Use CommitmentDiscountUsage_transform_v1_2() instead.', folder='Commitment discounts')\r\nCommitmentDiscountUsage_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n CommitmentDiscountUsage_raw\r\n //\r\n // Change real to decimal\r\n | extend\r\n InstanceFlexibilityRatio = todecimal(InstanceFlexibilityRatio),\r\n ReservedHours = todecimal(ReservedHours),\r\n TotalReservedQuantity = todecimal(TotalReservedQuantity),\r\n UsedHours = todecimal(UsedHours)\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Handle resource columns\r\n | extend ResourceId = tolower(InstanceId)\r\n | extend tmp_ResourceDetails = parse_resourceid(ResourceId)\r\n | extend ResourceName = tostring(tmp_ResourceDetails.ResourceName)\r\n | extend SubAccountId = tostring(tmp_ResourceDetails.SubAccountId)\r\n | extend x_ResourceGroupName = tostring(tmp_ResourceDetails.x_ResourceGroupName)\r\n | extend x_ResourceType = tostring(tmp_ResourceDetails.x_ResourceType)\r\n | lookup kind=leftouter (ResourceTypes | distinct x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n | lookup kind=leftouter (Services | distinct x_ResourceType, ServiceName, ServiceCategory, x_ServiceModel) on x_ResourceType\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n ChargePeriodEnd = UsageDate + 1d,\r\n ChargePeriodStart = UsageDate,\r\n CommitmentDiscountCategory = 'Usage',\r\n CommitmentDiscountId = tolower(strcat('/providers/microsoft.capacity/reservationorders/', ReservationOrderId, '/reservations/', ReservationId)),\r\n CommitmentDiscountType = 'Reservation',\r\n ConsumedQuantity = UsedHours,\r\n ProviderName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n SubAccountId,\r\n x_CommitmentDiscountCommittedCount = TotalReservedQuantity,\r\n x_CommitmentDiscountCommittedAmount = ReservedHours,\r\n // TODO: Is this needed? -- x_CommitmentDiscountKind = Kind,\r\n x_CommitmentDiscountNormalizedGroup = iff(InstanceFlexibilityGroup == 'NA', '', InstanceFlexibilityGroup),\r\n x_CommitmentDiscountNormalizedRatio = InstanceFlexibilityRatio,\r\n x_CommitmentDiscountQuantity = UsedHours * InstanceFlexibilityRatio,\r\n x_IngestionTime = ingestion_time(),\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n // x_RowId = hash_sha256(strcat(\r\n // // DO NOT CHANGE COLUMNS OR COLUMN ORDER\r\n // CommitmentDiscountId,\r\n // ResourceId,\r\n // ChargePeriodStart\r\n // )),\r\n x_ServiceModel,\r\n x_SkuOrderId = ReservationOrderId,\r\n x_SkuSize = iff(SkuName == 'NA', '', SkuName),\r\n x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName)),\r\n x_SourceProvider = coalesce(x_SourceProvider, ProviderName),\r\n x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationDetails', '')),\r\n x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2024-03-01', ''))\r\n}\r\n\r\n// CommitmentDiscountUsage_final_v1_0 table\r\n.create-merge table CommitmentDiscountUsage_final_v1_0 (\r\n ChargePeriodEnd: datetime, // Hubs add-on\r\n ChargePeriodStart: datetime, // MS 2023-03-01\r\n CommitmentDiscountCategory: string, // Hubs add-on\r\n CommitmentDiscountId: string, // MS 2023-03-01\r\n CommitmentDiscountType: string, // Hubs add-on\r\n ConsumedQuantity: decimal, // MS 2023-03-01\r\n ProviderName: string, // Hubs add-on\r\n ResourceId: string, // MS 2023-03-01\r\n ResourceName: string, // Hubs add-on\r\n ResourceType: string, // Hubs add-on\r\n ServiceCategory: string, // Hubs add-on\r\n ServiceName: string, // Hubs add-on\r\n SubAccountId: string, // Hubs add-on\r\n x_CommitmentDiscountCommittedCount: decimal, // MS 2023-03-01\r\n x_CommitmentDiscountCommittedAmount: decimal, // MS 2023-03-01\r\n x_CommitmentDiscountNormalizedGroup: string, // MS 2023-03-01\r\n x_CommitmentDiscountNormalizedRatio: decimal, // MS 2023-03-01\r\n x_CommitmentDiscountQuantity: decimal, // MS 2023-03-01\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_ResourceGroupName: string, // Hubs add-on\r\n x_ResourceType: string, // Hubs add-on\r\n x_ServiceModel: string, // Hubs add-on\r\n x_SkuOrderId: string, // MS 2023-03-01\r\n x_SkuSize: string, // MS 2023-03-01\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string // Hubs add-on\r\n)\r\n\r\n// Update policy for CommitmentDiscountUsage_raw -> CommitmentDiscountUsage_final_v1_0 table\r\n.alter table CommitmentDiscountUsage_final_v1_0 policy update\r\n```\r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"CommitmentDiscountUsage_raw\",\r\n \"Query\": \"CommitmentDiscountUsage_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Recommendations |================================================================================================\r\n// Supported datasets/versions:\r\n// - MS CM EA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-ea\r\n// - MS CM MCA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-mca\r\n//======================================================================================================================\r\n\r\n// Recommendations_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: All recommendations transformed to FOCUS 1.0. Use Recommendations_transform_v1_2() instead.', folder='Recommendations')\r\nRecommendations_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', toint(''),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n Recommendations_raw\r\n //\r\n // Change real to decimal\r\n | extend\r\n CostWithNoReservedInstances = todecimal(CostWithNoReservedInstances),\r\n InstanceFlexibilityRatio = todecimal(InstanceFlexibilityRatio),\r\n NetSavings = todecimal(NetSavings),\r\n RecommendedQuantity = todecimal(RecommendedQuantity),\r\n RecommendedQuantityNormalized = todecimal(RecommendedQuantityNormalized),\r\n TotalCostWithReservedInstances = todecimal(TotalCostWithReservedInstances)\r\n //\r\n | extend x_IngestionTime = ingestion_time()\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Set source columns\r\n | extend x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationRecommendations', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2023-05-01', ''))\r\n //\r\n // Convert JSON cost columns to decimal\r\n | extend CostWithNoReservedInstances = case(isnotempty(CostWithNoReservedInstances), CostWithNoReservedInstances, isnotempty(CostWithNoReservedInstancesJson), todecimal(extract(@'\"value\":([0-9\\.]+)', 1, CostWithNoReservedInstancesJson)), CostWithNoReservedInstances)\r\n | extend NetSavings = case(isnotempty(NetSavings), NetSavings, isnotempty(NetSavingsJson), todecimal(extract(@'\"value\":([0-9\\.]+)', 1, NetSavingsJson)), NetSavings)\r\n | extend TotalCostWithReservedInstances = case(isnotempty(TotalCostWithReservedInstances), TotalCostWithReservedInstances, isnotempty(TotalCostWithReservedInstancesJson), todecimal(extract(@'\"value\":([0-9\\.]+)', 1, TotalCostWithReservedInstancesJson)), TotalCostWithReservedInstances)\r\n //\r\n // Build recommendation details\r\n | lookup kind=leftouter (Regions | summarize RegionName = make_set(RegionName)[0] by Location = RegionId) on Location\r\n | extend x_RecommendationDetails = case(\r\n x_SourceType == 'ReservationRecommendations', bag_pack(\r\n 'CommitmentDiscountNormalizedGroup', InstanceFlexibilityGroup,\r\n 'CommitmentDiscountNormalizedRatio', InstanceFlexibilityRatio,\r\n 'CommitmentDiscountNormalizedSize', NormalizedSize,\r\n 'CommitmentDiscountResourceType', ResourceType,\r\n 'CommitmentDiscountScope', Scope,\r\n 'LookbackPeriodDuration', case(\r\n LookBackPeriod matches regex @'^Last([0-9]+)Days$', replace_regex(LookBackPeriod, @'^Last([0-9]+)Days$', @'P\\1D'),\r\n LookBackPeriod matches regex @'^[0-9]+$', strcat('P', LookBackPeriod, 'D'),\r\n ''\r\n ),\r\n 'LookbackPeriodStart', FirstUsageDate,\r\n 'RecommendedQuantity', RecommendedQuantity,\r\n 'RecommendedQuantityNormalized', RecommendedQuantityNormalized,\r\n 'RegionId', Location,\r\n 'RegionName', RegionName,\r\n 'SkuMeterId', MeterId,\r\n 'SkuPriceDetails', SkuProperties,\r\n 'SkuSize', coalesce(SKU, SkuName),\r\n 'SkuTerm', isoMonths(Term)\r\n ),\r\n dynamic({})\r\n )\r\n //\r\n // Sort columns and apply final transforms\r\n | extend x_RecommendationDate = FirstUsageDate + (toint(extract(@'^P([0-9]+)D$', 1, tostring(x_RecommendationDetails.LookbackPeriodDuration))) * 1d)\r\n | extend x_RecommendationDate = iff(x_RecommendationDate > now(), startofday(now()), x_RecommendationDate)\r\n | project\r\n ProviderName,\r\n SubAccountId = iff(isnotempty(SubscriptionId), strcat('/subscriptions/', SubscriptionId), ''),\r\n x_IngestionTime,\r\n x_EffectiveCostAfter = TotalCostWithReservedInstances,\r\n x_EffectiveCostBefore = CostWithNoReservedInstances,\r\n x_EffectiveCostSavings = NetSavings,\r\n x_RecommendationDate = FirstUsageDate + (toint(extract(@'^P([0-9]+)D$', 1, tostring(x_RecommendationDetails.LookbackPeriodDuration))) * 1d),\r\n x_RecommendationDetails,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n// Recommendations_final_v1_0 table\r\n.create-merge table Recommendations_final_v1_0 (\r\n ProviderName: string,\r\n SubAccountId: string,\r\n x_IngestionTime: datetime,\r\n x_EffectiveCostAfter: decimal,\r\n x_EffectiveCostBefore: decimal,\r\n x_EffectiveCostSavings: decimal,\r\n x_RecommendationDate: datetime,\r\n x_RecommendationDetails: dynamic,\r\n x_SourceName: string,\r\n x_SourceProvider: string,\r\n x_SourceType: string,\r\n x_SourceVersion: string\r\n)\r\n\r\n// Update policy for Recommendations_raw -> Recommendations_final_v1_0 table\r\n.alter table Recommendations_final_v1_0 policy update\r\n```\r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"Recommendations_raw\",\r\n \"Query\": \"Recommendations_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Transactions |===================================================================================================\r\n// Supported versions:\r\n// - MS CM EA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-ea\r\n// - MS CM MCA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-mca\r\n//======================================================================================================================\r\n\r\n// Transactions_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: All transactions transformed to FOCUS 1.0. Use Transactions_transform_v1_2() instead.', folder='Transactions')\r\nTransactions_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', toint(''),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n Transactions_raw\r\n //\r\n // Change real to decimal\r\n | extend\r\n Amount = todecimal(Amount),\r\n MonetaryCommitment = todecimal(MonetaryCommitment),\r\n Overage = todecimal(Overage),\r\n Quantity = todecimal(Quantity)\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Set source columns\r\n | extend x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationTransactions', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2023-05-01', ''))\r\n //\r\n // Handle BillingPeriodStart/End\r\n | extend BillingMonth = tostring(BillingMonth)\r\n | extend BillingPeriodStart = iff(isempty(BillingMonth), datetime(null), todatetime(strcat(substring(BillingMonth, 0, 4), \"-\", substring(BillingMonth, 4, 2), \"-\", substring(BillingMonth, 6, 2))))\r\n | extend BillingPeriodEnd = iff(isempty(BillingMonth), datetime(null), startofmonth(endofmonth(BillingPeriodStart) + 1d))\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n BilledCost = Amount,\r\n BillingAccountId = case(\r\n BillingProfileId startswith '/', BillingProfileId,\r\n isnotempty(CurrentEnrollmentId), strcat('/providers/Microsoft.Billing/billingAccounts/', CurrentEnrollmentId),\r\n isnotempty(BillingProfileId), strcat('/providers/Microsoft.Billing/billingProfiles/', BillingProfileId),\r\n ''\r\n ),\r\n BillingAccountName = coalesce(BillingProfileName, CurrentEnrollmentId),\r\n BillingCurrency = Currency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n ChargeCategory = case(\r\n EventType in ('Cancel', 'Purchase', 'Refund'), 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = case(\r\n EventType == 'Cancel', 'Cancel', // FOCUS does not handle this scenario\r\n EventType == 'Refund', 'Correction',\r\n ''\r\n ),\r\n ChargeDescription = Description,\r\n ChargeFrequency = case(\r\n BillingFrequency == 'OneTime', 'One-Time',\r\n BillingFrequency == 'Recurring', 'Recurring',\r\n BillingFrequency\r\n ),\r\n ChargePeriodStart = EventDate,\r\n PricingQuantity = Quantity,\r\n PricingUnit = 'Reservations',\r\n ProviderName,\r\n RegionId = Region,\r\n RegionName = Region,\r\n SubAccountId = iff(isempty(PurchasingSubscriptionGuid), '', strcat('/subscriptions/', PurchasingSubscriptionGuid)),\r\n SubAccountName = iff(isempty(PurchasingSubscriptionGuid), '', PurchasingSubscriptionName),\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerEmail,\r\n x_CostCenter = CostCenter,\r\n x_InvoiceId = InvoiceId,\r\n x_InvoiceNumber = Invoice,\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = coalesce(InvoiceSectionName, DepartmentName),\r\n x_IngestionTime = ingestion_time(),\r\n x_MonetaryCommitment = MonetaryCommitment,\r\n x_Overage = Overage,\r\n x_PurchasingBillingAccountId = PurchasingEnrollment,\r\n x_SkuOrderId = ReservationOrderId,\r\n x_SkuOrderName = ReservationOrderName,\r\n x_SkuSize = ArmSkuName,\r\n x_SkuTerm = isoMonths(Term),\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_SubscriptionId = PurchasingSubscriptionGuid,\r\n x_TransactionType = EventType\r\n}\r\n\r\n// Transactions_final_v1_0 table\r\n.create-merge table Transactions_final_v1_0 (\r\n BilledCost: decimal, // MS CM EA+MCA 2023-05-01\r\n BillingAccountId: string, // MS CM EA+MCA 2023-05-01\r\n BillingAccountName: string, // MS CM EA+MCA 2023-05-01\r\n BillingCurrency: string, // MS CM EA+MCA 2023-05-01\r\n BillingPeriodEnd: datetime, // MS CM EA+MCA 2023-05-01\r\n BillingPeriodStart: datetime, // MS CM EA+MCA 2023-05-01\r\n ChargeCategory: string, // Hubs add-on\r\n ChargeClass: string, // Hubs add-on\r\n ChargeDescription: string, // MS CM EA+MCA 2023-05-01\r\n ChargeFrequency: string, // MS CM EA+MCA 2023-05-01\r\n ChargePeriodStart: datetime, // MS CM EA+MCA 2023-05-01\r\n PricingQuantity: decimal, // MS CM EA+MCA 2023-05-01\r\n PricingUnit: string, // Hubs add-on\r\n ProviderName: string, // Hubs add-on\r\n RegionId: string, // MS CM EA+MCA 2023-05-01\r\n RegionName: string, // MS CM EA+MCA 2023-05-01\r\n SubAccountId: string, // MS CM EA+MCA 2023-05-01\r\n SubAccountName: string, // MS CM EA+MCA 2023-05-01\r\n x_AccountName: string, // MS CM EA 2023-05-01\r\n x_AccountOwnerId: string, // MS CM EA 2023-05-01\r\n x_CostCenter: string, // MS CM EA 2023-05-01\r\n x_InvoiceId: string, // MS CM MCA 2023-05-01\r\n x_InvoiceNumber: string, // MS CM MCA 2023-05-01\r\n x_InvoiceSectionId: string, // MS CM MCA 2023-05-01\r\n x_InvoiceSectionName: string, // MS CM MCA 2023-05-01\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_MonetaryCommitment: decimal, // MS CM EA 2023-05-01\r\n x_Overage: decimal, // MS CM EA 2023-05-01\r\n x_PurchasingBillingAccountId: string, // MS CM EA 2023-05-01\r\n x_SkuOrderId: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuOrderName: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuSize: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuTerm: int, // MS CM EA+MCA 2023-05-01\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_SubscriptionId: string, // MS CM EA+MCA 2023-05-01\r\n x_TransactionType: string // MS CM EA+MCA 2023-05-01\r\n)\r\n\r\n// Update policy for Transactions_raw -> Transactions_final_v1_0 table\r\n.alter table Transactions_final_v1_0 policy update\r\n```\r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"Transactions_raw\",\r\n \"Query\": \"Transactions_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n", - "dataExplorerPrivateDnsZoneName": "[replace(format('privatelink.{0}.{1}', parameters('location'), replace(environment().suffixes.storage, 'core', 'kusto')), '..', '.')]", - "ingestionCapacity": { - "Dev(No SLA)_Standard_E2a_v4": 1, - "Dev(No SLA)_Standard_D11_v2": 1, - "Standard_D11_v2": 2, - "Standard_D12_v2": 4, - "Standard_D13_v2": 8, - "Standard_D14_v2": 16, - "Standard_D16d_v5": 16, - "Standard_D32d_v4": 32, - "Standard_D32d_v5": 32, - "Standard_DS13_v2+1TB_PS": 8, - "Standard_DS13_v2+2TB_PS": 8, - "Standard_DS14_v2+3TB_PS": 16, - "Standard_DS14_v2+4TB_PS": 16, - "Standard_E2a_v4": 2, - "Standard_E2ads_v5": 2, - "Standard_E2d_v4": 2, - "Standard_E2d_v5": 2, - "Standard_E4a_v4": 4, - "Standard_E4ads_v5": 4, - "Standard_E4d_v4": 4, - "Standard_E4d_v5": 4, - "Standard_E8a_v4": 8, - "Standard_E8ads_v5": 8, - "Standard_E8as_v4+1TB_PS": 8, - "Standard_E8as_v4+2TB_PS": 8, - "Standard_E8as_v5+1TB_PS": 8, - "Standard_E8as_v5+2TB_PS": 8, - "Standard_E8d_v4": 8, - "Standard_E8d_v5": 8, - "Standard_E8s_v4+1TB_PS": 8, - "Standard_E8s_v4+2TB_PS": 8, - "Standard_E8s_v5+1TB_PS": 8, - "Standard_E8s_v5+2TB_PS": 8, - "Standard_E16a_v4": 16, - "Standard_E16ads_v5": 16, - "Standard_E16as_v4+3TB_PS": 16, - "Standard_E16as_v4+4TB_PS": 16, - "Standard_E16as_v5+3TB_PS": 16, - "Standard_E16as_v5+4TB_PS": 16, - "Standard_E16d_v4": 16, - "Standard_E16d_v5": 16, - "Standard_E16s_v4+3TB_PS": 16, - "Standard_E16s_v4+4TB_PS": 16, - "Standard_E16s_v5+3TB_PS": 16, - "Standard_E16s_v5+4TB_PS": 16, - "Standard_E64i_v3": 64, - "Standard_E80ids_v4": 80, - "Standard_EC8ads_v5": 8, - "Standard_EC8as_v5+1TB_PS": 8, - "Standard_EC8as_v5+2TB_PS": 8, - "Standard_EC16ads_v5": 16, - "Standard_EC16as_v5+3TB_PS": 16, - "Standard_EC16as_v5+4TB_PS": 16, - "Standard_L4s": 4, - "Standard_L8as_v3": 8, - "Standard_L8s": 8, - "Standard_L8s_v2": 8, - "Standard_L8s_v3": 8, - "Standard_L16as_v3": 16, - "Standard_L16s": 16, - "Standard_L16s_v2": 16, - "Standard_L16s_v3": 16, - "Standard_L32as_v3": 32, - "Standard_L32s_v3": 32 - } - }, - "resources": [ - { - "type": "Microsoft.Kusto/clusters/principalAssignments", - "apiVersion": "2023-08-15", - "name": "[format('{0}/{1}', parameters('clusterName'), 'adf-mi-cluster-admin')]", + "dataFactory::pipeline_ExecuteExportsETL": { + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_ExecuteETL', variables('MSEXPORTS')))]", + "properties": { + "activities": [ + { + "name": "Wait", + "description": "Files may not be available immediately after being created.", + "type": "Wait", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "waitTimeInSeconds": 60 + } + }, + { + "name": "Read Manifest", + "description": "Load the export manifest to determine the scope, dataset, and date range.", + "type": "Lookup", + "dependsOn": [ + { + "activity": "Wait", + "dependencyConditions": [ + "Completed" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "msexports_manifest", + "type": "DatasetReference", + "parameters": { + "fileName": { + "value": "@pipeline().parameters.fileName", + "type": "Expression" + }, + "folderPath": { + "value": "@pipeline().parameters.folderPath", + "type": "Expression" + } + } + } + } + }, + { + "name": "Set Has No Rows", + "description": "Check the row count ", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Read Manifest", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "hasNoRows", + "value": { + "value": "@or(equals(activity('Read Manifest').output.firstRow.blobCount, null), equals(activity('Read Manifest').output.firstRow.blobCount, 0))", + "type": "Expression" + } + } + }, + { + "name": "Set Export Dataset Type", + "description": "Save the dataset type from the export manifest.", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Read Manifest", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "exportDatasetType", + "value": { + "value": "@activity('Read Manifest').output.firstRow.exportConfig.type", + "type": "Expression" + } + } + }, + { + "name": "Set MCA Column", + "description": "Determines if the dataset schema has channel-specific columns and saves the column name that only exists in MCA to determine if it is an MCA dataset.", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Set Export Dataset Type", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "mcaColumnToCheck", + "value": { + "value": "@if(contains(createArray('pricesheet', 'reservationtransactions'), toLower(variables('exportDatasetType'))), 'BillingProfileId', if(equals(toLower(variables('exportDatasetType')), 'reservationrecommendations'), 'Net Savings', null))", + "type": "Expression" + } + } + }, + { + "name": "Set Export Dataset Version", + "description": "Save the dataset version from the export manifest.", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Read Manifest", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "exportDatasetVersion", + "value": { + "value": "@activity('Read Manifest').output.firstRow.exportConfig.dataVersion", + "type": "Expression" + } + } + }, + { + "name": "Detect Channel", + "description": "Determines what channel this export is from. Switch statement handles the different file types if the mcaColumnToCheck variable is set.", + "type": "Switch", + "dependsOn": [ + { + "activity": "Set Has No Rows", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set MCA Column", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set Export Dataset Version", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "on": { + "value": "@if(or(empty(variables('mcaColumnToCheck')), variables('hasNoRows')), 'ignore', last(array(split(activity('Read Manifest').output.firstRow.blobs[0].blobName, '.'))))", + "type": "Expression" + }, + "cases": [ + { + "value": "csv", + "activities": [ + { + "name": "Check for MCA Column in CSV", + "description": "Checks the dataset to determine if the applicable MCA-specific column exists.", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "DelimitedTextSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": false, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "DelimitedTextReadSettings" + } + }, + "dataset": { + "referenceName": "[replace(format('{0}', variables('MSEXPORTS')), '-', '_')]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@activity('Read Manifest').output.firstRow.blobs[0].blobName", + "type": "Expression" + } + } + } + } + }, + { + "name": "Set Schema File with Channel in CSV", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Check for MCA Column in CSV", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "schemaFile", + "value": { + "value": "@toLower(concat(variables('exportDatasetType'), '_', variables('exportDatasetVersion'), if(and(contains(activity('Check for MCA Column in CSV').output, 'firstRow'), contains(activity('Check for MCA Column in CSV').output.firstRow, variables('mcaColumnToCheck'))), '_mca', '_ea'), '.json'))", + "type": "Expression" + } + } + } + ] + }, + { + "value": "gz", + "activities": [ + { + "name": "Check for MCA Column in Gzip CSV", + "description": "Checks the dataset to determine if the applicable MCA-specific column exists.", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "DelimitedTextSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": false, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "DelimitedTextReadSettings" + } + }, + "dataset": { + "referenceName": "[format('{0}_gzip', variables('MSEXPORTS'))]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@activity('Read Manifest').output.firstRow.blobs[0].blobName", + "type": "Expression" + } + } + } + } + }, + { + "name": "Set Schema File with Channel in Gzip CSV", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Check for MCA Column in Gzip CSV", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "schemaFile", + "value": { + "value": "@toLower(concat(variables('exportDatasetType'), '_', variables('exportDatasetVersion'), if(and(contains(activity('Check for MCA Column in Gzip CSV').output, 'firstRow'), contains(activity('Check for MCA Column in Gzip CSV').output.firstRow, variables('mcaColumnToCheck'))), '_mca', '_ea'), '.json'))", + "type": "Expression" + } + } + } + ] + }, + { + "value": "parquet", + "activities": [ + { + "name": "Check for MCA Column in Parquet", + "description": "Checks the dataset to determine if the applicable MCA-specific column exists.", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "ParquetSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": false, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "ParquetReadSettings" + } + }, + "dataset": { + "referenceName": "[format('{0}_parquet', variables('MSEXPORTS'))]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@activity('Read Manifest').output.firstRow.blobs[0].blobName", + "type": "Expression" + } + } + } + } + }, + { + "name": "Set Schema File with Channel for Parquet", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Check for MCA Column in Parquet", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "schemaFile", + "value": { + "value": "@toLower(concat(variables('exportDatasetType'), '_', variables('exportDatasetVersion'), if(and(contains(activity('Check for MCA Column in Parquet').output, 'firstRow'), contains(activity('Check for MCA Column in Parquet').output.firstRow, variables('mcaColumnToCheck'))), '_mca', '_ea'), '.json'))", + "type": "Expression" + } + } + } + ] + } + ], + "defaultActivities": [ + { + "name": "Set Schema File", + "type": "SetVariable", + "dependsOn": [], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "schemaFile", + "value": { + "value": "@toLower(concat(variables('exportDatasetType'), '_', variables('exportDatasetVersion'), '.json'))", + "type": "Expression" + } + } + } + ] + } + }, + { + "name": "Set Scope", + "description": "Save the scope from the export manifest.", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Read Manifest", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "scope", + "value": { + "value": "@split(toLower(activity('Read Manifest').output.firstRow.exportConfig.resourceId), '/providers/microsoft.costmanagement/exports/')[0]", + "type": "Expression" + } + } + }, + { + "name": "Set Date", + "description": "Save the exported month from the export manifest.", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Read Manifest", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "date", + "value": { + "value": "@replace(substring(activity('Read Manifest').output.firstRow.runInfo.startDate, 0, 7), '-', '')", + "type": "Expression" + } + } + }, + { + "name": "Failed to Read Manifest", + "type": "Fail", + "dependsOn": [ + { + "activity": "Set Date", + "dependencyConditions": [ + "Failed" + ] + }, + { + "activity": "Set Export Dataset Type", + "dependencyConditions": [ + "Failed" + ] + }, + { + "activity": "Set Scope", + "dependencyConditions": [ + "Failed" + ] + }, + { + "activity": "Read Manifest", + "dependencyConditions": [ + "Failed" + ] + }, + { + "activity": "Set Export Dataset Version", + "dependencyConditions": [ + "Failed" + ] + }, + { + "activity": "Detect Channel", + "dependencyConditions": [ + "Failed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Failed to read the manifest file for this export run. Manifest path: ', pipeline().parameters.folderPath)", + "type": "Expression" + }, + "errorCode": "ManifestReadFailed" + } + }, + { + "name": "Check Schema", + "description": "Verify that the schema file exists in storage.", + "type": "GetMetadata", + "dependsOn": [ + { + "activity": "Set Scope", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set Date", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Detect Channel", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference", + "parameters": { + "fileName": { + "value": "@variables('schemaFile')", + "type": "Expression" + }, + "folderPath": "[format('{0}/schemas', reference('schemaFiles').outputs.containerName.value)]" + } + }, + "fieldList": [ + "exists" + ], + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + } + }, + { + "name": "Schema Not Found", + "type": "Fail", + "dependsOn": [ + { + "activity": "Check Schema", + "dependencyConditions": [ + "Failed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('The ', variables('schemaFile'), ' schema mapping file was not found. Please confirm version ', variables('exportDatasetVersion'), ' of the ', variables('exportDatasetType'), ' dataset is supported by this version of FinOps hubs. You may need to upgrade to a newer release. To add support for another dataset, you can create a custom mapping file.')", + "type": "Expression" + }, + "errorCode": "SchemaNotFound" + } + }, + { + "name": "Set Hub Dataset", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Set Export Dataset Type", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "hubDataset", + "value": { + "value": "@if(equals(toLower(variables('exportDatasetType')), 'focuscost'), 'Costs', if(equals(toLower(variables('exportDatasetType')), 'pricesheet'), 'Prices', if(equals(toLower(variables('exportDatasetType')), 'reservationdetails'), 'CommitmentDiscountUsage', if(equals(toLower(variables('exportDatasetType')), 'reservationrecommendations'), 'Recommendations', if(equals(toLower(variables('exportDatasetType')), 'reservationtransactions'), 'Transactions', if(equals(toLower(variables('exportDatasetType')), 'actualcost'), 'ActualCosts', if(equals(toLower(variables('exportDatasetType')), 'amortizedcost'), 'AmortizedCosts', toLower(variables('exportDatasetType')))))))))", + "type": "Expression" + } + } + }, + { + "name": "Set Destination Folder", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Check Schema", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set Hub Dataset", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "destinationFolder", + "value": { + "value": "@replace(concat(variables('hubDataset'),'/',substring(variables('date'), 0, 4),'/',substring(variables('date'), 4, 2),'/',toLower(variables('scope')), if(equals(variables('hubDataset'), 'Recommendations'), activity('Read Manifest').output.firstRow.exportConfig.exportName, '')),'//','/')", + "type": "Expression" + } + } + }, + { + "name": "For Each Blob", + "description": "Loop thru each exported file listed in the manifest.", + "type": "ForEach", + "dependsOn": [ + { + "activity": "Set Destination Folder", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@if(variables('hasNoRows'), json('[]'), activity('Read Manifest').output.firstRow.blobs)", + "type": "Expression" + }, + "batchCount": "[if(parameters('app').hub.options.privateRouting, 4, 30)]", + "isSequential": false, + "activities": [ + { + "name": "Execute", + "description": "Run the ingestion ETL pipeline.", + "type": "ExecutePipeline", + "dependsOn": [], + "policy": { + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "pipeline": { + "referenceName": "[format('{0}_ETL_{1}', variables('MSEXPORTS'), variables('INGESTION'))]", + "type": "PipelineReference" + }, + "waitOnCompletion": true, + "parameters": { + "blobPath": { + "value": "@item().blobName", + "type": "Expression" + }, + "destinationFolder": { + "value": "@variables('destinationFolder')", + "type": "Expression" + }, + "destinationFile": { + "value": "@last(array(split(replace(replace(item().blobName, '.gz', ''), '.csv', '.parquet'), '/')))", + "type": "Expression" + }, + "ingestionId": { + "value": "@activity('Read Manifest').output.firstRow.runInfo.runId", + "type": "Expression" + }, + "schemaFile": { + "value": "@variables('schemaFile')", + "type": "Expression" + }, + "exportDatasetType": { + "value": "@variables('exportDatasetType')", + "type": "Expression" + }, + "exportDatasetVersion": { + "value": "@variables('exportDatasetVersion')", + "type": "Expression" + } + } + } + } + ] + } + }, + { + "name": "Copy Manifest", + "description": "Copy the manifest to the ingestion container to trigger ADX ingestion", + "type": "Copy", + "dependsOn": [ + { + "activity": "For Each Blob", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "sink": { + "type": "JsonSink", + "storeSettings": { + "type": "AzureBlobFSWriteSettings" + }, + "formatSettings": { + "type": "JsonWriteSettings" + } + }, + "enableStaging": false + }, + "inputs": [ + { + "referenceName": "msexports_manifest", + "type": "DatasetReference", + "parameters": { + "fileName": "manifest.json", + "folderPath": { + "value": "@pipeline().parameters.folderPath", + "type": "Expression" + } + } + } + ], + "outputs": [ + { + "referenceName": "ingestion_manifest", + "type": "DatasetReference", + "parameters": { + "fileName": "manifest.json", + "folderPath": { + "value": "[format('@concat(''{0}/'', variables(''destinationFolder''))', variables('INGESTION'))]", + "type": "Expression" + } + } + } + ] + } + ], + "parameters": { + "folderPath": { + "type": "string" + }, + "fileName": { + "type": "string" + } + }, + "variables": { + "date": { + "type": "String" + }, + "destinationFolder": { + "type": "String" + }, + "exportDatasetType": { + "type": "String" + }, + "exportDatasetVersion": { + "type": "String" + }, + "hasNoRows": { + "type": "Boolean" + }, + "hubDataset": { + "type": "String" + }, + "mcaColumnToCheck": { + "type": "String" + }, + "schemaFile": { + "type": "String" + }, + "scope": { + "type": "String" + } + }, + "annotations": [ + "New export" + ] + }, + "dependsOn": [ + "appRegistration", + "dataFactory::dataset_msexports", + "dataFactory::dataset_msexports_gzip", + "dataFactory::dataset_msexports_manifest", + "dataFactory::dataset_msexports_parquet", + "dataFactory::pipeline_ToIngestion", + "schemaFiles" + ] + }, + "dataFactory::pipeline_ToIngestion": { + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_ETL_{1}', variables('MSEXPORTS'), variables('INGESTION')))]", "properties": { - "principalType": "App", - "principalId": "[reference(resourceId('Microsoft.DataFactory/factories', parameters('dataFactoryName')), '2018-06-01', 'full').identity.principalId]", - "tenantId": "[reference(resourceId('Microsoft.DataFactory/factories', parameters('dataFactoryName')), '2018-06-01', 'full').identity.tenantId]", - "role": "AllDatabasesAdmin" + "activities": [ + { + "name": "Get Existing Parquet Files", + "description": "Get the previously ingested files so we can remove any older data. This is necessary to avoid data duplication in reports.", + "type": "GetMetadata", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "dataset": { + "referenceName": "[format('{0}_files', variables('INGESTION'))]", + "type": "DatasetReference", + "parameters": { + "folderPath": "@pipeline().parameters.destinationFolder" + } + }, + "fieldList": [ + "childItems" + ], + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "ParquetReadSettings" + } + } + }, + { + "name": "Filter Out Current Exports", + "description": "Remove existing files from the current export so those files do not get deleted.", + "type": "Filter", + "dependsOn": [ + { + "activity": "Get Existing Parquet Files", + "dependencyConditions": [ + "Completed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@if(contains(activity('Get Existing Parquet Files').output, 'childItems'), activity('Get Existing Parquet Files').output.childItems, json('[]'))", + "type": "Expression" + }, + "condition": { + "value": "[format('@and(endswith(item().name, ''.parquet''), not(startswith(item().name, concat(pipeline().parameters.ingestionId, ''{0}''))))', variables('ingestionIdFileNameSeparator'))]", + "type": "Expression" + } + } + }, + { + "name": "Load Schema Mappings", + "description": "Get schema mapping file to use for the CSV to parquet conversion.", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference", + "parameters": { + "fileName": { + "value": "@toLower(pipeline().parameters.schemaFile)", + "type": "Expression" + }, + "folderPath": "[format('{0}/schemas', variables('CONFIG'))]" + } + } + } + }, + { + "name": "Failed to Load Schema", + "type": "Fail", + "dependsOn": [ + { + "activity": "Load Schema Mappings", + "dependencyConditions": [ + "Failed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Unable to load the ', pipeline().parameters.schemaFile, ' schema file. Please confirm the schema and version are supported for FinOps hubs ingestion. Unsupported files will remain in the msexports container.')", + "type": "Expression" + }, + "errorCode": "SchemaLoadFailed" + } + }, + { + "name": "Set Additional Columns", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Load Schema Mappings", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "additionalColumns", + "value": { + "value": "@intersection(array(json(concat('[{\"name\":\"x_SourceProvider\",\"value\":\"Microsoft\"},{\"name\":\"x_SourceName\",\"value\":\"Cost Management\"},{\"name\":\"x_SourceType\",\"value\":\"', pipeline().parameters.exportDatasetVersion, '\"},{\"name\":\"x_SourceVersion\",\"value\":\"', pipeline().parameters.exportDatasetVersion, '\"}'))), activity('Load Schema Mappings').output.firstRow.additionalColumns)", + "type": "Expression" + } + } + }, + { + "name": "For Each Old File", + "description": "Loop thru each of the existing files from previous exports.", + "type": "ForEach", + "dependsOn": [ + { + "activity": "Convert to Parquet", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Filter Out Current Exports", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@activity('Filter Out Current Exports').output.Value", + "type": "Expression" + }, + "activities": [ + { + "name": "Delete Old Ingested File", + "description": "Delete the previously ingested files from older exports.", + "type": "Delete", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "dataset": { + "referenceName": "[variables('INGESTION')]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@concat(pipeline().parameters.destinationFolder, '/', item().name)", + "type": "Expression" + } + } + }, + "enableLogging": false, + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": false, + "enablePartitionDiscovery": false + } + } + } + ] + } + }, + { + "name": "Set Destination Path", + "type": "SetVariable", + "dependsOn": [], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "destinationPath", + "value": { + "value": "[format('@concat(pipeline().parameters.destinationFolder, ''/'', pipeline().parameters.ingestionId, ''{0}'', pipeline().parameters.destinationFile)', variables('ingestionIdFileNameSeparator'))]", + "type": "Expression" + } + } + }, + { + "name": "Convert to Parquet", + "description": "[format('Convert CSV to parquet and move the file to the {0} container.', variables('INGESTION'))]", + "type": "Switch", + "dependsOn": [ + { + "activity": "Set Destination Path", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Load Schema Mappings", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set Additional Columns", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "on": { + "value": "@last(array(split(pipeline().parameters.blobPath, '.')))", + "type": "Expression" + }, + "cases": [ + { + "value": "csv", + "activities": [ + { + "name": "Convert CSV File", + "type": "Copy", + "dependsOn": [], + "policy": { + "timeout": "0.00:10:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "DelimitedTextSource", + "additionalColumns": { + "value": "@variables('additionalColumns')", + "type": "Expression" + }, + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "DelimitedTextReadSettings" + } + }, + "sink": { + "type": "ParquetSink", + "storeSettings": { + "type": "AzureBlobFSWriteSettings" + }, + "formatSettings": { + "type": "ParquetWriteSettings", + "fileExtension": ".parquet" + } + }, + "enableStaging": false, + "parallelCopies": 1, + "validateDataConsistency": false, + "translator": { + "value": "@activity('Load Schema Mappings').output.firstRow.translator", + "type": "Expression" + } + }, + "inputs": [ + { + "referenceName": "[replace(format('{0}', variables('MSEXPORTS')), '-', '_')]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@pipeline().parameters.blobPath", + "type": "Expression" + } + } + } + ], + "outputs": [ + { + "referenceName": "[variables('INGESTION')]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@variables('destinationPath')", + "type": "Expression" + } + } + } + ] + } + ] + }, + { + "value": "gz", + "activities": [ + { + "name": "Convert GZip CSV File", + "type": "Copy", + "dependsOn": [], + "policy": { + "timeout": "0.00:10:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "DelimitedTextSource", + "additionalColumns": { + "value": "@variables('additionalColumns')", + "type": "Expression" + }, + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "DelimitedTextReadSettings" + } + }, + "sink": { + "type": "ParquetSink", + "storeSettings": { + "type": "AzureBlobFSWriteSettings" + }, + "formatSettings": { + "type": "ParquetWriteSettings", + "fileExtension": ".parquet" + } + }, + "enableStaging": false, + "parallelCopies": 1, + "validateDataConsistency": false, + "translator": { + "value": "@activity('Load Schema Mappings').output.firstRow.translator", + "type": "Expression" + } + }, + "inputs": [ + { + "referenceName": "[format('{0}_gzip', variables('MSEXPORTS'))]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@pipeline().parameters.blobPath", + "type": "Expression" + } + } + } + ], + "outputs": [ + { + "referenceName": "[variables('INGESTION')]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@variables('destinationPath')", + "type": "Expression" + } + } + } + ] + } + ] + }, + { + "value": "parquet", + "activities": [ + { + "name": "Move Parquet File", + "type": "Copy", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "ParquetSource", + "additionalColumns": { + "value": "@variables('additionalColumns')", + "type": "Expression" + }, + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "ParquetReadSettings" + } + }, + "sink": { + "type": "ParquetSink", + "storeSettings": { + "type": "AzureBlobFSWriteSettings" + }, + "formatSettings": { + "type": "ParquetWriteSettings", + "fileExtension": ".parquet" + } + }, + "enableStaging": false, + "parallelCopies": 1, + "validateDataConsistency": false + }, + "inputs": [ + { + "referenceName": "[format('{0}_parquet', variables('MSEXPORTS'))]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@pipeline().parameters.blobPath", + "type": "Expression" + } + } + } + ], + "outputs": [ + { + "referenceName": "[variables('INGESTION')]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@variables('destinationPath')", + "type": "Expression" + } + } + } + ] + } + ] + } + ], + "defaultActivities": [ + { + "name": "Unsupported File Type", + "type": "Fail", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Unable to ingest the specified export file because the file type is not supported. File: ', pipeline().parameters.blobPath)", + "type": "Expression" + }, + "errorCode": "UnsupportedExportFileType" + } + } + ] + } + }, + { + "name": "Read Hub Config", + "description": "Read the hub config to determine if the export should be retained.", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": false, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference", + "parameters": { + "fileName": "settings.json", + "folderPath": "[variables('CONFIG')]" + } + } + } + }, + { + "name": "If Not Retaining Exports", + "description": "If the msexports retention period <= 0, delete the source file. The main reason to keep the source file is to allow for troubleshooting and reprocessing in the future.", + "type": "IfCondition", + "dependsOn": [ + { + "activity": "Convert to Parquet", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Read Hub Config", + "dependencyConditions": [ + "Completed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "expression": { + "value": "@lessOrEquals(coalesce(activity('Read Hub Config').output.firstRow.retention.msexports.days, 0), 0)", + "type": "Expression" + }, + "ifTrueActivities": [ + { + "name": "Delete Source File", + "description": "Delete the exported data file to keep storage costs down. This file is not referenced by any reporting systems.", + "type": "Delete", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "dataset": { + "referenceName": "[format('{0}_parquet', variables('MSEXPORTS'))]", + "type": "DatasetReference", + "parameters": { + "blobPath": { + "value": "@pipeline().parameters.blobPath", + "type": "Expression" + } + } + }, + "enableLogging": false, + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + } + } + } + ] + } + } + ], + "parameters": { + "blobPath": { + "type": "String" + }, + "destinationFile": { + "type": "string" + }, + "destinationFolder": { + "type": "string" + }, + "ingestionId": { + "type": "string" + }, + "schemaFile": { + "type": "string" + }, + "exportDatasetType": { + "type": "string" + }, + "exportDatasetVersion": { + "type": "string" + } + }, + "variables": { + "additionalColumns": { + "type": "Array" + }, + "destinationPath": { + "type": "String" + } + } }, "dependsOn": [ - "[resourceId('Microsoft.Kusto/clusters', parameters('clusterName'))]" - ] - }, - { - "type": "Microsoft.Kusto/clusters/databases", - "apiVersion": "2023-08-15", - "name": "[format('{0}/{1}', parameters('clusterName'), 'Ingestion')]", - "location": "[parameters('location')]", - "kind": "ReadWrite", - "dependsOn": [ - "[resourceId('Microsoft.Kusto/clusters', parameters('clusterName'))]" + "appRegistration", + "dataFactory::dataset_msexports", + "dataFactory::dataset_msexports_gzip", + "dataFactory::dataset_msexports_parquet" ] }, - { - "type": "Microsoft.Kusto/clusters/databases", - "apiVersion": "2023-08-15", - "name": "[format('{0}/{1}', parameters('clusterName'), 'Hub')]", - "location": "[parameters('location')]", - "kind": "ReadWrite", + "dataFactory": { + "existing": true, + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", "dependsOn": [ - "[resourceId('Microsoft.Kusto/clusters', parameters('clusterName'))]" + "appRegistration" ] }, - { - "type": "Microsoft.Kusto/clusters", - "apiVersion": "2023-08-15", - "name": "[parameters('clusterName')]", - "location": "[parameters('location')]", - "tags": "[union(parameters('tags'), coalesce(tryGet(parameters('tagsByResource'), 'Microsoft.Kusto/clusters'), createObject()))]", - "sku": { - "name": "[parameters('clusterSku')]", - "tier": "[if(startsWith(parameters('clusterSku'), 'Dev(No SLA)_'), 'Basic', 'Standard')]", - "capacity": "[if(startsWith(parameters('clusterSku'), 'Dev(No SLA)_'), 1, if(equals(parameters('clusterCapacity'), 1), 2, parameters('clusterCapacity')))]" - }, - "identity": { - "type": "SystemAssigned" - }, + "appRegistration": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.Exports_Register", "properties": { - "enableStreamingIngest": true, - "enableAutoStop": false, - "publicNetworkAccess": "[if(parameters('enablePublicAccess'), 'Enabled', 'Disabled')]" + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "version": { + "value": "[variables('finOpsToolkitVersion')]" + }, + "features": { + "value": [ + "Storage", + "DataFactory" + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "8267413868206701537" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppFeature": { + "type": "string", + "allowedValues": [ + "DataFactory", + "KeyVault", + "Storage" + ], + "metadata": { + "description": "FinOps hub app features.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "version": { + "type": "string", + "metadata": { + "description": "Required. Version number of the FinOps hub app." + } + }, + "features": { + "type": "array", + "items": { + "$ref": "#/definitions/HubAppFeature" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which features the app requires. Allowed values: \"DataFactory\", \"KeyVault\", \"Storage\". Default: [] (none)." + } + }, + "storageRoles": { + "type": "array", + "items": { + "type": "string" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which RBAC roles the Data Factory identity needs on the storage account, if created. This is in addition to Storage Blob Data Contributor for reading and managing content. Default: [] (none)." + } + }, + "telemetryString": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n<#\r\n.SYNOPSIS\r\nManages Azure Data Factory triggers and pipelines during FinOps hub deployment.\r\n\r\n.DESCRIPTION\r\nThis script is called by Bicep deployment scripts to start/stop Data Factory triggers\r\nand optionally run pipelines. It handles two types of triggers:\r\n\r\n1. Schedule triggers - Can be started/stopped directly\r\n2. BlobEventsTriggers - Require Event Grid subscription management before start/stop\r\n\r\nBlobEventsTriggers use Event Grid to listen for storage blob events. Before stopping,\r\nthe Event Grid subscription must be removed (unsubscribed). Before starting, the\r\nsubscription must be added and fully provisioned. This script handles this automatically\r\nby detecting BlobEventsTriggers via the BlobPathBeginsWith property.\r\n\r\nThe script uses retry logic with linear backoff to handle transient API failures and\r\nwait for Event Grid subscription provisioning/deprovisioning to complete.\r\n\r\n.PARAMETER DataFactoryResourceGroup\r\nThe resource group containing the Data Factory.\r\n\r\n.PARAMETER DataFactoryName\r\nThe name of the Data Factory instance.\r\n\r\n.PARAMETER Pipelines\r\nPipe-delimited list of pipeline names to run (e.g., \"pipeline1|pipeline2\").\r\n\r\n.PARAMETER StartTriggers\r\nSwitch to start all stopped triggers (with Event Grid subscription for blob triggers).\r\n\r\n.PARAMETER StopTriggers\r\nSwitch to stop all running triggers (with Event Grid unsubscription for blob triggers).\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StopTriggers\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StartTriggers\r\n#>\r\n\r\nparam(\r\n [string] $DataFactoryResourceGroup,\r\n [string] $DataFactoryName,\r\n [string] $Pipelines = \"\",\r\n [switch] $StartTriggers,\r\n [switch] $StopTriggers\r\n)\r\n\r\n$MAX_RETRIES = 20\r\n$DeploymentScriptOutputs = @{}\r\n\r\nfunction Write-Log($Message)\r\n{\r\n Write-Output \"$(Get-Date -Format 'HH:mm:ss') $Message\"\r\n}\r\n\r\nfunction Invoke-WithRetry([scriptblock]$Action, [string]$Name, [int]$Delay = 5)\r\n{\r\n for ($i = 1; $i -le $MAX_RETRIES; $i++)\r\n {\r\n try { return & $Action }\r\n catch\r\n {\r\n Write-Log \"$Name failed (attempt $i/${MAX_RETRIES}): $($_.Exception.Message)\"\r\n if ($i -eq $MAX_RETRIES) { throw }\r\n Start-Sleep -Seconds ($Delay * $i)\r\n }\r\n }\r\n}\r\n\r\nfunction Set-BlobTriggerSubscription([string]$TriggerName, [switch]$Subscribe)\r\n{\r\n $targetStatus = if ($Subscribe) { 'Enabled' } else { 'Disabled' }\r\n $action = if ($Subscribe) { 'Subscribing' } else { 'Unsubscribing' }\r\n\r\n Write-Log \"$action $TriggerName to events...\"\r\n Invoke-WithRetry -Name \"$action $TriggerName\" -Delay 5 -Action {\r\n if ($Subscribe)\r\n {\r\n Add-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n else\r\n {\r\n Remove-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n\r\n $status = Get-AzDataFactoryV2TriggerSubscriptionStatus `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName\r\n if ($status.Status -ne $targetStatus)\r\n {\r\n throw \"Subscription status is $($status.Status), expected $targetStatus\"\r\n }\r\n }\r\n}\r\n\r\nif ($StartTriggers -or $StopTriggers)\r\n{\r\n $triggers = Invoke-WithRetry -Name \"Get triggers\" -Action {\r\n Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n | Where-Object {\r\n ($StartTriggers -and $_.Properties.RuntimeState -ne \"Started\") `\r\n -or ($StopTriggers -and $_.Properties.RuntimeState -ne \"Stopped\")\r\n }\r\n }\r\n\r\n Write-Log \"Found $($triggers.Count) trigger(s) to $(if ($StartTriggers) { 'start' } else { 'stop' })\"\r\n\r\n $triggers | ForEach-Object {\r\n $triggerName = $_.Name\r\n $isBlobTrigger = $null -ne $_.Properties.BlobPathBeginsWith\r\n\r\n if ($StopTriggers)\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName }\r\n Write-Log \"Stopping trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Stop $triggerName\" -Action {\r\n Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n else\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName -Subscribe }\r\n Write-Log \"Starting trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Start $triggerName\" -Action {\r\n Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n\r\n Invoke-WithRetry -Name \"Wait for $triggerName\" -Action {\r\n $state = (Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName).Properties.RuntimeState\r\n $expected = if ($StartTriggers) { 'Started' } else { 'Stopped' }\r\n if ($state -ne $expected) { throw \"Trigger is $state, expected $expected\" }\r\n }\r\n\r\n Write-Log \"...done\"\r\n $DeploymentScriptOutputs[$triggerName] = $true\r\n }\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($Pipelines))\r\n{\r\n $Pipelines.Split('|') | ForEach-Object {\r\n Write-Log \"Running pipeline $_...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", + "usesDataFactory": "[contains(parameters('features'), 'DataFactory')]", + "usesKeyVault": "[contains(parameters('features'), 'KeyVault')]", + "usesStorage": "[contains(parameters('features'), 'Storage')]", + "telemetryId": "[format('ftk-hubapp-{0}{1}{2}', parameters('app').id, if(empty(parameters('telemetryString')), '', '_'), parameters('telemetryString'))]", + "telemetryProps": { + "mode": "Incremental", + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "[format('FTK: {0}', parameters('app').id)]", + "version": "[parameters('version')]" + } + }, + "resources": [] + } + }, + "autoStartRbacRoles": [ + "673868aa-7521-48a0-acc6-0f60742d39f5" + ], + "factoryStorageRoles": "[union(parameters('storageRoles'), createArray('17d1049b-9a84-46fb-8f53-869881c3d3ab', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe', 'acdd72a7-3385-48ef-bd42-f606fba81ae7'))]", + "storageInfrastructureEncryptionProperties": "[if(not(parameters('app').hub.options.storageInfrastructureEncryption), createObject(), createObject('encryption', createObject('keySource', 'Microsoft.Storage', 'requireInfrastructureEncryption', parameters('app').hub.options.storageInfrastructureEncryption)))]" + }, + "resources": { + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').storage)]", + "properties": { + "name": "[parameters('app').storage]", + "groupId": "dfs", + "privateLinkResourceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "fqdns": [ + "[reference('storageAccount').primaryEndpoints.dfs]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "storageAccount" + ] + }, + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').keyVault)]", + "properties": { + "name": "[parameters('app').keyVault]", + "groupId": "vault", + "privateLinkResourceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "fqdns": [ + "[reference('keyVault').vaultUri]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "keyVault" + ] + }, + "dataFactory::managedVirtualNetwork": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'default')]", + "properties": {}, + "dependsOn": [ + "dataFactory" + ] + }, + "dataFactory::managedIntegrationRuntime": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/integrationRuntimes", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ManagedIntegrationRuntime')]", + "properties": { + "type": "Managed", + "managedVirtualNetwork": { + "referenceName": "default", + "type": "ManagedVirtualNetworkReference" + }, + "typeProperties": { + "computeProperties": { + "location": "[parameters('app').hub.location]", + "dataFlowProperties": { + "computeType": "General", + "coreCount": 8, + "timeToLive": 10, + "cleanup": false, + "customProperties": [] + }, + "copyComputeScaleProperties": { + "dataIntegrationUnit": 16, + "timeToLive": 30 + }, + "pipelineExternalComputeScaleProperties": { + "timeToLive": 30, + "numberOfPipelineNodes": 1, + "numberOfExternalNodes": 1 + } + } + } + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedVirtualNetwork" + ] + }, + "dataFactory::linkedService_keyVault": { + "condition": "[and(variables('usesDataFactory'), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').keyVault)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureKeyVault", + "typeProperties": { + "baseUrl": "[reference(format('Microsoft.KeyVault/vaults/{0}', parameters('app').keyVault), '2023-02-01').vaultUri]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "keyVault" + ] + }, + "dataFactory::linkedService_storageAccount": { + "condition": "[and(variables('usesDataFactory'), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').storage)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureBlobFS", + "typeProperties": { + "url": "[reference(format('Microsoft.Storage/storageAccounts/{0}', parameters('app').storage), '2021-08-01').primaryEndpoints.dfs]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "storageAccount" + ] + }, + "storageAccount::blobService": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]", + "dependsOn": [ + "storageAccount" + ] + }, + "blobEndpoint::blobPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "blobEndpoint" + ] + }, + "dfsEndpoint::dfsPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.dfs.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "dfsEndpoint" + ] + }, + "keyVaultPrivateDnsZone::keyVaultPrivateDnsZoneLink": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), format('{0}-link', replace(format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "properties": { + "virtualNetwork": { + "id": "[parameters('app').hub.routing.networkId]" + }, + "registrationEnabled": false + }, + "dependsOn": [ + "keyVaultPrivateDnsZone" + ] + }, + "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')))]" + } + } + ] + }, + "dependsOn": [ + "keyVaultEndpoint", + "keyVaultPrivateDnsZone" + ] + }, + "appTelemetry": { + "condition": "[parameters('app').hub.options.enableTelemetry]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "[if(lessOrEquals(length(variables('telemetryId')), 64), variables('telemetryId'), substring(variables('telemetryId'), 0, 64))]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Resources/deployments')]", + "properties": "[variables('telemetryProps')]" + }, + "dataFactory": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.DataFactory/factories')]", + "identity": { + "type": "SystemAssigned" + }, + "properties": { + "globalConfigurations": { + "PipelineBillingEnabled": "true" + } + } + }, + "storageRoleAssignments": { + "copy": { + "name": "storageRoleAssignments", + "count": "[length(variables('factoryStorageRoles'))]" + }, + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "name": "[guid(resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage), variables('factoryStorageRoles')[copyIndex()], resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('factoryStorageRoles')[copyIndex()])]", + "principalId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "storageAccount" + ] + }, + "triggerManagerIdentity": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[format('{0}_triggerManager', parameters('app').dataFactory)]", + "location": "[parameters('app').hub.location]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "dependsOn": [ + "dataFactory" + ] + }, + "triggerManagerRoleAssignments": { + "copy": { + "name": "triggerManagerRoleAssignments", + "count": "[length(variables('autoStartRbacRoles'))]" + }, + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]", + "name": "[guid(resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory), variables('autoStartRbacRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', format('{0}_triggerManager', parameters('app').dataFactory)))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('autoStartRbacRoles')[copyIndex()])]", + "principalId": "[reference('triggerManagerIdentity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "triggerManagerIdentity" + ] + }, + "storageAccount": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]", + "location": "[parameters('app').hub.location]", + "sku": { + "name": "[parameters('app').hub.options.storageSku]" + }, + "kind": "BlockBlobStorage", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Storage/storageAccounts')]", + "properties": "[shallowMerge(createArray(variables('storageInfrastructureEncryptionProperties'), createObject('supportsHttpsTrafficOnly', true(), 'allowSharedKeyAccess', true(), 'isHnsEnabled', true(), 'minimumTlsVersion', 'TLS1_2', 'allowBlobPublicAccess', false(), 'publicNetworkAccess', 'Enabled', 'networkAcls', createObject('bypass', 'AzureServices', 'defaultAction', if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')))))]" + }, + "blobPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]" + }, + "blobEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-blob-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "blobLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "blob" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "dfsPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]" + }, + "dfsEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-dfs-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "dfsLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "dfs" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "keyVault": { + "condition": "[variables('usesKeyVault')]", + "type": "Microsoft.KeyVault/vaults", + "apiVersion": "2023-02-01", + "name": "[parameters('app').keyVault]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.KeyVault/vaults')]", + "properties": { + "sku": { + "name": "[parameters('app').hub.options.keyVaultSku]", + "family": "A" + }, + "enabledForDeployment": true, + "enabledForTemplateDeployment": true, + "enabledForDiskEncryption": true, + "enableSoftDelete": true, + "softDeleteRetentionInDays": 90, + "enablePurgeProtection": "[if(parameters('app').hub.options.keyVaultEnablePurgeProtection, true(), null())]", + "enableRbacAuthorization": false, + "createMode": "default", + "tenantId": "[subscription().tenantId]", + "accessPolicies": [ + { + "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "tenantId": "[subscription().tenantId]", + "permissions": { + "secrets": [ + "get" + ] + } + } + ], + "networkAcls": { + "bypass": "AzureServices", + "defaultAction": "[if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')]" + } + }, + "dependsOn": [ + "dataFactory" + ] + }, + "keyVaultPrivateDnsZone": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones')]", + "properties": {} + }, + "keyVaultEndpoint": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-ep', parameters('app').keyVault)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.keyVault]" + }, + "privateLinkServiceConnections": [ + { + "name": "keyVaultLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "groupIds": [ + "vault" + ] + } + } + ] + }, + "dependsOn": [ + "keyVault" + ] + }, + "getKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint", + "getStoragePrivateEndpointConnections", + "keyVault" + ] + }, + "approveKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + }, + "privateEndpointConnections": { + "value": "[reference('getKeyVaultPrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getKeyVaultPrivateEndpointConnections", + "keyVault" + ] + }, + "getStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint", + "stopTriggers", + "storageAccount" + ] + }, + "approveStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + }, + "privateEndpointConnections": { + "value": "[reference('getStoragePrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getStoragePrivateEndpointConnections", + "storageAccount" + ] + }, + "stopTriggers": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.{1}_ADF.StopTriggers', parameters('app').publisher, parameters('app').name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "arguments": { + "value": "[join(createArray(format('-DataFactoryResourceGroup \"{0}\"', resourceGroup().name), format('-DataFactoryName \"{0}\"', parameters('app').dataFactory), '-StopTriggers'), ' ')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + }, + "dependsOn": [ + "appTelemetry", + "dataFactory", + "triggerManagerIdentity", + "triggerManagerRoleAssignments" + ] + } + }, + "outputs": { + "dataFactoryId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Data Factory instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]" + }, + "keyVaultId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Key Vault instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]" + }, + "storageAccountId": { + "type": "string", + "metadata": { + "description": "Resource ID of the storage account instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Principal ID for the managed identity used by Data Factory." + }, + "value": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" + }, + "triggerManagerIdentityName": { + "type": "string", + "metadata": { + "description": "Name of the managed identity used to create and stop ADF triggers." + }, + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + } + } + } } }, - { - "type": "Microsoft.Authorization/roleAssignments", - "apiVersion": "2022-04-01", - "scope": "[format('Microsoft.Storage/storageAccounts/{0}', parameters('storageAccountName'))]", - "name": "[guid(parameters('clusterName'), subscription().id, 'Storage Blob Data Contributor')]", - "properties": { - "description": "Give \"Storage Blob Data Contributor\" to the cluster", - "principalId": "[reference(resourceId('Microsoft.Kusto/clusters', parameters('clusterName')), '2023-08-15', 'full').identity.principalId]", - "principalType": "ServicePrincipal", - "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe')]" - }, - "dependsOn": [ - "[resourceId('Microsoft.Kusto/clusters', parameters('clusterName'))]" - ] - }, - { - "condition": "[not(parameters('enablePublicAccess'))]", - "type": "Microsoft.Network/privateDnsZones", - "apiVersion": "2024-06-01", - "name": "[variables('dataExplorerPrivateDnsZoneName')]", - "location": "global", - "tags": "[union(parameters('tags'), coalesce(tryGet(parameters('tagsByResource'), 'Microsoft.Network/privateDnsZones'), createObject()))]", - "properties": {} - }, - { - "condition": "[not(parameters('enablePublicAccess'))]", - "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", - "apiVersion": "2024-06-01", - "name": "[format('{0}/{1}', variables('dataExplorerPrivateDnsZoneName'), format('{0}-link', replace(variables('dataExplorerPrivateDnsZoneName'), '.', '-')))]", - "location": "global", - "tags": "[union(parameters('tags'), coalesce(tryGet(parameters('tagsByResource'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks'), createObject()))]", + "schemaFiles": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.Exports_Storage.SchemaFiles", "properties": { - "virtualNetwork": { - "id": "[parameters('virtualNetworkId')]" + "expressionEvaluationOptions": { + "scope": "inner" }, - "registrationEnabled": false - }, - "dependsOn": [ - "[resourceId('Microsoft.Network/privateDnsZones', variables('dataExplorerPrivateDnsZoneName'))]" - ] - }, - { - "condition": "[not(parameters('enablePublicAccess'))]", - "type": "Microsoft.Network/privateEndpoints", - "apiVersion": "2023-11-01", - "name": "[format('{0}-ep', parameters('clusterName'))]", - "location": "[parameters('location')]", - "tags": "[union(parameters('tags'), coalesce(tryGet(parameters('tagsByResource'), 'Microsoft.Network/privateEndpoints'), createObject()))]", - "properties": { - "subnet": { - "id": "[parameters('privateEndpointSubnetId')]" + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "container": { + "value": "config" + }, + "files": { + "value": { + "schemas/actualcost_c360-2025-04.json": "[variables('$fxv#0')]", + "schemas/amortizedcost_c360-2025-04.json": "[variables('$fxv#1')]", + "schemas/focuscost_1.2.json": "[variables('$fxv#2')]", + "schemas/focuscost_1.2-preview.json": "[variables('$fxv#3')]", + "schemas/focuscost_1.0r2.json": "[variables('$fxv#4')]", + "schemas/focuscost_1.0.json": "[variables('$fxv#5')]", + "schemas/focuscost_1.0-preview(v1).json": "[variables('$fxv#6')]", + "schemas/pricesheet_2023-05-01_ea.json": "[variables('$fxv#7')]", + "schemas/pricesheet_2023-05-01_mca.json": "[variables('$fxv#8')]", + "schemas/reservationdetails_2023-03-01.json": "[variables('$fxv#9')]", + "schemas/reservationrecommendations_2023-05-01_ea.json": "[variables('$fxv#10')]", + "schemas/reservationrecommendations_2023-05-01_mca.json": "[variables('$fxv#11')]", + "schemas/reservationtransactions_2023-05-01_ea.json": "[variables('$fxv#12')]", + "schemas/reservationtransactions_2023-05-01_mca.json": "[variables('$fxv#13')]" + } + } }, - "privateLinkServiceConnections": [ - { - "name": "dataExplorerLink", - "properties": { - "privateLinkServiceId": "[resourceId('Microsoft.Kusto/clusters', parameters('clusterName'))]", - "groupIds": [ - "cluster" + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6646280599480816180" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app that storage is getting updated for." + } + }, + "container": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage container to create or update." + } + }, + "files": { + "type": "object", + "defaultValue": {}, + "metadata": { + "description": "Optional. Dictionary of key/value pairs for the files to upload to the specified container. The key is the target path under the container and the value is the contents of the file. Default: {} (no files to upload)." + } + }, + "forceCreateBlobManagerIdentity": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Indicates whether to create the blob manager user assigned identity even if files are not being uploaded. Default: false." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n# Get storage context\r\n$storageContext = @{\r\n Context = New-AzStorageContext -StorageAccountName $env:storageAccountName -UseConnectedAccount\r\n Container = $env:containerName\r\n}\r\n\r\n# Uploading files\r\n$files = $env:files | ConvertFrom-Json -Depth 10\r\nWrite-Output \"Uploading ${$files.PSObject.Properties.Count} files...\"\r\n$files.PSObject.Properties | ForEach-Object {\r\n $filePath = $_.Name\r\n $tempPath = \"./$($filePath -replace \"/\", \"_\")\"\r\n Write-Output \" Uploading $filePath...\"\r\n $_.Value | Out-File $tempPath\r\n Set-AzStorageBlobContent @storageContext -File $tempPath -Blob $filePath -Force | Out-Null\r\n}\r\n", + "fileCount": "[length(items(parameters('files')))]", + "hasFiles": "[greater(variables('fileCount'), 0)]" + }, + "resources": { + "storageAccount::blobService::targetContainer": { + "type": "Microsoft.Storage/storageAccounts/blobServices/containers", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}/{2}', parameters('app').storage, 'default', parameters('container'))]", + "properties": { + "publicAccess": "None", + "metadata": {} + } + }, + "storageAccount::blobService": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]" + }, + "storageAccount": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]" + }, + "identity": { + "condition": "[or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity'))]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.Identity', deployment().name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_blobManager', parameters('app').storage)]" + }, + "roleAssignmentResourceId": { + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "roles": { + "value": [ + "ba92f5b4-2d11-453d-a403-e96b0029c9fe", + "69566ab7-960f-475b-8e7c-b3118f30c6bd" + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "18138592202204453545" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the identity is associated with." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the user assigned identity." + } + }, + "roleAssignmentResourceId": { + "type": "string", + "metadata": { + "description": "Required. Resource ID of the resource access is being granted for." + } + }, + "roles": { + "type": "array", + "items": { + "type": "string" + }, + "metadata": { + "description": "Required. List of RBAC role assignment GUIDs." + } + } + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.ManagedIdentity/userAssignedIdentities')]", + "location": "[parameters('app').hub.location]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(parameters('roles'))]" + }, + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "name": "[guid(parameters('roleAssignmentResourceId'), parameters('roles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', parameters('roles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + } + }, + "outputs": { + "id": { + "type": "string", + "metadata": { + "description": "Resource ID of the user assigned identity." + }, + "value": "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName'))]" + }, + "name": { + "type": "string", + "metadata": { + "description": "Name of the user assigned identity." + }, + "value": "[parameters('identityName')]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Principal ID of the user assigned identity." + }, + "value": "[reference('identity').principalId]" + } + } + } + } + }, + "uploadFiles": { + "condition": "[variables('hasFiles')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.Upload', deployment().name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[reference('identity').outputs.name.value]" + }, + "environmentVariables": { + "value": [ + { + "name": "storageAccountName", + "value": "[parameters('app').storage]" + }, + { + "name": "containerName", + "value": "[parameters('container')]" + }, + { + "name": "files", + "value": "[string(parameters('files'))]" + } + ] + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + }, + "dependsOn": [ + "identity" ] } - } - ] - }, - "dependsOn": [ - "[resourceId('Microsoft.Kusto/clusters', parameters('clusterName'))]" - ] - }, - { - "condition": "[not(parameters('enablePublicAccess'))]", - "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", - "apiVersion": "2023-11-01", - "name": "[format('{0}/{1}', format('{0}-ep', parameters('clusterName')), 'dataExplorer-endpoint-zone')]", - "properties": { - "privateDnsZoneConfigs": [ - { - "name": "privatelink-westus-kusto-net", - "properties": { - "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', variables('dataExplorerPrivateDnsZoneName'))]" - } - }, - { - "name": "privatelink-blob-core-windows-net", - "properties": { - "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" - } - }, - { - "name": "privatelink-table-core-windows-net", - "properties": { - "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.table.{0}', environment().suffixes.storage))]" - } }, - { - "name": "privatelink-queue-core-windows-net", - "properties": { - "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.queue.{0}', environment().suffixes.storage))]" + "outputs": { + "containerName": { + "type": "string", + "metadata": { + "description": "The name of the storage container." + }, + "value": "[parameters('container')]" + }, + "filesUploaded": { + "type": "int", + "metadata": { + "description": "The number of files uploaded to the storage container." + }, + "value": "[variables('fileCount')]" + }, + "identityId": { + "type": "string", + "metadata": { + "description": "Resource ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.id.value, '')]" + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Name of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.name.value, '')]" + }, + "identityPrincipalId": { + "type": "string", + "metadata": { + "description": "Principal ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.principalId.value, '')]" } } - ] + } }, "dependsOn": [ - "[resourceId('Microsoft.Network/privateEndpoints', format('{0}-ep', parameters('clusterName')))]", - "[resourceId('Microsoft.Network/privateDnsZones', variables('dataExplorerPrivateDnsZoneName'))]" + "appRegistration" ] }, - { + "exportContainer": { "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "ingestion_OpenDataInternalScripts", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.Exports_Storage.ExportContainer", "properties": { "expressionEvaluationOptions": { "scope": "inner" }, "mode": "Incremental", "parameters": { - "clusterName": { - "value": "[parameters('clusterName')]" - }, - "databaseName": { - "value": "Ingestion" - }, - "scripts": { - "value": { - "OpenDataFunctions_resource_type_1": "[variables('$fxv#0')]", - "OpenDataFunctions_resource_type_2": "[variables('$fxv#1')]", - "OpenDataFunctions_resource_type_3": "[variables('$fxv#2')]", - "OpenDataFunctions_resource_type_4": "[variables('$fxv#3')]", - "OpenDataFunctions_resource_type_5": "[variables('$fxv#4')]" - } - }, - "continueOnErrors": { - "value": "[parameters('continueOnErrors')]" + "app": { + "value": "[parameters('app')]" }, - "forceUpdateTag": { - "value": "[parameters('forceUpdateTag')]" + "container": { + "value": "[variables('MSEXPORTS')]" } }, "template": { "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", "contentVersion": "1.0.0.0", "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "9449675944348204569" + "version": "0.40.2.10011", + "templateHash": "6646280599480816180" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } } }, "parameters": { - "clusterName": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app that storage is getting updated for." + } + }, + "container": { "type": "string", "metadata": { - "description": "Required. Name of the FinOps hub Data Explorer instance." + "description": "Required. Name of the storage container to create or update." + } + }, + "files": { + "type": "object", + "defaultValue": {}, + "metadata": { + "description": "Optional. Dictionary of key/value pairs for the files to upload to the specified container. The key is the target path under the container and the value is the contents of the file. Default: {} (no files to upload)." + } + }, + "forceCreateBlobManagerIdentity": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Indicates whether to create the blob manager user assigned identity even if files are not being uploaded. Default: false." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n# Get storage context\r\n$storageContext = @{\r\n Context = New-AzStorageContext -StorageAccountName $env:storageAccountName -UseConnectedAccount\r\n Container = $env:containerName\r\n}\r\n\r\n# Uploading files\r\n$files = $env:files | ConvertFrom-Json -Depth 10\r\nWrite-Output \"Uploading ${$files.PSObject.Properties.Count} files...\"\r\n$files.PSObject.Properties | ForEach-Object {\r\n $filePath = $_.Name\r\n $tempPath = \"./$($filePath -replace \"/\", \"_\")\"\r\n Write-Output \" Uploading $filePath...\"\r\n $_.Value | Out-File $tempPath\r\n Set-AzStorageBlobContent @storageContext -File $tempPath -Blob $filePath -Force | Out-Null\r\n}\r\n", + "fileCount": "[length(items(parameters('files')))]", + "hasFiles": "[greater(variables('fileCount'), 0)]" + }, + "resources": { + "storageAccount::blobService::targetContainer": { + "type": "Microsoft.Storage/storageAccounts/blobServices/containers", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}/{2}', parameters('app').storage, 'default', parameters('container'))]", + "properties": { + "publicAccess": "None", + "metadata": {} } }, - "databaseName": { - "type": "string", - "metadata": { - "description": "Required. Name of the FinOps hub Data Explorer database to create or update." - } + "storageAccount::blobService": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]" }, - "scripts": { - "type": "object", - "metadata": { - "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." - } + "storageAccount": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]" }, - "continueOnErrors": { - "type": "bool", - "defaultValue": false, - "metadata": { - "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + "identity": { + "condition": "[or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity'))]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.Identity', deployment().name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_blobManager', parameters('app').storage)]" + }, + "roleAssignmentResourceId": { + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "roles": { + "value": [ + "ba92f5b4-2d11-453d-a403-e96b0029c9fe", + "69566ab7-960f-475b-8e7c-b3118f30c6bd" + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "18138592202204453545" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the identity is associated with." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the user assigned identity." + } + }, + "roleAssignmentResourceId": { + "type": "string", + "metadata": { + "description": "Required. Resource ID of the resource access is being granted for." + } + }, + "roles": { + "type": "array", + "items": { + "type": "string" + }, + "metadata": { + "description": "Required. List of RBAC role assignment GUIDs." + } + } + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.ManagedIdentity/userAssignedIdentities')]", + "location": "[parameters('app').hub.location]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(parameters('roles'))]" + }, + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "name": "[guid(parameters('roleAssignmentResourceId'), parameters('roles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', parameters('roles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + } + }, + "outputs": { + "id": { + "type": "string", + "metadata": { + "description": "Resource ID of the user assigned identity." + }, + "value": "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName'))]" + }, + "name": { + "type": "string", + "metadata": { + "description": "Name of the user assigned identity." + }, + "value": "[parameters('identityName')]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Principal ID of the user assigned identity." + }, + "value": "[reference('identity').principalId]" + } + } + } } }, - "forceUpdateTag": { - "type": "string", - "defaultValue": "[utcNow()]", - "metadata": { - "description": "Optional. Forces the table to be updated if different from the last time it was deployed." - } - } - }, - "resources": [ - { - "copy": { - "name": "cluster::database::script", - "count": "[length(items(parameters('scripts')))]" - }, - "type": "Microsoft.Kusto/clusters/databases/scripts", - "apiVersion": "2023-08-15", - "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", + "uploadFiles": { + "condition": "[variables('hasFiles')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.Upload', deployment().name)]", "properties": { - "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", - "continueOnErrors": "[parameters('continueOnErrors')]", - "forceUpdateTag": "[parameters('forceUpdateTag')]" - } - } - ] - } - }, - "dependsOn": [ - "[resourceId('Microsoft.Kusto/clusters', parameters('clusterName'))]", - "[resourceId('Microsoft.Kusto/clusters/databases', parameters('clusterName'), 'Ingestion')]" - ] - }, - { - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "ingestion_InitScripts", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "clusterName": { - "value": "[parameters('clusterName')]" - }, - "databaseName": { - "value": "Ingestion" - }, - "scripts": { - "value": { - "openData": "[variables('$fxv#5')]", - "common": "[variables('$fxv#6')]", - "infra": "[variables('$fxv#7')]", - "rawTables": "[replace(variables('$fxv#8'), '$$rawRetentionInDays$$', string(parameters('rawRetentionInDays')))]" - } - }, - "continueOnErrors": { - "value": "[parameters('continueOnErrors')]" - }, - "forceUpdateTag": { - "value": "[parameters('forceUpdateTag')]" - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "9449675944348204569" - } - }, - "parameters": { - "clusterName": { - "type": "string", - "metadata": { - "description": "Required. Name of the FinOps hub Data Explorer instance." - } - }, - "databaseName": { - "type": "string", - "metadata": { - "description": "Required. Name of the FinOps hub Data Explorer database to create or update." - } - }, - "scripts": { - "type": "object", - "metadata": { - "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." - } - }, - "continueOnErrors": { - "type": "bool", - "defaultValue": false, - "metadata": { - "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." - } - }, - "forceUpdateTag": { - "type": "string", - "defaultValue": "[utcNow()]", - "metadata": { - "description": "Optional. Forces the table to be updated if different from the last time it was deployed." - } - } - }, - "resources": [ - { - "copy": { - "name": "cluster::database::script", - "count": "[length(items(parameters('scripts')))]" + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[reference('identity').outputs.name.value]" + }, + "environmentVariables": { + "value": [ + { + "name": "storageAccountName", + "value": "[parameters('app').storage]" + }, + { + "name": "containerName", + "value": "[parameters('container')]" + }, + { + "name": "files", + "value": "[string(parameters('files'))]" + } + ] + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } }, - "type": "Microsoft.Kusto/clusters/databases/scripts", - "apiVersion": "2023-08-15", - "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", - "properties": { - "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", - "continueOnErrors": "[parameters('continueOnErrors')]", - "forceUpdateTag": "[parameters('forceUpdateTag')]" - } - } - ] - } - }, - "dependsOn": [ - "[resourceId('Microsoft.Kusto/clusters', parameters('clusterName'))]", - "[resourceId('Microsoft.Kusto/clusters/databases', parameters('clusterName'), 'Ingestion')]", - "[resourceId('Microsoft.Resources/deployments', 'ingestion_OpenDataInternalScripts')]" - ] - }, - { - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "ingestion_VersionedScripts", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "clusterName": { - "value": "[parameters('clusterName')]" - }, - "databaseName": { - "value": "Ingestion" - }, - "scripts": { - "value": { - "v1_0": "[variables('$fxv#9')]", - "v1_2": "[variables('$fxv#10')]" - } - }, - "continueOnErrors": { - "value": "[parameters('continueOnErrors')]" - }, - "forceUpdateTag": { - "value": "[parameters('forceUpdateTag')]" - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "9449675944348204569" + "dependsOn": [ + "identity" + ] } }, - "parameters": { - "clusterName": { - "type": "string", - "metadata": { - "description": "Required. Name of the FinOps hub Data Explorer instance." - } - }, - "databaseName": { - "type": "string", - "metadata": { - "description": "Required. Name of the FinOps hub Data Explorer database to create or update." - } - }, - "scripts": { - "type": "object", - "metadata": { - "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." - } - }, - "continueOnErrors": { - "type": "bool", - "defaultValue": false, - "metadata": { - "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." - } - }, - "forceUpdateTag": { + "outputs": { + "containerName": { "type": "string", - "defaultValue": "[utcNow()]", "metadata": { - "description": "Optional. Forces the table to be updated if different from the last time it was deployed." - } - } - }, - "resources": [ - { - "copy": { - "name": "cluster::database::script", - "count": "[length(items(parameters('scripts')))]" + "description": "The name of the storage container." }, - "type": "Microsoft.Kusto/clusters/databases/scripts", - "apiVersion": "2023-08-15", - "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", - "properties": { - "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", - "continueOnErrors": "[parameters('continueOnErrors')]", - "forceUpdateTag": "[parameters('forceUpdateTag')]" - } - } - ] - } - }, - "dependsOn": [ - "[resourceId('Microsoft.Kusto/clusters', parameters('clusterName'))]", - "[resourceId('Microsoft.Kusto/clusters/databases', parameters('clusterName'), 'Ingestion')]", - "[resourceId('Microsoft.Resources/deployments', 'ingestion_InitScripts')]" - ] - }, - { - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "hub_InitScripts", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "clusterName": { - "value": "[parameters('clusterName')]" - }, - "databaseName": { - "value": "Hub" - }, - "scripts": { - "value": { - "common": "[variables('$fxv#11')]", - "openData": "[variables('$fxv#12')]" - } - }, - "continueOnErrors": { - "value": "[parameters('continueOnErrors')]" - }, - "forceUpdateTag": { - "value": "[parameters('forceUpdateTag')]" - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "9449675944348204569" - } - }, - "parameters": { - "clusterName": { - "type": "string", - "metadata": { - "description": "Required. Name of the FinOps hub Data Explorer instance." - } + "value": "[parameters('container')]" }, - "databaseName": { - "type": "string", + "filesUploaded": { + "type": "int", "metadata": { - "description": "Required. Name of the FinOps hub Data Explorer database to create or update." - } + "description": "The number of files uploaded to the storage container." + }, + "value": "[variables('fileCount')]" }, - "scripts": { - "type": "object", + "identityId": { + "type": "string", "metadata": { - "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." - } + "description": "Resource ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.id.value, '')]" }, - "continueOnErrors": { - "type": "bool", - "defaultValue": false, + "identityName": { + "type": "string", "metadata": { - "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." - } + "description": "Name of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." + }, + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.name.value, '')]" }, - "forceUpdateTag": { + "identityPrincipalId": { "type": "string", - "defaultValue": "[utcNow()]", "metadata": { - "description": "Optional. Forces the table to be updated if different from the last time it was deployed." - } - } - }, - "resources": [ - { - "copy": { - "name": "cluster::database::script", - "count": "[length(items(parameters('scripts')))]" + "description": "Principal ID of the user assigned identity used to upload files. Will be empty if no files are uploaded or forceCreateBlobManagerIdentity is false." }, - "type": "Microsoft.Kusto/clusters/databases/scripts", - "apiVersion": "2023-08-15", - "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", - "properties": { - "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", - "continueOnErrors": "[parameters('continueOnErrors')]", - "forceUpdateTag": "[parameters('forceUpdateTag')]" - } + "value": "[if(or(variables('hasFiles'), parameters('forceCreateBlobManagerIdentity')), reference('identity').outputs.principalId.value, '')]" } - ] + } } }, "dependsOn": [ - "[resourceId('Microsoft.Kusto/clusters', parameters('clusterName'))]", - "[resourceId('Microsoft.Kusto/clusters/databases', parameters('clusterName'), 'Hub')]", - "[resourceId('Microsoft.Resources/deployments', 'ingestion_InitScripts')]" + "appRegistration" ] }, - { + "trigger_ExportManifestAdded": { "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "hub_VersionedScripts", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.Exports_ADF.ExportManifestTrigger", "properties": { "expressionEvaluationOptions": { "scope": "inner" }, "mode": "Incremental", "parameters": { - "clusterName": { - "value": "[parameters('clusterName')]" + "dataFactoryName": { + "value": "[parameters('app').dataFactory]" }, - "databaseName": { - "value": "Hub" + "triggerName": { + "value": "[format('{0}_ManifestAdded', variables('MSEXPORTS'))]" }, - "scripts": { + "pipelineName": { + "value": "[format('{0}_ExecuteETL', variables('MSEXPORTS'))]" + }, + "pipelineParameters": { "value": { - "v1_0": "[variables('$fxv#13')]", - "v1_2": "[variables('$fxv#14')]" + "folderPath": "@triggerBody().folderPath", + "fileName": "@triggerBody().fileName" } }, - "continueOnErrors": { - "value": "[parameters('continueOnErrors')]" + "storageAccountName": { + "value": "[parameters('app').storage]" }, - "forceUpdateTag": { - "value": "[parameters('forceUpdateTag')]" + "storageContainer": { + "value": "[variables('MSEXPORTS')]" + }, + "storagePathEndsWith": { + "value": "manifest.json" } }, "template": { @@ -10075,228 +12387,136 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "9449675944348204569" + "version": "0.40.2.10011", + "templateHash": "12667288354191065454" } }, "parameters": { - "clusterName": { + "dataFactoryName": { "type": "string", "metadata": { - "description": "Required. Name of the FinOps hub Data Explorer instance." + "description": "Required. Name of the publisher-specific Data Factory instance." } }, - "databaseName": { + "triggerName": { "type": "string", "metadata": { - "description": "Required. Name of the FinOps hub Data Explorer database to create or update." + "description": "Required. Name of the Data Factory trigger to create or update." } }, - "scripts": { - "type": "object", + "storageAccountName": { + "type": "string", + "defaultValue": "", "metadata": { - "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." + "description": "Optional. Azure storage container to monitor for updates and trigger events for." } }, - "continueOnErrors": { - "type": "bool", - "defaultValue": false, + "storageContainer": { + "type": "string", + "defaultValue": "", "metadata": { - "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + "description": "Optional. Azure storage container to monitor for updates and trigger events for." } }, - "forceUpdateTag": { + "storagePathStartsWith": { "type": "string", - "defaultValue": "[utcNow()]", + "defaultValue": "", "metadata": { - "description": "Optional. Forces the table to be updated if different from the last time it was deployed." - } - } - }, - "resources": [ - { - "copy": { - "name": "cluster::database::script", - "count": "[length(items(parameters('scripts')))]" - }, - "type": "Microsoft.Kusto/clusters/databases/scripts", - "apiVersion": "2023-08-15", - "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", - "properties": { - "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", - "continueOnErrors": "[parameters('continueOnErrors')]", - "forceUpdateTag": "[parameters('forceUpdateTag')]" + "description": "Optional. Beginning of the storage path within the specified storageContainer to monitor for updates and trigger events for." } - } - ] - } - }, - "dependsOn": [ - "[resourceId('Microsoft.Kusto/clusters', parameters('clusterName'))]", - "[resourceId('Microsoft.Kusto/clusters/databases', parameters('clusterName'), 'Hub')]", - "[resourceId('Microsoft.Resources/deployments', 'hub_InitScripts')]", - "[resourceId('Microsoft.Resources/deployments', 'ingestion_VersionedScripts')]" - ] - }, - { - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "hub_LatestScripts", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "clusterName": { - "value": "[parameters('clusterName')]" - }, - "databaseName": { - "value": "Hub" - }, - "scripts": { - "value": { - "latest": "[variables('$fxv#15')]" - } - }, - "continueOnErrors": { - "value": "[parameters('continueOnErrors')]" - }, - "forceUpdateTag": { - "value": "[parameters('forceUpdateTag')]" - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "9449675944348204569" - } - }, - "parameters": { - "clusterName": { + }, + "storagePathEndsWith": { "type": "string", + "defaultValue": "", "metadata": { - "description": "Required. Name of the FinOps hub Data Explorer instance." + "description": "Optional. End of the storage path to monitor for updates and trigger events for." } }, - "databaseName": { + "pipelineName": { "type": "string", "metadata": { - "description": "Required. Name of the FinOps hub Data Explorer database to create or update." + "description": "Required. Name of the Data Factory pipeline to execute when the trigger is executed." } }, - "scripts": { + "pipelineParameters": { "type": "object", "metadata": { - "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." - } - }, - "continueOnErrors": { - "type": "bool", - "defaultValue": false, - "metadata": { - "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." - } - }, - "forceUpdateTag": { - "type": "string", - "defaultValue": "[utcNow()]", - "metadata": { - "description": "Optional. Forces the table to be updated if different from the last time it was deployed." + "description": "Required. Parameters to pass to the pipeline when the trigger is executed." } } }, "resources": [ { - "copy": { - "name": "cluster::database::script", - "count": "[length(items(parameters('scripts')))]" - }, - "type": "Microsoft.Kusto/clusters/databases/scripts", - "apiVersion": "2023-08-15", - "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", + "condition": "[not(empty(parameters('storageAccountName')))]", + "type": "Microsoft.DataFactory/factories/triggers", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('dataFactoryName'), parameters('triggerName'))]", "properties": { - "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", - "continueOnErrors": "[parameters('continueOnErrors')]", - "forceUpdateTag": "[parameters('forceUpdateTag')]" + "annotations": [], + "pipelines": [ + { + "pipelineReference": { + "referenceName": "[parameters('pipelineName')]", + "type": "PipelineReference" + }, + "parameters": "[parameters('pipelineParameters')]" + } + ], + "type": "BlobEventsTrigger", + "typeProperties": { + "blobPathBeginsWith": "[format('/{0}/blobs/{1}', parameters('storageContainer'), parameters('storagePathStartsWith'))]", + "blobPathEndsWith": "[parameters('storagePathEndsWith')]", + "ignoreEmptyBlobs": true, + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]", + "events": [ + "Microsoft.Storage.BlobCreated" + ] + } } } ] } }, "dependsOn": [ - "[resourceId('Microsoft.Kusto/clusters', parameters('clusterName'))]", - "[resourceId('Microsoft.Kusto/clusters/databases', parameters('clusterName'), 'Hub')]", - "[resourceId('Microsoft.Resources/deployments', 'hub_VersionedScripts')]" + "appRegistration", + "dataFactory::pipeline_ExecuteExportsETL" ] } - ], + }, "outputs": { - "clusterId": { - "type": "string", - "metadata": { - "description": "The resource ID of the cluster." - }, - "value": "[resourceId('Microsoft.Kusto/clusters', parameters('clusterName'))]" - }, - "principalId": { - "type": "string", - "metadata": { - "description": "The ID of the cluster system assigned managed identity." - }, - "value": "[reference(resourceId('Microsoft.Kusto/clusters', parameters('clusterName')), '2023-08-15', 'full').identity.principalId]" - }, - "clusterName": { - "type": "string", - "metadata": { - "description": "The name of the cluster." - }, - "value": "[parameters('clusterName')]" - }, - "clusterUri": { - "type": "string", - "metadata": { - "description": "The URI of the cluster." - }, - "value": "[reference(resourceId('Microsoft.Kusto/clusters', parameters('clusterName')), '2023-08-15').uri]" - }, - "ingestionDbName": { - "type": "string", + "app": { + "$ref": "#/definitions/HubAppProperties", "metadata": { - "description": "The name of the database for data ingestion." + "description": "Properties of the hub app." }, - "value": "Ingestion" + "value": "[parameters('app')]" }, - "hubDbName": { + "exportContainer": { "type": "string", "metadata": { - "description": "The name of the database for queries." + "description": "Name of the container used for Cost Management exports." }, - "value": "Hub" + "value": "[reference('exportContainer').outputs.containerName.value]" }, - "clusterIngestionCapacity": { + "schemaFilesUploaded": { "type": "int", "metadata": { - "description": "Max ingestion capacity of the cluster." + "description": "Number of schema files uploaded." }, - "value": "[coalesce(tryGet(variables('ingestionCapacity'), parameters('clusterSku')), 1)]" + "value": "[reference('schemaFiles').outputs.filesUploaded.value]" } } } }, "dependsOn": [ - "core", - "infrastructure" + "core" ] }, - "dataFactoryResources": { + "cmManagedExports": { + "condition": "[parameters('enableManagedExports')]", "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "dataFactoryResources", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.ManagedExports", "properties": { "expressionEvaluationOptions": { "scope": "inner" @@ -10304,50 +12524,7 @@ "mode": "Incremental", "parameters": { "app": { - "value": "[__bicep.newApp(variables('hub'), 'Microsoft FinOps hubs', 'Microsoft.FinOpsHubs', 'DataFactory', 'FinOps hub engine', variables('$fxv#1'))]" - }, - "hubName": { - "value": "[parameters('hubName')]" - }, - "dataFactoryName": { - "value": "[reference('core').outputs.dataFactoryName.value]" - }, - "location": { - "value": "[parameters('location')]" - }, - "tags": { - "value": "[reference('core').outputs.publisherTags.value]" - }, - "tagsByResource": { - "value": "[parameters('tagsByResource')]" - }, - "storageAccountName": { - "value": "[reference('core').outputs.storageAccountName.value]" - }, - "exportContainerName": { - "value": "[reference('cmExports').outputs.exportContainer.value]" - }, - "configContainerName": { - "value": "[reference('core').outputs.configContainer.value]" - }, - "ingestionContainerName": { - "value": "[reference('core').outputs.ingestionContainer.value]" - }, - "dataExplorerName": "[if(not(variables('deployDataExplorer')), createObject('value', ''), createObject('value', reference('dataExplorer').outputs.clusterName.value))]", - "dataExplorerPrincipalId": "[if(not(variables('deployDataExplorer')), createObject('value', ''), createObject('value', reference('dataExplorer').outputs.principalId.value))]", - "dataExplorerIngestionDatabase": "[if(variables('useFabric'), createObject('value', 'Ingestion'), if(not(variables('deployDataExplorer')), createObject('value', ''), createObject('value', reference('dataExplorer').outputs.ingestionDbName.value)))]", - "dataExplorerIngestionCapacity": "[if(variables('useFabric'), createObject('value', parameters('fabricCapacityUnits')), if(not(variables('deployDataExplorer')), createObject('value', 1), createObject('value', reference('dataExplorer').outputs.clusterIngestionCapacity.value)))]", - "dataExplorerUri": "[if(variables('useFabric'), createObject('value', parameters('fabricQueryUri')), if(not(variables('deployDataExplorer')), createObject('value', ''), createObject('value', reference('dataExplorer').outputs.clusterUri.value)))]", - "dataExplorerId": "[if(not(variables('deployDataExplorer')), createObject('value', ''), createObject('value', reference('dataExplorer').outputs.clusterId.value))]", - "enableManagedExports": { - "value": "[parameters('enableManagedExports')]" - }, - "enablePublicAccess": { - "value": "[parameters('enablePublicAccess')]" - }, - "keyVaultName": "[if(empty(parameters('remoteHubStorageKey')), createObject('value', ''), createObject('value', reference('remoteHub').outputs.keyVaultName.value))]", - "remoteHubStorageUri": { - "value": "[parameters('remoteHubStorageUri')]" + "value": "[__bicep.newApp(variables('hub'), 'Microsoft.CostManagement', 'ManagedExports')]" } }, "template": { @@ -10357,8 +12534,8 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "11163540491967572356" + "version": "0.40.2.10011", + "templateHash": "17365536975648713074" } }, "definitions": { @@ -10383,2514 +12560,4640 @@ "version": { "type": "string" }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getExportBody": { + "parameters": [ + { + "type": "string", + "name": "exportContainerName" + }, + { + "type": "string", + "name": "datasetType" + }, + { + "type": "string", + "name": "schemaVersion" + }, + { + "type": "bool", + "name": "isMonthly" + }, + { + "type": "string", + "name": "exportFormat" + }, + { + "type": "string", + "name": "compressionMode" + }, + { + "type": "string", + "name": "partitionData" + }, + { + "type": "string", + "name": "dataOverwriteBehavior" + } + ], + "output": { + "type": "string", + "value": "[format('{{ \"properties\": {{ \"definition\": {{ \"dataSet\": {{ \"configuration\": {{ \"dataVersion\": \"{0}\", \"filters\": [] }}, \"granularity\": \"Daily\" }}, \"timeframe\": \"{1}\", \"type\": \"{2}\" }}, \"deliveryInfo\": {{ \"destination\": {{ \"container\": \"{3}\", \"rootFolderPath\": \"@{{if(startswith(item().scope, ''/''), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}}\", \"type\": \"AzureBlob\", \"resourceId\": \"@{{variables(''storageAccountId'')}}\" }} }}, \"schedule\": {{ \"recurrence\": \"{4}\", \"recurrencePeriod\": {{ \"from\": \"2024-01-01T00:00:00.000Z\", \"to\": \"2050-02-01T00:00:00.000Z\" }}, \"status\": \"Inactive\" }}, \"format\": \"{5}\", \"partitionData\": \"{6}\", \"dataOverwriteBehavior\": \"{7}\", \"compressionMode\": \"{8}\" }}, \"id\": \"@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{variables(''exportName'')}}\", \"name\": \"@{{variables(''exportName'')}}\", \"type\": \"Microsoft.CostManagement/reports\", \"identity\": {{ \"type\": \"systemAssigned\" }}, \"location\": \"global\" }}', parameters('schemaVersion'), if(parameters('isMonthly'), 'TheLastMonth', 'MonthToDate'), parameters('datasetType'), parameters('exportContainerName'), if(parameters('isMonthly'), 'Monthly', 'Daily'), parameters('exportFormat'), parameters('partitionData'), parameters('dataOverwriteBehavior'), parameters('compressionMode'))]" + } + }, + "getExportBodyV2": { + "parameters": [ + { + "type": "string", + "name": "exportContainerName" }, - "keyVaultSku": { - "type": "string" + { + "type": "string", + "name": "datasetType" }, - "networkAddressPrefix": { - "type": "string" + { + "type": "bool", + "name": "isMonthly" }, - "privateRouting": { - "type": "bool" + { + "type": "string", + "name": "exportFormat" }, - "publisherIsolation": { - "type": "bool" + { + "type": "string", + "name": "compressionMode" }, - "storageInfrastructureEncryption": { - "type": "bool" + { + "type": "string", + "name": "partitionData" }, - "storageSku": { - "type": "string" - } - } - }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" - }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" + { + "type": "string", + "name": "dataOverwriteBehavior" + }, + { + "type": "string", + "name": "recommendationScope" + }, + { + "type": "string", + "name": "recommendationLookbackPeriod" + }, + { + "type": "string", + "name": "resourceType" } + ], + "output": { + "type": "string", + "value": "[if(equals(toLower(parameters('datasetType')), 'focuscost'), format('{{ \"properties\": {{ \"definition\": {{ \"dataSet\": {{ \"configuration\": {{ \"dataVersion\": \"{0}\", \"filters\": [] }}, \"granularity\": \"Daily\" }}, \"timeframe\": \"{1}\", \"type\": \"{2}\" }}, \"deliveryInfo\": {{ \"destination\": {{ \"container\": \"{3}\", \"rootFolderPath\": \"@{{if(startswith(item().scope, ''/''), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}}\", \"type\": \"AzureBlob\", \"resourceId\": \"@{{variables(''storageAccountId'')}}\" }} }}, \"schedule\": {{ \"recurrence\": \"{4}\", \"recurrencePeriod\": {{ \"from\": \"2024-01-01T00:00:00.000Z\", \"to\": \"2050-02-01T00:00:00.000Z\" }}, \"status\": \"Inactive\" }}, \"format\": \"{5}\", \"partitionData\": \"{6}\", \"dataOverwriteBehavior\": \"{7}\", \"compressionMode\": \"{8}\" }}, \"id\": \"@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-{9}-costdetails''))}}\", \"name\": \"@{{toLower(concat(variables(''finOpsHub''), ''-{10}-costdetails''))}}\", \"type\": \"Microsoft.CostManagement/reports\", \"identity\": {{ \"type\": \"systemAssigned\" }}, \"location\": \"global\" }}', variables('exportDataVersions')[toLower(parameters('datasetType'))], if(parameters('isMonthly'), 'TheLastMonth', 'MonthToDate'), parameters('datasetType'), parameters('exportContainerName'), if(parameters('isMonthly'), 'Monthly', 'Daily'), parameters('exportFormat'), parameters('partitionData'), parameters('dataOverwriteBehavior'), parameters('compressionMode'), if(parameters('isMonthly'), 'monthly', 'daily'), if(parameters('isMonthly'), 'monthly', 'daily')), if(equals(toLower(parameters('datasetType')), 'reservationdetails'), format('{{ \"properties\": {{ \"definition\": {{ \"dataSet\": {{ \"configuration\": {{ \"dataVersion\": \"{0}\", \"filters\": [] }}, \"granularity\": \"Daily\" }}, \"timeframe\": \"{1}\", \"type\": \"{2}\" }}, \"deliveryInfo\": {{ \"destination\": {{ \"container\": \"{3}\", \"rootFolderPath\": \"@{{if(startswith(item().scope, ''/''), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}}\", \"type\": \"AzureBlob\", \"resourceId\": \"@{{variables(''storageAccountId'')}}\" }} }}, \"schedule\": {{ \"recurrence\": \"{4}\", \"recurrencePeriod\": {{ \"from\": \"2024-01-01T00:00:00.000Z\", \"to\": \"2050-02-01T00:00:00.000Z\" }}, \"status\": \"Inactive\" }}, \"format\": \"{5}\", \"partitionData\": \"{6}\", \"dataOverwriteBehavior\": \"{7}\", \"compressionMode\": \"{8}\" }}, \"id\": \"@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-{9}-{10}''))}}\", \"name\": \"@{{toLower(concat(variables(''finOpsHub''), ''-{11}-{12}''))}}\", \"type\": \"Microsoft.CostManagement/reports\", \"identity\": {{ \"type\": \"systemAssigned\" }}, \"location\": \"global\" }}', variables('exportDataVersions')[toLower(parameters('datasetType'))], if(parameters('isMonthly'), 'TheLastMonth', 'MonthToDate'), parameters('datasetType'), parameters('exportContainerName'), if(parameters('isMonthly'), 'Monthly', 'Daily'), parameters('exportFormat'), parameters('partitionData'), parameters('dataOverwriteBehavior'), parameters('compressionMode'), if(parameters('isMonthly'), 'monthly', 'daily'), toLower(parameters('datasetType')), if(parameters('isMonthly'), 'monthly', 'daily'), toLower(parameters('datasetType'))), if(or(equals(toLower(parameters('datasetType')), 'pricesheet'), equals(toLower(parameters('datasetType')), 'reservationtransactions')), format('{{ \"properties\": {{ \"definition\": {{ \"dataSet\": {{ \"configuration\": {{ \"dataVersion\": \"{0}\", \"filters\": [] }}}}, \"timeframe\": \"{1}\", \"type\": \"{2}\" }}, \"deliveryInfo\": {{ \"destination\": {{ \"container\": \"{3}\", \"rootFolderPath\": \"@{{if(startswith(item().scope, ''/''), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}}\", \"type\": \"AzureBlob\", \"resourceId\": \"@{{variables(''storageAccountId'')}}\" }} }}, \"schedule\": {{ \"recurrence\": \"{4}\", \"recurrencePeriod\": {{ \"from\": \"2024-01-01T00:00:00.000Z\", \"to\": \"2050-02-01T00:00:00.000Z\" }}, \"status\": \"Inactive\" }}, \"format\": \"{5}\", \"partitionData\": \"{6}\", \"dataOverwriteBehavior\": \"{7}\", \"compressionMode\": \"{8}\" }}, \"id\": \"@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-{9}-{10}''))}}\", \"name\": \"@{{toLower(concat(variables(''finOpsHub''), ''-{11}-{12}''))}}\", \"type\": \"Microsoft.CostManagement/reports\", \"identity\": {{ \"type\": \"systemAssigned\" }}, \"location\": \"global\" }}', variables('exportDataVersions')[toLower(parameters('datasetType'))], if(parameters('isMonthly'), 'TheCurrentMonth', 'MonthToDate'), parameters('datasetType'), parameters('exportContainerName'), if(parameters('isMonthly'), 'Monthly', 'Daily'), parameters('exportFormat'), parameters('partitionData'), parameters('dataOverwriteBehavior'), parameters('compressionMode'), if(parameters('isMonthly'), 'monthly', 'daily'), toLower(parameters('datasetType')), if(parameters('isMonthly'), 'monthly', 'daily'), toLower(parameters('datasetType'))), if(equals(toLower(parameters('datasetType')), 'reservationrecommendations'), format('{{ \"properties\": {{ \"definition\": {{ \"dataSet\": {{ \"configuration\": {{ \"dataVersion\": \"{0}\", \"filters\": [ {{ \"name\": \"reservationScope\", \"value\": \"{1}\" }}, {{ \"name\": \"resourceType\", \"value\": \"{2}\" }}, {{ \"name\": \"lookBackPeriod\", \"value\": \"{3}\" }}] }}}}, \"timeframe\": \"{4}\", \"type\": \"{5}\" }}, \"deliveryInfo\": {{ \"destination\": {{ \"container\": \"{6}\", \"rootFolderPath\": \"@{{if(startswith(item().scope, ''/''), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}}\", \"type\": \"AzureBlob\", \"resourceId\": \"@{{variables(''storageAccountId'')}}\" }} }}, \"schedule\": {{ \"recurrence\": \"{7}\", \"recurrencePeriod\": {{ \"from\": \"2024-01-01T00:00:00.000Z\", \"to\": \"2050-02-01T00:00:00.000Z\" }}, \"status\": \"Inactive\" }}, \"format\": \"{8}\", \"partitionData\": \"{9}\", \"dataOverwriteBehavior\": \"{10}\", \"compressionMode\": \"{11}\" }}, \"id\": \"@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-{12}-costdetails''))}}\", \"name\": \"@{{toLower(concat(variables(''finOpsHub''), ''-{13}-costdetails''))}}\", \"type\": \"Microsoft.CostManagement/reports\", \"identity\": {{ \"type\": \"systemAssigned\" }}, \"location\": \"global\" }}', variables('exportDataVersions')[toLower(parameters('datasetType'))], parameters('recommendationScope'), parameters('resourceType'), parameters('recommendationLookbackPeriod'), if(parameters('isMonthly'), 'TheLastMonth', 'MonthToDate'), parameters('datasetType'), parameters('exportContainerName'), if(parameters('isMonthly'), 'Monthly', 'Daily'), parameters('exportFormat'), parameters('partitionData'), parameters('dataOverwriteBehavior'), parameters('compressionMode'), if(parameters('isMonthly'), 'monthly', 'daily'), if(parameters('isMonthly'), 'monthly', 'daily')), 'undefined'))))]" } } - }, + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", - "location": "Azure resource location of the FinOps hub instance.", - "tags": "Tags to apply to all FinOps hub resources.", - "tagsByResource": "Tags to apply to resources based on their resource type.", - "version": "FinOps hub version number.", - "options": { - "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", - "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", - "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", - "privateRouting": "Indicates whether private network routing is enabled.", - "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", - "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", - "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." - }, - "routing": "FinOps hub private network routing properties, if enabled.", - "core": { - "suffix": "Unique suffix used for shared resources." - }, - "apps": {}, - "description": "FinOps hub instance properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } + "description": "Required. FinOps hub app getting deployed." } + } + }, + "variables": { + "CONFIG": "config", + "MSEXPORTS": "msexports", + "exportsApiVersion": "2023-07-01-preview", + "exportDataVersions": { + "focuscost": "1.2-preview", + "pricesheet": "2023-03-01", + "reservationdetails": "2023-03-01", + "reservationrecommendations": "2023-05-01", + "reservationtransactions": "2023-05-01" }, - "_1.HubRoutingProperties": { - "type": "object", + "finOpsToolkitVersion": "13.0" + }, + "resources": { + "dataFactory::dataset_config": { + "existing": true, + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, variables('CONFIG'))]" + }, + "dataFactory::trigger_DailySchedule": { + "type": "Microsoft.DataFactory/factories/triggers", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_DailySchedule', variables('CONFIG')))]", "properties": { - "networkId": { - "type": "string" - }, - "networkName": { - "type": "string" - }, - "scriptStorage": { - "type": "string" - }, - "dnsZones": { - "type": "object", - "properties": { - "blob": { - "$ref": "#/definitions/_1.IdNameObject" + "pipelines": [ + { + "pipelineReference": { + "referenceName": "[format('{0}_StartExportProcess', variables('CONFIG'))]", + "type": "PipelineReference" }, - "dfs": { - "$ref": "#/definitions/_1.IdNameObject" + "parameters": { + "Recurrence": "Daily" + } + } + ], + "type": "ScheduleTrigger", + "typeProperties": { + "recurrence": { + "frequency": "Hour", + "interval": 24, + "startTime": "2023-01-01T01:01:00", + "timeZone": "[reference('timeZones').outputs.Timezone.value]" + } + } + }, + "dependsOn": [ + "dataFactory::pipeline_StartExportProcess", + "timeZones" + ] + }, + "dataFactory::trigger_MonthlySchedule": { + "type": "Microsoft.DataFactory/factories/triggers", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_MonthlySchedule', variables('CONFIG')))]", + "properties": { + "pipelines": [ + { + "pipelineReference": { + "referenceName": "[format('{0}_StartExportProcess', variables('CONFIG'))]", + "type": "PipelineReference" }, - "queue": { - "$ref": "#/definitions/_1.IdNameObject" + "parameters": { + "Recurrence": "Monthly" + } + } + ], + "type": "ScheduleTrigger", + "typeProperties": { + "recurrence": { + "frequency": "Month", + "interval": 1, + "startTime": "2023-01-05T01:11:00", + "timeZone": "[reference('timeZones').outputs.Timezone.value]", + "schedule": { + "monthDays": [ + 2, + 5, + 19 + ] + } + } + } + }, + "dependsOn": [ + "dataFactory::pipeline_StartExportProcess", + "timeZones" + ] + }, + "dataFactory::pipeline_StartBackfillProcess": { + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_StartBackfillProcess', variables('CONFIG')))]", + "properties": { + "activities": [ + { + "name": "Get Config", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false }, - "table": { - "$ref": "#/definitions/_1.IdNameObject" + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference", + "parameters": { + "fileName": { + "value": "@variables('fileName')", + "type": "Expression" + }, + "folderPath": { + "value": "@variables('folderPath')", + "type": "Expression" + } + } + } + } + }, + { + "name": "Set backfill end date", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Get Config", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "variableName": "endDate", + "value": { + "value": "@addDays(startOfMonth(utcNow()), -1)", + "type": "Expression" + } + } + }, + { + "name": "Set backfill start date", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Get Config", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "variableName": "startDate", + "value": { + "value": "@subtractFromTime(startOfMonth(utcNow()), activity('Get Config').output.firstRow.retention.ingestion.months, 'Month')", + "type": "Expression" + } } - } - }, - "subnets": { - "type": "object", - "properties": { - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "scripts": { - "type": "string" - }, - "storage": { - "type": "string" + }, + { + "name": "Set export start date", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Set backfill start date", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "variableName": "thisMonth", + "value": { + "value": "@startOfMonth(variables('endDate'))", + "type": "Expression" + } } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubAppProperties": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" + }, + { + "name": "Set export end date", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Set export start date", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "variableName": "nextMonth", + "value": { + "value": "@startOfMonth(subtractFromTime(variables('thisMonth'), 1, 'Month'))", + "type": "Expression" + } + } + }, + { + "name": "Every Month", + "type": "Until", + "dependsOn": [ + { + "activity": "Set export end date", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set backfill end date", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "expression": { + "value": "@less(variables('thisMonth'), variables('startDate'))", + "type": "Expression" + }, + "activities": [ + { + "name": "Update export start date", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Backfill data", + "dependencyConditions": [ + "Completed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "variableName": "thisMonth", + "value": { + "value": "@variables('nextMonth')", + "type": "Expression" + } + } + }, + { + "name": "Update export end date", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Update export start date", + "dependencyConditions": [ + "Completed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "variableName": "nextMonth", + "value": { + "value": "@subtractFromTime(variables('thisMonth'), 1, 'Month')", + "type": "Expression" + } + } + }, + { + "name": "Backfill data", + "type": "ExecutePipeline", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "pipeline": { + "referenceName": "[format('{0}_RunBackfillJob', variables('CONFIG'))]", + "type": "PipelineReference" + }, + "waitOnCompletion": true, + "parameters": { + "StartDate": { + "value": "@variables('thisMonth')", + "type": "Expression" + }, + "EndDate": { + "value": "@addDays(addToTime(variables('thisMonth'), 1, 'Month'), -1)", + "type": "Expression" + } + } + } + } + ], + "timeout": "0.02:00:00" } } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "storage": { - "type": "string" + ], + "concurrency": 1, + "variables": { + "exportName": { + "type": "String" + }, + "storageAccountId": { + "type": "String", + "defaultValue": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "finOpsHub": { + "type": "String", + "defaultValue": "[parameters('app').hub.name]" + }, + "resourceManagementUri": { + "type": "String", + "defaultValue": "[environment().resourceManager]" + }, + "fileName": { + "type": "String", + "defaultValue": "settings.json" + }, + "folderPath": { + "type": "String", + "defaultValue": "[variables('CONFIG')]" + }, + "endDate": { + "type": "String" + }, + "startDate": { + "type": "String" + }, + "thisMonth": { + "type": "String" + }, + "nextMonth": { + "type": "String" + } } }, - "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", - "dataFactory": "Name of the Data Factory instance for this publisher.", - "keyVault": "Name of the KeyVault instance for this publisher.", - "storage": "Name of the storage account for this publisher.", - "description": "FinOps hub app configuration settings.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - }, - "functions": [ - { - "namespace": "__bicep", - "members": { - "getExportBody": { - "parameters": [ - { - "type": "string", - "name": "exportContainerName" - }, - { - "type": "string", - "name": "datasetType" - }, - { - "type": "string", - "name": "schemaVersion" - }, - { - "type": "bool", - "name": "isMonthly" - }, - { - "type": "string", - "name": "exportFormat" + "dependsOn": [ + "dataFactory::pipeline_RunBackfillJob" + ] + }, + "dataFactory::pipeline_RunBackfillJob": { + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_RunBackfillJob', variables('CONFIG')))]", + "properties": { + "activities": [ + { + "name": "Get Config", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false }, - { - "type": "string", - "name": "compressionMode" + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference", + "parameters": { + "fileName": { + "value": "@variables('fileName')", + "type": "Expression" + }, + "folderPath": { + "value": "@variables('folderPath')", + "type": "Expression" + } + } + } + } + }, + { + "name": "Set Scopes", + "description": "Save scopes to test if it is an array", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Get Config", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false }, - { - "type": "string", - "name": "partitionData" + "userProperties": [], + "typeProperties": { + "variableName": "scopesArray", + "value": { + "value": "@activity('Get Config').output.firstRow.scopes", + "type": "Expression" + } + } + }, + { + "name": "Set Scopes as Array", + "description": "Wraps a single scope object into an array to work around the PowerShell bug where single-item arrays are sometimes written as a single object instead of an array.", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Set Scopes", + "dependencyConditions": [ + "Failed" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false }, - { - "type": "string", - "name": "dataOverwriteBehavior" + "userProperties": [], + "typeProperties": { + "variableName": "scopesArray", + "value": { + "value": "@createArray(activity('Get Config').output.firstRow.scopes)", + "type": "Expression" + } } - ], - "output": { - "type": "string", - "value": "[format('{{ \"properties\": {{ \"definition\": {{ \"dataSet\": {{ \"configuration\": {{ \"dataVersion\": \"{0}\", \"filters\": [] }}, \"granularity\": \"Daily\" }}, \"timeframe\": \"{1}\", \"type\": \"{2}\" }}, \"deliveryInfo\": {{ \"destination\": {{ \"container\": \"{3}\", \"rootFolderPath\": \"@{{if(startswith(item().scope, ''/''), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}}\", \"type\": \"AzureBlob\", \"resourceId\": \"@{{variables(''storageAccountId'')}}\" }} }}, \"schedule\": {{ \"recurrence\": \"{4}\", \"recurrencePeriod\": {{ \"from\": \"2024-01-01T00:00:00.000Z\", \"to\": \"2050-02-01T00:00:00.000Z\" }}, \"status\": \"Inactive\" }}, \"format\": \"{5}\", \"partitionData\": \"{6}\", \"dataOverwriteBehavior\": \"{7}\", \"compressionMode\": \"{8}\" }}, \"id\": \"@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{variables(''exportName'')}}\", \"name\": \"@{{variables(''exportName'')}}\", \"type\": \"Microsoft.CostManagement/reports\", \"identity\": {{ \"type\": \"systemAssigned\" }}, \"location\": \"global\" }}', parameters('schemaVersion'), if(parameters('isMonthly'), 'TheLastMonth', 'MonthToDate'), parameters('datasetType'), parameters('exportContainerName'), if(parameters('isMonthly'), 'Monthly', 'Daily'), parameters('exportFormat'), parameters('partitionData'), parameters('dataOverwriteBehavior'), parameters('compressionMode'))]" + }, + { + "name": "Filter Invalid Scopes", + "description": "Remove any invalid scopes to avoid errors.", + "type": "Filter", + "dependsOn": [ + { + "activity": "Set Scopes", + "dependencyConditions": [ + "Succeeded", + "Failed" + ] + }, + { + "activity": "Set Scopes as Array", + "dependencyConditions": [ + "Skipped", + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@variables('scopesArray')", + "type": "Expression" + }, + "condition": { + "value": "@and(not(empty(item().scope)), not(equals(item().scope, '/')))", + "type": "Expression" + } + } + }, + { + "name": "ForEach Export Scope", + "type": "ForEach", + "dependsOn": [ + { + "activity": "Filter Invalid Scopes", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@activity('Filter Invalid Scopes').output.Value", + "type": "Expression" + }, + "isSequential": true, + "activities": [ + { + "name": "Set backfill export name", + "type": "SetVariable", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "variableName": "exportName", + "value": { + "value": "@toLower(concat(variables('finOpsHub'), '-monthly-costdetails'))", + "type": "Expression" + } + } + }, + { + "name": "Trigger backfill export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Set backfill export name", + "dependencyConditions": [ + "Completed" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 1, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{variables(''exportName'')}}/run?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "POST", + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunBackfill@{0}', variables('finOpsToolkitVersion'))]", + "Content-Type": "application/json", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "body": "{\"timePeriod\" : { \"from\" : \"@{pipeline().parameters.StartDate}\", \"to\" : \"@{pipeline().parameters.EndDate}\" }}", + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + } + ] + } + } + ], + "concurrency": 1, + "parameters": { + "StartDate": { + "type": "string" + }, + "EndDate": { + "type": "string" } }, - "getExportBodyV2": { - "parameters": [ - { - "type": "string", - "name": "exportContainerName" - }, - { - "type": "string", - "name": "datasetType" - }, - { - "type": "string", - "name": "schemaVersion" - }, - { - "type": "bool", - "name": "isMonthly" - }, - { - "type": "string", - "name": "exportFormat" - }, - { - "type": "string", - "name": "compressionMode" - }, - { - "type": "string", - "name": "partitionData" - }, - { - "type": "string", - "name": "dataOverwriteBehavior" + "variables": { + "exportName": { + "type": "String" + }, + "storageAccountId": { + "type": "String", + "defaultValue": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "finOpsHub": { + "type": "String", + "defaultValue": "[parameters('app').hub.name]" + }, + "resourceManagementUri": { + "type": "String", + "defaultValue": "[environment().resourceManager]" + }, + "fileName": { + "type": "String", + "defaultValue": "settings.json" + }, + "folderPath": { + "type": "String", + "defaultValue": "[variables('CONFIG')]" + }, + "scopesArray": { + "type": "Array" + } + } + } + }, + "dataFactory::pipeline_StartExportProcess": { + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_StartExportProcess', variables('CONFIG')))]", + "properties": { + "activities": [ + { + "name": "Get Config", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false }, - { - "type": "string", - "name": "recommendationScope" + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference", + "parameters": { + "fileName": { + "value": "@variables('fileName')", + "type": "Expression" + }, + "folderPath": { + "value": "@variables('folderPath')", + "type": "Expression" + } + } + } + } + }, + { + "name": "Set Scopes", + "description": "Save scopes to test if it is an array", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Get Config", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false }, - { - "type": "string", - "name": "recommendationLookbackPeriod" + "userProperties": [], + "typeProperties": { + "variableName": "scopesArray", + "value": { + "value": "@activity('Get Config').output.firstRow.scopes", + "type": "Expression" + } + } + }, + { + "name": "Set Scopes as Array", + "description": "Wraps a single scope object into an array to work around the PowerShell bug where single-item arrays are sometimes written as a single object instead of an array.", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Set Scopes", + "dependencyConditions": [ + "Failed" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false }, - { - "type": "string", - "name": "resourceType" + "userProperties": [], + "typeProperties": { + "variableName": "scopesArray", + "value": { + "value": "@createArray(activity('Get Config').output.firstRow.scopes)", + "type": "Expression" + } + } + }, + { + "name": "Filter Invalid Scopes", + "description": "Remove any invalid scopes to avoid errors.", + "type": "Filter", + "dependsOn": [ + { + "activity": "Set Scopes", + "dependencyConditions": [ + "Succeeded", + "Failed" + ] + }, + { + "activity": "Set Scopes as Array", + "dependencyConditions": [ + "Succeeded", + "Skipped" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@variables('scopesArray')", + "type": "Expression" + }, + "condition": { + "value": "@and(not(empty(item().scope)), not(equals(item().scope, '/')))", + "type": "Expression" + } + } + }, + { + "name": "ForEach Export Scope", + "type": "ForEach", + "dependsOn": [ + { + "activity": "Filter Invalid Scopes", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@activity('Filter Invalid Scopes').output.Value", + "type": "Expression" + }, + "isSequential": true, + "activities": [ + { + "name": "Get exports for scope", + "type": "WebActivity", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "GET", + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Run exports for scope", + "type": "ExecutePipeline", + "dependsOn": [ + { + "activity": "Get exports for scope", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "pipeline": { + "referenceName": "[format('{0}_RunExportJobs', variables('CONFIG'))]", + "type": "PipelineReference" + }, + "waitOnCompletion": true, + "parameters": { + "ExportScopes": { + "value": "@activity('Get exports for scope').output.value", + "type": "Expression" + }, + "Recurrence": { + "value": "@pipeline().parameters.Recurrence", + "type": "Expression" + } + } + } + } + ] } - ], - "output": { + } + ], + "concurrency": 1, + "parameters": { + "Recurrence": { "type": "string", - "value": "[if(equals(toLower(parameters('datasetType')), 'focuscost'), format('{{ \"properties\": {{ \"definition\": {{ \"dataSet\": {{ \"configuration\": {{ \"dataVersion\": \"{0}\", \"filters\": [] }}, \"granularity\": \"Daily\" }}, \"timeframe\": \"{1}\", \"type\": \"{2}\" }}, \"deliveryInfo\": {{ \"destination\": {{ \"container\": \"{3}\", \"rootFolderPath\": \"@{{if(startswith(item().scope, ''/''), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}}\", \"type\": \"AzureBlob\", \"resourceId\": \"@{{variables(''storageAccountId'')}}\" }} }}, \"schedule\": {{ \"recurrence\": \"{4}\", \"recurrencePeriod\": {{ \"from\": \"2024-01-01T00:00:00.000Z\", \"to\": \"2050-02-01T00:00:00.000Z\" }}, \"status\": \"Inactive\" }}, \"format\": \"{5}\", \"partitionData\": \"{6}\", \"dataOverwriteBehavior\": \"{7}\", \"compressionMode\": \"{8}\" }}, \"id\": \"@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-{9}-costdetails''))}}\", \"name\": \"@{{toLower(concat(variables(''finOpsHub''), ''-{10}-costdetails''))}}\", \"type\": \"Microsoft.CostManagement/reports\", \"identity\": {{ \"type\": \"systemAssigned\" }}, \"location\": \"global\" }}', parameters('schemaVersion'), if(parameters('isMonthly'), 'TheLastMonth', 'MonthToDate'), parameters('datasetType'), parameters('exportContainerName'), if(parameters('isMonthly'), 'Monthly', 'Daily'), parameters('exportFormat'), parameters('partitionData'), parameters('dataOverwriteBehavior'), parameters('compressionMode'), if(parameters('isMonthly'), 'monthly', 'daily'), if(parameters('isMonthly'), 'monthly', 'daily')), if(equals(toLower(parameters('datasetType')), 'reservationdetails'), format('{{ \"properties\": {{ \"definition\": {{ \"dataSet\": {{ \"configuration\": {{ \"dataVersion\": \"{0}\", \"filters\": [] }}, \"granularity\": \"Daily\" }}, \"timeframe\": \"{1}\", \"type\": \"{2}\" }}, \"deliveryInfo\": {{ \"destination\": {{ \"container\": \"{3}\", \"rootFolderPath\": \"@{{if(startswith(item().scope, ''/''), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}}\", \"type\": \"AzureBlob\", \"resourceId\": \"@{{variables(''storageAccountId'')}}\" }} }}, \"schedule\": {{ \"recurrence\": \"{4}\", \"recurrencePeriod\": {{ \"from\": \"2024-01-01T00:00:00.000Z\", \"to\": \"2050-02-01T00:00:00.000Z\" }}, \"status\": \"Inactive\" }}, \"format\": \"{5}\", \"partitionData\": \"{6}\", \"dataOverwriteBehavior\": \"{7}\", \"compressionMode\": \"{8}\" }}, \"id\": \"@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-{9}-{10}''))}}\", \"name\": \"@{{toLower(concat(variables(''finOpsHub''), ''-{11}-{12}''))}}\", \"type\": \"Microsoft.CostManagement/reports\", \"identity\": {{ \"type\": \"systemAssigned\" }}, \"location\": \"global\" }}', parameters('schemaVersion'), if(parameters('isMonthly'), 'TheLastMonth', 'MonthToDate'), parameters('datasetType'), parameters('exportContainerName'), if(parameters('isMonthly'), 'Monthly', 'Daily'), parameters('exportFormat'), parameters('partitionData'), parameters('dataOverwriteBehavior'), parameters('compressionMode'), if(parameters('isMonthly'), 'monthly', 'daily'), toLower(parameters('datasetType')), if(parameters('isMonthly'), 'monthly', 'daily'), toLower(parameters('datasetType'))), if(or(equals(toLower(parameters('datasetType')), 'pricesheet'), equals(toLower(parameters('datasetType')), 'reservationtransactions')), format('{{ \"properties\": {{ \"definition\": {{ \"dataSet\": {{ \"configuration\": {{ \"dataVersion\": \"{0}\", \"filters\": [] }}}}, \"timeframe\": \"{1}\", \"type\": \"{2}\" }}, \"deliveryInfo\": {{ \"destination\": {{ \"container\": \"{3}\", \"rootFolderPath\": \"@{{if(startswith(item().scope, ''/''), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}}\", \"type\": \"AzureBlob\", \"resourceId\": \"@{{variables(''storageAccountId'')}}\" }} }}, \"schedule\": {{ \"recurrence\": \"{4}\", \"recurrencePeriod\": {{ \"from\": \"2024-01-01T00:00:00.000Z\", \"to\": \"2050-02-01T00:00:00.000Z\" }}, \"status\": \"Inactive\" }}, \"format\": \"{5}\", \"partitionData\": \"{6}\", \"dataOverwriteBehavior\": \"{7}\", \"compressionMode\": \"{8}\" }}, \"id\": \"@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-{9}-{10}''))}}\", \"name\": \"@{{toLower(concat(variables(''finOpsHub''), ''-{11}-{12}''))}}\", \"type\": \"Microsoft.CostManagement/reports\", \"identity\": {{ \"type\": \"systemAssigned\" }}, \"location\": \"global\" }}', parameters('schemaVersion'), if(parameters('isMonthly'), 'TheCurrentMonth', 'MonthToDate'), parameters('datasetType'), parameters('exportContainerName'), if(parameters('isMonthly'), 'Monthly', 'Daily'), parameters('exportFormat'), parameters('partitionData'), parameters('dataOverwriteBehavior'), parameters('compressionMode'), if(parameters('isMonthly'), 'monthly', 'daily'), toLower(parameters('datasetType')), if(parameters('isMonthly'), 'monthly', 'daily'), toLower(parameters('datasetType'))), if(equals(toLower(parameters('datasetType')), 'reservationrecommendations'), format('{{ \"properties\": {{ \"definition\": {{ \"dataSet\": {{ \"configuration\": {{ \"dataVersion\": \"{0}\", \"filters\": [ {{ \"name\": \"reservationScope\", \"value\": \"{1}\" }}, {{ \"name\": \"resourceType\", \"value\": \"{2}\" }}, {{ \"name\": \"lookBackPeriod\", \"value\": \"{3}\" }}] }}}}, \"timeframe\": \"{4}\", \"type\": \"{5}\" }}, \"deliveryInfo\": {{ \"destination\": {{ \"container\": \"{6}\", \"rootFolderPath\": \"@{{if(startswith(item().scope, ''/''), substring(item().scope, 1, sub(length(item().scope), 1)) ,item().scope)}}\", \"type\": \"AzureBlob\", \"resourceId\": \"@{{variables(''storageAccountId'')}}\" }} }}, \"schedule\": {{ \"recurrence\": \"{7}\", \"recurrencePeriod\": {{ \"from\": \"2024-01-01T00:00:00.000Z\", \"to\": \"2050-02-01T00:00:00.000Z\" }}, \"status\": \"Inactive\" }}, \"format\": \"{8}\", \"partitionData\": \"{9}\", \"dataOverwriteBehavior\": \"{10}\", \"compressionMode\": \"{11}\" }}, \"id\": \"@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-{12}-costdetails''))}}\", \"name\": \"@{{toLower(concat(variables(''finOpsHub''), ''-{13}-costdetails''))}}\", \"type\": \"Microsoft.CostManagement/reports\", \"identity\": {{ \"type\": \"systemAssigned\" }}, \"location\": \"global\" }}', parameters('schemaVersion'), parameters('recommendationScope'), parameters('resourceType'), parameters('recommendationLookbackPeriod'), if(parameters('isMonthly'), 'TheLastMonth', 'MonthToDate'), parameters('datasetType'), parameters('exportContainerName'), if(parameters('isMonthly'), 'Monthly', 'Daily'), parameters('exportFormat'), parameters('partitionData'), parameters('dataOverwriteBehavior'), parameters('compressionMode'), if(parameters('isMonthly'), 'monthly', 'daily'), if(parameters('isMonthly'), 'monthly', 'daily')), 'undefined'))))]" + "defaultValue": "Daily" + } + }, + "variables": { + "fileName": { + "type": "String", + "defaultValue": "settings.json" + }, + "folderPath": { + "type": "String", + "defaultValue": "[variables('CONFIG')]" + }, + "finOpsHub": { + "type": "String", + "defaultValue": "[parameters('app').hub.name]" + }, + "resourceManagementUri": { + "type": "String", + "defaultValue": "[environment().resourceManager]" + }, + "scopesArray": { + "type": "Array" } } - } - } - ], - "parameters": { - "app": { - "$ref": "#/definitions/HubAppProperties", - "metadata": { - "description": "Required. Temporary app placeholder for the deployments module." - } - }, - "hubName": { - "type": "string", - "metadata": { - "description": "Required. Name of the FinOps hub instance." - } - }, - "dataFactoryName": { - "type": "string", - "metadata": { - "description": "Required. Name of the Data Factory instance." - } - }, - "keyVaultName": { - "type": "string", - "metadata": { - "description": "Required. The name of the Azure Key Vault instance." - } - }, - "storageAccountName": { - "type": "string", - "metadata": { - "description": "Required. The name of the Azure storage account instance." - } - }, - "exportContainerName": { - "type": "string", - "metadata": { - "description": "Required. The name of the container where Cost Management data is exported." - } - }, - "ingestionContainerName": { - "type": "string", - "metadata": { - "description": "Required. The name of the container where normalized data is ingested." - } - }, - "configContainerName": { - "type": "string", - "metadata": { - "description": "Required. The name of the container where normalized data is ingested." - } - }, - "dataExplorerName": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Name of the Azure Data Explorer cluster to use for advanced analytics, if applicable." - } - }, - "dataExplorerId": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Resource ID of the Azure Data Explorer cluster to use for advanced analytics, if applicable." - } - }, - "dataExplorerPrincipalId": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. ID of the Azure Data Explorer cluster system assigned managed identity, if applicable." - } - }, - "dataExplorerUri": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. URI of the Azure Data Explorer cluster or Microsoft Fabric eventhouse query endpoint to use for advanced analytics, if applicable." - } - }, - "dataExplorerIngestionDatabase": { - "type": "string", - "defaultValue": "Ingestion", - "metadata": { - "description": "Optional. Name of the Azure Data Explorer ingestion database. Default: \"ingestion\"." - } - }, - "dataExplorerIngestionCapacity": { - "type": "int", - "defaultValue": 1, - "metadata": { - "description": "Optional. Azure Data Explorer ingestion capacity or Microsoft Fabric capacity units. Increase for non-dev/trial SKUs. Default: 1" - } - }, - "location": { - "type": "string", - "defaultValue": "[resourceGroup().location]", - "metadata": { - "description": "Optional. The location to use for the managed identity and deployment script to auto-start triggers. Default = (resource group location)." - } - }, - "remoteHubStorageUri": { - "type": "string", - "metadata": { - "description": "Optional. Remote storage account for ingestion dataset." - } - }, - "tags": { - "type": "object", - "defaultValue": {}, - "metadata": { - "description": "Optional. Tags to apply to all resources." - } - }, - "tagsByResource": { - "type": "object", - "defaultValue": {}, - "metadata": { - "description": "Optional. Tags to apply to resources based on their resource type. Resource type specific tags will be merged with tags for all resources." - } - }, - "enableManagedExports": { - "type": "bool", - "defaultValue": true, - "metadata": { - "description": "Optional. Enable managed exports where your FinOps hub instance will create and run Cost Management exports on your behalf. Not supported for Microsoft Customer Agreement (MCA) billing profiles. Requires the ability to grant User Access Administrator role to FinOps hubs, which is required to create Cost Management exports. Default: true." - } - }, - "enablePublicAccess": { - "type": "bool", - "metadata": { - "description": "Required. Enable public access." - } - } - }, - "variables": { - "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n# Init outputs\r\n$DeploymentScriptOutputs = @{}\r\n\r\n#\r\n$adfParams = @{\r\n ResourceGroupName = $env:DataFactoryResourceGroup\r\n DataFactoryName = $env:DataFactoryName\r\n}\r\n\r\n# Delete old triggers\r\n$triggers = Get-AzDataFactoryV2Trigger @adfParams -ErrorAction SilentlyContinue `\r\n| Where-Object { $_.Name -match '^msexports(_(setup|daily|monthly|extract|FileAdded))?$' }\r\n$DeploymentScriptOutputs[\"stopTriggers\"] = $triggers | Stop-AzDataFactoryV2Trigger -Force -ErrorAction SilentlyContinue\r\n$DeploymentScriptOutputs[\"deleteTriggers\"] = $triggers | Remove-AzDataFactoryV2Trigger -Force -ErrorAction SilentlyContinue\r\n\r\n# Delete old pipelines\r\n$DeploymentScriptOutputs[\"pipelines\"] = Get-AzDataFactoryV2Pipeline @adfParams -ErrorAction SilentlyContinue `\r\n| Where-Object { $_.Name -match '^(msexports_(backfill|extract|fill|get|run|setup|transform)|config_(BackfillData|ExportData|RunBackfill|RunExports))$' } `\r\n| Remove-AzDataFactoryV2Pipeline -Force -ErrorAction SilentlyContinue\r\n", - "$fxv#1": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\nParam(\r\n [switch] $Stop\r\n)\r\n\r\n# Init outputs\r\n$DeploymentScriptOutputs = @{}\r\n\r\nif (-not $Stop)\r\n{\r\n Start-Sleep -Seconds 10\r\n}\r\n\r\n# Loop thru triggers\r\n$env:Triggers.Split('|') `\r\n| ForEach-Object {\r\n $trigger = $_\r\n if ($Stop)\r\n {\r\n Write-Output \"Stopping trigger $trigger...\"\r\n $triggerOutput = Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $env:DataFactoryResourceGroup `\r\n -DataFactoryName $env:DataFactoryName `\r\n -Name $trigger `\r\n -Force `\r\n -ErrorAction SilentlyContinue # Ignore errors, since the trigger may not exist\r\n }\r\n else\r\n {\r\n Write-Output \"Starting trigger $trigger...\"\r\n $triggerOutput = Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $env:DataFactoryResourceGroup `\r\n -DataFactoryName $env:DataFactoryName `\r\n -Name $trigger `\r\n -Force\r\n }\r\n if ($triggerOutput)\r\n {\r\n Write-Output \"done...\"\r\n }\r\n else\r\n {\r\n Write-Output \"failed...\"\r\n }\r\n $DeploymentScriptOutputs[$trigger] = $triggerOutput\r\n}\r\n\r\nif ($Stop)\r\n{\r\n Start-Sleep -Seconds 10\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($env:Pipelines))\r\n{\r\n $env:Pipelines.Split('|') `\r\n | ForEach-Object {\r\n Write-Output \"Running the init pipeline...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $env:DataFactoryResourceGroup `\r\n -DataFactoryName $env:DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", - "$fxv#2": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\nParam(\r\n [switch] $Stop\r\n)\r\n\r\n# Init outputs\r\n$DeploymentScriptOutputs = @{}\r\n\r\nif (-not $Stop)\r\n{\r\n Start-Sleep -Seconds 10\r\n}\r\n\r\n# Loop thru triggers\r\n$env:Triggers.Split('|') `\r\n| ForEach-Object {\r\n $trigger = $_\r\n if ($Stop)\r\n {\r\n Write-Output \"Stopping trigger $trigger...\"\r\n $triggerOutput = Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $env:DataFactoryResourceGroup `\r\n -DataFactoryName $env:DataFactoryName `\r\n -Name $trigger `\r\n -Force `\r\n -ErrorAction SilentlyContinue # Ignore errors, since the trigger may not exist\r\n }\r\n else\r\n {\r\n Write-Output \"Starting trigger $trigger...\"\r\n $triggerOutput = Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $env:DataFactoryResourceGroup `\r\n -DataFactoryName $env:DataFactoryName `\r\n -Name $trigger `\r\n -Force\r\n }\r\n if ($triggerOutput)\r\n {\r\n Write-Output \"done...\"\r\n }\r\n else\r\n {\r\n Write-Output \"failed...\"\r\n }\r\n $DeploymentScriptOutputs[$trigger] = $triggerOutput\r\n}\r\n\r\nif ($Stop)\r\n{\r\n Start-Sleep -Seconds 10\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($env:Pipelines))\r\n{\r\n $env:Pipelines.Split('|') `\r\n | ForEach-Object {\r\n Write-Output \"Running the init pipeline...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $env:DataFactoryResourceGroup `\r\n -DataFactoryName $env:DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", - "focusSchemaVersion": "1.0", - "exportSchemaVersion": "2023-05-01", - "reservationDetailsSchemaVersion": "2023-03-01", - "ftkVersion": "12.0", - "ftkReleaseUri": "[if(endsWith(variables('ftkVersion'), '-dev'), 'https://github.com/microsoft/finops-toolkit/releases/latest/download', format('https://github.com/microsoft/finops-toolkit/releases/download/v{0}', variables('ftkVersion')))]", - "exportApiVersion": "2023-07-01-preview", - "hubDataExplorerName": "hubDataExplorer", - "deployDataExplorer": "[not(empty(parameters('dataExplorerId')))]", - "useFabric": "[and(not(variables('deployDataExplorer')), not(empty(parameters('dataExplorerUri'))))]", - "datasetPropsDefault": { - "location": { - "type": "AzureBlobFSLocation", - "fileName": { - "value": "@{dataset().fileName}", - "type": "Expression" - }, - "folderPath": { - "value": "@{dataset().folderPath}", - "type": "Expression" - } - } - }, - "safeExportContainerName": "[replace(format('{0}', parameters('exportContainerName')), '-', '_')]", - "safeIngestionContainerName": "[replace(format('{0}', parameters('ingestionContainerName')), '-', '_')]", - "safeConfigContainerName": "[replace(format('{0}', parameters('configContainerName')), '-', '_')]", - "managedVnetName": "default", - "ingestionIdFileNameSeparator": "__", - "exportManifestAddedTriggerName": "[format('{0}_ManifestAdded', variables('safeExportContainerName'))]", - "ingestionManifestAddedTriggerName": "[format('{0}_ManifestAdded', variables('safeIngestionContainerName'))]", - "updateConfigTriggerName": "[format('{0}_SettingsUpdated', variables('safeConfigContainerName'))]", - "dailyTriggerName": "[format('{0}_DailySchedule', variables('safeConfigContainerName'))]", - "monthlyTriggerName": "[format('{0}_MonthlySchedule', variables('safeConfigContainerName'))]", - "allHubTriggers": [ - "[variables('exportManifestAddedTriggerName')]", - "[variables('ingestionManifestAddedTriggerName')]", - "[variables('updateConfigTriggerName')]", - "[variables('dailyTriggerName')]", - "[variables('monthlyTriggerName')]" - ], - "autoStartRbacRoles": [ - "673868aa-7521-48a0-acc6-0f60742d39f5" - ], - "storageRbacRoles": "[union(createArray('17d1049b-9a84-46fb-8f53-869881c3d3ab', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe', 'acdd72a7-3385-48ef-bd42-f606fba81ae7'), if(not(parameters('enableManagedExports')), createArray(), createArray('18d7d88d-d35e-4fb5-a5c3-7773c20a72d9')))]", - "adxRbacRoles": [ - "b24988ac-6180-42a0-ab88-20f7382dd24c" - ] - }, - "resources": { - "dataFactory": { - "existing": true, - "type": "Microsoft.DataFactory/factories", - "apiVersion": "2018-06-01", - "name": "[parameters('dataFactoryName')]" - }, - "storageAccount": { - "existing": true, - "type": "Microsoft.Storage/storageAccounts", - "apiVersion": "2022-09-01", - "name": "[parameters('storageAccountName')]" - }, - "keyVault": { - "condition": "[not(empty(parameters('remoteHubStorageUri')))]", - "existing": true, - "type": "Microsoft.KeyVault/vaults", - "apiVersion": "2023-02-01", - "name": "[parameters('keyVaultName')]" - }, - "dataExplorerCluster": { - "condition": "[variables('deployDataExplorer')]", - "existing": true, - "type": "Microsoft.Kusto/clusters", - "apiVersion": "2023-08-15", - "name": "[parameters('dataExplorerName')]" + }, + "dependsOn": [ + "dataFactory::pipeline_RunExportJobs" + ] }, - "managedVirtualNetwork": { - "condition": "[not(parameters('enablePublicAccess'))]", - "type": "Microsoft.DataFactory/factories/managedVirtualNetworks", + "dataFactory::pipeline_RunExportJobs": { + "type": "Microsoft.DataFactory/factories/pipelines", "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), variables('managedVnetName'))]", - "properties": {} + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_RunExportJobs', variables('CONFIG')))]", + "properties": { + "activities": [ + { + "name": "ForEach export scope", + "type": "ForEach", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@pipeline().parameters.exportScopes", + "type": "Expression" + }, + "isSequential": true, + "activities": [ + { + "name": "If scheduled", + "type": "IfCondition", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "expression": { + "value": "@and( startswith(toLower(item().name), toLower(variables('hubName'))), and(contains(string(item().properties.schedule), 'recurrence'), equals(toLower(item().properties.schedule.recurrence), toLower(pipeline().parameters.Recurrence))))", + "type": "Expression" + }, + "ifTrueActivities": [ + { + "name": "Trigger export", + "type": "WebActivity", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "method": "POST", + "url": { + "value": "[format('@{{replace(toLower(concat(variables(''resourceManagementUri''),item().id)), ''com//'', ''com/'')}}/run?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "body": " ", + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + } + ] + } + } + ] + } + } + ], + "concurrency": 1, + "parameters": { + "ExportScopes": { + "type": "array" + }, + "Recurrence": { + "type": "string", + "defaultValue": "Daily" + } + }, + "variables": { + "resourceManagementUri": { + "type": "String", + "defaultValue": "[environment().resourceManager]" + }, + "hubName": { + "type": "String", + "defaultValue": "[parameters('app').hub.name]" + } + } + }, + "dependsOn": [ + "dataFactory::dataset_config" + ] }, - "managedIntegrationRuntime": { - "condition": "[not(parameters('enablePublicAccess'))]", - "type": "Microsoft.DataFactory/factories/integrationRuntimes", + "dataFactory::pipeline_ConfigureExports": { + "type": "Microsoft.DataFactory/factories/pipelines", "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), 'ManagedIntegrationRuntime')]", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_ConfigureExports', variables('CONFIG')))]", "properties": { - "type": "Managed", - "managedVirtualNetwork": { - "referenceName": "[variables('managedVnetName')]", - "type": "ManagedVirtualNetworkReference" - }, - "typeProperties": { - "computeProperties": { - "location": "[parameters('location')]", - "dataFlowProperties": { - "computeType": "General", - "coreCount": 8, - "timeToLive": 10, - "cleanup": false, - "customProperties": [] - }, - "copyComputeScaleProperties": { - "dataIntegrationUnit": 16, - "timeToLive": 30 - }, - "pipelineExternalComputeScaleProperties": { - "timeToLive": 30, - "numberOfPipelineNodes": 1, - "numberOfExternalNodes": 1 + "activities": [ + { + "name": "Get Config", + "type": "Lookup", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "source": { + "type": "JsonSource", + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" + } + }, + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference", + "parameters": { + "fileName": { + "value": "@variables('fileName')", + "type": "Expression" + }, + "folderPath": { + "value": "@variables('folderPath')", + "type": "Expression" + } + } + } + } + }, + { + "name": "Save Scopes", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Get Config", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "scopesArray", + "value": { + "value": "@activity('Get Config').output.firstRow.scopes", + "type": "Expression" + } + } + }, + { + "name": "Save Scopes as Array", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Save Scopes", + "dependencyConditions": [ + "Failed" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "scopesArray", + "value": { + "value": "@array(activity('Get Config').output.firstRow.scopes)", + "type": "Expression" + } + } + }, + { + "name": "Filter Invalid Scopes", + "type": "Filter", + "dependsOn": [ + { + "activity": "Save Scopes", + "dependencyConditions": [ + "Succeeded", + "Failed" + ] + }, + { + "activity": "Save Scopes as Array", + "dependencyConditions": [ + "Skipped", + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@variables('scopesArray')", + "type": "Expression" + }, + "condition": { + "value": "@and(not(empty(item().scope)), not(equals(item().scope, '/')))", + "type": "Expression" + } + } + }, + { + "name": "ForEach Export Scope", + "type": "ForEach", + "dependsOn": [ + { + "activity": "Filter Invalid Scopes", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "items": { + "value": "@activity('Filter Invalid Scopes').output.value", + "type": "Expression" + }, + "isSequential": true, + "activities": [ + { + "name": "Set Export Type", + "type": "SetVariable", + "dependsOn": [], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "exportScopeType", + "value": { + "value": "@if(contains(toLower(item().scope), 'providers/microsoft.billing/billingaccounts'), if(contains(toLower(item().scope), ':'), 'mca', if(contains(toLower(item().scope), '/departments/'), 'ea-department', 'ea')), if(contains(toLower(item().scope), 'subscriptions/'), 'subscription', 'undefined'))", + "type": "Expression" + } + } + }, + { + "name": "Switch Export Type", + "type": "Switch", + "dependsOn": [ + { + "activity": "Set Export Type", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "on": { + "value": "@toLower(variables('exportScopeType'))", + "type": "Expression" + }, + "cases": [ + { + "value": "ea", + "activities": [ + { + "name": "Open month focus export", + "type": "WebActivity", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-costdetails''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'FocusCost', false(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsDaily@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Closed month focus export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Open month focus export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-monthly-costdetails''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'FocusCost', true(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsMonthly@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Monthly pricesheet export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Closed month focus export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-monthly-pricesheet''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'Pricesheet', true(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.Prices@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Trigger EA monthly pricesheet export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Monthly pricesheet export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "method": "POST", + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-monthly-pricesheet''))}}/run?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.Prices@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "body": " ", + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Daily reservation details export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Monthly pricesheet export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-reservationdetails''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'ReservationDetails', false(), 'CSV', 'None', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.ReservationDetails@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Daily reservation transactions export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Daily reservation details export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-reservationtransactions''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'ReservationTransactions', false(), 'CSV', 'None', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.ReservationTransactions@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Daily shared 30day virtual machines", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Daily reservation transactions export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-recommendations-shared-last30days-virtualmachines''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'ReservationRecommendations', false(), 'CSV', 'None', 'true', 'CreateNewReport', 'Shared', 'Last30Days', 'VirtualMachines')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.ReservationRecommendations.VM.Shared.30d@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + } + ] + }, + { + "value": "ea-department", + "activities": [ + { + "name": "EA Department open month focus export", + "type": "WebActivity", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-costdetails''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'FocusCost', false(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsDaily@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "EA Department closed month focus export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "EA Department open month focus export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-monthly-costdetails''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'FocusCost', true(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsMonthly@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + } + ] + }, + { + "value": "subscription", + "activities": [ + { + "name": "Subscription open month focus export", + "type": "WebActivity", + "dependsOn": [], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-costdetails''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'FocusCost', false(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsDaily@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + }, + { + "name": "Subscription closed month focus export", + "type": "WebActivity", + "dependsOn": [ + { + "activity": "Subscription open month focus export", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.00:05:00", + "retry": 2, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "url": { + "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-monthly-costdetails''))}}?api-version={0}', variables('exportsApiVersion'))]", + "type": "Expression" + }, + "method": "PUT", + "body": { + "value": "[__bicep.getExportBodyV2(variables('MSEXPORTS'), 'FocusCost', true(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", + "type": "Expression" + }, + "headers": { + "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsMonthly@{0}', variables('finOpsToolkitVersion'))]", + "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('finOpsToolkitVersion'))]" + }, + "authentication": { + "type": "MSI", + "resource": { + "value": "@variables('resourceManagementUri')", + "type": "Expression" + } + } + } + } + ] + }, + { + "value": "mca", + "activities": [ + { + "name": "Export Type Unsupported Error", + "type": "Fail", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('MCA agreements are not supported for managed exports :',variables('exportScope'))", + "type": "Expression" + }, + "errorCode": "ExportTypeUnsupported" + } + } + ] + } + ], + "defaultActivities": [ + { + "name": "Export Type Not Defined Error", + "type": "Fail", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Unable to determine the export scope type for :',variables('exportScope'))", + "type": "Expression" + }, + "errorCode": "ExportTypeNotDefined" + } + } + ] + } + } + ] } } - } - }, - "dependsOn": [ - "managedVirtualNetwork" - ] - }, - "storageManagedPrivateEndpoint": { - "condition": "[not(parameters('enablePublicAccess'))]", - "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}/{2}', parameters('dataFactoryName'), variables('managedVnetName'), parameters('storageAccountName'))]", - "properties": { - "name": "[parameters('storageAccountName')]", - "groupId": "dfs", - "privateLinkResourceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]", - "fqdns": [ - "[reference('storageAccount').primaryEndpoints.dfs]" - ] - }, - "dependsOn": [ - "managedVirtualNetwork", - "storageAccount" - ] - }, - "keyVaultManagedPrivateEndpoint": { - "condition": "[and(not(empty(parameters('remoteHubStorageUri'))), not(parameters('enablePublicAccess')))]", - "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}/{2}', parameters('dataFactoryName'), variables('managedVnetName'), parameters('keyVaultName'))]", - "properties": { - "name": "[parameters('keyVaultName')]", - "groupId": "vault", - "privateLinkResourceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName'))]", - "fqdns": [ - "[reference('keyVault').vaultUri]" - ] - }, - "dependsOn": [ - "keyVault", - "managedVirtualNetwork" - ] - }, - "dataExplorerManagedPrivateEndpoint": { - "condition": "[and(variables('deployDataExplorer'), not(parameters('enablePublicAccess')))]", - "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}/{2}', parameters('dataFactoryName'), variables('managedVnetName'), variables('hubDataExplorerName'))]", - "properties": { - "name": "[variables('hubDataExplorerName')]", - "groupId": "cluster", - "privateLinkResourceId": "[parameters('dataExplorerId')]", - "fqdns": [ - "[parameters('dataExplorerUri')]" - ] - }, - "dependsOn": [ - "managedVirtualNetwork" - ] - }, - "triggerManagerIdentity": { - "type": "Microsoft.ManagedIdentity/userAssignedIdentities", - "apiVersion": "2023-01-31", - "name": "[format('{0}_triggerManager', parameters('dataFactoryName'))]", - "location": "[parameters('location')]", - "tags": "[union(parameters('tags'), coalesce(tryGet(parameters('tagsByResource'), 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]" - }, - "triggerManagerRoleAssignments": { - "copy": { - "name": "triggerManagerRoleAssignments", - "count": "[length(variables('autoStartRbacRoles'))]" - }, - "type": "Microsoft.Authorization/roleAssignments", - "apiVersion": "2022-04-01", - "scope": "[format('Microsoft.DataFactory/factories/{0}', parameters('dataFactoryName'))]", - "name": "[guid(resourceId('Microsoft.DataFactory/factories', parameters('dataFactoryName')), variables('autoStartRbacRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', format('{0}_triggerManager', parameters('dataFactoryName'))))]", - "properties": { - "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('autoStartRbacRoles')[copyIndex()])]", - "principalId": "[reference('triggerManagerIdentity').principalId]", - "principalType": "ServicePrincipal" - }, - "dependsOn": [ - "triggerManagerIdentity" - ] - }, - "factoryIdentityStorageRoleAssignments": { - "copy": { - "name": "factoryIdentityStorageRoleAssignments", - "count": "[length(variables('storageRbacRoles'))]" - }, - "type": "Microsoft.Authorization/roleAssignments", - "apiVersion": "2022-04-01", - "scope": "[format('Microsoft.Storage/storageAccounts/{0}', parameters('storageAccountName'))]", - "name": "[guid(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), variables('storageRbacRoles')[copyIndex()], resourceId('Microsoft.DataFactory/factories', parameters('dataFactoryName')))]", - "properties": { - "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('storageRbacRoles')[copyIndex()])]", - "principalId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", - "principalType": "ServicePrincipal" - }, - "dependsOn": [ - "dataFactory" - ] - }, - "factoryIdentityDataExplorerRoleAssignments": { - "copy": { - "name": "factoryIdentityDataExplorerRoleAssignments", - "count": "[length(variables('adxRbacRoles'))]" - }, - "condition": "[variables('deployDataExplorer')]", - "type": "Microsoft.Authorization/roleAssignments", - "apiVersion": "2022-04-01", - "scope": "[format('Microsoft.Kusto/clusters/{0}', parameters('dataExplorerName'))]", - "name": "[guid(resourceId('Microsoft.Kusto/clusters', parameters('dataExplorerName')), variables('adxRbacRoles')[copyIndex()], resourceId('Microsoft.DataFactory/factories', parameters('dataFactoryName')))]", - "properties": { - "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('adxRbacRoles')[copyIndex()])]", - "principalId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", - "principalType": "ServicePrincipal" - }, - "dependsOn": [ - "dataFactory" - ] - }, - "linkedService_keyVault": { - "condition": "[not(empty(parameters('remoteHubStorageUri')))]", - "type": "Microsoft.DataFactory/factories/linkedservices", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), parameters('keyVaultName'))]", - "properties": { - "annotations": [], - "parameters": {}, - "type": "AzureKeyVault", - "typeProperties": { - "baseUrl": "[reference(format('Microsoft.KeyVault/vaults/{0}', parameters('keyVaultName')), '2023-02-01').vaultUri]" - }, - "connectVia": "[if(parameters('enablePublicAccess'), null(), createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'))]" - }, - "dependsOn": [ - "managedIntegrationRuntime" - ] - }, - "linkedService_storageAccount": { - "type": "Microsoft.DataFactory/factories/linkedservices", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), parameters('storageAccountName'))]", - "properties": { - "annotations": [], - "parameters": {}, - "type": "AzureBlobFS", - "typeProperties": { - "url": "[reference(format('Microsoft.Storage/storageAccounts/{0}', parameters('storageAccountName')), '2021-08-01').primaryEndpoints.dfs]" - }, - "connectVia": "[if(parameters('enablePublicAccess'), null(), createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'))]" - }, - "dependsOn": [ - "managedIntegrationRuntime" - ] - }, - "linkedService_dataExplorer": { - "condition": "[or(variables('deployDataExplorer'), variables('useFabric'))]", - "type": "Microsoft.DataFactory/factories/linkedservices", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), variables('hubDataExplorerName'))]", - "properties": { - "type": "AzureDataExplorer", - "parameters": { - "database": { + ], + "concurrency": 1, + "variables": { + "scopesArray": { + "type": "Array" + }, + "exportName": { + "type": "String" + }, + "exportScope": { + "type": "String" + }, + "exportScopeType": { + "type": "String" + }, + "storageAccountId": { "type": "String", - "defaultValue": "[parameters('dataExplorerIngestionDatabase')]" - } - }, - "typeProperties": { - "endpoint": "[parameters('dataExplorerUri')]", - "database": "@{linkedService().database}", - "tenant": "[reference('dataFactory', '2018-06-01', 'full').identity.tenantId]", - "servicePrincipalId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" - }, - "connectVia": "[if(parameters('enablePublicAccess'), null(), createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'))]" - }, - "dependsOn": [ - "dataFactory", - "managedIntegrationRuntime" - ] - }, - "linkedService_remoteHubStorage": { - "condition": "[not(empty(parameters('remoteHubStorageUri')))]", - "type": "Microsoft.DataFactory/factories/linkedservices", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), 'remoteHubStorage')]", - "properties": { - "annotations": [], - "parameters": {}, - "type": "AzureBlobFS", - "typeProperties": { - "url": "[parameters('remoteHubStorageUri')]", - "accountKey": { - "type": "AzureKeyVaultSecret", - "store": { - "referenceName": "[parameters('keyVaultName')]", - "type": "LinkedServiceReference" - }, - "secretName": "[format('{0}-storage-key', toLower(parameters('hubName')))]" + "defaultValue": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "finOpsHub": { + "type": "String", + "defaultValue": "[parameters('app').hub.name]" + }, + "resourceManagementUri": { + "type": "String", + "defaultValue": "[environment().resourceManager]" + }, + "fileName": { + "type": "String", + "defaultValue": "settings.json" + }, + "folderPath": { + "type": "String", + "defaultValue": "[variables('CONFIG')]" } - }, - "connectVia": "[if(parameters('enablePublicAccess'), null(), createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'))]" - }, - "dependsOn": [ - "linkedService_keyVault", - "managedIntegrationRuntime" - ] + } + } }, - "linkedService_ftkRepo": { - "type": "Microsoft.DataFactory/factories/linkedservices", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), 'ftkRepo')]", - "properties": { - "parameters": { - "filePath": { - "type": "string" - } - }, - "annotations": [], - "type": "HttpServer", - "typeProperties": { - "url": "@concat('https://github.com/microsoft/finops-toolkit/', linkedService().filePath)", - "enableServerCertificateValidation": true, - "authenticationType": "Anonymous" - }, - "connectVia": "[if(parameters('enablePublicAccess'), null(), createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'))]" - }, - "dependsOn": [ - "managedIntegrationRuntime" - ] + "storageAccount": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]" }, - "dataset_config": { - "type": "Microsoft.DataFactory/factories/datasets", + "dataFactory": { + "existing": true, + "type": "Microsoft.DataFactory/factories", "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), variables('safeConfigContainerName'))]", + "name": "[parameters('app').dataFactory]" + }, + "appRegistration": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.ManagedExports_Register", "properties": { - "annotations": [], + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", "parameters": { - "fileName": { - "type": "String", - "defaultValue": "settings.json" + "app": { + "value": "[parameters('app')]" + }, + "version": { + "value": "[variables('finOpsToolkitVersion')]" + }, + "features": { + "value": [ + "DataFactory" + ] + }, + "storageRoles": { + "value": [ + "f58310d9-a9f6-439a-9e8d-f62e7b41a168" + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "8267413868206701537" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppFeature": { + "type": "string", + "allowedValues": [ + "DataFactory", + "KeyVault", + "Storage" + ], + "metadata": { + "description": "FinOps hub app features.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } }, - "folderPath": { - "type": "String", - "defaultValue": "[parameters('configContainerName')]" - } - }, - "type": "Json", - "typeProperties": "[variables('datasetPropsDefault')]", - "linkedServiceName": { - "parameters": {}, - "referenceName": "[parameters('storageAccountName')]", - "type": "LinkedServiceReference" - } - }, - "dependsOn": [ - "linkedService_storageAccount" - ] - }, - "dataset_manifest": { - "type": "Microsoft.DataFactory/factories/datasets", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), 'manifest')]", - "properties": { - "annotations": [], - "parameters": { - "fileName": { - "type": "String", - "defaultValue": "manifest.json" + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "version": { + "type": "string", + "metadata": { + "description": "Required. Version number of the FinOps hub app." + } + }, + "features": { + "type": "array", + "items": { + "$ref": "#/definitions/HubAppFeature" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which features the app requires. Allowed values: \"DataFactory\", \"KeyVault\", \"Storage\". Default: [] (none)." + } + }, + "storageRoles": { + "type": "array", + "items": { + "type": "string" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which RBAC roles the Data Factory identity needs on the storage account, if created. This is in addition to Storage Blob Data Contributor for reading and managing content. Default: [] (none)." + } + }, + "telemetryString": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." + } + } }, - "folderPath": { - "type": "String", - "defaultValue": "[parameters('exportContainerName')]" - } - }, - "type": "Json", - "typeProperties": "[variables('datasetPropsDefault')]", - "linkedServiceName": { - "parameters": {}, - "referenceName": "[parameters('storageAccountName')]", - "type": "LinkedServiceReference" - } - }, - "dependsOn": [ - "linkedService_storageAccount" - ] - }, - "dataset_msexports": { - "type": "Microsoft.DataFactory/factories/datasets", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), variables('safeExportContainerName'))]", - "properties": { - "annotations": [], - "parameters": { - "blobPath": { - "type": "String" - } - }, - "type": "DelimitedText", - "typeProperties": { - "location": { - "type": "AzureBlobFSLocation", - "fileName": { - "value": "@{dataset().blobPath}", - "type": "Expression" + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n<#\r\n.SYNOPSIS\r\nManages Azure Data Factory triggers and pipelines during FinOps hub deployment.\r\n\r\n.DESCRIPTION\r\nThis script is called by Bicep deployment scripts to start/stop Data Factory triggers\r\nand optionally run pipelines. It handles two types of triggers:\r\n\r\n1. Schedule triggers - Can be started/stopped directly\r\n2. BlobEventsTriggers - Require Event Grid subscription management before start/stop\r\n\r\nBlobEventsTriggers use Event Grid to listen for storage blob events. Before stopping,\r\nthe Event Grid subscription must be removed (unsubscribed). Before starting, the\r\nsubscription must be added and fully provisioned. This script handles this automatically\r\nby detecting BlobEventsTriggers via the BlobPathBeginsWith property.\r\n\r\nThe script uses retry logic with linear backoff to handle transient API failures and\r\nwait for Event Grid subscription provisioning/deprovisioning to complete.\r\n\r\n.PARAMETER DataFactoryResourceGroup\r\nThe resource group containing the Data Factory.\r\n\r\n.PARAMETER DataFactoryName\r\nThe name of the Data Factory instance.\r\n\r\n.PARAMETER Pipelines\r\nPipe-delimited list of pipeline names to run (e.g., \"pipeline1|pipeline2\").\r\n\r\n.PARAMETER StartTriggers\r\nSwitch to start all stopped triggers (with Event Grid subscription for blob triggers).\r\n\r\n.PARAMETER StopTriggers\r\nSwitch to stop all running triggers (with Event Grid unsubscription for blob triggers).\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StopTriggers\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StartTriggers\r\n#>\r\n\r\nparam(\r\n [string] $DataFactoryResourceGroup,\r\n [string] $DataFactoryName,\r\n [string] $Pipelines = \"\",\r\n [switch] $StartTriggers,\r\n [switch] $StopTriggers\r\n)\r\n\r\n$MAX_RETRIES = 20\r\n$DeploymentScriptOutputs = @{}\r\n\r\nfunction Write-Log($Message)\r\n{\r\n Write-Output \"$(Get-Date -Format 'HH:mm:ss') $Message\"\r\n}\r\n\r\nfunction Invoke-WithRetry([scriptblock]$Action, [string]$Name, [int]$Delay = 5)\r\n{\r\n for ($i = 1; $i -le $MAX_RETRIES; $i++)\r\n {\r\n try { return & $Action }\r\n catch\r\n {\r\n Write-Log \"$Name failed (attempt $i/${MAX_RETRIES}): $($_.Exception.Message)\"\r\n if ($i -eq $MAX_RETRIES) { throw }\r\n Start-Sleep -Seconds ($Delay * $i)\r\n }\r\n }\r\n}\r\n\r\nfunction Set-BlobTriggerSubscription([string]$TriggerName, [switch]$Subscribe)\r\n{\r\n $targetStatus = if ($Subscribe) { 'Enabled' } else { 'Disabled' }\r\n $action = if ($Subscribe) { 'Subscribing' } else { 'Unsubscribing' }\r\n\r\n Write-Log \"$action $TriggerName to events...\"\r\n Invoke-WithRetry -Name \"$action $TriggerName\" -Delay 5 -Action {\r\n if ($Subscribe)\r\n {\r\n Add-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n else\r\n {\r\n Remove-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n\r\n $status = Get-AzDataFactoryV2TriggerSubscriptionStatus `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName\r\n if ($status.Status -ne $targetStatus)\r\n {\r\n throw \"Subscription status is $($status.Status), expected $targetStatus\"\r\n }\r\n }\r\n}\r\n\r\nif ($StartTriggers -or $StopTriggers)\r\n{\r\n $triggers = Invoke-WithRetry -Name \"Get triggers\" -Action {\r\n Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n | Where-Object {\r\n ($StartTriggers -and $_.Properties.RuntimeState -ne \"Started\") `\r\n -or ($StopTriggers -and $_.Properties.RuntimeState -ne \"Stopped\")\r\n }\r\n }\r\n\r\n Write-Log \"Found $($triggers.Count) trigger(s) to $(if ($StartTriggers) { 'start' } else { 'stop' })\"\r\n\r\n $triggers | ForEach-Object {\r\n $triggerName = $_.Name\r\n $isBlobTrigger = $null -ne $_.Properties.BlobPathBeginsWith\r\n\r\n if ($StopTriggers)\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName }\r\n Write-Log \"Stopping trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Stop $triggerName\" -Action {\r\n Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n else\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName -Subscribe }\r\n Write-Log \"Starting trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Start $triggerName\" -Action {\r\n Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n\r\n Invoke-WithRetry -Name \"Wait for $triggerName\" -Action {\r\n $state = (Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName).Properties.RuntimeState\r\n $expected = if ($StartTriggers) { 'Started' } else { 'Stopped' }\r\n if ($state -ne $expected) { throw \"Trigger is $state, expected $expected\" }\r\n }\r\n\r\n Write-Log \"...done\"\r\n $DeploymentScriptOutputs[$triggerName] = $true\r\n }\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($Pipelines))\r\n{\r\n $Pipelines.Split('|') | ForEach-Object {\r\n Write-Log \"Running pipeline $_...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", + "usesDataFactory": "[contains(parameters('features'), 'DataFactory')]", + "usesKeyVault": "[contains(parameters('features'), 'KeyVault')]", + "usesStorage": "[contains(parameters('features'), 'Storage')]", + "telemetryId": "[format('ftk-hubapp-{0}{1}{2}', parameters('app').id, if(empty(parameters('telemetryString')), '', '_'), parameters('telemetryString'))]", + "telemetryProps": { + "mode": "Incremental", + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "[format('FTK: {0}', parameters('app').id)]", + "version": "[parameters('version')]" + } + }, + "resources": [] + } + }, + "autoStartRbacRoles": [ + "673868aa-7521-48a0-acc6-0f60742d39f5" + ], + "factoryStorageRoles": "[union(parameters('storageRoles'), createArray('17d1049b-9a84-46fb-8f53-869881c3d3ab', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe', 'acdd72a7-3385-48ef-bd42-f606fba81ae7'))]", + "storageInfrastructureEncryptionProperties": "[if(not(parameters('app').hub.options.storageInfrastructureEncryption), createObject(), createObject('encryption', createObject('keySource', 'Microsoft.Storage', 'requireInfrastructureEncryption', parameters('app').hub.options.storageInfrastructureEncryption)))]" + }, + "resources": { + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').storage)]", + "properties": { + "name": "[parameters('app').storage]", + "groupId": "dfs", + "privateLinkResourceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "fqdns": [ + "[reference('storageAccount').primaryEndpoints.dfs]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "storageAccount" + ] + }, + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').keyVault)]", + "properties": { + "name": "[parameters('app').keyVault]", + "groupId": "vault", + "privateLinkResourceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "fqdns": [ + "[reference('keyVault').vaultUri]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "keyVault" + ] + }, + "dataFactory::managedVirtualNetwork": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'default')]", + "properties": {}, + "dependsOn": [ + "dataFactory" + ] + }, + "dataFactory::managedIntegrationRuntime": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/integrationRuntimes", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ManagedIntegrationRuntime')]", + "properties": { + "type": "Managed", + "managedVirtualNetwork": { + "referenceName": "default", + "type": "ManagedVirtualNetworkReference" + }, + "typeProperties": { + "computeProperties": { + "location": "[parameters('app').hub.location]", + "dataFlowProperties": { + "computeType": "General", + "coreCount": 8, + "timeToLive": 10, + "cleanup": false, + "customProperties": [] + }, + "copyComputeScaleProperties": { + "dataIntegrationUnit": 16, + "timeToLive": 30 + }, + "pipelineExternalComputeScaleProperties": { + "timeToLive": 30, + "numberOfPipelineNodes": 1, + "numberOfExternalNodes": 1 + } + } + } + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedVirtualNetwork" + ] + }, + "dataFactory::linkedService_keyVault": { + "condition": "[and(variables('usesDataFactory'), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').keyVault)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureKeyVault", + "typeProperties": { + "baseUrl": "[reference(format('Microsoft.KeyVault/vaults/{0}', parameters('app').keyVault), '2023-02-01').vaultUri]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "keyVault" + ] + }, + "dataFactory::linkedService_storageAccount": { + "condition": "[and(variables('usesDataFactory'), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').storage)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureBlobFS", + "typeProperties": { + "url": "[reference(format('Microsoft.Storage/storageAccounts/{0}', parameters('app').storage), '2021-08-01').primaryEndpoints.dfs]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "storageAccount" + ] + }, + "storageAccount::blobService": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]", + "dependsOn": [ + "storageAccount" + ] + }, + "blobEndpoint::blobPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "blobEndpoint" + ] + }, + "dfsEndpoint::dfsPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.dfs.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "dfsEndpoint" + ] + }, + "keyVaultPrivateDnsZone::keyVaultPrivateDnsZoneLink": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), format('{0}-link', replace(format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "properties": { + "virtualNetwork": { + "id": "[parameters('app').hub.routing.networkId]" + }, + "registrationEnabled": false + }, + "dependsOn": [ + "keyVaultPrivateDnsZone" + ] }, - "fileSystem": "[variables('safeExportContainerName')]" - }, - "columnDelimiter": ",", - "escapeChar": "\"", - "quoteChar": "\"", - "firstRowAsHeader": true - }, - "linkedServiceName": { - "parameters": {}, - "referenceName": "[parameters('storageAccountName')]", - "type": "LinkedServiceReference" - } - }, - "dependsOn": [ - "linkedService_storageAccount" - ] - }, - "dataset_msexports_gzip": { - "type": "Microsoft.DataFactory/factories/datasets", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), format('{0}_gzip', variables('safeExportContainerName')))]", - "properties": { - "annotations": [], - "parameters": { - "blobPath": { - "type": "String" - } - }, - "type": "DelimitedText", - "typeProperties": { - "location": { - "type": "AzureBlobFSLocation", - "fileName": { - "value": "@{dataset().blobPath}", - "type": "Expression" + "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')))]" + } + } + ] + }, + "dependsOn": [ + "keyVaultEndpoint", + "keyVaultPrivateDnsZone" + ] }, - "fileSystem": "[variables('safeExportContainerName')]" - }, - "columnDelimiter": ",", - "escapeChar": "\"", - "quoteChar": "\"", - "firstRowAsHeader": true, - "compressionCodec": "Gzip" - }, - "linkedServiceName": { - "parameters": {}, - "referenceName": "[parameters('storageAccountName')]", - "type": "LinkedServiceReference" - } - }, - "dependsOn": [ - "linkedService_storageAccount" - ] - }, - "dataset_msexports_parquet": { - "type": "Microsoft.DataFactory/factories/datasets", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), format('{0}_parquet', variables('safeExportContainerName')))]", - "properties": { - "annotations": [], - "parameters": { - "blobPath": { - "type": "String" - } - }, - "type": "Parquet", - "typeProperties": { - "location": { - "type": "AzureBlobFSLocation", - "fileName": { - "value": "@{dataset().blobPath}", - "type": "Expression" + "appTelemetry": { + "condition": "[parameters('app').hub.options.enableTelemetry]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "[if(lessOrEquals(length(variables('telemetryId')), 64), variables('telemetryId'), substring(variables('telemetryId'), 0, 64))]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Resources/deployments')]", + "properties": "[variables('telemetryProps')]" }, - "fileSystem": "[variables('safeExportContainerName')]" - } - }, - "linkedServiceName": { - "parameters": {}, - "referenceName": "[parameters('storageAccountName')]", - "type": "LinkedServiceReference" - } - }, - "dependsOn": [ - "linkedService_storageAccount" - ] - }, - "dataset_ingestion": { - "type": "Microsoft.DataFactory/factories/datasets", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), variables('safeIngestionContainerName'))]", - "properties": { - "annotations": [], - "parameters": { - "blobPath": { - "type": "String" - } - }, - "type": "Parquet", - "typeProperties": { - "location": { - "type": "AzureBlobFSLocation", - "fileName": { - "value": "@{dataset().blobPath}", - "type": "Expression" + "dataFactory": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.DataFactory/factories')]", + "identity": { + "type": "SystemAssigned" + }, + "properties": { + "globalConfigurations": { + "PipelineBillingEnabled": "true" + } + } }, - "fileSystem": "[variables('safeIngestionContainerName')]" - } - }, - "linkedServiceName": { - "parameters": {}, - "referenceName": "[if(empty(parameters('remoteHubStorageUri')), parameters('storageAccountName'), 'remoteHubStorage')]", - "type": "LinkedServiceReference" - } - }, - "dependsOn": [ - "linkedService_remoteHubStorage", - "linkedService_storageAccount" - ] - }, - "dataset_ingestion_files": { - "type": "Microsoft.DataFactory/factories/datasets", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), format('{0}_files', variables('safeIngestionContainerName')))]", - "properties": { - "annotations": [], - "parameters": { - "folderPath": { - "type": "String" - } - }, - "type": "Parquet", - "typeProperties": { - "location": { - "type": "AzureBlobFSLocation", - "fileSystem": "[variables('safeIngestionContainerName')]", - "folderPath": { - "value": "@dataset().folderPath", - "type": "Expression" - } - } - }, - "linkedServiceName": { - "parameters": {}, - "referenceName": "[if(empty(parameters('remoteHubStorageUri')), parameters('storageAccountName'), 'remoteHubStorage')]", - "type": "LinkedServiceReference" - } - }, - "dependsOn": [ - "linkedService_remoteHubStorage", - "linkedService_storageAccount" - ] - }, - "dataset_dataExplorer": { - "condition": "[or(variables('deployDataExplorer'), variables('useFabric'))]", - "type": "Microsoft.DataFactory/factories/datasets", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), variables('hubDataExplorerName'))]", - "properties": { - "type": "AzureDataExplorerTable", - "linkedServiceName": { - "parameters": { - "database": "@dataset().database" - }, - "referenceName": "[variables('hubDataExplorerName')]", - "type": "LinkedServiceReference" - }, - "parameters": { - "database": { - "type": "String", - "defaultValue": "[parameters('dataExplorerIngestionDatabase')]" - }, - "table": { - "type": "String" - } - }, - "typeProperties": { - "table": { - "value": "@dataset().table", - "type": "Expression" - } - } - }, - "dependsOn": [ - "linkedService_dataExplorer" - ] - }, - "dataset_ftkReleaseFile": { - "type": "Microsoft.DataFactory/factories/datasets", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), 'ftkReleaseFile')]", - "properties": { - "linkedServiceName": { - "referenceName": "ftkRepo", - "type": "LinkedServiceReference" - }, - "parameters": { - "fileName": { - "type": "string" - }, - "version": { - "type": "string", - "defaultValue": "[variables('ftkVersion')]" - } - }, - "annotations": [], - "type": "DelimitedText", - "typeProperties": { - "location": { - "type": "HttpServerLocation", - "relativeUrl": { - "value": "@concat('releases/download/v', dataset().version, '/', dataset().fileName)", - "type": "Expression" - } - }, - "columnDelimiter": ",", - "escapeChar": "\\", - "firstRowAsHeader": true, - "quoteChar": "\"" - }, - "schema": [] - }, - "dependsOn": [ - "linkedService_ftkRepo" - ] - }, - "trigger_DailySchedule": { - "condition": "[parameters('enableManagedExports')]", - "type": "Microsoft.DataFactory/factories/triggers", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), variables('dailyTriggerName'))]", - "properties": { - "pipelines": [ - { - "pipelineReference": { - "referenceName": "[format('{0}_StartExportProcess', variables('safeConfigContainerName'))]", - "type": "PipelineReference" + "storageRoleAssignments": { + "copy": { + "name": "storageRoleAssignments", + "count": "[length(variables('factoryStorageRoles'))]" + }, + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "name": "[guid(resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage), variables('factoryStorageRoles')[copyIndex()], resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('factoryStorageRoles')[copyIndex()])]", + "principalId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "storageAccount" + ] + }, + "triggerManagerIdentity": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[format('{0}_triggerManager', parameters('app').dataFactory)]", + "location": "[parameters('app').hub.location]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "dependsOn": [ + "dataFactory" + ] + }, + "triggerManagerRoleAssignments": { + "copy": { + "name": "triggerManagerRoleAssignments", + "count": "[length(variables('autoStartRbacRoles'))]" + }, + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]", + "name": "[guid(resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory), variables('autoStartRbacRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', format('{0}_triggerManager', parameters('app').dataFactory)))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('autoStartRbacRoles')[copyIndex()])]", + "principalId": "[reference('triggerManagerIdentity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "triggerManagerIdentity" + ] }, - "parameters": { - "Recurrence": "Daily" - } - } - ], - "type": "ScheduleTrigger", - "typeProperties": { - "recurrence": { - "frequency": "Hour", - "interval": 24, - "startTime": "2023-01-01T01:01:00", - "timeZone": "[reference('azuretimezones').outputs.Timezone.value]" - } - } - }, - "dependsOn": [ - "azuretimezones", - "pipeline_StartExportProcess", - "stopTriggers" - ] - }, - "trigger_MonthlySchedule": { - "condition": "[parameters('enableManagedExports')]", - "type": "Microsoft.DataFactory/factories/triggers", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), variables('monthlyTriggerName'))]", - "properties": { - "pipelines": [ - { - "pipelineReference": { - "referenceName": "[format('{0}_StartExportProcess', variables('safeConfigContainerName'))]", - "type": "PipelineReference" + "storageAccount": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]", + "location": "[parameters('app').hub.location]", + "sku": { + "name": "[parameters('app').hub.options.storageSku]" + }, + "kind": "BlockBlobStorage", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Storage/storageAccounts')]", + "properties": "[shallowMerge(createArray(variables('storageInfrastructureEncryptionProperties'), createObject('supportsHttpsTrafficOnly', true(), 'allowSharedKeyAccess', true(), 'isHnsEnabled', true(), 'minimumTlsVersion', 'TLS1_2', 'allowBlobPublicAccess', false(), 'publicNetworkAccess', 'Enabled', 'networkAcls', createObject('bypass', 'AzureServices', 'defaultAction', if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')))))]" }, - "parameters": { - "Recurrence": "Monthly" - } - } - ], - "type": "ScheduleTrigger", - "typeProperties": { - "recurrence": { - "frequency": "Month", - "interval": 1, - "startTime": "2023-01-05T01:11:00", - "timeZone": "[reference('azuretimezones').outputs.Timezone.value]", - "schedule": { - "monthDays": [ - 2, - 5, - 19 + "blobPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]" + }, + "blobEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-blob-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "blobLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "blob" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" ] - } - } - } - }, - "dependsOn": [ - "azuretimezones", - "pipeline_StartExportProcess", - "stopTriggers" - ] - }, - "pipeline_InitializeHub": { - "condition": "[or(variables('deployDataExplorer'), variables('useFabric'))]", - "type": "Microsoft.DataFactory/factories/pipelines", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), format('{0}_InitializeHub', variables('safeConfigContainerName')))]", - "properties": { - "activities": [ - { - "name": "Get Config", - "type": "Lookup", - "dependsOn": [], - "policy": { - "timeout": "0.00:05:00", - "retry": 2, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false }, - "userProperties": [], - "typeProperties": { - "source": { - "type": "JsonSource", - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": true, - "enablePartitionDiscovery": false + "dfsPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]" + }, + "dfsEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-dfs-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" }, - "formatSettings": { - "type": "JsonReadSettings" + "privateLinkServiceConnections": [ + { + "name": "dfsLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "dfs" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "keyVault": { + "condition": "[variables('usesKeyVault')]", + "type": "Microsoft.KeyVault/vaults", + "apiVersion": "2023-02-01", + "name": "[parameters('app').keyVault]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.KeyVault/vaults')]", + "properties": { + "sku": { + "name": "[parameters('app').hub.options.keyVaultSku]", + "family": "A" + }, + "enabledForDeployment": true, + "enabledForTemplateDeployment": true, + "enabledForDiskEncryption": true, + "enableSoftDelete": true, + "softDeleteRetentionInDays": 90, + "enablePurgeProtection": "[if(parameters('app').hub.options.keyVaultEnablePurgeProtection, true(), null())]", + "enableRbacAuthorization": false, + "createMode": "default", + "tenantId": "[subscription().tenantId]", + "accessPolicies": [ + { + "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "tenantId": "[subscription().tenantId]", + "permissions": { + "secrets": [ + "get" + ] + } + } + ], + "networkAcls": { + "bypass": "AzureServices", + "defaultAction": "[if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')]" } }, - "dataset": { - "referenceName": "[variables('safeConfigContainerName')]", - "type": "DatasetReference" - } - } - }, - { - "name": "Set Version", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Get Config", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "variableName": "version", - "value": { - "value": "@activity('Get Config').output.firstRow.version", - "type": "Expression" - } - } - }, - { - "name": "Set Scopes", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Get Config", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "variableName": "scopes", - "value": { - "value": "@string(activity('Get Config').output.firstRow.scopes)", - "type": "Expression" - } - } - }, - { - "name": "Set Retention", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Get Config", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "variableName": "retention", - "value": { - "value": "@string(activity('Get Config').output.firstRow.retention)", - "type": "Expression" - } - } - }, - { - "name": "Until Capacity Is Available", - "type": "Until", - "dependsOn": [ - { - "activity": "Set Version", - "dependencyConditions": [ - "Succeeded" + "dependsOn": [ + "dataFactory" + ] + }, + "keyVaultPrivateDnsZone": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones')]", + "properties": {} + }, + "keyVaultEndpoint": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-ep', parameters('app').keyVault)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.keyVault]" + }, + "privateLinkServiceConnections": [ + { + "name": "keyVaultLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "groupIds": [ + "vault" + ] + } + } ] }, - { - "activity": "Set Scopes", - "dependencyConditions": [ - "Succeeded" - ] + "dependsOn": [ + "keyVault" + ] + }, + "getKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } }, - { - "activity": "Set Retention", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "expression": { - "value": "@equals(variables('tryAgain'), false)", - "type": "Expression" + "dependsOn": [ + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint", + "getStoragePrivateEndpointConnections", + "keyVault" + ] + }, + "approveKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + }, + "privateEndpointConnections": { + "value": "[reference('getKeyVaultPrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } }, - "activities": [ - { - "name": "Confirm Ingestion Capacity", - "type": "AzureDataExplorerCommand", - "dependsOn": [], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false + "dependsOn": [ + "getKeyVaultPrivateEndpointConnections", + "keyVault" + ] + }, + "getStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } }, - "userProperties": [], - "typeProperties": { - "command": ".show capacity | where Resource == 'Ingestions' | project Remaining", - "commandTimeout": "00:20:00" + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } }, - "linkedServiceName": { - "referenceName": "[variables('hubDataExplorerName')]", - "type": "LinkedServiceReference", - "parameters": { - "database": "[parameters('dataExplorerIngestionDatabase')]" + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" } } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint", + "stopTriggers", + "storageAccount" + ] + }, + "approveStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" }, - { - "name": "If Has Capacity", - "type": "IfCondition", - "dependsOn": [ + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + }, + "privateEndpointConnections": { + "value": "[reference('getStoragePrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ { - "activity": "Confirm Ingestion Capacity", - "dependencyConditions": [ - "Succeeded" - ] + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getStoragePrivateEndpointConnections", + "storageAccount" + ] + }, + "stopTriggers": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.{1}_ADF.StopTriggers', parameters('app').publisher, parameters('app').name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "arguments": { + "value": "[join(createArray(format('-DataFactoryResourceGroup \"{0}\"', resourceGroup().name), format('-DataFactoryName \"{0}\"', parameters('app').dataFactory), '-StopTriggers'), ' ')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" } - ], - "userProperties": [], - "typeProperties": { - "expression": { - "value": "@or(equals(activity('Confirm Ingestion Capacity').output.count, 0), greater(activity('Confirm Ingestion Capacity').output.value[0].Remaining, 0))", - "type": "Expression" - }, - "ifFalseActivities": [ - { - "name": "Wait for Ingestion", - "type": "Wait", - "dependsOn": [], - "userProperties": [], - "typeProperties": { - "waitTimeInSeconds": 15 - } - }, - { - "name": "Try Again", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Wait for Ingestion", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" }, - "userProperties": [], - "typeProperties": { - "variableName": "tryAgain", - "value": true + "value": { + "type": "string" } } - ], - "ifTrueActivities": [ - { - "name": "Set ingestion policy in ADX", - "type": "AzureDataExplorerCommand", - "dependsOn": [], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" }, - "userProperties": [], - "typeProperties": { - "command": { - "value": "[if(variables('useFabric'), format('.show database {0} policy managed_identity', parameters('dataExplorerIngestionDatabase')), format('.alter-merge database {0} policy managed_identity \"[ {{ ''ObjectId'' : ''{1}'', ''AllowedUsages'' : ''NativeIngestion'' }}]\"', parameters('dataExplorerIngestionDatabase'), parameters('dataExplorerPrincipalId')))]", - "type": "Expression" - }, - "commandTimeout": "00:20:00" + "name": { + "type": "string" }, - "linkedServiceName": { - "referenceName": "[variables('hubDataExplorerName')]", - "type": "LinkedServiceReference", - "parameters": { - "database": "[parameters('dataExplorerIngestionDatabase')]" + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } } } }, - { - "name": "Save Hub Settings in ADX", - "type": "AzureDataExplorerCommand", - "dependsOn": [ - { - "activity": "Set ingestion policy in ADX", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." }, - "userProperties": [], - "typeProperties": { - "command": { - "value": "@concat('.append HubSettingsLog <| print version=\"', variables('version'), '\",scopes=dynamic(', variables('scopes'), '),retention=dynamic(', variables('retention'), ') | extend scopes = iff(isnull(scopes[0]), pack_array(scopes), scopes) | mv-apply scopeObj = scopes on (where isnotempty(scopeObj.scope) | summarize scopes = make_set(scopeObj.scope))')", - "type": "Expression" - }, - "commandTimeout": "00:20:00" + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." }, - "linkedServiceName": { - "referenceName": "[variables('hubDataExplorerName')]", - "type": "LinkedServiceReference", - "parameters": { - "database": "[parameters('dataExplorerIngestionDatabase')]" + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } } } }, - { - "name": "Update PricingUnits in ADX", - "type": "AzureDataExplorerCommand", - "dependsOn": [ - { - "activity": "Save Hub Settings in ADX", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." }, - "userProperties": [], - "typeProperties": { - "command": "[format('.set-or-replace PricingUnits <| externaldata(x_PricingUnitDescription: string, AccountTypes: string, x_PricingBlockSize: decimal, PricingUnit: string)[@\"{0}/PricingUnits.csv\"] with (format=\"csv\", ignoreFirstRecord=true) | project-away AccountTypes', variables('ftkReleaseUri'))]", - "commandTimeout": "00:20:00" + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" }, - "linkedServiceName": { - "referenceName": "[variables('hubDataExplorerName')]", - "type": "LinkedServiceReference", - "parameters": { - "database": "[parameters('dataExplorerIngestionDatabase')]" - } + "name": { + "type": "string" } }, - { - "name": "Update Regions in ADX", - "type": "AzureDataExplorerCommand", - "dependsOn": [ - { - "activity": "Update PricingUnits in ADX", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" }, - "userProperties": [], - "typeProperties": { - "command": "[format('.set-or-replace Regions <| externaldata(ResourceLocation: string, RegionId: string, RegionName: string)[@\"{0}/Regions.csv\"] with (format=\"csv\", ignoreFirstRecord=true)', variables('ftkReleaseUri'))]", - "commandTimeout": "00:20:00" + "name": { + "type": "string" }, - "linkedServiceName": { - "referenceName": "[variables('hubDataExplorerName')]", - "type": "LinkedServiceReference", - "parameters": { - "database": "[parameters('dataExplorerIngestionDatabase')]" - } - } - }, - { - "name": "Update ResourceTypes in ADX", - "type": "AzureDataExplorerCommand", - "dependsOn": [ - { - "activity": "Update Regions in ADX", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false + "publisher": { + "type": "string" }, - "userProperties": [], - "typeProperties": { - "command": "[format('.set-or-replace ResourceTypes <| externaldata(x_ResourceType: string, SingularDisplayName: string, PluralDisplayName: string, LowerSingularDisplayName: string, LowerPluralDisplayName: string, IsPreview: bool, Description: string, IconUri: string, Links: string)[@\"{0}/ResourceTypes.csv\"] with (format=\"csv\", ignoreFirstRecord=true) | project-away Links', variables('ftkReleaseUri'))]", - "commandTimeout": "00:20:00" + "suffix": { + "type": "string" }, - "linkedServiceName": { - "referenceName": "[variables('hubDataExplorerName')]", - "type": "LinkedServiceReference", - "parameters": { - "database": "[parameters('dataExplorerIngestionDatabase')]" - } - } - }, - { - "name": "Update Services in ADX", - "type": "AzureDataExplorerCommand", - "dependsOn": [ - { - "activity": "Update ResourceTypes in ADX", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false + "tags": { + "type": "object" }, - "userProperties": [], - "typeProperties": { - "command": "[format('.set-or-replace Services <| externaldata(x_ConsumedService: string, x_ResourceType: string, ServiceName: string, ServiceCategory: string, ServiceSubcategory: string, PublisherName: string, x_PublisherCategory: string, x_Environment: string, x_ServiceModel: string)[@\"{0}/Services.csv\"] with (format=\"csv\", ignoreFirstRecord=true)', variables('ftkReleaseUri'))]", - "commandTimeout": "00:20:00" + "dataFactory": { + "type": "string" }, - "linkedServiceName": { - "referenceName": "[variables('hubDataExplorerName')]", - "type": "LinkedServiceReference", - "parameters": { - "database": "[parameters('dataExplorerIngestionDatabase')]" - } + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" } }, - { - "name": "Ingestion Complete", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Update Services in ADX", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "tryAgain", - "value": false + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" } } - ] - } - }, - { - "name": "Abort On Error", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "If Has Capacity", - "dependencyConditions": [ - "Failed" + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" ] } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "tryAgain", - "value": false } } - ], - "timeout": "0.02:00:00" + }, + "dependsOn": [ + "appTelemetry", + "dataFactory", + "triggerManagerIdentity", + "triggerManagerRoleAssignments" + ] } }, - { - "name": "Timeout Error", - "type": "Fail", - "dependsOn": [ - { - "activity": "Until Capacity Is Available", - "dependencyConditions": [ - "Failed" - ] - } - ], - "userProperties": [], - "typeProperties": { - "message": "Data Explorer ingestion timed out after 2 hours while waiting for available capacity. Please re-run this pipeline to re-attempt ingestion. If you continue to see this error, please report an issue at https://aka.ms/ftk/ideas.", - "errorCode": "DataExplorerIngestionTimeout" + "outputs": { + "dataFactoryId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Data Factory instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]" + }, + "keyVaultId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Key Vault instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]" + }, + "storageAccountId": { + "type": "string", + "metadata": { + "description": "Resource ID of the storage account instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Principal ID for the managed identity used by Data Factory." + }, + "value": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" + }, + "triggerManagerIdentityName": { + "type": "string", + "metadata": { + "description": "Name of the managed identity used to create and stop ADF triggers." + }, + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" } } - ], - "concurrency": 1, - "variables": { - "version": { - "type": "String" + } + } + }, + "timeZones": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.CostManagement.ManagedExports_TimeZones", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "location": { + "value": "[parameters('app').hub.location]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "7746202288364701295" + } }, - "scopes": { - "type": "String" + "parameters": { + "location": { + "type": "string", + "defaultValue": "[resourceGroup().location]", + "metadata": { + "description": "Optional. The location to use for the managed identity and deployment script to auto-start triggers. Default = (resource group location)." + } + }, + "timezoneobject": { + "type": "object", + "defaultValue": { + "australiaeast": "AUS Eastern Standard Time", + "australiacentral": "AUS Eastern Standard Time", + "australiacentral2": "AUS Eastern Standard Time", + "australiasoutheast": "AUS Eastern Standard Time", + "brazilsouth": "E. South America Standard Time", + "canadacentral": "Central Standard Time", + "canadaeast": "Eastern Standard Time", + "centralindia": "India Standard Time", + "centralus": "Central Standard Time", + "eastasia": "China Standard Time", + "eastus": "Eastern Standard Time", + "eastus2": "Eastern Standard Time", + "francecentral": "W. Europe Standard Time", + "germanynorth": "W. Europe Standard Time", + "germanywestcentral": "W. Europe Standard Time", + "japaneast": "Japan Standard Time", + "japanwest": "Japan Standard Time", + "koreacentral": "Korea Standard Time", + "koreasouth": "Korea Standard Time", + "northcentralus": "Central Standard Time", + "northeurope": "GMT Standard Time", + "norwayeast": "W. Europe Standard Time", + "norwaywest": "W. Europe Standard Time", + "southcentralus": "Central Standard Time", + "southindia": "India Standard Time", + "southeastasia": "Singapore Standard Time", + "switzerlandnorth": "W. Europe Standard Time", + "switzerlandwest": "W. Europe Standard Time", + "uksouth": "GMT Standard Time", + "ukwest": "GMT Standard Time", + "westcentralus": "Central Standard Time", + "westeurope": "W. Europe Standard Time", + "westindia": "India Standard Time", + "westus": "Pacific Standard Time", + "westus2": "Pacific Standard Time" + } + }, + "utchrs": { + "type": "string", + "defaultValue": "[utcNow('hh')]" + }, + "utcmins": { + "type": "string", + "defaultValue": "[utcNow('mm')]" + }, + "utcsecs": { + "type": "string", + "defaultValue": "[utcNow('ss')]" + } }, - "retention": { - "type": "String" + "variables": { + "loc": "[toLower(replace(parameters('location'), ' ', ''))]", + "timezone": "[coalesce(tryGet(parameters('timezoneobject'), variables('loc')), 'Universal Coordinated Time')]" }, - "tryAgain": { - "type": "Boolean", - "defaultValue": true + "resources": [], + "outputs": { + "AzureRegion": { + "type": "string", + "value": "[parameters('location')]" + }, + "Timezone": { + "type": "string", + "value": "[variables('timezone')]" + }, + "UtcHours": { + "type": "string", + "value": "[parameters('utchrs')]" + }, + "UtcMinutes": { + "type": "string", + "value": "[parameters('utcmins')]" + }, + "UtcSeconds": { + "type": "string", + "value": "[parameters('utcsecs')]" + } } } - }, - "dependsOn": [ - "dataset_config", - "linkedService_dataExplorer" - ], - "metadata": { - "description": "Initializes the hub instance based on the configuration settings." } }, - "pipeline_StartBackfillProcess": { - "condition": "[parameters('enableManagedExports')]", - "type": "Microsoft.DataFactory/factories/pipelines", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), format('{0}_StartBackfillProcess', variables('safeConfigContainerName')))]", + "trigger_SettingsUpdated": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core_SettingsUpdatedTrigger", "properties": { - "activities": [ - { - "name": "Get Config", - "type": "Lookup", - "dependsOn": [], - "policy": { - "timeout": "0.00:05:00", - "retry": 2, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "source": { - "type": "JsonSource", - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": true, - "enablePartitionDiscovery": false - }, - "formatSettings": { - "type": "JsonReadSettings" - } - }, - "dataset": { - "referenceName": "[variables('safeConfigContainerName')]", - "type": "DatasetReference", - "parameters": { - "fileName": { - "value": "@variables('fileName')", - "type": "Expression" - }, - "folderPath": { - "value": "@variables('folderPath')", - "type": "Expression" - } - } - } + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "dataFactoryName": { + "value": "[parameters('app').dataFactory]" + }, + "triggerName": { + "value": "[format('{0}_SettingsUpdated', variables('CONFIG'))]" + }, + "pipelineName": { + "value": "[format('{0}_ConfigureExports', variables('CONFIG'))]" + }, + "pipelineParameters": { + "value": {} + }, + "storageAccountName": { + "value": "[parameters('app').storage]" + }, + "storageContainer": { + "value": "[variables('CONFIG')]" + }, + "storagePathEndsWith": { + "value": "settings.json" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "12667288354191065454" } }, - { - "name": "Set backfill end date", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Get Config", - "dependencyConditions": [ - "Succeeded" - ] + "parameters": { + "dataFactoryName": { + "type": "string", + "metadata": { + "description": "Required. Name of the publisher-specific Data Factory instance." } - ], - "userProperties": [], - "typeProperties": { - "variableName": "endDate", - "value": { - "value": "@addDays(startOfMonth(utcNow()), -1)", - "type": "Expression" + }, + "triggerName": { + "type": "string", + "metadata": { + "description": "Required. Name of the Data Factory trigger to create or update." } - } - }, - { - "name": "Set backfill start date", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Get Config", - "dependencyConditions": [ - "Succeeded" - ] + }, + "storageAccountName": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Azure storage container to monitor for updates and trigger events for." } - ], - "userProperties": [], - "typeProperties": { - "variableName": "startDate", - "value": { - "value": "@subtractFromTime(startOfMonth(utcNow()), activity('Get Config').output.firstRow.retention.ingestion.months, 'Month')", - "type": "Expression" + }, + "storageContainer": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Azure storage container to monitor for updates and trigger events for." } - } - }, - { - "name": "Set export start date", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Set backfill start date", - "dependencyConditions": [ - "Succeeded" - ] + }, + "storagePathStartsWith": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Beginning of the storage path within the specified storageContainer to monitor for updates and trigger events for." } - ], - "userProperties": [], - "typeProperties": { - "variableName": "thisMonth", - "value": { - "value": "@startOfMonth(variables('endDate'))", - "type": "Expression" + }, + "storagePathEndsWith": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. End of the storage path to monitor for updates and trigger events for." + } + }, + "pipelineName": { + "type": "string", + "metadata": { + "description": "Required. Name of the Data Factory pipeline to execute when the trigger is executed." + } + }, + "pipelineParameters": { + "type": "object", + "metadata": { + "description": "Required. Parameters to pass to the pipeline when the trigger is executed." } } }, - { - "name": "Set export end date", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Set export start date", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "variableName": "nextMonth", - "value": { - "value": "@startOfMonth(subtractFromTime(variables('thisMonth'), 1, 'Month'))", - "type": "Expression" + "resources": [ + { + "condition": "[not(empty(parameters('storageAccountName')))]", + "type": "Microsoft.DataFactory/factories/triggers", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('dataFactoryName'), parameters('triggerName'))]", + "properties": { + "annotations": [], + "pipelines": [ + { + "pipelineReference": { + "referenceName": "[parameters('pipelineName')]", + "type": "PipelineReference" + }, + "parameters": "[parameters('pipelineParameters')]" + } + ], + "type": "BlobEventsTrigger", + "typeProperties": { + "blobPathBeginsWith": "[format('/{0}/blobs/{1}', parameters('storageContainer'), parameters('storagePathStartsWith'))]", + "blobPathEndsWith": "[parameters('storagePathEndsWith')]", + "ignoreEmptyBlobs": true, + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]", + "events": [ + "Microsoft.Storage.BlobCreated" + ] + } } } - }, - { - "name": "Every Month", - "type": "Until", - "dependsOn": [ - { - "activity": "Set export end date", - "dependencyConditions": [ - "Succeeded" - ] - }, - { - "activity": "Set backfill end date", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "expression": { - "value": "@less(variables('thisMonth'), variables('startDate'))", - "type": "Expression" - }, - "activities": [ - { - "name": "Update export start date", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Backfill data", - "dependencyConditions": [ - "Completed" - ] - } - ], - "userProperties": [], - "typeProperties": { - "variableName": "thisMonth", - "value": { - "value": "@variables('nextMonth')", - "type": "Expression" - } - } - }, - { - "name": "Update export end date", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Update export start date", - "dependencyConditions": [ - "Completed" - ] - } - ], - "userProperties": [], - "typeProperties": { - "variableName": "nextMonth", - "value": { - "value": "@subtractFromTime(variables('thisMonth'), 1, 'Month')", - "type": "Expression" - } - } - }, - { - "name": "Backfill data", - "type": "ExecutePipeline", - "dependsOn": [], - "userProperties": [], - "typeProperties": { - "pipeline": { - "referenceName": "[format('{0}_RunBackfillJob', variables('safeConfigContainerName'))]", - "type": "PipelineReference" - }, - "waitOnCompletion": true, - "parameters": { - "StartDate": { - "value": "@variables('thisMonth')", - "type": "Expression" - }, - "EndDate": { - "value": "@addDays(addToTime(variables('thisMonth'), 1, 'Month'), -1)", - "type": "Expression" - } - } - } - } - ], - "timeout": "0.02:00:00" + ] + } + }, + "dependsOn": [ + "dataFactory::pipeline_ConfigureExports" + ] + } + } + } + }, + "dependsOn": [ + "cmExports" + ] + }, + "analytics": { + "condition": "[or(variables('useFabric'), variables('useAzureDataExplorer'))]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[__bicep.newApp(variables('hub'), 'Microsoft.FinOpsHubs', 'Analytics')]" + }, + "fabricQueryUri": { + "value": "[parameters('fabricQueryUri')]" + }, + "fabricCapacityUnits": { + "value": "[parameters('fabricCapacityUnits')]" + }, + "clusterName": { + "value": "[parameters('dataExplorerName')]" + }, + "clusterSku": { + "value": "[parameters('dataExplorerSku')]" + }, + "clusterCapacity": { + "value": "[parameters('dataExplorerCapacity')]" + }, + "rawRetentionInDays": { + "value": "[parameters('dataExplorerRawRetentionInDays')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6187428357849376543" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" } } - ], - "concurrency": 1, - "variables": { - "exportName": { - "type": "String" - }, - "storageAccountId": { - "type": "String", - "defaultValue": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]" - }, - "finOpsHub": { - "type": "String", - "defaultValue": "[parameters('hubName')]" - }, - "resourceManagementUri": { - "type": "String", - "defaultValue": "[environment().resourceManager]" - }, - "fileName": { - "type": "String", - "defaultValue": "settings.json" - }, - "folderPath": { - "type": "String", - "defaultValue": "[parameters('configContainerName')]" - }, - "endDate": { - "type": "String" - }, - "startDate": { - "type": "String" - }, - "thisMonth": { - "type": "String" + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "privateRoutingForLinkedServices": { + "parameters": [ + { + "$ref": "#/definitions/_1.HubProperties", + "name": "hub" + } + ], + "output": { + "type": "object", + "value": "[if(parameters('hub').options.privateRouting, createObject('connectVia', createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference')), createObject())]" }, - "nextMonth": { - "type": "String" + "metadata": { + "description": "Returns an object that represents the properties needed to enable private routing for linked services. Use property expansion (`...value`) to apply to a linkedServices resource.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } } } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "clusterName": { + "type": "string", + "defaultValue": "", + "maxLength": 22, + "metadata": { + "description": "Optional. Name of the Azure Data Explorer cluster to use for advanced analytics. If empty, Azure Data Explorer will not be deployed. Required to use with Power BI if you have more than $2-5M/mo in costs being monitored. Default: \"\" (do not use)." + } + }, + "clusterSku": { + "type": "string", + "defaultValue": "Dev(No SLA)_Standard_E2a_v4", + "allowedValues": [ + "Dev(No SLA)_Standard_E2a_v4", + "Dev(No SLA)_Standard_D11_v2", + "Standard_D11_v2", + "Standard_D12_v2", + "Standard_D13_v2", + "Standard_D14_v2", + "Standard_D16d_v5", + "Standard_D32d_v4", + "Standard_D32d_v5", + "Standard_DS13_v2+1TB_PS", + "Standard_DS13_v2+2TB_PS", + "Standard_DS14_v2+3TB_PS", + "Standard_DS14_v2+4TB_PS", + "Standard_E2a_v4", + "Standard_E2ads_v5", + "Standard_E2d_v4", + "Standard_E2d_v5", + "Standard_E4a_v4", + "Standard_E4ads_v5", + "Standard_E4d_v4", + "Standard_E4d_v5", + "Standard_E8a_v4", + "Standard_E8ads_v5", + "Standard_E8as_v4+1TB_PS", + "Standard_E8as_v4+2TB_PS", + "Standard_E8as_v5+1TB_PS", + "Standard_E8as_v5+2TB_PS", + "Standard_E8d_v4", + "Standard_E8d_v5", + "Standard_E8s_v4+1TB_PS", + "Standard_E8s_v4+2TB_PS", + "Standard_E8s_v5+1TB_PS", + "Standard_E8s_v5+2TB_PS", + "Standard_E16a_v4", + "Standard_E16ads_v5", + "Standard_E16as_v4+3TB_PS", + "Standard_E16as_v4+4TB_PS", + "Standard_E16as_v5+3TB_PS", + "Standard_E16as_v5+4TB_PS", + "Standard_E16d_v4", + "Standard_E16d_v5", + "Standard_E16s_v4+3TB_PS", + "Standard_E16s_v4+4TB_PS", + "Standard_E16s_v5+3TB_PS", + "Standard_E16s_v5+4TB_PS", + "Standard_E64i_v3", + "Standard_E80ids_v4", + "Standard_EC8ads_v5", + "Standard_EC8as_v5+1TB_PS", + "Standard_EC8as_v5+2TB_PS", + "Standard_EC16ads_v5", + "Standard_EC16as_v5+3TB_PS", + "Standard_EC16as_v5+4TB_PS", + "Standard_L4s", + "Standard_L8as_v3", + "Standard_L8s", + "Standard_L8s_v2", + "Standard_L8s_v3", + "Standard_L16as_v3", + "Standard_L16s", + "Standard_L16s_v2", + "Standard_L16s_v3", + "Standard_L32as_v3", + "Standard_L32s_v3" + ], + "metadata": { + "description": "Optional. Name of the Azure Data Explorer SKU. Default: \"Dev(No SLA)_Standard_E2a_v4\"." + } + }, + "clusterCapacity": { + "type": "int", + "defaultValue": 1, + "minValue": 1, + "maxValue": 1000, + "metadata": { + "description": "Optional. Number of nodes to use in the cluster. Allowed values: 1 for the Basic SKU tier and 2-1000 for Standard. Default: 1 for dev/test SKUs, 2 for standard SKUs." + } + }, + "fabricQueryUri": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Microsoft Fabric eventhouse query URI. Default: \"\" (do not use)." + } + }, + "fabricCapacityUnits": { + "type": "int", + "defaultValue": 2, + "minValue": 1, + "maxValue": 2048, + "metadata": { + "description": "Optional. Number of capacity units for the Microsoft Fabric capacity. This is the number in your Fabric SKU (e.g., Trial = 1, F2 = 2, F64 = 64). This is used to manage parallelization in data pipelines. If you change capacity, please redeploy the template. Allowed values: 1 for the Fabric trial and 2-2048 based on the assigned Fabric capacity (e.g., F2-F2048). Default: 2." + } + }, + "forceUpdateTag": { + "type": "string", + "defaultValue": "[utcNow()]", + "metadata": { + "description": "Optional. Forces the table to be updated if different from the last time it was deployed." + } + }, + "continueOnErrors": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + } + }, + "rawRetentionInDays": { + "type": "int", + "metadata": { + "description": "Required. Number of days of data to retain in the Data Explorer *_raw tables." + } + } + }, + "variables": { + "$fxv#0": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData/Internal')\r\n_resource_type_1(id: string) {\r\n dynamic({\r\n \"arizeai.observabilityeval/organizations\": { \"SingularDisplayName\": \"Azure Native Arize AI Cloud Service\" }\r\n ,\"astronomer.astro/organizations\": { \"SingularDisplayName\": \"Astro Organization\" }\r\n ,\"citrix.services/xenappessentials\": { \"SingularDisplayName\": \"Citrix Virtual Apps Essentials\" }\r\n ,\"citrix.services/xendesktopessentials\": { \"SingularDisplayName\": \"Citrix Virtual Desktops Essentials\" }\r\n ,\"commvault.contentstore/cloudaccounts\": { \"SingularDisplayName\": \"Commvault Cloud Account\" }\r\n ,\"commvault.contentstore/cloudaccounts/plans\": { \"SingularDisplayName\": \"Commvault.ContentStore cloud accounts plan\" }\r\n ,\"commvault.contentstore/cloudaccounts/protectiongroups\": { \"SingularDisplayName\": \"Commvault.ContentStore cloud accounts protection group\" }\r\n ,\"commvault.contentstore/cloudaccounts/protectiongroups/protecteditems\": { \"SingularDisplayName\": \"Commvault.ContentStore cloud accounts protection groups protected item\" }\r\n ,\"commvault.contentstore/cloudaccounts/storages\": { \"SingularDisplayName\": \"Commvault.ContentStore cloud accounts storage\" }\r\n ,\"dell.storage/filesystems\": { \"SingularDisplayName\": \"Dell PowerScale\" }\r\n ,\"dynatrace.observability/monitors\": { \"SingularDisplayName\": \"Dynatrace\" }\r\n ,\"github.network/networksettings\": { \"SingularDisplayName\": \"GitHub.Network network setting\" }\r\n ,\"informatica.datamanagement/organizations\": { \"SingularDisplayName\": \"Informatica Organization\" }\r\n ,\"lambdatest.hyperexecute/organizations\": { \"SingularDisplayName\": \"Azure Native LambdaTest - HyperExecute Cloud Service\" }\r\n ,\"microsoft.aad/domainservices\": { \"SingularDisplayName\": \"Microsoft Entra Domain Services\" }\r\n ,\"microsoft.aadiam/diagnosticsettings\": { \"SingularDisplayName\": \"Microsoft.aadiam diagnostic setting\" }\r\n ,\"microsoft.aadiam/privatelinkforazuread\": { \"SingularDisplayName\": \"Private Link for Microsoft Entra ID\" }\r\n ,\"microsoft.advisor/advisorscore\": { \"SingularDisplayName\": \"Microsoft.Advisor advisor score\" }\r\n ,\"microsoft.advisor/assessments\": { \"SingularDisplayName\": \"Microsoft.Advisor assessment\" }\r\n ,\"microsoft.advisor/configurations\": { \"SingularDisplayName\": \"Microsoft.Advisor configuration\" }\r\n ,\"microsoft.advisor/generaterecommendations\": { \"SingularDisplayName\": \"Microsoft.Advisor generate recommendation\" }\r\n ,\"microsoft.advisor/metadata\": { \"SingularDisplayName\": \"Microsoft.Advisor metadata\" }\r\n ,\"microsoft.advisor/recommendations\": { \"SingularDisplayName\": \"Microsoft.Advisor recommendation\" }\r\n ,\"microsoft.advisor/recommendations/suppressions\": { \"SingularDisplayName\": \"Microsoft.Advisor recommendations suppression\" }\r\n ,\"microsoft.advisor/resiliencyreviews\": { \"SingularDisplayName\": \"Microsoft.Advisor resiliency review\" }\r\n ,\"microsoft.agfoodplatform/farmbeats\": { \"SingularDisplayName\": \"Azure Data Manager for Agriculture\" }\r\n ,\"microsoft.agfoodplatform/farmbeatsextensiondefinitions\": { \"SingularDisplayName\": \"Microsoft.AgFoodPlatform farm beats extension definition\" }\r\n ,\"microsoft.agfoodplatform/farmbeatssolutiondefinitions\": { \"SingularDisplayName\": \"Microsoft.AgFoodPlatform farm beats solution definition\" }\r\n ,\"microsoft.agricultureplatform/agriservices\": { \"SingularDisplayName\": \"Agriculture data solutions\" }\r\n ,\"microsoft.akshybrid/agentpools\": { \"SingularDisplayName\": \"Microsoft.AksHybrid agent pool\" }\r\n ,\"microsoft.akshybrid/provisionedclusters\": { \"SingularDisplayName\": \"Microsoft.AksHybrid provisioned cluster\" }\r\n ,\"microsoft.akshybrid/upgradeprofiles\": { \"SingularDisplayName\": \"Microsoft.AksHybrid upgrade profile\" }\r\n ,\"microsoft.alertsmanagement/actionrules\": { \"SingularDisplayName\": \"Alert processing rule\" }\r\n ,\"microsoft.alertsmanagement/alerts\": { \"SingularDisplayName\": \"Microsoft.AlertsManagement alert\" }\r\n ,\"microsoft.alertsmanagement/alerts/enrichments\": { \"SingularDisplayName\": \"Microsoft.AlertsManagement alerts enrichment\" }\r\n ,\"microsoft.alertsmanagement/prometheusrulegroups\": { \"SingularDisplayName\": \"Prometheus rule group\" }\r\n ,\"microsoft.alertsmanagement/smartdetectoralertrules\": { \"SingularDisplayName\": \"Smart detector alert rule\" }\r\n ,\"microsoft.alertsmanagement/smartgroups\": { \"SingularDisplayName\": \"Microsoft.AlertsManagement smart group\" }\r\n ,\"microsoft.alertsmanagement/tenantactivitylogalerts\": { \"SingularDisplayName\": \"Microsoft.AlertsManagement tenant activity log alert\" }\r\n ,\"microsoft.all/arcvirtualmachines\": { \"SingularDisplayName\": \"Azure Arc virtual machine\" }\r\n ,\"microsoft.all/hcivirtualmachines\": { \"SingularDisplayName\": \"Azure Local Virtual Machine - Azure Arc\" }\r\n ,\"microsoft.all/virtualmachines\": { \"SingularDisplayName\": \"Virtual machine\" }\r\n ,\"microsoft.analysisservices/servers\": { \"SingularDisplayName\": \"Analysis Services server\" }\r\n ,\"microsoft.anybuild/clusters\": { \"SingularDisplayName\": \"AnyBuild cluster\" }\r\n ,\"microsoft.apicenter/deletedservices\": { \"SingularDisplayName\": \"Microsoft.ApiCenter deleted service\" }\r\n ,\"microsoft.apicenter/services\": { \"SingularDisplayName\": \"API Center\" }\r\n ,\"microsoft.apicenter/services/workspaces\": { \"SingularDisplayName\": \"Workspace\" }\r\n ,\"microsoft.apimanagement/gateways\": { \"SingularDisplayName\": \"API Management gateway\" }\r\n ,\"microsoft.apimanagement/gateways/configconnections\": { \"SingularDisplayName\": \"Microsoft.ApiManagement gateways config connection\" }\r\n ,\"microsoft.apimanagement/service\": { \"SingularDisplayName\": \"API Management service\" }\r\n ,\"microsoft.apimanagement/service/workspaces\": { \"SingularDisplayName\": \"Workspace\" }\r\n ,\"microsoft.apisecurity/defendersettings\": { \"SingularDisplayName\": \"Microsoft.ApiSecurity defender setting\" }\r\n ,\"microsoft.app/agents\": { \"SingularDisplayName\": \"SRE Agent\" }\r\n ,\"microsoft.app/builders\": { \"SingularDisplayName\": \"Microsoft.App builder\" }\r\n ,\"microsoft.app/builders/builds\": { \"SingularDisplayName\": \"Microsoft.App builders build\" }\r\n ,\"microsoft.app/connectedenvironments\": { \"SingularDisplayName\": \"Container Apps Connected Environment\" }\r\n ,\"microsoft.app/containerapps\": { \"SingularDisplayName\": \"Container App\" }\r\n ,\"microsoft.app/jobs\": { \"SingularDisplayName\": \"Container App Job\" }\r\n ,\"microsoft.app/logicapps\": { \"SingularDisplayName\": \"Logic app\" }\r\n ,\"microsoft.app/logicapps/workflows\": { \"SingularDisplayName\": \"Logic app workflow\" }\r\n ,\"microsoft.app/managedenvironments\": { \"SingularDisplayName\": \"Container Apps Environment\" }\r\n ,\"microsoft.app/sessionpools\": { \"SingularDisplayName\": \"Container App Session Pool\" }\r\n ,\"microsoft.app/spaces\": { \"SingularDisplayName\": \"App Space\" }\r\n ,\"microsoft.appassessment/migrateprojects\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate project\" }\r\n ,\"microsoft.appassessment/migrateprojects/assessments\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects assessment\" }\r\n ,\"microsoft.appassessment/migrateprojects/assessments/assessedapplications\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects assessments assessed application\" }\r\n ,\"microsoft.appassessment/migrateprojects/assessments/assessedmachines\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects assessments assessed machine\" }\r\n ,\"microsoft.appassessment/migrateprojects/assessments/machinestoassess\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects assessments machines to asses\" }\r\n ,\"microsoft.appassessment/migrateprojects/sites\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects site\" }\r\n ,\"microsoft.appassessment/migrateprojects/sites/applianceconfigurations\": { \"SingularDisplayName\": \"Microsoft.AppAssessment migrate projects sites appliance configuration\" }\r\n ,\"microsoft.appcomplianceautomation/reports\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation report\" }\r\n ,\"microsoft.appcomplianceautomation/reports/evidences\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation reports evidence\" }\r\n ,\"microsoft.appcomplianceautomation/reports/scopingconfigurations\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation reports scoping configuration\" }\r\n ,\"microsoft.appcomplianceautomation/reports/snapshots\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation reports snapshot\" }\r\n ,\"microsoft.appcomplianceautomation/reports/snapshots/controls\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation reports snapshots control\" }\r\n ,\"microsoft.appcomplianceautomation/reports/webhooks\": { \"SingularDisplayName\": \"Microsoft.AppComplianceAutomation reports webhook\" }\r\n ,\"microsoft.appconfiguration/configurationstores\": { \"SingularDisplayName\": \"App Configuration\" }\r\n ,\"microsoft.applicationmigration/discoveryhubs\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration discovery hub\" }\r\n ,\"microsoft.applicationmigration/discoveryhubs/applications\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration discovery hubs application\" }\r\n ,\"microsoft.applicationmigration/discoveryhubs/applications/members\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration discovery hubs applications member\" }\r\n ,\"microsoft.applicationmigration/pgsqlsites\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration pgsqlsite\" }\r\n ,\"microsoft.applicationmigration/pgsqlsites/agents\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration pgsqlsites agent\" }\r\n ,\"microsoft.applicationmigration/pgsqlsites/pgsqldatabases\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration pgsqlsites pgsqldatabase\" }\r\n ,\"microsoft.applicationmigration/pgsqlsites/pgsqlinstances\": { \"SingularDisplayName\": \"Microsoft.ApplicationMigration pgsqlsites pgsqlinstance\" }\r\n ,\"microsoft.appplatform/spring\": { \"SingularDisplayName\": \"Azure Spring Apps\" }\r\n ,\"microsoft.appsecurity/appprotectmanagedrulesetmanifests\": { \"SingularDisplayName\": \"Microsoft.AppSecurity app protect managed rule set manifest\" }\r\n ,\"microsoft.appsecurity/policies\": { \"SingularDisplayName\": \"App Protect Policy\" }\r\n ,\"microsoft.arc/all\": { \"SingularDisplayName\": \"Azure Arc enabled resource\" }\r\n ,\"microsoft.arc/allfairfax\": { \"SingularDisplayName\": \"Azure Arc enabled resource\" }\r\n ,\"microsoft.arc/kubernetesresources\": { \"SingularDisplayName\": \"Azure Arc Kubernetes cluster\" }\r\n ,\"microsoft.arc/kubernetesresourcesfairfax\": { \"SingularDisplayName\": \"Azure Arc Kubernetes cluster\" }\r\n ,\"microsoft.arcnetworking/arcnwloadbalancers\": { \"SingularDisplayName\": \"Microsoft.ArcNetworking arc nw load balancer\" }\r\n ,\"microsoft.aszlabhardware/labservers\": { \"SingularDisplayName\": \"Microsoft.AszLabHardware labserver\" }\r\n ,\"microsoft.aszlabhardware/reservations\": { \"SingularDisplayName\": \"Microsoft.AszLabHardware reservation\" }\r\n ,\"microsoft.aszlabhardware/reservations/servers\": { \"SingularDisplayName\": \"Microsoft.AszLabHardware reservations server\" }\r\n ,\"microsoft.aszlabhardware/servers\": { \"SingularDisplayName\": \"Microsoft.AszLabHardware server\" }\r\n ,\"microsoft.attestation/attestationproviders\": { \"SingularDisplayName\": \"Attestation provider\" }\r\n ,\"microsoft.authorization/accessreviewhistorydefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization access review history definition\" }\r\n ,\"microsoft.authorization/accessreviewscheduledefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization access review schedule definition\" }\r\n ,\"microsoft.authorization/accessreviewscheduledefinitions/instances\": { \"SingularDisplayName\": \"Microsoft.Authorization access review schedule definitions instance\" }\r\n ,\"microsoft.authorization/accessreviewscheduledefinitions/instances/decisions\": { \"SingularDisplayName\": \"Microsoft.Authorization access review schedule definitions instances decision\" }\r\n ,\"microsoft.authorization/accessreviewschedulesettings\": { \"SingularDisplayName\": \"Microsoft.Authorization access review schedule setting\" }\r\n ,\"microsoft.authorization/datapolicymanifests\": { \"SingularDisplayName\": \"Microsoft.Authorization data policy manifest\" }\r\n ,\"microsoft.authorization/denyassignments\": { \"SingularDisplayName\": \"Microsoft.Authorization deny assignment\" }\r\n ,\"microsoft.authorization/locks\": { \"SingularDisplayName\": \"Microsoft.Authorization lock\" }\r\n ,\"microsoft.authorization/policyassignments\": { \"SingularDisplayName\": \"Microsoft.Authorization policy assignment\" }\r\n ,\"microsoft.authorization/policydefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization policy definition\" }\r\n ,\"microsoft.authorization/policydefinitions/versions\": { \"SingularDisplayName\": \"Microsoft.Authorization policy definitions version\" }\r\n ,\"microsoft.authorization/policyexemptions\": { \"SingularDisplayName\": \"Microsoft.Authorization policy exemption\" }\r\n ,\"microsoft.authorization/policysetdefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization policy set definition\" }\r\n ,\"microsoft.authorization/policysetdefinitions/versions\": { \"SingularDisplayName\": \"Microsoft.Authorization policy set definitions version\" }\r\n ,\"microsoft.authorization/privatelinkassociations\": { \"SingularDisplayName\": \"Microsoft.Authorization private link association\" }\r\n ,\"microsoft.authorization/provideroperations\": { \"SingularDisplayName\": \"Microsoft.Authorization provider operation\" }\r\n ,\"microsoft.authorization/resourcemanagementprivatelinks\": { \"SingularDisplayName\": \"Resource management private link\" }\r\n ,\"microsoft.authorization/roleassignmentapprovals\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment approval\" }\r\n ,\"microsoft.authorization/roleassignmentapprovals/stages\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment approvals stage\" }\r\n ,\"microsoft.authorization/roleassignments\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment\" }\r\n ,\"microsoft.authorization/roleassignmentscheduleinstances\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment schedule instance\" }\r\n ,\"microsoft.authorization/roleassignmentschedulerequests\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment schedule request\" }\r\n ,\"microsoft.authorization/roleassignmentschedules\": { \"SingularDisplayName\": \"Microsoft.Authorization role assignment schedule\" }\r\n ,\"microsoft.authorization/roledefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization role definition\" }\r\n ,\"microsoft.authorization/roleeligibilityscheduleinstances\": { \"SingularDisplayName\": \"Microsoft.Authorization role eligibility schedule instance\" }\r\n ,\"microsoft.authorization/roleeligibilityschedulerequests\": { \"SingularDisplayName\": \"Microsoft.Authorization role eligibility schedule request\" }\r\n ,\"microsoft.authorization/roleeligibilityschedules\": { \"SingularDisplayName\": \"Microsoft.Authorization role eligibility schedule\" }\r\n ,\"microsoft.authorization/rolemanagementalertconfigurations\": { \"SingularDisplayName\": \"Microsoft.Authorization role management alert configuration\" }\r\n ,\"microsoft.authorization/rolemanagementalertdefinitions\": { \"SingularDisplayName\": \"Microsoft.Authorization role management alert definition\" }\r\n ,\"microsoft.authorization/rolemanagementalertoperations\": { \"SingularDisplayName\": \"Microsoft.Authorization role management alert operation\" }\r\n ,\"microsoft.authorization/rolemanagementalerts\": { \"SingularDisplayName\": \"Microsoft.Authorization role management alert\" }\r\n ,\"microsoft.authorization/rolemanagementalerts/alertincidents\": { \"SingularDisplayName\": \"Microsoft.Authorization role management alerts alert incident\" }\r\n ,\"microsoft.authorization/rolemanagementpolicies\": { \"SingularDisplayName\": \"Microsoft.Authorization role management policy\" }\r\n ,\"microsoft.authorization/rolemanagementpolicyassignments\": { \"SingularDisplayName\": \"Microsoft.Authorization role management policy assignment\" }\r\n ,\"microsoft.automanage/bestpractices\": { \"SingularDisplayName\": \"Microsoft.Automanage best practice\" }\r\n ,\"microsoft.automanage/bestpractices/versions\": { \"SingularDisplayName\": \"Microsoft.Automanage best practices version\" }\r\n ,\"microsoft.automanage/configurationprofileassignments\": { \"SingularDisplayName\": \"Microsoft.Automanage configuration profile assignment\" }\r\n ,\"microsoft.automanage/configurationprofileassignments/reports\": { \"SingularDisplayName\": \"Microsoft.Automanage configuration profile assignments report\" }\r\n ,\"microsoft.automanage/configurationprofiles\": { \"SingularDisplayName\": \"Microsoft.Automanage configuration profile\" }\r\n ,\"microsoft.automanage/configurationprofiles/versions\": { \"SingularDisplayName\": \"Microsoft.Automanage configuration profiles version\" }\r\n ,\"microsoft.automanage/serviceprincipals\": { \"SingularDisplayName\": \"ServicePrincipals\" }\r\n ,\"microsoft.automation/automationaccounts\": { \"SingularDisplayName\": \"Automation account\" }\r\n ,\"microsoft.automation/automationaccounts/hybridrunbookworkergroups\": { \"SingularDisplayName\": \"Automation hybrid worker group\" }\r\n ,\"microsoft.automation/automationaccounts/runbooks\": { \"SingularDisplayName\": \"Automation runbook\" }\r\n ,\"microsoft.autonomousdevelopmentplatform/accounts\": { \"SingularDisplayName\": \"Microsoft.AutonomousDevelopmentPlatform account\" }\r\n ,\"microsoft.autonomousdevelopmentplatform/accounts/datapools\": { \"SingularDisplayName\": \"Microsoft.AutonomousDevelopmentPlatform accounts data pool\" }\r\n ,\"microsoft.autonomousdevelopmentplatform/workspaces\": { \"SingularDisplayName\": \"Microsoft.AutonomousDevelopmentPlatform workspace\" }\r\n ,\"microsoft.avs/privateclouds\": { \"SingularDisplayName\": \"Azure VMware Solution private cloud\" }\r\n ,\"microsoft.awsconnector/accessanalyzeranalyzers\": { \"SingularDisplayName\": \"Access Analyzer Analyzer\" }\r\n ,\"microsoft.awsconnector/acmcertificatesummaries\": { \"SingularDisplayName\": \"ACM Certificate Summary\" }\r\n ,\"microsoft.awsconnector/apigatewayrestapis\": { \"SingularDisplayName\": \"Api Gateway Rest Api\" }\r\n ,\"microsoft.awsconnector/apigatewaystages\": { \"SingularDisplayName\": \"Api Gateway Stage\" }\r\n ,\"microsoft.awsconnector/applicationautoscalingscalabletargets\": { \"SingularDisplayName\": \"Application Auto Scaling Scalable Target\" }\r\n ,\"microsoft.awsconnector/appsyncgraphqlapis\": { \"SingularDisplayName\": \"App Sync Graphql Api\" }\r\n ,\"microsoft.awsconnector/autoscalingautoscalinggroups\": { \"SingularDisplayName\": \"Auto Scaling Auto Scaling Group\" }\r\n ,\"microsoft.awsconnector/cloudformationstacks\": { \"SingularDisplayName\": \"Cloud Formation Stack\" }\r\n ,\"microsoft.awsconnector/cloudformationstacksets\": { \"SingularDisplayName\": \"Cloud Formation Stack Set\" }\r\n ,\"microsoft.awsconnector/cloudfrontdistributions\": { \"SingularDisplayName\": \"Cloud Front Distribution\" }\r\n ,\"microsoft.awsconnector/cloudtrailtrails\": { \"SingularDisplayName\": \"Cloud Trail Trail\" }\r\n ,\"microsoft.awsconnector/cloudwatchalarms\": { \"SingularDisplayName\": \"Cloud Watch Alarm\" }\r\n ,\"microsoft.awsconnector/codebuildprojects\": { \"SingularDisplayName\": \"Code Build Project\" }\r\n ,\"microsoft.awsconnector/codebuildsourcecredentialsinfos\": { \"SingularDisplayName\": \"Code Build Source Credentials Info\" }\r\n ,\"microsoft.awsconnector/configserviceconfigurationrecorders\": { \"SingularDisplayName\": \"Config Service Configuration Recorder\" }\r\n ,\"microsoft.awsconnector/configserviceconfigurationrecorderstatuses\": { \"SingularDisplayName\": \"Config Service Configuration Recorder Status\" }\r\n ,\"microsoft.awsconnector/configservicedeliverychannels\": { \"SingularDisplayName\": \"Config Service Delivery Channel\" }\r\n ,\"microsoft.awsconnector/databasemigrationservicereplicationinstances\": { \"SingularDisplayName\": \"Database Migration Service Replication Instance\" }\r\n ,\"microsoft.awsconnector/daxclusters\": { \"SingularDisplayName\": \"DAX Cluster\" }\r\n ,\"microsoft.awsconnector/dynamodbcontinuousbackupsdescriptions\": { \"SingularDisplayName\": \"Dynamo DB Continuous Backups Description\" }\r\n ,\"microsoft.awsconnector/dynamodbtables\": { \"SingularDisplayName\": \"Dynamo DB Table\" }\r\n ,\"microsoft.awsconnector/ec2accountattributes\": { \"SingularDisplayName\": \"EC2 Account Attribute\" }\r\n ,\"microsoft.awsconnector/ec2addresses\": { \"SingularDisplayName\": \"EC2 Address\" }\r\n ,\"microsoft.awsconnector/ec2flowlogs\": { \"SingularDisplayName\": \"EC2 Flow Log\" }\r\n ,\"microsoft.awsconnector/ec2images\": { \"SingularDisplayName\": \"EC2 Image\" }\r\n ,\"microsoft.awsconnector/ec2instances\": { \"SingularDisplayName\": \"Microsoft.AwsConnector ec2 instance\" }\r\n ,\"microsoft.awsconnector/ec2instancestatuses\": { \"SingularDisplayName\": \"EC2 Instance Status\" }\r\n ,\"microsoft.awsconnector/ec2ipams\": { \"SingularDisplayName\": \"EC2 Ipam\" }\r\n ,\"microsoft.awsconnector/ec2keypairs\": { \"SingularDisplayName\": \"EC2 Key Pair\" }\r\n ,\"microsoft.awsconnector/ec2networkacls\": { \"SingularDisplayName\": \"EC2 Network Acl\" }\r\n ,\"microsoft.awsconnector/ec2networkinterfaces\": { \"SingularDisplayName\": \"EC2 Network Interface\" }\r\n ,\"microsoft.awsconnector/ec2routetables\": { \"SingularDisplayName\": \"EC2 Route Table\" }\r\n ,\"microsoft.awsconnector/ec2securitygroups\": { \"SingularDisplayName\": \"EC2 Security Group\" }\r\n ,\"microsoft.awsconnector/ec2snapshots\": { \"SingularDisplayName\": \"EC2 Snapshot\" }\r\n ,\"microsoft.awsconnector/ec2subnets\": { \"SingularDisplayName\": \"EC2 Subnet\" }\r\n ,\"microsoft.awsconnector/ec2volumes\": { \"SingularDisplayName\": \"EC2 Volume\" }\r\n ,\"microsoft.awsconnector/ec2vpcendpoints\": { \"SingularDisplayName\": \"EC2 VPCEndpoint\" }\r\n ,\"microsoft.awsconnector/ec2vpcpeeringconnections\": { \"SingularDisplayName\": \"EC2 VPCPeering Connection\" }\r\n ,\"microsoft.awsconnector/ec2vpcs\": { \"SingularDisplayName\": \"EC2 VPC\" }\r\n ,\"microsoft.awsconnector/ecrimagedetails\": { \"SingularDisplayName\": \"ECR Image Detail\" }\r\n ,\"microsoft.awsconnector/ecrrepositories\": { \"SingularDisplayName\": \"ECR Repository\" }\r\n ,\"microsoft.awsconnector/ecsclusters\": { \"SingularDisplayName\": \"ECS Cluster\" }\r\n ,\"microsoft.awsconnector/ecsservices\": { \"SingularDisplayName\": \"ECS Service\" }\r\n ,\"microsoft.awsconnector/ecstaskdefinitions\": { \"SingularDisplayName\": \"ECS Task Definition\" }\r\n ,\"microsoft.awsconnector/efsfilesystems\": { \"SingularDisplayName\": \"EFS File System\" }\r\n ,\"microsoft.awsconnector/efsmounttargets\": { \"SingularDisplayName\": \"EFS Mount Target\" }\r\n ,\"microsoft.awsconnector/eksnodegroups\": { \"SingularDisplayName\": \"EKS Nodegroup\" }\r\n ,\"microsoft.awsconnector/elasticbeanstalkapplications\": { \"SingularDisplayName\": \"Elastic Beanstalk Application\" }\r\n ,\"microsoft.awsconnector/elasticbeanstalkconfigurationtemplates\": { \"SingularDisplayName\": \"Elastic Beanstalk Configuration Template\" }\r\n ,\"microsoft.awsconnector/elasticbeanstalkenvironments\": { \"SingularDisplayName\": \"Elastic Beanstalk Environment\" }\r\n ,\"microsoft.awsconnector/elasticloadbalancingv2listeners\": { \"SingularDisplayName\": \"Elastic Load Balancing V2 Listener\" }\r\n ,\"microsoft.awsconnector/elasticloadbalancingv2loadbalancers\": { \"SingularDisplayName\": \"Elastic Load Balancing V2 Load Balancer\" }\r\n ,\"microsoft.awsconnector/elasticloadbalancingv2targetgroups\": { \"SingularDisplayName\": \"Elastic Load Balancing V2 Target Group\" }\r\n ,\"microsoft.awsconnector/elasticloadbalancingv2targethealthdescriptions\": { \"SingularDisplayName\": \"Elastic Load Balancing v2 Target Health Description\" }\r\n ,\"microsoft.awsconnector/elasticsearchdomains\": { \"SingularDisplayName\": \"Elasticsearch Domain\" }\r\n ,\"microsoft.awsconnector/emrclusters\": { \"SingularDisplayName\": \"EMR Cluster\" }\r\n ,\"microsoft.awsconnector/guarddutydetectors\": { \"SingularDisplayName\": \"Guard Duty Detector\" }\r\n ,\"microsoft.awsconnector/iamaccesskeylastuseds\": { \"SingularDisplayName\": \"IAM Access Key Last Used\" }\r\n ,\"microsoft.awsconnector/iamaccesskeymetadata\": { \"SingularDisplayName\": \"IAM Access Key Metadata\" }\r\n ,\"microsoft.awsconnector/iamgroups\": { \"SingularDisplayName\": \"IAM Group\" }\r\n ,\"microsoft.awsconnector/iaminstanceprofiles\": { \"SingularDisplayName\": \"IAM Instance Profile\" }\r\n ,\"microsoft.awsconnector/iammanagedpolicies\": { \"SingularDisplayName\": \"IAM Managed Policy\" }\r\n ,\"microsoft.awsconnector/iammfadevices\": { \"SingularDisplayName\": \"IAM MFADevice\" }\r\n ,\"microsoft.awsconnector/iampasswordpolicies\": { \"SingularDisplayName\": \"IAM Password Policy\" }\r\n ,\"microsoft.awsconnector/iampolicyversions\": { \"SingularDisplayName\": \"IAM Policy Version\" }\r\n ,\"microsoft.awsconnector/iamroles\": { \"SingularDisplayName\": \"IAM Role\" }\r\n ,\"microsoft.awsconnector/iamservercertificates\": { \"SingularDisplayName\": \"IAM Server Certificate\" }\r\n ,\"microsoft.awsconnector/iamuserpolicies\": { \"SingularDisplayName\": \"IAM User Policy\" }\r\n ,\"microsoft.awsconnector/iamvirtualmfadevices\": { \"SingularDisplayName\": \"IAM Virtual MFADevice\" }\r\n ,\"microsoft.awsconnector/kmsaliases\": { \"SingularDisplayName\": \"KMS Alias\" }\r\n ,\"microsoft.awsconnector/kmskeys\": { \"SingularDisplayName\": \"KMS Key\" }\r\n ,\"microsoft.awsconnector/lambdafunctioncodelocations\": { \"SingularDisplayName\": \"Lambda Function Code Location\" }\r\n ,\"microsoft.awsconnector/lambdafunctionconfigurations\": { \"SingularDisplayName\": \"Microsoft.AwsConnector lambda function configuration\" }\r\n ,\"microsoft.awsconnector/lambdafunctions\": { \"SingularDisplayName\": \"Lambda Function\" }\r\n ,\"microsoft.awsconnector/licensemanagerlicenses\": { \"SingularDisplayName\": \"License Manager License\" }\r\n ,\"microsoft.awsconnector/lightsailbuckets\": { \"SingularDisplayName\": \"Lightsail Bucket\" }\r\n ,\"microsoft.awsconnector/lightsailinstances\": { \"SingularDisplayName\": \"Lightsail Instance\" }\r\n ,\"microsoft.awsconnector/logsloggroups\": { \"SingularDisplayName\": \"Logs Log Group\" }\r\n ,\"microsoft.awsconnector/logslogstreams\": { \"SingularDisplayName\": \"Logs Log Stream\" }\r\n ,\"microsoft.awsconnector/logsmetricfilters\": { \"SingularDisplayName\": \"Logs Metric Filter\" }\r\n ,\"microsoft.awsconnector/logssubscriptionfilters\": { \"SingularDisplayName\": \"Logs Subscription Filter\" }\r\n ,\"microsoft.awsconnector/macie2jobsummaries\": { \"SingularDisplayName\": \"Macie2 Job Summary\" }\r\n ,\"microsoft.awsconnector/macieallowlists\": { \"SingularDisplayName\": \"Macie Allow List\" }\r\n ,\"microsoft.awsconnector/networkfirewallfirewallpolicies\": { \"SingularDisplayName\": \"Network Firewall Firewall Policy\" }\r\n ,\"microsoft.awsconnector/networkfirewallfirewalls\": { \"SingularDisplayName\": \"Network Firewall Firewall\" }\r\n ,\"microsoft.awsconnector/networkfirewallrulegroups\": { \"SingularDisplayName\": \"Network Firewall Rule Group\" }\r\n ,\"microsoft.awsconnector/opensearchdomainstatuses\": { \"SingularDisplayName\": \"Open Search Domain Status\" }\r\n ,\"microsoft.awsconnector/opensearchservicedomains\": { \"SingularDisplayName\": \"Open Search Service Domain\" }\r\n ,\"microsoft.awsconnector/organizationsaccounts\": { \"SingularDisplayName\": \"Organizations Account\" }\r\n ,\"microsoft.awsconnector/organizationsorganizations\": { \"SingularDisplayName\": \"Organizations Organization\" }\r\n ,\"microsoft.awsconnector/rdsdbclusters\": { \"SingularDisplayName\": \"RDS DBCluster\" }\r\n ,\"microsoft.awsconnector/rdsdbinstances\": { \"SingularDisplayName\": \"RDS DBInstance\" }\r\n ,\"microsoft.awsconnector/rdsdbsnapshotattributesresults\": { \"SingularDisplayName\": \"RDS DBSnapshot Attributes Result\" }\r\n ,\"microsoft.awsconnector/rdsdbsnapshots\": { \"SingularDisplayName\": \"RDS DBSnapshot\" }\r\n ,\"microsoft.awsconnector/rdseventsubscriptions\": { \"SingularDisplayName\": \"RDS Event Subscription\" }\r\n ,\"microsoft.awsconnector/rdsexporttasks\": { \"SingularDisplayName\": \"RDS Export Task\" }\r\n ,\"microsoft.awsconnector/redshiftclusterparametergroups\": { \"SingularDisplayName\": \"Redshift Cluster Parameter Group\" }\r\n ,\"microsoft.awsconnector/redshiftclusters\": { \"SingularDisplayName\": \"Redshift Cluster\" }\r\n ,\"microsoft.awsconnector/route53domainsdomainsummaries\": { \"SingularDisplayName\": \"Route 53 Domains Domain Summary\" }\r\n ,\"microsoft.awsconnector/route53hostedzones\": { \"SingularDisplayName\": \"Route53 Hosted Zone\" }\r\n ,\"microsoft.awsconnector/route53resourcerecordsets\": { \"SingularDisplayName\": \"Route 53 Resource Record Set\" }\r\n ,\"microsoft.awsconnector/s3accesscontrolpolicies\": { \"SingularDisplayName\": \"S3 Access Control Policy\" }\r\n ,\"microsoft.awsconnector/s3accesspoints\": { \"SingularDisplayName\": \"S3 Access Point\" }\r\n ,\"microsoft.awsconnector/s3bucketpolicies\": { \"SingularDisplayName\": \"S3 Bucket Policy\" }\r\n ,\"microsoft.awsconnector/s3buckets\": { \"SingularDisplayName\": \"S3 Bucket\" }\r\n ,\"microsoft.awsconnector/s3controlmultiregionaccesspointpolicydocuments\": { \"SingularDisplayName\": \"S3 Control Multi Region Access Point Policy Document\" }\r\n ,\"microsoft.awsconnector/sagemakerapps\": { \"SingularDisplayName\": \"Sage Maker App\" }\r\n ,\"microsoft.awsconnector/sagemakerdevices\": { \"SingularDisplayName\": \"Sage Maker Device\" }\r\n ,\"microsoft.awsconnector/sagemakerimages\": { \"SingularDisplayName\": \"Sage Maker Image\" }\r\n ,\"microsoft.awsconnector/sagemakernotebookinstancesummaries\": { \"SingularDisplayName\": \"Sage Maker Notebook Instance Summary\" }\r\n ,\"microsoft.awsconnector/secretsmanagerresourcepolicies\": { \"SingularDisplayName\": \"Secrets Manager Resource Policy\" }\r\n ,\"microsoft.awsconnector/secretsmanagersecrets\": { \"SingularDisplayName\": \"Secrets Manager Secret\" }\r\n ,\"microsoft.awsconnector/snssubscriptions\": { \"SingularDisplayName\": \"SNS Subscription\" }\r\n ,\"microsoft.awsconnector/snstopics\": { \"SingularDisplayName\": \"SNS Topic\" }\r\n ,\"microsoft.awsconnector/sqsqueues\": { \"SingularDisplayName\": \"SQS Queue\" }\r\n ,\"microsoft.awsconnector/ssminstanceinformations\": { \"SingularDisplayName\": \"SSM Instance Information\" }\r\n ,\"microsoft.awsconnector/ssmparameters\": { \"SingularDisplayName\": \"SSM Parameter\" }\r\n ,\"microsoft.awsconnector/ssmresourcecompliancesummaryitems\": { \"SingularDisplayName\": \"SSM Resource Compliance Summary Item\" }\r\n ,\"microsoft.awsconnector/wafv2ipsets\": { \"SingularDisplayName\": \"WAFv2 IPSet\" }\r\n ,\"microsoft.awsconnector/wafv2loggingconfigurations\": { \"SingularDisplayName\": \"WAFv2 Logging Configuration\" }\r\n ,\"microsoft.awsconnector/wafv2webaclassociations\": { \"SingularDisplayName\": \"WAFv2 Web ACLAssociation\" }\r\n ,\"microsoft.awsconnector/wafwebaclsummaries\": { \"SingularDisplayName\": \"WAF Web ACLSummary\" }\r\n ,\"microsoft.azureactivedirectory/b2cdirectories\": { \"SingularDisplayName\": \"B2C tenant\" }\r\n ,\"microsoft.azureactivedirectory/ciamdirectories\": { \"SingularDisplayName\": \"External Configuration Tenant\" }\r\n ,\"microsoft.azureactivedirectory/guestusages\": { \"SingularDisplayName\": \"Guest Usage\" }\r\n ,\"microsoft.azurearcdata/datacontrollers\": { \"SingularDisplayName\": \"Azure Arc data controller\" }\r\n ,\"microsoft.azurearcdata/mysqlserver\": { \"SingularDisplayName\": \"MySql Server - Azure Arc\" }\r\n ,\"microsoft.azurearcdata/postgresinstances\": { \"SingularDisplayName\": \"PostgreSQL server ? Azure Arc\" }\r\n ,\"microsoft.azurearcdata/postgressqlserver\": { \"SingularDisplayName\": \"PostgresSql Server - Azure Arc\" }\r\n ,\"microsoft.azurearcdata/sqlmanagedinstances\": { \"SingularDisplayName\": \"SQL managed instance - Azure Arc\" }\r\n ,\"microsoft.azurearcdata/sqlserveresulicenses\": { \"SingularDisplayName\": \"SQL Server ESU license\" }\r\n ,\"microsoft.azurearcdata/sqlserverinstances\": { \"SingularDisplayName\": \"SQL Server - Azure Arc\" }\r\n ,\"microsoft.azurearcdata/sqlserverinstances/databases\": { \"SingularDisplayName\": \"SQL Server database - Azure Arc\" }\r\n ,\"microsoft.azurearcdata/sqlserverlicenses\": { \"SingularDisplayName\": \"SQL Server License\" }\r\n ,\"microsoft.azurebusinesscontinuity/deletedunifiedprotecteditems\": { \"SingularDisplayName\": \"Microsoft.AzureBusinessContinuity deleted unified protected item\" }\r\n ,\"microsoft.azurebusinesscontinuity/unifiedprotecteditems\": { \"SingularDisplayName\": \"Microsoft.AzureBusinessContinuity unified protected item\" }\r\n ,\"microsoft.azurecis/aadapplications\": { \"SingularDisplayName\": \"Microsoft.AzureCis AAD application\" }\r\n ,\"microsoft.azurecis/addressrecords\": { \"SingularDisplayName\": \"Microsoft.AzureCis address record\" }\r\n ,\"microsoft.azurecis/autopilotenvironments\": { \"SingularDisplayName\": \"Microsoft.AzureCis autopilot environment\" }\r\n ,\"microsoft.azurecis/autopilotmachinefunctions\": { \"SingularDisplayName\": \"Microsoft.AzureCis autopilot machine function\" }\r\n ,\"microsoft.azurecis/autopilotsoftwareloadbalancevirtualips\": { \"SingularDisplayName\": \"Microsoft.AzureCis auto pilot software load balance virtual IP\" }\r\n ,\"microsoft.azurecis/azcopies\": { \"SingularDisplayName\": \"Microsoft.AzureCis az copy\" }\r\n ,\"microsoft.azurecis/canonicalnamerecords\": { \"SingularDisplayName\": \"Microsoft.AzureCis canonical name record\" }\r\n ,\"microsoft.azurecis/dsmsallowlists\": { \"SingularDisplayName\": \"Microsoft.AzureCis ds msallowlist\" }\r\n ,\"microsoft.azurecis/dsmscertificates\": { \"SingularDisplayName\": \"Microsoft.AzureCis dsms certificate\" }\r\n ,\"microsoft.azurecis/dsmsrootfolders\": { \"SingularDisplayName\": \"Microsoft.AzureCis dsms root folder\" }\r\n ,\"microsoft.azurecis/dstsapplications\": { \"SingularDisplayName\": \"Microsoft.AzureCis dsts application\" }\r\n ,\"microsoft.azurecis/dstsserviceaccounts\": { \"SingularDisplayName\": \"Microsoft.AzureCis dsts service account\" }\r\n ,\"microsoft.azurecis/dstsserviceclientidentities\": { \"SingularDisplayName\": \"Microsoft.AzureCis dsts service client identity\" }\r\n ,\"microsoft.azurecis/genericgenevaactions\": { \"SingularDisplayName\": \"Microsoft.AzureCis generic geneva action\" }\r\n ,\"microsoft.azurecis/plannedquotas\": { \"SingularDisplayName\": \"Microsoft.AzureCis planned quota\" }\r\n ,\"microsoft.azurecis/pointerrecords\": { \"SingularDisplayName\": \"Microsoft.AzureCis pointer record\" }\r\n ,\"microsoft.azurecis/publishconfigvalues\": { \"SingularDisplayName\": \"Microsoft.AzureCis publish config value\" }\r\n ,\"microsoft.azurecis/pushagentv2accounts\": { \"SingularDisplayName\": \"Microsoft.AzureCis push agent v2 account\" }\r\n ,\"microsoft.azurecis/servicerecords\": { \"SingularDisplayName\": \"Microsoft.AzureCis service record\" }\r\n ,\"microsoft.azurecis/sharedconfigvalues\": { \"SingularDisplayName\": \"Microsoft.AzureCis shared config value\" }\r\n ,\"microsoft.azurecloudmetadata/clouds\": { \"SingularDisplayName\": \"Microsoft.AzureCloudMetadata cloud\" }\r\n ,\"microsoft.azurecloudmetadata/clouds/geographies\": { \"SingularDisplayName\": \"Microsoft.AzureCloudMetadata clouds geography\" }\r\n ,\"microsoft.azurecloudmetadata/clouds/geographies/regions\": { \"SingularDisplayName\": \"Microsoft.AzureCloudMetadata clouds geographies region\" }\r\n ,\"microsoft.azuredatatransfer/connections\": { \"SingularDisplayName\": \"Connection\" }\r\n ,\"microsoft.azuredatatransfer/connections/flows\": { \"SingularDisplayName\": \"Flow\" }\r\n ,\"microsoft.azuredatatransfer/pipelines\": { \"SingularDisplayName\": \"Pipeline\" }\r\n ,\"microsoft.azurefleet/fleets\": { \"SingularDisplayName\": \"Compute Fleet\" }\r\n ,\"microsoft.azurefleet/fleetscomputehub\": { \"SingularDisplayName\": \"Compute Fleet\" }\r\n ,\"microsoft.azureimagetestingforlinux/jobs\": { \"SingularDisplayName\": \"Microsoft.AzureImageTestingForLinux job\" }\r\n ,\"microsoft.azureimagetestingforlinux/jobtemplates\": { \"SingularDisplayName\": \"Microsoft.AzureImageTestingForLinux job template\" }\r\n ,\"microsoft.azurelargeinstance/azurelargeinstances\": { \"SingularDisplayName\": \"Azure Large Instance\" }\r\n ,\"microsoft.azurelargeinstance/azurelargestorageinstances\": { \"SingularDisplayName\": \"Microsoft.AzureLargeInstance Azure large storage instance\" }\r\n ,\"microsoft.azurepercept/accounts\": { \"SingularDisplayName\": \"Microsoft.AzurePercept account\" }\r\n ,\"microsoft.azurepercept/accounts/devices\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts device\" }\r\n ,\"microsoft.azurepercept/accounts/devices/sensors\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts devices sensor\" }\r\n ,\"microsoft.azurepercept/accounts/sensors\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts sensor\" }\r\n ,\"microsoft.azurepercept/accounts/solutioninstances\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts solutioninstance\" }\r\n ,\"microsoft.azurepercept/accounts/solutions\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts solution\" }\r\n ,\"microsoft.azurepercept/accounts/targets\": { \"SingularDisplayName\": \"Microsoft.AzurePercept accounts target\" }\r\n ,\"microsoft.azureplaywrightservice/accounts\": { \"SingularDisplayName\": \"Playwright Testing\" }\r\n ,\"microsoft.azurescan/scanningaccounts\": { \"SingularDisplayName\": \"ESRP Scan\" }\r\n ,\"microsoft.azuresphere/catalogs\": { \"SingularDisplayName\": \"Azure Sphere Catalog\" }\r\n ,\"microsoft.azurespherev2/catalogs\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalog\" }\r\n ,\"microsoft.azurespherev2/catalogs/artifacts\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs artifact\" }\r\n ,\"microsoft.azurespherev2/catalogs/certificates\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs certificate\" }\r\n ,\"microsoft.azurespherev2/catalogs/deviceregistrations\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs device registration\" }\r\n ,\"microsoft.azurespherev2/catalogs/provisioningpackages\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs provisioning package\" }\r\n ,\"microsoft.azurespherev2/catalogs/syndicationchannels\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs syndication channel\" }\r\n ,\"microsoft.azurespherev2/catalogs/syndicationchannels/deployments\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs syndication channels deployment\" }\r\n ,\"microsoft.azurespherev2/catalogs/updatepackages\": { \"SingularDisplayName\": \"Microsoft.AzureSphereV2 catalogs update package\" }\r\n ,\"microsoft.azurestack/cloudmanifestfiles\": { \"SingularDisplayName\": \"Microsoft.AzureStack cloud manifest file\" }\r\n ,\"microsoft.azurestack/linkedsubscriptions\": { \"SingularDisplayName\": \"Microsoft.AzureStack linked subscription\" }\r\n ,\"microsoft.azurestack/registrations\": { \"SingularDisplayName\": \"Microsoft.AzureStack registration\" }\r\n ,\"microsoft.azurestack/registrations/customersubscriptions\": { \"SingularDisplayName\": \"Microsoft.AzureStack registrations customer subscription\" }\r\n ,\"microsoft.azurestack/registrations/products\": { \"SingularDisplayName\": \"Microsoft.AzureStack registrations product\" }\r\n ,\"microsoft.azurestackhci/clusters\": { \"SingularDisplayName\": \"Azure Local\" }\r\n ,\"microsoft.azurestackhci/clusters/updates/updateruns\": { \"SingularDisplayName\": \"Azure Local\" }\r\n ,\"microsoft.azurestackhci/clusters/updatesummaries\": { \"SingularDisplayName\": \"Azure Local\" }\r\n ,\"microsoft.azurestackhci/devicepools\": { \"SingularDisplayName\": \"Azure Stack\" }\r\n ,\"microsoft.azurestackhci/edgedevices\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI edge device\" }\r\n ,\"microsoft.azurestackhci/edgedevices/jobs\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI edge devices job\" }\r\n ,\"microsoft.azurestackhci/edgemachines\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI edge machine\" }\r\n ,\"microsoft.azurestackhci/edgemachines/jobs\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI edge machines job\" }\r\n ,\"microsoft.azurestackhci/edgenodepools\": { \"SingularDisplayName\": \"Azure Stack\" }\r\n ,\"microsoft.azurestackhci/galleryimages\": { \"SingularDisplayName\": \"Azure Local Gallery image\" }\r\n ,\"microsoft.azurestackhci/logicalnetworks\": { \"SingularDisplayName\": \"Azure Local Logical network\" }\r\n ,\"microsoft.azurestackhci/marketplacegalleryimages\": { \"SingularDisplayName\": \"Azure Local Marketplace Gallery image\" }\r\n ,\"microsoft.azurestackhci/networkinterfaces\": { \"SingularDisplayName\": \"Azure Local VM Network Interface\" }\r\n ,\"microsoft.azurestackhci/networksecuritygroups\": { \"SingularDisplayName\": \"Azure Local Network Security Group\" }\r\n ,\"microsoft.azurestackhci/networksecuritygroups/securityrules\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI network security groups security rule\" }\r\n ,\"microsoft.azurestackhci/storagecontainers\": { \"SingularDisplayName\": \"Azure Local Storage path\" }\r\n ,\"microsoft.azurestackhci/virtualharddisks\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI virtual hard disk\" }\r\n ,\"microsoft.azurestackhci/virtualmachineinstances\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI virtual machine instance\" }\r\n ,\"microsoft.azurestackhci/virtualmachineinstances/guestagents\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI virtual machine instances guest agent\" }\r\n ,\"microsoft.azurestackhci/virtualmachineinstances/hybrididentitymetadata\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI virtual machine instances hybrid identity metadata\" }\r\n ,\"microsoft.azurestackhci/virtualmachines\": { \"SingularDisplayName\": \"Azure Local virtual machine - Azure Arc\" }\r\n ,\"microsoft.azurestackhci/virtualnetworks\": { \"SingularDisplayName\": \"Microsoft.AzureStackHCI virtual network\" }\r\n ,\"microsoft.backupsolutions/vmwareapplications\": { \"SingularDisplayName\": \"Microsoft.BackupSolutions vmware application\" }\r\n ,\"microsoft.bakeryhybrid/pies\": { \"SingularDisplayName\": \"Microsoft.BakeryHybrid py\" }\r\n ,\"microsoft.bakeryhybrid/pies/nestedresourcetype\": { \"SingularDisplayName\": \"Microsoft.BakeryHybrid pies nested resource type\" }\r\n ,\"microsoft.baremetal/baremetalconnections\": { \"SingularDisplayName\": \"Microsoft.BareMetal bare metal connection\" }\r\n ,\"microsoft.baremetal/crayservers\": { \"SingularDisplayName\": \"Cray Server\" }\r\n ,\"microsoft.baremetal/monitoringservers\": { \"SingularDisplayName\": \"Monitoring Server\" }\r\n ,\"microsoft.baremetal/peeringsettings\": { \"SingularDisplayName\": \"Microsoft.BareMetal peering setting\" }\r\n ,\"microsoft.baremetalinfrastructure/baremetalinstances\": { \"SingularDisplayName\": \"BareMetal Instance\" }\r\n ,\"microsoft.baremetalinfrastructure/baremetalstorageinstances\": { \"SingularDisplayName\": \"Microsoft.BareMetalInfrastructure bare metal storage instance\" }\r\n ,\"microsoft.batch/batchaccounts\": { \"SingularDisplayName\": \"Batch account\" }\r\n ,\"microsoft.billing/billingaccounts\": { \"SingularDisplayName\": \"Microsoft.Billing billing account\" }\r\n ,\"microsoft.billing/billingaccounts/agreements\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts agreement\" }\r\n ,\"microsoft.billing/billingaccounts/associatedtenants\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts associated tenant\" }\r\n ,\"microsoft.billing/billingaccounts/availablebalance\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts available balance\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profile\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/availablebalance\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles available balance\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/billingsubscriptions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles billing subscription\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/customers\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles customer\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/customers/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles customers billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/customers/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles customers billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/customers/transfers\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles customers transfer\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/instructions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles instruction\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoices\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice section\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice sections billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice sections billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections/billingsubscriptions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice sections billing subscription\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections/products\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice sections product\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/invoicesections/transfers\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles invoice sections transfer\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/paymentmethodlinks\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles payment method link\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/policies\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles policy\" }\r\n ,\"microsoft.billing/billingaccounts/billingprofiles/transactions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing profiles transaction\" }\r\n ,\"microsoft.billing/billingaccounts/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/billingsubscriptionaliases\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing subscription aliase\" }\r\n ,\"microsoft.billing/billingaccounts/billingsubscriptions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing subscription\" }\r\n ,\"microsoft.billing/billingaccounts/billingsubscriptions/invoices\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts billing subscriptions invoice\" }\r\n ,\"microsoft.billing/billingaccounts/customers\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts customer\" }\r\n ,\"microsoft.billing/billingaccounts/customers/billingsubscriptions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts customers billing subscription\" }\r\n ,\"microsoft.billing/billingaccounts/customers/policies\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts customers policy\" }\r\n ,\"microsoft.billing/billingaccounts/customers/products\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts customers product\" }\r\n ,\"microsoft.billing/billingaccounts/departments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts department\" }\r\n ,\"microsoft.billing/billingaccounts/departments/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts departments billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/departments/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts departments billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/departments/enrollmentaccounts\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts departments enrollment account\" }\r\n ,\"microsoft.billing/billingaccounts/enrollmentaccounts\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts enrollment account\" }\r\n ,\"microsoft.billing/billingaccounts/enrollmentaccounts/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts enrollment accounts billing role assignment\" }\r\n ,\"microsoft.billing/billingaccounts/enrollmentaccounts/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts enrollment accounts billing role definition\" }\r\n ,\"microsoft.billing/billingaccounts/incentiveschedules\": { \"SingularDisplayName\": \"Incentive Schedule\" }\r\n ,\"microsoft.billing/billingaccounts/incentiveschedules/milestones\": { \"SingularDisplayName\": \"Milestone\" }\r\n ,\"microsoft.billing/billingaccounts/invoices\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts invoice\" }\r\n ,\"microsoft.billing/billingaccounts/invoicesections\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts invoice section\" }\r\n ,\"microsoft.billing/billingaccounts/invoicesections/billingsubscriptions\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts invoice sections billing subscription\" }\r\n ,\"microsoft.billing/billingaccounts/invoicesections/products\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts invoice sections product\" }\r\n ,\"microsoft.billing/billingaccounts/invoicesections/transfers\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts invoice sections transfer\" }\r\n ,\"microsoft.billing/billingaccounts/lineofcredit\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts line of credit\" }\r\n ,\"microsoft.billing/billingaccounts/migrations\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts migration\" }\r\n ,\"microsoft.billing/billingaccounts/paymentmethods\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts payment method\" }\r\n ,\"microsoft.billing/billingaccounts/policies\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts policy\" }\r\n ,\"microsoft.billing/billingaccounts/products\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts product\" }\r\n ,\"microsoft.billing/billingaccounts/reservationorders\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts reservation order\" }\r\n ,\"microsoft.billing/billingaccounts/reservationorders/reservations\": { \"SingularDisplayName\": \"Microsoft.Billing billing accounts reservation orders reservation\" }\r\n ,\"microsoft.billing/billingaccounts/savingsplanorders\": { \"SingularDisplayName\": \"Savings plan order\" }\r\n ,\"microsoft.billing/billingaccounts/savingsplanorders/savingsplans\": { \"SingularDisplayName\": \"Savings plan\" }\r\n ,\"microsoft.billing/billingperiods\": { \"SingularDisplayName\": \"Microsoft.Billing billing period\" }\r\n ,\"microsoft.billing/billingproperty\": { \"SingularDisplayName\": \"Microsoft.Billing billing property\" }\r\n ,\"microsoft.billing/billingrequests\": { \"SingularDisplayName\": \"Microsoft.Billing billing request\" }\r\n ,\"microsoft.billing/billingroleassignments\": { \"SingularDisplayName\": \"Microsoft.Billing billing role assignment\" }\r\n ,\"microsoft.billing/billingroledefinitions\": { \"SingularDisplayName\": \"Microsoft.Billing billing role definition\" }\r\n ,\"microsoft.billing/enrollmentaccounts\": { \"SingularDisplayName\": \"Microsoft.Billing enrollment account\" }\r\n ,\"microsoft.billing/paymentmethods\": { \"SingularDisplayName\": \"Microsoft.Billing payment method\" }\r\n ,\"microsoft.billing/policies\": { \"SingularDisplayName\": \"Microsoft.Billing policy\" }\r\n ,\"microsoft.billing/promotions\": { \"SingularDisplayName\": \"Microsoft.Billing promotion\" }\r\n ,\"microsoft.billing/transfers\": { \"SingularDisplayName\": \"Microsoft.Billing transfer\" }\r\n ,\"microsoft.billingbenefits/credits\": { \"SingularDisplayName\": \"Credit\" }\r\n ,\"microsoft.billingbenefits/discounts\": { \"SingularDisplayName\": \"Discount\" }\r\n ,\"microsoft.billingbenefits/incentiveschedules\": { \"SingularDisplayName\": \"Incentive Schedule\" }\r\n ,\"microsoft.billingbenefits/incentiveschedules/milestones\": { \"SingularDisplayName\": \"Milestone\" }\r\n ,\"microsoft.billingbenefits/maccs\": { \"SingularDisplayName\": \"Microsoft Azure Consumption Commitment\" }\r\n ,\"microsoft.billingbenefits/reservationorderaliases\": { \"SingularDisplayName\": \"Microsoft.BillingBenefits reservation order aliase\" }\r\n ,\"microsoft.billingbenefits/savingsplanorderaliases\": { \"SingularDisplayName\": \"Microsoft.BillingBenefits savings plan order aliase\" }\r\n ,\"microsoft.billingbenefits/savingsplanorders\": { \"SingularDisplayName\": \"Savings plan order\" }\r\n ,\"microsoft.billingbenefits/savingsplanorders/savingsplans\": { \"SingularDisplayName\": \"Savings plan\" }\r\n ,\"microsoft.bing/accounts\": { \"SingularDisplayName\": \"Bing Resource\" }\r\n ,\"microsoft.blockchain/blockchainmembers\": { \"SingularDisplayName\": \"Microsoft.Blockchain blockchain member\" }\r\n ,\"microsoft.blockchain/blockchainmembers/transactionnodes\": { \"SingularDisplayName\": \"Microsoft.Blockchain blockchain members transaction node\" }\r\n ,\"microsoft.blockchaintokens/tokenservices\": { \"SingularDisplayName\": \"Microsoft.BlockchainTokens token service\" }\r\n ,\"microsoft.blockchaintokens/tokenservices/blockchainnetworks\": { \"SingularDisplayName\": \"Microsoft.BlockchainTokens token services blockchain network\" }\r\n ,\"microsoft.blockchaintokens/tokenservices/groups\": { \"SingularDisplayName\": \"Microsoft.BlockchainTokens token services group\" }\r\n ,\"microsoft.blockchaintokens/tokenservices/groups/accounts\": { \"SingularDisplayName\": \"Microsoft.BlockchainTokens token services groups account\" }\r\n ,\"microsoft.blockchaintokens/tokenservices/tokentemplates\": { \"SingularDisplayName\": \"Microsoft.BlockchainTokens token services token template\" }\r\n ,\"microsoft.bluefin/instances\": { \"SingularDisplayName\": \"Microsoft.Bluefin instance\" }\r\n ,\"microsoft.bluefin/instances/datasets\": { \"SingularDisplayName\": \"Microsoft.Bluefin instances dataset\" }\r\n ,\"microsoft.bluefin/instances/pipelines\": { \"SingularDisplayName\": \"Microsoft.Bluefin instances pipeline\" }\r\n ,\"microsoft.blueprint/blueprintassignments\": { \"SingularDisplayName\": \"Microsoft.Blueprint blueprint assignment\" }\r\n ,\"microsoft.blueprint/blueprints\": { \"SingularDisplayName\": \"Microsoft.Blueprint blueprint\" }\r\n ,\"microsoft.blueprint/blueprints/artifacts\": { \"SingularDisplayName\": \"Microsoft.Blueprint blueprints artifact\" }\r\n ,\"microsoft.blueprint/blueprints/versions\": { \"SingularDisplayName\": \"Microsoft.Blueprint blueprints version\" }\r\n ,\"microsoft.botservice/botservices\": { \"SingularDisplayName\": \"Bot Service\" }\r\n ,\"microsoft.cache/redis\": { \"SingularDisplayName\": \"Redis cache\" }\r\n ,\"microsoft.cache/redisenterprise\": { \"SingularDisplayName\": \"Azure Managed Redis\" }\r\n ,\"microsoft.cache/redisenterprise/databases\": { \"SingularDisplayName\": \"Redis Enterprise database\" }\r\n ,\"microsoft.capacity/reservationorders\": { \"SingularDisplayName\": \"Reservation order\" }\r\n ,\"microsoft.capacity/reservationorders/reservations\": { \"SingularDisplayName\": \"Reservation\" }\r\n ,\"microsoft.cascade/sites\": { \"SingularDisplayName\": \"Microsoft.Cascade site\" }\r\n ,\"microsoft.cdn/cdnwebapplicationfirewallpolicies\": { \"SingularDisplayName\": \"Content Delivery Network WAF policy\" }\r\n ,\"microsoft.cdn/edgeactions\": { \"SingularDisplayName\": \"Edge Action\" }\r\n ,\"microsoft.cdn/profiles\": { \"SingularDisplayName\": \"Front Door and CDN profile\" }\r\n ,\"microsoft.cdn/profiles/afdendpoints\": { \"SingularDisplayName\": \"Endpoint\" }\r\n ,\"microsoft.cdn/profiles/afdendpoints/routes\": { \"SingularDisplayName\": \"Route\" }\r\n ,\"microsoft.cdn/profiles/customdomains\": { \"SingularDisplayName\": \"Custom domain\" }\r\n ,\"microsoft.cdn/profiles/endpoints\": { \"SingularDisplayName\": \"CDN endpoint\" }\r\n ,\"microsoft.cdn/profiles/endpoints/customdomains\": { \"SingularDisplayName\": \"CDN custom domain\" }\r\n ,\"microsoft.cdn/profiles/endpoints/origins\": { \"SingularDisplayName\": \"CDN origin\" }\r\n ,\"microsoft.cdn/profiles/origingroups\": { \"SingularDisplayName\": \"Origin group\" }\r\n ,\"microsoft.cdn/profiles/origingroups/origins\": { \"SingularDisplayName\": \"Origin\" }\r\n ,\"microsoft.cdn/profiles/rulesets\": { \"SingularDisplayName\": \"Rule set\" }\r\n ,\"microsoft.cdn/profiles/rulesets/rules\": { \"SingularDisplayName\": \"Rule\" }\r\n ,\"microsoft.cdn/profiles/secrets\": { \"SingularDisplayName\": \"Secret\" }\r\n ,\"microsoft.cdn/profiles/securitypolicies\": { \"SingularDisplayName\": \"Security policy\" }\r\n ,\"microsoft.certificateregistration/certificateorders\": { \"SingularDisplayName\": \"App Service certificate\" }\r\n ,\"microsoft.certify/testsuites\": { \"SingularDisplayName\": \"Microsoft.Certify test suite\" }\r\n ,\"microsoft.certify/validationjobs\": { \"SingularDisplayName\": \"Microsoft.Certify validation job\" }\r\n ,\"microsoft.changeanalysis/profile\": { \"SingularDisplayName\": \"Microsoft.ChangeAnalysis profile\" }\r\n ,\"microsoft.changesafety/changestates\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety change state\" }\r\n ,\"microsoft.changesafety/changestates/stageprogressions\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety change states stage progression\" }\r\n ,\"microsoft.changesafety/stagemaps\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety stage map\" }\r\n ,\"microsoft.changesafety/validations\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety validation\" }\r\n ,\"microsoft.changesafety/validators\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety validator\" }\r\n ,\"microsoft.changesafety/validators/versions\": { \"SingularDisplayName\": \"Microsoft.ChangeSafety validators version\" }\r\n ,\"microsoft.chaos/experiments\": { \"SingularDisplayName\": \"Chaos Experiment\" }\r\n ,\"microsoft.chaos/privateaccesses\": { \"SingularDisplayName\": \"Agent Private Access\" }\r\n ,\"microsoft.chaos/targets\": { \"SingularDisplayName\": \"Microsoft.Chaos target\" }\r\n ,\"microsoft.chaos/targets/capabilities\": { \"SingularDisplayName\": \"Microsoft.Chaos targets capability\" }\r\n ,\"microsoft.classiccompute/domainnames\": { \"SingularDisplayName\": \"Cloud service (classic)\" }\r\n ,\"microsoft.classiccompute/domainnames/slots/roles\": { \"SingularDisplayName\": \"Cloud service role (classic)\" }\r\n ,\"microsoft.classiccompute/virtualmachines\": { \"SingularDisplayName\": \"Virtual machine (classic)\" }\r\n ,\"microsoft.classicnetwork/networksecuritygroups\": { \"SingularDisplayName\": \"Network security group (classic)\" }\r\n ,\"microsoft.classicnetwork/reservedips\": { \"SingularDisplayName\": \"Reserved IP address (classic)\" }\r\n ,\"microsoft.classicnetwork/virtualnetworks\": { \"SingularDisplayName\": \"Virtual network (classic)\" }\r\n })[tolower(id)]\r\n}\r\n", + "$fxv#1": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData/Internal')\r\n_resource_type_2(id: string) {\r\n dynamic({\r\n \"microsoft.classicstorage/storageaccounts\": { \"SingularDisplayName\": \"Storage account (classic)\" }\r\n ,\"microsoft.classicstorage/storageaccounts/disks\": { \"SingularDisplayName\": \"Disk (classic)\" }\r\n ,\"microsoft.classicstorage/storageaccounts/osimages\": { \"SingularDisplayName\": \"OS image (classic)\" }\r\n ,\"microsoft.classicstorage/storageaccounts/vmimages\": { \"SingularDisplayName\": \"VM image (classic)\" }\r\n ,\"microsoft.cleanroom/cleanrooms\": { \"SingularDisplayName\": \"Microsoft.CleanRoom cleanroom\" }\r\n ,\"microsoft.cleanroom/collaborations\": { \"SingularDisplayName\": \"Microsoft.CleanRoom collaboration\" }\r\n ,\"microsoft.cleanroom/collaborations/contracts\": { \"SingularDisplayName\": \"Microsoft.CleanRoom collaborations contract\" }\r\n ,\"microsoft.cleanroom/consortiums\": { \"SingularDisplayName\": \"Microsoft.CleanRoom consortium\" }\r\n ,\"microsoft.cleanroom/microservices\": { \"SingularDisplayName\": \"Microsoft.CleanRoom microservice\" }\r\n ,\"microsoft.cloud/hubs\": { \"SingularDisplayName\": \"FinOps hub\" }\r\n ,\"microsoft.clouddeviceplatform/delegatedidentities\": { \"SingularDisplayName\": \"Microsoft.CloudDevicePlatform delegated identity\" }\r\n ,\"microsoft.cloudhealth/healthmodels\": { \"SingularDisplayName\": \"Health Model\" }\r\n ,\"microsoft.cloudtest/accounts\": { \"SingularDisplayName\": \"CloudTest Account\" }\r\n ,\"microsoft.cloudtest/buildcaches\": { \"SingularDisplayName\": \"1ES Build Cache\" }\r\n ,\"microsoft.cloudtest/hostedpools\": { \"SingularDisplayName\": \"1ES Hosted Pool\" }\r\n ,\"microsoft.cloudtest/images\": { \"SingularDisplayName\": \"1ES Image\" }\r\n ,\"microsoft.cloudtest/pools\": { \"SingularDisplayName\": \"CloudTest Pool\" }\r\n ,\"microsoft.clusterstor/nodes\": { \"SingularDisplayName\": \"ClusterStor\" }\r\n ,\"microsoft.codesigning/codesigningaccounts\": { \"SingularDisplayName\": \"Trusted Signing Account\" }\r\n ,\"microsoft.codespaces/plans\": { \"SingularDisplayName\": \"Microsoft.Codespaces plan\" }\r\n ,\"microsoft.cognitiveservices/accounts\": { \"SingularDisplayName\": \"Azure AI Foundry\" }\r\n ,\"microsoft.cognitiveservices/accounts/projects\": { \"SingularDisplayName\": \"Azure AI Foundry project\" }\r\n ,\"microsoft.cognitiveservices/commitmentplans\": { \"SingularDisplayName\": \"Microsoft.CognitiveServices commitment plan\" }\r\n ,\"microsoft.cognitiveservices/commitmentplans/accountassociations\": { \"SingularDisplayName\": \"Microsoft.CognitiveServices commitment plans account association\" }\r\n ,\"microsoft.communication/communicationservices\": { \"SingularDisplayName\": \"Communication Service\" }\r\n ,\"microsoft.communication/emailservices\": { \"SingularDisplayName\": \"Email Communication Service\" }\r\n ,\"microsoft.communication/emailservices/domains\": { \"SingularDisplayName\": \"Email Communication Services Domain\" }\r\n ,\"microsoft.community/communitytrainings\": { \"SingularDisplayName\": \"Community Training\" }\r\n ,\"microsoft.compositesolutions/compositesolutiondefinitions\": { \"SingularDisplayName\": \"Microsoft.CompositeSolutions composite solution definition\" }\r\n ,\"microsoft.compositesolutions/compositesolutions\": { \"SingularDisplayName\": \"Microsoft.CompositeSolutions composite solution\" }\r\n ,\"microsoft.compute/availabilitysets\": { \"SingularDisplayName\": \"Availability set\" }\r\n ,\"microsoft.compute/capacityreservationgroups\": { \"SingularDisplayName\": \"Capacity Reservation Group\" }\r\n ,\"microsoft.compute/capacityreservationgroups/capacityreservations\": { \"SingularDisplayName\": \"Capacity reservation\" }\r\n ,\"microsoft.compute/capacityreservationgroupscomputehub\": { \"SingularDisplayName\": \"Capacity Reservation Group\" }\r\n ,\"microsoft.compute/cloudservices\": { \"SingularDisplayName\": \"Cloud service (extended support)\" }\r\n ,\"microsoft.compute/computefleetinstances\": { \"SingularDisplayName\": \"Instance\" }\r\n ,\"microsoft.compute/computefleetscalesets\": { \"SingularDisplayName\": \"Virtual machine scale set\" }\r\n ,\"microsoft.compute/diskaccesses\": { \"SingularDisplayName\": \"Disk Access\" }\r\n ,\"microsoft.compute/diskencryptionsets\": { \"SingularDisplayName\": \"Disk Encryption Set\" }\r\n ,\"microsoft.compute/disks\": { \"SingularDisplayName\": \"Disk\" }\r\n ,\"microsoft.compute/galleries\": { \"SingularDisplayName\": \"Azure compute gallery\" }\r\n ,\"microsoft.compute/galleries/applications\": { \"SingularDisplayName\": \"VM application definition\" }\r\n ,\"microsoft.compute/galleries/applications/versions\": { \"SingularDisplayName\": \"VM application version\" }\r\n ,\"microsoft.compute/galleries/images\": { \"SingularDisplayName\": \"VM image definition\" }\r\n ,\"microsoft.compute/galleries/images/versions\": { \"SingularDisplayName\": \"VM image version\" }\r\n ,\"microsoft.compute/galleries/imagescomputehub\": { \"SingularDisplayName\": \"VM image definition\" }\r\n ,\"microsoft.compute/hostgroups\": { \"SingularDisplayName\": \"Host group\" }\r\n ,\"microsoft.compute/hostgroups/hosts\": { \"SingularDisplayName\": \"Host\" }\r\n ,\"microsoft.compute/hostgroupscomputehub\": { \"SingularDisplayName\": \"Host group\" }\r\n ,\"microsoft.compute/images\": { \"SingularDisplayName\": \"Image\" }\r\n ,\"microsoft.compute/imagescomputehub\": { \"SingularDisplayName\": \"Image\" }\r\n ,\"microsoft.compute/locations/communitygalleries/images\": { \"SingularDisplayName\": \"Community image\" }\r\n ,\"microsoft.compute/locations/communitygalleries/imagescomputehub\": { \"SingularDisplayName\": \"Community image\" }\r\n ,\"microsoft.compute/proximityplacementgroups\": { \"SingularDisplayName\": \"Proximity placement group\" }\r\n ,\"microsoft.compute/proximityplacementgroupscomputehub\": { \"SingularDisplayName\": \"Proximity placement group\" }\r\n ,\"microsoft.compute/restorepointcollections\": { \"SingularDisplayName\": \"Restore Point Collection\" }\r\n ,\"microsoft.compute/restorepointcollections/restorepoints\": { \"SingularDisplayName\": \"Restore Point\" }\r\n ,\"microsoft.compute/snapshots\": { \"SingularDisplayName\": \"Snapshot\" }\r\n ,\"microsoft.compute/sshpublickeys\": { \"SingularDisplayName\": \"SSH key\" }\r\n ,\"microsoft.compute/standbypoolinstance\": { \"SingularDisplayName\": \"Standby pool\" }\r\n ,\"microsoft.compute/virtualmachinecomputehub\": { \"SingularDisplayName\": \"Virtual machine\" }\r\n ,\"microsoft.compute/virtualmachineflexinstances\": { \"SingularDisplayName\": \"Instance\" }\r\n ,\"microsoft.compute/virtualmachines\": { \"SingularDisplayName\": \"Virtual machine\" }\r\n ,\"microsoft.compute/virtualmachines/providers/guestconfigurationassignments\": { \"SingularDisplayName\": \"Guest Assignment\" }\r\n ,\"microsoft.compute/virtualmachinescalesets\": { \"SingularDisplayName\": \"Virtual machine scale set\" }\r\n ,\"microsoft.compute/virtualmachinescalesets/providers/guestconfigurationassignments\": { \"SingularDisplayName\": \"Guest Assignment\" }\r\n ,\"microsoft.compute/virtualmachinescalesets/virtualmachines\": { \"SingularDisplayName\": \"Virtual machine scale set instance\" }\r\n ,\"microsoft.compute/virtualmachinescalesets/virtualmachines/networkinterfaces/ipconfigurations/publicipaddresses\": { \"SingularDisplayName\": \"Public IP address\" }\r\n ,\"microsoft.compute/virtualmachinescalesetscomputehub\": { \"SingularDisplayName\": \"Virtual machine scale set\" }\r\n ,\"microsoft.computehub/advisorcost\": { \"SingularDisplayName\": \"Recommendations\" }\r\n ,\"microsoft.computehub/advisoroperationalexcellence\": { \"SingularDisplayName\": \"Recommendations\" }\r\n ,\"microsoft.computehub/advisorperformance\": { \"SingularDisplayName\": \"Recommendations\" }\r\n ,\"microsoft.computehub/advisorreliability\": { \"SingularDisplayName\": \"Recommendations\" }\r\n ,\"microsoft.computehub/advisorsecurity\": { \"SingularDisplayName\": \"Recommendations\" }\r\n ,\"microsoft.computehub/all\": { \"SingularDisplayName\": \"All resources\" }\r\n ,\"microsoft.computehub/backup\": { \"SingularDisplayName\": \"Backup job\" }\r\n ,\"microsoft.computehub/computehubmain\": { \"SingularDisplayName\": \"Compute infrastructure\" }\r\n ,\"microsoft.computehub/healthevents\": { \"SingularDisplayName\": \"Health events\" }\r\n ,\"microsoft.computehub/linuxostype\": { \"SingularDisplayName\": \"Linux OS\" }\r\n ,\"microsoft.computehub/microsoftdefenderfreetrialsubscription\": { \"SingularDisplayName\": \"Microsoft defender\" }\r\n ,\"microsoft.computehub/microsoftdefenderstandardsubscription\": { \"SingularDisplayName\": \"Microsoft defender\" }\r\n ,\"microsoft.computehub/outages\": { \"SingularDisplayName\": \"Outages\" }\r\n ,\"microsoft.computehub/powerstatedeallocated\": { \"SingularDisplayName\": \"Power states\" }\r\n ,\"microsoft.computehub/powerstaterunning\": { \"SingularDisplayName\": \"Power states\" }\r\n ,\"microsoft.computehub/powerstatestopped\": { \"SingularDisplayName\": \"Power states\" }\r\n ,\"microsoft.computehub/provisioningstatefailedresources\": { \"SingularDisplayName\": \"Provisioning states\" }\r\n ,\"microsoft.computehub/provisioningstatesucceededresources\": { \"SingularDisplayName\": \"Provisioning states\" }\r\n ,\"microsoft.computehub/windowsostype\": { \"SingularDisplayName\": \"Windows OS\" }\r\n ,\"microsoft.computeschedule/autoactions\": { \"SingularDisplayName\": \"Automatic Action\" }\r\n ,\"microsoft.computeschedule/autoactions/occurrences\": { \"SingularDisplayName\": \"Microsoft.ComputeSchedule auto actions occurrence\" }\r\n ,\"microsoft.confidentialledger/ledgers\": { \"SingularDisplayName\": \"Confidential Ledger\" }\r\n ,\"microsoft.confidentialledger/managedccfs\": { \"SingularDisplayName\": \"Managed CCF App\" }\r\n ,\"microsoft.confluent/agreements\": { \"SingularDisplayName\": \"Microsoft.Confluent agreement\" }\r\n ,\"microsoft.confluent/organizations\": { \"SingularDisplayName\": \"Confluent organization\" }\r\n ,\"microsoft.connectedcache/cachenodes\": { \"SingularDisplayName\": \"Connected Cache for ISP\" }\r\n ,\"microsoft.connectedcache/enterprisecustomers\": { \"SingularDisplayName\": \"Connected Cache for Enterprise & Education\" }\r\n ,\"microsoft.connectedcache/enterprisemcccustomers\": { \"SingularDisplayName\": \"Connected Cache for Enterprise & Education\" }\r\n ,\"microsoft.connectedcache/enterprisemcccustomers/enterprisemcccachenodes\": { \"SingularDisplayName\": \"MCC CacheNode for Enterprise\" }\r\n ,\"microsoft.connectedcache/ispcustomers\": { \"SingularDisplayName\": \"Connected Cache for ISP\" }\r\n ,\"microsoft.connectedcredentials/credentials\": { \"SingularDisplayName\": \"Microsoft.ConnectedCredentials credential\" }\r\n ,\"microsoft.connectedvehicle/platformaccounts\": { \"SingularDisplayName\": \"Microsoft.ConnectedVehicle platform account\" }\r\n ,\"microsoft.connectedvmwarevsphere/clusters\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere cluster\" }\r\n ,\"microsoft.connectedvmwarevsphere/datastores\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere datastore\" }\r\n ,\"microsoft.connectedvmwarevsphere/hosts\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere host\" }\r\n ,\"microsoft.connectedvmwarevsphere/resourcepools\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere resource pool\" }\r\n ,\"microsoft.connectedvmwarevsphere/vcenters\": { \"SingularDisplayName\": \"VMware vCenter\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachineinstances\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere virtual machine instance\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachineinstances/guestagents\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere virtual machine instances guest agent\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachineinstances/hybrididentitymetadata\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere virtual machine instances hybrid identity metadata\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachines\": { \"SingularDisplayName\": \"VMware + AVS virtual machine\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachines/providers/guestconfigurationassignments\": { \"SingularDisplayName\": \"Guest Assignment\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualmachinetemplates\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere virtual machine template\" }\r\n ,\"microsoft.connectedvmwarevsphere/virtualnetworks\": { \"SingularDisplayName\": \"Microsoft.ConnectedVMwarevSphere virtual network\" }\r\n ,\"microsoft.consumption/budgets\": { \"SingularDisplayName\": \"Microsoft.Consumption budget\" }\r\n ,\"microsoft.consumption/credits\": { \"SingularDisplayName\": \"Microsoft.Consumption credit\" }\r\n ,\"microsoft.consumption/pricesheets\": { \"SingularDisplayName\": \"Microsoft.Consumption pricesheet\" }\r\n ,\"microsoft.containerinstance/containergroupprofiles\": { \"SingularDisplayName\": \"Microsoft.ContainerInstance container group profile\" }\r\n ,\"microsoft.containerinstance/containergroupprofiles/revisions\": { \"SingularDisplayName\": \"Microsoft.ContainerInstance container group profiles revision\" }\r\n ,\"microsoft.containerinstance/containergroups\": { \"SingularDisplayName\": \"Container instances\" }\r\n ,\"microsoft.containerinstance/ngroups\": { \"SingularDisplayName\": \"Microsoft.ContainerInstance ngroup\" }\r\n ,\"microsoft.containerregistry/registries\": { \"SingularDisplayName\": \"Container registry\" }\r\n ,\"microsoft.containerregistry/registries/replications\": { \"SingularDisplayName\": \"Container registry replication\" }\r\n ,\"microsoft.containerregistry/registries/scopemaps\": { \"SingularDisplayName\": \"Container registry scope map\" }\r\n ,\"microsoft.containerregistry/registries/tokens\": { \"SingularDisplayName\": \"Container registry token\" }\r\n ,\"microsoft.containerregistry/registries/webhooks\": { \"SingularDisplayName\": \"Container registry webhook\" }\r\n ,\"microsoft.containerservice/fleets\": { \"SingularDisplayName\": \"Kubernetes fleet manager\" }\r\n ,\"microsoft.containerservice/managedclusters\": { \"SingularDisplayName\": \"Kubernetes service\" }\r\n ,\"microsoft.containerservice/managedclusters/managednamespaces\": { \"SingularDisplayName\": \"Managed namespace\" }\r\n ,\"microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/extensions\": { \"SingularDisplayName\": \"Kubernetes service extension\" }\r\n ,\"microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/fluxconfigurations\": { \"SingularDisplayName\": \"GitOps configuration\" }\r\n ,\"microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/namespaces\": { \"SingularDisplayName\": \"Kubernetes namespace\" }\r\n ,\"microsoft.containerservice/managedclusters/namespaces\": { \"SingularDisplayName\": \"Managed namespace\" }\r\n ,\"microsoft.containerservice/managedclustersnapshots\": { \"SingularDisplayName\": \"Microsoft.ContainerService managedclustersnapshot\" }\r\n ,\"microsoft.containerservice/snapshots\": { \"SingularDisplayName\": \"Microsoft.ContainerService snapshot\" }\r\n ,\"microsoft.containerstorage/pools\": { \"SingularDisplayName\": \"Container storage\" }\r\n ,\"microsoft.costmanagement/alerts\": { \"SingularDisplayName\": \"Microsoft.CostManagement alert\" }\r\n ,\"microsoft.costmanagement/budgets\": { \"SingularDisplayName\": \"Microsoft.CostManagement budget\" }\r\n ,\"microsoft.costmanagement/cloudconnectors\": { \"SingularDisplayName\": \"Microsoft.CostManagement cloud connector\" }\r\n ,\"microsoft.costmanagement/connectors\": { \"SingularDisplayName\": \"Microsoft.CostManagement connector\" }\r\n ,\"microsoft.costmanagement/costallocationrules\": { \"SingularDisplayName\": \"Microsoft.CostManagement cost allocation rule\" }\r\n ,\"microsoft.costmanagement/costdetailsoperationresults\": { \"SingularDisplayName\": \"Microsoft.CostManagement cost details operation result\" }\r\n ,\"microsoft.costmanagement/exports\": { \"SingularDisplayName\": \"Microsoft.CostManagement export\" }\r\n ,\"microsoft.costmanagement/externalbillingaccounts\": { \"SingularDisplayName\": \"Microsoft.CostManagement external billing account\" }\r\n ,\"microsoft.costmanagement/externalsubscriptions\": { \"SingularDisplayName\": \"Microsoft.CostManagement external subscription\" }\r\n ,\"microsoft.costmanagement/markuprules\": { \"SingularDisplayName\": \"Microsoft.CostManagement markup rule\" }\r\n ,\"microsoft.costmanagement/operationstatus\": { \"SingularDisplayName\": \"Microsoft.CostManagement operation statu\" }\r\n ,\"microsoft.costmanagement/reportconfigs\": { \"SingularDisplayName\": \"Microsoft.CostManagement reportconfig\" }\r\n ,\"microsoft.costmanagement/reports\": { \"SingularDisplayName\": \"Microsoft.CostManagement report\" }\r\n ,\"microsoft.costmanagement/scheduledactions\": { \"SingularDisplayName\": \"Microsoft.CostManagement scheduled action\" }\r\n ,\"microsoft.costmanagement/settings\": { \"SingularDisplayName\": \"Microsoft.CostManagement setting\" }\r\n ,\"microsoft.costmanagement/views\": { \"SingularDisplayName\": \"Microsoft.CostManagement view\" }\r\n ,\"microsoft.customerlockbox/requests\": { \"SingularDisplayName\": \"Microsoft.CustomerLockbox request\" }\r\n ,\"microsoft.customerlockbox/tenantoptedin\": { \"SingularDisplayName\": \"Microsoft.CustomerLockbox tenant opted in\" }\r\n ,\"microsoft.customproviders/associations\": { \"SingularDisplayName\": \"Microsoft.CustomProviders association\" }\r\n ,\"microsoft.customproviders/resourceproviders\": { \"SingularDisplayName\": \"Microsoft.CustomProviders resource provider\" }\r\n ,\"microsoft.dashboard/dashboards\": { \"SingularDisplayName\": \"Azure Monitor dashboards with Grafana\" }\r\n ,\"microsoft.dashboard/grafana\": { \"SingularDisplayName\": \"Azure Managed Grafana\" }\r\n ,\"microsoft.dataaccelerator/indexclusters\": { \"SingularDisplayName\": \"Microsoft.DataAccelerator index cluster\" }\r\n ,\"microsoft.databasefleetmanager/fleets\": { \"SingularDisplayName\": \"Database fleet manager\" }\r\n ,\"microsoft.databasefleetmanager/fleets/fleetspaces\": { \"SingularDisplayName\": \"Fleetspaces\" }\r\n ,\"microsoft.databasefleetmanager/fleets/fleetspaces/databases\": { \"SingularDisplayName\": \"Fleet managed database\" }\r\n ,\"microsoft.databasefleetmanager/fleets/tiers\": { \"SingularDisplayName\": \"tier\" }\r\n ,\"microsoft.databasewatcher/watchers\": { \"SingularDisplayName\": \"Database watcher\" }\r\n ,\"microsoft.databox/jobs\": { \"SingularDisplayName\": \"Azure Data Box\" }\r\n ,\"microsoft.databoxedge/databoxedgedevices\": { \"SingularDisplayName\": \"Azure Stack Edge / Data Box Gateway\" }\r\n ,\"microsoft.databricks/accessconnectors\": { \"SingularDisplayName\": \"Access Connector for Azure Databricks\" }\r\n ,\"microsoft.databricks/workspaces\": { \"SingularDisplayName\": \"Azure Databricks Service\" }\r\n ,\"microsoft.datacatalog/catalogs\": { \"SingularDisplayName\": \"Data catalog\" }\r\n ,\"microsoft.datacollaboration/workspaces\": { \"SingularDisplayName\": \"Project CI\" }\r\n ,\"microsoft.datadog/agreements\": { \"SingularDisplayName\": \"Microsoft.Datadog agreement\" }\r\n ,\"microsoft.datadog/monitors\": { \"SingularDisplayName\": \"Datadog\" }\r\n ,\"microsoft.datadog/subscriptionstatuses\": { \"SingularDisplayName\": \"Microsoft.Datadog subscription statuse\" }\r\n ,\"microsoft.datafactory/datafactories\": { \"SingularDisplayName\": \"Data factory\" }\r\n ,\"microsoft.datafactory/factories\": { \"SingularDisplayName\": \"Data factory (V2)\" }\r\n ,\"microsoft.datafactory/factories/pipelines\": { \"SingularDisplayName\": \"Data Factory pipeline\" }\r\n ,\"microsoft.datafactory/factories/triggers\": { \"SingularDisplayName\": \"Data Factory trigger\" }\r\n ,\"microsoft.datalakeanalytics/accounts\": { \"SingularDisplayName\": \"Data Lake Analytics account\" }\r\n ,\"microsoft.datalakestore/accounts\": { \"SingularDisplayName\": \"Data Lake Storage Gen1\" }\r\n ,\"microsoft.datamigration/databasemigrations\": { \"SingularDisplayName\": \"Microsoft.DataMigration database migration\" }\r\n ,\"microsoft.datamigration/migrationservices\": { \"SingularDisplayName\": \"Microsoft.DataMigration migration service\" }\r\n ,\"microsoft.datamigration/services\": { \"SingularDisplayName\": \"Azure Database Migration Service (classic)\" }\r\n ,\"microsoft.datamigration/services/projects\": { \"SingularDisplayName\": \"Azure Database Migration Project\" }\r\n ,\"microsoft.datamigration/sqlmigrationservices\": { \"SingularDisplayName\": \"Azure Database Migration Service\" }\r\n ,\"microsoft.dataprotection/backupvaults\": { \"SingularDisplayName\": \"Backup vault\" }\r\n ,\"microsoft.dataprotection/resourceguards\": { \"SingularDisplayName\": \"Resource Guard\" }\r\n ,\"microsoft.datareplication/replicationfabrics\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication fabric\" }\r\n ,\"microsoft.datareplication/replicationfabrics/fabricagents\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication fabrics fabric agent\" }\r\n ,\"microsoft.datareplication/replicationfabrics/fabricagents/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication fabrics fabric agents operation\" }\r\n ,\"microsoft.datareplication/replicationfabrics/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication fabrics operation\" }\r\n ,\"microsoft.datareplication/replicationvaults\": { \"SingularDisplayName\": \"Data replication vault\" }\r\n ,\"microsoft.datareplication/replicationvaults/alertsettings\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults alert setting\" }\r\n ,\"microsoft.datareplication/replicationvaults/events\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults event\" }\r\n ,\"microsoft.datareplication/replicationvaults/jobs\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults job\" }\r\n ,\"microsoft.datareplication/replicationvaults/jobs/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults jobs operation\" }\r\n ,\"microsoft.datareplication/replicationvaults/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults operation\" }\r\n ,\"microsoft.datareplication/replicationvaults/privateendpointconnectionproxies\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults private endpoint connection proxy\" }\r\n ,\"microsoft.datareplication/replicationvaults/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults private endpoint connection\" }\r\n ,\"microsoft.datareplication/replicationvaults/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults private link resource\" }\r\n ,\"microsoft.datareplication/replicationvaults/protecteditems\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults protected item\" }\r\n ,\"microsoft.datareplication/replicationvaults/protecteditems/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults protected items operation\" }\r\n ,\"microsoft.datareplication/replicationvaults/protecteditems/recoverypoints\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults protected items recovery point\" }\r\n ,\"microsoft.datareplication/replicationvaults/replicationextensions\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults replication extension\" }\r\n ,\"microsoft.datareplication/replicationvaults/replicationextensions/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults replication extensions operation\" }\r\n ,\"microsoft.datareplication/replicationvaults/replicationpolicies\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults replication policy\" }\r\n ,\"microsoft.datareplication/replicationvaults/replicationpolicies/operations\": { \"SingularDisplayName\": \"Microsoft.DataReplication replication vaults replication policies operation\" }\r\n ,\"microsoft.datashare/accounts\": { \"SingularDisplayName\": \"Data Share\" }\r\n ,\"microsoft.dbformariadb/servers\": { \"SingularDisplayName\": \"Azure Database for MariaDB server\" }\r\n ,\"microsoft.dbformysql/flexibleservers\": { \"SingularDisplayName\": \"Azure Database for MySQL flexible server\" }\r\n ,\"microsoft.dbformysql/servers\": { \"SingularDisplayName\": \"MySQL server\" }\r\n ,\"microsoft.dbforpostgresql/flexibleservers\": { \"SingularDisplayName\": \"Azure Database for PostgreSQL flexible server\" }\r\n ,\"microsoft.dbforpostgresql/servergroupsv2\": { \"SingularDisplayName\": \"Azure Cosmos DB for PostgreSQL Cluster\" }\r\n ,\"microsoft.dbforpostgresql/servers\": { \"SingularDisplayName\": \"PostgreSQL server\" }\r\n ,\"microsoft.delegatednetwork/controller\": { \"SingularDisplayName\": \"Microsoft.DelegatedNetwork controller\" }\r\n ,\"microsoft.delegatednetwork/delegatedsubnets\": { \"SingularDisplayName\": \"Microsoft.DelegatedNetwork delegated subnet\" }\r\n ,\"microsoft.delegatednetwork/orchestrators\": { \"SingularDisplayName\": \"Microsoft.DelegatedNetwork orchestrator\" }\r\n ,\"microsoft.dependencymap/maps\": { \"SingularDisplayName\": \"Microsoft.DependencyMap map\" }\r\n ,\"microsoft.dependencymap/maps/discoverysources\": { \"SingularDisplayName\": \"Microsoft.DependencyMap maps discovery source\" }\r\n ,\"microsoft.deploymentmanager/artifactsources\": { \"SingularDisplayName\": \"Microsoft.DeploymentManager artifact source\" }\r\n ,\"microsoft.deploymentmanager/rollouts\": { \"SingularDisplayName\": \"Rollout\" }\r\n ,\"microsoft.deploymentmanager/servicetopologies\": { \"SingularDisplayName\": \"Microsoft.DeploymentManager service topology\" }\r\n ,\"microsoft.deploymentmanager/servicetopologies/services\": { \"SingularDisplayName\": \"Microsoft.DeploymentManager service topologies service\" }\r\n ,\"microsoft.deploymentmanager/servicetopologies/services/serviceunits\": { \"SingularDisplayName\": \"Microsoft.DeploymentManager service topologies services service unit\" }\r\n ,\"microsoft.deploymentmanager/steps\": { \"SingularDisplayName\": \"Microsoft.DeploymentManager step\" }\r\n ,\"microsoft.desktopvirtualization/appattachpackages\": { \"SingularDisplayName\": \"App attach package\" }\r\n ,\"microsoft.desktopvirtualization/applicationgroups\": { \"SingularDisplayName\": \"Application group\" }\r\n ,\"microsoft.desktopvirtualization/hostpools\": { \"SingularDisplayName\": \"Host pool\" }\r\n ,\"microsoft.desktopvirtualization/scalingplans\": { \"SingularDisplayName\": \"Scaling plan\" }\r\n ,\"microsoft.desktopvirtualization/workspaces\": { \"SingularDisplayName\": \"Workspace\" }\r\n ,\"microsoft.devai/instances\": { \"SingularDisplayName\": \"Microsoft.DevAI instance\" }\r\n ,\"microsoft.devai/instances/experiments\": { \"SingularDisplayName\": \"Microsoft.DevAI instances experiment\" }\r\n ,\"microsoft.devai/instances/sandboxes\": { \"SingularDisplayName\": \"Microsoft.DevAI instances sandbox\" }\r\n ,\"microsoft.devai/instances/sandboxes/experiments\": { \"SingularDisplayName\": \"Microsoft.DevAI instances sandboxes experiment\" }\r\n ,\"microsoft.devcenter/devcenters\": { \"SingularDisplayName\": \"Dev center\" }\r\n ,\"microsoft.devcenter/devcenters/devboxdefinitions\": { \"SingularDisplayName\": \"Dev Box definition\" }\r\n ,\"microsoft.devcenter/networkconnections\": { \"SingularDisplayName\": \"Network connection\" }\r\n ,\"microsoft.devcenter/plans\": { \"SingularDisplayName\": \"Dev center plan\" }\r\n ,\"microsoft.devcenter/projects\": { \"SingularDisplayName\": \"Project\" }\r\n ,\"microsoft.devcenter/projects/pools\": { \"SingularDisplayName\": \"Pool\" }\r\n ,\"microsoft.developmentwindows365/developmentcloudpcdelegatedmsis\": { \"SingularDisplayName\": \"Microsoft.DevelopmentWindows365 development cloud pc delegated msi\" }\r\n ,\"microsoft.devhub/iacprofiles\": { \"SingularDisplayName\": \"Infrastructure as Code Automation\" }\r\n ,\"microsoft.devhub/templates\": { \"SingularDisplayName\": \"Microsoft.DevHub template\" }\r\n ,\"microsoft.devhub/templates/versions\": { \"SingularDisplayName\": \"Microsoft.DevHub templates version\" }\r\n ,\"microsoft.devhub/workflows\": { \"SingularDisplayName\": \"Microsoft.DevHub workflow\" }\r\n ,\"microsoft.deviceonboarding/discoveryservices\": { \"SingularDisplayName\": \"Microsoft.DeviceOnboarding discovery service\" }\r\n ,\"microsoft.deviceonboarding/discoveryservices/ownershipvoucherpublickeys\": { \"SingularDisplayName\": \"Microsoft.DeviceOnboarding discovery services ownership voucher public key\" }\r\n ,\"microsoft.deviceonboarding/onboardingservices\": { \"SingularDisplayName\": \"Microsoft.DeviceOnboarding onboarding service\" }\r\n ,\"microsoft.deviceonboarding/onboardingservices/policies\": { \"SingularDisplayName\": \"Microsoft.DeviceOnboarding onboarding services policy\" }\r\n ,\"microsoft.deviceregistry/assetendpointprofiles\": { \"SingularDisplayName\": \"IoT Asset Endpoint Profile\" }\r\n ,\"microsoft.deviceregistry/assets\": { \"SingularDisplayName\": \"IoT Asset\" }\r\n ,\"microsoft.deviceregistry/billingcontainers\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry billing container\" }\r\n ,\"microsoft.deviceregistry/devices\": { \"SingularDisplayName\": \"IoT Device\" }\r\n ,\"microsoft.deviceregistry/discoveredassetendpointprofiles\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry discovered asset endpoint profile\" }\r\n ,\"microsoft.deviceregistry/discoveredassets\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry discovered asset\" }\r\n ,\"microsoft.deviceregistry/namespaces\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespace\" }\r\n ,\"microsoft.deviceregistry/namespaces/assetendpointprofiles\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespaces asset endpoint profile\" }\r\n ,\"microsoft.deviceregistry/namespaces/assets\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespaces asset\" }\r\n ,\"microsoft.deviceregistry/namespaces/devices\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespaces device\" }\r\n ,\"microsoft.deviceregistry/namespaces/discoveredassetendpointprofiles\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespaces discovered asset endpoint profile\" }\r\n ,\"microsoft.deviceregistry/namespaces/discoveredassets\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry namespaces discovered asset\" }\r\n ,\"microsoft.deviceregistry/schemaregistries\": { \"SingularDisplayName\": \"IoT Schema Registry\" }\r\n ,\"microsoft.deviceregistry/schemaregistries/schemas\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry schema registries schema\" }\r\n ,\"microsoft.deviceregistry/schemaregistries/schemas/schemaversions\": { \"SingularDisplayName\": \"Microsoft.DeviceRegistry schema registries schemas schema version\" }\r\n ,\"microsoft.devices/iothubs\": { \"SingularDisplayName\": \"IoT hub\" }\r\n ,\"microsoft.devices/provisioningservices\": { \"SingularDisplayName\": \"Azure IoT Hub Device Provisioning Service (DPS)\" }\r\n ,\"microsoft.deviceupdate/accounts\": { \"SingularDisplayName\": \"Device Update for IoT Hub\" }\r\n ,\"microsoft.deviceupdate/updateaccounts\": { \"SingularDisplayName\": \"Device Update Account\" }\r\n ,\"microsoft.deviceupdate/updateaccounts/activedeployments\": { \"SingularDisplayName\": \"Device Update Active Deployment\" }\r\n ,\"microsoft.deviceupdate/updateaccounts/agents\": { \"SingularDisplayName\": \"Device Update Agent\" }\r\n ,\"microsoft.deviceupdate/updateaccounts/deployments\": { \"SingularDisplayName\": \"Device Update Deployment\" }\r\n ,\"microsoft.deviceupdate/updateaccounts/deviceclasses\": { \"SingularDisplayName\": \"Device Update Device Class\" }\r\n ,\"microsoft.deviceupdate/updateaccounts/updates\": { \"SingularDisplayName\": \"Device Update\" }\r\n ,\"microsoft.devops/pipelines\": { \"SingularDisplayName\": \"Microsoft.DevOps pipeline\" }\r\n ,\"microsoft.devopsinfrastructure/pools\": { \"SingularDisplayName\": \"Managed DevOps Pool\" }\r\n ,\"microsoft.devspaces/controllers\": { \"SingularDisplayName\": \"Microsoft.DevSpaces controller\" }\r\n ,\"microsoft.devtestlab/labs\": { \"SingularDisplayName\": \"DevTest lab\" }\r\n ,\"microsoft.devtestlab/labs/virtualmachines\": { \"SingularDisplayName\": \"DevTest Lab virtual machine\" }\r\n ,\"microsoft.devtestlab/schedules\": { \"SingularDisplayName\": \"Microsoft.DevTestLab schedule\" }\r\n ,\"microsoft.devtunnels/tunnelplans\": { \"SingularDisplayName\": \"Dev Tunnels Domain\" }\r\n ,\"microsoft.diagnostics/apollo\": { \"SingularDisplayName\": \"Microsoft.Diagnostics apollo\" }\r\n ,\"microsoft.digitaltwins/digitaltwinsinstances\": { \"SingularDisplayName\": \"Azure Digital Twins\" }\r\n ,\"microsoft.discovery/agents\": { \"SingularDisplayName\": \"Microsoft Discovery Agent\" }\r\n ,\"microsoft.discovery/bookshelves\": { \"SingularDisplayName\": \"Microsoft Discovery Bookshelf\" }\r\n ,\"microsoft.discovery/datacontainers\": { \"SingularDisplayName\": \"Microsoft Discovery Data Container\" }\r\n ,\"microsoft.discovery/datacontainers/dataassets\": { \"SingularDisplayName\": \"Data asset\" }\r\n ,\"microsoft.discovery/models\": { \"SingularDisplayName\": \"Microsoft Discovery Model\" }\r\n ,\"microsoft.discovery/storages\": { \"SingularDisplayName\": \"Microsoft Discovery Storage\" }\r\n ,\"microsoft.discovery/supercomputers\": { \"SingularDisplayName\": \"Microsoft Discovery Supercomputer\" }\r\n ,\"microsoft.discovery/supercomputers/nodepools\": { \"SingularDisplayName\": \"Nodepool\" }\r\n ,\"microsoft.discovery/tools\": { \"SingularDisplayName\": \"Microsoft Discovery Tool\" }\r\n ,\"microsoft.discovery/workflows\": { \"SingularDisplayName\": \"Microsoft Discovery Workflow\" }\r\n ,\"microsoft.discovery/workspaces\": { \"SingularDisplayName\": \"Microsoft Discovery Workspace\" }\r\n ,\"microsoft.discovery/workspaces/projects\": { \"SingularDisplayName\": \"Microsoft Discovery Project\" }\r\n ,\"microsoft.documentdb/cassandraclusters\": { \"SingularDisplayName\": \"Azure Managed Instance for Apache Cassandra\" }\r\n ,\"microsoft.documentdb/databaseaccounts\": { \"SingularDisplayName\": \"Cosmos DB account\" }\r\n ,\"microsoft.documentdb/fleets\": { \"SingularDisplayName\": \"Azure Cosmos DB Fleet\" }\r\n ,\"microsoft.documentdb/fleetspacepotentialdatabaseaccounts\": { \"SingularDisplayName\": \"Potential Azure Cosmos DB account\" }\r\n ,\"microsoft.documentdb/fleetspacepotentialdatabaseaccountswithlocations\": { \"SingularDisplayName\": \"Potential Azure Cosmos DB account\" }\r\n ,\"microsoft.documentdb/mongoclusters\": { \"SingularDisplayName\": \"Azure Cosmos DB for MongoDB (vCore)\" }\r\n ,\"microsoft.documentdb/throughputpools\": { \"SingularDisplayName\": \"Microsoft.DocumentDB throughput pool\" }\r\n ,\"microsoft.documentdb/throughputpools/throughputpoolaccounts\": { \"SingularDisplayName\": \"Microsoft.DocumentDB throughput pools throughput pool account\" }\r\n ,\"microsoft.domainregistration/domains\": { \"SingularDisplayName\": \"App Service Domain\" }\r\n ,\"microsoft.domainregistration/topleveldomains\": { \"SingularDisplayName\": \"Microsoft.DomainRegistration top level domain\" }\r\n ,\"microsoft.durabletask/namespaces\": { \"SingularDisplayName\": \"Microsoft.DurableTask namespace\" }\r\n ,\"microsoft.durabletask/namespaces/taskhubs\": { \"SingularDisplayName\": \"Task Hub\" }\r\n ,\"microsoft.durabletask/schedulers\": { \"SingularDisplayName\": \"Durable Task Scheduler\" }\r\n ,\"microsoft.durabletask/schedulers/taskhubs\": { \"SingularDisplayName\": \"Task Hub\" }\r\n ,\"microsoft.dynamics365fraudprotection/instances\": { \"SingularDisplayName\": \"Microsoft.Dynamics365FraudProtection instance\" }\r\n ,\"microsoft.easm/workspaces\": { \"SingularDisplayName\": \"Microsoft Defender EASM\" }\r\n ,\"microsoft.edge/configurations\": { \"SingularDisplayName\": \"Site configuration\" }\r\n ,\"microsoft.edge/configurations/arcgatewayconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations arc gateway configuration\" }\r\n ,\"microsoft.edge/configurations/connectivityconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations connectivity configuration\" }\r\n ,\"microsoft.edge/configurations/dynamicconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations dynamic configuration\" }\r\n ,\"microsoft.edge/configurations/dynamicconfigurations/versions\": { \"SingularDisplayName\": \"Microsoft.Edge configurations dynamic configurations version\" }\r\n ,\"microsoft.edge/configurations/networkconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations network configuration\" }\r\n ,\"microsoft.edge/configurations/securityconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations security configuration\" }\r\n ,\"microsoft.edge/configurations/timeserverconfigurations\": { \"SingularDisplayName\": \"Microsoft.Edge configurations time server configuration\" }\r\n ,\"microsoft.edge/connectivitystatuses\": { \"SingularDisplayName\": \"Microsoft.Edge connectivity statuse\" }\r\n ,\"microsoft.edge/disconnectedoperations\": { \"SingularDisplayName\": \"Azure Local - disconnected operations\" }\r\n ,\"microsoft.edge/siteawareresourcetypes\": { \"SingularDisplayName\": \"Microsoft.Edge site aware resource type\" }\r\n ,\"microsoft.edge/sites\": { \"SingularDisplayName\": \"Site manager - Azure Arc\" }\r\n ,\"microsoft.edge/updates\": { \"SingularDisplayName\": \"Microsoft.Edge update\" }\r\n ,\"microsoft.edgemarketplace/offers\": { \"SingularDisplayName\": \"Microsoft.EdgeMarketplace offer\" }\r\n ,\"microsoft.edgemarketplace/publishers\": { \"SingularDisplayName\": \"Microsoft.EdgeMarketplace publisher\" }\r\n ,\"microsoft.edgeorder/addresses\": { \"SingularDisplayName\": \"Azure Edge Hardware Center Address\" }\r\n ,\"microsoft.edgeorder/bootstrapconfigurations\": { \"SingularDisplayName\": \"Site Key\" }\r\n ,\"microsoft.edgeorder/orderitems\": { \"SingularDisplayName\": \"Azure Edge Hardware Center\" }\r\n ,\"microsoft.edgeorder/virtual_orderitems\": { \"SingularDisplayName\": \"Device\" }\r\n ,\"microsoft.edgezones/extendedzones\": { \"SingularDisplayName\": \"Microsoft.EdgeZones extended zone\" }\r\n ,\"microsoft.education/grants\": { \"SingularDisplayName\": \"Microsoft.Education grant\" }\r\n ,\"microsoft.education/labs\": { \"SingularDisplayName\": \"Microsoft.Education lab\" }\r\n ,\"microsoft.education/labs/joinrequests\": { \"SingularDisplayName\": \"Microsoft.Education labs join request\" }\r\n ,\"microsoft.education/labs/students\": { \"SingularDisplayName\": \"Microsoft.Education labs student\" }\r\n ,\"microsoft.education/studentlabs\": { \"SingularDisplayName\": \"Microsoft.Education student lab\" }\r\n ,\"microsoft.elastic/monitors\": { \"SingularDisplayName\": \"Elastic Cloud Resource\" }\r\n ,\"microsoft.elasticsan/elasticsans\": { \"SingularDisplayName\": \"Elastic SAN\" }\r\n ,\"microsoft.energydataplatform/energyservices\": { \"SingularDisplayName\": \"Microsoft.EnergyDataPlatform energy service\" }\r\n ,\"microsoft.enterpriseknowledgegraph/services\": { \"SingularDisplayName\": \"Microsoft.EnterpriseKnowledgeGraph service\" }\r\n ,\"microsoft.enterprisesupport/enterprisesupports\": { \"SingularDisplayName\": \"Microsoft.EnterpriseSupport enterprise support\" }\r\n ,\"microsoft.eventgrid/domains\": { \"SingularDisplayName\": \"Event Grid Domain\" }\r\n ,\"microsoft.eventgrid/domains/topics\": { \"SingularDisplayName\": \"Event Grid Domain Topic\" }\r\n ,\"microsoft.eventgrid/eventsubscriptions\": { \"SingularDisplayName\": \"Microsoft.EventGrid event subscription\" }\r\n ,\"microsoft.eventgrid/extensiontopics\": { \"SingularDisplayName\": \"Event Grid extension topic\" }\r\n ,\"microsoft.eventgrid/namespaces\": { \"SingularDisplayName\": \"Event Grid Namespace\" }\r\n ,\"microsoft.eventgrid/namespaces/topics\": { \"SingularDisplayName\": \"Event Grid Namespace Topic\" }\r\n ,\"microsoft.eventgrid/namespaces/topics/eventsubscriptions\": { \"SingularDisplayName\": \"Event Subscription\" }\r\n ,\"microsoft.eventgrid/namespaces/topicspaces\": { \"SingularDisplayName\": \"Event Grid Topic Space\" }\r\n ,\"microsoft.eventgrid/partnerconfigurations\": { \"SingularDisplayName\": \"Event Grid Partner Configuration\" }\r\n ,\"microsoft.eventgrid/partnerdestinations\": { \"SingularDisplayName\": \"Event Grid Partner Destination\" }\r\n ,\"microsoft.eventgrid/partnernamespaces\": { \"SingularDisplayName\": \"Event Grid Partner Namespace\" }\r\n ,\"microsoft.eventgrid/partnernamespaces/channels\": { \"SingularDisplayName\": \"Event Grid Channel\" }\r\n ,\"microsoft.eventgrid/partnerregistrations\": { \"SingularDisplayName\": \"Event Grid Partner Registration\" }\r\n ,\"microsoft.eventgrid/partnertopics\": { \"SingularDisplayName\": \"Event Grid Partner Topic\" }\r\n ,\"microsoft.eventgrid/systemtopics\": { \"SingularDisplayName\": \"Event Grid System Topic\" }\r\n ,\"microsoft.eventgrid/systemtopics/eventsubscriptions\": { \"SingularDisplayName\": \"Event Grid Subscriptions\" }\r\n ,\"microsoft.eventgrid/topics\": { \"SingularDisplayName\": \"Event Grid Topic\" }\r\n ,\"microsoft.eventgrid/topictypes\": { \"SingularDisplayName\": \"Microsoft.EventGrid topic type\" }\r\n ,\"microsoft.eventgrid/verifiedpartners\": { \"SingularDisplayName\": \"Microsoft.EventGrid verified partner\" }\r\n ,\"microsoft.eventhub/clusters\": { \"SingularDisplayName\": \"Event Hubs Cluster\" }\r\n ,\"microsoft.eventhub/namespaces\": { \"SingularDisplayName\": \"Event Hubs namespace\" }\r\n ,\"microsoft.eventhub/namespaces/disasterrecoveryconfigs\": { \"SingularDisplayName\": \"Event Hubs Geo-DR Alias\" }\r\n ,\"microsoft.eventhub/namespaces/eventhubs\": { \"SingularDisplayName\": \"Event Hubs Instance\" }\r\n ,\"microsoft.eventhub/namespaces/providers/diagnosticsettings\": { \"SingularDisplayName\": \"Diagnostic settings\" }\r\n ,\"microsoft.eventhub/namespaces/schemagroups\": { \"SingularDisplayName\": \"Schema Group\" }\r\n ,\"microsoft.experimentation/experimentworkspaces\": { \"SingularDisplayName\": \"Experiment Workspace\" }\r\n ,\"microsoft.extendedlocation/customlocations\": { \"SingularDisplayName\": \"Custom location\" }\r\n ,\"microsoft.fabric/capacities\": { \"SingularDisplayName\": \"Fabric Capacity\" }\r\n ,\"microsoft.fabric/privatelinkservicesforfabric\": { \"SingularDisplayName\": \"Microsoft.Fabric private link services for fabric\" }\r\n ,\"microsoft.fabric/privatelinkservicesforfabric/operationresults\": { \"SingularDisplayName\": \"Microsoft.Fabric private link services for fabric operation result\" }\r\n ,\"microsoft.fabric/privatelinkservicesforfabric/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.Fabric private link services for fabric private endpoint connection\" }\r\n ,\"microsoft.fabric/privatelinkservicesforfabric/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.Fabric private link services for fabric private link resource\" }\r\n ,\"microsoft.fairfieldgardens/deviceprovisioningstates\": { \"SingularDisplayName\": \"Microsoft.FairfieldGardens device provisioning state\" }\r\n ,\"microsoft.fairfieldgardens/provisioningresources\": { \"SingularDisplayName\": \"Fairfield Gardens\" }\r\n ,\"microsoft.fairfieldgardens/provisioningresources/provisioningpolicies\": { \"SingularDisplayName\": \"Provisioning policy\" }\r\n ,\"microsoft.falcon/namespaces\": { \"SingularDisplayName\": \"Microsoft.Falcon namespace\" }\r\n ,\"microsoft.features/featureprovidernamespaces/featureconfigurations\": { \"SingularDisplayName\": \"Preview features\" }\r\n ,\"microsoft.fidalgo/devcenters\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenter\" }\r\n ,\"microsoft.fidalgo/devcenters/attachednetworks\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters attachednetwork\" }\r\n ,\"microsoft.fidalgo/devcenters/catalogs\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters catalog\" }\r\n ,\"microsoft.fidalgo/devcenters/catalogs/items\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters catalogs item\" }\r\n ,\"microsoft.fidalgo/devcenters/devboxdefinitions\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters devboxdefinition\" }\r\n ,\"microsoft.fidalgo/devcenters/environmenttypes\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters environment type\" }\r\n ,\"microsoft.fidalgo/devcenters/galleries\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters gallery\" }\r\n ,\"microsoft.fidalgo/devcenters/galleries/images\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters galleries image\" }\r\n ,\"microsoft.fidalgo/devcenters/galleries/images/versions\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters galleries images version\" }\r\n ,\"microsoft.fidalgo/devcenters/mappings\": { \"SingularDisplayName\": \"Microsoft.Fidalgo devcenters mapping\" }\r\n ,\"microsoft.fidalgo/machinedefinitions\": { \"SingularDisplayName\": \"Microsoft.Fidalgo machinedefinition\" }\r\n ,\"microsoft.fidalgo/networksettings\": { \"SingularDisplayName\": \"Microsoft.Fidalgo networksetting\" }\r\n ,\"microsoft.fidalgo/networksettings/healthchecks\": { \"SingularDisplayName\": \"Microsoft.Fidalgo networksettings healthcheck\" }\r\n ,\"microsoft.fidalgo/projects\": { \"SingularDisplayName\": \"Microsoft.Fidalgo project\" }\r\n ,\"microsoft.fidalgo/projects/attachednetworks\": { \"SingularDisplayName\": \"Microsoft.Fidalgo projects attachednetwork\" }\r\n ,\"microsoft.fidalgo/projects/devboxdefinitions\": { \"SingularDisplayName\": \"Microsoft.Fidalgo projects devboxdefinition\" }\r\n ,\"microsoft.fidalgo/projects/environments\": { \"SingularDisplayName\": \"Microsoft.Fidalgo projects environment\" }\r\n ,\"microsoft.fidalgo/projects/pools\": { \"SingularDisplayName\": \"Microsoft.Fidalgo projects pool\" }\r\n ,\"microsoft.fileshares/fileshares\": { \"SingularDisplayName\": \"File share\" }\r\n ,\"microsoft.fluidrelay/fluidrelayservers\": { \"SingularDisplayName\": \"Fluid Relay\" }\r\n ,\"microsoft.footprintmonitoring/profiles\": { \"SingularDisplayName\": \"Microsoft.FootprintMonitoring profile\" }\r\n ,\"microsoft.footprintmonitoring/profiles/experiments\": { \"SingularDisplayName\": \"Microsoft.FootprintMonitoring profiles experiment\" }\r\n ,\"microsoft.footprintmonitoring/profiles/measurementendpoints\": { \"SingularDisplayName\": \"Microsoft.FootprintMonitoring profiles measurement endpoint\" }\r\n ,\"microsoft.footprintmonitoring/profiles/measurementendpoints/conditions\": { \"SingularDisplayName\": \"Microsoft.FootprintMonitoring profiles measurement endpoints condition\" }\r\n ,\"microsoft.gallery/myareas/galleryitems\": { \"SingularDisplayName\": \"Template\" }\r\n ,\"microsoft.genomics/accounts\": { \"SingularDisplayName\": \"Genomics account\" }\r\n ,\"microsoft.graph/azureadapplication\": { \"SingularDisplayName\": \"Entra application\" }\r\n ,\"microsoft.graph/azureadapplicationprototype\": { \"SingularDisplayName\": \"Microsoft.Graph Azure ad application prototype\" }\r\n ,\"microsoft.graphservices/accounts\": { \"SingularDisplayName\": \"Metered API account\" }\r\n ,\"microsoft.guestconfiguration/guestconfigurationassignments\": { \"SingularDisplayName\": \"Microsoft.GuestConfiguration guest configuration assignment\" }\r\n ,\"microsoft.guestconfiguration/guestconfigurationassignments/reports\": { \"SingularDisplayName\": \"Microsoft.GuestConfiguration guest configuration assignments report\" }\r\n ,\"microsoft.hanaonazure/hanainstances\": { \"SingularDisplayName\": \"SAP HANA on Azure\" }\r\n ,\"microsoft.hanaonazure/sapmonitors\": { \"SingularDisplayName\": \"Azure Monitor for SAP Solutions (classic)\" }\r\n ,\"microsoft.hardware/orders\": { \"SingularDisplayName\": \"Microsoft.Hardware order\" }\r\n ,\"microsoft.hardwaresecuritymodules/cloudhsmclusters\": { \"SingularDisplayName\": \"Azure Cloud HSM\" }\r\n ,\"microsoft.hdinsight/clusterpools\": { \"SingularDisplayName\": \"Azure HDInsight on AKS cluster pool\" }\r\n ,\"microsoft.hdinsight/clusterpools/clusters\": { \"SingularDisplayName\": \"Azure HDInsight on AKS cluster\" }\r\n ,\"microsoft.hdinsight/clusterpools/clusters/instanceviews\": { \"SingularDisplayName\": \"Microsoft.HDInsight clusterpools clusters instance view\" }\r\n ,\"microsoft.hdinsight/clusters\": { \"SingularDisplayName\": \"HDInsight cluster\" }\r\n ,\"microsoft.healthbot/healthbots\": { \"SingularDisplayName\": \"Healthcare agent service\" }\r\n ,\"microsoft.healthcareapis/services\": { \"SingularDisplayName\": \"Azure API for FHIR\" }\r\n ,\"microsoft.healthcareapis/workspaces\": { \"SingularDisplayName\": \"Health Data Services workspace\" }\r\n ,\"microsoft.healthcareapis/workspaces/dicomservices\": { \"SingularDisplayName\": \"DICOM service\" }\r\n ,\"microsoft.healthcareapis/workspaces/fhirservices\": { \"SingularDisplayName\": \"FHIR service\" }\r\n ,\"microsoft.healthcareapis/workspaces/iotconnectors\": { \"SingularDisplayName\": \"MedTech service\" }\r\n ,\"microsoft.healthdataaiservices/deidservices\": { \"SingularDisplayName\": \"De-identification Service\" }\r\n ,\"microsoft.healthmodel/healthmodels\": { \"SingularDisplayName\": \"Health Model\" }\r\n ,\"microsoft.healthplatform/accounts\": { \"SingularDisplayName\": \"Microsoft.HealthPlatform account\" }\r\n ,\"microsoft.help/diagnostics\": { \"SingularDisplayName\": \"Microsoft.Help diagnostic\" }\r\n ,\"microsoft.help/selfhelp\": { \"SingularDisplayName\": \"Microsoft.Help self help\" }\r\n ,\"microsoft.help/simplifiedsolutions\": { \"SingularDisplayName\": \"Microsoft.Help simplified solution\" }\r\n ,\"microsoft.help/solutions\": { \"SingularDisplayName\": \"Microsoft.Help solution\" }\r\n ,\"microsoft.help/troubleshooters\": { \"SingularDisplayName\": \"Microsoft.Help troubleshooter\" }\r\n ,\"microsoft.hpcworkbench/instances\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instance\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chamber\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers/accessprofiles\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chambers access profile\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers/filerequests\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chambers file request\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers/files\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chambers file\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers/storages\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chambers storage\" }\r\n ,\"microsoft.hpcworkbench/instances/chambers/workloads\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances chambers workload\" }\r\n ,\"microsoft.hpcworkbench/instances/consortiums\": { \"SingularDisplayName\": \"Microsoft.HpcWorkbench instances consortium\" }\r\n ,\"microsoft.hybridcloud/cloudconnections\": { \"SingularDisplayName\": \"Microsoft.HybridCloud cloud connection\" }\r\n ,\"microsoft.hybridcloud/cloudconnectors\": { \"SingularDisplayName\": \"Microsoft.HybridCloud cloud connector\" }\r\n ,\"microsoft.hybridcompute/arcgatewayassociatedresources\": { \"SingularDisplayName\": \"Arc gateway associated resource\" }\r\n ,\"microsoft.hybridcompute/arcserverwithwac\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/gateways\": { \"SingularDisplayName\": \"Arc gateway\" }\r\n ,\"microsoft.hybridcompute/licenses\": { \"SingularDisplayName\": \"Extended Security Updates - Windows Server 2012/R2\" }\r\n ,\"microsoft.hybridcompute/machines\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/machines/microsoft.awsconnector/ec2instances\": { \"SingularDisplayName\": \"Microsoft.AwsConnector ec2 instance\" }\r\n ,\"microsoft.hybridcompute/machines/microsoft.connectedvmwarevsphere/virtualmachineinstances\": { \"SingularDisplayName\": \"VMware + AVS virtual machine\" }\r\n ,\"microsoft.hybridcompute/machines/providers/guestconfigurationassignments\": { \"SingularDisplayName\": \"Guest Assignment\" }\r\n ,\"microsoft.hybridcompute/machinesesu\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/machinespaygo\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/machinessoftwareassurance\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/machinessovereign\": { \"SingularDisplayName\": \"Machine - Azure Arc\" }\r\n ,\"microsoft.hybridcompute/privatelinkscopes\": { \"SingularDisplayName\": \"Azure Arc Private Link Scope\" }\r\n ,\"microsoft.hybridcompute/settings\": { \"SingularDisplayName\": \"Microsoft.HybridCompute setting\" }\r\n ,\"microsoft.hybridconnectivity/endpoints\": { \"SingularDisplayName\": \"Microsoft.HybridConnectivity endpoint\" }\r\n ,\"microsoft.hybridconnectivity/endpoints/serviceconfigurations\": { \"SingularDisplayName\": \"Microsoft.HybridConnectivity endpoints service configuration\" }\r\n ,\"microsoft.hybridconnectivity/publiccloudconnectors\": { \"SingularDisplayName\": \"Multicloud connector\" }\r\n ,\"microsoft.hybridconnectivity/solutionconfigurations\": { \"SingularDisplayName\": \"Microsoft.HybridConnectivity solution configuration\" }\r\n ,\"microsoft.hybridconnectivity/solutionconfigurations/inventory\": { \"SingularDisplayName\": \"Microsoft.HybridConnectivity solution configurations inventory\" }\r\n ,\"microsoft.hybridconnectivity/solutiontypes\": { \"SingularDisplayName\": \"Microsoft.HybridConnectivity solution type\" }\r\n ,\"microsoft.hybridcontainerservice/kubernetesversions\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService kubernetes version\" }\r\n ,\"microsoft.hybridcontainerservice/provisionedclusterinstances\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService provisioned cluster instance\" }\r\n ,\"microsoft.hybridcontainerservice/provisionedclusterinstances/agentpools\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService provisioned cluster instances agent pool\" }\r\n ,\"microsoft.hybridcontainerservice/provisionedclusterinstances/hybrididentitymetadata\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService provisioned cluster instances hybrid identity metadata\" }\r\n ,\"microsoft.hybridcontainerservice/provisionedclusterinstances/upgradeprofiles\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService provisioned cluster instances upgrade profile\" }\r\n ,\"microsoft.hybridcontainerservice/provisionedclusters\": { \"SingularDisplayName\": \"Kubernetes hybrid - Azure Arc\" }\r\n ,\"microsoft.hybridcontainerservice/skus\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService SKU\" }\r\n ,\"microsoft.hybridcontainerservice/storagespaces\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService storage space\" }\r\n ,\"microsoft.hybridcontainerservice/virtualnetworks\": { \"SingularDisplayName\": \"Microsoft.HybridContainerService virtual network\" }\r\n ,\"microsoft.hybriddata/datamanagers\": { \"SingularDisplayName\": \"Microsoft.HybridData data manager\" }\r\n ,\"microsoft.hybriddata/datamanagers/dataservices\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers data service\" }\r\n ,\"microsoft.hybriddata/datamanagers/dataservices/jobdefinitions\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers data services job definition\" }\r\n ,\"microsoft.hybriddata/datamanagers/dataservices/jobdefinitions/jobs\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers data services job definitions job\" }\r\n ,\"microsoft.hybriddata/datamanagers/datastores\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers data store\" }\r\n ,\"microsoft.hybriddata/datamanagers/datastoretypes\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers data store type\" }\r\n ,\"microsoft.hybriddata/datamanagers/publickeys\": { \"SingularDisplayName\": \"Microsoft.HybridData data managers public key\" }\r\n ,\"microsoft.hybridnetwork/configurationgroupvalues\": { \"SingularDisplayName\": \"Configuration Group Value\" }\r\n ,\"microsoft.hybridnetwork/devices\": { \"SingularDisplayName\": \"Azure Network Function Manager ? Device\" }\r\n ,\"microsoft.hybridnetwork/networkfunctions\": { \"SingularDisplayName\": \"Azure Network Function Manager ? Network Function\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publisher\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/artifactstores\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers artifact store\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/configurationgroupschemas\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers configuration group schema\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/networkfunctiondefinitiongroups\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers network function definition group\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/networkfunctiondefinitiongroups/networkfunctiondefinitionversions\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers network function definition groups network function definition version\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/networkservicedesigngroups\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers network service design group\" }\r\n ,\"microsoft.hybridnetwork/proxypublishers/networkservicedesigngroups/networkservicedesignversions\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork proxy publishers network service design groups network service design version\" }\r\n ,\"microsoft.hybridnetwork/publishers\": { \"SingularDisplayName\": \"Publisher\" }\r\n ,\"microsoft.hybridnetwork/publishers/artifactstores\": { \"SingularDisplayName\": \"Publisher Artifact Store\" }\r\n ,\"microsoft.hybridnetwork/publishers/artifactstores/artifactmanifests\": { \"SingularDisplayName\": \"Publisher Artifact Manifest\" }\r\n ,\"microsoft.hybridnetwork/publishers/configurationgroupschemas\": { \"SingularDisplayName\": \"Configuration Group Schema\" }\r\n ,\"microsoft.hybridnetwork/publishers/networkfunctiondefinitiongroups\": { \"SingularDisplayName\": \"Network Function Definition\" }\r\n ,\"microsoft.hybridnetwork/publishers/networkfunctiondefinitiongroups/networkfunctiondefinitionversions\": { \"SingularDisplayName\": \"Network Function Definition Version\" }\r\n ,\"microsoft.hybridnetwork/publishers/networkservicedesigngroups\": { \"SingularDisplayName\": \"Network Service Design\" }\r\n ,\"microsoft.hybridnetwork/publishers/networkservicedesigngroups/networkservicedesignversions\": { \"SingularDisplayName\": \"Network Service Design Version\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management container\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers/rolloutsequences\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management containers rollout sequence\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers/rollouttiers\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management containers rollout tier\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers/updatespecifications\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management containers update specification\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers/updatespecifications/rollouts\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management containers update specifications rollout\" }\r\n ,\"microsoft.hybridnetwork/servicemanagementcontainers/updatespecifications/rollouts/statuses\": { \"SingularDisplayName\": \"Microsoft.HybridNetwork service management containers update specifications rollouts statuse\" }\r\n ,\"microsoft.hybridnetwork/sitenetworkservices\": { \"SingularDisplayName\": \"Site Network Service\" }\r\n ,\"microsoft.hybridnetwork/sites\": { \"SingularDisplayName\": \"Site\" }\r\n })[tolower(id)]\r\n}\r\n", + "$fxv#10": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Ingestion database\r\n// See known issues @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n//===| Prices |=========================================================================================================\r\n// Supported versions:\r\n// - MS EA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-ea\r\n// - MS MCA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-mca\r\n//======================================================================================================================\r\n\r\n// Prices_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='Transforms Prices_raw into FOCUS 1.2.', folder='Prices')\r\nPrices_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', int(null),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n let prices = materialize(\r\n Prices_raw\r\n | extend PricingCurrency = coalesce(Currency, CurrencyCode) // CurrencyCode last as a fallback only\r\n | extend x_SkuId = coalesce(SkuId, SkuID)\r\n | extend x_SkuMeterId = coalesce(MeterId, MeterID)\r\n | extend x_SkuProductId = coalesce(ProductId, ProductID)\r\n | extend x_SkuTerm = isoMonths(Term)\r\n | project-rename\r\n SkuMeter = MeterName,\r\n x_BaseUnitPrice = BasePrice,\r\n x_EffectivePeriodEnd = EffectiveEndDate,\r\n x_EffectivePeriodStart = EffectiveStartDate,\r\n x_PricingUnitDescription = UnitOfMeasure,\r\n x_SkuIncludedQuantity = IncludedQuantity,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuMeterType = MeterType,\r\n x_SkuOfferId = OfferID,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPriceType = PriceType,\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTier = TierMinimumUnits\r\n | extend ContractedUnitPrice = iff(x_SkuPriceType != 'SavingsPlan', UnitPrice, real(null)) // UnitPrice for savings plan is not the on-demand unit price\r\n | extend ListUnitPrice = iff(x_SkuPriceType != 'SavingsPlan', MarketPrice, real(null)) // MarketPrice for savings plan is not the list price\r\n | extend ChargeCategory = case(\r\n x_SkuPriceType == 'Consumption', 'Usage',\r\n x_SkuPriceType == 'ReservedInstance', 'Purchase',\r\n x_SkuPriceType == 'SavingsPlan', 'Usage', // Savings plan prices are for committed usage, not the purchase\r\n ''\r\n )\r\n | extend SkuPriceIdv2 = strcat(case(x_SkuPriceType == 'Consumption', 'OD', x_SkuPriceType == 'ReservedInstance', 'RI', x_SkuPriceType == 'SavingsPlan', 'SP', 'XX'), substring(ChargeCategory, 0, 1), x_SkuTerm, '_', x_SkuProductId, '_', x_SkuId, '_', x_SkuMeterType, '_', x_SkuTier, x_SkuOfferId)\r\n | extend x_BillingAccountId = iff(BillingAccountId startswith '/', split(BillingAccountId, '/')[-1], coalesce(BillingAccountId, EnrollmentNumber))\r\n | extend x_BillingProfileId = iff(BillingProfileId startswith '/', split(BillingProfileId, '/')[-1], coalesce(BillingProfileId, EnrollmentNumber))\r\n | extend tmp_SavingsPlanKey = strcat(x_SkuMeterId, x_SkuProductId, x_SkuId, x_SkuTier, x_SkuOfferId)\r\n //\r\n // Get latest ingested row based on the unique ID\r\n | extend x_IngestionTime = ingestion_time()\r\n );\r\n //\r\n // Meters for reservations and savings plans to identify commitment eligibility\r\n let riMeters = prices | where x_SkuPriceType == 'ReservedInstance' | distinct x_SkuMeterId;\r\n let spMeters = prices | where x_SkuPriceType == 'SavingsPlan' | distinct x_SkuMeterId;\r\n //\r\n // Copy list/base/contracted prices from on-demand SKUs\r\n prices\r\n | where x_SkuPriceType == 'SavingsPlan'\r\n // If we use join, specify the shuffle key\r\n // TODO: Compare join vs. lookup perf -- | join kind=leftouter hint.strategy=shuffle (prices | where x_SkuPriceType == 'Consumption' | where x_SkuMeterId in (spMeters) | distinct tmp_SavingsPlanKey, ListUnitPrice, ContractedUnitPrice, x_BaseUnitPrice) on tmp_SavingsPlanKey\r\n | lookup kind=leftouter (prices | where x_SkuPriceType == 'Consumption' | where x_SkuMeterId in (spMeters) | distinct tmp_SavingsPlanKey, ListUnitPrice, ContractedUnitPrice, x_BaseUnitPrice) on tmp_SavingsPlanKey\r\n | extend ListUnitPrice = coalesce(ListUnitPrice, ListUnitPrice1)\r\n | extend ContractedUnitPrice = coalesce(ContractedUnitPrice, ContractedUnitPrice1)\r\n | extend x_BaseUnitPrice = coalesce(x_BaseUnitPrice, x_BaseUnitPrice1)\r\n | project-away ListUnitPrice1, ContractedUnitPrice1, x_BaseUnitPrice1, tmp_SavingsPlanKey\r\n | union ((prices | where x_SkuPriceType != 'SavingsPlan'))\r\n //\r\n // Set CommitmentDiscountCategory for reuse\r\n | extend CommitmentDiscountCategory = case(\r\n x_SkuPriceType == 'ReservedInstance', 'Usage',\r\n x_SkuPriceType == 'SavingsPlan', 'Spend',\r\n ''\r\n )\r\n //\r\n // Calculate commitment discount eligibility\r\n // TODO: Would a join be faster?\r\n // TODO: Check this to ensure it's correct\r\n | extend x_CommitmentDiscountSpendEligibility = iff(x_SkuMeterId in (riMeters) and x_SkuPriceType != 'ReservedInstance', 'Eligible', 'Not Eligible')\r\n | extend x_CommitmentDiscountUsageEligibility = iff(x_SkuMeterId in (spMeters), 'Eligible', 'Not Eligible')\r\n //\r\n // TODO: Implement x_CommitmentDiscountNormalizedRatio\r\n | extend x_CommitmentDiscountNormalizedRatio = real(null)\r\n //\r\n // Add PricingUnit and x_PricingBlockSize\r\n // TODO: Compare join vs. lookup perf -- | join kind=leftouter (PricingUnits) on x_PricingUnitDescription | project-away x_PricingUnitDescription1\r\n | lookup kind=leftouter (PricingUnits) on x_PricingUnitDescription\r\n //\r\n | extend x_EffectiveUnitPrice = iff(x_SkuPriceType == 'SavingsPlan', UnitPrice, real(null)) // Savings plan prices are for the effective price, not the contracted price\r\n | extend x_EffectiveUnitPriceDiscount = ContractedUnitPrice - x_EffectiveUnitPrice\r\n | extend x_ContractedUnitPriceDiscount = ListUnitPrice - ContractedUnitPrice\r\n | extend x_TotalUnitPriceDiscount = ListUnitPrice - x_EffectiveUnitPrice\r\n | project\r\n BillingAccountId = tolower(case(\r\n BillingProfileId startswith '/', BillingProfileId,\r\n BillingAccountId startswith '/', BillingAccountId,\r\n strcat('/providers/microsoft.billing/billingaccounts/', x_BillingAccountId, iff(x_BillingProfileId == x_BillingAccountId, '', strcat('/billingprofiles/', x_BillingProfileId)))\r\n )),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName, x_BillingProfileId),\r\n BillingCurrency = coalesce(BillingCurrency, CurrencyCode, Currency), // Currency last as a fallback only\r\n ChargeCategory,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountType = case(\r\n x_SkuPriceType == 'ReservedInstance', 'Reservation',\r\n x_SkuPriceType == 'SavingsPlan', 'Savings plan',\r\n ''\r\n ),\r\n CommitmentDiscountUnit = case(\r\n isempty(CommitmentDiscountCategory), '',\r\n CommitmentDiscountCategory == 'Spend', PricingCurrency,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio == real(1), PricingUnit,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio > real(1), strcat('Normalized ', PricingUnit),\r\n ''\r\n ),\r\n ContractedUnitPrice,\r\n ListUnitPrice,\r\n PricingCategory = case(\r\n x_SkuPriceType == 'Consumption', 'Standard',\r\n x_SkuPriceType == 'ReservedInstance', 'Standard', // Reservation purchases are tracked as \"Standard\"\r\n x_SkuPriceType == 'SavingsPlan', 'Committed',\r\n ''\r\n ),\r\n PricingCurrency,\r\n PricingUnit,\r\n SkuId = coalesce(ProductId, ProductID),\r\n SkuMeter,\r\n SkuPriceId = strcat(x_SkuProductId, '_', x_SkuId, '_', x_SkuMeterType),\r\n SkuPriceIdv2,\r\n x_BaseUnitPrice,\r\n x_BillingAccountAgreement = case(\r\n strlen(x_BillingAccountId) > 32, 'MCA',\r\n strlen(x_BillingAccountId) < 32, 'EA',\r\n 'Unknown'\r\n ),\r\n x_BillingAccountId,\r\n x_BillingProfileId,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_CommitmentDiscountSpendEligibility,\r\n x_CommitmentDiscountUsageEligibility,\r\n x_ContractedUnitPriceDiscount,\r\n x_ContractedUnitPriceDiscountPercent = 1.0 * x_ContractedUnitPriceDiscount / ListUnitPrice * 100,\r\n x_EffectivePeriodEnd = startofmonth(x_EffectivePeriodEnd + 1h),\r\n x_EffectivePeriodStart,\r\n x_EffectiveUnitPrice,\r\n x_EffectiveUnitPriceDiscount,\r\n x_EffectiveUnitPriceDiscountPercent = 1.0 * x_EffectiveUnitPriceDiscount / ContractedUnitPrice * 100,\r\n x_IngestionTime,\r\n x_PricingBlockSize,\r\n x_PricingSubcategory = case(\r\n x_SkuPriceType == 'Consumption' and (x_SkuIncludedQuantity > 0 or x_SkuTier > 0), 'Tiered',\r\n x_SkuPriceType == 'Consumption', 'Standard',\r\n x_SkuPriceType == 'ReservedInstance', 'Standard', // Reservation purchases are tracked as \"Standard\"\r\n x_SkuPriceType == 'SavingsPlan', 'Committed Spend',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_SkuDescription = Product,\r\n x_SkuId,\r\n x_SkuIncludedQuantity,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterSubcategory,\r\n x_SkuMeterType,\r\n x_SkuPriceType,\r\n x_SkuProductId,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuOfferId,\r\n x_SkuPartNumber,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceName = coalesce(x_SourceName, 'Cost Management'),\r\n x_SourceProvider = coalesce(x_SourceProvider, 'Microsoft'),\r\n x_SourceType = coalesce(x_SourceType, 'PriceSheet'),\r\n x_SourceVersion = coalesce(x_SourceVersion, '2023-05-01'),\r\n x_TotalUnitPriceDiscount,\r\n x_TotalUnitPriceDiscountPercent = 1.0 * x_TotalUnitPriceDiscount / ListUnitPrice * 100\r\n}\r\n\r\n// Prices_final_v1_2 table\r\n.create-merge table Prices_final_v1_2 (\r\n BillingAccountId: string,\r\n BillingAccountName: string,\r\n BillingCurrency: string,\r\n ChargeCategory: string,\r\n CommitmentDiscountCategory: string,\r\n CommitmentDiscountType: string,\r\n CommitmentDiscountUnit: string,\r\n ContractedUnitPrice: real,\r\n ListUnitPrice: real,\r\n PricingCategory: string,\r\n PricingCurrency: string, // Azure\r\n PricingUnit: string,\r\n SkuId: string,\r\n SkuMeter: string, // Azure\r\n SkuPriceId: string,\r\n SkuPriceIdv2: string, // Hubs add-on\r\n x_BaseUnitPrice: real, // Azure\r\n x_BillingAccountAgreement: string, // Hubs add-on\r\n x_BillingAccountId: string, // Azure MCA\r\n x_BillingProfileId: string, // Azure MCA\r\n x_CommitmentDiscountNormalizedRatio: real, // Hubs add-on\r\n x_CommitmentDiscountSpendEligibility: string, // Hubs add-on\r\n x_CommitmentDiscountUsageEligibility: string, // Hubs add-on\r\n x_ContractedUnitPriceDiscount: real, // Hubs add-on\r\n x_ContractedUnitPriceDiscountPercent: real, // Hubs add-on\r\n x_EffectivePeriodEnd: datetime, // Azure\r\n x_EffectivePeriodStart: datetime, // Azure\r\n x_EffectiveUnitPrice: real, // Azure\r\n x_EffectiveUnitPriceDiscount: real, // Hubs add-on\r\n x_EffectiveUnitPriceDiscountPercent: real, // Hubs add-on\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_PricingBlockSize: real, // Hubs add-on\r\n x_PricingSubcategory: string, // Hubs add-on\r\n x_PricingUnitDescription: string, // Azure\r\n x_SkuDescription: string, // Azure\r\n x_SkuId: string, // Azure\r\n x_SkuIncludedQuantity: real, // Azure EA\r\n x_SkuMeterCategory: string, // Azure\r\n x_SkuMeterId: string, // Azure\r\n x_SkuMeterSubcategory: string, // Azure\r\n x_SkuMeterType: string, // Azure\r\n x_SkuPriceType: string, // Azure\r\n x_SkuProductId: string, // Azure\r\n x_SkuRegion: string, // Azure\r\n x_SkuServiceFamily: string, // Azure\r\n x_SkuOfferId: string, // Azure EA\r\n x_SkuPartNumber: string, // Azure EA\r\n x_SkuTerm: int, // Azure\r\n x_SkuTier: real, // Azure MCA\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_TotalUnitPriceDiscount: real, // Hubs add-on\r\n x_TotalUnitPriceDiscountPercent: real // Hubs add-on\r\n)\r\n\r\n// Update policy for Prices_raw -> Prices_final_v1_2\r\n.alter table Prices_final_v1_2 policy update\r\n```\r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"Prices_raw\",\r\n \"Query\": \"Prices_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Cost and usage |=================================================================================================\r\n// Supported versions:\r\n// - MS: 1.2-preview, 1.0, 1.0-preview(v1)\r\n// https://aka.ms/costmgmt/exports/focus\r\n// - AWS: 1.0\r\n// https://docs.aws.amazon.com/cur/latest/userguide/table-dictionary-focus-1-0-aws-columns.html\r\n// - GCP: Jan-Jun 2024\r\n// https://cloud.google.com/resources/google-cloud-focus?e=48754805&hl=en\r\n// Links to (Aug 2024): https://services.google.com/fh/files/misc/focus_guide_v1.pdf\r\n// See also:\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/standard-usage\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/detailed-usage\r\n// - OCI: 1.0 \r\n// https://docs.oracle.com/iaas/Content/Billing/Concepts/costusagereportsoverview.htm#costreports__focus-cost-report-schema\r\n//\r\n// Support for non-Azure data is limited to ingestion only. Data is not transformed across versions.\r\n//======================================================================================================================\r\n\r\n// Costs_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='All costs transformed to FOCUS 1.2.', folder='Costs')\r\nCosts_transform_v1_2()\r\n{\r\n let checkString = (column: string, oldValue: string, newValue: string) {\r\n iff(oldValue == newValue, dynamic({}), bag_pack(column, oldValue))\r\n };\r\n let checkInt = (column: string, oldValue: int, newValue: int) {\r\n iff(oldValue == newValue, dynamic({}), bag_pack(column, oldValue))\r\n };\r\n let checkReal = (column: string, oldValue: real, newValue: real) {\r\n iff(oldValue == newValue, dynamic({}), bag_pack(column, oldValue))\r\n };\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n Costs_raw\r\n //\r\n // Dedupe rows\r\n | extend x_IngestionTime = ingestion_time()\r\n | extend x_ChargeId = ''\r\n // TODO: Consider adding a unique charge ID per row\r\n // hash_sha256(strcat(\r\n // // DO NOT CHANGE COLUMNS OR COLUMN ORDER\r\n // // 1. Resource hierarchy (including resource name), highest to lowest\r\n // BillingAccountId,\r\n // x_InvoiceSectionId,\r\n // x_AccountOwnerId,\r\n // SubAccountId,\r\n // x_ResourceGroupName,\r\n // ResourceName,\r\n // // 2. Resource details\r\n // ResourceId,\r\n // RegionId,\r\n // Tags,\r\n // CommitmentDiscountId,\r\n // x_CostCenter,\r\n // // 4. Meter details\r\n // SkuPriceId,\r\n // x_SkuMeterId,\r\n // x_SkuPartNumber,\r\n // x_SkuOfferId,\r\n // x_SkuDetails,\r\n // // 5. Date\r\n // ChargePeriodStart\r\n // ))\r\n //\r\n // Identify data quality issues\r\n // TODO: Remove x_SourceChanges in v1_3 (or later)\r\n | extend x_SourceChanges = trim_end(',', strcat(\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and ChargeFrequency == 'Usage-Based', 'InvalidChargeFrequency,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and EffectiveCost > 0, 'InvalidEffectiveCost,', ''),\r\n iff((isempty(ContractedCost) or ContractedCost == 0) and EffectiveCost != 0, 'MissingContractedCost,', ''),\r\n iff((isempty(ContractedUnitPrice) or ContractedUnitPrice == 0) and x_EffectiveUnitPrice != 0, 'MissingContractedUnitPrice,', ''),\r\n iff(ListCost < ContractedCost, 'ListCostLessThanContractedCost,', ''),\r\n iff(ContractedCost < EffectiveCost, 'ContractedCostLessThanEffectiveCost,', ''),\r\n iff((isempty(ListCost) or ListCost == 0) and (ContractedCost != 0 or EffectiveCost != 0), 'MissingListCost,', ''),\r\n iff((isempty(ListUnitPrice) or ListUnitPrice == 0) and (ContractedUnitPrice != 0 or x_EffectiveUnitPrice != 0), 'MissingListUnitPrice,', ''),\r\n iff((isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(x_BilledUnitPrice - x_EffectiveUnitPrice) < 0.0001)\r\n or (isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(ContractedUnitPrice - x_EffectiveUnitPrice) < 0.0001),\r\n 'XEffectiveUnitPriceRoundingError,', ''),\r\n iff(ConsumedQuantity == 0 and (EffectiveCost != 0 or BilledCost != 0), 'MissingConsumedQuantity,', ''),\r\n iff(PricingQuantity == 0 and (EffectiveCost != 0 or BilledCost != 0), 'MissingPricingQuantity,', ''),\r\n iff(isempty(ProviderName), 'MissingProviderName,', ''),\r\n iff(isempty(PublisherName), 'MissingPublisherName,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceId), 'MissingResourceId,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceName), 'MissingResourceName,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceType), 'MissingResourceType,', ''),\r\n iff(BilledCost > 0 and x_BilledUnitPrice == 0, 'MissingXBilledUnitPrice,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(x_ResourceType), 'MissingXResourceType,', ''),\r\n iff(PricingCategory == 'Standard' and isnotempty(CommitmentDiscountId) and ChargeCategory == 'Usage', 'PricingCategoryShouldBeCommitted,', ''),\r\n iff(x_SkuTerm == '1Year' or x_SkuTerm == '3Years' or x_SkuTerm == '5Years', 'SkuTermShouldBeAnInteger,', '')\r\n ))\r\n //\r\n // Handle provider columns that moved to FOCUS\r\n | extend PricingCurrency = coalesce(PricingCurrency, x_PricingCurrency)\r\n //\r\n // Backup original prices/costs before the merge\r\n | extend old_ContractedCost = ContractedCost\r\n | extend old_ContractedUnitPrice = ContractedUnitPrice\r\n | extend old_ListCost = ListCost\r\n | extend old_ListUnitPrice = ListUnitPrice\r\n | extend old_x_EffectiveUnitPrice = x_EffectiveUnitPrice\r\n //\r\n // Fix columns needed in other changes\r\n | extend old_ProviderName = ProviderName, ProviderName = case(\r\n isnotempty(ProviderName), ProviderName,\r\n isnotempty(coalesce(x_CostCategories, x_Discount, x_Operation, x_ServiceCode, x_UsageType)), 'AWS',\r\n isnotempty(coalesce(tostring(UsageAmount), tostring(x_Cost), x_Credits, x_CostType, tostring(x_CurrencyConversionRate), tostring(x_ExportTime), x_Project, x_ServiceId)), 'GCP',\r\n isnotempty(coalesce(x_BillingProfileId, x_InvoiceSectionId)), 'Microsoft',\r\n ''\r\n )\r\n //\r\n // Identify source\r\n | extend x_SourceName = coalesce(x_SourceName, iff(isnotempty(x_BillingProfileId), 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(isnotempty(x_BillingProfileId), 'FocusCost', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, case(\r\n isnotempty(coalesce(InvoiceId, SkuMeter, PricingCurrency)) and isempty(SkuPriceDetails) and isnotempty(x_SkuDetails), '1.2-preview',\r\n isnotempty(coalesce(InvoiceId, SkuMeter, PricingCurrency)), '1.2',\r\n isnotempty(coalesce(ChargeClass, CommitmentDiscountStatus, tostring(ConsumedQuantity), ConsumedUnit, tostring(ContractedCost), tostring(ContractedUnitPrice), RegionId, RegionName)), '1.0',\r\n isnotempty(coalesce(ChargeSubcategory, Region, tostring(UsageQuantity), UsageUnit)), iff(ProviderName == 'Microsoft', '1.0-preview(v1)', '1.0-preview'),\r\n ''\r\n ))\r\n // Append version check error code\r\n | extend x_SourceChanges = iff(x_SourceVersion == '1.0', x_SourceChanges,\r\n strcat(x_SourceChanges, iff(isempty(x_SourceChanges), '', ','), iff(x_SourceVersion == '', 'UnknownFocusVersion', 'LegacyFocusVersion'))\r\n )\r\n //\r\n // Fix quantities\r\n | extend old_PricingQuantity = PricingQuantity, PricingQuantity = case(\r\n PricingQuantity != 0 or (EffectiveCost == 0 and BilledCost == 0), PricingQuantity,\r\n PricingQuantity == 0 and isnotempty(BilledCost) and BilledCost != 0 and isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, BilledCost / x_BilledUnitPrice,\r\n PricingQuantity == 0 and isnotempty(BilledCost) and BilledCost != 0 and isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, BilledCost / x_BilledUnitPrice,\r\n PricingQuantity == 0 and isnotempty(EffectiveCost) and EffectiveCost != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0, EffectiveCost / x_EffectiveUnitPrice,\r\n PricingQuantity\r\n )\r\n | extend old_ConsumedQuantity = ConsumedQuantity, ConsumedQuantity = case(\r\n isnotempty(ConsumedQuantity) and ConsumedQuantity != 0, ConsumedQuantity,\r\n ChargeCategory == 'Usage', PricingQuantity / coalesce(x_PricingBlockSize, real(1)),\r\n ConsumedQuantity\r\n )\r\n //\r\n // Populate missing prices -- mapping to on-demand prices requires meter ID and offer ID\r\n | extend tmp_MissingPrices = ProviderName == 'Microsoft'\r\n and (isempty(ListUnitPrice) or isempty(ContractedUnitPrice) or ListUnitPrice == 0 or ContractedUnitPrice == 0)\r\n and x_EffectiveUnitPrice != 0\r\n and not(CommitmentDiscountCategory == 'Spend' and CommitmentDiscountStatus == 'Unused')\r\n and isnotempty(strcat(x_SkuMeterId, x_SkuOfferId))\r\n | as allCosts\r\n | where tmp_MissingPrices\r\n | extend tmp_ReservationPriceLookupKey = tolower(strcat(x_BillingProfileId, substring(ChargePeriodStart, 0, 7), x_SkuMeterId, x_SkuOfferId))\r\n | as costsWithMissingPrices\r\n | join kind=leftouter (\r\n Prices_final_v1_2\r\n | extend tmp_ReservationPriceLookupKey = tolower(strcat(x_BillingProfileId, substring(x_EffectivePeriodStart, 0, 7), x_SkuMeterId, x_SkuOfferId))\r\n | where x_SkuPriceType == 'Consumption' and tmp_ReservationPriceLookupKey in ((costsWithMissingPrices | summarize by tmp_ReservationPriceLookupKey))\r\n | summarize ListUnitPrice = min(ListUnitPrice), ContractedUnitPrice = min(ContractedUnitPrice) by tmp_ReservationPriceLookupKey, x_PricingBlockSize, PricingUnit\r\n ) on tmp_ReservationPriceLookupKey\r\n //\r\n // Select the best price to use for each row\r\n | extend x_EffectiveUnitPrice = case(\r\n // If price is a rounding error away from the billed price, use the billed price\r\n isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(x_BilledUnitPrice - x_EffectiveUnitPrice) < 0.0001, x_BilledUnitPrice,\r\n // If price is a rounding error away from the contracted price, use the contracted price\r\n isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(ContractedUnitPrice - x_EffectiveUnitPrice) < 0.0001, ContractedUnitPrice,\r\n x_EffectiveUnitPrice\r\n )\r\n | extend ContractedUnitPrice = case(\r\n // If price is already correct, keep that\r\n (isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0) or (EffectiveCost == 0 and BilledCost == 0), ContractedUnitPrice,\r\n // If both prices use the same scale, use the new one\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize == x_PricingBlockSize1, ContractedUnitPrice1 * x_BillingExchangeRate,\r\n // If prices are the same unit but not the same scale, use the new one but correct the scale\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize != x_PricingBlockSize1 and isnotempty(x_PricingBlockSize) and isnotempty(x_PricingBlockSize1), ContractedUnitPrice1 * x_BillingExchangeRate / x_PricingBlockSize1 * x_PricingBlockSize,\r\n // If billed price is available, assume the billed price is the same as contracted price to support aggregations\r\n isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, x_EffectiveUnitPrice,\r\n // Otherwise, assume the effective price is the same as contracted price to support aggregations\r\n x_EffectiveUnitPrice\r\n )\r\n | extend ListUnitPrice = case(\r\n // If price is already correct, keep that\r\n (isnotempty(ListUnitPrice) and ListUnitPrice != 0) or (EffectiveCost == 0 and BilledCost == 0), ListUnitPrice,\r\n // If both prices use the same scale, use the new one\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize == x_PricingBlockSize1, ListUnitPrice1 * x_BillingExchangeRate,\r\n // If prices are the same unit but not the same scale, use the new one but correct the scale\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize != x_PricingBlockSize1 and isnotempty(x_PricingBlockSize) and isnotempty(x_PricingBlockSize1), ListUnitPrice1 * x_BillingExchangeRate / x_PricingBlockSize1 * x_PricingBlockSize,\r\n // Otherwise, assume the contracted price is the same as list price to support aggregations\r\n ContractedUnitPrice\r\n )\r\n // Calculate missing costs based on new prices -- If cost is already correct, keep that; if not and price is available, recalculate the cost; otherwise, keep the existing cost\r\n | extend ContractedCost = case(\r\n // If not set or there's no cost, keep the original value\r\n (isnotempty(ContractedCost) and ContractedCost != 0) or (EffectiveCost == 0 and BilledCost == 0), ContractedCost,\r\n // ContractedCost is 0 in all other scenarios...\r\n // If 0 and there's a billed cost and prices are the same, use BilledCost\r\n isnotempty(BilledCost) and BilledCost != 0 and ContractedUnitPrice == x_BilledUnitPrice, BilledCost,\r\n // If 0 and there's a billed cost and prices are the same, use EffectiveCost\r\n isnotempty(EffectiveCost) and EffectiveCost != 0 and ContractedUnitPrice == x_EffectiveUnitPrice, EffectiveCost,\r\n // If 0 and there's a price, calculate the cost based on the price\r\n isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0, ContractedUnitPrice * PricingQuantity,\r\n // If 0 and there's no price, assume EffectiveCost\r\n isempty(ContractedUnitPrice) or ContractedUnitPrice == 0, EffectiveCost,\r\n // Fall back to the original value for any unhandled scenarios\r\n ContractedCost\r\n )\r\n | extend ListCost = case(\r\n // If not set or there's no cost, keep the original value\r\n (isnotempty(ListCost) and ListCost != 0) or (EffectiveCost == 0 and BilledCost == 0), ListCost,\r\n // ListCost is 0 in all other scenarios...\r\n // If 0 and there's a contracted cost and prices are the same, use ContractedCost\r\n isnotempty(ContractedCost) and ContractedCost != 0 and ListUnitPrice == ContractedUnitPrice, ContractedCost,\r\n // If 0 and there's a price, calculate the cost based on the price\r\n isnotempty(ListUnitPrice) and ListUnitPrice != 0, ListUnitPrice * PricingQuantity,\r\n // If 0 and there's no price, assume ContractedCost\r\n isempty(ListUnitPrice) or ListUnitPrice == 0, ContractedCost,\r\n // Fall back to the original value for any unhandled scenarios\r\n ListCost\r\n )\r\n // Merge the rest of the unmodified cost records and remove excess columns\r\n | union (allCosts | where not(tmp_MissingPrices))\r\n | project-away x_PricingBlockSize1, PricingUnit1, ListUnitPrice1, ContractedUnitPrice1, tmp_MissingPrices, tmp_ReservationPriceLookupKey, tmp_ReservationPriceLookupKey1\r\n //\r\n | extend SkuPriceDetails = parse_json(SkuPriceDetails)\r\n | extend Tags = parse_json(Tags)\r\n | extend x_SkuDetails = parse_json(x_SkuDetails)\r\n //\r\n // Handle FOCUS 1.0-preview\r\n | extend old_ChargeSubcategory = ChargeSubcategory\r\n | extend old_ChargeCategory = ChargeCategory, ChargeCategory = case(\r\n // Handle FOCUS 1.0-preview ChargeSubcategory\r\n ChargeSubcategory == 'Credit', 'Credit',\r\n ChargeSubcategory == 'Refund', 'Purchase', // We are assuming purchase refunds since we don't have data to indicate usage refunds\r\n ChargeCategory\r\n )\r\n | extend old_ChargeClass = ChargeClass, ChargeClass = case(ChargeSubcategory == 'Refund', 'Correction', ChargeClass)\r\n //\r\n // Populate CapacityReservationId when not specified\r\n | extend CapacityReservationId = coalesce(CapacityReservationId, tostring(coalesce(x_SkuDetails.VMCapacityReservationId, SkuPriceDetails.VMCapacityReservationId, SkuPriceDetails.x_VMCapacityReservationId)))\r\n | extend old_CapacityReservationStatus = CapacityReservationStatus, CapacityReservationStatus = case(\r\n isempty(CapacityReservationId), '',\r\n isnotempty(CapacityReservationStatus), CapacityReservationStatus,\r\n tolower(x_ResourceType) == 'microsoft.compute/capacityreservationgroups/capacityreservations', 'Unused',\r\n 'Used'\r\n )\r\n //\r\n // BUG: ChargeFrequency shows \"Usage-Based\" for monthly recurring savings plan purchases\r\n | extend old_ChargeFrequency = ChargeFrequency, ChargeFrequency = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and ChargeFrequency == 'Usage-Based' and ProviderName == 'Microsoft' and x_SourceVersion startswith '1.0', 'Recurring', ChargeFrequency)\r\n //\r\n // Commitment discounts\r\n | extend x_CommitmentDiscountNormalizedRatio = case(\r\n // Calculate from CommitmentDiscountQuantity, if specified\r\n isnotempty(CommitmentDiscountQuantity) and CommitmentDiscountQuantity != 0, CommitmentDiscountQuantity / PricingQuantity / coalesce(x_PricingBlockSize, real(1)),\r\n // Not applicable\r\n isempty(CommitmentDiscountStatus), real(null),\r\n // Parse from SKU details if not specified explicitly\r\n toreal(coalesce(x_SkuDetails.RINormalizationRatio, SkuPriceDetails.RINormalizationRatio, SkuPriceDetails.x_RINormalizationRatio, dynamic(1)))\r\n )\r\n | extend old_CommitmentDiscountQuantity = CommitmentDiscountQuantity, CommitmentDiscountQuantity = case(\r\n // FOCUS 1.2\r\n isnotempty(CommitmentDiscountQuantity), CommitmentDiscountQuantity,\r\n // FOCUS 1.0-preview, 1.0\r\n isempty(CommitmentDiscountStatus), real(null),\r\n CommitmentDiscountCategory == 'Spend', EffectiveCost / coalesce(x_BillingExchangeRate, real(1)),\r\n CommitmentDiscountCategory == 'Usage' and isnotempty(x_CommitmentDiscountNormalizedRatio), PricingQuantity / coalesce(x_PricingBlockSize, real(1)) * x_CommitmentDiscountNormalizedRatio,\r\n real(null)\r\n )\r\n | extend old_CommitmentDiscountUnit = CommitmentDiscountUnit, CommitmentDiscountUnit = case(\r\n // FOCUS 1.2\r\n isnotempty(CommitmentDiscountUnit), CommitmentDiscountUnit,\r\n // FOCUS 1.0\r\n isempty(CommitmentDiscountQuantity), '',\r\n CommitmentDiscountCategory == 'Spend', PricingCurrency,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio == real(1), ConsumedUnit,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio > real(1), strcat('Normalized ', ConsumedUnit),\r\n ''\r\n )\r\n | extend old_CommitmentDiscountStatus = CommitmentDiscountStatus, CommitmentDiscountStatus = case(\r\n // FOCUS 1.0+\r\n isnotempty(CommitmentDiscountStatus), CommitmentDiscountStatus,\r\n // FOCUS 1.0-preview\r\n ChargeSubcategory == 'Used Commitment', 'Used',\r\n ChargeSubcategory == 'Unused Commitment', 'Unused',\r\n ''\r\n )\r\n | extend x_CommitmentDiscountUtilizationPotential = case(\r\n ChargeCategory == 'Purchase', real(0),\r\n ProviderName == 'Microsoft' and isnotempty(CommitmentDiscountCategory), EffectiveCost,\r\n CommitmentDiscountCategory == 'Usage', ConsumedQuantity,\r\n CommitmentDiscountCategory == 'Spend', EffectiveCost,\r\n real(0)\r\n )\r\n | extend x_CommitmentDiscountUtilizationAmount = iff(CommitmentDiscountStatus == 'Used', x_CommitmentDiscountUtilizationPotential, real(0))\r\n //\r\n // Pricing\r\n | extend old_x_AmortizationClass = x_AmortizationClass, x_AmortizationClass = case(\r\n // FOCUS 1.2\r\n isnotempty(x_AmortizationClass), x_AmortizationClass,\r\n // FOCUS 1.0-preview+\r\n ChargeCategory == 'Purchase' and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeCategory == 'Usage' and isnotempty(CommitmentDiscountId) and isnotempty(CommitmentDiscountStatus), 'Amortized Charge',\r\n ''\r\n )\r\n | extend old_PricingCategory = PricingCategory, PricingCategory = case(\r\n // FOCUS 1.0+\r\n isnotempty(PricingCategory), PricingCategory,\r\n // FOCUS 1.0-preview\r\n PricingCategory == 'On-Demand', 'Standard',\r\n PricingCategory == 'Commitment-Based', 'Committed',\r\n ''\r\n )\r\n //\r\n // Commitment discount utilization\r\n | extend x_CommitmentDiscountUtilizationPotential = case(\r\n ChargeCategory == 'Purchase', real(0),\r\n ProviderName == 'Microsoft' and isnotempty(CommitmentDiscountCategory), EffectiveCost,\r\n CommitmentDiscountCategory == 'Usage', ConsumedQuantity,\r\n CommitmentDiscountCategory == 'Spend', EffectiveCost,\r\n real(0)\r\n )\r\n | extend x_CommitmentDiscountUtilizationAmount = iff(CommitmentDiscountStatus == 'Used', x_CommitmentDiscountUtilizationPotential, real(0))\r\n //\r\n // BUG: Fix ContractedCost that has bad values\r\n | extend ContractedCost = iff(ProviderName == 'Microsoft' and isnotempty(PricingQuantity) and isnotempty(x_PricingBlockSize) and ContractedCost != ContractedUnitPrice * PricingQuantity, ContractedUnitPrice * PricingQuantity, ContractedCost)\r\n //\r\n // Handle FOCUS 1.0-preview UsageQuantity/Unit\r\n | extend ConsumedQuantity = iff(ChargeCategory == 'Usage', coalesce(ConsumedQuantity, UsageQuantity, UsageAmount), real(null))\r\n | extend old_ConsumedUnit = ConsumedUnit, ConsumedUnit = iff(ChargeCategory == 'Usage' and isnotempty(ConsumedQuantity), coalesce(ConsumedUnit, UsageUnit, 'Units'), '')\r\n //\r\n // Convert IDs to lowercase for consistency\r\n | extend BillingAccountId = tolower(BillingAccountId)\r\n | extend CommitmentDiscountId = tolower(CommitmentDiscountId)\r\n //\r\n // BUG: Remove EffectiveCost for commitment discount purchases\r\n | extend old_EffectiveCost = EffectiveCost, EffectiveCost = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), real(0), EffectiveCost)\r\n | extend old_x_EffectiveCostInUsd = x_EffectiveCostInUsd, x_EffectiveCostInUsd = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), real(0), x_EffectiveCostInUsd)\r\n //\r\n // Clean up resource columns\r\n | extend old_ResourceId = ResourceId, ResourceId = case(\r\n isnotempty(ResourceId), ResourceId,\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), CommitmentDiscountId,\r\n ResourceId\r\n )\r\n | extend old_ResourceName = ResourceName, ResourceName = tolower(case(\r\n isnotempty(ResourceName), ResourceName,\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountName), CommitmentDiscountName,\r\n isnotempty(ResourceId), parse_resourceid(ResourceId).ResourceName,\r\n ResourceName\r\n ))\r\n | extend old_x_ResourceType = x_ResourceType, x_ResourceType = case(\r\n isnotempty(x_ResourceType), x_ResourceType,\r\n isnotempty(ResourceId), parse_resourceid(ResourceId).x_ResourceType,\r\n x_ResourceType\r\n )\r\n | extend old_ResourceType = ResourceType, ResourceType = case(\r\n // Use existing resource type display name unless it's an internal resource type ID\r\n isnotempty(ResourceType) and tolower(ResourceType) != tolower(x_ResourceType) and ResourceType !contains '/', ResourceType,\r\n // Use CommitmentDiscountType for commitment discount purchases\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountType), CommitmentDiscountType,\r\n // Look up display name from internal type\r\n isnotempty(x_ResourceType), coalesce(tostring(resource_type(x_ResourceType).SingularDisplayName), ResourceType, x_ResourceType),\r\n ResourceType\r\n )\r\n //\r\n // Handle missing values\r\n | extend old_PublisherName = PublisherName, PublisherName = case(PublisherName == 'Microsoft Corporation', 'Microsoft', isnotempty(PublisherName), PublisherName, x_PublisherCategory == 'Cloud Provider', ProviderName, '')\r\n //\r\n // Handle FOCUS 1.0-preview Region column\r\n | extend old_Region = Region\r\n | extend old_RegionId = RegionId, RegionId = coalesce(RegionId, iff(ProviderName == 'Microsoft', replace_string(tolower(Region), ' ', ''), Region))\r\n | extend RegionName = coalesce(RegionName, Region)\r\n //\r\n // SKU properties\r\n | extend x_SkuCoreCount = toint(coalesce(SkuPriceDetails.CoreCount, SkuPriceDetails.x_VCPUs, x_SkuDetails.VCPUs, SkuPriceDetails.x_VCores, x_SkuDetails.VCores, SkuPriceDetails.x_vCores, x_SkuDetails.vCores))\r\n | extend x_SkuInstanceType = tostring(coalesce(SkuPriceDetails.InstanceType, SkuPriceDetails.x_ServiceType, x_SkuDetails.ServiceType, SkuPriceDetails.x_ServerSku, x_SkuDetails.ServerSku))\r\n | extend x_SkuOperatingSystem = case(\r\n isnotempty(SkuPriceDetails.OperatingSystem), SkuPriceDetails.OperatingSystem,\r\n coalesce(SkuPriceDetails.x_ImageType, x_SkuDetails.ImageType) == 'Canonical', 'Linux',\r\n coalesce(SkuPriceDetails.x_ImageType, x_SkuDetails.ImageType) == 'Windows Server BYOL', 'Windows Server',\r\n x_SkuMeterSubcategory endswith ' Series Windows', 'Windows Server',\r\n coalesce(SkuPriceDetails.x_ImageType, x_SkuDetails.ImageType)\r\n )\r\n | extend x_ConsumedCoreHours = iff(ConsumedUnit == 'Hours' and isnotempty(x_SkuCoreCount), x_SkuCoreCount * ConsumedQuantity, real(null))\r\n | extend SkuPriceDetails = case(\r\n // FOCUS 1.2\r\n isnotempty(SkuPriceDetails), SkuPriceDetails,\r\n // FOCUS 1.0-preview, 1.0\r\n parse_json(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(tostring(x_SkuDetails)\r\n // Prefix all keys with x_ first to avoid double-prefixing\r\n , @'([\\{,])\"', @'\\1\"x_')\r\n // CoreCount for number of CPUs/vCPUs/cores/vCores\r\n , @'\"x_(VCPUs|VCores|vCores)\":', @'\"CoreCount\":')\r\n // TODO: DiskMaxIops for disk I/O operations per second (IOPS)\r\n // TODO: DiskSpace for disk size in GiB\r\n // TODO: DiskType for the kind of disk (e.g., SSD, HDD, NVMe)\r\n // TODO: GpuCount for the number of GPUs\r\n // InstanceType for the resource size/SKU (e.g., ArmSkuName)\r\n , @'\"x_(ServerSku|ServiceType)\":', @'\"InstanceType\":')\r\n // TODO: InstanceSeries for the size family/series\r\n // TODO: MemorySize for the RAM in GiB\r\n // TODO: NetworkMaxIops for network I/O operations per second (IOPS)\r\n // TODO: NetworkMaxThroughput for network max throughput for data transfer in Mbps\r\n // OperatingSystem for the OS name\r\n , @'(\"x_ImageType\":\"Canonical\")', @'\\1,\"OperatingSystem\":\"Linux\"')\r\n , @'(\"x_ImageType\":\"Windows Server( BYOL)?\")', @'\\1,\"OperatingSystem\":\"Windows Server\"')\r\n , @'(\"x_ImageType\":(\"[^\"]+\"))', @'\\1,\"OperatingSystem\":\\2')\r\n // TODO: Redundancy for the level of redundancy (e.g., Local, Zonal, Global)\r\n // TODO: StorageClass for the tier of storage (e.g., Hot, Archive, Nearline)\r\n )\r\n )\r\n | extend SkuPriceDetails = iff(isempty(SkuPriceDetails.OperatingSystem) and isnotempty(x_SkuOperatingSystem),\r\n parse_json(replace_string(tostring(SkuPriceDetails), '}', strcat(@',\"OperatingSystem\":\"', x_SkuOperatingSystem, '\"}'))),\r\n SkuPriceDetails)\r\n //\r\n // Azure Hybrid Benefit\r\n | extend tmp_SqlAhb = tolower(coalesce(x_SkuDetails.AHB, SkuPriceDetails.x_AHB))\r\n | extend x_SkuLicenseType = case(\r\n ChargeCategory != 'Usage', '',\r\n x_SkuMeterCategory in ('Virtual Machines', 'Virtual Machine Licenses') and (x_SkuMeterSubcategory contains 'Windows' or coalesce(SkuPriceDetails.x_ImageType, x_SkuDetails.ImageType) == 'Windows Server BYOL'), 'Windows Server',\r\n isnotempty(tmp_SqlAhb) or x_SkuMeterSubcategory == 'SQL Server Azure Hybrid Benefit', 'SQL Server',\r\n ''\r\n )\r\n | extend x_SkuLicenseStatus = case(\r\n isempty(x_SkuLicenseType), '',\r\n coalesce(SkuPriceDetails.x_ImageType, x_SkuDetails.ImageType) == 'Windows Server BYOL' or tmp_SqlAhb == 'true' or x_SkuMeterSubcategory contains 'Azure Hybrid Benefit', 'Enabled',\r\n (x_SkuMeterSubcategory contains 'Windows') or tmp_SqlAhb == 'false', 'Not Enabled',\r\n ''\r\n )\r\n | extend x_SkuLicenseQuantity = case(\r\n isempty(x_SkuCoreCount) or isempty(x_SkuLicenseType), int(null),\r\n x_SkuCoreCount <= 8, int(8),\r\n x_SkuCoreCount > 8, x_SkuCoreCount,\r\n int(null)\r\n )\r\n | extend x_SkuLicenseUnit = iff(isnotempty(x_SkuLicenseQuantity), 'Cores', '')\r\n //\r\n // Savings\r\n | extend x_CommitmentDiscountSavings = iff(isempty(ContractedCost) or ContractedCost == 0 or ContractedCost - EffectiveCost < 0.0001, real(0), ContractedCost - EffectiveCost)\r\n | extend x_NegotiatedDiscountSavings = iff(isempty(ListCost) or ListCost == 0 or ListCost - ContractedCost < 0.0001, real(0), ListCost - ContractedCost)\r\n | extend x_TotalSavings = iff(isempty(ListCost) or ListCost == 0 or ListCost - EffectiveCost < 0.0001, real(0), ListCost - EffectiveCost)\r\n | extend x_CommitmentDiscountPercent = iff(isempty(ContractedUnitPrice) or ContractedUnitPrice == 0 or ContractedUnitPrice - x_EffectiveUnitPrice < 0.0001, real(0), (ContractedUnitPrice - x_EffectiveUnitPrice) / ContractedUnitPrice)\r\n | extend x_NegotiatedDiscountPercent = iff(isempty(ListUnitPrice) or ListUnitPrice == 0 or ListUnitPrice - ContractedUnitPrice < 0.0001, real(0), (ListUnitPrice - ContractedUnitPrice) / ListUnitPrice)\r\n | extend x_TotalDiscountPercent = iff(isempty(ListUnitPrice) or ListUnitPrice == 0 or ListUnitPrice - x_EffectiveUnitPrice < 0.0001, real(0), (ListUnitPrice - x_EffectiveUnitPrice) / ListUnitPrice)\r\n //\r\n // Minor fixes\r\n | extend old_BillingPeriodEnd = BillingPeriodEnd, BillingPeriodEnd = startofmonth(BillingPeriodEnd)\r\n | extend old_BillingPeriodStart = BillingPeriodStart, BillingPeriodStart = startofmonth(BillingPeriodStart)\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n AvailabilityZone,\r\n BilledCost,\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingAccountType,\r\n BillingCurrency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n CapacityReservationId,\r\n CapacityReservationStatus,\r\n ChargeCategory,\r\n ChargeClass,\r\n ChargeDescription,\r\n ChargeFrequency,\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId,\r\n CommitmentDiscountName,\r\n CommitmentDiscountQuantity,\r\n CommitmentDiscountStatus,\r\n CommitmentDiscountType,\r\n CommitmentDiscountUnit,\r\n ConsumedQuantity,\r\n ConsumedUnit,\r\n ContractedCost = coalesce(ContractedCost, x_OnDemandCost, x_Cost),\r\n ContractedUnitPrice = coalesce(ContractedUnitPrice, x_OnDemandUnitPrice),\r\n EffectiveCost,\r\n InvoiceId = coalesce(InvoiceId, x_InvoiceId),\r\n InvoiceIssuerName,\r\n ListCost,\r\n ListUnitPrice,\r\n PricingCategory,\r\n PricingCurrency = coalesce(PricingCurrency, x_PricingCurrency),\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n PublisherName,\r\n RegionId,\r\n RegionName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory, // TODO: Populate ServiceSubcategory from ServiceName when missing\r\n SkuId,\r\n SkuMeter = coalesce(SkuMeter, x_SkuMeterName),\r\n SkuPriceDetails,\r\n SkuPriceId,\r\n SubAccountId,\r\n SubAccountName = iff(isempty(SubAccountId), '', SubAccountName),\r\n SubAccountType,\r\n Tags,\r\n x_AccountId = iff(x_AccountId == '-2', '', x_AccountId),\r\n x_AccountName = iff(x_AccountId == '-2', '', x_AccountName),\r\n x_AccountOwnerId = iff(x_AccountId == '-2', '', x_AccountOwnerId),\r\n x_AmortizationClass,\r\n x_BilledCostInUsd,\r\n x_BilledUnitPrice,\r\n x_BillingAccountAgreement = case(\r\n ProviderName == 'Microsoft' and x_BillingAccountId == x_BillingProfileId, 'EA',\r\n ProviderName == 'Microsoft' and x_BillingAccountId != x_BillingProfileId, 'MCA',\r\n ProviderName\r\n ),\r\n x_BillingAccountId,\r\n x_BillingAccountName,\r\n x_BillingExchangeRate,\r\n x_BillingExchangeRateDate,\r\n x_BillingItemCode,\r\n x_BillingItemName,\r\n x_BillingProfileId,\r\n x_BillingProfileName,\r\n x_ChargeId,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_CommitmentDiscountPercent,\r\n x_CommitmentDiscountSavings,\r\n x_CommitmentDiscountSpendEligibility = '', // TODO: Add x_CommitmentDiscountSpendEligibility for Costs\r\n x_CommitmentDiscountUsageEligibility = '', // TODO: Add x_CommitmentDiscountUsageEligibility for Costs\r\n x_CommitmentDiscountUtilizationAmount,\r\n x_CommitmentDiscountUtilizationPotential,\r\n x_CommodityCode,\r\n x_CommodityName,\r\n x_ComponentName,\r\n x_ComponentType,\r\n x_ConsumedCoreHours,\r\n x_ContractedCostInUsd = coalesce(x_ContractedCostInUsd, x_OnDemandCostInUsd),\r\n x_CostAllocationRuleName,\r\n x_CostCategories = parse_json(x_CostCategories),\r\n x_CostCenter,\r\n x_CostType,\r\n x_Credits = parse_json(x_Credits),\r\n x_CurrencyConversionRate,\r\n x_CustomerId,\r\n x_CustomerName,\r\n x_Discount = parse_json(x_Discount),\r\n x_EffectiveCostInUsd,\r\n x_EffectiveUnitPrice,\r\n x_ExportTime,\r\n x_IngestionTime,\r\n x_InstanceID,\r\n x_InvoiceIssuerId,\r\n x_InvoiceSectionId = case(\r\n x_InvoiceSectionId == '-2', '',\r\n x_InvoiceSectionId\r\n ),\r\n x_InvoiceSectionName = case(\r\n x_InvoiceSectionName == 'Unassigned', '',\r\n x_InvoiceSectionName\r\n ),\r\n x_ListCostInUsd,\r\n x_Location,\r\n x_NegotiatedDiscountPercent,\r\n x_NegotiatedDiscountSavings,\r\n x_Operation,\r\n x_OwnerAccountID,\r\n x_PartnerCreditApplied,\r\n x_PartnerCreditRate,\r\n x_PricingBlockSize,\r\n x_PricingSubcategory,\r\n x_PricingUnitDescription = iff(x_PricingUnitDescription == 'Unassigned', '', x_PricingUnitDescription),\r\n x_Project,\r\n x_PublisherCategory,\r\n x_PublisherId,\r\n x_ResellerId,\r\n x_ResellerName,\r\n x_ResourceGroupName = tolower(x_ResourceGroupName),\r\n x_ResourceType,\r\n x_ServiceCode,\r\n x_ServiceId,\r\n x_ServiceModel, // TODO: Populate from ServiceName when missing\r\n x_ServicePeriodEnd,\r\n x_ServicePeriodStart,\r\n x_SkuCoreCount,\r\n x_SkuDescription,\r\n x_SkuDetails,\r\n x_SkuInstanceType,\r\n x_SkuIsCreditEligible,\r\n x_SkuLicenseQuantity,\r\n x_SkuLicenseStatus,\r\n x_SkuLicenseType,\r\n x_SkuLicenseUnit,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterSubcategory,\r\n x_SkuOfferId,\r\n x_SkuOperatingSystem,\r\n x_SkuOrderId,\r\n x_SkuOrderName,\r\n x_SkuPartNumber,\r\n x_SkuPlanName,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceChanges,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceValues = bag_merge(\r\n checkString('BillingPeriodEnd', old_BillingPeriodEnd, BillingPeriodEnd),\r\n checkString('BillingPeriodStart', old_BillingPeriodStart, BillingPeriodStart),\r\n checkString('CapacityReservationStatus', old_CapacityReservationStatus, CapacityReservationStatus),\r\n checkString('ChargeCategory', old_ChargeCategory, ChargeCategory),\r\n checkString('ChargeClass', old_ChargeClass, ChargeClass),\r\n checkString('ChargeSubcategory', old_ChargeSubcategory, ''), // Not included in final schema; use empty string\r\n checkString('ChargeFrequency', old_ChargeFrequency, ChargeFrequency),\r\n checkReal('CommitmentDiscountQuantity', old_CommitmentDiscountQuantity, CommitmentDiscountQuantity),\r\n checkString('CommitmentDiscountUnit', old_CommitmentDiscountUnit, CommitmentDiscountUnit),\r\n checkString('CommitmentDiscountStatus', old_CommitmentDiscountStatus, CommitmentDiscountStatus),\r\n checkReal('ConsumedQuantity', old_ConsumedQuantity, ConsumedQuantity),\r\n checkString('ConsumedUnit', old_ConsumedUnit, ConsumedUnit),\r\n checkReal('ContractedCost', old_ContractedCost, ContractedCost),\r\n checkReal('ContractedUnitPrice', old_ContractedUnitPrice, ContractedUnitPrice),\r\n checkReal('EffectiveCost', old_EffectiveCost, EffectiveCost),\r\n checkReal('ListCost', old_ListCost, ListCost),\r\n checkReal('ListUnitPrice', old_ListUnitPrice, ListUnitPrice),\r\n checkString('PricingCategory', old_PricingCategory, PricingCategory),\r\n checkReal('PricingQuantity', old_PricingQuantity, PricingQuantity),\r\n checkString('ProviderName', old_ProviderName, ProviderName),\r\n checkString('PublisherName', old_PublisherName, PublisherName),\r\n checkString('Region', old_Region, ''), // Not included in final schema; use empty string\r\n checkString('RegionId', old_RegionId, RegionId),\r\n checkString('ResourceId', old_ResourceId, ResourceId),\r\n checkString('ResourceName', old_ResourceName, ResourceName),\r\n checkString('ResourceType', old_ResourceType, ResourceType),\r\n checkString('x_AmortizationClass', old_x_AmortizationClass, x_AmortizationClass),\r\n checkReal('x_EffectiveCostInUsd', old_x_EffectiveCostInUsd, x_EffectiveCostInUsd),\r\n checkReal('x_EffectiveUnitPrice', old_x_EffectiveUnitPrice, x_EffectiveUnitPrice),\r\n checkString('x_ResourceType', old_x_ResourceType, x_ResourceType)\r\n ),\r\n x_SourceVersion,\r\n x_SubproductName,\r\n x_TotalDiscountPercent,\r\n x_TotalSavings,\r\n x_UsageType\r\n}\r\n\r\n// Costs_final_v1_2 table\r\n.create-merge table Costs_final_v1_2 (\r\n AvailabilityZone: string,\r\n BilledCost: real,\r\n BillingAccountId: string,\r\n BillingAccountName: string,\r\n BillingAccountType: string,\r\n BillingCurrency: string,\r\n BillingPeriodEnd: datetime,\r\n BillingPeriodStart: datetime,\r\n CapacityReservationId: string,\r\n CapacityReservationStatus: string,\r\n ChargeCategory: string,\r\n ChargeClass: string,\r\n ChargeDescription: string,\r\n ChargeFrequency: string,\r\n ChargePeriodEnd: datetime,\r\n ChargePeriodStart: datetime,\r\n CommitmentDiscountCategory: string,\r\n CommitmentDiscountId: string,\r\n CommitmentDiscountName: string,\r\n CommitmentDiscountQuantity: real,\r\n CommitmentDiscountStatus: string,\r\n CommitmentDiscountType: string,\r\n CommitmentDiscountUnit: string,\r\n ConsumedQuantity: real,\r\n ConsumedUnit: string,\r\n ContractedCost: real,\r\n ContractedUnitPrice: real,\r\n EffectiveCost: real,\r\n InvoiceId: string,\r\n InvoiceIssuerName: string,\r\n ListCost: real,\r\n ListUnitPrice: real,\r\n PricingCategory: string,\r\n PricingCurrency: string,\r\n PricingQuantity: real,\r\n PricingUnit: string,\r\n ProviderName: string,\r\n PublisherName: string,\r\n RegionId: string,\r\n RegionName: string,\r\n ResourceId: string,\r\n ResourceName: string,\r\n ResourceType: string,\r\n ServiceCategory: string,\r\n ServiceName: string,\r\n ServiceSubcategory: string,\r\n SkuId: string,\r\n SkuMeter: string,\r\n SkuPriceDetails: dynamic,\r\n SkuPriceId: string,\r\n SubAccountId: string,\r\n SubAccountName: string,\r\n SubAccountType: string,\r\n Tags: dynamic,\r\n x_AccountId: string, // Azure 1.0-preview(v1)+\r\n x_AccountName: string, // Azure 1.0-preview(v1)+\r\n x_AccountOwnerId: string, // Azure 1.0-preview(v1)+\r\n x_AmortizationClass: string, // Azure 1.2-preview+\r\n x_BilledCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_BilledUnitPrice: real, // Azure 1.0-preview(v1)+\r\n x_BillingAccountAgreement: string, // Hubs add-on\r\n x_BillingAccountId: string, // Azure 1.0-preview(v1)+\r\n x_BillingAccountName: string, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRate: real, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRateDate: datetime, // Azure 1.0-preview(v1)+\r\n x_BillingItemCode: string, // Alibaba 1.0\r\n x_BillingItemName: string, // Alibaba 1.0\r\n x_BillingProfileId: string, // Azure 1.0-preview(v1)+\r\n x_BillingProfileName: string, // Azure 1.0-preview(v1)+\r\n x_ChargeId: string, // Azure 1.0-preview(v1) only\r\n x_CommitmentDiscountNormalizedRatio: real, // Azure 1.2-preview+\r\n x_CommitmentDiscountPercent: real, // Hubs add-on\r\n x_CommitmentDiscountSavings: real, // Hubs add-on\r\n x_CommitmentDiscountSpendEligibility: string, // Hubs add-on\r\n x_CommitmentDiscountUsageEligibility: string, // Hubs add-on\r\n x_CommitmentDiscountUtilizationAmount: real, // Hubs add-on\r\n x_CommitmentDiscountUtilizationPotential: real, // Hubs add-on\r\n x_CommodityCode: string, // Alibaba 1.0\r\n x_CommodityName: string, // Alibaba 1.0\r\n x_ComponentName: string, // Tencent 1.0\r\n x_ComponentType: string, // Tencent 1.0\r\n x_ConsumedCoreHours: real, // Hubs add-on\r\n x_ContractedCostInUsd: real, // Azure 1.0+\r\n x_CostAllocationRuleName: string, // Azure 1.0-preview(v1)+\r\n x_CostCategories: dynamic, // AWS 1.0 (JSON)\r\n x_CostCenter: string, // Azure 1.0-preview(v1)+\r\n x_CostType: string, // GCP Jan 2024\r\n x_Credits: dynamic, // GCP Jan 2024\r\n x_CurrencyConversionRate: real, // GCP Jun 2024\r\n x_CustomerId: string, // Azure 1.0-preview(v1)+\r\n x_CustomerName: string, // Azure 1.0-preview(v1)+\r\n x_Discount: dynamic, // AWS 1.0 (JSON)\r\n x_EffectiveCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_EffectiveUnitPrice: real, // Azure 1.0-preview(v1)+\r\n x_ExportTime: datetime, // GCP Jan 2024 / Tencent 1.0\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_InstanceID: string, // Alibaba 1.0\r\n x_InvoiceIssuerId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionName: string, // Azure 1.0-preview(v1)+\r\n x_ListCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_Location: string, // GCP Jan 2024\r\n x_NegotiatedDiscountPercent:real, // Hubs add-on\r\n x_NegotiatedDiscountSavings:real, // Hubs add-on\r\n x_Operation: string, // AWS 1.0\r\n x_OwnerAccountID: string, // Tencent 1.0\r\n x_PartnerCreditApplied: string, // Azure 1.0-preview(v1)+\r\n x_PartnerCreditRate: string, // Azure 1.0-preview(v1)+\r\n x_PricingBlockSize: real, // Azure 1.0-preview(v1)+\r\n x_PricingSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_PricingUnitDescription: string, // Azure 1.0-preview(v1)+\r\n x_Project: string, // GCP Jan 2024\r\n x_PublisherCategory: string, // Azure 1.0-preview(v1)+\r\n x_PublisherId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceGroupName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceType: string, // Azure 1.0-preview(v1)+\r\n x_ServiceCode: string, // AWS 1.0\r\n x_ServiceId: string, // GCP Jan 2024\r\n x_ServiceModel: string, // Azure 1.2-preview+\r\n x_ServicePeriodEnd: datetime, // Azure 1.0-preview(v1)+\r\n x_ServicePeriodStart: datetime, // Azure 1.0-preview(v1)+\r\n x_SkuCoreCount: int, // Hubs add-on\r\n x_SkuDescription: string, // Azure 1.0-preview(v1)+\r\n x_SkuDetails: dynamic, // Azure 1.0-preview(v1)+\r\n x_SkuInstanceType: string, // Hubs add-on\r\n x_SkuIsCreditEligible: bool, // Azure 1.0-preview(v1)+\r\n x_SkuLicenseQuantity: int, // Hubs add-on\r\n x_SkuLicenseStatus: string, // Hubs add-on\r\n x_SkuLicenseType: string, // Hubs add-on\r\n x_SkuLicenseUnit: string, // Hubs add-on\r\n x_SkuMeterCategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterId: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuOfferId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOperatingSystem: string, // Hubs add-on\r\n x_SkuOrderId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOrderName: string, // Azure 1.0-preview(v1)+\r\n x_SkuPartNumber: string, // Azure 1.0-preview(v1)+\r\n x_SkuPlanName: string, // Azure 1.2-preview+\r\n x_SkuRegion: string, // Azure 1.0-preview(v1)+\r\n x_SkuServiceFamily: string, // Azure 1.0-preview(v1)+\r\n x_SkuTerm: int, // Azure 1.0-preview(v1)+\r\n x_SkuTier: string, // Azure 1.0-preview(v1)+\r\n x_SourceChanges: string, // Hubs add-on\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceValues: dynamic, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_SubproductName: string, // Tencent 1.0\r\n x_TotalDiscountPercent: real, // Hubs add-on\r\n x_TotalSavings: real, // Hubs add-on\r\n x_UsageType: string // AWS 1.0\r\n)\r\n\r\n// Update policy for Costs_raw -> Costs_final_v1_2 table\r\n.alter table Costs_final_v1_2 policy update\r\n```\r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"Costs_raw\",\r\n \"Query\": \"Costs_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Actual costs |===================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// ActualCosts_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='ActualCost exports transformed to FOCUS 1.2.', folder='Costs')\r\nActualCosts_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n ActualCosts_raw\r\n | where ChargeType in ('Purchase', 'Refund') and isnotempty(ReservationId)\r\n //\r\n //\r\n // !!! The rest of the query is reused for both the ActualCosts and AmortizedCosts queries -- Copy all changes in both transform functions !!!\r\n //\r\n //\r\n | extend x_AmortizationClass = case(\r\n ChargeType in ('Purchase', 'Refund') and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', 'Amortized Charge',\r\n ''\r\n )\r\n | extend tmp_ResourceInfo = parse_resourceid(ResourceId)\r\n // TODO: PricingCategory needs to include savings plan usage and spot usage\r\n | extend PricingCategory = case(\r\n x_AmortizationClass == 'Amortized Charge', 'Committed',\r\n ChargeType in ('Usage', 'Purchase'), 'Standard',\r\n ''\r\n )\r\n | extend x_ResourceType = tostring(tmp_ResourceInfo.x_ResourceType)\r\n | project-rename\r\n PricingQuantity = Quantity,\r\n x_PricingUnitDescription = UnitOfMeasure\r\n | join kind=leftouter (PricingUnits) on x_PricingUnitDescription\r\n | join kind=leftouter (Regions) on ResourceLocation\r\n | join kind=leftouter (ResourceTypes | project x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n // TODO: Add the following in 1.2: PublisherName, x_PublisherCategory, x_Environment\r\n | join kind=leftouter (Services | where isnotempty(x_ResourceType) | project x_ResourceType, ServiceName, ServiceCategory, ServiceSubcategory, x_ServiceModel) on x_ResourceType\r\n | join kind=leftouter (Services | where isnotempty(x_ConsumedService) | summarize take_any(ServiceName), take_any(ServiceCategory), take_any(ServiceSubcategory), take_any(x_ServiceModel) by ConsumedService = x_ConsumedService) on ConsumedService\r\n | extend CommitmentDiscountCategory = iff(isnotempty(ReservationId), 'Usage', '') // TODO: CommitmentDiscountCategory needs to handle savings plans\r\n | project\r\n AvailabilityZone = AvailabilityZone,\r\n BilledCost = iff(x_AmortizationClass == 'Amortized Charge', real(0), Cost),\r\n BillingAccountId = strcat('/providers/microsoft.billing/billingaccounts/', BillingAccountId, iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), '', strcat('/billingprofiles/', BillingProfileId))),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName),\r\n BillingAccountType = iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), 'Billing Profile', 'Billing Account'),\r\n BillingCurrency,\r\n BillingPeriodEnd = startofmonth(BillingPeriodEndDate, 1),\r\n BillingPeriodStart = startofmonth(BillingPeriodStartDate),\r\n CapacityReservationId = '',\r\n CapacityReservationStatus = '',\r\n ChargeCategory = case(\r\n ChargeType in ('Usage', 'Purchase', 'Credit', 'Tax'), ChargeType,\r\n ChargeType in ('UnusedReservation', 'UnusedSavingsPlan'), 'Usage',\r\n ChargeType == 'Refund', 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = iff(ChargeType == 'Refund', 'Correction', ''),\r\n ChargeDescription = Product,\r\n ChargeFrequency = case(\r\n Frequency == 'UsageBased', 'Usage-Based',\r\n Frequency == 'OneTime', 'One-Time',\r\n Frequency // \"Recurring\" and any fallback\r\n ),\r\n ChargePeriodEnd = Date + 1d,\r\n ChargePeriodStart = Date,\r\n ChargeSubcategory = '',\r\n // TODO: CommitmentDiscount* columns need to handle savings plans\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId = iff(isnotempty(ReservationId), strcat('/providers/microsoft.capacity/reservationorders/', ProductOrderId, '/reservations/', ReservationId), ''),\r\n CommitmentDiscountName = ReservationName,\r\n CommitmentDiscountQuantity = real(null),\r\n CommitmentDiscountStatus = case(\r\n isempty(ReservationId), '',\r\n isnotempty(ReservationId) and ChargeType == 'Usage', 'Used',\r\n ChargeType startswith 'Unused', 'Unused',\r\n 'Unused'\r\n ),\r\n CommitmentDiscountType = iff(isnotempty(ReservationId), 'Reservation', ''),\r\n CommitmentDiscountUnit = '',\r\n ConsumedQuantity = PricingQuantity * x_PricingBlockSize,\r\n ConsumedUnit = PricingUnit,\r\n ContractedCost = UnitPrice * PricingQuantity,\r\n ContractedUnitPrice = UnitPrice,\r\n EffectiveCost = iff(x_AmortizationClass == 'Principal', real(0), Cost),\r\n InvoiceId = '',\r\n InvoiceIssuerName = 'Microsoft',\r\n ListCost = real(null),\r\n ListUnitPrice = real(null),\r\n PricingCategory,\r\n PricingCurrency = iff(BillingAccountId == BillingProfileId, BillingCurrency, ''),\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName = 'Microsoft',\r\n PublisherName = iff(PublisherType == 'Marketplace', PublisherName, 'Microsoft'),\r\n Region = '',\r\n RegionId,\r\n RegionName,\r\n ResourceId = tostring(tmp_ResourceInfo.ResourceId),\r\n ResourceName = tostring(tmp_ResourceInfo.ResourceName),\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory,\r\n SkuId = '',\r\n SkuMeter = MeterName,\r\n SkuPriceDetails = '',\r\n SkuPriceId = '',\r\n SubAccountId = strcat('/subscriptions/', SubscriptionId),\r\n SubAccountName = iff(isempty(SubscriptionId), '', SubscriptionName),\r\n SubAccountType = iff(isempty(SubscriptionId), '', 'Subscription'),\r\n Tags = iff(Tags startswith '{', Tags, strcat('{', Tags, '}')),\r\n UsageAmount = real(null),\r\n UsageQuantity = real(null),\r\n UsageUnit = '',\r\n x_AccountId = '',\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerId,\r\n x_AmortizationClass = '',\r\n x_BilledCostInUsd = real(null),\r\n x_BilledUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), UnitPrice),\r\n x_BillingAccountId = BillingAccountId,\r\n x_BillingAccountName = BillingAccountName,\r\n x_BillingExchangeRate = real(null),\r\n x_BillingExchangeRateDate = datetime(null),\r\n x_BillingItemCode = '',\r\n x_BillingItemName = '',\r\n x_BillingProfileId = BillingProfileId,\r\n x_BillingProfileName = BillingProfileName,\r\n x_ChargeId = '',\r\n x_CommodityCode = '',\r\n x_CommodityName = '',\r\n x_ComponentType = '',\r\n x_ComponentName = '',\r\n x_ContractedCostInUsd = real(null),\r\n x_Cost = real(null),\r\n x_CostAllocationRuleName = '',\r\n x_CostCategories = '',\r\n x_CostCenter = CostCenter,\r\n x_CostType = '',\r\n x_Credits = '',\r\n x_CurrencyConversionRate = real(null),\r\n x_CustomerId = '',\r\n x_CustomerName = '',\r\n x_Discount = '',\r\n x_EffectiveCostInUsd = real(null),\r\n x_EffectiveUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), EffectivePrice),\r\n x_ExportTime = datetime(null),\r\n x_InstanceID = '',\r\n x_InvoiceId = '',\r\n x_InvoiceIssuerId = '',\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = InvoiceSection,\r\n x_ListCostInUsd = real(null),\r\n x_Location = '',\r\n x_OnDemandCost = real(null),\r\n x_OnDemandCostInUsd = real(null),\r\n x_OnDemandUnitPrice = real(null),\r\n x_Operation = '',\r\n x_OwnerAccountID = '',\r\n x_PartnerCreditApplied = '',\r\n x_PartnerCreditRate = '',\r\n x_PricingBlockSize,\r\n x_PricingCurrency = '',\r\n x_PricingSubcategory = case(\r\n // TODO: Add x_SkuTier when supported by C360 -- PricingCategory == 'Standard' and isnotempty(x_SkuTier), 'Tiered',\r\n PricingCategory == 'Standard', 'Standard',\r\n PricingCategory == 'Committed', strcat('Committed ', CommitmentDiscountCategory),\r\n PricingCategory == 'Dynamic', 'Spot',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_Project = '',\r\n x_PublisherCategory = iff(PublisherType == 'Marketplace', 'Vendor', 'Cloud Provider'),\r\n x_PublisherId = '',\r\n x_ResellerId = '',\r\n x_ResellerName = '',\r\n x_ResourceGroupName = ResourceGroup,\r\n x_ResourceType,\r\n x_ServiceCode = '',\r\n x_ServiceId = '',\r\n x_ServiceModel,\r\n x_ServicePeriodEnd = datetime(null),\r\n x_ServicePeriodStart = datetime(null),\r\n x_SkuDescription = Product,\r\n x_SkuDetails = iff(AdditionalInfo startswith '{', AdditionalInfo, strcat('{', AdditionalInfo, '}')),\r\n x_SkuIsCreditEligible = IsAzureCreditEligible,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterId = MeterId,\r\n x_SkuMeterName = MeterName,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuOfferId = OfferId,\r\n x_SkuOrderId = ProductOrderId,\r\n x_SkuOrderName = ProductOrderName,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPlanName = '',\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTerm = toint(Term),\r\n x_SkuTier = '',\r\n x_SourceName = 'C360',\r\n x_SourceProvider = 'Microsoft',\r\n x_SourceType = 'ActualCost',\r\n x_SourceVersion = 'C360-2025-04',\r\n x_SubproductName = '',\r\n x_UsageType = ''\r\n}\r\n\r\n// Update policy for ActualCosts_raw -> Costs_raw table\r\n.alter table Costs_raw policy update\r\n``` \r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"ActualCosts_raw\",\r\n \"Query\": \"ActualCosts_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Amortized costs |================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// AmortizedCosts_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='ActualCost exports transformed to FOCUS 1.2.', folder='Costs')\r\nAmortizedCosts_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n AmortizedCosts_raw\r\n //\r\n //\r\n // !!! The rest of the query is reused for both the ActualCosts and AmortizedCosts queries -- Copy all changes in both transform functions !!!\r\n //\r\n //\r\n | extend x_AmortizationClass = case(\r\n ChargeType in ('Purchase', 'Refund') and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', 'Amortized Charge',\r\n ''\r\n )\r\n | extend tmp_ResourceInfo = parse_resourceid(ResourceId)\r\n // TODO: PricingCategory needs to include savings plan usage and spot usage\r\n | extend PricingCategory = case(\r\n x_AmortizationClass == 'Amortized Charge', 'Committed',\r\n ChargeType in ('Usage', 'Purchase'), 'Standard',\r\n ''\r\n )\r\n | extend x_ResourceType = tostring(tmp_ResourceInfo.x_ResourceType)\r\n | project-rename\r\n PricingQuantity = Quantity,\r\n x_PricingUnitDescription = UnitOfMeasure\r\n | join kind=leftouter (PricingUnits) on x_PricingUnitDescription\r\n | join kind=leftouter (Regions) on ResourceLocation\r\n | join kind=leftouter (ResourceTypes | project x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n // TODO: Add the following in 1.2: PublisherName, x_PublisherCategory, x_Environment\r\n | join kind=leftouter (Services | where isnotempty(x_ResourceType) | project x_ResourceType, ServiceName, ServiceCategory, ServiceSubcategory, x_ServiceModel) on x_ResourceType\r\n | join kind=leftouter (Services | where isnotempty(x_ConsumedService) | summarize take_any(ServiceName), take_any(ServiceCategory), take_any(ServiceSubcategory), take_any(x_ServiceModel) by ConsumedService = x_ConsumedService) on ConsumedService\r\n | extend CommitmentDiscountCategory = iff(isnotempty(ReservationId), 'Usage', '') // TODO: CommitmentDiscountCategory needs to handle savings plans\r\n | project\r\n AvailabilityZone = AvailabilityZone,\r\n BilledCost = iff(x_AmortizationClass == 'Amortized Charge', real(0), Cost),\r\n BillingAccountId = strcat('/providers/microsoft.billing/billingaccounts/', BillingAccountId, iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), '', strcat('/billingprofiles/', BillingProfileId))),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName),\r\n BillingAccountType = iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), 'Billing Profile', 'Billing Account'),\r\n BillingCurrency,\r\n BillingPeriodEnd = startofmonth(BillingPeriodEndDate, 1),\r\n BillingPeriodStart = startofmonth(BillingPeriodStartDate),\r\n CapacityReservationId = '',\r\n CapacityReservationStatus = '',\r\n ChargeCategory = case(\r\n ChargeType in ('Usage', 'Purchase', 'Credit', 'Tax'), ChargeType,\r\n ChargeType in ('UnusedReservation', 'UnusedSavingsPlan'), 'Usage',\r\n ChargeType == 'Refund', 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = iff(ChargeType == 'Refund', 'Correction', ''),\r\n ChargeDescription = Product,\r\n ChargeFrequency = case(\r\n Frequency == 'UsageBased', 'Usage-Based',\r\n Frequency == 'OneTime', 'One-Time',\r\n Frequency // \"Recurring\" and any fallback\r\n ),\r\n ChargePeriodEnd = Date + 1d,\r\n ChargePeriodStart = Date,\r\n ChargeSubcategory = '',\r\n // TODO: CommitmentDiscount* columns need to handle savings plans\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId = iff(isnotempty(ReservationId), strcat('/providers/microsoft.capacity/reservationorders/', ProductOrderId, '/reservations/', ReservationId), ''),\r\n CommitmentDiscountName = ReservationName,\r\n CommitmentDiscountQuantity = real(null),\r\n CommitmentDiscountStatus = case(\r\n isempty(ReservationId), '',\r\n isnotempty(ReservationId) and ChargeType == 'Usage', 'Used',\r\n ChargeType startswith 'Unused', 'Unused',\r\n 'Unused'\r\n ),\r\n CommitmentDiscountType = iff(isnotempty(ReservationId), 'Reservation', ''),\r\n CommitmentDiscountUnit = '',\r\n ConsumedQuantity = PricingQuantity * x_PricingBlockSize,\r\n ConsumedUnit = PricingUnit,\r\n ContractedCost = UnitPrice * PricingQuantity,\r\n ContractedUnitPrice = UnitPrice,\r\n EffectiveCost = iff(x_AmortizationClass == 'Principal', real(0), Cost),\r\n InvoiceId = '',\r\n InvoiceIssuerName = 'Microsoft',\r\n ListCost = real(null),\r\n ListUnitPrice = real(null),\r\n PricingCategory,\r\n PricingCurrency = iff(BillingAccountId == BillingProfileId, BillingCurrency, ''),\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName = 'Microsoft',\r\n PublisherName = iff(PublisherType == 'Marketplace', PublisherName, 'Microsoft'),\r\n Region = '',\r\n RegionId,\r\n RegionName,\r\n ResourceId = tostring(tmp_ResourceInfo.ResourceId),\r\n ResourceName = tostring(tmp_ResourceInfo.ResourceName),\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory,\r\n SkuId = '',\r\n SkuMeter = MeterName,\r\n SkuPriceDetails = '',\r\n SkuPriceId = '',\r\n SubAccountId = strcat('/subscriptions/', SubscriptionId),\r\n SubAccountName = iff(isempty(SubscriptionId), '', SubscriptionName),\r\n SubAccountType = iff(isempty(SubscriptionId), '', 'Subscription'),\r\n Tags = iff(Tags startswith '{', Tags, strcat('{', Tags, '}')),\r\n UsageAmount = real(null),\r\n UsageQuantity = real(null),\r\n UsageUnit = '',\r\n x_AccountId = '',\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerId,\r\n x_AmortizationClass = '',\r\n x_BilledCostInUsd = real(null),\r\n x_BilledUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), UnitPrice),\r\n x_BillingAccountId = BillingAccountId,\r\n x_BillingAccountName = BillingAccountName,\r\n x_BillingExchangeRate = real(null),\r\n x_BillingExchangeRateDate = datetime(null),\r\n x_BillingItemCode = '',\r\n x_BillingItemName = '',\r\n x_BillingProfileId = BillingProfileId,\r\n x_BillingProfileName = BillingProfileName,\r\n x_ChargeId = '',\r\n x_CommodityCode = '',\r\n x_CommodityName = '',\r\n x_ComponentType = '',\r\n x_ComponentName = '',\r\n x_ContractedCostInUsd = real(null),\r\n x_Cost = real(null),\r\n x_CostAllocationRuleName = '',\r\n x_CostCategories = '',\r\n x_CostCenter = CostCenter,\r\n x_CostType = '',\r\n x_Credits = '',\r\n x_CurrencyConversionRate = real(null),\r\n x_CustomerId = '',\r\n x_CustomerName = '',\r\n x_Discount = '',\r\n x_EffectiveCostInUsd = real(null),\r\n x_EffectiveUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), EffectivePrice),\r\n x_ExportTime = datetime(null),\r\n x_InstanceID = '',\r\n x_InvoiceId = '',\r\n x_InvoiceIssuerId = '',\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = InvoiceSection,\r\n x_ListCostInUsd = real(null),\r\n x_Location = '',\r\n x_OnDemandCost = real(null),\r\n x_OnDemandCostInUsd = real(null),\r\n x_OnDemandUnitPrice = real(null),\r\n x_Operation = '',\r\n x_OwnerAccountID = '',\r\n x_PartnerCreditApplied = '',\r\n x_PartnerCreditRate = '',\r\n x_PricingBlockSize,\r\n x_PricingCurrency = '',\r\n x_PricingSubcategory = case(\r\n // TODO: Add x_SkuTier when supported by C360 -- PricingCategory == 'Standard' and isnotempty(x_SkuTier), 'Tiered',\r\n PricingCategory == 'Standard', 'Standard',\r\n PricingCategory == 'Committed', strcat('Committed ', CommitmentDiscountCategory),\r\n PricingCategory == 'Dynamic', 'Spot',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_Project = '',\r\n x_PublisherCategory = iff(PublisherType == 'Marketplace', 'Vendor', 'Cloud Provider'),\r\n x_PublisherId = '',\r\n x_ResellerId = '',\r\n x_ResellerName = '',\r\n x_ResourceGroupName = ResourceGroup,\r\n x_ResourceType,\r\n x_ServiceCode = '',\r\n x_ServiceId = '',\r\n x_ServiceModel,\r\n x_ServicePeriodEnd = datetime(null),\r\n x_ServicePeriodStart = datetime(null),\r\n x_SkuDescription = Product,\r\n x_SkuDetails = iff(AdditionalInfo startswith '{', AdditionalInfo, strcat('{', AdditionalInfo, '}')),\r\n x_SkuIsCreditEligible = IsAzureCreditEligible,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterId = MeterId,\r\n x_SkuMeterName = MeterName,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuOfferId = OfferId,\r\n x_SkuOrderId = ProductOrderId,\r\n x_SkuOrderName = ProductOrderName,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPlanName = '',\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTerm = toint(Term),\r\n x_SkuTier = '',\r\n x_SourceName = 'C360',\r\n x_SourceProvider = 'Microsoft',\r\n x_SourceType = 'ActualCost',\r\n x_SourceVersion = 'C360-2025-04',\r\n x_SubproductName = '',\r\n x_UsageType = ''\r\n}\r\n\r\n// Update policy for AmortizedCosts_raw -> Costs_raw table\r\n.alter table Costs_raw policy update\r\n``` \r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"AmortizedCosts_raw\",\r\n \"Query\": \"AmortizedCosts_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| CommitmentDiscountUsage |========================================================================================\r\n// Supported versions:\r\n// - MS EA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-ea\r\n// - MS MCA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-mca\r\n//======================================================================================================================\r\n\r\n// CommitmentDiscountUsage_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='All commitment discount usage transformed to FOCUS 1.2. This includes reservationdeatils_raw.', folder='Commitment discounts')\r\nCommitmentDiscountUsage_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n CommitmentDiscountUsage_raw\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Handle resource columns\r\n | extend ResourceId = tolower(InstanceId)\r\n | extend tmp_ResourceDetails = parse_resourceid(ResourceId)\r\n | extend ResourceName = tostring(tmp_ResourceDetails.ResourceName)\r\n | extend SubAccountId = tostring(tmp_ResourceDetails.SubAccountId)\r\n | extend x_ResourceGroupName = tostring(tmp_ResourceDetails.x_ResourceGroupName)\r\n | extend x_ResourceType = tostring(tmp_ResourceDetails.x_ResourceType)\r\n | lookup kind=leftouter (ResourceTypes | distinct x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n | lookup kind=leftouter (Services | distinct x_ResourceType, ServiceName, ServiceCategory, ServiceSubcategory, x_ServiceModel) on x_ResourceType\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n ChargePeriodEnd = UsageDate + 1d,\r\n ChargePeriodStart = UsageDate,\r\n CommitmentDiscountCategory = 'Usage',\r\n CommitmentDiscountId = tolower(strcat('/providers/microsoft.capacity/reservationorders/', ReservationOrderId, '/reservations/', ReservationId)),\r\n CommitmentDiscountQuantity = UsedHours * InstanceFlexibilityRatio,\r\n CommitmentDiscountType = 'Reservation',\r\n CommitmentDiscountUnit = case(\r\n InstanceFlexibilityRatio == 1, 'Hours',\r\n InstanceFlexibilityRatio != 1, 'Normalized Hours',\r\n ''\r\n ),\r\n ConsumedQuantity = UsedHours,\r\n ProviderName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory,\r\n SubAccountId,\r\n x_CommitmentDiscountCommittedCount = TotalReservedQuantity,\r\n x_CommitmentDiscountCommittedAmount = ReservedHours,\r\n // TODO: Is this needed? -- x_CommitmentDiscountKind = Kind,\r\n x_CommitmentDiscountNormalizedGroup = iff(InstanceFlexibilityGroup == 'NA', '', InstanceFlexibilityGroup),\r\n x_CommitmentDiscountNormalizedRatio = InstanceFlexibilityRatio,\r\n x_IngestionTime = ingestion_time(),\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n // x_RowId = hash_sha256(strcat(\r\n // // DO NOT CHANGE COLUMNS OR COLUMN ORDER\r\n // CommitmentDiscountId,\r\n // ResourceId,\r\n // ChargePeriodStart\r\n // )),\r\n x_ServiceModel,\r\n x_SkuOrderId = ReservationOrderId,\r\n x_SkuSize = iff(SkuName == 'NA', '', SkuName),\r\n x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName)),\r\n x_SourceProvider = coalesce(x_SourceProvider, ProviderName),\r\n x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationDetails', '')),\r\n x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2024-03-01', ''))\r\n}\r\n\r\n// CommitmentDiscountUsage_final_v1_2 table\r\n.create-merge table CommitmentDiscountUsage_final_v1_2 (\r\n ChargePeriodEnd: datetime, // Hubs add-on\r\n ChargePeriodStart: datetime, // MS 2023-03-01\r\n CommitmentDiscountCategory: string, // Hubs add-on\r\n CommitmentDiscountId: string, // MS 2023-03-01\r\n CommitmentDiscountQuantity: real, // MS 2023-03-01\r\n CommitmentDiscountType: string, // Hubs add-on\r\n CommitmentDiscountUnit: string, // Hubs add-on\r\n ConsumedQuantity: real, // MS 2023-03-01\r\n ProviderName: string, // Hubs add-on\r\n ResourceId: string, // MS 2023-03-01\r\n ResourceName: string, // Hubs add-on\r\n ResourceType: string, // Hubs add-on\r\n ServiceCategory: string, // Hubs add-on\r\n ServiceName: string, // Hubs add-on\r\n ServiceSubcategory: string, // Hubs add-on\r\n SubAccountId: string, // Hubs add-on\r\n x_CommitmentDiscountCommittedCount: real, // MS 2023-03-01\r\n x_CommitmentDiscountCommittedAmount: real, // MS 2023-03-01\r\n x_CommitmentDiscountNormalizedGroup: string, // MS 2023-03-01\r\n x_CommitmentDiscountNormalizedRatio: real, // MS 2023-03-01\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_ResourceGroupName: string, // Hubs add-on\r\n x_ResourceType: string, // Hubs add-on\r\n x_ServiceModel: string, // Hubs add-on\r\n x_SkuOrderId: string, // MS 2023-03-01\r\n x_SkuSize: string, // MS 2023-03-01\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string // Hubs add-on\r\n)\r\n\r\n// Update policy for CommitmentDiscountUsage_raw -> CommitmentDiscountUsage_final_v1_2 table\r\n.alter table CommitmentDiscountUsage_final_v1_2 policy update\r\n```\r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"CommitmentDiscountUsage_raw\",\r\n \"Query\": \"CommitmentDiscountUsage_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Recommendations |================================================================================================\r\n// Supported datasets/versions:\r\n// - MS CM EA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-ea\r\n// - MS CM MCA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-mca\r\n//======================================================================================================================\r\n\r\n// Recommendations_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='All recommendations transformed to FOCUS 1.2.', folder='Recommendations')\r\nRecommendations_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', int(null),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n Recommendations_raw\r\n | extend x_IngestionTime = ingestion_time()\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Set source columns\r\n | extend x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationRecommendations', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2023-05-01', ''))\r\n //\r\n // Convert JSON cost columns to real\r\n | extend CostWithNoReservedInstances = case(isnotempty(CostWithNoReservedInstances), CostWithNoReservedInstances, isnotempty(CostWithNoReservedInstancesJson), toreal(extract(@'\"value\":([0-9\\.]+)', 1, CostWithNoReservedInstancesJson)), CostWithNoReservedInstances)\r\n | extend NetSavings = case(isnotempty(NetSavings), NetSavings, isnotempty(NetSavingsJson), toreal(extract(@'\"value\":([0-9\\.]+)', 1, NetSavingsJson)), NetSavings)\r\n | extend TotalCostWithReservedInstances = case(isnotempty(TotalCostWithReservedInstances), TotalCostWithReservedInstances, isnotempty(TotalCostWithReservedInstancesJson), toreal(extract(@'\"value\":([0-9\\.]+)', 1, TotalCostWithReservedInstancesJson)), TotalCostWithReservedInstances)\r\n //\r\n // Build recommendation details\r\n | lookup kind=leftouter (Regions | summarize RegionName = make_set(RegionName)[0] by Location = RegionId) on Location\r\n | extend x_RecommendationDetails = case(\r\n // Use incoming x_RecommendationDetails first\r\n isnotempty(x_RecommendationDetails), x_RecommendationDetails,\r\n // Create one for reservation recommendations if needed\r\n x_SourceType == 'ReservationRecommendations', bag_pack(\r\n 'CommitmentDiscountNormalizedGroup', InstanceFlexibilityGroup,\r\n 'CommitmentDiscountNormalizedRatio', InstanceFlexibilityRatio,\r\n 'CommitmentDiscountNormalizedSize', NormalizedSize,\r\n 'CommitmentDiscountResourceType', ResourceType,\r\n 'CommitmentDiscountScope', Scope,\r\n 'LookbackPeriodDuration', case(\r\n LookBackPeriod matches regex @'^Last([0-9]+)Days$', replace_regex(LookBackPeriod, @'^Last([0-9]+)Days$', @'P\\1D'),\r\n LookBackPeriod matches regex @'^[0-9]+$', strcat('P', LookBackPeriod, 'D'),\r\n ''\r\n ),\r\n 'LookbackPeriodStart', FirstUsageDate,\r\n 'RecommendedQuantity', RecommendedQuantity,\r\n 'RecommendedQuantityNormalized', RecommendedQuantityNormalized,\r\n 'RegionId', Location,\r\n 'RegionName', RegionName,\r\n 'SkuMeterId', MeterId,\r\n 'SkuPriceDetails', SkuProperties,\r\n 'SkuSize', coalesce(SKU, SkuName),\r\n 'SkuTerm', isoMonths(Term)\r\n ),\r\n dynamic({})\r\n )\r\n //\r\n // Prefer specified date, then fall back to generating a date based on reservation recommendation lookback period, then validate to ensure it's not in the future\r\n | extend x_RecommendationDate = coalesce(x_RecommendationDate, FirstUsageDate + (toint(extract(@'^P([0-9]+)D$', 1, tostring(x_RecommendationDetails.LookbackPeriodDuration))) * 1d))\r\n | extend x_RecommendationDate = iff(x_RecommendationDate > now(), startofday(now()), x_RecommendationDate)\r\n //\r\n | project\r\n ProviderName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n SubAccountId = coalesce(SubAccountId, iff(isnotempty(SubscriptionId), strcat('/subscriptions/', SubscriptionId), '')),\r\n SubAccountName,\r\n x_EffectiveCostAfter = coalesce(x_EffectiveCostAfter, TotalCostWithReservedInstances),\r\n x_EffectiveCostBefore = coalesce(x_EffectiveCostBefore, CostWithNoReservedInstances),\r\n x_EffectiveCostSavings = coalesce(x_EffectiveCostSavings, NetSavings),\r\n x_IngestionTime,\r\n x_RecommendationCategory, // TODO: Set for reservation recommendations\r\n x_RecommendationDate,\r\n x_RecommendationDescription,\r\n x_RecommendationDetails,\r\n x_RecommendationId, // TODO: Set for reservation recommendations\r\n x_ResourceGroupName,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n// Recommendations_final_v1_2 table\r\n.create-merge table Recommendations_final_v1_2 (\r\n ProviderName: string,\r\n ResourceId: string,\r\n ResourceName: string,\r\n ResourceType: string,\r\n SubAccountId: string,\r\n SubAccountName: string,\r\n x_EffectiveCostAfter: real,\r\n x_EffectiveCostBefore: real,\r\n x_EffectiveCostSavings: real,\r\n x_IngestionTime: datetime,\r\n x_RecommendationCategory: string,\r\n x_RecommendationDate: datetime,\r\n x_RecommendationDescription: string,\r\n x_RecommendationDetails: dynamic,\r\n x_RecommendationId: string,\r\n x_ResourceGroupName: string,\r\n x_SourceName: string,\r\n x_SourceProvider: string,\r\n x_SourceType: string,\r\n x_SourceVersion: string\r\n)\r\n\r\n// Update policy for Recommendations_raw -> Recommendations_final_v1_2 table\r\n.alter table Recommendations_final_v1_2 policy update\r\n```\r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"Recommendations_raw\",\r\n \"Query\": \"Recommendations_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Transactions |===================================================================================================\r\n// Supported versions:\r\n// - MS CM EA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-ea\r\n// - MS CM MCA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-mca\r\n//======================================================================================================================\r\n\r\n// Transactions_transform_v1_2 function\r\n.create-or-alter function\r\nwith (docstring='All transactions transformed to FOCUS 1.2.', folder='Transactions')\r\nTransactions_transform_v1_2()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', int(null),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n Transactions_raw\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Set source columns\r\n | extend x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationTransactions', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2023-05-01', ''))\r\n //\r\n // Handle BillingPeriodStart/End\r\n | extend BillingMonth = tostring(BillingMonth)\r\n | extend BillingPeriodStart = iff(isempty(BillingMonth), datetime(null), todatetime(strcat(substring(BillingMonth, 0, 4), \"-\", substring(BillingMonth, 4, 2), \"-\", substring(BillingMonth, 6, 2))))\r\n | extend BillingPeriodEnd = iff(isempty(BillingMonth), datetime(null), startofmonth(endofmonth(BillingPeriodStart) + 1d))\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n BilledCost = Amount,\r\n BillingAccountId = case(\r\n BillingProfileId startswith '/', BillingProfileId,\r\n isnotempty(CurrentEnrollmentId), strcat('/providers/Microsoft.Billing/billingAccounts/', CurrentEnrollmentId),\r\n isnotempty(BillingProfileId), strcat('/providers/Microsoft.Billing/billingProfiles/', BillingProfileId),\r\n ''\r\n ),\r\n BillingAccountName = coalesce(BillingProfileName, CurrentEnrollmentId),\r\n BillingCurrency = Currency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n ChargeCategory = case(\r\n EventType in ('Cancel', 'Purchase', 'Refund'), 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = case(\r\n EventType == 'Cancel', 'Cancel', // FOCUS does not handle this scenario\r\n EventType == 'Refund', 'Correction',\r\n ''\r\n ),\r\n ChargeDescription = Description,\r\n ChargeFrequency = case(\r\n BillingFrequency == 'OneTime', 'One-Time',\r\n BillingFrequency == 'Recurring', 'Recurring',\r\n BillingFrequency\r\n ),\r\n ChargePeriodStart = EventDate,\r\n InvoiceId,\r\n PricingQuantity = Quantity,\r\n PricingUnit = 'Reservations',\r\n ProviderName,\r\n RegionId = Region,\r\n RegionName = Region,\r\n SubAccountId = iff(isempty(PurchasingSubscriptionGuid), '', strcat('/subscriptions/', PurchasingSubscriptionGuid)),\r\n SubAccountName = iff(isempty(PurchasingSubscriptionGuid), '', PurchasingSubscriptionName),\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerEmail,\r\n x_CostCenter = CostCenter,\r\n x_InvoiceNumber = Invoice,\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = coalesce(InvoiceSectionName, DepartmentName),\r\n x_IngestionTime = ingestion_time(),\r\n x_MonetaryCommitment = MonetaryCommitment,\r\n x_Overage = Overage,\r\n x_PurchasingBillingAccountId = PurchasingEnrollment,\r\n x_SkuOrderId = ReservationOrderId,\r\n x_SkuOrderName = ReservationOrderName,\r\n x_SkuSize = ArmSkuName,\r\n x_SkuTerm = isoMonths(Term),\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_SubscriptionId = PurchasingSubscriptionGuid,\r\n x_TransactionType = EventType\r\n}\r\n\r\n// Transactions_final_v1_2 table\r\n.create-merge table Transactions_final_v1_2 (\r\n BilledCost: real, // MS CM EA+MCA 2023-05-01\r\n BillingAccountId: string, // MS CM EA+MCA 2023-05-01\r\n BillingAccountName: string, // MS CM EA+MCA 2023-05-01\r\n BillingCurrency: string, // MS CM EA+MCA 2023-05-01\r\n BillingPeriodEnd: datetime, // MS CM EA+MCA 2023-05-01\r\n BillingPeriodStart: datetime, // MS CM EA+MCA 2023-05-01\r\n ChargeCategory: string, // Hubs add-on\r\n ChargeClass: string, // Hubs add-on\r\n ChargeDescription: string, // MS CM EA+MCA 2023-05-01\r\n ChargeFrequency: string, // MS CM EA+MCA 2023-05-01\r\n ChargePeriodStart: datetime, // MS CM EA+MCA 2023-05-01\r\n InvoiceId: string, // MS CM MCA 2023-05-01\r\n PricingQuantity: real, // MS CM EA+MCA 2023-05-01\r\n PricingUnit: string, // Hubs add-on\r\n ProviderName: string, // Hubs add-on\r\n RegionId: string, // MS CM EA+MCA 2023-05-01\r\n RegionName: string, // MS CM EA+MCA 2023-05-01\r\n SubAccountId: string, // MS CM EA+MCA 2023-05-01\r\n SubAccountName: string, // MS CM EA+MCA 2023-05-01\r\n x_AccountName: string, // MS CM EA 2023-05-01\r\n x_AccountOwnerId: string, // MS CM EA 2023-05-01\r\n x_CostCenter: string, // MS CM EA 2023-05-01\r\n x_InvoiceNumber: string, // MS CM MCA 2023-05-01\r\n x_InvoiceSectionId: string, // MS CM MCA 2023-05-01\r\n x_InvoiceSectionName: string, // MS CM MCA 2023-05-01\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_MonetaryCommitment: real, // MS CM EA 2023-05-01\r\n x_Overage: real, // MS CM EA 2023-05-01\r\n x_PurchasingBillingAccountId: string, // MS CM EA 2023-05-01\r\n x_SkuOrderId: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuOrderName: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuSize: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuTerm: int, // MS CM EA+MCA 2023-05-01\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_SubscriptionId: string, // MS CM EA+MCA 2023-05-01\r\n x_TransactionType: string // MS CM EA+MCA 2023-05-01\r\n)\r\n\r\n// Update policy for Transactions_raw -> Transactions_final_v1_2 table\r\n.alter table Transactions_final_v1_2 policy update\r\n```\r\n[{\r\n \"IsEnabled\": true,\r\n \"Source\": \"Transactions_raw\",\r\n \"Query\": \"Transactions_transform_v1_2()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n", + "$fxv#11": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Common utility functions\r\n//\r\n// TIP: Use Ctrl+K,Ctrl+0 to collapse all regions in VS Code\r\n//======================================================================================================================\r\n\r\n\r\n//===| Date functions |=================================================================================================\r\n\r\n// monthstring\r\n.create-or-alter function \r\nwith (docstring = @'Returns the name of the month for the specified date (e.g. Jan or January)', folder =@'Common') \r\nmonthstring(['date']: datetime, length: int = 9)\r\n{\r\n substring(dynamic(['January','February','March','April','May','June','July','August','September','October','November','December'])[getmonth(['date']) - 1], 0, length)\r\n}\r\n\r\n// datestring\r\n.create-or-alter function \r\nwith (docstring = @'Converts 2 dates into a simple, user-friendly date range (e.g. Jan 1-Jan 3)', folder =@'Common') \r\ndatestring(start: datetime, end: datetime = datetime('0001-01-01'))\r\n{\r\n let month = (d: datetime) { monthstring(d, 3) };\r\n let endDate = iff(end == datetime('0001-01-01'), start, end);\r\n let sameDate = startofday(start) == startofday(endDate);\r\n let sameMonth = startofmonth(start) == startofmonth(endDate);\r\n let sameYear = startofyear(start) == startofyear(endDate);\r\n let fullMonth = startofday(start) == startofmonth(start) and startofday(endDate) == startofday(endofmonth(endDate));\r\n let fullYear = startofday(start) == startofyear(start) and startofday(endDate) == startofday(endofyear(endDate));\r\n let currentYear = sameYear and startofyear(start) == startofyear(now());\r\n case(\r\n // Full year | yyyy (same year) / yyyy-yyyy (diff years)\r\n fullYear,\r\n strcat(getyear(start), iff(sameYear, '', strcat('-', getyear(endDate)))),\r\n // 1 full mo, same year | Mmm yyyy\r\n fullMonth and sameMonth and sameYear,\r\n strcat(month(start), ' ', getyear(start)),\r\n // 2+ full mo, same year | Mmm-Mmm (current year) / Mmm-Mmm yyyy (other year)\r\n fullMonth and sameYear,\r\n strcat(month(start), '-', month(endDate), iff(currentYear, '', strcat(' ', getyear(endDate)))),\r\n // Full mo, diff year | Mmm yyyy-Mmm yyyy\r\n fullMonth and not(sameYear),\r\n strcat(month(start), ' ', getyear(start), '-', month(endDate), ' ', getyear(endDate)),\r\n // Same date | Mmm d (current year) / Mmm d, yyyy (other year)\r\n sameDate,\r\n strcat(month(start), ' ', dayofmonth(start), iff(currentYear, '', strcat(', ', getyear(endDate)))),\r\n // 1 partial M, same Y | Mmm d-d (current year) / Mmm d-d, yyyy (other year)\r\n not(fullMonth) and sameMonth and sameYear,\r\n strcat(month(start), ' ', dayofmonth(start), '-', dayofmonth(endDate), iff(currentYear, '', strcat(' ', getyear(endDate)))),\r\n // 2+ partial M, same Y | Mmm d-Mmm d (current year) / Mmm d-Mmm d, yyyy (other year)\r\n not(fullMonth) and not(sameMonth) and sameYear,\r\n strcat(month(start), ' ', dayofmonth(start), '-', month(endDate), ' ', dayofmonth(endDate), iff(currentYear, '', strcat(', ', getyear(endDate)))),\r\n // All other cases | Mmm d, yyyy-Mmm d, yyyy\r\n strcat(month(start), ' ', dayofmonth(start), ', ', getyear(start), '-', month(endDate), ' ', dayofmonth(endDate), ', ', getyear(endDate))\r\n )\r\n}\r\n\r\n// daterange\r\n.create-or-alter function \r\nwith (docstring = @'DEPRECATED: Please use datestring(); function will be removed on or after the Jan 2026 release', folder =@'Common') \r\ndaterange(start: datetime, end: datetime = datetime('0001-01-01'))\r\n{\r\n datestring(start, end)\r\n}\r\n\r\n// monthsago\r\n.create-or-alter function \r\nwith (docstring = 'DEPRECATED: Please use startofmonth(now(), -<# of months>); function will be removed on or after the Jan 2026 release', folder = 'Common')\r\nmonthsago(months: int)\r\n{\r\n datetime_add('month', -months, startofmonth(now()))\r\n}\r\n\r\n\r\n//===| Number functions |===============================================================================================\r\n// NOTE: Must be defined before string converters\r\n\r\n// delta\r\n.create-or-alter function \r\nwith (docstring = @'Compares 2 values and returns the percentage change from oldval to newval', folder =@'Common') \r\ndelta(oldval: double, newval: double)\r\n{\r\n (newval - todouble(oldval))/oldval\r\n}\r\n\r\n// percentOfTotal\r\n// NOTE: Must be before percent() function\r\n.create-or-alter function \r\nwith (docstring = @'Calculates the percentage of each record based on a required Count column', folder =@'Common') \r\npercentOfTotal(t: (Count: long), tot: long)\r\n{\r\n let total = todouble(tot);\r\n t \r\n | extend Percent = round(Count / total * 100, 3) \r\n | order by Count desc\r\n}\r\n\r\n// percent\r\n.create-or-alter function \r\nwith (docstring = @'Calculates the percentage of each record based on a required Count column', folder =@'Common') \r\npercent(t: (Count: long))\r\n{\r\n let total = todouble(toscalar(t | summarize sum(Count)));\r\n percentOfTotal(t, total)\r\n}\r\n\r\n// plusminus\r\n.create-or-alter function \r\nwith (docstring = 'Shows a +/- sign based on the direction of the number', folder = 'Common')\r\nplusminus(val: string)\r\n{\r\n let neg = substring(val, 0, 1) == '-';\r\n iff(neg, val, strcat('+', val))\r\n}\r\n\r\n// updown\r\n.create-or-alter function \r\nwith (docstring = 'Shows an up/down arrow based on the direction of the number', folder = 'Common')\r\nupdown(val: string)\r\n{\r\n // TODO: Handle 0\r\n let neg = substring(val, 0, 1) == '-';\r\n iff(neg, strcat('↓', substring(val, 1)), strcat('↑', val))\r\n}\r\n\r\n\r\n//===| String functions |===============================================================================================\r\n\r\n// percentstring\r\n// NOTE: Must be defined before deltastring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate a percentage and render as a string', folder = 'Common')\r\npercentstring(num: double, total: double = 1.0, places: int = 9)\r\n{\r\n let value = 1.0 * num / total * 100;\r\n strcat(case(\r\n places != 9, round(value, places),\r\n value < 10, round(value, 2),\r\n round(value, 1)\r\n ), '%')\r\n}\r\n\r\n//----------------------------------------------------------------------------------------------------------------------\r\n\r\n// arraystring\r\n.create-or-alter function \r\nwith (docstring = 'Convert an array to a comma-delimited string', folder = 'Common')\r\narraystring(arr: dynamic)\r\n{\r\n replace_string(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(\r\n tostring(arr)\r\n , @'^\\[\"', '')\r\n , @'\"\\]$', '')\r\n , @'^, ', '')\r\n , @', $', '')\r\n , @'^\\[]$', '')\r\n , '\",\"', ', ')\r\n}\r\n\r\n// deltastring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate a delta percentage and render as a string', folder = 'Common')\r\ndeltastring(oldval: double, newval: double, places: int = 1, useArrows: bool = false)\r\n{\r\n let d = delta(oldval, newval);\r\n strcat(case(useArrows and d > 0, '↑', useArrows and d < 0, '↓', d < 0, '-', ''), percentstring(abs(d), 1, places))\r\n}\r\n\r\n// diffstring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate the difference and render as a string', folder = 'Common')\r\ndiffstring(oldval: double, newval: double, places: int = 1)\r\n{\r\n plusminus(round(newval - oldval, places))\r\n}\r\n\r\n// numberstring\r\n.create-or-alter function \r\nwith (docstring = 'Convert a number to a string', folder = 'Common')\r\nnumberstring(num: double, abbrev: bool = true)\r\n{\r\n replace_regex(case(\r\n num >= 10000000000000, strcat(round(1.0 * num / 1000000000000, 1), 'T'),\r\n num >= 1000000000000, strcat(round(1.0 * num / 1000000000000, 2), 'T'),\r\n num >= 10000000000, strcat(round(1.0 * num / 1000000000, 1), 'B'),\r\n num >= 1000000000, strcat(round(1.0 * num / 1000000000, 2), 'B'),\r\n num >= 10000000, strcat(round(1.0 * num / 1000000, 1), 'M'),\r\n num >= 1000000, strcat(round(1.0 * num / 1000000, 2), 'M'),\r\n num >= 10000, strcat(round(1.0 * num / 1000, 1), 'K'),\r\n // Kusto doesn't support back-refs yet -- num > 1000, replace_regex(tostring(num), @'(\\d)(?=(\\d{3})+\\.)', @'\\1,'), // See https://docs.microsoft.com/azure/data-explorer/kusto/query/re2-library\r\n num > 1000, replace_regex(tostring(num), @'([0-9]{3})$', @',\\1'), //num / 1000, ',', substring(tostring(num), 0) - (num / 1000 * 1000)),\r\n tostring(num)\r\n ), @'\\.0$', '')\r\n}\r\n\r\n\r\n//===| Other |==========================================================================================================\r\n\r\n// ifempty\r\n.create-or-alter function \r\nwith (docstring = 'Replaces an empty value with the specified default value', folder = 'Common')\r\nifempty(val: dynamic, defaultVal: dynamic)\r\n{\r\n iff(isempty(val), defaultVal, val)\r\n}\r\n", + "$fxv#12": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Hub database / Open data functions\r\n// Wrap Ingestion database tables for easy access.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n\r\n// PricingUnits\r\n.create-or-alter function\r\nwith (docstring = 'Gets pricing units from the FinOps toolkit PricingUnits open data.', folder = 'OpenData')\r\nPricingUnits()\r\n{\r\n database('Ingestion').PricingUnits\r\n}\r\n\r\n// Regions\r\n.create-or-alter function\r\nwith (docstring = 'Gets regions from the FinOps toolkit Regions open data.', folder = 'OpenData')\r\nRegion()\r\n{\r\n database('Ingestion').Regions\r\n}\r\n\r\n// ResourceTypes\r\n.create-or-alter function\r\nwith (docstring = 'Gets resource types from the FinOps toolkit ResourceTypes open data.', folder = 'OpenData')\r\nResourceType()\r\n{\r\n database('Ingestion').ResourceTypes\r\n}\r\n\r\n// Services\r\n.create-or-alter function\r\nwith (docstring = 'Gets services from the FinOps toolkit Services open data.', folder = 'OpenData')\r\nServices()\r\n{\r\n database('Ingestion').Services\r\n}\r\n", + "$fxv#13": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Hub database / FOCUS 1.0 functions\r\n// Used for reporting with backward compatibility.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n\r\n// CommitmentDiscountUsage_final_v1_0\r\n.create-or-alter function\r\nwith (docstring = 'Gets all commitment discount usage records aligned to FOCUS 1.0.', folder = 'CommitmentDiscountUsage')\r\nCommitmentDiscountUsage_v1_0()\r\n{\r\n database('Ingestion').CommitmentDiscountUsage_final_v1_0\r\n | union (\r\n database('Ingestion').CommitmentDiscountUsage_final_v1_2\r\n // Convert real to decimal\r\n | extend\r\n CommitmentDiscountQuantity = todecimal(CommitmentDiscountQuantity),\r\n ConsumedQuantity = todecimal(ConsumedQuantity),\r\n x_CommitmentDiscountCommittedCount = todecimal(x_CommitmentDiscountCommittedCount),\r\n x_CommitmentDiscountCommittedAmount = todecimal(x_CommitmentDiscountCommittedAmount),\r\n x_CommitmentDiscountNormalizedRatio = todecimal(x_CommitmentDiscountNormalizedRatio)\r\n )\r\n | project\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId,\r\n CommitmentDiscountType,\r\n ConsumedQuantity,\r\n ProviderName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n SubAccountId,\r\n x_CommitmentDiscountCommittedCount,\r\n x_CommitmentDiscountCommittedAmount,\r\n x_CommitmentDiscountNormalizedGroup,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_CommitmentDiscountQuantity,\r\n x_IngestionTime,\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n x_ServiceModel,\r\n x_SkuOrderId,\r\n x_SkuSize,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n\r\n// Costs_final_v1_0\r\n.create-or-alter function\r\nwith (docstring = 'Gets all cost and usage records aligned to FOCUS 1.0.', folder = 'Costs')\r\nCosts_v1_0()\r\n{\r\n database('Ingestion').Costs_final_v1_0\r\n | union (\r\n database('Ingestion').Costs_final_v1_2\r\n // Convert real to decimal\r\n | extend\r\n BilledCost = todecimal(BilledCost),\r\n CommitmentDiscountQuantity = todecimal(CommitmentDiscountQuantity),\r\n ConsumedQuantity = todecimal(ConsumedQuantity),\r\n ContractedCost = todecimal(ContractedCost),\r\n ContractedUnitPrice = todecimal(ContractedUnitPrice),\r\n EffectiveCost = todecimal(EffectiveCost),\r\n ListCost = todecimal(ListCost),\r\n ListUnitPrice = todecimal(ListUnitPrice),\r\n PricingQuantity = todecimal(PricingQuantity),\r\n x_BilledCostInUsd = todecimal(x_BilledCostInUsd),\r\n x_BilledUnitPrice = todecimal(x_BilledUnitPrice),\r\n x_BillingExchangeRate = todecimal(x_BillingExchangeRate),\r\n x_CommitmentDiscountNormalizedRatio = todecimal(x_CommitmentDiscountNormalizedRatio),\r\n x_ContractedCostInUsd = todecimal(x_ContractedCostInUsd),\r\n x_CurrencyConversionRate = todecimal(x_CurrencyConversionRate),\r\n x_EffectiveCostInUsd = todecimal(x_EffectiveCostInUsd),\r\n x_EffectiveUnitPrice = todecimal(x_EffectiveUnitPrice),\r\n x_ListCostInUsd = todecimal(x_ListCostInUsd),\r\n x_PricingBlockSize = todecimal(x_PricingBlockSize)\r\n // Rename columns\r\n | project-rename\r\n x_InvoiceId = InvoiceId,\r\n x_PricingCurrency = PricingCurrency,\r\n x_SkuMeterName = SkuMeter\r\n // Generate historical x_SkuDetails format from SkuPriceDetails\r\n | extend x_SkuDetails = iff(isnotempty(x_SkuDetails), x_SkuDetails, parse_json(replace_regex(tostring(SkuPriceDetails), @'([\\{,])\"x_', @'\\1\"')))\r\n )\r\n | project\r\n AvailabilityZone,\r\n BilledCost,\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingAccountType,\r\n BillingCurrency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n ChargeCategory,\r\n ChargeClass,\r\n ChargeDescription,\r\n ChargeFrequency,\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId,\r\n CommitmentDiscountName,\r\n CommitmentDiscountStatus,\r\n CommitmentDiscountType,\r\n ConsumedQuantity,\r\n ConsumedUnit,\r\n ContractedCost,\r\n ContractedUnitPrice,\r\n EffectiveCost,\r\n InvoiceIssuerName,\r\n ListCost,\r\n ListUnitPrice,\r\n PricingCategory,\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n PublisherName,\r\n RegionId,\r\n RegionName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n SkuId,\r\n SkuPriceId,\r\n SubAccountId,\r\n SubAccountName,\r\n SubAccountType,\r\n Tags,\r\n x_AccountId,\r\n x_AccountName,\r\n x_AccountOwnerId,\r\n x_BilledCostInUsd,\r\n x_BilledUnitPrice,\r\n x_BillingAccountAgreement,\r\n x_BillingAccountId,\r\n x_BillingAccountName,\r\n x_BillingExchangeRate,\r\n x_BillingExchangeRateDate,\r\n x_BillingProfileId,\r\n x_BillingProfileName,\r\n x_ChargeId,\r\n x_ContractedCostInUsd,\r\n x_CostAllocationRuleName,\r\n x_CostCategories,\r\n x_CostCenter,\r\n x_Credits,\r\n x_CostType,\r\n x_CurrencyConversionRate,\r\n x_CustomerId,\r\n x_CustomerName,\r\n x_Discount,\r\n x_EffectiveCostInUsd,\r\n x_EffectiveUnitPrice,\r\n x_ExportTime,\r\n x_IngestionTime,\r\n x_InvoiceId,\r\n x_InvoiceIssuerId,\r\n x_InvoiceSectionId,\r\n x_InvoiceSectionName,\r\n x_ListCostInUsd,\r\n x_Location,\r\n x_Operation,\r\n x_PartnerCreditApplied,\r\n x_PartnerCreditRate,\r\n x_PricingBlockSize,\r\n x_PricingCurrency,\r\n x_PricingSubcategory,\r\n x_PricingUnitDescription,\r\n x_Project,\r\n x_PublisherCategory,\r\n x_PublisherId,\r\n x_ResellerId,\r\n x_ResellerName,\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n x_ServiceCode,\r\n x_ServiceId,\r\n x_ServicePeriodEnd,\r\n x_ServicePeriodStart,\r\n x_SkuDescription,\r\n x_SkuDetails,\r\n x_SkuIsCreditEligible,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterName,\r\n x_SkuMeterSubcategory,\r\n x_SkuOfferId,\r\n x_SkuOrderId,\r\n x_SkuOrderName,\r\n x_SkuPartNumber,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceChanges,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_UsageType\r\n}\r\n\r\n\r\n// Prices_final_v1_0\r\n.create-or-alter function\r\nwith (docstring = 'Gets all prices aligned to FOCUS 1.0.', folder = 'Prices')\r\nPrices_v1_0()\r\n{\r\n database('Ingestion').Prices_final_v1_0\r\n | union (\r\n database('Ingestion').Prices_final_v1_2\r\n // Convert real to decimal\r\n | extend\r\n ContractedUnitPrice = todecimal(ContractedUnitPrice),\r\n ListUnitPrice = todecimal(ListUnitPrice),\r\n x_BaseUnitPrice = todecimal(x_BaseUnitPrice),\r\n x_CommitmentDiscountNormalizedRatio = todecimal(x_CommitmentDiscountNormalizedRatio),\r\n x_ContractedUnitPriceDiscount = todecimal(x_ContractedUnitPriceDiscount),\r\n x_ContractedUnitPriceDiscountPercent = todecimal(x_ContractedUnitPriceDiscountPercent),\r\n x_EffectiveUnitPrice = todecimal(x_EffectiveUnitPrice),\r\n x_EffectiveUnitPriceDiscount = todecimal(x_EffectiveUnitPriceDiscount),\r\n x_EffectiveUnitPriceDiscountPercent = todecimal(x_EffectiveUnitPriceDiscountPercent),\r\n x_PricingBlockSize = todecimal(x_PricingBlockSize),\r\n x_SkuIncludedQuantity = todecimal(x_SkuIncludedQuantity),\r\n x_SkuTier = todecimal(x_SkuTier),\r\n x_TotalUnitPriceDiscount = todecimal(x_TotalUnitPriceDiscount),\r\n x_TotalUnitPriceDiscountPercent = todecimal(x_TotalUnitPriceDiscountPercent) \r\n // Rename columns\r\n | project-rename\r\n x_PricingCurrency = PricingCurrency,\r\n x_SkuMeterName = SkuMeter\r\n )\r\n | project\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingCurrency,\r\n ChargeCategory,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountType,\r\n ContractedUnitPrice,\r\n ListUnitPrice,\r\n PricingCategory,\r\n PricingUnit,\r\n SkuId,\r\n SkuPriceId,\r\n SkuPriceIdv2,\r\n x_BaseUnitPrice,\r\n x_BillingAccountAgreement,\r\n x_BillingAccountId,\r\n x_BillingProfileId,\r\n x_CommitmentDiscountSpendEligibility,\r\n x_CommitmentDiscountUsageEligibility,\r\n x_ContractedUnitPriceDiscount,\r\n x_ContractedUnitPriceDiscountPercent,\r\n x_EffectivePeriodEnd,\r\n x_EffectivePeriodStart,\r\n x_EffectiveUnitPrice,\r\n x_EffectiveUnitPriceDiscount,\r\n x_EffectiveUnitPriceDiscountPercent,\r\n x_IngestionTime,\r\n x_PricingBlockSize,\r\n x_PricingCurrency,\r\n x_PricingSubcategory,\r\n x_PricingUnitDescription,\r\n x_SkuDescription,\r\n x_SkuId,\r\n x_SkuIncludedQuantity,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterName,\r\n x_SkuMeterSubcategory,\r\n x_SkuMeterType,\r\n x_SkuPriceType,\r\n x_SkuProductId,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuOfferId,\r\n x_SkuPartNumber,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_TotalUnitPriceDiscount,\r\n x_TotalUnitPriceDiscountPercent\r\n}\r\n\r\n\r\n// Recommendations_final_v1_0\r\n.create-or-alter function\r\nwith (docstring = 'Gets all recommendations aligned to FOCUS 1.0.', folder = 'Recommendations')\r\nRecommendations_v1_0()\r\n{\r\n database('Ingestion').Recommendations_final_v1_0\r\n | union (\r\n database('Ingestion').Recommendations_final_v1_2\r\n // Convert real to decimal\r\n | extend\r\n x_EffectiveCostAfter = todecimal(x_EffectiveCostAfter),\r\n x_EffectiveCostBefore = todecimal(x_EffectiveCostBefore),\r\n x_EffectiveCostSavings = todecimal(x_EffectiveCostSavings)\r\n )\r\n | project\r\n ProviderName,\r\n SubAccountId,\r\n x_IngestionTime,\r\n x_EffectiveCostAfter,\r\n x_EffectiveCostBefore,\r\n x_EffectiveCostSavings,\r\n x_RecommendationDate,\r\n x_RecommendationDetails,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n\r\n// Transactions_final_v1_0\r\n.create-or-alter function\r\nwith (docstring = 'Gets all transactions aligned to FOCUS 1.0.', folder = 'Transactions')\r\nTransactions_v1_0()\r\n{\r\n database('Ingestion').Transactions_final_v1_0\r\n | union (\r\n database('Ingestion').Transactions_final_v1_2\r\n // Convert real to decimal\r\n | extend\r\n BilledCost = todecimal(BilledCost),\r\n PricingQuantity = todecimal(PricingQuantity),\r\n x_MonetaryCommitment = todecimal(x_MonetaryCommitment),\r\n x_Overage = todecimal(x_Overage)\r\n // Rename columns\r\n | project-rename\r\n x_InvoiceId = InvoiceId\r\n )\r\n | project\r\n BilledCost,\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingCurrency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n ChargeCategory,\r\n ChargeClass,\r\n ChargeDescription,\r\n ChargeFrequency,\r\n ChargePeriodStart,\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n RegionId,\r\n RegionName,\r\n SubAccountId,\r\n SubAccountName,\r\n x_AccountName,\r\n x_AccountOwnerId,\r\n x_CostCenter,\r\n x_InvoiceId,\r\n x_InvoiceNumber,\r\n x_InvoiceSectionId,\r\n x_InvoiceSectionName,\r\n x_IngestionTime,\r\n x_MonetaryCommitment,\r\n x_Overage,\r\n x_PurchasingBillingAccountId,\r\n x_SkuOrderId,\r\n x_SkuOrderName,\r\n x_SkuSize,\r\n x_SkuTerm,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_SubscriptionId,\r\n x_TransactionType\r\n}\r\n", + "$fxv#14": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Hub database / FOCUS 1.2 functions\r\n// Used for reporting with backward compatibility.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n\r\n// CommitmentDiscountUsage_final_v1_2\r\n.create-or-alter function\r\nwith (docstring = 'Gets all commitment discount usage records aligned to FOCUS 1.2.', folder = 'CommitmentDiscountUsage')\r\nCommitmentDiscountUsage_v1_2()\r\n{\r\n database('Ingestion').CommitmentDiscountUsage_final_v1_2\r\n | union (\r\n database('Ingestion').CommitmentDiscountUsage_final_v1_0\r\n // Convert decimal to real\r\n | extend\r\n ConsumedQuantity = toreal(ConsumedQuantity),\r\n x_CommitmentDiscountCommittedCount = toreal(x_CommitmentDiscountCommittedCount),\r\n x_CommitmentDiscountCommittedAmount = toreal(x_CommitmentDiscountCommittedAmount),\r\n x_CommitmentDiscountNormalizedRatio = toreal(x_CommitmentDiscountNormalizedRatio)\r\n // Add new columns\r\n | lookup kind=leftouter (Services | distinct x_ResourceType, ServiceSubcategory) on x_ResourceType\r\n | extend CommitmentDiscountQuantity = ConsumedQuantity * x_CommitmentDiscountNormalizedRatio\r\n | extend CommitmentDiscountUnit = case(\r\n x_CommitmentDiscountNormalizedRatio == 1, 'Hours',\r\n x_CommitmentDiscountNormalizedRatio > 1, 'Normalized Hours',\r\n ''\r\n )\r\n )\r\n | project\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId,\r\n CommitmentDiscountQuantity,\r\n CommitmentDiscountType,\r\n CommitmentDiscountUnit,\r\n ConsumedQuantity,\r\n ProviderName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory,\r\n SubAccountId,\r\n x_CommitmentDiscountCommittedCount,\r\n x_CommitmentDiscountCommittedAmount,\r\n x_CommitmentDiscountNormalizedGroup,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_IngestionTime,\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n x_ServiceModel,\r\n x_SkuOrderId,\r\n x_SkuSize,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n\r\n// Costs_final_v1_2\r\n.create-or-alter function\r\nwith (docstring = 'Gets all cost and usage records aligned to FOCUS 1.2.', folder = 'Costs')\r\nCosts_v1_2()\r\n{\r\n database('Ingestion').Costs_final_v1_2\r\n | union (\r\n database('Ingestion').Costs_final_v1_0\r\n // Convert decimal to real\r\n | extend\r\n BilledCost = toreal(BilledCost),\r\n ConsumedQuantity = toreal(ConsumedQuantity),\r\n ContractedCost = toreal(ContractedCost),\r\n ContractedUnitPrice = toreal(ContractedUnitPrice),\r\n EffectiveCost = toreal(EffectiveCost),\r\n ListCost = toreal(ListCost),\r\n ListUnitPrice = toreal(ListUnitPrice),\r\n PricingQuantity = toreal(PricingQuantity),\r\n x_BilledCostInUsd = toreal(x_BilledCostInUsd),\r\n x_BilledUnitPrice = toreal(x_BilledUnitPrice),\r\n x_BillingExchangeRate = toreal(x_BillingExchangeRate),\r\n x_ContractedCostInUsd = toreal(x_ContractedCostInUsd),\r\n x_CurrencyConversionRate = toreal(x_CurrencyConversionRate),\r\n x_EffectiveCostInUsd = toreal(x_EffectiveCostInUsd),\r\n x_EffectiveUnitPrice = toreal(x_EffectiveUnitPrice),\r\n x_ListCostInUsd = toreal(x_ListCostInUsd),\r\n x_PricingBlockSize = toreal(x_PricingBlockSize)\r\n // Rename columns\r\n | project-rename\r\n InvoiceId = x_InvoiceId,\r\n PricingCurrency = x_PricingCurrency,\r\n SkuMeter = x_SkuMeterName\r\n // Add new columns\r\n | lookup kind=leftouter (Services | where isnotempty(x_ResourceType) | summarize take_any(ServiceSubcategory), take_any(x_ServiceModel) by x_ResourceType) on x_ResourceType\r\n | extend CapacityReservationId = tostring(x_SkuDetails.VMCapacityReservationId)\r\n | extend CapacityReservationStatus = case(\r\n isempty(CapacityReservationId), '',\r\n tolower(x_ResourceType) == 'microsoft.compute/capacityreservationgroups/capacityreservations', 'Unused',\r\n 'Used'\r\n )\r\n | extend x_CommitmentDiscountNormalizedRatio = case(\r\n // Not applicable\r\n isempty(CommitmentDiscountStatus), real(null),\r\n // Parse from SKU details if not specified explicitly\r\n toreal(coalesce(x_SkuDetails.RINormalizationRatio, dynamic(1)))\r\n )\r\n | extend CommitmentDiscountQuantity = case(\r\n isempty(CommitmentDiscountStatus), real(null),\r\n CommitmentDiscountCategory == 'Spend', EffectiveCost / coalesce(x_BillingExchangeRate, real(1)),\r\n CommitmentDiscountCategory == 'Usage' and isnotempty(x_CommitmentDiscountNormalizedRatio), PricingQuantity / coalesce(x_PricingBlockSize, real(1)) * x_CommitmentDiscountNormalizedRatio,\r\n real(null)\r\n )\r\n | extend CommitmentDiscountUnit = case(\r\n isempty(CommitmentDiscountQuantity), '',\r\n CommitmentDiscountCategory == 'Spend', PricingCurrency,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio == real(1), ConsumedUnit,\r\n CommitmentDiscountCategory == 'Usage' and x_CommitmentDiscountNormalizedRatio > real(1), strcat('Normalized ', ConsumedUnit),\r\n ''\r\n )\r\n | extend x_AmortizationClass = case(\r\n ChargeCategory == 'Purchase' and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeCategory == 'Usage' and isnotempty(CommitmentDiscountId) and isnotempty(CommitmentDiscountStatus), 'Amortized Charge',\r\n ''\r\n )\r\n // Hubs add-ons\r\n | extend x_CommitmentDiscountUtilizationPotential = case(\r\n ChargeCategory == 'Purchase', real(0),\r\n ProviderName == 'Microsoft' and isnotempty(CommitmentDiscountCategory), EffectiveCost,\r\n CommitmentDiscountCategory == 'Usage', ConsumedQuantity,\r\n CommitmentDiscountCategory == 'Spend', EffectiveCost,\r\n real(0)\r\n )\r\n | extend x_CommitmentDiscountUtilizationAmount = iff(CommitmentDiscountStatus == 'Used', x_CommitmentDiscountUtilizationPotential, real(0))\r\n | extend x_SkuCoreCount = toint(coalesce(x_SkuDetails.VCPUs, x_SkuDetails.VCores, x_SkuDetails.vCores))\r\n | extend x_SkuInstanceType = tostring(coalesce(x_SkuDetails.ServiceType, x_SkuDetails.ServerSku))\r\n | extend x_SkuOperatingSystem = case(\r\n x_SkuDetails.ImageType == 'Canonical', 'Linux',\r\n x_SkuDetails.ImageType == 'Windows Server BYOL', 'Windows Server',\r\n x_SkuMeterSubcategory endswith ' Series Windows', 'Windows Server',\r\n x_SkuDetails.ImageType\r\n )\r\n | extend x_ConsumedCoreHours = iff(ConsumedUnit == 'Hours' and isnotempty(x_SkuCoreCount), x_SkuCoreCount * ConsumedQuantity, real(null))\r\n | extend tmp_SqlAhb = tolower(x_SkuDetails.AHB)\r\n | extend x_SkuLicenseType = case(\r\n x_SkuDetails.ImageType contains 'Windows Server BYOL', 'Windows Server',\r\n x_SkuMeterSubcategory == 'SQL Server Azure Hybrid Benefit', 'SQL Server',\r\n ''\r\n )\r\n | extend x_SkuLicenseStatus = case(\r\n isnotempty(x_SkuLicenseType) or tmp_SqlAhb == 'true' or (x_SkuMeterSubcategory contains 'Azure Hybrid Benefit'), 'Enabled',\r\n (x_SkuMeterSubcategory contains 'Windows') or tmp_SqlAhb == 'false', 'Not enabled',\r\n ''\r\n )\r\n | extend x_SkuLicenseQuantity = case(\r\n isempty(x_SkuCoreCount), int(null),\r\n x_SkuCoreCount <= 8, int(8),\r\n x_SkuCoreCount > 8, x_SkuCoreCount,\r\n int(null)\r\n )\r\n | extend x_SkuLicenseUnit = iff(isnotempty(x_SkuLicenseQuantity), 'Cores', '')\r\n | extend x_CommitmentDiscountSavings = iff(ContractedCost < EffectiveCost, real(0), ContractedCost - EffectiveCost)\r\n | extend x_NegotiatedDiscountSavings = iff(ListCost < ContractedCost, real(0), ListCost - ContractedCost)\r\n | extend x_TotalSavings = iff(ListCost < EffectiveCost, real(0), ListCost - EffectiveCost)\r\n | extend x_CommitmentDiscountPercent = iff(ContractedUnitPrice == 0, real(0), (ContractedUnitPrice - x_EffectiveUnitPrice) / ContractedUnitPrice)\r\n | extend x_NegotiatedDiscountPercent = iff(ListUnitPrice == 0, real(0), (ListUnitPrice - ContractedUnitPrice) / ListUnitPrice)\r\n | extend x_TotalDiscountPercent = iff(ListUnitPrice == 0, real(0), (ListUnitPrice - x_EffectiveUnitPrice) / ListUnitPrice)\r\n // SkuPriceDetails conversion -- Must be after hubs add-ons\r\n | extend SkuPriceDetails = parse_json(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(tostring(x_SkuDetails)\r\n // Prefix all keys with x_ first to avoid double-prefixing\r\n , @'([\\{,])\"', @'\\1\"x_')\r\n // CoreCount for number of CPUs/vCPUs/cores/vCores\r\n , @'\"x_(VCPUs|VCores|vCores)\":', @'\"CoreCount\":')\r\n // TODO: DiskMaxIops for disk I/O operations per second (IOPS)\r\n // TODO: DiskSpace for disk size in GiB\r\n // TODO: DiskType for the kind of disk (e.g., SSD, HDD, NVMe)\r\n // TODO: GpuCount for the number of GPUs\r\n // InstanceType for the resource size/SKU (e.g., ArmSkuName)\r\n , @'\"x_(ServerSku|ServiceType)\":', @'\"InstanceType\":')\r\n // TODO: InstanceSeries for the size family/series\r\n // TODO: MemorySize for the RAM in GiB\r\n // TODO: NetworkMaxIops for network I/O operations per second (IOPS)\r\n // TODO: NetworkMaxThroughput for network max throughput for data transfer in Mbps\r\n // OperatingSystem for the OS name\r\n , @'(\"x_ImageType\":\"Canonical\")', @'\\1,\"OperatingSystem\":\"Linux\"')\r\n , @'(\"x_ImageType\":\"Windows Server( BYOL)?\")', @'\\1,\"OperatingSystem\":\"Windows Server\"')\r\n , @'(\"x_ImageType\":(\"[^\"]+\"))', @'\\1,\"OperatingSystem\":\\2')\r\n // TODO: Redundancy for the level of redundancy (e.g., Local, Zonal, Global)\r\n // TODO: StorageClass for the tier of storage (e.g., Hot, Archive, Nearline)\r\n )\r\n | extend SkuPriceDetails = iff(isempty(SkuPriceDetails.OperatingSystem) and isnotempty(x_SkuOperatingSystem),\r\n parse_json(replace_string(tostring(SkuPriceDetails), '}', strcat(@',\"OperatingSystem\":\"', x_SkuOperatingSystem, '\"}'))),\r\n SkuPriceDetails)\r\n )\r\n | extend SkuPriceDetails = iff(isnotempty(SkuPriceDetails), SkuPriceDetails, parse_json(replace_regex(tostring(x_SkuDetails), @'([\\{,])\"', @'\\1\"x_')))\r\n | project\r\n AvailabilityZone,\r\n BilledCost,\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingAccountType,\r\n BillingCurrency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n CapacityReservationId,\r\n CapacityReservationStatus,\r\n ChargeCategory,\r\n ChargeClass,\r\n ChargeDescription,\r\n ChargeFrequency,\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId,\r\n CommitmentDiscountName,\r\n CommitmentDiscountQuantity,\r\n CommitmentDiscountStatus,\r\n CommitmentDiscountType,\r\n CommitmentDiscountUnit,\r\n ConsumedQuantity,\r\n ConsumedUnit,\r\n ContractedCost,\r\n ContractedUnitPrice,\r\n EffectiveCost,\r\n InvoiceId,\r\n InvoiceIssuerName,\r\n ListCost,\r\n ListUnitPrice,\r\n PricingCategory,\r\n PricingCurrency,\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n PublisherName,\r\n RegionId,\r\n RegionName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory,\r\n SkuId,\r\n SkuMeter,\r\n SkuPriceDetails,\r\n SkuPriceId,\r\n SubAccountId,\r\n SubAccountName,\r\n SubAccountType,\r\n Tags,\r\n x_AccountId,\r\n x_AccountName,\r\n x_AccountOwnerId,\r\n x_AmortizationClass,\r\n x_BilledCostInUsd,\r\n x_BilledUnitPrice,\r\n x_BillingAccountAgreement,\r\n x_BillingAccountId,\r\n x_BillingAccountName,\r\n x_BillingExchangeRate,\r\n x_BillingExchangeRateDate,\r\n x_BillingItemCode,\r\n x_BillingItemName,\r\n x_BillingProfileId,\r\n x_BillingProfileName,\r\n x_ChargeId,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_CommitmentDiscountPercent,\r\n x_CommitmentDiscountSavings,\r\n x_CommitmentDiscountSpendEligibility,\r\n x_CommitmentDiscountUsageEligibility,\r\n x_CommitmentDiscountUtilizationAmount,\r\n x_CommitmentDiscountUtilizationPotential,\r\n x_CommodityCode,\r\n x_CommodityName,\r\n x_ComponentName,\r\n x_ComponentType,\r\n x_ConsumedCoreHours,\r\n x_ContractedCostInUsd,\r\n x_CostAllocationRuleName,\r\n x_CostCategories,\r\n x_CostCenter,\r\n x_CostType,\r\n x_Credits,\r\n x_CurrencyConversionRate,\r\n x_CustomerId,\r\n x_CustomerName,\r\n x_Discount,\r\n x_EffectiveCostInUsd,\r\n x_EffectiveUnitPrice,\r\n x_ExportTime,\r\n x_IngestionTime,\r\n x_InstanceID,\r\n x_InvoiceIssuerId,\r\n x_InvoiceSectionId,\r\n x_InvoiceSectionName,\r\n x_ListCostInUsd,\r\n x_Location,\r\n x_NegotiatedDiscountPercent,\r\n x_NegotiatedDiscountSavings,\r\n x_Operation,\r\n x_OwnerAccountID,\r\n x_PartnerCreditApplied,\r\n x_PartnerCreditRate,\r\n x_PricingBlockSize,\r\n x_PricingSubcategory,\r\n x_PricingUnitDescription,\r\n x_Project,\r\n x_PublisherCategory,\r\n x_PublisherId,\r\n x_ResellerId,\r\n x_ResellerName,\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n x_ServiceCode,\r\n x_ServiceId,\r\n x_ServiceModel,\r\n x_ServicePeriodEnd,\r\n x_ServicePeriodStart,\r\n x_SkuCoreCount,\r\n x_SkuDescription,\r\n x_SkuDetails,\r\n x_SkuInstanceType,\r\n x_SkuIsCreditEligible,\r\n x_SkuLicenseQuantity,\r\n x_SkuLicenseStatus,\r\n x_SkuLicenseType,\r\n x_SkuLicenseUnit,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterSubcategory,\r\n x_SkuOfferId,\r\n x_SkuOperatingSystem,\r\n x_SkuOrderId,\r\n x_SkuOrderName,\r\n x_SkuPartNumber,\r\n x_SkuPlanName,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceChanges,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceValues,\r\n x_SourceVersion,\r\n x_SubproductName,\r\n x_TotalDiscountPercent,\r\n x_TotalSavings,\r\n x_UsageType\r\n}\r\n\r\n\r\n// Prices_final_v1_2\r\n.create-or-alter function\r\nwith (docstring = 'Gets all prices aligned to FOCUS 1.2.', folder = 'Prices')\r\nPrices_v1_2()\r\n{\r\n database('Ingestion').Prices_final_v1_2\r\n | union (\r\n database('Ingestion').Prices_final_v1_0\r\n // Convert decimal to real\r\n | extend\r\n ContractedUnitPrice = toreal(ContractedUnitPrice),\r\n ListUnitPrice = toreal(ListUnitPrice),\r\n x_BaseUnitPrice = toreal(x_BaseUnitPrice),\r\n x_ContractedUnitPriceDiscount = toreal(x_ContractedUnitPriceDiscount),\r\n x_ContractedUnitPriceDiscountPercent = toreal(x_ContractedUnitPriceDiscountPercent),\r\n x_EffectiveUnitPrice = toreal(x_EffectiveUnitPrice),\r\n x_EffectiveUnitPriceDiscount = toreal(x_EffectiveUnitPriceDiscount),\r\n x_EffectiveUnitPriceDiscountPercent = toreal(x_EffectiveUnitPriceDiscountPercent),\r\n x_PricingBlockSize = toreal(x_PricingBlockSize),\r\n x_SkuIncludedQuantity = toreal(x_SkuIncludedQuantity),\r\n x_SkuTier = toreal(x_SkuTier),\r\n x_TotalUnitPriceDiscount = toreal(x_TotalUnitPriceDiscount),\r\n x_TotalUnitPriceDiscountPercent = toreal(x_TotalUnitPriceDiscountPercent) \r\n // Rename columns\r\n | project-rename\r\n PricingCurrency = x_PricingCurrency,\r\n SkuMeter = x_SkuMeterName\r\n )\r\n | project\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingCurrency,\r\n ChargeCategory,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountType,\r\n CommitmentDiscountUnit,\r\n ContractedUnitPrice,\r\n ListUnitPrice,\r\n PricingCategory,\r\n PricingCurrency,\r\n PricingUnit,\r\n SkuId,\r\n SkuMeter,\r\n SkuPriceId,\r\n SkuPriceIdv2,\r\n x_BaseUnitPrice,\r\n x_BillingAccountAgreement,\r\n x_BillingAccountId,\r\n x_BillingProfileId,\r\n x_CommitmentDiscountNormalizedRatio,\r\n x_CommitmentDiscountSpendEligibility,\r\n x_CommitmentDiscountUsageEligibility,\r\n x_ContractedUnitPriceDiscount,\r\n x_ContractedUnitPriceDiscountPercent,\r\n x_EffectivePeriodEnd,\r\n x_EffectivePeriodStart,\r\n x_EffectiveUnitPrice,\r\n x_EffectiveUnitPriceDiscount,\r\n x_EffectiveUnitPriceDiscountPercent,\r\n x_IngestionTime,\r\n x_PricingBlockSize,\r\n x_PricingSubcategory,\r\n x_PricingUnitDescription,\r\n x_SkuDescription,\r\n x_SkuId,\r\n x_SkuIncludedQuantity,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterSubcategory,\r\n x_SkuMeterType,\r\n x_SkuPriceType,\r\n x_SkuProductId,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuOfferId,\r\n x_SkuPartNumber,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_TotalUnitPriceDiscount,\r\n x_TotalUnitPriceDiscountPercent\r\n}\r\n\r\n\r\n// Recommendations_final_v1_2\r\n.create-or-alter function\r\nwith (docstring = 'Gets all recommendations aligned to FOCUS 1.2.', folder = 'Recommendations')\r\nRecommendations_v1_2()\r\n{\r\n database('Ingestion').Recommendations_final_v1_2\r\n | union (\r\n database('Ingestion').Recommendations_final_v1_0\r\n // Convert decimal to real\r\n | extend\r\n x_EffectiveCostAfter = toreal(x_EffectiveCostAfter),\r\n x_EffectiveCostBefore = toreal(x_EffectiveCostBefore),\r\n x_EffectiveCostSavings = toreal(x_EffectiveCostSavings)\r\n )\r\n | project\r\n ProviderName,\r\n SubAccountId,\r\n x_IngestionTime,\r\n x_EffectiveCostAfter,\r\n x_EffectiveCostBefore,\r\n x_EffectiveCostSavings,\r\n x_RecommendationDate,\r\n x_RecommendationDetails,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n\r\n// Transactions_final_v1_2\r\n.create-or-alter function\r\nwith (docstring = 'Gets all transactions aligned to FOCUS 1.2.', folder = 'Transactions')\r\nTransactions_v1_2()\r\n{\r\n database('Ingestion').Transactions_final_v1_2\r\n | union (\r\n database('Ingestion').Transactions_final_v1_0\r\n // Convert decimal to real\r\n | extend\r\n BilledCost = toreal(BilledCost),\r\n PricingQuantity = toreal(PricingQuantity),\r\n x_MonetaryCommitment = toreal(x_MonetaryCommitment),\r\n x_Overage = toreal(x_Overage)\r\n // Rename columns\r\n | project-rename\r\n InvoiceId = x_InvoiceId\r\n )\r\n | project\r\n BilledCost,\r\n BillingAccountId,\r\n BillingAccountName,\r\n BillingCurrency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n ChargeCategory,\r\n ChargeClass,\r\n ChargeDescription,\r\n ChargeFrequency,\r\n ChargePeriodStart,\r\n InvoiceId,\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n RegionId,\r\n RegionName,\r\n SubAccountId,\r\n SubAccountName,\r\n x_AccountName,\r\n x_AccountOwnerId,\r\n x_CostCenter,\r\n x_InvoiceNumber,\r\n x_InvoiceSectionId,\r\n x_InvoiceSectionName,\r\n x_IngestionTime,\r\n x_MonetaryCommitment,\r\n x_Overage,\r\n x_PurchasingBillingAccountId,\r\n x_SkuOrderId,\r\n x_SkuOrderName,\r\n x_SkuSize,\r\n x_SkuTerm,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_SubscriptionId,\r\n x_TransactionType\r\n}\r\n\r\n\r\n//======================================================================================================================\r\n// Latest FOCUS version\r\n//======================================================================================================================\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all commitment discount usage records with the latest supported version of the FOCUS schema.', folder = 'CommitmentDiscountUsage')\r\nCommitmentDiscountUsage()\r\n{\r\n CommitmentDiscountUsage_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all cost and usage records with the latest supported version of the FOCUS schema.', folder = 'Costs')\r\nCosts()\r\n{\r\n Costs_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all prices with the latest supported version of the FOCUS schema.', folder = 'Prices')\r\nPrices()\r\n{\r\n Prices_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all recommendations with the latest supported version of the FOCUS schema.', folder = 'Recommendations')\r\nRecommendations()\r\n{\r\n Recommendations_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all transactions with the latest supported version of the FOCUS schema.', folder = 'Transactions')\r\nTransactions()\r\n{\r\n Transactions_v1_2()\r\n}\r\n", + "$fxv#15": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Hub database / Latest FOCUS version functions\r\n// Used for ad hoc queries.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all commitment discount usage records with the latest supported version of the FOCUS schema.', folder = 'CommitmentDiscountUsage')\r\nCommitmentDiscountUsage()\r\n{\r\n CommitmentDiscountUsage_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all cost and usage records with the latest supported version of the FOCUS schema.', folder = 'Costs')\r\nCosts()\r\n{\r\n Costs_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all prices with the latest supported version of the FOCUS schema.', folder = 'Prices')\r\nPrices()\r\n{\r\n Prices_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all recommendations with the latest supported version of the FOCUS schema.', folder = 'Recommendations')\r\nRecommendations()\r\n{\r\n Recommendations_v1_2()\r\n}\r\n\r\n\r\n.create-or-alter function\r\nwith (docstring = 'Gets all transactions with the latest supported version of the FOCUS schema.', folder = 'Transactions')\r\nTransactions()\r\n{\r\n Transactions_v1_2()\r\n}\r\n", + "$fxv#2": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData/Internal')\r\n_resource_type_3(id: string) {\r\n dynamic({\r\n \"microsoft.hybridnetwork/vendors\": { \"SingularDisplayName\": \"Azure Network Function Manager ? vendor\" }\r\n ,\"microsoft.hybridonboarding/extensionmanagers\": { \"SingularDisplayName\": \"Microsoft.HybridOnboarding extension manager\" }\r\n ,\"microsoft.impact/connectors\": { \"SingularDisplayName\": \"Impact Reporting Connector\" }\r\n ,\"microsoft.impact/impactcategories\": { \"SingularDisplayName\": \"Microsoft.Impact impact category\" }\r\n ,\"microsoft.impact/topologyimpacts\": { \"SingularDisplayName\": \"Microsoft.Impact topology impact\" }\r\n ,\"microsoft.impact/workloadimpacts\": { \"SingularDisplayName\": \"Microsoft.Impact workload impact\" }\r\n ,\"microsoft.impact/workloadimpacts/insights\": { \"SingularDisplayName\": \"Microsoft.Impact workload impacts insight\" }\r\n ,\"microsoft.importexport/jobs\": { \"SingularDisplayName\": \"Microsoft.ImportExport job\" }\r\n ,\"microsoft.insights/actiongroups\": { \"SingularDisplayName\": \"Action group\" }\r\n ,\"microsoft.insights/activitylogalerts\": { \"SingularDisplayName\": \"Activity log alert rule\" }\r\n ,\"microsoft.insights/alertrules\": { \"SingularDisplayName\": \"Microsoft.Insights alertrule\" }\r\n ,\"microsoft.insights/alertrules/incidents\": { \"SingularDisplayName\": \"Microsoft.insights alertrules incident\" }\r\n ,\"microsoft.insights/autoscalesettings\": { \"SingularDisplayName\": \"Microsoft.Insights autoscalesetting\" }\r\n ,\"microsoft.insights/components\": { \"SingularDisplayName\": \"Application Insights app\" }\r\n ,\"microsoft.insights/datacollectionendpoints\": { \"SingularDisplayName\": \"Data collection endpoint\" }\r\n ,\"microsoft.insights/datacollectionruleassociations\": { \"SingularDisplayName\": \"Microsoft.Insights data collection rule association\" }\r\n ,\"microsoft.insights/datacollectionrules\": { \"SingularDisplayName\": \"Data collection rule\" }\r\n ,\"microsoft.insights/datacollectionrulesresources\": { \"SingularDisplayName\": \"Data collection rule associated resource\" }\r\n ,\"microsoft.insights/diagnosticsettings\": { \"SingularDisplayName\": \"Diagnostic settings\" }\r\n ,\"microsoft.insights/diagnosticsettingscategories\": { \"SingularDisplayName\": \"Microsoft.Insights diagnostic settings category\" }\r\n ,\"microsoft.insights/guestdiagnosticsettings\": { \"SingularDisplayName\": \"Microsoft.insights guest diagnostic setting\" }\r\n ,\"microsoft.insights/guestdiagnosticsettingsassociation\": { \"SingularDisplayName\": \"Microsoft.insights guest diagnostic settings association\" }\r\n ,\"microsoft.insights/logprofiles\": { \"SingularDisplayName\": \"Microsoft.Insights logprofile\" }\r\n ,\"microsoft.insights/metricalerts\": { \"SingularDisplayName\": \"Metric alert rule\" }\r\n ,\"microsoft.insights/notificationstatus\": { \"SingularDisplayName\": \"Microsoft.Insights notification statu\" }\r\n ,\"microsoft.insights/privatelinkscopeoperationstatuses\": { \"SingularDisplayName\": \"Microsoft.insights private link scope operation statuse\" }\r\n ,\"microsoft.insights/privatelinkscopes\": { \"SingularDisplayName\": \"Azure Monitor Private Link Scope\" }\r\n ,\"microsoft.insights/scheduledqueryrules\": { \"SingularDisplayName\": \"Log search alert rule\" }\r\n ,\"microsoft.insights/tenantactiongroups\": { \"SingularDisplayName\": \"Microsoft.Insights tenant action group\" }\r\n ,\"microsoft.insights/tenantactiongroups/notificationstatus\": { \"SingularDisplayName\": \"Microsoft.Insights tenant action groups notification statu\" }\r\n ,\"microsoft.insights/vminsightsonboardingstatuses\": { \"SingularDisplayName\": \"Microsoft.Insights VM insights onboarding statuse\" }\r\n ,\"microsoft.insights/webtests\": { \"SingularDisplayName\": \"Application Insights availability test\" }\r\n ,\"microsoft.insights/workbooks\": { \"SingularDisplayName\": \"Azure Workbook\" }\r\n ,\"microsoft.insights/workbooktemplates\": { \"SingularDisplayName\": \"Azure Workbook Template\" }\r\n ,\"microsoft.integrationspaces/spaces\": { \"SingularDisplayName\": \"Integration Environment\" }\r\n ,\"microsoft.intelligentitdigitaltwin/digitaltwins\": { \"SingularDisplayName\": \"Microsoft.IntelligentITDigitalTwin digital twin\" }\r\n ,\"microsoft.intelligentitdigitaltwin/digitaltwins/assets\": { \"SingularDisplayName\": \"Microsoft.IntelligentITDigitalTwin digital twins asset\" }\r\n ,\"microsoft.intelligentitdigitaltwin/digitaltwins/executionplans\": { \"SingularDisplayName\": \"Microsoft.IntelligentITDigitalTwin digital twins execution plan\" }\r\n ,\"microsoft.intelligentitdigitaltwin/digitaltwins/testplans\": { \"SingularDisplayName\": \"Microsoft.IntelligentITDigitalTwin digital twins test plan\" }\r\n ,\"microsoft.intelligentitdigitaltwin/digitaltwins/tests\": { \"SingularDisplayName\": \"Microsoft.IntelligentITDigitalTwin digital twins test\" }\r\n ,\"microsoft.inventory/subscriptioninternalproperties\": { \"SingularDisplayName\": \"Microsoft.Inventory subscription internal property\" }\r\n ,\"microsoft.iotcentral/iotapps\": { \"SingularDisplayName\": \"IoT Central Application\" }\r\n ,\"microsoft.iotfirmwaredefense/workspaces\": { \"SingularDisplayName\": \"Firmware analysis workspace\" }\r\n ,\"microsoft.iotfirmwaredefense/workspaces/firmwares\": { \"SingularDisplayName\": \"Microsoft.IoTFirmwareDefense workspaces firmware\" }\r\n ,\"microsoft.iotfirmwaredefense/workspaces/firmwares/summaries\": { \"SingularDisplayName\": \"Microsoft.IoTFirmwareDefense workspaces firmwares summary\" }\r\n ,\"microsoft.iotoperations/instances\": { \"SingularDisplayName\": \"Azure IoT Operations\" }\r\n ,\"microsoft.iotoperations/instances/brokers\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances broker\" }\r\n ,\"microsoft.iotoperations/instances/brokers/authentications\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances brokers authentication\" }\r\n ,\"microsoft.iotoperations/instances/brokers/authorizations\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances brokers authorization\" }\r\n ,\"microsoft.iotoperations/instances/brokers/listeners\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances brokers listener\" }\r\n ,\"microsoft.iotoperations/instances/dataflowendpoints\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances dataflow endpoint\" }\r\n ,\"microsoft.iotoperations/instances/dataflowprofiles\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances dataflow profile\" }\r\n ,\"microsoft.iotoperations/instances/dataflowprofiles/dataflows\": { \"SingularDisplayName\": \"Microsoft.IoTOperations instances dataflow profiles dataflow\" }\r\n ,\"microsoft.iotoperationsdataprocessor/instances\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsDataProcessor instance\" }\r\n ,\"microsoft.iotoperationsdataprocessor/instances/datasets\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsDataProcessor instances dataset\" }\r\n ,\"microsoft.iotoperationsdataprocessor/instances/pipelines\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsDataProcessor instances pipeline\" }\r\n ,\"microsoft.iotoperationsmq/mq\": { \"SingularDisplayName\": \"IoT Operations Ops MQ\" }\r\n ,\"microsoft.iotoperationsmq/mq/broker\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq broker\" }\r\n ,\"microsoft.iotoperationsmq/mq/broker/authentication\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq broker authentication\" }\r\n ,\"microsoft.iotoperationsmq/mq/broker/authorization\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq broker authorization\" }\r\n ,\"microsoft.iotoperationsmq/mq/broker/listener\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq broker listener\" }\r\n ,\"microsoft.iotoperationsmq/mq/datalakeconnector\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq data lake connector\" }\r\n ,\"microsoft.iotoperationsmq/mq/datalakeconnector/topicmap\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq data lake connector topic map\" }\r\n ,\"microsoft.iotoperationsmq/mq/diagnosticservice\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq diagnostic service\" }\r\n ,\"microsoft.iotoperationsmq/mq/kafkaconnector\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq kafka connector\" }\r\n ,\"microsoft.iotoperationsmq/mq/kafkaconnector/topicmap\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq kafka connector topic map\" }\r\n ,\"microsoft.iotoperationsmq/mq/mqttbridgeconnector\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq mqtt bridge connector\" }\r\n ,\"microsoft.iotoperationsmq/mq/mqttbridgeconnector/topicmap\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsMQ mq mqtt bridge connector topic map\" }\r\n ,\"microsoft.iotoperationsorchestrator/instances\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsOrchestrator instance\" }\r\n ,\"microsoft.iotoperationsorchestrator/solutions\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsOrchestrator solution\" }\r\n ,\"microsoft.iotoperationsorchestrator/targets\": { \"SingularDisplayName\": \"Microsoft.IoTOperationsOrchestrator target\" }\r\n ,\"microsoft.iotsecurity/alerttypes\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity alert type\" }\r\n ,\"microsoft.iotsecurity/defendersettings\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity defender setting\" }\r\n ,\"microsoft.iotsecurity/onpremisesensors\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity on premise sensor\" }\r\n ,\"microsoft.iotsecurity/recommendationtypes\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity recommendation type\" }\r\n ,\"microsoft.iotsecurity/sensors\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity sensor\" }\r\n ,\"microsoft.iotsecurity/sites\": { \"SingularDisplayName\": \"Microsoft.IoTSecurity site\" }\r\n ,\"microsoft.keyvault/managedhsms\": { \"SingularDisplayName\": \"Azure Key Vault Managed HSM\" }\r\n ,\"microsoft.keyvault/vaults\": { \"SingularDisplayName\": \"Key vault\" }\r\n ,\"microsoft.kubernetes/connectedclusters\": { \"SingularDisplayName\": \"Kubernetes - Azure Arc\" }\r\n ,\"microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/extensions\": { \"SingularDisplayName\": \"Kubernetes - Azure Arc extension\" }\r\n ,\"microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/fluxconfigurations\": { \"SingularDisplayName\": \"GitOps configuration\" }\r\n ,\"microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/namespaces\": { \"SingularDisplayName\": \"Kubernetes - Azure Arc namespace\" }\r\n ,\"microsoft.kubernetesconfiguration/extensions\": { \"SingularDisplayName\": \"Kubernetes service extension\" }\r\n ,\"microsoft.kubernetesconfiguration/extensiontypes\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration extension type\" }\r\n ,\"microsoft.kubernetesconfiguration/extensiontypes/versions\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration extension types version\" }\r\n ,\"microsoft.kubernetesconfiguration/fluxconfigurations\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration flux configuration\" }\r\n ,\"microsoft.kubernetesconfiguration/fluxconfigurations/operations\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration flux configurations operation\" }\r\n ,\"microsoft.kubernetesconfiguration/privatelinkscopes\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration private link scope\" }\r\n ,\"microsoft.kubernetesconfiguration/privatelinkscopes/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration private link scopes private endpoint connection\" }\r\n ,\"microsoft.kubernetesconfiguration/privatelinkscopes/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration private link scopes private link resource\" }\r\n ,\"microsoft.kubernetesconfiguration/sourcecontrolconfigurations\": { \"SingularDisplayName\": \"Microsoft.KubernetesConfiguration source control configuration\" }\r\n ,\"microsoft.kubernetesruntime/bgppeers\": { \"SingularDisplayName\": \"Microsoft.KubernetesRuntime bgp peer\" }\r\n ,\"microsoft.kubernetesruntime/loadbalancers\": { \"SingularDisplayName\": \"Arc Load Balancer\" }\r\n ,\"microsoft.kubernetesruntime/services\": { \"SingularDisplayName\": \"Microsoft.KubernetesRuntime service\" }\r\n ,\"microsoft.kubernetesruntime/storageclasses\": { \"SingularDisplayName\": \"Microsoft.KubernetesRuntime storage class\" }\r\n ,\"microsoft.kusto/clusters\": { \"SingularDisplayName\": \"Azure Data Explorer Cluster\" }\r\n ,\"microsoft.kusto/clusters/databases\": { \"SingularDisplayName\": \"Azure Data Explorer Database\" }\r\n ,\"microsoft.labservices/labaccounts\": { \"SingularDisplayName\": \"Lab account\" }\r\n ,\"microsoft.labservices/labaccounts/labs\": { \"SingularDisplayName\": \"Lab\" }\r\n ,\"microsoft.labservices/labplans\": { \"SingularDisplayName\": \"Lab plan\" }\r\n ,\"microsoft.labservices/labs\": { \"SingularDisplayName\": \"Lab\" }\r\n ,\"microsoft.liftrpilot/organizations\": { \"SingularDisplayName\": \"Azure Pilot\" }\r\n ,\"microsoft.loadtestservice/loadtestmappings\": { \"SingularDisplayName\": \"Microsoft.LoadTestService load test mapping\" }\r\n ,\"microsoft.loadtestservice/loadtestprofilemappings\": { \"SingularDisplayName\": \"Microsoft.LoadTestService load test profile mapping\" }\r\n ,\"microsoft.loadtestservice/loadtests\": { \"SingularDisplayName\": \"Azure Load Testing\" }\r\n ,\"microsoft.loadtestservice/playwrightworkspaces\": { \"SingularDisplayName\": \"Playwright Workspace\" }\r\n ,\"microsoft.logic/businessprocesses\": { \"SingularDisplayName\": \"Business Process\" }\r\n ,\"microsoft.logic/integrationaccounts\": { \"SingularDisplayName\": \"Logic app integration account\" }\r\n ,\"microsoft.logic/integrationserviceenvironments\": { \"SingularDisplayName\": \"Integration Service Environment\" }\r\n ,\"microsoft.logic/integrationserviceenvironments/health\": { \"SingularDisplayName\": \"Microsoft.Logic integration service environments health\" }\r\n ,\"microsoft.logic/integrationserviceenvironments/managedapis\": { \"SingularDisplayName\": \"Managed Connector\" }\r\n ,\"microsoft.logic/templates\": { \"SingularDisplayName\": \"Logic Apps Template\" }\r\n ,\"microsoft.logic/workflows\": { \"SingularDisplayName\": \"Logic app\" }\r\n ,\"microsoft.logz/monitors\": { \"SingularDisplayName\": \"Logz.io\" }\r\n ,\"microsoft.logz/monitors/accounts\": { \"SingularDisplayName\": \"Logz sub account\" }\r\n ,\"microsoft.m365/m365resources\": { \"SingularDisplayName\": \"Microsoft.M365 m365 resource\" }\r\n ,\"microsoft.m365consumptionservices/services\": { \"SingularDisplayName\": \"Microsoft.M365ConsumptionServices service\" }\r\n ,\"microsoft.machinelearning/commitmentplans\": { \"SingularDisplayName\": \"Microsoft.MachineLearning commitment plan\" }\r\n ,\"microsoft.machinelearning/commitmentplans/commitmentassociations\": { \"SingularDisplayName\": \"Microsoft.MachineLearning commitment plans commitment association\" }\r\n ,\"microsoft.machinelearning/webservices\": { \"SingularDisplayName\": \"Machine Learning Studio (classic) web service\" }\r\n ,\"microsoft.machinelearning/workspaces\": { \"SingularDisplayName\": \"Machine Learning Studio (classic) workspace\" }\r\n ,\"microsoft.machinelearningexperimentation/accounts\": { \"SingularDisplayName\": \"Microsoft.MachineLearningExperimentation account\" }\r\n ,\"microsoft.machinelearningexperimentation/accounts/workspaces\": { \"SingularDisplayName\": \"Microsoft.MachineLearningExperimentation accounts workspace\" }\r\n ,\"microsoft.machinelearningexperimentation/accounts/workspaces/projects\": { \"SingularDisplayName\": \"Microsoft.MachineLearningExperimentation accounts workspaces project\" }\r\n ,\"microsoft.machinelearningservices/aistudio\": { \"SingularDisplayName\": \"Azure AI Foundry\" }\r\n ,\"microsoft.machinelearningservices/aistudiocreate\": { \"SingularDisplayName\": \"Azure AI Foundry\" }\r\n ,\"microsoft.machinelearningservices/registries\": { \"SingularDisplayName\": \"Azure Machine Learning registry\" }\r\n ,\"microsoft.machinelearningservices/workspaces\": { \"SingularDisplayName\": \"Azure Machine Learning workspace\" }\r\n ,\"microsoft.machinelearningservices/workspaces/onlineendpoints\": { \"SingularDisplayName\": \"Machine learning online endpoint\" }\r\n ,\"microsoft.machinelearningservices/workspaces/onlineendpoints/deployments\": { \"SingularDisplayName\": \"Machine learning online deployment\" }\r\n ,\"microsoft.machinelearningservices/workspacescreate\": { \"SingularDisplayName\": \"Azure Machine Learning workspace\" }\r\n ,\"microsoft.maintenance/configurationassignments\": { \"SingularDisplayName\": \"Microsoft.Maintenance configuration assignment\" }\r\n ,\"microsoft.maintenance/maintenanceconfigurations\": { \"SingularDisplayName\": \"Maintenance Configuration\" }\r\n ,\"microsoft.maintenance/maintenanceconfigurationsaumbladeresource\": { \"SingularDisplayName\": \"Maintenance configuration\" }\r\n ,\"microsoft.maintenance/maintenanceconfigurationsbladeresource\": { \"SingularDisplayName\": \"Maintenance configuration\" }\r\n ,\"microsoft.maintenance/publicmaintenanceconfigurations\": { \"SingularDisplayName\": \"Microsoft.Maintenance public maintenance configuration\" }\r\n ,\"microsoft.managedidentity/identities\": { \"SingularDisplayName\": \"Microsoft.ManagedIdentity identity\" }\r\n ,\"microsoft.managedidentity/userassignedidentities\": { \"SingularDisplayName\": \"Managed Identity\" }\r\n ,\"microsoft.managednetwork/managednetworks\": { \"SingularDisplayName\": \"Microsoft.ManagedNetwork managed network\" }\r\n ,\"microsoft.managednetwork/managednetworks/managednetworkgroups\": { \"SingularDisplayName\": \"Microsoft.ManagedNetwork managed networks managed network group\" }\r\n ,\"microsoft.managednetwork/managednetworks/managednetworkpeeringpolicies\": { \"SingularDisplayName\": \"Microsoft.ManagedNetwork managed networks managed network peering policy\" }\r\n ,\"microsoft.managednetworkfabric/accesscontrollists\": { \"SingularDisplayName\": \"Access Control List (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/internetgatewayrules\": { \"SingularDisplayName\": \"Internet Gateway Rule (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/internetgateways\": { \"SingularDisplayName\": \"Internet Gateway (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/ipcommunities\": { \"SingularDisplayName\": \"IP Community (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/ipextendedcommunities\": { \"SingularDisplayName\": \"IP Extended Community (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/ipprefixes\": { \"SingularDisplayName\": \"IP Prefix (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/l2isolationdomains\": { \"SingularDisplayName\": \"Layer 2 Isolation Domain (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/l3isolationdomains\": { \"SingularDisplayName\": \"Layer 3 Isolation Domain (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/l3isolationdomains/externalnetworks\": { \"SingularDisplayName\": \"External Network (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/l3isolationdomains/internalnetworks\": { \"SingularDisplayName\": \"Internal Network (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/neighborgroups\": { \"SingularDisplayName\": \"Neighbor Group (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkdevices\": { \"SingularDisplayName\": \"Network Device (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkdevices/networkinterfaces\": { \"SingularDisplayName\": \"Network Interface (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkfabriccontrollers\": { \"SingularDisplayName\": \"Network Fabric Controller (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkfabrics\": { \"SingularDisplayName\": \"Network Fabric (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkfabrics/networktonetworkinterconnects\": { \"SingularDisplayName\": \"Network to Network Interconnect (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkfabricskus\": { \"SingularDisplayName\": \"Network Fabric SKU (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkmonitors\": { \"SingularDisplayName\": \"Microsoft.ManagedNetworkFabric network monitor\" }\r\n ,\"microsoft.managednetworkfabric/networkpacketbrokers\": { \"SingularDisplayName\": \"Network Packet Broker (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networkracks\": { \"SingularDisplayName\": \"Network Rack (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networktaprules\": { \"SingularDisplayName\": \"Network Tap Rule (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/networktaps\": { \"SingularDisplayName\": \"Network Tap (Operator Nexus)\" }\r\n ,\"microsoft.managednetworkfabric/routepolicies\": { \"SingularDisplayName\": \"Route Policy (Operator Nexus)\" }\r\n ,\"microsoft.managedservices/marketplaceregistrationdefinitions\": { \"SingularDisplayName\": \"Microsoft.ManagedServices marketplace registration definition\" }\r\n ,\"microsoft.managedservices/registrationassignments\": { \"SingularDisplayName\": \"Microsoft.ManagedServices registration assignment\" }\r\n ,\"microsoft.managedservices/registrationdefinitions\": { \"SingularDisplayName\": \"Azure Lighthouse\" }\r\n ,\"microsoft.management/managementgroups\": { \"SingularDisplayName\": \"Microsoft.Management management group\" }\r\n ,\"microsoft.management/managementgroups/microsoft.resources/deploymentstacks\": { \"SingularDisplayName\": \"Deployment stack\" }\r\n ,\"microsoft.management/managementgroups/providers/privatelinkassociations\": { \"SingularDisplayName\": \"Application Gateway\" }\r\n ,\"microsoft.management/managementgroups/providers/templatespecs\": { \"SingularDisplayName\": \"Template spec\" }\r\n ,\"microsoft.management/managementgroups/settings\": { \"SingularDisplayName\": \"Microsoft.Management management groups setting\" }\r\n ,\"microsoft.management/managementgroups/subscriptions\": { \"SingularDisplayName\": \"Microsoft.Management management groups subscription\" }\r\n ,\"microsoft.management/servicegroups\": { \"SingularDisplayName\": \"Service group\" }\r\n ,\"microsoft.managementpartner/partners\": { \"SingularDisplayName\": \"Microsoft.ManagementPartner partner\" }\r\n ,\"microsoft.manufacturingplatform/manufacturingdataservices\": { \"SingularDisplayName\": \"Factory Operations Agent in Azure AI Foundry\" }\r\n ,\"microsoft.maps/accounts\": { \"SingularDisplayName\": \"Azure Maps Account\" }\r\n ,\"microsoft.maps/accounts/creators\": { \"SingularDisplayName\": \"Azure Maps Creator Resource\" }\r\n ,\"microsoft.marketplace/privatestores\": { \"SingularDisplayName\": \"Microsoft.Marketplace private store\" }\r\n ,\"microsoft.marketplace/privatestores/adminrequestapprovals\": { \"SingularDisplayName\": \"Microsoft.Marketplace private stores admin request approval\" }\r\n ,\"microsoft.marketplace/privatestores/collections\": { \"SingularDisplayName\": \"Microsoft.Marketplace private stores collection\" }\r\n ,\"microsoft.marketplace/privatestores/collections/offers\": { \"SingularDisplayName\": \"Microsoft.Marketplace private stores collections offer\" }\r\n ,\"microsoft.marketplace/privatestores/offers\": { \"SingularDisplayName\": \"Microsoft.Marketplace private stores offer\" }\r\n ,\"microsoft.marketplace/privatestores/requestapprovals\": { \"SingularDisplayName\": \"Microsoft.Marketplace private stores request approval\" }\r\n ,\"microsoft.media/mediaservices\": { \"SingularDisplayName\": \"Media service\" }\r\n ,\"microsoft.media/mediaservices/accountfilters\": { \"SingularDisplayName\": \"Microsoft.Media media services account filter\" }\r\n ,\"microsoft.media/mediaservices/assets\": { \"SingularDisplayName\": \"Microsoft.Media media services asset\" }\r\n ,\"microsoft.media/mediaservices/assets/assetfilters\": { \"SingularDisplayName\": \"Microsoft.Media media services assets asset filter\" }\r\n ,\"microsoft.media/mediaservices/assets/tracks\": { \"SingularDisplayName\": \"Microsoft.Media media services assets track\" }\r\n ,\"microsoft.media/mediaservices/assets/tracks/operationresults\": { \"SingularDisplayName\": \"Microsoft.Media media services assets tracks operation result\" }\r\n ,\"microsoft.media/mediaservices/assets/tracks/operationstatuses\": { \"SingularDisplayName\": \"Microsoft.Media media services assets tracks operation statuse\" }\r\n ,\"microsoft.media/mediaservices/contentkeypolicies\": { \"SingularDisplayName\": \"Microsoft.Media media services content key policy\" }\r\n ,\"microsoft.media/mediaservices/liveevents\": { \"SingularDisplayName\": \"Live event\" }\r\n ,\"microsoft.media/mediaservices/liveevents/liveoutputs\": { \"SingularDisplayName\": \"Microsoft.Media mediaservices live events live output\" }\r\n ,\"microsoft.media/mediaservices/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.Media mediaservices private endpoint connection\" }\r\n ,\"microsoft.media/mediaservices/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.Media mediaservices private link resource\" }\r\n ,\"microsoft.media/mediaservices/streamingendpoints\": { \"SingularDisplayName\": \"Streaming Endpoint\" }\r\n ,\"microsoft.media/mediaservices/streaminglocators\": { \"SingularDisplayName\": \"Microsoft.Media media services streaming locator\" }\r\n ,\"microsoft.media/mediaservices/streamingpolicies\": { \"SingularDisplayName\": \"Microsoft.Media media services streaming policy\" }\r\n ,\"microsoft.media/mediaservices/transforms\": { \"SingularDisplayName\": \"Microsoft.Media media services transform\" }\r\n ,\"microsoft.media/mediaservices/transforms/jobs\": { \"SingularDisplayName\": \"Microsoft.Media media services transforms job\" }\r\n ,\"microsoft.mesh/worlds\": { \"SingularDisplayName\": \"Microsoft.Mesh world\" }\r\n ,\"microsoft.mesh/worlds/events\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds event\" }\r\n ,\"microsoft.mesh/worlds/events/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds events access policy\" }\r\n ,\"microsoft.mesh/worlds/spaces\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds space\" }\r\n ,\"microsoft.mesh/worlds/spaces/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds spaces access policy\" }\r\n ,\"microsoft.mesh/worlds/templates\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds template\" }\r\n ,\"microsoft.mesh/worlds/templates/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Mesh worlds templates access policy\" }\r\n ,\"microsoft.messagingcatalog/catalogs\": { \"SingularDisplayName\": \"Microsoft.MessagingCatalog catalog\" }\r\n ,\"microsoft.messagingconnectors/connectors\": { \"SingularDisplayName\": \"Microsoft.MessagingConnectors connector\" }\r\n ,\"microsoft.metaverse/metaverses\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverse\" }\r\n ,\"microsoft.metaverse/metaverses/events\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses event\" }\r\n ,\"microsoft.metaverse/metaverses/events/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses events access policy\" }\r\n ,\"microsoft.metaverse/metaverses/spaces\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses space\" }\r\n ,\"microsoft.metaverse/metaverses/spaces/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses spaces access policy\" }\r\n ,\"microsoft.metaverse/metaverses/templates\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses template\" }\r\n ,\"microsoft.metaverse/metaverses/templates/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.Metaverse metaverses templates access policy\" }\r\n ,\"microsoft.migrate/assessmentprojects\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment project\" }\r\n ,\"microsoft.migrate/assessmentprojects/aksassessmentoptions\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects aks assessment option\" }\r\n ,\"microsoft.migrate/assessmentprojects/aksassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects aks assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/aksassessments/assessedwebapps\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects aks assessments assessed web app\" }\r\n ,\"microsoft.migrate/assessmentprojects/aksassessments/clusters\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects aks assessments cluster\" }\r\n ,\"microsoft.migrate/assessmentprojects/aksassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects aks assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/assessmentoptions\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects assessment option\" }\r\n ,\"microsoft.migrate/assessmentprojects/assessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/assessments/assessedmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects assessments assessed machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/assessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/avsassessmentoptions\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects avs assessment option\" }\r\n ,\"microsoft.migrate/assessmentprojects/avsassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects avs assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/avsassessments/avsassessedmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects avs assessments avs assessed machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/avsassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects avs assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business case\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/avssummaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases avs summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/evaluatedavsmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases evaluated avs machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/evaluatedmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases evaluated machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/evaluatedsqlentities\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases evaluated sql entity\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/evaluatedwebapps\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases evaluated web app\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/iaassummaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases iaas summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/overviewsummaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases overview summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/businesscases/paassummaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects business cases paas summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects group\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/assessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/assessments/assessedmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups assessments assessed machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/avsassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups avs assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/avsassessments/avsassessedmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups avs assessments avs assessed machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments/assessedsqldatabases\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessments assessed sql database\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments/assessedsqlinstances\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessments assessed sql instance\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments/assessedsqlmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessments assessed sql machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments/recommendedassessedentities\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessments recommended assessed entity\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/sqlassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups sql assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/webappassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups web app assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/webappassessments/assessedwebapps\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups web app assessments assessed web app\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/webappassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups web app assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/groups/webappassessments/webappserviceplans\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects groups web app assessments web app service plan\" }\r\n ,\"microsoft.migrate/assessmentprojects/heterogeneousassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects heterogeneous assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/heterogeneousassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects heterogeneous assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/hypervcollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects hypervcollector\" }\r\n ,\"microsoft.migrate/assessmentprojects/importcollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects importcollector\" }\r\n ,\"microsoft.migrate/assessmentprojects/importsqlcollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects import sql collector\" }\r\n ,\"microsoft.migrate/assessmentprojects/machines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects private endpoint connection\" }\r\n ,\"microsoft.migrate/assessmentprojects/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects private link resource\" }\r\n ,\"microsoft.migrate/assessmentprojects/projectsummary\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects project summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/servercollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects servercollector\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessmentoptions\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessment option\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessments/assessedsqldatabases\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessments assessed sql database\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessments/assessedsqlinstances\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessments assessed sql instance\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessments/assessedsqlmachines\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessments assessed sql machine\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sql assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/sqlcollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects sqlcollector\" }\r\n ,\"microsoft.migrate/assessmentprojects/vmwarecollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects vmwarecollector\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappassessmentoptions\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app assessment option\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappassessments/assessedwebapps\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app assessments assessed web app\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app assessments summary\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappassessments/webappserviceplans\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app assessments web app service plan\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappcollectors\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app collector\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappcompoundassessments\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app compound assessment\" }\r\n ,\"microsoft.migrate/assessmentprojects/webappcompoundassessments/summaries\": { \"SingularDisplayName\": \"Microsoft.Migrate assessment projects web app compound assessments summary\" }\r\n ,\"microsoft.migrate/migrateprojects\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate project\" }\r\n ,\"microsoft.migrate/migrateprojects/databaseinstances\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate projects database instance\" }\r\n ,\"microsoft.migrate/migrateprojects/databases\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate projects database\" }\r\n ,\"microsoft.migrate/migrateprojects/machines\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate projects machine\" }\r\n ,\"microsoft.migrate/migrateprojects/migrateevents\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate projects migrate event\" }\r\n ,\"microsoft.migrate/migrateprojects/solutions\": { \"SingularDisplayName\": \"Microsoft.Migrate migrate projects solution\" }\r\n ,\"microsoft.migrate/modernizeprojects\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize project\" }\r\n ,\"microsoft.migrate/modernizeprojects/deployedresources\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects deployed resource\" }\r\n ,\"microsoft.migrate/modernizeprojects/jobs\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects job\" }\r\n ,\"microsoft.migrate/modernizeprojects/jobs/operations\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects jobs operation\" }\r\n ,\"microsoft.migrate/modernizeprojects/migrateagents\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects migrate agent\" }\r\n ,\"microsoft.migrate/modernizeprojects/migrateagents/operations\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects migrate agents operation\" }\r\n ,\"microsoft.migrate/modernizeprojects/operations\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects operation\" }\r\n ,\"microsoft.migrate/modernizeprojects/workloaddeployments\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects workload deployment\" }\r\n ,\"microsoft.migrate/modernizeprojects/workloaddeployments/operations\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects workload deployments operation\" }\r\n ,\"microsoft.migrate/modernizeprojects/workloadinstances\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects workload instance\" }\r\n ,\"microsoft.migrate/modernizeprojects/workloadinstances/operations\": { \"SingularDisplayName\": \"Microsoft.Migrate modernize projects workload instances operation\" }\r\n ,\"microsoft.migrate/movecollections\": { \"SingularDisplayName\": \"Microsoft.Migrate move collection\" }\r\n ,\"microsoft.migrate/movecollections/moveresources\": { \"SingularDisplayName\": \"Microsoft.Migrate move collections move resource\" }\r\n ,\"microsoft.migrate/projects\": { \"SingularDisplayName\": \"Migration project\" }\r\n ,\"microsoft.mission/approvals\": { \"SingularDisplayName\": \"Approval\" }\r\n ,\"microsoft.mission/catalogs\": { \"SingularDisplayName\": \"Catalog\" }\r\n ,\"microsoft.mission/communities\": { \"SingularDisplayName\": \"Community\" }\r\n ,\"microsoft.mission/communities/communityendpoints\": { \"SingularDisplayName\": \"Community endpoint\" }\r\n ,\"microsoft.mission/communities/transithubs\": { \"SingularDisplayName\": \"Transit hub\" }\r\n ,\"microsoft.mission/enclaveconnections\": { \"SingularDisplayName\": \"Enclave connection\" }\r\n ,\"microsoft.mission/externalconnections\": { \"SingularDisplayName\": \"Microsoft.Mission external connection\" }\r\n ,\"microsoft.mission/internalconnections\": { \"SingularDisplayName\": \"Microsoft.Mission internal connection\" }\r\n ,\"microsoft.mission/virtualenclaves\": { \"SingularDisplayName\": \"Enclave\" }\r\n ,\"microsoft.mission/virtualenclaves/enclaveendpoints\": { \"SingularDisplayName\": \"Enclave endpoint\" }\r\n ,\"microsoft.mission/virtualenclaves/endpoints\": { \"SingularDisplayName\": \"Endpoint\" }\r\n ,\"microsoft.mission/virtualenclaves/workloads\": { \"SingularDisplayName\": \"Workload\" }\r\n ,\"microsoft.mixedreality/objectanchorsaccounts\": { \"SingularDisplayName\": \"Object Anchors Account\" }\r\n ,\"microsoft.mixedreality/objectunderstandingaccounts\": { \"SingularDisplayName\": \"Object Understanding Account\" }\r\n ,\"microsoft.mixedreality/remoterenderingaccounts\": { \"SingularDisplayName\": \"Remote Rendering Account\" }\r\n ,\"microsoft.mixedreality/spatialanchorsaccounts\": { \"SingularDisplayName\": \"Spatial Anchors Account\" }\r\n ,\"microsoft.mixedreality/spatialmapsaccounts\": { \"SingularDisplayName\": \"Microsoft.MixedReality spatial maps account\" }\r\n ,\"microsoft.mobilenetwork/amfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork amf deployment\" }\r\n ,\"microsoft.mobilenetwork/clusterservices\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork cluster service\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks\": { \"SingularDisplayName\": \"Mobile Network\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks/datanetworks\": { \"SingularDisplayName\": \"Data Network\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks/services\": { \"SingularDisplayName\": \"Service\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks/simpolicies\": { \"SingularDisplayName\": \"SIM Policy\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks/sites\": { \"SingularDisplayName\": \"Mobile Network Site\" }\r\n ,\"microsoft.mobilenetwork/mobilenetworks/slices\": { \"SingularDisplayName\": \"Slice\" }\r\n ,\"microsoft.mobilenetwork/nrfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork nrf deployment\" }\r\n ,\"microsoft.mobilenetwork/nssfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork nssf deployment\" }\r\n ,\"microsoft.mobilenetwork/observabilityservices\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork observability service\" }\r\n ,\"microsoft.mobilenetwork/packetcorecontrolplanes\": { \"SingularDisplayName\": \"Packet Core Control Plane\" }\r\n ,\"microsoft.mobilenetwork/packetcorecontrolplanes/packetcoredataplanes\": { \"SingularDisplayName\": \"Packet Core Data Plane\" }\r\n ,\"microsoft.mobilenetwork/packetcorecontrolplanes/packetcoredataplanes/attacheddatanetworks\": { \"SingularDisplayName\": \"Attached Data Network\" }\r\n ,\"microsoft.mobilenetwork/packetcorecontrolplaneversions\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork packet core control plane version\" }\r\n ,\"microsoft.mobilenetwork/radioaccessnetworks\": { \"SingularDisplayName\": \"Radio Access Network Insights\" }\r\n ,\"microsoft.mobilenetwork/sdmdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork sdm deployment\" }\r\n ,\"microsoft.mobilenetwork/simgroups\": { \"SingularDisplayName\": \"SIM Group\" }\r\n ,\"microsoft.mobilenetwork/simgroups/sims\": { \"SingularDisplayName\": \"SIM\" }\r\n ,\"microsoft.mobilenetwork/sims\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork sim\" }\r\n ,\"microsoft.mobilenetwork/smfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork smf deployment\" }\r\n ,\"microsoft.mobilenetwork/upfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork upf deployment\" }\r\n ,\"microsoft.mobilenetwork/virtualizedmmedeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork virtualized mme deployment\" }\r\n ,\"microsoft.mobilenetwork/vnfagentdeployments\": { \"SingularDisplayName\": \"Microsoft.MobileNetwork vnf agent deployment\" }\r\n ,\"microsoft.mobilepacketcore/amfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore amf deployment\" }\r\n ,\"microsoft.mobilepacketcore/clusterservices\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore cluster service\" }\r\n ,\"microsoft.mobilepacketcore/networkfunctions\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore network function\" }\r\n ,\"microsoft.mobilepacketcore/nrfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore nrf deployment\" }\r\n ,\"microsoft.mobilepacketcore/nssfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore nssf deployment\" }\r\n ,\"microsoft.mobilepacketcore/observabilityservices\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore observability service\" }\r\n ,\"microsoft.mobilepacketcore/smfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore smf deployment\" }\r\n ,\"microsoft.mobilepacketcore/upfdeployments\": { \"SingularDisplayName\": \"Microsoft.MobilePacketCore upf deployment\" }\r\n ,\"microsoft.modsimworkbench/workbenches\": { \"SingularDisplayName\": \"Modeling and Simulation Workbench\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers\": { \"SingularDisplayName\": \"Chamber\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/connectors\": { \"SingularDisplayName\": \"Chamber Connector\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/filerequests\": { \"SingularDisplayName\": \"Chamber Data Pipeline File Request\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/files\": { \"SingularDisplayName\": \"Chamber Data Pipeline File\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/licenses\": { \"SingularDisplayName\": \"Chamber License\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/storages\": { \"SingularDisplayName\": \"Chamber Storage\" }\r\n ,\"microsoft.modsimworkbench/workbenches/chambers/workloads\": { \"SingularDisplayName\": \"Chamber VM\" }\r\n ,\"microsoft.modsimworkbench/workbenches/sharedstorages\": { \"SingularDisplayName\": \"Shared Storage\" }\r\n ,\"microsoft.monitor/accounts\": { \"SingularDisplayName\": \"Azure Monitor workspace\" }\r\n ,\"microsoft.monitor/investigations\": { \"SingularDisplayName\": \"Microsoft.Monitor investigation\" }\r\n ,\"microsoft.monitor/pipelinegroups\": { \"SingularDisplayName\": \"Azure Monitor pipeline\" }\r\n ,\"microsoft.mysqldiscovery/mysqlsites\": { \"SingularDisplayName\": \"Microsoft.MySQLDiscovery my sqlsite\" }\r\n ,\"microsoft.mysqldiscovery/mysqlsites/agents\": { \"SingularDisplayName\": \"Microsoft.MySQLDiscovery my sqlsites agent\" }\r\n ,\"microsoft.mysqldiscovery/mysqlsites/errorsummaries\": { \"SingularDisplayName\": \"Microsoft.MySQLDiscovery my sqlsites error summary\" }\r\n ,\"microsoft.mysqldiscovery/mysqlsites/mysqlservers\": { \"SingularDisplayName\": \"Microsoft.MySQLDiscovery my sqlsites my sqlserver\" }\r\n ,\"microsoft.mysqldiscovery/mysqlsites/summaries\": { \"SingularDisplayName\": \"Microsoft.MySQLDiscovery my sqlsites summary\" }\r\n ,\"microsoft.netapp/netappaccounts\": { \"SingularDisplayName\": \"NetApp account\" }\r\n ,\"microsoft.netapp/netappaccounts/backuppolicies\": { \"SingularDisplayName\": \"Backup Policy\" }\r\n ,\"microsoft.netapp/netappaccounts/backupvaults\": { \"SingularDisplayName\": \"Backup vault\" }\r\n ,\"microsoft.netapp/netappaccounts/capacitypools\": { \"SingularDisplayName\": \"Capacity pool\" }\r\n ,\"microsoft.netapp/netappaccounts/capacitypools/volumes\": { \"SingularDisplayName\": \"Volume\" }\r\n ,\"microsoft.netapp/netappaccounts/capacitypools/volumes/snapshots\": { \"SingularDisplayName\": \"Snapshot\" }\r\n ,\"microsoft.netapp/netappaccounts/capacitypools/volumes/volumequotarules\": { \"SingularDisplayName\": \"User and group quota\" }\r\n ,\"microsoft.netapp/netappaccounts/snapshotpolicies\": { \"SingularDisplayName\": \"Snapshot policy\" }\r\n ,\"microsoft.netapp/netappaccounts/volumegroups\": { \"SingularDisplayName\": \"VolumeGroup\" }\r\n ,\"microsoft.network/applicationgatewayavailablessloptions\": { \"SingularDisplayName\": \"Microsoft.Network application gateway available ssl option\" }\r\n ,\"microsoft.network/applicationgatewayavailablessloptions/predefinedpolicies\": { \"SingularDisplayName\": \"Microsoft.Network application gateway available ssl options predefined policy\" }\r\n ,\"microsoft.network/applicationgateways\": { \"SingularDisplayName\": \"Application gateway\" }\r\n ,\"microsoft.network/applicationgatewaywebapplicationfirewallpolicies\": { \"SingularDisplayName\": \"Application Gateway WAF policy\" }\r\n ,\"microsoft.network/applicationsecuritygroups\": { \"SingularDisplayName\": \"Application security group\" }\r\n ,\"microsoft.network/azurefirewalls\": { \"SingularDisplayName\": \"Firewall\" }\r\n ,\"microsoft.network/azurewebcategories\": { \"SingularDisplayName\": \"Microsoft.Network Azure web category\" }\r\n ,\"microsoft.network/bastionhosts\": { \"SingularDisplayName\": \"Bastion\" }\r\n ,\"microsoft.network/cloudserviceslots\": { \"SingularDisplayName\": \"Microsoft.Network cloud service slot\" }\r\n ,\"microsoft.network/connections\": { \"SingularDisplayName\": \"Connection\" }\r\n ,\"microsoft.network/customipprefixes\": { \"SingularDisplayName\": \"Custom IP Prefix\" }\r\n ,\"microsoft.network/ddoscustompolicies\": { \"SingularDisplayName\": \"Microsoft.Network DDoS custom policy\" }\r\n ,\"microsoft.network/ddosprotectionplans\": { \"SingularDisplayName\": \"DDoS protection plan\" }\r\n ,\"microsoft.network/dnsforwardingrulesets\": { \"SingularDisplayName\": \"DNS forwarding ruleset\" }\r\n ,\"microsoft.network/dnsresolverdomainlists\": { \"SingularDisplayName\": \"DNS Domain List\" }\r\n ,\"microsoft.network/dnsresolverpolicies\": { \"SingularDisplayName\": \"DNS Security Policy\" }\r\n ,\"microsoft.network/dnsresolvers\": { \"SingularDisplayName\": \"DNS private resolver\" }\r\n ,\"microsoft.network/dnszones\": { \"SingularDisplayName\": \"DNS zone\" }\r\n ,\"microsoft.network/dscpconfigurations\": { \"SingularDisplayName\": \"Microsoft.Network DSCP configuration\" }\r\n ,\"microsoft.network/expressroutecircuits\": { \"SingularDisplayName\": \"ExpressRoute circuit\" }\r\n ,\"microsoft.network/expressroutecrossconnections\": { \"SingularDisplayName\": \"Microsoft.Network express route cross connection\" }\r\n ,\"microsoft.network/expressroutecrossconnections/peerings\": { \"SingularDisplayName\": \"Microsoft.Network express route cross connections peering\" }\r\n ,\"microsoft.network/expressroutegateways\": { \"SingularDisplayName\": \"ExpressRoute Gateway\" }\r\n ,\"microsoft.network/expressroutegateways/expressrouteconnections\": { \"SingularDisplayName\": \"Microsoft.Network express route gateways express route connection\" }\r\n ,\"microsoft.network/expressrouteports\": { \"SingularDisplayName\": \"ExpressRoute Direct\" }\r\n ,\"microsoft.network/expressrouteportslocations\": { \"SingularDisplayName\": \"Microsoft.Network express route ports location\" }\r\n ,\"microsoft.network/firewallpolicies\": { \"SingularDisplayName\": \"Firewall Policy\" }\r\n ,\"microsoft.network/frontdoors\": { \"SingularDisplayName\": \"Front Door and CDN profiles\" }\r\n ,\"microsoft.network/frontdoorwebapplicationfirewallpolicies\": { \"SingularDisplayName\": \"Front Door WAF policy\" }\r\n ,\"microsoft.network/ipallocations\": { \"SingularDisplayName\": \"Microsoft.Network IP allocation\" }\r\n ,\"microsoft.network/ipgroups\": { \"SingularDisplayName\": \"IP Group\" }\r\n ,\"microsoft.network/loadbalancers\": { \"SingularDisplayName\": \"Load balancer\" }\r\n ,\"microsoft.network/localnetworkgateways\": { \"SingularDisplayName\": \"Local network gateway\" }\r\n ,\"microsoft.network/natgateways\": { \"SingularDisplayName\": \"NAT gateway\" }\r\n ,\"microsoft.network/networkexperimentprofiles\": { \"SingularDisplayName\": \"Microsoft.Network network experiment profile\" }\r\n ,\"microsoft.network/networkexperimentprofiles/experiments\": { \"SingularDisplayName\": \"Microsoft.Network network experiment profiles experiment\" }\r\n ,\"microsoft.network/networkinterfaces\": { \"SingularDisplayName\": \"Network interface\" }\r\n ,\"microsoft.network/networkmanagerconnections\": { \"SingularDisplayName\": \"Microsoft.Network network manager connection\" }\r\n ,\"microsoft.network/networkmanagers\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/connectivityconfigurations\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/ipampools\": { \"SingularDisplayName\": \"IP address pool\" }\r\n ,\"microsoft.network/networkmanagers/networkgroups\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/routingconfigurations\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/securityadminconfigurations\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/securityuserconfigurations\": { \"SingularDisplayName\": \"Network manager\" }\r\n ,\"microsoft.network/networkmanagers/verifierworkspaces\": { \"SingularDisplayName\": \"Verifier Workspace\" }\r\n ,\"microsoft.network/networkprofiles\": { \"SingularDisplayName\": \"Microsoft.Network network profile\" }\r\n ,\"microsoft.network/networksecuritygroups\": { \"SingularDisplayName\": \"Network security group\" }\r\n ,\"microsoft.network/networksecurityperimeters\": { \"SingularDisplayName\": \"Network Security Perimeter\" }\r\n ,\"microsoft.network/networksecurityperimeters/profiles\": { \"SingularDisplayName\": \"Network Security Perimeter Profile\" }\r\n ,\"microsoft.network/networkverifiers\": { \"SingularDisplayName\": \"Virtual Network Verifier\" }\r\n ,\"microsoft.network/networkvirtualappliances\": { \"SingularDisplayName\": \"Microsoft.Network network virtual appliance\" }\r\n ,\"microsoft.network/networkwatchers\": { \"SingularDisplayName\": \"Network Watcher\" }\r\n ,\"microsoft.network/networkwatchers/flowlogs\": { \"SingularDisplayName\": \"Flow log\" }\r\n ,\"microsoft.network/p2svpngateways\": { \"SingularDisplayName\": \"VPN Gateway (Point to Site)\" }\r\n ,\"microsoft.network/privatednszones\": { \"SingularDisplayName\": \"Private DNS zone\" }\r\n ,\"microsoft.network/privatednszones/virtualnetworklinks\": { \"SingularDisplayName\": \"Virtual network link\" }\r\n ,\"microsoft.network/privateendpoints\": { \"SingularDisplayName\": \"Private endpoint\" }\r\n ,\"microsoft.network/privatelinkservices\": { \"SingularDisplayName\": \"Private link service\" }\r\n ,\"microsoft.network/publicipaddresses\": { \"SingularDisplayName\": \"Public IP address\" }\r\n ,\"microsoft.network/publicipprefixes\": { \"SingularDisplayName\": \"Public IP Prefix\" }\r\n ,\"microsoft.network/routefilters\": { \"SingularDisplayName\": \"Route filter\" }\r\n ,\"microsoft.network/routetables\": { \"SingularDisplayName\": \"Route table\" }\r\n ,\"microsoft.network/securitypartnerproviders\": { \"SingularDisplayName\": \"Microsoft.Network security partner provider\" }\r\n ,\"microsoft.network/serviceendpointpolicies\": { \"SingularDisplayName\": \"Service endpoint policy\" }\r\n ,\"microsoft.network/trafficmanagergeographichierarchies\": { \"SingularDisplayName\": \"Microsoft.Network traffic manager geographic hierarchy\" }\r\n ,\"microsoft.network/trafficmanagerprofiles\": { \"SingularDisplayName\": \"Traffic Manager profile\" }\r\n ,\"microsoft.network/trafficmanagerusermetricskeys\": { \"SingularDisplayName\": \"Microsoft.Network traffic manager user metrics key\" }\r\n ,\"microsoft.network/virtualhubs\": { \"SingularDisplayName\": \"Microsoft.Network/virtualHub\" }\r\n ,\"microsoft.network/virtualnetworkgateways\": { \"SingularDisplayName\": \"Virtual network gateway\" }\r\n ,\"microsoft.network/virtualnetworks\": { \"SingularDisplayName\": \"Virtual network\" }\r\n ,\"microsoft.network/virtualnetworktaps\": { \"SingularDisplayName\": \"Virtual network terminal access point\" }\r\n ,\"microsoft.network/virtualrouters\": { \"SingularDisplayName\": \"Microsoft.Network virtual router\" }\r\n ,\"microsoft.network/virtualrouters/peerings\": { \"SingularDisplayName\": \"Microsoft.Network virtual routers peering\" }\r\n ,\"microsoft.network/virtualwans\": { \"SingularDisplayName\": \"Virtual WAN\" }\r\n ,\"microsoft.network/vpngateways\": { \"SingularDisplayName\": \"VPN Gateway (Site to Site)\" }\r\n ,\"microsoft.network/vpngateways/vpnconnections\": { \"SingularDisplayName\": \"Microsoft.Network VPN gateways VPN connection\" }\r\n ,\"microsoft.network/vpngateways/vpnconnections/vpnlinkconnections\": { \"SingularDisplayName\": \"Microsoft.Network VPN gateways VPN connections VPN link connection\" }\r\n ,\"microsoft.network/vpnserverconfigurations\": { \"SingularDisplayName\": \"Microsoft.Network VPN server configuration\" }\r\n ,\"microsoft.network/vpnsites\": { \"SingularDisplayName\": \"Microsoft.Network VPN site\" }\r\n ,\"microsoft.network/vpnsites/vpnsitelinks\": { \"SingularDisplayName\": \"Microsoft.Network VPN sites VPN site link\" }\r\n ,\"microsoft.networkanalytics/dataconnectors\": { \"SingularDisplayName\": \"AIOps - Data Connector\" }\r\n ,\"microsoft.networkanalytics/datalakehouses\": { \"SingularDisplayName\": \"AIOps - Data LakeHouse\" }\r\n ,\"microsoft.networkanalytics/dataproducts\": { \"SingularDisplayName\": \"Azure Operator Insights ? Data Product\" }\r\n ,\"microsoft.networkanalytics/dataproducts/datatypes\": { \"SingularDisplayName\": \"Data Type\" }\r\n ,\"microsoft.networkanalytics/dataproductscatalogs\": { \"SingularDisplayName\": \"Microsoft.NetworkAnalytics data products catalog\" }\r\n ,\"microsoft.networkanalytics/metricsingestionendpoints\": { \"SingularDisplayName\": \"Microsoft.NetworkAnalytics metrics ingestion endpoint\" }\r\n ,\"microsoft.networkanalytics/networkanalyticsproducts\": { \"SingularDisplayName\": \"Microsoft.NetworkAnalytics network analytics product\" }\r\n ,\"microsoft.networkcloud/baremetalmachines\": { \"SingularDisplayName\": \"Bare Metal Machine (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/cloudservicesnetworks\": { \"SingularDisplayName\": \"Cloud Services Network (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/clustermanagers\": { \"SingularDisplayName\": \"Cluster Manager (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/clusters\": { \"SingularDisplayName\": \"Cluster (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/clusters/baremetalmachinekeysets\": { \"SingularDisplayName\": \"Cluster Bare Metal Machine Key Set (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/clusters/bmckeysets\": { \"SingularDisplayName\": \"Cluster Baseboard Management Controller Key Set (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/clusters/metricsconfigurations\": { \"SingularDisplayName\": \"Cluster Metrics Configuration (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/edgeclustermachineskus\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud edge cluster machine SKU\" }\r\n ,\"microsoft.networkcloud/edgeclusterruntimeversions\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud edge cluster runtime version\" }\r\n ,\"microsoft.networkcloud/edgeclusters\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud edge cluster\" }\r\n ,\"microsoft.networkcloud/edgeclusters/nodes\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud edge clusters node\" }\r\n ,\"microsoft.networkcloud/edgeclusterskus\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud edge cluster SKU\" }\r\n ,\"microsoft.networkcloud/kubernetesclusters\": { \"SingularDisplayName\": \"Kubernetes Cluster (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/kubernetesclusters/agentpools\": { \"SingularDisplayName\": \"Agent Pool (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/kubernetesclusters/features\": { \"SingularDisplayName\": \"Kubernetes Cluster Feature (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/l2networks\": { \"SingularDisplayName\": \"Layer 2 Network (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/l3networks\": { \"SingularDisplayName\": \"Layer 3 Network (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/racks\": { \"SingularDisplayName\": \"Compute Rack (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/rackskus\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud rack SKU\" }\r\n ,\"microsoft.networkcloud/registrationhubs\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud registration hub\" }\r\n ,\"microsoft.networkcloud/registrationhubs/images\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud registration hubs image\" }\r\n ,\"microsoft.networkcloud/registrationhubs/machines\": { \"SingularDisplayName\": \"Microsoft.NetworkCloud registration hubs machine\" }\r\n ,\"microsoft.networkcloud/storageappliances\": { \"SingularDisplayName\": \"Storage Appliance (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/trunkednetworks\": { \"SingularDisplayName\": \"Trunked Network (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/virtualmachines\": { \"SingularDisplayName\": \"Virtual Machine (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/virtualmachines/consoles\": { \"SingularDisplayName\": \"Virtual Machine Console (Operator Nexus)\" }\r\n ,\"microsoft.networkcloud/volumes\": { \"SingularDisplayName\": \"Volume (Operator Nexus)\" }\r\n ,\"microsoft.networkfunction/azuretrafficcollectors\": { \"SingularDisplayName\": \"ExpressRoute traffic collector\" }\r\n ,\"microsoft.networkfunction/meshvpns\": { \"SingularDisplayName\": \"Mesh VPN\" }\r\n ,\"microsoft.nexusidentity/identitycontrollers\": { \"SingularDisplayName\": \"Microsoft.NexusIdentity identity controller\" }\r\n ,\"microsoft.nexusidentity/identitysets\": { \"SingularDisplayName\": \"Microsoft.NexusIdentity identity set\" }\r\n ,\"microsoft.notebooks/notebookproxies\": { \"SingularDisplayName\": \"Microsoft.Notebooks notebook proxy\" }\r\n ,\"microsoft.notificationhubs/namespaces\": { \"SingularDisplayName\": \"Notification Hub Namespace\" }\r\n ,\"microsoft.notificationhubs/namespaces/notificationhubs\": { \"SingularDisplayName\": \"Notification Hub\" }\r\n ,\"microsoft.objectstore/osnamespaces\": { \"SingularDisplayName\": \"Microsoft.ObjectStore os namespace\" }\r\n })[tolower(id)]\r\n}\r\n", + "$fxv#3": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData/Internal')\r\n_resource_type_4(id: string) {\r\n dynamic({\r\n \"microsoft.offazure/hypervsites\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv site\" }\r\n ,\"microsoft.offazure/hypervsites/clusters\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites cluster\" }\r\n ,\"microsoft.offazure/hypervsites/hosts\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites host\" }\r\n ,\"microsoft.offazure/hypervsites/jobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites job\" }\r\n ,\"microsoft.offazure/hypervsites/machines\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites machine\" }\r\n ,\"microsoft.offazure/hypervsites/machines/softwareinventories\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites machines software inventory\" }\r\n ,\"microsoft.offazure/hypervsites/operationsstatus\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites operations statu\" }\r\n ,\"microsoft.offazure/hypervsites/runasaccounts\": { \"SingularDisplayName\": \"Microsoft.OffAzure hyperv sites run as account\" }\r\n ,\"microsoft.offazure/importsites\": { \"SingularDisplayName\": \"Microsoft.OffAzure import site\" }\r\n ,\"microsoft.offazure/importsites/deletejobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure import sites delete job\" }\r\n ,\"microsoft.offazure/importsites/exportjobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure import sites export job\" }\r\n ,\"microsoft.offazure/importsites/importjobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure import sites import job\" }\r\n ,\"microsoft.offazure/importsites/jobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure import sites job\" }\r\n ,\"microsoft.offazure/importsites/machines\": { \"SingularDisplayName\": \"Microsoft.OffAzure import sites machine\" }\r\n ,\"microsoft.offazure/mastersites\": { \"SingularDisplayName\": \"Microsoft.OffAzure master site\" }\r\n ,\"microsoft.offazure/mastersites/operationsstatus\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites operations statu\" }\r\n ,\"microsoft.offazure/mastersites/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites private endpoint connection\" }\r\n ,\"microsoft.offazure/mastersites/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites private link resource\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql site\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/discoverysitedatasources\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites discovery site data source\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/jobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites job\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/operationsstatus\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites operations statu\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/runasaccounts\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites run as account\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/sqlavailabilitygroups\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites sql availability group\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/sqldatabases\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites sql database\" }\r\n ,\"microsoft.offazure/mastersites/sqlsites/sqlservers\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites sql sites sql server\" }\r\n ,\"microsoft.offazure/mastersites/webappsites\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app site\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/discoverysitedatasources\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites discovery site data source\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/extendedmachines\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites extended machine\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/iiswebapplications\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites iis web application\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/iiswebservers\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites iis web server\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/runasaccounts\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites runasaccount\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/tomcatwebapplications\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites tomcat web application\" }\r\n ,\"microsoft.offazure/mastersites/webappsites/tomcatwebservers\": { \"SingularDisplayName\": \"Microsoft.OffAzure master sites web app sites tomcat web server\" }\r\n ,\"microsoft.offazure/serversites\": { \"SingularDisplayName\": \"Microsoft.OffAzure server site\" }\r\n ,\"microsoft.offazure/serversites/jobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure server sites job\" }\r\n ,\"microsoft.offazure/serversites/machines\": { \"SingularDisplayName\": \"Microsoft.OffAzure server sites machine\" }\r\n ,\"microsoft.offazure/serversites/machines/softwareinventories\": { \"SingularDisplayName\": \"Microsoft.OffAzure server sites machines software inventory\" }\r\n ,\"microsoft.offazure/serversites/operationsstatus\": { \"SingularDisplayName\": \"Microsoft.OffAzure server sites operations statu\" }\r\n ,\"microsoft.offazure/serversites/runasaccounts\": { \"SingularDisplayName\": \"Microsoft.OffAzure server sites run as account\" }\r\n ,\"microsoft.offazure/vmwaresites\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware site\" }\r\n ,\"microsoft.offazure/vmwaresites/hosts\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites host\" }\r\n ,\"microsoft.offazure/vmwaresites/jobs\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites job\" }\r\n ,\"microsoft.offazure/vmwaresites/machines\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites machine\" }\r\n ,\"microsoft.offazure/vmwaresites/machines/softwareinventories\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites machines software inventory\" }\r\n ,\"microsoft.offazure/vmwaresites/operationsstatus\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites operations statu\" }\r\n ,\"microsoft.offazure/vmwaresites/runasaccounts\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites run as account\" }\r\n ,\"microsoft.offazure/vmwaresites/vcenters\": { \"SingularDisplayName\": \"Microsoft.OffAzure vmware sites vcenter\" }\r\n ,\"microsoft.offazurespringboot/springbootsites\": { \"SingularDisplayName\": \"Microsoft.OffAzureSpringBoot springbootsite\" }\r\n ,\"microsoft.offazurespringboot/springbootsites/errorsummaries\": { \"SingularDisplayName\": \"Microsoft.OffAzureSpringBoot springbootsites error summary\" }\r\n ,\"microsoft.offazurespringboot/springbootsites/springbootapps\": { \"SingularDisplayName\": \"Microsoft.OffAzureSpringBoot springbootsites springbootapp\" }\r\n ,\"microsoft.offazurespringboot/springbootsites/springbootservers\": { \"SingularDisplayName\": \"Microsoft.OffAzureSpringBoot springbootsites springbootserver\" }\r\n ,\"microsoft.offazurespringboot/springbootsites/summaries\": { \"SingularDisplayName\": \"Microsoft.OffAzureSpringBoot springbootsites summary\" }\r\n ,\"microsoft.onlineexperimentation/workspaces\": { \"SingularDisplayName\": \"Online Experimentation Workspace\" }\r\n ,\"microsoft.openenergyplatform/energyservices\": { \"SingularDisplayName\": \"Azure Data Manager for Energy\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspace\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces/applicationregistrations\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspaces application registration\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces/applications\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspaces application\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces/eventgridfilters\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspaces event grid filter\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces/shares\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspaces share\" }\r\n ,\"microsoft.openlogisticsplatform/workspaces/sharesubscriptions\": { \"SingularDisplayName\": \"Microsoft.OpenLogisticsPlatform workspaces share subscription\" }\r\n ,\"microsoft.operationalinsights/clusters\": { \"SingularDisplayName\": \"Log Analytics dedicated cluster\" }\r\n ,\"microsoft.operationalinsights/querypacks\": { \"SingularDisplayName\": \"Log Analytics query pack\" }\r\n ,\"microsoft.operationalinsights/workspaces\": { \"SingularDisplayName\": \"Log Analytics workspace\" }\r\n ,\"microsoft.operationsmanagement/managementassociations\": { \"SingularDisplayName\": \"Microsoft.OperationsManagement management association\" }\r\n ,\"microsoft.operationsmanagement/solutions\": { \"SingularDisplayName\": \"Solution\" }\r\n ,\"microsoft.operatorvoicemail/operatorvoicemailinstances\": { \"SingularDisplayName\": \"Microsoft.OperatorVoicemail operator voicemail instance\" }\r\n ,\"microsoft.oraclediscovery/oraclesites\": { \"SingularDisplayName\": \"Microsoft.OracleDiscovery oracle site\" }\r\n ,\"microsoft.oraclediscovery/oraclesites/errorsummaries\": { \"SingularDisplayName\": \"Microsoft.OracleDiscovery oracle sites error summary\" }\r\n ,\"microsoft.oraclediscovery/oraclesites/oracledatabases\": { \"SingularDisplayName\": \"Microsoft.OracleDiscovery oracle sites oracle database\" }\r\n ,\"microsoft.oraclediscovery/oraclesites/oracleservers\": { \"SingularDisplayName\": \"Microsoft.OracleDiscovery oracle sites oracle server\" }\r\n ,\"microsoft.oraclediscovery/oraclesites/summaries\": { \"SingularDisplayName\": \"Microsoft.OracleDiscovery oracle sites summary\" }\r\n ,\"microsoft.orbital/cloudaccessrouters\": { \"SingularDisplayName\": \"Cloud Access Router\" }\r\n ,\"microsoft.orbital/contactprofiles\": { \"SingularDisplayName\": \"Contact Profile\" }\r\n ,\"microsoft.orbital/edgesites\": { \"SingularDisplayName\": \"Edge Site\" }\r\n ,\"microsoft.orbital/geocatalogs\": { \"SingularDisplayName\": \"GeoCatalog\" }\r\n ,\"microsoft.orbital/globalcommunicationssites\": { \"SingularDisplayName\": \"Microsoft.Orbital global communications site\" }\r\n ,\"microsoft.orbital/groundstations\": { \"SingularDisplayName\": \"Ground Station\" }\r\n ,\"microsoft.orbital/l2connections\": { \"SingularDisplayName\": \"L2 Connection\" }\r\n ,\"microsoft.orbital/sdwancontrollers\": { \"SingularDisplayName\": \"SDWAN Controller\" }\r\n ,\"microsoft.orbital/spacecrafts\": { \"SingularDisplayName\": \"Spacecraft\" }\r\n ,\"microsoft.orbital/spacecrafts/contacts\": { \"SingularDisplayName\": \"Contact\" }\r\n ,\"microsoft.orbital/terminals\": { \"SingularDisplayName\": \"Cloud Access Terminal\" }\r\n ,\"microsoft.partnermanagedconsumerrecurrence/recurrences\": { \"SingularDisplayName\": \"Microsoft.PartnerManagedConsumerRecurrence recurrence\" }\r\n ,\"microsoft.partnermanagedconsumerrecurrence/recurrences/operationresult\": { \"SingularDisplayName\": \"Microsoft.PartnerManagedConsumerRecurrence recurrences operation result\" }\r\n ,\"microsoft.peering/peerasns\": { \"SingularDisplayName\": \"Microsoft.Peering peer asn\" }\r\n ,\"microsoft.peering/peerings\": { \"SingularDisplayName\": \"Peering\" }\r\n ,\"microsoft.peering/peerings/registeredasns\": { \"SingularDisplayName\": \"Registered ASN\" }\r\n ,\"microsoft.peering/peerings/registeredprefixes\": { \"SingularDisplayName\": \"Registered prefix\" }\r\n ,\"microsoft.peering/peeringservices\": { \"SingularDisplayName\": \"Peering Service\" }\r\n ,\"microsoft.peering/peeringservices/prefixes\": { \"SingularDisplayName\": \"Peering Service Prefix\" }\r\n ,\"microsoft.pki/pkis\": { \"SingularDisplayName\": \"Microsoft.Pki PKI\" }\r\n ,\"microsoft.pki/pkis/certificateauthorities\": { \"SingularDisplayName\": \"Microsoft.Pki pkis certificate authority\" }\r\n ,\"microsoft.pki/pkis/enrollmentpolicies\": { \"SingularDisplayName\": \"Microsoft.Pki pkis enrollment policy\" }\r\n ,\"microsoft.policyinsights/attestations\": { \"SingularDisplayName\": \"Microsoft.PolicyInsights attestation\" }\r\n ,\"microsoft.policyinsights/policymetadata\": { \"SingularDisplayName\": \"Microsoft.PolicyInsights policy metadata\" }\r\n ,\"microsoft.policyinsights/remediations\": { \"SingularDisplayName\": \"Microsoft.PolicyInsights remediation\" }\r\n ,\"microsoft.portal/consoles\": { \"SingularDisplayName\": \"Microsoft.Portal console\" }\r\n ,\"microsoft.portal/dashboards\": { \"SingularDisplayName\": \"Shared dashboard\" }\r\n ,\"microsoft.portal/tenantconfigurations\": { \"SingularDisplayName\": \"Microsoft.Portal tenant configuration\" }\r\n ,\"microsoft.portal/usersettings\": { \"SingularDisplayName\": \"Microsoft.Portal user setting\" }\r\n ,\"microsoft.portal/virtual-privatedashboards\": { \"SingularDisplayName\": \"Private dashboard\" }\r\n ,\"microsoft.portalservices/copilotsettings\": { \"SingularDisplayName\": \"Microsoft.PortalServices copilot setting\" }\r\n ,\"microsoft.portalservices/dashboards\": { \"SingularDisplayName\": \"Shared dashboard\" }\r\n ,\"microsoft.portalservices/extensions\": { \"SingularDisplayName\": \"Portal Extension\" }\r\n ,\"microsoft.portalservices/extensions/deployments\": { \"SingularDisplayName\": \"Extension Deployment\" }\r\n ,\"microsoft.portalservices/extensions/slots\": { \"SingularDisplayName\": \"Extension Slot\" }\r\n ,\"microsoft.portalservices/extensions/versions\": { \"SingularDisplayName\": \"Extension Version\" }\r\n ,\"microsoft.portalservices/settings\": { \"SingularDisplayName\": \"Microsoft.PortalServices setting\" }\r\n ,\"microsoft.powerbi/privatelinkservicesforpowerbi\": { \"SingularDisplayName\": \"Microsoft.PowerBI private link services for power bi\" }\r\n ,\"microsoft.powerbi/privatelinkservicesforpowerbi/privateendpointconnections\": { \"SingularDisplayName\": \"Microsoft.PowerBI private link services for power bi private endpoint connection\" }\r\n ,\"microsoft.powerbi/privatelinkservicesforpowerbi/privatelinkresources\": { \"SingularDisplayName\": \"Microsoft.PowerBI private link services for power bi private link resource\" }\r\n ,\"microsoft.powerbi/workspacecollections\": { \"SingularDisplayName\": \"Microsoft.PowerBI workspace collection\" }\r\n ,\"microsoft.powerbidedicated/autoscalevcores\": { \"SingularDisplayName\": \"Microsoft.PowerBIDedicated auto scale vcore\" }\r\n ,\"microsoft.powerbidedicated/capacities\": { \"SingularDisplayName\": \"Power BI Embedded\" }\r\n ,\"microsoft.powerplatform/accounts\": { \"SingularDisplayName\": \"Microsoft.PowerPlatform account\" }\r\n ,\"microsoft.premonition/libraries\": { \"SingularDisplayName\": \"Microsoft.Premonition library\" }\r\n ,\"microsoft.premonition/libraries/analyses\": { \"SingularDisplayName\": \"Microsoft.Premonition libraries analyse\" }\r\n ,\"microsoft.premonition/libraries/samples\": { \"SingularDisplayName\": \"Microsoft.Premonition libraries sample\" }\r\n ,\"microsoft.professionalservice/resources\": { \"SingularDisplayName\": \"Professional Service\" }\r\n ,\"microsoft.programmableconnectivity/gateways\": { \"SingularDisplayName\": \"APC Gateway\" }\r\n ,\"microsoft.programmableconnectivity/operatorapiconnections\": { \"SingularDisplayName\": \"APC Operator API Connection\" }\r\n ,\"microsoft.programmableconnectivity/operatorapiplans\": { \"SingularDisplayName\": \"APC Operator API Plan\" }\r\n ,\"microsoft.proposal/proposals\": { \"SingularDisplayName\": \"Microsoft.Proposal proposal\" }\r\n ,\"microsoft.providerhub/providerregistrations\": { \"SingularDisplayName\": \"Resource Provider as a Service\" }\r\n ,\"microsoft.providerhub/providerregistrations/customrollouts\": { \"SingularDisplayName\": \"Rollout\" }\r\n ,\"microsoft.providerhub/providerregistrations/defaultrollouts\": { \"SingularDisplayName\": \"Rollout\" }\r\n ,\"microsoft.providerhub/providerregistrations/resourcetyperegistrations\": { \"SingularDisplayName\": \"Resource Type\" }\r\n ,\"microsoft.providerhub/providerregistrations/resourcetyperegistrations/resourcetyperegistrations\": { \"SingularDisplayName\": \"Resource Type\" }\r\n ,\"microsoft.providerhubdevtest/regionalstresstests\": { \"SingularDisplayName\": \"Microsoft.ProviderHubDevTest regional stresstest\" }\r\n ,\"microsoft.providerhubdevtest/stresstests\": { \"SingularDisplayName\": \"Microsoft.ProviderHubDevTest stresstest\" }\r\n ,\"microsoft.purview/accounts\": { \"SingularDisplayName\": \"Microsoft Purview account\" }\r\n ,\"microsoft.quantum/provideraccounts\": { \"SingularDisplayName\": \"Microsoft.Quantum provider account\" }\r\n ,\"microsoft.quantum/workspaces\": { \"SingularDisplayName\": \"Quantum Workspace\" }\r\n ,\"microsoft.quota/groupquotas\": { \"SingularDisplayName\": \"Microsoft.Quota group quota\" }\r\n ,\"microsoft.quota/groupquotas/groupquotarequests\": { \"SingularDisplayName\": \"Microsoft.Quota group quotas group quota request\" }\r\n ,\"microsoft.quota/groupquotas/quotaallocationrequests\": { \"SingularDisplayName\": \"Microsoft.Quota group quotas quota allocation request\" }\r\n ,\"microsoft.quota/groupquotas/quotaallocations\": { \"SingularDisplayName\": \"Microsoft.Quota group quotas quota allocation\" }\r\n ,\"microsoft.quota/groupquotas/subscriptionrequests\": { \"SingularDisplayName\": \"Microsoft.Quota group quotas subscription request\" }\r\n ,\"microsoft.quota/groupquotas/subscriptions\": { \"SingularDisplayName\": \"Microsoft.Quota group quotas subscription\" }\r\n ,\"microsoft.quota/quotarequests\": { \"SingularDisplayName\": \"Microsoft.Quota quota request\" }\r\n ,\"microsoft.quota/quotas\": { \"SingularDisplayName\": \"Microsoft.Quota quota\" }\r\n ,\"microsoft.quota/usages\": { \"SingularDisplayName\": \"Microsoft.Quota usage\" }\r\n ,\"microsoft.recommendationsservice/accounts\": { \"SingularDisplayName\": \"Intelligent Recommendations Account\" }\r\n ,\"microsoft.recommendationsservice/accounts/modeling\": { \"SingularDisplayName\": \"Modeling\" }\r\n ,\"microsoft.recommendationsservice/accounts/serviceendpoints\": { \"SingularDisplayName\": \"Service Endpoint\" }\r\n ,\"microsoft.recoveryservices/replicationeligibilityresults\": { \"SingularDisplayName\": \"Microsoft.RecoveryServices replication eligibility result\" }\r\n ,\"microsoft.recoveryservices/vaults\": { \"SingularDisplayName\": \"Recovery Services vault\" }\r\n ,\"microsoft.recoveryservices/vaults/backupfabrics/protectioncontainers/protecteditems\": { \"SingularDisplayName\": \"Backup Item\" }\r\n ,\"microsoft.recoveryservicesbvtd/vaults\": { \"SingularDisplayName\": \"Recovery Services BVTD\" }\r\n ,\"microsoft.recoveryservicesbvtd2/vaults\": { \"SingularDisplayName\": \"Recovery Services BVTD2\" }\r\n ,\"microsoft.recoveryservicesintd/vaults\": { \"SingularDisplayName\": \"Recovery Services INTD\" }\r\n ,\"microsoft.recoveryservicesintd2/vaults\": { \"SingularDisplayName\": \"Recovery Services INTD2\" }\r\n ,\"microsoft.redhatopenshift/openshiftclusters\": { \"SingularDisplayName\": \"Azure Red Hat OpenShift cluster\" }\r\n ,\"microsoft.relationships/dependencyof\": { \"SingularDisplayName\": \"Dependency Relationship\" }\r\n ,\"microsoft.relationships/servicegroupmember\": { \"SingularDisplayName\": \"Service group member relationship\" }\r\n ,\"microsoft.relationships/servicegrouprelationships\": { \"SingularDisplayName\": \"Connected Resource\" }\r\n ,\"microsoft.relay/namespaces\": { \"SingularDisplayName\": \"Relay\" }\r\n ,\"microsoft.relay/namespaces/hybridconnections\": { \"SingularDisplayName\": \"Hybrid connection\" }\r\n ,\"microsoft.relay/namespaces/wcfrelays\": { \"SingularDisplayName\": \"WCF relay\" }\r\n ,\"microsoft.resilience/resiliencestates\": { \"SingularDisplayName\": \"Microsoft.Resilience resilience state\" }\r\n ,\"microsoft.resourceconnector/appliances\": { \"SingularDisplayName\": \"Resource bridge\" }\r\n ,\"microsoft.resourcegraph/queries\": { \"SingularDisplayName\": \"Resource Graph query\" }\r\n ,\"microsoft.resourcehealth/availabilitystatuses\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth availability statuse\" }\r\n ,\"microsoft.resourcehealth/childavailabilitystatuses\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth child availability statuse\" }\r\n ,\"microsoft.resourcehealth/emergingissues\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth emerging issue\" }\r\n ,\"microsoft.resourcehealth/events\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth event\" }\r\n ,\"microsoft.resourcehealth/events/impactedresources\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth events impacted resource\" }\r\n ,\"microsoft.resourcehealth/metadata\": { \"SingularDisplayName\": \"Microsoft.ResourceHealth metadata\" }\r\n ,\"microsoft.resources/builtintemplatespecs\": { \"SingularDisplayName\": \"Built-in template spec\" }\r\n ,\"microsoft.resources/changes\": { \"SingularDisplayName\": \"Microsoft.Resources change\" }\r\n ,\"microsoft.resources/databoundaries\": { \"SingularDisplayName\": \"Microsoft.Resources data boundary\" }\r\n ,\"microsoft.resources/deletedresources\": { \"SingularDisplayName\": \"Recycle Bin\" }\r\n ,\"microsoft.resources/deployments\": { \"SingularDisplayName\": \"Microsoft.Resources deployment\" }\r\n ,\"microsoft.resources/deployments/operations\": { \"SingularDisplayName\": \"Microsoft.Resources deployments operation\" }\r\n ,\"microsoft.resources/deploymentscripts\": { \"SingularDisplayName\": \"Deployment Script\" }\r\n ,\"microsoft.resources/deploymentstacks\": { \"SingularDisplayName\": \"Deployment stack\" }\r\n ,\"microsoft.resources/mobobrokers\": { \"SingularDisplayName\": \"Microsoft.Resources mobo broker\" }\r\n ,\"microsoft.resources/resourcechange\": { \"SingularDisplayName\": \"Change Analysis\" }\r\n ,\"microsoft.resources/resourcechanges\": { \"SingularDisplayName\": \"Resource change\" }\r\n ,\"microsoft.resources/resourcegraphvisualizer\": { \"SingularDisplayName\": \"Resource Graph Visualizer\" }\r\n ,\"microsoft.resources/resourcegroups\": { \"SingularDisplayName\": \"Microsoft.Resources resource group\" }\r\n ,\"microsoft.resources/resources\": { \"SingularDisplayName\": \"Resource\" }\r\n ,\"microsoft.resources/snapshots\": { \"SingularDisplayName\": \"Microsoft.Resources snapshot\" }\r\n ,\"microsoft.resources/subscriptions\": { \"SingularDisplayName\": \"Subscription\" }\r\n ,\"microsoft.resources/subscriptions/resourcegroups\": { \"SingularDisplayName\": \"Resource group\" }\r\n ,\"microsoft.resources/tags\": { \"SingularDisplayName\": \"Microsoft.Resources tag\" }\r\n ,\"microsoft.resources/templatespecs\": { \"SingularDisplayName\": \"Template spec\" }\r\n ,\"microsoft.resources/virtualsubscriptionsforresourcepicker\": { \"SingularDisplayName\": \"Subscription\" }\r\n ,\"microsoft.saas/applications\": { \"SingularDisplayName\": \"Software as a Service (classic)\" }\r\n ,\"microsoft.saas/resources\": { \"SingularDisplayName\": \"SaaS\" }\r\n ,\"microsoft.saas/saasresources\": { \"SingularDisplayName\": \"SaaS (classic)\" }\r\n ,\"microsoft.saashub/cloudservices\": { \"SingularDisplayName\": \"Microsoft.SaaSHub cloud service\" }\r\n ,\"microsoft.saashub/cloudservices/hidden\": { \"SingularDisplayName\": \"Microsoft SaaS\" }\r\n ,\"microsoft.saashub/saasresources\": { \"SingularDisplayName\": \"Microsoft.SaaSHub saas resource\" }\r\n ,\"microsoft.salescopilot/conversationintelligencerecordingaccounts\": { \"SingularDisplayName\": \"Microsoft.SalesCopilot conversation intelligence recording account\" }\r\n ,\"microsoft.scheduler/jobcollections\": { \"SingularDisplayName\": \"Scheduler job collection\" }\r\n ,\"microsoft.scheduler/jobcollections/jobs\": { \"SingularDisplayName\": \"Scheduler job\" }\r\n ,\"microsoft.scom/managedinstances\": { \"SingularDisplayName\": \"SCOM managed instance\" }\r\n ,\"microsoft.scvmm/availabilitysets\": { \"SingularDisplayName\": \"Microsoft.ScVmm availability set\" }\r\n ,\"microsoft.scvmm/clouds\": { \"SingularDisplayName\": \"Microsoft.ScVmm cloud\" }\r\n ,\"microsoft.scvmm/virtualmachineinstances\": { \"SingularDisplayName\": \"Microsoft.ScVmm virtual machine instance\" }\r\n ,\"microsoft.scvmm/virtualmachineinstances/guestagents\": { \"SingularDisplayName\": \"Microsoft.ScVmm virtual machine instances guest agent\" }\r\n ,\"microsoft.scvmm/virtualmachineinstances/hybrididentitymetadata\": { \"SingularDisplayName\": \"Microsoft.ScVmm virtual machine instances hybrid identity metadata\" }\r\n ,\"microsoft.scvmm/virtualmachines\": { \"SingularDisplayName\": \"SCVMM virtual machine - Azure Arc\" }\r\n ,\"microsoft.scvmm/virtualmachinetemplates\": { \"SingularDisplayName\": \"Microsoft.ScVmm virtual machine template\" }\r\n ,\"microsoft.scvmm/virtualnetworks\": { \"SingularDisplayName\": \"Microsoft.ScVmm virtual network\" }\r\n ,\"microsoft.scvmm/vmmservers\": { \"SingularDisplayName\": \"SCVMM management server\" }\r\n ,\"microsoft.search/searchservices\": { \"SingularDisplayName\": \"Search service\" }\r\n ,\"microsoft.secretmanagementsampleprovider/forecasts\": { \"SingularDisplayName\": \"Microsoft.SecretManagementSampleProvider forecast\" }\r\n ,\"microsoft.secretsynccontroller/azurekeyvaultsecretproviderclasses\": { \"SingularDisplayName\": \"Microsoft.SecretSyncController Azure key vault secret provider class\" }\r\n ,\"microsoft.secretsynccontroller/secretsyncs\": { \"SingularDisplayName\": \"Microsoft.SecretSyncController secret sync\" }\r\n ,\"microsoft.security/adaptivenetworkhardenings\": { \"SingularDisplayName\": \"Microsoft.Security adaptive network hardening\" }\r\n ,\"microsoft.security/advancedthreatprotectionsettings\": { \"SingularDisplayName\": \"Microsoft.Security advanced threat protection setting\" }\r\n ,\"microsoft.security/alertssuppressionrules\": { \"SingularDisplayName\": \"Microsoft.Security alerts suppression rule\" }\r\n ,\"microsoft.security/apicollections\": { \"SingularDisplayName\": \"Microsoft.Security API collection\" }\r\n ,\"microsoft.security/applications\": { \"SingularDisplayName\": \"Microsoft.Security application\" }\r\n ,\"microsoft.security/assessmentmetadata\": { \"SingularDisplayName\": \"Microsoft.Security assessment metadata\" }\r\n ,\"microsoft.security/assessments\": { \"SingularDisplayName\": \"Microsoft.Security assessment\" }\r\n ,\"microsoft.security/assessments/governanceassignments\": { \"SingularDisplayName\": \"Microsoft.Security assessments governance assignment\" }\r\n ,\"microsoft.security/assessments/subassessments\": { \"SingularDisplayName\": \"Microsoft.Security assessments sub assessment\" }\r\n ,\"microsoft.security/assignments\": { \"SingularDisplayName\": \"Microsoft.Security assignment\" }\r\n ,\"microsoft.security/automations\": { \"SingularDisplayName\": \"Microsoft.Security automation\" }\r\n ,\"microsoft.security/autoprovisioningsettings\": { \"SingularDisplayName\": \"Microsoft.Security auto provisioning setting\" }\r\n ,\"microsoft.security/complianceresults\": { \"SingularDisplayName\": \"Microsoft.Security compliance result\" }\r\n ,\"microsoft.security/compliances\": { \"SingularDisplayName\": \"Microsoft.Security compliance\" }\r\n ,\"microsoft.security/connectors\": { \"SingularDisplayName\": \"Microsoft.Security connector\" }\r\n ,\"microsoft.security/customassessmentautomations\": { \"SingularDisplayName\": \"Microsoft.Security custom assessment automation\" }\r\n ,\"microsoft.security/defenderforstoragesettings\": { \"SingularDisplayName\": \"Microsoft.Security defender for storage setting\" }\r\n ,\"microsoft.security/defenderforstoragesettings/malwarescans\": { \"SingularDisplayName\": \"Microsoft.Security defender for storage settings malware scan\" }\r\n ,\"microsoft.security/devicesecuritygroups\": { \"SingularDisplayName\": \"Microsoft.Security device security group\" }\r\n ,\"microsoft.security/governancerules\": { \"SingularDisplayName\": \"Microsoft.Security governance rule\" }\r\n ,\"microsoft.security/governancerules/operationresults\": { \"SingularDisplayName\": \"Microsoft.Security governance rules operation result\" }\r\n ,\"microsoft.security/healthreports\": { \"SingularDisplayName\": \"Microsoft.Security health report\" }\r\n ,\"microsoft.security/informationprotectionpolicies\": { \"SingularDisplayName\": \"Microsoft.Security information protection policy\" }\r\n ,\"microsoft.security/iotsecuritysolutions\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solution\" }\r\n ,\"microsoft.security/iotsecuritysolutions/analyticsmodels\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions analytics model\" }\r\n ,\"microsoft.security/iotsecuritysolutions/analyticsmodels/aggregatedalerts\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions analytics models aggregated alert\" }\r\n ,\"microsoft.security/iotsecuritysolutions/analyticsmodels/aggregatedrecommendations\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions analytics models aggregated recommendation\" }\r\n ,\"microsoft.security/iotsecuritysolutions/iotalerts\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions IoT alert\" }\r\n ,\"microsoft.security/iotsecuritysolutions/iotalerttypes\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions IoT alert type\" }\r\n ,\"microsoft.security/iotsecuritysolutions/iotrecommendations\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions IoT recommendation\" }\r\n ,\"microsoft.security/iotsecuritysolutions/iotrecommendationtypes\": { \"SingularDisplayName\": \"Microsoft.Security IoT security solutions IoT recommendation type\" }\r\n ,\"microsoft.security/locations/alerts\": { \"SingularDisplayName\": \"Security Alert\" }\r\n ,\"microsoft.security/mdeonboardings\": { \"SingularDisplayName\": \"Microsoft.Security mde onboarding\" }\r\n ,\"microsoft.security/pricings\": { \"SingularDisplayName\": \"Defender for Cloud\" }\r\n ,\"microsoft.security/pricings/securityoperators\": { \"SingularDisplayName\": \"Microsoft.Security pricings security operator\" }\r\n ,\"microsoft.security/regulatorycompliancestandards\": { \"SingularDisplayName\": \"Microsoft.Security regulatory compliance standard\" }\r\n ,\"microsoft.security/regulatorycompliancestandards/regulatorycompliancecontrols\": { \"SingularDisplayName\": \"Microsoft.Security regulatory compliance standards regulatory compliance control\" }\r\n ,\"microsoft.security/regulatorycompliancestandards/regulatorycompliancecontrols/regulatorycomplianceassessments\": { \"SingularDisplayName\": \"Microsoft.Security regulatory compliance standards regulatory compliance controls regulatory compliance assessment\" }\r\n ,\"microsoft.security/securescores\": { \"SingularDisplayName\": \"Microsoft.Security secure score\" }\r\n ,\"microsoft.security/securityconnectors\": { \"SingularDisplayName\": \"Microsoft.Security security connector\" }\r\n ,\"microsoft.security/securityconnectors/devops\": { \"SingularDisplayName\": \"Microsoft.Security security connectors devop\" }\r\n ,\"microsoft.security/securitycontacts\": { \"SingularDisplayName\": \"Microsoft.Security security contact\" }\r\n ,\"microsoft.security/sensitivitysettings\": { \"SingularDisplayName\": \"Microsoft.Security sensitivity setting\" }\r\n ,\"microsoft.security/servervulnerabilityassessments\": { \"SingularDisplayName\": \"Microsoft.Security server vulnerability assessment\" }\r\n ,\"microsoft.security/servervulnerabilityassessmentssettings\": { \"SingularDisplayName\": \"Microsoft.Security server vulnerability assessments setting\" }\r\n ,\"microsoft.security/settings\": { \"SingularDisplayName\": \"Microsoft.Security setting\" }\r\n ,\"microsoft.security/standards\": { \"SingularDisplayName\": \"Microsoft.Security standard\" }\r\n ,\"microsoft.security/workspacesettings\": { \"SingularDisplayName\": \"Microsoft.Security workspace setting\" }\r\n ,\"microsoft.securitycopilot/capacities\": { \"SingularDisplayName\": \"Microsoft Security compute capacity\" }\r\n ,\"microsoft.securitydetonation/chambers\": { \"SingularDisplayName\": \"Security Detonation Chamber\" }\r\n ,\"microsoft.securityinsightsarg/sentinel\": { \"SingularDisplayName\": \"Microsoft Sentinel\" }\r\n ,\"microsoft.sentinelplatformservices/sentinelplatformservices\": { \"SingularDisplayName\": \"Microsoft.SentinelPlatformServices sentinel platform service\" }\r\n ,\"microsoft.serialconsole/consoleservices\": { \"SingularDisplayName\": \"Microsoft.SerialConsole console service\" }\r\n ,\"microsoft.serialconsole/serialports\": { \"SingularDisplayName\": \"Microsoft.SerialConsole serial port\" }\r\n ,\"microsoft.servicebus/namespaces\": { \"SingularDisplayName\": \"Service Bus namespace\" }\r\n ,\"microsoft.servicebus/namespaces/disasterrecoveryconfigs\": { \"SingularDisplayName\": \"Service Bus Geo-DR Alias\" }\r\n ,\"microsoft.servicebus/namespaces/queues\": { \"SingularDisplayName\": \"Service Bus queue\" }\r\n ,\"microsoft.servicebus/namespaces/topics\": { \"SingularDisplayName\": \"Service Bus topic\" }\r\n ,\"microsoft.servicebus/namespaces/topics/subscriptions\": { \"SingularDisplayName\": \"Service Bus Subscription\" }\r\n ,\"microsoft.servicefabric/clusters\": { \"SingularDisplayName\": \"Service Fabric cluster\" }\r\n ,\"microsoft.servicefabric/managedclusters\": { \"SingularDisplayName\": \"Service Fabric managed cluster\" }\r\n ,\"microsoft.servicefabricmesh/applications\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh application\" }\r\n ,\"microsoft.servicefabricmesh/applications/services\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh applications service\" }\r\n ,\"microsoft.servicefabricmesh/applications/services/replicas\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh applications services replica\" }\r\n ,\"microsoft.servicefabricmesh/gateways\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh gateway\" }\r\n ,\"microsoft.servicefabricmesh/networks\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh network\" }\r\n ,\"microsoft.servicefabricmesh/secrets\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh secret\" }\r\n ,\"microsoft.servicefabricmesh/secrets/values\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh secrets value\" }\r\n ,\"microsoft.servicefabricmesh/volumes\": { \"SingularDisplayName\": \"Microsoft.ServiceFabricMesh volume\" }\r\n ,\"microsoft.servicelinker/dryruns\": { \"SingularDisplayName\": \"Microsoft.ServiceLinker dryrun\" }\r\n ,\"microsoft.servicelinker/linkers\": { \"SingularDisplayName\": \"Microsoft.ServiceLinker linker\" }\r\n ,\"microsoft.servicenetworking/trafficcontrollers\": { \"SingularDisplayName\": \"Application Gateway for Containers\" }\r\n ,\"microsoft.serviceshub/connectors\": { \"SingularDisplayName\": \"Services Hub Connector\" }\r\n ,\"microsoft.signalrservice/signalr\": { \"SingularDisplayName\": \"SignalR\" }\r\n ,\"microsoft.signalrservice/signalr/replicas\": { \"SingularDisplayName\": \"SignalR Replica\" }\r\n ,\"microsoft.signalrservice/webpubsub\": { \"SingularDisplayName\": \"Web PubSub Service\" }\r\n ,\"microsoft.signalrservice/webpubsub/replicas\": { \"SingularDisplayName\": \"Web PubSub Service Replica\" }\r\n ,\"microsoft.skytap/billingnodes\": { \"SingularDisplayName\": \"Microsoft.Skytap billing node\" }\r\n ,\"microsoft.skytap/interfaces\": { \"SingularDisplayName\": \"Microsoft.Skytap interface\" }\r\n ,\"microsoft.skytap/nodes\": { \"SingularDisplayName\": \"Microsoft.Skytap node\" }\r\n ,\"microsoft.softwareplan/hybridusebenefits\": { \"SingularDisplayName\": \"Microsoft.SoftwarePlan hybrid use benefit\" }\r\n ,\"microsoft.solutions/applicationdefinitions\": { \"SingularDisplayName\": \"Service catalog managed application definition\" }\r\n ,\"microsoft.solutions/applications\": { \"SingularDisplayName\": \"Managed application\" }\r\n ,\"microsoft.solutions/jitrequests\": { \"SingularDisplayName\": \"Microsoft.Solutions JIT request\" }\r\n ,\"microsoft.sovereign/landingzoneaccounts\": { \"SingularDisplayName\": \"Landing zone account\" }\r\n ,\"microsoft.sovereign/landingzoneaccounts/landingzoneconfigurations\": { \"SingularDisplayName\": \"Landing Zone Configuration\" }\r\n ,\"microsoft.sovereign/landingzoneaccounts/landingzoneregistrations\": { \"SingularDisplayName\": \"Landing Zone Registration\" }\r\n ,\"microsoft.sovereign/landingzoneconfigurations\": { \"SingularDisplayName\": \"Landing Zone Configuration\" }\r\n ,\"microsoft.sovereign/landingzoneregistrations\": { \"SingularDisplayName\": \"Landing Zone Registration\" }\r\n ,\"microsoft.sovereign/transparencylogs\": { \"SingularDisplayName\": \"Transparency log\" }\r\n ,\"microsoft.sql/azuresql\": { \"SingularDisplayName\": \"Azure SQL resource\" }\r\n ,\"microsoft.sql/instancepools\": { \"SingularDisplayName\": \"Instance pool\" }\r\n ,\"microsoft.sql/managedinstances\": { \"SingularDisplayName\": \"SQL managed instance\" }\r\n ,\"microsoft.sql/managedinstances/databases\": { \"SingularDisplayName\": \"Managed database\" }\r\n ,\"microsoft.sql/servers\": { \"SingularDisplayName\": \"SQL server\" }\r\n ,\"microsoft.sql/servers/databases\": { \"SingularDisplayName\": \"SQL database\" }\r\n ,\"microsoft.sql/servers/elasticpools\": { \"SingularDisplayName\": \"SQL elastic pool\" }\r\n ,\"microsoft.sql/servers/jobagents\": { \"SingularDisplayName\": \"Elastic Job agent\" }\r\n ,\"microsoft.sql/virtualclusters\": { \"SingularDisplayName\": \"Virtual cluster\" }\r\n ,\"microsoft.sqlvirtualmachine/sqlvirtualmachinegroups\": { \"SingularDisplayName\": \"Microsoft.SqlVirtualMachine sql virtual machine group\" }\r\n ,\"microsoft.sqlvirtualmachine/sqlvirtualmachinegroups/availabilitygrouplisteners\": { \"SingularDisplayName\": \"Microsoft.SqlVirtualMachine sql virtual machine groups availability group listener\" }\r\n ,\"microsoft.sqlvirtualmachine/sqlvirtualmachines\": { \"SingularDisplayName\": \"SQL virtual machine\" }\r\n ,\"microsoft.standbypool/standbycontainergrouppools\": { \"SingularDisplayName\": \"Microsoft.StandbyPool standby container group pool\" }\r\n ,\"microsoft.standbypool/standbycontainergrouppools/runtimeviews\": { \"SingularDisplayName\": \"Microsoft.StandbyPool standby container group pools runtime view\" }\r\n ,\"microsoft.standbypool/standbyvirtualmachinepools\": { \"SingularDisplayName\": \"Microsoft.StandbyPool standby virtual machine pool\" }\r\n ,\"microsoft.standbypool/standbyvirtualmachinepools/runtimeviews\": { \"SingularDisplayName\": \"Microsoft.StandbyPool standby virtual machine pools runtime view\" }\r\n ,\"microsoft.standbypool/standbyvirtualmachinepools/standbyvirtualmachines\": { \"SingularDisplayName\": \"Microsoft.StandbyPool standby virtual machine pools standby virtual machine\" }\r\n ,\"microsoft.storage/storageaccounts\": { \"SingularDisplayName\": \"Storage account\" }\r\n ,\"microsoft.storageactions/storagetasks\": { \"SingularDisplayName\": \"Storage task - Azure Storage Actions\" }\r\n ,\"microsoft.storagecache/amlfilesystems\": { \"SingularDisplayName\": \"Azure Managed Lustre\" }\r\n ,\"microsoft.storagecache/caches\": { \"SingularDisplayName\": \"HPC cache\" }\r\n ,\"microsoft.storagediscovery/storagediscoveryworkspaces\": { \"SingularDisplayName\": \"Storage Discovery workspace\" }\r\n ,\"microsoft.storagehub/all\": { \"SingularDisplayName\": \"All resources\" }\r\n ,\"microsoft.storagehub/policycomplianceresources\": { \"SingularDisplayName\": \"Policy compliance\" }\r\n ,\"microsoft.storageinsights/storagecollectionrules\": { \"SingularDisplayName\": \"Microsoft.StorageInsights storage collection rule\" }\r\n ,\"microsoft.storagemover/storagemovers\": { \"SingularDisplayName\": \"Storage mover\" }\r\n ,\"microsoft.storagepool/diskpools\": { \"SingularDisplayName\": \"Microsoft.StoragePool disk pool\" }\r\n ,\"microsoft.storagepool/diskpools/iscsitargets\": { \"SingularDisplayName\": \"Microsoft.StoragePool disk pools iscsi target\" }\r\n ,\"microsoft.storagesync/storagesyncservices\": { \"SingularDisplayName\": \"Storage Sync Service\" }\r\n ,\"microsoft.storagetasks/storagetasks\": { \"SingularDisplayName\": \"Microsoft.StorageTasks storage task\" }\r\n ,\"microsoft.storsimple/managers\": { \"SingularDisplayName\": \"StorSimple device manager\" }\r\n ,\"microsoft.storsimple/managers/accesscontrolrecords\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers access control record\" }\r\n ,\"microsoft.storsimple/managers/bandwidthsettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers bandwidth setting\" }\r\n ,\"microsoft.storsimple/managers/certificates\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers certificate\" }\r\n ,\"microsoft.storsimple/managers/devices\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers device\" }\r\n ,\"microsoft.storsimple/managers/devices/alertsettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices alert setting\" }\r\n ,\"microsoft.storsimple/managers/devices/backuppolicies\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices backup policy\" }\r\n ,\"microsoft.storsimple/managers/devices/backuppolicies/schedules\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices backup policies schedule\" }\r\n ,\"microsoft.storsimple/managers/devices/backupschedulegroups\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices backup schedule group\" }\r\n ,\"microsoft.storsimple/managers/devices/chapsettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices chap setting\" }\r\n ,\"microsoft.storsimple/managers/devices/fileservers\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices fileserver\" }\r\n ,\"microsoft.storsimple/managers/devices/fileservers/shares\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices fileservers share\" }\r\n ,\"microsoft.storsimple/managers/devices/iscsiservers\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices iscsiserver\" }\r\n ,\"microsoft.storsimple/managers/devices/iscsiservers/disks\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices iscsiservers disk\" }\r\n ,\"microsoft.storsimple/managers/devices/jobs\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices job\" }\r\n ,\"microsoft.storsimple/managers/devices/networksettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices network setting\" }\r\n ,\"microsoft.storsimple/managers/devices/securitysettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices security setting\" }\r\n ,\"microsoft.storsimple/managers/devices/timesettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices time setting\" }\r\n ,\"microsoft.storsimple/managers/devices/updatesummary\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices update summary\" }\r\n ,\"microsoft.storsimple/managers/devices/volumecontainers\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices volume container\" }\r\n ,\"microsoft.storsimple/managers/devices/volumecontainers/volumes\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers devices volume containers volume\" }\r\n ,\"microsoft.storsimple/managers/encryptionsettings\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers encryption setting\" }\r\n ,\"microsoft.storsimple/managers/extendedinformation\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers extended information\" }\r\n ,\"microsoft.storsimple/managers/storageaccountcredentials\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers storage account credential\" }\r\n ,\"microsoft.storsimple/managers/storagedomains\": { \"SingularDisplayName\": \"Microsoft.StorSimple managers storage domain\" }\r\n ,\"microsoft.streamanalytics/clusters\": { \"SingularDisplayName\": \"Stream Analytics cluster\" }\r\n ,\"microsoft.streamanalytics/streamingjobs\": { \"SingularDisplayName\": \"Stream Analytics job\" }\r\n ,\"microsoft.subscription/aliases\": { \"SingularDisplayName\": \"Microsoft.Subscription aliase\" }\r\n ,\"microsoft.subscription/changetenantrequest\": { \"SingularDisplayName\": \"Microsoft.Subscription change tenant request\" }\r\n ,\"microsoft.subscription/policies\": { \"SingularDisplayName\": \"Microsoft.Subscription policy\" }\r\n ,\"microsoft.subscription/subscriptiondefinitions\": { \"SingularDisplayName\": \"Microsoft.Subscription subscription definition\" }\r\n ,\"microsoft.subscription/subscriptionoperations\": { \"SingularDisplayName\": \"Microsoft.Subscription subscription operation\" }\r\n ,\"microsoft.support/fileworkspaces\": { \"SingularDisplayName\": \"Microsoft.Support file workspace\" }\r\n ,\"microsoft.support/fileworkspaces/files\": { \"SingularDisplayName\": \"Microsoft.Support file workspaces file\" }\r\n ,\"microsoft.support/services\": { \"SingularDisplayName\": \"Microsoft.Support service\" }\r\n ,\"microsoft.support/services/problemclassifications\": { \"SingularDisplayName\": \"Microsoft.Support services problem classification\" }\r\n ,\"microsoft.support/supporttickets\": { \"SingularDisplayName\": \"Support Request\" }\r\n ,\"microsoft.sustainabilityservices/calculations\": { \"SingularDisplayName\": \"Project Sustainability Calculator\" }\r\n ,\"microsoft.symphony/instances\": { \"SingularDisplayName\": \"Microsoft.Symphony instance\" }\r\n ,\"microsoft.symphony/solutions\": { \"SingularDisplayName\": \"Microsoft.Symphony solution\" }\r\n ,\"microsoft.symphony/targets\": { \"SingularDisplayName\": \"Microsoft.Symphony target\" }\r\n ,\"microsoft.synapse/privatelinkhubs\": { \"SingularDisplayName\": \"Synapse private link hub\" }\r\n ,\"microsoft.synapse/workspaces\": { \"SingularDisplayName\": \"Synapse workspace\" }\r\n ,\"microsoft.synapse/workspaces/bigdatapools\": { \"SingularDisplayName\": \"Apache Spark pool\" }\r\n ,\"microsoft.synapse/workspaces/kustopools\": { \"SingularDisplayName\": \"Data Explorer pool\" }\r\n ,\"microsoft.synapse/workspaces/kustopools/databases\": { \"SingularDisplayName\": \"Data Explorer Database\" }\r\n ,\"microsoft.synapse/workspaces/scopepools\": { \"SingularDisplayName\": \"SCOPE pool\" }\r\n ,\"microsoft.synapse/workspaces/sqlpools\": { \"SingularDisplayName\": \"Dedicated SQL pool\" }\r\n ,\"microsoft.syntex/accounts\": { \"SingularDisplayName\": \"Microsoft.Syntex account\" }\r\n ,\"microsoft.syntex/documentprocessors\": { \"SingularDisplayName\": \"Microsoft.Syntex document processor\" }\r\n ,\"microsoft.test/healthdataaiservices\": { \"SingularDisplayName\": \"Azure Health Data and AI Services\" }\r\n ,\"microsoft.timeseriesinsights/environments\": { \"SingularDisplayName\": \"Microsoft.TimeSeriesInsights environment\" }\r\n ,\"microsoft.timeseriesinsights/environments/accesspolicies\": { \"SingularDisplayName\": \"Microsoft.TimeSeriesInsights environments access policy\" }\r\n ,\"microsoft.timeseriesinsights/environments/eventsources\": { \"SingularDisplayName\": \"Microsoft.TimeSeriesInsights environments event source\" }\r\n ,\"microsoft.timeseriesinsights/environments/referencedatasets\": { \"SingularDisplayName\": \"Microsoft.TimeSeriesInsights environments reference data set\" }\r\n ,\"microsoft.toolchainorchestrator/activations\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator activation\" }\r\n ,\"microsoft.toolchainorchestrator/campaigns\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator campaign\" }\r\n ,\"microsoft.toolchainorchestrator/campaigns/versions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator campaigns version\" }\r\n ,\"microsoft.toolchainorchestrator/catalogs\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator catalog\" }\r\n ,\"microsoft.toolchainorchestrator/catalogs/versions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator catalogs version\" }\r\n ,\"microsoft.toolchainorchestrator/diagnostics\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator diagnostic\" }\r\n ,\"microsoft.toolchainorchestrator/instances\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator instance\" }\r\n ,\"microsoft.toolchainorchestrator/instances/versions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator instances version\" }\r\n ,\"microsoft.toolchainorchestrator/solutions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator solution\" }\r\n ,\"microsoft.toolchainorchestrator/solutions/versions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator solutions version\" }\r\n ,\"microsoft.toolchainorchestrator/targets\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator target\" }\r\n ,\"microsoft.toolchainorchestrator/targets/versions\": { \"SingularDisplayName\": \"Microsoft.ToolchainOrchestrator targets version\" }\r\n ,\"microsoft.updatemanager/updaterules\": { \"SingularDisplayName\": \"Update Rule\" }\r\n ,\"microsoft.usagebilling/accounts\": { \"SingularDisplayName\": \"Microsoft.UsageBilling account\" }\r\n ,\"microsoft.usagebilling/accounts/dataexports\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts data export\" }\r\n ,\"microsoft.usagebilling/accounts/inputs\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts input\" }\r\n ,\"microsoft.usagebilling/accounts/metricexports\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts metric export\" }\r\n ,\"microsoft.usagebilling/accounts/pav2outputs\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts pav2output\" }\r\n ,\"microsoft.usagebilling/accounts/pipelines\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts pipeline\" }\r\n ,\"microsoft.usagebilling/accounts/pipelines/outputselectors\": { \"SingularDisplayName\": \"Microsoft.UsageBilling accounts pipelines output selector\" }\r\n ,\"microsoft.verifiedid/authorities\": { \"SingularDisplayName\": \"Microsoft.VerifiedId authority\" }\r\n ,\"microsoft.videoindexer/accounts\": { \"SingularDisplayName\": \"Azure AI Video Indexer\" }\r\n ,\"microsoft.virtualmachineimages/imagetemplates\": { \"SingularDisplayName\": \"Image template\" }\r\n ,\"microsoft.visualstudio/account\": { \"SingularDisplayName\": \"Azure DevOps organization\" }\r\n ,\"microsoft.vmware/resourcepools\": { \"SingularDisplayName\": \"Microsoft.VMware resource pool\" }\r\n ,\"microsoft.vmware/vcenters\": { \"SingularDisplayName\": \"Microsoft.VMware vcenter\" }\r\n ,\"microsoft.vmware/vcenters/inventoryitems\": { \"SingularDisplayName\": \"Microsoft.VMware vcenters inventory item\" }\r\n ,\"microsoft.vmware/virtualmachines\": { \"SingularDisplayName\": \"Microsoft.VMware virtual machine\" }\r\n ,\"microsoft.vmware/virtualmachinetemplates\": { \"SingularDisplayName\": \"Microsoft.VMware virtual machine template\" }\r\n ,\"microsoft.vmware/virtualnetworks\": { \"SingularDisplayName\": \"Microsoft.VMware virtual network\" }\r\n ,\"microsoft.vmwarecloudsimple/dedicatedcloudnodes\": { \"SingularDisplayName\": \"Microsoft.VMwareCloudSimple dedicated cloud node\" }\r\n ,\"microsoft.vmwarecloudsimple/dedicatedcloudservices\": { \"SingularDisplayName\": \"Microsoft.VMwareCloudSimple dedicated cloud service\" }\r\n ,\"microsoft.vmwarecloudsimple/virtualmachines\": { \"SingularDisplayName\": \"Microsoft.VMwareCloudSimple virtual machine\" }\r\n ,\"microsoft.vnfmanager/devices\": { \"SingularDisplayName\": \"Microsoft.VnfManager device\" }\r\n ,\"microsoft.vnfmanager/vendors\": { \"SingularDisplayName\": \"Microsoft.VnfManager vendor\" }\r\n ,\"microsoft.vnfmanager/vendors/skus\": { \"SingularDisplayName\": \"Microsoft.VnfManager vendors SKU\" }\r\n ,\"microsoft.vnfmanager/vnfs\": { \"SingularDisplayName\": \"Microsoft.VnfManager vnf\" }\r\n ,\"microsoft.voiceservices/communicationsgateways\": { \"SingularDisplayName\": \"Communications Gateway\" }\r\n ,\"microsoft.voiceservices/communicationsgateways/testlines\": { \"SingularDisplayName\": \"Communications Gateway Test Line\" }\r\n ,\"microsoft.vsonline/accounts\": { \"SingularDisplayName\": \"Microsoft.VSOnline account\" }\r\n ,\"microsoft.vsonline/plans\": { \"SingularDisplayName\": \"Visual Studio Online Plan\" }\r\n ,\"microsoft.web/certificates\": { \"SingularDisplayName\": \"Microsoft.Web certificate\" }\r\n ,\"microsoft.web/connectiongateways\": { \"SingularDisplayName\": \"App Service on-premises data gateway\" }\r\n ,\"microsoft.web/connections\": { \"SingularDisplayName\": \"App Service API connection\" }\r\n ,\"microsoft.web/containerapps\": { \"SingularDisplayName\": \"Microsoft.Web container app\" }\r\n ,\"microsoft.web/containerapps/revisions\": { \"SingularDisplayName\": \"Microsoft.Web container apps revision\" }\r\n ,\"microsoft.web/customapis\": { \"SingularDisplayName\": \"Logic apps custom connector\" }\r\n ,\"microsoft.web/deletedsites\": { \"SingularDisplayName\": \"Microsoft.Web deleted site\" }\r\n ,\"microsoft.web/hostingenvironments\": { \"SingularDisplayName\": \"App Service Environment\" }\r\n ,\"microsoft.web/ishostingenvironmentnameavailable\": { \"SingularDisplayName\": \"Microsoft.Web ishostingenvironmentnameavailable\" }\r\n ,\"microsoft.web/kubeenvironments\": { \"SingularDisplayName\": \"App Service Kubernetes Environment\" }\r\n ,\"microsoft.web/logicappstemplate\": { \"SingularDisplayName\": \"Logic Apps Template\" }\r\n ,\"microsoft.web/publishingusers\": { \"SingularDisplayName\": \"Microsoft.Web publishing user\" }\r\n ,\"microsoft.web/serverfarms\": { \"SingularDisplayName\": \"App Service plan\" }\r\n ,\"microsoft.web/sites\": { \"SingularDisplayName\": \"App Service web app\" }\r\n ,\"microsoft.web/sites/slots\": { \"SingularDisplayName\": \"App Service deployment slot\" }\r\n ,\"microsoft.web/sourcecontrols\": { \"SingularDisplayName\": \"Microsoft.Web sourcecontrol\" }\r\n ,\"microsoft.web/staticsites\": { \"SingularDisplayName\": \"Static Web App\" }\r\n ,\"microsoft.weightsandbiases/instances\": { \"SingularDisplayName\": \"Azure Native Weights & Biases Cloud Service\" }\r\n ,\"microsoft.whiteboxcadlprovider/whiteboxresources\": { \"SingularDisplayName\": \"Microsoft.WhiteBoxCadlProvider white box resource\" }\r\n ,\"microsoft.windows365/cloudpcdelegatedmsis\": { \"SingularDisplayName\": \"Microsoft.Windows365 cloud pc delegated msi\" }\r\n ,\"microsoft.windowsesu/multipleactivationkeys\": { \"SingularDisplayName\": \"Microsoft.WindowsESU multiple activation key\" }\r\n ,\"microsoft.windowsiot/deviceservices\": { \"SingularDisplayName\": \"Microsoft.WindowsIoT device service\" }\r\n ,\"microsoft.windowspushnotificationservices/registrations\": { \"SingularDisplayName\": \"Windows Push Notification Service\" }\r\n ,\"microsoft.workloadmonitor/monitors\": { \"SingularDisplayName\": \"Microsoft.WorkloadMonitor monitor\" }\r\n ,\"microsoft.workloadmonitor/monitors/history\": { \"SingularDisplayName\": \"Microsoft.WorkloadMonitor monitors history\" }\r\n ,\"microsoft.workloads/configurationvalidationresults\": { \"SingularDisplayName\": \"Microsoft.Workloads configuration validation result\" }\r\n ,\"microsoft.workloads/connectors\": { \"SingularDisplayName\": \"Microsoft.Workloads connector\" }\r\n ,\"microsoft.workloads/connectors/acssbackups\": { \"SingularDisplayName\": \"Microsoft.Workloads connectors acss backup\" }\r\n ,\"microsoft.workloads/connectors/amsinsights\": { \"SingularDisplayName\": \"Microsoft.Workloads connectors ams insight\" }\r\n ,\"microsoft.workloads/connectors/sapvirtualinstancemonitors\": { \"SingularDisplayName\": \"Microsoft.Workloads connectors sap virtual instance monitor\" }\r\n ,\"microsoft.workloads/epicvirtualinstances\": { \"SingularDisplayName\": \"Virtual Instance for Epic solution\" }\r\n ,\"microsoft.workloads/insights\": { \"SingularDisplayName\": \"Microsoft.Workloads insight\" }\r\n ,\"microsoft.workloads/instancegroupmonitors\": { \"SingularDisplayName\": \"Microsoft.Workloads instance group monitor\" }\r\n ,\"microsoft.workloads/instancehealthdefinitions\": { \"SingularDisplayName\": \"Microsoft.Workloads instance health definition\" }\r\n ,\"microsoft.workloads/instancehealthdefinitions/signaldefinitions\": { \"SingularDisplayName\": \"Microsoft.Workloads instance health definitions signal definition\" }\r\n ,\"microsoft.workloads/instancemonitors\": { \"SingularDisplayName\": \"Microsoft.Workloads instance monitor\" }\r\n ,\"microsoft.workloads/monitors\": { \"SingularDisplayName\": \"Azure Monitor for SAP solutions\" }\r\n ,\"microsoft.workloads/oraclevirtualinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads oracle virtual instance\" }\r\n ,\"microsoft.workloads/oraclevirtualinstances/databaseinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads oracle virtual instances database instance\" }\r\n ,\"microsoft.workloads/phpworkloads\": { \"SingularDisplayName\": \"Microsoft.Workloads php workload\" }\r\n ,\"microsoft.workloads/phpworkloads/wordpressinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads php workloads wordpress instance\" }\r\n ,\"microsoft.workloads/sapdiscoverysites\": { \"SingularDisplayName\": \"Microsoft.Workloads sap discovery site\" }\r\n ,\"microsoft.workloads/sapdiscoverysites/sapinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads sap discovery sites sap instance\" }\r\n ,\"microsoft.workloads/sapdiscoverysites/sapinstances/serverinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads sap discovery sites sap instances server instance\" }\r\n ,\"microsoft.workloads/sapvirtualinstances\": { \"SingularDisplayName\": \"Virtual Instance for SAP solutions\" }\r\n ,\"microsoft.workloads/sapvirtualinstances/applicationinstances\": { \"SingularDisplayName\": \"App server instance for SAP solutions\" }\r\n ,\"microsoft.workloads/sapvirtualinstances/centralinstances\": { \"SingularDisplayName\": \"Central service instance for SAP solutions\" }\r\n ,\"microsoft.workloads/sapvirtualinstances/databaseinstances\": { \"SingularDisplayName\": \"Database for SAP solutions\" }\r\n ,\"microsoft.workloads/virtualinstances\": { \"SingularDisplayName\": \"Microsoft.Workloads virtual instance\" }\r\n ,\"microsoft.workloads/virtualinstances/components\": { \"SingularDisplayName\": \"Microsoft.Workloads virtual instances component\" }\r\n ,\"microsoft.workloads/workloadinstance\": { \"SingularDisplayName\": \"My Resource\" }\r\n ,\"microsoft.zerotrustsegmentation/segmentationmanagers\": { \"SingularDisplayName\": \"Segmentation Manager\" }\r\n ,\"mongodb.atlas/organizations\": { \"SingularDisplayName\": \"MongoDB Atlas Organization\" }\r\n ,\"neon.postgres/organizations\": { \"SingularDisplayName\": \"Neon Serverless Postgres Organization\" }\r\n ,\"newrelic.observability/monitors\": { \"SingularDisplayName\": \"New Relic\" }\r\n ,\"nginx.nginxplus/nginxdeployments\": { \"SingularDisplayName\": \"NGINXaaS\" }\r\n ,\"oracle.database/autonomousdatabases\": { \"SingularDisplayName\": \"Autonomous Database\" }\r\n ,\"oracle.database/basedb\": { \"SingularDisplayName\": \"Autonomous Database\" }\r\n ,\"oracle.database/cloudexadatainfrastructures\": { \"SingularDisplayName\": \"Oracle Exadata Infrastructure\" }\r\n ,\"oracle.database/cloudvmclusters\": { \"SingularDisplayName\": \"Oracle Exadata VM Cluster\" }\r\n ,\"oracle.database/exadbvmclusters\": { \"SingularDisplayName\": \"Oracle Exascale VM Cluster\" }\r\n ,\"oracle.database/exascaledbstoragevaults\": { \"SingularDisplayName\": \"Oracle Exascale DB Storage Vault\" }\r\n ,\"oracle.database/networkanchors\": { \"SingularDisplayName\": \"Network Anchor\" }\r\n ,\"oracle.database/oraclesubscriptions\": { \"SingularDisplayName\": \"OracleSubscription\" }\r\n ,\"oracle.database/resourceanchors\": { \"SingularDisplayName\": \"Resource Anchor\" }\r\n ,\"paloaltonetworks.cloudngfw/firewalls\": { \"SingularDisplayName\": \"Cloud NGFW by Palo Alto Networks\" }\r\n ,\"paloaltonetworks.cloudngfw/globalrulestacks\": { \"SingularDisplayName\": \"Global Rulestack\" }\r\n ,\"paloaltonetworks.cloudngfw/localrulestacks\": { \"SingularDisplayName\": \"Local Rulestack for Cloud NGFW by Palo Alto Networks\" }\r\n ,\"pinecone.vectordb/organizations\": { \"SingularDisplayName\": \"Azure Native Pinecone Cloud Service\" }\r\n ,\"purestorage.block/reservations\": { \"SingularDisplayName\": \"Azure Native Pure Storage Cloud Service\" }\r\n ,\"purestorage.block/storagepools\": { \"SingularDisplayName\": \"Storage pool\" }\r\n ,\"purestorage.block/storagepools/avsstoragecontainers\": { \"SingularDisplayName\": \"PureStorage.Block storage pools avs storage container\" }\r\n })[tolower(id)]\r\n}\r\n", + "$fxv#4": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData/Internal')\r\n_resource_type_5(id: string) {\r\n dynamic({\r\n \"qumulo.qaas/storages\": { \"SingularDisplayName\": \"Qumulo.QaaS storage\" }\r\n ,\"qumulo.storage/filesystems\": { \"SingularDisplayName\": \"Azure Native Qumulo Scalable File Service\" }\r\n ,\"solarwinds.observability/organizations\": { \"SingularDisplayName\": \"SolarWinds Observability\" }\r\n ,\"splitio.experimentation/experimentationworkspaces\": { \"SingularDisplayName\": \"Split Experimentation Workspace\" }\r\n ,\"wandisco.fusion/migrators\": { \"SingularDisplayName\": \"LiveData Migrator\" }\r\n ,\"wandisco.fusion/migrators/datatransferagents\": { \"SingularDisplayName\": \"Data Transfer Agent\" }\r\n ,\"wandisco.fusion/migrators/exclusiontemplates\": { \"SingularDisplayName\": \"Exclusion\" }\r\n ,\"wandisco.fusion/migrators/livedatamigrations\": { \"SingularDisplayName\": \"Migration\" }\r\n ,\"wandisco.fusion/migrators/metadatamigrations\": { \"SingularDisplayName\": \"Metadata Migration\" }\r\n ,\"wandisco.fusion/migrators/metadatatargets\": { \"SingularDisplayName\": \"Metadata Target\" }\r\n ,\"wandisco.fusion/migrators/pathmappings\": { \"SingularDisplayName\": \"Path Mapping\" }\r\n ,\"wandisco.fusion/migrators/targets\": { \"SingularDisplayName\": \"Target\" }\r\n ,\"wandisco.fusion/migrators/verifications\": { \"SingularDisplayName\": \"Verification\" }\r\n })[tolower(id)]\r\n}\r\n", + "$fxv#5": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n// resource_type\r\n.create-or-alter function \r\nwith (docstring = 'Return details about the specified ID.', folder = 'OpenData')\r\nresource_type(id: string) {\r\n coalesce(_resource_type_1(id), _resource_type_2(id), _resource_type_3(id), _resource_type_4(id), _resource_type_5(id))\r\n}\r\n", + "$fxv#6": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Common utility functions\r\n//\r\n// TIP: Use Ctrl+K,Ctrl+0 to collapse all regions in VS Code\r\n//======================================================================================================================\r\n\r\n\r\n//===| Date functions |=================================================================================================\r\n\r\n// monthstring\r\n.create-or-alter function \r\nwith (docstring = @'Returns the name of the month for the specified date (e.g. Jan or January)', folder =@'Common') \r\nmonthstring(['date']: datetime, length: int = 9)\r\n{\r\n substring(dynamic(['January','February','March','April','May','June','July','August','September','October','November','December'])[getmonth(['date']) - 1], 0, length)\r\n}\r\n\r\n// datestring\r\n.create-or-alter function \r\nwith (docstring = @'Converts 2 dates into a simple, user-friendly date range (e.g. Jan 1-Jan 3)', folder =@'Common') \r\ndatestring(start: datetime, end: datetime = datetime('0001-01-01'))\r\n{\r\n let month = (d: datetime) { monthstring(d, 3) };\r\n let endDate = iff(end == datetime('0001-01-01'), start, end);\r\n let sameDate = startofday(start) == startofday(endDate);\r\n let sameMonth = startofmonth(start) == startofmonth(endDate);\r\n let sameYear = startofyear(start) == startofyear(endDate);\r\n let fullMonth = startofday(start) == startofmonth(start) and startofday(endDate) == startofday(endofmonth(endDate));\r\n let fullYear = startofday(start) == startofyear(start) and startofday(endDate) == startofday(endofyear(endDate));\r\n let currentYear = sameYear and startofyear(start) == startofyear(now());\r\n case(\r\n // Full year | yyyy (same year) / yyyy-yyyy (diff years)\r\n fullYear,\r\n strcat(getyear(start), iff(sameYear, '', strcat('-', getyear(endDate)))),\r\n // 1 full mo, same year | Mmm yyyy\r\n fullMonth and sameMonth and sameYear,\r\n strcat(month(start), ' ', getyear(start)),\r\n // 2+ full mo, same year | Mmm-Mmm (current year) / Mmm-Mmm yyyy (other year)\r\n fullMonth and sameYear,\r\n strcat(month(start), '-', month(endDate), iff(currentYear, '', strcat(' ', getyear(endDate)))),\r\n // Full mo, diff year | Mmm yyyy-Mmm yyyy\r\n fullMonth and not(sameYear),\r\n strcat(month(start), ' ', getyear(start), '-', month(endDate), ' ', getyear(endDate)),\r\n // Same date | Mmm d (current year) / Mmm d, yyyy (other year)\r\n sameDate,\r\n strcat(month(start), ' ', dayofmonth(start), iff(currentYear, '', strcat(', ', getyear(endDate)))),\r\n // 1 partial M, same Y | Mmm d-d (current year) / Mmm d-d, yyyy (other year)\r\n not(fullMonth) and sameMonth and sameYear,\r\n strcat(month(start), ' ', dayofmonth(start), '-', dayofmonth(endDate), iff(currentYear, '', strcat(' ', getyear(endDate)))),\r\n // 2+ partial M, same Y | Mmm d-Mmm d (current year) / Mmm d-Mmm d, yyyy (other year)\r\n not(fullMonth) and not(sameMonth) and sameYear,\r\n strcat(month(start), ' ', dayofmonth(start), '-', month(endDate), ' ', dayofmonth(endDate), iff(currentYear, '', strcat(', ', getyear(endDate)))),\r\n // All other cases | Mmm d, yyyy-Mmm d, yyyy\r\n strcat(month(start), ' ', dayofmonth(start), ', ', getyear(start), '-', month(endDate), ' ', dayofmonth(endDate), ', ', getyear(endDate))\r\n )\r\n}\r\n\r\n// daterange\r\n.create-or-alter function \r\nwith (docstring = @'DEPRECATED: Please use datestring(); function will be removed on or after the Jan 2026 release', folder =@'Common') \r\ndaterange(start: datetime, end: datetime = datetime('0001-01-01'))\r\n{\r\n datestring(start, end)\r\n}\r\n\r\n// monthsago\r\n.create-or-alter function \r\nwith (docstring = 'DEPRECATED: Please use startofmonth(now(), -<# of months>); function will be removed on or after the Jan 2026 release', folder = 'Common')\r\nmonthsago(months: int)\r\n{\r\n datetime_add('month', -months, startofmonth(now()))\r\n}\r\n\r\n\r\n//===| Number functions |===============================================================================================\r\n// NOTE: Must be defined before string converters\r\n\r\n// delta\r\n.create-or-alter function \r\nwith (docstring = @'Compares 2 values and returns the percentage change from oldval to newval', folder =@'Common') \r\ndelta(oldval: double, newval: double)\r\n{\r\n (newval - todouble(oldval))/oldval\r\n}\r\n\r\n// percentOfTotal\r\n// NOTE: Must be before percent() function\r\n.create-or-alter function \r\nwith (docstring = @'Calculates the percentage of each record based on a required Count column', folder =@'Common') \r\npercentOfTotal(t: (Count: long), tot: long)\r\n{\r\n let total = todouble(tot);\r\n t \r\n | extend Percent = round(Count / total * 100, 3) \r\n | order by Count desc\r\n}\r\n\r\n// percent\r\n.create-or-alter function \r\nwith (docstring = @'Calculates the percentage of each record based on a required Count column', folder =@'Common') \r\npercent(t: (Count: long))\r\n{\r\n let total = todouble(toscalar(t | summarize sum(Count)));\r\n percentOfTotal(t, total)\r\n}\r\n\r\n// plusminus\r\n.create-or-alter function \r\nwith (docstring = 'Shows a +/- sign based on the direction of the number', folder = 'Common')\r\nplusminus(val: string)\r\n{\r\n let neg = substring(val, 0, 1) == '-';\r\n iff(neg, val, strcat('+', val))\r\n}\r\n\r\n// updown\r\n.create-or-alter function \r\nwith (docstring = 'Shows an up/down arrow based on the direction of the number', folder = 'Common')\r\nupdown(val: string)\r\n{\r\n // TODO: Handle 0\r\n let neg = substring(val, 0, 1) == '-';\r\n iff(neg, strcat('↓', substring(val, 1)), strcat('↑', val))\r\n}\r\n\r\n\r\n//===| String functions |===============================================================================================\r\n\r\n// percentstring\r\n// NOTE: Must be defined before deltastring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate a percentage and render as a string', folder = 'Common')\r\npercentstring(num: double, total: double = 1.0, places: int = 9)\r\n{\r\n let value = 1.0 * num / total * 100;\r\n strcat(case(\r\n places != 9, round(value, places),\r\n value < 10, round(value, 2),\r\n round(value, 1)\r\n ), '%')\r\n}\r\n\r\n//----------------------------------------------------------------------------------------------------------------------\r\n\r\n// arraystring\r\n.create-or-alter function \r\nwith (docstring = 'Convert an array to a comma-delimited string', folder = 'Common')\r\narraystring(arr: dynamic)\r\n{\r\n replace_string(replace_regex(replace_regex(replace_regex(replace_regex(replace_regex(\r\n tostring(arr)\r\n , @'^\\[\"', '')\r\n , @'\"\\]$', '')\r\n , @'^, ', '')\r\n , @', $', '')\r\n , @'^\\[]$', '')\r\n , '\",\"', ', ')\r\n}\r\n\r\n// deltastring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate a delta percentage and render as a string', folder = 'Common')\r\ndeltastring(oldval: double, newval: double, places: int = 1, useArrows: bool = false)\r\n{\r\n let d = delta(oldval, newval);\r\n strcat(case(useArrows and d > 0, '↑', useArrows and d < 0, '↓', d < 0, '-', ''), percentstring(abs(d), 1, places))\r\n}\r\n\r\n// diffstring\r\n.create-or-alter function \r\nwith (docstring = 'Calculate the difference and render as a string', folder = 'Common')\r\ndiffstring(oldval: double, newval: double, places: int = 1)\r\n{\r\n plusminus(round(newval - oldval, places))\r\n}\r\n\r\n// numberstring\r\n.create-or-alter function \r\nwith (docstring = 'Convert a number to a string', folder = 'Common')\r\nnumberstring(num: double, abbrev: bool = true)\r\n{\r\n replace_regex(case(\r\n num >= 10000000000000, strcat(round(1.0 * num / 1000000000000, 1), 'T'),\r\n num >= 1000000000000, strcat(round(1.0 * num / 1000000000000, 2), 'T'),\r\n num >= 10000000000, strcat(round(1.0 * num / 1000000000, 1), 'B'),\r\n num >= 1000000000, strcat(round(1.0 * num / 1000000000, 2), 'B'),\r\n num >= 10000000, strcat(round(1.0 * num / 1000000, 1), 'M'),\r\n num >= 1000000, strcat(round(1.0 * num / 1000000, 2), 'M'),\r\n num >= 10000, strcat(round(1.0 * num / 1000, 1), 'K'),\r\n // Kusto doesn't support back-refs yet -- num > 1000, replace_regex(tostring(num), @'(\\d)(?=(\\d{3})+\\.)', @'\\1,'), // See https://docs.microsoft.com/azure/data-explorer/kusto/query/re2-library\r\n num > 1000, replace_regex(tostring(num), @'([0-9]{3})$', @',\\1'), //num / 1000, ',', substring(tostring(num), 0) - (num / 1000 * 1000)),\r\n tostring(num)\r\n ), @'\\.0$', '')\r\n}\r\n\r\n\r\n//===| Other |==========================================================================================================\r\n\r\n// ifempty\r\n.create-or-alter function \r\nwith (docstring = 'Replaces an empty value with the specified default value', folder = 'Common')\r\nifempty(val: dynamic, defaultVal: dynamic)\r\n{\r\n iff(isempty(val), defaultVal, val)\r\n}\r\n", + "$fxv#7": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Ingestion database\r\n// Used for data ingestion, normalization, and cleansing.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n//===| Settings |=======================================================================================================\r\n\r\n.create-merge table HubSettingsLog (\r\n version: string,\r\n scopes: dynamic,\r\n retention: dynamic\r\n)\r\n\r\n//----------------------------------------------------------------------------------------------------------------------\r\n\r\n// HubSettings function\r\n.create-or-alter function\r\nwith (docstring='Gets the latest version of hub settings.', folder='Settings')\r\nHubSettings()\r\n{\r\n HubSettingsLog\r\n | extend timestamp = ingestion_time()\r\n | summarize arg_max(timestamp, *)\r\n}\r\n\r\n//----------------------------------------------------------------------------------------------------------------------\r\n\r\n// HubScopes function\r\n.create-or-alter function\r\nwith (docstring='Gets the currently configured scopes.', folder='Settings')\r\nHubScopes()\r\n{\r\n HubSettings\r\n | project scopes\r\n | mv-expand scopes\r\n}\r\n\r\n\r\n//===| Open data |======================================================================================================\r\n\r\n// PricingUnits -- Create table if it doesn't exist\r\n.create-merge table PricingUnits ( ignore: string )\r\n\r\n// PricingUnits -- Remove all columns\r\n.alter table PricingUnits ( ignore: string )\r\n\r\n// PricingUnits -- Redefine all columns to change types\r\n.alter table PricingUnits (\r\n x_PricingUnitDescription: string,\r\n x_PricingBlockSize: real,\r\n PricingUnit: string\r\n)\r\n\r\n// Regions\r\n.create-merge table Regions(\r\n ResourceLocation: string,\r\n RegionId: string,\r\n RegionName: string\r\n)\r\n\r\n// ResourceTypes\r\n.create-merge table ResourceTypes(\r\n x_ResourceType: string,\r\n SingularDisplayName: string,\r\n PluralDisplayName: string,\r\n LowerSingularDisplayName: string,\r\n LowerPluralDisplayName: string,\r\n IsPreview: bool,\r\n Description: string,\r\n IconUri: string\r\n)\r\n\r\n// Services\r\n.create-merge table Services(\r\n x_ConsumedService: string,\r\n x_ResourceType: string,\r\n ServiceName: string,\r\n ServiceCategory: string,\r\n ServiceSubcategory: string,\r\n PublisherName: string,\r\n x_PublisherCategory: string,\r\n x_Environment: string,\r\n x_ServiceModel: string\r\n)\r\n\r\n//----------------------------------------------------------------------------------------------------------------------\r\n\r\n// parse_resourceid\r\n.create-or-alter function\r\nwith (docstring = 'Parses an Azure resource ID to extract resource attributes like the name, type, resource group, and subaccount ID.', folder = 'Common')\r\nparse_resourceid(resourceId: string) {\r\n let ResourceId = tolower(resourceId);\r\n // let ResourceId = tolower('/providers/Microsoft.BillingBenefits/savingsPlanOrders/2d2e284b-0638-427e-b8c6-1b874d4f17c8/sp/xxx');\r\n let SubAccountId = tostring(extract('/subscriptions/[^/]+', 1, ResourceId));\r\n let x_ResourceGroupName = tostring(extract('/resourcegroups/[^/]+', 1, ResourceId));\r\n let providerPath = iff(ResourceId !contains '/providers/', '', split(iff(ResourceId startswith '/subscriptions/', strcat('/providers/microsoft.resources/', ResourceId), ResourceId), '/providers/')[-1]);\r\n let x_ResourceProvider = iff(isempty(providerPath), '', split(providerPath, '/')[0]);\r\n let tmp_ResourceProviderPath = iff(isempty(providerPath), '', substring(providerPath, strlen(x_ResourceProvider) + 1));\r\n let segments = split(tmp_ResourceProviderPath, '/');\r\n let ResourceName = trim(@'/+', replace_string(strcat_array(array_iff(\r\n dynamic([false, true, false, true, false, true, false, true, false, true, false, true, false, true, false, true, false, true]),\r\n segments, dynamic([])), '/'), '//', '/'));\r\n let x_ResourceTypePath = trim(@'/+', replace_string(strcat_array(array_iff(\r\n dynamic([true, false, true, false, true, false, true, false, true, false, true, false, true, false, true, false, true, false]),\r\n segments, dynamic([])), '/'), '//', '/'));\r\n let xRT = iff(isempty(x_ResourceProvider) or isempty(x_ResourceTypePath), '', strcat(x_ResourceProvider, '/', x_ResourceTypePath));\r\n // TODO: Remove ResourceType in 0.9\r\n bag_pack('ResourceId', ResourceId, 'ResourceName', ResourceName, 'ResourceType', xRT, 'SubAccountId', SubAccountId, 'x_ResourceGroupName', x_ResourceGroupName, 'x_ResourceProvider', x_ResourceProvider, 'x_ResourceType', xRT)\r\n}\r\n", + "$fxv#8": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Ingestion database\r\n// Used for data ingestion, normalization, and cleansing.\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n//===| ActualCosts |====================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// ActualCosts_raw table -- Create the table if it doesn't exist\r\n.create-merge table ActualCosts_raw ( ignore: string )\r\n\r\n// ActualCosts_raw table -- Remove all columns to allow changing column types\r\n.alter table ActualCosts_raw ( ignore: string )\r\n\r\n// ActualCosts_raw table -- Redefine all columns\r\n.alter table ActualCosts_raw (\r\n AccountName: string,\r\n AccountOwnerId: string,\r\n AdditionalInfo: string,\r\n AvailabilityZone: string,\r\n BillingAccountId: string, \r\n BillingAccountName: string,\r\n BillingCurrency: string,\r\n BillingPeriodEndDate: datetime,\r\n BillingPeriodStartDate: datetime,\r\n BillingProfileId: string,\r\n BillingProfileName: string,\r\n ChargeType: string,\r\n ConsumedService: string,\r\n CostCenter: string,\r\n Cost: real,\r\n Date: datetime,\r\n EffectivePrice: real,\r\n Frequency: string,\r\n InvoiceSection: string,\r\n InvoiceSectionId: string,\r\n IsAzureCreditEligible: bool,\r\n MeterCategory: string,\r\n MeterId: string,\r\n MeterName: string,\r\n MeterRegion: string,\r\n MeterSubCategory: string,\r\n OfferId: string,\r\n PartNumber: string,\r\n PlanName: string,\r\n Product: string,\r\n ProductOrderId: string,\r\n ProductOrderName: string,\r\n PublisherName: string,\r\n PublisherType: string,\r\n Quantity: real,\r\n ReservationId: string,\r\n ReservationName: string,\r\n ResourceGroup: string,\r\n ResourceId: string,\r\n ResourceLocation: string,\r\n ResourceName: string,\r\n ServiceFamily: string,\r\n ServiceInfo1: string,\r\n ServiceInfo2: string,\r\n SubscriptionId: string,\r\n SubscriptionName: string,\r\n Tags: string,\r\n Term: string,\r\n UnitOfMeasure: string,\r\n UnitPrice: real\r\n)\r\n\r\n// ActualCosts_raw ingestion mapping\r\n.create-or-alter table ActualCosts_raw ingestion parquet mapping \"ActualCosts_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"AccountName\", \"Properties\": { \"Field\": \"AccountName\" } },\r\n { \"Column\": \"AccountOwnerId\", \"Properties\": { \"Field\": \"AccountOwnerId\" } },\r\n { \"Column\": \"AdditionalInfo\", \"Properties\": { \"Field\": \"AdditionalInfo\" } },\r\n { \"Column\": \"AvailabilityZone\", \"Properties\": { \"Field\": \"AvailabilityZone\" } },\r\n { \"Column\": \"BillingAccountId\", \"Properties\": { \"Field\": \"BillingAccountId\" } },\r\n { \"Column\": \"BillingAccountName\", \"Properties\": { \"Field\": \"BillingAccountName\" } },\r\n { \"Column\": \"BillingCurrency\", \"Properties\": { \"Field\": \"BillingCurrency\" } },\r\n { \"Column\": \"BillingPeriodEndDate\", \"Properties\": { \"Field\": \"BillingPeriodEndDate\" } },\r\n { \"Column\": \"BillingPeriodStartDate\", \"Properties\": { \"Field\": \"BillingPeriodStartDate\" } },\r\n { \"Column\": \"BillingProfileId\", \"Properties\": { \"Field\": \"BillingProfileId\" } },\r\n { \"Column\": \"BillingProfileName\", \"Properties\": { \"Field\": \"BillingProfileName\" } },\r\n { \"Column\": \"ChargeType\", \"Properties\": { \"Field\": \"ChargeType\" } },\r\n { \"Column\": \"ConsumedService\", \"Properties\": { \"Field\": \"ConsumedService\" } },\r\n { \"Column\": \"CostCenter\", \"Properties\": { \"Field\": \"CostCenter\" } },\r\n { \"Column\": \"Cost\", \"Properties\": { \"Field\": \"Cost\" } },\r\n { \"Column\": \"Date\", \"Properties\": { \"Field\": \"Date\" } },\r\n { \"Column\": \"EffectivePrice\", \"Properties\": { \"Field\": \"EffectivePrice\" } },\r\n { \"Column\": \"Frequency\", \"Properties\": { \"Field\": \"Frequency\" } },\r\n { \"Column\": \"InvoiceSection\", \"Properties\": { \"Field\": \"InvoiceSection\" } },\r\n { \"Column\": \"InvoiceSectionId\", \"Properties\": { \"Field\": \"InvoiceSectionId\" } },\r\n { \"Column\": \"IsAzureCreditEligible\", \"Properties\": { \"Field\": \"IsAzureCreditEligible\" } },\r\n { \"Column\": \"MeterCategory\", \"Properties\": { \"Field\": \"MeterCategory\" } },\r\n { \"Column\": \"MeterId\", \"Properties\": { \"Field\": \"MeterId\" } },\r\n { \"Column\": \"MeterName\", \"Properties\": { \"Field\": \"MeterName\" } },\r\n { \"Column\": \"MeterRegion\", \"Properties\": { \"Field\": \"MeterRegion\" } },\r\n { \"Column\": \"MeterSubCategory\", \"Properties\": { \"Field\": \"MeterSubCategory\" } },\r\n { \"Column\": \"OfferId\", \"Properties\": { \"Field\": \"OfferId\" } },\r\n { \"Column\": \"PartNumber\", \"Properties\": { \"Field\": \"PartNumber\" } },\r\n { \"Column\": \"PlanName\", \"Properties\": { \"Field\": \"PlanName\" } },\r\n { \"Column\": \"Product\", \"Properties\": { \"Field\": \"Product\" } },\r\n { \"Column\": \"ProductOrderId\", \"Properties\": { \"Field\": \"ProductOrderId\" } },\r\n { \"Column\": \"ProductOrderName\", \"Properties\": { \"Field\": \"ProductOrderName\" } },\r\n { \"Column\": \"PublisherName\", \"Properties\": { \"Field\": \"PublisherName\" } },\r\n { \"Column\": \"PublisherType\", \"Properties\": { \"Field\": \"PublisherType\" } },\r\n { \"Column\": \"Quantity\", \"Properties\": { \"Field\": \"Quantity\" } },\r\n { \"Column\": \"ReservationId\", \"Properties\": { \"Field\": \"ReservationId\" } },\r\n { \"Column\": \"ReservationName\", \"Properties\": { \"Field\": \"ReservationName\" } },\r\n { \"Column\": \"ResourceGroup\", \"Properties\": { \"Field\": \"ResourceGroup\" } },\r\n { \"Column\": \"ResourceId\", \"Properties\": { \"Field\": \"ResourceId\" } },\r\n { \"Column\": \"ResourceLocation\", \"Properties\": { \"Field\": \"ResourceLocation\" } },\r\n { \"Column\": \"ResourceName\", \"Properties\": { \"Field\": \"ResourceName\" } },\r\n { \"Column\": \"ServiceFamily\", \"Properties\": { \"Field\": \"ServiceFamily\" } },\r\n { \"Column\": \"ServiceInfo1\", \"Properties\": { \"Field\": \"ServiceInfo1\" } },\r\n { \"Column\": \"ServiceInfo2\", \"Properties\": { \"Field\": \"ServiceInfo2\" } },\r\n { \"Column\": \"SubscriptionId\", \"Properties\": { \"Field\": \"SubscriptionId\" } },\r\n { \"Column\": \"SubscriptionName\", \"Properties\": { \"Field\": \"SubscriptionName\" } },\r\n { \"Column\": \"Tags\", \"Properties\": { \"Field\": \"Tags\" } },\r\n { \"Column\": \"Term\", \"Properties\": { \"Field\": \"Term\" } },\r\n { \"Column\": \"UnitOfMeasure\", \"Properties\": { \"Field\": \"UnitOfMeasure\" } },\r\n { \"Column\": \"UnitPrice\", \"Properties\": { \"Field\": \"UnitPrice\" } }\r\n]\r\n```\r\n\r\n// ActualCosts_raw retention policy (clear historical data)\r\n.alter-merge table ActualCosts_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// ActualCosts_raw retention policy (set the user-defined retention period)\r\n.alter-merge table ActualCosts_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable ActualCosts_raw streaming ingestion (required for Fabric)\r\n.alter table ActualCosts_raw policy streamingingestion disable\r\n\r\n\r\n//===| AmortizedCosts |=================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// AmortizedCosts_raw table -- Create the table if it doesn't exist\r\n.create-merge table AmortizedCosts_raw ( ignore: string )\r\n\r\n// AmortizedCosts_raw table -- Remove all columns to allow changing column types\r\n.alter table AmortizedCosts_raw ( ignore: string )\r\n\r\n// AmortizedCosts_raw table -- Redefine all columns\r\n.alter table AmortizedCosts_raw (\r\n AccountName: string,\r\n AccountOwnerId: string,\r\n AdditionalInfo: string,\r\n AvailabilityZone: string,\r\n BillingAccountId: string, \r\n BillingAccountName: string,\r\n BillingCurrency: string,\r\n BillingPeriodEndDate: datetime,\r\n BillingPeriodStartDate: datetime,\r\n BillingProfileId: string,\r\n BillingProfileName: string,\r\n ChargeType: string,\r\n ConsumedService: string,\r\n CostCenter: string,\r\n Cost: real,\r\n Date: datetime,\r\n EffectivePrice: real,\r\n Frequency: string,\r\n InvoiceSection: string,\r\n InvoiceSectionId: string,\r\n IsAzureCreditEligible: bool,\r\n MeterCategory: string,\r\n MeterId: string,\r\n MeterName: string,\r\n MeterRegion: string,\r\n MeterSubCategory: string,\r\n OfferId: string,\r\n PartNumber: string,\r\n PlanName: string,\r\n Product: string,\r\n ProductOrderId: string,\r\n ProductOrderName: string,\r\n PublisherName: string,\r\n PublisherType: string,\r\n Quantity: real,\r\n ReservationId: string,\r\n ReservationName: string,\r\n ResourceGroup: string,\r\n ResourceId: string,\r\n ResourceLocation: string,\r\n ResourceName: string,\r\n ServiceFamily: string,\r\n ServiceInfo1: string,\r\n ServiceInfo2: string,\r\n SubscriptionId: string,\r\n SubscriptionName: string,\r\n Tags: string,\r\n Term: string,\r\n UnitOfMeasure: string,\r\n UnitPrice: real\r\n)\r\n\r\n// AmortizedCosts_raw ingestion mapping\r\n.create-or-alter table AmortizedCosts_raw ingestion parquet mapping \"AmortizedCosts_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"AccountName\", \"Properties\": { \"Field\": \"AccountName\" } },\r\n { \"Column\": \"AccountOwnerId\", \"Properties\": { \"Field\": \"AccountOwnerId\" } },\r\n { \"Column\": \"AdditionalInfo\", \"Properties\": { \"Field\": \"AdditionalInfo\" } },\r\n { \"Column\": \"AvailabilityZone\", \"Properties\": { \"Field\": \"AvailabilityZone\" } },\r\n { \"Column\": \"BillingAccountId\", \"Properties\": { \"Field\": \"BillingAccountId\" } },\r\n { \"Column\": \"BillingAccountName\", \"Properties\": { \"Field\": \"BillingAccountName\" } },\r\n { \"Column\": \"BillingCurrency\", \"Properties\": { \"Field\": \"BillingCurrency\" } },\r\n { \"Column\": \"BillingPeriodEndDate\", \"Properties\": { \"Field\": \"BillingPeriodEndDate\" } },\r\n { \"Column\": \"BillingPeriodStartDate\", \"Properties\": { \"Field\": \"BillingPeriodStartDate\" } },\r\n { \"Column\": \"BillingProfileId\", \"Properties\": { \"Field\": \"BillingProfileId\" } },\r\n { \"Column\": \"BillingProfileName\", \"Properties\": { \"Field\": \"BillingProfileName\" } },\r\n { \"Column\": \"ChargeType\", \"Properties\": { \"Field\": \"ChargeType\" } },\r\n { \"Column\": \"ConsumedService\", \"Properties\": { \"Field\": \"ConsumedService\" } },\r\n { \"Column\": \"CostCenter\", \"Properties\": { \"Field\": \"CostCenter\" } },\r\n { \"Column\": \"Cost\", \"Properties\": { \"Field\": \"Cost\" } },\r\n { \"Column\": \"Date\", \"Properties\": { \"Field\": \"Date\" } },\r\n { \"Column\": \"EffectivePrice\", \"Properties\": { \"Field\": \"EffectivePrice\" } },\r\n { \"Column\": \"Frequency\", \"Properties\": { \"Field\": \"Frequency\" } },\r\n { \"Column\": \"InvoiceSection\", \"Properties\": { \"Field\": \"InvoiceSection\" } },\r\n { \"Column\": \"InvoiceSectionId\", \"Properties\": { \"Field\": \"InvoiceSectionId\" } },\r\n { \"Column\": \"IsAzureCreditEligible\", \"Properties\": { \"Field\": \"IsAzureCreditEligible\" } },\r\n { \"Column\": \"MeterCategory\", \"Properties\": { \"Field\": \"MeterCategory\" } },\r\n { \"Column\": \"MeterId\", \"Properties\": { \"Field\": \"MeterId\" } },\r\n { \"Column\": \"MeterName\", \"Properties\": { \"Field\": \"MeterName\" } },\r\n { \"Column\": \"MeterRegion\", \"Properties\": { \"Field\": \"MeterRegion\" } },\r\n { \"Column\": \"MeterSubCategory\", \"Properties\": { \"Field\": \"MeterSubCategory\" } },\r\n { \"Column\": \"OfferId\", \"Properties\": { \"Field\": \"OfferId\" } },\r\n { \"Column\": \"PartNumber\", \"Properties\": { \"Field\": \"PartNumber\" } },\r\n { \"Column\": \"PlanName\", \"Properties\": { \"Field\": \"PlanName\" } },\r\n { \"Column\": \"Product\", \"Properties\": { \"Field\": \"Product\" } },\r\n { \"Column\": \"ProductOrderId\", \"Properties\": { \"Field\": \"ProductOrderId\" } },\r\n { \"Column\": \"ProductOrderName\", \"Properties\": { \"Field\": \"ProductOrderName\" } },\r\n { \"Column\": \"PublisherName\", \"Properties\": { \"Field\": \"PublisherName\" } },\r\n { \"Column\": \"PublisherType\", \"Properties\": { \"Field\": \"PublisherType\" } },\r\n { \"Column\": \"Quantity\", \"Properties\": { \"Field\": \"Quantity\" } },\r\n { \"Column\": \"ReservationId\", \"Properties\": { \"Field\": \"ReservationId\" } },\r\n { \"Column\": \"ReservationName\", \"Properties\": { \"Field\": \"ReservationName\" } },\r\n { \"Column\": \"ResourceGroup\", \"Properties\": { \"Field\": \"ResourceGroup\" } },\r\n { \"Column\": \"ResourceId\", \"Properties\": { \"Field\": \"ResourceId\" } },\r\n { \"Column\": \"ResourceLocation\", \"Properties\": { \"Field\": \"ResourceLocation\" } },\r\n { \"Column\": \"ResourceName\", \"Properties\": { \"Field\": \"ResourceName\" } },\r\n { \"Column\": \"ServiceFamily\", \"Properties\": { \"Field\": \"ServiceFamily\" } },\r\n { \"Column\": \"ServiceInfo1\", \"Properties\": { \"Field\": \"ServiceInfo1\" } },\r\n { \"Column\": \"ServiceInfo2\", \"Properties\": { \"Field\": \"ServiceInfo2\" } },\r\n { \"Column\": \"SubscriptionId\", \"Properties\": { \"Field\": \"SubscriptionId\" } },\r\n { \"Column\": \"SubscriptionName\", \"Properties\": { \"Field\": \"SubscriptionName\" } },\r\n { \"Column\": \"Tags\", \"Properties\": { \"Field\": \"Tags\" } },\r\n { \"Column\": \"Term\", \"Properties\": { \"Field\": \"Term\" } },\r\n { \"Column\": \"UnitOfMeasure\", \"Properties\": { \"Field\": \"UnitOfMeasure\" } },\r\n { \"Column\": \"UnitPrice\", \"Properties\": { \"Field\": \"UnitPrice\" } }\r\n]\r\n```\r\n\r\n// AmortizedCosts_raw retention policy (clear historical data)\r\n.alter-merge table AmortizedCosts_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// AmortizedCosts_raw retention policy (set the user-defined retention period)\r\n.alter-merge table AmortizedCosts_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable AmortizedCosts_raw streaming ingestion (required for Fabric)\r\n.alter table AmortizedCosts_raw policy streamingingestion disable\r\n\r\n\r\n//===| CommitmentDiscountUsage |========================================================================================\r\n// Supported versions:\r\n// - MS EA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-ea\r\n// - MS MCA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-mca\r\n//======================================================================================================================\r\n\r\n// CommitmentDiscountUsage_raw table -- Create the table if it doesn't exist\r\n.create-merge table CommitmentDiscountUsage_raw ( ignore: string )\r\n\r\n// CommitmentDiscountUsage_raw table -- Remove all columns to allow changing column types\r\n.alter table CommitmentDiscountUsage_raw ( ignore: string )\r\n\r\n// CommitmentDiscountUsage_raw table -- Redefine all columns\r\n.alter table CommitmentDiscountUsage_raw (\r\n InstanceFlexibilityGroup: string,\r\n InstanceFlexibilityRatio: real,\r\n InstanceId: string,\r\n Kind: string,\r\n ReservationId: string,\r\n ReservationOrderId: string,\r\n ReservedHours: real,\r\n SkuName: string,\r\n TotalReservedQuantity: real,\r\n UsageDate: datetime,\r\n UsedHours: real,\r\n x_SourceName: string, // Hubs v1_0+\r\n x_SourceProvider: string, // Hubs v1_0+\r\n x_SourceType: string, // Hubs v1_0+\r\n x_SourceVersion: string // Hubs v1_0+\r\n)\r\n\r\n// CommitmentDiscountUsage_raw ingestion mapping\r\n.create-or-alter table CommitmentDiscountUsage_raw ingestion parquet mapping \"CommitmentDiscountUsage_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"InstanceFlexibilityGroup\", \"Properties\": { \"Field\": \"InstanceFlexibilityGroup\" } },\r\n { \"Column\": \"InstanceFlexibilityRatio\", \"Properties\": { \"Field\": \"InstanceFlexibilityRatio\" } },\r\n { \"Column\": \"InstanceId\", \"Properties\": { \"Field\": \"InstanceId\" } },\r\n { \"Column\": \"Kind\", \"Properties\": { \"Field\": \"Kind\" } },\r\n { \"Column\": \"ReservationId\", \"Properties\": { \"Field\": \"ReservationId\" } },\r\n { \"Column\": \"ReservationOrderId\", \"Properties\": { \"Field\": \"ReservationOrderId\" } },\r\n { \"Column\": \"ReservedHours\", \"Properties\": { \"Field\": \"ReservedHours\" } },\r\n { \"Column\": \"SkuName\", \"Properties\": { \"Field\": \"SkuName\" } },\r\n { \"Column\": \"TotalReservedQuantity\", \"Properties\": { \"Field\": \"TotalReservedQuantity\" } },\r\n { \"Column\": \"UsageDate\", \"Properties\": { \"Field\": \"UsageDate\" } },\r\n { \"Column\": \"UsedHours\", \"Properties\": { \"Field\": \"UsedHours\" } },\r\n { \"Column\": \"x_SourceName\", \"Properties\": { \"Field\": \"x_SourceName\" } },\r\n { \"Column\": \"x_SourceProvider\", \"Properties\": { \"Field\": \"x_SourceProvider\" } },\r\n { \"Column\": \"x_SourceType\", \"Properties\": { \"Field\": \"x_SourceType\" } },\r\n { \"Column\": \"x_SourceVersion\", \"Properties\": { \"Field\": \"x_SourceVersion\" } }\r\n]\r\n```\r\n\r\n// CommitmentDiscountUsage_raw retention policy (clear historical data)\r\n.alter-merge table CommitmentDiscountUsage_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// CommitmentDiscountUsage_raw retention policy (set the user-defined retention period)\r\n.alter-merge table CommitmentDiscountUsage_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable CommitmentDiscountUsage_raw streaming ingestion (required for Fabric)\r\n.alter table CommitmentDiscountUsage_raw policy streamingingestion disable\r\n\r\n\r\n//===| Costs |==========================================================================================================\r\n// Supported versions:\r\n// - MS: 1.0, 1.0-preview(v1) -- See https://aka.ms/costmgmt/exports/focus\r\n// - AWS: 1.0 -- See https://docs.aws.amazon.com/cur/latest/userguide/table-dictionary-focus-1-0-aws-columns.html\r\n// - GCP: Jan-Jun 2024 -- See https://cloud.google.com/resources/google-cloud-focus?e=48754805&hl=en\r\n// Links to (Aug 2024): https://services.google.com/fh/files/misc/focus_guide_v1.pdf\r\n// See also:\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/standard-usage\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/detailed-usage\r\n// - OCI: 1.0 -- See https://docs.oracle.com/iaas/Content/Billing/Concepts/costusagereportsoverview.htm#costreports__focus-cost-report-schema\r\n// - Tencent: 1.0 -- See https://www.tencentcloud.com/document/product/555/67495 / https://www.tencentcloud.com/document/product/555/67496\r\n// - Alibaba: 1.0 -- See https://www.alibabacloud.com/help/en/user-center/user-guide/export-alibaba-cloud-standard-billing-focus\r\n//\r\n// Support for non-Azure data is limited to ingestion only. Data is not transformed across versions.\r\n//======================================================================================================================\r\n\r\n// Costs_raw table -- Create the table if it doesn't exist\r\n.create-merge table Costs_raw ( ignore: string )\r\n\r\n// Costs_raw table -- Remove all columns to allow changing column types\r\n.alter table Costs_raw ( ignore: string )\r\n\r\n// Costs_raw table -- Redefine all columns\r\n.alter table Costs_raw (\r\n AvailabilityZone: string, // FOCUS 0.5+\r\n BilledCost: real, // FOCUS 0.5+\r\n BillingAccountId: string, // FOCUS 0.5+\r\n BillingAccountName: string, // FOCUS 0.5+\r\n BillingAccountType: string, // Azure 1.0-preview(v1)+\r\n BillingCurrency: string, // FOCUS 0.5+\r\n BillingPeriodEnd: datetime, // FOCUS 0.5+\r\n BillingPeriodStart: datetime, // FOCUS 0.5+\r\n CapacityReservationId: string, // FOCUS 1.1+\r\n CapacityReservationStatus: string, // FOCUS 1.1+\r\n ChargeCategory: string, // FOCUS 1.0-preview+\r\n ChargeClass: string, // FOCUS 1.0+\r\n ChargeDescription: string, // FOCUS 1.0+\r\n ChargeFrequency: string, // FOCUS 1.0+\r\n ChargePeriodEnd: datetime, // FOCUS 0.5+\r\n ChargePeriodStart: datetime, // FOCUS 0.5+\r\n ChargeSubcategory: string, // FOCUS 1.0-preview only\r\n CommitmentDiscountCategory: string, // FOCUS 1.0-preview+\r\n CommitmentDiscountId: string, // FOCUS 1.0-preview+\r\n CommitmentDiscountName: string, // FOCUS 1.0-preview+\r\n CommitmentDiscountQuantity: real, // FOCUS 1.1+\r\n CommitmentDiscountStatus: string, // FOCUS 1.0+\r\n CommitmentDiscountType: string, // FOCUS 1.0-preview+\r\n CommitmentDiscountUnit: string, // FOCUS 1.1+\r\n ConsumedQuantity: real, // FOCUS 1.0+\r\n ConsumedUnit: string, // FOCUS 1.0+\r\n ContractedCost: real, // FOCUS 1.0+\r\n ContractedUnitPrice: real, // FOCUS 1.0+\r\n EffectiveCost: real, // FOCUS 1.0-preview+\r\n InvoiceId: string, // FOCUS 1.2+\r\n InvoiceIssuerName: string, // FOCUS 0.5+\r\n ListCost: real, // FOCUS 1.0-preview+\r\n ListUnitPrice: real, // FOCUS 1.0-preview+\r\n PricingCategory: string, // FOCUS 1.0-preview+\r\n PricingCurrency: string, // FOCUS 1.2+\r\n PricingQuantity: real, // FOCUS 1.0-preview+\r\n PricingUnit: string, // FOCUS 1.0-preview+\r\n ProviderName: string, // FOCUS 0.5+\r\n PublisherName: string, // FOCUS 0.5+\r\n Region: string, // FOCUS 0.5-1.0-preview (deprecated)\r\n RegionId: string, // FOCUS 1.0+\r\n RegionName: string, // FOCUS 1.0+\r\n ResourceId: string, // FOCUS 0.5+\r\n ResourceName: string, // FOCUS 0.5+\r\n ResourceType: string, // FOCUS 1.0-preview+\r\n ServiceCategory: string, // FOCUS 0.5+\r\n ServiceName: string, // FOCUS 0.5+\r\n ServiceSubcategory: string, // FOCUS 1.1+\r\n SkuId: string, // FOCUS 1.0-preview+\r\n SkuMeter: string, // FOCUS 1.1+\r\n SkuPriceDetails: string, // FOCUS 1.1+\r\n SkuPriceId: string, // FOCUS 1.0-preview+\r\n SubAccountId: string, // FOCUS 0.5+\r\n SubAccountName: string, // FOCUS 0.5+\r\n SubAccountType: string, // Azure 1.0-preview(v1)+\r\n Tags: string, // FOCUS 1.0-preview+\r\n UsageAmount: real, // GCP Jan 2024 -- Removed Mar 2024 (UsageQuantity)\r\n UsageQuantity: real, // FOCUS 1.0-preview only\r\n UsageUnit: string, // FOCUS 1.0-preview only\r\n x_AccountId: string, // Azure 1.0-preview(v1)+\r\n x_AccountName: string, // Azure 1.0-preview(v1)+\r\n x_AccountOwnerId: string, // Azure 1.0-preview(v1)+\r\n x_AmortizationClass: string, // Azure 1.2-preview+\r\n x_BilledCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_BilledUnitPrice: real, // Azure 1.0-preview(v1)+\r\n x_BillingAccountId: string, // Azure 1.0-preview(v1)+\r\n x_BillingAccountName: string, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRate: real, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRateDate: datetime, // Azure 1.0-preview(v1)+\r\n x_BillingItemCode: string, // Alibaba 1.0+\r\n x_BillingItemName: string, // Alibaba 1.0+\r\n x_BillingProfileId: string, // Azure 1.0-preview(v1)+\r\n x_BillingProfileName: string, // Azure 1.0-preview(v1)+\r\n x_ChargeId: string, // Azure 1.0-preview(v1) only\r\n x_CommodityCode: string, // Alibaba 1.0+\r\n x_CommodityName: string, // Alibaba 1.0+\r\n x_ComponentName: string, // Tencent 1.0+\r\n x_ComponentType: string, // Tencent 1.0+\r\n x_ContractedCostInUsd: real, // Azure 1.0+\r\n x_Cost: real, // GCP Jan 2024 -- Removed Jun 2024 (ContractedCost)\r\n x_CostAllocationRuleName: string, // Azure 1.0-preview(v1)+\r\n x_CostCategories: string, // AWS 1.0 (JSON)\r\n x_CostCenter: string, // Azure 1.0-preview(v1)+\r\n x_CostType: string, // GCP Jan 2024\r\n x_Credits: string, // GCP Jan 2024\r\n x_CurrencyConversionRate: real, // GCP Jun 2024\r\n x_CustomerId: string, // Azure 1.0-preview(v1)+\r\n x_CustomerName: string, // Azure 1.0-preview(v1)+\r\n x_Discount: string, // AWS 1.0 (JSON)\r\n x_EffectiveCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_EffectiveUnitPrice: real, // Azure 1.0-preview(v1)+\r\n x_ExportTime: datetime, // GCP Jan 2024 / Tencent 1.0+\r\n x_InstanceID: string, // Alibaba 1.0+\r\n x_InvoiceId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceIssuerId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionName: string, // Azure 1.0-preview(v1)+\r\n x_ListCostInUsd: real, // Azure 1.0-preview(v1)+\r\n x_Location: string, // GCP Jan 2024\r\n x_OnDemandCost: real, // Azure 1.0-preview(v1) only\r\n x_OnDemandCostInUsd: real, // Azure 1.0-preview(v1) only\r\n x_OnDemandUnitPrice: real, // Azure 1.0-preview(v1) only\r\n x_Operation: string, // AWS 1.0\r\n x_OwnerAccountID: string, // Tencent 1.0+\r\n x_PartnerCreditApplied: string, // Azure 1.0-preview(v1)+\r\n x_PartnerCreditRate: string, // Azure 1.0-preview(v1)+\r\n x_PricingBlockSize: real, // Azure 1.0-preview(v1)+\r\n x_PricingCurrency: string, // Azure 1.0-preview(v1)-1.0r2\r\n x_PricingSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_PricingUnitDescription: string, // Azure 1.0-preview(v1)+\r\n x_Project: string, // GCP Jan 2024\r\n x_PublisherCategory: string, // Azure 1.0-preview(v1)+\r\n x_PublisherId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceGroupName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceType: string, // Azure 1.0-preview(v1)+\r\n x_ServiceCode: string, // AWS 1.0\r\n x_ServiceId: string, // GCP Jan 2024\r\n x_ServiceModel: string, // Azure 1.2-preview+\r\n x_ServicePeriodEnd: datetime, // Azure 1.0-preview(v1)+\r\n x_ServicePeriodStart: datetime, // Azure 1.0-preview(v1)+\r\n x_SkuDescription: string, // Azure 1.0-preview(v1)+\r\n x_SkuDetails: string, // Azure 1.0-preview(v1)-1.2-preview\r\n x_SkuIsCreditEligible: bool, // Azure 1.0-preview(v1)+\r\n x_SkuMeterCategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterId: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterName: string, // Azure 1.0-preview(v1)-1.0r2\r\n x_SkuMeterSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuOfferId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOrderId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOrderName: string, // Azure 1.0-preview(v1)+\r\n x_SkuPartNumber: string, // Azure 1.0-preview(v1)+\r\n x_SkuPlanName: string, // Azure 1.2-preview+\r\n x_SkuRegion: string, // Azure 1.0-preview(v1)+\r\n x_SkuServiceFamily: string, // Azure 1.0-preview(v1)+\r\n x_SkuTerm: int, // Azure 1.0-preview(v1)+\r\n x_SkuTier: string, // Azure 1.0-preview(v1)+\r\n x_SourceName: string, // Hubs v1_0+\r\n x_SourceProvider: string, // Hubs v1_0+\r\n x_SourceType: string, // Hubs v1_0+\r\n x_SourceVersion: string, // Hubs v1_0+\r\n x_SubproductName: string, // Tencent 1.0+ // cSpell:ignore Subproduct\r\n x_UsageType: string // AWS 1.0\r\n)\r\n\r\n// Costs_raw ingestion mapping\r\n.create-or-alter table Costs_raw ingestion parquet mapping \"Costs_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"AvailabilityZone\", \"Properties\": { \"Field\": \"AvailabilityZone\" } },\r\n { \"Column\": \"BilledCost\", \"Properties\": { \"Field\": \"BilledCost\" } },\r\n { \"Column\": \"BillingAccountId\", \"Properties\": { \"Field\": \"BillingAccountId\" } },\r\n { \"Column\": \"BillingAccountName\", \"Properties\": { \"Field\": \"BillingAccountName\" } },\r\n { \"Column\": \"BillingAccountType\", \"Properties\": { \"Field\": \"BillingAccountType\" } },\r\n { \"Column\": \"BillingCurrency\", \"Properties\": { \"Field\": \"BillingCurrency\" } },\r\n { \"Column\": \"BillingPeriodEnd\", \"Properties\": { \"Field\": \"BillingPeriodEnd\" } },\r\n { \"Column\": \"BillingPeriodStart\", \"Properties\": { \"Field\": \"BillingPeriodStart\" } },\r\n { \"Column\": \"CapacityReservationId\", \"Properties\": { \"Field\": \"CapacityReservationId\" } },\r\n { \"Column\": \"CapacityReservationStatus\", \"Properties\": { \"Field\": \"CapacityReservationStatus\" } },\r\n { \"Column\": \"ChargeCategory\", \"Properties\": { \"Field\": \"ChargeCategory\" } },\r\n { \"Column\": \"ChargeClass\", \"Properties\": { \"Field\": \"ChargeClass\" } },\r\n { \"Column\": \"ChargeDescription\", \"Properties\": { \"Field\": \"ChargeDescription\" } },\r\n { \"Column\": \"ChargeFrequency\", \"Properties\": { \"Field\": \"ChargeFrequency\" } },\r\n { \"Column\": \"ChargePeriodEnd\", \"Properties\": { \"Field\": \"ChargePeriodEnd\" } },\r\n { \"Column\": \"ChargePeriodStart\", \"Properties\": { \"Field\": \"ChargePeriodStart\" } },\r\n { \"Column\": \"ChargeSubcategory\", \"Properties\": { \"Field\": \"ChargeSubcategory\" } },\r\n { \"Column\": \"CommitmentDiscountCategory\", \"Properties\": { \"Field\": \"CommitmentDiscountCategory\" } },\r\n { \"Column\": \"CommitmentDiscountId\", \"Properties\": { \"Field\": \"CommitmentDiscountId\" } },\r\n { \"Column\": \"CommitmentDiscountName\", \"Properties\": { \"Field\": \"CommitmentDiscountName\" } },\r\n { \"Column\": \"CommitmentDiscountQuantity\", \"Properties\": { \"Field\": \"CommitmentDiscountQuantity\" } },\r\n { \"Column\": \"CommitmentDiscountStatus\", \"Properties\": { \"Field\": \"CommitmentDiscountStatus\" } },\r\n { \"Column\": \"CommitmentDiscountType\", \"Properties\": { \"Field\": \"CommitmentDiscountType\" } },\r\n { \"Column\": \"CommitmentDiscountUnit\", \"Properties\": { \"Field\": \"CommitmentDiscountUnit\" } },\r\n { \"Column\": \"ConsumedQuantity\", \"Properties\": { \"Field\": \"ConsumedQuantity\" } },\r\n { \"Column\": \"ConsumedUnit\", \"Properties\": { \"Field\": \"ConsumedUnit\" } },\r\n { \"Column\": \"ContractedCost\", \"Properties\": { \"Field\": \"ContractedCost\" } },\r\n { \"Column\": \"ContractedUnitPrice\", \"Properties\": { \"Field\": \"ContractedUnitPrice\" } },\r\n { \"Column\": \"EffectiveCost\", \"Properties\": { \"Field\": \"EffectiveCost\" } },\r\n { \"Column\": \"InvoiceId\", \"Properties\": { \"Field\": \"InvoiceId\" } },\r\n { \"Column\": \"InvoiceIssuerName\", \"Properties\": { \"Field\": \"InvoiceIssuerName\" } },\r\n { \"Column\": \"ListCost\", \"Properties\": { \"Field\": \"ListCost\" } },\r\n { \"Column\": \"ListUnitPrice\", \"Properties\": { \"Field\": \"ListUnitPrice\" } },\r\n { \"Column\": \"PricingCategory\", \"Properties\": { \"Field\": \"PricingCategory\" } },\r\n { \"Column\": \"PricingCurrency\", \"Properties\": { \"Field\": \"PricingCurrency\" } },\r\n { \"Column\": \"PricingQuantity\", \"Properties\": { \"Field\": \"PricingQuantity\" } },\r\n { \"Column\": \"PricingUnit\", \"Properties\": { \"Field\": \"PricingUnit\" } },\r\n { \"Column\": \"ProviderName\", \"Properties\": { \"Field\": \"ProviderName\" } },\r\n { \"Column\": \"PublisherName\", \"Properties\": { \"Field\": \"PublisherName\" } },\r\n { \"Column\": \"Region\", \"Properties\": { \"Field\": \"Region\" } },\r\n { \"Column\": \"RegionId\", \"Properties\": { \"Field\": \"RegionId\" } },\r\n { \"Column\": \"RegionName\", \"Properties\": { \"Field\": \"RegionName\" } },\r\n { \"Column\": \"ResourceId\", \"Properties\": { \"Field\": \"ResourceId\" } },\r\n { \"Column\": \"ResourceName\", \"Properties\": { \"Field\": \"ResourceName\" } },\r\n { \"Column\": \"ResourceType\", \"Properties\": { \"Field\": \"ResourceType\" } },\r\n { \"Column\": \"ServiceCategory\", \"Properties\": { \"Field\": \"ServiceCategory\" } },\r\n { \"Column\": \"ServiceName\", \"Properties\": { \"Field\": \"ServiceName\" } },\r\n { \"Column\": \"ServiceSubcategory\", \"Properties\": { \"Field\": \"ServiceSubcategory\" } },\r\n { \"Column\": \"SkuId\", \"Properties\": { \"Field\": \"SkuId\" } },\r\n { \"Column\": \"SkuMeter\", \"Properties\": { \"Field\": \"SkuMeter\" } },\r\n { \"Column\": \"SkuPriceDetails\", \"Properties\": { \"Field\": \"SkuPriceDetails\" } },\r\n { \"Column\": \"SkuPriceId\", \"Properties\": { \"Field\": \"SkuPriceId\" } },\r\n { \"Column\": \"SubAccountId\", \"Properties\": { \"Field\": \"SubAccountId\" } },\r\n { \"Column\": \"SubAccountName\", \"Properties\": { \"Field\": \"SubAccountName\" } },\r\n { \"Column\": \"SubAccountType\", \"Properties\": { \"Field\": \"SubAccountType\" } },\r\n { \"Column\": \"Tags\", \"Properties\": { \"Field\": \"Tags\" } },\r\n { \"Column\": \"UsageAmount\", \"Properties\": { \"Field\": \"UsageAmount\" } },\r\n { \"Column\": \"UsageQuantity\", \"Properties\": { \"Field\": \"UsageQuantity\" } },\r\n { \"Column\": \"UsageUnit\", \"Properties\": { \"Field\": \"UsageUnit\" } },\r\n { \"Column\": \"x_AccountId\", \"Properties\": { \"Field\": \"x_AccountId\" } },\r\n { \"Column\": \"x_AccountName\", \"Properties\": { \"Field\": \"x_AccountName\" } },\r\n { \"Column\": \"x_AccountOwnerId\", \"Properties\": { \"Field\": \"x_AccountOwnerId\" } },\r\n { \"Column\": \"x_AmortizationClass\", \"Properties\": { \"Field\": \"x_AmortizationClass\" } },\r\n { \"Column\": \"x_BilledCostInUsd\", \"Properties\": { \"Field\": \"x_BilledCostInUsd\" } },\r\n { \"Column\": \"x_BilledUnitPrice\", \"Properties\": { \"Field\": \"x_BilledUnitPrice\" } },\r\n { \"Column\": \"x_BillingAccountId\", \"Properties\": { \"Field\": \"x_BillingAccountId\" } },\r\n { \"Column\": \"x_BillingAccountName\", \"Properties\": { \"Field\": \"x_BillingAccountName\" } },\r\n { \"Column\": \"x_BillingExchangeRate\", \"Properties\": { \"Field\": \"x_BillingExchangeRate\" } },\r\n { \"Column\": \"x_BillingExchangeRateDate\", \"Properties\": { \"Field\": \"x_BillingExchangeRateDate\" } },\r\n { \"Column\": \"x_BillingItemCode\", \"Properties\": { \"Field\": \"x_BillingItemCode\" } },\r\n { \"Column\": \"x_BillingItemName\", \"Properties\": { \"Field\": \"x_BillingItemName\" } },\r\n { \"Column\": \"x_BillingProfileId\", \"Properties\": { \"Field\": \"x_BillingProfileId\" } },\r\n { \"Column\": \"x_BillingProfileName\", \"Properties\": { \"Field\": \"x_BillingProfileName\" } },\r\n { \"Column\": \"x_ChargeId\", \"Properties\": { \"Field\": \"x_ChargeId\" } },\r\n { \"Column\": \"x_ContractedCostInUsd\", \"Properties\": { \"Field\": \"x_ContractedCostInUsd\" } },\r\n { \"Column\": \"x_CommodityCode\", \"Properties\": { \"Field\": \"x_CommodityCode\" } },\r\n { \"Column\": \"x_CommodityName\", \"Properties\": { \"Field\": \"x_CommodityName\" } },\r\n { \"Column\": \"x_ComponentName\", \"Properties\": { \"Field\": \"x_ComponentName\" } },\r\n { \"Column\": \"x_ComponentType\", \"Properties\": { \"Field\": \"x_ComponentType\" } },\r\n { \"Column\": \"x_Cost\", \"Properties\": { \"Field\": \"x_Cost\" } },\r\n { \"Column\": \"x_CostAllocationRuleName\", \"Properties\": { \"Field\": \"x_CostAllocationRuleName\" } },\r\n { \"Column\": \"x_CostCategories\", \"Properties\": { \"Field\": \"x_CostCategories\" } },\r\n { \"Column\": \"x_CostCenter\", \"Properties\": { \"Field\": \"x_CostCenter\" } },\r\n { \"Column\": \"x_Credits\", \"Properties\": { \"Field\": \"x_Credits\" } },\r\n { \"Column\": \"x_CostType\", \"Properties\": { \"Field\": \"x_CostType\" } },\r\n { \"Column\": \"x_CurrencyConversionRate\", \"Properties\": { \"Field\": \"x_CurrencyConversionRate\" } },\r\n { \"Column\": \"x_CustomerId\", \"Properties\": { \"Field\": \"x_CustomerId\" } },\r\n { \"Column\": \"x_CustomerName\", \"Properties\": { \"Field\": \"x_CustomerName\" } },\r\n { \"Column\": \"x_Discount\", \"Properties\": { \"Field\": \"x_Discount\" } },\r\n { \"Column\": \"x_EffectiveCostInUsd\", \"Properties\": { \"Field\": \"x_EffectiveCostInUsd\" } },\r\n { \"Column\": \"x_EffectiveUnitPrice\", \"Properties\": { \"Field\": \"x_EffectiveUnitPrice\" } },\r\n { \"Column\": \"x_ExportTime\", \"Properties\": { \"Field\": \"x_ExportTime\" } },\r\n { \"Column\": \"x_InstanceID\", \"Properties\": { \"Field\": \"x_InstanceID\" } },\r\n { \"Column\": \"x_InvoiceId\", \"Properties\": { \"Field\": \"x_InvoiceId\" } },\r\n { \"Column\": \"x_InvoiceIssuerId\", \"Properties\": { \"Field\": \"x_InvoiceIssuerId\" } },\r\n { \"Column\": \"x_InvoiceSectionId\", \"Properties\": { \"Field\": \"x_InvoiceSectionId\" } },\r\n { \"Column\": \"x_InvoiceSectionName\", \"Properties\": { \"Field\": \"x_InvoiceSectionName\" } },\r\n { \"Column\": \"x_ListCostInUsd\", \"Properties\": { \"Field\": \"x_ListCostInUsd\" } },\r\n { \"Column\": \"x_Location\", \"Properties\": { \"Field\": \"x_Location\" } },\r\n { \"Column\": \"x_OnDemandCost\", \"Properties\": { \"Field\": \"x_OnDemandCost\" } },\r\n { \"Column\": \"x_OnDemandCostInUsd\", \"Properties\": { \"Field\": \"x_OnDemandCostInUsd\" } },\r\n { \"Column\": \"x_OnDemandUnitPrice\", \"Properties\": { \"Field\": \"x_OnDemandUnitPrice\" } },\r\n { \"Column\": \"x_Operation\", \"Properties\": { \"Field\": \"x_Operation\" } },\r\n { \"Column\": \"x_OwnerAccountID\", \"Properties\": { \"Field\": \"x_OwnerAccountID\" } },\r\n { \"Column\": \"x_PartnerCreditApplied\", \"Properties\": { \"Field\": \"x_PartnerCreditApplied\" } },\r\n { \"Column\": \"x_PartnerCreditRate\", \"Properties\": { \"Field\": \"x_PartnerCreditRate\" } },\r\n { \"Column\": \"x_PricingBlockSize\", \"Properties\": { \"Field\": \"x_PricingBlockSize\" } },\r\n { \"Column\": \"x_PricingCurrency\", \"Properties\": { \"Field\": \"x_PricingCurrency\" } },\r\n { \"Column\": \"x_PricingSubcategory\", \"Properties\": { \"Field\": \"x_PricingSubcategory\" } },\r\n { \"Column\": \"x_PricingUnitDescription\", \"Properties\": { \"Field\": \"x_PricingUnitDescription\" } },\r\n { \"Column\": \"x_Project\", \"Properties\": { \"Field\": \"x_Project\" } },\r\n { \"Column\": \"x_PublisherCategory\", \"Properties\": { \"Field\": \"x_PublisherCategory\" } },\r\n { \"Column\": \"x_PublisherId\", \"Properties\": { \"Field\": \"x_PublisherId\" } },\r\n { \"Column\": \"x_ResellerId\", \"Properties\": { \"Field\": \"x_ResellerId\" } },\r\n { \"Column\": \"x_ResellerName\", \"Properties\": { \"Field\": \"x_ResellerName\" } },\r\n { \"Column\": \"x_ResourceGroupName\", \"Properties\": { \"Field\": \"x_ResourceGroupName\" } },\r\n { \"Column\": \"x_ResourceType\", \"Properties\": { \"Field\": \"x_ResourceType\" } },\r\n { \"Column\": \"x_ServiceCode\", \"Properties\": { \"Field\": \"x_ServiceCode\" } },\r\n { \"Column\": \"x_ServiceId\", \"Properties\": { \"Field\": \"x_ServiceId\" } },\r\n { \"Column\": \"x_ServiceModel\", \"Properties\": { \"Field\": \"x_ServiceModel\" } },\r\n { \"Column\": \"x_ServicePeriodEnd\", \"Properties\": { \"Field\": \"x_ServicePeriodEnd\" } },\r\n { \"Column\": \"x_ServicePeriodStart\", \"Properties\": { \"Field\": \"x_ServicePeriodStart\" } },\r\n { \"Column\": \"x_SkuDescription\", \"Properties\": { \"Field\": \"x_SkuDescription\" } },\r\n { \"Column\": \"x_SkuDetails\", \"Properties\": { \"Field\": \"x_SkuDetails\" } },\r\n { \"Column\": \"x_SkuIsCreditEligible\", \"Properties\": { \"Field\": \"x_SkuIsCreditEligible\" } },\r\n { \"Column\": \"x_SkuMeterCategory\", \"Properties\": { \"Field\": \"x_SkuMeterCategory\" } },\r\n { \"Column\": \"x_SkuMeterId\", \"Properties\": { \"Field\": \"x_SkuMeterId\" } },\r\n { \"Column\": \"x_SkuMeterName\", \"Properties\": { \"Field\": \"x_SkuMeterName\" } },\r\n { \"Column\": \"x_SkuMeterSubcategory\", \"Properties\": { \"Field\": \"x_SkuMeterSubcategory\" } },\r\n { \"Column\": \"x_SkuOfferId\", \"Properties\": { \"Field\": \"x_SkuOfferId\" } },\r\n { \"Column\": \"x_SkuOrderId\", \"Properties\": { \"Field\": \"x_SkuOrderId\" } },\r\n { \"Column\": \"x_SkuOrderName\", \"Properties\": { \"Field\": \"x_SkuOrderName\" } },\r\n { \"Column\": \"x_SkuPartNumber\", \"Properties\": { \"Field\": \"x_SkuPartNumber\" } },\r\n { \"Column\": \"x_SkuPlanName\", \"Properties\": { \"Field\": \"x_SkuPlanName\" } },\r\n { \"Column\": \"x_SkuRegion\", \"Properties\": { \"Field\": \"x_SkuRegion\" } },\r\n { \"Column\": \"x_SkuServiceFamily\", \"Properties\": { \"Field\": \"x_SkuServiceFamily\" } },\r\n { \"Column\": \"x_SkuTerm\", \"Properties\": { \"Field\": \"x_SkuTerm\" } },\r\n { \"Column\": \"x_SkuTier\", \"Properties\": { \"Field\": \"x_SkuTier\" } },\r\n { \"Column\": \"x_SourceName\", \"Properties\": { \"Field\": \"x_SourceName\" } },\r\n { \"Column\": \"x_SourceProvider\", \"Properties\": { \"Field\": \"x_SourceProvider\" } },\r\n { \"Column\": \"x_SourceType\", \"Properties\": { \"Field\": \"x_SourceType\" } },\r\n { \"Column\": \"x_SourceVersion\", \"Properties\": { \"Field\": \"x_SourceVersion\" } },\r\n { \"Column\": \"x_SubproductName\", \"Properties\": { \"Field\": \"x_SubproductName\" } },\r\n { \"Column\": \"x_UsageType\", \"Properties\": { \"Field\": \"x_UsageType\" } }\r\n]\r\n```\r\n\r\n// Costs_raw retention policy (clear historical data)\r\n.alter-merge table Costs_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// Costs_raw retention policy (set the user-defined retention period)\r\n.alter-merge table Costs_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable Costs_raw streaming ingestion (required for Fabric)\r\n.alter table Costs_raw policy streamingingestion disable\r\n\r\n\r\n//===| Prices |=========================================================================================================\r\n// NOTE: Must be before cost details.\r\n//\r\n// Supported versions:\r\n// - MS EA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-ea\r\n// - MS MCA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-mca\r\n//======================================================================================================================\r\n\r\n// Prices_raw table -- Create the table if it doesn't exist\r\n.create-merge table Prices_raw ( ignore: string )\r\n\r\n// Prices_raw table -- Remove all columns to allow changing column types\r\n.alter table Prices_raw ( ignore: string )\r\n\r\n// Prices_raw table -- Redefine all columns\r\n.alter table Prices_raw (\r\n BasePrice: real, // Azure EA + MCA\r\n BillingAccountId: string, // Azure MCA\r\n BillingAccountName: string, // Azure MCA\r\n BillingCurrency: string, // Azure MCA\r\n BillingProfileId: string, // Azure MCA\r\n BillingProfileName: string, // Azure MCA\r\n Currency: string, // Azure MCA\r\n CurrencyCode: string, // Azure EA\r\n EffectiveEndDate: datetime, // Azure MCA\r\n EffectiveStartDate: datetime, // Azure EA + MCA\r\n EnrollmentNumber: string, // Azure EA\r\n IncludedQuantity: real, // Azure EA\r\n MarketPrice: real, // Azure EA + MCA\r\n MeterCategory: string, // Azure EA + MCA\r\n MeterId: string, // Azure MCA\r\n MeterID: string, // Azure EA\r\n MeterName: string, // Azure EA + MCA\r\n MeterRegion: string, // Azure EA + MCA\r\n MeterSubCategory: string, // Azure EA + MCA\r\n MeterType: string, // Azure EA + MCA\r\n OfferID: string, // Azure EA\r\n PartNumber: string, // Azure EA\r\n PriceType: string, // Azure EA + MCA\r\n Product: string, // Azure EA + MCA\r\n ProductId: string, // Azure MCA\r\n ProductID: string, // Azure EA\r\n ServiceFamily: string, // Azure EA + MCA\r\n SkuId: string, // Azure MCA\r\n SkuID: string, // Azure EA\r\n Term: string, // Azure EA + MCA\r\n TierMinimumUnits: real, // Azure MCA\r\n UnitOfMeasure: string, // Azure EA + MCA\r\n UnitPrice: real, // Azure EA + MCA\r\n x_SourceName: string, // Hubs v1_0+\r\n x_SourceProvider: string, // Hubs v1_0+\r\n x_SourceType: string, // Hubs v1_0+\r\n x_SourceVersion: string // Hubs v1_0+\r\n)\r\n\r\n// Prices_raw ingestion mapping\r\n.create-or-alter table Prices_raw ingestion parquet mapping \"Prices_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"BasePrice\", \"Properties\": { \"Field\": \"BasePrice\" } },\r\n { \"Column\": \"BillingAccountId\", \"Properties\": { \"Field\": \"BillingAccountId\" } },\r\n { \"Column\": \"BillingAccountName\", \"Properties\": { \"Field\": \"BillingAccountName\" } },\r\n { \"Column\": \"BillingCurrency\", \"Properties\": { \"Field\": \"BillingCurrency\" } },\r\n { \"Column\": \"BillingProfileId\", \"Properties\": { \"Field\": \"BillingProfileId\" } },\r\n { \"Column\": \"BillingProfileName\", \"Properties\": { \"Field\": \"BillingProfileName\" } },\r\n { \"Column\": \"Currency\", \"Properties\": { \"Field\": \"Currency\" } },\r\n { \"Column\": \"CurrencyCode\", \"Properties\": { \"Field\": \"CurrencyCode\" } },\r\n { \"Column\": \"EffectiveEndDate\", \"Properties\": { \"Field\": \"EffectiveEndDate\" } },\r\n { \"Column\": \"EffectiveStartDate\", \"Properties\": { \"Field\": \"EffectiveStartDate\" } },\r\n { \"Column\": \"EnrollmentNumber\", \"Properties\": { \"Field\": \"EnrollmentNumber\" } },\r\n { \"Column\": \"IncludedQuantity\", \"Properties\": { \"Field\": \"IncludedQuantity\" } },\r\n { \"Column\": \"MarketPrice\", \"Properties\": { \"Field\": \"MarketPrice\" } },\r\n { \"Column\": \"MeterCategory\", \"Properties\": { \"Field\": \"MeterCategory\" } },\r\n { \"Column\": \"MeterId\", \"Properties\": { \"Field\": \"MeterId\" } },\r\n { \"Column\": \"MeterID\", \"Properties\": { \"Field\": \"MeterID\" } },\r\n { \"Column\": \"MeterName\", \"Properties\": { \"Field\": \"MeterName\" } },\r\n { \"Column\": \"MeterRegion\", \"Properties\": { \"Field\": \"MeterRegion\" } },\r\n { \"Column\": \"MeterSubCategory\", \"Properties\": { \"Field\": \"MeterSubCategory\" } },\r\n { \"Column\": \"MeterType\", \"Properties\": { \"Field\": \"MeterType\" } },\r\n { \"Column\": \"OfferID\", \"Properties\": { \"Field\": \"OfferID\" } },\r\n { \"Column\": \"PartNumber\", \"Properties\": { \"Field\": \"PartNumber\" } },\r\n { \"Column\": \"PriceType\", \"Properties\": { \"Field\": \"PriceType\" } },\r\n { \"Column\": \"Product\", \"Properties\": { \"Field\": \"Product\" } },\r\n { \"Column\": \"ProductId\", \"Properties\": { \"Field\": \"ProductId\" } },\r\n { \"Column\": \"ProductID\", \"Properties\": { \"Field\": \"ProductID\" } },\r\n { \"Column\": \"ServiceFamily\", \"Properties\": { \"Field\": \"ServiceFamily\" } },\r\n { \"Column\": \"SkuId\", \"Properties\": { \"Field\": \"SkuId\" } },\r\n { \"Column\": \"SkuID\", \"Properties\": { \"Field\": \"SkuID\" } },\r\n { \"Column\": \"Term\", \"Properties\": { \"Field\": \"Term\" } },\r\n { \"Column\": \"TierMinimumUnits\", \"Properties\": { \"Field\": \"TierMinimumUnits\" } },\r\n { \"Column\": \"UnitOfMeasure\", \"Properties\": { \"Field\": \"UnitOfMeasure\" } },\r\n { \"Column\": \"UnitPrice\", \"Properties\": { \"Field\": \"UnitPrice\" } },\r\n { \"Column\": \"x_SourceName\", \"Properties\": { \"Field\": \"x_SourceName\" } },\r\n { \"Column\": \"x_SourceProvider\", \"Properties\": { \"Field\": \"x_SourceProvider\" } },\r\n { \"Column\": \"x_SourceType\", \"Properties\": { \"Field\": \"x_SourceType\" } },\r\n { \"Column\": \"x_SourceVersion\", \"Properties\": { \"Field\": \"x_SourceVersion\" } }\r\n]\r\n```\r\n\r\n// Prices_raw retention policy (clear historical data)\r\n.alter-merge table Prices_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// Prices_raw retention policy (set the user-defined retention period)\r\n.alter-merge table Prices_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable Prices_raw streaming ingestion (required for Fabric)\r\n.alter table Prices_raw policy streamingingestion disable\r\n\r\n\r\n//===| Recommendations |================================================================================================\r\n// Supported datasets/versions:\r\n// - MS CM EA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-ea\r\n// - MS CM MCA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-mca\r\n//======================================================================================================================\r\n\r\n// Recommendations_raw table -- Create the table if it doesn't exist\r\n.create-merge table Recommendations_raw ( ignore: string )\r\n\r\n// Recommendations_raw table -- Remove all columns to allow changing column types\r\n.alter table Recommendations_raw ( ignore: string )\r\n\r\n// Recommendations_raw table -- Redefine all columns\r\n.alter table Recommendations_raw (\r\n CostWithNoReservedInstances: real, // MS CM EA resv reco 2024-05-01\r\n CostWithNoReservedInstancesJson: string, // MS CM MCA resv reco 2024-05-01 -- Renamed to remove spaces and flag as JSON\r\n FirstUsageDate: datetime, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n InstanceFlexibilityGroup: string, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n InstanceFlexibilityRatio: real, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n Location: string, // MS CM EA+MCA resv reco 2024-05-01\r\n LookBackPeriod: string, // MS CM EA+MCA resv reco 2024-05-01\r\n MeterId: string, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n NetSavings: real, // MS CM EA resv reco 2024-05-01\r\n NetSavingsJson: string, // MS CM MCA resv reco 2024-05-01 -- Renamed to remove spaces and flag as JSON\r\n NormalizedSize: string, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n ProviderName: string, // Hubs v1_2\r\n RecommendedQuantity: real, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n RecommendedQuantityNormalized: real, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n ResourceId: string, // Hubs v1_2\r\n ResourceName: string, // Hubs v1_2\r\n ResourceType: string, // Hubs v1_2, MS CM EA+MCA resv reco 2024-05-01\r\n Scope: string, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n SKU: string, // MS CM EA resv reco 2024-05-01\r\n SkuName: string, // MS CM MCA resv reco 2024-05-01 -- Renamed to remove spaces\r\n SkuProperties: string, // MS CM EA/MCA resv reco 2024-05-01 -- Renamed for MCA\r\n SubAccountId: string, // Hubs v1_2\r\n SubAccountName: string, // Hubs v1_2\r\n SubscriptionId: string, // MS CM EA+MCA resv reco 2024-05-01\r\n Term: string, // MS CM EA+MCA resv reco 2024-05-01\r\n TotalCostWithReservedInstances: real, // MS CM EA resv reco 2024-05-01\r\n TotalCostWithReservedInstancesJson: string, // MS CM MCA resv reco 2024-05-01 -- Renamed to remove spaces and flag as JSON\r\n x_EffectiveCostAfter: real, // Hubs v1_2\r\n x_EffectiveCostBefore: real, // Hubs v1_2\r\n x_EffectiveCostSavings: real, // Hubs v1_2\r\n x_RecommendationCategory: string, // Hubs v1_2\r\n x_RecommendationDate: datetime, // Hubs v1_2\r\n x_RecommendationDescription: string, // Hubs v1_2\r\n x_RecommendationDetails: dynamic, // Hubs v1_2\r\n x_RecommendationId: string, // Hubs v1_2\r\n x_ResourceGroupName: string, // Hubs v1_2\r\n x_SourceName: string, // Hubs v1_0+\r\n x_SourceProvider: string, // Hubs v1_0+\r\n x_SourceType: string, // Hubs v1_0+\r\n x_SourceVersion: string // Hubs v1_0+\r\n)\r\n\r\n// Recommendations_raw ingestion mapping\r\n.create-or-alter table Recommendations_raw ingestion parquet mapping \"Recommendations_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"CostWithNoReservedInstances\", \"Properties\": { \"Field\": \"CostWithNoReservedInstances\" } },\r\n { \"Column\": \"CostWithNoReservedInstancesJson\", \"Properties\": { \"Field\": \"CostWithNoReservedInstancesJson\" } },\r\n { \"Column\": \"FirstUsageDate\", \"Properties\": { \"Field\": \"FirstUsageDate\" } },\r\n { \"Column\": \"InstanceFlexibilityGroup\", \"Properties\": { \"Field\": \"InstanceFlexibilityGroup\" } },\r\n { \"Column\": \"InstanceFlexibilityRatio\", \"Properties\": { \"Field\": \"InstanceFlexibilityRatio\" } },\r\n { \"Column\": \"Location\", \"Properties\": { \"Field\": \"Location\" } },\r\n { \"Column\": \"LookBackPeriod\", \"Properties\": { \"Field\": \"LookBackPeriod\" } },\r\n { \"Column\": \"MeterId\", \"Properties\": { \"Field\": \"MeterId\" } },\r\n { \"Column\": \"NetSavings\", \"Properties\": { \"Field\": \"NetSavings\" } },\r\n { \"Column\": \"NetSavingsJson\", \"Properties\": { \"Field\": \"NetSavingsJson\" } },\r\n { \"Column\": \"NormalizedSize\", \"Properties\": { \"Field\": \"NormalizedSize\" } },\r\n { \"Column\": \"ProviderName\", \"Properties\": { \"Field\": \"ProviderName\" } },\r\n { \"Column\": \"RecommendedQuantity\", \"Properties\": { \"Field\": \"RecommendedQuantity\" } },\r\n { \"Column\": \"RecommendedQuantityNormalized\", \"Properties\": { \"Field\": \"RecommendedQuantityNormalized\" } },\r\n { \"Column\": \"ResourceId\", \"Properties\": { \"Field\": \"ResourceId\" } },\r\n { \"Column\": \"ResourceName\", \"Properties\": { \"Field\": \"ResourceName\" } },\r\n { \"Column\": \"ResourceType\", \"Properties\": { \"Field\": \"ResourceType\" } },\r\n { \"Column\": \"Scope\", \"Properties\": { \"Field\": \"Scope\" } },\r\n { \"Column\": \"SKU\", \"Properties\": { \"Field\": \"SKU\" } },\r\n { \"Column\": \"SkuName\", \"Properties\": { \"Field\": \"SkuName\" } },\r\n { \"Column\": \"SkuProperties\", \"Properties\": { \"Field\": \"SkuProperties\" } },\r\n { \"Column\": \"SubAccountId\", \"Properties\": { \"Field\": \"SubAccountId\" } },\r\n { \"Column\": \"SubAccountName\", \"Properties\": { \"Field\": \"SubAccountName\" } },\r\n { \"Column\": \"SubscriptionId\", \"Properties\": { \"Field\": \"SubscriptionId\" } },\r\n { \"Column\": \"Term\", \"Properties\": { \"Field\": \"Term\" } },\r\n { \"Column\": \"TotalCostWithReservedInstances\", \"Properties\": { \"Field\": \"TotalCostWithReservedInstances\" } },\r\n { \"Column\": \"TotalCostWithReservedInstancesJson\", \"Properties\": { \"Field\": \"TotalCostWithReservedInstancesJson\" } },\r\n { \"Column\": \"x_EffectiveCostAfter\", \"Properties\": { \"Field\": \"x_EffectiveCostAfter\" } },\r\n { \"Column\": \"x_EffectiveCostBefore\", \"Properties\": { \"Field\": \"x_EffectiveCostBefore\" } },\r\n { \"Column\": \"x_EffectiveCostSavings\", \"Properties\": { \"Field\": \"x_EffectiveCostSavings\" } },\r\n { \"Column\": \"x_RecommendationCategory\", \"Properties\": { \"Field\": \"x_RecommendationCategory\" } },\r\n { \"Column\": \"x_RecommendationDate\", \"Properties\": { \"Field\": \"x_RecommendationDate\" } },\r\n { \"Column\": \"x_RecommendationDescription\", \"Properties\": { \"Field\": \"x_RecommendationDescription\" } },\r\n { \"Column\": \"x_RecommendationDetails\", \"Properties\": { \"Field\": \"x_RecommendationDetails\" } },\r\n { \"Column\": \"x_RecommendationId\", \"Properties\": { \"Field\": \"x_RecommendationId\" } },\r\n { \"Column\": \"x_ResourceGroupName\", \"Properties\": { \"Field\": \"x_ResourceGroupName\" } },\r\n { \"Column\": \"x_SourceName\", \"Properties\": { \"Field\": \"x_SourceName\" } },\r\n { \"Column\": \"x_SourceProvider\", \"Properties\": { \"Field\": \"x_SourceProvider\" } },\r\n { \"Column\": \"x_SourceType\", \"Properties\": { \"Field\": \"x_SourceType\" } },\r\n { \"Column\": \"x_SourceVersion\", \"Properties\": { \"Field\": \"x_SourceVersion\" } }\r\n]\r\n```\r\n\r\n// Recommendations_raw retention policy (clear historical data)\r\n.alter-merge table Recommendations_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// Recommendations_raw retention policy (set the user-defined retention period)\r\n.alter-merge table Recommendations_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable Recommendations_raw streaming ingestion (required for Fabric)\r\n.alter table Recommendations_raw policy streamingingestion disable\r\n\r\n\r\n//===| Transactions |===================================================================================================\r\n// Supported versions:\r\n// - MS CM EA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-ea\r\n// - MS CM MCA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-mca\r\n//======================================================================================================================\r\n\r\n// Transactions_raw table -- Create the table if it doesn't exist\r\n.create-merge table Transactions_raw ( ignore: string )\r\n\r\n// Transactions_raw table -- Remove all columns to allow changing column types\r\n.alter table Transactions_raw ( ignore: string )\r\n\r\n// Transactions_raw table -- Redefine all columns\r\n.alter table Transactions_raw (\r\n AccountName: string, // MS CM EA resv trans 2023-05-01\r\n AccountOwnerEmail: string, // MS CM EA resv trans 2023-05-01\r\n Amount: real, // MS CM EA+MCA resv trans 2023-05-01\r\n ArmSkuName: string, // MS CM EA+MCA resv trans 2023-05-01\r\n BillingFrequency: string, // MS CM EA+MCA resv trans 2023-05-01\r\n BillingMonth: string, // MS CM EA resv trans 2023-05-01\r\n BillingProfileId: string, // MS CM MCA resv trans 2023-05-01\r\n BillingProfileName: string, // MS CM MCA resv trans 2023-05-01\r\n CostCenter: string, // MS CM EA resv trans 2023-05-01\r\n Currency: string, // MS CM EA+MCA resv trans 2023-05-01\r\n CurrentEnrollmentId: string, // MS CM EA resv trans 2023-05-01\r\n DepartmentName: string, // MS CM EA resv trans 2023-05-01\r\n Description: string, // MS CM EA+MCA resv trans 2023-05-01\r\n EventDate: datetime, // MS CM EA+MCA resv trans 2023-05-01\r\n EventType: string, // MS CM EA+MCA resv trans 2023-05-01\r\n Invoice: string, // MS CM EA+MCA resv trans 2023-05-01\r\n InvoiceId: string, // MS CM EA+MCA resv trans 2023-05-01\r\n InvoiceSectionId: string, // MS CM MCA resv trans 2023-05-01\r\n InvoiceSectionName: string, // MS CM MCA resv trans 2023-05-01\r\n MonetaryCommitment: real, // MS CM EA resv trans 2023-05-01\r\n Overage: real, // MS CM EA resv trans 2023-05-01\r\n PurchasingEnrollment: string, // MS CM EA resv trans 2023-05-01\r\n PurchasingSubscriptionGuid: string, // MS CM EA+MCA resv trans 2023-05-01\r\n PurchasingSubscriptionName: string, // MS CM EA+MCA resv trans 2023-05-01\r\n Quantity: real, // MS CM EA+MCA resv trans 2023-05-01\r\n Region: string, // MS CM EA+MCA resv trans 2023-05-01\r\n ReservationOrderId: string, // MS CM EA+MCA resv trans 2023-05-01\r\n ReservationOrderName: string, // MS CM EA+MCA resv trans 2023-05-01\r\n Term: string, // MS CM EA+MCA resv trans 2023-05-01\r\n x_SourceName: string, // Hubs v1_0+\r\n x_SourceProvider: string, // Hubs v1_0+\r\n x_SourceType: string, // Hubs v1_0+\r\n x_SourceVersion: string // Hubs v1_0+\r\n)\r\n\r\n// Transactions_raw ingestion mapping\r\n.create-or-alter table Transactions_raw ingestion parquet mapping \"Transactions_raw_mapping\"\r\n```\r\n[\r\n { \"Column\": \"AccountName\", \"Properties\": { \"Field\": \"AccountName\" } },\r\n { \"Column\": \"AccountOwnerEmail\", \"Properties\": { \"Field\": \"AccountOwnerEmail\" } },\r\n { \"Column\": \"Amount\", \"Properties\": { \"Field\": \"Amount\" } },\r\n { \"Column\": \"ArmSkuName\", \"Properties\": { \"Field\": \"ArmSkuName\" } },\r\n { \"Column\": \"BillingFrequency\", \"Properties\": { \"Field\": \"BillingFrequency\" } },\r\n { \"Column\": \"BillingMonth\", \"Properties\": { \"Field\": \"BillingMonth\" } },\r\n { \"Column\": \"BillingProfileId\", \"Properties\": { \"Field\": \"BillingProfileId\" } },\r\n { \"Column\": \"BillingProfileName\", \"Properties\": { \"Field\": \"BillingProfileName\" } },\r\n { \"Column\": \"CostCenter\", \"Properties\": { \"Field\": \"CostCenter\" } },\r\n { \"Column\": \"Currency\", \"Properties\": { \"Field\": \"Currency\" } },\r\n { \"Column\": \"CurrentEnrollmentId\", \"Properties\": { \"Field\": \"CurrentEnrollmentId\" } },\r\n { \"Column\": \"DepartmentName\", \"Properties\": { \"Field\": \"DepartmentName\" } },\r\n { \"Column\": \"Description\", \"Properties\": { \"Field\": \"Description\" } },\r\n { \"Column\": \"EventDate\", \"Properties\": { \"Field\": \"EventDate\" } },\r\n { \"Column\": \"EventType\", \"Properties\": { \"Field\": \"EventType\" } },\r\n { \"Column\": \"Invoice\", \"Properties\": { \"Field\": \"Invoice\" } },\r\n { \"Column\": \"InvoiceId\", \"Properties\": { \"Field\": \"InvoiceId\" } },\r\n { \"Column\": \"InvoiceSectionId\", \"Properties\": { \"Field\": \"InvoiceSectionId\" } },\r\n { \"Column\": \"InvoiceSectionName\", \"Properties\": { \"Field\": \"InvoiceSectionName\" } },\r\n { \"Column\": \"MonetaryCommitment\", \"Properties\": { \"Field\": \"MonetaryCommitment\" } },\r\n { \"Column\": \"Overage\", \"Properties\": { \"Field\": \"Overage\" } },\r\n { \"Column\": \"PurchasingEnrollment\", \"Properties\": { \"Field\": \"PurchasingEnrollment\" } },\r\n { \"Column\": \"PurchasingSubscriptionGuid\", \"Properties\": { \"Field\": \"PurchasingSubscriptionGuid\" } },\r\n { \"Column\": \"PurchasingSubscriptionName\", \"Properties\": { \"Field\": \"PurchasingSubscriptionName\" } },\r\n { \"Column\": \"Quantity\", \"Properties\": { \"Field\": \"Quantity\" } },\r\n { \"Column\": \"Region\", \"Properties\": { \"Field\": \"Region\" } },\r\n { \"Column\": \"ReservationOrderId\", \"Properties\": { \"Field\": \"ReservationOrderId\" } },\r\n { \"Column\": \"ReservationOrderName\", \"Properties\": { \"Field\": \"ReservationOrderName\" } },\r\n { \"Column\": \"Term\", \"Properties\": { \"Field\": \"Term\" } },\r\n { \"Column\": \"x_SourceName\", \"Properties\": { \"Field\": \"x_SourceName\" } },\r\n { \"Column\": \"x_SourceProvider\", \"Properties\": { \"Field\": \"x_SourceProvider\" } },\r\n { \"Column\": \"x_SourceType\", \"Properties\": { \"Field\": \"x_SourceType\" } },\r\n { \"Column\": \"x_SourceVersion\", \"Properties\": { \"Field\": \"x_SourceVersion\" } }\r\n]\r\n```\r\n\r\n// Transactions_raw retention policy (clear historical data)\r\n.alter-merge table Transactions_raw policy retention softdelete = 0d recoverability = disabled\r\n\r\n// Transactions_raw retention policy (set the user-defined retention period)\r\n.alter-merge table Transactions_raw policy retention softdelete = $$rawRetentionInDays$$d recoverability = disabled\r\n\r\n// Disable Transactions_raw streaming ingestion (required for Fabric)\r\n.alter table Transactions_raw policy streamingingestion disable\r\n\r\n", + "$fxv#9": "// Copyright (c) Microsoft Corporation.\r\n// Licensed under the MIT License.\r\n\r\n//======================================================================================================================\r\n// Ingestion database\r\n// Used for data ingestion, normalization, and cleansing.\r\n// See known issues @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n//======================================================================================================================\r\n\r\n// For allowed commands, see https://learn.microsoft.com/azure/data-explorer/database-script\r\n\r\n//===| Prices |=========================================================================================================\r\n// Supported versions:\r\n// - MS EA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-ea\r\n// - MS MCA 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/price-sheet-mca\r\n//======================================================================================================================\r\n\r\n// Prices_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: All prices transformed to FOCUS 1.0. Use Prices_transform_v1_2() instead.', folder='Prices')\r\nPrices_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', toint(''),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n let prices = materialize(\r\n Prices_raw\r\n //\r\n // Change real to decimal\r\n | extend\r\n BasePrice = todecimal(BasePrice),\r\n IncludedQuantity = todecimal(IncludedQuantity),\r\n MarketPrice = todecimal(MarketPrice),\r\n TierMinimumUnits = todecimal(TierMinimumUnits),\r\n UnitPrice = todecimal(UnitPrice)\r\n //\r\n | extend x_SkuId = coalesce(SkuId, SkuID)\r\n | extend x_SkuMeterId = coalesce(MeterId, MeterID)\r\n | extend x_SkuProductId = coalesce(ProductId, ProductID)\r\n | extend x_SkuTerm = isoMonths(Term)\r\n | project-rename\r\n x_BaseUnitPrice = BasePrice,\r\n x_EffectivePeriodEnd = EffectiveEndDate,\r\n x_EffectivePeriodStart = EffectiveStartDate,\r\n x_PricingUnitDescription = UnitOfMeasure,\r\n x_SkuIncludedQuantity = IncludedQuantity,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterName = MeterName,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuMeterType = MeterType,\r\n x_SkuOfferId = OfferID,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPriceType = PriceType,\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTier = TierMinimumUnits\r\n | extend ContractedUnitPrice = iff(x_SkuPriceType != 'SavingsPlan', UnitPrice, todecimal('')) // UnitPrice for savings plan is not the on-demand unit price\r\n | extend ListUnitPrice = iff(x_SkuPriceType != 'SavingsPlan', MarketPrice, todecimal('')) // MarketPrice for savings plan is not the list price\r\n | extend ChargeCategory = case(\r\n x_SkuPriceType == 'Consumption', 'Usage',\r\n x_SkuPriceType == 'ReservedInstance', 'Purchase',\r\n x_SkuPriceType == 'SavingsPlan', 'Usage', // Savings plan prices are for committed usage, not the purchase\r\n ''\r\n )\r\n | extend SkuPriceIdv2 = strcat(case(x_SkuPriceType == 'Consumption', 'OD', x_SkuPriceType == 'ReservedInstance', 'RI', x_SkuPriceType == 'SavingsPlan', 'SP', 'XX'), substring(ChargeCategory, 0, 1), x_SkuTerm, '_', x_SkuProductId, '_', x_SkuId, '_', x_SkuMeterType, '_', x_SkuTier, x_SkuOfferId)\r\n | extend x_BillingAccountId = iff(BillingAccountId startswith '/', split(BillingAccountId, '/')[-1], coalesce(BillingAccountId, EnrollmentNumber))\r\n | extend x_BillingProfileId = iff(BillingProfileId startswith '/', split(BillingProfileId, '/')[-1], coalesce(BillingProfileId, EnrollmentNumber))\r\n | extend tmp_SavingsPlanKey = strcat(x_SkuMeterId, x_SkuProductId, x_SkuId, x_SkuTier, x_SkuOfferId)\r\n //\r\n // Get latest ingested row based on the unique ID\r\n | extend x_IngestionTime = ingestion_time()\r\n );\r\n //\r\n // Meters for reservations and savings plans to identify commitment eligibility\r\n let riMeters = prices | where x_SkuPriceType == 'ReservedInstance' | distinct x_SkuMeterId;\r\n let spMeters = prices | where x_SkuPriceType == 'SavingsPlan' | distinct x_SkuMeterId;\r\n //\r\n // Copy list/base/contracted prices from on-demand SKUs\r\n prices\r\n | where x_SkuPriceType == 'SavingsPlan'\r\n // If we use join, specify the shuffle key\r\n // TODO: Compare join vs. lookup perf -- | join kind=leftouter hint.strategy=shuffle (prices | where x_SkuPriceType == 'Consumption' | where x_SkuMeterId in (spMeters) | distinct tmp_SavingsPlanKey, ListUnitPrice, ContractedUnitPrice, x_BaseUnitPrice) on tmp_SavingsPlanKey\r\n | lookup kind=leftouter (prices | where x_SkuPriceType == 'Consumption' | where x_SkuMeterId in (spMeters) | distinct tmp_SavingsPlanKey, ListUnitPrice, ContractedUnitPrice, x_BaseUnitPrice) on tmp_SavingsPlanKey\r\n | extend ListUnitPrice = coalesce(ListUnitPrice, ListUnitPrice1)\r\n | extend ContractedUnitPrice = coalesce(ContractedUnitPrice, ContractedUnitPrice1)\r\n | extend x_BaseUnitPrice = coalesce(x_BaseUnitPrice, x_BaseUnitPrice1)\r\n | project-away ListUnitPrice1, ContractedUnitPrice1, x_BaseUnitPrice1, tmp_SavingsPlanKey\r\n | union ((prices | where x_SkuPriceType != 'SavingsPlan'))\r\n //\r\n // Calculate commitment discount elgibility\r\n // TODO: Would a join be faster?\r\n | extend x_CommitmentDiscountSpendEligibility = iff(x_SkuMeterId in (riMeters) and x_SkuPriceType != 'ReservedInstance', 'Eligible', 'Not Eligible')\r\n | extend x_CommitmentDiscountUsageEligibility = iff(x_SkuMeterId in (spMeters), 'Eligible', 'Not Eligible')\r\n //\r\n // Add PricingUnit and x_PricingBlockSize\r\n // TODO: Compare join vs. lookup perf -- | join kind=leftouter (PricingUnits) on x_PricingUnitDescription | project-away x_PricingUnitDescription1\r\n | lookup kind=leftouter (PricingUnits | extend x_PricingBlockSize = todecimal(x_PricingBlockSize)) on x_PricingUnitDescription\r\n //\r\n | extend x_EffectiveUnitPrice = iff(x_SkuPriceType == 'SavingsPlan', UnitPrice, todecimal('')) // Savings plan prices are for the effective price, not the contracted price\r\n | extend x_EffectiveUnitPriceDiscount = ContractedUnitPrice - x_EffectiveUnitPrice\r\n | extend x_ContractedUnitPriceDiscount = ListUnitPrice - ContractedUnitPrice\r\n | extend x_TotalUnitPriceDiscount = ListUnitPrice - x_EffectiveUnitPrice\r\n | project\r\n BillingAccountId = tolower(case(\r\n BillingProfileId startswith '/', BillingProfileId,\r\n BillingAccountId startswith '/', BillingAccountId,\r\n strcat('/providers/microsoft.billing/billingaccounts/', x_BillingAccountId, iff(x_BillingProfileId == x_BillingAccountId, '', strcat('/billingprofiles/', x_BillingProfileId)))\r\n )),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName, x_BillingProfileId),\r\n BillingCurrency = coalesce(BillingCurrency, CurrencyCode, Currency), // Currency last as a fallback only\r\n ChargeCategory,\r\n CommitmentDiscountCategory = case(\r\n x_SkuPriceType == 'ReservedInstance', 'Usage',\r\n x_SkuPriceType == 'SavingsPlan', 'Spend',\r\n ''\r\n ),\r\n CommitmentDiscountType = case(\r\n x_SkuPriceType == 'ReservedInstance', 'Reservation',\r\n x_SkuPriceType == 'SavingsPlan', 'Savings plan',\r\n ''\r\n ),\r\n ContractedUnitPrice,\r\n ListUnitPrice,\r\n PricingCategory = case(\r\n x_SkuPriceType == 'Consumption', 'Standard',\r\n x_SkuPriceType == 'ReservedInstance', 'Standard', // Reservation purchases are tracked as \"Standard\"\r\n x_SkuPriceType == 'SavingsPlan', 'Committed',\r\n ''\r\n ),\r\n PricingUnit,\r\n SkuId = coalesce(ProductId, ProductID),\r\n SkuPriceId = strcat(x_SkuProductId, '_', x_SkuId, '_', x_SkuMeterType),\r\n SkuPriceIdv2,\r\n x_BaseUnitPrice,\r\n x_BillingAccountAgreement = case(\r\n strlen(x_BillingAccountId) > 32, 'MCA',\r\n strlen(x_BillingAccountId) < 32, 'EA',\r\n 'Unknown'\r\n ),\r\n x_BillingAccountId,\r\n x_BillingProfileId,\r\n x_CommitmentDiscountSpendEligibility,\r\n x_CommitmentDiscountUsageEligibility,\r\n x_ContractedUnitPriceDiscount,\r\n x_ContractedUnitPriceDiscountPercent = 1.0 * x_ContractedUnitPriceDiscount / ListUnitPrice * 100,\r\n x_EffectivePeriodEnd = startofmonth(x_EffectivePeriodEnd + 1h),\r\n x_EffectivePeriodStart,\r\n x_EffectiveUnitPrice,\r\n x_EffectiveUnitPriceDiscount,\r\n x_EffectiveUnitPriceDiscountPercent = 1.0 * x_EffectiveUnitPriceDiscount / ContractedUnitPrice * 100,\r\n x_IngestionTime,\r\n x_PricingBlockSize,\r\n x_PricingCurrency = coalesce(Currency, CurrencyCode), // CurrencyCode last as a fallback only\r\n x_PricingSubcategory = case(\r\n x_SkuPriceType == 'Consumption' and (x_SkuIncludedQuantity > 0 or x_SkuTier > 0), 'Tiered',\r\n x_SkuPriceType == 'Consumption', 'Standard',\r\n x_SkuPriceType == 'ReservedInstance', 'Standard', // Reservation purchases are tracked as \"Standard\"\r\n x_SkuPriceType == 'SavingsPlan', 'Committed Spend',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_SkuDescription = Product,\r\n x_SkuId,\r\n x_SkuIncludedQuantity,\r\n x_SkuMeterCategory,\r\n x_SkuMeterId,\r\n x_SkuMeterName,\r\n x_SkuMeterSubcategory,\r\n x_SkuMeterType,\r\n x_SkuPriceType,\r\n x_SkuProductId,\r\n x_SkuRegion,\r\n x_SkuServiceFamily,\r\n x_SkuOfferId,\r\n x_SkuPartNumber,\r\n x_SkuTerm,\r\n x_SkuTier,\r\n x_SourceName = coalesce(x_SourceName, 'Cost Management'),\r\n x_SourceProvider = coalesce(x_SourceProvider, 'Microsoft'),\r\n x_SourceType = coalesce(x_SourceType, 'PriceSheet'),\r\n x_SourceVersion = coalesce(x_SourceVersion, '2023-05-01'),\r\n x_TotalUnitPriceDiscount,\r\n x_TotalUnitPriceDiscountPercent = 1.0 * x_TotalUnitPriceDiscount / ListUnitPrice * 100\r\n}\r\n\r\n// Prices_final_v1_0 table\r\n.create-merge table Prices_final_v1_0 (\r\n BillingAccountId: string,\r\n BillingAccountName: string,\r\n BillingCurrency: string,\r\n ChargeCategory: string,\r\n CommitmentDiscountCategory: string,\r\n CommitmentDiscountType: string,\r\n ContractedUnitPrice: decimal,\r\n ListUnitPrice: decimal,\r\n PricingCategory: string,\r\n PricingUnit: string,\r\n SkuId: string,\r\n SkuPriceId: string,\r\n SkuPriceIdv2: string, // Hubs add-on\r\n x_BaseUnitPrice: decimal, // Azure\r\n x_BillingAccountAgreement: string, // Hubs add-on\r\n x_BillingAccountId: string, // Azure MCA\r\n x_BillingProfileId: string, // Azure MCA\r\n x_CommitmentDiscountSpendEligibility: string, // Hubs add-on\r\n x_CommitmentDiscountUsageEligibility: string, // Hubs add-on\r\n x_ContractedUnitPriceDiscount: decimal, // Hubs add-on\r\n x_ContractedUnitPriceDiscountPercent: decimal, // Hubs add-on\r\n x_EffectivePeriodEnd: datetime, // Azure\r\n x_EffectivePeriodStart: datetime, // Azure\r\n x_EffectiveUnitPrice: decimal, // Azure\r\n x_EffectiveUnitPriceDiscount: decimal, // Hubs add-on\r\n x_EffectiveUnitPriceDiscountPercent: decimal, // Hubs add-on\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_PricingBlockSize: decimal, // Hubs add-on\r\n x_PricingCurrency: string, // Azure\r\n x_PricingSubcategory: string, // Hubs add-on\r\n x_PricingUnitDescription: string, // Azure\r\n x_SkuDescription: string, // Azure\r\n x_SkuId: string, // Azure\r\n x_SkuIncludedQuantity: decimal, // Azure EA\r\n x_SkuMeterCategory: string, // Azure\r\n x_SkuMeterId: string, // Azure\r\n x_SkuMeterName: string, // Azure\r\n x_SkuMeterSubcategory: string, // Azure\r\n x_SkuMeterType: string, // Azure\r\n x_SkuPriceType: string, // Azure\r\n x_SkuProductId: string, // Azure\r\n x_SkuRegion: string, // Azure\r\n x_SkuServiceFamily: string, // Azure\r\n x_SkuOfferId: string, // Azure EA\r\n x_SkuPartNumber: string, // Azure EA\r\n x_SkuTerm: int, // Azure\r\n x_SkuTier: decimal, // Azure MCA\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_TotalUnitPriceDiscount: decimal, // Hubs add-on\r\n x_TotalUnitPriceDiscountPercent: decimal // Hubs add-on\r\n)\r\n\r\n// Update policy for Prices_raw -> Prices_final_v1_0\r\n.alter table Prices_final_v1_0 policy update\r\n```\r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"Prices_raw\",\r\n \"Query\": \"Prices_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Cost and usage |=================================================================================================\r\n// Supported versions:\r\n// - MS: 1.0, 1.0-preview(v1) -- See https://aka.ms/costmgmt/exports/focus\r\n// - AWS: 1.0 -- See https://docs.aws.amazon.com/cur/latest/userguide/table-dictionary-focus-1-0-aws-columns.html\r\n// - GCP: Jan-Jun 2024 -- See https://cloud.google.com/resources/google-cloud-focus?e=48754805&hl=en\r\n// Links to (Aug 2024): https://services.google.com/fh/files/misc/focus_guide_v1.pdf\r\n// See also:\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/standard-usage\r\n// - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/detailed-usage\r\n// - OCI: 1.0 -- See https://docs.oracle.com/iaas/Content/Billing/Concepts/costusagereportsoverview.htm#costreports__focus-cost-report-schema\r\n//\r\n// Support for non-Azure data is limited to ingestion only. Data is not transformed across versions.\r\n//======================================================================================================================\r\n\r\n// Costs_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: All costs transformed to FOCUS 1.0. Use Costs_transform_v1_2() instead.', folder='Costs')\r\nCosts_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n Costs_raw\r\n //\r\n // Change real to decimal\r\n | extend\r\n BilledCost = todecimal(BilledCost),\r\n CommitmentDiscountQuantity = todecimal(CommitmentDiscountQuantity),\r\n ConsumedQuantity = todecimal(ConsumedQuantity),\r\n ContractedCost = todecimal(ContractedCost),\r\n ContractedUnitPrice = todecimal(ContractedUnitPrice),\r\n EffectiveCost = todecimal(EffectiveCost),\r\n ListCost = todecimal(ListCost),\r\n ListUnitPrice = todecimal(ListUnitPrice),\r\n PricingQuantity = todecimal(PricingQuantity),\r\n UsageAmount = todecimal(UsageAmount),\r\n UsageQuantity = todecimal(UsageQuantity),\r\n x_BilledCostInUsd = todecimal(x_BilledCostInUsd),\r\n x_BilledUnitPrice = todecimal(x_BilledUnitPrice),\r\n x_BillingExchangeRate = todecimal(x_BillingExchangeRate),\r\n x_ContractedCostInUsd = todecimal(x_ContractedCostInUsd),\r\n x_Cost = todecimal(x_Cost),\r\n x_CurrencyConversionRate = todecimal(x_CurrencyConversionRate),\r\n x_EffectiveCostInUsd = todecimal(x_EffectiveCostInUsd),\r\n x_EffectiveUnitPrice = todecimal(x_EffectiveUnitPrice),\r\n x_ListCostInUsd = todecimal(x_ListCostInUsd),\r\n x_OnDemandCost = todecimal(x_OnDemandCost),\r\n x_OnDemandCostInUsd = todecimal(x_OnDemandCostInUsd),\r\n x_OnDemandUnitPrice = todecimal(x_OnDemandUnitPrice),\r\n x_PricingBlockSize = todecimal(x_PricingBlockSize)\r\n //\r\n // Dedupe rows\r\n | extend x_IngestionTime = ingestion_time()\r\n | extend x_ChargeId = ''\r\n // TODO: Consider adding a unique charge ID per row\r\n // hash_sha256(strcat(\r\n // // DO NOT CHANGE COLUMNS OR COLUMN ORDER\r\n // // 1. Resource hierarchy (including resource name), highest to lowest\r\n // BillingAccountId,\r\n // x_InvoiceSectionId,\r\n // x_AccountOwnerId,\r\n // SubAccountId,\r\n // x_ResourceGroupName,\r\n // ResourceName,\r\n // // 2. Resource details\r\n // ResourceId,\r\n // RegionId,\r\n // Tags,\r\n // CommitmentDiscountId,\r\n // x_CostCenter,\r\n // // 4. Meter details\r\n // SkuPriceId,\r\n // x_SkuMeterId,\r\n // x_SkuPartNumber,\r\n // x_SkuOfferId,\r\n // x_SkuDetails,\r\n // // 5. Date\r\n // ChargePeriodStart\r\n // ))\r\n //\r\n // Identify data quality issues\r\n | extend x_SourceChanges = trim_end(',', strcat(\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and ChargeFrequency == 'Usage-Based', 'InvalidChargeFrequency,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and EffectiveCost > 0, 'InvalidEffectiveCost,', ''),\r\n iff((isempty(ContractedCost) or ContractedCost == 0) and EffectiveCost != 0, 'MissingContractedCost,', ''),\r\n iff((isempty(ContractedUnitPrice) or ContractedUnitPrice == 0) and x_EffectiveUnitPrice != 0, 'MissingContractedUnitPrice,', ''),\r\n iff(ListCost < ContractedCost, 'ListCostLessThanContractedCost,', ''),\r\n iff(ContractedCost < EffectiveCost, 'ContractedCostLessThanEffectiveCost,', ''),\r\n iff((isempty(ListCost) or ListCost == 0) and (ContractedCost != 0 or EffectiveCost != 0), 'MissingListCost,', ''),\r\n iff((isempty(ListUnitPrice) or ListUnitPrice == 0) and (ContractedUnitPrice != 0 or x_EffectiveUnitPrice != 0), 'MissingListUnitPrice,', ''),\r\n iff((isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(x_BilledUnitPrice - x_EffectiveUnitPrice) < 0.0001)\r\n or (isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(ContractedUnitPrice - x_EffectiveUnitPrice) < 0.0001),\r\n 'XEffectiveUnitPriceRoundingError,', ''),\r\n iff(ConsumedQuantity == 0 and (EffectiveCost != 0 or BilledCost != 0), 'MissingConsumedQuantity,', ''),\r\n iff(PricingQuantity == 0 and (EffectiveCost != 0 or BilledCost != 0), 'MissingPricingQuantity,', ''),\r\n iff(isempty(ProviderName), 'MissingProviderName,', ''),\r\n iff(isempty(PublisherName), 'MissingPublisherName,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceId), 'MissingResourceId,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceName), 'MissingResourceName,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(ResourceType), 'MissingResourceType,', ''),\r\n iff(BilledCost > 0 and x_BilledUnitPrice == 0, 'MissingXBilledUnitPrice,', ''),\r\n iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and isempty(x_ResourceType), 'MissingXResourceType,', ''),\r\n iff(PricingCategory == 'Standard' and isnotempty(CommitmentDiscountId) and ChargeCategory == 'Usage', 'PricingCategoryShouldBeCommitted,', ''),\r\n iff(x_SkuTerm == '1Year' or x_SkuTerm == '3Years' or x_SkuTerm == '5Years', 'SkuTermShouldBeAnInteger,', '')\r\n ))\r\n //\r\n // Fix columns needed in other changes\r\n | extend ProviderName = case(\r\n isnotempty(ProviderName), ProviderName,\r\n isnotempty(coalesce(x_CostCategories, x_Discount, x_Operation, x_ServiceCode, x_UsageType)), 'AWS',\r\n isnotempty(coalesce(tostring(UsageAmount), tostring(x_Cost), x_Credits, x_CostType, tostring(x_CurrencyConversionRate), tostring(x_ExportTime), x_Project, x_ServiceId)), 'GCP',\r\n isnotempty(coalesce(x_BillingProfileId, x_InvoiceSectionId)), 'Microsoft',\r\n ''\r\n )\r\n //\r\n // Identify source\r\n | extend x_SourceName = coalesce(x_SourceName, iff(isnotempty(x_BillingProfileId), 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(isnotempty(x_BillingProfileId), 'FocusCost', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, case(\r\n isnotempty(coalesce(InvoiceId, SkuMeter, PricingCurrency)) and isempty(SkuPriceDetails) and isnotempty(x_SkuDetails), '1.2-preview',\r\n isnotempty(coalesce(InvoiceId, SkuMeter, PricingCurrency)), '1.2',\r\n isnotempty(coalesce(ChargeClass, CommitmentDiscountStatus, tostring(ConsumedQuantity), ConsumedUnit, tostring(ContractedCost), tostring(ContractedUnitPrice), RegionId, RegionName)), '1.0',\r\n isnotempty(coalesce(ChargeSubcategory, Region, tostring(UsageQuantity), UsageUnit)), iff(ProviderName == 'Microsoft', '1.0-preview(v1)', '1.0-preview'),\r\n ''\r\n ))\r\n // Append version check error code\r\n | extend x_SourceChanges = iff(x_SourceVersion == '1.0', x_SourceChanges,\r\n strcat(x_SourceChanges, iff(isempty(x_SourceChanges), '', ','), iff(x_SourceVersion == '', 'UnknownFocusVersion', 'LegacyFocusVersion'))\r\n )\r\n //\r\n // Fix quantities\r\n | extend PricingQuantity = case(\r\n PricingQuantity != 0 or (EffectiveCost == 0 and BilledCost == 0), PricingQuantity,\r\n PricingQuantity == 0 and isnotempty(BilledCost) and BilledCost != 0 and isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, BilledCost / x_BilledUnitPrice,\r\n PricingQuantity == 0 and isnotempty(BilledCost) and BilledCost != 0 and isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, BilledCost / x_BilledUnitPrice,\r\n PricingQuantity == 0 and isnotempty(EffectiveCost) and EffectiveCost != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0, EffectiveCost / x_EffectiveUnitPrice,\r\n PricingQuantity\r\n )\r\n | extend ConsumedQuantity = case(\r\n isnotempty(ConsumedQuantity) and ConsumedQuantity != 0, ConsumedQuantity,\r\n ChargeCategory == 'Usage', PricingQuantity / coalesce(x_PricingBlockSize, decimal(1)),\r\n ConsumedQuantity\r\n )\r\n //\r\n // Populate missing prices -- mapping to on-demand prices requires meter ID and offer ID\r\n | extend tmp_MissingPrices = ProviderName == 'Microsoft'\r\n and (ListUnitPrice == 0 or ContractedUnitPrice == 0)\r\n and x_EffectiveUnitPrice != 0\r\n and not(CommitmentDiscountCategory == 'Spend' and CommitmentDiscountStatus == 'Unused')\r\n and isnotempty(strcat(x_SkuMeterId, x_SkuOfferId))\r\n | as allCosts\r\n | where tmp_MissingPrices\r\n | extend tmp_ReservationPriceLookupKey = tolower(strcat(x_BillingProfileId, substring(ChargePeriodStart, 0, 7), x_SkuMeterId, x_SkuOfferId))\r\n | as costsWithMissingPrices\r\n | join kind=leftouter (\r\n Prices_final_v1_0\r\n | extend tmp_ReservationPriceLookupKey = tolower(strcat(x_BillingProfileId, substring(x_EffectivePeriodStart, 0, 7), x_SkuMeterId, x_SkuOfferId))\r\n | where x_SkuPriceType == 'Consumption' and tmp_ReservationPriceLookupKey in ((costsWithMissingPrices | summarize by tmp_ReservationPriceLookupKey))\r\n | summarize ListUnitPrice = min(ListUnitPrice), ContractedUnitPrice = min(ContractedUnitPrice) by tmp_ReservationPriceLookupKey, x_PricingBlockSize, PricingUnit\r\n ) on tmp_ReservationPriceLookupKey\r\n //\r\n // Select the best price to use for each row\r\n // TODO: Save values before changing -- | extend x_old_ContractedUnitPrice = ContractedUnitPrice, x_old_EffectiveUnitPrice = x_EffectiveUnitPrice, x_old_ListUnitPrice = ListUnitPrice, x_old_ListCost = ListCost, x_old_ContractedCost = ContractedCost\r\n | extend x_EffectiveUnitPrice = case(\r\n // If price is a rounding error away from the billed price, use the billed price\r\n isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(x_BilledUnitPrice - x_EffectiveUnitPrice) < 0.0001, x_BilledUnitPrice,\r\n // If price is a rounding error away from the contracted price, use the contracted price\r\n isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0 and isnotempty(x_EffectiveUnitPrice) and x_EffectiveUnitPrice != 0 and abs(ContractedUnitPrice - x_EffectiveUnitPrice) < 0.0001, ContractedUnitPrice,\r\n x_EffectiveUnitPrice\r\n )\r\n | extend ContractedUnitPrice = case(\r\n // If price is already correct, keep that\r\n (isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0) or (EffectiveCost == 0 and BilledCost == 0), ContractedUnitPrice,\r\n // If both prices use the same scale, use the new one\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize == x_PricingBlockSize1, ContractedUnitPrice1 * x_BillingExchangeRate,\r\n // If prices are the same unit but not the same scale, use the new one but correct the scale\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize != x_PricingBlockSize1 and isnotempty(x_PricingBlockSize) and isnotempty(x_PricingBlockSize1), ContractedUnitPrice1 * x_BillingExchangeRate / x_PricingBlockSize1 * x_PricingBlockSize,\r\n // If billed price is available, assume the billed price is the same as contracted price to support aggregations\r\n isnotempty(x_BilledUnitPrice) and x_BilledUnitPrice != 0, x_EffectiveUnitPrice,\r\n // Otherwise, assume the effective price is the same as contracted price to support aggregations\r\n x_EffectiveUnitPrice\r\n )\r\n | extend ListUnitPrice = case(\r\n // If price is already correct, keep that\r\n (isnotempty(ListUnitPrice) and ListUnitPrice != 0) or (EffectiveCost == 0 and BilledCost == 0), ListUnitPrice,\r\n // If both prices use the same scale, use the new one\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize == x_PricingBlockSize1, ListUnitPrice1 * x_BillingExchangeRate,\r\n // If prices are the same unit but not the same scale, use the new one but correct the scale\r\n PricingUnit == PricingUnit1 and x_PricingBlockSize != x_PricingBlockSize1 and isnotempty(x_PricingBlockSize) and isnotempty(x_PricingBlockSize1), ListUnitPrice1 * x_BillingExchangeRate / x_PricingBlockSize1 * x_PricingBlockSize,\r\n // Otherwise, assume the contracted price is the same as list price to support aggregations\r\n ContractedUnitPrice\r\n )\r\n // Calculate missing costs based on new prices -- If cost is already correct, keep that; if not and price is available, recalculate the cost; otherwise, keep the existing cost\r\n | extend ContractedCost = case(\r\n // If not set or there's no cost, keep the original value\r\n (isnotempty(ContractedCost) and ContractedCost != 0) or (EffectiveCost == 0 and BilledCost == 0), ContractedCost,\r\n // ContractedCost is 0 in all other scenarios...\r\n // If 0 and there's a billed cost and prices are the same, use BilledCost\r\n isnotempty(BilledCost) and BilledCost != 0 and ContractedUnitPrice == x_BilledUnitPrice, BilledCost,\r\n // If 0 and there's a billed cost and prices are the same, use EffectiveCost\r\n isnotempty(EffectiveCost) and EffectiveCost != 0 and ContractedUnitPrice == x_EffectiveUnitPrice, EffectiveCost,\r\n // If 0 and there's a price, calculate the cost based on the price\r\n isnotempty(ContractedUnitPrice) and ContractedUnitPrice != 0, ContractedUnitPrice * PricingQuantity,\r\n // If 0 and there's no price, assume EffectiveCost\r\n isempty(ContractedUnitPrice) or ContractedUnitPrice == 0, EffectiveCost,\r\n // Fall back to the original value for any unhandled scenarios\r\n ContractedCost\r\n )\r\n | extend ListCost = case(\r\n // If not set or there's no cost, keep the original value\r\n (isnotempty(ListCost) and ListCost != 0) or (EffectiveCost == 0 and BilledCost == 0), ListCost,\r\n // ListCost is 0 in all other scenarios...\r\n // If 0 and there's a contracted cost and prices are the same, use ContractedCost\r\n isnotempty(ContractedCost) and ContractedCost != 0 and ListUnitPrice == ContractedUnitPrice, ContractedCost,\r\n // If 0 and there's a price, calculate the cost based on the price\r\n isnotempty(ListUnitPrice) and ListUnitPrice != 0, ListUnitPrice * PricingQuantity,\r\n // If 0 and there's no price, assume ContractedCost\r\n isempty(ListUnitPrice) or ListUnitPrice == 0, ContractedCost,\r\n // Fall back to the original value for any unhandled scenarios\r\n ListCost\r\n )\r\n // Merge the rest of the unmodified cost records and remove excess columns\r\n | union (allCosts | where not(tmp_MissingPrices))\r\n | project-away x_PricingBlockSize1, PricingUnit1, ListUnitPrice1, ContractedUnitPrice1, tmp_MissingPrices, tmp_ReservationPriceLookupKey, tmp_ReservationPriceLookupKey1\r\n //\r\n // BUG: Fix ContractedCost that has bad values\r\n | extend ContractedCost = iff(ProviderName == 'Microsoft' and isnotempty(PricingQuantity) and isnotempty(x_PricingBlockSize) and ContractedCost != ContractedUnitPrice * PricingQuantity, ContractedUnitPrice * PricingQuantity, ContractedCost)\r\n //\r\n // Handle FOCUS 1.0-preview UsageQuantity/Unit\r\n | extend ConsumedQuantity = iff(ChargeCategory == 'Usage', coalesce(ConsumedQuantity, UsageQuantity, UsageAmount), todecimal(''))\r\n | extend ConsumedUnit = iff(ChargeCategory == 'Usage' and isnotempty(ConsumedQuantity), coalesce(ConsumedUnit, UsageUnit, 'Units'), '')\r\n //\r\n // Convert IDs to lowercase for consistency\r\n | extend CommitmentDiscountId = tolower(CommitmentDiscountId)\r\n //\r\n // BUG: Remove EffectiveCost for commitment discount purchases\r\n | extend EffectiveCost = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), decimal(0), EffectiveCost)\r\n | extend x_EffectiveCostInUsd = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), decimal(0), x_EffectiveCostInUsd)\r\n //\r\n // Clean up resource columns\r\n | extend ResourceId = case(\r\n isnotempty(ResourceId), ResourceId,\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId), CommitmentDiscountId,\r\n ResourceId)\r\n | extend ResourceName = tolower(case(\r\n isnotempty(ResourceName), ResourceName,\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountName), CommitmentDiscountName,\r\n isnotempty(ResourceId), parse_resourceid(ResourceId).ResourceName,\r\n ResourceName))\r\n | extend x_ResourceType = case(\r\n isnotempty(x_ResourceType), x_ResourceType,\r\n isnotempty(ResourceId), parse_resourceid(ResourceId).x_ResourceType,\r\n x_ResourceType)\r\n | extend ResourceType = case(\r\n // Use existing resource type display name unless it's an internal resource type ID\r\n isnotempty(ResourceType) and tolower(ResourceType) != tolower(x_ResourceType) and ResourceType !contains '/', ResourceType,\r\n // Use CommitmentDiscountType for commitment discount purchases\r\n ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountType), CommitmentDiscountType,\r\n // Look up display name from internal type\r\n isnotempty(x_ResourceType), coalesce(resource_type(x_ResourceType).SingularDisplayName, ResourceType, x_ResourceType),\r\n ResourceType)\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n AvailabilityZone,\r\n BilledCost,\r\n BillingAccountId = tolower(BillingAccountId),\r\n BillingAccountName,\r\n BillingAccountType,\r\n BillingCurrency,\r\n BillingPeriodEnd = startofmonth(BillingPeriodEnd),\r\n BillingPeriodStart = startofmonth(BillingPeriodStart),\r\n ChargeCategory = case(\r\n // Handle FOCUS 1.0-preview ChargeSubcategory\r\n ChargeSubcategory == 'Credit', 'Credit',\r\n ChargeSubcategory == 'Refund', 'Purchase', // We are assuming purchase refunds since we don't have data to indicate usage refunds\r\n ChargeCategory\r\n ),\r\n ChargeClass = case(ChargeSubcategory == 'Refund', 'Correction', ChargeClass),\r\n ChargeDescription,\r\n // BUG: ChargeFrequency shows \"Usage-Based\" for monthly recurring savings plan purchases\r\n ChargeFrequency = iff(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountId) and ChargeFrequency == 'Usage-Based' and ProviderName == 'Microsoft' and x_SourceVersion startswith '1.0', 'Recurring', ChargeFrequency),\r\n ChargePeriodEnd,\r\n ChargePeriodStart,\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId = tolower(CommitmentDiscountId),\r\n CommitmentDiscountName,\r\n CommitmentDiscountStatus = case(\r\n // Handle FOCUS 1.0-preview ChargeSubcategory\r\n ChargeSubcategory == 'Used Commitment', 'Used',\r\n ChargeSubcategory == 'Unused Commitment', 'Unused',\r\n CommitmentDiscountStatus\r\n ),\r\n CommitmentDiscountType,\r\n ConsumedQuantity,\r\n ConsumedUnit,\r\n ContractedCost = coalesce(ContractedCost, x_OnDemandCost, x_Cost),\r\n ContractedUnitPrice = coalesce(ContractedUnitPrice, x_OnDemandUnitPrice),\r\n EffectiveCost,\r\n InvoiceIssuerName,\r\n ListCost,\r\n ListUnitPrice,\r\n PricingCategory = case(\r\n // Handle FOCUS 1.0-preview PricingCategory values\r\n PricingCategory == 'On-Demand', 'Standard',\r\n PricingCategory == 'Commitment-Based', 'Committed',\r\n PricingCategory\r\n ),\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName,\r\n // Handle missing PublisherName values\r\n PublisherName = case(PublisherName == 'Microsoft Corporation', 'Microsoft', isnotempty(PublisherName), PublisherName, x_PublisherCategory == 'Cloud Provider', ProviderName, ''),\r\n // Handle FOCUS 1.0-preview Region column\r\n RegionId = coalesce(RegionId, iff(ProviderName == 'Microsoft', replace_string(tolower(Region), ' ', ''), Region)),\r\n RegionName = coalesce(RegionName, Region),\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n SkuId,\r\n SkuPriceId,\r\n SubAccountId,\r\n SubAccountName,\r\n SubAccountType, // Azure 1.0-preview(v1)+\r\n Tags = parse_json(Tags),\r\n x_AccountId, // Azure 1.0-preview(v1)+\r\n x_AccountName, // Azure 1.0-preview(v1)+\r\n x_AccountOwnerId, // Azure 1.0-preview(v1)+\r\n x_BilledCostInUsd, // Azure 1.0-preview(v1)+\r\n x_BilledUnitPrice, // Azure 1.0-preview(v1)+\r\n x_BillingAccountAgreement = case(\r\n ProviderName == 'Microsoft' and x_BillingAccountId == x_BillingProfileId, 'EA',\r\n ProviderName == 'Microsoft' and x_BillingAccountId != x_BillingProfileId, 'MCA',\r\n ProviderName\r\n ), // Hubs add-on\r\n x_BillingAccountId, // Azure 1.0-preview(v1)+\r\n x_BillingAccountName, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRate, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRateDate, // Azure 1.0-preview(v1)+\r\n x_BillingProfileId, // Azure 1.0-preview(v1)+\r\n x_BillingProfileName, // Azure 1.0-preview(v1)+\r\n x_ChargeId, // Azure 1.0-preview(v1) only\r\n x_ContractedCostInUsd = coalesce(x_ContractedCostInUsd, x_OnDemandCostInUsd), // Azure 1.0+\r\n x_CostAllocationRuleName, // Azure 1.0-preview(v1)+\r\n x_CostCategories = parse_json(x_CostCategories), // AWS 1.0 (JSON)\r\n x_CostCenter, // Azure 1.0-preview(v1)+\r\n x_Credits = parse_json(x_Credits), // GCP Jan 2024\r\n x_CostType, // GCP Jan 2024\r\n x_CurrencyConversionRate, // GCP Jun 2024\r\n x_CustomerId, // Azure 1.0-preview(v1)+\r\n x_CustomerName, // Azure 1.0-preview(v1)+\r\n x_Discount = parse_json(x_Discount), // AWS 1.0 (JSON)\r\n x_EffectiveCostInUsd, // Azure 1.0-preview(v1)+\r\n x_EffectiveUnitPrice, // Azure 1.0-preview(v1)+\r\n x_ExportTime, // GCP Jan 2024\r\n x_IngestionTime, // Hubs add-on\r\n x_InvoiceId = coalesce(InvoiceId, x_InvoiceId), // Azure 1.0-preview(v1)+\r\n x_InvoiceIssuerId, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionId = case( // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionId == '-2', '',\r\n x_InvoiceSectionId\r\n ),\r\n x_InvoiceSectionName = case( // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionName == 'Unassigned', '',\r\n x_InvoiceSectionName\r\n ),\r\n x_ListCostInUsd, // Azure 1.0-preview(v1)+\r\n x_Location, // GCP Jan 2024\r\n x_Operation, // AWS 1.0\r\n x_PartnerCreditApplied, // Azure 1.0-preview(v1)+\r\n x_PartnerCreditRate, // Azure 1.0-preview(v1)+\r\n x_PricingBlockSize, // Azure 1.0-preview(v1)+\r\n x_PricingCurrency = coalesce(PricingCurrency, x_PricingCurrency), // Azure 1.0-preview(v1)+\r\n x_PricingSubcategory, // Azure 1.0-preview(v1)+\r\n x_PricingUnitDescription, // Azure 1.0-preview(v1)+\r\n x_Project, // GCP Jan 2024\r\n x_PublisherCategory, // Azure 1.0-preview(v1)+\r\n x_PublisherId, // Azure 1.0-preview(v1)+\r\n x_ResellerId, // Azure 1.0-preview(v1)+\r\n x_ResellerName, // Azure 1.0-preview(v1)+\r\n x_ResourceGroupName = tolower(x_ResourceGroupName), // Azure 1.0-preview(v1)+\r\n x_ResourceType, // Azure 1.0-preview(v1)+\r\n x_ServiceCode, // AWS 1.0\r\n x_ServiceId, // GCP Jan 2024\r\n x_ServicePeriodEnd, // Azure 1.0-preview(v1)+\r\n x_ServicePeriodStart, // Azure 1.0-preview(v1)+\r\n x_SkuDescription, // Azure 1.0-preview(v1)+\r\n x_SkuDetails = parse_json(x_SkuDetails), // Azure 1.0-preview(v1)+\r\n x_SkuIsCreditEligible, // Azure 1.0-preview(v1)+\r\n x_SkuMeterCategory, // Azure 1.0-preview(v1)+\r\n x_SkuMeterId, // Azure 1.0-preview(v1)+\r\n x_SkuMeterName = coalesce(SkuMeter, x_SkuMeterName), // Azure 1.0-preview(v1)+\r\n x_SkuMeterSubcategory, // Azure 1.0-preview(v1)+\r\n x_SkuOfferId, // Azure 1.0-preview(v1)+\r\n x_SkuOrderId, // Azure 1.0-preview(v1)+\r\n x_SkuOrderName, // Azure 1.0-preview(v1)+\r\n x_SkuPartNumber, // Azure 1.0-preview(v1)+\r\n x_SkuRegion, // Azure 1.0-preview(v1)+\r\n x_SkuServiceFamily, // Azure 1.0-preview(v1)+\r\n x_SkuTerm, // Azure 1.0-preview(v1)+\r\n x_SkuTier, // Azure 1.0-preview(v1)+\r\n x_SourceChanges, // Hubs add-on\r\n x_SourceName, // Hubs add-on\r\n x_SourceProvider, // Hubs add-on\r\n x_SourceType, // Hubs add-on\r\n x_SourceVersion, // Hubs add-on\r\n x_UsageType // AWS 1.0\r\n}\r\n\r\n// Costs_final_v1_0 table\r\n.create-merge table Costs_final_v1_0 (\r\n AvailabilityZone: string,\r\n BilledCost: decimal,\r\n BillingAccountId: string,\r\n BillingAccountName: string,\r\n BillingAccountType: string, // Azure 1.0-preview(v1)+\r\n BillingCurrency: string,\r\n BillingPeriodEnd: datetime,\r\n BillingPeriodStart: datetime,\r\n ChargeCategory: string,\r\n ChargeClass: string,\r\n ChargeDescription: string,\r\n ChargeFrequency: string,\r\n ChargePeriodEnd: datetime,\r\n ChargePeriodStart: datetime,\r\n CommitmentDiscountCategory: string, // FOCUS 1.0-preview only\r\n CommitmentDiscountId: string,\r\n CommitmentDiscountName: string,\r\n CommitmentDiscountStatus: string,\r\n CommitmentDiscountType: string,\r\n ConsumedQuantity: decimal,\r\n ConsumedUnit: string,\r\n ContractedCost: decimal,\r\n ContractedUnitPrice: decimal,\r\n EffectiveCost: decimal,\r\n InvoiceIssuerName: string,\r\n ListCost: decimal,\r\n ListUnitPrice: decimal,\r\n PricingCategory: string,\r\n PricingQuantity: decimal,\r\n PricingUnit: string,\r\n ProviderName: string,\r\n PublisherName: string,\r\n RegionId: string,\r\n RegionName: string,\r\n ResourceId: string,\r\n ResourceName: string,\r\n ResourceType: string,\r\n ServiceCategory: string,\r\n ServiceName: string,\r\n SkuId: string,\r\n SkuPriceId: string,\r\n SubAccountId: string,\r\n SubAccountName: string,\r\n SubAccountType: string,\r\n Tags: dynamic,\r\n x_AccountId: string, // Azure 1.0-preview(v1)+\r\n x_AccountName: string, // Azure 1.0-preview(v1)+\r\n x_AccountOwnerId: string, // Azure 1.0-preview(v1)+\r\n x_BilledCostInUsd: decimal, // Azure 1.0-preview(v1)+\r\n x_BilledUnitPrice: decimal, // Azure 1.0-preview(v1)+\r\n x_BillingAccountAgreement: string, // Hubs add-on\r\n x_BillingAccountId: string, // Azure 1.0-preview(v1)+\r\n x_BillingAccountName: string, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRate: decimal, // Azure 1.0-preview(v1)+\r\n x_BillingExchangeRateDate: datetime, // Azure 1.0-preview(v1)+\r\n x_BillingProfileId: string, // Azure 1.0-preview(v1)+\r\n x_BillingProfileName: string, // Azure 1.0-preview(v1)+\r\n x_ChargeId: string, // Azure 1.0-preview(v1) only\r\n x_ContractedCostInUsd: decimal, // Azure 1.0+\r\n x_CostAllocationRuleName: string, // Azure 1.0-preview(v1)+\r\n x_CostCategories: dynamic, // AWS 1.0 (JSON)\r\n x_CostCenter: string, // Azure 1.0-preview(v1)+\r\n x_Credits: dynamic, // GCP Jan 2024\r\n x_CostType: string, // GCP Jan 2024\r\n x_CurrencyConversionRate: decimal, // GCP Jun 2024\r\n x_CustomerId: string, // Azure 1.0-preview(v1)+\r\n x_CustomerName: string, // Azure 1.0-preview(v1)+\r\n x_Discount: dynamic, // AWS 1.0 (JSON)\r\n x_EffectiveCostInUsd: decimal, // Azure 1.0-preview(v1)+\r\n x_EffectiveUnitPrice: decimal, // Azure 1.0-preview(v1)+\r\n x_ExportTime: datetime, // GCP Jan 2024\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_InvoiceId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceIssuerId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionId: string, // Azure 1.0-preview(v1)+\r\n x_InvoiceSectionName: string, // Azure 1.0-preview(v1)+\r\n x_ListCostInUsd: decimal, // Azure 1.0-preview(v1)+\r\n x_Location: string, // GCP Jan 2024\r\n x_Operation: string, // AWS 1.0\r\n x_PartnerCreditApplied: string, // Azure 1.0-preview(v1)+\r\n x_PartnerCreditRate: string, // Azure 1.0-preview(v1)+\r\n x_PricingBlockSize: decimal, // Azure 1.0-preview(v1)+\r\n x_PricingCurrency: string, // Azure 1.0-preview(v1)+\r\n x_PricingSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_PricingUnitDescription: string, // Azure 1.0-preview(v1)+\r\n x_Project: string, // GCP Jan 2024\r\n x_PublisherCategory: string, // Azure 1.0-preview(v1)+\r\n x_PublisherId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerId: string, // Azure 1.0-preview(v1)+\r\n x_ResellerName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceGroupName: string, // Azure 1.0-preview(v1)+\r\n x_ResourceType: string, // Azure 1.0-preview(v1)+\r\n x_ServiceCode: string, // AWS 1.0\r\n x_ServiceId: string, // GCP Jan 2024\r\n x_ServicePeriodEnd: datetime, // Azure 1.0-preview(v1)+\r\n x_ServicePeriodStart: datetime, // Azure 1.0-preview(v1)+\r\n x_SkuDescription: string, // Azure 1.0-preview(v1)+\r\n x_SkuDetails: dynamic, // Azure 1.0-preview(v1)+\r\n x_SkuIsCreditEligible: bool, // Azure 1.0-preview(v1)+\r\n x_SkuMeterCategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterId: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterName: string, // Azure 1.0-preview(v1)+\r\n x_SkuMeterSubcategory: string, // Azure 1.0-preview(v1)+\r\n x_SkuOfferId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOrderId: string, // Azure 1.0-preview(v1)+\r\n x_SkuOrderName: string, // Azure 1.0-preview(v1)+\r\n x_SkuPartNumber: string, // Azure 1.0-preview(v1)+\r\n x_SkuRegion: string, // Azure 1.0-preview(v1)+\r\n x_SkuServiceFamily: string, // Azure 1.0-preview(v1)+\r\n x_SkuTerm: int, // Azure 1.0-preview(v1)+\r\n x_SkuTier: string, // Azure 1.0-preview(v1)+\r\n x_SourceChanges: string, // Hubs add-on\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_UsageType: string // AWS 1.0\r\n)\r\n\r\n// Update policy for Costs_raw -> Costs_final_v1_0 table\r\n.alter table Costs_final_v1_0 policy update\r\n```\r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"Costs_raw\",\r\n \"Query\": \"Costs_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Actual costs |===================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// ActualCosts_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: ActualCost exports transformed to FOCUS 1.0. Use ActualCosts_transform_v1_2() instead.', folder='Costs')\r\nActualCosts_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n // TODO: Transform actual costs to FOCUS 1.0 format\r\n ActualCosts_raw\r\n | where ChargeType in ('Purchase', 'Refund') and isnotempty(ReservationId)\r\n //\r\n //\r\n // !!! The rest of the query is reused for both the ActualCosts and AmortizedCosts queries -- Copy all changes in both transform functions !!!\r\n //\r\n //\r\n | extend x_AmortizationClass = case(\r\n ChargeType in ('Purchase', 'Refund') and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', 'Amortized Charge',\r\n ''\r\n )\r\n | extend tmp_ResourceInfo = parse_resourceid(ResourceId)\r\n // TODO: PricingCategory needs to include savings plan usage and spot usage\r\n | extend PricingCategory = case(\r\n x_AmortizationClass == 'Amortized Charge', 'Committed',\r\n ChargeType in ('Usage', 'Purchase'), 'Standard',\r\n ''\r\n )\r\n | extend x_ResourceType = tostring(tmp_ResourceInfo.x_ResourceType)\r\n | project-rename\r\n PricingQuantity = Quantity,\r\n x_PricingUnitDescription = UnitOfMeasure\r\n | join kind=leftouter (PricingUnits) on x_PricingUnitDescription\r\n | join kind=leftouter (Regions) on ResourceLocation\r\n | join kind=leftouter (ResourceTypes | project x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n // TODO: Add the following in 1.2: ServiceSubcategory, PublisherName, x_PublisherCategory, x_Environment, x_ServiceModel\r\n | join kind=leftouter (Services | where isnotempty(x_ResourceType) | project x_ResourceType, ServiceName, ServiceCategory) on x_ResourceType\r\n | join kind=leftouter (Services | where isnotempty(x_ConsumedService) | summarize take_any(ServiceName), take_any(ServiceCategory) by ConsumedService = x_ConsumedService) on ConsumedService\r\n | extend CommitmentDiscountCategory = iff(isnotempty(ReservationId), 'Usage', '') // TODO: CommitmentDiscountCategory needs to handle savings plans\r\n | project\r\n AvailabilityZone = AvailabilityZone,\r\n BilledCost = iff(x_AmortizationClass == 'Amortized Charge', real(0), Cost),\r\n BillingAccountId = strcat('/providers/microsoft.billing/billingaccounts/', BillingAccountId, iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), '', strcat('/billingprofiles/', BillingProfileId))),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName),\r\n BillingAccountType = iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), 'Billing Profile', 'Billing Account'),\r\n BillingCurrency,\r\n BillingPeriodEnd = startofmonth(BillingPeriodEndDate, 1),\r\n BillingPeriodStart = startofmonth(BillingPeriodStartDate),\r\n CapacityReservationId = '',\r\n CapacityReservationStatus = '',\r\n ChargeCategory = case(\r\n ChargeType in ('Usage', 'Purchase', 'Credit', 'Tax'), ChargeType,\r\n ChargeType in ('UnusedReservation', 'UnusedSavingsPlan'), 'Usage',\r\n ChargeType == 'Refund', 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = iff(ChargeType == 'Refund', 'Correction', ''),\r\n ChargeDescription = Product,\r\n ChargeFrequency = case(\r\n Frequency == 'UsageBased', 'Usage-Based',\r\n Frequency == 'OneTime', 'One-Time',\r\n Frequency // \"Recurring\" and any fallback\r\n ),\r\n ChargePeriodStart = Date,\r\n ChargePeriodEnd = Date + 1d,\r\n ChargeSubcategory = '',\r\n // TODO: CommitmentDiscount* columns need to handle savings plans\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId = iff(isnotempty(ReservationId), strcat('/providers/microsoft.capacity/reservationorders/', ProductOrderId, '/reservations/', ReservationId), ''),\r\n CommitmentDiscountName = ReservationName,\r\n CommitmentDiscountQuantity = real(null),\r\n CommitmentDiscountStatus = case(\r\n isempty(ReservationId), '',\r\n isnotempty(ReservationId) and ChargeType == 'Usage', 'Used',\r\n ChargeType startswith 'Unused', 'Unused',\r\n 'Unused'\r\n ),\r\n CommitmentDiscountType = iff(isnotempty(ReservationId), 'Reservation', ''),\r\n CommitmentDiscountUnit = '',\r\n ConsumedQuantity = PricingQuantity * x_PricingBlockSize,\r\n ConsumedUnit = PricingUnit,\r\n ContractedCost = UnitPrice * PricingQuantity,\r\n ContractedUnitPrice = UnitPrice,\r\n EffectiveCost = iff(x_AmortizationClass == 'Principal', real(0), Cost),\r\n InvoiceId = '',\r\n InvoiceIssuerName = 'Microsoft',\r\n ListCost = real(null),\r\n ListUnitPrice = real(null),\r\n PricingCategory,\r\n PricingCurrency = '',\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName = 'Microsoft',\r\n PublisherName = iff(PublisherType == 'Marketplace', PublisherName, 'Microsoft'),\r\n Region = '',\r\n RegionId,\r\n RegionName,\r\n ResourceId = tostring(tmp_ResourceInfo.ResourceId),\r\n ResourceName = tostring(tmp_ResourceInfo.ResourceName),\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory = '',\r\n SkuId = '',\r\n SkuMeter = '',\r\n SkuPriceDetails = '',\r\n SkuPriceId = '',\r\n SubAccountId = strcat('/subscriptions/', SubscriptionId),\r\n SubAccountName = SubscriptionName,\r\n SubAccountType = iff(isempty(SubscriptionId), '', 'Subscription'),\r\n Tags = iff(Tags startswith '{', Tags, strcat('{', Tags, '}')),\r\n UsageAmount = real(null),\r\n UsageQuantity = real(null),\r\n UsageUnit = '',\r\n x_AccountId = '',\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerId,\r\n x_AmortizationClass = '',\r\n x_BilledCostInUsd = real(null),\r\n x_BilledUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), UnitPrice),\r\n x_BillingAccountId = BillingAccountId,\r\n x_BillingAccountName = BillingAccountName,\r\n x_BillingExchangeRate = real(null),\r\n x_BillingExchangeRateDate = datetime(null),\r\n x_BillingProfileId = BillingProfileId,\r\n x_BillingProfileName = BillingProfileName,\r\n x_BillingItemCode = '',\r\n x_BillingItemName = '',\r\n x_ChargeId = '',\r\n x_CommodityCode = '',\r\n x_CommodityName = '',\r\n x_ComponentName = '',\r\n x_ComponentType = '',\r\n x_ContractedCostInUsd = real(null),\r\n x_Cost = real(null),\r\n x_CostAllocationRuleName = '',\r\n x_CostCategories = '',\r\n x_CostCenter = CostCenter,\r\n x_CostType = '',\r\n x_Credits = '',\r\n x_CurrencyConversionRate = real(null),\r\n x_CustomerId = '',\r\n x_CustomerName = '',\r\n x_Discount = '',\r\n x_EffectiveCostInUsd = real(null),\r\n x_EffectiveUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), EffectivePrice),\r\n x_ExportTime = datetime(null),\r\n x_InstanceID = '',\r\n x_InvoiceId = '',\r\n x_InvoiceIssuerId = '',\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = InvoiceSection,\r\n x_ListCostInUsd = real(null),\r\n x_Location = '',\r\n x_OnDemandCost = real(null),\r\n x_OnDemandCostInUsd = real(null),\r\n x_OnDemandUnitPrice = real(null),\r\n x_Operation = '',\r\n x_OwnerAccountID = '',\r\n x_PartnerCreditApplied = '',\r\n x_PartnerCreditRate = '',\r\n x_PricingBlockSize,\r\n x_PricingCurrency = iff(BillingAccountId == BillingProfileId, BillingCurrency, ''),\r\n x_PricingSubcategory = case(\r\n // TODO: Add x_SkuTier when supported by C360 -- PricingCategory == 'Standard' and isnotempty(x_SkuTier), 'Tiered',\r\n PricingCategory == 'Standard', 'Standard',\r\n PricingCategory == 'Committed', strcat('Committed ', CommitmentDiscountCategory),\r\n PricingCategory == 'Dynamic', 'Spot',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_Project = '',\r\n x_PublisherCategory = iff(PublisherType == 'Marketplace', 'Vendor', 'Cloud Provider'),\r\n x_PublisherId = '',\r\n x_ResellerId = '',\r\n x_ResellerName = '',\r\n x_ResourceGroupName = ResourceGroup,\r\n x_ResourceType,\r\n x_ServiceCode = '',\r\n x_ServiceId = '',\r\n x_ServiceModel = '',\r\n x_ServicePeriodEnd = datetime(null),\r\n x_ServicePeriodStart = datetime(null),\r\n x_SkuDescription = Product,\r\n x_SkuDetails = iff(AdditionalInfo startswith '{', AdditionalInfo, strcat('{', AdditionalInfo, '}')),\r\n x_SkuIsCreditEligible = IsAzureCreditEligible,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterId = MeterId,\r\n x_SkuMeterName = MeterName,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuOfferId = OfferId,\r\n x_SkuOrderId = ProductOrderId,\r\n x_SkuOrderName = ProductOrderName,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPlanName = '',\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTerm = toint(Term),\r\n x_SkuTier = '',\r\n x_SourceName = 'C360',\r\n x_SourceProvider = 'Microsoft',\r\n x_SourceType = 'ActualCost',\r\n x_SourceVersion = 'C360-2025-04',\r\n x_SubproductName = '',\r\n x_UsageType = ''\r\n}\r\n\r\n// Update policy for ActualCosts_raw -> Costs_raw table\r\n.alter table Costs_raw policy update\r\n``` \r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"ActualCosts_raw\",\r\n \"Query\": \"ActualCosts_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Amortized costs |================================================================================================\r\n// Supported versions:\r\n// - C360-2025-04\r\n//======================================================================================================================\r\n\r\n// AmortizedCosts_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: ActualCost exports transformed to FOCUS 1.0. Use AmortizedCosts_transform_v1_2() instead.', folder='Costs')\r\nAmortizedCosts_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n // TODO: Transform actual costs to FOCUS 1.0 format\r\n AmortizedCosts_raw\r\n //\r\n //\r\n // !!! The rest of the query is reused for both the ActualCosts and AmortizedCosts queries -- Copy all changes in both transform functions !!!\r\n //\r\n //\r\n | extend x_AmortizationClass = case(\r\n ChargeType in ('Purchase', 'Refund') and (tolower(ResourceId) contains '/microsoft.capacity/reservationorders/' or tolower(ResourceId) contains '/microsoft.billingbenefits/savingsplanorders/'), 'Principal',\r\n ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', 'Amortized Charge',\r\n ''\r\n )\r\n | extend tmp_ResourceInfo = parse_resourceid(ResourceId)\r\n // TODO: PricingCategory needs to include savings plan usage and spot usage\r\n | extend PricingCategory = case(\r\n x_AmortizationClass == 'Amortized Charge', 'Committed',\r\n ChargeType in ('Usage', 'Purchase'), 'Standard',\r\n ''\r\n )\r\n | extend x_ResourceType = tostring(tmp_ResourceInfo.x_ResourceType)\r\n | project-rename\r\n PricingQuantity = Quantity,\r\n x_PricingUnitDescription = UnitOfMeasure\r\n | join kind=leftouter (PricingUnits) on x_PricingUnitDescription\r\n | join kind=leftouter (Regions) on ResourceLocation\r\n | join kind=leftouter (ResourceTypes | project x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n // TODO: Add the following in 1.2: ServiceSubcategory, PublisherName, x_PublisherCategory, x_Environment, x_ServiceModel\r\n | join kind=leftouter (Services | where isnotempty(x_ResourceType) | project x_ResourceType, ServiceName, ServiceCategory) on x_ResourceType\r\n | join kind=leftouter (Services | where isnotempty(x_ConsumedService) | summarize take_any(ServiceName), take_any(ServiceCategory) by ConsumedService = x_ConsumedService) on ConsumedService\r\n | extend CommitmentDiscountCategory = iff(isnotempty(ReservationId), 'Usage', '') // TODO: CommitmentDiscountCategory needs to handle savings plans\r\n | project\r\n AvailabilityZone = AvailabilityZone,\r\n BilledCost = iff(x_AmortizationClass == 'Amortized Charge', real(0), Cost),\r\n BillingAccountId = strcat('/providers/microsoft.billing/billingaccounts/', BillingAccountId, iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), '', strcat('/billingprofiles/', BillingProfileId))),\r\n BillingAccountName = coalesce(BillingProfileName, BillingAccountName),\r\n BillingAccountType = iff(BillingAccountId == BillingProfileId or isempty(BillingProfileId), 'Billing Profile', 'Billing Account'),\r\n BillingCurrency,\r\n BillingPeriodEnd = startofmonth(BillingPeriodEndDate, 1),\r\n BillingPeriodStart = startofmonth(BillingPeriodStartDate),\r\n CapacityReservationId = '',\r\n CapacityReservationStatus = '',\r\n ChargeCategory = case(\r\n ChargeType in ('Usage', 'Purchase', 'Credit', 'Tax'), ChargeType,\r\n ChargeType in ('UnusedReservation', 'UnusedSavingsPlan'), 'Usage',\r\n ChargeType == 'Refund', 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = iff(ChargeType == 'Refund', 'Correction', ''),\r\n ChargeDescription = Product,\r\n ChargeFrequency = case(\r\n Frequency == 'UsageBased', 'Usage-Based',\r\n Frequency == 'OneTime', 'One-Time',\r\n Frequency // \"Recurring\" and any fallback\r\n ),\r\n ChargePeriodStart = Date,\r\n ChargePeriodEnd = Date + 1d,\r\n ChargeSubcategory = '',\r\n // TODO: CommitmentDiscount* columns need to handle savings plans\r\n CommitmentDiscountCategory,\r\n CommitmentDiscountId = iff(isnotempty(ReservationId), strcat('/providers/microsoft.capacity/reservationorders/', ProductOrderId, '/reservations/', ReservationId), ''),\r\n CommitmentDiscountName = ReservationName,\r\n CommitmentDiscountQuantity = real(null),\r\n CommitmentDiscountStatus = case(\r\n isempty(ReservationId), '',\r\n isnotempty(ReservationId) and ChargeType == 'Usage', 'Used',\r\n ChargeType startswith 'Unused', 'Unused',\r\n 'Unused'\r\n ),\r\n CommitmentDiscountType = iff(isnotempty(ReservationId), 'Reservation', ''),\r\n CommitmentDiscountUnit = '',\r\n ConsumedQuantity = PricingQuantity * x_PricingBlockSize,\r\n ConsumedUnit = PricingUnit,\r\n ContractedCost = UnitPrice * PricingQuantity,\r\n ContractedUnitPrice = UnitPrice,\r\n EffectiveCost = iff(x_AmortizationClass == 'Principal', real(0), Cost),\r\n InvoiceId = '',\r\n InvoiceIssuerName = 'Microsoft',\r\n ListCost = real(null),\r\n ListUnitPrice = real(null),\r\n PricingCategory,\r\n PricingCurrency = '',\r\n PricingQuantity,\r\n PricingUnit,\r\n ProviderName = 'Microsoft',\r\n PublisherName = iff(PublisherType == 'Marketplace', PublisherName, 'Microsoft'),\r\n Region = '',\r\n RegionId,\r\n RegionName,\r\n ResourceId = tostring(tmp_ResourceInfo.ResourceId),\r\n ResourceName = tostring(tmp_ResourceInfo.ResourceName),\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n ServiceSubcategory = '',\r\n SkuId = '',\r\n SkuMeter = '',\r\n SkuPriceDetails = '',\r\n SkuPriceId = '',\r\n SubAccountId = strcat('/subscriptions/', SubscriptionId),\r\n SubAccountName = SubscriptionName,\r\n SubAccountType = iff(isempty(SubscriptionId), '', 'Subscription'),\r\n Tags = iff(Tags startswith '{', Tags, strcat('{', Tags, '}')),\r\n UsageAmount = real(null),\r\n UsageQuantity = real(null),\r\n UsageUnit = '',\r\n x_AccountId = '',\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerId,\r\n x_AmortizationClass = '',\r\n x_BilledCostInUsd = real(null),\r\n x_BilledUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), UnitPrice),\r\n x_BillingAccountId = BillingAccountId,\r\n x_BillingAccountName = BillingAccountName,\r\n x_BillingExchangeRate = real(null),\r\n x_BillingExchangeRateDate = datetime(null),\r\n x_BillingProfileId = BillingProfileId,\r\n x_BillingProfileName = BillingProfileName,\r\n x_BillingItemCode = '',\r\n x_BillingItemName = '',\r\n x_ChargeId = '',\r\n x_CommodityCode = '',\r\n x_CommodityName = '',\r\n x_ComponentName = '',\r\n x_ComponentType = '',\r\n x_ContractedCostInUsd = real(null),\r\n x_Cost = real(null),\r\n x_CostAllocationRuleName = '',\r\n x_CostCategories = '',\r\n x_CostCenter = CostCenter,\r\n x_CostType = '',\r\n x_Credits = '',\r\n x_CurrencyConversionRate = real(null),\r\n x_CustomerId = '',\r\n x_CustomerName = '',\r\n x_Discount = '',\r\n x_EffectiveCostInUsd = real(null),\r\n x_EffectiveUnitPrice = iff(ChargeType == 'Usage' and isnotempty(ReservationId) and ChargeType != 'UnusedSavingsPlan', real(0), EffectivePrice),\r\n x_ExportTime = datetime(null),\r\n x_InstanceID = '',\r\n x_InvoiceId = '',\r\n x_InvoiceIssuerId = '',\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = InvoiceSection,\r\n x_ListCostInUsd = real(null),\r\n x_Location = '',\r\n x_OnDemandCost = real(null),\r\n x_OnDemandCostInUsd = real(null),\r\n x_OnDemandUnitPrice = real(null),\r\n x_Operation = '',\r\n x_OwnerAccountID = '',\r\n x_PartnerCreditApplied = '',\r\n x_PartnerCreditRate = '',\r\n x_PricingBlockSize,\r\n x_PricingCurrency = iff(BillingAccountId == BillingProfileId, BillingCurrency, ''),\r\n x_PricingSubcategory = case(\r\n // TODO: Add x_SkuTier when supported by C360 -- PricingCategory == 'Standard' and isnotempty(x_SkuTier), 'Tiered',\r\n PricingCategory == 'Standard', 'Standard',\r\n PricingCategory == 'Committed', strcat('Committed ', CommitmentDiscountCategory),\r\n PricingCategory == 'Dynamic', 'Spot',\r\n ''\r\n ),\r\n x_PricingUnitDescription,\r\n x_Project = '',\r\n x_PublisherCategory = iff(PublisherType == 'Marketplace', 'Vendor', 'Cloud Provider'),\r\n x_PublisherId = '',\r\n x_ResellerId = '',\r\n x_ResellerName = '',\r\n x_ResourceGroupName = ResourceGroup,\r\n x_ResourceType,\r\n x_ServiceCode = '',\r\n x_ServiceId = '',\r\n x_ServiceModel = '',\r\n x_ServicePeriodEnd = datetime(null),\r\n x_ServicePeriodStart = datetime(null),\r\n x_SkuDescription = Product,\r\n x_SkuDetails = iff(AdditionalInfo startswith '{', AdditionalInfo, strcat('{', AdditionalInfo, '}')),\r\n x_SkuIsCreditEligible = IsAzureCreditEligible,\r\n x_SkuMeterCategory = MeterCategory,\r\n x_SkuMeterId = MeterId,\r\n x_SkuMeterName = MeterName,\r\n x_SkuMeterSubcategory = MeterSubCategory,\r\n x_SkuOfferId = OfferId,\r\n x_SkuOrderId = ProductOrderId,\r\n x_SkuOrderName = ProductOrderName,\r\n x_SkuPartNumber = PartNumber,\r\n x_SkuPlanName = '',\r\n x_SkuRegion = MeterRegion,\r\n x_SkuServiceFamily = ServiceFamily,\r\n x_SkuTerm = toint(Term),\r\n x_SkuTier = '',\r\n x_SourceName = 'C360',\r\n x_SourceProvider = 'Microsoft',\r\n x_SourceType = 'ActualCost',\r\n x_SourceVersion = 'C360-2025-04',\r\n x_SubproductName = '',\r\n x_UsageType = ''\r\n}\r\n\r\n// Update policy for AmortizedCosts_raw -> Costs_raw table\r\n.alter table Costs_raw policy update\r\n``` \r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"AmortizedCosts_raw\",\r\n \"Query\": \"AmortizedCosts_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| CommitmentDiscountUsage |========================================================================================\r\n// Supported versions:\r\n// - MS EA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-ea\r\n// - MS MCA reservation details: 2023-03-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-details-mca\r\n//======================================================================================================================\r\n\r\n// CommitmentDiscountUsage_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: All commitment discount usage transformed to FOCUS 1.0. This includes reservationdeatils_raw. Use CommitmentDiscountUsage_transform_v1_2() instead.', folder='Commitment discounts')\r\nCommitmentDiscountUsage_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n CommitmentDiscountUsage_raw\r\n //\r\n // Change real to decimal\r\n | extend\r\n InstanceFlexibilityRatio = todecimal(InstanceFlexibilityRatio),\r\n ReservedHours = todecimal(ReservedHours),\r\n TotalReservedQuantity = todecimal(TotalReservedQuantity),\r\n UsedHours = todecimal(UsedHours)\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Handle resource columns\r\n | extend ResourceId = tolower(InstanceId)\r\n | extend tmp_ResourceDetails = parse_resourceid(ResourceId)\r\n | extend ResourceName = tostring(tmp_ResourceDetails.ResourceName)\r\n | extend SubAccountId = tostring(tmp_ResourceDetails.SubAccountId)\r\n | extend x_ResourceGroupName = tostring(tmp_ResourceDetails.x_ResourceGroupName)\r\n | extend x_ResourceType = tostring(tmp_ResourceDetails.x_ResourceType)\r\n | lookup kind=leftouter (ResourceTypes | distinct x_ResourceType, ResourceType = SingularDisplayName) on x_ResourceType\r\n | lookup kind=leftouter (Services | distinct x_ResourceType, ServiceName, ServiceCategory, x_ServiceModel) on x_ResourceType\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n ChargePeriodEnd = UsageDate + 1d,\r\n ChargePeriodStart = UsageDate,\r\n CommitmentDiscountCategory = 'Usage',\r\n CommitmentDiscountId = tolower(strcat('/providers/microsoft.capacity/reservationorders/', ReservationOrderId, '/reservations/', ReservationId)),\r\n CommitmentDiscountType = 'Reservation',\r\n ConsumedQuantity = UsedHours,\r\n ProviderName,\r\n ResourceId,\r\n ResourceName,\r\n ResourceType,\r\n ServiceCategory,\r\n ServiceName,\r\n SubAccountId,\r\n x_CommitmentDiscountCommittedCount = TotalReservedQuantity,\r\n x_CommitmentDiscountCommittedAmount = ReservedHours,\r\n // TODO: Is this needed? -- x_CommitmentDiscountKind = Kind,\r\n x_CommitmentDiscountNormalizedGroup = iff(InstanceFlexibilityGroup == 'NA', '', InstanceFlexibilityGroup),\r\n x_CommitmentDiscountNormalizedRatio = InstanceFlexibilityRatio,\r\n x_CommitmentDiscountQuantity = UsedHours * InstanceFlexibilityRatio,\r\n x_IngestionTime = ingestion_time(),\r\n x_ResourceGroupName,\r\n x_ResourceType,\r\n // x_RowId = hash_sha256(strcat(\r\n // // DO NOT CHANGE COLUMNS OR COLUMN ORDER\r\n // CommitmentDiscountId,\r\n // ResourceId,\r\n // ChargePeriodStart\r\n // )),\r\n x_ServiceModel,\r\n x_SkuOrderId = ReservationOrderId,\r\n x_SkuSize = iff(SkuName == 'NA', '', SkuName),\r\n x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName)),\r\n x_SourceProvider = coalesce(x_SourceProvider, ProviderName),\r\n x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationDetails', '')),\r\n x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2024-03-01', ''))\r\n}\r\n\r\n// CommitmentDiscountUsage_final_v1_0 table\r\n.create-merge table CommitmentDiscountUsage_final_v1_0 (\r\n ChargePeriodEnd: datetime, // Hubs add-on\r\n ChargePeriodStart: datetime, // MS 2023-03-01\r\n CommitmentDiscountCategory: string, // Hubs add-on\r\n CommitmentDiscountId: string, // MS 2023-03-01\r\n CommitmentDiscountType: string, // Hubs add-on\r\n ConsumedQuantity: decimal, // MS 2023-03-01\r\n ProviderName: string, // Hubs add-on\r\n ResourceId: string, // MS 2023-03-01\r\n ResourceName: string, // Hubs add-on\r\n ResourceType: string, // Hubs add-on\r\n ServiceCategory: string, // Hubs add-on\r\n ServiceName: string, // Hubs add-on\r\n SubAccountId: string, // Hubs add-on\r\n x_CommitmentDiscountCommittedCount: decimal, // MS 2023-03-01\r\n x_CommitmentDiscountCommittedAmount: decimal, // MS 2023-03-01\r\n x_CommitmentDiscountNormalizedGroup: string, // MS 2023-03-01\r\n x_CommitmentDiscountNormalizedRatio: decimal, // MS 2023-03-01\r\n x_CommitmentDiscountQuantity: decimal, // MS 2023-03-01\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_ResourceGroupName: string, // Hubs add-on\r\n x_ResourceType: string, // Hubs add-on\r\n x_ServiceModel: string, // Hubs add-on\r\n x_SkuOrderId: string, // MS 2023-03-01\r\n x_SkuSize: string, // MS 2023-03-01\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string // Hubs add-on\r\n)\r\n\r\n// Update policy for CommitmentDiscountUsage_raw -> CommitmentDiscountUsage_final_v1_0 table\r\n.alter table CommitmentDiscountUsage_final_v1_0 policy update\r\n```\r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"CommitmentDiscountUsage_raw\",\r\n \"Query\": \"CommitmentDiscountUsage_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Recommendations |================================================================================================\r\n// Supported datasets/versions:\r\n// - MS CM EA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-ea\r\n// - MS CM MCA reservation recommendations: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-recommendations-mca\r\n//======================================================================================================================\r\n\r\n// Recommendations_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: All recommendations transformed to FOCUS 1.0. Use Recommendations_transform_v1_2() instead.', folder='Recommendations')\r\nRecommendations_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', toint(''),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n Recommendations_raw\r\n //\r\n // Change real to decimal\r\n | extend\r\n CostWithNoReservedInstances = todecimal(CostWithNoReservedInstances),\r\n InstanceFlexibilityRatio = todecimal(InstanceFlexibilityRatio),\r\n NetSavings = todecimal(NetSavings),\r\n RecommendedQuantity = todecimal(RecommendedQuantity),\r\n RecommendedQuantityNormalized = todecimal(RecommendedQuantityNormalized),\r\n TotalCostWithReservedInstances = todecimal(TotalCostWithReservedInstances)\r\n //\r\n | extend x_IngestionTime = ingestion_time()\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Set source columns\r\n | extend x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationRecommendations', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2023-05-01', ''))\r\n //\r\n // Convert JSON cost columns to decimal\r\n | extend CostWithNoReservedInstances = case(isnotempty(CostWithNoReservedInstances), CostWithNoReservedInstances, isnotempty(CostWithNoReservedInstancesJson), todecimal(extract(@'\"value\":([0-9\\.]+)', 1, CostWithNoReservedInstancesJson)), CostWithNoReservedInstances)\r\n | extend NetSavings = case(isnotempty(NetSavings), NetSavings, isnotempty(NetSavingsJson), todecimal(extract(@'\"value\":([0-9\\.]+)', 1, NetSavingsJson)), NetSavings)\r\n | extend TotalCostWithReservedInstances = case(isnotempty(TotalCostWithReservedInstances), TotalCostWithReservedInstances, isnotempty(TotalCostWithReservedInstancesJson), todecimal(extract(@'\"value\":([0-9\\.]+)', 1, TotalCostWithReservedInstancesJson)), TotalCostWithReservedInstances)\r\n //\r\n // Build recommendation details\r\n | lookup kind=leftouter (Regions | summarize RegionName = make_set(RegionName)[0] by Location = RegionId) on Location\r\n | extend x_RecommendationDetails = case(\r\n x_SourceType == 'ReservationRecommendations', bag_pack(\r\n 'CommitmentDiscountNormalizedGroup', InstanceFlexibilityGroup,\r\n 'CommitmentDiscountNormalizedRatio', InstanceFlexibilityRatio,\r\n 'CommitmentDiscountNormalizedSize', NormalizedSize,\r\n 'CommitmentDiscountResourceType', ResourceType,\r\n 'CommitmentDiscountScope', Scope,\r\n 'LookbackPeriodDuration', case(\r\n LookBackPeriod matches regex @'^Last([0-9]+)Days$', replace_regex(LookBackPeriod, @'^Last([0-9]+)Days$', @'P\\1D'),\r\n LookBackPeriod matches regex @'^[0-9]+$', strcat('P', LookBackPeriod, 'D'),\r\n ''\r\n ),\r\n 'LookbackPeriodStart', FirstUsageDate,\r\n 'RecommendedQuantity', RecommendedQuantity,\r\n 'RecommendedQuantityNormalized', RecommendedQuantityNormalized,\r\n 'RegionId', Location,\r\n 'RegionName', RegionName,\r\n 'SkuMeterId', MeterId,\r\n 'SkuPriceDetails', SkuProperties,\r\n 'SkuSize', coalesce(SKU, SkuName),\r\n 'SkuTerm', isoMonths(Term)\r\n ),\r\n dynamic({})\r\n )\r\n //\r\n // Sort columns and apply final transforms\r\n | extend x_RecommendationDate = FirstUsageDate + (toint(extract(@'^P([0-9]+)D$', 1, tostring(x_RecommendationDetails.LookbackPeriodDuration))) * 1d)\r\n | extend x_RecommendationDate = iff(x_RecommendationDate > now(), startofday(now()), x_RecommendationDate)\r\n | project\r\n ProviderName,\r\n SubAccountId = iff(isnotempty(SubscriptionId), strcat('/subscriptions/', SubscriptionId), ''),\r\n x_IngestionTime,\r\n x_EffectiveCostAfter = TotalCostWithReservedInstances,\r\n x_EffectiveCostBefore = CostWithNoReservedInstances,\r\n x_EffectiveCostSavings = NetSavings,\r\n x_RecommendationDate = FirstUsageDate + (toint(extract(@'^P([0-9]+)D$', 1, tostring(x_RecommendationDetails.LookbackPeriodDuration))) * 1d),\r\n x_RecommendationDetails,\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion\r\n}\r\n\r\n// Recommendations_final_v1_0 table\r\n.create-merge table Recommendations_final_v1_0 (\r\n ProviderName: string,\r\n SubAccountId: string,\r\n x_IngestionTime: datetime,\r\n x_EffectiveCostAfter: decimal,\r\n x_EffectiveCostBefore: decimal,\r\n x_EffectiveCostSavings: decimal,\r\n x_RecommendationDate: datetime,\r\n x_RecommendationDetails: dynamic,\r\n x_SourceName: string,\r\n x_SourceProvider: string,\r\n x_SourceType: string,\r\n x_SourceVersion: string\r\n)\r\n\r\n// Update policy for Recommendations_raw -> Recommendations_final_v1_0 table\r\n.alter table Recommendations_final_v1_0 policy update\r\n```\r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"Recommendations_raw\",\r\n \"Query\": \"Recommendations_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n\r\n\r\n//===| Transactions |===================================================================================================\r\n// Supported versions:\r\n// - MS CM EA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-ea\r\n// - MS CM MCA reservation transactions: 2023-05-01 -- See https://learn.microsoft.com/azure/cost-management-billing/dataset-schema/reservation-transactions-mca\r\n//======================================================================================================================\r\n\r\n// Transactions_transform_v1_0 function\r\n.create-or-alter function\r\nwith (docstring='DEPRECATED: All transactions transformed to FOCUS 1.0. Use Transactions_transform_v1_2() instead.', folder='Transactions')\r\nTransactions_transform_v1_0()\r\n{\r\n // NOTE: All open issues and questions are tracked @ https://github.com/microsoft/finops-toolkit/issues/1111\r\n let isoMonths = (duration: string) {\r\n let number = toint(replace_regex(duration, @'[PMY]', ''));\r\n toint(case(\r\n duration == '', toint(''),\r\n duration endswith \"Y\", number * 12,\r\n duration endswith \"M\", number,\r\n -1\r\n ))\r\n };\r\n Transactions_raw\r\n //\r\n // Change real to decimal\r\n | extend\r\n Amount = todecimal(Amount),\r\n MonetaryCommitment = todecimal(MonetaryCommitment),\r\n Overage = todecimal(Overage),\r\n Quantity = todecimal(Quantity)\r\n //\r\n // Set ProviderName\r\n | extend ProviderName = 'Microsoft'\r\n //\r\n // Set source columns\r\n | extend x_SourceName = coalesce(x_SourceName, iff(ProviderName == 'Microsoft', 'Cost Management', ProviderName))\r\n | extend x_SourceProvider = coalesce(x_SourceProvider, ProviderName)\r\n | extend x_SourceType = coalesce(x_SourceType, iff(ProviderName == 'Microsoft', 'ReservationTransactions', ''))\r\n | extend x_SourceVersion = coalesce(x_SourceVersion, iff(ProviderName == 'Microsoft', '2023-05-01', ''))\r\n //\r\n // Handle BillingPeriodStart/End\r\n | extend BillingMonth = tostring(BillingMonth)\r\n | extend BillingPeriodStart = iff(isempty(BillingMonth), datetime(null), todatetime(strcat(substring(BillingMonth, 0, 4), \"-\", substring(BillingMonth, 4, 2), \"-\", substring(BillingMonth, 6, 2))))\r\n | extend BillingPeriodEnd = iff(isempty(BillingMonth), datetime(null), startofmonth(endofmonth(BillingPeriodStart) + 1d))\r\n //\r\n // Sort columns and apply final transforms\r\n | project\r\n BilledCost = Amount,\r\n BillingAccountId = case(\r\n BillingProfileId startswith '/', BillingProfileId,\r\n isnotempty(CurrentEnrollmentId), strcat('/providers/Microsoft.Billing/billingAccounts/', CurrentEnrollmentId),\r\n isnotempty(BillingProfileId), strcat('/providers/Microsoft.Billing/billingProfiles/', BillingProfileId),\r\n ''\r\n ),\r\n BillingAccountName = coalesce(BillingProfileName, CurrentEnrollmentId),\r\n BillingCurrency = Currency,\r\n BillingPeriodEnd,\r\n BillingPeriodStart,\r\n ChargeCategory = case(\r\n EventType in ('Cancel', 'Purchase', 'Refund'), 'Purchase',\r\n 'Adjustment'\r\n ),\r\n ChargeClass = case(\r\n EventType == 'Cancel', 'Cancel', // FOCUS does not handle this scenario\r\n EventType == 'Refund', 'Correction',\r\n ''\r\n ),\r\n ChargeDescription = Description,\r\n ChargeFrequency = case(\r\n BillingFrequency == 'OneTime', 'One-Time',\r\n BillingFrequency == 'Recurring', 'Recurring',\r\n BillingFrequency\r\n ),\r\n ChargePeriodStart = EventDate,\r\n PricingQuantity = Quantity,\r\n PricingUnit = 'Reservations',\r\n ProviderName,\r\n RegionId = Region,\r\n RegionName = Region,\r\n SubAccountId = iff(isempty(PurchasingSubscriptionGuid), '', strcat('/subscriptions/', PurchasingSubscriptionGuid)),\r\n SubAccountName = iff(isempty(PurchasingSubscriptionGuid), '', PurchasingSubscriptionName),\r\n x_AccountName = AccountName,\r\n x_AccountOwnerId = AccountOwnerEmail,\r\n x_CostCenter = CostCenter,\r\n x_InvoiceId = InvoiceId,\r\n x_InvoiceNumber = Invoice,\r\n x_InvoiceSectionId = InvoiceSectionId,\r\n x_InvoiceSectionName = coalesce(InvoiceSectionName, DepartmentName),\r\n x_IngestionTime = ingestion_time(),\r\n x_MonetaryCommitment = MonetaryCommitment,\r\n x_Overage = Overage,\r\n x_PurchasingBillingAccountId = PurchasingEnrollment,\r\n x_SkuOrderId = ReservationOrderId,\r\n x_SkuOrderName = ReservationOrderName,\r\n x_SkuSize = ArmSkuName,\r\n x_SkuTerm = isoMonths(Term),\r\n x_SourceName,\r\n x_SourceProvider,\r\n x_SourceType,\r\n x_SourceVersion,\r\n x_SubscriptionId = PurchasingSubscriptionGuid,\r\n x_TransactionType = EventType\r\n}\r\n\r\n// Transactions_final_v1_0 table\r\n.create-merge table Transactions_final_v1_0 (\r\n BilledCost: decimal, // MS CM EA+MCA 2023-05-01\r\n BillingAccountId: string, // MS CM EA+MCA 2023-05-01\r\n BillingAccountName: string, // MS CM EA+MCA 2023-05-01\r\n BillingCurrency: string, // MS CM EA+MCA 2023-05-01\r\n BillingPeriodEnd: datetime, // MS CM EA+MCA 2023-05-01\r\n BillingPeriodStart: datetime, // MS CM EA+MCA 2023-05-01\r\n ChargeCategory: string, // Hubs add-on\r\n ChargeClass: string, // Hubs add-on\r\n ChargeDescription: string, // MS CM EA+MCA 2023-05-01\r\n ChargeFrequency: string, // MS CM EA+MCA 2023-05-01\r\n ChargePeriodStart: datetime, // MS CM EA+MCA 2023-05-01\r\n PricingQuantity: decimal, // MS CM EA+MCA 2023-05-01\r\n PricingUnit: string, // Hubs add-on\r\n ProviderName: string, // Hubs add-on\r\n RegionId: string, // MS CM EA+MCA 2023-05-01\r\n RegionName: string, // MS CM EA+MCA 2023-05-01\r\n SubAccountId: string, // MS CM EA+MCA 2023-05-01\r\n SubAccountName: string, // MS CM EA+MCA 2023-05-01\r\n x_AccountName: string, // MS CM EA 2023-05-01\r\n x_AccountOwnerId: string, // MS CM EA 2023-05-01\r\n x_CostCenter: string, // MS CM EA 2023-05-01\r\n x_InvoiceId: string, // MS CM MCA 2023-05-01\r\n x_InvoiceNumber: string, // MS CM MCA 2023-05-01\r\n x_InvoiceSectionId: string, // MS CM MCA 2023-05-01\r\n x_InvoiceSectionName: string, // MS CM MCA 2023-05-01\r\n x_IngestionTime: datetime, // Hubs add-on\r\n x_MonetaryCommitment: decimal, // MS CM EA 2023-05-01\r\n x_Overage: decimal, // MS CM EA 2023-05-01\r\n x_PurchasingBillingAccountId: string, // MS CM EA 2023-05-01\r\n x_SkuOrderId: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuOrderName: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuSize: string, // MS CM EA+MCA 2023-05-01\r\n x_SkuTerm: int, // MS CM EA+MCA 2023-05-01\r\n x_SourceName: string, // Hubs add-on\r\n x_SourceProvider: string, // Hubs add-on\r\n x_SourceType: string, // Hubs add-on\r\n x_SourceVersion: string, // Hubs add-on\r\n x_SubscriptionId: string, // MS CM EA+MCA 2023-05-01\r\n x_TransactionType: string // MS CM EA+MCA 2023-05-01\r\n)\r\n\r\n// Update policy for Transactions_raw -> Transactions_final_v1_0 table\r\n.alter table Transactions_final_v1_0 policy update\r\n```\r\n[{\r\n \"IsEnabled\": false,\r\n \"Source\": \"Transactions_raw\",\r\n \"Query\": \"Transactions_transform_v1_0()\",\r\n \"IsTransactional\": true,\r\n \"PropagateIngestionProperties\": true\r\n}]\r\n```\r\n", + "CONFIG": "config", + "HUB_DATA_EXPLORER": "hubDataExplorer", + "HUB_DB": "Hub", + "INGESTION": "ingestion", + "INGESTION_DB": "Ingestion", + "INGESTION_ID_SEPARATOR": "__", + "ftkGitTag": "13", + "ftkReleaseUri": "[if(endsWith(variables('finOpsToolkitVersion'), '-dev'), 'https://raw.githubusercontent.com/microsoft/finops-toolkit/refs/heads/dev/src/open-data', format('https://raw.githubusercontent.com/microsoft/finops-toolkit/refs/tags/v{0}/src/open-data', variables('ftkGitTag')))]", + "useFabric": "[not(empty(parameters('fabricQueryUri')))]", + "useAzure": "[and(not(variables('useFabric')), not(empty(parameters('clusterName'))))]", + "dataExplorerPrivateDnsZoneName": "[replace(format('privatelink.{0}.{1}', parameters('app').hub.location, replace(environment().suffixes.storage, 'core', 'kusto')), '..', '.')]", + "ingestionCapacity": { + "Dev(No SLA)_Standard_E2a_v4": 1, + "Dev(No SLA)_Standard_D11_v2": 1, + "Standard_D11_v2": 2, + "Standard_D12_v2": 4, + "Standard_D13_v2": 8, + "Standard_D14_v2": 16, + "Standard_D16d_v5": 16, + "Standard_D32d_v4": 32, + "Standard_D32d_v5": 32, + "Standard_DS13_v2+1TB_PS": 8, + "Standard_DS13_v2+2TB_PS": 8, + "Standard_DS14_v2+3TB_PS": 16, + "Standard_DS14_v2+4TB_PS": 16, + "Standard_E2a_v4": 2, + "Standard_E2ads_v5": 2, + "Standard_E2d_v4": 2, + "Standard_E2d_v5": 2, + "Standard_E4a_v4": 4, + "Standard_E4ads_v5": 4, + "Standard_E4d_v4": 4, + "Standard_E4d_v5": 4, + "Standard_E8a_v4": 8, + "Standard_E8ads_v5": 8, + "Standard_E8as_v4+1TB_PS": 8, + "Standard_E8as_v4+2TB_PS": 8, + "Standard_E8as_v5+1TB_PS": 8, + "Standard_E8as_v5+2TB_PS": 8, + "Standard_E8d_v4": 8, + "Standard_E8d_v5": 8, + "Standard_E8s_v4+1TB_PS": 8, + "Standard_E8s_v4+2TB_PS": 8, + "Standard_E8s_v5+1TB_PS": 8, + "Standard_E8s_v5+2TB_PS": 8, + "Standard_E16a_v4": 16, + "Standard_E16ads_v5": 16, + "Standard_E16as_v4+3TB_PS": 16, + "Standard_E16as_v4+4TB_PS": 16, + "Standard_E16as_v5+3TB_PS": 16, + "Standard_E16as_v5+4TB_PS": 16, + "Standard_E16d_v4": 16, + "Standard_E16d_v5": 16, + "Standard_E16s_v4+3TB_PS": 16, + "Standard_E16s_v4+4TB_PS": 16, + "Standard_E16s_v5+3TB_PS": 16, + "Standard_E16s_v5+4TB_PS": 16, + "Standard_E64i_v3": 64, + "Standard_E80ids_v4": 80, + "Standard_EC8ads_v5": 8, + "Standard_EC8as_v5+1TB_PS": 8, + "Standard_EC8as_v5+2TB_PS": 8, + "Standard_EC16ads_v5": 16, + "Standard_EC16as_v5+3TB_PS": 16, + "Standard_EC16as_v5+4TB_PS": 16, + "Standard_L4s": 4, + "Standard_L8as_v3": 8, + "Standard_L8s": 8, + "Standard_L8s_v2": 8, + "Standard_L8s_v3": 8, + "Standard_L16as_v3": 16, + "Standard_L16s": 16, + "Standard_L16s_v2": 16, + "Standard_L16s_v3": 16, + "Standard_L32as_v3": 32, + "Standard_L32s_v3": 32 + }, + "dataExplorerIngestionCapacity": "[if(variables('useFabric'), parameters('fabricCapacityUnits'), if(not(variables('useAzure')), 1, coalesce(tryGet(variables('ingestionCapacity'), parameters('clusterSku')), 1)))]", + "dataExplorerUri": "[if(variables('useFabric'), parameters('fabricQueryUri'), format('https://{0}.{1}.kusto.windows.net', replace(parameters('clusterName'), '_', '-'), parameters('app').hub.location))]", + "finOpsToolkitVersion": "13.0" + }, + "resources": { + "cluster::adfClusterAdmin": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Kusto/clusters/principalAssignments", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}', replace(parameters('clusterName'), '_', '-'), 'adf-mi-cluster-admin')]", + "properties": { + "principalType": "App", + "principalId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "tenantId": "[reference('dataFactory', '2018-06-01', 'full').identity.tenantId]", + "role": "AllDatabasesAdmin" }, "dependsOn": [ - "dataset_config", - "pipeline_RunBackfillJob" - ], - "metadata": { - "description": "Runs the backfill job for each month based on retention settings." - } + "cluster", + "dataFactory" + ] + }, + "cluster::ingestionDb": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Kusto/clusters/databases", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}', replace(parameters('clusterName'), '_', '-'), variables('INGESTION_DB'))]", + "location": "[parameters('app').hub.location]", + "kind": "ReadWrite", + "dependsOn": [ + "cluster" + ] + }, + "cluster::hubDb": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Kusto/clusters/databases", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}', replace(parameters('clusterName'), '_', '-'), variables('HUB_DB'))]", + "location": "[parameters('app').hub.location]", + "kind": "ReadWrite", + "dependsOn": [ + "cluster" + ] }, - "pipeline_RunBackfillJob": { - "condition": "[parameters('enableManagedExports')]", - "type": "Microsoft.DataFactory/factories/pipelines", + "dataFactoryVNet::dataExplorerManagedPrivateEndpoint": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), format('{0}_RunBackfillJob', variables('safeConfigContainerName')))]", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', variables('HUB_DATA_EXPLORER'))]", "properties": { - "activities": [ - { - "name": "Get Config", - "type": "Lookup", - "dependsOn": [], - "policy": { - "timeout": "0.00:05:00", - "retry": 2, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "source": { - "type": "JsonSource", - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": true, - "enablePartitionDiscovery": false - }, - "formatSettings": { - "type": "JsonReadSettings" - } - }, - "dataset": { - "referenceName": "[variables('safeConfigContainerName')]", - "type": "DatasetReference", - "parameters": { - "fileName": { - "value": "@variables('fileName')", - "type": "Expression" - }, - "folderPath": { - "value": "@variables('folderPath')", - "type": "Expression" - } - } - } - } - }, - { - "name": "Set Scopes", - "description": "Save scopes to test if it is an array", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Get Config", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "scopesArray", - "value": { - "value": "@activity('Get Config').output.firstRow.scopes", - "type": "Expression" - } - } - }, - { - "name": "Set Scopes as Array", - "description": "Wraps a single scope object into an array to work around the PowerShell bug where single-item arrays are sometimes written as a single object instead of an array.", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Set Scopes", - "dependencyConditions": [ - "Failed" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "scopesArray", - "value": { - "value": "@createArray(activity('Get Config').output.firstRow.scopes)", - "type": "Expression" - } - } - }, - { - "name": "Filter Invalid Scopes", - "description": "Remove any invalid scopes to avoid errors.", - "type": "Filter", - "dependsOn": [ - { - "activity": "Set Scopes", - "dependencyConditions": [ - "Succeeded" - ] - }, - { - "activity": "Set Scopes as Array", - "dependencyConditions": [ - "Skipped", - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "items": { - "value": "@variables('scopesArray')", - "type": "Expression" - }, - "condition": { - "value": "@and(not(empty(item().scope)), not(equals(item().scope, '/')))", - "type": "Expression" - } - } - }, - { - "name": "ForEach Export Scope", - "type": "ForEach", - "dependsOn": [ - { - "activity": "Filter Invalid Scopes", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "items": { - "value": "@activity('Filter Invalid Scopes').output.Value", - "type": "Expression" - }, - "isSequential": true, - "activities": [ - { - "name": "Set backfill export name", - "type": "SetVariable", - "dependsOn": [], - "userProperties": [], - "typeProperties": { - "variableName": "exportName", - "value": { - "value": "@toLower(concat(variables('finOpsHub'), '-monthly-costdetails'))", - "type": "Expression" - } - } - }, - { - "name": "Trigger backfill export", - "type": "WebActivity", - "dependsOn": [ - { - "activity": "Set backfill export name", - "dependencyConditions": [ - "Completed" - ] - } - ], - "policy": { - "timeout": "0.00:05:00", - "retry": 1, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "url": { - "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{variables(''exportName'')}}/run?api-version={0}', variables('exportApiVersion'))]", - "type": "Expression" - }, - "method": "POST", - "headers": { - "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunBackfill@{0}', variables('ftkVersion'))]", - "Content-Type": "application/json", - "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('ftkVersion'))]" - }, - "body": "{\"timePeriod\" : { \"from\" : \"@{pipeline().parameters.StartDate}\", \"to\" : \"@{pipeline().parameters.EndDate}\" }}", - "authentication": { - "type": "MSI", - "resource": { - "value": "@variables('resourceManagementUri')", - "type": "Expression" - } - } - } - } - ] - } - } - ], - "concurrency": 1, - "parameters": { - "StartDate": { - "type": "string" - }, - "EndDate": { - "type": "string" - } - }, - "variables": { - "exportName": { - "type": "String" - }, - "storageAccountId": { - "type": "String", - "defaultValue": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]" - }, - "finOpsHub": { - "type": "String", - "defaultValue": "[parameters('hubName')]" - }, - "resourceManagementUri": { - "type": "String", - "defaultValue": "[environment().resourceManager]" - }, - "fileName": { - "type": "String", - "defaultValue": "settings.json" - }, - "folderPath": { - "type": "String", - "defaultValue": "[parameters('configContainerName')]" - }, - "scopesArray": { - "type": "Array" - } - } + "name": "[variables('HUB_DATA_EXPLORER')]", + "groupId": "cluster", + "privateLinkResourceId": "[resourceId('Microsoft.Kusto/clusters', replace(parameters('clusterName'), '_', '-'))]", + "fqdns": [ + "[format('https://{0}.{1}.kusto.windows.net', replace(parameters('clusterName'), '_', '-'), parameters('app').hub.location)]" + ] + }, + "dependsOn": [ + "appRegistration", + "cluster" + ] + }, + "dataFactory": { + "existing": true, + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "dependsOn": [ + "appRegistration" + ] + }, + "blobPrivateDnsZone": { + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]", + "dependsOn": [ + "appRegistration" + ] + }, + "queuePrivateDnsZone": { + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.queue.{0}', environment().suffixes.storage)]", + "dependsOn": [ + "appRegistration" + ] + }, + "tablePrivateDnsZone": { + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.table.{0}', environment().suffixes.storage)]", + "dependsOn": [ + "appRegistration" + ] + }, + "storage": { + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]", + "dependsOn": [ + "appRegistration" + ] + }, + "cluster": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Kusto/clusters", + "apiVersion": "2023-08-15", + "name": "[replace(parameters('clusterName'), '_', '-')]", + "location": "[parameters('app').hub.location]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Kusto/clusters'), createObject()))]", + "sku": { + "name": "[parameters('clusterSku')]", + "tier": "[if(startsWith(parameters('clusterSku'), 'Dev(No SLA)_'), 'Basic', 'Standard')]", + "capacity": "[if(startsWith(parameters('clusterSku'), 'Dev(No SLA)_'), 1, if(equals(parameters('clusterCapacity'), 1), 2, parameters('clusterCapacity')))]" + }, + "identity": { + "type": "SystemAssigned" + }, + "properties": { + "enableStreamingIngest": true, + "enableAutoStop": false, + "publicNetworkAccess": "[if(parameters('app').hub.options.privateRouting, 'Disabled', 'Enabled')]" }, "dependsOn": [ - "dataset_config" - ], - "metadata": { - "description": "Creates and triggers exports for all defined scopes for the specified date range." - } + "appRegistration" + ] }, - "pipeline_StartExportProcess": { - "condition": "[parameters('enableManagedExports')]", - "type": "Microsoft.DataFactory/factories/pipelines", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), format('{0}_StartExportProcess', variables('safeConfigContainerName')))]", + "clusterStorageAccess": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "name": "[guid(replace(parameters('clusterName'), '_', '-'), subscription().id, 'Storage Blob Data Contributor')]", "properties": { - "activities": [ - { - "name": "Get Config", - "type": "Lookup", - "dependsOn": [], - "policy": { - "timeout": "0.00:05:00", - "retry": 2, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "source": { - "type": "JsonSource", - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": true, - "enablePartitionDiscovery": false - }, - "formatSettings": { - "type": "JsonReadSettings" - } - }, - "dataset": { - "referenceName": "[variables('safeConfigContainerName')]", - "type": "DatasetReference", - "parameters": { - "fileName": { - "value": "@variables('fileName')", - "type": "Expression" - }, - "folderPath": { - "value": "@variables('folderPath')", - "type": "Expression" - } - } - } - } - }, - { - "name": "Set Scopes", - "description": "Save scopes to test if it is an array", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Get Config", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "scopesArray", - "value": { - "value": "@activity('Get Config').output.firstRow.scopes", - "type": "Expression" - } - } - }, - { - "name": "Set Scopes as Array", - "description": "Wraps a single scope object into an array to work around the PowerShell bug where single-item arrays are sometimes written as a single object instead of an array.", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Set Scopes", - "dependencyConditions": [ - "Failed" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "scopesArray", - "value": { - "value": "@createArray(activity('Get Config').output.firstRow.scopes)", - "type": "Expression" - } - } - }, - { - "name": "Filter Invalid Scopes", - "description": "Remove any invalid scopes to avoid errors.", - "type": "Filter", - "dependsOn": [ - { - "activity": "Set Scopes", - "dependencyConditions": [ - "Succeeded" - ] - }, - { - "activity": "Set Scopes as Array", - "dependencyConditions": [ - "Succeeded", - "Skipped" - ] - } - ], - "userProperties": [], - "typeProperties": { - "items": { - "value": "@variables('scopesArray')", - "type": "Expression" - }, - "condition": { - "value": "@and(not(empty(item().scope)), not(equals(item().scope, '/')))", - "type": "Expression" - } - } - }, - { - "name": "ForEach Export Scope", - "type": "ForEach", - "dependsOn": [ - { - "activity": "Filter Invalid Scopes", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "items": { - "value": "@activity('Filter Invalid Scopes').output.Value", - "type": "Expression" - }, - "isSequential": true, - "activities": [ - { - "name": "Get exports for scope", - "type": "WebActivity", - "dependsOn": [], - "policy": { - "timeout": "0.00:05:00", - "retry": 2, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "url": { - "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports?api-version={0}', variables('exportApiVersion'))]", - "type": "Expression" - }, - "method": "GET", - "authentication": { - "type": "MSI", - "resource": { - "value": "@variables('resourceManagementUri')", - "type": "Expression" - } - } - } - }, - { - "name": "Run exports for scope", - "type": "ExecutePipeline", - "dependsOn": [ - { - "activity": "Get exports for scope", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "pipeline": { - "referenceName": "[format('{0}_RunExportJobs', variables('safeConfigContainerName'))]", - "type": "PipelineReference" - }, - "waitOnCompletion": true, - "parameters": { - "ExportScopes": { - "value": "@activity('Get exports for scope').output.value", - "type": "Expression" - }, - "Recurrence": { - "value": "@pipeline().parameters.Recurrence", - "type": "Expression" - } - } - } - } + "description": "Give \"Storage Blob Data Contributor\" to the cluster", + "principalId": "[reference('cluster', '2023-08-15', 'full').identity.principalId]", + "principalType": "ServicePrincipal", + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe')]" + }, + "dependsOn": [ + "appRegistration", + "cluster" + ] + }, + "dataExplorerPrivateDnsZone": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[variables('dataExplorerPrivateDnsZoneName')]", + "location": "global", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Network/privateDnsZones'), createObject()))]", + "properties": {} + }, + "dataExplorerPrivateDnsZoneLink": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', variables('dataExplorerPrivateDnsZoneName'), format('{0}-link', replace(variables('dataExplorerPrivateDnsZoneName'), '.', '-')))]", + "location": "global", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Network/privateDnsZones/virtualNetworkLinks'), createObject()))]", + "properties": { + "virtualNetwork": { + "id": "[parameters('app').hub.routing.networkId]" + }, + "registrationEnabled": false + }, + "dependsOn": [ + "dataExplorerPrivateDnsZone" + ] + }, + "dataExplorerEndpoint": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-ep', replace(parameters('clusterName'), '_', '-'))]", + "location": "[parameters('app').hub.location]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Network/privateEndpoints'), createObject()))]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.dataExplorer]" + }, + "privateLinkServiceConnections": [ + { + "name": "dataExplorerLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Kusto/clusters', replace(parameters('clusterName'), '_', '-'))]", + "groupIds": [ + "cluster" ] } } - ], - "concurrency": 1, - "parameters": { - "Recurrence": { - "type": "string", - "defaultValue": "Daily" - } - }, - "variables": { - "fileName": { - "type": "String", - "defaultValue": "settings.json" + ] + }, + "dependsOn": [ + "cluster" + ] + }, + "dataExplorerPrivateDnsZoneGroup": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-ep', replace(parameters('clusterName'), '_', '-')), 'dataExplorer-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "privatelink-westus-kusto-net", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', variables('dataExplorerPrivateDnsZoneName'))]" + } }, - "folderPath": { - "type": "String", - "defaultValue": "[parameters('configContainerName')]" + { + "name": "privatelink-blob-core-windows-net", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" + } }, - "finOpsHub": { - "type": "String", - "defaultValue": "[parameters('hubName')]" + { + "name": "privatelink-table-core-windows-net", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.table.{0}', environment().suffixes.storage))]" + } }, - "resourceManagementUri": { + { + "name": "privatelink-queue-core-windows-net", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.queue.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "appRegistration", + "dataExplorerEndpoint", + "dataExplorerPrivateDnsZone" + ] + }, + "dataFactoryVNet": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'default')]", + "dependsOn": [ + "appRegistration" + ] + }, + "linkedService_dataExplorer": { + "condition": "[or(variables('useAzure'), variables('useFabric'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, variables('HUB_DATA_EXPLORER'))]", + "properties": "[shallowMerge(createArray(createObject('type', 'AzureDataExplorer', 'parameters', createObject('database', createObject('type', 'String', 'defaultValue', variables('INGESTION_DB'))), 'typeProperties', createObject('endpoint', variables('dataExplorerUri'), 'database', '@{linkedService().database}', 'tenant', reference('dataFactory', '2018-06-01', 'full').identity.tenantId, 'servicePrincipalId', reference('dataFactory', '2018-06-01', 'full').identity.principalId)), __bicep.privateRoutingForLinkedServices(parameters('app').hub)))]", + "dependsOn": [ + "appRegistration", + "cluster", + "dataFactory" + ] + }, + "linkedService_ftkRepo": { + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ftkRepo')]", + "properties": "[shallowMerge(createArray(createObject('type', 'HttpServer', 'parameters', createObject('filePath', createObject('type', 'string')), 'typeProperties', createObject('url', '@concat(''https://gitapp.hub.com/microsoft/finops-toolkit/'', linkedService().filePath)', 'enableServerCertificateValidation', true(), 'authenticationType', 'Anonymous')), __bicep.privateRoutingForLinkedServices(parameters('app').hub)))]", + "dependsOn": [ + "appRegistration" + ] + }, + "dataset_dataExplorer": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, variables('HUB_DATA_EXPLORER'))]", + "properties": { + "type": "AzureDataExplorerTable", + "linkedServiceName": { + "parameters": { + "database": "@dataset().database" + }, + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference" + }, + "parameters": { + "database": { "type": "String", - "defaultValue": "[environment().resourceManager]" + "defaultValue": "[variables('INGESTION_DB')]" }, - "scopesArray": { - "type": "Array" + "table": { + "type": "String" + } + }, + "typeProperties": { + "table": { + "value": "@dataset().table", + "type": "Expression" } } }, "dependsOn": [ - "dataset_config", - "pipeline_RunExportJobs" - ], - "metadata": { - "description": "Gets a list of all Cost Management exports configured for this hub based on the scopes defined in settings.json, then runs each export using the config_RunExportJobs pipeline." - } + "appRegistration", + "linkedService_dataExplorer" + ] }, - "pipeline_RunExportJobs": { - "condition": "[parameters('enableManagedExports')]", - "type": "Microsoft.DataFactory/factories/pipelines", + "dataset_ftkReleaseFile": { + "type": "Microsoft.DataFactory/factories/datasets", "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), format('{0}_RunExportJobs', variables('safeConfigContainerName')))]", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ftkReleaseFile')]", "properties": { - "activities": [ - { - "name": "ForEach export scope", - "type": "ForEach", - "dependsOn": [], - "userProperties": [], - "typeProperties": { - "items": { - "value": "@pipeline().parameters.exportScopes", - "type": "Expression" - }, - "isSequential": true, - "activities": [ - { - "name": "If scheduled", - "type": "IfCondition", - "dependsOn": [], - "userProperties": [], - "typeProperties": { - "expression": { - "value": "@and( startswith(toLower(item().name), toLower(variables('hubName'))), and(contains(string(item().properties.schedule), 'recurrence'), equals(toLower(item().properties.schedule.recurrence), toLower(pipeline().parameters.Recurrence))))", - "type": "Expression" - }, - "ifTrueActivities": [ - { - "name": "Trigger export", - "type": "WebActivity", - "dependsOn": [], - "policy": { - "timeout": "0.00:05:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "method": "POST", - "url": { - "value": "[format('@{{replace(toLower(concat(variables(''resourceManagementUri''),item().id)), ''com//'', ''com/'')}}/run?api-version={0}', variables('exportApiVersion'))]", - "type": "Expression" - }, - "headers": { - "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs@{0}', variables('ftkVersion'))]", - "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('ftkVersion'))]" - }, - "body": " ", - "authentication": { - "type": "MSI", - "resource": { - "value": "@variables('resourceManagementUri')", - "type": "Expression" - } - } - } - } - ] - } - } - ] - } - } - ], - "concurrency": 1, + "linkedServiceName": { + "referenceName": "ftkRepo", + "type": "LinkedServiceReference" + }, "parameters": { - "ExportScopes": { - "type": "array" + "fileName": { + "type": "string" }, - "Recurrence": { + "version": { "type": "string", - "defaultValue": "Daily" + "defaultValue": "[variables('ftkGitTag')]" } }, - "variables": { - "resourceManagementUri": { - "type": "String", - "defaultValue": "[environment().resourceManager]" + "annotations": [], + "type": "DelimitedText", + "typeProperties": { + "location": { + "type": "HttpServerLocation", + "relativeUrl": { + "value": "@concat('releases/download/v', dataset().version, '/', dataset().fileName)", + "type": "Expression" + } }, - "hubName": { - "type": "String", - "defaultValue": "[parameters('hubName')]" - } - } + "columnDelimiter": ",", + "escapeChar": "\\", + "firstRowAsHeader": true, + "quoteChar": "\"" + }, + "schema": [] }, "dependsOn": [ - "dataset_config" - ], - "metadata": { - "description": "Runs the specified Cost Management exports." - } + "appRegistration", + "linkedService_ftkRepo" + ] }, - "pipeline_ConfigureExports": { - "condition": "[parameters('enableManagedExports')]", + "pipeline_InitializeHub": { "type": "Microsoft.DataFactory/factories/pipelines", "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), format('{0}_ConfigureExports', variables('safeConfigContainerName')))]", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_InitializeHub', variables('CONFIG')))]", "properties": { "activities": [ { @@ -12908,33 +17211,23 @@ "typeProperties": { "source": { "type": "JsonSource", - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": true, - "enablePartitionDiscovery": false - }, - "formatSettings": { - "type": "JsonReadSettings" - } - }, - "dataset": { - "referenceName": "[variables('safeConfigContainerName')]", - "type": "DatasetReference", - "parameters": { - "fileName": { - "value": "@variables('fileName')", - "type": "Expression" - }, - "folderPath": { - "value": "@variables('folderPath')", - "type": "Expression" - } + "storeSettings": { + "type": "AzureBlobFSReadSettings", + "recursive": true, + "enablePartitionDiscovery": false + }, + "formatSettings": { + "type": "JsonReadSettings" } + }, + "dataset": { + "referenceName": "[variables('CONFIG')]", + "type": "DatasetReference" } } }, { - "name": "Save Scopes", + "name": "Set Version", "type": "SetVariable", "dependsOn": [ { @@ -12944,609 +17237,448 @@ ] } ], - "policy": { - "secureOutput": false, - "secureInput": false - }, "userProperties": [], "typeProperties": { - "variableName": "scopesArray", + "variableName": "version", "value": { - "value": "@activity('Get Config').output.firstRow.scopes", + "value": "@activity('Get Config').output.firstRow.version", "type": "Expression" } } }, { - "name": "Save Scopes as Array", + "name": "Set Scopes", "type": "SetVariable", "dependsOn": [ { - "activity": "Save Scopes", + "activity": "Get Config", "dependencyConditions": [ - "Failed" + "Succeeded" ] } ], - "policy": { - "secureOutput": false, - "secureInput": false - }, "userProperties": [], "typeProperties": { - "variableName": "scopesArray", + "variableName": "scopes", "value": { - "value": "@array(activity('Get Config').output.firstRow.scopes)", + "value": "@string(activity('Get Config').output.firstRow.scopes)", "type": "Expression" } } }, { - "name": "Filter Invalid Scopes", - "type": "Filter", + "name": "Set Retention", + "type": "SetVariable", "dependsOn": [ { - "activity": "Save Scopes", - "dependencyConditions": [ - "Succeeded" - ] - }, - { - "activity": "Save Scopes as Array", + "activity": "Get Config", "dependencyConditions": [ - "Skipped", "Succeeded" ] } ], "userProperties": [], "typeProperties": { - "items": { - "value": "@variables('scopesArray')", - "type": "Expression" - }, - "condition": { - "value": "@and(not(empty(item().scope)), not(equals(item().scope, '/')))", + "variableName": "retention", + "value": { + "value": "@string(activity('Get Config').output.firstRow.retention)", "type": "Expression" } } }, { - "name": "ForEach Export Scope", - "type": "ForEach", + "name": "Until Capacity Is Available", + "type": "Until", "dependsOn": [ { - "activity": "Filter Invalid Scopes", + "activity": "Set Version", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set Scopes", + "dependencyConditions": [ + "Succeeded" + ] + }, + { + "activity": "Set Retention", "dependencyConditions": [ "Succeeded" ] } ], - "userProperties": [], - "typeProperties": { - "items": { - "value": "@activity('Filter Invalid Scopes').output.value", - "type": "Expression" - }, - "isSequential": true, - "activities": [ - { - "name": "Set Export Type", - "type": "SetVariable", - "dependsOn": [], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "exportScopeType", - "value": { - "value": "@if(contains(toLower(item().scope), 'providers/microsoft.billing/billingaccounts'), if(contains(toLower(item().scope), ':'), 'mca', 'ea'), if(contains(toLower(item().scope), 'subscriptions/'), 'subscription', 'undefined'))", - "type": "Expression" - } - } - }, - { - "name": "Switch Export Type", - "type": "Switch", - "dependsOn": [ - { - "activity": "Set Export Type", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "on": { - "value": "@toLower(variables('exportScopeType'))", - "type": "Expression" - }, - "cases": [ - { - "value": "ea", - "activities": [ - { - "name": "EA open month focus export", - "type": "WebActivity", - "dependsOn": [], - "policy": { - "timeout": "0.00:05:00", - "retry": 2, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "url": { - "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-costdetails''))}}?api-version={0}', variables('exportApiVersion'))]", - "type": "Expression" - }, - "method": "PUT", - "body": { - "value": "[__bicep.getExportBodyV2(parameters('exportContainerName'), 'FocusCost', variables('focusSchemaVersion'), false(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", - "type": "Expression" - }, - "headers": { - "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsDaily@{0}', variables('ftkVersion'))]", - "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('ftkVersion'))]" - }, - "authentication": { - "type": "MSI", - "resource": { - "value": "@variables('resourceManagementUri')", - "type": "Expression" - } - } - } - }, - { - "name": "EA closed month focus export", - "type": "WebActivity", - "dependsOn": [ - { - "activity": "EA open month focus export", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "timeout": "0.00:05:00", - "retry": 2, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "url": { - "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-monthly-costdetails''))}}?api-version={0}', variables('exportApiVersion'))]", - "type": "Expression" - }, - "method": "PUT", - "body": { - "value": "[__bicep.getExportBodyV2(parameters('exportContainerName'), 'FocusCost', variables('focusSchemaVersion'), true(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", - "type": "Expression" - }, - "headers": { - "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsMonthly@{0}', variables('ftkVersion'))]", - "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('ftkVersion'))]" - }, - "authentication": { - "type": "MSI", - "resource": { - "value": "@variables('resourceManagementUri')", - "type": "Expression" - } - } - } - }, - { - "name": "EA monthly pricesheet export", - "type": "WebActivity", - "dependsOn": [ - { - "activity": "EA closed month focus export", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "timeout": "0.00:05:00", - "retry": 2, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "url": { - "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-monthly-pricesheet''))}}?api-version={0}', variables('exportApiVersion'))]", - "type": "Expression" - }, - "method": "PUT", - "body": { - "value": "[__bicep.getExportBodyV2(parameters('exportContainerName'), 'Pricesheet', variables('exportSchemaVersion'), true(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", - "type": "Expression" - }, - "headers": { - "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.Prices@{0}', variables('ftkVersion'))]", - "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('ftkVersion'))]" - }, - "authentication": { - "type": "MSI", - "resource": { - "value": "@variables('resourceManagementUri')", - "type": "Expression" - } - } - } - }, - { - "name": "Trigger EA monthly pricesheet export", - "type": "WebActivity", - "dependsOn": [ - { - "activity": "EA monthly pricesheet export", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "timeout": "0.00:05:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "method": "POST", - "url": { - "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-monthly-pricesheet''))}}/run?api-version={0}', variables('exportApiVersion'))]", - "type": "Expression" - }, - "headers": { - "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.Prices@{0}', variables('ftkVersion'))]", - "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('ftkVersion'))]" - }, - "body": " ", - "authentication": { - "type": "MSI", - "resource": { - "value": "@variables('resourceManagementUri')", - "type": "Expression" - } - } - } - }, + "userProperties": [], + "typeProperties": { + "expression": { + "value": "@equals(variables('tryAgain'), false)", + "type": "Expression" + }, + "activities": [ + { + "name": "Confirm Ingestion Capacity", + "type": "AzureDataExplorerCommand", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": ".show capacity | where Resource == 'Ingestions' | project Remaining", + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } + }, + { + "name": "If Has Capacity", + "type": "IfCondition", + "dependsOn": [ + { + "activity": "Confirm Ingestion Capacity", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "expression": { + "value": "@or(equals(activity('Confirm Ingestion Capacity').output.count, 0), greater(activity('Confirm Ingestion Capacity').output.value[0].Remaining, 0))", + "type": "Expression" + }, + "ifFalseActivities": [ + { + "name": "Wait for Ingestion", + "type": "Wait", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "waitTimeInSeconds": 15 + } + }, + { + "name": "Try Again", + "type": "SetVariable", + "dependsOn": [ { - "name": "EA daily reservation details export", - "type": "WebActivity", - "dependsOn": [ - { - "activity": "EA monthly pricesheet export", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "timeout": "0.00:05:00", - "retry": 2, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "url": { - "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-reservationdetails''))}}?api-version={0}', variables('exportApiVersion'))]", - "type": "Expression" - }, - "method": "PUT", - "body": { - "value": "[__bicep.getExportBodyV2(parameters('exportContainerName'), 'ReservationDetails', variables('reservationDetailsSchemaVersion'), false(), 'CSV', 'None', 'true', 'CreateNewReport', '', '', '')]", - "type": "Expression" - }, - "headers": { - "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.ReservationDetails@{0}', variables('ftkVersion'))]", - "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('ftkVersion'))]" - }, - "authentication": { - "type": "MSI", - "resource": { - "value": "@variables('resourceManagementUri')", - "type": "Expression" - } - } - } + "activity": "Wait for Ingestion", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "tryAgain", + "value": true + } + } + ], + "ifTrueActivities": [ + { + "name": "Set ingestion policy in ADX", + "type": "AzureDataExplorerCommand", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": { + "value": "[if(variables('useFabric'), format('.show database {0} policy managed_identity', variables('INGESTION_DB')), format('.alter-merge database {0} policy managed_identity \"[ {{ ''ObjectId'' : ''{1}'', ''AllowedUsages'' : ''NativeIngestion'' }}]\"', variables('INGESTION_DB'), reference('cluster', '2023-08-15', 'full').identity.principalId))]", + "type": "Expression" }, + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } + }, + { + "name": "Save Hub Settings in ADX", + "type": "AzureDataExplorerCommand", + "dependsOn": [ { - "name": "EA daily reservation transactions export", - "type": "WebActivity", - "dependsOn": [ - { - "activity": "EA daily reservation details export", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "timeout": "0.00:05:00", - "retry": 2, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "url": { - "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-reservationtransactions''))}}?api-version={0}', variables('exportApiVersion'))]", - "type": "Expression" - }, - "method": "PUT", - "body": { - "value": "[__bicep.getExportBodyV2(parameters('exportContainerName'), 'ReservationTransactions', variables('exportSchemaVersion'), false(), 'CSV', 'None', 'true', 'CreateNewReport', '', '', '')]", - "type": "Expression" - }, - "headers": { - "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.ReservationTransactions@{0}', variables('ftkVersion'))]", - "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('ftkVersion'))]" - }, - "authentication": { - "type": "MSI", - "resource": { - "value": "@variables('resourceManagementUri')", - "type": "Expression" - } - } - } + "activity": "Set ingestion policy in ADX", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": { + "value": "@concat('.append HubSettingsLog <| print version=\"', variables('version'), '\",scopes=dynamic(', variables('scopes'), '),retention=dynamic(', variables('retention'), ') | extend scopes = iff(isnull(scopes[0]), pack_array(scopes), scopes) | mv-apply scopeObj = scopes on (where isnotempty(scopeObj.scope) | summarize scopes = make_set(scopeObj.scope))')", + "type": "Expression" }, + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } + }, + { + "name": "Update PricingUnits in ADX", + "type": "AzureDataExplorerCommand", + "dependsOn": [ { - "name": "EA daily shared 30day virtualmachines", - "type": "WebActivity", - "dependsOn": [ - { - "activity": "EA daily reservation transactions export", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "timeout": "0.00:05:00", - "retry": 2, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "url": { - "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-recommendations-shared-last30days-virtualmachines''))}}?api-version={0}', variables('exportApiVersion'))]", - "type": "Expression" - }, - "method": "PUT", - "body": { - "value": "[__bicep.getExportBodyV2(parameters('exportContainerName'), 'ReservationRecommendations', variables('exportSchemaVersion'), false(), 'CSV', 'None', 'true', 'CreateNewReport', 'Shared', 'Last30Days', 'VirtualMachines')]", - "type": "Expression" - }, - "headers": { - "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.ReservationRecommendations.VM.Shared.30d@{0}', variables('ftkVersion'))]", - "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('ftkVersion'))]" - }, - "authentication": { - "type": "MSI", - "resource": { - "value": "@variables('resourceManagementUri')", - "type": "Expression" - } - } - } + "activity": "Save Hub Settings in ADX", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": "[format('.set-or-replace PricingUnits <| externaldata(x_PricingUnitDescription: string, AccountTypes: string, x_PricingBlockSize: real, PricingUnit: string)[@\"{0}/PricingUnits.csv\"] with (format=\"csv\", ignoreFirstRecord=true) | project-away AccountTypes', variables('ftkReleaseUri'))]", + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" } - ] + } }, { - "value": "subscription", - "activities": [ + "name": "Update Regions in ADX", + "type": "AzureDataExplorerCommand", + "dependsOn": [ { - "name": "Subscription open month focus export", - "type": "WebActivity", - "dependsOn": [], - "policy": { - "timeout": "0.00:05:00", - "retry": 2, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "url": { - "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-daily-costdetails''))}}?api-version={0}', variables('exportApiVersion'))]", - "type": "Expression" - }, - "method": "PUT", - "body": { - "value": "[__bicep.getExportBodyV2(parameters('exportContainerName'), 'FocusCost', variables('focusSchemaVersion'), false(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", - "type": "Expression" - }, - "headers": { - "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsDaily@{0}', variables('ftkVersion'))]", - "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('ftkVersion'))]" - }, - "authentication": { - "type": "MSI", - "resource": { - "value": "@variables('resourceManagementUri')", - "type": "Expression" - } - } - } - }, + "activity": "Update PricingUnits in ADX", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": "[format('.set-or-replace Regions <| externaldata(ResourceLocation: string, RegionId: string, RegionName: string)[@\"{0}/Regions.csv\"] with (format=\"csv\", ignoreFirstRecord=true)', variables('ftkReleaseUri'))]", + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } + }, + { + "name": "Update ResourceTypes in ADX", + "type": "AzureDataExplorerCommand", + "dependsOn": [ { - "name": "Subscription closed month focus export", - "type": "WebActivity", - "dependsOn": [ - { - "activity": "Subscription open month focus export", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "timeout": "0.00:05:00", - "retry": 2, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "url": { - "value": "[format('@{{variables(''resourceManagementUri'')}}@{{item().scope}}/providers/Microsoft.CostManagement/exports/@{{toLower(concat(variables(''finOpsHub''), ''-monthly-costdetails''))}}?api-version={0}', variables('exportApiVersion'))]", - "type": "Expression" - }, - "method": "PUT", - "body": { - "value": "[__bicep.getExportBodyV2(parameters('exportContainerName'), 'FocusCost', variables('focusSchemaVersion'), true(), 'Parquet', 'Snappy', 'true', 'CreateNewReport', '', '', '')]", - "type": "Expression" - }, - "headers": { - "x-ms-command-name": "[format('FinOpsToolkit.Hubs.config_RunExportJobs.CostsMonthly@{0}', variables('ftkVersion'))]", - "ClientType": "[format('FinOpsToolkit.Hubs@{0}', variables('ftkVersion'))]" - }, - "authentication": { - "type": "MSI", - "resource": { - "value": "@variables('resourceManagementUri')", - "type": "Expression" - } - } - } + "activity": "Update Regions in ADX", + "dependencyConditions": [ + "Succeeded" + ] } - ] + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": "[format('.set-or-replace ResourceTypes <| externaldata(x_ResourceType: string, SingularDisplayName: string, PluralDisplayName: string, LowerSingularDisplayName: string, LowerPluralDisplayName: string, IsPreview: bool, Description: string, IconUri: string, Links: string)[@\"{0}/ResourceTypes.csv\"] with (format=\"csv\", ignoreFirstRecord=true) | project-away Links', variables('ftkReleaseUri'))]", + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } }, { - "value": "mca", - "activities": [ + "name": "Update Services in ADX", + "type": "AzureDataExplorerCommand", + "dependsOn": [ { - "name": "Export Type Unsupported Error", - "type": "Fail", - "dependsOn": [], - "userProperties": [], - "typeProperties": { - "message": { - "value": "@concat('MCA agreements are not supported for managed exports :',variables('exportScope'))", - "type": "Expression" - }, - "errorCode": "ExportTypeUnsupported" - } + "activity": "Update ResourceTypes in ADX", + "dependencyConditions": [ + "Succeeded" + ] } - ] - } - ], - "defaultActivities": [ + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": "[format('.set-or-replace Services <| externaldata(x_ConsumedService: string, x_ResourceType: string, ServiceName: string, ServiceCategory: string, ServiceSubcategory: string, PublisherName: string, x_PublisherCategory: string, x_Environment: string, x_ServiceModel: string)[@\"{0}/Services.csv\"] with (format=\"csv\", ignoreFirstRecord=true)', variables('ftkReleaseUri'))]", + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } + } + }, { - "name": "Export Type Not Defined Error", - "type": "Fail", - "dependsOn": [], + "name": "Ingestion Complete", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Update Services in ADX", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, "userProperties": [], "typeProperties": { - "message": { - "value": "@concat('Unable to determine the export scope type for :',variables('exportScope'))", - "type": "Expression" - }, - "errorCode": "ExportTypeNotDefined" + "variableName": "tryAgain", + "value": false } } ] } + }, + { + "name": "Abort On Error", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "If Has Capacity", + "dependencyConditions": [ + "Failed" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "tryAgain", + "value": false + } } - ] + ], + "timeout": "0.02:00:00" + } + }, + { + "name": "Timeout Error", + "type": "Fail", + "dependsOn": [ + { + "activity": "Until Capacity Is Available", + "dependencyConditions": [ + "Failed" + ] + } + ], + "userProperties": [], + "typeProperties": { + "message": "Data Explorer ingestion timed out after 2 hours while waiting for available capacity. Please re-run this pipeline to re-attempt ingestion. If you continue to see this error, please report an issue at https://aka.ms/ftk/ideas.", + "errorCode": "DataExplorerIngestionTimeout" } } ], "concurrency": 1, "variables": { - "scopesArray": { - "type": "Array" - }, - "exportName": { + "version": { "type": "String" }, - "exportScope": { + "scopes": { "type": "String" }, - "exportScopeType": { + "retention": { "type": "String" }, - "storageAccountId": { - "type": "String", - "defaultValue": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]" - }, - "finOpsHub": { - "type": "String", - "defaultValue": "[parameters('hubName')]" - }, - "resourceManagementUri": { - "type": "String", - "defaultValue": "[environment().resourceManager]" - }, - "fileName": { - "type": "String", - "defaultValue": "settings.json" - }, - "folderPath": { - "type": "String", - "defaultValue": "[parameters('configContainerName')]" + "tryAgain": { + "type": "Boolean", + "defaultValue": true } } }, "dependsOn": [ - "dataset_config" + "appRegistration", + "cluster", + "linkedService_dataExplorer" ], "metadata": { - "description": "Creates Cost Management exports for supported scopes." + "description": "Initializes the hub instance based on the configuration settings." } }, - "pipeline_ExecuteExportsETL": { + "pipeline_ToDataExplorer": { + "condition": "[or(variables('useAzure'), variables('useFabric'))]", "type": "Microsoft.DataFactory/factories/pipelines", "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), format('{0}_ExecuteETL', variables('safeExportContainerName')))]", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_ETL_dataExplorer', variables('INGESTION')))]", "properties": { "activities": [ { - "name": "Wait", - "description": "Files may not be available immediately after being created.", - "type": "Wait", - "dependsOn": [], - "userProperties": [], - "typeProperties": { - "waitTimeInSeconds": 60 - } - }, - { - "name": "Read Manifest", - "description": "Load the export manifest to determine the scope, dataset, and date range.", + "name": "Read Hub Config", + "description": "Read the hub config to determine how long data should be retained.", "type": "Lookup", - "dependsOn": [ - { - "activity": "Wait", - "dependencyConditions": [ - "Completed" - ] - } - ], "policy": { "timeout": "0.12:00:00", "retry": 0, @@ -13560,7 +17692,7 @@ "type": "JsonSource", "storeSettings": { "type": "AzureBlobFSReadSettings", - "recursive": true, + "recursive": false, "enablePartitionDiscovery": false }, "formatSettings": { @@ -13568,103 +17700,21 @@ } }, "dataset": { - "referenceName": "manifest", + "referenceName": "[variables('CONFIG')]", "type": "DatasetReference", "parameters": { - "fileName": { - "value": "@pipeline().parameters.fileName", - "type": "Expression" - }, - "folderPath": { - "value": "@pipeline().parameters.folderPath", - "type": "Expression" - } + "fileName": "settings.json", + "folderPath": "[variables('CONFIG')]" } } } }, { - "name": "Set Has No Rows", - "description": "Check the row count ", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Read Manifest", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "hasNoRows", - "value": { - "value": "@or(equals(activity('Read Manifest').output.firstRow.blobCount, null), equals(activity('Read Manifest').output.firstRow.blobCount, 0))", - "type": "Expression" - } - } - }, - { - "name": "Set Export Dataset Type", - "description": "Save the dataset type from the export manifest.", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Read Manifest", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "exportDatasetType", - "value": { - "value": "@activity('Read Manifest').output.firstRow.exportConfig.type", - "type": "Expression" - } - } - }, - { - "name": "Set MCA Column", - "description": "Determines if the dataset schema has channel-specific columns and saves the column name that only exists in MCA to determine if it is an MCA dataset.", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Set Export Dataset Type", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "mcaColumnToCheck", - "value": { - "value": "@if(contains(createArray('pricesheet', 'reservationtransactions'), toLower(variables('exportDatasetType'))), 'BillingProfileId', if(equals(toLower(variables('exportDatasetType')), 'reservationrecommendations'), 'Net Savings', null))", - "type": "Expression" - } - } - }, - { - "name": "Set Export Dataset Version", - "description": "Save the dataset version from the export manifest.", + "name": "Set Final Retention Months", "type": "SetVariable", "dependsOn": [ { - "activity": "Read Manifest", + "activity": "Read Hub Config", "dependencyConditions": [ "Succeeded" ] @@ -13676,296 +17726,415 @@ }, "userProperties": [], "typeProperties": { - "variableName": "exportDatasetVersion", + "variableName": "finalRetentionMonths", "value": { - "value": "@activity('Read Manifest').output.firstRow.exportConfig.dataVersion", + "value": "@coalesce(activity('Read Hub Config').output.firstRow.retention.final.months, 999)", "type": "Expression" } } }, { - "name": "Detect Channel", - "description": "Determines what channel this export is from. Switch statement handles the different file types if the mcaColumnToCheck variable is set.", - "type": "Switch", + "name": "Until Capacity Is Available", + "type": "Until", "dependsOn": [ { - "activity": "Set Has No Rows", - "dependencyConditions": [ - "Succeeded" - ] - }, - { - "activity": "Set MCA Column", - "dependencyConditions": [ - "Succeeded" - ] - }, - { - "activity": "Set Export Dataset Version", + "activity": "Set Final Retention Months", "dependencyConditions": [ - "Succeeded" + "Completed", + "Skipped" ] } ], "userProperties": [], "typeProperties": { - "on": { - "value": "@if(or(empty(variables('mcaColumnToCheck')), variables('hasNoRows')), 'ignore', last(array(split(activity('Read Manifest').output.firstRow.blobs[0].blobName, '.'))))", + "expression": { + "value": "@equals(variables('tryAgain'), false)", "type": "Expression" }, - "cases": [ + "activities": [ { - "value": "csv", - "activities": [ + "name": "Confirm Ingestion Capacity", + "type": "AzureDataExplorerCommand", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": ".show capacity | where Resource == 'Ingestions' | project Remaining", + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference" + } + }, + { + "name": "If Has Capacity", + "type": "IfCondition", + "dependsOn": [ { - "name": "Check for MCA Column in CSV", - "description": "Checks the dataset to determine if the applicable MCA-specific column exists.", - "type": "Lookup", - "dependsOn": [], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false + "activity": "Confirm Ingestion Capacity", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "expression": { + "value": "@or(equals(activity('Confirm Ingestion Capacity').output.count, 0), greater(activity('Confirm Ingestion Capacity').output.value[0].Remaining, 0))", + "type": "Expression" + }, + "ifFalseActivities": [ + { + "name": "Wait for Ingestion", + "type": "Wait", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "waitTimeInSeconds": 15 + } }, - "userProperties": [], - "typeProperties": { - "source": { - "type": "DelimitedTextSource", - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": false, - "enablePartitionDiscovery": false + { + "name": "Try Again", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Wait for Ingestion", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "tryAgain", + "value": true + } + } + ], + "ifTrueActivities": [ + { + "name": "Pre-Ingest Cleanup", + "description": "Cost Management exports include all month-to-date data from the previous export run. To ensure data is not double-reported, it must be dropped from the raw table before ingestion completes. Remove previous ingestions into the raw table for the month and any previous runs of the current ingestion month file in any table.", + "type": "AzureDataExplorerCommand", + "dependsOn": [], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false + }, + "typeProperties": { + "command": { + "value": "@concat('.drop extents <| .show extents | where (TableName == \"', pipeline().parameters.table, '\" and Tags !has \"drop-by:', pipeline().parameters.ingestionId, '\" and Tags has \"drop-by:', pipeline().parameters.folderPath, '\") or (Tags has \"drop-by:', pipeline().parameters.ingestionId, '\" and Tags has \"drop-by:', pipeline().parameters.folderPath, '/', pipeline().parameters.originalFileName, '\")')", + "type": "Expression" }, - "formatSettings": { - "type": "DelimitedTextReadSettings" - } + "commandTimeout": "00:20:00" }, - "dataset": { - "referenceName": "[variables('safeExportContainerName')]", - "type": "DatasetReference", + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", "parameters": { - "blobPath": { - "value": "@activity('Read Manifest').output.firstRow.blobs[0].blobName", - "type": "Expression" - } + "database": "[variables('INGESTION_DB')]" } } - } - }, - { - "name": "Set Schema File with Channel in CSV", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Check for MCA Column in CSV", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false }, - "userProperties": [], - "typeProperties": { - "variableName": "schemaFile", - "value": { - "value": "@toLower(concat(variables('exportDatasetType'), '_', variables('exportDatasetVersion'), if(and(contains(activity('Check for MCA Column in CSV').output, 'firstRow'), contains(activity('Check for MCA Column in CSV').output.firstRow, variables('mcaColumnToCheck'))), '_mca', '_ea'), '.json'))", - "type": "Expression" + { + "name": "Ingest Data", + "type": "AzureDataExplorerCommand", + "dependsOn": [ + { + "activity": "Pre-Ingest Cleanup", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 3, + "retryIntervalInSeconds": 120, + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "command": { + "value": "[format('@concat(''.ingest into table '', pipeline().parameters.table, '' (\"abfss://{0}@{1}.dfs.{2}/'', pipeline().parameters.folderPath, ''/'', pipeline().parameters.fileName, '';{3}\") with (format=\"parquet\", ingestionMappingReference=\"'', pipeline().parameters.table, ''_mapping\", tags=\"[\\\"drop-by:'', pipeline().parameters.ingestionId, ''\\\", \\\"drop-by:'', pipeline().parameters.folderPath, ''/'', pipeline().parameters.originalFileName, ''\\\", \\\"drop-by:ftk-version-{4}\\\"]\"); print Success = assert(iff(toscalar($command_results | project-keep HasErrors) == false, true, false), \"Ingestion Failed\")'')', variables('INGESTION'), parameters('app').storage, environment().suffixes.storage, if(variables('useFabric'), 'impersonate', 'managed_identity=system'), variables('finOpsToolkitVersion'))]", + "type": "Expression" + }, + "commandTimeout": "01:00:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", + "parameters": { + "database": "[variables('INGESTION_DB')]" + } } - } - } - ] - }, - { - "value": "gz", - "activities": [ - { - "name": "Check for MCA Column in Gzip CSV", - "description": "Checks the dataset to determine if the applicable MCA-specific column exists.", - "type": "Lookup", - "dependsOn": [], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false }, - "userProperties": [], - "typeProperties": { - "source": { - "type": "DelimitedTextSource", - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": false, - "enablePartitionDiscovery": false - }, - "formatSettings": { - "type": "DelimitedTextReadSettings" + { + "name": "Post-Ingest Cleanup", + "description": "Cost Management exports include all month-to-date data from the previous export run. To ensure data is not double-reported, it must be dropped after ingestion completes. Remove the current ingestion month file from raw and any old ingestions for the month from the final table.", + "type": "AzureDataExplorerCommand", + "dependsOn": [ + { + "activity": "Ingest Data", + "dependencyConditions": [ + "Completed" + ] } + ], + "policy": { + "timeout": "0.12:00:00", + "retry": 0, + "retryIntervalInSeconds": 30, + "secureOutput": false, + "secureInput": false }, - "dataset": { - "referenceName": "[format('{0}_gzip', variables('safeExportContainerName'))]", - "type": "DatasetReference", + "typeProperties": { + "command": { + "value": "@concat('.drop extents <| .show extents | extend isOldFinalData = (TableName startswith \"', replace(pipeline().parameters.table, '_raw', '_final_v'), '\" and Tags !has \"drop-by:', pipeline().parameters.ingestionId, '\" and Tags has \"drop-by:', pipeline().parameters.folderPath, '\") | extend isPastFinalRetention = (TableName startswith \"', replace(pipeline().parameters.table, '_raw', '_final_v'), '\" and todatetime(substring(strcat(replace_string(extract(\"drop-by:[A-Za-z]+/(\\\\d{4}/\\\\d{2}(/\\\\d{2})?)\", 1, Tags), \"/\", \"-\"), \"-01\"), 0, 10)) < datetime_add(\"month\", -', if(lessOrEquals(variables('finalRetentionMonths'), 0), 0, variables('finalRetentionMonths')), ', startofmonth(now()))) | where isOldFinalData or isPastFinalRetention')", + "type": "Expression" + }, + "commandTimeout": "00:20:00" + }, + "linkedServiceName": { + "referenceName": "[variables('HUB_DATA_EXPLORER')]", + "type": "LinkedServiceReference", "parameters": { - "blobPath": { - "value": "@activity('Read Manifest').output.firstRow.blobs[0].blobName", - "type": "Expression" - } + "database": "[variables('INGESTION_DB')]" } } - } - }, - { - "name": "Set Schema File with Channel in Gzip CSV", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Check for MCA Column in Gzip CSV", - "dependencyConditions": [ - "Succeeded" - ] + }, + { + "name": "Ingestion Complete", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Post-Ingest Cleanup", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "tryAgain", + "value": false } - ], - "policy": { - "secureOutput": false, - "secureInput": false }, - "userProperties": [], - "typeProperties": { - "variableName": "schemaFile", - "value": { - "value": "@toLower(concat(variables('exportDatasetType'), '_', variables('exportDatasetVersion'), if(and(contains(activity('Check for MCA Column in Gzip CSV').output, 'firstRow'), contains(activity('Check for MCA Column in Gzip CSV').output.firstRow, variables('mcaColumnToCheck'))), '_mca', '_ea'), '.json'))", - "type": "Expression" + { + "name": "Abort On Ingestion Error", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Ingest Data", + "dependencyConditions": [ + "Failed" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "tryAgain", + "value": false } - } - } - ] - }, - { - "value": "parquet", - "activities": [ - { - "name": "Check for MCA Column in Parquet", - "description": "Checks the dataset to determine if the applicable MCA-specific column exists.", - "type": "Lookup", - "dependsOn": [], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false }, - "userProperties": [], - "typeProperties": { - "source": { - "type": "ParquetSource", - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": false, - "enablePartitionDiscovery": false + { + "name": "Ingestion Failed Error", + "type": "Fail", + "dependsOn": [ + { + "activity": "Abort On Ingestion Error", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Data Explorer ingestion into the ', pipeline().parameters.table, ' table failed. Please fix the error and rerun ingestion for the following folder path: \"', pipeline().parameters.folderPath, '\". File: ', pipeline().parameters.originalFileName, '. Error: ', if(greater(length(activity('Ingest Data').output.errors), 0), activity('Ingest Data').output.errors[0].Message, 'Unknown'), ' (Code: ', if(greater(length(activity('Ingest Data').output.errors), 0), activity('Ingest Data').output.errors[0].Code, 'None'), ')')", + "type": "Expression" + }, + "errorCode": "DataExplorerIngestionFailed" + } + }, + { + "name": "Abort On Pre-Ingest Drop Error", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Pre-Ingest Cleanup", + "dependencyConditions": [ + "Failed" + ] + } + ], + "policy": { + "secureOutput": false, + "secureInput": false + }, + "userProperties": [], + "typeProperties": { + "variableName": "tryAgain", + "value": false + } + }, + { + "name": "Pre-Ingest Drop Failed Error", + "type": "Fail", + "dependsOn": [ + { + "activity": "Abort On Pre-Ingest Drop Error", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Data Explorer pre-ingestion cleanup (drop extents from raw table) for the ', pipeline().parameters.table, ' table failed. Ingestion was not completed. Please fix the error and rerun ingestion for the following folder path: \"', pipeline().parameters.folderPath, '\". File: ', pipeline().parameters.originalFileName, '. Error: ', if(greater(length(activity('Pre-Ingest Cleanup').output.errors), 0), activity('Pre-Ingest Cleanup').output.errors[0].Message, 'Unknown'), ' (Code: ', if(greater(length(activity('Pre-Ingest Cleanup').output.errors), 0), activity('Pre-Ingest Cleanup').output.errors[0].Code, 'None'), ')')", + "type": "Expression" }, - "formatSettings": { - "type": "ParquetReadSettings" + "errorCode": "DataExplorerPreIngestionDropFailed" + } + }, + { + "name": "Abort On Post-Ingest Drop Error", + "type": "SetVariable", + "dependsOn": [ + { + "activity": "Post-Ingest Cleanup", + "dependencyConditions": [ + "Failed" + ] } + ], + "policy": { + "secureOutput": false, + "secureInput": false }, - "dataset": { - "referenceName": "[format('{0}_parquet', variables('safeExportContainerName'))]", - "type": "DatasetReference", - "parameters": { - "blobPath": { - "value": "@activity('Read Manifest').output.firstRow.blobs[0].blobName", - "type": "Expression" - } - } - } - } - }, - { - "name": "Set Schema File with Channel for Parquet", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Check for MCA Column in Parquet", - "dependencyConditions": [ - "Succeeded" - ] + "userProperties": [], + "typeProperties": { + "variableName": "tryAgain", + "value": false } - ], - "policy": { - "secureOutput": false, - "secureInput": false }, - "userProperties": [], - "typeProperties": { - "variableName": "schemaFile", - "value": { - "value": "@toLower(concat(variables('exportDatasetType'), '_', variables('exportDatasetVersion'), if(and(contains(activity('Check for MCA Column in Parquet').output, 'firstRow'), contains(activity('Check for MCA Column in Parquet').output.firstRow, variables('mcaColumnToCheck'))), '_mca', '_ea'), '.json'))", - "type": "Expression" + { + "name": "Post-Ingest Drop Failed Error", + "type": "Fail", + "dependsOn": [ + { + "activity": "Abort On Post-Ingest Drop Error", + "dependencyConditions": [ + "Succeeded" + ] + } + ], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Data Explorer post-ingestion cleanup (drop extents from final tables) for the ', replace(pipeline().parameters.table, '_raw', '_final_*'), ' table failed. Please fix the error and rerun ingestion for the following folder path: \"', pipeline().parameters.folderPath, '\". File: ', pipeline().parameters.originalFileName, '. Error: ', if(greater(length(activity('Post-Ingest Cleanup').output.errors), 0), activity('Post-Ingest Cleanup').output.errors[0].Message, 'Unknown'), ' (Code: ', if(greater(length(activity('Post-Ingest Cleanup').output.errors), 0), activity('Post-Ingest Cleanup').output.errors[0].Code, 'None'), ')')", + "type": "Expression" + }, + "errorCode": "DataExplorerPostIngestionDropFailed" } } - } - ] - } - ], - "defaultActivities": [ - { - "name": "Set Schema File", - "type": "SetVariable", - "dependsOn": [], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "schemaFile", - "value": { - "value": "@toLower(concat(variables('exportDatasetType'), '_', variables('exportDatasetVersion'), '.json'))", - "type": "Expression" - } + ] } } - ] + ], + "timeout": "0.02:00:00" } + } + ], + "parameters": { + "folderPath": { + "type": "string" + }, + "fileName": { + "type": "string" + }, + "originalFileName": { + "type": "string" + }, + "ingestionId": { + "type": "string" + }, + "table": { + "type": "string" + } + }, + "variables": { + "tryAgain": { + "type": "Boolean", + "defaultValue": true + }, + "logRetentionDays": { + "type": "Integer", + "defaultValue": 0 }, + "finalRetentionMonths": { + "type": "Integer", + "defaultValue": 999 + } + }, + "annotations": [] + }, + "dependsOn": [ + "appRegistration", + "linkedService_dataExplorer" + ], + "metadata": { + "description": "Ingests parquet data into an Azure Data Explorer cluster." + } + }, + "pipeline_ExecuteIngestionETL": { + "condition": "[or(variables('useAzure'), variables('useFabric'))]", + "type": "Microsoft.DataFactory/factories/pipelines", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_ExecuteETL', variables('INGESTION')))]", + "properties": { + "concurrency": 1, + "activities": [ { - "name": "Set Scope", - "description": "Save the scope from the export manifest.", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Read Manifest", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, + "name": "Wait", + "description": "Files may not be available immediately after being created.", + "type": "Wait", + "dependsOn": [], "userProperties": [], "typeProperties": { - "variableName": "scope", - "value": { - "value": "@split(toLower(activity('Read Manifest').output.firstRow.exportConfig.resourceId), '/providers/microsoft.costmanagement/exports/')[0]", - "type": "Expression" - } + "waitTimeInSeconds": 60 } }, { - "name": "Set Date", - "description": "Save the exported month from the export manifest.", + "name": "Set Container Folder Path", "type": "SetVariable", "dependsOn": [ { - "activity": "Read Manifest", + "activity": "Wait", "dependencyConditions": [ "Succeeded" ] @@ -13977,82 +18146,20 @@ }, "userProperties": [], "typeProperties": { - "variableName": "date", + "variableName": "containerFolderPath", "value": { - "value": "@replace(substring(activity('Read Manifest').output.firstRow.runInfo.startDate, 0, 7), '-', '')", + "value": "@join(skip(array(split(pipeline().parameters.folderPath, '/')), 1), '/')", "type": "Expression" } } }, { - "name": "Failed to Read Manifest", - "type": "Fail", - "dependsOn": [ - { - "activity": "Set Date", - "dependencyConditions": [ - "Failed" - ] - }, - { - "activity": "Set Export Dataset Type", - "dependencyConditions": [ - "Failed" - ] - }, - { - "activity": "Set Scope", - "dependencyConditions": [ - "Failed" - ] - }, - { - "activity": "Read Manifest", - "dependencyConditions": [ - "Failed" - ] - }, - { - "activity": "Set Export Dataset Version", - "dependencyConditions": [ - "Failed" - ] - }, - { - "activity": "Detect Channel", - "dependencyConditions": [ - "Failed" - ] - } - ], - "userProperties": [], - "typeProperties": { - "message": { - "value": "@concat('Failed to read the manifest file for this export run. Manifest path: ', pipeline().parameters.folderPath)", - "type": "Expression" - }, - "errorCode": "ManifestReadFailed" - } - }, - { - "name": "Check Schema", - "description": "Verify that the schema file exists in storage.", + "name": "Get Existing Parquet Files", + "description": "Get the previously ingested files so we can get file paths.", "type": "GetMetadata", "dependsOn": [ { - "activity": "Set Scope", - "dependencyConditions": [ - "Succeeded" - ] - }, - { - "activity": "Set Date", - "dependencyConditions": [ - "Succeeded" - ] - }, - { - "activity": "Detect Channel", + "activity": "Set Container Folder Path", "dependencyConditions": [ "Succeeded" ] @@ -14068,55 +18175,54 @@ "userProperties": [], "typeProperties": { "dataset": { - "referenceName": "[variables('safeConfigContainerName')]", + "referenceName": "ingestion_files", "type": "DatasetReference", "parameters": { - "fileName": { - "value": "@variables('schemaFile')", - "type": "Expression" - }, - "folderPath": "[format('{0}/schemas', parameters('configContainerName'))]" + "folderPath": "@variables('containerFolderPath')" } }, "fieldList": [ - "exists" + "childItems" ], "storeSettings": { "type": "AzureBlobFSReadSettings", - "recursive": true, "enablePartitionDiscovery": false }, "formatSettings": { - "type": "JsonReadSettings" + "type": "ParquetReadSettings" } } }, { - "name": "Schema Not Found", - "type": "Fail", + "name": "Filter Out Folders", + "description": "Remove any folders or manifest files.", + "type": "Filter", "dependsOn": [ { - "activity": "Check Schema", + "activity": "Get Existing Parquet Files", "dependencyConditions": [ - "Failed" + "Succeeded" ] } ], "userProperties": [], "typeProperties": { - "message": { - "value": "@concat('The ', variables('schemaFile'), ' schema mapping file was not found. Please confirm version ', variables('exportDatasetVersion'), ' of the ', variables('exportDatasetType'), ' dataset is supported by this version of FinOps hubs. You may need to upgrade to a newer release. To add support for another dataset, you can create a custom mapping file.')", + "items": { + "value": "@if(contains(activity('Get Existing Parquet Files').output, 'childItems'), activity('Get Existing Parquet Files').output.childItems, json('[]'))", "type": "Expression" }, - "errorCode": "SchemaNotFound" + "condition": { + "value": "@and(equals(item().type, 'File'), not(contains(toLower(item().name), 'manifest.json')))", + "type": "Expression" + } } }, { - "name": "Set Hub Dataset", + "name": "Set Ingestion Timestamp", "type": "SetVariable", "dependsOn": [ { - "activity": "Set Export Dataset Type", + "activity": "Wait", "dependencyConditions": [ "Succeeded" ] @@ -14128,50 +18234,26 @@ }, "userProperties": [], "typeProperties": { - "variableName": "hubDataset", + "variableName": "timestamp", "value": { - "value": "@if(equals(toLower(variables('exportDatasetType')), 'focuscost'), 'Costs', if(equals(toLower(variables('exportDatasetType')), 'pricesheet'), 'Prices', if(equals(toLower(variables('exportDatasetType')), 'reservationdetails'), 'CommitmentDiscountUsage', if(equals(toLower(variables('exportDatasetType')), 'reservationrecommendations'), 'Recommendations', if(equals(toLower(variables('exportDatasetType')), 'reservationtransactions'), 'Transactions', if(equals(toLower(variables('exportDatasetType')), 'actualcost'), 'ActualCosts', if(equals(toLower(variables('exportDatasetType')), 'amortizedcost'), 'AmortizedCosts', toLower(variables('exportDatasetType')))))))))", + "value": "@utcNow()", "type": "Expression" } } }, { - "name": "Set Destination Folder", - "type": "SetVariable", + "name": "For Each Old File", + "description": "Loop thru each of the existing files.", + "type": "ForEach", "dependsOn": [ { - "activity": "Check Schema", + "activity": "Filter Out Folders", "dependencyConditions": [ "Succeeded" ] }, { - "activity": "Set Hub Dataset", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "destinationFolder", - "value": { - "value": "@replace(concat(variables('hubDataset'),'/',substring(variables('date'), 0, 4),'/',substring(variables('date'), 4, 2),'/',toLower(variables('scope')), if(equals(variables('hubDataset'), 'Recommendations'), activity('Read Manifest').output.firstRow.exportConfig.exportName, '')),'//','/')", - "type": "Expression" - } - } - }, - { - "name": "For Each Blob", - "description": "Loop thru each exported file listed in the manifest.", - "type": "ForEach", - "dependsOn": [ - { - "activity": "Set Destination Folder", + "activity": "Set Ingestion Timestamp", "dependencyConditions": [ "Succeeded" ] @@ -14179,16 +18261,15 @@ ], "userProperties": [], "typeProperties": { + "batchCount": "[variables('dataExplorerIngestionCapacity')]", "items": { - "value": "@if(variables('hasNoRows'), json('[]'), activity('Read Manifest').output.firstRow.blobs)", + "value": "@activity('Filter Out Folders').output.Value", "type": "Expression" }, - "batchCount": "[if(parameters('enablePublicAccess'), 30, 4)]", - "isSequential": false, "activities": [ { "name": "Execute", - "description": "Run the ingestion ETL pipeline.", + "description": "Run the ADX ETL pipeline.", "type": "ExecutePipeline", "dependsOn": [], "policy": { @@ -14197,37 +18278,29 @@ "userProperties": [], "typeProperties": { "pipeline": { - "referenceName": "[format('{0}_ETL_{1}', variables('safeExportContainerName'), variables('safeIngestionContainerName'))]", + "referenceName": "[format('{0}_ETL_dataExplorer', variables('INGESTION'))]", "type": "PipelineReference" }, "waitOnCompletion": true, "parameters": { - "blobPath": { - "value": "@item().blobName", + "folderPath": { + "value": "@variables('containerFolderPath')", "type": "Expression" }, - "destinationFolder": { - "value": "@variables('destinationFolder')", + "fileName": { + "value": "@item().name", "type": "Expression" }, - "destinationFile": { - "value": "@last(array(split(replace(replace(item().blobName, '.gz', ''), '.csv', '.parquet'), '/')))", + "originalFileName": { + "value": "[format('@last(array(split(item().name, ''{0}'')))', variables('INGESTION_ID_SEPARATOR'))]", "type": "Expression" }, "ingestionId": { - "value": "@activity('Read Manifest').output.firstRow.runInfo.runId", - "type": "Expression" - }, - "schemaFile": { - "value": "@variables('schemaFile')", - "type": "Expression" - }, - "exportDatasetType": { - "value": "@variables('exportDatasetType')", + "value": "[format('@concat(first(array(split(item().name, ''{0}''))), ''_'', variables(''timestamp''))', variables('INGESTION_ID_SEPARATOR'))]", "type": "Expression" }, - "exportDatasetVersion": { - "value": "@variables('exportDatasetVersion')", + "table": { + "value": "@concat(first(array(split(variables('containerFolderPath'), '/'))), '_raw')", "type": "Expression" } } @@ -14237,1604 +18310,1834 @@ } }, { - "name": "Copy Manifest", - "description": "Copy the manifest to the ingestion container to trigger ADX ingestion", - "type": "Copy", + "name": "If No Files", + "description": "If there are no files found, fail the pipeline.", + "type": "IfCondition", "dependsOn": [ { - "activity": "For Each Blob", + "activity": "Filter Out Folders", "dependencyConditions": [ "Succeeded" ] } ], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, "userProperties": [], "typeProperties": { - "source": { - "type": "JsonSource", - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": true, - "enablePartitionDiscovery": false + "expression": { + "value": "@equals(length(activity('Filter Out Folders').output.Value), 0)", + "type": "Expression" + }, + "ifTrueActivities": [ + { + "name": "Files Not Found", + "type": "Fail", + "dependsOn": [], + "userProperties": [], + "typeProperties": { + "message": { + "value": "@concat('Unable to locate parquet files to ingest from the ', pipeline().parameters.folderPath, ' path. Please confirm the folder path is the full path, including the \"ingestion\" container and not starting with or ending with a slash (\"/\").')", + "type": "Expression" + }, + "errorCode": "IngestionFilesNotFound" + } + } + ] + } + } + ], + "parameters": { + "folderPath": { + "type": "string" + } + }, + "variables": { + "containerFolderPath": { + "type": "string" + }, + "timestamp": { + "type": "string" + } + }, + "annotations": [ + "New ingestion" + ] + }, + "dependsOn": [ + "appRegistration", + "pipeline_ToDataExplorer" + ], + "metadata": { + "description": "Queues the ingestion_ETL_dataExplorer pipeline to account for Data Factory pipeline trigger limits." + } + }, + "appRegistration": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_Register", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "version": { + "value": "[variables('finOpsToolkitVersion')]" + }, + "features": { + "value": [ + "DataFactory", + "Storage" + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "8267413868206701537" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" }, - "formatSettings": { - "type": "JsonReadSettings" + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } } }, - "sink": { - "type": "JsonSink", - "storeSettings": { - "type": "AzureBlobFSWriteSettings" + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." }, - "formatSettings": { - "type": "JsonWriteSettings" + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" } - }, - "enableStaging": false + } }, - "inputs": [ - { - "referenceName": "manifest", - "type": "DatasetReference", - "parameters": { - "fileName": "manifest.json", - "folderPath": { - "value": "@pipeline().parameters.folderPath", - "type": "Expression" + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } } - } - } - ], - "outputs": [ - { - "referenceName": "manifest", - "type": "DatasetReference", - "parameters": { - "fileName": "manifest.json", - "folderPath": { - "value": "[format('@concat(''{0}/'', variables(''destinationFolder''))', parameters('ingestionContainerName'))]", - "type": "Expression" + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } } } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } } - ] - } - ], - "parameters": { - "folderPath": { - "type": "string" - }, - "fileName": { - "type": "string" - } - }, - "variables": { - "date": { - "type": "String" - }, - "destinationFolder": { - "type": "String" - }, - "exportDatasetType": { - "type": "String" - }, - "exportDatasetVersion": { - "type": "String" - }, - "hasNoRows": { - "type": "Boolean" - }, - "hubDataset": { - "type": "String" - }, - "mcaColumnToCheck": { - "type": "String" - }, - "schemaFile": { - "type": "String" - }, - "scope": { - "type": "String" - } - }, - "annotations": [ - "New export" - ] - }, - "dependsOn": [ - "dataset_config", - "dataset_manifest", - "dataset_msexports", - "dataset_msexports_gzip", - "dataset_msexports_parquet", - "pipeline_ToIngestion" - ], - "metadata": { - "description": "Queues the msexports_ETL_ingestion pipeline." - } - }, - "pipeline_ToIngestion": { - "type": "Microsoft.DataFactory/factories/pipelines", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), format('{0}_ETL_{1}', variables('safeExportContainerName'), variables('safeIngestionContainerName')))]", - "properties": { - "activities": [ - { - "name": "Get Existing Parquet Files", - "description": "Get the previously ingested files so we can remove any older data. This is necessary to avoid data duplication in reports.", - "type": "GetMetadata", - "dependsOn": [], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false }, - "userProperties": [], - "typeProperties": { - "dataset": { - "referenceName": "[format('{0}_files', variables('safeIngestionContainerName'))]", - "type": "DatasetReference", - "parameters": { - "folderPath": "@pipeline().parameters.destinationFolder" + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" } }, - "fieldList": [ - "childItems" - ], - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "enablePartitionDiscovery": false - }, - "formatSettings": { - "type": "ParquetReadSettings" - } - } - }, - { - "name": "Filter Out Current Exports", - "description": "Remove existing files from the current export so those files do not get deleted.", - "type": "Filter", - "dependsOn": [ - { - "activity": "Get Existing Parquet Files", - "dependencyConditions": [ - "Completed" - ] - } - ], - "userProperties": [], - "typeProperties": { - "items": { - "value": "@if(contains(activity('Get Existing Parquet Files').output, 'childItems'), activity('Get Existing Parquet Files').output.childItems, json('[]'))", - "type": "Expression" - }, - "condition": { - "value": "[format('@and(endswith(item().name, ''.parquet''), not(startswith(item().name, concat(pipeline().parameters.ingestionId, ''{0}''))))', variables('ingestionIdFileNameSeparator'))]", - "type": "Expression" + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppFeature": { + "type": "string", + "allowedValues": [ + "DataFactory", + "KeyVault", + "Storage" + ], + "metadata": { + "description": "FinOps hub app features.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } } - } - }, - { - "name": "Load Schema Mappings", - "description": "Get schema mapping file to use for the CSV to parquet conversion.", - "type": "Lookup", - "dependsOn": [], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false }, - "userProperties": [], - "typeProperties": { - "source": { - "type": "JsonSource", - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": true, - "enablePartitionDiscovery": false + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" }, - "formatSettings": { - "type": "JsonReadSettings" + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" } }, - "dataset": { - "referenceName": "[variables('safeConfigContainerName')]", - "type": "DatasetReference", - "parameters": { - "fileName": { - "value": "@toLower(pipeline().parameters.schemaFile)", - "type": "Expression" - }, - "folderPath": "[format('{0}/schemas', parameters('configContainerName'))]" + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" } } } }, - { - "name": "Failed to Load Schema", - "type": "Fail", - "dependsOn": [ - { - "activity": "Load Schema Mappings", - "dependencyConditions": [ - "Failed" - ] + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } } - ], - "userProperties": [], - "typeProperties": { - "message": { - "value": "@concat('Unable to load the ', pipeline().parameters.schemaFile, ' schema file. Please confirm the schema and version are supported for FinOps hubs ingestion. Unsupported files will remain in the msexports container.')", - "type": "Expression" - }, - "errorCode": "SchemaLoadFailed" } - }, - { - "name": "Set Additional Columns", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Load Schema Mappings", - "dependencyConditions": [ - "Succeeded" - ] + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." } - ], - "policy": { - "secureOutput": false, - "secureInput": false }, - "userProperties": [], - "typeProperties": { - "variableName": "additionalColumns", - "value": { - "value": "@intersection(array(json(concat('[{\"name\":\"x_SourceProvider\",\"value\":\"Microsoft\"},{\"name\":\"x_SourceName\",\"value\":\"Cost Management\"},{\"name\":\"x_SourceType\",\"value\":\"', pipeline().parameters.exportDatasetVersion, '\"},{\"name\":\"x_SourceVersion\",\"value\":\"', pipeline().parameters.exportDatasetVersion, '\"}'))), activity('Load Schema Mappings').output.firstRow.additionalColumns)", - "type": "Expression" + "version": { + "type": "string", + "metadata": { + "description": "Required. Version number of the FinOps hub app." } - } - }, - { - "name": "For Each Old File", - "description": "Loop thru each of the existing files from previous exports.", - "type": "ForEach", - "dependsOn": [ - { - "activity": "Convert to Parquet", - "dependencyConditions": [ - "Succeeded" - ] + }, + "features": { + "type": "array", + "items": { + "$ref": "#/definitions/HubAppFeature" }, - { - "activity": "Filter Out Current Exports", - "dependencyConditions": [ - "Succeeded" - ] + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which features the app requires. Allowed values: \"DataFactory\", \"KeyVault\", \"Storage\". Default: [] (none)." } - ], - "userProperties": [], - "typeProperties": { + }, + "storageRoles": { + "type": "array", "items": { - "value": "@activity('Filter Out Current Exports').output.Value", - "type": "Expression" + "type": "string" }, - "activities": [ - { - "name": "Delete Old Ingested File", - "description": "Delete the previously ingested files from older exports.", - "type": "Delete", - "dependsOn": [], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "dataset": { - "referenceName": "[variables('safeIngestionContainerName')]", - "type": "DatasetReference", - "parameters": { - "blobPath": { - "value": "@concat(pipeline().parameters.destinationFolder, '/', item().name)", - "type": "Expression" - } - } - }, - "enableLogging": false, - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": false, - "enablePartitionDiscovery": false - } - } - } - ] - } - }, - { - "name": "Set Destination Path", - "type": "SetVariable", - "dependsOn": [], - "policy": { - "secureOutput": false, - "secureInput": false + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which RBAC roles the Data Factory identity needs on the storage account, if created. This is in addition to Storage Blob Data Contributor for reading and managing content. Default: [] (none)." + } }, - "userProperties": [], - "typeProperties": { - "variableName": "destinationPath", - "value": { - "value": "[format('@concat(pipeline().parameters.destinationFolder, ''/'', pipeline().parameters.ingestionId, ''{0}'', pipeline().parameters.destinationFile)', variables('ingestionIdFileNameSeparator'))]", - "type": "Expression" + "telemetryString": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." } } }, - { - "name": "Convert to Parquet", - "description": "[format('Convert CSV to parquet and move the file to the {0} container.', parameters('ingestionContainerName'))]", - "type": "Switch", - "dependsOn": [ - { - "activity": "Set Destination Path", - "dependencyConditions": [ - "Succeeded" - ] - }, - { - "activity": "Load Schema Mappings", - "dependencyConditions": [ - "Succeeded" + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n<#\r\n.SYNOPSIS\r\nManages Azure Data Factory triggers and pipelines during FinOps hub deployment.\r\n\r\n.DESCRIPTION\r\nThis script is called by Bicep deployment scripts to start/stop Data Factory triggers\r\nand optionally run pipelines. It handles two types of triggers:\r\n\r\n1. Schedule triggers - Can be started/stopped directly\r\n2. BlobEventsTriggers - Require Event Grid subscription management before start/stop\r\n\r\nBlobEventsTriggers use Event Grid to listen for storage blob events. Before stopping,\r\nthe Event Grid subscription must be removed (unsubscribed). Before starting, the\r\nsubscription must be added and fully provisioned. This script handles this automatically\r\nby detecting BlobEventsTriggers via the BlobPathBeginsWith property.\r\n\r\nThe script uses retry logic with linear backoff to handle transient API failures and\r\nwait for Event Grid subscription provisioning/deprovisioning to complete.\r\n\r\n.PARAMETER DataFactoryResourceGroup\r\nThe resource group containing the Data Factory.\r\n\r\n.PARAMETER DataFactoryName\r\nThe name of the Data Factory instance.\r\n\r\n.PARAMETER Pipelines\r\nPipe-delimited list of pipeline names to run (e.g., \"pipeline1|pipeline2\").\r\n\r\n.PARAMETER StartTriggers\r\nSwitch to start all stopped triggers (with Event Grid subscription for blob triggers).\r\n\r\n.PARAMETER StopTriggers\r\nSwitch to stop all running triggers (with Event Grid unsubscription for blob triggers).\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StopTriggers\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StartTriggers\r\n#>\r\n\r\nparam(\r\n [string] $DataFactoryResourceGroup,\r\n [string] $DataFactoryName,\r\n [string] $Pipelines = \"\",\r\n [switch] $StartTriggers,\r\n [switch] $StopTriggers\r\n)\r\n\r\n$MAX_RETRIES = 20\r\n$DeploymentScriptOutputs = @{}\r\n\r\nfunction Write-Log($Message)\r\n{\r\n Write-Output \"$(Get-Date -Format 'HH:mm:ss') $Message\"\r\n}\r\n\r\nfunction Invoke-WithRetry([scriptblock]$Action, [string]$Name, [int]$Delay = 5)\r\n{\r\n for ($i = 1; $i -le $MAX_RETRIES; $i++)\r\n {\r\n try { return & $Action }\r\n catch\r\n {\r\n Write-Log \"$Name failed (attempt $i/${MAX_RETRIES}): $($_.Exception.Message)\"\r\n if ($i -eq $MAX_RETRIES) { throw }\r\n Start-Sleep -Seconds ($Delay * $i)\r\n }\r\n }\r\n}\r\n\r\nfunction Set-BlobTriggerSubscription([string]$TriggerName, [switch]$Subscribe)\r\n{\r\n $targetStatus = if ($Subscribe) { 'Enabled' } else { 'Disabled' }\r\n $action = if ($Subscribe) { 'Subscribing' } else { 'Unsubscribing' }\r\n\r\n Write-Log \"$action $TriggerName to events...\"\r\n Invoke-WithRetry -Name \"$action $TriggerName\" -Delay 5 -Action {\r\n if ($Subscribe)\r\n {\r\n Add-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n else\r\n {\r\n Remove-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n\r\n $status = Get-AzDataFactoryV2TriggerSubscriptionStatus `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName\r\n if ($status.Status -ne $targetStatus)\r\n {\r\n throw \"Subscription status is $($status.Status), expected $targetStatus\"\r\n }\r\n }\r\n}\r\n\r\nif ($StartTriggers -or $StopTriggers)\r\n{\r\n $triggers = Invoke-WithRetry -Name \"Get triggers\" -Action {\r\n Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n | Where-Object {\r\n ($StartTriggers -and $_.Properties.RuntimeState -ne \"Started\") `\r\n -or ($StopTriggers -and $_.Properties.RuntimeState -ne \"Stopped\")\r\n }\r\n }\r\n\r\n Write-Log \"Found $($triggers.Count) trigger(s) to $(if ($StartTriggers) { 'start' } else { 'stop' })\"\r\n\r\n $triggers | ForEach-Object {\r\n $triggerName = $_.Name\r\n $isBlobTrigger = $null -ne $_.Properties.BlobPathBeginsWith\r\n\r\n if ($StopTriggers)\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName }\r\n Write-Log \"Stopping trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Stop $triggerName\" -Action {\r\n Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n else\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName -Subscribe }\r\n Write-Log \"Starting trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Start $triggerName\" -Action {\r\n Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n\r\n Invoke-WithRetry -Name \"Wait for $triggerName\" -Action {\r\n $state = (Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName).Properties.RuntimeState\r\n $expected = if ($StartTriggers) { 'Started' } else { 'Stopped' }\r\n if ($state -ne $expected) { throw \"Trigger is $state, expected $expected\" }\r\n }\r\n\r\n Write-Log \"...done\"\r\n $DeploymentScriptOutputs[$triggerName] = $true\r\n }\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($Pipelines))\r\n{\r\n $Pipelines.Split('|') | ForEach-Object {\r\n Write-Log \"Running pipeline $_...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", + "usesDataFactory": "[contains(parameters('features'), 'DataFactory')]", + "usesKeyVault": "[contains(parameters('features'), 'KeyVault')]", + "usesStorage": "[contains(parameters('features'), 'Storage')]", + "telemetryId": "[format('ftk-hubapp-{0}{1}{2}', parameters('app').id, if(empty(parameters('telemetryString')), '', '_'), parameters('telemetryString'))]", + "telemetryProps": { + "mode": "Incremental", + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "[format('FTK: {0}', parameters('app').id)]", + "version": "[parameters('version')]" + } + }, + "resources": [] + } + }, + "autoStartRbacRoles": [ + "673868aa-7521-48a0-acc6-0f60742d39f5" + ], + "factoryStorageRoles": "[union(parameters('storageRoles'), createArray('17d1049b-9a84-46fb-8f53-869881c3d3ab', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe', 'acdd72a7-3385-48ef-bd42-f606fba81ae7'))]", + "storageInfrastructureEncryptionProperties": "[if(not(parameters('app').hub.options.storageInfrastructureEncryption), createObject(), createObject('encryption', createObject('keySource', 'Microsoft.Storage', 'requireInfrastructureEncryption', parameters('app').hub.options.storageInfrastructureEncryption)))]" + }, + "resources": { + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').storage)]", + "properties": { + "name": "[parameters('app').storage]", + "groupId": "dfs", + "privateLinkResourceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "fqdns": [ + "[reference('storageAccount').primaryEndpoints.dfs]" ] }, - { - "activity": "Set Additional Columns", - "dependencyConditions": [ - "Succeeded" + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "storageAccount" + ] + }, + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').keyVault)]", + "properties": { + "name": "[parameters('app').keyVault]", + "groupId": "vault", + "privateLinkResourceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "fqdns": [ + "[reference('keyVault').vaultUri]" ] - } - ], - "userProperties": [], - "typeProperties": { - "on": { - "value": "@last(array(split(pipeline().parameters.blobPath, '.')))", - "type": "Expression" }, - "cases": [ - { - "value": "csv", - "activities": [ - { - "name": "Convert CSV File", - "type": "Copy", - "dependsOn": [], - "policy": { - "timeout": "0.00:10:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "source": { - "type": "DelimitedTextSource", - "additionalColumns": { - "value": "@variables('additionalColumns')", - "type": "Expression" - }, - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": true, - "enablePartitionDiscovery": false - }, - "formatSettings": { - "type": "DelimitedTextReadSettings" - } - }, - "sink": { - "type": "ParquetSink", - "storeSettings": { - "type": "AzureBlobFSWriteSettings" - }, - "formatSettings": { - "type": "ParquetWriteSettings", - "fileExtension": ".parquet" - } - }, - "enableStaging": false, - "parallelCopies": 1, - "validateDataConsistency": false, - "translator": { - "value": "@activity('Load Schema Mappings').output.firstRow.translator", - "type": "Expression" - } - }, - "inputs": [ - { - "referenceName": "[variables('safeExportContainerName')]", - "type": "DatasetReference", - "parameters": { - "blobPath": { - "value": "@pipeline().parameters.blobPath", - "type": "Expression" - } - } - } - ], - "outputs": [ - { - "referenceName": "[variables('safeIngestionContainerName')]", - "type": "DatasetReference", - "parameters": { - "blobPath": { - "value": "@variables('destinationPath')", - "type": "Expression" - } - } - } - ] - } - ] + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "keyVault" + ] + }, + "dataFactory::managedVirtualNetwork": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'default')]", + "properties": {}, + "dependsOn": [ + "dataFactory" + ] + }, + "dataFactory::managedIntegrationRuntime": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/integrationRuntimes", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ManagedIntegrationRuntime')]", + "properties": { + "type": "Managed", + "managedVirtualNetwork": { + "referenceName": "default", + "type": "ManagedVirtualNetworkReference" }, - { - "value": "gz", - "activities": [ - { - "name": "Convert GZip CSV File", - "type": "Copy", - "dependsOn": [], - "policy": { - "timeout": "0.00:10:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "source": { - "type": "DelimitedTextSource", - "additionalColumns": { - "value": "@variables('additionalColumns')", - "type": "Expression" - }, - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": true, - "enablePartitionDiscovery": false - }, - "formatSettings": { - "type": "DelimitedTextReadSettings" - } - }, - "sink": { - "type": "ParquetSink", - "storeSettings": { - "type": "AzureBlobFSWriteSettings" - }, - "formatSettings": { - "type": "ParquetWriteSettings", - "fileExtension": ".parquet" - } - }, - "enableStaging": false, - "parallelCopies": 1, - "validateDataConsistency": false, - "translator": { - "value": "@activity('Load Schema Mappings').output.firstRow.translator", - "type": "Expression" - } - }, - "inputs": [ - { - "referenceName": "[format('{0}_gzip', variables('safeExportContainerName'))]", - "type": "DatasetReference", - "parameters": { - "blobPath": { - "value": "@pipeline().parameters.blobPath", - "type": "Expression" - } - } - } - ], - "outputs": [ - { - "referenceName": "[variables('safeIngestionContainerName')]", - "type": "DatasetReference", - "parameters": { - "blobPath": { - "value": "@variables('destinationPath')", - "type": "Expression" - } - } - } - ] + "typeProperties": { + "computeProperties": { + "location": "[parameters('app').hub.location]", + "dataFlowProperties": { + "computeType": "General", + "coreCount": 8, + "timeToLive": 10, + "cleanup": false, + "customProperties": [] + }, + "copyComputeScaleProperties": { + "dataIntegrationUnit": 16, + "timeToLive": 30 + }, + "pipelineExternalComputeScaleProperties": { + "timeToLive": 30, + "numberOfPipelineNodes": 1, + "numberOfExternalNodes": 1 } - ] + } + } + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedVirtualNetwork" + ] + }, + "dataFactory::linkedService_keyVault": { + "condition": "[and(variables('usesDataFactory'), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').keyVault)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureKeyVault", + "typeProperties": { + "baseUrl": "[reference(format('Microsoft.KeyVault/vaults/{0}', parameters('app').keyVault), '2023-02-01').vaultUri]" }, - { - "value": "parquet", - "activities": [ - { - "name": "Move Parquet File", - "type": "Copy", - "dependsOn": [], - "policy": { - "timeout": "0.00:05:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "source": { - "type": "ParquetSource", - "additionalColumns": { - "value": "@variables('additionalColumns')", - "type": "Expression" - }, - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": true, - "enablePartitionDiscovery": false - }, - "formatSettings": { - "type": "ParquetReadSettings" - } - }, - "sink": { - "type": "ParquetSink", - "storeSettings": { - "type": "AzureBlobFSWriteSettings" - }, - "formatSettings": { - "type": "ParquetWriteSettings", - "fileExtension": ".parquet" - } - }, - "enableStaging": false, - "parallelCopies": 1, - "validateDataConsistency": false - }, - "inputs": [ - { - "referenceName": "[format('{0}_parquet', variables('safeExportContainerName'))]", - "type": "DatasetReference", - "parameters": { - "blobPath": { - "value": "@pipeline().parameters.blobPath", - "type": "Expression" - } - } - } - ], - "outputs": [ - { - "referenceName": "[variables('safeIngestionContainerName')]", - "type": "DatasetReference", - "parameters": { - "blobPath": { - "value": "@variables('destinationPath')", - "type": "Expression" - } - } - } - ] + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "keyVault" + ] + }, + "dataFactory::linkedService_storageAccount": { + "condition": "[and(variables('usesDataFactory'), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').storage)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureBlobFS", + "typeProperties": { + "url": "[reference(format('Microsoft.Storage/storageAccounts/{0}', parameters('app').storage), '2021-08-01').primaryEndpoints.dfs]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "storageAccount" + ] + }, + "storageAccount::blobService": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]", + "dependsOn": [ + "storageAccount" + ] + }, + "blobEndpoint::blobPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" } - ] - } - ], - "defaultActivities": [ - { - "name": "Unsupported File Type", - "type": "Fail", - "dependsOn": [], - "userProperties": [], - "typeProperties": { - "message": { - "value": "@concat('Unable to ingest the specified export file because the file type is not supported. File: ', pipeline().parameters.blobPath)", - "type": "Expression" - }, - "errorCode": "UnsupportedExportFileType" } - } + ] + }, + "dependsOn": [ + "blobEndpoint" ] - } - }, - { - "name": "Read Hub Config", - "description": "Read the hub config to determine if the export should be retained.", - "type": "Lookup", - "dependsOn": [], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false }, - "userProperties": [], - "typeProperties": { - "source": { - "type": "JsonSource", - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": false, - "enablePartitionDiscovery": false + "dfsEndpoint::dfsPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.dfs.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "dfsEndpoint" + ] + }, + "keyVaultPrivateDnsZone::keyVaultPrivateDnsZoneLink": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), format('{0}-link', replace(format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", + "properties": { + "virtualNetwork": { + "id": "[parameters('app').hub.routing.networkId]" }, - "formatSettings": { - "type": "JsonReadSettings" - } + "registrationEnabled": false }, - "dataset": { - "referenceName": "[variables('safeConfigContainerName')]", - "type": "DatasetReference", - "parameters": { - "fileName": "settings.json", - "folderPath": "[parameters('configContainerName')]" + "dependsOn": [ + "keyVaultPrivateDnsZone" + ] + }, + "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')))]" + } + } + ] + }, + "dependsOn": [ + "keyVaultEndpoint", + "keyVaultPrivateDnsZone" + ] + }, + "appTelemetry": { + "condition": "[parameters('app').hub.options.enableTelemetry]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "[if(lessOrEquals(length(variables('telemetryId')), 64), variables('telemetryId'), substring(variables('telemetryId'), 0, 64))]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Resources/deployments')]", + "properties": "[variables('telemetryProps')]" + }, + "dataFactory": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.DataFactory/factories')]", + "identity": { + "type": "SystemAssigned" + }, + "properties": { + "globalConfigurations": { + "PipelineBillingEnabled": "true" } } - } - }, - { - "name": "If Not Retaining Exports", - "description": "If the msexports retention period <= 0, delete the source file. The main reason to keep the source file is to allow for troubleshooting and reprocessing in the future.", - "type": "IfCondition", - "dependsOn": [ - { - "activity": "Convert to Parquet", - "dependencyConditions": [ - "Succeeded" - ] + }, + "storageRoleAssignments": { + "copy": { + "name": "storageRoleAssignments", + "count": "[length(variables('factoryStorageRoles'))]" }, - { - "activity": "Read Hub Config", - "dependencyConditions": [ - "Completed" + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "name": "[guid(resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage), variables('factoryStorageRoles')[copyIndex()], resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('factoryStorageRoles')[copyIndex()])]", + "principalId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "storageAccount" + ] + }, + "triggerManagerIdentity": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[format('{0}_triggerManager', parameters('app').dataFactory)]", + "location": "[parameters('app').hub.location]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "dependsOn": [ + "dataFactory" + ] + }, + "triggerManagerRoleAssignments": { + "copy": { + "name": "triggerManagerRoleAssignments", + "count": "[length(variables('autoStartRbacRoles'))]" + }, + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]", + "name": "[guid(resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory), variables('autoStartRbacRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', format('{0}_triggerManager', parameters('app').dataFactory)))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('autoStartRbacRoles')[copyIndex()])]", + "principalId": "[reference('triggerManagerIdentity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "triggerManagerIdentity" + ] + }, + "storageAccount": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]", + "location": "[parameters('app').hub.location]", + "sku": { + "name": "[parameters('app').hub.options.storageSku]" + }, + "kind": "BlockBlobStorage", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Storage/storageAccounts')]", + "properties": "[shallowMerge(createArray(variables('storageInfrastructureEncryptionProperties'), createObject('supportsHttpsTrafficOnly', true(), 'allowSharedKeyAccess', true(), 'isHnsEnabled', true(), 'minimumTlsVersion', 'TLS1_2', 'allowBlobPublicAccess', false(), 'publicNetworkAccess', 'Enabled', 'networkAcls', createObject('bypass', 'AzureServices', 'defaultAction', if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')))))]" + }, + "blobPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]" + }, + "blobEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-blob-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "blobLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "blob" + ] + } + } ] - } - ], - "userProperties": [], - "typeProperties": { - "expression": { - "value": "@lessOrEquals(coalesce(activity('Read Hub Config').output.firstRow.retention.msexports.days, 0), 0)", - "type": "Expression" }, - "ifTrueActivities": [ - { - "name": "Delete Source File", - "description": "Delete the exported data file to keep storage costs down. This file is not referenced by any reporting systems.", - "type": "Delete", - "dependsOn": [], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "dataset": { - "referenceName": "[format('{0}_parquet', variables('safeExportContainerName'))]", - "type": "DatasetReference", - "parameters": { - "blobPath": { - "value": "@pipeline().parameters.blobPath", - "type": "Expression" - } - } - }, - "enableLogging": false, - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": true, - "enablePartitionDiscovery": false + "dependsOn": [ + "storageAccount" + ] + }, + "dfsPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]" + }, + "dfsEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-dfs-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "dfsLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "dfs" + ] } } - } + ] + }, + "dependsOn": [ + "storageAccount" ] - } - } - ], - "parameters": { - "blobPath": { - "type": "String" - }, - "destinationFile": { - "type": "string" - }, - "destinationFolder": { - "type": "string" - }, - "ingestionId": { - "type": "string" - }, - "schemaFile": { - "type": "string" - }, - "exportDatasetType": { - "type": "string" - }, - "exportDatasetVersion": { - "type": "string" - } - }, - "variables": { - "additionalColumns": { - "type": "Array" - }, - "destinationPath": { - "type": "String" - } - }, - "annotations": [] - }, - "dependsOn": [ - "dataset_config", - "dataset_ingestion", - "dataset_ingestion_files", - "dataset_msexports", - "dataset_msexports_gzip", - "dataset_msexports_parquet" - ], - "metadata": { - "description": "Transforms CSV data to a standard schema and converts to Parquet." - } - }, - "pipeline_ToDataExplorer": { - "condition": "[or(variables('deployDataExplorer'), variables('useFabric'))]", - "type": "Microsoft.DataFactory/factories/pipelines", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), format('{0}_ETL_dataExplorer', variables('safeIngestionContainerName')))]", - "properties": { - "activities": [ - { - "name": "Read Hub Config", - "description": "Read the hub config to determine how long data should be retained.", - "type": "Lookup", - "dependsOn": [], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false }, - "userProperties": [], - "typeProperties": { - "source": { - "type": "JsonSource", - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "recursive": false, - "enablePartitionDiscovery": false + "keyVault": { + "condition": "[variables('usesKeyVault')]", + "type": "Microsoft.KeyVault/vaults", + "apiVersion": "2023-02-01", + "name": "[parameters('app').keyVault]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.KeyVault/vaults')]", + "properties": { + "sku": { + "name": "[parameters('app').hub.options.keyVaultSku]", + "family": "A" }, - "formatSettings": { - "type": "JsonReadSettings" + "enabledForDeployment": true, + "enabledForTemplateDeployment": true, + "enabledForDiskEncryption": true, + "enableSoftDelete": true, + "softDeleteRetentionInDays": 90, + "enablePurgeProtection": "[if(parameters('app').hub.options.keyVaultEnablePurgeProtection, true(), null())]", + "enableRbacAuthorization": false, + "createMode": "default", + "tenantId": "[subscription().tenantId]", + "accessPolicies": [ + { + "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "tenantId": "[subscription().tenantId]", + "permissions": { + "secrets": [ + "get" + ] + } + } + ], + "networkAcls": { + "bypass": "AzureServices", + "defaultAction": "[if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')]" } }, - "dataset": { - "referenceName": "[variables('safeConfigContainerName')]", - "type": "DatasetReference", - "parameters": { - "fileName": "settings.json", - "folderPath": "[parameters('configContainerName')]" - } - } - } - }, - { - "name": "Set Final Retention Months", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Read Hub Config", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false + "dependsOn": [ + "dataFactory" + ] }, - "userProperties": [], - "typeProperties": { - "variableName": "finalRetentionMonths", - "value": { - "value": "@coalesce(activity('Read Hub Config').output.firstRow.retention.final.months, 999)", - "type": "Expression" - } - } - }, - { - "name": "Until Capacity Is Available", - "type": "Until", - "dependsOn": [ - { - "activity": "Set Final Retention Months", - "dependencyConditions": [ - "Completed", - "Skipped" - ] - } - ], - "userProperties": [], - "typeProperties": { - "expression": { - "value": "@equals(variables('tryAgain'), false)", - "type": "Expression" - }, - "activities": [ - { - "name": "Confirm Ingestion Capacity", - "type": "AzureDataExplorerCommand", - "dependsOn": [], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "command": ".show capacity | where Resource == 'Ingestions' | project Remaining", - "commandTimeout": "00:20:00" - }, - "linkedServiceName": { - "referenceName": "[variables('hubDataExplorerName')]", - "type": "LinkedServiceReference" - } + "keyVaultPrivateDnsZone": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones')]", + "properties": {} + }, + "keyVaultEndpoint": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-ep', parameters('app').keyVault)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.keyVault]" }, - { - "name": "If Has Capacity", - "type": "IfCondition", - "dependsOn": [ - { - "activity": "Confirm Ingestion Capacity", - "dependencyConditions": [ - "Succeeded" + "privateLinkServiceConnections": [ + { + "name": "keyVaultLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "groupIds": [ + "vault" ] } - ], - "userProperties": [], - "typeProperties": { - "expression": { - "value": "@or(equals(activity('Confirm Ingestion Capacity').output.count, 0), greater(activity('Confirm Ingestion Capacity').output.value[0].Remaining, 0))", - "type": "Expression" - }, - "ifFalseActivities": [ - { - "name": "Wait for Ingestion", - "type": "Wait", - "dependsOn": [], - "userProperties": [], - "typeProperties": { - "waitTimeInSeconds": 15 - } - }, - { - "name": "Try Again", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Wait for Ingestion", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "tryAgain", - "value": true - } + } + ] + }, + "dependsOn": [ + "keyVault" + ] + }, + "getKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." } - ], - "ifTrueActivities": [ - { - "name": "Pre-Ingest Cleanup", - "description": "Cost Management exports include all month-to-date data from the previous export run. To ensure data is not double-reported, it must be dropped from the raw table before ingestion completes. Remove previous ingestions into the raw table for the month and any previous runs of the current ingestion month file in any table.", - "type": "AzureDataExplorerCommand", - "dependsOn": [], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "typeProperties": { - "command": { - "value": "@concat('.drop extents <| .show extents | where (TableName == \"', pipeline().parameters.table, '\" and Tags !has \"drop-by:', pipeline().parameters.ingestionId, '\" and Tags has \"drop-by:', pipeline().parameters.folderPath, '\") or (Tags has \"drop-by:', pipeline().parameters.ingestionId, '\" and Tags has \"drop-by:', pipeline().parameters.folderPath, '/', pipeline().parameters.originalFileName, '\")')", - "type": "Expression" - }, - "commandTimeout": "00:20:00" - }, - "linkedServiceName": { - "referenceName": "[variables('hubDataExplorerName')]", - "type": "LinkedServiceReference", - "parameters": { - "database": "[parameters('dataExplorerIngestionDatabase')]" - } - } - }, - { - "name": "Ingest Data", - "type": "AzureDataExplorerCommand", - "dependsOn": [ - { - "activity": "Pre-Ingest Cleanup", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "timeout": "0.12:00:00", - "retry": 3, - "retryIntervalInSeconds": 120, - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "command": { - "value": "[format('@concat(''.ingest into table '', pipeline().parameters.table, '' (\"abfss://{0}@{1}.dfs.{2}/'', pipeline().parameters.folderPath, ''/'', pipeline().parameters.fileName, '';{3}\") with (format=\"parquet\", ingestionMappingReference=\"'', pipeline().parameters.table, ''_mapping\", tags=\"[\\\"drop-by:'', pipeline().parameters.ingestionId, ''\\\", \\\"drop-by:'', pipeline().parameters.folderPath, ''/'', pipeline().parameters.originalFileName, ''\\\", \\\"drop-by:ftk-version-{4}\\\"]\"); print Success = assert(iff(toscalar($command_results | project-keep HasErrors) == false, true, false), \"Ingestion Failed\")'')', parameters('ingestionContainerName'), parameters('storageAccountName'), environment().suffixes.storage, if(variables('useFabric'), 'impersonate', 'managed_identity=system'), variables('ftkVersion'))]", - "type": "Expression" - }, - "commandTimeout": "01:00:00" - }, - "linkedServiceName": { - "referenceName": "[variables('hubDataExplorerName')]", - "type": "LinkedServiceReference", - "parameters": { - "database": "[parameters('dataExplorerIngestionDatabase')]" - } - } - }, - { - "name": "Post-Ingest Cleanup", - "description": "Cost Management exports include all month-to-date data from the previous export run. To ensure data is not double-reported, it must be dropped after ingestion completes. Remove the current ingestion month file from raw and any old ingestions for the month from the final table.", - "type": "AzureDataExplorerCommand", - "dependsOn": [ - { - "activity": "Ingest Data", - "dependencyConditions": [ - "Completed" - ] - } - ], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false - }, - "typeProperties": { - "command": { - "value": "@concat('.drop extents <| .show extents | extend isOldFinalData = (TableName startswith \"', replace(pipeline().parameters.table, '_raw', '_final_v'), '\" and Tags !has \"drop-by:', pipeline().parameters.ingestionId, '\" and Tags has \"drop-by:', pipeline().parameters.folderPath, '\") | extend isPastFinalRetention = (TableName startswith \"', replace(pipeline().parameters.table, '_raw', '_final_v'), '\" and todatetime(substring(strcat(replace_string(extract(\"drop-by:[A-Za-z]+/(\\\\d{4}/\\\\d{2}(/\\\\d{2})?)\", 1, Tags), \"/\", \"-\"), \"-01\"), 0, 10)) < datetime_add(\"month\", -', if(lessOrEquals(variables('finalRetentionMonths'), 0), 0, variables('finalRetentionMonths')), ', startofmonth(now()))) | where isOldFinalData or isPastFinalRetention')", - "type": "Expression" - }, - "commandTimeout": "00:20:00" - }, - "linkedServiceName": { - "referenceName": "[variables('hubDataExplorerName')]", - "type": "LinkedServiceReference", - "parameters": { - "database": "[parameters('dataExplorerIngestionDatabase')]" - } - } - }, - { - "name": "Ingestion Complete", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Post-Ingest Cleanup", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "tryAgain", - "value": false - } - }, - { - "name": "Abort On Ingestion Error", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Ingest Data", - "dependencyConditions": [ - "Failed" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "tryAgain", - "value": false - } - }, - { - "name": "Ingestion Failed Error", - "type": "Fail", - "dependsOn": [ - { - "activity": "Abort On Ingestion Error", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "message": { - "value": "@concat('Data Explorer ingestion into the ', pipeline().parameters.table, ' table failed. Please fix the error and rerun ingestion for the following folder path: \"', pipeline().parameters.folderPath, '\". File: ', pipeline().parameters.originalFileName, '. Error: ', if(greater(length(activity('Ingest Data').output.errors), 0), activity('Ingest Data').output.errors[0].Message, 'Unknown'), ' (Code: ', if(greater(length(activity('Ingest Data').output.errors), 0), activity('Ingest Data').output.errors[0].Code, 'None'), ')')", - "type": "Expression" - }, - "errorCode": "DataExplorerIngestionFailed" - } - }, - { - "name": "Abort On Pre-Ingest Drop Error", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Pre-Ingest Cleanup", - "dependencyConditions": [ - "Failed" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "tryAgain", - "value": false - } - }, - { - "name": "Pre-Ingest Drop Failed Error", - "type": "Fail", - "dependsOn": [ - { - "activity": "Abort On Pre-Ingest Drop Error", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "message": { - "value": "@concat('Data Explorer pre-ingestion cleanup (drop extents from raw table) for the ', pipeline().parameters.table, ' table failed. Ingestion was not completed. Please fix the error and rerun ingestion for the following folder path: \"', pipeline().parameters.folderPath, '\". File: ', pipeline().parameters.originalFileName, '. Error: ', if(greater(length(activity('Pre-Ingest Cleanup').output.errors), 0), activity('Pre-Ingest Cleanup').output.errors[0].Message, 'Unknown'), ' (Code: ', if(greater(length(activity('Pre-Ingest Cleanup').output.errors), 0), activity('Pre-Ingest Cleanup').output.errors[0].Code, 'None'), ')')", - "type": "Expression" - }, - "errorCode": "DataExplorerPreIngestionDropFailed" - } - }, - { - "name": "Abort On Post-Ingest Drop Error", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Post-Ingest Cleanup", - "dependencyConditions": [ - "Failed" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "tryAgain", - "value": false - } - }, - { - "name": "Post-Ingest Drop Failed Error", - "type": "Fail", - "dependsOn": [ - { - "activity": "Abort On Post-Ingest Drop Error", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "message": { - "value": "@concat('Data Explorer post-ingestion cleanup (drop extents from final tables) for the ', replace(pipeline().parameters.table, '_raw', '_final_*'), ' table failed. Please fix the error and rerun ingestion for the following folder path: \"', pipeline().parameters.folderPath, '\". File: ', pipeline().parameters.originalFileName, '. Error: ', if(greater(length(activity('Post-Ingest Cleanup').output.errors), 0), activity('Post-Ingest Cleanup').output.errors[0].Message, 'Unknown'), ' (Code: ', if(greater(length(activity('Post-Ingest Cleanup').output.errors), 0), activity('Post-Ingest Cleanup').output.errors[0].Code, 'None'), ')')", - "type": "Expression" - }, - "errorCode": "DataExplorerPostIngestionDropFailed" + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" } } - ] + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } } } - ], - "timeout": "0.02:00:00" - } - } - ], - "parameters": { - "folderPath": { - "type": "string" - }, - "fileName": { - "type": "string" - }, - "originalFileName": { - "type": "string" - }, - "ingestionId": { - "type": "string" - }, - "table": { - "type": "string" - } - }, - "variables": { - "tryAgain": { - "type": "Boolean", - "defaultValue": true - }, - "logRetentionDays": { - "type": "Integer", - "defaultValue": 0 - }, - "finalRetentionMonths": { - "type": "Integer", - "defaultValue": 999 - } - }, - "annotations": [] - }, - "dependsOn": [ - "dataset_config", - "linkedService_dataExplorer" - ], - "metadata": { - "description": "Ingests parquet data into an Azure Data Explorer cluster." - } - }, - "pipeline_ExecuteIngestionETL": { - "condition": "[or(variables('deployDataExplorer'), variables('useFabric'))]", - "type": "Microsoft.DataFactory/factories/pipelines", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), format('{0}_ExecuteETL', variables('safeIngestionContainerName')))]", - "properties": { - "concurrency": 1, - "activities": [ - { - "name": "Wait", - "description": "Files may not be available immediately after being created.", - "type": "Wait", - "dependsOn": [], - "userProperties": [], - "typeProperties": { - "waitTimeInSeconds": 60 - } - }, - { - "name": "Set Container Folder Path", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Wait", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false - }, - "userProperties": [], - "typeProperties": { - "variableName": "containerFolderPath", - "value": { - "value": "@join(skip(array(split(pipeline().parameters.folderPath, '/')), 1), '/')", - "type": "Expression" - } - } - }, - { - "name": "Get Existing Parquet Files", - "description": "Get the previously ingested files so we can get file paths.", - "type": "GetMetadata", - "dependsOn": [ - { - "activity": "Set Container Folder Path", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint", + "getStoragePrivateEndpointConnections", + "keyVault" + ] }, - "userProperties": [], - "typeProperties": { - "dataset": { - "referenceName": "[format('{0}_files', variables('safeIngestionContainerName'))]", - "type": "DatasetReference", + "approveKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", "parameters": { - "folderPath": "@variables('containerFolderPath')" + "keyVaultName": { + "value": "[parameters('app').keyVault]" + }, + "privateEndpointConnections": { + "value": "[reference('getKeyVaultPrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } } }, - "fieldList": [ - "childItems" - ], - "storeSettings": { - "type": "AzureBlobFSReadSettings", - "enablePartitionDiscovery": false - }, - "formatSettings": { - "type": "ParquetReadSettings" - } - } - }, - { - "name": "Filter Out Folders", - "description": "Remove any folders or manifest files.", - "type": "Filter", - "dependsOn": [ - { - "activity": "Get Existing Parquet Files", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "items": { - "value": "@if(contains(activity('Get Existing Parquet Files').output, 'childItems'), activity('Get Existing Parquet Files').output.childItems, json('[]'))", - "type": "Expression" - }, - "condition": { - "value": "@and(equals(item().type, 'File'), not(contains(toLower(item().name), 'manifest.json')))", - "type": "Expression" - } - } - }, - { - "name": "Set Ingestion Timestamp", - "type": "SetVariable", - "dependsOn": [ - { - "activity": "Wait", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "policy": { - "secureOutput": false, - "secureInput": false + "dependsOn": [ + "getKeyVaultPrivateEndpointConnections", + "keyVault" + ] }, - "userProperties": [], - "typeProperties": { - "variableName": "timestamp", - "value": { - "value": "@utcNow()", - "type": "Expression" - } - } - }, - { - "name": "For Each Old File", - "description": "Loop thru each of the existing files.", - "type": "ForEach", - "dependsOn": [ - { - "activity": "Filter Out Folders", - "dependencyConditions": [ - "Succeeded" - ] + "getStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } }, - { - "activity": "Data Explorer validation", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "batchCount": "[parameters('dataExplorerIngestionCapacity')]", - "items": { - "value": "@activity('Filter Out Folders').output.Value", - "type": "Expression" + "dependsOn": [ + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint", + "stopTriggers", + "storageAccount" + ] + }, + "approveStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + }, + "privateEndpointConnections": { + "value": "[reference('getStoragePrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } }, - "activities": [ - { - "name": "Execute", - "description": "Run the ADX ETL pipeline.", - "type": "ExecutePipeline", - "dependsOn": [], - "policy": { - "secureInput": false + "dependsOn": [ + "getStoragePrivateEndpointConnections", + "storageAccount" + ] + }, + "stopTriggers": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.{1}_ADF.StopTriggers', parameters('app').publisher, parameters('app').name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" }, - "userProperties": [], - "typeProperties": { - "pipeline": { - "referenceName": "[format('{0}_ETL_dataExplorer', variables('safeIngestionContainerName'))]", - "type": "PipelineReference" + "identityName": { + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "arguments": { + "value": "[join(createArray(format('-DataFactoryResourceGroup \"{0}\"', resourceGroup().name), format('-DataFactoryName \"{0}\"', parameters('app').dataFactory), '-StopTriggers'), ' ')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } }, - "waitOnCompletion": true, - "parameters": { - "folderPath": { - "value": "@variables('containerFolderPath')", - "type": "Expression" + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } }, - "fileName": { - "value": "@item().name", - "type": "Expression" + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } }, - "originalFileName": { - "value": "[format('@last(array(split(item().name, ''{0}'')))', variables('ingestionIdFileNameSeparator'))]", - "type": "Expression" + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } }, - "ingestionId": { - "value": "[format('@concat(first(array(split(item().name, ''{0}''))), ''_'', variables(''timestamp''))', variables('ingestionIdFileNameSeparator'))]", - "type": "Expression" + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } }, - "table": { - "value": "@concat(first(array(split(variables('containerFolderPath'), '/'))), '_raw')", - "type": "Expression" + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } } } - } - } - ] - } - }, - { - "name": "If No Files", - "description": "If there are no files found, fail the pipeline.", - "type": "IfCondition", - "dependsOn": [ - { - "activity": "Filter Out Folders", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "expression": { - "value": "@equals(length(activity('Filter Out Folders').output.Value), 0)", - "type": "Expression" - }, - "ifTrueActivities": [ - { - "name": "Files Not Found", - "type": "Fail", - "dependsOn": [], - "userProperties": [], - "typeProperties": { - "message": { - "value": "@concat('Unable to locate parquet files to ingest from the ', pipeline().parameters.folderPath, ' path. Please confirm the folder path is the full path, including the \"ingestion\" container and not starting with or ending with a slash (\"/\").')", - "type": "Expression" - }, - "errorCode": "IngestionFilesNotFound" - } - } - ] - } - }, - { - "name": "Data Explorer validation", - "description": "If Data Explorer is stopped, start it", - "type": "IfCondition", - "dependsOn": [ - { - "activity": "Set Ingestion Timestamp", - "dependencyConditions": [ - "Succeeded" - ] - } - ], - "userProperties": [], - "typeProperties": { - "expression": { - "value": "[format('@equals({0}, true)', variables('deployDataExplorer'))]", - "type": "Expression" - }, - "ifTrueActivities": [ - { - "name": "Start ADX Cluster", - "type": "WebActivity", - "dependsOn": [], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false }, - "userProperties": [], - "typeProperties": { - "method": "POST", - "url": { - "value": "[format('{0}{1}/start?api-version=2024-04-13', environment().resourceManager, resourceId('Microsoft.Kusto/clusters', parameters('dataExplorerName')))]", - "type": "Expression" + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } }, - "body": "{}", - "authentication": { - "type": "MSI", - "resource": { - "value": "[environment().resourceManager]", - "type": "Expression" + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." } - } - } - }, - { - "name": "Error ADX Start", - "type": "Fail", - "dependsOn": [ - { - "activity": "Start ADX Cluster After Error", - "dependencyConditions": [ - "Failed" - ] - } - ], - "userProperties": [], - "typeProperties": { - "message": { - "value": "@concat('Failed to start the Data Explorer instance. Message: ', activity('Start ADX Cluster After Error').output.error.message)", - "type": "Expression" }, - "errorCode": { - "value": "@activity('Start ADX Cluster After Error').output.error.code", - "type": "Expression" - } - } - }, - { - "name": "Wait ADX Provision State", - "type": "Wait", - "dependsOn": [ - { - "activity": "Start ADX Cluster", - "dependencyConditions": [ - "Failed" - ] - } - ], - "userProperties": [], - "typeProperties": { - "waitTimeInSeconds": 600 - } - }, - { - "name": "Start ADX Cluster After Error", - "type": "WebActivity", - "dependsOn": [ - { - "activity": "Wait ADX Provision State", - "dependencyConditions": [ - "Succeeded" - ] + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } } - ], - "policy": { - "timeout": "0.12:00:00", - "retry": 0, - "retryIntervalInSeconds": 30, - "secureOutput": false, - "secureInput": false }, - "userProperties": [], - "typeProperties": { - "method": "POST", - "url": { - "value": "[format('{0}{1}/start?api-version=2024-04-13', environment().resourceManager, resourceId('Microsoft.Kusto/clusters', parameters('dataExplorerName')))]", - "type": "Expression", - "body": "{}" + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" }, - "authentication": { - "type": "MSI", - "resource": { - "value": "[environment().resourceManager]", - "type": "Expression" - } + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] } } } + }, + "dependsOn": [ + "appTelemetry", + "dataFactory", + "triggerManagerIdentity", + "triggerManagerRoleAssignments" ] } + }, + "outputs": { + "dataFactoryId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Data Factory instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]" + }, + "keyVaultId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Key Vault instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]" + }, + "storageAccountId": { + "type": "string", + "metadata": { + "description": "Resource ID of the storage account instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "Principal ID for the managed identity used by Data Factory." + }, + "value": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" + }, + "triggerManagerIdentityName": { + "type": "string", + "metadata": { + "description": "Name of the managed identity used to create and stop ADF triggers." + }, + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + } } - ], + } + } + }, + "ingestion_OpenDataInternalScripts": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_ADX.IngestionOpenDataInternal", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", "parameters": { - "folderPath": { - "type": "string" + "clusterName": { + "value": "[replace(parameters('clusterName'), '_', '-')]" + }, + "databaseName": { + "value": "[variables('INGESTION_DB')]" + }, + "scripts": { + "value": { + "OpenDataFunctions_resource_type_1": "[variables('$fxv#0')]", + "OpenDataFunctions_resource_type_2": "[variables('$fxv#1')]", + "OpenDataFunctions_resource_type_3": "[variables('$fxv#2')]", + "OpenDataFunctions_resource_type_4": "[variables('$fxv#3')]", + "OpenDataFunctions_resource_type_5": "[variables('$fxv#4')]" + } + }, + "continueOnErrors": { + "value": "[parameters('continueOnErrors')]" + }, + "forceUpdateTag": { + "value": "[parameters('forceUpdateTag')]" } }, - "variables": { - "containerFolderPath": { - "type": "string" + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "18008676165167943669" + } + }, + "parameters": { + "clusterName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer instance." + } + }, + "databaseName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer database to create or update." + } + }, + "scripts": { + "type": "object", + "metadata": { + "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." + } + }, + "continueOnErrors": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + } + }, + "forceUpdateTag": { + "type": "string", + "defaultValue": "[utcNow()]", + "metadata": { + "description": "Optional. Forces the table to be updated if different from the last time it was deployed." + } + } + }, + "resources": [ + { + "copy": { + "name": "cluster::database::script", + "count": "[length(items(parameters('scripts')))]" + }, + "type": "Microsoft.Kusto/clusters/databases/scripts", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", + "properties": { + "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", + "continueOnErrors": "[parameters('continueOnErrors')]", + "forceUpdateTag": "[parameters('forceUpdateTag')]" + } + } + ] + } + }, + "dependsOn": [ + "cluster", + "cluster::ingestionDb" + ] + }, + "ingestion_InitScripts": { + "condition": "[variables('useAzure')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_ADX.IngestionInit", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "clusterName": { + "value": "[replace(parameters('clusterName'), '_', '-')]" + }, + "databaseName": { + "value": "[variables('INGESTION_DB')]" + }, + "scripts": { + "value": { + "openData": "[variables('$fxv#5')]", + "common": "[variables('$fxv#6')]", + "infra": "[variables('$fxv#7')]", + "rawTables": "[replace(variables('$fxv#8'), '$$rawRetentionInDays$$', string(parameters('rawRetentionInDays')))]" + } + }, + "continueOnErrors": { + "value": "[parameters('continueOnErrors')]" + }, + "forceUpdateTag": { + "value": "[parameters('forceUpdateTag')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "18008676165167943669" + } + }, + "parameters": { + "clusterName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer instance." + } + }, + "databaseName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer database to create or update." + } + }, + "scripts": { + "type": "object", + "metadata": { + "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." + } + }, + "continueOnErrors": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + } + }, + "forceUpdateTag": { + "type": "string", + "defaultValue": "[utcNow()]", + "metadata": { + "description": "Optional. Forces the table to be updated if different from the last time it was deployed." + } + } }, - "timestamp": { - "type": "string" - } - }, - "annotations": [ - "New ingestion" - ] + "resources": [ + { + "copy": { + "name": "cluster::database::script", + "count": "[length(items(parameters('scripts')))]" + }, + "type": "Microsoft.Kusto/clusters/databases/scripts", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", + "properties": { + "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", + "continueOnErrors": "[parameters('continueOnErrors')]", + "forceUpdateTag": "[parameters('forceUpdateTag')]" + } + } + ] + } }, "dependsOn": [ - "dataset_ingestion_files", - "pipeline_ToDataExplorer" - ], - "metadata": { - "description": "Queues the ingestion_ETL_dataExplorer pipeline to account for Data Factory pipeline trigger limits." - } + "cluster", + "cluster::ingestionDb", + "ingestion_OpenDataInternalScripts" + ] }, - "azuretimezones": { + "ingestion_VersionedScripts": { + "condition": "[variables('useAzure')]", "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "azuretimezones", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_ADX.IngestionVersioned", "properties": { "expressionEvaluationOptions": { "scope": "inner" }, "mode": "Incremental", "parameters": { - "location": { - "value": "[parameters('location')]" + "clusterName": { + "value": "[replace(parameters('clusterName'), '_', '-')]" + }, + "databaseName": { + "value": "[variables('INGESTION_DB')]" + }, + "scripts": { + "value": { + "v1_0": "[variables('$fxv#9')]", + "v1_2": "[variables('$fxv#10')]" + } + }, + "continueOnErrors": { + "value": "[parameters('continueOnErrors')]" + }, + "forceUpdateTag": { + "value": "[parameters('forceUpdateTag')]" } }, "template": { @@ -15843,114 +20146,96 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "4022825617953122148" + "version": "0.40.2.10011", + "templateHash": "18008676165167943669" } }, "parameters": { - "location": { + "clusterName": { "type": "string", - "defaultValue": "[resourceGroup().location]", "metadata": { - "description": "Optional. The location to use for the managed identity and deployment script to auto-start triggers. Default = (resource group location)." + "description": "Required. Name of the FinOps hub Data Explorer instance." } }, - "timezoneobject": { - "type": "object", - "defaultValue": { - "australiaeast": "AUS Eastern Standard Time", - "australiacentral": "AUS Eastern Standard Time", - "australiacentral2": "AUS Eastern Standard Time", - "australiasoutheast": "AUS Eastern Standard Time", - "brazilsouth": "E. South America Standard Time", - "canadacentral": "Central Standard Time", - "canadaeast": "Eastern Standard Time", - "centralindia": "India Standard Time", - "centralus": "Central Standard Time", - "eastasia": "China Standard Time", - "eastus": "Eastern Standard Time", - "eastus2": "Eastern Standard Time", - "francecentral": "W. Europe Standard Time", - "germanynorth": "W. Europe Standard Time", - "germanywestcentral": "W. Europe Standard Time", - "japaneast": "Japan Standard Time", - "japanwest": "Japan Standard Time", - "koreacentral": "Korea Standard Time", - "koreasouth": "Korea Standard Time", - "northcentralus": "Central Standard Time", - "northeurope": "GMT Standard Time", - "norwayeast": "W. Europe Standard Time", - "norwaywest": "W. Europe Standard Time", - "southcentralus": "Central Standard Time", - "southindia": "India Standard Time", - "southeastasia": "Singapore Standard Time", - "switzerlandnorth": "W. Europe Standard Time", - "switzerlandwest": "W. Europe Standard Time", - "uksouth": "GMT Standard Time", - "ukwest": "GMT Standard Time", - "westcentralus": "Central Standard Time", - "westeurope": "W. Europe Standard Time", - "westindia": "India Standard Time", - "westus": "Pacific Standard Time", - "westus2": "Pacific Standard Time" + "databaseName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer database to create or update." } }, - "utchrs": { - "type": "string", - "defaultValue": "[utcNow('hh')]" + "scripts": { + "type": "object", + "metadata": { + "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." + } }, - "utcmins": { - "type": "string", - "defaultValue": "[utcNow('mm')]" + "continueOnErrors": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + } }, - "utcsecs": { + "forceUpdateTag": { "type": "string", - "defaultValue": "[utcNow('ss')]" + "defaultValue": "[utcNow()]", + "metadata": { + "description": "Optional. Forces the table to be updated if different from the last time it was deployed." + } } }, - "variables": { - "loc": "[toLower(replace(parameters('location'), ' ', ''))]", - "timezone": "[coalesce(tryGet(parameters('timezoneobject'), variables('loc')), 'Universal Coordinated Time')]" - }, - "resources": [], - "outputs": { - "AzureRegion": { - "type": "string", - "value": "[parameters('location')]" - }, - "Timezone": { - "type": "string", - "value": "[variables('timezone')]" - }, - "UtcHours": { - "type": "string", - "value": "[parameters('utchrs')]" - }, - "UtcMinutes": { - "type": "string", - "value": "[parameters('utcmins')]" - }, - "UtcSeconds": { - "type": "string", - "value": "[parameters('utcsecs')]" + "resources": [ + { + "copy": { + "name": "cluster::database::script", + "count": "[length(items(parameters('scripts')))]" + }, + "type": "Microsoft.Kusto/clusters/databases/scripts", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", + "properties": { + "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", + "continueOnErrors": "[parameters('continueOnErrors')]", + "forceUpdateTag": "[parameters('forceUpdateTag')]" + } } - } + ] } - } + }, + "dependsOn": [ + "cluster", + "cluster::ingestionDb", + "ingestion_InitScripts" + ] }, - "getStoragePrivateEndpointConnections": { - "condition": "[not(parameters('enablePublicAccess'))]", + "hub_InitScripts": { + "condition": "[variables('useAzure')]", "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "GetStoragePrivateEndpointConnections", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_ADX.HubInit", "properties": { "expressionEvaluationOptions": { "scope": "inner" }, "mode": "Incremental", "parameters": { - "storageAccountName": { - "value": "[parameters('storageAccountName')]" + "clusterName": { + "value": "[replace(parameters('clusterName'), '_', '-')]" + }, + "databaseName": { + "value": "[variables('HUB_DB')]" + }, + "scripts": { + "value": { + "common": "[variables('$fxv#11')]", + "openData": "[variables('$fxv#12')]" + } + }, + "continueOnErrors": { + "value": "[parameters('continueOnErrors')]" + }, + "forceUpdateTag": { + "value": "[parameters('forceUpdateTag')]" } }, "template": { @@ -15959,72 +20244,96 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "491732910990436410" + "version": "0.40.2.10011", + "templateHash": "18008676165167943669" } }, "parameters": { - "privateEndpointConnections": { - "type": "array", - "defaultValue": [], + "clusterName": { + "type": "string", "metadata": { - "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + "description": "Required. Name of the FinOps hub Data Explorer instance." } }, - "storageAccountName": { + "databaseName": { "type": "string", "metadata": { - "description": "Required. Name of the storage account." + "description": "Required. Name of the FinOps hub Data Explorer database to create or update." + } + }, + "scripts": { + "type": "object", + "metadata": { + "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." + } + }, + "continueOnErrors": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + } + }, + "forceUpdateTag": { + "type": "string", + "defaultValue": "[utcNow()]", + "metadata": { + "description": "Optional. Forces the table to be updated if different from the last time it was deployed." } } }, "resources": [ { "copy": { - "name": "privateEndpointConnection", - "count": "[length(parameters('privateEndpointConnections'))]" + "name": "cluster::database::script", + "count": "[length(items(parameters('scripts')))]" }, - "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", - "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", - "apiVersion": "2023-04-01", - "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "type": "Microsoft.Kusto/clusters/databases/scripts", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", "properties": { - "privateLinkServiceConnectionState": { - "status": "Approved", - "description": "Approved-by-pipeline", - "actionRequired": "None" - } + "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", + "continueOnErrors": "[parameters('continueOnErrors')]", + "forceUpdateTag": "[parameters('forceUpdateTag')]" } } - ], - "outputs": { - "privateEndpointConnections": { - "type": "array", - "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" - } - } + ] } }, "dependsOn": [ - "storageManagedPrivateEndpoint" + "cluster", + "cluster::hubDb", + "ingestion_InitScripts" ] }, - "approveStoragePrivateEndpointConnections": { - "condition": "[not(parameters('enablePublicAccess'))]", + "hub_VersionedScripts": { + "condition": "[variables('useAzure')]", "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "ApproveStoragePrivateEndpointConnections", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_ADX.HubVersioned", "properties": { "expressionEvaluationOptions": { "scope": "inner" }, "mode": "Incremental", "parameters": { - "storageAccountName": { - "value": "[parameters('storageAccountName')]" + "clusterName": { + "value": "[replace(parameters('clusterName'), '_', '-')]" }, - "privateEndpointConnections": { - "value": "[reference('getStoragePrivateEndpointConnections').outputs.privateEndpointConnections.value]" + "databaseName": { + "value": "[variables('HUB_DB')]" + }, + "scripts": { + "value": { + "v1_0": "[variables('$fxv#13')]", + "v1_2": "[variables('$fxv#14')]" + } + }, + "continueOnErrors": { + "value": "[parameters('continueOnErrors')]" + }, + "forceUpdateTag": { + "value": "[parameters('forceUpdateTag')]" } }, "template": { @@ -16033,69 +20342,96 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "491732910990436410" + "version": "0.40.2.10011", + "templateHash": "18008676165167943669" } }, "parameters": { - "privateEndpointConnections": { - "type": "array", - "defaultValue": [], + "clusterName": { + "type": "string", "metadata": { - "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + "description": "Required. Name of the FinOps hub Data Explorer instance." } }, - "storageAccountName": { + "databaseName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer database to create or update." + } + }, + "scripts": { + "type": "object", + "metadata": { + "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." + } + }, + "continueOnErrors": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + } + }, + "forceUpdateTag": { "type": "string", + "defaultValue": "[utcNow()]", "metadata": { - "description": "Required. Name of the storage account." + "description": "Optional. Forces the table to be updated if different from the last time it was deployed." } } }, "resources": [ { "copy": { - "name": "privateEndpointConnection", - "count": "[length(parameters('privateEndpointConnections'))]" + "name": "cluster::database::script", + "count": "[length(items(parameters('scripts')))]" }, - "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", - "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", - "apiVersion": "2023-04-01", - "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "type": "Microsoft.Kusto/clusters/databases/scripts", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", "properties": { - "privateLinkServiceConnectionState": { - "status": "Approved", - "description": "Approved-by-pipeline", - "actionRequired": "None" - } + "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", + "continueOnErrors": "[parameters('continueOnErrors')]", + "forceUpdateTag": "[parameters('forceUpdateTag')]" } } - ], - "outputs": { - "privateEndpointConnections": { - "type": "array", - "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" - } - } + ] } }, "dependsOn": [ - "getStoragePrivateEndpointConnections" + "cluster", + "cluster::hubDb", + "hub_InitScripts", + "ingestion_VersionedScripts" ] }, - "getKeyVaultPrivateEndpointConnections": { - "condition": "[and(not(empty(parameters('remoteHubStorageUri'))), not(parameters('enablePublicAccess')))]", + "hub_LatestScripts": { + "condition": "[variables('useAzure')]", "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "GetKeyVaultPrivateEndpointConnections", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_ADX.HubLatest", "properties": { "expressionEvaluationOptions": { "scope": "inner" }, "mode": "Incremental", "parameters": { - "keyVaultName": { - "value": "[parameters('keyVaultName')]" + "clusterName": { + "value": "[replace(parameters('clusterName'), '_', '-')]" + }, + "databaseName": { + "value": "[variables('HUB_DB')]" + }, + "scripts": { + "value": { + "latest": "[variables('$fxv#15')]" + } + }, + "continueOnErrors": { + "value": "[parameters('continueOnErrors')]" + }, + "forceUpdateTag": { + "value": "[parameters('forceUpdateTag')]" } }, "template": { @@ -16104,71 +20440,81 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "11127712826844297340" + "version": "0.40.2.10011", + "templateHash": "18008676165167943669" } }, "parameters": { - "privateEndpointConnections": { - "type": "array", - "defaultValue": [], + "clusterName": { + "type": "string", "metadata": { - "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + "description": "Required. Name of the FinOps hub Data Explorer instance." + } + }, + "databaseName": { + "type": "string", + "metadata": { + "description": "Required. Name of the FinOps hub Data Explorer database to create or update." + } + }, + "scripts": { + "type": "object", + "metadata": { + "description": "Required. List of database scripts to run. The key is the name of the database script and the value is the KQL script content." } }, - "keyVaultName": { + "continueOnErrors": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. If true, ingestion will continue even if some rows fail to ingest. Default: false." + } + }, + "forceUpdateTag": { "type": "string", + "defaultValue": "[utcNow()]", "metadata": { - "description": "Required. Name of the KeyVault." + "description": "Optional. Forces the table to be updated if different from the last time it was deployed." } } }, "resources": [ { "copy": { - "name": "privateEndpointConnection", - "count": "[length(parameters('privateEndpointConnections'))]" + "name": "cluster::database::script", + "count": "[length(items(parameters('scripts')))]" }, - "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", - "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", - "apiVersion": "2023-07-01", - "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "type": "Microsoft.Kusto/clusters/databases/scripts", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}/{2}', parameters('clusterName'), parameters('databaseName'), items(parameters('scripts'))[copyIndex()].key)]", "properties": { - "privateLinkServiceConnectionState": { - "status": "Approved", - "description": "Approved-by-pipeline" - } + "scriptContent": "[items(parameters('scripts'))[copyIndex()].value]", + "continueOnErrors": "[parameters('continueOnErrors')]", + "forceUpdateTag": "[parameters('forceUpdateTag')]" } } - ], - "outputs": { - "privateEndpointConnections": { - "type": "array", - "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" - } - } + ] } }, "dependsOn": [ - "keyVaultManagedPrivateEndpoint" + "cluster", + "cluster::hubDb", + "hub_VersionedScripts" ] }, - "approveKeyVaultPrivateEndpointConnections": { - "condition": "[and(not(empty(parameters('remoteHubStorageUri'))), not(parameters('enablePublicAccess')))]", + "getDataExplorerPrivateEndpointConnections": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "ApproveKeyVaultPrivateEndpointConnections", + "apiVersion": "2025-04-01", + "name": "GetDataExplorerPrivateEndpointConnections", "properties": { "expressionEvaluationOptions": { "scope": "inner" }, "mode": "Incremental", "parameters": { - "keyVaultName": { - "value": "[parameters('keyVaultName')]" - }, - "privateEndpointConnections": { - "value": "[reference('getKeyVaultPrivateEndpointConnections').outputs.privateEndpointConnections.value]" + "dataExplorerName": { + "value": "[replace(parameters('clusterName'), '_', '-')]" } }, "template": { @@ -16177,8 +20523,8 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "11127712826844297340" + "version": "0.40.2.10011", + "templateHash": "17977949049147119573" } }, "parameters": { @@ -16189,10 +20535,10 @@ "description": "Optional. Array of private endpoint connections. Pending ones will be approved." } }, - "keyVaultName": { + "dataExplorerName": { "type": "string", "metadata": { - "description": "Required. Name of the KeyVault." + "description": "Required. Name of the ADX cluster." } } }, @@ -16203,9 +20549,9 @@ "count": "[length(parameters('privateEndpointConnections'))]" }, "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", - "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", - "apiVersion": "2023-07-01", - "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "type": "Microsoft.Kusto/clusters/privateEndpointConnections", + "apiVersion": "2023-08-15", + "name": "[format('{0}/{1}', parameters('dataExplorerName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", "properties": { "privateLinkServiceConnectionState": { "status": "Approved", @@ -16217,20 +20563,21 @@ "outputs": { "privateEndpointConnections": { "type": "array", - "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + "value": "[reference(resourceId('Microsoft.Kusto/clusters', parameters('dataExplorerName')), '2023-08-15').privateEndpointConnections]" } } } }, "dependsOn": [ - "getKeyVaultPrivateEndpointConnections" + "cluster", + "dataFactoryVNet::dataExplorerManagedPrivateEndpoint" ] }, - "getDataExplorerPrivateEndpointConnections": { - "condition": "[and(variables('deployDataExplorer'), not(parameters('enablePublicAccess')))]", + "approveDataExplorerPrivateEndpointConnections": { + "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "GetDataExplorerPrivateEndpointConnections", + "apiVersion": "2025-04-01", + "name": "ApproveDataExplorerPrivateEndpointConnections", "properties": { "expressionEvaluationOptions": { "scope": "inner" @@ -16238,7 +20585,10 @@ "mode": "Incremental", "parameters": { "dataExplorerName": { - "value": "[parameters('dataExplorerName')]" + "value": "[replace(parameters('clusterName'), '_', '-')]" + }, + "privateEndpointConnections": { + "value": "[reference('getDataExplorerPrivateEndpointConnections').outputs.privateEndpointConnections.value]" } }, "template": { @@ -16247,8 +20597,8 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "9394304748737938982" + "version": "0.40.2.10011", + "templateHash": "17977949049147119573" } }, "parameters": { @@ -16293,25 +20643,42 @@ } }, "dependsOn": [ - "dataExplorerManagedPrivateEndpoint" + "cluster", + "getDataExplorerPrivateEndpointConnections" ] }, - "approveDataExplorerPrivateEndpointConnections": { - "condition": "[and(variables('deployDataExplorer'), not(parameters('enablePublicAccess')))]", + "trigger_IngestionManifestAdded": { "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "ApproveDataExplorerPrivateEndpointConnections", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Core_IngestionManifestAddedTrigger", "properties": { "expressionEvaluationOptions": { "scope": "inner" }, "mode": "Incremental", "parameters": { - "dataExplorerName": { - "value": "[parameters('dataExplorerName')]" + "dataFactoryName": { + "value": "[parameters('app').dataFactory]" }, - "privateEndpointConnections": { - "value": "[reference('getDataExplorerPrivateEndpointConnections').outputs.privateEndpointConnections.value]" + "triggerName": { + "value": "[format('{0}_ManifestAdded', variables('INGESTION'))]" + }, + "pipelineName": { + "value": "[format('{0}_ExecuteETL', variables('INGESTION'))]" + }, + "pipelineParameters": { + "value": { + "folderPath": "@triggerBody().folderPath" + } + }, + "storageAccountName": { + "value": "[parameters('app').storage]" + }, + "storageContainer": { + "value": "[variables('INGESTION')]" + }, + "storagePathEndsWith": { + "value": "manifest.json" } }, "template": { @@ -16320,59 +20687,106 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "9394304748737938982" + "version": "0.40.2.10011", + "templateHash": "12667288354191065454" } }, "parameters": { - "privateEndpointConnections": { - "type": "array", - "defaultValue": [], + "dataFactoryName": { + "type": "string", "metadata": { - "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + "description": "Required. Name of the publisher-specific Data Factory instance." } }, - "dataExplorerName": { + "triggerName": { "type": "string", "metadata": { - "description": "Required. Name of the ADX cluster." + "description": "Required. Name of the Data Factory trigger to create or update." + } + }, + "storageAccountName": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Azure storage container to monitor for updates and trigger events for." + } + }, + "storageContainer": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Azure storage container to monitor for updates and trigger events for." + } + }, + "storagePathStartsWith": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Beginning of the storage path within the specified storageContainer to monitor for updates and trigger events for." + } + }, + "storagePathEndsWith": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. End of the storage path to monitor for updates and trigger events for." + } + }, + "pipelineName": { + "type": "string", + "metadata": { + "description": "Required. Name of the Data Factory pipeline to execute when the trigger is executed." + } + }, + "pipelineParameters": { + "type": "object", + "metadata": { + "description": "Required. Parameters to pass to the pipeline when the trigger is executed." } } }, "resources": [ { - "copy": { - "name": "privateEndpointConnection", - "count": "[length(parameters('privateEndpointConnections'))]" - }, - "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", - "type": "Microsoft.Kusto/clusters/privateEndpointConnections", - "apiVersion": "2023-08-15", - "name": "[format('{0}/{1}', parameters('dataExplorerName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "condition": "[not(empty(parameters('storageAccountName')))]", + "type": "Microsoft.DataFactory/factories/triggers", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('dataFactoryName'), parameters('triggerName'))]", "properties": { - "privateLinkServiceConnectionState": { - "status": "Approved", - "description": "Approved-by-pipeline" + "annotations": [], + "pipelines": [ + { + "pipelineReference": { + "referenceName": "[parameters('pipelineName')]", + "type": "PipelineReference" + }, + "parameters": "[parameters('pipelineParameters')]" + } + ], + "type": "BlobEventsTrigger", + "typeProperties": { + "blobPathBeginsWith": "[format('/{0}/blobs/{1}', parameters('storageContainer'), parameters('storagePathStartsWith'))]", + "blobPathEndsWith": "[parameters('storagePathEndsWith')]", + "ignoreEmptyBlobs": true, + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]", + "events": [ + "Microsoft.Storage.BlobCreated" + ] } } } - ], - "outputs": { - "privateEndpointConnections": { - "type": "array", - "value": "[reference(resourceId('Microsoft.Kusto/clusters', parameters('dataExplorerName')), '2023-08-15').privateEndpointConnections]" - } - } + ] } }, "dependsOn": [ - "getDataExplorerPrivateEndpointConnections" + "appRegistration", + "pipeline_ExecuteIngestionETL" ] }, - "deleteOldResources": { + "runInitializationPipeline": { + "condition": "[or(variables('useAzure'), variables('useFabric'))]", "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.FinOpsHubs.Core_ADF.DeleteOldResources", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.Analytics_InitializeHub", "properties": { "expressionEvaluationOptions": { "scope": "inner" @@ -16382,26 +20796,17 @@ "app": { "value": "[parameters('app')]" }, + "dataFactoryInstances": { + "value": [ + "[parameters('app').dataFactory]" + ] + }, "identityName": { - "value": "[format('{0}_triggerManager', parameters('dataFactoryName'))]" + "value": "[reference('appRegistration').outputs.triggerManagerIdentityName.value]" }, - "scriptContent": { - "value": "[variables('$fxv#0')]" - }, - "environmentVariables": { + "startPipelines": { "value": [ - { - "name": "DataFactorySubscriptionId", - "value": "[subscription().id]" - }, - { - "name": "DataFactoryResourceGroup", - "value": "[resourceGroup().name]" - }, - { - "name": "DataFactoryName", - "value": "[parameters('dataFactoryName')]" - } + "[format('{0}_InitializeHub', variables('CONFIG'))]" ] } }, @@ -16412,22 +20817,11 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "588615643779078900" + "version": "0.40.2.10011", + "templateHash": "3919636936819908918" } }, "definitions": { - "EnvironmentVariable": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "value": { - "type": "string" - } - } - }, "_1.HubProperties": { "type": "object", "properties": { @@ -16458,6 +20852,9 @@ "keyVaultSku": { "type": "string" }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, "networkAddressPrefix": { "type": "string" }, @@ -16497,6 +20894,7 @@ "options": { "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", "privateRouting": "Indicates whether private network routing is enabled.", "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", @@ -16546,302 +20944,67 @@ "subnets": { "type": "object", "properties": { - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "scripts": { - "type": "string" - }, - "storage": { - "type": "string" - } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubAppProperties": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { + "dataExplorer": { "type": "string" }, - "suffix": { + "dataFactory": { "type": "string" }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "storage": { - "type": "string" - } - }, - "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", - "dataFactory": "Name of the Data Factory instance for this publisher.", - "keyVault": "Name of the KeyVault instance for this publisher.", - "storage": "Name of the storage account for this publisher.", - "description": "FinOps hub app configuration settings.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - }, - "parameters": { - "app": { - "$ref": "#/definitions/HubAppProperties", - "metadata": { - "description": "Required. FinOps hub app the deployment script is being run for." - } - }, - "identityName": { - "type": "string", - "metadata": { - "description": "Required. Name of the managed identity to create." - } - }, - "scriptName": { - "type": "string", - "defaultValue": "[deployment().name]", - "metadata": { - "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." - } - }, - "scriptContent": { - "type": "string", - "metadata": { - "description": "Required. Name of the deployment script to create." - } - }, - "arguments": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Additional arguments to pass into the deployment script." - } - }, - "environmentVariables": { - "type": "array", - "items": { - "$ref": "#/definitions/EnvironmentVariable" - }, - "defaultValue": [], - "metadata": { - "description": "Optional. Environment variables to use for the deployment script." - } - } - }, - "variables": { - "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", - "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', format('{0}cg', parameters('app').hub.routing.scriptStorage), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" - }, - "resources": { - "identity": { - "type": "Microsoft.ManagedIdentity/userAssignedIdentities", - "apiVersion": "2023-01-31", - "name": "[parameters('identityName')]", - "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", - "location": "[parameters('app').hub.location]" - }, - "scriptStorageAccount": { - "condition": "[parameters('app').hub.options.privateRouting]", - "existing": true, - "type": "Microsoft.Storage/storageAccounts", - "apiVersion": "2022-09-01", - "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" - }, - "identityRoleAssignments": { - "copy": { - "name": "identityRoleAssignments", - "count": "[length(variables('privateEndpointDeploymentRoles'))]" - }, - "condition": "[parameters('app').hub.options.privateRouting]", - "type": "Microsoft.Authorization/roleAssignments", - "apiVersion": "2022-04-01", - "scope": "[format('Microsoft.Storage/storageAccounts/{0}', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", - "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", - "properties": { - "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", - "principalId": "[reference('identity').principalId]", - "principalType": "ServicePrincipal" - }, - "dependsOn": [ - "identity" - ] - }, - "script": { - "type": "Microsoft.Resources/deploymentScripts", - "apiVersion": "2023-08-01", - "name": "[parameters('scriptName')]", - "kind": "AzurePowerShell", - "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", - "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", - "identity": { - "type": "UserAssigned", - "userAssignedIdentities": { - "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} - } - }, - "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '9.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", - "dependsOn": [ - "identity", - "identityRoleAssignments" - ] - } - } - } - }, - "dependsOn": [ - "stopTriggers", - "triggerManagerIdentity", - "triggerManagerRoleAssignments" - ] - }, - "stopTriggers": { - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.FinOpsHubs.Core_ADF.StopTriggers", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "app": { - "value": "[parameters('app')]" - }, - "identityName": { - "value": "[format('{0}_triggerManager', parameters('dataFactoryName'))]" - }, - "scriptContent": { - "value": "[variables('$fxv#1')]" - }, - "arguments": { - "value": "-Stop" - }, - "environmentVariables": { - "value": [ - { - "name": "DataFactorySubscriptionId", - "value": "[subscription().id]" - }, - { - "name": "DataFactoryResourceGroup", - "value": "[resourceGroup().name]" - }, - { - "name": "DataFactoryName", - "value": "[parameters('dataFactoryName')]" + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } }, - { - "name": "Triggers", - "value": "[join(variables('allHubTriggers'), '|')]" + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } } - ] - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "languageVersion": "2.0", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "588615643779078900" - } - }, - "definitions": { - "EnvironmentVariable": { + }, + "_1.IdNameObject": { "type": "object", "properties": { - "name": { + "id": { "type": "string" }, - "value": { + "name": { "type": "string" } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } } }, - "_1.HubProperties": { + "HubAppProperties": { "type": "object", "properties": { "id": { @@ -16850,753 +21013,1005 @@ "name": { "type": "string" }, - "location": { + "publisher": { + "type": "string" + }, + "suffix": { "type": "string" }, "tags": { "type": "object" }, - "tagsByResource": { - "type": "object" + "dataFactory": { + "type": "string" }, - "version": { + "keyVault": { "type": "string" }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" - }, - "keyVaultSku": { - "type": "string" - }, - "networkAddressPrefix": { - "type": "string" + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "dataFactoryInstances": { + "type": "array", + "items": { + "type": "string" + }, + "metadata": { + "description": "Required. List of Azure Data Factory instances to start triggers for. Can be up to 1 per publisher." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to use when starting the triggers." + } + }, + "startAllTriggers": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Start all triggers for the Data Factory instances. Default: false." + } + }, + "startPipelines": { + "type": "array", + "items": { + "type": "string" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. List of pipelines to run. Default: [] (no pipelines)." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n<#\r\n.SYNOPSIS\r\nManages Azure Data Factory triggers and pipelines during FinOps hub deployment.\r\n\r\n.DESCRIPTION\r\nThis script is called by Bicep deployment scripts to start/stop Data Factory triggers\r\nand optionally run pipelines. It handles two types of triggers:\r\n\r\n1. Schedule triggers - Can be started/stopped directly\r\n2. BlobEventsTriggers - Require Event Grid subscription management before start/stop\r\n\r\nBlobEventsTriggers use Event Grid to listen for storage blob events. Before stopping,\r\nthe Event Grid subscription must be removed (unsubscribed). Before starting, the\r\nsubscription must be added and fully provisioned. This script handles this automatically\r\nby detecting BlobEventsTriggers via the BlobPathBeginsWith property.\r\n\r\nThe script uses retry logic with linear backoff to handle transient API failures and\r\nwait for Event Grid subscription provisioning/deprovisioning to complete.\r\n\r\n.PARAMETER DataFactoryResourceGroup\r\nThe resource group containing the Data Factory.\r\n\r\n.PARAMETER DataFactoryName\r\nThe name of the Data Factory instance.\r\n\r\n.PARAMETER Pipelines\r\nPipe-delimited list of pipeline names to run (e.g., \"pipeline1|pipeline2\").\r\n\r\n.PARAMETER StartTriggers\r\nSwitch to start all stopped triggers (with Event Grid subscription for blob triggers).\r\n\r\n.PARAMETER StopTriggers\r\nSwitch to stop all running triggers (with Event Grid unsubscription for blob triggers).\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StopTriggers\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StartTriggers\r\n#>\r\n\r\nparam(\r\n [string] $DataFactoryResourceGroup,\r\n [string] $DataFactoryName,\r\n [string] $Pipelines = \"\",\r\n [switch] $StartTriggers,\r\n [switch] $StopTriggers\r\n)\r\n\r\n$MAX_RETRIES = 20\r\n$DeploymentScriptOutputs = @{}\r\n\r\nfunction Write-Log($Message)\r\n{\r\n Write-Output \"$(Get-Date -Format 'HH:mm:ss') $Message\"\r\n}\r\n\r\nfunction Invoke-WithRetry([scriptblock]$Action, [string]$Name, [int]$Delay = 5)\r\n{\r\n for ($i = 1; $i -le $MAX_RETRIES; $i++)\r\n {\r\n try { return & $Action }\r\n catch\r\n {\r\n Write-Log \"$Name failed (attempt $i/${MAX_RETRIES}): $($_.Exception.Message)\"\r\n if ($i -eq $MAX_RETRIES) { throw }\r\n Start-Sleep -Seconds ($Delay * $i)\r\n }\r\n }\r\n}\r\n\r\nfunction Set-BlobTriggerSubscription([string]$TriggerName, [switch]$Subscribe)\r\n{\r\n $targetStatus = if ($Subscribe) { 'Enabled' } else { 'Disabled' }\r\n $action = if ($Subscribe) { 'Subscribing' } else { 'Unsubscribing' }\r\n\r\n Write-Log \"$action $TriggerName to events...\"\r\n Invoke-WithRetry -Name \"$action $TriggerName\" -Delay 5 -Action {\r\n if ($Subscribe)\r\n {\r\n Add-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n else\r\n {\r\n Remove-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n\r\n $status = Get-AzDataFactoryV2TriggerSubscriptionStatus `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName\r\n if ($status.Status -ne $targetStatus)\r\n {\r\n throw \"Subscription status is $($status.Status), expected $targetStatus\"\r\n }\r\n }\r\n}\r\n\r\nif ($StartTriggers -or $StopTriggers)\r\n{\r\n $triggers = Invoke-WithRetry -Name \"Get triggers\" -Action {\r\n Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n | Where-Object {\r\n ($StartTriggers -and $_.Properties.RuntimeState -ne \"Started\") `\r\n -or ($StopTriggers -and $_.Properties.RuntimeState -ne \"Stopped\")\r\n }\r\n }\r\n\r\n Write-Log \"Found $($triggers.Count) trigger(s) to $(if ($StartTriggers) { 'start' } else { 'stop' })\"\r\n\r\n $triggers | ForEach-Object {\r\n $triggerName = $_.Name\r\n $isBlobTrigger = $null -ne $_.Properties.BlobPathBeginsWith\r\n\r\n if ($StopTriggers)\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName }\r\n Write-Log \"Stopping trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Stop $triggerName\" -Action {\r\n Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n else\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName -Subscribe }\r\n Write-Log \"Starting trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Start $triggerName\" -Action {\r\n Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n\r\n Invoke-WithRetry -Name \"Wait for $triggerName\" -Action {\r\n $state = (Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName).Properties.RuntimeState\r\n $expected = if ($StartTriggers) { 'Started' } else { 'Stopped' }\r\n if ($state -ne $expected) { throw \"Trigger is $state, expected $expected\" }\r\n }\r\n\r\n Write-Log \"...done\"\r\n $DeploymentScriptOutputs[$triggerName] = $true\r\n }\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($Pipelines))\r\n{\r\n $Pipelines.Split('|') | ForEach-Object {\r\n Write-Log \"Running pipeline $_...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", + "uniqueInstances": "[union(filter(parameters('dataFactoryInstances'), lambda('adf', not(empty(lambdaVariables('adf'))))), createArray())]" + }, + "resources": { + "initialize": { + "copy": { + "name": "initialize", + "count": "[length(variables('uniqueInstances'))]" + }, + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[if(lessOrEquals(length(format('Microsoft.FinOpsHubs.Init_{0}', variables('uniqueInstances')[copyIndex()])), 64), format('Microsoft.FinOpsHubs.Init_{0}', variables('uniqueInstances')[copyIndex()]), substring(format('Microsoft.FinOpsHubs.Init_{0}', variables('uniqueInstances')[copyIndex()]), 0, 64))]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[parameters('identityName')]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "arguments": { + "value": "[join(filter(createArray(format('-DataFactoryResourceGroup \"{0}\"', resourceGroup().name), format('-DataFactoryName \"{0}\"', variables('uniqueInstances')[copyIndex()]), if(not(empty(parameters('startPipelines'))), format('-Pipelines \"{0}\"', join(parameters('startPipelines'), '|')), ''), if(parameters('startAllTriggers'), '-StartTriggers', '')), lambda('arg', not(empty(lambdaVariables('arg'))))), ' ')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } }, - "privateRouting": { - "type": "bool" + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } }, - "publisherIsolation": { - "type": "bool" + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } }, - "storageInfrastructureEncryption": { - "type": "bool" + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } }, - "storageSku": { - "type": "string" - } - } - }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" - }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } } - } - } - }, - "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", - "location": "Azure resource location of the FinOps hub instance.", - "tags": "Tags to apply to all FinOps hub resources.", - "tagsByResource": "Tags to apply to resources based on their resource type.", - "version": "FinOps hub version number.", - "options": { - "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", - "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", - "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", - "privateRouting": "Indicates whether private network routing is enabled.", - "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", - "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", - "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." - }, - "routing": "FinOps hub private network routing properties, if enabled.", - "core": { - "suffix": "Unique suffix used for shared resources." - }, - "apps": {}, - "description": "FinOps hub instance properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.HubRoutingProperties": { - "type": "object", - "properties": { - "networkId": { - "type": "string" - }, - "networkName": { - "type": "string" - }, - "scriptStorage": { - "type": "string" - }, - "dnsZones": { - "type": "object", - "properties": { - "blob": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "dfs": { - "$ref": "#/definitions/_1.IdNameObject" + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } }, - "queue": { - "$ref": "#/definitions/_1.IdNameObject" + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } }, - "table": { - "$ref": "#/definitions/_1.IdNameObject" - } - } - }, - "subnets": { - "type": "object", - "properties": { - "dataFactory": { - "type": "string" + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } }, - "keyVault": { - "type": "string" + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } }, - "scripts": { - "type": "string" + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } }, - "storage": { - "type": "string" + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubAppProperties": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" }, - "displayName": { - "type": "string" + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" }, - "suffix": { - "type": "string" + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] }, - "tags": { - "type": "object" + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] } } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "storage": { - "type": "string" } - }, - "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", - "dataFactory": "Name of the Data Factory instance for this publisher.", - "keyVault": "Name of the KeyVault instance for this publisher.", - "storage": "Name of the storage account for this publisher.", - "description": "FinOps hub app configuration settings.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - }, - "parameters": { - "app": { - "$ref": "#/definitions/HubAppProperties", - "metadata": { - "description": "Required. FinOps hub app the deployment script is being run for." - } - }, - "identityName": { - "type": "string", - "metadata": { - "description": "Required. Name of the managed identity to create." - } - }, - "scriptName": { - "type": "string", - "defaultValue": "[deployment().name]", - "metadata": { - "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." - } - }, - "scriptContent": { - "type": "string", - "metadata": { - "description": "Required. Name of the deployment script to create." - } - }, - "arguments": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Additional arguments to pass into the deployment script." - } - }, - "environmentVariables": { - "type": "array", - "items": { - "$ref": "#/definitions/EnvironmentVariable" - }, - "defaultValue": [], - "metadata": { - "description": "Optional. Environment variables to use for the deployment script." } } - }, - "variables": { - "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", - "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', format('{0}cg', parameters('app').hub.routing.scriptStorage), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" - }, - "resources": { - "identity": { - "type": "Microsoft.ManagedIdentity/userAssignedIdentities", - "apiVersion": "2023-01-31", - "name": "[parameters('identityName')]", - "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", - "location": "[parameters('app').hub.location]" - }, - "scriptStorageAccount": { - "condition": "[parameters('app').hub.options.privateRouting]", - "existing": true, - "type": "Microsoft.Storage/storageAccounts", - "apiVersion": "2022-09-01", - "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" - }, - "identityRoleAssignments": { - "copy": { - "name": "identityRoleAssignments", - "count": "[length(variables('privateEndpointDeploymentRoles'))]" - }, - "condition": "[parameters('app').hub.options.privateRouting]", - "type": "Microsoft.Authorization/roleAssignments", - "apiVersion": "2022-04-01", - "scope": "[format('Microsoft.Storage/storageAccounts/{0}', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", - "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", - "properties": { - "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", - "principalId": "[reference('identity').principalId]", - "principalType": "ServicePrincipal" - }, - "dependsOn": [ - "identity" - ] - }, - "script": { - "type": "Microsoft.Resources/deploymentScripts", - "apiVersion": "2023-08-01", - "name": "[parameters('scriptName')]", - "kind": "AzurePowerShell", - "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", - "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", - "identity": { - "type": "UserAssigned", - "userAssignedIdentities": { - "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} - } - }, - "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '9.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", - "dependsOn": [ - "identity", - "identityRoleAssignments" - ] - } } } }, - "dependsOn": [ - "triggerManagerIdentity", - "triggerManagerRoleAssignments" - ] + "dependsOn": [ + "appRegistration", + "ingestion_InitScripts", + "ingestion_OpenDataInternalScripts", + "ingestion_VersionedScripts", + "pipeline_InitializeHub" + ] + } + }, + "outputs": { + "clusterId": { + "type": "string", + "metadata": { + "description": "The resource ID of the cluster." + }, + "value": "[if(variables('useFabric'), '', resourceId('Microsoft.Kusto/clusters', replace(parameters('clusterName'), '_', '-')))]" + }, + "principalId": { + "type": "string", + "metadata": { + "description": "The ID of the cluster system assigned managed identity." + }, + "value": "[if(variables('useFabric'), '', reference('cluster', '2023-08-15', 'full').identity.principalId)]" }, - "trigger_ExportManifestAdded": { - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.FinOpsHubs.Core_ExportManifestAddedTrigger", + "clusterName": { + "type": "string", + "metadata": { + "description": "The name of the cluster." + }, + "value": "[if(variables('useFabric'), '', replace(parameters('clusterName'), '_', '-'))]" + }, + "clusterUri": { + "type": "string", + "metadata": { + "description": "The URI of the cluster." + }, + "value": "[variables('dataExplorerUri')]" + }, + "ingestionDbName": { + "type": "string", + "metadata": { + "description": "The name of the database for data ingestion." + }, + "value": "[variables('INGESTION_DB')]" + }, + "hubDbName": { + "type": "string", + "metadata": { + "description": "The name of the database for queries." + }, + "value": "[variables('HUB_DB')]" + }, + "clusterIngestionCapacity": { + "type": "int", + "metadata": { + "description": "Max ingestion capacity of the cluster." + }, + "value": "[variables('dataExplorerIngestionCapacity')]" + } + } + } + }, + "dependsOn": [ + "cmExports", + "core", + "deleteOldResources" + ] + }, + "remoteHub": { + "condition": "[not(empty(parameters('remoteHubStorageKey')))]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.RemoteHub", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[__bicep.newApp(variables('hub'), 'Microsoft.FinOpsHubs', 'RemoteHub')]" + }, + "remoteStorageKey": { + "value": "[parameters('remoteHubStorageKey')]" + }, + "remoteHubStorageUri": { + "value": "[parameters('remoteHubStorageUri')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6617373141576044697" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", "properties": { - "expressionEvaluationOptions": { - "scope": "inner" + "id": { + "type": "string" }, - "mode": "Incremental", - "parameters": { - "dataFactoryName": { - "value": "[parameters('dataFactoryName')]" - }, - "triggerName": { - "value": "[variables('exportManifestAddedTriggerName')]" - }, - "pipelineName": { - "value": "[format('{0}_ExecuteETL', variables('safeExportContainerName'))]" - }, - "pipelineParameters": { - "value": { - "folderPath": "@triggerBody().folderPath", - "fileName": "@triggerBody().fileName" - } - }, - "storageAccountName": { - "value": "[parameters('storageAccountName')]" - }, - "storageContainer": { - "value": "[parameters('exportContainerName')]" - }, - "storagePathEndsWith": { - "value": "manifest.json" - } + "name": { + "type": "string" }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "10717799137710795976" - } - }, - "parameters": { - "dataFactoryName": { - "type": "string", - "metadata": { - "description": "Required. Name of the publisher-specific Data Factory instance." - } + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" }, - "triggerName": { - "type": "string", - "metadata": { - "description": "Required. Name of the Data Factory trigger to create or update." - } + "keyVaultSku": { + "type": "string" }, - "storageAccountName": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Azure storage container to monitor for updates and trigger events for." - } + "keyVaultEnablePurgeProtection": { + "type": "bool" }, - "storageContainer": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Azure storage container to monitor for updates and trigger events for." - } + "networkAddressPrefix": { + "type": "string" }, - "storagePathStartsWith": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Beginning of the storage path within the specified storageContainer to monitor for updates and trigger events for." - } + "privateRouting": { + "type": "bool" }, - "storagePathEndsWith": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. End of the storage path to monitor for updates and trigger events for." - } + "publisherIsolation": { + "type": "bool" }, - "pipelineName": { - "type": "string", - "metadata": { - "description": "Required. Name of the Data Factory pipeline to execute when the trigger is executed." - } + "storageInfrastructureEncryption": { + "type": "bool" }, - "pipelineParameters": { - "type": "object", - "metadata": { - "description": "Required. Parameters to pass to the pipeline when the trigger is executed." - } + "storageSku": { + "type": "string" } - }, - "resources": [ - { - "condition": "[not(empty(parameters('storageAccountName')))]", - "type": "Microsoft.DataFactory/factories/triggers", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), parameters('triggerName'))]", - "properties": { - "annotations": [], - "pipelines": [ - { - "pipelineReference": { - "referenceName": "[parameters('pipelineName')]", - "type": "PipelineReference" - }, - "parameters": "[parameters('pipelineParameters')]" - } - ], - "type": "BlobEventsTrigger", - "typeProperties": { - "blobPathBeginsWith": "[format('/{0}/blobs/{1}', parameters('storageContainer'), parameters('storagePathStartsWith'))]", - "blobPathEndsWith": "[parameters('storagePathEndsWith')]", - "ignoreEmptyBlobs": true, - "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]", - "events": [ - "Microsoft.Storage.BlobCreated" - ] - } - } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" } - ] + } } }, - "dependsOn": [ - "pipeline_ExecuteExportsETL", - "stopTriggers" - ] - }, - "trigger_IngestionManifestAdded": { - "condition": "[or(variables('deployDataExplorer'), variables('useFabric'))]", - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.FinOpsHubs.Core_IngestionManifestAddedTrigger", + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", "properties": { - "expressionEvaluationOptions": { - "scope": "inner" + "networkId": { + "type": "string" }, - "mode": "Incremental", - "parameters": { - "dataFactoryName": { - "value": "[parameters('dataFactoryName')]" - }, - "triggerName": { - "value": "[variables('ingestionManifestAddedTriggerName')]" - }, - "pipelineName": { - "value": "[format('{0}_ExecuteETL', variables('safeIngestionContainerName'))]" - }, - "pipelineParameters": { - "value": { - "folderPath": "@triggerBody().folderPath" - } - }, - "storageAccountName": { - "value": "[parameters('storageAccountName')]" - }, - "storageContainer": { - "value": "[parameters('ingestionContainerName')]" - }, - "storagePathEndsWith": { - "value": "manifest.json" - } + "networkName": { + "type": "string" }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "10717799137710795976" - } - }, - "parameters": { - "dataFactoryName": { - "type": "string", - "metadata": { - "description": "Required. Name of the publisher-specific Data Factory instance." - } + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" }, - "triggerName": { - "type": "string", - "metadata": { - "description": "Required. Name of the Data Factory trigger to create or update." - } + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" }, - "storageAccountName": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Azure storage container to monitor for updates and trigger events for." - } + "queue": { + "$ref": "#/definitions/_1.IdNameObject" }, - "storageContainer": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Azure storage container to monitor for updates and trigger events for." - } + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" }, - "storagePathStartsWith": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Beginning of the storage path within the specified storageContainer to monitor for updates and trigger events for." - } + "dataFactory": { + "type": "string" }, - "storagePathEndsWith": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. End of the storage path to monitor for updates and trigger events for." - } + "keyVault": { + "type": "string" }, - "pipelineName": { - "type": "string", - "metadata": { - "description": "Required. Name of the Data Factory pipeline to execute when the trigger is executed." - } + "scripts": { + "type": "string" }, - "pipelineParameters": { - "type": "object", - "metadata": { - "description": "Required. Parameters to pass to the pipeline when the trigger is executed." - } + "storage": { + "type": "string" } - }, - "resources": [ + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "privateRoutingForLinkedServices": { + "parameters": [ { - "condition": "[not(empty(parameters('storageAccountName')))]", - "type": "Microsoft.DataFactory/factories/triggers", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), parameters('triggerName'))]", - "properties": { - "annotations": [], - "pipelines": [ - { - "pipelineReference": { - "referenceName": "[parameters('pipelineName')]", - "type": "PipelineReference" - }, - "parameters": "[parameters('pipelineParameters')]" - } - ], - "type": "BlobEventsTrigger", - "typeProperties": { - "blobPathBeginsWith": "[format('/{0}/blobs/{1}', parameters('storageContainer'), parameters('storagePathStartsWith'))]", - "blobPathEndsWith": "[parameters('storagePathEndsWith')]", - "ignoreEmptyBlobs": true, - "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]", - "events": [ - "Microsoft.Storage.BlobCreated" - ] - } - } + "$ref": "#/definitions/_1.HubProperties", + "name": "hub" } - ] + ], + "output": { + "type": "object", + "value": "[if(parameters('hub').options.privateRouting, createObject('connectVia', createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference')), createObject())]" + }, + "metadata": { + "description": "Returns an object that represents the properties needed to enable private routing for linked services. Use property expansion (`...value`) to apply to a linkedServices resource.", + "__bicep_imported_from!": { + "sourceTemplate": "../../fx/hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "remoteStorageKey": { + "type": "securestring", + "metadata": { + "description": "Required. Create and store a key for a remote storage account." + } + }, + "remoteHubStorageUri": { + "type": "string", + "metadata": { + "description": "Required. Remote storage account for ingestion dataset." + } + }, + "ingestionContainerName": { + "type": "string", + "defaultValue": "ingestion", + "metadata": { + "description": "Optional. Name of the ingestion container. Default: ingestion." + } + } + }, + "variables": { + "storageKeySecretName": "[format('{0}-storage-key', toLower(parameters('app').hub.name))]", + "finOpsToolkitVersion": "13.0" + }, + "resources": { + "dataFactory::linkedService_remoteHubStorage": { + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'remoteHubStorage')]", + "properties": "[shallowMerge(createArray(createObject('annotations', createArray(), 'parameters', createObject(), 'type', 'AzureBlobFS', 'typeProperties', createObject('url', parameters('remoteHubStorageUri'), 'accountKey', createObject('type', 'AzureKeyVaultSecret', 'store', createObject('referenceName', parameters('app').keyVault, 'type', 'LinkedServiceReference'), 'secretName', variables('storageKeySecretName')))), __bicep.privateRoutingForLinkedServices(parameters('app').hub)))]", + "dependsOn": [ + "appRegistration" + ] + }, + "dataFactory::dataset_ingestion": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('ingestionContainerName'))]", + "properties": { + "annotations": [], + "parameters": { + "blobPath": { + "type": "String" + } + }, + "type": "Parquet", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().blobPath}", + "type": "Expression" + }, + "fileSystem": "[parameters('ingestionContainerName')]" + } + }, + "linkedServiceName": { + "parameters": {}, + "referenceName": "remoteHubStorage", + "type": "LinkedServiceReference" } }, "dependsOn": [ - "pipeline_ExecuteIngestionETL", - "stopTriggers" + "appRegistration", + "dataFactory::linkedService_remoteHubStorage" ] }, - "trigger_SettingsUpdated": { - "condition": "[parameters('enableManagedExports')]", - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.FinOpsHubs.Core_SettingsUpdatedTrigger", + "dataFactory::dataset_ingestion_files": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, format('{0}_files', parameters('ingestionContainerName')))]", "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", + "annotations": [], "parameters": { - "dataFactoryName": { - "value": "[parameters('dataFactoryName')]" - }, - "triggerName": { - "value": "[variables('updateConfigTriggerName')]" - }, - "pipelineName": { - "value": "[format('{0}_ConfigureExports', variables('safeConfigContainerName'))]" - }, - "pipelineParameters": { - "value": {} - }, - "storageAccountName": { - "value": "[parameters('storageAccountName')]" - }, - "storageContainer": { - "value": "[parameters('configContainerName')]" - }, - "storagePathEndsWith": { - "value": "settings.json" + "folderPath": { + "type": "String" } }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "10717799137710795976" + "type": "Parquet", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileSystem": "[parameters('ingestionContainerName')]", + "folderPath": { + "value": "@dataset().folderPath", + "type": "Expression" } + } + }, + "linkedServiceName": { + "parameters": {}, + "referenceName": "remoteHubStorage", + "type": "LinkedServiceReference" + } + }, + "dependsOn": [ + "appRegistration", + "dataFactory::linkedService_remoteHubStorage" + ] + }, + "dataFactory::dataset_ingestion_manifest": { + "type": "Microsoft.DataFactory/factories/datasets", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ingestion_manifest')]", + "properties": { + "annotations": [], + "parameters": { + "fileName": { + "type": "String", + "defaultValue": "manifest.json" }, - "parameters": { - "dataFactoryName": { - "type": "string", - "metadata": { - "description": "Required. Name of the publisher-specific Data Factory instance." - } - }, - "triggerName": { - "type": "string", - "metadata": { - "description": "Required. Name of the Data Factory trigger to create or update." - } - }, - "storageAccountName": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Azure storage container to monitor for updates and trigger events for." - } - }, - "storageContainer": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Azure storage container to monitor for updates and trigger events for." - } - }, - "storagePathStartsWith": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Beginning of the storage path within the specified storageContainer to monitor for updates and trigger events for." - } - }, - "storagePathEndsWith": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. End of the storage path to monitor for updates and trigger events for." - } - }, - "pipelineName": { - "type": "string", - "metadata": { - "description": "Required. Name of the Data Factory pipeline to execute when the trigger is executed." - } + "folderPath": { + "type": "String", + "defaultValue": "[parameters('ingestionContainerName')]" + } + }, + "type": "Json", + "typeProperties": { + "location": { + "type": "AzureBlobFSLocation", + "fileName": { + "value": "@{dataset().fileName}", + "type": "Expression" }, - "pipelineParameters": { - "type": "object", - "metadata": { - "description": "Required. Parameters to pass to the pipeline when the trigger is executed." - } - } - }, - "resources": [ - { - "condition": "[not(empty(parameters('storageAccountName')))]", - "type": "Microsoft.DataFactory/factories/triggers", - "apiVersion": "2018-06-01", - "name": "[format('{0}/{1}', parameters('dataFactoryName'), parameters('triggerName'))]", - "properties": { - "annotations": [], - "pipelines": [ - { - "pipelineReference": { - "referenceName": "[parameters('pipelineName')]", - "type": "PipelineReference" - }, - "parameters": "[parameters('pipelineParameters')]" - } - ], - "type": "BlobEventsTrigger", - "typeProperties": { - "blobPathBeginsWith": "[format('/{0}/blobs/{1}', parameters('storageContainer'), parameters('storagePathStartsWith'))]", - "blobPathEndsWith": "[parameters('storagePathEndsWith')]", - "ignoreEmptyBlobs": true, - "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]", - "events": [ - "Microsoft.Storage.BlobCreated" - ] - } - } + "folderPath": { + "value": "@{dataset().folderPath}", + "type": "Expression" } - ] + } + }, + "linkedServiceName": { + "parameters": {}, + "referenceName": "remoteHubStorage", + "type": "LinkedServiceReference" } }, "dependsOn": [ - "pipeline_ConfigureExports", - "stopTriggers" + "appRegistration", + "dataFactory::linkedService_remoteHubStorage" + ] + }, + "keyVault": { + "existing": true, + "type": "Microsoft.KeyVault/vaults", + "apiVersion": "2023-02-01", + "name": "[parameters('app').keyVault]", + "dependsOn": [ + "appRegistration" + ] + }, + "dataFactory": { + "existing": true, + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "dependsOn": [ + "appRegistration" ] }, - "startTriggers": { + "appRegistration": { "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.FinOpsHubs.Core_ADF.StartTriggers", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.RemoteHub_Register", "properties": { "expressionEvaluationOptions": { "scope": "inner" @@ -17606,34 +22021,14 @@ "app": { "value": "[parameters('app')]" }, - "identityName": { - "value": "[format('{0}_triggerManager', parameters('dataFactoryName'))]" - }, - "scriptContent": { - "value": "[variables('$fxv#2')]" + "version": { + "value": "[variables('finOpsToolkitVersion')]" }, - "environmentVariables": { + "features": { "value": [ - { - "name": "DataFactorySubscriptionId", - "value": "[subscription().id]" - }, - { - "name": "DataFactoryResourceGroup", - "value": "[resourceGroup().name]" - }, - { - "name": "DataFactoryName", - "value": "[parameters('dataFactoryName')]" - }, - { - "name": "Triggers", - "value": "[join(variables('allHubTriggers'), '|')]" - }, - { - "name": "Pipelines", - "value": "[join(createArray(format('{0}_InitializeHub', variables('safeConfigContainerName'))), '|')]" - } + "DataFactory", + "KeyVault", + "Storage" ] } }, @@ -17641,25 +22036,14 @@ "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", "languageVersion": "2.0", "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "588615643779078900" - } - }, - "definitions": { - "EnvironmentVariable": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "value": { - "type": "string" - } - } - }, + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "8267413868206701537" + } + }, + "definitions": { "_1.HubProperties": { "type": "object", "properties": { @@ -17690,6 +22074,9 @@ "keyVaultSku": { "type": "string" }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, "networkAddressPrefix": { "type": "string" }, @@ -17729,6 +22116,7 @@ "options": { "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", "privateRouting": "Indicates whether private network routing is enabled.", "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", @@ -17778,6 +22166,9 @@ "subnets": { "type": "object", "properties": { + "dataExplorer": { + "type": "string" + }, "dataFactory": { "type": "string" }, @@ -17804,6 +22195,7 @@ "table": "Resource ID and name for the table storage DNS zone." }, "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", "dataFactory": "Resource ID of the subnet for Data Factory instances.", "keyVault": "Resource ID of the subnet for Key Vault instances.", "scripts": "Resource ID of the subnet for deployment script storage.", @@ -17815,1421 +22207,2568 @@ } } }, - "_1.IdNameObject": { - "type": "object", + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppFeature": { + "type": "string", + "allowedValues": [ + "DataFactory", + "KeyVault", + "Storage" + ], + "metadata": { + "description": "FinOps hub app features.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "functions": [ + { + "namespace": "__bicep", + "members": { + "getAppPublisherTags": { + "parameters": [ + { + "$ref": "#/definitions/HubAppProperties", + "name": "app" + }, + { + "type": "string", + "name": "resourceType" + } + ], + "output": { + "type": "object", + "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" + }, + "metadata": { + "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + } + } + ], + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "version": { + "type": "string", + "metadata": { + "description": "Required. Version number of the FinOps hub app." + } + }, + "features": { + "type": "array", + "items": { + "$ref": "#/definitions/HubAppFeature" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which features the app requires. Allowed values: \"DataFactory\", \"KeyVault\", \"Storage\". Default: [] (none)." + } + }, + "storageRoles": { + "type": "array", + "items": { + "type": "string" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Indicate which RBAC roles the Data Factory identity needs on the storage account, if created. This is in addition to Storage Blob Data Contributor for reading and managing content. Default: [] (none)." + } + }, + "telemetryString": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n<#\r\n.SYNOPSIS\r\nManages Azure Data Factory triggers and pipelines during FinOps hub deployment.\r\n\r\n.DESCRIPTION\r\nThis script is called by Bicep deployment scripts to start/stop Data Factory triggers\r\nand optionally run pipelines. It handles two types of triggers:\r\n\r\n1. Schedule triggers - Can be started/stopped directly\r\n2. BlobEventsTriggers - Require Event Grid subscription management before start/stop\r\n\r\nBlobEventsTriggers use Event Grid to listen for storage blob events. Before stopping,\r\nthe Event Grid subscription must be removed (unsubscribed). Before starting, the\r\nsubscription must be added and fully provisioned. This script handles this automatically\r\nby detecting BlobEventsTriggers via the BlobPathBeginsWith property.\r\n\r\nThe script uses retry logic with linear backoff to handle transient API failures and\r\nwait for Event Grid subscription provisioning/deprovisioning to complete.\r\n\r\n.PARAMETER DataFactoryResourceGroup\r\nThe resource group containing the Data Factory.\r\n\r\n.PARAMETER DataFactoryName\r\nThe name of the Data Factory instance.\r\n\r\n.PARAMETER Pipelines\r\nPipe-delimited list of pipeline names to run (e.g., \"pipeline1|pipeline2\").\r\n\r\n.PARAMETER StartTriggers\r\nSwitch to start all stopped triggers (with Event Grid subscription for blob triggers).\r\n\r\n.PARAMETER StopTriggers\r\nSwitch to stop all running triggers (with Event Grid unsubscription for blob triggers).\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StopTriggers\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StartTriggers\r\n#>\r\n\r\nparam(\r\n [string] $DataFactoryResourceGroup,\r\n [string] $DataFactoryName,\r\n [string] $Pipelines = \"\",\r\n [switch] $StartTriggers,\r\n [switch] $StopTriggers\r\n)\r\n\r\n$MAX_RETRIES = 20\r\n$DeploymentScriptOutputs = @{}\r\n\r\nfunction Write-Log($Message)\r\n{\r\n Write-Output \"$(Get-Date -Format 'HH:mm:ss') $Message\"\r\n}\r\n\r\nfunction Invoke-WithRetry([scriptblock]$Action, [string]$Name, [int]$Delay = 5)\r\n{\r\n for ($i = 1; $i -le $MAX_RETRIES; $i++)\r\n {\r\n try { return & $Action }\r\n catch\r\n {\r\n Write-Log \"$Name failed (attempt $i/${MAX_RETRIES}): $($_.Exception.Message)\"\r\n if ($i -eq $MAX_RETRIES) { throw }\r\n Start-Sleep -Seconds ($Delay * $i)\r\n }\r\n }\r\n}\r\n\r\nfunction Set-BlobTriggerSubscription([string]$TriggerName, [switch]$Subscribe)\r\n{\r\n $targetStatus = if ($Subscribe) { 'Enabled' } else { 'Disabled' }\r\n $action = if ($Subscribe) { 'Subscribing' } else { 'Unsubscribing' }\r\n\r\n Write-Log \"$action $TriggerName to events...\"\r\n Invoke-WithRetry -Name \"$action $TriggerName\" -Delay 5 -Action {\r\n if ($Subscribe)\r\n {\r\n Add-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n else\r\n {\r\n Remove-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n\r\n $status = Get-AzDataFactoryV2TriggerSubscriptionStatus `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName\r\n if ($status.Status -ne $targetStatus)\r\n {\r\n throw \"Subscription status is $($status.Status), expected $targetStatus\"\r\n }\r\n }\r\n}\r\n\r\nif ($StartTriggers -or $StopTriggers)\r\n{\r\n $triggers = Invoke-WithRetry -Name \"Get triggers\" -Action {\r\n Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n | Where-Object {\r\n ($StartTriggers -and $_.Properties.RuntimeState -ne \"Started\") `\r\n -or ($StopTriggers -and $_.Properties.RuntimeState -ne \"Stopped\")\r\n }\r\n }\r\n\r\n Write-Log \"Found $($triggers.Count) trigger(s) to $(if ($StartTriggers) { 'start' } else { 'stop' })\"\r\n\r\n $triggers | ForEach-Object {\r\n $triggerName = $_.Name\r\n $isBlobTrigger = $null -ne $_.Properties.BlobPathBeginsWith\r\n\r\n if ($StopTriggers)\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName }\r\n Write-Log \"Stopping trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Stop $triggerName\" -Action {\r\n Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n else\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName -Subscribe }\r\n Write-Log \"Starting trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Start $triggerName\" -Action {\r\n Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n\r\n Invoke-WithRetry -Name \"Wait for $triggerName\" -Action {\r\n $state = (Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName).Properties.RuntimeState\r\n $expected = if ($StartTriggers) { 'Started' } else { 'Stopped' }\r\n if ($state -ne $expected) { throw \"Trigger is $state, expected $expected\" }\r\n }\r\n\r\n Write-Log \"...done\"\r\n $DeploymentScriptOutputs[$triggerName] = $true\r\n }\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($Pipelines))\r\n{\r\n $Pipelines.Split('|') | ForEach-Object {\r\n Write-Log \"Running pipeline $_...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", + "usesDataFactory": "[contains(parameters('features'), 'DataFactory')]", + "usesKeyVault": "[contains(parameters('features'), 'KeyVault')]", + "usesStorage": "[contains(parameters('features'), 'Storage')]", + "telemetryId": "[format('ftk-hubapp-{0}{1}{2}', parameters('app').id, if(empty(parameters('telemetryString')), '', '_'), parameters('telemetryString'))]", + "telemetryProps": { + "mode": "Incremental", + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "[format('FTK: {0}', parameters('app').id)]", + "version": "[parameters('version')]" + } + }, + "resources": [] + } + }, + "autoStartRbacRoles": [ + "673868aa-7521-48a0-acc6-0f60742d39f5" + ], + "factoryStorageRoles": "[union(parameters('storageRoles'), createArray('17d1049b-9a84-46fb-8f53-869881c3d3ab', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe', 'acdd72a7-3385-48ef-bd42-f606fba81ae7'))]", + "storageInfrastructureEncryptionProperties": "[if(not(parameters('app').hub.options.storageInfrastructureEncryption), createObject(), createObject('encryption', createObject('keySource', 'Microsoft.Storage', 'requireInfrastructureEncryption', parameters('app').hub.options.storageInfrastructureEncryption)))]" + }, + "resources": { + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').storage)]", + "properties": { + "name": "[parameters('app').storage]", + "groupId": "dfs", + "privateLinkResourceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "fqdns": [ + "[reference('storageAccount').primaryEndpoints.dfs]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "storageAccount" + ] + }, + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint": { + "condition": "[and(and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}/{2}', parameters('app').dataFactory, 'default', parameters('app').keyVault)]", + "properties": { + "name": "[parameters('app').keyVault]", + "groupId": "vault", + "privateLinkResourceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "fqdns": [ + "[reference('keyVault').vaultUri]" + ] + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork", + "keyVault" + ] + }, + "dataFactory::managedVirtualNetwork": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/managedVirtualNetworks", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'default')]", + "properties": {}, + "dependsOn": [ + "dataFactory" + ] + }, + "dataFactory::managedIntegrationRuntime": { + "condition": "[and(variables('usesDataFactory'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.DataFactory/factories/integrationRuntimes", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, 'ManagedIntegrationRuntime')]", + "properties": { + "type": "Managed", + "managedVirtualNetwork": { + "referenceName": "default", + "type": "ManagedVirtualNetworkReference" + }, + "typeProperties": { + "computeProperties": { + "location": "[parameters('app').hub.location]", + "dataFlowProperties": { + "computeType": "General", + "coreCount": 8, + "timeToLive": 10, + "cleanup": false, + "customProperties": [] + }, + "copyComputeScaleProperties": { + "dataIntegrationUnit": 16, + "timeToLive": 30 + }, + "pipelineExternalComputeScaleProperties": { + "timeToLive": 30, + "numberOfPipelineNodes": 1, + "numberOfExternalNodes": 1 + } + } + } + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedVirtualNetwork" + ] + }, + "dataFactory::linkedService_keyVault": { + "condition": "[and(variables('usesDataFactory'), variables('usesKeyVault'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').keyVault)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureKeyVault", + "typeProperties": { + "baseUrl": "[reference(format('Microsoft.KeyVault/vaults/{0}', parameters('app').keyVault), '2023-02-01').vaultUri]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "keyVault" + ] + }, + "dataFactory::linkedService_storageAccount": { + "condition": "[and(variables('usesDataFactory'), variables('usesStorage'))]", + "type": "Microsoft.DataFactory/factories/linkedservices", + "apiVersion": "2018-06-01", + "name": "[format('{0}/{1}', parameters('app').dataFactory, parameters('app').storage)]", + "properties": { + "annotations": [], + "parameters": {}, + "type": "AzureBlobFS", + "typeProperties": { + "url": "[reference(format('Microsoft.Storage/storageAccounts/{0}', parameters('app').storage), '2021-08-01').primaryEndpoints.dfs]" + }, + "connectVia": "[if(parameters('app').hub.options.privateRouting, createObject('referenceName', 'ManagedIntegrationRuntime', 'type', 'IntegrationRuntimeReference'), null())]" + }, + "dependsOn": [ + "dataFactory", + "dataFactory::managedIntegrationRuntime", + "storageAccount" + ] + }, + "storageAccount::blobService": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2022-09-01", + "name": "[format('{0}/{1}', parameters('app').storage, 'default')]", + "dependsOn": [ + "storageAccount" + ] + }, + "blobEndpoint::blobPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "blobEndpoint" + ] + }, + "dfsEndpoint::dfsPrivateDnsZoneGroup": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", + "properties": { + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.dfs.{0}', environment().suffixes.storage))]" + } + } + ] + }, + "dependsOn": [ + "dfsEndpoint" + ] + }, + "keyVaultPrivateDnsZone::keyVaultPrivateDnsZoneLink": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", + "apiVersion": "2024-06-01", + "name": "[format('{0}/{1}', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), format('{0}-link', replace(format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), '.', '-')))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", "properties": { - "id": { - "type": "string" + "virtualNetwork": { + "id": "[parameters('app').hub.routing.networkId]" }, - "name": { - "type": "string" - } + "registrationEnabled": false }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } + "dependsOn": [ + "keyVaultPrivateDnsZone" + ] }, - "HubAppProperties": { - "type": "object", + "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", + "apiVersion": "2023-11-01", + "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" + "privateDnsZoneConfigs": [ + { + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "properties": { + "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')))]" } } - }, - "hub": { - "$ref": "#/definitions/_1.HubProperties" - }, - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "storage": { - "type": "string" - } + ] }, - "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", - "dataFactory": "Name of the Data Factory instance for this publisher.", - "keyVault": "Name of the KeyVault instance for this publisher.", - "storage": "Name of the storage account for this publisher.", - "description": "FinOps hub app configuration settings.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - }, - "parameters": { - "app": { - "$ref": "#/definitions/HubAppProperties", - "metadata": { - "description": "Required. FinOps hub app the deployment script is being run for." - } - }, - "identityName": { - "type": "string", - "metadata": { - "description": "Required. Name of the managed identity to create." - } - }, - "scriptName": { - "type": "string", - "defaultValue": "[deployment().name]", - "metadata": { - "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." - } + "dependsOn": [ + "keyVaultEndpoint", + "keyVaultPrivateDnsZone" + ] }, - "scriptContent": { - "type": "string", - "metadata": { - "description": "Required. Name of the deployment script to create." - } + "appTelemetry": { + "condition": "[parameters('app').hub.options.enableTelemetry]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "[if(lessOrEquals(length(variables('telemetryId')), 64), variables('telemetryId'), substring(variables('telemetryId'), 0, 64))]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Resources/deployments')]", + "properties": "[variables('telemetryProps')]" }, - "arguments": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Additional arguments to pass into the deployment script." + "dataFactory": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.DataFactory/factories", + "apiVersion": "2018-06-01", + "name": "[parameters('app').dataFactory]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.DataFactory/factories')]", + "identity": { + "type": "SystemAssigned" + }, + "properties": { + "globalConfigurations": { + "PipelineBillingEnabled": "true" + } } }, - "environmentVariables": { - "type": "array", - "items": { - "$ref": "#/definitions/EnvironmentVariable" + "storageRoleAssignments": { + "copy": { + "name": "storageRoleAssignments", + "count": "[length(variables('factoryStorageRoles'))]" }, - "defaultValue": [], - "metadata": { - "description": "Optional. Environment variables to use for the deployment script." - } - } - }, - "variables": { - "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", - "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', format('{0}cg', parameters('app').hub.routing.scriptStorage), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" - }, - "resources": { - "identity": { + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "name": "[guid(resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage), variables('factoryStorageRoles')[copyIndex()], resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('factoryStorageRoles')[copyIndex()])]", + "principalId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "dataFactory", + "storageAccount" + ] + }, + "triggerManagerIdentity": { + "condition": "[variables('usesDataFactory')]", "type": "Microsoft.ManagedIdentity/userAssignedIdentities", "apiVersion": "2023-01-31", - "name": "[parameters('identityName')]", + "name": "[format('{0}_triggerManager', parameters('app').dataFactory)]", + "location": "[parameters('app').hub.location]", "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", - "location": "[parameters('app').hub.location]" - }, - "scriptStorageAccount": { - "condition": "[parameters('app').hub.options.privateRouting]", - "existing": true, - "type": "Microsoft.Storage/storageAccounts", - "apiVersion": "2022-09-01", - "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + "dependsOn": [ + "dataFactory" + ] }, - "identityRoleAssignments": { + "triggerManagerRoleAssignments": { "copy": { - "name": "identityRoleAssignments", - "count": "[length(variables('privateEndpointDeploymentRoles'))]" + "name": "triggerManagerRoleAssignments", + "count": "[length(variables('autoStartRbacRoles'))]" }, - "condition": "[parameters('app').hub.options.privateRouting]", + "condition": "[variables('usesDataFactory')]", "type": "Microsoft.Authorization/roleAssignments", "apiVersion": "2022-04-01", - "scope": "[format('Microsoft.Storage/storageAccounts/{0}', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", - "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "scope": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]", + "name": "[guid(resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory), variables('autoStartRbacRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', format('{0}_triggerManager', parameters('app').dataFactory)))]", "properties": { - "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", - "principalId": "[reference('identity').principalId]", + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('autoStartRbacRoles')[copyIndex()])]", + "principalId": "[reference('triggerManagerIdentity').principalId]", "principalType": "ServicePrincipal" }, "dependsOn": [ - "identity" + "dataFactory", + "triggerManagerIdentity" + ] + }, + "storageAccount": { + "condition": "[variables('usesStorage')]", + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[parameters('app').storage]", + "location": "[parameters('app').hub.location]", + "sku": { + "name": "[parameters('app').hub.options.storageSku]" + }, + "kind": "BlockBlobStorage", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Storage/storageAccounts')]", + "properties": "[shallowMerge(createArray(variables('storageInfrastructureEncryptionProperties'), createObject('supportsHttpsTrafficOnly', true(), 'allowSharedKeyAccess', true(), 'isHnsEnabled', true(), 'minimumTlsVersion', 'TLS1_2', 'allowBlobPublicAccess', false(), 'publicNetworkAccess', 'Enabled', 'networkAcls', createObject('bypass', 'AzureServices', 'defaultAction', if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')))))]" + }, + "blobPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]" + }, + "blobEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-blob-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "blobLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "blob" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "dfsPrivateDnsZone": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "existing": true, + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]" + }, + "dfsEndpoint": { + "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-dfs-ep', parameters('app').storage)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.storage]" + }, + "privateLinkServiceConnections": [ + { + "name": "dfsLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]", + "groupIds": [ + "dfs" + ] + } + } + ] + }, + "dependsOn": [ + "storageAccount" + ] + }, + "keyVault": { + "condition": "[variables('usesKeyVault')]", + "type": "Microsoft.KeyVault/vaults", + "apiVersion": "2023-02-01", + "name": "[parameters('app').keyVault]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.KeyVault/vaults')]", + "properties": { + "sku": { + "name": "[parameters('app').hub.options.keyVaultSku]", + "family": "A" + }, + "enabledForDeployment": true, + "enabledForTemplateDeployment": true, + "enabledForDiskEncryption": true, + "enableSoftDelete": true, + "softDeleteRetentionInDays": 90, + "enablePurgeProtection": "[if(parameters('app').hub.options.keyVaultEnablePurgeProtection, true(), null())]", + "enableRbacAuthorization": false, + "createMode": "default", + "tenantId": "[subscription().tenantId]", + "accessPolicies": [ + { + "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", + "tenantId": "[subscription().tenantId]", + "permissions": { + "secrets": [ + "get" + ] + } + } + ], + "networkAcls": { + "bypass": "AzureServices", + "defaultAction": "[if(parameters('app').hub.options.privateRouting, 'Deny', 'Allow')]" + } + }, + "dependsOn": [ + "dataFactory" ] }, - "script": { - "type": "Microsoft.Resources/deploymentScripts", - "apiVersion": "2023-08-01", - "name": "[parameters('scriptName')]", - "kind": "AzurePowerShell", - "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", - "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", - "identity": { - "type": "UserAssigned", - "userAssignedIdentities": { - "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} - } + "keyVaultPrivateDnsZone": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateDnsZones", + "apiVersion": "2024-06-01", + "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", + "location": "global", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateDnsZones')]", + "properties": {} + }, + "keyVaultEndpoint": { + "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Network/privateEndpoints", + "apiVersion": "2023-11-01", + "name": "[format('{0}-ep', parameters('app').keyVault)]", + "location": "[parameters('app').hub.location]", + "tags": "[__bicep.getAppPublisherTags(parameters('app'), 'Microsoft.Network/privateEndpoints')]", + "properties": { + "subnet": { + "id": "[parameters('app').hub.routing.subnets.keyVault]" + }, + "privateLinkServiceConnections": [ + { + "name": "keyVaultLink", + "properties": { + "privateLinkServiceId": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]", + "groupIds": [ + "vault" + ] + } + } + ] }, - "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '9.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", "dependsOn": [ - "identity", - "identityRoleAssignments" + "keyVault" ] - } - } - } - }, - "dependsOn": [ - "deleteOldResources", - "pipeline_InitializeHub", - "trigger_DailySchedule", - "trigger_ExportManifestAdded", - "trigger_IngestionManifestAdded", - "trigger_MonthlySchedule", - "trigger_SettingsUpdated", - "triggerManagerIdentity", - "triggerManagerRoleAssignments" - ] - } - }, - "outputs": { - "resourceId": { - "type": "string", - "metadata": { - "description": "The Resource ID of the Data factory." - }, - "value": "[resourceId('Microsoft.DataFactory/factories', parameters('dataFactoryName'))]" - }, - "name": { - "type": "string", - "metadata": { - "description": "The Name of the Azure Data Factory instance." - }, - "value": "[parameters('dataFactoryName')]" - } - } - } - }, - "dependsOn": [ - "cmExports", - "core", - "dataExplorer", - "remoteHub" - ] - }, - "remoteHub": { - "condition": "[not(empty(parameters('remoteHubStorageKey')))]", - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.FinOpsHubs.RemoteHub", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "hub": { - "value": "[variables('hub')]" - }, - "remoteStorageKey": { - "value": "[parameters('remoteHubStorageKey')]" - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "languageVersion": "2.0", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "3708155483370559900" - } - }, - "definitions": { - "_1.HubRoutingProperties": { - "type": "object", - "properties": { - "networkId": { - "type": "string" - }, - "networkName": { - "type": "string" - }, - "scriptStorage": { - "type": "string" - }, - "dnsZones": { - "type": "object", - "properties": { - "blob": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "dfs": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "queue": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "table": { - "$ref": "#/definitions/_1.IdNameObject" - } - } - }, - "subnets": { - "type": "object", - "properties": { - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "scripts": { - "type": "string" }, - "storage": { - "type": "string" - } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, - "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubProperties": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - }, - "location": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "tagsByResource": { - "type": "object" - }, - "version": { - "type": "string" - }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" + "getKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::keyVaultManagedPrivateEndpoint", + "getStoragePrivateEndpointConnections", + "keyVault" + ] }, - "keyVaultSku": { - "type": "string" + "approveKeyVaultPrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesKeyVault')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveKeyVaultPrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "keyVaultName": { + "value": "[parameters('app').keyVault]" + }, + "privateEndpointConnections": { + "value": "[reference('getKeyVaultPrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "743018309241196676" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Required. Name of the KeyVault." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.KeyVault/vaults/privateEndpointConnections", + "apiVersion": "2023-07-01", + "name": "[format('{0}/{1}', parameters('keyVaultName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName')), '2023-07-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getKeyVaultPrivateEndpointConnections", + "keyVault" + ] }, - "networkAddressPrefix": { - "type": "string" + "getStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "GetStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "dataFactory::managedVirtualNetwork::storageManagedPrivateEndpoint", + "stopTriggers", + "storageAccount" + ] + }, + "approveStoragePrivateEndpointConnections": { + "condition": "[and(and(variables('usesDataFactory'), variables('usesStorage')), parameters('app').hub.options.privateRouting)]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "ApproveStoragePrivateEndpointConnections", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "storageAccountName": { + "value": "[parameters('app').storage]" + }, + "privateEndpointConnections": { + "value": "[reference('getStoragePrivateEndpointConnections').outputs.privateEndpointConnections.value]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "5719643816033951501" + } + }, + "parameters": { + "privateEndpointConnections": { + "type": "array", + "defaultValue": [], + "metadata": { + "description": "Optional. Array of private endpoint connections. Pending ones will be approved." + } + }, + "storageAccountName": { + "type": "string", + "metadata": { + "description": "Required. Name of the storage account." + } + } + }, + "resources": [ + { + "copy": { + "name": "privateEndpointConnection", + "count": "[length(parameters('privateEndpointConnections'))]" + }, + "condition": "[equals(parameters('privateEndpointConnections')[copyIndex()].properties.privateLinkServiceConnectionState.status, 'Pending')]", + "type": "Microsoft.Storage/storageAccounts/privateEndpointConnections", + "apiVersion": "2023-04-01", + "name": "[format('{0}/{1}', parameters('storageAccountName'), last(array(split(parameters('privateEndpointConnections')[copyIndex()].id, '/'))))]", + "properties": { + "privateLinkServiceConnectionState": { + "status": "Approved", + "description": "Approved-by-pipeline", + "actionRequired": "None" + } + } + } + ], + "outputs": { + "privateEndpointConnections": { + "type": "array", + "value": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2023-04-01').privateEndpointConnections]" + } + } + } + }, + "dependsOn": [ + "getStoragePrivateEndpointConnections", + "storageAccount" + ] + }, + "stopTriggers": { + "condition": "[variables('usesDataFactory')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}.{1}_ADF.StopTriggers', parameters('app').publisher, parameters('app').name)]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "arguments": { + "value": "[join(createArray(format('-DataFactoryResourceGroup \"{0}\"', resourceGroup().name), format('-DataFactoryName \"{0}\"', parameters('app').dataFactory), '-StopTriggers'), ' ')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + }, + "dependsOn": [ + "appTelemetry", + "dataFactory", + "triggerManagerIdentity", + "triggerManagerRoleAssignments" + ] + } + }, + "outputs": { + "dataFactoryId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Data Factory instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.DataFactory/factories', parameters('app').dataFactory)]" }, - "privateRouting": { - "type": "bool" + "keyVaultId": { + "type": "string", + "metadata": { + "description": "Resource ID of the Key Vault instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.KeyVault/vaults', parameters('app').keyVault)]" }, - "publisherIsolation": { - "type": "bool" + "storageAccountId": { + "type": "string", + "metadata": { + "description": "Resource ID of the storage account instance used by the FinOps hub app." + }, + "value": "[resourceId('Microsoft.Storage/storageAccounts', parameters('app').storage)]" }, - "storageInfrastructureEncryption": { - "type": "bool" + "principalId": { + "type": "string", + "metadata": { + "description": "Principal ID for the managed identity used by Data Factory." + }, + "value": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" }, - "storageSku": { - "type": "string" - } - } - }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" - }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" + "triggerManagerIdentityName": { + "type": "string", + "metadata": { + "description": "Name of the managed identity used to create and stop ADF triggers." + }, + "value": "[format('{0}_triggerManager', parameters('app').dataFactory)]" } } } - }, - "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", - "location": "Azure resource location of the FinOps hub instance.", - "tags": "Tags to apply to all FinOps hub resources.", - "tagsByResource": "Tags to apply to resources based on their resource type.", - "version": "FinOps hub version number.", - "options": { - "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", - "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", - "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", - "privateRouting": "Indicates whether private network routing is enabled.", - "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", - "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", - "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." - }, - "routing": "FinOps hub private network routing properties, if enabled.", - "core": { - "suffix": "Unique suffix used for shared resources." - }, - "apps": {}, - "description": "FinOps hub instance properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - }, - "parameters": { - "hub": { - "$ref": "#/definitions/HubProperties", - "metadata": { - "description": "Required. FinOps hub instance properties." } }, - "remoteStorageKey": { - "type": "securestring", - "metadata": { - "description": "Required. Create and store a key for a remote storage account." - } - } - }, - "variables": { - "$fxv#0": "12.0" - }, - "resources": { - "appRegistration": { + "keyVault_secret": { "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "Microsoft.FinOpsHubs.RemoteHub_Register", + "apiVersion": "2025-04-01", + "name": "keyVault_secret", "properties": { "expressionEvaluationOptions": { "scope": "inner" }, "mode": "Incremental", "parameters": { - "hub": { - "value": "[parameters('hub')]" - }, - "publisher": { - "value": "Microsoft FinOps hubs" - }, - "namespace": { - "value": "Microsoft.FinOpsHubs" + "vaultName": { + "value": "[parameters('app').keyVault]" }, - "appName": { - "value": "RemoteHub" + "secretName": { + "value": "[variables('storageKeySecretName')]" }, - "displayName": { - "value": "FinOps hub remote relay" + "secretValue": { + "value": "[parameters('remoteStorageKey')]" }, - "appVersion": { - "value": "[variables('$fxv#0')]" + "secretExpirationInSeconds": { + "value": 1702648632 }, - "features": { - "value": [ - "KeyVault", - "Storage" - ] + "secretNotBeforeInSeconds": { + "value": 10000 } }, "template": { "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "languageVersion": "2.0", "contentVersion": "1.0.0.0", "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "15179190433979236138" + "version": "0.40.2.10011", + "templateHash": "11304809191141616403" } }, - "definitions": { - "_1.HubRoutingProperties": { - "type": "object", - "properties": { - "networkId": { - "type": "string" - }, - "networkName": { - "type": "string" - }, - "scriptStorage": { - "type": "string" - }, - "dnsZones": { - "type": "object", - "properties": { - "blob": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "dfs": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "queue": { - "$ref": "#/definitions/_1.IdNameObject" - }, - "table": { - "$ref": "#/definitions/_1.IdNameObject" - } - } - }, - "subnets": { - "type": "object", - "properties": { - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "scripts": { - "type": "string" - }, - "storage": { - "type": "string" - } - } - } - }, - "metadata": { - "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", - "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", - "scriptStorage": "Name of the storage account used for deployment scripts.", - "dnsZones": { - "blob": "Resource ID and name for the blob storage DNS zone.", - "dfs": "Resource ID and name for the DFS storage DNS zone.", - "queue": "Resource ID and name for the queue storage DNS zone.", - "table": "Resource ID and name for the table storage DNS zone." - }, - "subnets": { - "dataFactory": "Resource ID of the subnet for Data Factory instances.", - "keyVault": "Resource ID of the subnet for Key Vault instances.", - "scripts": "Resource ID of the subnet for deployment script storage.", - "storage": "Resource ID of the subnet for storage accounts." - }, - "description": "FinOps hub private network routing properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "_1.IdNameObject": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - } - }, + "parameters": { + "vaultName": { + "type": "string", "metadata": { - "id": "Fully-qualified resource ID.", - "name": "Resource name.", - "description": "Resource ID and name.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } + "description": "Required. Name of the publisher-specific Key Vault instance." } }, - "HubAppFeature": { + "secretName": { "type": "string", - "allowedValues": [ - "DataFactory", - "KeyVault", - "Storage" - ], "metadata": { - "description": "FinOps hub app features.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "HubAppProperties": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "publisher": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "displayName": { - "type": "string" - }, - "suffix": { - "type": "string" - }, - "tags": { - "type": "object" - } - } - }, - "hub": { - "$ref": "#/definitions/HubProperties" - }, - "dataFactory": { - "type": "string" - }, - "keyVault": { - "type": "string" - }, - "storage": { - "type": "string" - } - }, + "description": "Required. Name of the Key Vault secret to create or update." + } + }, + "secretValue": { + "type": "securestring", "metadata": { - "name": "Short name of the FinOps hub app (not including the publisher namespace).", - "displayName": "Display name of the FinOps hub app.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app.", - "publisher": { - "name": "Fully-qualified namespace of the FinOps hub app publisher.", - "displayName": "Display name of the FinOps hub app publisher.", - "suffix": "Unique suffix used for publisher resources.", - "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher." - }, - "hub": "FinOps hub instance the app is deployed to.", - "dataFactory": "Name of the Data Factory instance for this publisher.", - "keyVault": "Name of the KeyVault instance for this publisher.", - "storage": "Name of the storage account for this publisher.", - "description": "FinOps hub app configuration settings.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } + "description": "Required. Value of the Key Vault secret." } }, - "HubProperties": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "name": { - "type": "string" - }, - "location": { - "type": "string" - }, - "tags": { - "type": "object" - }, - "tagsByResource": { - "type": "object" - }, - "version": { - "type": "string" - }, - "options": { - "type": "object", - "properties": { - "enableTelemetry": { - "type": "bool" - }, - "keyVaultSku": { - "type": "string" - }, - "networkAddressPrefix": { - "type": "string" - }, - "privateRouting": { - "type": "bool" - }, - "publisherIsolation": { - "type": "bool" - }, - "storageInfrastructureEncryption": { - "type": "bool" - }, - "storageSku": { - "type": "string" - } - } - }, - "routing": { - "$ref": "#/definitions/_1.HubRoutingProperties" - }, - "core": { - "type": "object", - "properties": { - "suffix": { - "type": "string" - } - } - } - }, + "secretExpirationInSeconds": { + "type": "int", + "defaultValue": -1, "metadata": { - "id": "FinOps hub resource ID.", - "name": "FinOps hub instance name.", - "location": "Azure resource location of the FinOps hub instance.", - "tags": "Tags to apply to all FinOps hub resources.", - "tagsByResource": "Tags to apply to resources based on their resource type.", - "version": "FinOps hub version number.", - "options": { - "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", - "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", - "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", - "privateRouting": "Indicates whether private network routing is enabled.", - "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", - "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", - "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." - }, - "routing": "FinOps hub private network routing properties, if enabled.", - "core": { - "suffix": "Unique suffix used for shared resources." - }, - "apps": {}, - "description": "FinOps hub instance properties.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } + "description": "Optional. Value of the Key Vault secret expiration date (exp) property. This is represented as seconds since Jan 1, 1970." + } + }, + "secretNotBeforeInSeconds": { + "type": "int", + "defaultValue": -1, + "metadata": { + "description": "Optional. Value of the Key Vault secret not before date (nbf) property. This is represented as seconds since Jan 1, 1970." + } + } + }, + "resources": [ + { + "type": "Microsoft.KeyVault/vaults/secrets", + "apiVersion": "2023-02-01", + "name": "[format('{0}/{1}', parameters('vaultName'), parameters('secretName'))]", + "properties": { + "attributes": "[union(createObject('enabled', true()), if(lessOrEquals(parameters('secretExpirationInSeconds'), 0), createObject(), createObject('exp', parameters('secretExpirationInSeconds'))), if(lessOrEquals(parameters('secretNotBeforeInSeconds'), 0), createObject(), createObject('nbf', parameters('secretNotBeforeInSeconds'))))]", + "value": "[parameters('secretValue')]" } } - }, - "functions": [ - { - "namespace": "__bicep", - "members": { - "getAppTags": { - "parameters": [ - { - "$ref": "#/definitions/HubAppProperties", - "name": "app" - }, - { - "type": "string", - "name": "resourceType" - }, - { - "type": "bool", - "nullable": true, - "name": "forceAppTags" - } - ], - "output": { - "type": "object", - "value": "[union(if(or(parameters('app').hub.options.publisherIsolation, coalesce(parameters('forceAppTags'), false())), parameters('app').tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" - }, - "metadata": { - "description": "Returns a tags dictionary that includes tags for the FinOps hub app.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "getPublisherTags": { - "parameters": [ - { - "$ref": "#/definitions/HubAppProperties", - "name": "app" - }, - { - "type": "string", - "name": "resourceType" - } - ], - "output": { - "type": "object", - "value": "[union(if(parameters('app').hub.options.publisherIsolation, parameters('app').publisher.tags, parameters('app').hub.tags), coalesce(tryGet(parameters('app').hub.tagsByResource, parameters('resourceType')), createObject()))]" - }, - "metadata": { - "description": "Returns a tags dictionary that includes tags for the FinOps hub app publisher.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "newApp": { - "parameters": [ - { - "$ref": "#/definitions/HubProperties", - "name": "hub" - }, - { - "type": "string", - "name": "publisherDisplayName" - }, - { - "type": "string", - "name": "publisherName" - }, - { - "type": "string", - "name": "appPartialName" - }, - { - "type": "string", - "name": "appDisplayName" - }, - { - "type": "string", - "name": "version" - } - ], - "output": { - "$ref": "#/definitions/HubAppProperties", - "value": "[_1.newAppInternal(parameters('hub'), parameters('publisherName'), parameters('publisherDisplayName'), if(or(not(parameters('hub').options.publisherIsolation), equals(parameters('publisherName'), 'Microsoft.FinOpsHubs')), parameters('hub').core.suffix, uniqueString(parameters('publisherName'))), createObject('ftk-hubapp-publisher', parameters('publisherName')), format('{0}.{1}', parameters('publisherName'), parameters('appPartialName')), parameters('appDisplayName'), parameters('version'))]" - }, - "metadata": { - "description": "Creates a new FinOps hub app configuration object.", - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - } + ], + "outputs": { + "secretName": { + "type": "string", + "metadata": { + "description": "Name of the Key Vault secret." + }, + "value": "[parameters('secretName')]" + } + } + } + }, + "dependsOn": [ + "appRegistration" + ] + } + }, + "outputs": { + "keyVaultName": { + "type": "string", + "metadata": { + "description": "Name of the Key Vault instance." + }, + "value": "[parameters('app').keyVault]" + } + } + } + }, + "dependsOn": [ + "core" + ] + }, + "deleteOldResources": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.DeleteOldResources", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[reference('core').outputs.app.value]" + }, + "identityName": { + "value": "[reference('core').outputs.triggerManagerIdentityName.value]" + }, + "scriptContent": { + "value": "[variables('$fxv#1')]" + }, + "environmentVariables": { + "value": [ + { + "name": "DataFactorySubscriptionId", + "value": "[subscription().id]" + }, + { + "name": "DataFactoryResourceGroup", + "value": "[resourceGroup().name]" + }, + { + "name": "DataFactoryName", + "value": "[reference('core').outputs.app.value.dataFactory]" + } + ] + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" + } + }, + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "value": { + "type": "string" + } + } + }, + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" + } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app the deployment script is being run for." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to create." + } + }, + "scriptName": { + "type": "string", + "defaultValue": "[deployment().name]", + "metadata": { + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." + } + }, + "scriptContent": { + "type": "string", + "metadata": { + "description": "Required. Name of the deployment script to create." + } + }, + "arguments": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "Optional. Additional arguments to pass into the deployment script." + } + }, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. Environment variables to use for the deployment script." + } + } + }, + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", + "properties": { + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" + }, + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] + } + } + } + }, + "dependsOn": [ + "core" + ] + }, + "startTriggers": { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "Microsoft.FinOpsHubs.StartTriggers", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[reference('core').outputs.app.value]" + }, + "dataFactoryInstances": { + "value": [ + "[reference('core').outputs.app.value.dataFactory]", + "[reference('cmExports').outputs.app.value.dataFactory]" + ] + }, + "identityName": { + "value": "[reference('core').outputs.triggerManagerIdentityName.value]" + }, + "startAllTriggers": { + "value": true + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "3919636936819908918" + } + }, + "definitions": { + "_1.HubProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" }, - { - "namespace": "_1", - "members": { - "newAppInternal": { - "parameters": [ - { - "$ref": "#/definitions/HubProperties", - "name": "hub" - }, - { - "type": "string", - "name": "publisherName" - }, - { - "type": "string", - "name": "publisherDisplayName" - }, - { - "type": "string", - "name": "publisherSuffix" - }, - { - "type": "object", - "name": "publisherTags" - }, - { - "type": "string", - "name": "appName" - }, - { - "type": "string", - "name": "appDisplayName" - }, - { - "type": "string", - "name": "version" - } - ], - "output": { - "$ref": "#/definitions/HubAppProperties", - "value": { - "name": "[parameters('appName')]", - "displayName": "[parameters('appDisplayName')]", - "tags": "[union(parameters('hub').tags, parameters('publisherTags'), createObject('ftk-hubapp', parameters('appName'), 'ftk-hubapp-version', parameters('version')))]", - "publisher": { - "name": "[parameters('publisherName')]", - "displayName": "[parameters('publisherDisplayName')]", - "suffix": "[parameters('publisherSuffix')]", - "tags": "[union(parameters('hub').tags, parameters('publisherTags'))]" - }, - "hub": "[parameters('hub')]", - "dataFactory": "[replace(format('{0}-{1}', take(format('{0}-engine', replace(parameters('hub').name, '_', '-')), sub(sub(63, length(parameters('publisherSuffix'))), 1)), parameters('publisherSuffix')), '--', '-')]", - "keyVault": "[replace(format('{0}-{1}', take(format('{0}-vault', replace(parameters('hub').name, '_', '-')), sub(sub(24, length(parameters('publisherSuffix'))), 1)), parameters('publisherSuffix')), '--', '-')]", - "storage": "[format('{0}{1}', take(_1.safeStorageName(parameters('hub').name), sub(24, length(parameters('publisherSuffix')))), parameters('publisherSuffix'))]" - } - }, - "metadata": { - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - }, - "safeStorageName": { - "parameters": [ - { - "type": "string", - "name": "name" - } - ], - "output": { - "type": "string", - "value": "[replace(replace(toLower(parameters('name')), '-', ''), '_', '')]" - }, - "metadata": { - "__bicep_imported_from!": { - "sourceTemplate": "hub-types.bicep" - } - } - } - } + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" } - ], - "parameters": { - "hub": { - "$ref": "#/definitions/HubProperties", - "metadata": { - "description": "Required. FinOps hub instance properties." - } + } + }, + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" + } + } + } + }, + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.HubRoutingProperties": { + "type": "object", + "properties": { + "networkId": { + "type": "string" + }, + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" }, - "publisher": { - "type": "string", - "metadata": { - "description": "Required. Display name of the FinOps hub app publisher." - } + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" }, - "namespace": { - "type": "string", - "metadata": { - "description": "Required. Namespace to use for the FinOps hub app publisher. Will be combined with appName to form a fully-qualified identifier. Must be an alphanumeric string without spaces or special characters except for periods. This value should never change and will be used to uniquely identify the publisher. A change would require migrating content to the new publisher. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." - } + "queue": { + "$ref": "#/definitions/_1.IdNameObject" }, - "appName": { - "type": "string", - "metadata": { - "description": "Required. Unique identifier of the FinOps hub app within the publisher namespace. Must be an alphanumeric string without spaces or special characters. This name should never change and will be used with the namespace to fully qualify the app. A change would require migrating content to the new app. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." - } + "table": { + "$ref": "#/definitions/_1.IdNameObject" + } + } + }, + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" }, - "displayName": { - "type": "string", - "metadata": { - "description": "Required. Display name of the FinOps hub app." - } + "dataFactory": { + "type": "string" }, - "appVersion": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Version number of the FinOps hub app." - } + "keyVault": { + "type": "string" }, - "features": { - "type": "array", - "items": { - "$ref": "#/definitions/HubAppFeature" - }, - "defaultValue": [], - "metadata": { - "description": "Optional. Indicate which features the app requires. Allowed values: \"Storage\". Default: [] (none)." - } + "scripts": { + "type": "string" }, - "telemetryString": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Optional. Custom string with additional metadata to log. Must an alphanumeric string without spaces or special characters except for underscores and dashes. Namespace + appName + telemetryString must be 50 characters or less - additional characters will be trimmed." - } + "storage": { + "type": "string" + } + } + } + }, + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." + }, + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + }, + "HubAppProperties": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } + }, + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } + } + }, + "parameters": { + "app": { + "$ref": "#/definitions/HubAppProperties", + "metadata": { + "description": "Required. FinOps hub app getting deployed." + } + }, + "dataFactoryInstances": { + "type": "array", + "items": { + "type": "string" + }, + "metadata": { + "description": "Required. List of Azure Data Factory instances to start triggers for. Can be up to 1 per publisher." + } + }, + "identityName": { + "type": "string", + "metadata": { + "description": "Required. Name of the managed identity to use when starting the triggers." + } + }, + "startAllTriggers": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "Optional. Start all triggers for the Data Factory instances. Default: false." + } + }, + "startPipelines": { + "type": "array", + "items": { + "type": "string" + }, + "defaultValue": [], + "metadata": { + "description": "Optional. List of pipelines to run. Default: [] (no pipelines)." + } + } + }, + "variables": { + "$fxv#0": "# Copyright (c) Microsoft Corporation.\r\n# Licensed under the MIT License.\r\n\r\n<#\r\n.SYNOPSIS\r\nManages Azure Data Factory triggers and pipelines during FinOps hub deployment.\r\n\r\n.DESCRIPTION\r\nThis script is called by Bicep deployment scripts to start/stop Data Factory triggers\r\nand optionally run pipelines. It handles two types of triggers:\r\n\r\n1. Schedule triggers - Can be started/stopped directly\r\n2. BlobEventsTriggers - Require Event Grid subscription management before start/stop\r\n\r\nBlobEventsTriggers use Event Grid to listen for storage blob events. Before stopping,\r\nthe Event Grid subscription must be removed (unsubscribed). Before starting, the\r\nsubscription must be added and fully provisioned. This script handles this automatically\r\nby detecting BlobEventsTriggers via the BlobPathBeginsWith property.\r\n\r\nThe script uses retry logic with linear backoff to handle transient API failures and\r\nwait for Event Grid subscription provisioning/deprovisioning to complete.\r\n\r\n.PARAMETER DataFactoryResourceGroup\r\nThe resource group containing the Data Factory.\r\n\r\n.PARAMETER DataFactoryName\r\nThe name of the Data Factory instance.\r\n\r\n.PARAMETER Pipelines\r\nPipe-delimited list of pipeline names to run (e.g., \"pipeline1|pipeline2\").\r\n\r\n.PARAMETER StartTriggers\r\nSwitch to start all stopped triggers (with Event Grid subscription for blob triggers).\r\n\r\n.PARAMETER StopTriggers\r\nSwitch to stop all running triggers (with Event Grid unsubscription for blob triggers).\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StopTriggers\r\n\r\n.EXAMPLE\r\n.\\Init-DataFactory.ps1 -DataFactoryResourceGroup \"rg-hub\" -DataFactoryName \"adf-hub\" -StartTriggers\r\n#>\r\n\r\nparam(\r\n [string] $DataFactoryResourceGroup,\r\n [string] $DataFactoryName,\r\n [string] $Pipelines = \"\",\r\n [switch] $StartTriggers,\r\n [switch] $StopTriggers\r\n)\r\n\r\n$MAX_RETRIES = 20\r\n$DeploymentScriptOutputs = @{}\r\n\r\nfunction Write-Log($Message)\r\n{\r\n Write-Output \"$(Get-Date -Format 'HH:mm:ss') $Message\"\r\n}\r\n\r\nfunction Invoke-WithRetry([scriptblock]$Action, [string]$Name, [int]$Delay = 5)\r\n{\r\n for ($i = 1; $i -le $MAX_RETRIES; $i++)\r\n {\r\n try { return & $Action }\r\n catch\r\n {\r\n Write-Log \"$Name failed (attempt $i/${MAX_RETRIES}): $($_.Exception.Message)\"\r\n if ($i -eq $MAX_RETRIES) { throw }\r\n Start-Sleep -Seconds ($Delay * $i)\r\n }\r\n }\r\n}\r\n\r\nfunction Set-BlobTriggerSubscription([string]$TriggerName, [switch]$Subscribe)\r\n{\r\n $targetStatus = if ($Subscribe) { 'Enabled' } else { 'Disabled' }\r\n $action = if ($Subscribe) { 'Subscribing' } else { 'Unsubscribing' }\r\n\r\n Write-Log \"$action $TriggerName to events...\"\r\n Invoke-WithRetry -Name \"$action $TriggerName\" -Delay 5 -Action {\r\n if ($Subscribe)\r\n {\r\n Add-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n else\r\n {\r\n Remove-AzDataFactoryV2TriggerSubscription `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName | Out-Null\r\n }\r\n\r\n $status = Get-AzDataFactoryV2TriggerSubscriptionStatus `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $TriggerName\r\n if ($status.Status -ne $targetStatus)\r\n {\r\n throw \"Subscription status is $($status.Status), expected $targetStatus\"\r\n }\r\n }\r\n}\r\n\r\nif ($StartTriggers -or $StopTriggers)\r\n{\r\n $triggers = Invoke-WithRetry -Name \"Get triggers\" -Action {\r\n Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n | Where-Object {\r\n ($StartTriggers -and $_.Properties.RuntimeState -ne \"Started\") `\r\n -or ($StopTriggers -and $_.Properties.RuntimeState -ne \"Stopped\")\r\n }\r\n }\r\n\r\n Write-Log \"Found $($triggers.Count) trigger(s) to $(if ($StartTriggers) { 'start' } else { 'stop' })\"\r\n\r\n $triggers | ForEach-Object {\r\n $triggerName = $_.Name\r\n $isBlobTrigger = $null -ne $_.Properties.BlobPathBeginsWith\r\n\r\n if ($StopTriggers)\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName }\r\n Write-Log \"Stopping trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Stop $triggerName\" -Action {\r\n Stop-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n else\r\n {\r\n if ($isBlobTrigger) { Set-BlobTriggerSubscription -TriggerName $triggerName -Subscribe }\r\n Write-Log \"Starting trigger $triggerName...\"\r\n Invoke-WithRetry -Name \"Start $triggerName\" -Action {\r\n Start-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName -Force\r\n }\r\n }\r\n\r\n Invoke-WithRetry -Name \"Wait for $triggerName\" -Action {\r\n $state = (Get-AzDataFactoryV2Trigger `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -Name $triggerName).Properties.RuntimeState\r\n $expected = if ($StartTriggers) { 'Started' } else { 'Stopped' }\r\n if ($state -ne $expected) { throw \"Trigger is $state, expected $expected\" }\r\n }\r\n\r\n Write-Log \"...done\"\r\n $DeploymentScriptOutputs[$triggerName] = $true\r\n }\r\n}\r\n\r\nif (-not [string]::IsNullOrWhiteSpace($Pipelines))\r\n{\r\n $Pipelines.Split('|') | ForEach-Object {\r\n Write-Log \"Running pipeline $_...\"\r\n Invoke-AzDataFactoryV2Pipeline `\r\n -ResourceGroupName $DataFactoryResourceGroup `\r\n -DataFactoryName $DataFactoryName `\r\n -PipelineName $_\r\n }\r\n}\r\n", + "uniqueInstances": "[union(filter(parameters('dataFactoryInstances'), lambda('adf', not(empty(lambdaVariables('adf'))))), createArray())]" + }, + "resources": { + "initialize": { + "copy": { + "name": "initialize", + "count": "[length(variables('uniqueInstances'))]" + }, + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[if(lessOrEquals(length(format('Microsoft.FinOpsHubs.Init_{0}', variables('uniqueInstances')[copyIndex()])), 64), format('Microsoft.FinOpsHubs.Init_{0}', variables('uniqueInstances')[copyIndex()]), substring(format('Microsoft.FinOpsHubs.Init_{0}', variables('uniqueInstances')[copyIndex()]), 0, 64))]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "app": { + "value": "[parameters('app')]" + }, + "identityName": { + "value": "[parameters('identityName')]" + }, + "scriptContent": { + "value": "[variables('$fxv#0')]" + }, + "arguments": { + "value": "[join(filter(createArray(format('-DataFactoryResourceGroup \"{0}\"', resourceGroup().name), format('-DataFactoryName \"{0}\"', variables('uniqueInstances')[copyIndex()]), if(not(empty(parameters('startPipelines'))), format('-Pipelines \"{0}\"', join(parameters('startPipelines'), '|')), ''), if(parameters('startAllTriggers'), '-StartTriggers', '')), lambda('arg', not(empty(lambdaVariables('arg'))))), ' ')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "languageVersion": "2.0", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "6505423328364225738" } }, - "variables": { - "app": "[__bicep.newApp(parameters('hub'), parameters('publisher'), parameters('namespace'), parameters('appName'), parameters('displayName'), parameters('appVersion'))]", - "usesDataFactory": "[contains(parameters('features'), 'DataFactory')]", - "usesKeyVault": "[contains(parameters('features'), 'KeyVault')]", - "usesStorage": "[contains(parameters('features'), 'Storage')]", - "telemetryId": "[format('ftk-hubapp-{0}{1}{2}', variables('app').name, if(empty(parameters('telemetryString')), '', '_'), parameters('telemetryString'))]", - "telemetryProps": { - "mode": "Incremental", - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "[format('FTK: {0} - {1} {2}', parameters('publisher'), parameters('displayName'), variables('telemetryId'))]", - "version": "[parameters('appVersion')]" - } + "definitions": { + "EnvironmentVariable": { + "type": "object", + "properties": { + "name": { + "type": "string" }, - "resources": [] + "value": { + "type": "string" + } } }, - "storageInfrastructureEncryptionProperties": "[if(not(parameters('hub').options.storageInfrastructureEncryption), createObject(), createObject('encryption', createObject('keySource', 'Microsoft.Storage', 'requireInfrastructureEncryption', parameters('hub').options.storageInfrastructureEncryption)))]" - }, - "resources": { - "storageAccount::blobService": { - "condition": "[variables('usesStorage')]", - "type": "Microsoft.Storage/storageAccounts/blobServices", - "apiVersion": "2022-09-01", - "name": "[format('{0}/{1}', variables('app').storage, 'default')]", - "dependsOn": [ - "storageAccount" - ] - }, - "blobEndpoint::blobPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", - "apiVersion": "2023-11-01", - "name": "[format('{0}/{1}', format('{0}-blob-ep', variables('app').storage), 'storage-endpoint-zone')]", - "properties": { - "privateDnsZoneConfigs": [ - { - "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]", - "properties": { - "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]" - } - } - ] - }, - "dependsOn": [ - "blobEndpoint" - ] - }, - "dfsEndpoint::dfsPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", - "apiVersion": "2023-11-01", - "name": "[format('{0}/{1}', format('{0}-dfs-ep', variables('app').storage), 'dfs-endpoint-zone')]", - "properties": { - "privateDnsZoneConfigs": [ - { - "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]", - "properties": { - "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink.dfs.{0}', environment().suffixes.storage))]" - } - } - ] - }, - "dependsOn": [ - "dfsEndpoint" - ] - }, - "keyVault::keyVault_accessPolicies": { - "condition": "[variables('usesKeyVault')]", - "type": "Microsoft.KeyVault/vaults/accessPolicies", - "apiVersion": "2023-02-01", - "name": "[format('{0}/{1}', variables('app').keyVault, 'add')]", + "_1.HubProperties": { + "type": "object", "properties": { - "accessPolicies": [ - { - "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", - "tenantId": "[subscription().tenantId]", - "permissions": { - "secrets": [ - "get" - ] + "id": { + "type": "string" + }, + "name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "tagsByResource": { + "type": "object" + }, + "version": { + "type": "string" + }, + "options": { + "type": "object", + "properties": { + "enableTelemetry": { + "type": "bool" + }, + "keyVaultSku": { + "type": "string" + }, + "keyVaultEnablePurgeProtection": { + "type": "bool" + }, + "networkAddressPrefix": { + "type": "string" + }, + "privateRouting": { + "type": "bool" + }, + "publisherIsolation": { + "type": "bool" + }, + "storageInfrastructureEncryption": { + "type": "bool" + }, + "storageSku": { + "type": "string" } } - ] - }, - "dependsOn": [ - "dataFactory", - "keyVault" - ] - }, - "keyVaultPrivateDnsZone::keyVaultPrivateDnsZoneLink": { - "condition": "[and(variables('usesKeyVault'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", - "apiVersion": "2024-06-01", - "name": "[format('{0}/{1}', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), format('{0}-link', replace(format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')), '.', '-')))]", - "location": "global", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Network/privateDnsZones/virtualNetworkLinks')]", - "properties": { - "virtualNetwork": { - "id": "[parameters('hub').routing.networkId]" }, - "registrationEnabled": false - }, - "dependsOn": [ - "keyVaultPrivateDnsZone" - ] - }, - "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { - "condition": "[and(variables('usesKeyVault'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", - "apiVersion": "2023-11-01", - "name": "[format('{0}/{1}', format('{0}-ep', variables('app').keyVault), 'keyvault-endpoint-zone')]", - "properties": { - "privateDnsZoneConfigs": [ - { - "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", - "properties": { - "privateDnsZoneId": "[resourceId('Microsoft.Network/privateDnsZones', format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')))]" + "routing": { + "$ref": "#/definitions/_1.HubRoutingProperties" + }, + "core": { + "type": "object", + "properties": { + "suffix": { + "type": "string" } } - ] - }, - "dependsOn": [ - "keyVaultEndpoint", - "keyVaultPrivateDnsZone" - ] - }, - "appTelemetry": { - "condition": "[parameters('hub').options.enableTelemetry]", - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "[if(lessOrEquals(length(variables('telemetryId')), 64), variables('telemetryId'), substring(variables('telemetryId'), 0, 64))]", - "tags": "[__bicep.getAppTags(variables('app'), 'Microsoft.Resources/deployments', true())]", - "properties": "[variables('telemetryProps')]" - }, - "dataFactory": { - "condition": "[variables('usesDataFactory')]", - "type": "Microsoft.DataFactory/factories", - "apiVersion": "2018-06-01", - "name": "[variables('app').dataFactory]", - "location": "[variables('app').hub.location]", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.DataFactory/factories')]", - "identity": { - "type": "SystemAssigned" + } }, - "properties": { - "globalConfigurations": { - "PipelineBillingEnabled": "true" + "metadata": { + "id": "FinOps hub resource ID.", + "name": "FinOps hub instance name.", + "location": "Azure resource location of the FinOps hub instance.", + "tags": "Tags to apply to all FinOps hub resources.", + "tagsByResource": "Tags to apply to resources based on their resource type.", + "version": "FinOps hub version number.", + "options": { + "enableTelemetry": "Indicates whether telemetry should be enabled for deployments.", + "keyVaultSku": "KeyVault SKU. Allowed values: \"standard\", \"premium\".", + "keyVaultEnablePurgeProtection": "Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.", + "networkAddressPrefix": "Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.", + "privateRouting": "Indicates whether private network routing is enabled.", + "publisherIsolation": "Indicates whether FinOps hub resources should be separated by publisher for advanced security.", + "storageInfrastructureEncryption": "Indicates whether infrastructure encryption is enabled for the storage account.", + "storageSku": "Storage account SKU. Allowed values: \"Premium_LRS\", \"Premium_ZRS\"." + }, + "routing": "FinOps hub private network routing properties, if enabled.", + "core": { + "suffix": "Unique suffix used for shared resources." + }, + "apps": {}, + "description": "FinOps hub instance properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" } } }, - "storageAccount": { - "condition": "[variables('usesStorage')]", - "type": "Microsoft.Storage/storageAccounts", - "apiVersion": "2022-09-01", - "name": "[variables('app').storage]", - "location": "[parameters('hub').location]", - "sku": { - "name": "[parameters('hub').options.storageSku]" - }, - "kind": "BlockBlobStorage", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Storage/storageAccounts')]", - "properties": "[shallowMerge(createArray(variables('storageInfrastructureEncryptionProperties'), createObject('supportsHttpsTrafficOnly', true(), 'allowSharedKeyAccess', true(), 'isHnsEnabled', true(), 'minimumTlsVersion', 'TLS1_2', 'allowBlobPublicAccess', false(), 'publicNetworkAccess', 'Enabled', 'networkAcls', createObject('bypass', 'AzureServices', 'defaultAction', if(parameters('hub').options.privateRouting, 'Deny', 'Allow')))))]" - }, - "blobPrivateDnsZone": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "existing": true, - "type": "Microsoft.Network/privateDnsZones", - "apiVersion": "2024-06-01", - "name": "[format('privatelink.blob.{0}', environment().suffixes.storage)]" - }, - "blobEndpoint": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints", - "apiVersion": "2023-11-01", - "name": "[format('{0}-blob-ep', variables('app').storage)]", - "location": "[parameters('hub').location]", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Network/privateEndpoints')]", + "_1.HubRoutingProperties": { + "type": "object", "properties": { - "subnet": { - "id": "[parameters('hub').routing.subnets.storage]" + "networkId": { + "type": "string" }, - "privateLinkServiceConnections": [ - { - "name": "blobLink", - "properties": { - "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', variables('app').storage)]", - "groupIds": [ - "blob" - ] + "networkName": { + "type": "string" + }, + "scriptStorage": { + "type": "string" + }, + "dnsZones": { + "type": "object", + "properties": { + "blob": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "dfs": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "queue": { + "$ref": "#/definitions/_1.IdNameObject" + }, + "table": { + "$ref": "#/definitions/_1.IdNameObject" } } - ] - }, - "dependsOn": [ - "storageAccount" - ] - }, - "dfsPrivateDnsZone": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "existing": true, - "type": "Microsoft.Network/privateDnsZones", - "apiVersion": "2024-06-01", - "name": "[format('privatelink.dfs.{0}', environment().suffixes.storage)]" - }, - "dfsEndpoint": { - "condition": "[and(variables('usesStorage'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints", - "apiVersion": "2023-11-01", - "name": "[format('{0}-dfs-ep', variables('app').storage)]", - "location": "[parameters('hub').location]", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Network/privateEndpoints')]", - "properties": { - "subnet": { - "id": "[parameters('hub').routing.subnets.storage]" }, - "privateLinkServiceConnections": [ - { - "name": "dfsLink", - "properties": { - "privateLinkServiceId": "[resourceId('Microsoft.Storage/storageAccounts', variables('app').storage)]", - "groupIds": [ - "dfs" - ] + "subnets": { + "type": "object", + "properties": { + "dataExplorer": { + "type": "string" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "scripts": { + "type": "string" + }, + "storage": { + "type": "string" } } - ] + } }, - "dependsOn": [ - "storageAccount" - ] - }, - "keyVault": { - "condition": "[variables('usesKeyVault')]", - "type": "Microsoft.KeyVault/vaults", - "apiVersion": "2023-02-01", - "name": "[variables('app').keyVault]", - "location": "[parameters('hub').location]", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.KeyVault/vaults')]", - "properties": { - "sku": { - "name": "[parameters('hub').options.keyVaultSku]", - "family": "A" + "metadata": { + "networkId": "Resource ID of the FinOps hub isolated virtual network, if private network routing is enabled.", + "networkName": "Name of the FinOps hub isolated virtual network, if private network routing is enabled.", + "scriptStorage": "Name of the storage account used for deployment scripts.", + "dnsZones": { + "blob": "Resource ID and name for the blob storage DNS zone.", + "dfs": "Resource ID and name for the DFS storage DNS zone.", + "queue": "Resource ID and name for the queue storage DNS zone.", + "table": "Resource ID and name for the table storage DNS zone." }, - "enabledForDeployment": true, - "enabledForTemplateDeployment": true, - "enabledForDiskEncryption": true, - "enableSoftDelete": true, - "softDeleteRetentionInDays": 90, - "enableRbacAuthorization": false, - "createMode": "default", - "tenantId": "[subscription().tenantId]", - "accessPolicies": [ - { - "objectId": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]", - "tenantId": "[subscription().tenantId]", - "permissions": { - "secrets": [ - "get" - ] - } - } - ], - "networkAcls": { - "bypass": "AzureServices", - "defaultAction": "[if(parameters('hub').options.privateRouting, 'Deny', 'Allow')]" + "subnets": { + "dataExplorer": "Resource ID of the subnet for the Data Explorer instance.", + "dataFactory": "Resource ID of the subnet for Data Factory instances.", + "keyVault": "Resource ID of the subnet for Key Vault instances.", + "scripts": "Resource ID of the subnet for deployment script storage.", + "storage": "Resource ID of the subnet for storage accounts." + }, + "description": "FinOps hub private network routing properties.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" } - }, - "dependsOn": [ - "dataFactory" - ] + } }, - "keyVaultPrivateDnsZone": { - "condition": "[and(variables('usesKeyVault'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateDnsZones", - "apiVersion": "2024-06-01", - "name": "[format('privatelink{0}', replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore'))]", - "location": "global", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Network/privateDnsZones')]", - "properties": {} + "_1.IdNameObject": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "name": { + "type": "string" + } + }, + "metadata": { + "id": "Fully-qualified resource ID.", + "name": "Resource name.", + "description": "Resource ID and name.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } }, - "keyVaultEndpoint": { - "condition": "[and(variables('usesKeyVault'), parameters('hub').options.privateRouting)]", - "type": "Microsoft.Network/privateEndpoints", - "apiVersion": "2023-11-01", - "name": "[format('{0}-ep', variables('app').keyVault)]", - "location": "[parameters('hub').location]", - "tags": "[__bicep.getPublisherTags(variables('app'), 'Microsoft.Network/privateEndpoints')]", + "HubAppProperties": { + "type": "object", "properties": { - "subnet": { - "id": "[parameters('hub').routing.subnets.keyVault]" + "id": { + "type": "string" }, - "privateLinkServiceConnections": [ - { - "name": "keyVaultLink", - "properties": { - "privateLinkServiceId": "[resourceId('Microsoft.KeyVault/vaults', variables('app').keyVault)]", - "groupIds": [ - "vault" - ] - } - } - ] + "name": { + "type": "string" + }, + "publisher": { + "type": "string" + }, + "suffix": { + "type": "string" + }, + "tags": { + "type": "object" + }, + "dataFactory": { + "type": "string" + }, + "keyVault": { + "type": "string" + }, + "storage": { + "type": "string" + }, + "hub": { + "$ref": "#/definitions/_1.HubProperties" + } }, - "dependsOn": [ - "keyVault" - ] + "metadata": { + "id": "Fully-qualified name of the publisher and app, separated by a dot.", + "name": "Short name of the FinOps hub app. Last segment of the app ID.", + "publisher": "Fully-qualified namespace of the FinOps hub app publisher.", + "suffix": "Unique suffix used for publisher resources.", + "tags": "Tags to apply to all FinOps hub resources for this FinOps hub app publisher. Tags are not specific to the app since resources are shared.", + "dataFactory": "Name of the Data Factory instance for this publisher.", + "keyVault": "Name of the KeyVault instance for this publisher.", + "storage": "Name of the storage account for this publisher.", + "hub": "FinOps hub instance the app is deployed to.", + "description": "FinOps hub app configuration settings.", + "__bicep_imported_from!": { + "sourceTemplate": "hub-types.bicep" + } + } } }, - "outputs": { + "parameters": { "app": { "$ref": "#/definitions/HubAppProperties", "metadata": { - "description": "FinOps hub app configuration." - }, - "value": "[variables('app')]" + "description": "Required. FinOps hub app the deployment script is being run for." + } }, - "principalId": { - "type": "string", - "metadata": { - "description": "Principal ID for the managed identity used by Data Factory." - }, - "value": "[reference('dataFactory', '2018-06-01', 'full').identity.principalId]" - } - } - } - } - }, - "keyVault_secret": { - "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", - "name": "keyVault_secret", - "properties": { - "expressionEvaluationOptions": { - "scope": "inner" - }, - "mode": "Incremental", - "parameters": { - "vaultName": { - "value": "[reference('appRegistration').outputs.app.value.keyVault]" - }, - "secretName": { - "value": "[format('{0}-storage-key', toLower(reference('appRegistration').outputs.app.value.hub.name))]" - }, - "secretValue": { - "value": "[parameters('remoteStorageKey')]" - }, - "secretExpirationInSeconds": { - "value": 1702648632 - }, - "secretNotBeforeInSeconds": { - "value": 10000 - } - }, - "template": { - "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", - "contentVersion": "1.0.0.0", - "metadata": { - "_generator": { - "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "338893459125049689" - } - }, - "parameters": { - "vaultName": { + "identityName": { "type": "string", "metadata": { - "description": "Required. Name of the publisher-specific Key Vault instance." + "description": "Required. Name of the managed identity to create." } }, - "secretName": { + "scriptName": { "type": "string", + "defaultValue": "[deployment().name]", "metadata": { - "description": "Required. Name of the Key Vault secret to create or update." + "description": "Optional. Name of the deployment script to create. Default = (same as deployment)." } }, - "secretValue": { - "type": "securestring", + "scriptContent": { + "type": "string", "metadata": { - "description": "Required. Value of the Key Vault secret." + "description": "Required. Name of the deployment script to create." } }, - "secretExpirationInSeconds": { - "type": "int", - "defaultValue": -1, + "arguments": { + "type": "string", + "defaultValue": "", "metadata": { - "description": "Optional. Value of the Key Vault secret expiration date (exp) property. This is represented as seconds since Jan 1, 1970." + "description": "Optional. Additional arguments to pass into the deployment script." } }, - "secretNotBeforeInSeconds": { - "type": "int", - "defaultValue": -1, + "environmentVariables": { + "type": "array", + "items": { + "$ref": "#/definitions/EnvironmentVariable" + }, + "defaultValue": [], "metadata": { - "description": "Optional. Value of the Key Vault secret not before date (nbf) property. This is represented as seconds since Jan 1, 1970." + "description": "Optional. Environment variables to use for the deployment script." } } }, - "resources": [ - { - "type": "Microsoft.KeyVault/vaults/secrets", - "apiVersion": "2023-02-01", - "name": "[format('{0}/{1}', parameters('vaultName'), parameters('secretName'))]", + "variables": { + "privateEndpointDeploymentRoles": "[if(not(parameters('app').hub.options.privateRouting), createArray(), createArray('69566ab7-960f-475b-8e7c-b3118f30c6bd'))]", + "containerGroupName": "[replace(replace(replace(parameters('scriptName'), '/', '-'), '.', '-'), '_', '-')]", + "privateEndpointDeploymentProperties": "[if(not(parameters('app').hub.options.privateRouting), createObject(), createObject('storageAccountSettings', createObject('storageAccountName', coalesce(parameters('app').hub.routing.scriptStorage, '')), 'containerSettings', createObject('containerGroupName', if(greater(length(variables('containerGroupName')), 63), substring(variables('containerGroupName'), 0, 62), variables('containerGroupName')), 'subnetIds', createArray(createObject('id', coalesce(parameters('app').hub.routing.subnets.scripts, ''))))))]" + }, + "resources": { + "identity": { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[parameters('identityName')]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.ManagedIdentity/userAssignedIdentities'), createObject()))]", + "location": "[parameters('app').hub.location]" + }, + "scriptStorageAccount": { + "condition": "[parameters('app').hub.options.privateRouting]", + "existing": true, + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2022-09-01", + "name": "[coalesce(parameters('app').hub.routing.scriptStorage, '')]" + }, + "identityRoleAssignments": { + "copy": { + "name": "identityRoleAssignments", + "count": "[length(variables('privateEndpointDeploymentRoles'))]" + }, + "condition": "[parameters('app').hub.options.privateRouting]", + "type": "Microsoft.Authorization/roleAssignments", + "apiVersion": "2022-04-01", + "scope": "[resourceId('Microsoft.Storage/storageAccounts', coalesce(parameters('app').hub.routing.scriptStorage, ''))]", + "name": "[guid(variables('privateEndpointDeploymentRoles')[copyIndex()], resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]", "properties": { - "attributes": "[union(createObject('enabled', true()), if(lessOrEquals(parameters('secretExpirationInSeconds'), 0), createObject(), createObject('exp', parameters('secretExpirationInSeconds'))), if(lessOrEquals(parameters('secretNotBeforeInSeconds'), 0), createObject(), createObject('nbf', parameters('secretNotBeforeInSeconds'))))]", - "value": "[parameters('secretValue')]" - } - } - ], - "outputs": { - "secretName": { - "type": "string", - "metadata": { - "description": "Name of the Key Vault secret." + "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('privateEndpointDeploymentRoles')[copyIndex()])]", + "principalId": "[reference('identity').principalId]", + "principalType": "ServicePrincipal" }, - "value": "[parameters('secretName')]" + "dependsOn": [ + "identity" + ] + }, + "script": { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2023-08-01", + "name": "[parameters('scriptName')]", + "kind": "AzurePowerShell", + "location": "[if(startsWith(parameters('app').hub.location, 'china'), 'chinaeast2', parameters('app').hub.location)]", + "tags": "[union(parameters('app').tags, coalesce(tryGet(parameters('app').hub.tagsByResource, 'Microsoft.Resources/deploymentScripts'), createObject()))]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName')))]": {} + } + }, + "properties": "[shallowMerge(createArray(variables('privateEndpointDeploymentProperties'), createObject('azPowerShellVersion', '11.0', 'retentionInterval', 'PT1H', 'cleanupPreference', 'OnSuccess', 'scriptContent', parameters('scriptContent'), 'arguments', parameters('arguments'), 'environmentVariables', parameters('environmentVariables'))))]", + "dependsOn": [ + "identity", + "identityRoleAssignments" + ] } } } - }, - "dependsOn": [ - "appRegistration" - ] - } - }, - "outputs": { - "keyVaultName": { - "type": "string", - "metadata": { - "description": "Name of the Key Vault instance." - }, - "value": "[reference('appRegistration').outputs.app.value.keyVault]" + } } } } - } + }, + "dependsOn": [ + "analytics", + "cmExports", + "cmManagedExports", + "core", + "deleteOldResources", + "remoteHub" + ] } }, "outputs": { @@ -19280,28 +24819,28 @@ "metadata": { "description": "The resource ID of the Data Explorer cluster." }, - "value": "[if(not(variables('deployDataExplorer')), '', reference('dataExplorer').outputs.clusterId.value)]" + "value": "[if(not(variables('useAzureDataExplorer')), '', reference('analytics').outputs.clusterId.value)]" }, "clusterUri": { "type": "string", "metadata": { "description": "The URI of the Data Explorer cluster." }, - "value": "[if(variables('useFabric'), parameters('fabricQueryUri'), if(not(variables('deployDataExplorer')), '', reference('dataExplorer').outputs.clusterUri.value))]" + "value": "[if(variables('useFabric'), parameters('fabricQueryUri'), if(not(variables('useAzureDataExplorer')), '', reference('analytics').outputs.clusterUri.value))]" }, "ingestionDbName": { "type": "string", "metadata": { "description": "The name of the Data Explorer database used for ingesting data." }, - "value": "[if(variables('useFabric'), 'Ingestion', if(not(variables('deployDataExplorer')), '', reference('dataExplorer').outputs.ingestionDbName.value))]" + "value": "[if(or(variables('useFabric'), variables('useAzureDataExplorer')), reference('analytics').outputs.ingestionDbName.value, '')]" }, "hubDbName": { "type": "string", "metadata": { "description": "The name of the Data Explorer database used for querying data." }, - "value": "[if(variables('useFabric'), 'Hub', if(not(variables('deployDataExplorer')), '', reference('dataExplorer').outputs.hubDbName.value))]" + "value": "[if(or(variables('useFabric'), variables('useAzureDataExplorer')), reference('analytics').outputs.hubDbName.value, '')]" }, "managedIdentityId": { "type": "string", @@ -19342,70 +24881,70 @@ "metadata": { "description": "Name of the Data Factory instance." }, - "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2022-09-01').outputs.dataFactoryName.value]" + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.dataFactoryName.value]" }, "storageAccountId": { "type": "string", "metadata": { "description": "Resource ID of the deployed storage account." }, - "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2022-09-01').outputs.storageAccountId.value]" + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.storageAccountId.value]" }, "storageAccountName": { "type": "string", "metadata": { "description": "Name of the storage account created for the hub instance. This must be used when connecting FinOps toolkit Power BI reports to your data." }, - "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2022-09-01').outputs.storageAccountName.value]" + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.storageAccountName.value]" }, "storageUrlForPowerBI": { "type": "string", "metadata": { "description": "URL to use when connecting custom Power BI reports to your data." }, - "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2022-09-01').outputs.storageUrlForPowerBI.value]" + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.storageUrlForPowerBI.value]" }, "clusterId": { "type": "string", "metadata": { "description": "Resource ID of the Data Explorer cluster." }, - "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2022-09-01').outputs.clusterId.value]" + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.clusterId.value]" }, "clusterUri": { "type": "string", "metadata": { "description": "URI of the Data Explorer cluster." }, - "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2022-09-01').outputs.clusterUri.value]" + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.clusterUri.value]" }, "ingestionDbName": { "type": "string", "metadata": { "description": "Name of the Data Explorer database used for ingesting data." }, - "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2022-09-01').outputs.ingestionDbName.value]" + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.ingestionDbName.value]" }, "hubDbName": { "type": "string", "metadata": { "description": "Name of the Data Explorer database used for querying data." }, - "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2022-09-01').outputs.hubDbName.value]" + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.hubDbName.value]" }, "managedIdentityId": { "type": "string", "metadata": { "description": "Object ID of the Data Factory managed identity. This will be needed when configuring managed exports." }, - "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2022-09-01').outputs.managedIdentityId.value]" + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.managedIdentityId.value]" }, "managedIdentityTenantId": { "type": "string", "metadata": { "description": "Azure AD tenant ID. This will be needed when configuring managed exports." }, - "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2022-09-01').outputs.managedIdentityTenantId.value]" + "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub'), '2025-04-01').outputs.managedIdentityTenantId.value]" } } } \ No newline at end of file diff --git a/docs/deploy/finops-hub-latest.ui.json b/docs/deploy/finops-hub-latest.ui.json index 16b32e2ef..201aefd47 100644 --- a/docs/deploy/finops-hub-latest.ui.json +++ b/docs/deploy/finops-hub-latest.ui.json @@ -696,6 +696,51 @@ } ], "visible": true + }, + { + "name": "remoteHub", + "type": "Microsoft.Common.Section", + "label": "Remote hub configuration", + "elements": [ + { + "name": "remoteHubIntro", + "type": "Microsoft.Common.TextBlock", + "visible": true, + "options": { + "text": "Configure this hub to send data to a remote FinOps hub in another tenant or subscription. This enables cross-tenant cost management scenarios where a central tenant collects cost data from multiple tenants. Leave these fields empty if this is not a remote hub setup." + } + }, + { + "name": "remoteHubStorageUri", + "type": "Microsoft.Common.TextBox", + "label": "Remote hub storage URI", + "toolTip": "Data Lake storage endpoint from the remote hub storage account. Copy from the storage account Settings > Endpoints > Data Lake storage. Example: https://myremotehub.dfs.core.windows.net/", + "constraints": { + "required": false, + "regex": "^$|^https://.*\\.dfs\\.core\\.windows\\.net/?$", + "validationMessage": "Must be a valid Data Lake storage endpoint URL in the format: https://storageaccount.dfs.core.windows.net/" + }, + "visible": true + }, + { + "name": "remoteHubStorageKey", + "type": "Microsoft.Common.PasswordBox", + "label": { + "password": "Remote hub storage key" + }, + "toolTip": "Storage account access key for the remote hub. Copy from the remote hub storage account Security + networking > Access keys > key1/2 > Key.", + "constraints": { + "required": false, + "regex": "^$|^[A-Za-z0-9+/]{86}==$", + "validationMessage": "Must be a valid storage account access key (base64 encoded, ending with ==)" + }, + "options": { + "hideConfirmation": true + }, + "visible": true + } + ], + "visible": true } ] }, @@ -739,6 +784,8 @@ "ingestionRetentionInMonths": "[steps('retention').storage.ingestionMonths]", "dataExplorerRawRetentionInDays": "[steps('retention').dataExplorer.rawDays]", "dataExplorerFinalRetentionInMonths": "[steps('retention').dataExplorer.finalMonths]", + "remoteHubStorageUri": "[steps('advanced').remoteHub.remoteHubStorageUri]", + "remoteHubStorageKey": "[steps('advanced').remoteHub.remoteHubStorageKey]", "tagsByResource": "[steps('tags').tagsByResource]" } } diff --git a/docs/deploy/finops-workbooks-13.0.json b/docs/deploy/finops-workbooks-13.0.json new file mode 100644 index 000000000..cad19f780 --- /dev/null +++ b/docs/deploy/finops-workbooks-13.0.json @@ -0,0 +1,20933 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "14432727458044791742" + } + }, + "parameters": { + "displayNamePrefix": { + "type": "string", + "defaultValue": "FinOps", + "metadata": { + "description": "Optional. Display name prefix to use for all workbooks. Default: \"FinOps\"." + } + }, + "includeOptimization": { + "type": "bool", + "defaultValue": true, + "metadata": { + "description": "Optional. Indicates whether to deploy the optimization workbook. Default: true." + } + }, + "includeGovernance": { + "type": "bool", + "defaultValue": true, + "metadata": { + "description": "Optional. Indicates whether to deploy the governance workbook. Default: true." + } + }, + "location": { + "type": "string", + "defaultValue": "[resourceGroup().location]", + "metadata": { + "description": "Optional. Location of the resources. Default: Same as deployment. See https://aka.ms/azureregions." + } + }, + "tags": { + "type": "object", + "defaultValue": {}, + "metadata": { + "description": "Optional. Tags for all resources." + } + }, + "tagsByResource": { + "type": "object", + "defaultValue": {}, + "metadata": { + "description": "Optional. Tags to apply to resources based on their resource type. Resource type specific tags will be merged with tags for all resources." + } + }, + "enableDefaultTelemetry": { + "type": "bool", + "defaultValue": true, + "metadata": { + "description": "Optional. Enable telemetry to track anonymous module usage trends, monitor for bugs, and improve future releases." + } + } + }, + "variables": { + "telemetryId": "00f120b5-2007-6120-0000-a7730126b006", + "finOpsToolkitVersion": "13.0", + "resourceTags": "[union(parameters('tags'), coalesce(tryGet(parameters('tagsByResource'), 'Microsoft.Insights/workbooks'), createObject()), createObject('ftk-version', variables('finOpsToolkitVersion'), 'ftk-tool', 'FinOps workbooks'))]" + }, + "resources": [ + { + "condition": "[parameters('enableDefaultTelemetry')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "[format('pid-{0}-{1}', variables('telemetryId'), uniqueString(deployment().name, parameters('location')))]", + "properties": { + "mode": "Incremental", + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "FinOps toolkit", + "version": "[variables('finOpsToolkitVersion')]" + } + }, + "resources": [] + } + } + }, + { + "condition": "[parameters('includeOptimization')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}-Optimization', parameters('displayNamePrefix'))]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "displayName": { + "value": "[format('{0} - Optimization', parameters('displayNamePrefix'))]" + }, + "location": { + "value": "[parameters('location')]" + }, + "tags": { + "value": "[variables('resourceTags')]" + }, + "enableDefaultTelemetry": { + "value": false + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "16879565610975211575" + } + }, + "parameters": { + "displayName": { + "type": "string", + "defaultValue": "Cost optimization", + "metadata": { + "description": "Optional. Display name for the workbook used in the Gallery. Must be unique in the resource group." + } + }, + "location": { + "type": "string", + "defaultValue": "[resourceGroup().location]", + "metadata": { + "description": "Optional. Location of the resources. Default: Same as deployment. See https://aka.ms/azureregions." + } + }, + "description": { + "type": "string", + "defaultValue": "Reports to help you optimize your cost.", + "metadata": { + "description": "Optional. Workbook description." + } + }, + "tags": { + "type": "object", + "defaultValue": {}, + "metadata": { + "description": "Optional. Tags for all resources." + } + }, + "enableDefaultTelemetry": { + "type": "bool", + "defaultValue": true, + "metadata": { + "description": "Optional. Enable telemetry to track anonymous module usage trends, monitor for bugs, and improve future releases." + } + } + }, + "variables": { + "$fxv#0": { + "version": "Notebook/1.0", + "items": [ + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "ca40468d-4518-43bf-ac6e-0a11d7331e12", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Overview", + "subTarget": "Welcome", + "style": "link" + }, + { + "id": "f280fc2a-f42a-42a4-ad4b-be37ab3e8b48", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Rate optimization", + "subTarget": "RateOptimization", + "style": "link" + }, + { + "id": "26b3c7ef-1a00-4a3f-a773-677f00db9343", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Usage optimization", + "subTarget": "UsageOptimization", + "style": "link" + } + ] + }, + "name": "links - MainTabs" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "28fdc6e9-2946-4016-8e75-b812ff8f853d", + "cellValue": "SelectedRateOptimizationTab", + "linkTarget": "parameter", + "linkLabel": "Compute", + "subTarget": "Compute", + "style": "link" + }, + { + "id": "4e0a0d2d-1d61-4d04-a35d-93e38d1bac29", + "cellValue": "SelectedRateOptimizationTab", + "linkTarget": "parameter", + "linkLabel": "Storage", + "subTarget": "Storage", + "style": "link" + }, + { + "id": "22d04714-50f4-4d72-baec-e8ccddddc7f3", + "cellValue": "SelectedRateOptimizationTab", + "linkTarget": "parameter", + "linkLabel": "Networking", + "subTarget": "Networking", + "style": "link" + }, + { + "id": "eaedbb0e-e895-4940-80ad-f743c3ab1041", + "cellValue": "SelectedRateOptimizationTab", + "linkTarget": "parameter", + "linkLabel": "Top 10 services", + "subTarget": "Top10Services", + "style": "link" + } + ] + }, + "name": "links - UsageOptimization tabs" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "51aa3a9b-14e0-4c22-a60d-abdbf8813f00", + "version": "KqlParameterItem/1.0", + "name": "Subscription", + "type": 6, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "includeAll": false, + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::all", + "label": " Subscription", + "value": [ + "value::all" + ] + }, + { + "id": "f342a111-002a-47fd-807f-0d4ccac0618a", + "version": "KqlParameterItem/1.0", + "name": "ResourceGroup", + "label": "Resource Group", + "type": 2, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": "resources\r\n| distinct resourceGroup", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "showDefault": false + }, + "defaultValue": "value::all", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + { + "id": "2336f06b-ddaa-4a9e-b72f-a2bec1ea84a9", + "version": "KqlParameterItem/1.0", + "name": "SingleSubHidden", + "type": 1, + "isRequired": true, + "query": "resourcecontainers\r\n| where type==\"microsoft.resources/subscriptions\"\r\n| take 1\r\n| project subscriptionId", + "crossComponentResources": [ + "{Subscription}" + ], + "isHiddenWhenLocked": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "label": "Hidden Subscription" + }, + { + "id": "d6776ffe-e4f6-4c08-8f9e-a2fe2b3b6634", + "version": "KqlParameterItem/1.0", + "name": "TagName", + "type": 2, + "query": "Resources\r\n| where tags != '' and tags != '[]'\r\n| mvexpand tags\r\n| extend tagName = tostring(bag_keys(tags)[0])\r\n| distinct tagName\r\n| sort by tagName asc", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [] + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null, + "label": "Tag Name" + }, + { + "id": "f73dc4a1-ef8b-45c5-a30b-a11bb077a3cc", + "version": "KqlParameterItem/1.0", + "name": "TagValue", + "type": 2, + "query": "Resources\r\n| mvexpand tags\r\n| extend tagName = tostring(bag_keys(tags)[0])\r\n| extend tagValue = tostring(tags[tagName])\r\n| where tags != '' and tags != '[]' and tostring(bag_keys(tags)[0]) == '{TagName}'\r\n| distinct tagValue\r\n| sort by tagValue asc", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [] + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null, + "label": "Tag Value" + } + ], + "style": "pills", + "queryType": 0, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "75", + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "UsageOptimization" + }, + "name": "parameters - Filters" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "template", + "loadFromTemplateId": "", + "items": [ + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "37ceb1c3-3930-4689-a90b-22f26e42bd81", + "version": "KqlParameterItem/1.0", + "name": "Subscription", + "type": 6, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "includeAll": false, + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::all", + "label": " Subscription" + }, + { + "id": "08f5fe68-c2e3-4882-9300-b3e33f572dfe", + "version": "KqlParameterItem/1.0", + "name": "ResourceGroup", + "label": "Resource Group", + "type": 2, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": "resources\r\n| distinct resourceGroup", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "showDefault": false + }, + "defaultValue": "value::all", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + { + "id": "4fea3013-df84-4930-a453-8a6bd0375130", + "version": "KqlParameterItem/1.0", + "name": "SingleSubHidden", + "type": 1, + "isRequired": true, + "query": "resourcecontainers\r\n| where type==\"microsoft.resources/subscriptions\"\r\n| take 1\r\n| project subscriptionId", + "crossComponentResources": [ + "{Subscription}" + ], + "isHiddenWhenLocked": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "label": "Hidden Subscription" + }, + { + "id": "8412f39d-ee67-4979-b887-47463b8848c2", + "version": "KqlParameterItem/1.0", + "name": "TagName", + "type": 2, + "query": "Resources\r\n| where tags != '' and tags != '[]'\r\n| mvexpand tags\r\n| extend tagName = tostring(bag_keys(tags)[0])\r\n| distinct tagName\r\n| sort by tagName asc", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [] + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null, + "label": "Tag Name" + }, + { + "id": "50c68f38-13a0-4aff-a259-4426c83b7cc0", + "version": "KqlParameterItem/1.0", + "name": "TagValue", + "type": 2, + "query": "Resources\r\n| mvexpand tags\r\n| extend tagName = tostring(bag_keys(tags)[0])\r\n| extend tagValue = tostring(tags[tagName])\r\n| where tags != '' and tags != '[]' and tostring(bag_keys(tags)[0]) == '{TagName}'\r\n| distinct tagValue\r\n| sort by tagValue asc", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [] + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null, + "label": "Tag Value" + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "75", + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "CostInformation" + }, + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "Welcome" + } + ], + "name": "parameters - Filters" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "eae8a0d2-14e6-4cd1-a2d2-fd6b207cf517", + "version": "KqlParameterItem/1.0", + "name": "Location", + "type": 2, + "query": "Resources\r\n| where type =~ 'Microsoft.Compute/virtualMachines'\r\n| project name, location\r\n| summarize count () by location\r\n| project location", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::1" + ] + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::1", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "label": "Resource Location" + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "25", + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "AHB" + }, + "name": "parameters - location" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "eae8a0d2-14e6-4cd1-a2d2-fd6b207cf517", + "version": "KqlParameterItem/1.0", + "name": "Location", + "type": 2, + "query": "Resources\r\n| where type =~ 'Microsoft.Compute/virtualMachines'\r\n| project name, location\r\n| summarize count () by location\r\n| project location", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::1" + ] + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::1", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "label": "Resource Location" + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "25", + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "AHB" + }, + "name": "parameters - location" + }, + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "6d563f46-7150-458c-9ee4-0558abe8e29b", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Azure App Service", + "subTarget": "webapp", + "style": "link" + }, + { + "id": "dbe9a7fb-6ab1-4de1-a98b-4ec8a9af906c", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Azure Kubernetes Service", + "subTarget": "AKS", + "style": "link" + }, + { + "id": "0211f413-9f36-4750-9ef2-d382ba30ba6c", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Azure Synapse", + "subTarget": "Synapse", + "preText": "VM", + "style": "link" + }, + { + "id": "820d600c-8ab3-4622-ba5a-52f60574d111", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Monitoring", + "subTarget": "Monitoring", + "style": "link" + } + ] + }, + "name": "links - Storage" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Synapse\r\nA Synapse Workspace is considered unused if it doesn't have any SQL pools attached to it\r\n", + "style": "upsell" + }, + "name": "Synapse" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources\r\n| where type =~ 'Microsoft.Synapse/workspaces'\r\n| join kind=leftouter (\r\n Resources\r\n | where type =~ 'Microsoft.Synapse/workspaces/sqlPools'\r\n | extend SynapseWorkspaceResourceId = substring(id, 0, indexof(id, '/sqlPools/'))\r\n | summarize sqlpoolCount = count() by SynapseWorkspaceResourceId\r\n) on $left.id == $right.SynapseWorkspaceResourceId\r\n| join kind=leftouter (\r\n Resources\r\n | where type =~ 'Microsoft.Synapse/workspaces/bigDataPools'\r\n | extend SynapseWorkspaceResourceId = substring(id, 0, indexof(id, '/bigDataPools/'))\r\n | summarize bigdatapoolCount = count() by SynapseWorkspaceResourceId\r\n) on $left.id == $right.SynapseWorkspaceResourceId\r\n| where (isnull(sqlpoolCount) or sqlpoolCount == 0) and (isnull(bigdatapoolCount) or bigdatapoolCount == 0)\r\n| project id, resourceGroup, subscriptionId, location\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n )\r\n on id", + "size": 0, + "title": "Unused Synapase workspace", + "noDataMessage": "All of your Synapse workspaces have SQL pools.", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "id1", + "formatter": 5 + }, + { + "columnMatch": "storageaccount", + "formatter": 13, + "formatOptions": { + "linkTarget": "Resource", + "subTarget": "insights", + "linkIsContextBlade": true, + "showIcon": true + } + } + ], + "labelSettings": [ + { + "columnId": "id", + "label": "Resource ID" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "subscriptionId", + "label": "Subscription Name" + } + ] + } + }, + "name": "Get-Synapse1" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "Synapse" + }, + "name": "SynapseGroup" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Azure Kubernetes Service\r\n- Enable cluster autoscaler to automatically adjust the number of agent nodes in response to resource constraints\r\n\r\n- Consider using Azure Spot VMs for workloads that can handle interruptions, early terminations, or evictions. For example, workloads such as batch processing jobs, development and testing environments, and large compute workloads may be good candidates to be scheduled on a spot node pool.\r\n\r\n- Utilize the Horizontal pod autoscaler to adjust the number of pods in a deployment depending on CPU utilization or other select metrics.\r\n\r\n- Use the Start/Stop feature in Azure Kubernetes Services (AKS).\r\n\r\n", + "style": "upsell" + }, + "name": "Azure Kubernetes Service" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "\tresources\r\n | where resourceGroup in ({ResourceGroup})\r\n\t| where type == \"microsoft.containerservice/managedclusters\"\r\n\t| extend AKSname=name,location=location,Sku=tostring(sku.name),Tier=tostring(sku.tier),AgentPoolProfiles=properties.agentPoolProfiles\r\n | project id,AKSname,resourceGroup,subscriptionId,Sku,Tier,AgentPoolProfiles,location\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n )\r\n on id\r\n\t| mvexpand AgentPoolProfiles\r\n\t| extend ProfileName = tostring(AgentPoolProfiles.name) ,mode=AgentPoolProfiles.mode,AutoScaleEnabled = AgentPoolProfiles.enableAutoScaling ,SpotVM=AgentPoolProfiles.scaleSetPriority, VMSize=tostring(AgentPoolProfiles.vmSize),minCount=tostring(AgentPoolProfiles.minCount),maxCount=tostring(AgentPoolProfiles.maxCount) , nodeCount=tostring(AgentPoolProfiles.['count'])\r\n | project id,ProfileName,Sku,Tier,mode,AutoScaleEnabled,SpotVM, VMSize,nodeCount,minCount,maxCount,location,resourceGroup,subscriptionId,AKSname\r\n \r\n", + "size": 0, + "noDataMessage": "You have no AKS clusters!", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "AKS Name", + "formatter": 1 + }, + { + "columnMatch": "id", + "formatter": 13, + "formatOptions": { + "linkTarget": "Resource", + "subTarget": "Insights", + "showIcon": true + } + }, + { + "columnMatch": "mode", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "System", + "representation": "Gear", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "User", + "representation": "Person", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "AutoScaleEnabled", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "true", + "representation": "success", + "text": "Enabled" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "disabled", + "text": "Disabled" + } + ] + } + }, + { + "columnMatch": "SpotVM", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "is Empty", + "representation": "2", + "text": "{0}{1}Not Spot VM" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "Group", + "formatter": 1 + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "AKSname", + "formatter": 5 + } + ], + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "AKSname" + ], + "expandTopLevel": true + }, + "labelSettings": [ + { + "columnId": "id", + "label": "ID" + }, + { + "columnId": "ProfileName", + "label": "Profile Name" + }, + { + "columnId": "Sku", + "label": "SKU" + }, + { + "columnId": "Tier", + "label": "SKU Tier" + }, + { + "columnId": "mode", + "label": "Mode" + }, + { + "columnId": "AutoScaleEnabled", + "label": "Autoscale enabled?" + }, + { + "columnId": "SpotVM", + "label": "Spot VM?" + }, + { + "columnId": "VMSize", + "label": "VM SKU" + }, + { + "columnId": "nodeCount", + "label": "Number of nodes" + }, + { + "columnId": "minCount", + "label": "Minimum nodes" + }, + { + "columnId": "maxCount", + "label": "Maximum nodes" + }, + { + "columnId": "location", + "label": "Location" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "subscriptionId", + "label": "Subscription Name" + }, + { + "columnId": "AKSname", + "label": "AKS Name" + } + ] + } + }, + "name": "Get-All-AKS" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "AKS" + }, + "name": "AKSGroup" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Azure App Service\r\n## Save with Premium v3 reserved instances\r\nWhen you commit to an Azure App Service Premium v3 reserved instance you can save money. The reservation discount is applied automatically to the number of running instances that match the reservation scope and attributes - you don't need to assign a reservation to a specific instance to get the discounts.\r\n\r\n## Determine the right reserved instance size before you buy\r\nBefore you buy a reservation, you should determine the size of the Premium v3 reserved instance that you need. The following sections will help you determine the right Premium v3 reserved instance size.\r\n\r\n## Use Autoscale appropriately\r\nAutoscale can be used to provision resources for when they're needed or on demand, which allows you to minimize costs when your environment is idle.\r\n", + "style": "upsell" + }, + "name": "Azure App Service" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where resourceGroup in ({ResourceGroup})\r\n| where type =~ 'Microsoft.Web/sites'\r\n| extend WebAppRG=resourceGroup, WebAppName=name, AppServicePlan=tostring(properties.serverFarmId), SKU=tostring(properties.sku), Type=kind, Status=tostring(properties.state), WebAppLocation=location, SubscriptionName=subscriptionId\r\n| project id,WebAppName, Type, Status, WebAppLocation, AppServicePlan, WebAppRG,SubscriptionName\r\n| order by id asc\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n )\r\n on id", + "size": 0, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "isVisible", + "comparison": "isEqualTo", + "value": "Never" + }, + "name": "query - WebFunctionStatus" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where resourceGroup in ({ResourceGroup})\r\n| where type == \"microsoft.web/serverfarms\" and sku.tier !~ 'Free'\r\n| extend planId=tolower(tostring(id)),skuname = tostring(sku.name) , skutier = tostring(sku.tier), workers=tostring(properties.numberOfWorkers),webRG=resourceGroup,maxworkers=tostring(properties.maximumNumberOfWorkers), Sites=tostring(properties.numberOfSites), SubscriptionName=subscriptionId\r\n| project planId, name, skuname, skutier, workers, maxworkers, webRG, Sites, SubscriptionName\r\n| join kind=leftouter (resources | where type ==\"microsoft.insights/autoscalesettings\" | project planId=tolower(tostring(properties.targetResourceUri)), PredictiveAutoscale=properties.predictiveAutoscalePolicy.scaleMode, AutoScaleProfiles=properties.profiles,resourceGroup) on planId\r\n", + "size": 0, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "isVisible", + "comparison": "isEqualTo", + "value": "Never" + }, + "name": "query - AppServiceplandetails" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"3fddbdd9-c4eb-46ae-b6b0-654c0da7b1a8\",\"mergeType\":\"inner\",\"leftTable\":\"query - AppServiceplandetails\",\"rightTable\":\"query - WebFunctionStatus\",\"leftColumn\":\"planId\",\"rightColumn\":\"AppServicePlan\"}],\"projectRename\":[{\"originalName\":\"[query - AppServiceplandetails].type\",\"mergedName\":\"type\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].tenantId\",\"mergedName\":\"tenantId\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].kind\",\"mergedName\":\"kind\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].location\",\"mergedName\":\"location\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].managedBy\",\"mergedName\":\"managedBy\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].sku\",\"mergedName\":\"sku\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].plan\",\"mergedName\":\"plan\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].properties\",\"mergedName\":\"properties\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].tags\",\"mergedName\":\"tags\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].identity\",\"mergedName\":\"identity\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].zones\",\"mergedName\":\"zones\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].extendedLocation\",\"mergedName\":\"extendedLocation\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].planId\",\"mergedName\":\"planId\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - WebFunctionStatus].id\",\"mergedName\":\"id\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].name\",\"mergedName\":\"name\",\"fromId\":\"3fddbdd9-c4eb-46ae-b6b0-654c0da7b1a8\"},{\"originalName\":\"[query - WebFunctionStatus].Status\",\"mergedName\":\"Status\",\"fromId\":\"3fddbdd9-c4eb-46ae-b6b0-654c0da7b1a8\"},{\"originalName\":\"[query - WebFunctionStatus].Type\",\"mergedName\":\"Type\",\"fromId\":\"3fddbdd9-c4eb-46ae-b6b0-654c0da7b1a8\"},{\"originalName\":\"[query - AppServiceplandetails].skuname\",\"mergedName\":\"skuname\",\"fromId\":\"3fddbdd9-c4eb-46ae-b6b0-654c0da7b1a8\"},{\"originalName\":\"[query - AppServiceplandetails].skutier\",\"mergedName\":\"skutier\",\"fromId\":\"3fddbdd9-c4eb-46ae-b6b0-654c0da7b1a8\"},{\"originalName\":\"[query - AppServiceplandetails].PredictiveAutoscale\",\"mergedName\":\"PredictiveAutoscale\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].AutoScaleProfiles\",\"mergedName\":\"AutoScaleProfiles\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - AppServiceplandetails].workers\",\"mergedName\":\"workers\",\"fromId\":\"3fddbdd9-c4eb-46ae-b6b0-654c0da7b1a8\"},{\"originalName\":\"[query - AppServiceplandetails].maxworkers\",\"mergedName\":\"maxworkers\",\"fromId\":\"3fddbdd9-c4eb-46ae-b6b0-654c0da7b1a8\"},{\"originalName\":\"[query - AppServiceplandetails].webRG\",\"mergedName\":\"webRG\",\"fromId\":\"3fddbdd9-c4eb-46ae-b6b0-654c0da7b1a8\"},{\"originalName\":\"[query - AppServiceplandetails].planId1\",\"mergedName\":\"planId1\",\"fromId\":\"3fddbdd9-c4eb-46ae-b6b0-654c0da7b1a8\"},{\"originalName\":\"[query - AppServiceplandetails].resourceGroup\",\"mergedName\":\"resourceGroup\",\"fromId\":\"3fddbdd9-c4eb-46ae-b6b0-654c0da7b1a8\"},{\"originalName\":\"[query - WebFunctionStatus].WebAppName\",\"mergedName\":\"WebAppName\",\"fromId\":\"3fddbdd9-c4eb-46ae-b6b0-654c0da7b1a8\"},{\"originalName\":\"[query - WebFunctionStatus].WebAppLocation\",\"mergedName\":\"WebAppLocation\",\"fromId\":\"3fddbdd9-c4eb-46ae-b6b0-654c0da7b1a8\"},{\"originalName\":\"[query - WebFunctionStatus].AppServicePlan\",\"mergedName\":\"AppServicePlan\",\"fromId\":\"3fddbdd9-c4eb-46ae-b6b0-654c0da7b1a8\"},{\"originalName\":\"[query - WebFunctionStatus].WebAppRG\",\"mergedName\":\"WebAppRG\",\"fromId\":\"3fddbdd9-c4eb-46ae-b6b0-654c0da7b1a8\"},{\"originalName\":\"[query - AppServiceplandetails].Sites\",\"mergedName\":\"Sites\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - WebFunctionStatus].SubscriptionName\"},{\"originalName\":\"[query - WebFunctionStatus].id1\"},{\"originalName\":\"[query - AppServiceplandetails].resourceGroup1\"}]}", + "size": 0, + "title": "Web Apps", + "noDataMessage": "You have no WebApps!", + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "Name", + "formatter": 1 + }, + { + "columnMatch": "SubscriptionName", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "name", + "formatter": 5 + }, + { + "columnMatch": "Status", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "Running", + "representation": "success", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Stopped", + "representation": "disabled", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "webRG", + "formatter": 5 + }, + { + "columnMatch": "planId1", + "formatter": 5 + }, + { + "columnMatch": "resourceGroup", + "formatter": 5 + }, + { + "columnMatch": "WebAppName", + "formatter": 5 + }, + { + "columnMatch": "AppServicePlan", + "formatter": 5 + }, + { + "columnMatch": "WebAppRG", + "formatter": 14, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Group", + "formatter": 1 + } + ], + "rowLimit": 1000, + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "name" + ], + "expandTopLevel": true + }, + "labelSettings": [ + { + "columnId": "SubscriptionName", + "label": "Subscription Name" + }, + { + "columnId": "planId", + "label": "Plan ID" + }, + { + "columnId": "id", + "label": "ID" + }, + { + "columnId": "name", + "label": "Name" + }, + { + "columnId": "skuname", + "label": "SKU" + }, + { + "columnId": "skutier", + "label": "SKU Tier" + }, + { + "columnId": "PredictiveAutoscale", + "label": "Autoscale Enabled?" + }, + { + "columnId": "AutoScaleProfiles", + "label": "Autoscale Profile" + }, + { + "columnId": "workers", + "label": "Workers" + }, + { + "columnId": "maxworkers", + "label": "Max. Workers" + }, + { + "columnId": "webRG", + "label": "Application Resource Group" + }, + { + "columnId": "WebAppName", + "label": "Application Name" + }, + { + "columnId": "WebAppLocation", + "label": "Application Location" + }, + { + "columnId": "AppServicePlan", + "label": "App Service Plan" + }, + { + "columnId": "WebAppRG", + "label": "Application Resource Group" + } + ] + } + }, + "name": "Get-Idle-WebApp" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where resourceGroup in ({ResourceGroup})\r\n| where type == \"microsoft.web/serverfarms\" and properties.numberOfSites == \"0\"\r\n| extend id, planId=tolower(tostring(id)),skuname = tostring(sku.name) , skutier = tostring(sku.tier), workers=tostring(properties.numberOfWorkers),webRG=resourceGroup,maxworkers=tostring(properties.maximumNumberOfWorkers), Sites=tostring(properties.numberOfSites), SubscriptionName=subscriptionId\r\n| project id, planId, name, skuname, skutier, workers, maxworkers, webRG, Sites, SubscriptionName\r\n| join kind=leftouter (resources | where type ==\"microsoft.insights/autoscalesettings\" | project planId=tolower(tostring(properties.targetResourceUri)), PredictiveAutoscale=properties.predictiveAutoscalePolicy.scaleMode, AutoScaleProfiles=properties.profiles,resourceGroup) on planId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n ) on id", + "size": 0, + "noDataMessage": "All of your App Service's plan have at least one website.", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "id", + "formatter": 5 + }, + { + "columnMatch": "name", + "formatter": 5 + }, + { + "columnMatch": "maxworkers", + "formatter": 5 + }, + { + "columnMatch": "SubscriptionName", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "planId1", + "formatter": 5 + }, + { + "columnMatch": "PredictiveAutoscale", + "formatter": 5 + }, + { + "columnMatch": "AutoScaleProfiles", + "formatter": 5 + }, + { + "columnMatch": "resourceGroup", + "formatter": 5 + }, + { + "columnMatch": "id1", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "planId", + "label": "App Service Plan " + }, + { + "columnId": "name", + "label": "SKU Name" + }, + { + "columnId": "skuname", + "label": "SKU Name" + }, + { + "columnId": "skutier", + "label": "SKU Tier" + }, + { + "columnId": "workers", + "label": "Number of Workers " + }, + { + "columnId": "maxworkers", + "label": "Number of websites" + }, + { + "columnId": "webRG", + "label": "Resource Group " + }, + { + "columnId": "Sites", + "label": "Number of websites" + }, + { + "columnId": "SubscriptionName", + "label": "Subscription Name" + } + ] + } + }, + "name": "query - IdleServicePlans" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "webapp" + }, + "name": "WebAppGroup" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "## Log Analytics workspace\r\nA [Log Analytics workspace](https://learn.microsoft.com/azure/azure-monitor/logs/log-analytics-workspace-overview) is a unique environment for log data from Azure Monitor and other Azure services, such as Microsoft Sentinel and Microsoft Defender for Cloud. Each workspace has its own data repository and configuration but might combine data from multiple services. The following advices could be of help in cost optimization:\r\n\r\n1. Adopt [commitment tiers](https://learn.microsoft.com/azure/azure-monitor/logs/cost-logs#commitment-tiers) where applicable.\r\n2. Adopt [Azure Monitor Logs dedicated cluster](https://learn.microsoft.com/azure/azure-monitor/logs/cost-logs#dedicated-clusters) if a single workspace does not ingest enough data as per the minimum commitment tier (100 GB/day) or if it is possible to aggregate ingestion costs from more than one workspace in the same region.\r\n3. Convert the free tier based workspace to **Pay-as-you-go** model and add them to an Azure Monitor Logs dedicated cluster where possible.", + "style": "upsell" + }, + "name": "MonitoringRecommendations" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type =~ 'microsoft.operationalinsights/workspaces'\r\n| where resourceGroup in ({ResourceGroup})\r\n| extend \r\n state = trim(' ', tostring(properties.provisioningState)),\r\n sku = trim(' ', tostring(properties.sku.name)),\r\n skuUpdate = trim(' ', tostring(properties.sku.lastSkuUpdate)),\r\n retentionDays = toint(properties.retentionInDays),\r\n dailyquotaGB = trim(' ', tostring(properties.workspaceCapping.dailyQuotaGb))\r\n| extend dailyquotaGB = iif(dailyquotaGB !=-1.0, dailyquotaGB,\"--\")\r\n| project id, resourceGroup, location, retentionDays, dailyquotaGB, sku, subscriptionId\r\n| join kind = inner (\r\n resources\r\n | where type =~ 'microsoft.operationalinsights/workspaces'\r\n | where resourceGroup in ({ResourceGroup})\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags[tagName])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | summarize arg_max(tagName, tagValue) by id\r\n) on id\r\n| extend resourceGroup = tostring(split(id,'/providers/')[0])\r\n| project-away id1", + "size": 0, + "title": "Log Analytics Workspaces", + "showRefreshButton": true, + "exportMultipleValues": true, + "exportedParameters": [ + { + "fieldName": "id", + "parameterName": "selectedWorkspaceId", + "parameterType": 1 + } + ], + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "id", + "formatter": 13, + "formatOptions": { + "linkTarget": "Resource", + "subTarget": "insights", + "linkIsContextBlade": true, + "showIcon": true + } + }, + { + "columnMatch": "resourceGroup", + "formatter": 14, + "formatOptions": { + "linkTarget": "Resource", + "linkIsContextBlade": true, + "showIcon": true + } + }, + { + "columnMatch": "retentionDays", + "formatter": 4, + "formatOptions": { + "min": 1, + "max": 730, + "palette": "blue", + "customColumnWidthSetting": "10%" + } + }, + { + "columnMatch": "dailyquotaGB", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "10%" + } + }, + { + "columnMatch": "sku", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "colors", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "lacluster", + "representation": "green", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "free", + "representation": "gray", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "capacityreservation", + "representation": "green", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "red", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": "Resource", + "linkIsContextBlade": true, + "showIcon": true + } + }, + { + "columnMatch": "tagName", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "is Empty", + "representation": "Blank", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Tags", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "tagValue", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "is Empty", + "representation": "Blank", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Tags", + "text": "{0}{1}" + } + ] + } + } + ], + "rowLimit": 10000, + "filter": true, + "labelSettings": [ + { + "columnId": "id", + "label": "Workspace" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "location", + "label": "Location" + }, + { + "columnId": "retentionDays", + "label": "Retention (days)" + }, + { + "columnId": "dailyquotaGB", + "label": "Daily Cap (GB)" + }, + { + "columnId": "sku", + "label": "Pricing Tier" + }, + { + "columnId": "subscriptionId", + "label": "Subscription" + }, + { + "columnId": "tagName", + "label": "Tag Name" + }, + { + "columnId": "tagValue", + "label": "Tag Value" + } + ] + }, + "sortBy": [] + }, + "name": "logAnalyticsWorkspaces", + "styleSettings": { + "showBorder": true + } + }, + { + "type": 1, + "content": { + "json": "💡_Select one or more workspaces from the list above to see daily ingestion trend_" + }, + "conditionalVisibility": { + "parameterName": "selectedWorkspaceId", + "comparison": "isEqualTo" + }, + "name": "text - 3", + "styleSettings": { + "showBorder": true + } + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "d9c04e61-453f-4f85-8d7e-1a34037d836b", + "version": "KqlParameterItem/1.0", + "name": "selectedWorkspaces", + "type": 5, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": "where type =~ 'microsoft.operationalinsights/workspaces'\r\n| where id in ({selectedWorkspaceId})", + "crossComponentResources": [ + "{Subscription}" + ], + "isHiddenWhenLocked": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "timeContext": { + "durationMs": 2592000000 + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null + }, + { + "id": "2108523c-fb80-49b3-9ff1-ea5e5eca2091", + "version": "KqlParameterItem/1.0", + "name": "TimeRange", + "label": "Time range", + "type": 4, + "isRequired": true, + "typeSettings": { + "selectableValues": [ + { + "durationMs": 172800000 + }, + { + "durationMs": 604800000 + }, + { + "durationMs": 1209600000 + }, + { + "durationMs": 2592000000 + } + ] + }, + "timeContext": { + "durationMs": 2592000000 + }, + "value": { + "durationMs": 2592000000 + } + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "conditionalVisibility": { + "parameterName": "_", + "comparison": "isEqualTo", + "value": "_" + }, + "name": "parameters - 2" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Usage\r\n| where StartTime >= startofday({TimeRange:start}) and EndTime < startofday(now())\r\n| where IsBillable == true\r\n| project Quantity, ResourceUri, TimeGenerated\r\n| summarize BillableDataGB = sum(Quantity / 1024.) by bin(TimeGenerated, 1d)\r\n| project TimeGenerated, BillableDataGB", + "size": 0, + "aggregation": 5, + "title": "Total Daily Ingestion for selected workspaces - Trend by {TimeRange:label}", + "timeContextFromParameter": "TimeRange", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces", + "crossComponentResources": [ + "{selectedWorkspaces}" + ], + "visualization": "barchart", + "chartSettings": { + "seriesLabelSettings": [ + { + "seriesName": "BillableDataGB", + "label": "Ingested data" + } + ], + "ySettings": { + "numberFormatSettings": { + "unit": 39, + "options": { + "style": "decimal", + "useGrouping": true, + "maximumFractionDigits": 2 + } + } + } + } + }, + "conditionalVisibility": { + "parameterName": "selectedWorkspaceId", + "comparison": "isNotEqualTo" + }, + "name": "dailyIngestionTrend", + "styleSettings": { + "showBorder": true + } + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "Monitoring" + }, + "name": "MonitoringGroup" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "advisorresources\r\n| where type =~ 'microsoft.advisor/recommendations'\r\n| where resourceGroup in ({ResourceGroup})\r\n| where properties.category == 'Cost' and properties.lastUpdated >= ago(1d)\r\n| where properties.impactedField has \"Workspaces\"\r\n| extend AffectedResource=tostring(properties.resourceMetadata.resourceId), Category=properties.category, SubCategory=properties.impactedField, Impact=properties.impact,resourceGroup,subscriptionId,Recommendation=tostring(properties.shortDescription.problem), id, stableId = name, recommendationTypeId = tostring(properties.recommendationTypeId), maxCpuP95 = properties.extendedProperties.MaxCpuP95\r\n| join kind = leftouter\r\n(advisorresources | where type=~'microsoft.advisor/suppressions'\r\n| extend tokens = split(id, '/')\r\n| extend stableId = iff(array_length(tokens) > 3, tokens[(array_length(tokens)-3)], '')\r\n| extend expirationTimeStamp = todatetime(iff(strcmp(tostring(properties.ttl), '-1') == 0, '9999-12-31', properties.expirationTimeStamp))\r\n| where expirationTimeStamp > now()\r\n| project stableId, expirationTimeStamp)\r\non stableId\r\n| where isempty(expirationTimeStamp)\r\n| project AffectedResource=tostring(properties.resourceMetadata.resourceId), Category=properties.category, SubCategory=properties.impactedField, Recommendation=tostring(properties.shortDescription.problem), Impact=properties.impact,resourceGroup,subscriptionId, id, stableId = name, recommendationTypeId = tostring(properties.recommendationTypeId), maxCpuP95 = properties.extendedProperties.MaxCpuP95\r\n| join kind = leftouter\r\n(advisorresources | where type =~ 'microsoft.advisor/configurations' | where isempty(resourceGroup) == true\r\n| project subscriptionId, excludeRecomm = properties.exclude, lowCpuThreshold = properties.lowCpuThreshold, AffectedResource=tostring(properties.resourceMetadata.resourceId),Impact=properties.impact,resourceGroup,AdditionaInfo=properties.extendedProperties,Recommendation=tostring(properties.shortDescription.problem))\r\non subscriptionId\r\n| extend isActive1 = iff(isnull(excludeRecomm), true, tobool(excludeRecomm) == false)\r\n| extend isActive2 = iff(recommendationTypeId == 'e10b1381-5f0a-47ff-8c7b-37bd13d7c974', iff((isnotempty(lowCpuThreshold) and isnotempty(maxCpuP95)), toint(maxCpuP95) < toint(lowCpuThreshold), iff((isempty(maxCpuP95) or toint(maxCpuP95) < 5), true, false)), true)\r\n| where isActive1 == true and isActive2 == true\r\n| join kind = leftouter\r\n(advisorresources | where type =~ 'microsoft.advisor/configurations' | where isnotempty(resourceGroup) == true\r\n| project subscriptionId, resourceGroup, excludeProperty = properties.exclude)\r\non subscriptionId, resourceGroup\r\n| extend isActive3 = iff(isnull(excludeProperty), true, tobool(excludeProperty) == false)\r\n| where isActive3 == true\r\n| where Category == 'Cost' \r\n| where SubCategory has \"Workspaces\"\r\n| project-away subscriptionId1, subscriptionId2, AffectedResource1, isActive2, isActive3, Impact1, Recommendation1, resourceGroup1, resourceGroup2", + "size": 0, + "title": "Azure Advisor Cost recommendations", + "noDataMessage": "You are following all of our cost recommendations for Monitoring", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "Group", + "formatter": 1 + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "id", + "formatter": 5 + }, + { + "columnMatch": "stableId", + "formatter": 5 + }, + { + "columnMatch": "recommendationTypeId", + "formatter": 5 + }, + { + "columnMatch": "maxCpuP95", + "formatter": 5 + }, + { + "columnMatch": "excludeRecomm", + "formatter": 5 + }, + { + "columnMatch": "lowCpuThreshold", + "formatter": 5 + }, + { + "columnMatch": "AdditionaInfo", + "formatter": 5, + "formatOptions": { + "customColumnWidthSetting": "19ch" + } + }, + { + "columnMatch": "isActive1", + "formatter": 5 + }, + { + "columnMatch": "excludeProperty", + "formatter": 5 + } + ], + "rowLimit": 1000, + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "Recommendation" + ], + "expandTopLevel": true + }, + "labelSettings": [ + { + "columnId": "AffectedResource", + "label": "Affected Resource" + }, + { + "columnId": "Category", + "label": "Recommendation Category" + }, + { + "columnId": "SubCategory", + "label": "Affected Resource Type" + }, + { + "columnId": "Recommendation", + "label": "Recommendation" + }, + { + "columnId": "Impact", + "label": "Impact" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "subscriptionId", + "label": "Subscription ID" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "isVisible", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "Get-AdvisorRecommendations-Monitoring" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": " resources\r\n | where resourceGroup in ({ResourceGroup})\r\n | where type has \"microsoft.operationalinsights/workspaces\"\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend AffectedResource=id,ResourceRG=resourceGroup\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n | project id", + "size": 0, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "rowLimit": 10000 + } + }, + "conditionalVisibility": { + "parameterName": "isVisible", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "query - tags - list all network resources" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\",\"mergeType\":\"innerunique\",\"leftTable\":\"Get-AdvisorRecommendations-Monitoring\",\"rightTable\":\"query - tags - list all network resources\",\"leftColumn\":\"AffectedResource\",\"rightColumn\":\"id\"}],\"projectRename\":[{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].AffectedResource\",\"mergedName\":\"Affected Resource\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].Category\",\"mergedName\":\"Recommendation Category\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].SubCategory\",\"mergedName\":\"Affected Resource Type\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].Recommendation\",\"mergedName\":\"Recommendation\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].Impact\",\"mergedName\":\"Impact\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].resourceGroup\",\"mergedName\":\"Resource Group\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].subscriptionId\",\"mergedName\":\"Subscription ID\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].id\",\"mergedName\":\"id\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].stableId\",\"mergedName\":\"stableId\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].recommendationTypeId\",\"mergedName\":\"recommendationTypeId\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].maxCpuP95\",\"mergedName\":\"maxCpuP95\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].excludeRecomm\",\"mergedName\":\"excludeRecomm\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].lowCpuThreshold\",\"mergedName\":\"lowCpuThreshold\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].AdditionaInfo\",\"mergedName\":\"AdditionaInfo\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].isActive1\",\"mergedName\":\"isActive1\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Monitoring].excludeProperty\",\"mergedName\":\"excludeProperty\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[query - tags - list all network resources].id\",\"mergedName\":\"id1\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].excludeProperty\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].isActive1\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].AdditionaInfo\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].lowCpuThreshold\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].excludeRecomm\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].maxCpuP95\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].recommendationTypeId\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].stableId\"},{\"originalName\":\"[query - tags - list all network resources].id\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].id\"}]}", + "size": 0, + "title": "Azure Advisor Cost recommendations", + "noDataMessage": "You are following all of our cost recommendations for Monitoring", + "noDataMessageStyle": 3, + "queryType": 7 + }, + "showPin": false, + "name": "query - Merge - Monitoring Advisor recommendations" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "Monitoring" + }, + "name": "AdvisorGroupMonitoring" + } + ] + }, + "name": "group - 0 " + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "UsageOptimization" + }, + { + "parameterName": "SelectedRateOptimizationTab", + "comparison": "isEqualTo", + "value": "Top10Services" + } + ], + "name": "group - Top10Services" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "template", + "loadFromTemplateId": "", + "items": [ + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "7a720abf-5b4a-4fb1-adaf-2383e70f625d", + "version": "KqlParameterItem/1.0", + "name": "Subscription", + "type": 6, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "includeAll": false, + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::all", + "label": " Subscription" + }, + { + "id": "a29babbc-5092-46c5-b03b-932c90aa61c9", + "version": "KqlParameterItem/1.0", + "name": "ResourceGroup", + "label": "Resource Group", + "type": 2, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": "resources\r\n| distinct resourceGroup", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "showDefault": false + }, + "defaultValue": "value::all", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + { + "id": "4b9c84b6-14ab-4663-b8b7-8bf0c351bbb5", + "version": "KqlParameterItem/1.0", + "name": "SingleSubHidden", + "type": 1, + "isRequired": true, + "query": "resourcecontainers\r\n| where type==\"microsoft.resources/subscriptions\"\r\n| take 1\r\n| project subscriptionId", + "crossComponentResources": [ + "{Subscription}" + ], + "isHiddenWhenLocked": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "label": "Hidden Subscription" + }, + { + "id": "6637e003-5323-4c6d-9990-426388c833e9", + "version": "KqlParameterItem/1.0", + "name": "TagName", + "type": 2, + "query": "Resources\r\n| where tags != '' and tags != '[]'\r\n| mvexpand tags\r\n| extend tagName = tostring(bag_keys(tags)[0])\r\n| distinct tagName\r\n| sort by tagName asc", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [] + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null, + "label": "Tag Name" + }, + { + "id": "d390e2b5-aa2f-494b-bbb8-0b18c8de9063", + "version": "KqlParameterItem/1.0", + "name": "TagValue", + "type": 2, + "query": "Resources\r\n| mvexpand tags\r\n| extend tagName = tostring(bag_keys(tags)[0])\r\n| extend tagValue = tostring(tags[tagName])\r\n| where tags != '' and tags != '[]' and tostring(bag_keys(tags)[0]) == '{TagName}'\r\n| distinct tagValue\r\n| sort by tagValue asc", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [] + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null, + "label": "Tag Value" + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "75", + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "CostInformation" + }, + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "Welcome" + } + ], + "name": "parameters - Filters" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "ae7eb928-8873-46f8-a3ff-77f45c207fb3", + "version": "KqlParameterItem/1.0", + "name": "Location", + "type": 2, + "query": "Resources\r\n| where type =~ 'Microsoft.Compute/virtualMachines'\r\n| project name, location\r\n| summarize count () by location\r\n| project location", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::1" + ] + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::1", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "label": "Resource Location" + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "25", + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "AHB" + }, + "name": "parameters - location" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "title": "Networking cost optimization recommendations", + "items": [ + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "ae7eb928-8873-46f8-a3ff-77f45c207fb3", + "version": "KqlParameterItem/1.0", + "name": "Location", + "type": 2, + "query": "Resources\r\n| where type =~ 'Microsoft.Compute/virtualMachines'\r\n| project name, location\r\n| summarize count () by location\r\n| project location", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::1" + ] + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::1", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "label": "Resource Location" + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "25", + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "AHB" + }, + "name": "parameters - location" + }, + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "e5d97e9d-97e6-45f2-871c-376799213b6a", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Azure Firewall", + "subTarget": "firewall", + "style": "link" + }, + { + "id": "0211f413-9f36-4750-9ef2-d382ba30ba6c", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Application Gateway", + "subTarget": "appGateway", + "preText": "VM", + "style": "link" + }, + { + "id": "61595d5e-9f25-4919-95a6-1462739f4657", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Load Balancer", + "subTarget": "loadBalancer", + "style": "link" + }, + { + "id": "dbe9a7fb-6ab1-4de1-a98b-4ec8a9af906c", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Public IP Address", + "subTarget": "publicIP", + "style": "link" + }, + { + "id": "79e7a97a-1413-41e8-b4c6-ebd1d0a45e2e", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Virtual Network Gateway", + "subTarget": "vpnGw", + "style": "link" + }, + { + "id": "5655ef75-a5ec-4f4b-badf-a99191a0493f", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "NAT Gateway", + "subTarget": "natgw", + "style": "link" + }, + { + "id": "68a77162-06c2-4648-83e0-f8f41c4fbda7", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "ExpressRoute", + "subTarget": "ER", + "style": "link" + }, + { + "id": "5dd4cb39-5aa1-4de9-bc4c-338e15b8d389", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Private DNS & Private Endpoint", + "subTarget": "privatedns", + "style": "link" + }, + { + "id": "6d563f46-7150-458c-9ee4-0558abe8e29b", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Advisor recommendations", + "subTarget": "advisorNetworking", + "style": "link" + } + ] + }, + "name": "links - Networking" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Recommendations for Application Gateways\r\nReview Application Gateways which include backend pools with no targets. Resources listed with 2 red signs are considered idle.", + "style": "upsell" + }, + "name": "Recommendations for Application Gateways" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type =~ 'Microsoft.Network/applicationGateways' and resourceGroup in ({ResourceGroup})\r\n| extend backendPoolsCount = array_length(properties.backendAddressPools),SKUName= tostring(properties.sku.name), SKUTier= tostring(properties.sku.tier),SKUCapacity=properties.sku.capacity,backendPools=properties.backendAddressPools,resourceGroup=strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup)\r\n| project id, name, SKUName, SKUTier, SKUCapacity,resourceGroup,subscriptionId\r\n| join (\r\n resources\r\n | where type =~ 'Microsoft.Network/applicationGateways' and resourceGroup in ({ResourceGroup})\r\n | mvexpand backendPools = properties.backendAddressPools\r\n | extend backendIPCount = array_length(backendPools.properties.backendIPConfigurations)\r\n | extend backendAddressesCount = array_length(backendPools.properties.backendAddresses)\r\n | extend backendPoolName = backendPools.properties.backendAddressPools.name\r\n | summarize backendIPCount = sum(backendIPCount) ,backendAddressesCount=sum(backendAddressesCount) by id\r\n) on id\r\n| project-away id1\r\n| where (backendIPCount == 0 or isempty(backendIPCount)) and (backendAddressesCount==0 or isempty(backendAddressesCount))\r\n| order by id asc\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n )\r\n on id", + "size": 0, + "title": "Application gateways with empty backend pools", + "noDataMessage": "You don't have any Application Gateways with empty backendpools", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "SKUCapacity", + "formatter": 1 + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "backendIPCount", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "0", + "representation": "disabled", + "text": "No Backend IPs" + }, + { + "operator": ">", + "thresholdValue": "0", + "representation": "success", + "text": "Backend IP configured" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "backendAddressesCount", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "0", + "representation": "disabled", + "text": "No Backend targets" + }, + { + "operator": ">", + "thresholdValue": "0", + "representation": "success", + "text": "Backend targets available" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "id1", + "formatter": 5 + }, + { + "columnMatch": "Recommendation", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "colors", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "No Backend targets", + "representation": "redBright", + "text": "No Backend targets" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "green", + "text": "Backend targets enabled" + } + ] + } + }, + { + "columnMatch": "backendPoolIPTarget", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": ">", + "thresholdValue": "0", + "representation": "success", + "text": "" + }, + { + "operator": "==", + "thresholdValue": "0", + "representation": "disabled", + "text": "" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "backendPoolVMTarget", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "is Empty", + "representation": "disabled", + "text": "" + }, + { + "operator": ">", + "thresholdValue": "0", + "representation": "success", + "text": "" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "Recommednation", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "colors", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "No Backend targets", + "representation": "redBright", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "green", + "text": "{0}{1}" + } + ] + } + } + ], + "labelSettings": [ + { + "columnId": "id", + "label": "ID" + }, + { + "columnId": "name", + "label": "Name" + }, + { + "columnId": "SKUName", + "label": "SKU" + }, + { + "columnId": "SKUTier", + "label": "SKU Tier" + }, + { + "columnId": "SKUCapacity", + "label": "Capacity" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "subscriptionId", + "label": "Subscription ID" + }, + { + "columnId": "backendIPCount", + "label": "Has backend pool for IPs?" + }, + { + "columnId": "backendAddressesCount", + "label": "Has backend pool for VMs?" + }, + { + "columnId": "id1", + "label": "ResourceID" + } + ] + } + }, + "name": "Get-Idle-AppGW" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "appGateway" + }, + "name": "NetworkingAppGateway" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Recommendations for Load Balancers\r\nReview Load balancers with no backend pools, and remove them if not needed.", + "style": "upsell" + }, + "name": "Recommendations for Load Balancers" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where resourceGroup in ({ResourceGroup})\r\n| extend resourceGroup=strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup), SKUName=tostring(sku.name),SKUTier=tostring(sku.tier),location,backendAddressPools = properties.backendAddressPools\r\n| where type =~ 'microsoft.network/loadbalancers' and array_length(backendAddressPools) == 0 and sku.name!='Basic'\r\n| order by id asc\r\n| project id,name, SKUName,SKUTier,backendAddressPools, location,resourceGroup, subscriptionId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n )\r\n on id", + "size": 0, + "title": "Load Balancers with empty backend pools", + "noDataMessage": "You don't have any Load Balancers with empty backendpools", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "backendAddressPools", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "0", + "representation": "disabled", + "text": "Empty Backend Pool" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "" + } + ] + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "id1", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "id", + "label": "Resource ID" + }, + { + "columnId": "name", + "label": "Name" + }, + { + "columnId": "SKUName", + "label": "SKU" + }, + { + "columnId": "SKUTier", + "label": "SKU Tier" + }, + { + "columnId": "backendAddressPools", + "label": "Has backend pool?" + }, + { + "columnId": "location", + "label": "Location" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "subscriptionId", + "label": "Subscription Name" + }, + { + "columnId": "id1", + "label": "ResourceID" + } + ] + } + }, + "name": "Get-Idle-LB" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "loadBalancer" + }, + "name": "LoadBalancerGroup" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Recommendations for Public IP Addresses\r\nReview unattached Public IP addresses, as they may represent additional cost.\r\n
This query will also show Public IPs attached to Idle network cards.\r\n", + "style": "upsell" + }, + "name": "Recommendations for PIP" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where resourceGroup in ({ResourceGroup})\r\n| where type =~ 'Microsoft.Network/publicIPAddresses' and isempty(properties.ipConfiguration) and isempty(properties.natGateway) and properties.publicIPAllocationMethod =~ 'Static'\r\n| extend PublicIpId=id, IPName=name, AllocationMethod=tostring(properties.publicIPAllocationMethod), SKUName=sku.name, Location=location ,resourceGroup=strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup)\r\n| project PublicIpId,IPName, SKUName, resourceGroup, Location, AllocationMethod, subscriptionId\r\n| union (\r\n Resources \r\n | where type =~ 'microsoft.network/networkinterfaces' and isempty(properties.virtualMachine) and isnull(properties.privateEndpoint) and isnotempty(properties.ipConfigurations) \r\n | extend IPconfig = properties.ipConfigurations \r\n | mv-expand IPconfig \r\n | extend PublicIpId= tostring(IPconfig.properties.publicIPAddress.id)\r\n | project PublicIpId\r\n | join ( \r\n resources \r\n | where type =~ 'Microsoft.Network/publicIPAddresses'\r\n | extend PublicIpId=id, IPName=name, AllocationMethod=tostring(properties.publicIPAllocationMethod), SKUName=sku.name, resourceGroup, Location=location \r\n ) on PublicIpId\r\n | project PublicIpId,IPName, SKUName, resourceGroup, Location, AllocationMethod, subscriptionId\r\n)\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | extend PublicIpId=id\r\n | distinct PublicIpId\r\n )\r\n on PublicIpId\r\n", + "size": 0, + "title": "Unattached Public IPs", + "noDataMessage": "You have no unattached Public IPs", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "PublicIpId1", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "PublicIpId", + "label": "ID" + }, + { + "columnId": "IPName", + "label": "Name" + }, + { + "columnId": "SKUName", + "label": "SKU" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "AllocationMethod", + "label": "Allocation Method" + }, + { + "columnId": "subscriptionId", + "label": "Subscription Name" + }, + { + "columnId": "PublicIpId1", + "label": "Resource ID" + } + ] + } + }, + "name": "Get-Idle-PIP" + }, + { + "type": 1, + "content": { + "json": "# Routing Preference\r\n\r\nAzure routing preference enables you to choose how your traffic routes between Azure and the Internet. You can choose to route traffic either via the Microsoft network or via the ISP network (public internet). By default, traffic is routed via the Microsoft global network for all Azure services.\r\n\r\nRouting preference choices include:\r\n\r\n- **Microsoft Network**: Both ingress and egress traffic stays bulk of the travel on the Microsoft global network. This routing is also known as cold potato routing. This option has a higher ingress/egress cost.\r\n\r\n- **Public Internet (ISP network)**: The new routing choice Internet routing minimizes travel on the Microsoft global network and uses the transit ISP network to route your traffic. This routing is also known as hot potato routing.\r\n\r\nFor more information about routing preference, see [What is routing preference?](https://learn.microsoft.com/azure/virtual-network/ip-services/ip-services-overview#routing-preference).\r\n\r\n", + "style": "upsell" + }, + "name": "text - 3" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where resourceGroup in ({ResourceGroup})\r\n| where type =~ 'Microsoft.Network/publicIPAddresses' and isnotempty(properties.ipConfiguration)\r\n| where tostring(properties.ipTags)== \"[]\"\r\n| extend PublicIpId=id, RoutingMethod=id, IPName=name, AllocationMethod=tostring(properties.publicIPAllocationMethod), SKUName=sku.name, Location=location ,resourceGroup=strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup)\r\n| project PublicIpId,IPName, RoutingMethod,SKUName, resourceGroup, Location, AllocationMethod, subscriptionId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | extend PublicIpId=id\r\n | distinct PublicIpId\r\n )\r\n on PublicIpId", + "size": 0, + "title": "Public IP Addresses ", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "RoutingMethod", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "Default", + "thresholdValue": null, + "representation": "info", + "text": "Microsoft Network" + } + ] + } + }, + { + "columnMatch": "PublicIpId1", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "PublicIpId", + "label": "ID" + }, + { + "columnId": "IPName", + "label": "Name" + }, + { + "columnId": "RoutingMethod", + "label": "Routing Method" + }, + { + "columnId": "SKUName", + "label": "SKU Name" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "Location", + "label": "Location" + }, + { + "columnId": "AllocationMethod", + "label": "Allocation Method" + }, + { + "columnId": "subscriptionId", + "label": "SubscriptionId" + }, + { + "columnId": "PublicIpId1", + "label": "Resource ID" + } + ] + } + }, + "name": "Query-PIP-RoutingPreference" + }, + { + "type": 1, + "content": { + "json": "# DDoS IP Protection\r\nIf you need to protect fewer than 15 public IP resources, the IP Protection tier is the more cost-effective option. However, if you have more than 15 public IP resources to protect, then the Network Protection tier becomes more cost-effective. \r\n\r\nThis query will surface all Public IP (PIP) addressess with the DDoS Protection enabled. If there are more than 15 Public IP Addresses with DDoS protection in the same virtual network, then it is cheaper to enable DDoS Network protection.\r\n\r\nThe Network Protection tier also provides additional features, including:\r\n\r\n- DDoS Protection Rapid Response (DRR)\r\n- Cost protection guarantees\r\n- Web Application Firewall (WAF) discounts\r\n\r\nFor more information about DDoS protection, see [Which Azure DDoS Protection tier should I choose?](https://learn.microsoft.com/azure/ddos-protection/ddos-faq?source=recommendations#which-azure-ddos-protection-tier-should-i-choose-).", + "style": "upsell" + }, + "name": "text - 5" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type == \"microsoft.network/publicipaddresses\"\r\n| project ddosProtection=tostring(properties.ddosSettings), name\r\n| where ddosProtection has \"Enabled\"\r\n| count\r\n| project TotalIpsProtected = Count\r\n| extend CheckIpsProtected = iff(TotalIpsProtected >= 15,\"Enable Network Protection tier\", \"Enable PIP DDoS Protection\")", + "size": 0, + "title": "Public IP Addresses DDoS Protection", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "RoutingMethod", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "Default", + "thresholdValue": null, + "representation": "info", + "text": "Microsoft Network" + } + ] + } + }, + { + "columnMatch": "PublicIpId1", + "formatter": 5 + } + ] + } + }, + "name": "Query-PIP-DDoSProtection" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "publicIP" + }, + "name": "PIPGroup" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Recommendations for Virtual Network Gateways\r\nReview idle Virtual Network Gateways that have no connections defined, as they may represent additional cost.\r\n", + "style": "upsell" + }, + "name": "Recommendations for idle virtualNetworkGateways" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type == \"microsoft.network/virtualnetworkgateways\"\r\n| extend resourceGroup =strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup)\r\n| project id, GWName=name,resourceGroup,location,subscriptionId\r\n| join kind = leftouter(\r\n resources\r\n | where type == \"microsoft.network/connections\"\r\n | extend id = tostring(properties.virtualNetworkGateway1.id)\r\n | project id\r\n | union (\r\n resources\r\n | where type == \"microsoft.network/connections\"\r\n | extend id = tostring(properties.virtualNetworkGateway2.id)\r\n | project id\r\n )\r\n) on id\r\n| where isempty(id1)\r\n| project id, GWName,resourceGroup,location,subscriptionId,status=id", + "size": 0, + "title": "Idle Virtual Network Gateways", + "noDataMessage": "No Idle Virtual Network Gateways found", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "status", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "Default", + "thresholdValue": null, + "representation": "warning", + "text": "Error-Connection not configured" + } + ] + } + } + ], + "filter": true, + "labelSettings": [ + { + "columnId": "id", + "label": "Resource ID" + }, + { + "columnId": "GWName", + "label": "VPN Gateway Name" + }, + { + "columnId": "status", + "label": "Is connected?" + } + ] + } + }, + "name": "query - Idle Virtual Network gateways" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "vpnGw" + }, + "name": "VPNGW Group" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "advisorresources\r\n| where type =~ 'microsoft.advisor/recommendations'\r\n| where resourceGroup in ({ResourceGroup})\r\n| where properties.category == 'Cost' and properties.lastUpdated >= ago(1d)\r\n| where properties.impactedField has \"Network\"\r\n| extend AffectedResource=tostring(properties.resourceMetadata.resourceId), Category=properties.category, SubCategory=properties.impactedField, Impact=properties.impact,resourceGroup,subscriptionId,Recommendation=tostring(properties.shortDescription.problem), id, stableId = name, recommendationTypeId = tostring(properties.recommendationTypeId), maxCpuP95 = properties.extendedProperties.MaxCpuP95\r\n| join kind = leftouter\r\n(advisorresources | where type=~'microsoft.advisor/suppressions'\r\n| extend tokens = split(id, '/')\r\n| extend stableId = iff(array_length(tokens) > 3, tokens[(array_length(tokens)-3)], '')\r\n| extend expirationTimeStamp = todatetime(iff(strcmp(tostring(properties.ttl), '-1') == 0, '9999-12-31', properties.expirationTimeStamp))\r\n| where expirationTimeStamp > now()\r\n| project stableId, expirationTimeStamp)\r\non stableId\r\n| where isempty(expirationTimeStamp)\r\n| project AffectedResource=tostring(properties.resourceMetadata.resourceId), Category=properties.category, SubCategory=properties.impactedField, Recommendation=tostring(properties.shortDescription.problem), Impact=properties.impact,resourceGroup,subscriptionId, id, stableId = name, recommendationTypeId = tostring(properties.recommendationTypeId), maxCpuP95 = properties.extendedProperties.MaxCpuP95\r\n| join kind = leftouter\r\n(advisorresources | where type =~ 'microsoft.advisor/configurations' | where isempty(resourceGroup) == true\r\n| project subscriptionId, excludeRecomm = properties.exclude, lowCpuThreshold = properties.lowCpuThreshold, AffectedResource=tostring(properties.resourceMetadata.resourceId),Impact=properties.impact,resourceGroup,AdditionaInfo=properties.extendedProperties,Recommendation=tostring(properties.shortDescription.problem))\r\non subscriptionId\r\n| extend isActive1 = iff(isnull(excludeRecomm), true, tobool(excludeRecomm) == false)\r\n| extend isActive2 = iff(recommendationTypeId == 'e10b1381-5f0a-47ff-8c7b-37bd13d7c974', iff((isnotempty(lowCpuThreshold) and isnotempty(maxCpuP95)), toint(maxCpuP95) < toint(lowCpuThreshold), iff((isempty(maxCpuP95) or toint(maxCpuP95) < 5), true, false)), true)\r\n| where isActive1 == true and isActive2 == true\r\n| join kind = leftouter\r\n(advisorresources | where type =~ 'microsoft.advisor/configurations' | where isnotempty(resourceGroup) == true\r\n| project subscriptionId, resourceGroup, excludeProperty = properties.exclude)\r\non subscriptionId, resourceGroup\r\n| extend isActive3 = iff(isnull(excludeProperty), true, tobool(excludeProperty) == false)\r\n| where isActive3 == true\r\n| where Category == 'Cost' \r\n| where SubCategory has \"Network\"\r\n| project-away subscriptionId1, subscriptionId2, AffectedResource1, isActive2, isActive3, Impact1, Recommendation1, resourceGroup1, resourceGroup2", + "size": 0, + "title": "Azure Advisor Cost recommendations", + "noDataMessage": "You are following all of our cost recommendations for Networking", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "Group", + "formatter": 1 + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "id", + "formatter": 5 + }, + { + "columnMatch": "stableId", + "formatter": 5 + }, + { + "columnMatch": "recommendationTypeId", + "formatter": 5 + }, + { + "columnMatch": "maxCpuP95", + "formatter": 5 + }, + { + "columnMatch": "excludeRecomm", + "formatter": 5 + }, + { + "columnMatch": "lowCpuThreshold", + "formatter": 5 + }, + { + "columnMatch": "AdditionaInfo", + "formatter": 5, + "formatOptions": { + "customColumnWidthSetting": "19ch" + } + }, + { + "columnMatch": "isActive1", + "formatter": 5 + }, + { + "columnMatch": "excludeProperty", + "formatter": 5 + } + ], + "rowLimit": 1000, + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "Recommendation" + ], + "expandTopLevel": true + }, + "labelSettings": [ + { + "columnId": "AffectedResource", + "label": "Affected Resource" + }, + { + "columnId": "Category", + "label": "Recommendation Category" + }, + { + "columnId": "SubCategory", + "label": "Affected Resource Type" + }, + { + "columnId": "Recommendation", + "label": "Recommendation" + }, + { + "columnId": "Impact", + "label": "Impact" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "subscriptionId", + "label": "Subscription ID" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "isVisible", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "Get-AdvisorRecommendations-Networking" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": " resources\r\n | where resourceGroup in ({ResourceGroup})\r\n | where type has \"Microsoft.Network\"\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend AffectedResource=id,ResourceRG=resourceGroup\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n | project id", + "size": 0, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "isVisible", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "query - tags - list all network resources" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\",\"mergeType\":\"innerunique\",\"leftTable\":\"Get-AdvisorRecommendations-Networking\",\"rightTable\":\"query - tags - list all network resources\",\"leftColumn\":\"AffectedResource\",\"rightColumn\":\"id\"}],\"projectRename\":[{\"originalName\":\"[Get-AdvisorRecommendations-Networking].AffectedResource\",\"mergedName\":\"AffectedResource\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].Category\",\"mergedName\":\"Category\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].SubCategory\",\"mergedName\":\"SubCategory\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].Recommendation\",\"mergedName\":\"Recommendation\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].Impact\",\"mergedName\":\"Impact\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].resourceGroup\",\"mergedName\":\"resourceGroup\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d87008d\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].excludeProperty\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].isActive1\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].AdditionaInfo\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].lowCpuThreshold\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].excludeRecomm\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].maxCpuP95\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].recommendationTypeId\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].stableId\"},{\"originalName\":\"[query - tags - list all network resources].id\"},{\"originalName\":\"[Get-AdvisorRecommendations-Networking].id\"}]}", + "size": 0, + "title": "Azure Advisor Cost recommendations", + "noDataMessage": "You are following all of our cost recommendations for Networking", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 7 + }, + "showPin": false, + "name": "query - Merge - Network Advisor recommendations" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "advisorNetworking" + }, + "name": "AdvisorGroupNetworking" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Recommendations for NAT Gateways\r\nReview idle NAT Gateways that have no subnet defined, as they may represent additional cost.\r\n", + "style": "upsell" + }, + "name": "Recommendations for idle NAT Gateway" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type == \"microsoft.network/natgateways\" and isnull(properties.subnets)\r\n| project id, GWName=name, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), Location=location ,resourceGroup=tostring(strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup)),subnet=tostring(properties.subnet), subscriptionId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n )\r\n on id", + "size": 0, + "title": "Idle NAT Gateways", + "noDataMessage": "No idle NAT gateways found", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "subnet", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "Default", + "thresholdValue": null, + "representation": "warning", + "text": "Not associated." + } + ] + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "id1", + "formatter": 5 + } + ], + "filter": true, + "labelSettings": [ + { + "columnId": "id", + "label": "Resource ID" + }, + { + "columnId": "GWName", + "label": "NAT Gateway Name" + }, + { + "columnId": "SKUName", + "label": "SKU Name" + }, + { + "columnId": "SKUTier", + "label": "SKU Tier" + }, + { + "columnId": "Location", + "label": "Location" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "subnet", + "label": "Subnet" + }, + { + "columnId": "subscriptionId", + "label": "Subscription" + } + ] + } + }, + "name": "query - Idle NAT gateways" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "natgw" + }, + "name": "NATGW Group" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Recommendations for Private DNS\r\nReview private DNS without [Virtual Network Links](https://learn.microsoft.com/azure/dns/private-dns-virtual-network-links).\r\n", + "style": "upsell" + }, + "name": "Recommendations for idle private dns" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type == \"microsoft.network/privatednszones\" and properties.numberOfVirtualNetworkLinks == 0\r\n| project id, PrivateDNSName=name, NumberOfRecordSets=tostring(properties.numberOfRecordSets),resourceGroup=tostring(strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup)),vNets=tostring(properties.properties.numberOfVirtualNetworkLinks), subscriptionId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n )\r\n on id", + "size": 0, + "title": "Idle private DNS ", + "noDataMessage": "No idle private DNS found", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "vNets", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "0", + "representation": "2", + "text": "Not associated to any vNET" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "warning", + "text": "Not associated to any vNET" + } + ] + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "id1", + "formatter": 5 + }, + { + "columnMatch": "subnet", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "Default", + "thresholdValue": null, + "representation": "warning", + "text": "Not associated." + } + ] + } + } + ], + "filter": true, + "labelSettings": [ + { + "columnId": "id", + "label": "Resource ID" + }, + { + "columnId": "PrivateDNSName", + "label": "Private DNS name" + }, + { + "columnId": "NumberOfRecordSets", + "label": "Number of DNS records" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "vNets", + "label": "vNETs associated" + }, + { + "columnId": "subscriptionId", + "label": "Subscription" + } + ] + } + }, + "name": "query - Idle private DNS" + }, + { + "type": 1, + "content": { + "json": "# Recommendations for Private endpoints\r\nReview [Private Endpoints](https://learn.microsoft.com/azure/private-link/private-endpoint-overview) that are not connected to any resource.", + "style": "upsell" + }, + "name": "Recommendations for idle private endpoints" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type =~ \"microsoft.network/privateendpoints\"\r\n| extend connection = iff(array_length(properties.manualPrivateLinkServiceConnections) > 0, properties.manualPrivateLinkServiceConnections[0], properties.privateLinkServiceConnections[0])\r\n| extend subnetId = properties.subnet.id\r\n| extend subnetIdSplit = split(subnetId, \"/\")\r\n| extend vnetId = strcat_array(array_slice(subnetIdSplit,0,8), \"/\")\r\n| extend serviceId = tostring(connection.properties.privateLinkServiceId)\r\n| extend serviceIdSplit = split(serviceId, \"/\")\r\n| extend serviceName = tostring(serviceIdSplit[8])\r\n| extend serviceTypeEnum = iff(isnotnull(serviceIdSplit[6]), tolower(strcat(serviceIdSplit[6], \"/\", serviceIdSplit[7])), \"microsoft.network/privatelinkservices\")\r\n| extend stateEnum = tostring(connection.properties.privateLinkServiceConnectionState.status)\r\n| extend stateDescription = tostring(connection.properties.privateLinkServiceConnectionState.description)\r\n| extend groupIds = tostring(connection.properties.groupIds[0])\r\n| where stateEnum == \"Disconnected\"\r\n| extend Details = pack_all()\r\n| project id, PrivateDNSName=name, stateEnum, stateDescription, resourceGroup=tostring(strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup)),serviceName, serviceTypeEnum, groupIds, vnetId, subnetId,subscriptionId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n )\r\n on id", + "size": 0, + "title": "Idle private endpoints", + "noDataMessage": "No idle private endpoints found", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "serviceTypeEnum", + "formatter": 16, + "formatOptions": { + "showIcon": true + } + }, + { + "columnMatch": "vnetId", + "formatter": 13, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "subnetId", + "formatter": 13, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "id1", + "formatter": 5 + } + ], + "filter": true, + "labelSettings": [ + { + "columnId": "id", + "label": "Resource ID" + }, + { + "columnId": "PrivateDNSName", + "label": "Private Endpoint name" + }, + { + "columnId": "stateEnum", + "label": "State" + }, + { + "columnId": "stateDescription", + "label": "State description" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "serviceName", + "label": "Resource Name" + }, + { + "columnId": "serviceTypeEnum", + "label": "Service Type" + }, + { + "columnId": "groupIds", + "label": "Resource Sub-type" + }, + { + "columnId": "vnetId", + "label": "Subnet" + }, + { + "columnId": "subnetId", + "label": "Subscription" + } + ] + } + }, + "name": "query - Idle private endpoint" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "privatedns" + }, + "name": "Private DNS and Private Endpoints Group" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Recommendations for Azure Firewall\r\n\r\n## Azure Firewall Premium SKU\r\nThis table identifies Azure Firewalls with Premium SKU and evaluates whether the associated policy incorporates premium-only features or not. If a Premium SKU Firewall lacks a policy with premium features, such as TLS or intrusion detection it will be shown here. To learn more about Azure Firewall skus, check this [SKU comparison table](https://learn.microsoft.com/azure/firewall/choose-firewall-sku). ", + "style": "upsell" + }, + "name": "Recommendations for premium Firewall" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where type =~ 'Microsoft.Network/azureFirewalls' and properties.sku.tier==\"Premium\"\r\n| project FWID=id, firewallName = name, SkuTier = tostring(properties.sku.tier), resourceGroup, location\r\n| join kind=inner (\r\n resources\r\n | where type =~ 'microsoft.network/firewallpolicies'\r\n | mv-expand properties.firewalls\r\n | extend intrusionDetection = tostring(properties.intrusionDetection contains \"Alert\" or properties.intrusionDetection contains \"Deny\"), transportSecurity = tostring(properties.transportSecurity contains \"keyVaultSecretId\")\r\n | extend FWID=tostring(properties_firewalls.id)\r\n | where intrusionDetection == \"False\" and transportSecurity == \"False\"\r\n | project PolicyName = name, PolicySKU=tostring(properties.sku.tier), intrusionDetection, transportSecurity, FWID\r\n) on FWID", + "size": 0, + "title": "Azure Firewall Premium", + "noDataMessage": "No Azure Firewall Premium found", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "firewallName", + "formatter": 5 + }, + { + "columnMatch": "FWID1", + "formatter": 5 + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "status", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "Default", + "thresholdValue": null, + "representation": "warning", + "text": "Error-Connection not configured" + } + ] + } + } + ], + "filter": true, + "labelSettings": [ + { + "columnId": "FWID", + "label": "Firewall Name" + }, + { + "columnId": "firewallName", + "label": "FWName" + }, + { + "columnId": "SkuTier", + "label": "SKU" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "location", + "label": "Location" + }, + { + "columnId": "PolicyName", + "label": "Policy Name" + }, + { + "columnId": "PolicySKU", + "label": "Policy SKU" + }, + { + "columnId": "intrusionDetection", + "label": "Is Intrusion Detection enabled?" + }, + { + "columnId": "transportSecurity", + "label": "Is TLS enabled?" + } + ] + } + }, + "name": "query - Optimize Premium AZ Firewall" + }, + { + "type": 1, + "content": { + "json": "## Avoid multiple Firewall instances in the same region\r\nOptimize the use of Azure Firewall by having a central instance of Azure Firewall in the hub virtual network or Virtual WAN secure hub and share the same firewall across many spoke virtual networks that are connected to the same hub from the same region. Ensure there's no unexpected cross-region traffic as part of the hub-spoke topology nor multiple Azure firewall instances deployed to the same region. To learn more about Azure Firewall design principles, check [Azure Well-Architected Framework review - Azure Firewall](https://learn.microsoft.com/azure/well-architected/service-guides/azure-firewall#cost-optimization).", + "style": "upsell" + }, + "name": "text - 3" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where type =~ 'Microsoft.Network/azureFirewalls'\r\n| mv-expand properties.ipConfigurations\r\n| project FWID=id, firewallName = name, SkuTier = tostring(properties.sku.tier), FWRG=resourceGroup, FWLocation=location, SubnetID=tostring(properties_ipConfigurations.properties.subnet.id)\r\n| join (\r\nresources\r\n| where type =~ 'Microsoft.Network/virtualNetworks' \r\n| mv-expand properties.subnets\r\n| where properties_subnets.id has 'AzureFirewallSubnet'\r\n| extend SubnetID=tostring(properties_subnets.id), SubnetName=name, SubnetLocation=location, SubnetRG=resourceGroup) on SubnetID\r\n| project FWID, FWRG,FWLocation, SubnetID,SubnetName, SubnetRG, SubnetLocation\r\n", + "size": 0, + "title": "Azure Firewall per location", + "noDataMessage": "No Firewall deployed", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "SubnetName", + "formatter": 5 + }, + { + "columnMatch": "firewallName", + "formatter": 5 + }, + { + "columnMatch": "FWID1", + "formatter": 5 + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "status", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "Default", + "thresholdValue": null, + "representation": "warning", + "text": "Error-Connection not configured" + } + ] + } + } + ], + "filter": true, + "labelSettings": [ + { + "columnId": "FWID", + "label": "Firewall Name" + }, + { + "columnId": "FWRG", + "label": "Firewall Resource Group" + }, + { + "columnId": "FWLocation", + "label": "Firewall Location" + }, + { + "columnId": "SubnetID", + "label": "Vnet / Subnet Name" + }, + { + "columnId": "SubnetName", + "label": "Subnet extended Name" + }, + { + "columnId": "SubnetRG", + "label": "Subnet Resource Group" + }, + { + "columnId": "SubnetLocation", + "label": "Subnet Location" + } + ] + } + }, + "name": "query - Firewall per Location" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "firewall" + }, + "name": "Firewall Group" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Recommendations for ExpressRoute\r\n\r\nReview idle ExpressRoute circuits that has not been provisioned (service provider has not completed provisioning or has deprovisioned), as they may represent additional cost.", + "style": "upsell" + }, + "name": "Recommendations for ExpressRoute" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type =~ 'Microsoft.Network/expressRouteCircuits' and properties.serviceProviderProvisioningState == \"NotProvisioned\"\r\n| extend ServiceLocation=tostring(properties.serviceProviderProperties.peeringLocation), ServiceProvider=tostring(properties.serviceProviderProperties.serviceProviderName), BandwidthInMbps=tostring(properties.serviceProviderProperties.bandwidthInMbps)\r\n| project ERId=id,ERName = name, ERRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), SKUFamily=tostring(sku.family), ERLocation = location, ServiceLocation, ServiceProvider, BandwidthInMbps\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | extend ERId=id\r\n | distinct ERId\r\n )\r\n on ERId\r\n\r\n", + "size": 0, + "title": "Idle ExpressRoute circuits", + "noDataMessage": "No idle ExpressRoute circuits found", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "ERId1", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "ERId", + "label": "ExpressRoute ID" + }, + { + "columnId": "ERName", + "label": "ER Name" + }, + { + "columnId": "ERRG", + "label": "Resource Group" + }, + { + "columnId": "SKUName", + "label": "SKU Name" + }, + { + "columnId": "SKUTier", + "label": "SKU Tier" + }, + { + "columnId": "SKUFamily", + "label": "SKU Family" + }, + { + "columnId": "ERLocation", + "label": "Location" + }, + { + "columnId": "ServiceLocation", + "label": "Service Location" + }, + { + "columnId": "ServiceProvider", + "label": "Service Provider" + }, + { + "columnId": "BandwidthInMbps", + "label": "Bandwidth in Mbps" + } + ] + } + }, + "name": "Idle ExpressRoute circuits" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "ER" + }, + "name": "ExpressRoute Group" + } + ] + }, + "name": "networking - Subscription" + } + ] + }, + "name": "group - 0" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "UsageOptimization" + }, + { + "parameterName": "SelectedRateOptimizationTab", + "comparison": "isEqualTo", + "value": "Networking" + } + ], + "name": "NetworkingGroup", + "styleSettings": { + "showBorder": true + } + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "template", + "title": "Storage cost optimization recommendations", + "loadFromTemplateId": "", + "items": [ + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "37ceb1c3-3930-4689-a90b-22f26e42bd81", + "version": "KqlParameterItem/1.0", + "name": "Subscription", + "type": 6, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "includeAll": false, + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::all", + "label": " Subscription" + }, + { + "id": "08f5fe68-c2e3-4882-9300-b3e33f572dfe", + "version": "KqlParameterItem/1.0", + "name": "ResourceGroup", + "label": "Resource Group", + "type": 2, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": "resources\r\n| distinct resourceGroup", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "showDefault": false + }, + "defaultValue": "value::all", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + { + "id": "4fea3013-df84-4930-a453-8a6bd0375130", + "version": "KqlParameterItem/1.0", + "name": "SingleSubHidden", + "type": 1, + "isRequired": true, + "query": "resourcecontainers\r\n| where type==\"microsoft.resources/subscriptions\"\r\n| take 1\r\n| project subscriptionId", + "crossComponentResources": [ + "{Subscription}" + ], + "isHiddenWhenLocked": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "label": "Hidden Subscription" + }, + { + "id": "8412f39d-ee67-4979-b887-47463b8848c2", + "version": "KqlParameterItem/1.0", + "name": "TagName", + "type": 2, + "query": "Resources\r\n| where tags != '' and tags != '[]'\r\n| mvexpand tags\r\n| extend tagName = tostring(bag_keys(tags)[0])\r\n| distinct tagName\r\n| sort by tagName asc", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [] + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null, + "label": "Tag Name" + }, + { + "id": "50c68f38-13a0-4aff-a259-4426c83b7cc0", + "version": "KqlParameterItem/1.0", + "name": "TagValue", + "type": 2, + "query": "Resources\r\n| mvexpand tags\r\n| extend tagName = tostring(bag_keys(tags)[0])\r\n| extend tagValue = tostring(tags[tagName])\r\n| where tags != '' and tags != '[]' and tostring(bag_keys(tags)[0]) == '{TagName}'\r\n| distinct tagValue\r\n| sort by tagValue asc", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [] + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null, + "label": "Tag Value" + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "75", + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "CostInformation" + }, + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "Welcome" + } + ], + "name": "parameters - Filters" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "eae8a0d2-14e6-4cd1-a2d2-fd6b207cf517", + "version": "KqlParameterItem/1.0", + "name": "Location", + "type": 2, + "query": "Resources\r\n| where type =~ 'Microsoft.Compute/virtualMachines'\r\n| project name, location\r\n| summarize count () by location\r\n| project location", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::1" + ] + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::1", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "label": "Resource Location" + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "25", + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "AHB" + }, + "name": "parameters - location" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "eae8a0d2-14e6-4cd1-a2d2-fd6b207cf517", + "version": "KqlParameterItem/1.0", + "name": "Location", + "type": 2, + "query": "Resources\r\n| where type =~ 'Microsoft.Compute/virtualMachines'\r\n| project name, location\r\n| summarize count () by location\r\n| project location", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::1" + ] + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::1", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "label": "Resource Location" + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "25", + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "AHB" + }, + "name": "parameters - location" + }, + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "0211f413-9f36-4750-9ef2-d382ba30ba6c", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Storage Accounts", + "subTarget": "Storage", + "preText": "VM", + "style": "link" + }, + { + "id": "dbe9a7fb-6ab1-4de1-a98b-4ec8a9af906c", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Managed Disks", + "subTarget": "Disks", + "style": "link" + }, + { + "id": "86ff248b-1ce4-4194-8cd4-b1e0a9956b5d", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Backup", + "subTarget": "Backup", + "style": "link" + }, + { + "id": "6d563f46-7150-458c-9ee4-0558abe8e29b", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Advisor recommendations", + "subTarget": "advisorStorage", + "style": "link" + } + ] + }, + "name": "links - Storage" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "## Idle backups\r\n\r\nReview protected items backup activity to determine if there are items that have not been backed up in the last 90 days. This could either mean that the underlying resource that's being backed up doesn't exist anymore or there's some issue with the resource that's preventing backups from being taken reliably.\r\n", + "style": "upsell" + }, + "name": "text - idleBackup" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "recoveryservicesresources\r\n| where type =~ 'microsoft.recoveryservices/vaults/backupfabrics/protectioncontainers/protecteditems'\r\n| extend vaultId = tostring(properties.vaultId),resourceId = tostring(properties.sourceResourceId),idleBackup= datetime_diff('day', now(), todatetime(properties.lastBackupTime)) > 90, resourceType=tostring(properties.workloadType), protectionState=tostring(properties.protectionState),lastBackupTime=tostring(properties.lastBackupTime), resourceGroup=strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup),lastBackupDate=todatetime(properties.lastBackupTime)\r\n| where idleBackup != 0\r\n| project resourceId,vaultId,idleBackup,lastBackupDate,resourceType,protectionState,lastBackupTime,location,resourceGroup,subscriptionId\r\n| join kind = inner(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | extend vaultId = id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | project vaultId\r\n )\r\n on vaultId\r\n | project-away vaultId1", + "size": 0, + "title": "Idle backups", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "idleBackup", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": ">", + "thresholdValue": "0", + "representation": "2", + "text": "No backup in the last 90 days" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "" + } + ] + } + }, + { + "columnMatch": "resourceGroup", + "formatter": 14, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "labelSettings": [ + { + "columnId": "resourceId", + "label": "Resource ID" + }, + { + "columnId": "idleBackup", + "label": "Backup activity" + }, + { + "columnId": "lastBackupDate", + "label": "Last backup date" + }, + { + "columnId": "resourceType", + "label": "Resource type" + }, + { + "columnId": "protectionState", + "label": "Protection state" + }, + { + "columnId": "lastBackupTime", + "label": "Last backup time" + }, + { + "columnId": "location", + "label": "Location" + }, + { + "columnId": "resourceGroup", + "label": "Resource group" + }, + { + "columnId": "subscriptionId", + "label": "Subscription ID" + } + ] + }, + "sortBy": [] + }, + "name": "query - idleBackups" + }, + { + "type": 1, + "content": { + "json": "## Backup storage redundancy settings\r\n\r\nBy default, when you configure backup for resources, geo-redundant storage (GRS) replication is applied to these backups. While this is the recommended storage replication option as it creates more redundancy for your critical data, you can choose to protect items using locally-redundant storage (LRS) if that meets your backup availability needs for dev-test workloads. Using LRS instead of GRS halves the cost of your backup storage. \r\n\r\n🖱️Click on each vault to see the configured storage replication\r\n", + "style": "upsell" + }, + "name": "text - backupReplication" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources\r\n| where type == 'microsoft.recoveryservices/vaults'\r\n| where resourceGroup in ({ResourceGroup})\r\n| extend skuTier = tostring(sku['tier']), skuName = tostring(sku['name']), resourceGroup=strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup),redundancySettings = tostring(properties.redundancySettings['standardTierStorageRedundancy'])\r\n| order by id asc\r\n| project id,redundancySettings, resourceGroup, location,subscriptionId, skuTier, skuName\r\n| join kind = innerunique (\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | extend vaultId = id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | project id\r\n)\r\non id\r\n| project-away id1\r\n", + "size": 0, + "title": "Recovery vaults storage replication ", + "exportedParameters": [ + { + "fieldName": "RGVault", + "parameterName": "resourceGroupVault", + "parameterType": 1 + }, + { + "fieldName": "subscriptionId", + "parameterName": "subscriptionId", + "parameterType": 1 + }, + { + "fieldName": "name", + "parameterName": "vaultName", + "parameterType": 1 + } + ], + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "redundancySettings", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "GeoRedundant", + "representation": "Globe", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "ResourceFlat", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "RGVault", + "formatter": 5 + }, + { + "columnMatch": "name", + "formatter": 5 + } + ] + } + }, + "name": "query - backupStorageReplication" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "Backup" + }, + "name": "group - Backup" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Storage accounts\r\nGeneral-purpose v2 storage accounts support the latest Azure Storage features and incorporate all of the functionality of general-purpose v1 and Blob storage accounts. General-purpose v2 accounts are recommended for most storage scenarios.\r\n\r\n1. General-purpose v2 accounts deliver the lowest per-gigabyte capacity prices for Azure Storage, as well as industry-competitive transaction prices.\r\n2. General-purpose v2 accounts support default account access tiers of hot or cool and blob level tiering between hot, cool, or archive.\r\n3. General-purpose v2 accounts allows you to also use lifecycle management to optimize your storage cost", + "style": "upsell" + }, + "name": "Storage accounts" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where type =~ 'Microsoft.Storage/StorageAccounts' and kind !='StorageV2' and kind !='FileStorage' and kind != 'BlockBlobStorage'\r\n| where resourceGroup in ({ResourceGroup})\r\n| extend StorageAccountName=name, SAKind=kind,AccessTier=tostring(properties.accessTier),SKUName=sku.name, SKUTier=sku.tier, Location=location\r\n| order by id asc\r\n| project id,StorageAccountName, SKUName, SKUTier, SAKind,AccessTier, resourceGroup, Location, subscriptionId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n )\r\n on id", + "size": 0, + "title": "Storage accounts which are not v2", + "noDataMessage": "All storage accounts are General-purpose v2", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "SKUName", + "formatter": 1 + }, + { + "columnMatch": "SKUTier", + "formatter": 1 + }, + { + "columnMatch": "SAKind", + "formatter": 1 + }, + { + "columnMatch": "AccessTier", + "formatter": 1 + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "id1", + "formatter": 5 + }, + { + "columnMatch": "storageaccount", + "formatter": 13, + "formatOptions": { + "linkTarget": "Resource", + "subTarget": "insights", + "linkIsContextBlade": true, + "showIcon": true + } + } + ], + "sortBy": [ + { + "itemKey": "$gen_link_id_0", + "sortOrder": 1 + } + ], + "labelSettings": [ + { + "columnId": "id", + "label": "Resource ID" + }, + { + "columnId": "StorageAccountName", + "label": "Name" + }, + { + "columnId": "SKUName", + "label": "SKU" + }, + { + "columnId": "SKUTier", + "label": "SKU Tier" + }, + { + "columnId": "SAKind", + "label": "Kind" + }, + { + "columnId": "AccessTier", + "label": "Access Tier" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "Location", + "label": "Location" + }, + { + "columnId": "subscriptionId", + "label": "Subscription Name" + } + ] + }, + "sortBy": [ + { + "itemKey": "$gen_link_id_0", + "sortOrder": 1 + } + ] + }, + "name": "Get-Storagev1" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "Storage" + }, + "name": "group - StorageAccount" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "## Unattached Managed Disks\r\n\r\nReview Managed Disks that are not attached to any Virtual machine.\r\n\r\n## Last Modified Date\r\nClick on a cell in the specified row to view the last modified date. This may help identify when the disk became idle.\r\n\r\n", + "style": "upsell" + }, + "name": "text - 3" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where type =~ 'microsoft.compute/disks' and managedBy == \"\"\r\n| extend diskState = tostring(properties.diskState)\r\n| where (tags !contains \"kubernetes.io-created-for-pvc\") and tags !contains \"ASR-ReplicaDisk\" and tags !contains \"asrseeddisk\" and tags !contains \"RSVaultBackup\"\r\n| where (managedBy == \"\" and diskState != 'ActiveSAS')\r\nor (diskState == 'Unattached' and diskState != 'ActiveSAS')\r\n| extend DiskId=id, DiskIDfull=id, DiskName=name, SKUName=sku.name, SKUTier=sku.tier, DiskSizeGB=tostring(properties.diskSizeGB), Location=location, TimeCreated=tostring(properties.timeCreated), QuickFix=id, SubId=subscriptionId\r\n| order by DiskId asc \r\n| project DiskId,DiskIDfull, DiskName, DiskSizeGB, SKUName, SKUTier, resourceGroup, QuickFix, Location, TimeCreated, subscriptionId,SubId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | extend DiskId = id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct DiskId\r\n )\r\n on DiskId", + "size": 0, + "title": "Unattached disks", + "noDataMessage": "There aren't any unattached disks!", + "noDataMessageStyle": 3, + "exportedParameters": [ + { + "fieldName": "DiskIDfull", + "parameterName": "DiskID" + }, + { + "fieldName": "DiskName", + "parameterName": "DiskName", + "parameterType": 1 + }, + { + "fieldName": "resourceGroup", + "parameterName": "ResourceGroup", + "parameterType": 1 + }, + { + "fieldName": "SubId", + "parameterName": "subscriptionId", + "parameterType": 1 + } + ], + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "table", + "gridSettings": { + "formatters": [ + { + "columnMatch": "DiskIDfull", + "formatter": 5 + }, + { + "columnMatch": "QuickFix", + "formatter": 7, + "formatOptions": { + "linkTarget": "ArmAction", + "linkLabel": "Remove Idle Disk", + "linkIsContextBlade": true, + "templateRunContext": { + "componentIdSource": "column", + "componentId": "DiskId", + "templateUriSource": "static", + "templateUri": "https://raw.githubusercontent.com/sebassem/MS-learn-Workbooks/main/Deploy-Tag.json", + "templateParameters": [ + { + "name": "DiskID", + "source": "static", + "value": "DiskId", + "kind": "stringValue" + } + ], + "titleSource": "static", + "title": "Remove Idle Disk", + "descriptionSource": "static", + "description": "# Description\r\nThis ARM Template will remove the selected disk.\r\n\r\n# Actions:\r\n- Click \"Remove Idle Disk\" to remove the selected item.\r\n- Click View Template to examine the template and parameters used during deployment\r\n\r\n\r\n\r\n", + "runLabelSource": "static", + "runLabel": "Remove Idle Disk" + }, + "armActionContext": { + "path": "/{DiskID}?api-version=2021-04-01", + "headers": [], + "params": [ + { + "key": "DiskID", + "value": "" + } + ], + "httpMethod": "DELETE", + "title": "Remove Idle Disks", + "description": "# Disk Deletion Warning: {DiskName}\r\n\r\n**Attention!**\r\n\r\nThis action will permanently remove the disk with the name **{DiskName}**. Please ensure that this disk is not currently in use and that you are deleting the correct disk.\r\n\r\n**Resource Details:**\r\n\r\n- Disk Name: {DiskName}\r\n- Resource Group: {ResourceGroup}\r\n\r\n### Required RBAC Permissions\r\n\r\nTo perform this action, you need to have **Contributor** permissions on the Resource Group where the disk is located.\r\n\r\nPlease review the information carefully before proceeding with the deletion.\r\n", + "actionName": "Removing Idle Dsk", + "runLabel": "I understand, remove disk {DiskName}" + } + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "rowLimit": 1000, + "labelSettings": [ + { + "columnId": "DiskId", + "label": "Resource ID" + }, + { + "columnId": "DiskName", + "label": "Name" + }, + { + "columnId": "DiskSizeGB", + "label": "Disk Size (GB)" + }, + { + "columnId": "SKUName", + "label": "SKU" + }, + { + "columnId": "SKUTier", + "label": "SKU Tier" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "QuickFix", + "label": "Delete disk?" + }, + { + "columnId": "Location", + "label": "Location" + }, + { + "columnId": "TimeCreated", + "label": "Time Created" + }, + { + "columnId": "subscriptionId", + "label": "Subscription Name" + } + ] + }, + "sortBy": [] + }, + "customWidth": "80", + "name": "Get-Idle-Disk" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"ARMEndpoint/1.0\",\"data\":null,\"headers\":[],\"method\":\"GET\",\"path\":\"/subscriptions/{subscriptionId}/resources?\",\"urlParams\":[{\"key\":\"api-version\",\"value\":\"2021-04-01\"},{\"key\":\"$expand\",\"value\":\"createdTime,changedTime,provisioningState\"},{\"key\":\"$filter\",\"value\":\"name eq '{DiskName}' and resourceGroup eq'{ResourceGroup}'\"}],\"batchDisabled\":false,\"transformers\":[{\"type\":\"jsonpath\",\"settings\":{\"tablePath\":\"$.value\",\"columns\":[{\"path\":\"$..id\",\"columnid\":\"id\"},{\"path\":\"$..createdTime\",\"columnid\":\"createdTime\"},{\"path\":\"$..changedTime\",\"columnid\":\"changedTime\"},{\"path\":\"$.name\",\"columnid\":\"name\"}]}}]}", + "size": 0, + "title": "Disk last modified date", + "showExportToExcel": true, + "queryType": 12, + "gridSettings": { + "formatters": [ + { + "columnMatch": "id", + "formatter": 5 + }, + { + "columnMatch": "createdTime", + "formatter": 5 + }, + { + "columnMatch": "name", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "id", + "label": "Name" + }, + { + "columnId": "createdTime", + "label": "Created time" + }, + { + "columnId": "changedTime", + "label": "Last change time" + } + ] + } + }, + "customWidth": "20", + "conditionalVisibility": { + "parameterName": "DiskID", + "comparison": "isNotEqualTo", + "value": "" + }, + "name": "IdleDisk date" + } + ] + }, + "name": "Idle Disks Group" + }, + { + "type": 1, + "content": { + "json": "# Premium disks attached to powered off virtual machines\r\nIf the VM associated with these premium disks has been deallocated for an extended period, consider changing the disk SKU to a less expensive option to save on costs. Premium disks are typically used for high-performance workloads, and if the VM is not in use, it might be more economical to downgrade the disk.", + "style": "upsell" + }, + "name": "text - premiumAttachedToPoweredOffVMs" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type == \"microsoft.compute/virtualmachines\"\r\n| extend vmId = tolower(tostring(id)), vmName = name, vmState = tostring(properties.extended.instanceView.powerState.displayStatus),VMRG=resourceGroup\r\n| where vmState == \"VM stopped\" or vmState == \"VM deallocated\"\r\n| extend storageProfile = parse_json(tostring(properties.storageProfile.osDisk))\r\n| extend managedDiskId = tolower(tostring(storageProfile.managedDisk.id))\r\n| join kind=inner (\r\n resources\r\n | where type == \"microsoft.compute/disks\"\r\n | where sku.name == \"Premium_LRS\" or sku.name == \"Premium_ZRS\"\r\n | extend diskId = tolower(tostring(id)), diskName = name, diskSKU=tostring(sku.name), diskTier=tostring(sku.tier)\r\n) on $left.managedDiskId == $right.diskId\r\n| project vmId, vmName, vmState, diskName,VMRG, diskId, diskSKU,diskTier\r\n", + "size": 0, + "title": "Premium disks attached to powered off VMs", + "noDataMessage": "None of your deallocated VMs have premium disks", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "vmName", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "vmId", + "label": "VM ID" + }, + { + "columnId": "vmName", + "label": "VM name" + }, + { + "columnId": "vmState", + "label": "VM state" + }, + { + "columnId": "diskName", + "label": "Disk name" + }, + { + "columnId": "VMRG", + "label": "VM RG" + }, + { + "columnId": "diskId", + "label": "Disk ID" + }, + { + "columnId": "diskSKU", + "label": "Disk SKU" + }, + { + "columnId": "diskTier", + "label": "Disk Tier" + } + ] + } + }, + "name": "query-PremiumDiskDeallocatedVM" + }, + { + "type": 1, + "content": { + "json": "## Old Managed Disks snapshots\r\n\r\nReview Managed Disks snapshots that are older than 30 days\r\n", + "style": "upsell" + }, + "name": "text - 4" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type == 'microsoft.compute/snapshots'\r\n| where resourceGroup in ({ResourceGroup})\r\n| extend TimeCreated = properties.timeCreated\r\n| extend resourceGroup=strcat(\"/subscriptions/\",subscriptionId,\"/resourceGroups/\",resourceGroup)\r\n| where TimeCreated < ago(30d)\r\n| order by id asc \r\n| project id, resourceGroup, location, TimeCreated ,subscriptionId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n )\r\n on id", + "size": 0, + "title": "Disk Snapshots with + 30 Days", + "noDataMessage": "No Snapshots with more than 30 days.", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "resourceGroup", + "formatter": 14, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "TimeCreated", + "formatter": 1 + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "id1", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "id", + "label": "Name" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "location", + "label": "Location" + }, + { + "columnId": "TimeCreated", + "label": "Time Created" + }, + { + "columnId": "subscriptionId", + "label": "Subscription Name" + } + ] + } + }, + "name": "Get-Old-Snapshots" + }, + { + "type": 1, + "content": { + "json": "## Managed Disks snapshots using Premium storage\r\n\r\nTo save 60% of cost, we recommend storing your snapshots in Standard Storage, regardless of the storage type of the parent disk. It is the default option for Managed Disks snapshots. Migrate your snapshot from Premium to Standard Storage.\r\n", + "style": "upsell" + }, + "name": "text - 5" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type == 'microsoft.compute/snapshots'\r\n| where resourceGroup in ({ResourceGroup})\r\n| extend StorageSku = tostring(sku.tier), resourceGroup=strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup),diskSize=tostring(properties.diskSizeGB)\r\n| where StorageSku == \"Premium\"\r\n| project id,name,StorageSku,diskSize,location,resourceGroup,subscriptionId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n )\r\n on id\r\n", + "size": 0, + "title": "Snapshots using premium storage", + "noDataMessage": "No snapshots are using Premium storage", + "noDataMessageStyle": 3, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "resourceGroup", + "formatter": 14, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "id1", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "id", + "label": "Resource Id" + }, + { + "columnId": "name", + "label": "Name" + }, + { + "columnId": "StorageSku", + "label": "SKU" + }, + { + "columnId": "diskSize", + "label": "Disk Size (GB)" + }, + { + "columnId": "location", + "label": "Location" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "subscriptionId", + "label": "Subscription Id" + } + ] + } + }, + "name": "query - Snapshots using premium storage" + }, + { + "type": 1, + "content": { + "json": "## Orphaned Managed Disks snapshots\r\n\r\nReview snapshots with deleted source disks.\r\n", + "style": "upsell" + }, + "name": "text - 6" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type == 'microsoft.compute/snapshots'\r\n| where resourceGroup in ({ResourceGroup})\r\n| extend parentDisk = properties.creationData.sourceResourceId, diskSize=tostring(properties.diskSizeGB),resourceGroup=strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup)\r\n| project id,parentDisk,diskSize,location,resourceGroup,subscriptionId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n )\r\n on id\r\n", + "size": 0, + "title": "All Managed Disks snapshots", + "noDataMessage": "No snapshots found", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "labelSettings": [ + { + "columnId": "id", + "label": "Resource Id" + }, + { + "columnId": "parentDisk", + "label": "Parent Disk Resource Id" + }, + { + "columnId": "diskSize", + "label": "Disk size (GB)" + }, + { + "columnId": "location", + "label": "Location" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "subscriptionId", + "label": "Subscription Id" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "IsVisible", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "query - Retrieve all snapshots" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type == 'microsoft.compute/disks'\r\n| project id\r\n", + "size": 0, + "title": "All managed disks", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "labelSettings": [ + { + "columnId": "id", + "label": "Resource Id" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "isVisible", + "comparison": "isEqualTo", + "value": "True" + }, + "name": "query - Retrieve all managed disks" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"d0a11ffb-579b-4259-827d-7ea62e3021fe\",\"mergeType\":\"leftanti\",\"leftTable\":\"query - Retrieve all snapshots\",\"rightTable\":\"query - Retrieve all managed disks\",\"leftColumn\":\"parentDisk\",\"rightColumn\":\"id\"}],\"projectRename\":[{\"originalName\":\"[query - Retrieve all snapshots].id\",\"mergedName\":\"Resource Id\",\"fromId\":\"d0a11ffb-579b-4259-827d-7ea62e3021fe\"},{\"originalName\":\"[query - Retrieve all snapshots].parentDisk\",\"mergedName\":\"Parent Disk Resource Id\",\"fromId\":\"d0a11ffb-579b-4259-827d-7ea62e3021fe\"},{\"originalName\":\"[query - Retrieve all snapshots].diskSize\",\"mergedName\":\"Disk size (GB)\",\"fromId\":\"d0a11ffb-579b-4259-827d-7ea62e3021fe\"},{\"originalName\":\"[query - Retrieve all snapshots].location\",\"mergedName\":\"Location\",\"fromId\":\"d0a11ffb-579b-4259-827d-7ea62e3021fe\"},{\"originalName\":\"[query - Retrieve all snapshots].resourceGroup\",\"mergedName\":\"Resource Group\",\"fromId\":\"d0a11ffb-579b-4259-827d-7ea62e3021fe\"},{\"originalName\":\"[query - Retrieve all snapshots].subscriptionId\",\"mergedName\":\"Subscription Id\",\"fromId\":\"d0a11ffb-579b-4259-827d-7ea62e3021fe\"},{\"originalName\":\"[query - Retrieve all snapshots].id1\",\"mergedName\":\"id1\",\"fromId\":\"d0a11ffb-579b-4259-827d-7ea62e3021fe\"}]}", + "size": 0, + "title": "Snapshots with deleted source disk", + "noDataMessage": "No orphaned snapshots found", + "noDataMessageStyle": 3, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "Parent Disk Resource Id", + "formatter": 5 + }, + { + "columnMatch": "Subscription Id", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "id1", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "Resource Id", + "label": "Resource Id" + }, + { + "columnId": "Parent Disk Resource Id", + "label": "Parent Disk resource Id" + }, + { + "columnId": "Disk size (GB)", + "label": "Disk size (GB)" + }, + { + "columnId": "Location", + "label": "Location" + }, + { + "columnId": "Resource Group", + "label": "Resource Group" + }, + { + "columnId": "Subscription Id", + "label": "Subscription Id" + } + ] + } + }, + "showPin": false, + "name": "query - orphaned snapshots" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "Disks" + }, + "name": "Managed Disks Group" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "advisorresources\r\n| where type =~ 'microsoft.advisor/recommendations'\r\n| where resourceGroup in ({ResourceGroup})\r\n| where properties.category == 'Cost' and properties.lastUpdated >= ago(1d)\r\n| extend AffectedResource=tostring(properties.resourceMetadata.resourceId), Category=tostring(properties.category), SubCategory=tostring(properties.impactedField), Impact=tostring(properties.impact),resourceGroup,subscriptionId,Recommendation=tostring(properties.shortDescription.problem), id, stableId = name, recommendationTypeId = tostring(properties.recommendationTypeId), maxCpuP95 = properties.extendedProperties.MaxCpuP95\r\n| where SubCategory has \"Microsoft.Storage\"\r\n| join kind = leftouter\r\n(advisorresources | where type=~'microsoft.advisor/suppressions'\r\n| extend tokens = split(id, '/')\r\n| extend stableId = iff(array_length(tokens) > 3, tokens[(array_length(tokens)-3)], '')\r\n| extend expirationTimeStamp = todatetime(iff(strcmp(tostring(properties.ttl), '-1') == 0, '9999-12-31', properties.expirationTimeStamp))\r\n| where expirationTimeStamp > now()\r\n| project stableId, expirationTimeStamp)\r\non stableId\r\n| where isempty(expirationTimeStamp)\r\n| project AffectedResource=tostring(properties.resourceMetadata.resourceId), Category=tostring(properties.category), SubCategory=tostring(properties.impactedField), Recommendation=tostring(properties.shortDescription.problem), Impact=tostring(properties.impact),resourceGroup,subscriptionId, id, stableId = name, recommendationTypeId = tostring(properties.recommendationTypeId), maxCpuP95 = properties.extendedProperties.MaxCpuP95\r\n| join kind = leftouter\r\n(advisorresources | where type =~ 'microsoft.advisor/configurations' | where isempty(resourceGroup) == true\r\n| project subscriptionId, excludeRecomm = properties.exclude, lowCpuThreshold = properties.lowCpuThreshold, AffectedResource=tostring(properties.resourceMetadata.resourceId),Impact=properties.impact,resourceGroup,AdditionaInfo=properties.extendedProperties,Recommendation=tostring(properties.shortDescription.problem))\r\non subscriptionId\r\n| extend isActive1 = iff(isnull(excludeRecomm), true, tobool(excludeRecomm) == false)\r\n| extend isActive2 = iff(recommendationTypeId == 'e10b1381-5f0a-47ff-8c7b-37bd13d7c974', iff((isnotempty(lowCpuThreshold) and isnotempty(maxCpuP95)), toint(maxCpuP95) < toint(lowCpuThreshold), iff((isempty(maxCpuP95) or toint(maxCpuP95) < 5), true, false)), true)\r\n| where isActive1 == true and isActive2 == true\r\n| join kind = leftouter\r\n(advisorresources | where type =~ 'microsoft.advisor/configurations' | where isnotempty(resourceGroup) == true\r\n| project subscriptionId, resourceGroup, excludeProperty = properties.exclude)\r\non subscriptionId, resourceGroup\r\n| extend isActive3 = iff(isnull(excludeProperty), true, tobool(excludeProperty) == false)\r\n| where isActive3 == true\r\n| project-away subscriptionId1, subscriptionId2, AffectedResource1, isActive2, isActive3, Impact1, Recommendation1, resourceGroup1, resourceGroup2\r\n| where resourceGroup in ({ResourceGroup})", + "size": 0, + "title": "Azure Advisor Cost recommendations", + "noDataMessage": "You are following all of our cost recommendations for Storage", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "Group", + "formatter": 1 + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "id", + "formatter": 5 + }, + { + "columnMatch": "stableId", + "formatter": 5 + }, + { + "columnMatch": "recommendationTypeId", + "formatter": 5 + }, + { + "columnMatch": "maxCpuP95", + "formatter": 5 + }, + { + "columnMatch": "excludeRecomm", + "formatter": 5 + }, + { + "columnMatch": "lowCpuThreshold", + "formatter": 5 + }, + { + "columnMatch": "AdditionaInfo", + "formatter": 5, + "formatOptions": { + "customColumnWidthSetting": "19ch" + } + }, + { + "columnMatch": "isActive1", + "formatter": 5 + }, + { + "columnMatch": "excludeProperty", + "formatter": 5 + } + ], + "rowLimit": 1000, + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "Recommendation" + ], + "expandTopLevel": true + }, + "labelSettings": [ + { + "columnId": "AffectedResource", + "label": "Affected Resource" + }, + { + "columnId": "Category", + "label": "Recommendation Category" + }, + { + "columnId": "SubCategory", + "label": "Affected Resource Type" + }, + { + "columnId": "Recommendation", + "label": "Recommendation" + }, + { + "columnId": "Impact", + "label": "Impact" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "subscriptionId", + "label": "Subscription ID" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "isVisible", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "Get-AdvisorRecommendations-Storage" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": " resources\r\n | where resourceGroup in ({ResourceGroup})\r\n | where type has \"Microsoft.Storage\"\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend AffectedResource=id,ResourceRG=resourceGroup\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n | project id", + "size": 0, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "isVisible", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "query - tags - list all storageresources" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"e84cba0d-e501-4f55-a761-9126fb305030\",\"mergeType\":\"innerunique\",\"leftTable\":\"Get-AdvisorRecommendations-Storage\",\"rightTable\":\"query - tags - list all storageresources\",\"leftColumn\":\"AffectedResource\",\"rightColumn\":\"id\"}],\"projectRename\":[{\"originalName\":\"[Get-AdvisorRecommendations-Storage].AffectedResource\",\"mergedName\":\"Affected Resource\",\"fromId\":\"e84cba0d-e501-4f55-a761-9126fb305030\"},{\"originalName\":\"[Get-AdvisorRecommendations-Storage].Category\",\"mergedName\":\"Recommendation Category\",\"fromId\":\"e84cba0d-e501-4f55-a761-9126fb305030\"},{\"originalName\":\"[Get-AdvisorRecommendations-Storage].SubCategory\",\"mergedName\":\"Affected Resource Type\",\"fromId\":\"e84cba0d-e501-4f55-a761-9126fb305030\"},{\"originalName\":\"[Get-AdvisorRecommendations-Storage].Recommendation\",\"mergedName\":\"Recommendation\",\"fromId\":\"e84cba0d-e501-4f55-a761-9126fb305030\"},{\"originalName\":\"[Get-AdvisorRecommendations-Storage].Impact\",\"mergedName\":\"Impact\",\"fromId\":\"e84cba0d-e501-4f55-a761-9126fb305030\"},{\"originalName\":\"[Get-AdvisorRecommendations-Storage].resourceGroup\",\"mergedName\":\"Resource Group\",\"fromId\":\"e84cba0d-e501-4f55-a761-9126fb305030\"},{\"originalName\":\"[Get-AdvisorRecommendations-Storage].subscriptionId\",\"mergedName\":\"Subscription ID\",\"fromId\":\"e84cba0d-e501-4f55-a761-9126fb305030\"},{\"originalName\":\"[query - tags - list all storageresources].id\"},{\"originalName\":\"[Get-AdvisorRecommendations-Storage].excludeProperty\"},{\"originalName\":\"[Get-AdvisorRecommendations-Storage].isActive1\"},{\"originalName\":\"[Get-AdvisorRecommendations-Storage].AdditionaInfo\"},{\"originalName\":\"[Get-AdvisorRecommendations-Storage].lowCpuThreshold\"},{\"originalName\":\"[Get-AdvisorRecommendations-Storage].excludeRecomm\"},{\"originalName\":\"[Get-AdvisorRecommendations-Storage].recommendationTypeId\"},{\"originalName\":\"[Get-AdvisorRecommendations-Storage].maxCpuP95\"},{\"originalName\":\"[Get-AdvisorRecommendations-Storage].stableId\"},{\"originalName\":\"[Get-AdvisorRecommendations-Storage].id\"}]}", + "size": 0, + "title": "Azure Advisor Cost recommendations", + "noDataMessage": "You are following all of our cost recommendations for Storage", + "noDataMessageStyle": 3, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "Affected Resource Type", + "formatter": 5 + }, + { + "columnMatch": "Subscription ID", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "rowLimit": 1000, + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "Recommendation" + ] + } + } + }, + "showPin": false, + "name": "query - Merge - Storage Advisor recommendations" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "advisorStorage" + }, + "name": "AdvisorGroupStorage" + } + ] + }, + "name": "group - 0" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "UsageOptimization" + }, + { + "parameterName": "SelectedRateOptimizationTab", + "comparison": "isEqualTo", + "value": "Storage" + } + ], + "name": "StorageGroup", + "styleSettings": { + "showBorder": true + } + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "template", + "loadFromTemplateId": "", + "items": [ + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "94bd2bd0-5aa8-4df6-8cf7-603407f4e2d8", + "version": "KqlParameterItem/1.0", + "name": "Subscription", + "type": 6, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "includeAll": false, + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::all", + "label": " Subscription" + }, + { + "id": "faa42c49-ab77-42a1-9aaf-d8508b9408af", + "version": "KqlParameterItem/1.0", + "name": "ResourceGroup", + "label": "Resource Group", + "type": 2, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": "resources\r\n| distinct resourceGroup", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "showDefault": false + }, + "defaultValue": "value::all", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + { + "id": "99a44dfa-30e2-4b2e-80a8-e05d2daab672", + "version": "KqlParameterItem/1.0", + "name": "SingleSubHidden", + "type": 1, + "isRequired": true, + "query": "resourcecontainers\r\n| where type==\"microsoft.resources/subscriptions\"\r\n| take 1\r\n| project subscriptionId", + "crossComponentResources": [ + "{Subscription}" + ], + "isHiddenWhenLocked": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "label": "Hidden Subscription" + }, + { + "id": "a02c21a6-cd5e-4e02-bb87-00993a06d8e8", + "version": "KqlParameterItem/1.0", + "name": "TagName", + "type": 2, + "query": "Resources\r\n| where tags != '' and tags != '[]'\r\n| mvexpand tags\r\n| extend tagName = tostring(bag_keys(tags)[0])\r\n| distinct tagName\r\n| sort by tagName asc", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [] + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null, + "label": "Tag Name" + }, + { + "id": "add52b5b-2e8d-45d3-a304-f6d8f4b205f7", + "version": "KqlParameterItem/1.0", + "name": "TagValue", + "type": 2, + "query": "Resources\r\n| mvexpand tags\r\n| extend tagName = tostring(bag_keys(tags)[0])\r\n| extend tagValue = tostring(tags[tagName])\r\n| where tags != '' and tags != '[]' and tostring(bag_keys(tags)[0]) == '{TagName}'\r\n| distinct tagValue\r\n| sort by tagValue asc", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [] + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null, + "label": "Tag Value" + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "75", + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "CostInformation" + }, + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "Welcome" + } + ], + "name": "parameters - Filters" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "1fc44b9a-2dd3-4b1f-bebd-b89d4ba6dfec", + "version": "KqlParameterItem/1.0", + "name": "Location", + "type": 2, + "query": "Resources\r\n| where type =~ 'Microsoft.Compute/virtualMachines'\r\n| project name, location\r\n| summarize count () by location\r\n| project location", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::1" + ] + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::1", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "label": "Resource Location" + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "25", + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "AHB" + }, + "name": "parameters - location" + }, + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "0211f413-9f36-4750-9ef2-d382ba30ba6c", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Virtual machines", + "subTarget": "VM", + "preText": "VM", + "style": "link" + }, + { + "id": "8a2fa734-a30e-404e-bf99-927c1891d4b9", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Virtual machine scale sets", + "subTarget": "VMSS", + "style": "link" + }, + { + "id": "6d563f46-7150-458c-9ee4-0558abe8e29b", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Advisor recommendations", + "subTarget": "advisorCompute", + "style": "link" + } + ] + }, + "name": "links - Compute" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Virtual Machines\r\n## Stopped virtual machines\r\nA virtual machine in a stopped state is still allocated the resources it was assigned, such as CPU and memory, but the VM itself is powered off. This allows for a quick startup when needed, but you are still billed for the allocated resources.", + "style": "upsell" + }, + "name": "text - StoppedVM" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where type =~ 'microsoft.compute/virtualmachines' and tostring(properties.extended.instanceView.powerState.displayStatus) != 'VM deallocated' and tostring(properties.extended.instanceView.powerState.displayStatus) != 'VM running'\r\n| where resourceGroup in ({ResourceGroup})\r\n| extend PowerState=tostring(properties.extended.instanceView.powerState.displayStatus), VMLocation=location, resourceGroup=strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup)\r\n| order by id asc\r\n| project id, PowerState, VMLocation, resourceGroup, subscriptionId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n )\r\n on id\r\n | project-away id1", + "size": 0, + "title": "Virtual Machines in a Stopped State", + "noDataMessage": "You have no VMs in a stopped state", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ] + } + }, + "name": "Get-StoppedVM" + }, + { + "type": 1, + "content": { + "json": "## Deallocated virtual machines\r\nA virtual machine in a deallocated state is not only powered off, but the underlying host infrastructure is also released, resulting in no charges for the allocated resources while the VM is in this state. However, some Azure resources such as disks and networking continue to incur charges.", + "style": "upsell" + }, + "name": "text - DeallocatedVM" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where type =~ 'microsoft.compute/virtualmachines' and tostring(properties.extended.instanceView.powerState.displayStatus) == 'VM deallocated'\r\n| where resourceGroup in ({ResourceGroup})\r\n| extend PowerState=tostring(properties.extended.instanceView.powerState.displayStatus), VMLocation=location, resourceGroup=strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup)\r\n| order by id asc\r\n| project id, PowerState, VMLocation, resourceGroup, subscriptionId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n )\r\n on id\r\n | project-away id1", + "size": 0, + "title": "Virtual Machines in a deallocated State", + "noDataMessage": "You have no VMs in a deallocated state", + "noDataMessageStyle": 3, + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "filter": true + } + }, + "name": "query - vmDeallocatedState" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "### Explore Different Processor Architectures to Optimize Costs\r\n\r\nDifferent processor architectures may offer cost advantages depending on your workload requirements. By exploring various processor types, you may find opportunities to reduce compute costs.\r\n\r\nConsider evaluating different architectures to determine the best fit for your needs.\r\n", + "style": "info" + }, + "name": "Text Processor type" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type == 'microsoft.compute/virtualmachines'\r\n| extend vmSize = properties.hardwareProfile.vmSize\r\n| extend processorType = case(\r\n // ARM Processors\r\n vmSize has \"Epsv5\" or vmSize has \"Epdsv5\" or vmSize has \"Dpsv5\" or vmSize has \"Dpdsv\", \"ARM\",\r\n // AMD Processors\r\n vmSize has \"Standard_D2a\" or vmSize has \"Standard_D4a\" or vmSize has \"Standard_D8a\" or vmSize has \"Standard_D16a\" or vmSize has \"Standard_D32a\" or vmSize has \"Standard_D48a\" or vmSize has \"Standard_D64a\" or vmSize has \"Standard_D96a\" or vmSize has \"Standard_D2as\" or vmSize has \"Standard_D4as\" or vmSize has \"Standard_D8as\" or vmSize has \"Standard_D16as\" or vmSize has \"Standard_D32as\" or vmSize has \"Standard_D48as\" or vmSize has \"Standard_D64as\" or vmSize has \"Standard_D96as\", \"AMD\",\r\n \"Intel\"\r\n)\r\n| summarize count() by processorType\r\n", + "size": 0, + "title": "ProcessorType per VM", + "noDataMessage": "There are no VMs in your environment.", + "noDataMessageStyle": 5, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart", + "tileSettings": { + "showBorder": false + } + }, + "customWidth": "50", + "name": "ProcessorType per VM" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type == 'microsoft.compute/virtualmachines'\r\n| extend vmSize = properties.hardwareProfile.vmSize\r\n| extend processorType = case(\r\n // ARM Processors\r\n vmSize has \"Epsv5\" or vmSize has \"Epdsv5\" or vmSize has \"Dpsv5\" or vmSize has \"Dpdsv\", \"ARM\",\r\n // AMD Processors\r\n vmSize has \"Standard_D2a\" or vmSize has \"Standard_D4a\" or vmSize has \"Standard_D8a\" or vmSize has \"Standard_D16a\" or vmSize has \"Standard_D32a\" or vmSize has \"Standard_D48a\" or vmSize has \"Standard_D64a\" or vmSize has \"Standard_D96a\" or vmSize has \"Standard_D2as\" or vmSize has \"Standard_D4as\" or vmSize has \"Standard_D8as\" or vmSize has \"Standard_D16as\" or vmSize has \"Standard_D32as\" or vmSize has \"Standard_D48as\" or vmSize has \"Standard_D64as\" or vmSize has \"Standard_D96as\" or vmSize has \"Standard_D2ads\" or vmSize has \"Standard_D4ads\"or vmSize has \"Standard_D8ads\" or vmSize has \"Standard_D16ads\" or vmSize has \"Standard_D32ads\"or vmSize has \"Standard_D48ads\"or vmSize has \"Standard_D64ads\"or vmSize has \"Standard_D96ads\", \"AMD\",\r\n \"Intel\"\r\n)\r\n| project vmName = name, processorType, vmSize, resourceGroup\r\n", + "size": 0, + "title": "List of VMs per processor type", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "customWidth": "50", + "name": "query - 1" + } + ] + }, + "name": "Group VM per Processor Type" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "VM" + }, + "name": "group - VMs" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "advisorresources\r\n| where type =~ 'microsoft.advisor/recommendations'\r\n| where resourceGroup in ({ResourceGroup})\r\n| where properties.category == 'Cost' and properties.lastUpdated >= ago(1d)\r\n| extend AffectedResource=tostring(properties.resourceMetadata.resourceId), Category=tostring(properties.category), SubCategory=tostring(properties.impactedField), Impact=tostring(properties.impact),subscriptionId,Recommendation=tostring(properties.shortDescription.problem), id, stableId = name, recommendationTypeId = tostring(properties.recommendationTypeId), maxCpuP95 = properties.extendedProperties.MaxCpuP95,resourceGroup=strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup)\r\n| join kind = leftouter\r\n(advisorresources | where type=~'microsoft.advisor/suppressions'\r\n| extend tokens = split(id, '/')\r\n| extend stableId = iff(array_length(tokens) > 3, tokens[(array_length(tokens)-3)], '')\r\n| extend expirationTimeStamp = todatetime(iff(strcmp(tostring(properties.ttl), '-1') == 0, '9999-12-31', properties.expirationTimeStamp))\r\n| where expirationTimeStamp > now()\r\n| project stableId, expirationTimeStamp)\r\non stableId\r\n| where isempty(expirationTimeStamp)\r\n| project AffectedResource=tostring(properties.resourceMetadata.resourceId), Category=tostring(properties.category), SubCategory=tostring(properties.impactedField), Recommendation=tostring(properties.shortDescription.problem), Impact=tostring(properties.impact),resourceGroup,subscriptionId, id, stableId = name, recommendationTypeId = tostring(properties.recommendationTypeId), maxCpuP95 = properties.extendedProperties.MaxCpuP95\r\n| join kind = leftouter\r\n(advisorresources | where type =~ 'microsoft.advisor/configurations' | where isempty(resourceGroup) == true\r\n| project subscriptionId, excludeRecomm = properties.exclude, lowCpuThreshold = properties.lowCpuThreshold, AffectedResource=tostring(properties.resourceMetadata.resourceId),Impact=tostring(properties.impact),resourceGroup,AdditionaInfo=properties.extendedProperties,Recommendation=tostring(properties.shortDescription.problem))\r\non subscriptionId\r\n| extend isActive1 = iff(isnull(excludeRecomm), true, tobool(excludeRecomm) == false)\r\n| extend isActive2 = iff(recommendationTypeId == 'e10b1381-5f0a-47ff-8c7b-37bd13d7c974', iff((isnotempty(lowCpuThreshold) and isnotempty(maxCpuP95)), toint(maxCpuP95) < toint(lowCpuThreshold), iff((isempty(maxCpuP95) or toint(maxCpuP95) < 5), true, false)), true)\r\n| where isActive1 == true and isActive2 == true\r\n| join kind = leftouter\r\n(advisorresources | where type =~ 'microsoft.advisor/configurations' | where isnotempty(resourceGroup) == true\r\n| project subscriptionId, resourceGroup, excludeProperty = properties.exclude)\r\non subscriptionId, resourceGroup\r\n| extend isActive3 = iff(isnull(excludeProperty), true, tobool(excludeProperty) == false)\r\n| where isActive3 == true\r\n| where Category == 'Cost' \r\n| where SubCategory has \"Microsoft.Compute\" or SubCategory has \"Container\" or SubCategory has \"Web\"\r\n| where SubCategory !has \"Microsoft.Compute/disks\"\r\n| project-away subscriptionId1, subscriptionId2, AffectedResource1, isActive2, isActive3, Impact1, Recommendation1, resourceGroup1, resourceGroup2", + "size": 0, + "title": "Azure Advisor Cost recommendations", + "noDataMessage": "You are following all of our cost recommendations for Compute", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "Group", + "formatter": 1 + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "id", + "formatter": 5 + }, + { + "columnMatch": "stableId", + "formatter": 5 + }, + { + "columnMatch": "recommendationTypeId", + "formatter": 5 + }, + { + "columnMatch": "maxCpuP95", + "formatter": 5 + }, + { + "columnMatch": "excludeRecomm", + "formatter": 5 + }, + { + "columnMatch": "lowCpuThreshold", + "formatter": 5 + }, + { + "columnMatch": "AdditionaInfo", + "formatter": 5, + "formatOptions": { + "customColumnWidthSetting": "19ch" + } + }, + { + "columnMatch": "isActive1", + "formatter": 5 + }, + { + "columnMatch": "excludeProperty", + "formatter": 5 + } + ], + "rowLimit": 1000, + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "Recommendation" + ], + "expandTopLevel": true + }, + "labelSettings": [ + { + "columnId": "AffectedResource", + "label": "Affected Resource" + }, + { + "columnId": "Category", + "label": "Recommendation Category" + }, + { + "columnId": "SubCategory", + "label": "Affected Resource Type" + }, + { + "columnId": "Recommendation", + "label": "Recommendation" + }, + { + "columnId": "Impact", + "label": "Impact" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "subscriptionId", + "label": "Subscription ID" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "IsVisible", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "Get-AdvisorRecommendations-Compute" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": " resources\r\n | where resourceGroup in ({ResourceGroup})\r\n | where (type has \"Microsoft.Compute\" or type has \"Microsoft.ContainerService\" or type has \"serverfarms\") and type !has \"Disks\"\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend AffectedResource=id,ResourceRG=resourceGroup\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n | project id", + "size": 0, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "isVisible", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "query - tags - list all compute resources" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"d446799d-b1af-4bca-9d72-84ba2d870039\",\"mergeType\":\"innerunique\",\"leftTable\":\"Get-AdvisorRecommendations-Compute\",\"rightTable\":\"query - tags - list all compute resources\",\"leftColumn\":\"AffectedResource\",\"rightColumn\":\"id\"}],\"projectRename\":[{\"originalName\":\"[Get-AdvisorRecommendations-Compute].AffectedResource\",\"mergedName\":\"Affected Resource\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].Category\",\"mergedName\":\"Recommendation Category\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].SubCategory\",\"mergedName\":\"Affected Resource Type\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].Recommendation\",\"mergedName\":\"Recommendation\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].Impact\",\"mergedName\":\"Impact\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].resourceGroup\",\"mergedName\":\"Resource Group\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].subscriptionId\",\"mergedName\":\"Subscription ID\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].id\",\"mergedName\":\"id\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].stableId\",\"mergedName\":\"stableId\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].recommendationTypeId\",\"mergedName\":\"recommendationTypeId\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].maxCpuP95\",\"mergedName\":\"maxCpuP95\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].excludeRecomm\",\"mergedName\":\"excludeRecomm\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].lowCpuThreshold\",\"mergedName\":\"lowCpuThreshold\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].AdditionaInfo\",\"mergedName\":\"AdditionaInfo\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].isActive1\",\"mergedName\":\"isActive1\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].excludeProperty\",\"mergedName\":\"excludeProperty\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[query - tags - list all compute resources].id\",\"mergedName\":\"id1\",\"fromId\":\"d446799d-b1af-4bca-9d72-84ba2d870039\"},{\"originalName\":\"[Get-AdvisorRecommendations-Compute].location\",\"mergedName\":\"location\",\"fromId\":\"unknown\"}]}", + "size": 0, + "title": "Azure Advisor Cost recommendations", + "noDataMessage": "You are following all of our cost recommendations for Compute", + "noDataMessageStyle": 3, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "Group", + "formatter": 1 + }, + { + "columnMatch": "Affected Resource Type", + "formatter": 5 + }, + { + "columnMatch": "Resource Group", + "formatter": 14, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Subscription ID", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "id", + "formatter": 5 + }, + { + "columnMatch": "stableId", + "formatter": 5 + }, + { + "columnMatch": "recommendationTypeId", + "formatter": 5 + }, + { + "columnMatch": "maxCpuP95", + "formatter": 5 + }, + { + "columnMatch": "excludeRecomm", + "formatter": 5 + }, + { + "columnMatch": "lowCpuThreshold", + "formatter": 5 + }, + { + "columnMatch": "AdditionaInfo", + "formatter": 5 + }, + { + "columnMatch": "isActive1", + "formatter": 5 + }, + { + "columnMatch": "excludeProperty", + "formatter": 5 + }, + { + "columnMatch": "id1", + "formatter": 5 + } + ], + "rowLimit": 1000, + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "Recommendation" + ] + } + } + }, + "showPin": false, + "name": "query - Merge - Compute Advisor recommendations" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "advisorCompute" + }, + "name": "AdvisorGroup" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Virtual Machine Scale Sets\r\n## Save with Azure Spot VMs on Virtual Machine Scale Sets\r\nUsing Azure Spot Virtual Machines on scale sets allows you to take advantage of our unused capacity at a significant cost savings. At any point in time when Azure needs the capacity back, the Azure infrastructure will evict Azure Spot Virtual Machine instances. Therefore, Azure Spot Virtual Machine instances are great for workloads that can handle interruptions like batch processing jobs, dev/test environments, large compute workloads, and more.\r\n\r\n## Spot Priority Mix\r\nAzure allows you to have the flexibility of running a mix of uninterruptible standard VMs and interruptible Spot VMs for Virtual Machine Scale Set deployments. You're able to deploy this Spot Priority Mix using Flexible orchestration to easily balance between high-capacity availability and lower infrastructure costs according to your workload requirements\r\n", + "style": "upsell" + }, + "name": "text - 8" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where type =~ 'microsoft.compute/virtualmachinescalesets'\r\n| where resourceGroup in ({ResourceGroup})\r\n| extend SpotVMs=tostring(properties.virtualMachineProfile.priority), SpotPriorityMix=tostring(properties.priorityMixPolicy), SKU=tostring(sku.name), resourceGroup=strcat('/subscriptions/',subscriptionId,'/resourceGroups/',resourceGroup)\r\n| project id, SKU, SpotVMs,SpotPriorityMix,subscriptionId,resourceGroup, location\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}'])\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct id\r\n )\r\n on id", + "size": 0, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "SpotVMs", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "Spot", + "representation": "success", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "warning", + "text": "Not using Spot VMs" + } + ] + } + }, + { + "columnMatch": "SpotPriorityMix", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "is Empty", + "representation": "2", + "text": "Not using Spot Priority Mix" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "resourceGroup", + "formatter": 14, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "id1", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "id", + "label": "ID" + }, + { + "columnId": "SKU", + "label": "SKU" + }, + { + "columnId": "SpotVMs", + "label": "Spot VMs" + }, + { + "columnId": "SpotPriorityMix", + "label": "Spot Priority Mix" + }, + { + "columnId": "subscriptionId", + "label": "Subscription Name" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "location", + "label": "Location" + } + ] + } + }, + "name": "query - 9" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "VMSS" + }, + "name": "group - VMSS" + } + ] + }, + "name": "Compute - Subscription" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "UsageOptimization" + }, + { + "parameterName": "SelectedRateOptimizationTab", + "comparison": "isEqualTo", + "value": "Compute" + } + ], + "name": "ComputeGroup", + "styleSettings": { + "showBorder": true + } + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "UsageOptimization" + }, + "name": "group - usage optimization" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "## Commitment-based savings\r\nTo maximize your Azure savings, consider savings plans for flexible usage and reserved instances for persistent needs. Azure Savings plans offer reduced rates with a fixed hourly spend and reserved instances allow pre-purchasing VM base price. Both options provide discounts and adapt to your usage patterns, helping you manage costs effectively. Below is an estimate of how much you can potentially save with 1-Year commitment for each option based on your usage pattern for the last 30 days.​", + "style": "upsell" + }, + "customWidth": "50", + "name": "text - P1YTotalSavings" + }, + { + "type": 1, + "content": { + "json": "## Commitment-based savings\r\nTo maximize your Azure savings, consider savings plans for flexible usage and reserved instances for persistent needs. Savings plans offer reduced rates with a fixed hourly spend, while reserved instances allow pre-purchasing VM base price. Both options provide discounts and adapt to your usage patterns, helping you manage costs effectively. Below is an estimate of how much you can save with 3-Year commitment for each option based on your usage pattern for the last 30 days.​", + "style": "upsell" + }, + "customWidth": "50", + "name": "text - P3YTotalSavings - Copy" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "AdvisorResources \r\n| where type == 'microsoft.advisor/recommendations' \r\n| where properties.category == 'Cost' and (properties.shortDescription.solution contains \"Reserved Instance\" or properties.shortDescription.solution contains \"savings plan\")\r\n| extend\r\nrecommendationTypeId = tostring(properties.recommendationTypeId),\r\nreservedResourceType=tostring(properties.extendedProperties.reservedResourceType),\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nterm=tostring(properties.extendedProperties.term),\r\nstableId = name,\r\nsubscriptionId = tostring(properties.extendedProperties.subId)\r\n| where term == \"P1Y\" and lookbackPeriod == \"Last 30 days\"\r\n| extend subscriptionId,stableId\r\n| join kind = leftouter\r\n(advisorresources | where type=~'microsoft.advisor/suppressions'\r\n| extend tokens = split(id, '/')\r\n| extend stableId = iff(array_length(tokens) > 3, tokens[(array_length(tokens)-3)], '')\r\n| extend expirationTimeStamp = todatetime(iff(strcmp(tostring(properties.ttl), '-1') == 0, '9999-12-31', properties.expirationTimeStamp))\r\n| where expirationTimeStamp > now()\r\n| project stableId, expirationTimeStamp)\r\non stableId\r\n| where isempty(expirationTimeStamp)\r\n| extend subscriptionId,stableId\r\n| join kind = leftouter\r\n(advisorresources \r\n| where type =~ 'microsoft.advisor/configurations'\r\n| where isempty(resourceGroup) == true\r\n| extend\r\nmaxCpuP95 = properties.extendedProperties.MaxCpuP95,\r\nlowCpuThreshold = properties.lowCpuThreshold,\r\nexcludeRecomm = properties.exclude,\r\nreservedResourceType=tostring(properties.extendedProperties.reservedResourceType),\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nscope=tostring(properties.extendedProperties.scope),\r\nterm=tostring(properties.extendedProperties.term),\r\nsavings=todouble(properties.extendedProperties.annualSavingsAmount),\r\nsavingsAmount = todouble(properties.extendedProperties.savingsAmount),\r\nRecommendation=tostring(properties.shortDescription.solution), \r\ncurrency = tostring(properties.extendedProperties.savingsCurrency),\r\ndisplayQty = tostring(properties.extendedProperties.displayQty),\r\ndisplaySKU = tostring(properties.extendedProperties.displaySKU),\r\nregion = tostring(properties.extendedProperties.region),\r\nstableId = name,\r\nsubscriptionId = tostring(properties.extendedProperties.subId))\r\non subscriptionId\r\n| extend isActive1 = iff(isnull(excludeRecomm), true, tobool(excludeRecomm) == false)\r\n| extend isActive2 = iff(recommendationTypeId == 'e10b1381-5f0a-47ff-8c7b-37bd13d7c974', iff((isnotempty(lowCpuThreshold) and isnotempty(maxCpuP95)), toint(maxCpuP95) < toint(lowCpuThreshold), iff((isempty(maxCpuP95) or toint(maxCpuP95) < 5), true, false)), true)\r\n| where isActive1 == true and isActive2 == true\r\n| join kind = leftouter\r\n(advisorresources | where type =~ 'microsoft.advisor/configurations' | where isnotempty(resourceGroup) == true\r\n| project subscriptionId, resourceGroup, excludeProperty = properties.exclude)\r\non subscriptionId, resourceGroup\r\n| extend isActive3 = iff(isnull(excludeProperty), true, tobool(excludeProperty) == false)\r\n| where isActive3 == true\r\n| extend reservedResourceType=tostring(properties.extendedProperties.reservedResourceType),\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nscope=tostring(properties.extendedProperties.scope),\r\nterm=tostring(properties.extendedProperties.term),\r\nsavings=todouble(properties.extendedProperties.annualSavingsAmount),\r\nsavingsAmount = todouble(properties.extendedProperties.savingsAmount),\r\nRecommendation=tostring(properties.shortDescription.solution), \r\ncurrency = tostring(properties.extendedProperties.savingsCurrency),\r\ndisplayQty = tostring(properties.extendedProperties.displayQty),\r\ndisplaySKU = tostring(properties.extendedProperties.displaySKU),\r\nregion = tostring(properties.extendedProperties.region),\r\nresources=tostring(properties.resourceMetadata.resourceId), \r\nsubscription = tostring(properties.extendedProperties.subId),\r\ntypeOfRecommendation = iif(properties.shortDescription.solution contains \"Reserved Instance\", \"Reservations\", \"Savings plan\")\r\n| where term == \"P1Y\" and lookbackPeriod == \"Last 30 days\"\r\n| summarize bin (sum(savings), 0.01) by typeOfRecommendation,currency\r\n| order by sum_savings desc\r\n", + "size": 0, + "title": "1 year total commitment-based savings", + "noDataMessage": "There are no commitment-based recommendations", + "noDataMessageStyle": 3, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "value::all" + ], + "visualization": "piechart", + "chartSettings": { + "seriesLabelSettings": [ + { + "seriesName": "Reservations", + "label": "Azure Reservations" + }, + { + "seriesName": "Savings plan", + "label": "Azure Savings Plan for Compute" + } + ] + } + }, + "customWidth": "50", + "name": "query - CommitmentBasedSavingsP1Y" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "AdvisorResources \r\n| where type == 'microsoft.advisor/recommendations' \r\n| where properties.category == 'Cost' and (properties.shortDescription.solution contains \"Reserved Instance\" or properties.shortDescription.solution contains \"savings plan\")\r\n| extend\r\nrecommendationTypeId = tostring(properties.recommendationTypeId),\r\nreservedResourceType=tostring(properties.extendedProperties.reservedResourceType),\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nterm=tostring(properties.extendedProperties.term),\r\nstableId = name,\r\nsubscriptionId = tostring(properties.extendedProperties.subId)\r\n| where term == \"P3Y\" and lookbackPeriod == \"Last 30 days\"\r\n| extend subscriptionId,stableId\r\n| join kind = leftouter\r\n(advisorresources | where type=~'microsoft.advisor/suppressions'\r\n| extend tokens = split(id, '/')\r\n| extend stableId = iff(array_length(tokens) > 3, tokens[(array_length(tokens)-3)], '')\r\n| extend expirationTimeStamp = todatetime(iff(strcmp(tostring(properties.ttl), '-1') == 0, '9999-12-31', properties.expirationTimeStamp))\r\n| where expirationTimeStamp > now()\r\n| project stableId, expirationTimeStamp)\r\non stableId\r\n| where isempty(expirationTimeStamp)\r\n| extend subscriptionId,stableId\r\n| join kind = leftouter\r\n(advisorresources \r\n| where type =~ 'microsoft.advisor/configurations'\r\n| where isempty(resourceGroup) == true\r\n| extend\r\nmaxCpuP95 = properties.extendedProperties.MaxCpuP95,\r\nlowCpuThreshold = properties.lowCpuThreshold,\r\nexcludeRecomm = properties.exclude,\r\nreservedResourceType=tostring(properties.extendedProperties.reservedResourceType),\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nscope=tostring(properties.extendedProperties.scope),\r\nterm=tostring(properties.extendedProperties.term),\r\nsavings=todouble(properties.extendedProperties.annualSavingsAmount),\r\nsavingsAmount = todouble(properties.extendedProperties.savingsAmount),\r\nRecommendation=tostring(properties.shortDescription.solution), \r\ncurrency = tostring(properties.extendedProperties.savingsCurrency),\r\ndisplayQty = tostring(properties.extendedProperties.displayQty),\r\ndisplaySKU = tostring(properties.extendedProperties.displaySKU),\r\nregion = tostring(properties.extendedProperties.region),\r\nstableId = name,\r\nsubscriptionId = tostring(properties.extendedProperties.subId))\r\non subscriptionId\r\n| extend isActive1 = iff(isnull(excludeRecomm), true, tobool(excludeRecomm) == false)\r\n| extend isActive2 = iff(recommendationTypeId == 'e10b1381-5f0a-47ff-8c7b-37bd13d7c974', iff((isnotempty(lowCpuThreshold) and isnotempty(maxCpuP95)), toint(maxCpuP95) < toint(lowCpuThreshold), iff((isempty(maxCpuP95) or toint(maxCpuP95) < 5), true, false)), true)\r\n| where isActive1 == true and isActive2 == true\r\n| join kind = leftouter\r\n(advisorresources | where type =~ 'microsoft.advisor/configurations' | where isnotempty(resourceGroup) == true\r\n| project subscriptionId, resourceGroup, excludeProperty = properties.exclude)\r\non subscriptionId, resourceGroup\r\n| extend isActive3 = iff(isnull(excludeProperty), true, tobool(excludeProperty) == false)\r\n| where isActive3 == true\r\n| extend reservedResourceType=tostring(properties.extendedProperties.reservedResourceType),\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nscope=tostring(properties.extendedProperties.scope),\r\nterm=tostring(properties.extendedProperties.term),\r\nsavings=todouble(properties.extendedProperties.annualSavingsAmount),\r\nsavingsAmount = todouble(properties.extendedProperties.savingsAmount),\r\nRecommendation=tostring(properties.shortDescription.solution), \r\ncurrency = tostring(properties.extendedProperties.savingsCurrency),\r\ndisplayQty = tostring(properties.extendedProperties.displayQty),\r\ndisplaySKU = tostring(properties.extendedProperties.displaySKU),\r\nregion = tostring(properties.extendedProperties.region),\r\nresources=tostring(properties.resourceMetadata.resourceId), \r\nsubscription = tostring(properties.extendedProperties.subId),\r\ntypeOfRecommendation = iif(properties.shortDescription.solution contains \"Reserved Instance\", \"Reservations\", \"Savings plan\")\r\n| where term == \"P3Y\" and lookbackPeriod == \"Last 30 days\"\r\n| summarize bin (sum(savings), 0.01) by typeOfRecommendation,currency\r\n| order by sum_savings desc\r\n", + "size": 0, + "title": "3 years total commitment-based savings", + "noDataMessage": "There are no commitment-based recommendations", + "noDataMessageStyle": 3, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "value::all" + ], + "visualization": "piechart", + "chartSettings": { + "seriesLabelSettings": [ + { + "seriesName": "Reservations", + "label": "Azure Reservations" + }, + { + "seriesName": "Savings plan", + "label": "Azure Savings Plan for Compute" + } + ] + } + }, + "customWidth": "50", + "name": "query - CommitmentBasedSavingsP3Y" + }, + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "792df0b2-35da-403d-999d-ff81ea8d4f56", + "cellValue": "selectedRateOptimizationTab", + "linkTarget": "parameter", + "linkLabel": "Azure Hybrid Benefit", + "subTarget": "AHB", + "style": "link" + }, + { + "id": "56eb4166-cb7c-4384-94a9-c5f201e1316d", + "cellValue": "selectedRateOptimizationTab", + "linkTarget": "parameter", + "linkLabel": "Azure Reservations", + "subTarget": "Reservations", + "style": "link" + }, + { + "id": "799d4fc7-5790-467c-84cc-ce4b4cc34a3f", + "cellValue": "selectedRateOptimizationTab", + "linkTarget": "parameter", + "linkLabel": "Azure savings plan for compute", + "subTarget": "SavingsPlan", + "style": "link" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RateOptimization" + }, + "name": "links - rate optimization tabs" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "template", + "loadFromTemplateId": "", + "items": [ + { + "type": 1, + "content": { + "json": "**Reserved instances** can provide a significant discount over on-demand prices. With reserved instances, you can pre-purchase the base costs for your virtual machines. \r\n
Discounts will automatically apply to new or existing VMs that have the same size and region as your reserved instance.
We analyzed your usage over selected Term, look-back period and recommend money-saving reserved instances​.\r\n
This query will only provide you recommendations for single scope reserved instances. *To learn more about Reserved Instances, go to this [link.](https://learn.microsoft.com/azure/cost-management-billing/manage/understand-vm-reservation-charges)*", + "style": "info" + }, + "name": "text - advisorReservationdDisclaimer" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "a1960768-9da4-455d-b6f6-6d43098cff76", + "version": "KqlParameterItem/1.0", + "name": "Subscription", + "type": 6, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "includeAll": false, + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::all", + "label": " Subscription", + "value": [ + "value::all" + ] + } + ], + "style": "pills", + "queryType": 0, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "75", + "name": "parameters - Filters" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "2b8ca845-75ba-4f4b-acad-54ee50d66d54", + "version": "KqlParameterItem/1.0", + "name": "LookBackPeriod", + "label": "Look back period", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[[\r\n {\"value\": \"Last 7 days\"},\r\n {\"value\": \"Last 30 days\"},\r\n {\"value\": \"Last 60 days\"}\r\n]", + "timeContext": { + "durationMs": 86400000 + }, + "value": "Last 60 days" + }, + { + "id": "953c9e4c-af03-4fb7-bf30-3f1bfdf09199", + "version": "KqlParameterItem/1.0", + "name": "term", + "label": "Term", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [] + }, + "jsonData": "[[\r\n {\r\n \"value\": \"P1Y\",\r\n \"Selected\": \"true\"\r\n },\r\n {\r\n \"value\": \"P3Y\"\r\n }\r\n]", + "timeContext": { + "durationMs": 86400000 + }, + "value": "P3Y" + }, + { + "id": "c46193fe-f1b2-49d1-a9bc-c9f5149f0194", + "version": "KqlParameterItem/1.0", + "name": "resourceType", + "label": "Resource type", + "type": 2, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": "AdvisorResources \r\n| where type == 'microsoft.advisor/recommendations' \r\n| where properties.category == 'Cost' and properties.shortDescription.solution contains \"Reserved Instance\"\r\n| extend reservedResourceType=tostring(properties.extendedProperties.reservedResourceType)\r\n| distinct reservedResourceType", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "showDefault": false + }, + "defaultValue": "value::all", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": [ + "value::all" + ] + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "name": "parameters - reservationsParams" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "AdvisorResources \r\n| where type == 'microsoft.advisor/recommendations' \r\n| where properties.category == 'Cost' and properties.shortDescription.solution contains \"Reserved Instance\" \r\n| extend\r\nrecommendationTypeId = tostring(properties.recommendationTypeId),\r\nreservedResourceType=tostring(properties.extendedProperties.reservedResourceType),\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nterm=tostring(properties.extendedProperties.term),\r\nstableId = name,\r\nsubscriptionId = tostring(properties.extendedProperties.subId)\r\n| where term == \"{term}\" and lookbackPeriod == \"{LookBackPeriod}\" and reservedResourceType in ({resourceType})\r\n| extend subscriptionId,stableId\r\n| join kind = leftouter\r\n(advisorresources | where type=~'microsoft.advisor/suppressions'\r\n| extend tokens = split(id, '/')\r\n| extend stableId = iff(array_length(tokens) > 3, tokens[(array_length(tokens)-3)], '')\r\n| extend expirationTimeStamp = todatetime(iff(strcmp(tostring(properties.ttl), '-1') == 0, '9999-12-31', properties.expirationTimeStamp))\r\n| where expirationTimeStamp > now()\r\n| project stableId, expirationTimeStamp)\r\non stableId\r\n| where isempty(expirationTimeStamp)\r\n| extend subscriptionId,stableId\r\n| join kind = leftouter\r\n(advisorresources \r\n| where type =~ 'microsoft.advisor/configurations'\r\n| where isempty(resourceGroup) == true\r\n| extend\r\nmaxCpuP95 = properties.extendedProperties.MaxCpuP95,\r\nlowCpuThreshold = properties.lowCpuThreshold,\r\nexcludeRecomm = properties.exclude,\r\nreservedResourceType=tostring(properties.extendedProperties.reservedResourceType),\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nscope=tostring(properties.extendedProperties.scope),\r\nterm=tostring(properties.extendedProperties.term),\r\nsavings=todouble(properties.extendedProperties.annualSavingsAmount),\r\nsavingsAmount = todouble(properties.extendedProperties.savingsAmount),\r\nRecommendation=tostring(properties.shortDescription.solution), \r\ncurrency = tostring(properties.extendedProperties.savingsCurrency),\r\ndisplayQty = tostring(properties.extendedProperties.displayQty),\r\ndisplaySKU = tostring(properties.extendedProperties.displaySKU),\r\nregion = tostring(properties.extendedProperties.region),\r\nstableId = name,\r\nsubscriptionId = tostring(properties.extendedProperties.subId))\r\non subscriptionId\r\n| extend isActive1 = iff(isnull(excludeRecomm), true, tobool(excludeRecomm) == false)\r\n| extend isActive2 = iff(recommendationTypeId == 'e10b1381-5f0a-47ff-8c7b-37bd13d7c974', iff((isnotempty(lowCpuThreshold) and isnotempty(maxCpuP95)), toint(maxCpuP95) < toint(lowCpuThreshold), iff((isempty(maxCpuP95) or toint(maxCpuP95) < 5), true, false)), true)\r\n| where isActive1 == true and isActive2 == true\r\n| join kind = leftouter\r\n(advisorresources | where type =~ 'microsoft.advisor/configurations' | where isnotempty(resourceGroup) == true\r\n| project subscriptionId, resourceGroup, excludeProperty = properties.exclude)\r\non subscriptionId, resourceGroup\r\n| extend isActive3 = iff(isnull(excludeProperty), true, tobool(excludeProperty) == false)\r\n| where isActive3 == true\r\n| extend reservedResourceType=tostring(properties.extendedProperties.reservedResourceType),\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nscope=tostring(properties.extendedProperties.scope),\r\nterm=tostring(properties.extendedProperties.term),\r\nsavings=todouble(properties.extendedProperties.annualSavingsAmount),\r\nsavingsAmount = todouble(properties.extendedProperties.savingsAmount),\r\nRecommendation=tostring(properties.shortDescription.solution), \r\ncurrency = tostring(properties.extendedProperties.savingsCurrency),\r\ndisplayQty = tostring(properties.extendedProperties.displayQty),\r\ndisplaySKU = tostring(properties.extendedProperties.displaySKU),\r\nregion = tostring(properties.extendedProperties.region),\r\nresources=tostring(properties.resourceMetadata.resourceId), \r\nsubscription = tostring(properties.extendedProperties.subId)\r\n| where term == \"{term}\" and lookbackPeriod == \"{LookBackPeriod}\" and reservedResourceType in ({resourceType})\r\n| summarize Subscriptions=dcount(resources), \r\n bin (sum(savings), 0.01) by Recommendation ,reservedResourceType ,lookbackPeriod,scope,term ,currency\r\n| order by sum_savings desc\r\n", + "size": 0, + "title": "Reservations Summary", + "noDataMessage": "No reservations recommendations found!", + "noDataMessageStyle": 3, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "categoricalbar", + "gridSettings": { + "filter": true, + "labelSettings": [ + { + "columnId": "reservedResourceType", + "label": "Resource type" + }, + { + "columnId": "lookbackPeriod", + "label": "Look back period" + }, + { + "columnId": "scope", + "label": "Scope" + }, + { + "columnId": "term", + "label": "Term" + }, + { + "columnId": "currency", + "label": "Currency" + }, + { + "columnId": "sum_savings", + "label": "Total annual savings" + } + ] + }, + "chartSettings": { + "xAxis": "reservedResourceType", + "yAxis": [ + "sum_savings" + ], + "group": "reservedResourceType", + "createOtherGroup": 0, + "showLegend": true, + "ySettings": { + "numberFormatSettings": { + "unit": 0, + "options": { + "style": "decimal", + "useGrouping": true + } + } + } + } + }, + "name": "query - Reservations Summary" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "AdvisorResources \r\n| where type == 'microsoft.advisor/recommendations' \r\n| where properties.category == 'Cost' and properties.shortDescription.solution contains \"Reserved Instance\" \r\n| extend\r\nrecommendationTypeId = tostring(properties.recommendationTypeId),\r\nreservedResourceType=tostring(properties.extendedProperties.reservedResourceType),\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nterm=tostring(properties.extendedProperties.term),\r\nstableId = name,\r\nsubscriptionId = tostring(properties.extendedProperties.subId)\r\n| where term == \"{term}\" and lookbackPeriod == \"{LookBackPeriod}\" and reservedResourceType in ({resourceType})\r\n| extend subscriptionId,stableId\r\n| join kind = leftouter\r\n(advisorresources | where type=~'microsoft.advisor/suppressions'\r\n| extend tokens = split(id, '/')\r\n| extend stableId = iff(array_length(tokens) > 3, tokens[(array_length(tokens)-3)], '')\r\n| extend expirationTimeStamp = todatetime(iff(strcmp(tostring(properties.ttl), '-1') == 0, '9999-12-31', properties.expirationTimeStamp))\r\n| where expirationTimeStamp > now()\r\n| project stableId, expirationTimeStamp)\r\non stableId\r\n| where isempty(expirationTimeStamp)\r\n| extend subscriptionId,stableId\r\n| join kind = leftouter\r\n(advisorresources \r\n| where type =~ 'microsoft.advisor/configurations'\r\n| where isempty(resourceGroup) == true\r\n| extend\r\nmaxCpuP95 = properties.extendedProperties.MaxCpuP95,\r\nlowCpuThreshold = properties.lowCpuThreshold,\r\nexcludeRecomm = properties.exclude,\r\nreservedResourceType=tostring(properties.extendedProperties.reservedResourceType),\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nscope=tostring(properties.extendedProperties.scope),\r\nterm=tostring(properties.extendedProperties.term),\r\nRecommendation=tostring(properties.shortDescription.solution), \r\nstableId = name,\r\nsubscriptionId = tostring(properties.extendedProperties.subId))\r\non subscriptionId\r\n| extend isActive1 = iff(isnull(excludeRecomm), true, tobool(excludeRecomm) == false)\r\n| extend isActive2 = iff(recommendationTypeId == 'e10b1381-5f0a-47ff-8c7b-37bd13d7c974', iff((isnotempty(lowCpuThreshold) and isnotempty(maxCpuP95)), toint(maxCpuP95) < toint(lowCpuThreshold), iff((isempty(maxCpuP95) or toint(maxCpuP95) < 5), true, false)), true)\r\n| where isActive1 == true and isActive2 == true\r\n| join kind = leftouter\r\n(advisorresources | where type =~ 'microsoft.advisor/configurations' | where isnotempty(resourceGroup) == true\r\n| project subscriptionId, resourceGroup, excludeProperty = properties.exclude)\r\non subscriptionId, resourceGroup\r\n| extend isActive3 = iff(isnull(excludeProperty), true, tobool(excludeProperty) == false)\r\n| where isActive3 == true\r\n| extend reservedResourceType=tostring(properties.extendedProperties.reservedResourceType),\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nscope=tostring(properties.extendedProperties.scope),\r\nterm=tostring(properties.extendedProperties.term),\r\nsavings=todouble(properties.extendedProperties.annualSavingsAmount),\r\nsavingsAmount = todouble(properties.extendedProperties.savingsAmount),\r\nRecommendation=tostring(properties.shortDescription.solution), \r\ncurrency = tostring(properties.extendedProperties.savingsCurrency),\r\ndisplayQty = tostring(properties.extendedProperties.displayQty),\r\ndisplaySKU = tostring(properties.extendedProperties.displaySKU),\r\nregion = tostring(properties.extendedProperties.region),\r\nsubscription = tostring(properties.extendedProperties.subId)\r\n| where term == \"{term}\" and lookbackPeriod == \"{LookBackPeriod}\" and reservedResourceType in ({resourceType})\r\n| project Recommendation,reservedResourceType,displaySKU,displayQty,savings,currency,lookbackPeriod,term,region,subscription\r\n| order by savings desc\r\n", + "size": 0, + "title": "Reservations details", + "noDataMessage": "No reservations recommendations found!", + "noDataMessageStyle": 3, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Recommendation", + "formatter": 5 + }, + { + "columnMatch": "reservedResourceType", + "formatter": 5 + }, + { + "columnMatch": "subscription", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "subscription", + "reservedResourceType" + ], + "expandTopLevel": false + }, + "labelSettings": [ + { + "columnId": "displaySKU", + "label": "SKU" + }, + { + "columnId": "displayQty", + "label": "Quantity" + }, + { + "columnId": "savings", + "label": "Total annual savings" + }, + { + "columnId": "currency", + "label": "Currency" + }, + { + "columnId": "lookbackPeriod", + "label": "Look back period" + }, + { + "columnId": "term", + "label": "Term" + }, + { + "columnId": "region", + "label": "Region" + }, + { + "columnId": "subscription", + "label": "Subscription" + } + ] + }, + "chartSettings": { + "xAxis": "reservedResourceType", + "yAxis": [ + "sum_savings" + ], + "group": "reservedResourceType", + "createOtherGroup": 0, + "showLegend": true, + "ySettings": { + "numberFormatSettings": { + "unit": 0, + "options": { + "style": "decimal", + "useGrouping": true + } + } + } + } + }, + "name": "query - Reservations details" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RateOptimization" + }, + { + "parameterName": "selectedRateOptimizationTab", + "comparison": "isEqualTo", + "value": "Reservations" + } + ], + "name": "group - Reservations" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "template", + "loadFromTemplateId": "", + "items": [ + { + "type": 1, + "content": { + "json": "We analyzed your compute usage over the last 30 days and recommend adding a savings plan to increase your savings.
The savings plan unlocks lower prices on select compute services when you commit to spend a fixed hourly amount for 1 or 3 years.
As you use select compute services globally, your usage is covered by the plan at reduced prices. During the times when your usage is above your hourly commitment, you’ll simply be billed at your regular pay-as-you-go prices. With savings automatically applying across compute usage globally, you’ll continue saving even as your usage needs change over time.
Savings plan are more suited for dynamic workloads while accommodating for planned or unplanned changes while reservations are more suited for stable, predictable workloads with no planned changes.
Saving estimates are calculated for individual subscriptions and the usage pattern observed over last 30 days. **Shared scope savings plans are available in purchase experience and can further increase savings.**
\r\nTo learn more about Savings Plan, check out this [link.](https://learn.microsoft.com/azure/cost-management-billing/savings-plan/purchase-recommendations)​", + "style": "info" + }, + "name": "text - advisorSavingsPlanDisclaimer" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "a1960768-9da4-455d-b6f6-6d43098cff76", + "version": "KqlParameterItem/1.0", + "name": "Subscription", + "type": 6, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "includeAll": false, + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::all", + "label": " Subscription", + "value": [ + "value::all" + ] + } + ], + "style": "pills", + "queryType": 0, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "75", + "name": "parameters - Filters" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "2b8ca845-75ba-4f4b-acad-54ee50d66d54", + "version": "KqlParameterItem/1.0", + "name": "LookBackPeriod", + "label": "Look back period", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[[\r\n {\"value\": \"Last 7 days\"},\r\n {\"value\": \"Last 30 days\"},\r\n {\"value\": \"Last 60 days\"}\r\n]", + "timeContext": { + "durationMs": 86400000 + }, + "value": "Last 30 days" + }, + { + "id": "953c9e4c-af03-4fb7-bf30-3f1bfdf09199", + "version": "KqlParameterItem/1.0", + "name": "term", + "label": "Term", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [] + }, + "jsonData": "[[\r\n {\r\n \"value\": \"P1Y\",\r\n \"Selected\": \"true\"\r\n },\r\n {\r\n \"value\": \"P3Y\"\r\n }\r\n]", + "timeContext": { + "durationMs": 86400000 + }, + "value": "P1Y" + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "name": "parameters - savingsPlanParams" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "AdvisorResources \r\n| where type == 'microsoft.advisor/recommendations' \r\n| where properties.category == 'Cost' and properties.shortDescription.solution contains \"savings plan\"\r\n| extend recommendationTypeId = tostring(properties.recommendationTypeId),\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nterm=tostring(properties.extendedProperties.term),\r\nstableId = name,\r\nsubscriptionId = tostring(properties.extendedProperties.subId),\r\ncommitment = tostring(properties.extendedProperties.commitment)\r\n| where term == \"{term}\" and lookbackPeriod == \"{LookBackPeriod}\"\r\n| extend subscriptionId,stableId\r\n| join kind = leftouter\r\n(advisorresources | where type=~'microsoft.advisor/suppressions'\r\n| extend tokens = split(id, '/')\r\n| extend stableId = iff(array_length(tokens) > 3, tokens[(array_length(tokens)-3)], '')\r\n| extend expirationTimeStamp = todatetime(iff(strcmp(tostring(properties.ttl), '-1') == 0, '9999-12-31', properties.expirationTimeStamp))\r\n| where expirationTimeStamp > now()\r\n| project stableId, expirationTimeStamp)\r\non stableId\r\n| where isempty(expirationTimeStamp)\r\n| extend subscriptionId,stableId\r\n| join kind = leftouter\r\n(advisorresources \r\n| where type =~ 'microsoft.advisor/configurations'\r\n| where isempty(resourceGroup) == true\r\n| extend\r\nmaxCpuP95 = properties.extendedProperties.MaxCpuP95,\r\nlowCpuThreshold = properties.lowCpuThreshold,\r\nexcludeRecomm = properties.exclude,\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nscope=tostring(properties.extendedProperties.scope),\r\nterm=tostring(properties.extendedProperties.term),\r\nsavings=todouble(properties.extendedProperties.annualSavingsAmount),\r\nsavingsAmount = todouble(properties.extendedProperties.savingsAmount),\r\nRecommendation=tostring(properties.shortDescription.solution), \r\ncurrency = tostring(properties.extendedProperties.savingsCurrency),\r\ndisplayQty = tostring(properties.extendedProperties.displayQty),\r\ndisplaySKU = tostring(properties.extendedProperties.displaySKU),\r\nregion = tostring(properties.extendedProperties.region),\r\nstableId = name,\r\ncommitment = tostring(properties.extendedProperties.commitment),\r\nsubscriptionId = tostring(properties.extendedProperties.subId))\r\non subscriptionId\r\n| extend isActive1 = iff(isnull(excludeRecomm), true, tobool(excludeRecomm) == false)\r\n| extend isActive2 = iff(recommendationTypeId == 'e10b1381-5f0a-47ff-8c7b-37bd13d7c974', iff((isnotempty(lowCpuThreshold) and isnotempty(maxCpuP95)), toint(maxCpuP95) < toint(lowCpuThreshold), iff((isempty(maxCpuP95) or toint(maxCpuP95) < 5), true, false)), true)\r\n| where isActive1 == true and isActive2 == true\r\n| join kind = leftouter\r\n(advisorresources | where type =~ 'microsoft.advisor/configurations' | where isnotempty(resourceGroup) == true\r\n| project subscriptionId, resourceGroup, excludeProperty = properties.exclude)\r\non subscriptionId, resourceGroup\r\n| extend isActive3 = iff(isnull(excludeProperty), true, tobool(excludeProperty) == false)\r\n| where isActive3 == true\r\n| extend lookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nscope=tostring(properties.extendedProperties.scope),\r\nterm=tostring(properties.extendedProperties.term),\r\nsavings=todouble(properties.extendedProperties.annualSavingsAmount),\r\nsavingsAmount = todouble(properties.extendedProperties.savingsAmount),\r\nRecommendation=tostring(properties.shortDescription.solution), \r\ncurrency = tostring(properties.extendedProperties.savingsCurrency),\r\ndisplayQty = tostring(properties.extendedProperties.displayQty),\r\ndisplaySKU = tostring(properties.extendedProperties.displaySKU),\r\nregion = tostring(properties.extendedProperties.region),\r\nresources=tostring(properties.resourceMetadata.resourceId), \r\nsubscription = tostring(properties.extendedProperties.subId),\r\ncommitment = tostring(properties.extendedProperties.commitment)\r\n| where term == \"{term}\" and lookbackPeriod == \"{LookBackPeriod}\"\r\n| summarize Subscriptions=dcount(resources), \r\n bin (sum(savings), 0.01) by subscription ,commitment ,lookbackPeriod,scope,term ,currency\r\n| order by sum_savings desc\r\n| join (\r\nresourcecontainers\r\n| where type == 'microsoft.resources/subscriptions'\r\n| extend subscription = subscriptionId\r\n| project name,subscription\r\n) on subscription\r\n| project-away subscription1,subscription\r\n", + "size": 0, + "title": "Savings plan Summary", + "noDataMessage": "No savings plan recommendations found!", + "noDataMessageStyle": 3, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "categoricalbar", + "gridSettings": { + "filter": true + }, + "chartSettings": { + "xAxis": "name", + "yAxis": [ + "sum_savings" + ], + "group": "reservedResourceType", + "createOtherGroup": 0, + "showLegend": true, + "ySettings": { + "numberFormatSettings": { + "unit": 0, + "options": { + "style": "decimal", + "useGrouping": true + } + } + } + } + }, + "name": "query - Saving plan Summary" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "AdvisorResources \r\n| where type == 'microsoft.advisor/recommendations' \r\n| where properties.category == 'Cost' and properties.shortDescription.solution contains \"savings plan\"\r\n| extend\r\nrecommendationTypeId = tostring(properties.recommendationTypeId),\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nterm=tostring(properties.extendedProperties.term),\r\nstableId = name,\r\nsubscriptionId = tostring(properties.extendedProperties.subId),\r\ncommitment = tostring(properties.extendedProperties.commitment)\r\n| where term == \"{term}\" and lookbackPeriod == \"{LookBackPeriod}\"\r\n| extend subscriptionId,stableId\r\n| join kind = leftouter\r\n(advisorresources | where type=~'microsoft.advisor/suppressions'\r\n| extend tokens = split(id, '/')\r\n| extend stableId = iff(array_length(tokens) > 3, tokens[(array_length(tokens)-3)], '')\r\n| extend expirationTimeStamp = todatetime(iff(strcmp(tostring(properties.ttl), '-1') == 0, '9999-12-31', properties.expirationTimeStamp))\r\n| where expirationTimeStamp > now()\r\n| project stableId, expirationTimeStamp)\r\non stableId\r\n| where isempty(expirationTimeStamp)\r\n| extend subscriptionId,stableId\r\n| join kind = leftouter\r\n(advisorresources \r\n| where type =~ 'microsoft.advisor/configurations'\r\n| where isempty(resourceGroup) == true\r\n| extend\r\nmaxCpuP95 = properties.extendedProperties.MaxCpuP95,\r\nlowCpuThreshold = properties.lowCpuThreshold,\r\nexcludeRecomm = properties.exclude,\r\nlookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nscope=tostring(properties.extendedProperties.scope),\r\nterm=tostring(properties.extendedProperties.term),\r\nRecommendation=tostring(properties.shortDescription.solution), \r\nstableId = name,\r\ncommitment = tostring(properties.extendedProperties.commitment),\r\nsubscriptionId = tostring(properties.extendedProperties.subId))\r\non subscriptionId\r\n| extend isActive1 = iff(isnull(excludeRecomm), true, tobool(excludeRecomm) == false)\r\n| extend isActive2 = iff(recommendationTypeId == 'e10b1381-5f0a-47ff-8c7b-37bd13d7c974', iff((isnotempty(lowCpuThreshold) and isnotempty(maxCpuP95)), toint(maxCpuP95) < toint(lowCpuThreshold), iff((isempty(maxCpuP95) or toint(maxCpuP95) < 5), true, false)), true)\r\n| where isActive1 == true and isActive2 == true\r\n| join kind = leftouter\r\n(advisorresources | where type =~ 'microsoft.advisor/configurations' | where isnotempty(resourceGroup) == true\r\n| project subscriptionId, resourceGroup, excludeProperty = properties.exclude)\r\non subscriptionId, resourceGroup\r\n| extend isActive3 = iff(isnull(excludeProperty), true, tobool(excludeProperty) == false)\r\n| where isActive3 == true\r\n| extend lookbackPeriod=tostring(strcat(\"Last \",properties.extendedProperties.lookbackPeriod,\" days\")),\r\nscope=tostring(properties.extendedProperties.scope),\r\nterm=tostring(properties.extendedProperties.term),\r\nsavings=todouble(properties.extendedProperties.annualSavingsAmount),\r\nsavingsAmount = todouble(properties.extendedProperties.savingsAmount),\r\nRecommendation=tostring(properties.shortDescription.solution), \r\ncurrency = tostring(properties.extendedProperties.savingsCurrency),\r\ndisplayQty = tostring(properties.extendedProperties.displayQty),\r\ndisplaySKU = tostring(properties.extendedProperties.displaySKU),\r\ncommitment = tostring(properties.extendedProperties.commitment),\r\nregion = tostring(properties.extendedProperties.region),\r\nsubscription = tostring(properties.extendedProperties.subId)\r\n| where term == \"{term}\" and lookbackPeriod == \"{LookBackPeriod}\"\r\n| project Recommendation,savings,commitment,currency,lookbackPeriod,term,subscription\r\n| order by savings desc\r\n| join (\r\nresourcecontainers\r\n| where type == 'microsoft.resources/subscriptions'\r\n| extend subscription = subscriptionId\r\n| project id,name,subscription\r\n) on subscription\r\n| project-away subscription1,subscription\r\n", + "size": 0, + "title": "Savings plan details", + "noDataMessage": "No savings plan recommendations found!", + "noDataMessageStyle": 3, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Recommendation", + "formatter": 5 + }, + { + "columnMatch": "id", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "name", + "formatter": 5 + }, + { + "columnMatch": "reservedResourceType", + "formatter": 5 + }, + { + "columnMatch": "subscription", + "formatter": 5 + } + ], + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "id" + ] + }, + "labelSettings": [ + { + "columnId": "savings", + "label": "Total annual savings" + }, + { + "columnId": "commitment", + "label": "Commitment" + }, + { + "columnId": "currency", + "label": "Currency" + }, + { + "columnId": "lookbackPeriod", + "label": "Look back period" + }, + { + "columnId": "term", + "label": "Term" + } + ] + }, + "chartSettings": { + "xAxis": "reservedResourceType", + "yAxis": [ + "sum_savings" + ], + "group": "reservedResourceType", + "createOtherGroup": 0, + "showLegend": true, + "ySettings": { + "numberFormatSettings": { + "unit": 0, + "options": { + "style": "decimal", + "useGrouping": true + } + } + } + } + }, + "name": "query - Savings plan details" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RateOptimization" + }, + { + "parameterName": "selectedRateOptimizationTab", + "comparison": "isEqualTo", + "value": "SavingsPlan" + } + ], + "name": "group - SavingsPlan" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "2b43eb64-bca3-444a-8003-003554236fe7", + "version": "KqlParameterItem/1.0", + "name": "Subscription", + "type": 6, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "includeAll": false, + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::all", + "label": " Subscription", + "value": [ + "value::all" + ] + }, + { + "id": "03fbf28a-892d-4b68-929c-3ba5056f4b94", + "version": "KqlParameterItem/1.0", + "name": "ResourceGroup", + "label": "Resource Group", + "type": 2, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": "resources\r\n| distinct resourceGroup", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "showDefault": false + }, + "defaultValue": "value::all", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + { + "id": "566c43ae-f300-43be-aa0d-61d92ba8da87", + "version": "KqlParameterItem/1.0", + "name": "SingleSubHidden", + "type": 1, + "isRequired": true, + "query": "resourcecontainers\r\n| where type==\"microsoft.resources/subscriptions\"\r\n| take 1\r\n| project subscriptionId", + "crossComponentResources": [ + "{Subscription}" + ], + "isHiddenWhenLocked": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "label": "Hidden Subscription" + }, + { + "id": "a9df02ed-7100-4130-952f-a3d9d5d364af", + "version": "KqlParameterItem/1.0", + "name": "TagName", + "type": 2, + "query": "Resources\r\n| where tags != '' and tags != '[]'\r\n| mvexpand tags\r\n| extend tagName = tostring(bag_keys(tags)[0])\r\n| distinct tagName\r\n| sort by tagName asc", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [] + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null, + "label": "Tag Name" + }, + { + "id": "66406915-1f07-448f-8170-2f3b0dc6dc00", + "version": "KqlParameterItem/1.0", + "name": "TagValue", + "type": 2, + "query": "Resources\r\n| mvexpand tags\r\n| extend tagName = tostring(bag_keys(tags)[0])\r\n| extend tagValue = tostring(tags[tagName])\r\n| where tags != '' and tags != '[]' and tostring(bag_keys(tags)[0]) == '{TagName}'\r\n| distinct tagValue\r\n| sort by tagValue asc", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [] + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null, + "label": "Tag Value" + } + ], + "style": "pills", + "queryType": 0, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "75", + "conditionalVisibility": { + "parameterName": "selectedRateOptimizationTab", + "comparison": "isEqualTo", + "value": "AHB" + }, + "name": "parameters - Filters" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "f74bc7f5-2b16-4440-8053-106e040b73b6", + "version": "KqlParameterItem/1.0", + "name": "Location", + "type": 2, + "query": "Resources\r\n| where type =~ 'Microsoft.Compute/virtualMachines'\r\n| project name, location\r\n| summarize count () by location\r\n| project location", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::1" + ] + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::1", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "label": "Resource Location" + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "25", + "conditionalVisibility": { + "parameterName": "selectedRateOptimizationTab", + "comparison": "isEqualTo", + "value": "AHB" + }, + "name": "parameters - location" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "template", + "loadType": "always", + "loadFromTemplateId": "", + "items": [ + { + "type": 1, + "content": { + "json": "# Azure Hybrid Benefit\r\nFor customers with Software Assurance, Azure Hybrid Benefit for Windows Server allows you to use your on-premises Windows Server licenses to run Windows virtual machines on Azure at a reduced cost. This article discusses how to deploy new VMs with Azure Hybrid Benefit for Windows Server enabled, and how you can update any existing running VMs. For more information about Azure Hybrid Benefit for Windows Server licensing and cost savings, see the [Azure Hybrid Benefit for Windows Server licensing page](https://azure.microsoft.com/pricing/hybrid-use-benefit/)\r\n\r\n", + "style": "upsell" + }, + "name": "Azure Hybrid Benefit" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"ARMEndpoint/1.0\",\"data\":null,\"headers\":[],\"method\":\"GET\",\"path\":\"/subscriptions/{SingleSubHidden}/providers/Microsoft.Compute/skus?$filter=location eq '{Location}'\",\"urlParams\":[{\"key\":\"api-version\",\"value\":\"2021-07-01\"}],\"batchDisabled\":false,\"transformers\":[{\"type\":\"jsonpath\",\"settings\":{\"tablePath\":\"$.*[?(@.resourceType=='virtualMachines')]\",\"columns\":[{\"path\":\"name\",\"columnid\":\"Name\"},{\"path\":\"capabilities[?(@.name=='vCPUs')].value\",\"columnid\":\"vCPUs\"},{\"path\":\"capabilities[?(@.name=='MemoryGB')].value\",\"columnid\":\"MemoryGB\"},{\"path\":\"capabilities[?(@.name=='MaxNetworkInterfaces')].value\",\"columnid\":\"MaxNetworkInterfaces\"},{\"path\":\"capabilities[?(@.name=='HyperVGenerations')].value\",\"columnid\":\"HyperVGenerations\"},{\"path\":\"capabilities[?(@.name=='vCPUsPerCore')].value\",\"columnid\":\"vCPUsPerCore\"}]}}]}", + "size": 0, + "title": "Get VM vCPU", + "exportParameterName": "ResourceSKU", + "showExportToExcel": true, + "queryType": 12, + "gridSettings": { + "rowLimit": 5000 + } + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "API-Get_VM_SKU" + }, + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "3f12a4b6-b18d-4191-8c1c-6045a7edcb6b", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "VM/VMSS", + "subTarget": "VM", + "style": "link" + }, + { + "id": "78ac1878-4b69-4f32-af1f-a8f095afbed5", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "SQL", + "subTarget": "SQL", + "style": "link" + } + ] + }, + "name": "links - 1" + }, + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "0211f413-9f36-4750-9ef2-d382ba30ba6c", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Windows Virtual Machines", + "subTarget": "VM", + "preText": "VM", + "style": "link" + }, + { + "id": "dbe9a7fb-6ab1-4de1-a98b-4ec8a9af906c", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Linux Virtual Machines", + "subTarget": "LinuxVM", + "style": "link" + }, + { + "id": "79e7a97a-1413-41e8-b4c6-ebd1d0a45e2e", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "VM Scale Set", + "subTarget": "VMSS", + "style": "link" + }, + { + "id": "be820ada-a0f4-4c51-b17b-3e506edd1410", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Custom Images", + "subTarget": "CustomImages", + "style": "link" + }, + { + "id": "1dda3cc4-59ba-4758-9c51-d6c9fab18647", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Azure Arc", + "subTarget": "arc", + "style": "link" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "VM" + }, + "name": "links - 4" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources \r\n| where resourceGroup in ({ResourceGroup})\r\n| where type has \"microsoft.compute/virtualmachines\" or type =~ 'microsoft.compute/virtualMachineScaleSets'\r\n| where properties.storageProfile.osDisk.osType == \"Windows\"\r\n| extend OS=properties.storageProfile.imageReference.id\r\n| where isnotnull(OS)\r\n| extend ActualCores = toint(extract(\".[A-Z]([0-9]+)\", 1, tostring(properties.hardwareProfile.vmSize)))\r\n| where tostring(properties.['licenseType']) has 'Windows'\r\n| extend WindowsId=id, VMIDFull=id, VMName=name, VMLocation=location, VMRG=resourceGroup, OSType=tostring(properties.storageProfile.osDisk.osType), VMSize=tostring (properties.hardwareProfile.vmSize), ActualCores, LicenseType = tostring(properties.['licenseType']), VMSSize=tostring(sku.name), QuickFix=id\r\n ) on subscriptionId \r\n| order by type asc \r\n| project WindowsId,VMName,VMRG,VMSize, ActualCores,VMLocation,OSType, LicenseType, subscriptionId, QuickFix, VMIDFull\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), WindowsId=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct WindowsId\r\n )\r\n on WindowsId", + "size": 0, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "CustomImageAHBEnabled" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources \r\n| where resourceGroup in ({ResourceGroup})\r\n| where type has \"microsoft.compute/virtualmachines\" or type =~ 'microsoft.compute/virtualMachineScaleSets'\r\n| where properties.storageProfile.osDisk.osType != \"Windows\"\r\n| extend ActualCores = toint(extract(\".[A-Z]([0-9]+)\", 1, tostring(properties.hardwareProfile.vmSize)))\r\n| extend OS=properties.storageProfile.imageReference.id\r\n| where isnotnull(OS)\r\n| where tostring(properties.['licenseType']) has 'Windows'\r\n| extend WindowsId=id, VMIDFull=id, VMName=name, VMLocation=location, VMRG=resourceGroup, OSType=tostring(properties.storageProfile.osDisk.osType), VMSize=tostring (properties.hardwareProfile.vmSize), ActualCores, LicenseType = tostring(properties.['licenseType']), VMSSize=tostring(sku.name), QuickFix=id\r\n ) on subscriptionId \r\n| order by type asc \r\n| project WindowsId,VMName,VMRG,VMSize, ActualCores,VMLocation,OSType, LicenseType, subscriptionId, QuickFix, VMIDFull\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), WindowsId=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct WindowsId\r\n )\r\n on WindowsId", + "size": 0, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "True" + }, + "name": "CustomImageAHBDisabled" + }, + { + "type": 1, + "content": { + "json": "## Custom Images - Windows Azure Hybrid Benefit (AHB) Overview\r\n#### Note: This tab contains information only about Custom Images where the OS is Windows." + }, + "name": "AHB Overview" + }, + { + "type": 1, + "content": { + "json": "Each two-processor license or each set of 16-core licenses, either Datacenter or Standard editions, are entitled to two instances of up to 8 cores, or one instance of up to 16 cores.\r\n\r\nThe virtual machines (VMs) with less than 8 cores are categorized as **Low Priority**, while those with 8 or more cores are classified as **High Priority**. In situations where there are insufficient Azure Hybrid benefit licenses to cover all the VMs in the environment, it is recommended to prioritize the High Priority VMs.", + "style": "info" + }, + "name": "NUmber of Processors-CustomImages", + "styleSettings": { + "margin": "10px", + "showBorder": true + } + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name, SubId=id\r\n| join (\r\nresources \r\n| where resourceGroup in ({ResourceGroup})\r\n| where type has \"microsoft.compute/virtualmachines\" or type =~ 'microsoft.compute/virtualMachineScaleSets'\r\n| where properties.storageProfile.osDisk.osType == \"Windows\"\r\n| extend OS=properties.storageProfile.imageReference.id\r\n| where isnotnull(OS)\r\n| extend WindowsId=id\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), WindowsId=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct WindowsId\r\n )\r\n on WindowsId\r\n| extend LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckAHBWindows = case(\r\n type == 'microsoft.compute/virtualmachines' or type =~ 'microsoft.compute/virtualMachineScaleSets', iif((properties.['licenseType'])\r\n !has 'Windows' and (properties.virtualMachineProfile.['licenseType']) !has 'Windows' , \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not Windows\"\r\n )\r\n) on subscriptionId \r\n| summarize count() by CheckAHBWindows, SubId, SubscriptionName\r\n", + "size": 0, + "title": "Summary of Windows VMs with or without AHB per Subscription", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "table", + "gridSettings": { + "formatters": [ + { + "columnMatch": "SubId", + "formatter": 5 + }, + { + "columnMatch": "SubscriptionName", + "formatter": 5 + } + ], + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "SubscriptionName" + ], + "finalBy": "SubscriptionName" + }, + "labelSettings": [ + { + "columnId": "CheckAHBWindows", + "label": "Is AHB enabled?" + }, + { + "columnId": "SubscriptionName", + "label": "Subscription Name" + }, + { + "columnId": "count_", + "label": "Number of resources" + } + ] + }, + "sortBy": [], + "tileSettings": { + "titleContent": { + "columnMatch": "CheckAHBWindows", + "formatter": 1 + }, + "subtitleContent": { + "columnMatch": "SubscriptionName", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + } + }, + "showBorder": false, + "size": "auto" + }, + "chartSettings": { + "xAxis": "SubscriptionName" + } + }, + "customWidth": "50", + "name": "Summary of Windows VMs with or without AHB per Subscription - Custom Images" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources \r\n| where resourceGroup in ({ResourceGroup})\r\n| where type has \"microsoft.compute/virtualmachines\" or type =~ 'microsoft.compute/virtualMachineScaleSets'\r\n| where properties.storageProfile.osDisk.osType == \"Windows\"\r\n| extend OS=properties.storageProfile.imageReference.id\r\n| where isnotnull(OS)\r\n| extend WindowsId=id\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), WindowsId=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct WindowsId\r\n )\r\n on WindowsId\r\n| extend LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckAHBWindows = case(\r\n type == 'microsoft.compute/virtualmachines' or type =~ 'microsoft.compute/virtualMachineScaleSets', iif((properties.['licenseType'])\r\n !has 'Windows' and (properties.virtualMachineProfile.['licenseType']) !has 'Windows' , \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not Windows\"\r\n )\r\n) on subscriptionId \r\n| summarize count() by CheckAHBWindows", + "size": 0, + "title": "Summary of Windows VMs with or without AHB", + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart" + }, + "customWidth": "50", + "name": "Summary of Windows VMs with or without AHB-CustomImages" + }, + { + "type": 1, + "content": { + "json": "### Consumed Licenses\r\nTotal number of Windows licenses cores consumed by all Windows virtual machines.\r\n", + "style": "info" + }, + "customWidth": "50", + "name": "Consumed Licenses-CustomImages" + }, + { + "type": 1, + "content": { + "json": "### Number of required Cores to enable Windows Azure Hybrid Benefit\r\nNumber of cores required to enable AHB across the entire environment.", + "style": "info" + }, + "customWidth": "50", + "name": "Number of required Cores-CustomImages" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\",\"mergeType\":\"table\",\"leftTable\":\"CustomImageAHBEnabled\"}],\"projectRename\":[{\"originalName\":\"[CustomImageAHBEnabled].WindowsId\",\"mergedName\":\"WindowsId\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].ActualCores\",\"mergedName\":\"ActualCores\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].QuickFix\",\"mergedName\":\"QuickFix\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].VMIDFull\",\"mergedName\":\"VMIDFull\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"ActualCores\",\"operator\":\"<=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"static\",\"resultVal\":\"Low Priority\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"static\",\"resultVal\":\"High Priority\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"ConsumedCores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"ActualCores\",\"operator\":\"<=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"8\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"column\",\"resultVal\":\"ActualCores\"}}]}]}", + "size": 0, + "title": "Consumed Cores per AHB Priority", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 7, + "visualization": "piechart", + "chartSettings": { + "yAxis": [ + "ConsumedCores" + ], + "group": "Prioritize AHB", + "createOtherGroup": null + } + }, + "customWidth": "33", + "showPin": false, + "name": "ConsumedCoresPerAhubpriority-CustomImages" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\",\"mergeType\":\"table\",\"leftTable\":\"CustomImageAHBEnabled\"}],\"projectRename\":[{\"originalName\":\"[CustomImageAHBEnabled].WindowsId\",\"mergedName\":\"WindowsId\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].ActualCores\",\"mergedName\":\"ActualCores\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].QuickFix\",\"mergedName\":\"QuickFix\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].VMIDFull\",\"mergedName\":\"VMIDFull\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBEnabled].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"ActualCores\",\"operator\":\"<=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"static\",\"resultVal\":\"Low Priority\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"static\",\"resultVal\":\"High Priority\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"ConsumedCores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"ActualCores\",\"operator\":\"<=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"8\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"column\",\"resultVal\":\"ActualCores\"}}]}]}", + "size": 0, + "title": "Consumed Cores per VM", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 7, + "visualization": "piechart", + "chartSettings": { + "yAxis": [ + "ConsumedCores" + ], + "group": "VMName", + "createOtherGroup": null + } + }, + "customWidth": "30", + "showPin": false, + "name": "ConsumedCoresPerVM-CustomImages" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\",\"mergeType\":\"table\",\"leftTable\":\"CustomImageAHBDisabled\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"ActualCores\",\"operator\":\"<=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"static\",\"resultVal\":\"Low Priority\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"static\",\"resultVal\":\"High Priority\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"ConsumedCores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"ActualCores\",\"operator\":\"<=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"8\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"column\",\"resultVal\":\"ActualCores\"}}]},{\"originalName\":\"[CustomImageAHBDisabled].WindowsId\",\"mergedName\":\"WindowsId\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBDisabled].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBDisabled].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBDisabled].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBDisabled].ActualCores\",\"mergedName\":\"ActualCores\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBDisabled].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBDisabled].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBDisabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBDisabled].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBDisabled].QuickFix\",\"mergedName\":\"QuickFix\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBDisabled].VMIDFull\",\"mergedName\":\"VMIDFull\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"},{\"originalName\":\"[CustomImageAHBDisabled].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02ac\"}]}", + "size": 0, + "title": "Required CPU cores for AHB activation on remaining VMs", + "noDataMessage": "All VMs within the current scope have AHB enabled", + "noDataMessageStyle": 3, + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 7, + "visualization": "piechart", + "chartSettings": { + "yAxis": [ + "ConsumedCores" + ], + "group": "VMName", + "createOtherGroup": null + } + }, + "customWidth": "36", + "showPin": false, + "name": "CoresNotEnabledperAHBPriority-CustomImages" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "ae5e8765-47ef-46a6-803b-6b7124c098d2", + "version": "KqlParameterItem/1.0", + "name": "AHBEnabled", + "label": "See VMs with AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n", + "timeContext": { + "durationMs": 86400000 + } + }, + { + "id": "f1ac5e53-253c-4afb-8bc5-b1ba2efea3eb", + "version": "KqlParameterItem/1.0", + "name": "AHBDisabled", + "label": "See VMs without AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n" + } + ], + "style": "pills", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "VM AHB Enabled - CustomImages" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\",\"mergeType\":\"table\",\"leftTable\":\"ConsumedCoresPerAhubpriority-CustomImages\"}],\"projectRename\":[{\"originalName\":\"[ConsumedCoresPerAhubpriority-CustomImages].WindowsId\",\"mergedName\":\"WindowsId\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[ConsumedCoresPerAhubpriority-CustomImages].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[ConsumedCoresPerAhubpriority-CustomImages].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[ConsumedCoresPerAhubpriority-CustomImages].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[ConsumedCoresPerAhubpriority-CustomImages].ActualCores\",\"mergedName\":\"ActualCores\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[ConsumedCoresPerAhubpriority-CustomImages].ConsumedCores\",\"mergedName\":\"ConsumedCores\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[ConsumedCoresPerAhubpriority-CustomImages].Prioritize AHB\",\"mergedName\":\"Prioritize AHB\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[ConsumedCoresPerAhubpriority-CustomImages].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[ConsumedCoresPerAhubpriority-CustomImages].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[ConsumedCoresPerAhubpriority-CustomImages].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[ConsumedCoresPerAhubpriority-CustomImages].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[ConsumedCoresPerAhubpriority-CustomImages].QuickFix\",\"mergedName\":\"QuickFix\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[ConsumedCoresPerAhubpriority-CustomImages].VMIDFull\",\"mergedName\":\"VMIDFull\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[ConsumedCoresPerAhubpriority-CustomImages].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"}]}", + "size": 0, + "title": "VMs with Azure Hybrid Benefit enabled", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "WindowsId", + "formatter": 13, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Prioritize AHB", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "success", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 5 + }, + { + "columnMatch": "QuickFix", + "formatter": 5 + }, + { + "columnMatch": "VMIDFull", + "formatter": 5 + }, + { + "columnMatch": "WindowsId1", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "WindowsId", + "label": "ResourceId" + }, + { + "columnId": "VMName", + "label": "VM name" + }, + { + "columnId": "VMRG", + "label": "Resource group" + }, + { + "columnId": "VMSize", + "label": "SKU" + }, + { + "columnId": "ActualCores", + "label": "Number of cores" + }, + { + "columnId": "ConsumedCores", + "label": "Consumed cores" + }, + { + "columnId": "Prioritize AHB", + "label": "AHB priority" + }, + { + "columnId": "VMLocation", + "label": "Location" + }, + { + "columnId": "OSType", + "label": "OS" + }, + { + "columnId": "LicenseType", + "label": "License" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "AHBEnabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "showPin": false, + "name": "VmsAHBEnabeld-CustomImages" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\",\"mergeType\":\"table\",\"leftTable\":\"CoresNotEnabledperAHBPriority-CustomImages\"}],\"projectRename\":[{\"originalName\":\"[CoresNotEnabledperAHBPriority-CustomImages].WindowsId\",\"mergedName\":\"WindowsId\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[CoresNotEnabledperAHBPriority-CustomImages].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[CoresNotEnabledperAHBPriority-CustomImages].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[CoresNotEnabledperAHBPriority-CustomImages].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[CoresNotEnabledperAHBPriority-CustomImages].ActualCores\",\"mergedName\":\"ActualCores\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[CoresNotEnabledperAHBPriority-CustomImages].ConsumedCores\",\"mergedName\":\"ConsumedCores\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[CoresNotEnabledperAHBPriority-CustomImages].Prioritize AHB\",\"mergedName\":\"Prioritize AHB\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[CoresNotEnabledperAHBPriority-CustomImages].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[CoresNotEnabledperAHBPriority-CustomImages].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[CoresNotEnabledperAHBPriority-CustomImages].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[CoresNotEnabledperAHBPriority-CustomImages].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[CoresNotEnabledperAHBPriority-CustomImages].QuickFix\",\"mergedName\":\"QuickFix\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[CoresNotEnabledperAHBPriority-CustomImages].VMIDFull\",\"mergedName\":\"VMIDFull\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"},{\"originalName\":\"[CoresNotEnabledperAHBPriority-CustomImages].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"0b494d5e-27b1-4cf6-bcd5-3e4e813a02e8\"}]}", + "size": 0, + "title": "VMs with Azure Hybrid Benefit not enabled", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "WindowsId", + "formatter": 13, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Prioritize AHB", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "success", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 5 + }, + { + "columnMatch": "QuickFix", + "formatter": 5 + }, + { + "columnMatch": "VMIDFull", + "formatter": 5 + }, + { + "columnMatch": "WindowsId1", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "WindowsId", + "label": "ResourceId" + }, + { + "columnId": "VMName", + "label": "VM name" + }, + { + "columnId": "VMRG", + "label": "Resource group" + }, + { + "columnId": "VMSize", + "label": "SKU" + }, + { + "columnId": "ActualCores", + "label": "Number of cores" + }, + { + "columnId": "ConsumedCores", + "label": "Consumed cores" + }, + { + "columnId": "Prioritize AHB", + "label": "AHB priority" + }, + { + "columnId": "VMLocation", + "label": "Location" + }, + { + "columnId": "OSType", + "label": "OS" + }, + { + "columnId": "LicenseType", + "label": "License" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "AHBDisabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "showPin": false, + "name": "VmsAHBDisabled-CustomImages" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "CustomImages" + }, + "name": "CustomImages" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "3f12a4b6-b18d-4191-8c1c-6045a7edcb6b", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Windows Server ESU licenses", + "subTarget": "windowsEsu", + "style": "link" + }, + { + "id": "0ca197b7-6f96-4def-9be4-dc26bb09538a", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "Windows Server Management enabled by Azure Arc", + "subTarget": "arcAHB", + "style": "link" + } + ] + }, + "name": "links - 1" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "title": "Windows Server ESU licenses", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "// Run query to see results.\r\nresources\r\n| where type =~ 'microsoft.hybridcompute/machines'\r\n| extend esuEligibility = properties.licenseProfile.esuProfile.esuEligibility\r\n| where esuEligibility =~ \"Eligible\"\r\n| extend agentVersion = tostring(properties.agentVersion)\r\n| extend parsedAgentVersion = parse_version(agentVersion)\r\n| where parsedAgentVersion >= parse_version(\"1.34\")\r\n| extend machineId = tolower(id)\r\n| join kind=leftouter(\r\n resources\r\n | where type =~ \"microsoft.hybridcompute/machines/licenseProfiles\"\r\n | extend machineId = tolower(tostring(trim_end(@\"\\/\\w+\\/(\\w|\\.)+\", id)))\r\n | extend licenseId = tolower(tostring(properties.esuProfile.assignedLicense))\r\n) on $left.machineId == $right.machineId\r\n| join kind=leftouter (\r\n resources\r\n | where type =~ \"microsoft.hybridcompute/licenses\"\r\n | extend licenseId = tolower(id)\r\n) on licenseId\r\n| extend status = case(\r\n properties.status =~ 'Connected', 'Connected',\r\n properties.status =~ 'Disconnected', 'Offline',\r\n properties.status =~ 'Error', 'Error',\r\n properties.status =~ 'Expired', 'Expired',\r\n 'Unknown')\r\n| extend operatingSystem = properties.osSku\r\n| extend esuStatus = case(\r\n (properties.licenseProfile.esuProfile.licenseAssignmentState =~ 'Assigned' and properties2.licenseDetails.state =~ 'Activated'), 'Enabled',\r\n properties.licenseProfile.esuProfile.licenseAssignmentState =~ 'NotAssigned' and properties.licenseProfile.esuProfile.esuKeyState =~ 'Active', 'Enabled by Volume License',\r\n properties.licenseProfile.esuProfile.licenseAssignmentState =~ 'Assigned' and properties2.licenseDetails.state =~ 'Deactivated' and properties.licenseProfile.esuProfile.esuKeyState =~ 'Inactive', 'License deactivated',\r\n properties.licenseProfile.esuProfile.licenseAssignmentState =~ 'NotAssigned' and properties.licenseProfile.esuProfile.esuKeyState =~ 'Inactive', 'Not enabled',\r\n 'Unknown'\r\n)\r\n| project name, status, operatingSystem, esuStatus\r\n| summarize count() by esuStatus", + "size": 0, + "title": "Coverage Summary", + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart", + "chartSettings": { + "yAxis": [ + "count_" + ], + "showLegend": true, + "seriesLabelSettings": [ + { + "seriesName": "Not enabled", + "color": "red" + }, + { + "seriesName": "Enabled", + "color": "green" + }, + { + "seriesName": "Enabled by Volume License", + "color": "greenDarkDark" + }, + { + "seriesName": "License deactivated", + "color": "yellow" + } + ] + } + }, + "name": "coverageSummary" + }, + { + "type": 1, + "content": { + "json": "## Review Your Current License Usage\r\n\r\nTo review your current license usage, go to the [Azure Arc](https://ms.portal.azure.com/#view/Microsoft_Azure_ArcCenterUX/ArcCenterMenuBlade/~/usageOverview) resource blade.\r\n\r\n", + "style": "info" + }, + "name": "text - 5" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "16791a28-f78e-4b26-a2e9-a9fbbda915df", + "version": "KqlParameterItem/1.0", + "name": "eligibleResources", + "label": "View eligible resources?", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n", + "timeContext": { + "durationMs": 86400000 + } + }, + { + "id": "e24ddae0-37c5-46a7-b52f-b307baaa6f51", + "version": "KqlParameterItem/1.0", + "name": "eligibleLicenses", + "label": "View eligible licenses?", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n", + "timeContext": { + "durationMs": 86400000 + } + } + ], + "style": "pills", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "parameters - 5" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "title": "Eligible Resources", + "items": [ + { + "type": 1, + "content": { + "json": "Windows Server 2012 or 2012 R2 machines running Arc agent version below 1.34 are ineligible for Extended Security Updates (ESUs). Upgrade to the latest version of the Azure Arc agent to allow enabling ESU on these machines.\r\n\r\n", + "style": "info" + }, + "name": "txtEligibleResources" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "// Run query to see results.\r\nresources\r\n| where type =~ 'microsoft.hybridcompute/machines'\r\n| extend esuEligibility = properties.licenseProfile.esuProfile.esuEligibility\r\n| where esuEligibility =~ \"Eligible\"\r\n| extend agentVersion = tostring(properties.agentVersion)\r\n| extend parsedAgentVersion = parse_version(agentVersion)\r\n| where parsedAgentVersion >= parse_version(\"1.34\")\r\n| extend machineId = tolower(id)\r\n| join kind=leftouter(\r\n resources\r\n | where type =~ \"microsoft.hybridcompute/machines/licenseProfiles\"\r\n | extend machineId = tolower(tostring(trim_end(@\"\\/\\w+\\/(\\w|\\.)+\", id)))\r\n | extend licenseId = tolower(tostring(properties.esuProfile.assignedLicense))\r\n) on $left.machineId == $right.machineId\r\n| join kind=leftouter (\r\n resources\r\n | where type =~ \"microsoft.hybridcompute/licenses\"\r\n | extend licenseId = tolower(id)\r\n) on licenseId\r\n| extend status = case(\r\n properties.status =~ 'Connected', 'Connected',\r\n properties.status =~ 'Disconnected', 'Offline',\r\n properties.status =~ 'Error', 'Error',\r\n properties.status =~ 'Expired', 'Expired',\r\n 'Unknown')\r\n| extend operatingSystem = properties.osSku\r\n| extend esuStatus = case(\r\n (properties.licenseProfile.esuProfile.licenseAssignmentState =~ 'Assigned' and properties2.licenseDetails.state =~ 'Activated'), 'Enabled',\r\n properties.licenseProfile.esuProfile.licenseAssignmentState =~ 'NotAssigned' and properties.licenseProfile.esuProfile.esuKeyState =~ 'Active', 'Enabled by Volume License',\r\n properties.licenseProfile.esuProfile.licenseAssignmentState =~ 'Assigned' and properties2.licenseDetails.state =~ 'Deactivated' and properties.licenseProfile.esuProfile.esuKeyState =~ 'Inactive', 'License deactivated',\r\n properties.licenseProfile.esuProfile.licenseAssignmentState =~ 'NotAssigned' and properties.licenseProfile.esuProfile.esuKeyState =~ 'Inactive', 'Not enabled',\r\n 'Unknown'\r\n)\r\n| extend esuStatusIcon = case(\r\n (properties.licenseProfile.esuProfile.licenseAssignmentState =~ 'Assigned' and properties2.licenseDetails.state =~ 'Activated'), '8',\r\n properties.licenseProfile.esuProfile.licenseAssignmentState =~ 'NotAssigned' and properties.licenseProfile.esuProfile.esuKeyState =~ 'Active', '9',\r\n properties.licenseProfile.esuProfile.licenseAssignmentState =~ 'Assigned' and properties2.licenseDetails.state =~ 'Deactivated' and properties.licenseProfile.esuProfile.esuKeyState =~ 'Inactive', '9',\r\n properties.licenseProfile.esuProfile.licenseAssignmentState =~ 'NotAssigned' or properties.licenseProfile.esuProfile.esuKeyState =~ 'Inactive', '7',\r\n '91'\r\n)\r\n| project name, status, resourceGroup, subscriptionId, operatingSystem, id, type, location, kind, tags, esuStatus, esuStatusIcon, agentVersion\r\n| extend subscriptionDisplayName=case(subscriptionId =~ 'e75c95f3-27b4-410f-a40e-2b9153a807dd','AEther Dev',subscriptionId =~ '823ca539-d44d-43ee-8dc8-023fd4f27396','AIOps_FailureSimulation_DevTest',subscriptionId =~ 'b2a328a7-ffff-4c09-b643-a4758cf170bc','AISC-DEV-02',subscriptionId =~ 'eef8b6d5-94da-4b36-9327-a662f2674efb','AISC-EngSys-01',subscriptionId =~ 'e0fd569c-e34a-4249-8c24-e8d723c7f054','AML - Responsible AI R&D',subscriptionId =~ 'f1d79e73-f8e3-4b10-bfdb-4207ca0723ed','AML Infra - Engineering System',subscriptionId =~ 'dad45786-32e5-4ef3-b90e-8e0838fbadb6','AnE.ExP.NonProduction',subscriptionId =~ '017ad81c-b730-4564-9109-9b763243a9a6','Anomaly Real Time-DevTest',subscriptionId =~ 'e686ef8c-d35d-4e9b-92f8-caaaa7948c0a','Applied Sciences Group Dev',subscriptionId =~ '6e967edb-425b-4a33-ae98-f1d2c509dda3','Arc enabled Infrastructure - Demo',subscriptionId =~ '710c6ed8-d8d9-4c51-b100-dc4d9e10dd18','ARC-ESU-TESTING-03',subscriptionId =~ '2145a411-d149-4010-84d4-40fe8a55db44','ARM Test Environment',subscriptionId =~ 'e2a8a7b3-cb7a-4930-9c79-2b0eb7bf843c','ASZ_Lab_Hardware_Dev_1',subscriptionId =~ '4c33a6bb-d566-4d0b-8951-00b55a91fba8','AzPIESecurity-DEV-Apple',subscriptionId =~ 'b65b516b-415b-4c68-a254-bfa7411275f8','Azure CloudES CDP ER - Test',subscriptionId =~ '6785ea1f-ac40-4244-a9ce-94b12fd832ca','Azure Migrate Demo Subscription',subscriptionId =~ '8c3c936a-c09b-4de3-830b-3f5f244d72e9','Azure Migrate Program Management Team',subscriptionId =~ '432a7068-99ae-4975-ad38-d96b71172cdf','Azure Profiler - Testing',subscriptionId =~ 'caffe3c0-acbd-4d01-af76-a45f421bfb64','Azure_Base_LivesiteArmory_test',subscriptionId =~ 'a1920ebd-59b7-4f19-af9f-5e80599e88e4','Babel',subscriptionId =~ 'b43a6159-1bea-4fa2-9407-e875fdc0ff55','BAP IA EU Schrems (P)',subscriptionId =~ 'ae71ef11-a03f-4b4f-a0e6-ef144727c711','Bing MM Measurement',subscriptionId =~ '8ef80208-601f-4c83-802e-751d211745f9','CloudAnalytics_Prod1',subscriptionId =~ 'f7b4a5ce-d3bd-4a03-84fe-8bcad772c8c1','Cog Services Computer Vision',subscriptionId =~ '38dbd059-bd26-4172-af17-bcc5a00744de','Core-ES-STREAM-DeXDI',subscriptionId =~ '87e8a61c-4832-4cfe-8b72-c12554f2309a','Cosmos_C&E_Azure_Azure Resource Manager_100424',subscriptionId =~ '8c4b5b03-3b24-4ed0-91f5-a703cd91b412','Cosmos_C&E_Azure_AzureEngineeringSystems_100200',subscriptionId =~ 'b988efc7-1bc8-4a73-b4a2-5ad864580250','Cosmos_C&E_DPG_Big Data_100036',subscriptionId =~ '6121bf73-2dc9-47ac-8b6a-c6ae4b88ef39','Cost Management PM',subscriptionId =~ '44074499-ad9f-45b9-9b0b-f594f8736a45','CSX-WSD-CFE-TNT-QPP',subscriptionId =~ '3cd01953-be46-4f9b-a4e0-26f19f428765','CSX-WSD-CFE-TNT-TOOLKITHUB',subscriptionId =~ '60214a3c-65a2-46f8-8b32-f959f2454106','Data Pillar Security Tooling',subscriptionId =~ 'f7c445af-a4de-4264-9e87-3196d6bc384d','Efficiency Pack Services (MSIT)',subscriptionId =~ '86f010b8-6473-4cb9-90c1-671c74faa4ee','emilyzhu dev subscription',subscriptionId =~ 'f3326bfd-5406-4136-a835-f64ccbfd9050','FTK @ MS',subscriptionId =~ 'd2c9544f-4329-4642-b73d-020e7fef844f','HPCScrub1',subscriptionId =~ 'bac420ed-c6fc-4a05-8ac1-8c0c52da1d6e','IDEAs MS Reporting',subscriptionId =~ '3cd95ff9-ac62-4b5c-8240-0cd046687ea0','Internal App Insights Resources for Perflens',subscriptionId =~ 'a386d5ea-ea90-441a-8263-d816368c84a1','IOT_PLATFORM_UPX_TEST',subscriptionId =~ '6a0ab98c-24d4-4131-8cd7-0a78bfce58f5','ITSM Integration - Production - New',subscriptionId =~ '13723929-6644-4060-a50a-cc38ebc5e8b1','LinuxMdsd Test',subscriptionId =~ '98b02a69-28a5-4ee7-a622-cd69c7a59c4e','Marvel Intelligence Model Training Subscription',subscriptionId =~ '1278a874-89fc-418c-b6b9-ac763b000415','Microsoft Azure Internal Consumption (nbrady)',subscriptionId =~ '29de2cfc-f00a-43bb-bdc8-3108795bd282','MSFT-Modern Device-Modern Mgmt-Imaging00',subscriptionId =~ '54b875cc-a81a-4914-8bfd-1a36bc7ddf4d','MSFT-WindowsVirtualDesktop-01',subscriptionId =~ '4dc2cd39-7a89-43d8-bebe-8bb501359891','Observability_AzMon_Grafana_Dev',subscriptionId =~ 'a471d615-ff98-4e80-b375-a19543d4691e','ODC-OneBox-Resources',subscriptionId =~ '1163fbbe-27e7-4b0f-8466-195fe5417043','Python_AI_Tools_PM_Team_Sub (jbinder)',subscriptionId =~ '3905431d-c062-4c17-8fd9-c51f89f334c4','Pytorch Build',subscriptionId =~ '3f577935-3138-4d07-86b3-75651b696483','Responsible & OpenAi Research',subscriptionId =~ '0f301386-8979-4981-acca-973d553078e7','Scope Team Test Subscription',subscriptionId =~ '1b0a5c20-7373-41a2-8fec-7364cceb7bbf','Services Hub Demo Open',subscriptionId =~ 'e72e5254-f265-4e95-9bd2-9ee8e7329051','Speech Services - DEV - SDK (carbon)',subscriptionId =~ 'a1c3dc6b-8630-4bb7-a29e-4ed4407c329b','Speech Services - EXP2',subscriptionId =~ '736af2bf-9fcb-4145-a19b-5b30b2b8949d','Trey International UK',subscriptionId =~ '51f73f67-1f29-4120-863e-dd315f743fc1','Trey Partner Lab 2',subscriptionId =~ '9ec51cfd-5ca7-4d76-8101-dd0a4abc5674','Trey Research Corporate',subscriptionId =~ '73c0021f-a37d-433f-8baa-7450cb54eea6','Trey Research Finance',subscriptionId =~ 'ed570627-0265-4620-bb42-bae06bcfa914','Trey Research IT',subscriptionId =~ '64e355d7-997c-491d-b0c1-8414dccfcf42','Trey Research R&D Playground',subscriptionId =~ '586f1d47-9dd9-43d5-b196-6e28f8405ff8','Trey Research R&D Production',subscriptionId =~ '9ec1d932-0f3f-486c-acc6-e7d78b358f9b','TScience',subscriptionId =~ '79f57c16-00fe-48da-87d4-5192e86cd047','TScienceGPU',subscriptionId =~ 'bac044cf-49e1-4843-8dda-1ce9662606c8','UI Fabric',subscriptionId =~ '6760347d-9ffe-41a9-ba11-c139dcea5ce6','xt-teams-migration-dev',subscriptionId =~ '45f9252d-e27e-4ed8-ab4e-dc5054de13fa','Contoso IT - Retail - Prod',subscriptionId)\r\n| where (type !~ ('dell.storage/filesystems'))\r\n| where (type !~ ('arizeai.observabilityeval/organizations'))\r\n| where (type !~ ('lambdatest.hyperexecute/organizations'))\r\n| where (type !~ ('pinecone.vectordb/organizations'))\r\n| where (type !~ ('microsoft.weightsandbiases/instances'))\r\n| where (type !~ ('purestorage.block/storagepools/avsstoragecontainers'))\r\n| where (type !~ ('purestorage.block/reservations'))\r\n| where (type !~ ('purestorage.block/storagepools'))\r\n| where (type !~ ('solarwinds.observability/organizations'))\r\n| where (type !~ ('splitio.experimentation/experimentationworkspaces'))\r\n| where (type !~ ('microsoft.agfoodplatform/farmbeats'))\r\n| where (type !~ ('microsoft.appsecurity/policies'))\r\n| where (type !~ ('microsoft.arc/all'))\r\n| where (type !~ ('microsoft.arc/allfairfax'))\r\n| where (type !~ ('microsoft.cdn/profiles/customdomains'))\r\n| where (type !~ ('microsoft.cdn/profiles/afdendpoints'))\r\n| where (type !~ ('microsoft.cdn/profiles/origingroups/origins'))\r\n| where (type !~ ('microsoft.cdn/profiles/origingroups'))\r\n| where (type !~ ('microsoft.cdn/profiles/afdendpoints/routes'))\r\n| where (type !~ ('microsoft.cdn/profiles/rulesets/rules'))\r\n| where (type !~ ('microsoft.cdn/profiles/rulesets'))\r\n| where (type !~ ('microsoft.cdn/profiles/secrets'))\r\n| where (type !~ ('microsoft.cdn/profiles/securitypolicies'))\r\n| where (type !~ ('microsoft.chaos/privateaccesses'))\r\n| where (type !~ ('microsoft.sovereign/transparencylogs'))\r\n| where (type !~ ('microsoft.sovereign/landingzoneconfigurations'))\r\n| where (type !~ ('microsoft.hardwaresecuritymodules/cloudhsmclusters'))\r\n| where (type !~ ('microsoft.compute/computefleetinstances'))\r\n| where (type !~ ('microsoft.compute/standbypoolinstance'))\r\n| where (type !~ ('microsoft.compute/virtualmachineflexinstances'))\r\n| where (type !~ ('microsoft.kubernetesconfiguration/extensions'))\r\n| where (type !~ ('microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/extensions'))\r\n| where (type !~ ('microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/namespaces'))\r\n| where (type !~ ('microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/namespaces'))\r\n| where (type !~ ('microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/fluxconfigurations'))\r\n| where (type !~ ('microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/fluxconfigurations'))\r\n| where (type !~ ('microsoft.portalservices/extensions/deployments'))\r\n| where (type !~ ('microsoft.portalservices/extensions'))\r\n| where (type !~ ('microsoft.portalservices/extensions/slots'))\r\n| where (type !~ ('microsoft.portalservices/extensions/versions'))\r\n| where (type !~ ('microsoft.datacollaboration/workspaces'))\r\n| where (type !~ ('microsoft.deviceregistry/devices'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/activedeployments'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/agents'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/deployments'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/deviceclasses'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/updates'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts'))\r\n| where (type !~ ('private.devtunnels/tunnelplans'))\r\n| where (type !~ ('private.easm/workspaces'))\r\n| where (type !~ ('microsoft.edgeorder/virtual_orderitems'))\r\n| where (type !~ ('microsoft.workloads/epicvirtualinstances'))\r\n| where (type !~ ('microsoft.fairfieldgardens/provisioningresources/provisioningpolicies'))\r\n| where (type !~ ('microsoft.fairfieldgardens/provisioningresources'))\r\n| where (type !~ ('microsoft.healthmodel/healthmodels'))\r\n| where (type !~ ('microsoft.hybridcompute/arcserverwithwac'))\r\n| where (type !~ ('microsoft.hybridcompute/machinessovereign'))\r\n| where (type !~ ('microsoft.hybridcompute/machinespaygo'))\r\n| where (type !~ ('microsoft.hybridcompute/machinessoftwareassurance'))\r\n| where (type !~ ('microsoft.network/virtualhubs')) or ((kind =~ ('routeserver')))\r\n| where (type !~ ('microsoft.network/networkvirtualappliances'))\r\n| where (type !~ ('microsoft.devhub/iacprofiles'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/files'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/filerequests'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/licenses'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/connectors'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/sharedstorages'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/storages'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/workloads'))\r\n| where (type !~ ('microsoft.insights/diagnosticsettings'))\r\n| where not((type =~ ('microsoft.network/serviceendpointpolicies')) and ((kind =~ ('internal'))))\r\n| where (type !~ ('microsoft.resources/resourcegraphvisualizer'))\r\n| where (type !~ ('microsoft.iotoperationsmq/mq'))\r\n| where (type !~ ('microsoft.orbital/cloudaccessrouters'))\r\n| where (type !~ ('microsoft.orbital/terminals'))\r\n| where (type !~ ('microsoft.orbital/sdwancontrollers'))\r\n| where (type !~ ('microsoft.orbital/geocatalogs'))\r\n| where (type !~ ('microsoft.recommendationsservice/accounts/modeling'))\r\n| where (type !~ ('microsoft.recommendationsservice/accounts/serviceendpoints'))\r\n| where (type !~ ('microsoft.recoveryservicesbvtd/vaults'))\r\n| where (type !~ ('microsoft.recoveryservicesbvtd2/vaults'))\r\n| where (type !~ ('microsoft.recoveryservicesintd/vaults'))\r\n| where (type !~ ('microsoft.recoveryservicesintd2/vaults'))\r\n| where (type !~ ('microsoft.resources/deletedresources'))\r\n| where (type !~ ('microsoft.deploymentmanager/rollouts'))\r\n| where (type !~ ('microsoft.features/featureprovidernamespaces/featureconfigurations'))\r\n| where (type !~ ('microsoft.saashub/cloudservices/hidden'))\r\n| where (type !~ ('microsoft.providerhub/providerregistrations'))\r\n| where (type !~ ('microsoft.providerhub/providerregistrations/customrollouts'))\r\n| where (type !~ ('microsoft.providerhub/providerregistrations/defaultrollouts'))\r\n| where (type !~ ('microsoft.edge/configurations'))\r\n| where not((type =~ ('microsoft.synapse/workspaces/sqlpools')) and ((kind =~ ('v3'))))\r\n| where (type !~ ('microsoft.mission/approvals'))\r\n| where (type !~ ('microsoft.mission/catalogs'))\r\n| where (type !~ ('microsoft.mission/communities'))\r\n| where (type !~ ('microsoft.mission/communities/communityendpoints'))\r\n| where (type !~ ('microsoft.mission/enclaveconnections'))\r\n| where (type !~ ('microsoft.mission/virtualenclaves/enclaveendpoints'))\r\n| where (type !~ ('microsoft.mission/communities/transithubs'))\r\n| where (type !~ ('microsoft.mission/virtualenclaves'))\r\n| where (type !~ ('microsoft.mission/virtualenclaves/workloads'))\r\n| where (type !~ ('microsoft.workloads/insights'))\r\n| where (type !~ ('microsoft.hanaonazure/sapmonitors'))\r\n| where (type !~ ('microsoft.cloudhealth/healthmodels'))\r\n| where (type !~ ('microsoft.connectedcache/enterprisemcccustomers/enterprisemcccachenodes'))\r\n| where not((type =~ ('microsoft.sql/servers/databases')) and ((kind in~ ('system','v2.0,system','v12.0,system','v12.0,system,serverless','v12.0,user,datawarehouse,gen2,analytics'))))\r\n| where not((type =~ ('microsoft.sql/servers')) and ((kind =~ ('v12.0,analytics'))))\r\n| where (type in~ ('Microsoft.HybridCompute/machinesEsu','Microsoft.HybridCompute/machines'))\r\n| project name,esuStatus,esuStatusIcon,operatingSystem,resourceGroup,subscriptionDisplayName,status,type,id,kind,location,subscriptionId,tags\r\n| sort by (tolower(tostring(name))) asc", + "size": 0, + "title": "Eligible resources", + "noDataMessage": "There are no eligible resources.", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "esuStatusIcon", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "7", + "representation": "4", + "text": "" + }, + { + "operator": "==", + "thresholdValue": "9", + "representation": "warning", + "text": "" + }, + { + "operator": "==", + "thresholdValue": "8", + "representation": "success", + "text": "" + }, + { + "operator": "==", + "thresholdValue": "91", + "representation": "unknown", + "text": "{0}{1}" + }, + { + "sourceColumn": "esuStatus", + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "" + } + ] + } + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "eligibleResources", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "arcResources" + } + ] + }, + "conditionalVisibility": { + "parameterName": "eligibleResources", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "group-eligibleResources" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "title": "Eligible licenses", + "items": [ + { + "type": 1, + "content": { + "json": "Keep in mind, subscription read permissions are required to give accurate core count and usage information for your ESU licenses. Physical core based licenses are subject to different licensing guidelines and are excluded from this view.[Learn more.](https://learn.microsoft.com/azure/azure-arc/servers/license-extended-security-updates)", + "style": "info" + }, + "name": "txtLearnMoreLicense" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "// Run query to see results.\r\nresources\r\n| where type =~ \"microsoft.hybridcompute/licenses\"\r\n| extend sku = properties.licenseDetails.edition\r\n| extend totalCores = properties.licenseDetails.processors\r\n| extend coreType = case(\r\n properties.licenseDetails.type =~ 'vCore','Virtual core',\r\n properties.licenseDetails.type =~ 'pCore','Physical core',\r\n 'Unknown'\r\n)\r\n| extend statusIcon = case(\r\n properties.licenseDetails.state =~ 'Activated', '8',\r\n properties.licenseDetails.state =~ 'Deactivated', '7',\r\n '91'\r\n)\r\n| extend status = case(\r\n properties.licenseDetails.state =~ 'Activated', 'Activated',\r\n properties.licenseDetails.state =~ 'Deactivated','Deactivated',\r\n 'Unknown'\r\n)\r\n| extend licenseId = tolower(tostring(id)) // Depending on what is stored in license profile, might have to get the immutableId instead\r\n| join kind=leftouter(\r\n resources\r\n | where type =~ \"microsoft.hybridcompute/machines/licenseProfiles\"\r\n | extend machineId = tolower(tostring(trim_end(@\"\\/\\w+\\/(\\w|\\.)+\", id)))\r\n | extend licenseId = tolower(tostring(properties.esuProfile.assignedLicense))\r\n | summarize resources = count() by licenseId\r\n) on licenseId // Get count of license profile per license, a license profile is created for each machine that is assigned a license\r\n| extend resources = iff(isnull(resources), 0, resources)\r\n| project id, name, type, location, subscriptionId, resourceGroup, kind, tags, extendedLocation, sku, totalCores, coreType, status, statusIcon, resources\r\n| extend subscriptionDisplayName=case(subscriptionId =~ 'e75c95f3-27b4-410f-a40e-2b9153a807dd','AEther Dev',subscriptionId =~ '823ca539-d44d-43ee-8dc8-023fd4f27396','AIOps_FailureSimulation_DevTest',subscriptionId =~ 'b2a328a7-ffff-4c09-b643-a4758cf170bc','AISC-DEV-02',subscriptionId =~ 'eef8b6d5-94da-4b36-9327-a662f2674efb','AISC-EngSys-01',subscriptionId =~ 'e0fd569c-e34a-4249-8c24-e8d723c7f054','AML - Responsible AI R&D',subscriptionId =~ 'f1d79e73-f8e3-4b10-bfdb-4207ca0723ed','AML Infra - Engineering System',subscriptionId =~ 'dad45786-32e5-4ef3-b90e-8e0838fbadb6','AnE.ExP.NonProduction',subscriptionId =~ '017ad81c-b730-4564-9109-9b763243a9a6','Anomaly Real Time-DevTest',subscriptionId =~ 'e686ef8c-d35d-4e9b-92f8-caaaa7948c0a','Applied Sciences Group Dev',subscriptionId =~ '6e967edb-425b-4a33-ae98-f1d2c509dda3','Arc enabled Infrastructure - Demo',subscriptionId =~ '710c6ed8-d8d9-4c51-b100-dc4d9e10dd18','ARC-ESU-TESTING-03',subscriptionId =~ '2145a411-d149-4010-84d4-40fe8a55db44','ARM Test Environment',subscriptionId =~ 'e2a8a7b3-cb7a-4930-9c79-2b0eb7bf843c','ASZ_Lab_Hardware_Dev_1',subscriptionId =~ '4c33a6bb-d566-4d0b-8951-00b55a91fba8','AzPIESecurity-DEV-Apple',subscriptionId =~ 'b65b516b-415b-4c68-a254-bfa7411275f8','Azure CloudES CDP ER - Test',subscriptionId =~ '6785ea1f-ac40-4244-a9ce-94b12fd832ca','Azure Migrate Demo Subscription',subscriptionId =~ '8c3c936a-c09b-4de3-830b-3f5f244d72e9','Azure Migrate Program Management Team',subscriptionId =~ '432a7068-99ae-4975-ad38-d96b71172cdf','Azure Profiler - Testing',subscriptionId =~ 'caffe3c0-acbd-4d01-af76-a45f421bfb64','Azure_Base_LivesiteArmory_test',subscriptionId =~ 'a1920ebd-59b7-4f19-af9f-5e80599e88e4','Babel',subscriptionId =~ 'b43a6159-1bea-4fa2-9407-e875fdc0ff55','BAP IA EU Schrems (P)',subscriptionId =~ 'ae71ef11-a03f-4b4f-a0e6-ef144727c711','Bing MM Measurement',subscriptionId =~ '8ef80208-601f-4c83-802e-751d211745f9','CloudAnalytics_Prod1',subscriptionId =~ 'f7b4a5ce-d3bd-4a03-84fe-8bcad772c8c1','Cog Services Computer Vision',subscriptionId =~ '38dbd059-bd26-4172-af17-bcc5a00744de','Core-ES-STREAM-DeXDI',subscriptionId =~ '87e8a61c-4832-4cfe-8b72-c12554f2309a','Cosmos_C&E_Azure_Azure Resource Manager_100424',subscriptionId =~ '8c4b5b03-3b24-4ed0-91f5-a703cd91b412','Cosmos_C&E_Azure_AzureEngineeringSystems_100200',subscriptionId =~ 'b988efc7-1bc8-4a73-b4a2-5ad864580250','Cosmos_C&E_DPG_Big Data_100036',subscriptionId =~ '6121bf73-2dc9-47ac-8b6a-c6ae4b88ef39','Cost Management PM',subscriptionId =~ '44074499-ad9f-45b9-9b0b-f594f8736a45','CSX-WSD-CFE-TNT-QPP',subscriptionId =~ '3cd01953-be46-4f9b-a4e0-26f19f428765','CSX-WSD-CFE-TNT-TOOLKITHUB',subscriptionId =~ '60214a3c-65a2-46f8-8b32-f959f2454106','Data Pillar Security Tooling',subscriptionId =~ 'f7c445af-a4de-4264-9e87-3196d6bc384d','Efficiency Pack Services (MSIT)',subscriptionId =~ '86f010b8-6473-4cb9-90c1-671c74faa4ee','emilyzhu dev subscription',subscriptionId =~ 'f3326bfd-5406-4136-a835-f64ccbfd9050','FTK @ MS',subscriptionId =~ 'd2c9544f-4329-4642-b73d-020e7fef844f','HPCScrub1',subscriptionId =~ 'bac420ed-c6fc-4a05-8ac1-8c0c52da1d6e','IDEAs MS Reporting',subscriptionId =~ '3cd95ff9-ac62-4b5c-8240-0cd046687ea0','Internal App Insights Resources for Perflens',subscriptionId =~ 'a386d5ea-ea90-441a-8263-d816368c84a1','IOT_PLATFORM_UPX_TEST',subscriptionId =~ '6a0ab98c-24d4-4131-8cd7-0a78bfce58f5','ITSM Integration - Production - New',subscriptionId =~ '13723929-6644-4060-a50a-cc38ebc5e8b1','LinuxMdsd Test',subscriptionId =~ '98b02a69-28a5-4ee7-a622-cd69c7a59c4e','Marvel Intelligence Model Training Subscription',subscriptionId =~ '1278a874-89fc-418c-b6b9-ac763b000415','Microsoft Azure Internal Consumption (nbrady)',subscriptionId =~ '29de2cfc-f00a-43bb-bdc8-3108795bd282','MSFT-Modern Device-Modern Mgmt-Imaging00',subscriptionId =~ '54b875cc-a81a-4914-8bfd-1a36bc7ddf4d','MSFT-WindowsVirtualDesktop-01',subscriptionId =~ '4dc2cd39-7a89-43d8-bebe-8bb501359891','Observability_AzMon_Grafana_Dev',subscriptionId =~ 'a471d615-ff98-4e80-b375-a19543d4691e','ODC-OneBox-Resources',subscriptionId =~ '1163fbbe-27e7-4b0f-8466-195fe5417043','Python_AI_Tools_PM_Team_Sub (jbinder)',subscriptionId =~ '3905431d-c062-4c17-8fd9-c51f89f334c4','Pytorch Build',subscriptionId =~ '3f577935-3138-4d07-86b3-75651b696483','Responsible & OpenAi Research',subscriptionId =~ '0f301386-8979-4981-acca-973d553078e7','Scope Team Test Subscription',subscriptionId =~ '1b0a5c20-7373-41a2-8fec-7364cceb7bbf','Services Hub Demo Open',subscriptionId =~ 'e72e5254-f265-4e95-9bd2-9ee8e7329051','Speech Services - DEV - SDK (carbon)',subscriptionId =~ 'a1c3dc6b-8630-4bb7-a29e-4ed4407c329b','Speech Services - EXP2',subscriptionId =~ '736af2bf-9fcb-4145-a19b-5b30b2b8949d','Trey International UK',subscriptionId =~ '51f73f67-1f29-4120-863e-dd315f743fc1','Trey Partner Lab 2',subscriptionId =~ '9ec51cfd-5ca7-4d76-8101-dd0a4abc5674','Trey Research Corporate',subscriptionId =~ '73c0021f-a37d-433f-8baa-7450cb54eea6','Trey Research Finance',subscriptionId =~ 'ed570627-0265-4620-bb42-bae06bcfa914','Trey Research IT',subscriptionId =~ '64e355d7-997c-491d-b0c1-8414dccfcf42','Trey Research R&D Playground',subscriptionId =~ '586f1d47-9dd9-43d5-b196-6e28f8405ff8','Trey Research R&D Production',subscriptionId =~ '9ec1d932-0f3f-486c-acc6-e7d78b358f9b','TScience',subscriptionId =~ '79f57c16-00fe-48da-87d4-5192e86cd047','TScienceGPU',subscriptionId =~ 'bac044cf-49e1-4843-8dda-1ce9662606c8','UI Fabric',subscriptionId =~ '6760347d-9ffe-41a9-ba11-c139dcea5ce6','xt-teams-migration-dev',subscriptionId =~ '45f9252d-e27e-4ed8-ab4e-dc5054de13fa','Contoso IT - Retail - Prod',subscriptionId)\r\n| where (type !~ ('dell.storage/filesystems'))\r\n| where (type !~ ('arizeai.observabilityeval/organizations'))\r\n| where (type !~ ('lambdatest.hyperexecute/organizations'))\r\n| where (type !~ ('pinecone.vectordb/organizations'))\r\n| where (type !~ ('microsoft.weightsandbiases/instances'))\r\n| where (type !~ ('purestorage.block/storagepools/avsstoragecontainers'))\r\n| where (type !~ ('purestorage.block/reservations'))\r\n| where (type !~ ('purestorage.block/storagepools'))\r\n| where (type !~ ('solarwinds.observability/organizations'))\r\n| where (type !~ ('splitio.experimentation/experimentationworkspaces'))\r\n| where (type !~ ('microsoft.agfoodplatform/farmbeats'))\r\n| where (type !~ ('microsoft.appsecurity/policies'))\r\n| where (type !~ ('microsoft.arc/all'))\r\n| where (type !~ ('microsoft.arc/allfairfax'))\r\n| where (type !~ ('microsoft.cdn/profiles/customdomains'))\r\n| where (type !~ ('microsoft.cdn/profiles/afdendpoints'))\r\n| where (type !~ ('microsoft.cdn/profiles/origingroups/origins'))\r\n| where (type !~ ('microsoft.cdn/profiles/origingroups'))\r\n| where (type !~ ('microsoft.cdn/profiles/afdendpoints/routes'))\r\n| where (type !~ ('microsoft.cdn/profiles/rulesets/rules'))\r\n| where (type !~ ('microsoft.cdn/profiles/rulesets'))\r\n| where (type !~ ('microsoft.cdn/profiles/secrets'))\r\n| where (type !~ ('microsoft.cdn/profiles/securitypolicies'))\r\n| where (type !~ ('microsoft.chaos/privateaccesses'))\r\n| where (type !~ ('microsoft.sovereign/transparencylogs'))\r\n| where (type !~ ('microsoft.sovereign/landingzoneconfigurations'))\r\n| where (type !~ ('microsoft.hardwaresecuritymodules/cloudhsmclusters'))\r\n| where (type !~ ('microsoft.compute/computefleetinstances'))\r\n| where (type !~ ('microsoft.compute/standbypoolinstance'))\r\n| where (type !~ ('microsoft.compute/virtualmachineflexinstances'))\r\n| where (type !~ ('microsoft.kubernetesconfiguration/extensions'))\r\n| where (type !~ ('microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/extensions'))\r\n| where (type !~ ('microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/namespaces'))\r\n| where (type !~ ('microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/namespaces'))\r\n| where (type !~ ('microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/fluxconfigurations'))\r\n| where (type !~ ('microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/fluxconfigurations'))\r\n| where (type !~ ('microsoft.portalservices/extensions/deployments'))\r\n| where (type !~ ('microsoft.portalservices/extensions'))\r\n| where (type !~ ('microsoft.portalservices/extensions/slots'))\r\n| where (type !~ ('microsoft.portalservices/extensions/versions'))\r\n| where (type !~ ('microsoft.datacollaboration/workspaces'))\r\n| where (type !~ ('microsoft.deviceregistry/devices'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/activedeployments'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/agents'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/deployments'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/deviceclasses'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/updates'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts'))\r\n| where (type !~ ('private.devtunnels/tunnelplans'))\r\n| where (type !~ ('private.easm/workspaces'))\r\n| where (type !~ ('microsoft.edgeorder/virtual_orderitems'))\r\n| where (type !~ ('microsoft.workloads/epicvirtualinstances'))\r\n| where (type !~ ('microsoft.fairfieldgardens/provisioningresources/provisioningpolicies'))\r\n| where (type !~ ('microsoft.fairfieldgardens/provisioningresources'))\r\n| where (type !~ ('microsoft.healthmodel/healthmodels'))\r\n| where (type !~ ('microsoft.hybridcompute/arcserverwithwac'))\r\n| where (type !~ ('microsoft.hybridcompute/machinessovereign'))\r\n| where (type !~ ('microsoft.hybridcompute/machinesesu'))\r\n| where (type !~ ('microsoft.hybridcompute/machinespaygo'))\r\n| where (type !~ ('microsoft.hybridcompute/machinessoftwareassurance'))\r\n| where (type !~ ('microsoft.network/virtualhubs')) or ((kind =~ ('routeserver')))\r\n| where (type !~ ('microsoft.network/networkvirtualappliances'))\r\n| where (type !~ ('microsoft.devhub/iacprofiles'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/files'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/filerequests'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/licenses'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/connectors'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/sharedstorages'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/storages'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/workloads'))\r\n| where (type !~ ('microsoft.insights/diagnosticsettings'))\r\n| where not((type =~ ('microsoft.network/serviceendpointpolicies')) and ((kind =~ ('internal'))))\r\n| where (type !~ ('microsoft.resources/resourcegraphvisualizer'))\r\n| where (type !~ ('microsoft.iotoperationsmq/mq'))\r\n| where (type !~ ('microsoft.orbital/cloudaccessrouters'))\r\n| where (type !~ ('microsoft.orbital/terminals'))\r\n| where (type !~ ('microsoft.orbital/sdwancontrollers'))\r\n| where (type !~ ('microsoft.orbital/geocatalogs'))\r\n| where (type !~ ('microsoft.recommendationsservice/accounts/modeling'))\r\n| where (type !~ ('microsoft.recommendationsservice/accounts/serviceendpoints'))\r\n| where (type !~ ('microsoft.recoveryservicesbvtd/vaults'))\r\n| where (type !~ ('microsoft.recoveryservicesbvtd2/vaults'))\r\n| where (type !~ ('microsoft.recoveryservicesintd/vaults'))\r\n| where (type !~ ('microsoft.recoveryservicesintd2/vaults'))\r\n| where (type !~ ('microsoft.resources/deletedresources'))\r\n| where (type !~ ('microsoft.deploymentmanager/rollouts'))\r\n| where (type !~ ('microsoft.features/featureprovidernamespaces/featureconfigurations'))\r\n| where (type !~ ('microsoft.saashub/cloudservices/hidden'))\r\n| where (type !~ ('microsoft.providerhub/providerregistrations'))\r\n| where (type !~ ('microsoft.providerhub/providerregistrations/customrollouts'))\r\n| where (type !~ ('microsoft.providerhub/providerregistrations/defaultrollouts'))\r\n| where (type !~ ('microsoft.edge/configurations'))\r\n| where not((type =~ ('microsoft.synapse/workspaces/sqlpools')) and ((kind =~ ('v3'))))\r\n| where (type !~ ('microsoft.mission/approvals'))\r\n| where (type !~ ('microsoft.mission/catalogs'))\r\n| where (type !~ ('microsoft.mission/communities'))\r\n| where (type !~ ('microsoft.mission/communities/communityendpoints'))\r\n| where (type !~ ('microsoft.mission/enclaveconnections'))\r\n| where (type !~ ('microsoft.mission/virtualenclaves/enclaveendpoints'))\r\n| where (type !~ ('microsoft.mission/communities/transithubs'))\r\n| where (type !~ ('microsoft.mission/virtualenclaves'))\r\n| where (type !~ ('microsoft.mission/virtualenclaves/workloads'))\r\n| where (type !~ ('microsoft.workloads/insights'))\r\n| where (type !~ ('microsoft.hanaonazure/sapmonitors'))\r\n| where (type !~ ('microsoft.cloudhealth/healthmodels'))\r\n| where (type !~ ('microsoft.connectedcache/enterprisemcccustomers/enterprisemcccachenodes'))\r\n| where not((type =~ ('microsoft.sql/servers/databases')) and ((kind in~ ('system','v2.0,system','v12.0,system','v12.0,system,serverless','v12.0,user,datawarehouse,gen2,analytics'))))\r\n| where not((type =~ ('microsoft.sql/servers')) and ((kind =~ ('v12.0,analytics'))))\r\n| project name,sku,totalCores,coreType,status,statusIcon,resources,id,resourceGroup,subscriptionDisplayName,type,kind,location,subscriptionId,tags\r\n| sort by (tolower(tostring(name))) asc", + "size": 0, + "title": "Eligible licenses", + "noDataMessage": "There are no eligible licenses.", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "eligibleLicenses", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "arclicenses" + } + ] + }, + "conditionalVisibility": { + "parameterName": "eligibleLicenses", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "group-eligibleLicenses" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "windowsEsu" + }, + "name": "group-Esu" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "title": "Windows Server Management enabled by Azure Arc", + "items": [ + { + "type": 1, + "content": { + "json": "## Benefits of Windows Server Management with Azure Arc\r\n\r\nWindows Server Management enabled by Azure Arc offers customers with Windows Server licenses that have active Software Assurances or Windows Server licenses that are active subscription licenses a few benefits, including access to Azure Update Management, Azure Change Tracking and Inventory and Windwos best practices assessment. FOr the full list of benefits, visit [Windows Server Management enabled by Azure Arc.](https://learn.microsoft.com/azure/azure-arc/servers/windows-server-management-overview?tabs=portal)\r\n\r\n", + "style": "info" + }, + "name": "text - 5" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "// Run query to see results.\r\nresources\r\n| where type =~ \"microsoft.hybridcompute/machines\" and isempty(kind)\r\n| extend status = properties.status\r\n| extend operatingSystem = properties.osSku\r\n| where properties.osType =~ 'windows'\r\n| extend licenseProfile = properties.licenseProfile\r\n| extend licenseStatus = tostring(licenseProfile.licenseStatus)\r\n| extend licenseChannel = tostring(licenseProfile.licenseChannel)\r\n| extend productSubscriptionStatus = tostring(licenseProfile.productProfile.subscriptionStatus)\r\n| extend softwareAssurance = licenseProfile.softwareAssurance\r\n| extend softwareAssuranceCustomer = licenseProfile.softwareAssurance.softwareAssuranceCustomer\r\n| extend benefitsStatus = case(\r\n softwareAssuranceCustomer == true, \"Activated\",\r\n (licenseStatus =~ \"Licensed\" and licenseChannel =~ \"PGS:TB\") or productSubscriptionStatus =~ \"Enabled\", \"Activated via Pay-as-you-go\",\r\n isnull(softwareAssurance) or isnull(softwareAssuranceCustomer) or softwareAssuranceCustomer == false, \"Not activated\",\r\n \"Not activated\")\r\n| extend benefitsStatusIcon = case(\r\n softwareAssuranceCustomer == true, \"8\",\r\n softwareAssuranceCustomer == true, \"8\",\r\n (licenseStatus =~ \"Licensed\" and licenseChannel =~ \"PGS:TB\") or productSubscriptionStatus =~ \"Enabled\", \"8\",\r\n isnull(softwareAssurance) or isnull(softwareAssuranceCustomer) or softwareAssuranceCustomer == false, \"7\",\r\n \"7\")\r\n| project name, status, benefitsStatus, benefitsStatusIcon, resourceGroup, subscriptionId, operatingSystem, id, type, location, kind, tags\r\n| where (type !~ ('dell.storage/filesystems'))\r\n| where (type !~ ('arizeai.observabilityeval/organizations'))\r\n| where (type !~ ('lambdatest.hyperexecute/organizations'))\r\n| where (type !~ ('pinecone.vectordb/organizations'))\r\n| where (type !~ ('microsoft.weightsandbiases/instances'))\r\n| where (type !~ ('purestorage.block/storagepools/avsstoragecontainers'))\r\n| where (type !~ ('purestorage.block/reservations'))\r\n| where (type !~ ('purestorage.block/storagepools'))\r\n| where (type !~ ('solarwinds.observability/organizations'))\r\n| where (type !~ ('splitio.experimentation/experimentationworkspaces'))\r\n| where (type !~ ('microsoft.agfoodplatform/farmbeats'))\r\n| where (type !~ ('microsoft.appsecurity/policies'))\r\n| where (type !~ ('microsoft.arc/all'))\r\n| where (type !~ ('microsoft.arc/allfairfax'))\r\n| where (type !~ ('microsoft.cdn/profiles/customdomains'))\r\n| where (type !~ ('microsoft.cdn/profiles/afdendpoints'))\r\n| where (type !~ ('microsoft.cdn/profiles/origingroups/origins'))\r\n| where (type !~ ('microsoft.cdn/profiles/origingroups'))\r\n| where (type !~ ('microsoft.cdn/profiles/afdendpoints/routes'))\r\n| where (type !~ ('microsoft.cdn/profiles/rulesets/rules'))\r\n| where (type !~ ('microsoft.cdn/profiles/rulesets'))\r\n| where (type !~ ('microsoft.cdn/profiles/secrets'))\r\n| where (type !~ ('microsoft.cdn/profiles/securitypolicies'))\r\n| where (type !~ ('microsoft.chaos/privateaccesses'))\r\n| where (type !~ ('microsoft.sovereign/transparencylogs'))\r\n| where (type !~ ('microsoft.sovereign/landingzoneconfigurations'))\r\n| where (type !~ ('microsoft.hardwaresecuritymodules/cloudhsmclusters'))\r\n| where (type !~ ('microsoft.compute/computefleetinstances'))\r\n| where (type !~ ('microsoft.compute/standbypoolinstance'))\r\n| where (type !~ ('microsoft.compute/virtualmachineflexinstances'))\r\n| where (type !~ ('microsoft.kubernetesconfiguration/extensions'))\r\n| where (type !~ ('microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/extensions'))\r\n| where (type !~ ('microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/namespaces'))\r\n| where (type !~ ('microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/namespaces'))\r\n| where (type !~ ('microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/fluxconfigurations'))\r\n| where (type !~ ('microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/fluxconfigurations'))\r\n| where (type !~ ('microsoft.portalservices/extensions/deployments'))\r\n| where (type !~ ('microsoft.portalservices/extensions'))\r\n| where (type !~ ('microsoft.portalservices/extensions/slots'))\r\n| where (type !~ ('microsoft.portalservices/extensions/versions'))\r\n| where (type !~ ('microsoft.datacollaboration/workspaces'))\r\n| where (type !~ ('microsoft.deviceregistry/devices'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/activedeployments'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/agents'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/deployments'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/deviceclasses'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/updates'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts'))\r\n| where (type !~ ('private.devtunnels/tunnelplans'))\r\n| where (type !~ ('private.easm/workspaces'))\r\n| where (type !~ ('microsoft.edgeorder/virtual_orderitems'))\r\n| where (type !~ ('microsoft.workloads/epicvirtualinstances'))\r\n| where (type !~ ('microsoft.fairfieldgardens/provisioningresources/provisioningpolicies'))\r\n| where (type !~ ('microsoft.fairfieldgardens/provisioningresources'))\r\n| where (type !~ ('microsoft.healthmodel/healthmodels'))\r\n| where (type !~ ('microsoft.hybridcompute/arcserverwithwac'))\r\n| where (type !~ ('microsoft.hybridcompute/machinessovereign'))\r\n| where (type !~ ('microsoft.hybridcompute/machinesesu'))\r\n| where (type !~ ('microsoft.hybridcompute/machinespaygo'))\r\n| where (type !~ ('microsoft.network/virtualhubs')) or ((kind =~ ('routeserver')))\r\n| where (type !~ ('microsoft.network/networkvirtualappliances'))\r\n| where (type !~ ('microsoft.devhub/iacprofiles'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/files'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/filerequests'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/licenses'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/connectors'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/sharedstorages'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/storages'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/workloads'))\r\n| where (type !~ ('microsoft.insights/diagnosticsettings'))\r\n| where not((type =~ ('microsoft.network/serviceendpointpolicies')) and ((kind =~ ('internal'))))\r\n| where (type !~ ('microsoft.resources/resourcegraphvisualizer'))\r\n| where (type !~ ('microsoft.iotoperationsmq/mq'))\r\n| where (type !~ ('microsoft.orbital/cloudaccessrouters'))\r\n| where (type !~ ('microsoft.orbital/terminals'))\r\n| where (type !~ ('microsoft.orbital/sdwancontrollers'))\r\n| where (type !~ ('microsoft.orbital/geocatalogs'))\r\n| where (type !~ ('microsoft.recommendationsservice/accounts/modeling'))\r\n| where (type !~ ('microsoft.recommendationsservice/accounts/serviceendpoints'))\r\n| where (type !~ ('microsoft.recoveryservicesbvtd/vaults'))\r\n| where (type !~ ('microsoft.recoveryservicesbvtd2/vaults'))\r\n| where (type !~ ('microsoft.recoveryservicesintd/vaults'))\r\n| where (type !~ ('microsoft.recoveryservicesintd2/vaults'))\r\n| where (type !~ ('microsoft.resources/deletedresources'))\r\n| where (type !~ ('microsoft.deploymentmanager/rollouts'))\r\n| where (type !~ ('microsoft.features/featureprovidernamespaces/featureconfigurations'))\r\n| where (type !~ ('microsoft.saashub/cloudservices/hidden'))\r\n| where (type !~ ('microsoft.providerhub/providerregistrations'))\r\n| where (type !~ ('microsoft.providerhub/providerregistrations/customrollouts'))\r\n| where (type !~ ('microsoft.providerhub/providerregistrations/defaultrollouts'))\r\n| where (type !~ ('microsoft.edge/configurations'))\r\n| where not((type =~ ('microsoft.synapse/workspaces/sqlpools')) and ((kind =~ ('v3'))))\r\n| where (type !~ ('microsoft.mission/approvals'))\r\n| where (type !~ ('microsoft.mission/catalogs'))\r\n| where (type !~ ('microsoft.mission/communities'))\r\n| where (type !~ ('microsoft.mission/communities/communityendpoints'))\r\n| where (type !~ ('microsoft.mission/enclaveconnections'))\r\n| where (type !~ ('microsoft.mission/virtualenclaves/enclaveendpoints'))\r\n| where (type !~ ('microsoft.mission/communities/transithubs'))\r\n| where (type !~ ('microsoft.mission/virtualenclaves'))\r\n| where (type !~ ('microsoft.mission/virtualenclaves/workloads'))\r\n| where (type !~ ('microsoft.workloads/insights'))\r\n| where (type !~ ('microsoft.hanaonazure/sapmonitors'))\r\n| where (type !~ ('microsoft.cloudhealth/healthmodels'))\r\n| where (type !~ ('microsoft.connectedcache/enterprisemcccustomers/enterprisemcccachenodes'))\r\n| where not((type =~ ('microsoft.sql/servers/databases')) and ((kind in~ ('system','v2.0,system','v12.0,system','v12.0,system,serverless','v12.0,user,datawarehouse,gen2,analytics'))))\r\n| where not((type =~ ('microsoft.sql/servers')) and ((kind =~ ('v12.0,analytics'))))\r\n| where (type in~ ('Microsoft.HybridCompute/machinesSoftwareAssurance','Microsoft.HybridCompute/machines'))\r\n| project name,benefitsStatus,benefitsStatusIcon,status,resourceGroup,operatingSystem,id,type,kind,location,subscriptionId\r\n| summarize count() by benefitsStatus", + "size": 0, + "title": "Coverage Summary", + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart", + "chartSettings": { + "yAxis": [ + "count_" + ], + "showLegend": true, + "seriesLabelSettings": [ + { + "seriesName": "Not activated", + "color": "red" + }, + { + "seriesName": "Activated", + "color": "green" + }, + { + "seriesName": "Activated via Pay-as-you-go", + "color": "greenDarkDark" + } + ] + } + }, + "name": "coverageSummary" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "16791a28-f78e-4b26-a2e9-a9fbbda915df", + "version": "KqlParameterItem/1.0", + "name": "eligibleResources", + "label": "View eligible resources?", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n", + "timeContext": { + "durationMs": 86400000 + }, + "value": "Yes" + } + ], + "style": "pills", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "parameters - 5" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "title": "Eligible Resources", + "items": [ + { + "type": 1, + "content": { + "json": "To enable Windows Server Management with Azure Arc, visit the [Arc service blade.](https://ms.portal.azure.com/#view/Microsoft_Azure_ArcCenterUX/ArcCenterMenuBlade/~/softwareAssurance)\r\n\r\n", + "style": "info" + }, + "name": "txtEligibleAHBResources" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "// Run query to see results.\r\nresources\r\n| where type =~ \"microsoft.hybridcompute/machines\" and isempty(kind)\r\n| extend status = properties.status\r\n| extend operatingSystem = properties.osSku\r\n| where properties.osType =~ 'windows'\r\n| extend licenseProfile = properties.licenseProfile\r\n| extend licenseStatus = tostring(licenseProfile.licenseStatus)\r\n| extend licenseChannel = tostring(licenseProfile.licenseChannel)\r\n| extend productSubscriptionStatus = tostring(licenseProfile.productProfile.subscriptionStatus)\r\n| extend softwareAssurance = licenseProfile.softwareAssurance\r\n| extend softwareAssuranceCustomer = licenseProfile.softwareAssurance.softwareAssuranceCustomer\r\n| extend benefitsStatus = case(\r\n softwareAssuranceCustomer == true, \"Activated\",\r\n (licenseStatus =~ \"Licensed\" and licenseChannel =~ \"PGS:TB\") or productSubscriptionStatus =~ \"Enabled\", \"Activated via Pay-as-you-go\",\r\n isnull(softwareAssurance) or isnull(softwareAssuranceCustomer) or softwareAssuranceCustomer == false, \"Not activated\",\r\n \"Not activated\")\r\n| extend benefitsStatusIcon = case(\r\n softwareAssuranceCustomer == true, \"8\",\r\n softwareAssuranceCustomer == true, \"8\",\r\n (licenseStatus =~ \"Licensed\" and licenseChannel =~ \"PGS:TB\") or productSubscriptionStatus =~ \"Enabled\", \"8\",\r\n isnull(softwareAssurance) or isnull(softwareAssuranceCustomer) or softwareAssuranceCustomer == false, \"7\",\r\n \"7\")\r\n| project name, status, benefitsStatus, benefitsStatusIcon, resourceGroup, subscriptionId, operatingSystem, id, type, location, kind, tags\r\n| where (type !~ ('dell.storage/filesystems'))\r\n| where (type !~ ('arizeai.observabilityeval/organizations'))\r\n| where (type !~ ('lambdatest.hyperexecute/organizations'))\r\n| where (type !~ ('pinecone.vectordb/organizations'))\r\n| where (type !~ ('microsoft.weightsandbiases/instances'))\r\n| where (type !~ ('purestorage.block/storagepools/avsstoragecontainers'))\r\n| where (type !~ ('purestorage.block/reservations'))\r\n| where (type !~ ('purestorage.block/storagepools'))\r\n| where (type !~ ('solarwinds.observability/organizations'))\r\n| where (type !~ ('splitio.experimentation/experimentationworkspaces'))\r\n| where (type !~ ('microsoft.agfoodplatform/farmbeats'))\r\n| where (type !~ ('microsoft.appsecurity/policies'))\r\n| where (type !~ ('microsoft.arc/all'))\r\n| where (type !~ ('microsoft.arc/allfairfax'))\r\n| where (type !~ ('microsoft.cdn/profiles/customdomains'))\r\n| where (type !~ ('microsoft.cdn/profiles/afdendpoints'))\r\n| where (type !~ ('microsoft.cdn/profiles/origingroups/origins'))\r\n| where (type !~ ('microsoft.cdn/profiles/origingroups'))\r\n| where (type !~ ('microsoft.cdn/profiles/afdendpoints/routes'))\r\n| where (type !~ ('microsoft.cdn/profiles/rulesets/rules'))\r\n| where (type !~ ('microsoft.cdn/profiles/rulesets'))\r\n| where (type !~ ('microsoft.cdn/profiles/secrets'))\r\n| where (type !~ ('microsoft.cdn/profiles/securitypolicies'))\r\n| where (type !~ ('microsoft.chaos/privateaccesses'))\r\n| where (type !~ ('microsoft.sovereign/transparencylogs'))\r\n| where (type !~ ('microsoft.sovereign/landingzoneconfigurations'))\r\n| where (type !~ ('microsoft.hardwaresecuritymodules/cloudhsmclusters'))\r\n| where (type !~ ('microsoft.compute/computefleetinstances'))\r\n| where (type !~ ('microsoft.compute/standbypoolinstance'))\r\n| where (type !~ ('microsoft.compute/virtualmachineflexinstances'))\r\n| where (type !~ ('microsoft.kubernetesconfiguration/extensions'))\r\n| where (type !~ ('microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/extensions'))\r\n| where (type !~ ('microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/namespaces'))\r\n| where (type !~ ('microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/namespaces'))\r\n| where (type !~ ('microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/fluxconfigurations'))\r\n| where (type !~ ('microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/fluxconfigurations'))\r\n| where (type !~ ('microsoft.portalservices/extensions/deployments'))\r\n| where (type !~ ('microsoft.portalservices/extensions'))\r\n| where (type !~ ('microsoft.portalservices/extensions/slots'))\r\n| where (type !~ ('microsoft.portalservices/extensions/versions'))\r\n| where (type !~ ('microsoft.datacollaboration/workspaces'))\r\n| where (type !~ ('microsoft.deviceregistry/devices'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/activedeployments'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/agents'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/deployments'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/deviceclasses'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/updates'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts'))\r\n| where (type !~ ('private.devtunnels/tunnelplans'))\r\n| where (type !~ ('private.easm/workspaces'))\r\n| where (type !~ ('microsoft.edgeorder/virtual_orderitems'))\r\n| where (type !~ ('microsoft.workloads/epicvirtualinstances'))\r\n| where (type !~ ('microsoft.fairfieldgardens/provisioningresources/provisioningpolicies'))\r\n| where (type !~ ('microsoft.fairfieldgardens/provisioningresources'))\r\n| where (type !~ ('microsoft.healthmodel/healthmodels'))\r\n| where (type !~ ('microsoft.hybridcompute/arcserverwithwac'))\r\n| where (type !~ ('microsoft.hybridcompute/machinessovereign'))\r\n| where (type !~ ('microsoft.hybridcompute/machinesesu'))\r\n| where (type !~ ('microsoft.hybridcompute/machinespaygo'))\r\n| where (type !~ ('microsoft.network/virtualhubs')) or ((kind =~ ('routeserver')))\r\n| where (type !~ ('microsoft.network/networkvirtualappliances'))\r\n| where (type !~ ('microsoft.devhub/iacprofiles'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/files'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/filerequests'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/licenses'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/connectors'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/sharedstorages'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/storages'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/workloads'))\r\n| where (type !~ ('microsoft.insights/diagnosticsettings'))\r\n| where not((type =~ ('microsoft.network/serviceendpointpolicies')) and ((kind =~ ('internal'))))\r\n| where (type !~ ('microsoft.resources/resourcegraphvisualizer'))\r\n| where (type !~ ('microsoft.iotoperationsmq/mq'))\r\n| where (type !~ ('microsoft.orbital/cloudaccessrouters'))\r\n| where (type !~ ('microsoft.orbital/terminals'))\r\n| where (type !~ ('microsoft.orbital/sdwancontrollers'))\r\n| where (type !~ ('microsoft.orbital/geocatalogs'))\r\n| where (type !~ ('microsoft.recommendationsservice/accounts/modeling'))\r\n| where (type !~ ('microsoft.recommendationsservice/accounts/serviceendpoints'))\r\n| where (type !~ ('microsoft.recoveryservicesbvtd/vaults'))\r\n| where (type !~ ('microsoft.recoveryservicesbvtd2/vaults'))\r\n| where (type !~ ('microsoft.recoveryservicesintd/vaults'))\r\n| where (type !~ ('microsoft.recoveryservicesintd2/vaults'))\r\n| where (type !~ ('microsoft.resources/deletedresources'))\r\n| where (type !~ ('microsoft.deploymentmanager/rollouts'))\r\n| where (type !~ ('microsoft.features/featureprovidernamespaces/featureconfigurations'))\r\n| where (type !~ ('microsoft.saashub/cloudservices/hidden'))\r\n| where (type !~ ('microsoft.providerhub/providerregistrations'))\r\n| where (type !~ ('microsoft.providerhub/providerregistrations/customrollouts'))\r\n| where (type !~ ('microsoft.providerhub/providerregistrations/defaultrollouts'))\r\n| where (type !~ ('microsoft.edge/configurations'))\r\n| where not((type =~ ('microsoft.synapse/workspaces/sqlpools')) and ((kind =~ ('v3'))))\r\n| where (type !~ ('microsoft.mission/approvals'))\r\n| where (type !~ ('microsoft.mission/catalogs'))\r\n| where (type !~ ('microsoft.mission/communities'))\r\n| where (type !~ ('microsoft.mission/communities/communityendpoints'))\r\n| where (type !~ ('microsoft.mission/enclaveconnections'))\r\n| where (type !~ ('microsoft.mission/virtualenclaves/enclaveendpoints'))\r\n| where (type !~ ('microsoft.mission/communities/transithubs'))\r\n| where (type !~ ('microsoft.mission/virtualenclaves'))\r\n| where (type !~ ('microsoft.mission/virtualenclaves/workloads'))\r\n| where (type !~ ('microsoft.workloads/insights'))\r\n| where (type !~ ('microsoft.hanaonazure/sapmonitors'))\r\n| where (type !~ ('microsoft.cloudhealth/healthmodels'))\r\n| where (type !~ ('microsoft.connectedcache/enterprisemcccustomers/enterprisemcccachenodes'))\r\n| where not((type =~ ('microsoft.sql/servers/databases')) and ((kind in~ ('system','v2.0,system','v12.0,system','v12.0,system,serverless','v12.0,user,datawarehouse,gen2,analytics'))))\r\n| where not((type =~ ('microsoft.sql/servers')) and ((kind =~ ('v12.0,analytics'))))\r\n| where (type in~ ('Microsoft.HybridCompute/machinesSoftwareAssurance','Microsoft.HybridCompute/machines'))\r\n| project name,benefitsStatus,benefitsStatusIcon,status,resourceGroup,operatingSystem,id,type,kind,location,subscriptionId\r\n| sort by (tolower(tostring(name))) asc", + "size": 0, + "title": "Eligible resources", + "noDataMessage": "There are no eligible resources.", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "benefitsStatusIcon", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "7", + "representation": "3", + "text": "" + }, + { + "operator": "==", + "thresholdValue": "8", + "representation": "success", + "text": "" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "" + } + ] + } + }, + { + "columnMatch": "esuStatusIcon", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "7", + "representation": "4", + "text": "" + }, + { + "operator": "==", + "thresholdValue": "9", + "representation": "warning", + "text": "" + }, + { + "operator": "==", + "thresholdValue": "8", + "representation": "success", + "text": "" + }, + { + "operator": "==", + "thresholdValue": "91", + "representation": "unknown", + "text": "{0}{1}" + }, + { + "sourceColumn": "esuStatus", + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "" + } + ] + } + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "eligibleResources", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "arcAHBResources" + } + ] + }, + "conditionalVisibility": { + "parameterName": "eligibleResources", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "group-eligibleResources" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "title": "Eligible licenses", + "items": [ + { + "type": 1, + "content": { + "json": "Keep in mind, subscription read permissions are required to give accurate core count and usage information for your ESU licenses. Physical core based licenses are subject to different licensing guidelines and are excluded from this view.[Learn more.](https://learn.microsoft.com/azure/azure-arc/servers/license-extended-security-updates)", + "style": "info" + }, + "name": "txtLearnMoreLicense" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "// Run query to see results.\r\nresources\r\n| where type =~ \"microsoft.hybridcompute/licenses\"\r\n| extend sku = properties.licenseDetails.edition\r\n| extend totalCores = properties.licenseDetails.processors\r\n| extend coreType = case(\r\n properties.licenseDetails.type =~ 'vCore','Virtual core',\r\n properties.licenseDetails.type =~ 'pCore','Physical core',\r\n 'Unknown'\r\n)\r\n| extend statusIcon = case(\r\n properties.licenseDetails.state =~ 'Activated', '8',\r\n properties.licenseDetails.state =~ 'Deactivated', '7',\r\n '91'\r\n)\r\n| extend status = case(\r\n properties.licenseDetails.state =~ 'Activated', 'Activated',\r\n properties.licenseDetails.state =~ 'Deactivated','Deactivated',\r\n 'Unknown'\r\n)\r\n| extend licenseId = tolower(tostring(id)) // Depending on what is stored in license profile, might have to get the immutableId instead\r\n| join kind=leftouter(\r\n resources\r\n | where type =~ \"microsoft.hybridcompute/machines/licenseProfiles\"\r\n | extend machineId = tolower(tostring(trim_end(@\"\\/\\w+\\/(\\w|\\.)+\", id)))\r\n | extend licenseId = tolower(tostring(properties.esuProfile.assignedLicense))\r\n | summarize resources = count() by licenseId\r\n) on licenseId // Get count of license profile per license, a license profile is created for each machine that is assigned a license\r\n| extend resources = iff(isnull(resources), 0, resources)\r\n| project id, name, type, location, subscriptionId, resourceGroup, kind, tags, extendedLocation, sku, totalCores, coreType, status, statusIcon, resources\r\n| extend subscriptionDisplayName=case(subscriptionId =~ 'e75c95f3-27b4-410f-a40e-2b9153a807dd','AEther Dev',subscriptionId =~ '823ca539-d44d-43ee-8dc8-023fd4f27396','AIOps_FailureSimulation_DevTest',subscriptionId =~ 'b2a328a7-ffff-4c09-b643-a4758cf170bc','AISC-DEV-02',subscriptionId =~ 'eef8b6d5-94da-4b36-9327-a662f2674efb','AISC-EngSys-01',subscriptionId =~ 'e0fd569c-e34a-4249-8c24-e8d723c7f054','AML - Responsible AI R&D',subscriptionId =~ 'f1d79e73-f8e3-4b10-bfdb-4207ca0723ed','AML Infra - Engineering System',subscriptionId =~ 'dad45786-32e5-4ef3-b90e-8e0838fbadb6','AnE.ExP.NonProduction',subscriptionId =~ '017ad81c-b730-4564-9109-9b763243a9a6','Anomaly Real Time-DevTest',subscriptionId =~ 'e686ef8c-d35d-4e9b-92f8-caaaa7948c0a','Applied Sciences Group Dev',subscriptionId =~ '6e967edb-425b-4a33-ae98-f1d2c509dda3','Arc enabled Infrastructure - Demo',subscriptionId =~ '710c6ed8-d8d9-4c51-b100-dc4d9e10dd18','ARC-ESU-TESTING-03',subscriptionId =~ '2145a411-d149-4010-84d4-40fe8a55db44','ARM Test Environment',subscriptionId =~ 'e2a8a7b3-cb7a-4930-9c79-2b0eb7bf843c','ASZ_Lab_Hardware_Dev_1',subscriptionId =~ '4c33a6bb-d566-4d0b-8951-00b55a91fba8','AzPIESecurity-DEV-Apple',subscriptionId =~ 'b65b516b-415b-4c68-a254-bfa7411275f8','Azure CloudES CDP ER - Test',subscriptionId =~ '6785ea1f-ac40-4244-a9ce-94b12fd832ca','Azure Migrate Demo Subscription',subscriptionId =~ '8c3c936a-c09b-4de3-830b-3f5f244d72e9','Azure Migrate Program Management Team',subscriptionId =~ '432a7068-99ae-4975-ad38-d96b71172cdf','Azure Profiler - Testing',subscriptionId =~ 'caffe3c0-acbd-4d01-af76-a45f421bfb64','Azure_Base_LivesiteArmory_test',subscriptionId =~ 'a1920ebd-59b7-4f19-af9f-5e80599e88e4','Babel',subscriptionId =~ 'b43a6159-1bea-4fa2-9407-e875fdc0ff55','BAP IA EU Schrems (P)',subscriptionId =~ 'ae71ef11-a03f-4b4f-a0e6-ef144727c711','Bing MM Measurement',subscriptionId =~ '8ef80208-601f-4c83-802e-751d211745f9','CloudAnalytics_Prod1',subscriptionId =~ 'f7b4a5ce-d3bd-4a03-84fe-8bcad772c8c1','Cog Services Computer Vision',subscriptionId =~ '38dbd059-bd26-4172-af17-bcc5a00744de','Core-ES-STREAM-DeXDI',subscriptionId =~ '87e8a61c-4832-4cfe-8b72-c12554f2309a','Cosmos_C&E_Azure_Azure Resource Manager_100424',subscriptionId =~ '8c4b5b03-3b24-4ed0-91f5-a703cd91b412','Cosmos_C&E_Azure_AzureEngineeringSystems_100200',subscriptionId =~ 'b988efc7-1bc8-4a73-b4a2-5ad864580250','Cosmos_C&E_DPG_Big Data_100036',subscriptionId =~ '6121bf73-2dc9-47ac-8b6a-c6ae4b88ef39','Cost Management PM',subscriptionId =~ '44074499-ad9f-45b9-9b0b-f594f8736a45','CSX-WSD-CFE-TNT-QPP',subscriptionId =~ '3cd01953-be46-4f9b-a4e0-26f19f428765','CSX-WSD-CFE-TNT-TOOLKITHUB',subscriptionId =~ '60214a3c-65a2-46f8-8b32-f959f2454106','Data Pillar Security Tooling',subscriptionId =~ 'f7c445af-a4de-4264-9e87-3196d6bc384d','Efficiency Pack Services (MSIT)',subscriptionId =~ '86f010b8-6473-4cb9-90c1-671c74faa4ee','emilyzhu dev subscription',subscriptionId =~ 'f3326bfd-5406-4136-a835-f64ccbfd9050','FTK @ MS',subscriptionId =~ 'd2c9544f-4329-4642-b73d-020e7fef844f','HPCScrub1',subscriptionId =~ 'bac420ed-c6fc-4a05-8ac1-8c0c52da1d6e','IDEAs MS Reporting',subscriptionId =~ '3cd95ff9-ac62-4b5c-8240-0cd046687ea0','Internal App Insights Resources for Perflens',subscriptionId =~ 'a386d5ea-ea90-441a-8263-d816368c84a1','IOT_PLATFORM_UPX_TEST',subscriptionId =~ '6a0ab98c-24d4-4131-8cd7-0a78bfce58f5','ITSM Integration - Production - New',subscriptionId =~ '13723929-6644-4060-a50a-cc38ebc5e8b1','LinuxMdsd Test',subscriptionId =~ '98b02a69-28a5-4ee7-a622-cd69c7a59c4e','Marvel Intelligence Model Training Subscription',subscriptionId =~ '1278a874-89fc-418c-b6b9-ac763b000415','Microsoft Azure Internal Consumption (nbrady)',subscriptionId =~ '29de2cfc-f00a-43bb-bdc8-3108795bd282','MSFT-Modern Device-Modern Mgmt-Imaging00',subscriptionId =~ '54b875cc-a81a-4914-8bfd-1a36bc7ddf4d','MSFT-WindowsVirtualDesktop-01',subscriptionId =~ '4dc2cd39-7a89-43d8-bebe-8bb501359891','Observability_AzMon_Grafana_Dev',subscriptionId =~ 'a471d615-ff98-4e80-b375-a19543d4691e','ODC-OneBox-Resources',subscriptionId =~ '1163fbbe-27e7-4b0f-8466-195fe5417043','Python_AI_Tools_PM_Team_Sub (jbinder)',subscriptionId =~ '3905431d-c062-4c17-8fd9-c51f89f334c4','Pytorch Build',subscriptionId =~ '3f577935-3138-4d07-86b3-75651b696483','Responsible & OpenAi Research',subscriptionId =~ '0f301386-8979-4981-acca-973d553078e7','Scope Team Test Subscription',subscriptionId =~ '1b0a5c20-7373-41a2-8fec-7364cceb7bbf','Services Hub Demo Open',subscriptionId =~ 'e72e5254-f265-4e95-9bd2-9ee8e7329051','Speech Services - DEV - SDK (carbon)',subscriptionId =~ 'a1c3dc6b-8630-4bb7-a29e-4ed4407c329b','Speech Services - EXP2',subscriptionId =~ '736af2bf-9fcb-4145-a19b-5b30b2b8949d','Trey International UK',subscriptionId =~ '51f73f67-1f29-4120-863e-dd315f743fc1','Trey Partner Lab 2',subscriptionId =~ '9ec51cfd-5ca7-4d76-8101-dd0a4abc5674','Trey Research Corporate',subscriptionId =~ '73c0021f-a37d-433f-8baa-7450cb54eea6','Trey Research Finance',subscriptionId =~ 'ed570627-0265-4620-bb42-bae06bcfa914','Trey Research IT',subscriptionId =~ '64e355d7-997c-491d-b0c1-8414dccfcf42','Trey Research R&D Playground',subscriptionId =~ '586f1d47-9dd9-43d5-b196-6e28f8405ff8','Trey Research R&D Production',subscriptionId =~ '9ec1d932-0f3f-486c-acc6-e7d78b358f9b','TScience',subscriptionId =~ '79f57c16-00fe-48da-87d4-5192e86cd047','TScienceGPU',subscriptionId =~ 'bac044cf-49e1-4843-8dda-1ce9662606c8','UI Fabric',subscriptionId =~ '6760347d-9ffe-41a9-ba11-c139dcea5ce6','xt-teams-migration-dev',subscriptionId =~ '45f9252d-e27e-4ed8-ab4e-dc5054de13fa','Contoso IT - Retail - Prod',subscriptionId)\r\n| where (type !~ ('dell.storage/filesystems'))\r\n| where (type !~ ('arizeai.observabilityeval/organizations'))\r\n| where (type !~ ('lambdatest.hyperexecute/organizations'))\r\n| where (type !~ ('pinecone.vectordb/organizations'))\r\n| where (type !~ ('microsoft.weightsandbiases/instances'))\r\n| where (type !~ ('purestorage.block/storagepools/avsstoragecontainers'))\r\n| where (type !~ ('purestorage.block/reservations'))\r\n| where (type !~ ('purestorage.block/storagepools'))\r\n| where (type !~ ('solarwinds.observability/organizations'))\r\n| where (type !~ ('splitio.experimentation/experimentationworkspaces'))\r\n| where (type !~ ('microsoft.agfoodplatform/farmbeats'))\r\n| where (type !~ ('microsoft.appsecurity/policies'))\r\n| where (type !~ ('microsoft.arc/all'))\r\n| where (type !~ ('microsoft.arc/allfairfax'))\r\n| where (type !~ ('microsoft.cdn/profiles/customdomains'))\r\n| where (type !~ ('microsoft.cdn/profiles/afdendpoints'))\r\n| where (type !~ ('microsoft.cdn/profiles/origingroups/origins'))\r\n| where (type !~ ('microsoft.cdn/profiles/origingroups'))\r\n| where (type !~ ('microsoft.cdn/profiles/afdendpoints/routes'))\r\n| where (type !~ ('microsoft.cdn/profiles/rulesets/rules'))\r\n| where (type !~ ('microsoft.cdn/profiles/rulesets'))\r\n| where (type !~ ('microsoft.cdn/profiles/secrets'))\r\n| where (type !~ ('microsoft.cdn/profiles/securitypolicies'))\r\n| where (type !~ ('microsoft.chaos/privateaccesses'))\r\n| where (type !~ ('microsoft.sovereign/transparencylogs'))\r\n| where (type !~ ('microsoft.sovereign/landingzoneconfigurations'))\r\n| where (type !~ ('microsoft.hardwaresecuritymodules/cloudhsmclusters'))\r\n| where (type !~ ('microsoft.compute/computefleetinstances'))\r\n| where (type !~ ('microsoft.compute/standbypoolinstance'))\r\n| where (type !~ ('microsoft.compute/virtualmachineflexinstances'))\r\n| where (type !~ ('microsoft.kubernetesconfiguration/extensions'))\r\n| where (type !~ ('microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/extensions'))\r\n| where (type !~ ('microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/namespaces'))\r\n| where (type !~ ('microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/namespaces'))\r\n| where (type !~ ('microsoft.kubernetes/connectedclusters/microsoft.kubernetesconfiguration/fluxconfigurations'))\r\n| where (type !~ ('microsoft.containerservice/managedclusters/microsoft.kubernetesconfiguration/fluxconfigurations'))\r\n| where (type !~ ('microsoft.portalservices/extensions/deployments'))\r\n| where (type !~ ('microsoft.portalservices/extensions'))\r\n| where (type !~ ('microsoft.portalservices/extensions/slots'))\r\n| where (type !~ ('microsoft.portalservices/extensions/versions'))\r\n| where (type !~ ('microsoft.datacollaboration/workspaces'))\r\n| where (type !~ ('microsoft.deviceregistry/devices'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/activedeployments'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/agents'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/deployments'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/deviceclasses'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts/updates'))\r\n| where (type !~ ('microsoft.deviceupdate/updateaccounts'))\r\n| where (type !~ ('private.devtunnels/tunnelplans'))\r\n| where (type !~ ('private.easm/workspaces'))\r\n| where (type !~ ('microsoft.edgeorder/virtual_orderitems'))\r\n| where (type !~ ('microsoft.workloads/epicvirtualinstances'))\r\n| where (type !~ ('microsoft.fairfieldgardens/provisioningresources/provisioningpolicies'))\r\n| where (type !~ ('microsoft.fairfieldgardens/provisioningresources'))\r\n| where (type !~ ('microsoft.healthmodel/healthmodels'))\r\n| where (type !~ ('microsoft.hybridcompute/arcserverwithwac'))\r\n| where (type !~ ('microsoft.hybridcompute/machinessovereign'))\r\n| where (type !~ ('microsoft.hybridcompute/machinesesu'))\r\n| where (type !~ ('microsoft.hybridcompute/machinespaygo'))\r\n| where (type !~ ('microsoft.hybridcompute/machinessoftwareassurance'))\r\n| where (type !~ ('microsoft.network/virtualhubs')) or ((kind =~ ('routeserver')))\r\n| where (type !~ ('microsoft.network/networkvirtualappliances'))\r\n| where (type !~ ('microsoft.devhub/iacprofiles'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/files'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/filerequests'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/licenses'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/connectors'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/sharedstorages'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/storages'))\r\n| where (type !~ ('microsoft.modsimworkbench/workbenches/chambers/workloads'))\r\n| where (type !~ ('microsoft.insights/diagnosticsettings'))\r\n| where not((type =~ ('microsoft.network/serviceendpointpolicies')) and ((kind =~ ('internal'))))\r\n| where (type !~ ('microsoft.resources/resourcegraphvisualizer'))\r\n| where (type !~ ('microsoft.iotoperationsmq/mq'))\r\n| where (type !~ ('microsoft.orbital/cloudaccessrouters'))\r\n| where (type !~ ('microsoft.orbital/terminals'))\r\n| where (type !~ ('microsoft.orbital/sdwancontrollers'))\r\n| where (type !~ ('microsoft.orbital/geocatalogs'))\r\n| where (type !~ ('microsoft.recommendationsservice/accounts/modeling'))\r\n| where (type !~ ('microsoft.recommendationsservice/accounts/serviceendpoints'))\r\n| where (type !~ ('microsoft.recoveryservicesbvtd/vaults'))\r\n| where (type !~ ('microsoft.recoveryservicesbvtd2/vaults'))\r\n| where (type !~ ('microsoft.recoveryservicesintd/vaults'))\r\n| where (type !~ ('microsoft.recoveryservicesintd2/vaults'))\r\n| where (type !~ ('microsoft.resources/deletedresources'))\r\n| where (type !~ ('microsoft.deploymentmanager/rollouts'))\r\n| where (type !~ ('microsoft.features/featureprovidernamespaces/featureconfigurations'))\r\n| where (type !~ ('microsoft.saashub/cloudservices/hidden'))\r\n| where (type !~ ('microsoft.providerhub/providerregistrations'))\r\n| where (type !~ ('microsoft.providerhub/providerregistrations/customrollouts'))\r\n| where (type !~ ('microsoft.providerhub/providerregistrations/defaultrollouts'))\r\n| where (type !~ ('microsoft.edge/configurations'))\r\n| where not((type =~ ('microsoft.synapse/workspaces/sqlpools')) and ((kind =~ ('v3'))))\r\n| where (type !~ ('microsoft.mission/approvals'))\r\n| where (type !~ ('microsoft.mission/catalogs'))\r\n| where (type !~ ('microsoft.mission/communities'))\r\n| where (type !~ ('microsoft.mission/communities/communityendpoints'))\r\n| where (type !~ ('microsoft.mission/enclaveconnections'))\r\n| where (type !~ ('microsoft.mission/virtualenclaves/enclaveendpoints'))\r\n| where (type !~ ('microsoft.mission/communities/transithubs'))\r\n| where (type !~ ('microsoft.mission/virtualenclaves'))\r\n| where (type !~ ('microsoft.mission/virtualenclaves/workloads'))\r\n| where (type !~ ('microsoft.workloads/insights'))\r\n| where (type !~ ('microsoft.hanaonazure/sapmonitors'))\r\n| where (type !~ ('microsoft.cloudhealth/healthmodels'))\r\n| where (type !~ ('microsoft.connectedcache/enterprisemcccustomers/enterprisemcccachenodes'))\r\n| where not((type =~ ('microsoft.sql/servers/databases')) and ((kind in~ ('system','v2.0,system','v12.0,system','v12.0,system,serverless','v12.0,user,datawarehouse,gen2,analytics'))))\r\n| where not((type =~ ('microsoft.sql/servers')) and ((kind =~ ('v12.0,analytics'))))\r\n| project name,sku,totalCores,coreType,status,statusIcon,resources,id,resourceGroup,subscriptionDisplayName,type,kind,location,subscriptionId,tags\r\n| sort by (tolower(tostring(name))) asc", + "size": 0, + "title": "Eligible licenses", + "noDataMessage": "There are no eligible licenses.", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "eligibleLicenses", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "arclicenses" + } + ] + }, + "conditionalVisibility": { + "parameterName": "eligibleLicenses", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "group-eligibleLicenses" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "arcAHB" + }, + "name": "group-ahbArc" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "arc" + }, + "name": "Arc" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "title": "Virtual Machines", + "loadType": "always", + "items": [ + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "loadType": "always", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources \r\n| where resourceGroup in ({ResourceGroup})\r\n| where type =~ 'microsoft.compute/virtualmachines' or type =~ 'microsoft.compute/virtualMachineScaleSets'\r\n| where tostring(properties.storageProfile.imageReference.publisher ) == \"MicrosoftWindowsServer\" or tostring(properties.virtualMachineProfile.storageProfile.osDisk.osType) == 'Windows' or tostring(properties.storageProfile.imageReference.publisher ) == \"microsoftsqlserver\"\r\n| extend OS=properties.storageProfile.imageReference.id \r\n| where isnull(OS)\r\n| where tostring(properties.['licenseType']) !has 'Windows' and tostring(properties.virtualMachineProfile.['licenseType']) != 'Windows_Server'\r\n| extend WindowsId=id, VMIDFull=id, VMName=name, VMLocation=location, VMRG=resourceGroup, OSType=tostring(properties.storageProfile.imageReference.offer), OsVersion = tostring(properties.storageProfile.imageReference.sku), VMSize=tostring (properties.hardwareProfile.vmSize), LicenseType = tostring(properties.['licenseType']), VMSSize=tostring(sku.name), QuickFix=id\r\n ) on subscriptionId \r\n| order by type asc \r\n| project WindowsId,VMName,VMRG,VMSize, VMSSize, VMLocation,OSType, OsVersion,LicenseType, subscriptionId, QuickFix, VMIDFull\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), WindowsId=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct WindowsId\r\n )\r\n on WindowsId", + "size": 0, + "title": "AHB Disabled", + "noDataMessage": "All of your VMs have AHB enabled.", + "noDataMessageStyle": 3, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "AHB Disabled" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources \r\n| where resourceGroup in ({ResourceGroup})\r\n| where type =~ 'microsoft.compute/virtualmachines'\r\n| where tostring(properties.storageProfile.imageReference.publisher ) == \"MicrosoftWindowsServer\" or tostring(properties.virtualMachineProfile.storageProfile.osDisk.osType) == 'Windows' or tostring(properties.storageProfile.imageReference.publisher ) == \"microsoftsqlserver\"\r\n| extend OS=properties.storageProfile.imageReference.id \r\n| where isnull(OS)\r\n| where tostring(properties.['licenseType']) has \"Windows\"\r\n| extend WindowsId=id, VMName=name, VMLocation=location, VMRG=resourceGroup, OSType=tostring(properties.storageProfile.imageReference.offer), OsVersion = tostring(properties.storageProfile.imageReference.sku), VMSize=tostring (properties.hardwareProfile.vmSize), LicenseType = tostring(properties.['licenseType']), VMSSize=tostring(sku.name)\r\n) on subscriptionId \r\n| order by type asc \r\n| project WindowsId,VMName,VMRG,VMSize, VMSSize, VMLocation,OSType, OsVersion,LicenseType, subscriptionId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), WindowsId=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct WindowsId\r\n )\r\n on WindowsId", + "size": 0, + "title": "AHB Enabled", + "noDataMessage": "None of your VMs have AHB enabled.", + "noDataMessageStyle": 4, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "VMRG", + "formatter": 0, + "tooltipFormat": { + "tooltip": "test" + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "WindowsAHBEnabled" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resourcechanges\r\n| where properties.changeType == \"Update\" and properties.targetResourceType == \"microsoft.compute/virtualmachines\"\r\n| mv-expand changes = properties.changes\r\n| mv-expand LicenseChanges=changes.['properties.licenseType']\r\n| extend WindowsId=id\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), WindowsId=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct WindowsId\r\n )\r\n on WindowsId\r\n| where isnotnull(LicenseChanges)\r\n| where tostring(LicenseChanges.newValue) has \"Windows\"\r\n| project VMID=properties.targetResourceId, NewLicense=tostring(LicenseChanges.newValue), DateofChange=todatetime(properties.changeAttributes.timestamp)\r\n", + "size": 0, + "title": "VM Latest Change Last 7 days", + "noDataMessage": "AHB was not enabled in the last 7 days.", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "VM Latest Change Last 7 days" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"ARMEndpoint/1.0\",\"data\":null,\"headers\":[],\"method\":\"GET\",\"path\":\"/subscriptions/{SingleSubHidden}/providers/Microsoft.Compute/skus?$filter=location eq '{Location}'\",\"urlParams\":[{\"key\":\"api-version\",\"value\":\"2021-07-01\"}],\"batchDisabled\":false,\"transformers\":[{\"type\":\"jsonpath\",\"settings\":{\"tablePath\":\"$.*[?(@.resourceType=='virtualMachines')]\",\"columns\":[{\"path\":\"name\",\"columnid\":\"Name\"},{\"path\":\"capabilities[?(@.name=='vCPUs')].value\",\"columnid\":\"vCPUs\"},{\"path\":\"capabilities[?(@.name=='MemoryGB')].value\",\"columnid\":\"MemoryGB\"},{\"path\":\"capabilities[?(@.name=='MaxNetworkInterfaces')].value\",\"columnid\":\"MaxNetworkInterfaces\"},{\"path\":\"capabilities[?(@.name=='HyperVGenerations')].value\",\"columnid\":\"HyperVGenerations\"},{\"path\":\"capabilities[?(@.name=='vCPUsPerCore')].value\",\"columnid\":\"vCPUsPerCore\"}]}}]}", + "size": 0, + "title": "Get VM vCPU", + "exportParameterName": "ResourceSKU", + "showExportToExcel": true, + "queryType": 12, + "gridSettings": { + "rowLimit": 5000 + } + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "query - Get VM vCPU" + }, + { + "type": 1, + "content": { + "json": "## Windows Azure Hybrid Benefit (AHB) Overview" + }, + "name": "AHB Overview" + }, + { + "type": 1, + "content": { + "json": "Each two-processor license or each set of 16-core licenses, either Datacenter or Standard editions, are entitled to two instances of up to 8 cores, or one instance of up to 16 cores.\r\n\r\nThe virtual machines (VMs) with less than 8 cores are categorized as **Low Priority**, while those with 8 or more cores are classified as **High Priority**. In situations where there are insufficient Azure Hybrid benefit licenses to cover all the VMs in the environment, it is recommended to prioritize the High Priority VMs.", + "style": "info" + }, + "name": "NUmber of Processors", + "styleSettings": { + "margin": "10px", + "showBorder": true + } + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources \r\n| where resourceGroup in ({ResourceGroup})\r\n| where type =~ 'microsoft.compute/virtualmachines'\r\n| where tostring(properties.storageProfile.imageReference.publisher ) == \"MicrosoftWindowsServer\" or tostring(properties.virtualMachineProfile.storageProfile.osDisk.osType) == 'Windows' or tostring(properties.storageProfile.imageReference.publisher ) == \"microsoftsqlserver\"\r\n| extend OS=properties.storageProfile.imageReference.id \r\n| where isnull(OS)\r\n| extend WindowsId=id\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), WindowsId=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct WindowsId\r\n )\r\n on WindowsId\r\n| extend LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckAHBWindows = case(\r\n type == 'microsoft.compute/virtualmachines' or type =~ 'microsoft.compute/virtualMachineScaleSets', iif((properties.['licenseType'])\r\n !has 'Windows' and (properties.virtualMachineProfile.['licenseType']) !has 'Windows' , \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not Windows\"\r\n )\r\n) on subscriptionId \r\n| summarize count() by SubscriptionName, CheckAHBWindows\r\n", + "size": 0, + "title": "Summary of Windows VMs with or without AHB per Subscription", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "table", + "gridSettings": { + "sortBy": [ + { + "itemKey": "SubscriptionName", + "sortOrder": 1 + } + ], + "labelSettings": [ + { + "columnId": "SubscriptionName", + "label": "Subscription Name" + }, + { + "columnId": "CheckAHBWindows", + "label": "Is AHB enabled?" + }, + { + "columnId": "count_", + "label": "Number of resources" + } + ] + }, + "sortBy": [ + { + "itemKey": "SubscriptionName", + "sortOrder": 1 + } + ], + "tileSettings": { + "titleContent": { + "columnMatch": "CheckAHBWindows", + "formatter": 1 + }, + "subtitleContent": { + "columnMatch": "SubscriptionName", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + } + }, + "showBorder": false, + "size": "auto" + }, + "chartSettings": { + "xAxis": "SubscriptionName" + } + }, + "customWidth": "50", + "name": "Summary of Windows VMs with or without AHB per Subscription" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources \r\n| where resourceGroup in ({ResourceGroup})\r\n| where tostring(properties.storageProfile.imageReference.publisher ) == \"MicrosoftWindowsServer\" or tostring(properties.virtualMachineProfile.storageProfile.osDisk.osType) == 'Windows' or tostring(properties.storageProfile.imageReference.publisher ) == \"microsoftsqlserver\"\r\n| where tostring(properties.storageProfile.osDisk.osType) == 'Windows'\r\n| extend OS=properties.storageProfile.imageReference.id \r\n| where isnull(OS)\r\n| extend WindowsId=id\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), WindowsId=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct WindowsId\r\n )\r\n on WindowsId\r\n| extend LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckAHBWindows = case(\r\n type == 'microsoft.compute/virtualmachines' or type =~ 'microsoft.compute/virtualMachineScaleSets', iif((properties.['licenseType'])\r\n !has 'Windows' and (properties.virtualMachineProfile.['licenseType']) !has 'Windows' , \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not Windows\"\r\n )\r\n) on subscriptionId \r\n| summarize count() by CheckAHBWindows", + "size": 0, + "title": "Summary of Windows VMs with or without AHB", + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart" + }, + "customWidth": "50", + "name": "Summary of Windows VMs with or without AHB" + }, + { + "type": 1, + "content": { + "json": "### Consumed Licenses\r\nTotal number of Windows licenses cores consumed by all Windows virtual machines.\r\n", + "style": "info" + }, + "customWidth": "50", + "name": "Consumed Licenses" + }, + { + "type": 1, + "content": { + "json": "### Number of required Cores to enable Windows Azure Hybrid Benefit\r\nNumber of cores required to enable AHB across the entire environment.", + "style": "info" + }, + "customWidth": "50", + "name": "Number of required Cores" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"50d79765-aad4-437e-a90b-8cc7865e7081\",\"mergeType\":\"inner\",\"leftTable\":\"WindowsAHBEnabled\",\"rightTable\":\"query - Get VM vCPU\",\"leftColumn\":\"VMSize\",\"rightColumn\":\"Name\"}],\"projectRename\":[{\"originalName\":\"[WindowsAHBEnabled].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[WindowsAHBEnabled].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - Get VM vCPU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\">=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores per VM\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"8\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"[\\\"vCPUs\\\"]\"}}]},{\"originalName\":\"[WindowsAHBEnabled].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[WindowsAHBEnabled].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"unknown\"},{\"originalName\":\"[WindowsAHBEnabled].WindowsId\"},{\"originalName\":\"[WindowsAHBEnabled].VMRG\"},{\"originalName\":\"[WindowsAHBEnabled].VMLocation\"},{\"originalName\":\"[WindowsAHBEnabled].OSType\"},{\"originalName\":\"[WindowsAHBEnabled].OsVersion\"},{\"originalName\":\"[WindowsAHBEnabled].LicenseType\"},{\"originalName\":\"[query - Get VM vCPU].Name\"},{\"originalName\":\"[query - Get VM vCPU].MemoryGB\"},{\"originalName\":\"[query - Get VM vCPU].MaxNetworkInterfaces\"},{\"originalName\":\"[query - Get VM vCPU].HyperVGenerations\"},{\"originalName\":\"[query - Get VM vCPU].vCPUsPerCore\"},{\"originalName\":\"[WindowsAHBEnabled].VMSSize\"}]}", + "size": 0, + "title": "Consumed Cores per AHB Priority", + "noDataMessage": "None of your VMs have AHB enabled", + "noDataMessageStyle": 4, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Sev4", + "text": "{0}{1}" + } + ] + }, + "numberFormat": { + "unit": 17, + "options": { + "style": "decimal" + } + } + } + ] + }, + "chartSettings": { + "yAxis": [ + "Consumed Cores per VM" + ], + "group": "Prioritize AHB?", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Consumed Cores per AHB Priority" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"d8deb22b-a596-43ee-acc4-180849d26130\",\"mergeType\":\"inner\",\"leftTable\":\"WindowsAHBEnabled\",\"rightTable\":\"query - Get VM vCPU\",\"leftColumn\":\"VMSize\",\"rightColumn\":\"Name\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"ConsumedCores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"8\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"[\\\"vCPUs\\\"]\"}}]},{\"originalName\":\"[WindowsAHBEnabled].WindowsId\",\"mergedName\":\"WindowsId\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[WindowsAHBEnabled].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[WindowsAHBEnabled].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[WindowsAHBEnabled].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[WindowsAHBEnabled].VMSSize\",\"mergedName\":\"VMSSize\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[WindowsAHBEnabled].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[WindowsAHBEnabled].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[WindowsAHBEnabled].OsVersion\",\"mergedName\":\"OsVersion\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[WindowsAHBEnabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[query - Get VM vCPU].Name\",\"mergedName\":\"Name\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[query - Get VM vCPU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[query - Get VM vCPU].MemoryGB\",\"mergedName\":\"MemoryGB\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[query - Get VM vCPU].MaxNetworkInterfaces\",\"mergedName\":\"MaxNetworkInterfaces\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[query - Get VM vCPU].HyperVGenerations\",\"mergedName\":\"HyperVGenerations\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[query - Get VM vCPU].vCPUsPerCore\",\"mergedName\":\"vCPUsPerCore\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[WindowsAHBEnabled].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[WindowsAHBEnabled].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"unknown\"}]}", + "size": 0, + "title": "Consumed Cores per VM", + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "ConsumedCores", + "formatter": 0, + "formatOptions": { + "aggregation": "Sum" + } + } + ] + }, + "tileSettings": { + "titleContent": {}, + "leftContent": { + "columnMatch": "ConsumedCores", + "formatter": 12, + "formatOptions": { + "palette": "blue" + } + }, + "showBorder": false + }, + "graphSettings": { + "type": 0 + }, + "chartSettings": { + "yAxis": [ + "ConsumedCores" + ], + "group": "VMName", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Consumed Cores per VM" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"50d79765-aad4-437e-a90b-8cc7865e7081\",\"mergeType\":\"inner\",\"leftTable\":\"AHB Disabled\",\"rightTable\":\"query - Get VM vCPU\",\"leftColumn\":\"VMSize\",\"rightColumn\":\"Name\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\">=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores per VM\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"8\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"[\\\"vCPUs\\\"]\"}}]},{\"originalName\":\"[query - 0].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - 0].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"unknown\"},{\"originalName\":\"[AHB Disabled].WindowsId\",\"mergedName\":\"WindowsId\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].VMSSize\",\"mergedName\":\"VMSSize\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].OsVersion\",\"mergedName\":\"OsVersion\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - Get VM vCPU].Name\",\"mergedName\":\"Name\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - Get VM vCPU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - Get VM vCPU].MemoryGB\",\"mergedName\":\"MemoryGB\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - Get VM vCPU].MaxNetworkInterfaces\",\"mergedName\":\"MaxNetworkInterfaces\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - Get VM vCPU].HyperVGenerations\",\"mergedName\":\"HyperVGenerations\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - Get VM vCPU].vCPUsPerCore\",\"mergedName\":\"vCPUsPerCore\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].QuickFix\",\"mergedName\":\"QuickFix\",\"fromId\":\"unknown\"},{\"originalName\":\"[AHB Disabled].VMIDFull\",\"mergedName\":\"VMIDFull\",\"fromId\":\"unknown\"},{\"originalName\":\"[WindowsAHBEnabled].WindowsId\"},{\"originalName\":\"[WindowsAHBEnabled].VMRG\"},{\"originalName\":\"[WindowsAHBEnabled].VMLocation\"},{\"originalName\":\"[WindowsAHBEnabled].OSType\"},{\"originalName\":\"[WindowsAHBEnabled].OsVersion\"},{\"originalName\":\"[WindowsAHBEnabled].LicenseType\"},{\"originalName\":\"[query - Get VM vCPU].Name\"},{\"originalName\":\"[query - Get VM vCPU].MemoryGB\"},{\"originalName\":\"[query - Get VM vCPU].MaxNetworkInterfaces\"},{\"originalName\":\"[query - Get VM vCPU].HyperVGenerations\"},{\"originalName\":\"[query - Get VM vCPU].vCPUsPerCore\"},{\"originalName\":\"[WindowsAHBEnabled].VMSSize\"}]}", + "size": 0, + "title": "Cores not enabled per AHB Priority", + "noDataMessage": "All of your VMs have AHB enabled", + "noDataMessageStyle": 4, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Sev4", + "text": "{0}{1}" + } + ] + }, + "numberFormat": { + "unit": 17, + "options": { + "style": "decimal" + } + } + } + ] + }, + "chartSettings": { + "yAxis": [ + "Consumed Cores per VM" + ], + "group": "Prioritize AHB?", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Cores NOT enabled per AHB Priority" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "ae5e8765-47ef-46a6-803b-6b7124c098d2", + "version": "KqlParameterItem/1.0", + "name": "AHBEnabled", + "label": "See VMs with AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n", + "timeContext": { + "durationMs": 86400000 + } + }, + { + "id": "f1ac5e53-253c-4afb-8bc5-b1ba2efea3eb", + "version": "KqlParameterItem/1.0", + "name": "AHBDisabled", + "label": "See VMs without AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n" + }, + { + "id": "20a00706-a89b-42aa-8dea-9c44c93e8014", + "version": "KqlParameterItem/1.0", + "name": "LastAHB", + "label": "See VMs AHB enabled in the last 7 days", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n" + } + ], + "style": "pills", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "VM AHB Enabled" + }, + { + "type": 1, + "content": { + "json": "List of Windows VMs without Hybrid Benefit groupped by Subscription.", + "style": "info" + }, + "conditionalVisibility": { + "parameterName": "AHBDisabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "List of Windows VMs without AHB" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"50d79765-aad4-437e-a90b-8cc7865e7081\",\"mergeType\":\"inner\",\"leftTable\":\"AHB Disabled\",\"rightTable\":\"query - Get VM vCPU\",\"leftColumn\":\"VMSize\",\"rightColumn\":\"Name\"}],\"projectRename\":[{\"originalName\":\"[query - 0].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - 0].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[AHB Disabled].VMName\",\"mergedName\":\"VM Name\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\">=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"static\",\"resultVal\":\"High Priority\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"static\",\"resultVal\":\"Low Priority\"}}]},{\"originalName\":\"[AHB Disabled].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].QuickFix\",\"mergedName\":\"QuickFix\",\"fromId\":\"unknown\"},{\"originalName\":\"[AHB Disabled].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - Get VM vCPU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].OsVersion\",\"mergedName\":\"OsVersion\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - Get VM vCPU].Name\",\"mergedName\":\"Name\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - Get VM vCPU].MemoryGB\",\"mergedName\":\"MemoryGB\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - Get VM vCPU].MaxNetworkInterfaces\",\"mergedName\":\"MaxNetworkInterfaces\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - Get VM vCPU].HyperVGenerations\",\"mergedName\":\"HyperVGenerations\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - Get VM vCPU].vCPUsPerCore\",\"mergedName\":\"vCPUsPerCore\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[AHB Disabled].VMIDFull\",\"mergedName\":\"VMIDFull\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - 0].VMName\"},{\"originalName\":\"[query - 0].VMSSize\"},{\"originalName\":\"[query - Get VM vCPU].Name\"},{\"originalName\":\"[query - Get VM vCPU].MemoryGB\"},{\"originalName\":\"[query - Get VM vCPU].MaxNetworkInterfaces\"},{\"originalName\":\"[query - Get VM vCPU].HyperVGenerations\"},{\"originalName\":\"[query - Get VM vCPU].vCPUsPerCore\"},{\"originalName\":\"[AHB Disabled].WindowsId\"},{\"originalName\":\"[AHB Disabled].VMSSize\"}]}", + "size": 0, + "title": "VMs without AHB", + "noDataMessage": "All of your VMs have AHB enabled", + "noDataMessageStyle": 3, + "exportedParameters": [ + { + "fieldName": "VMIDFull", + "parameterName": "WindowsID" + }, + { + "fieldName": "VMRG", + "parameterName": "ResourceGroup", + "parameterType": 1 + }, + { + "fieldName": "VM Name", + "parameterName": "VMName", + "parameterType": 1 + }, + { + "fieldName": "Prioritize AHB?", + "parameterName": "AHBPriority", + "parameterType": 1 + } + ], + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "QuickFix", + "formatter": 7, + "formatOptions": { + "linkTarget": "ArmAction", + "linkLabel": "Apply Hybrid Benefit", + "linkIsContextBlade": true, + "armActionContext": { + "path": "/{WindowsID}?api-version=2023-03-01", + "headers": [], + "params": [], + "body": "{\r\n \"properties\": {\r\n \"licenseType\": \"Windows_Server\"\r\n }\r\n}\r\n\r\n", + "httpMethod": "PATCH", + "title": "Apply Hybrid Benefit to VM {VMName}", + "description": "# Windows Hybrid Benefit Application Information: VM \"{VMName}\"\n\n\n{WindowsID}\n\n**Attention!**\n\nThis action will apply the Windows Hybrid Benefit to the virtual machine with the name **{VMName}**. Please ensure that you are applying the benefit to the correct VM.\n\n**Resource Details:**\n\n- VM Name: {VMName}\n- Resource Group: {ResourceGroup}\n- Prioritize AHB: {AHBPriority}\n\n### Required RBAC Permissions\n\nTo perform this action, you need to have **Contributor** permissions on the Resource Group where the VM is located.\n\nPlease review the information carefully before proceeding with applying the Windows Hybrid Benefit.\n", + "actionName": "Applying Hybrid benefit to VM {VMName}", + "runLabel": "Apply Hybrid Benefit to VM: \"{VMName}\"" + } + } + }, + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Sev4", + "text": "{0}{1}" + } + ] + }, + "numberFormat": { + "unit": 17, + "options": { + "style": "decimal" + } + }, + "tooltipFormat": { + "tooltip": "The virtual machines (VMs) with less than 8 cores are categorized as Low Priority, while those with 8 or more cores are classified as High Priority. " + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "WindowsId1", + "formatter": 5 + }, + { + "columnMatch": "Name", + "formatter": 5 + }, + { + "columnMatch": "HyperVGenerations", + "formatter": 5 + }, + { + "columnMatch": "vCPUsPerCore", + "formatter": 5 + }, + { + "columnMatch": "VMIDFull", + "formatter": 5 + } + ], + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "subscriptionId" + ], + "expandTopLevel": true + }, + "labelSettings": [ + { + "columnId": "VM Name", + "label": "VM Name" + }, + { + "columnId": "VMRG", + "label": "Resource Group" + }, + { + "columnId": "VMLocation", + "label": "Location" + }, + { + "columnId": "QuickFix", + "label": "Enable AHB" + }, + { + "columnId": "Prioritize AHB?", + "label": "AHB Priority" + }, + { + "columnId": "VMSize", + "label": "SKU" + }, + { + "columnId": "OSType", + "label": "OS Type" + }, + { + "columnId": "OsVersion", + "label": "OS Version" + }, + { + "columnId": "LicenseType", + "label": "License Type" + }, + { + "columnId": "subscriptionId", + "label": "Subscription Name" + }, + { + "columnId": "MemoryGB", + "label": "Memory" + }, + { + "columnId": "MaxNetworkInterfaces", + "label": "Max. NICs" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "AHBDisabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "VM+SKU+vCores" + }, + { + "type": 1, + "content": { + "json": "List of Windows VMs with Hybrid Benefit groupped by Subscription.", + "style": "info" + }, + "conditionalVisibility": { + "parameterName": "AHBEnabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "AHB By SUbscription" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"50d79765-aad4-437e-a90b-8cc7865e7081\",\"mergeType\":\"inner\",\"leftTable\":\"WindowsAHBEnabled\",\"rightTable\":\"query - Get VM vCPU\",\"leftColumn\":\"VMSize\",\"rightColumn\":\"Name\"}],\"projectRename\":[{\"originalName\":\"[WindowsAHBEnabled].WindowsId\",\"mergedName\":\"VM Name\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[WindowsAHBEnabled].VMRG\",\"mergedName\":\"Resource Group\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[WindowsAHBEnabled].VMSize\",\"mergedName\":\"VM SKU\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - Get VM vCPU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\">=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"static\",\"resultVal\":\"High Priority\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"static\",\"resultVal\":\"Low Priority\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores per VM\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"8\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"([\\\"vCPUs\\\"] + 7) & ~7\"}}]},{\"originalName\":\"[WindowsAHBEnabled].LicenseType\",\"mergedName\":\"License Type\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[WindowsAHBEnabled].VMLocation\",\"mergedName\":\"Location\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[WindowsAHBEnabled].OSType\",\"mergedName\":\"OS Type\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[WindowsAHBEnabled].OsVersion\",\"mergedName\":\"OS Version\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[WindowsAHBEnabled].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[WindowsAHBEnabled].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - 0].VMName\"},{\"originalName\":\"[query - 0].VMSSize\"},{\"originalName\":\"[query - Get VM vCPU].Name\"},{\"originalName\":\"[query - Get VM vCPU].MemoryGB\"},{\"originalName\":\"[query - Get VM vCPU].MaxNetworkInterfaces\"},{\"originalName\":\"[query - Get VM vCPU].HyperVGenerations\"},{\"originalName\":\"[query - Get VM vCPU].vCPUsPerCore\"},{\"originalName\":\"[WindowsAHBEnabled].VMSSize\"},{\"originalName\":\"[WindowsAHBEnabled].VMName\"}]}", + "size": 0, + "title": "VMs with AHB", + "noDataMessage": "None of your VMs have AHB enabled", + "noDataMessageStyle": 4, + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "2", + "text": "{0}{1}" + } + ] + }, + "numberFormat": { + "unit": 17, + "options": { + "style": "decimal" + } + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "WindowsId1", + "formatter": 5 + }, + { + "columnMatch": "Subscription Name", + "formatter": 5 + } + ], + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "subscriptionId" + ], + "expandTopLevel": true + }, + "labelSettings": [ + { + "columnId": "WindowsId1", + "label": "VM ID" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "AHBEnabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "VM+SKU+vCores-AHB" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"d8deb22b-a596-43ee-acc4-180849d26168\",\"mergeType\":\"inner\",\"leftTable\":\"VM Latest Change Last 7 days\",\"rightTable\":\"VM+SKU+vCores-AHB\",\"leftColumn\":\"VMID\",\"rightColumn\":\"VM Name\"}],\"projectRename\":[{\"originalName\":\"[VM Latest Change Last 7 days].VMID\",\"mergedName\":\"VMID\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM Latest Change Last 7 days].NewLicense\",\"mergedName\":\"NewLicense\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM Latest Change Last 7 days].DateofChange\",\"mergedName\":\"DateofChange\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].VM Name\",\"mergedName\":\"VM Name\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].Resource Group\",\"mergedName\":\"Resource Group\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].VM SKU\",\"mergedName\":\"VM SKU\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].Prioritize AHB?\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].License Type\",\"mergedName\":\"License Type\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].Consumed Cores per VM\",\"mergedName\":\"Consumed Cores per VM\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].Location\",\"mergedName\":\"Location\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].OS Type\",\"mergedName\":\"OS Type\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].OS Version\",\"mergedName\":\"OS Version\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[VM+SKU+vCores-AHB].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"unknown\"}]}", + "size": 0, + "title": "Total Cores Enabled last 7 Days", + "noDataMessage": "Windows AHB hasn't been enabled in the last 7 days", + "showRefreshButton": true, + "queryType": 7, + "visualization": "barchart", + "tileSettings": { + "showBorder": false, + "titleContent": { + "columnMatch": "NewLicense", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "vCPUs", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + }, + "chartSettings": { + "xAxis": "VM Name", + "yAxis": [ + "Consumed Cores per VM" + ], + "group": null, + "createOtherGroup": 0, + "seriesLabelSettings": [ + { + "seriesName": "Consumed Cores per VM", + "color": "grayBlue" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "LastAHB", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "Total Cores Enabled last 7 Days" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"d8deb22b-a596-43ee-acc4-180849d26168\",\"mergeType\":\"inner\",\"leftTable\":\"VM Latest Change Last 7 days\",\"rightTable\":\"VM+SKU+vCores-AHB\",\"leftColumn\":\"VMID\",\"rightColumn\":\"VM Name\"}],\"projectRename\":[{\"originalName\":\"[VM+SKU+vCores-AHB].VM Name\",\"mergedName\":\"VM Name\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].Resource Group\",\"mergedName\":\"Resource Group\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM Latest Change Last 7 days].NewLicense\",\"mergedName\":\"NewLicense\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM Latest Change Last 7 days].DateofChange\",\"mergedName\":\"DateofChange\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].VM SKU\",\"mergedName\":\"VM SKU\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].Consumed Cores per VM\",\"mergedName\":\"Consumed Cores per VM\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].Prioritize AHB?\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].Location\",\"mergedName\":\"Location\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26168\"},{\"originalName\":\"[VM+SKU+vCores-AHB].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[VM+SKU+vCores-AHB].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"unknown\"},{\"originalName\":\"[VM Latest Change Last 7 days].VMID\"},{\"originalName\":\"[VM+SKU+vCores-AHB].OS Type\"},{\"originalName\":\"[VM+SKU+vCores-AHB].OS Version\"},{\"originalName\":\"[VM+SKU+vCores-AHB].License Type\"}]}", + "size": 0, + "title": "Total Cores Enabled last 7 Days - Detailed view", + "noDataMessage": "No AHB has been enabled in the last 7 days", + "showExportToExcel": true, + "queryType": 7, + "visualization": "table", + "tileSettings": { + "showBorder": false, + "titleContent": { + "columnMatch": "NewLicense", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "vCPUs", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + }, + "chartSettings": { + "xAxis": "VM Name", + "yAxis": [ + "Consumed Cores per VM" + ], + "group": null, + "createOtherGroup": 0, + "seriesLabelSettings": [ + { + "seriesName": "Consumed Cores per VM", + "color": "grayBlue" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "LastAHB", + "comparison": "isEqualTo", + "value": "Yes" + }, + "showPin": false, + "name": "Total Cores Enabled last 7 Days - Details" + } + ] + }, + "name": "VM" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "VM" + }, + { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "VM" + } + ], + "name": "VM/VMSS-RGFilter" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "0211f413-9f36-4750-9ef2-d382ba30ba6c", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "SQL Server VMs", + "subTarget": "SQLVM", + "preText": "VM", + "style": "link" + }, + { + "id": "79e7a97a-1413-41e8-b4c6-ebd1d0a45e2e", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "SQL DB", + "subTarget": "SQLDB", + "style": "link" + }, + { + "id": "1f381e5b-7071-41ce-a354-c2df93445cae", + "cellValue": "SelectedSubTab", + "linkTarget": "parameter", + "linkLabel": "SQL Managed Instances", + "subTarget": "SQLMI", + "style": "link" + } + ] + }, + "name": "links - 4" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\n resources | where type =~ 'Microsoft.SqlVirtualMachine/SqlVirtualMachines' and tostring(properties.['sqlServerLicenseType']) != 'AHUB' and resourceGroup in ({ResourceGroup})\r\n | extend SQLID=id, VMName = name, VMRG = resourceGroup, VMLocation = location, LicenseType = tostring(properties.['sqlServerLicenseType']), OSType=tostring(properties.storageProfile.imageReference.offer), SQLAgentType = tostring(properties.['sqlManagement']), SQLVersion = tostring(properties.['sqlImageOffer']), SQLSKU=tostring(properties.['sqlImageSku'])\r\n ) on subscriptionId \r\n| join (\r\n resources\r\n | where type =~ 'Microsoft.Compute/virtualmachines'\r\n | project VMName = tolower(name), VMSize = tostring(properties.hardwareProfile.vmSize)\r\n ) on VMName\r\n| order by id asc \r\n| project SQLID,VMName,VMRG, VMLocation, VMSize, SQLVersion, SQLSKU, SQLAgentType, LicenseType, SubscriptionName\r\n| where SQLSKU != \"Developer\" and SQLSKU != \"Express\"\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLID\r\n )\r\n on SQLID", + "size": 0, + "title": "SQL VM AHB Disabled", + "noDataMessage": "All of your VMs have AHB enabled.", + "noDataMessageStyle": 3, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "Get-SQL-AHB-Disabled" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\n resources | where type =~ 'Microsoft.SqlVirtualMachine/SqlVirtualMachines' and tostring(properties.['sqlServerLicenseType']) == 'AHUB' and resourceGroup in ({ResourceGroup})\r\n | extend SQLID=id, VMName = name, VMRG = resourceGroup, VMLocation = location, LicenseType = tostring(properties.['sqlServerLicenseType']), OSType=tostring(properties.storageProfile.imageReference.offer), SQLAgentType = tostring(properties.['sqlManagement']), SQLVersion = tostring(properties.['sqlImageOffer']), SQLSKU=tostring(properties.['sqlImageSku'])\r\n ) on subscriptionId \r\n| join (\r\n resources\r\n | where type =~ 'Microsoft.Compute/virtualmachines'\r\n | project VMName = tolower(name), VMSize = tostring(properties.hardwareProfile.vmSize)\r\n ) on VMName\r\n| order by id asc \r\n| project SQLID,VMName,VMRG, VMLocation, VMSize, SQLVersion, SQLSKU, SQLAgentType, LicenseType, SubscriptionName\r\n| where SQLSKU != \"Developer\" and SQLSKU != \"Express\"\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLID\r\n )\r\n on SQLID", + "size": 0, + "title": "SQL VM AHB Enabled", + "noDataMessage": "None of your VMs have AHB enabled.", + "noDataMessageStyle": 5, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "Get-SQL-AHB-Enabled" + }, + { + "type": 1, + "content": { + "json": "## SQL Virtual Machines Azure Hybrid Benefit (AHB) Overview" + }, + "name": "SQL Text" + }, + { + "type": 1, + "content": { + "json": "Apply to SQL Server 1 to 4 vCPUs exchange: For every 1 core of SQL Server Enterprise Edition, you get 4 vCPUs of SQL Managed Instance or Azure SQL Database general purpose and Hyperscale tiers, or 4 vCPUs of SQL Server Standard edition on Azure VMs.\r\n\r\nThe SQL virtual machines (VMs) with less than 4 cores are categorized as **Low Priority**, while those with 8 or more cores are classified as **High Priority**. In situations where there are insufficient Azure Hybrid benefit licenses to cover all the VMs in the environment, it is recommended to prioritize the High Priority VMs.", + "style": "info" + }, + "name": "SQL License Info", + "styleSettings": { + "margin": "10px", + "showBorder": true + } + }, + { + "type": 1, + "content": { + "json": "### AHB Overview\r\nSummary of all SQL on VMs with and without SQL AHB.", + "style": "info" + }, + "name": "AHB Overview21" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "loadType": "explicit", + "loadButtonText": "Load SQL Info", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\n resources | where type =~ 'Microsoft.SqlVirtualMachine/SqlVirtualMachines' and resourceGroup in ({ResourceGroup})\r\n | extend SQLID=id, VMName = name, VMRG = resourceGroup, VMLocation = location, LicenseType = tostring(properties.['sqlServerLicenseType']), OSType=tostring(properties.storageProfile.imageReference.offer), SQLAgentType = tostring(properties.['sqlManagement']), SQLVersion = tostring(properties.['sqlImageOffer']), SQLSKU=tostring(properties.['sqlImageSku'])\r\n | where SQLSKU != \"Developer\" and SQLSKU != \"Express\"\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLID\r\n )\r\n on SQLID\r\n | extend CheckSQLVMAHB = case(\r\n type =~ 'Microsoft.SqlVirtualMachine/SqlVirtualMachines', iif((properties.['sqlServerLicenseType'])\r\n !has 'AHUB', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by SubscriptionName, CheckSQLVMAHB", + "size": 0, + "title": "Summary of SQL on VMs with or without AHB per Subscription", + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "table", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ], + "labelSettings": [ + { + "columnId": "SubscriptionName", + "label": "Subscription Name" + }, + { + "columnId": "CheckSQLVMAHB", + "label": "Is AHB enabled?" + }, + { + "columnId": "count_", + "label": "Number of Resources" + } + ] + }, + "tileSettings": { + "titleContent": { + "columnMatch": "CheckSQLVMAHB", + "formatter": 1 + }, + "subtitleContent": { + "columnMatch": "SubscriptionName", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + } + }, + "showBorder": false, + "size": "auto" + }, + "chartSettings": { + "xAxis": "SubscriptionName" + } + }, + "customWidth": "50", + "name": "Summary of SQL on VMs with or without AHB" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\n resources | where type =~ 'Microsoft.SqlVirtualMachine/SqlVirtualMachines' and resourceGroup in ({ResourceGroup})\r\n | extend SQLID=id, VMName = name, VMRG = resourceGroup, VMLocation = location, LicenseType = tostring(properties.['sqlServerLicenseType']), OSType=tostring(properties.storageProfile.imageReference.offer), SQLAgentType = tostring(properties.['sqlManagement']), SQLVersion = tostring(properties.['sqlImageOffer']), SQLSKU=tostring(properties.['sqlImageSku'])\r\n | where SQLSKU != \"Developer\" and SQLSKU != \"Express\"\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLID\r\n )\r\n on SQLID\r\n | extend CheckSQLVMAHB = case(\r\n type =~ 'Microsoft.SqlVirtualMachine/SqlVirtualMachines', iif((properties.['sqlServerLicenseType'])\r\n !has 'AHUB', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by CheckSQLVMAHB", + "size": 0, + "title": "Summary SQL Enabled and Disabled", + "noDataMessage": "You don't have any SQL VM", + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart" + }, + "customWidth": "50", + "name": "Summary SQL Enabled and Disabled" + }, + { + "type": 1, + "content": { + "json": "### Consumed Licenses\r\nTotal number of SQL licenses cores consumed by all SQL running on Virtual Machines.\r\n", + "style": "info" + }, + "customWidth": "50", + "name": "Consumed Licenses123" + }, + { + "type": 1, + "content": { + "json": "### Number of required Cores to enable SQL Azure Hybrid Benefit\r\nNumber of cores required to enable SQL AHB across the entire environment.\r\n\r\n\r\n", + "style": "info" + }, + "customWidth": "50", + "name": "Number of required Cores to enable SQL AHB" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"cd7477ac-acd6-4894-b929-53348c7640e8\",\"mergeType\":\"inner\",\"leftTable\":\"API-Get_VM_SKU\",\"rightTable\":\"Get-SQL-AHB-Enabled\",\"leftColumn\":\"Name\",\"rightColumn\":\"VMSize\"}],\"projectRename\":[{\"originalName\":\"[Get-SQL-AHB-Enabled].SQLID\",\"mergedName\":\"SQLID\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"4\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"round([\\\"vCPUs\\\"] / 4) * 4\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}}]},{\"originalName\":\"[Get-SQL-AHB-Enabled].SQLVersion\",\"mergedName\":\"SQLVersion\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].SQLSKU\",\"mergedName\":\"SQLSKU\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].SQLAgentType\",\"mergedName\":\"SQLAgentType\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[API-Get_VM_SKU].Name\",\"mergedName\":\"Name\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[API-Get_VM_SKU].MemoryGB\",\"mergedName\":\"MemoryGB\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[API-Get_VM_SKU].MaxNetworkInterfaces\",\"mergedName\":\"MaxNetworkInterfaces\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[API-Get_VM_SKU].HyperVGenerations\",\"mergedName\":\"HyperVGenerations\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUsPerCore\",\"mergedName\":\"vCPUsPerCore\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].SQLID1\",\"mergedName\":\"SQLID1\",\"fromId\":\"unknown\"}]}", + "size": 0, + "title": "SQL VM AHB Consumed Cores per VM", + "noDataMessage": "None of your VMs have AHB enabled.", + "noDataMessageStyle": 4, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ] + }, + "chartSettings": { + "yAxis": [ + "Consumed Cores" + ], + "group": "VMName", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Summary SQL+SKU AHB Enabled - per VM" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"cd7477ac-acd6-4894-b929-53348c7640e8\",\"mergeType\":\"inner\",\"leftTable\":\"API-Get_VM_SKU\",\"rightTable\":\"Get-SQL-AHB-Enabled\",\"leftColumn\":\"Name\",\"rightColumn\":\"VMSize\"}],\"projectRename\":[{\"originalName\":\"[Get-SQL-AHB-Enabled].SQLID\",\"mergedName\":\"SQLID\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"4\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"round([\\\"vCPUs\\\"] / 4) * 4\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}}]},{\"originalName\":\"[Get-SQL-AHB-Enabled].SQLVersion\",\"mergedName\":\"SQLVersion\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].SQLSKU\",\"mergedName\":\"SQLSKU\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].SQLAgentType\",\"mergedName\":\"SQLAgentType\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].SQLID1\",\"mergedName\":\"SQLID1\",\"fromId\":\"unknown\"},{\"originalName\":\"[API-Get_VM_SKU].Name\"},{\"originalName\":\"[API-Get_VM_SKU].MemoryGB\"},{\"originalName\":\"[API-Get_VM_SKU].MaxNetworkInterfaces\"},{\"originalName\":\"[API-Get_VM_SKU].HyperVGenerations\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUsPerCore\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].VMName\"}]}", + "size": 0, + "title": "SQL VM AHB Consumed Cores per Priority", + "noDataMessage": "None of your VMs have AHB enabled.", + "noDataMessageStyle": 4, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ] + }, + "chartSettings": { + "group": "Prioritize AHB?", + "createOtherGroup": null + } + }, + "customWidth": "33", + "showPin": false, + "name": "Summary SQL+SKU AHB Enabled -" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"cd7477ac-acd6-4894-b929-53348c7640b5\",\"mergeType\":\"inner\",\"leftTable\":\"API-Get_VM_SKU\",\"rightTable\":\"Get-SQL-AHB-Disabled\",\"leftColumn\":\"Name\",\"rightColumn\":\"VMSize\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"4\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"([\\\"vCPUs\\\"] +3) & ~3\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}}]},{\"originalName\":\"[API-Get_VM_SKU].Name\",\"mergedName\":\"Name\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[API-Get_VM_SKU].MemoryGB\",\"mergedName\":\"MemoryGB\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[API-Get_VM_SKU].MaxNetworkInterfaces\",\"mergedName\":\"MaxNetworkInterfaces\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[API-Get_VM_SKU].HyperVGenerations\",\"mergedName\":\"HyperVGenerations\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUsPerCore\",\"mergedName\":\"vCPUsPerCore\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].SQLID\",\"mergedName\":\"SQLID\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].SQLVersion\",\"mergedName\":\"SQLVersion\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].SQLSKU\",\"mergedName\":\"SQLSKU\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].SQLAgentType\",\"mergedName\":\"SQLAgentType\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].SQLID1\",\"mergedName\":\"SQLID1\",\"fromId\":\"unknown\"}]}", + "size": 0, + "title": "Cores not enabled per AHB Priority", + "noDataMessage": "All of your VMs have AHB enabled.", + "noDataMessageStyle": 3, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "warning", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + } + } + ] + }, + "sortBy": [], + "chartSettings": { + "yAxis": [ + "Consumed Cores" + ], + "showMetrics": false, + "showLegend": true + } + }, + "customWidth": "33", + "name": " Summary - SQL Cores AHB Disabled " + } + ] + }, + "name": "SQL Overview RG" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "ae5e8765-47ef-46a6-803b-6b7124c098d2", + "version": "KqlParameterItem/1.0", + "name": "SQLAVMHUBEnabled", + "label": "See SQL VMs with AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n", + "timeContext": { + "durationMs": 86400000 + } + }, + { + "id": "f1ac5e53-253c-4afb-8bc5-b1ba2efea3eb", + "version": "KqlParameterItem/1.0", + "name": "SQLVMAHBDisabled", + "label": "See SQL VMs without AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n", + "value": "Yes" + } + ], + "style": "pills", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "SQL AHB Disabled" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"cd7477ac-acd6-4894-b929-53348c7640b5\",\"mergeType\":\"inner\",\"leftTable\":\"API-Get_VM_SKU\",\"rightTable\":\"Get-SQL-AHB-Disabled\",\"leftColumn\":\"Name\",\"rightColumn\":\"VMSize\"}],\"projectRename\":[{\"originalName\":\"[Get-SQL-AHB-Disabled].SQLID\",\"mergedName\":\"VM Name\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"4\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"round([\\\"vCPUs\\\"] / 4) * 4\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}}]},{\"originalName\":\"[Get-SQL-AHB-Disabled].SQLVersion\",\"mergedName\":\"SQLVersion\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].SQLSKU\",\"mergedName\":\"SQLSKU\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].SQLAgentType\",\"mergedName\":\"SQLAgentType\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640b5\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].SQLID1\",\"mergedName\":\"SQLID1\",\"fromId\":\"unknown\"},{\"originalName\":\"[API-Get_VM_SKU].Name\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUs\"},{\"originalName\":\"[API-Get_VM_SKU].MemoryGB\"},{\"originalName\":\"[API-Get_VM_SKU].MaxNetworkInterfaces\"},{\"originalName\":\"[API-Get_VM_SKU].HyperVGenerations\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUsPerCore\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].SQLID\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].VMName\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].VMRG\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].VMLocation\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].SQLVersion\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].SQLSKU\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].SQLAgentType\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].LicenseType\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].SubscriptionName\"},{\"originalName\":\"[Get-SQL-AHB-Disabled].VMSize\"}]}", + "size": 0, + "title": "SQL VM AHB Disabled", + "noDataMessage": "All of your VMs have AHB enabled.", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "warning", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + } + } + ], + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "SubscriptionName" + ], + "expandTopLevel": true + }, + "labelSettings": [ + { + "columnId": "VM Name", + "label": "Name" + }, + { + "columnId": "VMRG", + "label": "Resource Group" + }, + { + "columnId": "VMLocation", + "label": "Location" + }, + { + "columnId": "VMSize", + "label": "SKU" + }, + { + "columnId": "vCPUs", + "label": "Number of vCPU" + }, + { + "columnId": "Consumed Cores", + "label": "Consumed Cores" + }, + { + "columnId": "SQLVersion", + "label": "SQL Version" + }, + { + "columnId": "SQLSKU", + "label": "SQL SKU" + }, + { + "columnId": "SQLAgentType", + "label": "SQL Agent" + }, + { + "columnId": "LicenseType", + "label": "License Type" + }, + { + "columnId": "SubscriptionName", + "label": "Subscription Name" + }, + { + "columnId": "SQLID1", + "label": "Resource ID" + } + ] + }, + "sortBy": [] + }, + "conditionalVisibility": { + "parameterName": "SQLVMAHBDisabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "SQL+SKU AHB Disabled" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"cd7477ac-acd6-4894-b929-53348c7640e8\",\"mergeType\":\"inner\",\"leftTable\":\"API-Get_VM_SKU\",\"rightTable\":\"Get-SQL-AHB-Enabled\",\"leftColumn\":\"Name\",\"rightColumn\":\"VMSize\"}],\"projectRename\":[{\"originalName\":\"[Get-SQL-AHB-Enabled].SQLID\",\"mergedName\":\"SQLID\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"4\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"round([\\\"vCPUs\\\"] / 4) * 4\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}}]},{\"originalName\":\"[Get-SQL-AHB-Enabled].SQLVersion\",\"mergedName\":\"SQLVersion\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].SQLSKU\",\"mergedName\":\"SQLSKU\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].SQLAgentType\",\"mergedName\":\"SQLAgentType\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"cd7477ac-acd6-4894-b929-53348c7640e8\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].SQLID1\",\"mergedName\":\"SQLID1\",\"fromId\":\"unknown\"},{\"originalName\":\"[API-Get_VM_SKU].Name\"},{\"originalName\":\"[API-Get_VM_SKU].MemoryGB\"},{\"originalName\":\"[API-Get_VM_SKU].MaxNetworkInterfaces\"},{\"originalName\":\"[API-Get_VM_SKU].HyperVGenerations\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUsPerCore\"},{\"originalName\":\"[Get-SQL-AHB-Enabled].VMName\"}]}", + "size": 0, + "title": "SQL VM AHB Enabled", + "noDataMessage": "None of your VMs have AHB enabled.", + "noDataMessageStyle": 4, + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "Subscription", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "SubscriptionName" + ], + "expandTopLevel": true + }, + "labelSettings": [ + { + "columnId": "SQLID", + "label": "Name" + }, + { + "columnId": "VMRG", + "label": "Resource Group" + }, + { + "columnId": "VMLocation", + "label": "Location" + }, + { + "columnId": "VMSize", + "label": "SKU" + }, + { + "columnId": "vCPUs", + "label": "Number of vCPU" + }, + { + "columnId": "Consumed Cores", + "label": "Consumed Cores" + }, + { + "columnId": "SQLVersion", + "label": "SQL Version" + }, + { + "columnId": "SQLSKU", + "label": "SQL SKU" + }, + { + "columnId": "SQLAgentType", + "label": "SQL Agent" + }, + { + "columnId": "LicenseType", + "label": "License Type" + }, + { + "columnId": "SubscriptionName", + "label": "Subscription Name" + }, + { + "columnId": "SQLID1", + "label": "Resource ID" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "SQLAVMHUBEnabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "SQL+SKU AHB Enabled" + } + ] + }, + "name": "SQL Detailed Info" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "SQLVM" + }, + "name": "SQL VM" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "title": "SQL Database", + "items": [ + { + "type": 1, + "content": { + "json": "## SQL Databases Azure Hybrid Benefit (AHB) Overview" + }, + "name": "SQL Databases AHB" + }, + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "e4aa368f-dcf2-44a6-88f9-a395c04eb21f", + "cellValue": "SQLType", + "linkTarget": "parameter", + "linkLabel": "SQL Database", + "subTarget": "SQLDatabase", + "style": "link" + }, + { + "id": "a94e8dc2-34be-4d97-934d-c27e1816c4fe", + "cellValue": "SQLType", + "linkTarget": "parameter", + "linkLabel": "SQL ElasticPool", + "subTarget": "SQLElastic", + "style": "link" + } + ] + }, + "name": "links - 8" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "SQLDB" + }, + "name": "text - 0" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and tostring(properties.['licenseType']) == 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/databases', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n )\r\n) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, CheckSQLDBAHB,SQLLocation, LicenseType, StorageAccountType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n", + "size": 0, + "title": "AHB Disabled", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "SQLDB AHB Disabled" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and tostring(properties.['licenseType']) != 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=sku.name, SKUTier=sku.tier, SQLLocation = location, vCores=toint(sku.capacity), LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, SQLLocation, LicenseType, StorageAccountType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n\r\n", + "size": 0, + "title": "AHB Enabled", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "SQLDB AHB Enabled" + }, + { + "type": 1, + "content": { + "json": "Apply to SQL Server 1 to 4 vCPUs exchange: For every 1 core of SQL Server Enterprise Edition, you get 4 vCPUs of SQL Managed Instance or Azure SQL Database general purpose and Hyperscale tiers, or 4 vCPUs of SQL Server Standard edition on Azure VMs.\r\n\r\nThe SQL virtual machines (VMs) with less than 4 cores are categorized as **Low Priority**, while those with 8 or more cores are classified as **High Priority**. In situations where there are insufficient Azure Hybrid benefit licenses to cover all the VMs in the environment, it is recommended to prioritize the High Priority VMs.\r\n\r\nFor Azure SQL Database, Azure Hybrid Benefit is only available when using the provisioned compute tier of the vCore-based purchasing model. Azure Hybrid Benefit doesn't apply to DTU-based purchasing models or the serverless compute tier.", + "style": "info" + }, + "name": "Apply to SQL Server 1 to 4 vCPUs " + }, + { + "type": 1, + "content": { + "json": "### AHB Overview\r\nSummary of all SQL Databases with and without SQL AHB.", + "style": "info" + }, + "name": " AHB Overview SQL DB" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "loadType": "explicit", + "loadButtonText": "Load SQL DB Info", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\n resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n | extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=sku.name, SKUTier=sku.tier, SQLLocation = location, LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/databases', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by SubscriptionName, CheckSQLDBAHB", + "size": 0, + "title": "Summary of SQL Databases with or without AHB per Subscription", + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "table", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ], + "labelSettings": [ + { + "columnId": "SubscriptionName", + "label": "Subscription Name" + }, + { + "columnId": "CheckSQLDBAHB", + "label": "Is AHB enabled?" + }, + { + "columnId": "count_", + "label": "Number of resources" + } + ] + }, + "tileSettings": { + "titleContent": { + "columnMatch": "CheckSQLDBAHB", + "formatter": 1 + }, + "subtitleContent": { + "columnMatch": "SubscriptionName", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + } + }, + "showBorder": false, + "size": "auto" + }, + "chartSettings": { + "xAxis": "SubscriptionName" + } + }, + "customWidth": "50", + "name": "Summary of SQL DBs with or without AHB per subs" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\n resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n | extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=sku.name, SKUTier=sku.tier, SQLLocation = location, LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/databases', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by CheckSQLDBAHB", + "size": 0, + "title": "Summary of SQL Databases with or without AHB", + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ] + }, + "tileSettings": { + "titleContent": { + "columnMatch": "CheckSQLDBAHB", + "formatter": 1 + }, + "subtitleContent": { + "columnMatch": "SubscriptionName", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + } + }, + "showBorder": false, + "size": "auto" + }, + "chartSettings": { + "xAxis": "SubscriptionName" + } + }, + "customWidth": "50", + "name": "Summary of SQL DBs with or without AHB " + }, + { + "type": 1, + "content": { + "json": "### Consumed Licenses\r\nTotal number of SQL licenses cores consumed by all SQL Databases\r\n", + "style": "info" + }, + "customWidth": "50", + "name": "Total number of SQL licenses cores consumed" + }, + { + "type": 1, + "content": { + "json": "### Number of required Cores to enable SQL Azure Hybrid Benefit\r\nNumber of cores required to enable SQL AHB across the entire environment.\r\n\r\n\r\n", + "style": "info" + }, + "customWidth": "50", + "name": "Text SQL DB" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"f65bea23-bb49-4498-b331-c20c618071c2\",\"mergeType\":\"table\",\"leftTable\":\"SQLDB AHB Enabled\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"4\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"round([\\\"vCPUs\\\"] / 4) * 4\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}}]},{\"originalName\":\"[SQLDB AHB Enabled].SQLDBID\",\"mergedName\":\"SQLDBID\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].SQLName\",\"mergedName\":\"SQLName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].SQLRG\",\"mergedName\":\"SQLRG\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].SKUName\",\"mergedName\":\"SKUName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].SKUTier\",\"mergedName\":\"SKUTier\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].SQLLocation\",\"mergedName\":\"SQLLocation\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].StorageAccountType\",\"mergedName\":\"StorageAccountType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"unknown\"},{\"originalName\":\"[SQLDB AHB Enabled].SQLDBID1\",\"mergedName\":\"SQLDBID1\",\"fromId\":\"unknown\"}]}", + "size": 0, + "title": "SQL DB AHB Consumed Cores per VM", + "noDataMessage": "None of your SQL DB have AHB enabled.", + "noDataMessageStyle": 4, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ] + }, + "chartSettings": { + "yAxis": [ + "Consumed Cores" + ], + "group": "SQLName", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Summary SQLDB+SKU AHB Enabled - per VM" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"f65bea23-bb49-4498-b331-c20c618071c2\",\"mergeType\":\"table\",\"leftTable\":\"SQLDB AHB Enabled\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"4\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"round([\\\"vCPUs\\\"] / 4) * 4\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}}]},{\"originalName\":\"[SQLDB AHB Enabled].SQLDBID\",\"mergedName\":\"SQLDBID\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].SQLName\",\"mergedName\":\"SQLName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].SQLRG\",\"mergedName\":\"SQLRG\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].SKUName\",\"mergedName\":\"SKUName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].SKUTier\",\"mergedName\":\"SKUTier\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].SQLLocation\",\"mergedName\":\"SQLLocation\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].StorageAccountType\",\"mergedName\":\"StorageAccountType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Enabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"unknown\"},{\"originalName\":\"[SQLDB AHB Enabled].SQLDBID1\",\"mergedName\":\"SQLDBID1\",\"fromId\":\"unknown\"}]}", + "size": 0, + "title": "SQL DB AHB Consumed Cores per Priority", + "noDataMessage": "None of your SQL DB have AHB enabled.", + "noDataMessageStyle": 4, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ] + }, + "chartSettings": { + "yAxis": [ + "Consumed Cores" + ], + "group": "Prioritize AHB?", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Summary SQLDB+SKU AHB Enabled - per Priority" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"f65bea23-bb49-4498-b331-c20c618071c2\",\"mergeType\":\"table\",\"leftTable\":\"SQLDB AHB Disabled\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"4\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"round([\\\"vCPUs\\\"] / 4) * 4\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}}]},{\"originalName\":\"[SQLDB AHB Disabled].SQLDBID\",\"mergedName\":\"SQLDBID\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Disabled].SQLName\",\"mergedName\":\"SQLName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Disabled].SQLRG\",\"mergedName\":\"SQLRG\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Disabled].SKUName\",\"mergedName\":\"SKUName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Disabled].SKUTier\",\"mergedName\":\"SKUTier\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Disabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Disabled].CheckSQLDBAHB\",\"mergedName\":\"CheckSQLDBAHB\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Disabled].SQLLocation\",\"mergedName\":\"SQLLocation\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Disabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Disabled].StorageAccountType\",\"mergedName\":\"StorageAccountType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Disabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLDB AHB Disabled].SQLDBID1\",\"mergedName\":\"SQLDBID1\",\"fromId\":\"unknown\"}]}", + "size": 0, + "title": "SQL DB AHB Cores not enabled per AHB Priority", + "noDataMessage": "All of your SQL DB have AHB enabled.", + "noDataMessageStyle": 3, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ] + }, + "chartSettings": { + "yAxis": [ + "Consumed Cores" + ], + "group": "Prioritize AHB?", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Summary SQLDB+SKU AHB Disabled - per Priority" + } + ] + }, + "name": "SQL DB Info" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "ae5e8765-47ef-46a6-803b-6b7124c098d2", + "version": "KqlParameterItem/1.0", + "name": "SQLDBHUBEnabled", + "label": "See SQL DBs with AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n", + "timeContext": { + "durationMs": 86400000 + } + }, + { + "id": "f1ac5e53-253c-4afb-8bc5-b1ba2efea3eb", + "version": "KqlParameterItem/1.0", + "name": "SQLDBAHBDisabled", + "label": "See SQL DBs without AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n" + } + ], + "style": "pills", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "SQL DB Without AHB" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"f65bea23-bb49-4498-b331-c20c618071ed\",\"mergeType\":\"table\",\"leftTable\":\"SQLDB AHB Disabled\"}],\"projectRename\":[{\"originalName\":\"[SQLDB AHB Disabled].SQLDBID\",\"mergedName\":\"SQLDBID\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLDB AHB Disabled].SQLName\",\"mergedName\":\"SQLName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLDB AHB Disabled].SQLRG\",\"mergedName\":\"SQLRG\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLDB AHB Disabled].SKUName\",\"mergedName\":\"SKUName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLDB AHB Disabled].SKUTier\",\"mergedName\":\"SKUTier\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLDB AHB Disabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLDB AHB Disabled].CheckSQLDBAHB\",\"mergedName\":\"CheckSQLDBAHB\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLDB AHB Disabled].SQLLocation\",\"mergedName\":\"SQLLocation\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLDB AHB Disabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLDB AHB Disabled].StorageAccountType\",\"mergedName\":\"StorageAccountType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLDB AHB Disabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLDB AHB Disabled].SQLDBID1\",\"mergedName\":\"SQLDBID1\",\"fromId\":\"unknown\"}]}", + "size": 0, + "title": "SQL DB AHB Disabled", + "noDataMessage": "All of your SQL DBs have AHB enabled.", + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "SubscriptionName", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "SubscriptionName" + ], + "expandTopLevel": true + }, + "labelSettings": [ + { + "columnId": "SQLDBID", + "label": "Database Name" + }, + { + "columnId": "SQLName", + "label": "Server Name" + }, + { + "columnId": "SQLRG", + "label": "Resource Group" + }, + { + "columnId": "SKUName", + "label": "SKU" + }, + { + "columnId": "SKUTier", + "label": "SKU Tier" + }, + { + "columnId": "vCores", + "label": "Number of vCore" + }, + { + "columnId": "CheckSQLDBAHB", + "label": "Is AHB enabled?" + }, + { + "columnId": "SQLLocation", + "label": "Location" + }, + { + "columnId": "LicenseType", + "label": "License Type" + }, + { + "columnId": "StorageAccountType", + "label": "Storage Account Type" + }, + { + "columnId": "SubscriptionName", + "label": "Subscription Name" + }, + { + "columnId": "SQLDBID1", + "label": "Resource ID" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "SQLDBAHBDisabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "SQL DB Disabled" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"f65bea23-bb49-4498-b331-c20c618071f9\",\"mergeType\":\"table\",\"leftTable\":\"SQLDB AHB Enabled\"}],\"projectRename\":[{\"originalName\":\"[SQLDB AHB Enabled].SQLDBID\",\"mergedName\":\"SQLDBID\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLDB AHB Enabled].SQLName\",\"mergedName\":\"SQLName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLDB AHB Enabled].SQLRG\",\"mergedName\":\"SQLRG\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLDB AHB Enabled].SKUName\",\"mergedName\":\"SKUName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLDB AHB Enabled].SKUTier\",\"mergedName\":\"SKUTier\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLDB AHB Enabled].SQLLocation\",\"mergedName\":\"SQLLocation\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLDB AHB Enabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLDB AHB Enabled].StorageAccountType\",\"mergedName\":\"StorageAccountType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLDB AHB Enabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLDB AHB Enabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"unknown\"},{\"originalName\":\"[SQLDB AHB Enabled].SQLDBID1\",\"mergedName\":\"SQLDBID1\",\"fromId\":\"unknown\"}]}", + "size": 0, + "title": "SQL DB AHB Enabled", + "noDataMessage": "None of you SQL DBs have AHB enabled.", + "noDataMessageStyle": 4, + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "labelSettings": [ + { + "columnId": "SQLDBID", + "label": "Name" + }, + { + "columnId": "SQLName", + "label": "Database Name" + }, + { + "columnId": "SQLRG", + "label": "Resource Group" + }, + { + "columnId": "SKUName", + "label": "SKU" + }, + { + "columnId": "SKUTier", + "label": "SKU Tier" + }, + { + "columnId": "SQLLocation", + "label": "Location" + }, + { + "columnId": "LicenseType", + "label": "License Type" + }, + { + "columnId": "StorageAccountType", + "label": "Storage Account Type" + }, + { + "columnId": "SubscriptionName", + "label": "Subscription Name" + }, + { + "columnId": "vCores", + "label": "Number of vCore" + }, + { + "columnId": "SQLDBID1", + "label": "Resource ID" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "SQLDBHUBEnabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "SQL DB AHB Enabled" + } + ] + }, + "name": "Load SQL DB Detailed Info" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SQLType", + "comparison": "isEqualTo", + "value": "SQLDatabase" + }, + "name": "SQLDatabase" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "SQL Elastic Pool" + }, + "name": "text - 0" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and tostring(properties.['licenseType']) == 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n )\r\n) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, CheckSQLDBAHB,SQLLocation, LicenseType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n", + "size": 0, + "title": "AHB Disabled", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "SQLElastic AHB Disabled" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and tostring(properties.['licenseType']) != 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, SQLLocation, LicenseType, CheckSQLDBAHB, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n\r\n", + "size": 0, + "title": "AHB Enabled", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "SQLElastic AHB Enabled" + }, + { + "type": 1, + "content": { + "json": "Apply to SQL Server 1 to 4 vCPUs exchange: For every 1 core of SQL Server Enterprise Edition, you get 4 vCPUs of SQL Managed Instance or Azure SQL Database general purpose and Hyperscale tiers, or 4 vCPUs of SQL Server Standard edition on Azure VMs.\r\n\r\nThe SQL virtual machines (VMs) with less than 4 cores are categorized as **Low Priority**, while those with 8 or more cores are classified as **High Priority**. In situations where there are insufficient Azure Hybrid benefit licenses to cover all the VMs in the environment, it is recommended to prioritize the High Priority VMs.\r\n\r\nFor Azure SQL Database, Azure Hybrid Benefit is only available when using the provisioned compute tier of the vCore-based purchasing model. Azure Hybrid Benefit doesn't apply to DTU-based purchasing models or the serverless compute tier.", + "style": "info" + }, + "name": "Apply to SQL Elastic Server 1 to 4 vCPUs " + }, + { + "type": 1, + "content": { + "json": "### AHB Overview\r\nSummary of all SQL Databases with and without SQL AHB.", + "style": "info" + }, + "name": " AHB Overview SQL Elastic" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "loadType": "explicit", + "loadButtonText": "Load SQL DB Info", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by SubscriptionName, CheckSQLDBAHB", + "size": 0, + "title": "Summary of SQL Databases with or without AHB per Subscription", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "table", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ], + "labelSettings": [ + { + "columnId": "SubscriptionName", + "label": "Subscription Name" + }, + { + "columnId": "CheckSQLDBAHB", + "label": "Is AHB enabled?" + }, + { + "columnId": "count_", + "label": "Number of resources" + } + ] + }, + "tileSettings": { + "titleContent": { + "columnMatch": "CheckSQLDBAHB", + "formatter": 1 + }, + "subtitleContent": { + "columnMatch": "SubscriptionName", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + } + }, + "showBorder": false, + "size": "auto" + }, + "chartSettings": { + "xAxis": "SubscriptionName" + } + }, + "customWidth": "50", + "name": "Summary of SQL Elastic with or without AHB per subs" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by CheckSQLDBAHB", + "size": 0, + "title": "Summary of SQL Databases with or without AHB", + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ] + }, + "tileSettings": { + "titleContent": { + "columnMatch": "CheckSQLDBAHB", + "formatter": 1 + }, + "subtitleContent": { + "columnMatch": "SubscriptionName", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + } + }, + "showBorder": false, + "size": "auto" + }, + "chartSettings": { + "xAxis": "SubscriptionName" + } + }, + "customWidth": "50", + "name": "Summary of SQL DBs with or without AHB " + }, + { + "type": 1, + "content": { + "json": "### Consumed Licenses\r\nTotal number of SQL licenses cores consumed by all SQL Databases\r\n", + "style": "info" + }, + "customWidth": "50", + "name": "Total number of SQL licenses cores consumed" + }, + { + "type": 1, + "content": { + "json": "### Number of required Cores to enable SQL Azure Hybrid Benefit\r\nNumber of cores required to enable SQL AHB across the entire environment.\r\n\r\n\r\n", + "style": "info" + }, + "customWidth": "50", + "name": "Text SQL DB" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"f65bea23-bb49-4498-b331-c20c618071c2\",\"mergeType\":\"table\",\"leftTable\":\"SQLElastic AHB Enabled\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"4\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"round([\\\"vCPUs\\\"] / 4) * 4\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}}]},{\"originalName\":\"[SQLElastic AHB Enabled].SQLDBID\",\"mergedName\":\"SQLDBID\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].SQLName\",\"mergedName\":\"SQLName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].SQLRG\",\"mergedName\":\"SQLRG\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].SKUName\",\"mergedName\":\"SKUName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].SKUTier\",\"mergedName\":\"SKUTier\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].SQLLocation\",\"mergedName\":\"SQLLocation\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].CheckSQLDBAHB\",\"mergedName\":\"CheckSQLDBAHB\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].SQLDBID1\",\"mergedName\":\"SQLDBID1\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"}]}", + "size": 0, + "title": "SQL DB Elastic Pools AHB Consumed Cores per VM", + "noDataMessage": "None of your SQL DB Elastic Pools have AHB enabled.", + "noDataMessageStyle": 4, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ] + }, + "chartSettings": { + "yAxis": [ + "vCores" + ], + "group": "SQLName", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Summary SQLElastic+SKU AHB Enabled - per VM" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"f65bea23-bb49-4498-b331-c20c618071c2\",\"mergeType\":\"table\",\"leftTable\":\"SQLElastic AHB Enabled\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"4\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"round([\\\"vCPUs\\\"] / 4) * 4\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}}]},{\"originalName\":\"[SQLElastic AHB Enabled].SQLDBID\",\"mergedName\":\"SQLDBID\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].SQLName\",\"mergedName\":\"SQLName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].SQLRG\",\"mergedName\":\"SQLRG\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].SKUName\",\"mergedName\":\"SKUName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].SKUTier\",\"mergedName\":\"SKUTier\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].SQLLocation\",\"mergedName\":\"SQLLocation\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].CheckSQLDBAHB\",\"mergedName\":\"CheckSQLDBAHB\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Enabled].SQLDBID1\",\"mergedName\":\"SQLDBID1\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"}]}", + "size": 0, + "title": "SQL Elastic AHB Consumed Cores per Priority", + "noDataMessage": "None of your SQL DB Elastic Pools have AHB enabled.", + "noDataMessageStyle": 4, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ] + }, + "chartSettings": { + "yAxis": [ + "vCores" + ], + "group": "Prioritize AHB?", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Summary SQLElastic+SKU AHB Enabled - per Priority" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"f65bea23-bb49-4498-b331-c20c618071c2\",\"mergeType\":\"table\",\"leftTable\":\"SQLElastic AHB Disabled\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"4\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"round([\\\"vCPUs\\\"] / 4) * 4\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}}]},{\"originalName\":\"[SQLElastic AHB Disabled].SQLDBID\",\"mergedName\":\"SQLDBID\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Disabled].SQLName\",\"mergedName\":\"SQLName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Disabled].SQLRG\",\"mergedName\":\"SQLRG\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Disabled].SKUName\",\"mergedName\":\"SKUName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Disabled].SKUTier\",\"mergedName\":\"SKUTier\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Disabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Disabled].CheckSQLDBAHB\",\"mergedName\":\"CheckSQLDBAHB\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Disabled].SQLLocation\",\"mergedName\":\"SQLLocation\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Disabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Disabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLElastic AHB Disabled].SQLDBID1\",\"mergedName\":\"SQLDBID1\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"}]}", + "size": 0, + "title": "SQL DB AHB Cores not enabled per AHB Priority", + "noDataMessage": "All of your SQL DB have AHB enabled.", + "noDataMessageStyle": 3, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ] + }, + "chartSettings": { + "yAxis": [ + "vCores" + ], + "group": "Prioritize AHB?", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Summary SQLDB+SKU AHB Disabled - per Priority" + } + ] + }, + "name": "SQL Elastic Info" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "ae5e8765-47ef-46a6-803b-6b7124c098d2", + "version": "KqlParameterItem/1.0", + "name": "SQLDBHUBEnabled", + "label": "See SQL DBs with AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n", + "timeContext": { + "durationMs": 86400000 + } + }, + { + "id": "f1ac5e53-253c-4afb-8bc5-b1ba2efea3eb", + "version": "KqlParameterItem/1.0", + "name": "SQLDBAHBDisabled", + "label": "See SQL DBs without AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n" + } + ], + "style": "pills", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "SQL DB Without AHB" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"f65bea23-bb49-4498-b331-c20c618071ed\",\"mergeType\":\"table\",\"leftTable\":\"SQLElastic AHB Disabled\"}],\"projectRename\":[{\"originalName\":\"[SQLDB AHB Disabled].SQLDBID1\",\"mergedName\":\"SQLDBID1\",\"fromId\":\"unknown\"},{\"originalName\":\"[SQLElastic AHB Disabled].SQLDBID\",\"mergedName\":\"SQLDBID\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLElastic AHB Disabled].SQLName\",\"mergedName\":\"SQLName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLElastic AHB Disabled].SQLRG\",\"mergedName\":\"SQLRG\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLElastic AHB Disabled].SKUName\",\"mergedName\":\"SKUName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLElastic AHB Disabled].SKUTier\",\"mergedName\":\"SKUTier\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLElastic AHB Disabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLElastic AHB Disabled].CheckSQLDBAHB\",\"mergedName\":\"CheckSQLDBAHB\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLElastic AHB Disabled].SQLLocation\",\"mergedName\":\"SQLLocation\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLElastic AHB Disabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLElastic AHB Disabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLElastic AHB Disabled].SQLDBID1\",\"mergedName\":\"SQLDBID1\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"}]}", + "size": 0, + "title": "SQL DB AHB Disabled", + "noDataMessage": "All of your SQL DBs have AHB enabled.", + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "SubscriptionName", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "SubscriptionName" + ], + "expandTopLevel": true + }, + "labelSettings": [ + { + "columnId": "SQLDBID", + "label": "Database Name" + }, + { + "columnId": "SQLName", + "label": "Server Name" + }, + { + "columnId": "SQLRG", + "label": "Resource Group" + }, + { + "columnId": "SKUName", + "label": "SKU" + }, + { + "columnId": "SKUTier", + "label": "SKU Tier" + }, + { + "columnId": "vCores", + "label": "Number of vCore" + }, + { + "columnId": "CheckSQLDBAHB", + "label": "Is AHB enabled?" + }, + { + "columnId": "SQLLocation", + "label": "Location" + }, + { + "columnId": "LicenseType", + "label": "License Type" + }, + { + "columnId": "SubscriptionName", + "label": "Subscription Name" + }, + { + "columnId": "SQLDBID1", + "label": "Resource ID" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "SQLDBAHBDisabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "SQL DB Disabled" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"f65bea23-bb49-4498-b331-c20c618071f9\",\"mergeType\":\"table\",\"leftTable\":\"SQLElastic AHB Enabled\"}],\"projectRename\":[{\"originalName\":\"[SQLDB AHB Enabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"unknown\"},{\"originalName\":\"[SQLDB AHB Enabled].SQLDBID1\",\"mergedName\":\"SQLDBID1\",\"fromId\":\"unknown\"},{\"originalName\":\"[SQLElastic AHB Enabled].SQLDBID\",\"mergedName\":\"SQLDBID\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLElastic AHB Enabled].SQLName\",\"mergedName\":\"SQLName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLElastic AHB Enabled].SQLRG\",\"mergedName\":\"SQLRG\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLElastic AHB Enabled].SKUName\",\"mergedName\":\"SKUName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLElastic AHB Enabled].SKUTier\",\"mergedName\":\"SKUTier\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLElastic AHB Enabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLElastic AHB Enabled].SQLLocation\",\"mergedName\":\"SQLLocation\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLElastic AHB Enabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLElastic AHB Enabled].CheckSQLDBAHB\",\"mergedName\":\"CheckSQLDBAHB\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLElastic AHB Enabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLElastic AHB Enabled].SQLDBID1\",\"mergedName\":\"SQLDBID1\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"}]}", + "size": 0, + "title": "SQL DB AHB Enabled", + "noDataMessage": "None of you SQL DBs have AHB enabled.", + "noDataMessageStyle": 4, + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "labelSettings": [ + { + "columnId": "SQLDBID", + "label": "Name" + }, + { + "columnId": "SQLName", + "label": "Database Name" + }, + { + "columnId": "SQLRG", + "label": "Resource Group" + }, + { + "columnId": "SKUName", + "label": "SKU" + }, + { + "columnId": "SKUTier", + "label": "SKU Tier" + }, + { + "columnId": "vCores", + "label": "Number of vCore" + }, + { + "columnId": "SQLLocation", + "label": "Location" + }, + { + "columnId": "LicenseType", + "label": "License Type" + }, + { + "columnId": "SubscriptionName", + "label": "Subscription Name" + }, + { + "columnId": "SQLDBID1", + "label": "Resource ID" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "SQLDBHUBEnabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "SQL DB AHB Enabled" + } + ] + }, + "name": "Load SQL DB Detailed Info" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SQLType", + "comparison": "isEqualTo", + "value": "SQLElastic" + }, + "name": "SQLElasticPool" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "SQLDB" + }, + "name": "SQLDBGroup" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "title": "SQL Managed Instance", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' and tostring(properties.['licenseType']) == 'LicenseIncluded'\r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n | project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n | where SQLRG in ({ResourceGroup})\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance", + "size": 0, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "SQLMIAHBDisabled" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' and tostring(properties.['licenseType']) != 'LicenseIncluded'\r\n | extend ManagedInstance=id, SQLName=name, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n | project ManagedInstance, SQLName, SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n | where SQLRG in ({ResourceGroup})\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n ", + "size": 0, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "SQLMIAHBEnabled" + }, + { + "type": 1, + "content": { + "json": "# SQL Managed Instances Azure Hybrid Benefit (AHB) Overview\r\n" + }, + "name": "SQL Managed Instances AHB" + }, + { + "type": 1, + "content": { + "json": "Apply to SQL Server 1 to 4 vCPUs exchange: For every 1 core of SQL Server Enterprise Edition, you get 4 vCPUs of SQL Managed Instance or Azure SQL Database general purpose and Hyperscale tiers, or 4 vCPUs of SQL Server Standard edition on Azure VMs.\r\n\r\nThe SQL virtual machines (VMs) with less than 4 cores are categorized as **Low Priority**, while those with 8 or more cores are classified as **High Priority**. In situations where there are insufficient Azure Hybrid benefit licenses to cover all the VMs in the environment, it is recommended to prioritize the High Priority VMs.", + "style": "info" + }, + "name": "Apply to SQL Server 1 to 4 vCPUs exchange" + }, + { + "type": 1, + "content": { + "json": "### AHB Overview\r\nSummary of all SQL Databases with and without SQL AHB.", + "style": "info" + }, + "name": "SQL Databases with and without SQL AHB." + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "loadType": "explicit", + "loadButtonText": "Load SQL MI Info", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' \r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n| summarize count() by SubscriptionName, CheckSQLMIAHB", + "size": 0, + "title": "Summary of SQL MI with or without AHB per Subscription", + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "table", + "tileSettings": { + "titleContent": { + "columnMatch": "CheckSQLMIAHB", + "formatter": 1 + }, + "subtitleContent": { + "columnMatch": "SubscriptionName" + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + }, + "showBorder": false + }, + "chartSettings": { + "yAxis": [ + "count_" + ], + "group": "CheckSQLMIAHB", + "createOtherGroup": null + } + }, + "customWidth": "50", + "name": "Summary of SQL MI with or without AHB per subs" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" \r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances'\r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n| summarize count() by SubscriptionName, CheckSQLMIAHB", + "size": 0, + "title": "Summary of SQL Managed Instance with or without AHB", + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart", + "chartSettings": { + "yAxis": [ + "count_" + ], + "group": "CheckSQLMIAHB", + "createOtherGroup": null + } + }, + "customWidth": "50", + "name": "Summary of SQL MI with or without AHB " + }, + { + "type": 1, + "content": { + "json": "### Consumed Licenses\r\nTotal number of SQL licenses cores consumed by all SQL Managed Instances.\r\n", + "style": "info" + }, + "customWidth": "50", + "name": "Consumed Licenses" + }, + { + "type": 1, + "content": { + "json": "### Number of required Cores to enable SQL Azure Hybrid Benefit\r\nNumber of cores required to enable SQL AHB across the entire environment.\r\n\r\n\r\n", + "style": "info" + }, + "customWidth": "50", + "name": "Number of required Cores to enable SQL " + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"f65bea23-bb49-4498-b331-c20c618071c2\",\"mergeType\":\"table\",\"leftTable\":\"SQLMIAHBEnabled\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"4\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"round([\\\"vCPUs\\\"] / 4) * 4\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}}]},{\"originalName\":\"[SQLDB AHB Enabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"unknown\"},{\"originalName\":\"[SQLMIAHBEnabled].ManagedInstance\",\"mergedName\":\"ManagedInstance\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBEnabled].SQLRG\",\"mergedName\":\"SQLRG\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBEnabled].SQLLocation\",\"mergedName\":\"SQLLocation\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBEnabled].CheckSQLMIAHB\",\"mergedName\":\"CheckSQLMIAHB\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBEnabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBEnabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBEnabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBEnabled].ManagedInstance1\",\"mergedName\":\"ManagedInstance1\",\"fromId\":\"unknown\"},{\"originalName\":\"[SQLMIAHBEnabled].SQLName\",\"mergedName\":\"SQLName\",\"fromId\":\"unknown\"}]}", + "size": 0, + "title": "SQL Managed Instance AHB Consumed Cores per VM", + "noDataMessage": "None of your SQL MI have AHB enabled.", + "noDataMessageStyle": 4, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ] + }, + "chartSettings": { + "yAxis": [ + "vCores" + ], + "group": "SQLName", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Summary SQLMI+SKU AHB Enabled - per VM" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"f65bea23-bb49-4498-b331-c20c618071c2\",\"mergeType\":\"table\",\"leftTable\":\"SQLMIAHBEnabled\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"4\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"round([\\\"vCPUs\\\"] / 4) * 4\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}}]},{\"originalName\":\"[SQLDB AHB Enabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"unknown\"},{\"originalName\":\"[SQLMIAHBEnabled].ManagedInstance\",\"mergedName\":\"ManagedInstance\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBEnabled].SQLRG\",\"mergedName\":\"SQLRG\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBEnabled].SQLLocation\",\"mergedName\":\"SQLLocation\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBEnabled].CheckSQLMIAHB\",\"mergedName\":\"CheckSQLMIAHB\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBEnabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBEnabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBEnabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBEnabled].ManagedInstance1\",\"mergedName\":\"ManagedInstance1\",\"fromId\":\"unknown\"},{\"originalName\":\"[SQLMIAHBEnabled].SQLName\",\"mergedName\":\"SQLName\",\"fromId\":\"unknown\"}]}", + "size": 0, + "title": "SQL Managed Instance AHB Consumed Cores per Priority", + "noDataMessage": "None of your SQL MI have AHB enabled.", + "noDataMessageStyle": 4, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ] + }, + "chartSettings": { + "yAxis": [ + "vCores" + ], + "group": "Prioritize AHB?", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Summary SQLMI+SKU AHB Enabled - per Priority" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"f65bea23-bb49-4498-b331-c20c618071c2\",\"mergeType\":\"table\",\"leftTable\":\"SQLMIAHBDisabled\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"4\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"round([\\\"vCPUs\\\"] / 4) * 4\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCores\",\"operator\":\"<\",\"rightValType\":\"static\",\"rightVal\":\"4\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}}]},{\"originalName\":\"[SQLMIAHBDisabled].ManagedInstance\",\"mergedName\":\"ManagedInstance\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBDisabled].SQLRG\",\"mergedName\":\"SQLRG\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBDisabled].SQLLocation\",\"mergedName\":\"SQLLocation\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBDisabled].CheckSQLMIAHB\",\"mergedName\":\"CheckSQLMIAHB\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBDisabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBDisabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBDisabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071c2\"},{\"originalName\":\"[SQLMIAHBDisabled].ManagedInstance1\",\"mergedName\":\"ManagedInstance1\",\"fromId\":\"unknown\"}]}", + "size": 0, + "title": "SQL Managed Instances AHB Cores not enabled per AHB Priority", + "noDataMessage": "All of your SQL MI have AHB enabled.", + "noDataMessageStyle": 3, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low Priority", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "unknown", + "text": "{0}{1}" + } + ] + } + } + ] + }, + "chartSettings": { + "yAxis": [ + "vCores" + ], + "group": "Prioritize AHB?", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Summary SQLMI+SKU AHB Disabled - per Priority" + } + ] + }, + "name": "SQL MI Info" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "ae5e8765-47ef-46a6-803b-6b7124c098d2", + "version": "KqlParameterItem/1.0", + "name": "SQLMIAHBEnabled", + "label": "See SQL MIs with AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n", + "timeContext": { + "durationMs": 86400000 + } + }, + { + "id": "f1ac5e53-253c-4afb-8bc5-b1ba2efea3eb", + "version": "KqlParameterItem/1.0", + "name": "SQLMIAHBDisabled", + "label": "See SQL MIs without AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n" + } + ], + "style": "pills", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "SQL MI AHB Disabled" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"f65bea23-bb49-4498-b331-c20c618071ed\",\"mergeType\":\"table\",\"leftTable\":\"SQLMIAHBDisabled\"}],\"projectRename\":[{\"originalName\":\"[SQLMIAHBDisabled].ManagedInstance\",\"mergedName\":\"ManagedInstance\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLMIAHBDisabled].SQLRG\",\"mergedName\":\"SQLRG\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLMIAHBDisabled].SQLLocation\",\"mergedName\":\"SQLLocation\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLMIAHBDisabled].CheckSQLMIAHB\",\"mergedName\":\"CheckSQLMIAHB\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLMIAHBDisabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLMIAHBDisabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"},{\"originalName\":\"[SQLMIAHBDisabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071ed\"}]}", + "size": 0, + "title": "SQL Managed Instance AHB Disabled", + "noDataMessage": "All of your SQL MIs have AHB enabled.", + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "SubscriptionName", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "SubscriptionName" + ], + "expandTopLevel": true + } + } + }, + "conditionalVisibility": { + "parameterName": "SQLMIAHBDisabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "SQL MI Disabled" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"f65bea23-bb49-4498-b331-c20c618071f9\",\"mergeType\":\"table\",\"leftTable\":\"SQLMIAHBEnabled\"}],\"projectRename\":[{\"originalName\":\"[SQLDB AHB Enabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"unknown\"},{\"originalName\":\"[SQLMIAHBEnabled].ManagedInstance\",\"mergedName\":\"ManagedInstance\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLMIAHBEnabled].SQLRG\",\"mergedName\":\"SQLRG\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLMIAHBEnabled].SQLLocation\",\"mergedName\":\"SQLLocation\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLMIAHBEnabled].CheckSQLMIAHB\",\"mergedName\":\"CheckSQLMIAHB\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLMIAHBEnabled].vCores\",\"mergedName\":\"vCores\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLMIAHBEnabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"},{\"originalName\":\"[SQLMIAHBEnabled].SubscriptionName\",\"mergedName\":\"SubscriptionName\",\"fromId\":\"f65bea23-bb49-4498-b331-c20c618071f9\"}]}", + "size": 0, + "title": "SQL Managed Instance AHB Enabled", + "noDataMessage": "None of you SQL MIs have AHB enabled.", + "showExportToExcel": true, + "queryType": 7 + }, + "conditionalVisibility": { + "parameterName": "SQLMIAHBEnabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "SQL MI AHB Enabled" + } + ] + }, + "name": "SQL MI Detailed" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "SQLMI" + }, + "name": "SQL MI" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "SQL" + }, + "name": "SQLAHB" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "title": "Linux Hybrid Benefit", + "loadType": "explicit", + "loadButtonText": "Load Linux Recommendations", + "items": [ + { + "type": 1, + "content": { + "json": "## Linux Azure Hybrid Benefit (AHB) Overview" + }, + "name": "Linux Text" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"ARMEndpoint/1.0\",\"data\":null,\"headers\":[],\"method\":\"GET\",\"path\":\"/subscriptions/{SingleSubHidden}/providers/Microsoft.Compute/skus?$filter=location eq '{Location}'\",\"urlParams\":[{\"key\":\"api-version\",\"value\":\"2021-07-01\"}],\"batchDisabled\":false,\"transformers\":[{\"type\":\"jsonpath\",\"settings\":{\"tablePath\":\"$.*[?(@.resourceType=='virtualMachines')]\",\"columns\":[{\"path\":\"name\",\"columnid\":\"Name\"},{\"path\":\"capabilities[?(@.name=='vCPUs')].value\",\"columnid\":\"vCPUs\"},{\"path\":\"capabilities[?(@.name=='MemoryGB')].value\",\"columnid\":\"MemoryGB\"},{\"path\":\"capabilities[?(@.name=='MaxNetworkInterfaces')].value\",\"columnid\":\"MaxNetworkInterfaces\"},{\"path\":\"capabilities[?(@.name=='HyperVGenerations')].value\",\"columnid\":\"HyperVGenerations\"},{\"path\":\"capabilities[?(@.name=='vCPUsPerCore')].value\",\"columnid\":\"vCPUsPerCore\"}]}}]}", + "size": 0, + "title": "Get VM vCPU", + "exportParameterName": "ResourceSKU", + "showExportToExcel": true, + "queryType": 12, + "gridSettings": { + "rowLimit": 5000 + } + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "API-Get_VMLinux_SKU" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where resourceGroup in ({ResourceGroup})\r\n| where type =~ 'microsoft.compute/virtualmachines' and (properties.storageProfile.imageReference.publisher == 'suse' or properties.storageProfile.imageReference.publisher=='RedHat')\r\n| where isnull ((properties.['licenseType']))\r\n| extend LinuxId=id, VMName=name, VMLocation=location, VMRG=resourceGroup, OSType=tostring(properties.storageProfile.imageReference.publisher), OsVersion = tostring(properties.storageProfile.imageReference.sku), VMSize=tostring (properties.hardwareProfile.vmSize), LicenseType = tostring(properties.['licenseType']), VMSSize=tostring(sku.name)\r\n| order by type asc \r\n| project LinuxId,VMName,VMRG,VMSize, VMSSize, VMLocation,OSType, OsVersion,LicenseType\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), LinuxId=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct LinuxId\r\n )\r\n on LinuxId", + "size": 0, + "title": "AHB Disabled", + "noDataMessage": "None of your Linux VMs have AHB enabled.", + "noDataMessageStyle": 4, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "LinuxAHBDisabled" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where resourceGroup in ({ResourceGroup})\r\n| where type =~ 'microsoft.compute/virtualmachines' and (properties.storageProfile.imageReference.publisher == 'suse' or properties.storageProfile.imageReference.publisher=='RedHat')\r\n| where isnotnull ((properties.['licenseType']))\r\n| extend LinuxId=id, VMName=name, VMLocation=location, VMRG=resourceGroup, OSType=tostring(properties.storageProfile.imageReference.publisher), OsVersion = tostring(properties.storageProfile.imageReference.sku), VMSize=tostring (properties.hardwareProfile.vmSize), LicenseType = tostring(properties.['licenseType']), VMSSize=tostring(sku.name)\r\n| order by type asc \r\n| project LinuxId,VMName,VMRG,VMSize, VMSSize, VMLocation,OSType, OsVersion,LicenseType\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), LinuxId=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct LinuxId\r\n )\r\n on LinuxId", + "size": 0, + "title": "AHB Enabled", + "noDataMessage": "All of your Linux VMs have AHB enabled.", + "noDataMessageStyle": 3, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibility": { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "true" + }, + "name": "LinuxAHBRGEnabled" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where resourceGroup in ({ResourceGroup})\r\n| where type =~ 'microsoft.compute/virtualmachines' and (properties.storageProfile.imageReference.publisher == 'suse' or properties.storageProfile.imageReference.publisher=='RedHat')\r\n| extend LicenseType = tostring(properties.['licenseType'])\r\n| extend LinuxId=id\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), LinuxId=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct LinuxId\r\n )\r\n on LinuxId\r\n| extend CheckAHBLinux = case(\r\n type == 'microsoft.compute/virtualmachines' or type =~ 'microsoft.compute/virtualMachineScaleSets',\r\n iff(isnull((properties.['licenseType'])),\r\n \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not Linux\"\r\n )\r\n| summarize count() by CheckAHBLinux", + "size": 0, + "title": "Summary of Linux VMs with or without AHB", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart" + }, + "customWidth": "50", + "name": "Summary of Linux VMs with or without AHB" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"d8deb22b-a596-43ee-acc4-180849d26130\",\"mergeType\":\"inner\",\"leftTable\":\"LinuxAHBRGEnabled\",\"rightTable\":\"API-Get_VMLinux_SKU\",\"leftColumn\":\"VMSize\",\"rightColumn\":\"Name\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"ConsumedCores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"8\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"[\\\"vCPUs\\\"]\"}}]},{\"originalName\":\"[WindowsAHBEnabled].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[WindowsAHBEnabled].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"unknown\"},{\"originalName\":\"[LinuxAHBRGEnabled].LinuxId\",\"mergedName\":\"LinuxId\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[LinuxAHBRGEnabled].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[LinuxAHBRGEnabled].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[LinuxAHBRGEnabled].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[LinuxAHBRGEnabled].VMSSize\",\"mergedName\":\"VMSSize\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[LinuxAHBRGEnabled].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[LinuxAHBRGEnabled].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[LinuxAHBRGEnabled].OsVersion\",\"mergedName\":\"OsVersion\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[LinuxAHBRGEnabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[LinuxAHBRGEnabled].LinuxId1\",\"mergedName\":\"LinuxId1\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[API-Get_VMLinux_SKU].Name\",\"mergedName\":\"Name\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[API-Get_VMLinux_SKU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[API-Get_VMLinux_SKU].MemoryGB\",\"mergedName\":\"MemoryGB\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[API-Get_VMLinux_SKU].MaxNetworkInterfaces\",\"mergedName\":\"MaxNetworkInterfaces\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[API-Get_VMLinux_SKU].HyperVGenerations\",\"mergedName\":\"HyperVGenerations\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[API-Get_VMLinux_SKU].vCPUsPerCore\",\"mergedName\":\"vCPUsPerCore\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"}]}", + "size": 0, + "title": "Consumed Cores per VM", + "noDataMessage": "None of your Linux VM have AHB enabled", + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "ConsumedCores", + "formatter": 0, + "formatOptions": { + "aggregation": "Sum" + } + } + ] + }, + "tileSettings": { + "titleContent": {}, + "leftContent": { + "columnMatch": "ConsumedCores", + "formatter": 12, + "formatOptions": { + "palette": "blue" + } + }, + "showBorder": false + }, + "graphSettings": { + "type": 0 + }, + "chartSettings": { + "yAxis": [ + "ConsumedCores" + ], + "group": "VMName", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Linux Consumed Cores per VM" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "ae5e8765-47ef-46a6-803b-6b7124c098d2", + "version": "KqlParameterItem/1.0", + "name": "LinuxAHBEnabled", + "label": "See Linux VMs with AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n", + "timeContext": { + "durationMs": 86400000 + } + }, + { + "id": "4c3ff9fa-d9c8-4d35-94d4-48ba3a1547fd", + "version": "KqlParameterItem/1.0", + "name": "LinuxAHBDisabled", + "label": "See Linux VMs without AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [] + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n", + "timeContext": { + "durationMs": 86400000 + } + } + ], + "style": "pills", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "Linux VMs without AHB" + }, + { + "type": 1, + "content": { + "json": "List of Linux VMs with Hybrid Benefit groupped by Subscription." + }, + "conditionalVisibility": { + "parameterName": "LinuxAHBEnabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "Linux VMs with Hybrid Benefit" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\",\"mergeType\":\"inner\",\"leftTable\":\"LinuxAHBRGEnabled\",\"rightTable\":\"API-Get_VMLinux_SKU\",\"leftColumn\":\"VMSize\",\"rightColumn\":\"Name\"}],\"projectRename\":[{\"originalName\":\"[LinuxAHBRGEnabled].LinuxId\",\"mergedName\":\"VM ID\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBRGEnabled].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBRGEnabled].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[API-Get_VMLinux_SKU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBRGEnabled].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBRGEnabled].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBRGEnabled].OsVersion\",\"mergedName\":\"OsVersion\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBRGEnabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[API-Get_VMLinux_SKU].Name\",\"mergedName\":\"Name\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[API-Get_VMLinux_SKU].MemoryGB\",\"mergedName\":\"MemoryGB\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[API-Get_VMLinux_SKU].MaxNetworkInterfaces\",\"mergedName\":\"MaxNetworkInterfaces\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[API-Get_VMLinux_SKU].HyperVGenerations\",\"mergedName\":\"HyperVGenerations\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[API-Get_VMLinux_SKU].vCPUsPerCore\",\"mergedName\":\"vCPUsPerCore\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBRGEnabled].LinuxId1\",\"mergedName\":\"LinuxId1\",\"fromId\":\"unknown\"},{\"originalName\":\"[LinuxAHBEnabled].VMName\"},{\"originalName\":\"[API-Get_VM_SKU].MemoryGB\"},{\"originalName\":\"[API-Get_VM_SKU].MaxNetworkInterfaces\"},{\"originalName\":\"[API-Get_VM_SKU].Name\"},{\"originalName\":\"[API-Get_VM_SKU].HyperVGenerations\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUsPerCore\"},{\"originalName\":\"[LinuxAHBEnabled].VMSSize\"},{\"originalName\":\"[LinuxAHBRGEnabled].VMName\"},{\"originalName\":\"[LinuxAHBRGEnabled].VMSSize\"}]}", + "size": 0, + "title": "Linux VMs with AHB", + "noDataMessage": "None of your Linux VMs have AHB enabled", + "noDataMessageStyle": 4, + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Sev4", + "text": "{0}{1}" + } + ] + }, + "numberFormat": { + "unit": 17, + "options": { + "style": "decimal" + } + } + } + ] + }, + "tileSettings": { + "showBorder": false, + "titleContent": { + "columnMatch": "NewLicense", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "vCPUs", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + }, + "chartSettings": { + "xAxis": "VM Name", + "yAxis": [ + "Consumed Cores per VM" + ], + "group": null, + "createOtherGroup": 0, + "seriesLabelSettings": [ + { + "seriesName": "Consumed Cores per VM", + "color": "grayBlue" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "LinuxAHBEnabled", + "comparison": "isEqualTo", + "value": "yes" + }, + "name": "Linux-VM+SKU+vCores-AHB" + }, + { + "type": 1, + "content": { + "json": "List of Linux VMs without Hybrid Benefit groupped by Subscription." + }, + "conditionalVisibility": { + "parameterName": "LinuxAHBDisabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "LinuxAHBDisabled" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\",\"mergeType\":\"inner\",\"leftTable\":\"LinuxAHBDisabled\",\"rightTable\":\"API-Get_VMLinux_SKU\",\"leftColumn\":\"VMSize\",\"rightColumn\":\"Name\"}],\"projectRename\":[{\"originalName\":\"[LinuxAHBDisabled].LinuxId\",\"mergedName\":\"LinuxId\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBDisabled].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBDisabled].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBDisabled].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBDisabled].VMSSize\",\"mergedName\":\"VMSSize\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBDisabled].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBDisabled].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBDisabled].OsVersion\",\"mergedName\":\"OsVersion\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBDisabled].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[API-Get_VMLinux_SKU].Name\",\"mergedName\":\"Name\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[API-Get_VMLinux_SKU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[API-Get_VMLinux_SKU].MemoryGB\",\"mergedName\":\"MemoryGB\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[API-Get_VMLinux_SKU].MaxNetworkInterfaces\",\"mergedName\":\"MaxNetworkInterfaces\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[API-Get_VMLinux_SKU].HyperVGenerations\",\"mergedName\":\"HyperVGenerations\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[API-Get_VMLinux_SKU].vCPUsPerCore\",\"mergedName\":\"vCPUsPerCore\",\"fromId\":\"8a27a7d8-5ea8-4408-ac20-2fc4e65ca095\"},{\"originalName\":\"[LinuxAHBDisabled].LinuxId1\",\"mergedName\":\"LinuxId1\",\"fromId\":\"unknown\"},{\"originalName\":\"[LinuxAHBEnabled].VMName\"},{\"originalName\":\"[API-Get_VM_SKU].MemoryGB\"},{\"originalName\":\"[API-Get_VM_SKU].MaxNetworkInterfaces\"},{\"originalName\":\"[API-Get_VM_SKU].Name\"},{\"originalName\":\"[API-Get_VM_SKU].HyperVGenerations\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUsPerCore\"},{\"originalName\":\"[LinuxAHBEnabled].VMSSize\"}]}", + "size": 0, + "title": "Linux VMs without AHB", + "noDataMessage": "None of your Linux VMs have AHB enabled", + "noDataMessageStyle": 4, + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Sev4", + "text": "{0}{1}" + } + ] + }, + "numberFormat": { + "unit": 17, + "options": { + "style": "decimal" + } + } + } + ] + }, + "tileSettings": { + "showBorder": false, + "titleContent": { + "columnMatch": "NewLicense", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "vCPUs", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + }, + "chartSettings": { + "xAxis": "VM Name", + "yAxis": [ + "Consumed Cores per VM" + ], + "group": null, + "createOtherGroup": 0, + "seriesLabelSettings": [ + { + "seriesName": "Consumed Cores per VM", + "color": "grayBlue" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "LinuxAHBDisabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "Linux-VM+SKU+vCores-AHBDisabled" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "LinuxVM" + }, + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "VM" + } + ], + "name": "Linux" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "title": "VMSS", + "items": [ + { + "type": 1, + "content": { + "json": "## Windows Azure Hybrid Benefit (AHB) Overview - VM Scale Set" + }, + "name": "AHB Overview - VM Scale Set" + }, + { + "type": 1, + "content": { + "json": "Each two-processor license or each set of 16-core licenses, either Datacenter or Standard editions, are entitled to two instances of up to 8 cores, or one instance of up to 16 cores.\r\n\r\nThe virtual machines (VMs) with less than 8 cores are categorized as **Low Priority**, while those with 8 or more cores are classified as **High Priority**. In situations where there are insufficient Azure Hybrid benefit licenses to cover all the VMs in the environment, it is recommended to prioritize the High Priority VMs.", + "style": "info" + }, + "name": "Each two-processor license or each set of 16-core licenses" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | extend SubscriptionName=name \r\n| join (\r\nresources \r\n| where resourceGroup in ({ResourceGroup})\r\n| where type =~ 'microsoft.compute/virtualMachineScaleSets'\r\n| where tostring(properties.virtualMachineProfile.storageProfile.osDisk.osType) == 'Windows' and tostring(properties.virtualMachineProfile.licenseType) == \"Windows_Server\"\r\n| extend WindowsId=id, VMName=name, VMLocation=location, VMRG=resourceGroup, OSType=tostring(properties.virtualMachineProfile.storageProfile.osDisk.osType), OSVersion = tostring(properties.virtualMachineProfile.storageProfile.imageReference.sku), VMSize=tostring (properties.hardwareProfile.vmSize), LicenseType = tostring(properties.virtualMachineProfile.licenseType), VMSSize=tostring(sku.name)\r\n ) on subscriptionId \r\n| order by type asc \r\n| project WindowsId,VMName,VMRG,VMSize, VMSSize, VMLocation,OSType, OSVersion,LicenseType, subscriptionId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), WindowsId=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct WindowsId\r\n )\r\n on WindowsId", + "size": 0, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "LoadVMSSTab", + "comparison": "isEqualTo", + "value": "Yes" + }, + { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "True" + } + ], + "name": "VMSSAHBEnabled-RG" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | extend SubscriptionName=name \r\n| join (\r\nresources \r\n| where resourceGroup in ({ResourceGroup})\r\n| where type =~ 'microsoft.compute/virtualMachineScaleSets'\r\n| where tostring(properties.storageProfile.osDisk.osType) == 'Windows' or tostring(properties.virtualMachineProfile.storageProfile.osDisk.osType) == 'Windows'\r\n| where tostring(properties.['licenseType']) !has 'Windows' and tostring(properties.virtualMachineProfile.['licenseType']) !has 'Windows'\r\n| extend WindowsId=id, VMName=name, VMLocation=location, VMRG=resourceGroup, OSType=tostring(properties.virtualMachineProfile.storageProfile.osDisk.osType), OsVersion = tostring(properties.virtualMachineProfile.storageProfile.imageReference.sku), VMSize=tostring (properties.hardwareProfile.vmSize), LicenseType = tostring(properties.virtualMachineProfile.licenseType), VMSSize=tostring(sku.name)\r\n ) on subscriptionId \r\n| order by type asc \r\n| project WindowsId,VMName,VMRG,VMSize, VMSSize, VMLocation,OSType, OsVersion,LicenseType, subscriptionId\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), WindowsId=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct WindowsId\r\n )\r\n on WindowsId\r\n", + "size": 0, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "LoadVMSSTab", + "comparison": "isEqualTo", + "value": "Yes" + }, + { + "parameterName": "AlwaysHidden", + "comparison": "isEqualTo", + "value": "True" + } + ], + "name": "VMSSAHBDisabled-RG" + }, + { + "type": 1, + "content": { + "json": "### Consumed Licenses\r\nTotal number of Windows licenses cores consumed by all Windows virtual machines.\r\n", + "style": "info" + }, + "customWidth": "50", + "name": "Windows virtual machine" + }, + { + "type": 1, + "content": { + "json": "### Number of required Cores to enable Windows Azure Hybrid Benefit\r\nNumber of cores required to enable AHB across the entire environment.", + "style": "info" + }, + "customWidth": "50", + "name": "Number of required Cores to AHB" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "loadType": "explicit", + "loadButtonText": "Load VMSS Info", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"50d79765-aad4-437e-a90b-8cc7865e7081\",\"mergeType\":\"inner\",\"leftTable\":\"VMSSAHBEnabled-RG\",\"rightTable\":\"API-Get_VM_SKU\",\"leftColumn\":\"VMSSize\",\"rightColumn\":\"Name\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\">=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores per VM\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"8\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\" ([\\\"vCPUs\\\"] + 7) & ~7\"}}]},{\"originalName\":\"[query - 0].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[VMSSAHBEnabled-RG].WindowsId\",\"mergedName\":\"WindowsId\",\"fromId\":\"unknown\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"unknown\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"unknown\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"unknown\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMSSize\",\"mergedName\":\"VMSSize\",\"fromId\":\"unknown\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"unknown\"},{\"originalName\":\"[VMSSAHBEnabled-RG].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"unknown\"},{\"originalName\":\"[VMSSAHBEnabled-RG].OSVersion\",\"mergedName\":\"OSVersion\",\"fromId\":\"unknown\"},{\"originalName\":\"[VMSSAHBEnabled-RG].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"unknown\"},{\"originalName\":\"[VMSSAHBEnabled-RG].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[API-Get_VM_SKU].Name\",\"mergedName\":\"Name\",\"fromId\":\"unknown\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"unknown\"},{\"originalName\":\"[API-Get_VM_SKU].MemoryGB\",\"mergedName\":\"MemoryGB\",\"fromId\":\"unknown\"},{\"originalName\":\"[API-Get_VM_SKU].MaxNetworkInterfaces\",\"mergedName\":\"MaxNetworkInterfaces\",\"fromId\":\"unknown\"},{\"originalName\":\"[API-Get_VM_SKU].HyperVGenerations\",\"mergedName\":\"HyperVGenerations\",\"fromId\":\"unknown\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUsPerCore\",\"mergedName\":\"vCPUsPerCore\",\"fromId\":\"unknown\"},{\"originalName\":\"[VMSSAHBEnabled-RG].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"unknown\"},{\"originalName\":\"[WindowsAHBEnabled].WindowsId\"},{\"originalName\":\"[WindowsAHBEnabled].VMRG\"},{\"originalName\":\"[WindowsAHBEnabled].VMLocation\"},{\"originalName\":\"[WindowsAHBEnabled].OSType\"},{\"originalName\":\"[WindowsAHBEnabled].OsVersion\"},{\"originalName\":\"[WindowsAHBEnabled].LicenseType\"},{\"originalName\":\"[query - Get VM vCPU].Name\"},{\"originalName\":\"[query - Get VM vCPU].MemoryGB\"},{\"originalName\":\"[query - Get VM vCPU].MaxNetworkInterfaces\"},{\"originalName\":\"[query - Get VM vCPU].HyperVGenerations\"},{\"originalName\":\"[query - Get VM vCPU].vCPUsPerCore\"},{\"originalName\":\"[WindowsAHBEnabled].VMSSize\"}]}", + "size": 0, + "title": "Cores not enabled per AHB Priority", + "noDataMessage": "All of your VMs have AHB enabled", + "noDataMessageStyle": 4, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Sev4", + "text": "{0}{1}" + } + ] + }, + "numberFormat": { + "unit": 17, + "options": { + "style": "decimal" + } + } + } + ] + }, + "chartSettings": { + "yAxis": [ + "Consumed Cores per VM" + ], + "group": "Prioritize AHB?", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Cores NOT enabled per AHB Priority" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"50d79765-aad4-437e-a90b-8cc7865e7081\",\"mergeType\":\"inner\",\"leftTable\":\"VMSSAHBEnabled-RG\",\"rightTable\":\"API-Get_VM_SKU\",\"leftColumn\":\"VMSSize\",\"rightColumn\":\"Name\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\">=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"High Priority\\\"\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"\\\"Low Priority\\\"\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores per VM\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"8\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\" ([\\\"vCPUs\\\"] + 7) & ~7\"}}]},{\"originalName\":\"[WindowsAHBEnabled].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[VMSSAHBEnabled-RG].WindowsId\",\"mergedName\":\"WindowsId\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMSSize\",\"mergedName\":\"VMSSize\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].OSVersion\",\"mergedName\":\"OSVersion\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].Name\",\"mergedName\":\"Name\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].MemoryGB\",\"mergedName\":\"MemoryGB\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].MaxNetworkInterfaces\",\"mergedName\":\"MaxNetworkInterfaces\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].HyperVGenerations\",\"mergedName\":\"HyperVGenerations\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUsPerCore\",\"mergedName\":\"vCPUsPerCore\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"unknown\"},{\"originalName\":\"[WindowsAHBEnabled].WindowsId\"},{\"originalName\":\"[WindowsAHBEnabled].VMRG\"},{\"originalName\":\"[WindowsAHBEnabled].VMLocation\"},{\"originalName\":\"[WindowsAHBEnabled].OSType\"},{\"originalName\":\"[WindowsAHBEnabled].OsVersion\"},{\"originalName\":\"[WindowsAHBEnabled].LicenseType\"},{\"originalName\":\"[query - Get VM vCPU].Name\"},{\"originalName\":\"[query - Get VM vCPU].MemoryGB\"},{\"originalName\":\"[query - Get VM vCPU].MaxNetworkInterfaces\"},{\"originalName\":\"[query - Get VM vCPU].HyperVGenerations\"},{\"originalName\":\"[query - Get VM vCPU].vCPUsPerCore\"},{\"originalName\":\"[WindowsAHBEnabled].VMSSize\"}]}", + "size": 0, + "title": "Consumed Cores per AHB Priority", + "noDataMessage": "None of your VMs have AHB enabled", + "noDataMessageStyle": 4, + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Sev4", + "text": "{0}{1}" + } + ] + }, + "numberFormat": { + "unit": 17, + "options": { + "style": "decimal" + } + } + } + ] + } + }, + "customWidth": "33", + "name": "Consumed Cores per AHB Priority" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"d8deb22b-a596-43ee-acc4-180849d26130\",\"mergeType\":\"inner\",\"leftTable\":\"VMSSAHBEnabled-RG\",\"rightTable\":\"API-Get_VM_SKU\",\"leftColumn\":\"VMSSize\",\"rightColumn\":\"Name\"}],\"projectRename\":[{\"originalName\":\"[Added column]\",\"mergedName\":\"ConsumedCores\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"8\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"([\\\"vCPUs\\\"] + 7) & ~7\"}}]},{\"originalName\":\"[WindowsAHBEnabled].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[VMSSAHBEnabled-RG].WindowsId\",\"mergedName\":\"WindowsId\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMSSize\",\"mergedName\":\"VMSSize\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[VMSSAHBEnabled-RG].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[VMSSAHBEnabled-RG].OSVersion\",\"mergedName\":\"OSVersion\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[VMSSAHBEnabled-RG].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[VMSSAHBEnabled-RG].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[API-Get_VM_SKU].Name\",\"mergedName\":\"Name\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[API-Get_VM_SKU].MemoryGB\",\"mergedName\":\"MemoryGB\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[API-Get_VM_SKU].MaxNetworkInterfaces\",\"mergedName\":\"MaxNetworkInterfaces\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[API-Get_VM_SKU].HyperVGenerations\",\"mergedName\":\"HyperVGenerations\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUsPerCore\",\"mergedName\":\"vCPUsPerCore\",\"fromId\":\"d8deb22b-a596-43ee-acc4-180849d26130\"},{\"originalName\":\"[VMSSAHBEnabled-RG].WindowsId1\",\"mergedName\":\"WindowsId1\",\"fromId\":\"unknown\"}]}", + "size": 0, + "title": "Consumed Cores per VMSS", + "showRefreshButton": true, + "queryType": 7, + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "ConsumedCores", + "formatter": 0, + "formatOptions": { + "aggregation": "Sum" + } + } + ] + }, + "tileSettings": { + "titleContent": {}, + "leftContent": { + "columnMatch": "ConsumedCores", + "formatter": 12, + "formatOptions": { + "palette": "blue" + } + }, + "showBorder": false + }, + "graphSettings": { + "type": 0 + }, + "chartSettings": { + "yAxis": [ + "ConsumedCores" + ], + "group": "VMName", + "createOtherGroup": null + } + }, + "customWidth": "33", + "name": "Consumed Cores per VMSS" + } + ] + }, + "name": "VMSS RG Overview" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "ae5e8765-47ef-46a6-803b-6b7124c098d2", + "version": "KqlParameterItem/1.0", + "name": "VMSSAHBEnabled", + "label": "See VMSS with AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n", + "timeContext": { + "durationMs": 86400000 + } + }, + { + "id": "f1ac5e53-253c-4afb-8bc5-b1ba2efea3eb", + "version": "KqlParameterItem/1.0", + "name": "VMSSAHBDisabled", + "label": "See VMSS without AHB", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[\r\n {\"value\":\"Yes\"},\r\n {\"value\":\"No\", \"selected\":true}\r\n]\r\n\r\n" + } + ], + "style": "pills", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "VMSS Without AHB" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"50d79765-aad4-437e-a90b-8cc7865e7081\",\"mergeType\":\"inner\",\"leftTable\":\"VMSSAHBEnabled-RG\",\"rightTable\":\"API-Get_VM_SKU\",\"leftColumn\":\"VMSSize\",\"rightColumn\":\"Name\"}],\"projectRename\":[{\"originalName\":\"[WindowsAHBEnabled].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\">=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"static\",\"resultVal\":\"High Priority\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"static\",\"resultVal\":\"Low Priority\"}}]},{\"originalName\":\"[Added column]\",\"mergedName\":\"Consumed Cores per VM\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\"<=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"expression\",\"resultVal\":\"8\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"expression\",\"resultVal\":\"([\\\"vCPUs\\\"] + 7) & ~7\"}}]},{\"originalName\":\"[VMSSAHBEnabled-RG].WindowsId\",\"mergedName\":\"WindowsId\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMSSize\",\"mergedName\":\"VMSSize\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].OSVersion\",\"mergedName\":\"OSVersion\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBEnabled-RG].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].Name\",\"mergedName\":\"Name\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].MemoryGB\",\"mergedName\":\"MemoryGB\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].MaxNetworkInterfaces\",\"mergedName\":\"MaxNetworkInterfaces\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].HyperVGenerations\",\"mergedName\":\"HyperVGenerations\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUsPerCore\",\"mergedName\":\"vCPUsPerCore\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - 0].VMName\"},{\"originalName\":\"[query - 0].VMSSize\"},{\"originalName\":\"[query - Get VM vCPU].Name\"},{\"originalName\":\"[query - Get VM vCPU].MemoryGB\"},{\"originalName\":\"[query - Get VM vCPU].MaxNetworkInterfaces\"},{\"originalName\":\"[query - Get VM vCPU].HyperVGenerations\"},{\"originalName\":\"[query - Get VM vCPU].vCPUsPerCore\"},{\"originalName\":\"[WindowsAHBEnabled].VMSSize\"},{\"originalName\":\"[WindowsAHBEnabled].VMName\"},{\"originalName\":\"[VMSSAHBEnabled].VMSize\"},{\"originalName\":\"[VMSSAHBEnabled].VMName\"},{\"originalName\":\"[VMSSAHBEnabled-Tag].VMName\"},{\"originalName\":\"[VMSSAHBEnabled-Tag].VMSize\"}]}", + "size": 0, + "title": "VMSS with AHB", + "noDataMessage": "None of your VMSS have AHB enabled", + "noDataMessageStyle": 4, + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "2", + "text": "{0}{1}" + } + ] + }, + "numberFormat": { + "unit": 17, + "options": { + "style": "decimal" + } + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Subscription Name", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "subscriptionId" + ], + "expandTopLevel": true + } + } + }, + "conditionalVisibility": { + "parameterName": "VMSSAHBEnabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "VMSS+SKU+vCores-AHB" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"Merge/1.0\",\"merges\":[{\"id\":\"50d79765-aad4-437e-a90b-8cc7865e7081\",\"mergeType\":\"inner\",\"leftTable\":\"VMSSAHBDisabled-RG\",\"rightTable\":\"API-Get_VM_SKU\",\"leftColumn\":\"VMSSize\",\"rightColumn\":\"Name\"}],\"projectRename\":[{\"originalName\":\"[query - 0].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[query - 0].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"unknown\"},{\"originalName\":\"[Added column]\",\"mergedName\":\"Prioritize AHB?\",\"fromId\":null,\"isNewItem\":true,\"newItemData\":[{\"criteriaContext\":{\"leftOperand\":\"vCPUs\",\"operator\":\">=\",\"rightValType\":\"static\",\"rightVal\":\"8\",\"resultValType\":\"static\",\"resultVal\":\"High Priority\"}},{\"criteriaContext\":{\"operator\":\"Default\",\"rightValType\":\"column\",\"resultValType\":\"static\",\"resultVal\":\"Low Priority\"}}]},{\"originalName\":\"[VMSSAHBDisabled-RG].WindowsId\",\"mergedName\":\"WindowsId\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBDisabled-RG].VMName\",\"mergedName\":\"VMName\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBDisabled-RG].VMRG\",\"mergedName\":\"VMRG\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBDisabled-RG].VMSize\",\"mergedName\":\"VMSize\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBDisabled-RG].VMSSize\",\"mergedName\":\"VMSSize\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBDisabled-RG].VMLocation\",\"mergedName\":\"VMLocation\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBDisabled-RG].OSType\",\"mergedName\":\"OSType\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBDisabled-RG].OsVersion\",\"mergedName\":\"OsVersion\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBDisabled-RG].LicenseType\",\"mergedName\":\"LicenseType\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[VMSSAHBDisabled-RG].subscriptionId\",\"mergedName\":\"subscriptionId\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].Name\",\"mergedName\":\"Name\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUs\",\"mergedName\":\"vCPUs\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].MemoryGB\",\"mergedName\":\"MemoryGB\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].MaxNetworkInterfaces\",\"mergedName\":\"MaxNetworkInterfaces\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].HyperVGenerations\",\"mergedName\":\"HyperVGenerations\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[API-Get_VM_SKU].vCPUsPerCore\",\"mergedName\":\"vCPUsPerCore\",\"fromId\":\"50d79765-aad4-437e-a90b-8cc7865e7081\"},{\"originalName\":\"[query - 0].VMName\"},{\"originalName\":\"[query - 0].VMSSize\"},{\"originalName\":\"[query - Get VM vCPU].Name\"},{\"originalName\":\"[query - Get VM vCPU].MemoryGB\"},{\"originalName\":\"[query - Get VM vCPU].MaxNetworkInterfaces\"},{\"originalName\":\"[query - Get VM vCPU].HyperVGenerations\"},{\"originalName\":\"[query - Get VM vCPU].vCPUsPerCore\"},{\"originalName\":\"[VMSS-AHB-Disabled].VMName\"},{\"originalName\":\"[VMSS-AHB-Disabled-Tag].VMSize\"},{\"originalName\":\"[VMSS-AHB-Disabled-Tag].VMName\"}]}", + "size": 0, + "title": "VMSS without AHB", + "noDataMessage": "All of your VMSS have AHB enabled", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 7, + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Group", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Prioritize AHB?", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High Priority", + "representation": "Sev0", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Sev4", + "text": "{0}{1}" + } + ] + }, + "numberFormat": { + "unit": 17, + "options": { + "style": "decimal" + } + } + } + ], + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "subscriptionId" + ], + "expandTopLevel": true + } + } + }, + "conditionalVisibility": { + "parameterName": "VMSSAHBDisabled", + "comparison": "isEqualTo", + "value": "Yes" + }, + "name": "VMSS+SKU+vCores" + } + ] + }, + "name": "VMSS RG Details" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedSubTab", + "comparison": "isEqualTo", + "value": "VMSS" + }, + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "VM" + } + ], + "name": "VMSS-RG" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "selectedRateOptimizationTab", + "comparison": "isEqualTo", + "value": "AHB" + }, + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RateOptimization" + } + ], + "name": "AHB Overview" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RateOptimization" + }, + "name": "group - RateOptimization group" + }, + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "6b8c0a46-6867-498b-9a3e-799a2475a11a", + "cellValue": "selectedOverviewTab", + "linkTarget": "parameter", + "linkLabel": "Welcome", + "subTarget": "instructions", + "style": "link" + }, + { + "id": "da748ed1-f329-42d4-962d-9b2339baf7c4", + "cellValue": "selectedOverviewTab", + "linkTarget": "parameter", + "linkLabel": "Resources overview", + "subTarget": "resourcesMap", + "style": "link" + }, + { + "id": "a4b4de18-b90e-4212-86a2-ea5fabc4f40c", + "cellValue": "selectedOverviewTab", + "linkTarget": "parameter", + "linkLabel": "Security recommendations", + "subTarget": "securityRecommendations", + "style": "link" + }, + { + "id": "a18f24d2-3320-4c53-a86d-db32c920c8f7", + "cellValue": "selectedOverviewTab", + "linkTarget": "parameter", + "linkLabel": "Reliability recommendations", + "subTarget": "reliabilityRecommendations", + "style": "link" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "Welcome" + }, + "name": "tabs - overview tabs" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "6a9ccf8c-9f3e-4ee0-b45b-f511401f8656", + "version": "KqlParameterItem/1.0", + "name": "mapSubscriptions", + "label": "Subscriptions", + "type": 6, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "includeAll": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::all", + "value": [ + "value::all" + ] + } + ], + "style": "pills", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "Welcome" + }, + { + "parameterName": "selectedOverviewTab", + "comparison": "isNotEqualTo", + "value": "instructions" + } + ], + "name": "parameters - OverviewSubscriptions" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "advisorresources\r\n| where type == \"microsoft.advisor/recommendations\"\r\n| where tostring (properties.category) has \"Security\"\r\n| project AffectedResource=tostring(properties.resourceMetadata.resourceId),Impact=tostring(properties.impact),Recommendation=tostring(properties.shortDescription.problem),subscriptionId", + "size": 0, + "title": "Azure Advisor security recommendations", + "noDataMessage": "You are following all of our security recommendations for the selected subscriptions.", + "noDataMessageStyle": 3, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{mapSubscriptions}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "Impact", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "colors", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High", + "representation": "red", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Medium", + "representation": "orange", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low", + "representation": "blue", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "gray", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "resourceGroup", + "formatter": 14, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "Impact" + ] + } + } + }, + "name": "query - advisorSecurityRecommendations" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "Welcome" + }, + { + "parameterName": "selectedOverviewTab", + "comparison": "isEqualTo", + "value": "securityRecommendations" + } + ], + "name": "group - securityRecommendations" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Welcome to the cost optimization workbook" + }, + "name": "Welcome" + }, + { + "type": 1, + "content": { + "json": "### Reference: [Microsoft Azure Well-Architected Framework - cost optimization pillar](https://learn.microsoft.com/azure/architecture/framework/cost/overview)", + "style": "upsell" + }, + "name": "Reference" + }, + { + "type": 1, + "content": { + "json": "This workbook aims to offer a comprehensive overview of your Azure environment's resource usage, aligning with the WAF Cost Optimization pillar. It identifies recommendations to optimize efficiency, providing guidance on potential opportunities. Please note that the workbook serves as guidance to highlight optimization opportunities, and the extent of cost reduction depends on their implementation.\r\n\r\n## Overview of the cost optimization pillar\r\n\r\n* The cost optimization pillar provides principles for balancing business goals with technology needs to create a cost-effective workload while avoiding capital-intensive solutions.The workbook emphasizes the importance of reducing waste and improving operational efficiencies.\r\n\r\n* To assess your workload based on the principles outlined in the [Microsoft Azure Well-Architected Framework](https://learn.microsoft.com/azure/architecture/framework/), reference the [Microsoft Azure Well-Architected Review](https://learn.microsoft.com/assessments/?id=azure-architecture-review&mode=pre-assessment&session=20dc50e4-5b71-4f38-bc49-51cc1d9f205c) tool.\r\n\r\n\r\n\r\n\r\n" + }, + "name": "objective" + }, + { + "type": 1, + "content": { + "json": "Indicates an implemented recommendation that can result in a environment that is following the Cost Optimization & Cost Governance principles.", + "style": "success" + }, + "customWidth": "50", + "name": "Greenlight", + "styleSettings": { + "margin": "10px", + "showBorder": true + } + }, + { + "type": 1, + "content": { + "json": "## Prerequisites\r\n\r\nThis workbook requires the following least-privileged (minimum) roles on your Subscriptions:\r\n\r\n * **Reader** : allows you to import the workbook without saving it and view all of the workbook tabs.\r\n * **Workbook Contributor** : allows you to import and save the workbook\r\n\r\nThis workbook includes \"Quick Fix\" actions within certain queries. The permissions necessary to execute these actions may vary and are documented for each specific action.\r\n\r\n\r\n" + }, + "name": "Prerequisites" + }, + { + "type": 1, + "content": { + "json": "## Feedback\r\n\r\n [ Submit feedback here ](https://aka.ms/advisor_cost_wb_feedback) on your experience with workbooks at any time.\r\n\r\n\r\n\r\n [Submit any issues ](https://aka.ms/costworkbookfeedback) with the workbook template to GitHub." + }, + "name": "text - 5" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "Welcome" + }, + { + "parameterName": "selectedOverviewTab", + "comparison": "isEqualTo", + "value": "instructions" + } + ], + "name": "Welcome" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "summarize count() by location", + "size": 2, + "title": "Resource distribution per region", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{mapSubscriptions}" + ], + "visualization": "map", + "mapSettings": { + "locInfo": "AzureLoc", + "locInfoColumn": "location", + "sizeSettings": "count_", + "sizeAggregation": "Sum", + "labelSettings": "location", + "legendMetric": "count_", + "legendAggregation": "Sum", + "itemColorSettings": { + "nodeColorField": "count_", + "colorAggregation": "Sum", + "type": "heatmap", + "heatmapPalette": "greenRed" + } + } + }, + "name": "query - resourcesMap" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "selectedOverviewTab", + "comparison": "isEqualTo", + "value": "resourcesMap" + }, + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "Welcome" + } + ], + "name": "group - resourceOverview" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "advisorresources\r\n| where type == \"microsoft.advisor/recommendations\"\r\n| where tostring (properties.category) has \"HighAvailability\"\r\n| project AffectedResource=tostring(properties.resourceMetadata.resourceId),Impact=tostring(properties.impact),Recommendation=tostring(properties.shortDescription.problem),subscriptionId", + "size": 0, + "title": "Azure Advisor reliability recommendations", + "noDataMessage": "You are following all of our reliability recommendations for the selected subscriptions.", + "noDataMessageStyle": 3, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{mapSubscriptions}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "Impact", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "colors", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "High", + "representation": "red", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Medium", + "representation": "orange", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Low", + "representation": "blue", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "gray", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "resourceGroup", + "formatter": 14, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "Impact" + ] + } + } + }, + "name": "query - advisorReliabilityRecommendations" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "Welcome" + }, + { + "parameterName": "selectedOverviewTab", + "comparison": "isEqualTo", + "value": "reliabilityRecommendations" + } + ], + "name": "group - reliabilityRecommendations" + } + ], + "fallbackResourceIds": [ + "Azure Monitor" + ], + "$schema": "https://github.com/Microsoft/Application-Insights-Workbooks/blob/master/schema/workbook.json" + }, + "version": "", + "workbookJson": "[string(variables('$fxv#0'))]", + "workbookId": "0b2", + "telemetryId": "[format('00f120b5-2007-6120-0000-{0}30126b006', variables('workbookId'))]", + "finOpsToolkitVersion": "13.0", + "resourceTags": "[if(contains(parameters('tags'), 'ftk-tool'), parameters('tags'), union(parameters('tags'), createObject('ftk-version', variables('finOpsToolkitVersion'), 'ftk-tool', format('{0} workbook', parameters('displayName')))))]" + }, + "resources": [ + { + "condition": "[parameters('enableDefaultTelemetry')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "[format('pid-{0}-{1}', variables('telemetryId'), uniqueString(deployment().name, parameters('location')))]", + "properties": { + "mode": "Incremental", + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "FinOps toolkit", + "version": "[variables('finOpsToolkitVersion')]" + } + }, + "resources": [] + } + } + }, + { + "type": "Microsoft.Insights/workbooks", + "apiVersion": "2022-04-01", + "name": "[guid(resourceGroup().id, 'Microsoft.Insights/workbooks', parameters('displayName'))]", + "location": "[parameters('location')]", + "tags": "[variables('resourceTags')]", + "kind": "shared", + "properties": { + "category": "workbook", + "description": "[parameters('description')]", + "displayName": "[parameters('displayName')]", + "serializedData": "[variables('workbookJson')]", + "sourceId": "Azure Monitor", + "version": "[variables('version')]" + } + } + ], + "outputs": { + "workbookId": { + "type": "string", + "metadata": { + "description": "The resource ID of the workbook." + }, + "value": "[resourceId('Microsoft.Insights/workbooks', guid(resourceGroup().id, 'Microsoft.Insights/workbooks', parameters('displayName')))]" + }, + "workbookUrl": { + "type": "string", + "metadata": { + "description": "Link to the workbook in the Azure portal." + }, + "value": "[format('{0}/#view/AppInsightsExtension/UsageNotebookBlade/ComponentId/Azure%20Monitor/ConfigurationId/{1}/Type/{2}/WorkbookTemplateName/{3}', environment().portal, uriComponent(resourceId('Microsoft.Insights/workbooks', guid(resourceGroup().id, 'Microsoft.Insights/workbooks', parameters('displayName')))), reference(resourceId('Microsoft.Insights/workbooks', guid(resourceGroup().id, 'Microsoft.Insights/workbooks', parameters('displayName'))), '2022-04-01').category, uriComponent(reference(resourceId('Microsoft.Insights/workbooks', guid(resourceGroup().id, 'Microsoft.Insights/workbooks', parameters('displayName'))), '2022-04-01').displayName))]" + } + } + } + } + }, + { + "condition": "[parameters('includeGovernance')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('{0}-Governance', parameters('displayNamePrefix'))]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "displayName": { + "value": "[format('{0} - Governance', parameters('displayNamePrefix'))]" + }, + "location": { + "value": "[parameters('location')]" + }, + "tags": { + "value": "[variables('resourceTags')]" + }, + "enableDefaultTelemetry": { + "value": false + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.40.2.10011", + "templateHash": "10362789456967541266" + } + }, + "parameters": { + "displayName": { + "type": "string", + "defaultValue": "Governance", + "metadata": { + "description": "Optional. Display name for the workbook used in the Gallery. Must be unique in the resource group." + } + }, + "location": { + "type": "string", + "defaultValue": "[resourceGroup().location]", + "metadata": { + "description": "Optional. Location of the resources. Default: Same as deployment. See https://aka.ms/azureregions." + } + }, + "description": { + "type": "string", + "defaultValue": "Reports to help you optimize your cost.", + "metadata": { + "description": "Optional. Workbook description." + } + }, + "tags": { + "type": "object", + "defaultValue": {}, + "metadata": { + "description": "Optional. Tags for all resources." + } + }, + "enableDefaultTelemetry": { + "type": "bool", + "defaultValue": true, + "metadata": { + "description": "Optional. Enable telemetry to track anonymous module usage trends, monitor for bugs, and improve future releases." + } + } + }, + "variables": { + "$fxv#0": { + "version": "Notebook/1.0", + "items": [ + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "19b06e9e-eec2-4a7e-935d-92d77b2f87a3", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Overview", + "subTarget": "RC_Overview", + "preText": "", + "style": "link" + }, + { + "id": "528e35b9-aca4-423f-9267-50f62011a3cb", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Virtual machine", + "subTarget": "RC_VM", + "style": "link" + }, + { + "id": "7faacfc6-663e-4ff5-bb64-f86d995f9563", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Storage + backup", + "subTarget": "RC_Storage", + "style": "link" + }, + { + "id": "c17ce2c0-83e6-4e5c-9c3e-f34cbf887e73", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Network", + "subTarget": "RC_Network", + "style": "link" + }, + { + "id": "2f4e49d7-3198-4173-af1c-4cf4c5178000", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "PaaS", + "subTarget": "RC_PaaS", + "style": "link" + }, + { + "id": "f8f7e1fc-8f5d-442a-9788-3eabbf8ab275", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Security", + "subTarget": "RC_Security", + "style": "link" + }, + { + "id": "80ad2db8-a21e-43e9-bd28-75d8d606eaf5", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Monitoring", + "subTarget": "RC_Monitoring", + "style": "link" + }, + { + "id": "6fc0fef0-a016-4923-9239-b641eb5bdc4f", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Services retirement", + "subTarget": "RC_ServicesRetirement", + "style": "link" + }, + { + "id": "e40dbf66-2abe-4bcf-acd7-1ee6d8fc950b", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Resource age", + "subTarget": "RC_Age", + "style": "link" + }, + { + "id": "e112c6e1-db5e-4b0e-99e9-2edac0eba177", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Tag explorer", + "subTarget": "RC_Tag", + "style": "link" + }, + { + "id": "840cd5ea-6b74-484b-846f-01d424b295cd", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Cost management", + "subTarget": "RC_Cost", + "style": "link" + }, + { + "id": "5436a8c9-73c4-4121-a814-dd6fbb0c0d0c", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Usage + limits", + "subTarget": "RC_Quota", + "style": "link" + }, + { + "id": "fa81b57a-8f3c-4502-beb0-128a7fc35f7c", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Compliance", + "subTarget": "RC_Compliance", + "style": "link" + }, + { + "id": "e3acf38e-2dc4-423e-b91d-a173280b5808", + "cellValue": "SelectedTab", + "linkTarget": "parameter", + "linkLabel": "Governance", + "subTarget": "RC_Governance", + "style": "link" + } + ] + }, + "name": "RC_Menu" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "value::tenant" + ], + "parameters": [ + { + "id": "30297a43-7d69-4daf-93c9-8170d5a995b0", + "version": "KqlParameterItem/1.0", + "name": "Subscription", + "label": "Subscriptions", + "type": 6, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "includeAll": false, + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::all" + } + ], + "style": "above", + "queryType": 1, + "resourceType": "microsoft.resources/tenants" + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "RC_Age" + }, + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "RC_Cost" + }, + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "RC_Quota" + }, + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "RC_Compliance" + }, + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "RC_ServicesRetirement" + } + ], + "name": "parameters - Subscriptions" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "# Welcome the Azure governance workbook" + }, + "name": "Welcome" + }, + { + "type": 1, + "content": { + "json": "### Reference: [Governance in the Microsoft Cloud Adoption Framework for Azure](https://learn.microsoft.com/azure/cloud-adoption-framework/govern/)", + "style": "upsell" + }, + "name": "Reference" + }, + { + "type": 1, + "content": { + "json": "The objective of this workbook is to provide a comprehensive overview of the governance posture of your Azure environment. It offers the standard metrics aligned with the Cloud Adoption Framework and has the capability to identify and apply recommendations to identify non compliance. This workbook is part of the [FinOps toolkit](https://aka.ms/finops/toolkit).\r\n\r\n## Overview of the Cloud Adoption Framework\r\n\r\n* The CAF Govern methodology provides a structured approach for establishing and optimizing cloud governance in Azure. The guidance is relevant for organizations across any industry. It covers essential categories of cloud governance, such as regulatory compliance, security, operations, cost, data, resource management, and artificial intelligence (AI).\r\n\r\n* Cloud governance is how you control cloud use across your organization. Cloud governance sets up guardrails that regulate cloud interactions. These guardrails are a framework of policies, procedures, and tools you use to establish control. Policies define acceptable and unacceptable cloud activity, and the procedures and tools you use ensure all cloud usage aligns with those policies. Successful cloud governance prevents all unauthorized or unmanaged cloud usage.\r\n\r\n* To assess your transformation journey, try the [governance benchmark tool](https://learn.microsoft.com/assessments/b1891add-7646-4d60-a875-32a4ab26327e/).\r\n\r\n\r\n\r\n\r\n" + }, + "name": "text - Overview" + }, + { + "type": 1, + "content": { + "json": "## Prerequisites\r\n\r\nThis workbook will present various cost-related details in the form of governance, networking, storage, VMs, web apps, SQL, and cost information to educate the business about cost related to various resources.\r\n\r\nThis workbook requires the following least-privileged (minimum) roles:\r\n\r\n * **Reader** : allows you to import the workbook without saving it and view all of the workbook tabs except the *Cost management* tab.\r\n * **Cost Management Reader**: allows you to view the costs in the *Cost management* tab \r\n * **Workbook Contributor** : allows you to import and save the workbook\r\n\r\n\r\n" + }, + "name": "text - 7" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources\n| summarize count()", + "size": 3, + "title": "Count of all resources", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "tiles", + "tileSettings": { + "titleContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + } + }, + "showBorder": false, + "size": "auto" + } + }, + "name": "Count of all resources" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| summarize Count=count(id) by subscriptionId\r\n| order by Count desc", + "size": 3, + "title": "Resource count per subscription (Top 10)", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "tiles", + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "type", + "formatter": 1 + } + ], + "rowLimit": 10, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "subscriptionId" + ], + "expandTopLevel": true + }, + "labelSettings": [ + { + "columnId": "subscriptionId", + "label": "Subscription name" + } + ] + }, + "sortBy": [], + "tileSettings": { + "titleContent": { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": "Resource", + "showIcon": true + } + }, + "leftContent": { + "columnMatch": "Count", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "style": "decimal", + "maximumFractionDigits": 2, + "maximumSignificantDigits": 3 + } + } + }, + "showBorder": false, + "rowLimit": 10, + "sortCriteriaField": "count_type", + "sortOrderField": 2 + }, + "graphSettings": { + "type": 0, + "topContent": { + "columnMatch": "subscriptionId", + "formatter": 1 + }, + "centerContent": { + "columnMatch": "Count", + "formatter": 1, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + }, + "chartSettings": { + "xAxis": "subscriptionId", + "yAxis": [ + "Count" + ], + "showLegend": true, + "seriesLabelSettings": [ + { + "seriesName": "subscriptionId", + "color": "greenDark" + } + ] + }, + "mapSettings": { + "locInfo": "LatLong", + "sizeSettings": "Count", + "sizeAggregation": "Sum", + "legendMetric": "Count", + "legendAggregation": "Sum", + "itemColorSettings": { + "type": "heatmap", + "colorAggregation": "Sum", + "nodeColorField": "Count", + "heatmapPalette": "greenRed" + } + } + }, + "name": "Resource count per subscription (Top 10)" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources \r\n| extend type = case(\r\ntype contains 'microsoft.netapp/netappaccounts', 'NetApp Accounts',\r\ntype contains \"microsoft.compute\", \"Azure Compute\",\r\ntype contains \"microsoft.logic\", \"LogicApps\",\r\ntype contains 'microsoft.keyvault/vaults', \"Key Vaults\",\r\ntype contains 'microsoft.storage/storageaccounts', \"Storage Accounts\",\r\ntype contains 'microsoft.compute/availabilitysets', 'Availability Sets',\r\ntype contains 'microsoft.operationalinsights/workspaces', 'Azure Monitor Resources',\r\ntype contains 'microsoft.operationsmanagement', 'Operations Management Resources',\r\ntype contains 'microsoft.insights', 'Azure Monitor Resources',\r\ntype contains 'microsoft.desktopvirtualization/applicationgroups', 'WVD Application Groups',\r\ntype contains 'microsoft.desktopvirtualization/workspaces', 'WVD Workspaces',\r\ntype contains 'microsoft.desktopvirtualization/hostpools', 'WVD Hostpools',\r\ntype contains 'microsoft.recoveryservices/vaults', 'Backup Vaults',\r\ntype contains 'microsoft.web', 'App Services',\r\ntype contains 'microsoft.managedidentity/userassignedidentities','Managed Identities',\r\ntype contains 'microsoft.storagesync/storagesyncservices', 'Azure File Sync',\r\ntype contains 'microsoft.hybridcompute/machines', 'ARC Machines',\r\ntype contains 'Microsoft.EventHub', 'Event Hub',\r\ntype contains 'Microsoft.EventGrid', 'Event Grid',\r\ntype contains 'Microsoft.Sql', 'SQL Resources',\r\ntype contains 'Microsoft.HDInsight/clusters', 'HDInsight Clusters',\r\ntype contains 'microsoft.devtestlab', 'DevTest Labs Resources',\r\ntype contains 'microsoft.containerinstance', 'Container Instances Resources',\r\ntype contains 'microsoft.portal/dashboards', 'Azure Dashboards',\r\ntype contains 'microsoft.containerregistry/registries', 'Container Registry',\r\ntype contains 'microsoft.automation', 'Automation Resources',\r\ntype contains 'sendgrid.email/accounts', 'SendGrid Accounts',\r\ntype contains 'microsoft.datafactory/factories', 'Data Factory',\r\ntype contains 'microsoft.databricks/workspaces', 'Databricks Workspaces',\r\ntype contains 'microsoft.machinelearningservices/workspaces', 'Machine Learnings Workspaces',\r\ntype contains 'microsoft.alertsmanagement/smartdetectoralertrules', 'Azure Monitor Resources',\r\ntype contains 'microsoft.apimanagement/service', 'API Management Services',\r\ntype contains 'microsoft.dbforpostgresql', 'PostgreSQL Resources',\r\ntype contains 'microsoft.scheduler/jobcollections', 'Scheduler Job Collections',\r\ntype contains 'microsoft.visualstudio/account', 'Azure DevOps Organization',\r\ntype contains 'microsoft.network/', 'Network Resources',\r\ntype contains 'microsoft.migrate/' or type contains 'microsoft.offazure', 'Azure Migrate Resources',\r\ntype contains 'microsoft.servicebus/namespaces', 'Service Bus Namespaces',\r\ntype contains 'microsoft.classic', 'ASM Obsolete Resources',\r\ntype contains 'microsoft.resources/templatespecs', 'Template Spec Resources',\r\ntype contains 'microsoft.virtualmachineimages', 'VM Image Templates',\r\ntype contains 'microsoft.documentdb', 'CosmosDB DB Resources',\r\ntype contains 'microsoft.alertsmanagement/actionrules', 'Azure Monitor Resources',\r\ntype contains 'microsoft.kubernetes/connectedclusters', 'ARC Kubernetes Clusters',\r\ntype contains 'microsoft.purview', 'Purview Resources',\r\ntype contains 'microsoft.security', 'Security Resources',\r\ntype contains 'microsoft.cdn', 'CDN Resources',\r\ntype contains 'microsoft.devices','IoT Resources',\r\ntype contains 'microsoft.datamigration', 'Data Migraiton Services',\r\ntype contains 'microsoft.cognitiveservices', 'Congitive Services',\r\ntype contains 'microsoft.customproviders', 'Custom Providers',\r\ntype contains 'microsoft.appconfiguration', 'App Services',\r\ntype contains 'microsoft.search', 'Search Services',\r\ntype contains 'microsoft.maps', 'Maps',\r\ntype contains 'microsoft.containerservice/managedclusters', 'AKS',\r\ntype contains 'microsoft.signalrservice', 'SignalR',\r\ntype contains 'microsoft.resourcegraph/queries', 'Resource Graph Queries',\r\ntype contains 'microsoft.batch', 'MS Batch',\r\ntype contains 'microsoft.analysisservices', 'Analysis Services',\r\ntype contains 'microsoft.synapse/workspaces', 'Synapse Workspaces',\r\ntype contains 'microsoft.synapse/workspaces/sqlpools', 'Synapse SQL Pools',\r\ntype contains 'microsoft.kusto/clusters', 'ADX Clusters',\r\ntype contains 'microsoft.resources/deploymentscripts', 'Deployment Scripts',\r\ntype contains 'microsoft.aad/domainservices', 'AD Domain Services',\r\ntype contains 'microsoft.labservices/labaccounts', 'Lab Accounts',\r\ntype contains 'microsoft.automanage/accounts', 'Automanage Accounts',\r\ntype contains 'microsoft.relay/namespaces', 'Azure Relay',\r\ntype contains 'microsoft.notificationhubs/namespaces', 'Notification Hubs',\r\ntype contains 'microsoft.digitaltwins/digitaltwinsinstances', 'Digital Twins',\r\nstrcat(\"Not Translated: \", type))\r\n| summarize count() by type\r\n| order by count_ desc", + "size": 3, + "title": "Resource number by type (Top 10)", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "tiles", + "tileSettings": { + "showBorder": false, + "titleContent": { + "columnMatch": "type", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + }, + "rowLimit": 10 + } + }, + "name": "Resource number by type" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| summarize count() by location", + "size": 3, + "title": "Resource number by location", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "map", + "mapSettings": { + "locInfo": "AzureLoc", + "sizeSettings": "count_", + "sizeAggregation": "Sum", + "legendMetric": "count_", + "legendAggregation": "Sum", + "itemColorSettings": { + "type": "heatmap", + "colorAggregation": "Sum", + "nodeColorField": "count_", + "heatmapPalette": "greenRed" + }, + "labelSettings": "location", + "locInfoColumn": "location" + } + }, + "name": "Resource number by location" + } + ] + }, + "conditionalVisibility": { + "parameterName": "Subscription", + "comparison": "isNotEqualTo" + }, + "name": "group - Overview metrics" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RC_Overview" + }, + "name": "RC_Overview" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "## Things to know before creating a virtual machine\r\nThere's always a multitude of design considerations when you build out an application infrastructure in Azure. These aspects of a virtual machine are important to think about to manage virtual machine properly:\r\n- The names of your application resources\r\n- The location where the resources are stored\r\n- The size of the virtual machine\r\n- The maximum number of virtual machines that can be created\r\n- The operating system that the virtual machine runs\r\n- The configuration of the virtual machine after it starts\r\n- The related resources that the virtual machine needs\r\n" + }, + "name": "text - 13" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources | where type =~ 'Microsoft.Compute/virtualMachines'\n| summarize count() by tostring(properties.storageProfile.osDisk.osType)", + "size": 3, + "title": "Virtual machine count per OS type", + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "tiles", + "tileSettings": { + "showBorder": false, + "titleContent": { + "columnMatch": "properties_storageProfile_osDisk_osType", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + } + }, + "name": "Virtual machine count per OS type" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources\r\n| where type =~ 'Microsoft.Compute/virtualMachines'\r\n| project SKU = tostring(properties.hardwareProfile.vmSize)\r\n| summarize count() by SKU\r\n| order by count_ desc", + "size": 1, + "title": "VM by VM type/size", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "barchart", + "tileSettings": { + "showBorder": false, + "titleContent": { + "columnMatch": "SKU", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + }, + "rowLimit": 10 + }, + "graphSettings": { + "type": 0, + "topContent": { + "columnMatch": "SKU", + "formatter": 1 + }, + "centerContent": { + "columnMatch": "count_", + "formatter": 1, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + } + }, + "name": "VM by VM type/size" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type=~ 'microsoft.compute/virtualmachinescalesets'\r\n| project subscriptionId, name, location, resourceGroup, Capacity = toint(sku.capacity), Tier = sku.name\r\n| order by Capacity desc", + "size": 0, + "title": "Virtual machine scale set capacity and size", + "noDataMessageStyle": 3, + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "name": "query - virtual machine scale set capacity and size" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources | where type == \"microsoft.compute/virtualmachines\"\r\n| extend osDiskId= tostring(properties.storageProfile.osDisk.managedDisk.id)\r\n | join kind=leftouter(resources\r\n | where type =~ 'microsoft.compute/disks'\r\n | where properties !has 'Unattached'\r\n | where properties has 'osType'\r\n | project OS = tostring(properties.osType), osSku = tostring(sku.name), osDiskSizeGB = toint(properties.diskSizeGB), osDiskId=tostring(id)) on osDiskId\r\n | join kind=leftouter(Resources\r\n | where type =~ 'microsoft.compute/disks'\r\n | where properties !has \"osType\"\r\n | where properties !has 'Unattached'\r\n | project sku = tostring(sku.name), diskSizeGB = toint(properties.diskSizeGB), id = managedBy\r\n | summarize sum(diskSizeGB), count(sku) by id, sku) on id\r\n| project vmId=id, subscriptionId, resourceGroup, OS, location, osDiskId, osSku, osDiskSizeGB, DataDisksGB=sum_diskSizeGB, diskSkuCount=count_sku\r\n| sort by diskSkuCount desc", + "size": 0, + "title": "Compute disks", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "table", + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "location", + "formatter": 17 + } + ], + "filter": true, + "labelSettings": [ + { + "columnId": "vmId", + "label": "Resource Name" + }, + { + "columnId": "subscriptionId", + "label": "Subscription" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "location", + "label": "Region" + }, + { + "columnId": "osDiskId", + "label": "OS Disk" + }, + { + "columnId": "osSku", + "label": "OS Disk SKU" + }, + { + "columnId": "osDiskSizeGB", + "label": "OS Disk Size" + } + ] + } + }, + "name": "Compute disks" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources\r\n| where type =~ 'microsoft.compute/virtualmachines'\r\n| extend nics=array_length(properties.networkProfile.networkInterfaces)\r\n| mv-expand nic=properties.networkProfile.networkInterfaces\r\n| where nics == 1 or nic.properties.primary =~ 'true' or isempty(nic)\r\n| project vmId = id, vmName = name, vmSize=tostring(properties.hardwareProfile.vmSize), nicId = tostring(nic.id)\r\n\t| join kind=leftouter (\r\n \t\tResources\r\n \t\t| where type =~ 'microsoft.network/networkinterfaces'\r\n \t\t| extend ipConfigsCount=array_length(properties.ipConfigurations)\r\n \t\t| mv-expand ipconfig=properties.ipConfigurations\r\n \t\t| where ipConfigsCount == 1 or ipconfig.properties.primary =~ 'true'\r\n \t\t| project nicId = id, privateIP= tostring(ipconfig.properties.privateIPAddress), publicIpId = tostring(ipconfig.properties.publicIPAddress.id), subscriptionId) on nicId\r\n| project-away nicId1\r\n| summarize by vmId, subscriptionId, vmSize, nicId, privateIP, publicIpId\r\n\t| join kind=leftouter (\r\n \t\tResources\r\n \t\t| where type =~ 'microsoft.network/publicipaddresses'\r\n \t\t| project publicIpId = id, publicIpAddress = tostring(properties.ipAddress)) on publicIpId\r\n| project-away publicIpId1\r\n| sort by publicIpAddress desc", + "size": 0, + "title": "Compute networking", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "table", + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "filter": true, + "labelSettings": [ + { + "columnId": "vmId", + "label": "Resource name" + }, + { + "columnId": "subscriptionId", + "label": "Subscription" + }, + { + "columnId": "vmSize", + "label": "VM size" + }, + { + "columnId": "nicId", + "label": "Network interface" + }, + { + "columnId": "privateIP", + "label": "Private IP" + }, + { + "columnId": "publicIpId", + "label": "Public IP" + }, + { + "columnId": "publicIpAddress", + "label": "Public IP address" + } + ] + } + }, + "name": "Compute networking" + }, + { + "type": 1, + "content": { + "json": "# Managed disk utilization" + }, + "name": "text - 16" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "ec135d58-9c6b-4998-bd1e-75871c540d7f", + "version": "KqlParameterItem/1.0", + "name": "laworkspace", + "label": "Log Analytics workspace", + "type": 5, + "description": "LA workspaces configured in virtual machines insight settings", + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": "where type =~ 'microsoft.operationalinsights/workspaces' | project id", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": [] + } + ], + "style": "above", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "name": "Log Analytics workspace selector" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "InsightsMetrics\n| where Origin == \"vm.azm.ms\"\n| where Namespace == \"LogicalDisk\"\n| where Name == \"FreeSpacePercentage\"\n| extend t=parse_json(Tags)\n| summarize arg_max(TimeGenerated, *) by tostring(t[\"vm.azm.ms/mountId\"]), Computer // arg_max over TimeGenerated returns the latest record\n| project Computer, TimeGenerated, t[\"vm.azm.ms/mountId\"], Val\n", + "size": 4, + "title": "Managed disks free space", + "timeContext": { + "durationMs": 604800000 + }, + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces", + "crossComponentResources": [ + "{laworkspace}" + ], + "visualization": "table", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Val", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": ">=", + "thresholdValue": "90", + "representation": "4", + "text": "{0}{1}" + }, + { + "operator": ">=", + "thresholdValue": "50", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + }, + "numberFormat": { + "unit": 1, + "options": { + "style": "decimal", + "useGrouping": false, + "maximumFractionDigits": 0 + } + } + } + ], + "labelSettings": [ + { + "columnId": "Computer", + "label": "Computer" + }, + { + "columnId": "TimeGenerated", + "label": "TimeGenerated" + }, + { + "columnId": "t_vm.azm.ms/mountId", + "label": "Drive" + }, + { + "columnId": "Val", + "label": "Free space percentage" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "laworkspace", + "comparison": "isNotEqualTo" + }, + "name": "Managed disks free space" + }, + { + "type": 1, + "content": { + "json": "# Compute optimization" + }, + "name": "text - 9" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "advisorresources\r\n| where type == \"microsoft.advisor/recommendations\"\r\n| where tostring (properties.category) has \"Cost\"\r\n| where properties.shortDescription.problem has \"underutilized\"\r\n| where properties.impactedField has \"Compute\" or properties.impactedField has \"Container\" or properties.impactedField has \"Web\"\r\n| project AffectedResource=tostring(properties.resourceMetadata.resourceId),Impact=properties.impact,resourceGroup,AdditionaInfo=properties.extendedProperties,subscriptionId,Recommendation=tostring(properties.shortDescription.problem)\r\n", + "size": 0, + "title": "Underused assets", + "noDataMessage": "No underused asset", + "noDataMessageStyle": 3, + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "customWidth": "100", + "name": "Underused assets" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "37cdc20d-07c3-466c-84bb-4d8050932641", + "version": "KqlParameterItem/1.0", + "name": "OrphanDisks", + "label": "Orphaned", + "type": 10, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[[\r\n { \"value\":\"!=\", \"label\":\"No\" },\r\n { \"value\":\"==\", \"label\":\"Yes\" }\r\n]", + "timeContext": { + "durationMs": 86400000 + }, + "value": "!=" + } + ], + "style": "above", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "Disks" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources \r\n| where type contains \"microsoft.compute/disks\" \r\n| extend diskState = tostring(properties.diskState)\r\n| where managedBy {OrphanDisks} \"\" or diskState {OrphanDisks} 'Unattached'\r\n| project id, subscriptionId, resourceGroup, diskState, location", + "size": 0, + "title": "Managed disks", + "noDataMessageStyle": 3, + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "location", + "formatter": 17 + } + ], + "labelSettings": [ + { + "columnId": "id", + "label": "Resource Name" + }, + { + "columnId": "subscriptionId", + "label": "Subscription" + }, + { + "columnId": "resourceGroup", + "label": "Resource group" + }, + { + "columnId": "diskState", + "label": "Disk state" + }, + { + "columnId": "location", + "label": "Region" + } + ] + } + }, + "name": "Managed disks" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "2d9b8893-0af4-480a-9ac7-639efb771ecb", + "version": "KqlParameterItem/1.0", + "name": "OrphanNIC", + "label": "Orphaned", + "type": 10, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[[\r\n { \"value\":\"has 'virtualmachine' or isnotnull(privateEndPoint)\", \"label\":\"No\" },\r\n { \"value\":\"!has 'virtualmachine' and isnull(privateEndPoint)\", \"label\":\"Yes\" }\r\n]", + "timeContext": { + "durationMs": 86400000 + }, + "value": "has 'virtualmachine' or isnotnull(privateEndPoint)" + } + ], + "style": "above", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "NICs - Copy" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources\r\n| where type has \"microsoft.network/networkinterfaces\"\r\n| extend VM = properties.virtualMachine.id\r\n| extend privateEndPoint = properties['privateEndpoint']['id']\r\n| where properties {OrphanNIC}\r\n| where properties['linkedResourceType'] != \"Microsoft.Netapp/volumes\"\r\n| project id, subscriptionId, resourceGroup, location, VM, privateEndPoint, properties\r\n", + "size": 0, + "title": "NICs", + "noDataMessageStyle": 3, + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "location", + "formatter": 17 + }, + { + "columnMatch": "properties", + "formatter": 7, + "formatOptions": { + "linkTarget": "CellDetails", + "linkLabel": "🔍 View details", + "linkIsContextBlade": true + } + } + ], + "labelSettings": [ + { + "columnId": "id", + "label": "Resource name" + }, + { + "columnId": "subscriptionId", + "label": "Subscription" + }, + { + "columnId": "resourceGroup", + "label": "Resource group" + }, + { + "columnId": "location", + "label": "Region" + }, + { + "columnId": "VM", + "label": "Virtual machine" + }, + { + "columnId": "privateEndPoint", + "label": "Private end point" + }, + { + "columnId": "properties", + "label": "Details" + } + ] + } + }, + "name": "NICs" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "98d786aa-8835-493f-9fe4-fe5da150392b", + "version": "KqlParameterItem/1.0", + "name": "VMState", + "label": "Virtual machine state", + "type": 2, + "query": "resources\r\n| where type == \"microsoft.compute/virtualmachines\"\r\n| extend state = properties['extended']['instanceView']['powerState']['displayStatus']\r\n| summarize by tostring(state)", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null + } + ], + "style": "above", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "name": "parameters - VMState" + }, + { + "type": 1, + "content": { + "json": "Select a virtual machine state to display the list of resource.", + "style": "info" + }, + "conditionalVisibility": { + "parameterName": "VMState", + "comparison": "isEqualTo" + }, + "name": "text - VMState" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources | where type == \"microsoft.compute/virtualmachines\"\r\n| extend vmState = tostring(properties.extended.instanceView.powerState.displayStatus)\r\n| extend vmState = iif(isempty(vmState), \"VM State Unknown\", (vmState))\r\n| summarize count() by vmState", + "size": 3, + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "tiles", + "tileSettings": { + "titleContent": { + "columnMatch": "vmState", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + }, + "showBorder": false + }, + "graphSettings": { + "type": 0, + "topContent": { + "columnMatch": "vmState", + "formatter": 1 + }, + "centerContent": { + "columnMatch": "count_", + "formatter": 1, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + } + }, + "name": "query - VM state chart" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type == \"microsoft.compute/virtualmachines\"\r\n| extend vmSize = tostring(properties.hardwareProfile.vmSize)\r\n| extend vmState = properties['extended']['instanceView']['powerState']['displayStatus']\r\n| where vmState == '{VMState}'\r\n| project id, subscriptionId, resourceGroup, vmState, vmSize, location", + "size": 0, + "title": "Virtual machine list by powerstate", + "noDataMessageStyle": 3, + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "resourceGroup", + "formatter": 14, + "formatOptions": { + "linkTarget": null, + "showIcon": false + } + }, + { + "columnMatch": "id", + "formatter": 13, + "formatOptions": { + "linkTarget": "Resource", + "showIcon": true + } + }, + { + "columnMatch": "vmSize", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "19.1429ch" + } + }, + { + "columnMatch": "location", + "formatter": 17, + "formatOptions": { + "customColumnWidthSetting": "108px" + } + } + ], + "filter": true, + "labelSettings": [ + { + "columnId": "id", + "label": "Resource Name" + }, + { + "columnId": "subscriptionId", + "label": "Subscription" + }, + { + "columnId": "resourceGroup", + "label": "Resource Group" + }, + { + "columnId": "vmState", + "label": "VM State" + }, + { + "columnId": "vmSize", + "label": "VM Size" + }, + { + "columnId": "location", + "label": "Region" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "VMState", + "comparison": "isNotEqualTo" + }, + "name": "query - VM list by powerstate" + }, + { + "type": 1, + "content": { + "json": "States and billing status of Azure virtual machines : https://learn.microsoft.com/azure/virtual-machines/states-billing", + "style": "info" + }, + "name": "Info VM states" + } + ] + }, + "conditionalVisibility": { + "parameterName": "Subscription", + "comparison": "isNotEqualTo" + }, + "name": "group - VMQueries" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RC_VM" + }, + "name": "RC_VM" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "## Storage account + backup" + }, + "name": "text - 9" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where type =~ 'microsoft.storagesync/storagesyncservices'\r\n\tor type =~ 'microsoft.recoveryservices/vaults'\r\n\tor type =~ 'microsoft.storage/storageaccounts'\r\n\tor type =~ 'microsoft.keyvault/vaults'\r\n| extend type = case(\r\n\ttype =~ 'microsoft.storagesync/storagesyncservices', 'Azure File Sync',\r\n\ttype =~ 'microsoft.recoveryservices/vaults', 'Azure Backup',\r\n\ttype =~ 'microsoft.storage/storageaccounts', 'Storage Accounts',\r\n\ttype =~ 'microsoft.keyvault/vaults', 'Key Vaults',\r\n\tstrcat(\"Not Translated: \", type))\r\n| where type !has \"Not Translated\"\r\n| summarize count() by type", + "size": 3, + "title": "Count of all resource types", + "noDataMessage": "No resources found", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "tiles", + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 13, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Resource", + "formatter": 5 + }, + { + "columnMatch": "subscriptionId", + "formatter": 5 + } + ], + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "subscriptionId" + ], + "expandTopLevel": true, + "finalBy": "Resource" + } + }, + "tileSettings": { + "titleContent": { + "columnMatch": "type", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + }, + "showBorder": false, + "sortCriteriaField": "type", + "sortOrderField": 1 + } + }, + "name": "query - Storage - Resource Overview " + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where type =~ 'microsoft.storagesync/storagesyncservices'\r\n\tor type =~ 'microsoft.recoveryservices/vaults'\r\n\tor type =~ 'microsoft.storage/storageaccounts'\r\n\tor type =~ 'microsoft.keyvault/vaults'\r\n| extend type = case(\r\n\ttype =~ 'microsoft.storagesync/storagesyncservices', 'Azure File Sync',\r\n\ttype =~ 'microsoft.recoveryservices/vaults', 'Azure Backup',\r\n\ttype =~ 'microsoft.storage/storageaccounts', 'Storage Accounts',\r\n\ttype =~ 'microsoft.keyvault/vaults', 'Key Vaults',\r\n\tstrcat(\"Not Translated: \", type))\r\n| extend Sku = case(\r\n\ttype !has 'Key Vaults', sku.name,\r\n\ttype =~ 'Key Vaults', properties.sku.name,\r\n\t' ')\r\n| extend Details = pack_all()\r\n| project Resource=id, type, kind, subscriptionId, resourceGroup, Sku, Details", + "size": 0, + "title": "Resource details", + "noDataMessage": "No resources found", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 13, + "formatOptions": { + "linkTarget": "Resource", + "showIcon": true + } + }, + { + "columnMatch": "Resource", + "formatter": 5 + }, + { + "columnMatch": "subscriptionId", + "formatter": 5 + }, + { + "columnMatch": "Details", + "formatter": 7, + "formatOptions": { + "linkTarget": "CellDetails", + "linkLabel": "🔍 View Details", + "linkIsContextBlade": true + } + } + ], + "rowLimit": 1000, + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "subscriptionId" + ], + "expandTopLevel": true, + "finalBy": "Resource" + } + } + }, + "name": "query - Storage - Resource Detailed" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "e94aafa3-c5d9-4523-89f0-4e87aa754511", + "version": "KqlParameterItem/1.0", + "name": "Resources", + "label": "Storage accounts", + "type": 5, + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": "where type =~ 'microsoft.storage/storageaccounts'\n| order by name asc\n| extend Rank = row_number()\n| project value = id, label = id, selected = Rank <= 5", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "resourceTypeFilter": { + "microsoft.storage/storageaccounts": true + }, + "additionalResourceOptions": [ + "value::all" + ], + "showDefault": false + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": [ + "value::all" + ] + }, + { + "id": "c4b69c01-2263-4ada-8d9c-43433b739ff3", + "version": "KqlParameterItem/1.0", + "name": "TimeRange", + "type": 4, + "isRequired": true, + "typeSettings": { + "selectableValues": [ + { + "durationMs": 300000, + "createdTime": "2018-08-06T23:52:38.87Z", + "isInitialTime": false, + "grain": 1, + "useDashboardTimeRange": false + }, + { + "durationMs": 900000, + "createdTime": "2018-08-06T23:52:38.871Z", + "isInitialTime": false, + "grain": 1, + "useDashboardTimeRange": false + }, + { + "durationMs": 1800000, + "createdTime": "2018-08-06T23:52:38.871Z", + "isInitialTime": false, + "grain": 1, + "useDashboardTimeRange": false + }, + { + "durationMs": 3600000, + "createdTime": "2018-08-06T23:52:38.871Z", + "isInitialTime": false, + "grain": 1, + "useDashboardTimeRange": false + }, + { + "durationMs": 14400000, + "createdTime": "2018-08-06T23:52:38.871Z", + "isInitialTime": false, + "grain": 1, + "useDashboardTimeRange": false + }, + { + "durationMs": 43200000, + "createdTime": "2018-08-06T23:52:38.871Z", + "isInitialTime": false, + "grain": 1, + "useDashboardTimeRange": false + }, + { + "durationMs": 86400000, + "createdTime": "2018-08-06T23:52:38.871Z", + "isInitialTime": false, + "grain": 1, + "useDashboardTimeRange": false + }, + { + "durationMs": 172800000, + "createdTime": "2018-08-06T23:52:38.871Z", + "isInitialTime": false, + "grain": 1, + "useDashboardTimeRange": false + }, + { + "durationMs": 259200000, + "createdTime": "2018-08-06T23:52:38.871Z", + "isInitialTime": false, + "grain": 1, + "useDashboardTimeRange": false + }, + { + "durationMs": 604800000, + "createdTime": "2018-08-06T23:52:38.871Z", + "isInitialTime": false, + "grain": 1, + "useDashboardTimeRange": false + } + ], + "allowCustom": true + }, + "value": { + "durationMs": 172800000 + }, + "label": "Time range" + }, + { + "id": "9b48988f-dcd2-48cc-b233-5999ed32149f", + "version": "KqlParameterItem/1.0", + "name": "Message", + "type": 1, + "query": "where type == 'microsoft.storage/storageaccounts' \n| summarize Selected = countif(id in ({Resources:value})), Total = count()\n| extend Selected = iff(Selected > 200, 200, Selected)\n| project Message = strcat('# ', Selected, ' / ', Total)", + "crossComponentResources": [ + "{Subscription}" + ], + "isHiddenWhenLocked": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + { + "id": "070b2474-4e01-478d-a7fa-6c20ad8ea1ad", + "version": "KqlParameterItem/1.0", + "name": "ResourceName", + "type": 1, + "isRequired": true, + "isHiddenWhenLocked": true, + "criteriaData": [ + { + "condition": "else result = 'Storage account'", + "criteriaContext": { + "operator": "Default", + "rightValType": "param", + "resultValType": "static", + "resultVal": "Storage account" + } + } + ] + }, + { + "id": "c6c32b32-6eb4-44d5-9cad-156d5d50ec3e", + "version": "KqlParameterItem/1.0", + "name": "ResourceImageUrl", + "type": 1, + "description": "used as a parameter for No Subcriptions workbook template", + "isHiddenWhenLocked": true + } + ], + "style": "above", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "parameters - 1", + "styleSettings": { + "margin": "15px 0 0 0" + } + }, + { + "type": 1, + "content": { + "json": "## Storage accounts details" + }, + "name": "text - 8" + }, + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "27d282bb-38ae-4ceb-b2bb-063db08ec6bc", + "cellValue": "selectedTab", + "linkTarget": "parameter", + "linkLabel": "Overview", + "subTarget": "Overview" + }, + { + "id": "9a52f588-fff8-47fe-b56d-81b8068ff6f7", + "cellValue": "selectedTab", + "linkTarget": "parameter", + "linkLabel": "Capacity", + "subTarget": "Capacity" + } + ] + }, + "name": "Navigation links", + "styleSettings": { + "margin": "10px 0 0 0" + } + }, + { + "type": 1, + "content": { + "json": "### Overview section" + }, + "conditionalVisibility": { + "parameterName": "1", + "comparison": "isEqualTo", + "value": "2" + }, + "name": "text - 4" + }, + { + "type": 10, + "content": { + "chartId": "workbookdb19a8d8-91af-44ea-951d-5ffa133b2ebe", + "version": "MetricsItem/2.0", + "size": 2, + "chartType": 0, + "resourceType": "microsoft.storage/storageaccounts", + "metricScope": 0, + "resourceParameter": "Resources", + "resourceIds": [ + "{Resources}" + ], + "timeContextFromParameter": "TimeRange", + "timeContext": { + "durationMs": 172800000 + }, + "metrics": [ + { + "namespace": "microsoft.storage/storageaccounts", + "metric": "microsoft.storage/storageaccounts-Transaction-Transactions", + "aggregation": 1 + }, + { + "namespace": "microsoft.storage/storageaccounts", + "metric": "microsoft.storage/storageaccounts-Transaction-SuccessE2ELatency", + "aggregation": 4 + }, + { + "namespace": "microsoft.storage/storageaccounts", + "metric": "microsoft.storage/storageaccounts-Transaction-SuccessServerLatency", + "aggregation": 4 + }, + { + "namespace": "microsoft.storage/storageaccounts", + "metric": "microsoft.storage/storageaccounts-Transaction-Transactions", + "aggregation": 1, + "splitBy": [ + "ResponseType" + ], + "splitBySortOrder": -1, + "splitByLimit": 4, + "columnName": "Errors" + } + ], + "resourceLimit": 200, + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 13, + "formatOptions": { + "linkTarget": "Resource", + "subTarget": "insights", + "showIcon": true + } + }, + { + "columnMatch": "Subscription", + "formatter": 5, + "formatOptions": { + "showIcon": true + } + }, + { + "columnMatch": "Name", + "formatter": 5, + "formatOptions": { + "showIcon": true + } + }, + { + "columnMatch": "microsoft.storage/storageaccounts-Transaction-Transactions$|Transactions$", + "formatter": 8, + "formatOptions": { + "min": 0, + "palette": "blue", + "showIcon": true, + "aggregation": "Sum" + }, + "numberFormat": { + "unit": 17, + "options": { + "style": "decimal", + "maximumFractionDigits": 1 + } + } + }, + { + "columnMatch": "microsoft.storage/storageaccounts-Transaction-Transactions Timeline$|Transactions Timeline$", + "formatter": 21, + "formatOptions": { + "min": 0, + "palette": "blue", + "showIcon": true + } + }, + { + "columnMatch": "microsoft.storage/storageaccounts-Transaction-SuccessE2ELatency$|microsoft.storage/storageaccounts-Transaction-SuccessServerLatency$|E2E Latency$|Server Latency$", + "formatter": 8, + "formatOptions": { + "min": 0, + "palette": "blue", + "linkTarget": "WorkbookTemplate", + "showIcon": true, + "workbookContext": { + "componentIdSource": "column", + "componentId": "Name", + "resourceIdsSource": "column", + "resourceIds": "Name", + "templateIdSource": "static", + "templateId": "Community-Workbooks/Individual Storage/Performance", + "typeSource": "static", + "type": "workbook", + "gallerySource": "static", + "gallery": "microsoft.storage/storageaccounts" + } + }, + "numberFormat": { + "unit": 23, + "options": { + "style": "decimal", + "maximumFractionDigits": 2 + } + } + }, + { + "columnMatch": "microsoft.storage/storageaccounts-Transaction-SuccessE2ELatency Timeline$|E2E Latency Timeline$", + "formatter": 5, + "formatOptions": { + "showIcon": true + } + }, + { + "columnMatch": "microsoft.storage/storageaccounts-Transaction-SuccessServerLatency Timeline", + "formatter": 5, + "formatOptions": { + "showIcon": true + } + }, + { + "columnMatch": "Success/Errors", + "formatter": 5, + "formatOptions": { + "showIcon": true + } + }, + { + "columnMatch": "success/Errors", + "formatter": 5, + "formatOptions": { + "showIcon": true + } + }, + { + "columnMatch": ".*\\/Errors", + "formatter": 8, + "formatOptions": { + "min": 0, + "palette": "gray", + "linkTarget": "WorkbookTemplate", + "showIcon": true, + "workbookContext": { + "componentIdSource": "column", + "componentId": "Name", + "resourceIdsSource": "column", + "resourceIds": "Name", + "templateIdSource": "static", + "templateId": "Community-Workbooks/Individual Storage/Failures", + "typeSource": "static", + "type": "workbook", + "gallerySource": "static", + "gallery": "microsoft.storage/storageaccounts" + } + }, + "numberFormat": { + "unit": 17, + "options": { + "style": "decimal", + "maximumFractionDigits": 1 + } + } + }, + { + "columnMatch": "Server Latency Timeline", + "formatter": 5, + "formatOptions": { + "showIcon": true + } + } + ], + "rowLimit": 10000, + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "Subscription" + ], + "expandTopLevel": true, + "finalBy": "Name" + }, + "sortBy": [ + { + "itemKey": "$gen_heatmap_microsoft.storage/storageaccounts-Transaction-Transactions$|Transactions$_3", + "sortOrder": 2 + } + ], + "labelSettings": [ + { + "columnId": "Subscription", + "label": "Subscription" + }, + { + "columnId": "microsoft.storage/storageaccounts-Transaction-Transactions", + "label": "Transactions" + }, + { + "columnId": "microsoft.storage/storageaccounts-Transaction-Transactions Timeline", + "label": "Transactions timeline" + }, + { + "columnId": "microsoft.storage/storageaccounts-Transaction-SuccessE2ELatency", + "label": "E2E latency" + }, + { + "columnId": "microsoft.storage/storageaccounts-Transaction-SuccessE2ELatency Timeline", + "label": "E2E latency timeline" + }, + { + "columnId": "microsoft.storage/storageaccounts-Transaction-SuccessServerLatency", + "label": "Server latency" + }, + { + "columnId": "microsoft.storage/storageaccounts-Transaction-SuccessServerLatency Timeline", + "label": "Server latency timeline" + } + ] + }, + "sortBy": [ + { + "itemKey": "$gen_heatmap_microsoft.storage/storageaccounts-Transaction-Transactions$|Transactions$_3", + "sortOrder": 2 + } + ], + "showExportToExcel": true + }, + "conditionalVisibility": { + "parameterName": "selectedTab", + "comparison": "isEqualTo", + "value": "Overview" + }, + "showPin": true, + "name": "storage account metrics", + "styleSettings": { + "margin": "0 10px 0 10px" + } + }, + { + "type": 1, + "content": { + "json": "## Capacity section" + }, + "conditionalVisibility": { + "parameterName": "1", + "comparison": "isEqualTo", + "value": "2" + }, + "name": "text - 6" + }, + { + "type": 10, + "content": { + "chartId": "workbookdb19a8d8-91af-44ea-951d-5ffa133b2ebe", + "version": "MetricsItem/2.0", + "size": 3, + "chartType": 0, + "resourceType": "microsoft.storage/storageaccounts", + "metricScope": 0, + "resourceParameter": "Resources", + "resourceIds": [ + "{Resources}" + ], + "timeContextFromParameter": "TimeRange", + "timeContext": { + "durationMs": 172800000 + }, + "metrics": [ + { + "namespace": "microsoft.storage/storageaccounts", + "metric": "microsoft.storage/storageaccounts-Capacity-UsedCapacity", + "aggregation": 4 + }, + { + "namespace": "microsoft.storage/storageaccounts/blobservices", + "metric": "microsoft.storage/storageaccounts/blobservices-Capacity-BlobCapacity", + "aggregation": 4 + }, + { + "namespace": "microsoft.storage/storageaccounts/fileservices", + "metric": "microsoft.storage/storageaccounts/fileservices-Capacity-FileCapacity", + "aggregation": 4 + }, + { + "namespace": "microsoft.storage/storageaccounts/queueservices", + "metric": "microsoft.storage/storageaccounts/queueservices-Capacity-QueueCapacity", + "aggregation": 4 + }, + { + "namespace": "microsoft.storage/storageaccounts/tableservices", + "metric": "microsoft.storage/storageaccounts/tableservices-Capacity-TableCapacity", + "aggregation": 4 + } + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 13, + "formatOptions": { + "linkTarget": "Resource", + "subTarget": "insights", + "showIcon": true + } + }, + { + "columnMatch": "Subscription", + "formatter": 5, + "formatOptions": { + "showIcon": true + } + }, + { + "columnMatch": "Name", + "formatter": 5, + "formatOptions": { + "showIcon": true + } + }, + { + "columnMatch": "microsoft.storage/storageaccounts-Capacity-UsedCapacity$|microsoft.storage/storageaccounts/blobservices-Capacity-BlobCapacity$|microsoft.storage/storageaccounts/fileservices-Capacity-FileCapacity$|microsoft.storage/storageaccounts/queueservices-Capacity-QueueCapacity$|microsoft.storage/storageaccounts/tableservices-Capacity-TableCapacity$", + "formatter": 8, + "formatOptions": { + "min": 0, + "palette": "blue", + "linkTarget": "WorkbookTemplate", + "showIcon": true, + "workbookContext": { + "componentIdSource": "column", + "componentId": "Name", + "resourceIdsSource": "column", + "resourceIds": "Name", + "templateIdSource": "static", + "templateId": "Community-Workbooks/Individual Storage/Capacity", + "typeSource": "static", + "type": "workbook", + "gallerySource": "static", + "gallery": "microsoft.storage/storageaccounts" + } + }, + "numberFormat": { + "unit": 2, + "options": { + "style": "decimal", + "maximumFractionDigits": 1 + } + } + }, + { + "columnMatch": "microsoft.storage/storageaccounts-Capacity-UsedCapacity Timeline$|Account used capacity Timeline$", + "formatter": 21, + "formatOptions": { + "min": 0, + "palette": "blue", + "showIcon": true + } + }, + { + "columnMatch": "microsoft.storage/storageaccounts/blobservices-Capacity-BlobCapacity Timeline$|Blob capacity Timeline$", + "formatter": 5, + "formatOptions": { + "showIcon": true + } + }, + { + "columnMatch": "microsoft.storage/storageaccounts/fileservices-Capacity-FileCapacity Timeline$|File capacity Timeline$", + "formatter": 5, + "formatOptions": { + "showIcon": true + } + }, + { + "columnMatch": "microsoft.storage/storageaccounts/queueservices-Capacity-QueueCapacity Timeline$|Queue capacity Timeline$", + "formatter": 5, + "formatOptions": { + "showIcon": true + } + }, + { + "columnMatch": "microsoft.storage/storageaccounts/tableservices-Capacity-TableCapacity Timeline$|Table capacity Timeline$", + "formatter": 5, + "formatOptions": { + "showIcon": true + } + } + ], + "rowLimit": 10000, + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "Subscription" + ], + "expandTopLevel": true, + "finalBy": "Name" + }, + "sortBy": [ + { + "itemKey": "$gen_link_$gen_group_0", + "sortOrder": 1 + } + ], + "labelSettings": [ + { + "columnId": "Subscription", + "label": "Subscription" + }, + { + "columnId": "microsoft.storage/storageaccounts-Capacity-UsedCapacity", + "label": "Account used capacity" + }, + { + "columnId": "microsoft.storage/storageaccounts-Capacity-UsedCapacity Timeline", + "label": "Account used capacity timeline" + }, + { + "columnId": "microsoft.storage/storageaccounts/blobservices-Capacity-BlobCapacity", + "label": "Blob capacity" + }, + { + "columnId": "microsoft.storage/storageaccounts/blobservices-Capacity-BlobCapacity Timeline", + "label": "Blob capacity timeline" + }, + { + "columnId": "microsoft.storage/storageaccounts/fileservices-Capacity-FileCapacity", + "label": "File capacity" + }, + { + "columnId": "microsoft.storage/storageaccounts/fileservices-Capacity-FileCapacity Timeline", + "label": "File capacity timeline" + }, + { + "columnId": "microsoft.storage/storageaccounts/queueservices-Capacity-QueueCapacity", + "label": "Queue capacity" + }, + { + "columnId": "microsoft.storage/storageaccounts/queueservices-Capacity-QueueCapacity Timeline", + "label": "Queue capacity timeline" + }, + { + "columnId": "microsoft.storage/storageaccounts/tableservices-Capacity-TableCapacity", + "label": "Table capacity" + }, + { + "columnId": "microsoft.storage/storageaccounts/tableservices-Capacity-TableCapacity Timeline", + "label": "Table capacity timeline" + } + ] + }, + "sortBy": [ + { + "itemKey": "$gen_link_$gen_group_0", + "sortOrder": 1 + } + ], + "showExportToExcel": true + }, + "conditionalVisibility": { + "parameterName": "selectedTab", + "comparison": "isEqualTo", + "value": "Capacity" + }, + "showPin": true, + "name": "storage account capacity metrics", + "styleSettings": { + "margin": "0 10px 0 10px" + } + } + ] + }, + "conditionalVisibility": { + "parameterName": "Subscription", + "comparison": "isNotEqualTo", + "value": "" + }, + "name": "Storage account + backup" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "Azure Backup now provides a set of customizable reporting templates to help you generate audit evidence reports for backup in an easier way. [Learn more](https://aka.ms/BCDRAuditReportTemplates).", + "style": "upsell" + }, + "name": "AuditText" + }, + { + "type": 1, + "content": { + "json": "## Backup details\r\n### Manage and securely backup your resources\r\nExplore and monitor backup estate at scale in real time across vaults." + }, + "name": "text - 8" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "2373a24f-ad32-4909-a7f6-59b373dcde6c", + "version": "KqlParameterItem/1.0", + "name": "Workspaces", + "label": "Workspace", + "type": 5, + "description": "LA workspaces configured in vault diagnostic settings", + "isRequired": true, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": "where type =~ 'microsoft.operationalinsights/workspaces' | project id", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": [] + } + ], + "style": "above", + "doNotRunWhenHidden": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "100", + "name": "Filters1" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Workspaces}" + ], + "parameters": [ + { + "id": "2965ad33-1401-47c9-8f4b-9b7126f87014", + "version": "KqlParameterItem/1.0", + "name": "TimeRange", + "label": "Time Range", + "type": 4, + "description": "Period of time for which reports should be viewed", + "isRequired": true, + "typeSettings": { + "selectableValues": [ + { + "durationMs": 259200000 + }, + { + "durationMs": 604800000 + }, + { + "durationMs": 1209600000 + }, + { + "durationMs": 2419200000 + }, + { + "durationMs": 2592000000 + }, + { + "durationMs": 5184000000 + }, + { + "durationMs": 7776000000 + } + ], + "allowCustom": true + }, + "value": { + "durationMs": 604800000 + } + }, + { + "id": "efede5fa-f577-4766-b9b6-6ba4e525f844", + "version": "KqlParameterItem/1.0", + "name": "DataSourceSubscription", + "label": "Datasource Subscription", + "type": 6, + "description": "Use to filter for datasources within a specific subscription", + "isRequired": true, + "multiSelect": true, + "quote": "", + "delimiter": ",", + "query": "let RangeStart = startofday({TimeRange:start});\r\nlet RangeEnd = iff(startofday({TimeRange:end}) == startofday(now()) ,startofday({TimeRange:end}) - 1d , startofday({TimeRange:end}));\r\nlet VaultSubscriptionList = \"*\";\r\nlet VaultLocationList = \"*\";\r\nlet VaultList = \"*\";\r\nlet VaultTypeList = \"*\";\r\nlet ExcludeLegacyEvent = true;\r\nlet BackupSolutionList = \"*\";\r\nlet ProtectionInfoList = \"*\";\r\nlet Item_search = \"*;*\";\r\nlet ItemArray = split(Item_search, \";\");\r\nlet ItemArray_length = array_length(ItemArray);\r\nlet BackupInstanceName = iff(ItemArray_length == 2, ItemArray[1], ItemArray[0] );\r\nlet DatasourceSetName = iff(ItemArray_length == 2, ItemArray[0], \"\");\r\nlet DisplayAllFields = false;\r\n_AzureBackup_GetBackupInstances(RangeStart, RangeEnd, VaultSubscriptionList, VaultLocationList, VaultList, VaultTypeList, ExcludeLegacyEvent, BackupSolutionList, ProtectionInfoList, DatasourceSetName, BackupInstanceName, DisplayAllFields)\r\n| distinct tostring(split(tostring(todynamic(DatasourceResourceId)),\"/\")[2])", + "crossComponentResources": [ + "{Workspaces}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "selectAllValue": "*", + "showDefault": false + }, + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces", + "value": [] + }, + { + "id": "256c7e33-df90-4956-aaf3-699aeaad912f", + "version": "KqlParameterItem/1.0", + "name": "DataSourceLocation", + "label": "Data source location", + "type": 2, + "description": "Use to filter for data sources within a specific location", + "isRequired": true, + "multiSelect": true, + "quote": "", + "delimiter": ",", + "query": "let RangeStart = startofday({TimeRange:start});\r\nlet RangeEnd = iff(startofday({TimeRange:end}) == startofday(now()) ,startofday({TimeRange:end}) - 1d , startofday({TimeRange:end}));\r\nlet VaultSubscriptionList = todynamic( replace(\"/subscriptions/\", \"\", @\"{DataSourceSubscription}\"));\r\nlet VaultLocationList = \"*\";\r\nlet VaultList = \"*\";\r\nlet VaultTypeList = \"*\";\r\nlet ExcludeLegacyEvent = true;\r\nlet BackupSolutionList = \"*\";\r\nlet ProtectionInfoList = \"*\";\r\nlet Item_search = \"*;*\";\r\nlet ItemArray = split(Item_search, \";\");\r\nlet ItemArray_length = array_length(ItemArray);\r\nlet BackupInstanceName = iff(ItemArray_length == 2, ItemArray[1], ItemArray[0] );\r\nlet DatasourceSetName = iff(ItemArray_length == 2, ItemArray[0], \"\");\r\nlet DisplayAllFields = false;\r\n_AzureBackup_GetBackupInstances(RangeStart, RangeEnd, VaultSubscriptionList, VaultLocationList, VaultList, VaultTypeList, ExcludeLegacyEvent, BackupSolutionList, ProtectionInfoList, DatasourceSetName, BackupInstanceName, DisplayAllFields)\r\n| distinct VaultLocation", + "crossComponentResources": [ + "{Workspaces}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "selectAllValue": "*", + "showDefault": false + }, + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces", + "value": [ + "value::all" + ] + }, + { + "id": "16ad110f-4ea3-44d6-826b-4ea3bbd68c93", + "version": "KqlParameterItem/1.0", + "name": "JobOperation", + "label": "Job Operation", + "type": 2, + "description": "Use to filter for a particular operation type", + "isRequired": true, + "multiSelect": true, + "quote": "", + "delimiter": ",", + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "selectAllValue": "*", + "showDefault": false + }, + "jsonData": "\r\n[ \r\n{ \"value\": \"Backup\", \t\t\t\t\t\t\"label\": \"Backup\" },\r\n{ \"value\": \"Restore\", \t\t\t\t\t\t\"label\": \"Restore\" }\r\n]", + "value": [ + "value::all" + ] + }, + { + "id": "6a6222bf-a28a-4c98-9d74-838e74497167", + "version": "KqlParameterItem/1.0", + "name": "JobStatus", + "label": "Job Status", + "type": 2, + "description": "Use to filter for a particular job status", + "isRequired": true, + "multiSelect": true, + "quote": "", + "delimiter": ",", + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "selectAllValue": "*", + "showDefault": false + }, + "jsonData": "\r\n[ \r\n{ \"value\": \"Completed\", \t\t\t\t\t\t\"label\": \"Completed\" },\r\n{ \"value\": \"Failed\", \t\t\t\"label\": \"Failed\" },\r\n\r\n{ \"value\": \"CompletedWithWarnings\", \t\t\t\t\t\t\"label\": \"CompletedWithWarnings\" },\r\n{ \"value\": \"Cancelled\", \"label\": \"Cancelled\" }\r\n]", + "value": [ + "value::all" + ] + }, + { + "id": "849a6401-cbaf-44b9-a733-0819f8923791", + "version": "KqlParameterItem/1.0", + "name": "SearchItem", + "label": "Search Item", + "type": 1, + "description": "Use to search for an item by name" + } + ], + "style": "above", + "doNotRunWhenHidden": true, + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "customWidth": "100", + "conditionalVisibility": { + "parameterName": "Workspaces", + "comparison": "isNotEqualTo" + }, + "name": "Filters2" + }, + { + "type": 1, + "content": { + "json": "## Backup job history" + }, + "conditionalVisibility": { + "parameterName": "Workspaces", + "comparison": "isNotEqualTo" + }, + "name": "Heading2" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "\r\nlet RangeStart = startofday({TimeRange:start});\r\nlet RangeEnd = iff(startofday({TimeRange:end}) == startofday(now()) ,startofday({TimeRange:end}) - 1d , startofday({TimeRange:end}));\r\nlet DataSourceSubscriptionList = todynamic( replace(\"/subscriptions/\", \"\", @\"{DataSourceSubscription}\"));\r\nlet DataSourceLocationList = todynamic( @\"{DataSourceLocation}\"); \r\nlet VaultTypeList = \"*\";\r\nlet VaultList = \"*\";\r\nlet ExcludeLegacyEvent = true;\r\nlet BackupSolutionList = \"*\";\r\nlet ProtectionInfoList = \"*\";\r\nlet Item_search = \"*;*\";\r\nlet ItemArray = split(Item_search, \";\");\r\nlet ItemArray_length = array_length(ItemArray);\r\nlet BackupInstanceName = iff(ItemArray_length == 2, ItemArray[1], ItemArray[0] );\r\nlet DatasourceSetName = iff(ItemArray_length == 2, ItemArray[0], \"\");\r\nlet JobOperationList = todynamic( @\"{JobOperation}\"); \r\nlet JobStatusList = todynamic( @\"{JobStatus}\");\r\nlet JobFailureCodeList = \"*\";\r\nlet ExcludeLog = true; \r\n_AzureBackup_GetJobs(RangeStart, RangeEnd, DataSourceSubscriptionList, DataSourceLocationList, VaultList, VaultTypeList, ExcludeLegacyEvent, BackupSolutionList, JobOperationList, JobStatusList, JobFailureCodeList, DatasourceSetName, BackupInstanceName, ExcludeLog)\r\n| where BackupInstanceFriendlyName contains iff(isnotempty('{SearchItem}'),'{SearchItem}',BackupInstanceFriendlyName)\r\n| sort by BackupInstanceId\r\n| summarize count() by Status", + "size": 3, + "title": "Jobs by Status", + "noDataMessage": "No record found for the selected time and scope.", + "showRefreshButton": true, + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces", + "crossComponentResources": [ + "{Workspaces}" + ], + "visualization": "piechart", + "tileSettings": { + "showBorder": false, + "titleContent": { + "columnMatch": "UniqueId", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "DurationInSecs", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + } + }, + "customWidth": "0", + "conditionalVisibility": { + "parameterName": "Workspaces", + "comparison": "isNotEqualTo" + }, + "name": "Chart1", + "styleSettings": { + "showBorder": true + } + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Workspaces}" + ], + "parameters": [ + { + "id": "7a64467f-eec7-495b-9099-233fb7bceb08", + "version": "KqlParameterItem/1.0", + "name": "RowsPerPage", + "label": "Rows per page", + "type": 2, + "description": "Number of rows to display in a single page", + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[[\r\n { \"value\":10, \"label\":\"10\", \"selected\":true },\r\n { \"value\":25, \"label\":\"25\" },\r\n { \"value\":50, \"label\":\"50\" },\r\n { \"value\":100, \"label\":\"100\" },\r\n { \"value\":250, \"label\":\"250\" },\r\n { \"value\":500, \"label\":\"500\" },\r\n { \"value\":1000, \"label\":\"1000\" }\r\n]" + }, + { + "id": "5c65bc61-a721-42b7-960b-3fe7a6170eb6", + "version": "KqlParameterItem/1.0", + "name": "Page", + "type": 2, + "description": "Page number", + "isRequired": true, + "query": "\r\nlet RangeStart = startofday({TimeRange:start});\r\nlet RangeEnd = iff(startofday({TimeRange:end}) == startofday(now()) ,startofday({TimeRange:end}) - 1d , startofday({TimeRange:end}));\r\nlet DataSourceSubscriptionList = todynamic( replace(\"/subscriptions/\", \"\", @\"{DataSourceSubscription}\"));\r\nlet DataSourceLocationList = todynamic( @\"{DataSourceLocation}\"); \r\nlet VaultTypeList = \"*\";\r\nlet VaultList = \"*\";\r\nlet ExcludeLegacyEvent = true;\r\nlet BackupSolutionList = \"*\";\r\nlet ProtectionInfoList = \"*\";\r\nlet Item_search = \"*;*\";\r\nlet ItemArray = split(Item_search, \";\");\r\nlet ItemArray_length = array_length(ItemArray);\r\nlet BackupInstanceName = iff(ItemArray_length == 2, ItemArray[1], ItemArray[0] );\r\nlet DatasourceSetName = iff(ItemArray_length == 2, ItemArray[0], \"\");\r\nlet JobOperationList = todynamic( @\"{JobOperation}\"); \r\nlet JobStatusList = todynamic( @\"{JobStatus}\");\r\nlet JobFailureCodeList = \"*\";\r\nlet ExcludeLog = true; \r\nlet backupItem = '{SearchItem}';\r\n_AzureBackup_GetJobs(RangeStart, RangeEnd, DataSourceSubscriptionList, DataSourceLocationList, VaultList, VaultTypeList, ExcludeLegacyEvent, BackupSolutionList, JobOperationList, JobStatusList, JobFailureCodeList, DatasourceSetName, BackupInstanceName, ExcludeLog)\r\n| where BackupInstanceFriendlyName contains backupItem\r\n| summarize c=count()\r\n| project num = (c-1)/toint('{RowsPerPage}') + 1\r\n| project nums = range(1,num,1), num\r\n| mvexpand nums\r\n| project nums = tostring(nums), num = strcat(tostring(nums),\" of \",tostring(num))\r\n\r\n", + "crossComponentResources": [ + "{Workspaces}" + ], + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces", + "value": "1" + } + ], + "style": "above", + "doNotRunWhenHidden": true, + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "customWidth": "100", + "conditionalVisibility": { + "parameterName": "Workspaces", + "comparison": "isNotEqualTo" + }, + "name": "Filters3" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "\r\nlet RangeStart = startofday({TimeRange:start});\r\nlet RangeEnd = iff(startofday({TimeRange:end}) == startofday(now()) ,startofday({TimeRange:end}) - 1d , startofday({TimeRange:end}));\r\nlet DataSourceSubscriptionList = todynamic( replace(\"/subscriptions/\", \"\", @\"{DataSourceSubscription}\"));\r\nlet DataSourceLocationList = todynamic( @\"{DataSourceLocation}\"); \r\nlet VaultTypeList = \"*\";\r\nlet VaultList = \"*\";\r\nlet ExcludeLegacyEvent = true;\r\nlet BackupSolutionList = \"*\";\r\nlet ProtectionInfoList = \"*\";\r\nlet Item_search = \"*;*\";\r\nlet ItemArray = split(Item_search, \";\");\r\nlet ItemArray_length = array_length(ItemArray);\r\nlet BackupInstanceName = iff(ItemArray_length == 2, ItemArray[1], ItemArray[0] );\r\nlet DatasourceSetName = iff(ItemArray_length == 2, ItemArray[0], \"\");\r\nlet JobOperationList = todynamic( @\"{JobOperation}\"); \r\nlet JobStatusList = todynamic( @\"{JobStatus}\");\r\nlet JobFailureCodeList = \"*\";\r\nlet ExcludeLog = true; \r\nlet backupItem = '{SearchItem}';\r\n_AzureBackup_GetJobs(RangeStart, RangeEnd, DataSourceSubscriptionList, DataSourceLocationList, VaultList, VaultTypeList, ExcludeLegacyEvent, BackupSolutionList, JobOperationList, JobStatusList, JobFailureCodeList, DatasourceSetName, BackupInstanceName, ExcludeLog)\r\n| where BackupInstanceFriendlyName contains iff(isnotempty('{SearchItem}'),'{SearchItem}',BackupInstanceFriendlyName)\r\n| sort by BackupInstanceId\r\n| extend row_num = row_number()\r\n| extend page_num = tostring(((row_num-1)/toint('{RowsPerPage}') + 1))\r\n| where page_num has ('{Page}')\r\n| project BackupItem = BackupInstanceId,BackupItemFriendlyName = BackupInstanceFriendlyName ,Vault = VaultResourceId,Subscription = VaultSubscriptionId, VaultLocation = VaultLocation,JobOperation = OperationCategory,JobStartTime = StartTime,JobDuration = tostring(todouble(DurationInSecs)/60/60),JobStatus = Status,FailureCode = ErrorTitle\r\n", + "size": 3, + "title": "List of jobs in period", + "noDataMessage": "No record found for the selected time and scope.", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces", + "crossComponentResources": [ + "{Workspaces}" + ], + "visualization": "table", + "gridSettings": { + "formatters": [ + { + "columnMatch": "BackupItem", + "formatter": 5 + }, + { + "columnMatch": "BackupItemFriendlyName", + "formatter": 16, + "formatOptions": { + "linkColumn": "BackupItem", + "linkTarget": "Resource", + "showIcon": true, + "customColumnWidthSetting": "10%" + } + }, + { + "columnMatch": "Vault", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "10%" + } + }, + { + "columnMatch": "Subscription", + "formatter": 15, + "formatOptions": { + "linkTarget": "Resource", + "showIcon": true, + "customColumnWidthSetting": "10%" + } + }, + { + "columnMatch": "VaultLocation", + "formatter": 17, + "formatOptions": { + "customColumnWidthSetting": "10%" + } + }, + { + "columnMatch": "JobOperation", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "10%" + } + }, + { + "columnMatch": "JobStartTime", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "10%" + } + }, + { + "columnMatch": "JobDuration", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "10%" + }, + "numberFormat": { + "unit": 0, + "options": { + "style": "decimal", + "minimumFractionDigits": 2, + "maximumFractionDigits": 2 + } + } + }, + { + "columnMatch": "JobStatus", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "contains", + "thresholdValue": "Warning", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "contains", + "thresholdValue": "Failed", + "representation": "failed", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Blank", + "text": "{0}{1}" + } + ], + "customColumnWidthSetting": "10%" + } + }, + { + "columnMatch": "FailureCode", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "10%" + } + } + ], + "rowLimit": 1000, + "labelSettings": [ + { + "columnId": "BackupItemFriendlyName", + "label": "Backup instance" + }, + { + "columnId": "Vault", + "label": "Vault" + }, + { + "columnId": "Subscription", + "label": "Subscription" + }, + { + "columnId": "VaultLocation", + "label": "Location" + }, + { + "columnId": "JobOperation", + "label": "Job operation" + }, + { + "columnId": "JobStartTime", + "label": "Job start time (UTC)" + }, + { + "columnId": "JobDuration", + "label": "Job duration (hours)" + }, + { + "columnId": "JobStatus", + "label": "Job status" + }, + { + "columnId": "FailureCode", + "label": "Job failure code" + } + ] + }, + "sortBy": [] + }, + "customWidth": "100", + "conditionalVisibility": { + "parameterName": "Workspaces", + "comparison": "isNotEqualTo" + }, + "name": "Grid1", + "styleSettings": { + "margin": "5px", + "padding": "5px", + "showBorder": true + } + } + ] + }, + "conditionalVisibility": { + "parameterName": "Subscription", + "comparison": "isNotEqualTo" + }, + "name": "Backup" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RC_Storage" + }, + { + "parameterName": "Subscription", + "comparison": "isNotEqualTo" + } + ], + "name": "RC_Storage" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "where type has \"microsoft.network\"\r\n| extend type = case(\r\n\ttype == 'microsoft.network/networkinterfaces', \"NICs\",\r\n\ttype == 'microsoft.network/networksecuritygroups', \"NSGs\", \r\n\ttype == \"microsoft.network/publicipaddresses\", \"Public IPs\", \r\n\ttype == 'microsoft.network/virtualnetworks', \"vNets\",\r\n\ttype == 'microsoft.network/networkwatchers/connectionmonitors', \"Connection Monitors\",\r\n\ttype == 'microsoft.network/privatednszones', \"Private DNS\",\r\n\ttype == 'microsoft.network/virtualnetworkgateways', @\"vNet Gateways\",\r\n\ttype == 'microsoft.network/connections', \"Connections\",\r\n\ttype == 'microsoft.network/networkwatchers', \"Network Watchers\",\r\n\ttype == 'microsoft.network/privateendpoints', \"Private Endpoints\",\r\n\ttype == 'microsoft.network/localnetworkgateways', \"Local Network Gateways\",\r\n\ttype == 'microsoft.network/privatednszones/virtualnetworklinks', \"vNet Links\",\r\n\ttype == 'microsoft.network/dnszones', 'DNS Zones',\r\n\ttype == 'microsoft.network/networkwatchers/flowlogs', 'Flow Logs',\r\n\ttype == 'microsoft.network/routetables', 'Route Tables',\r\n\ttype == 'microsoft.network/loadbalancers', 'Load Balancers',\r\n type =~ 'Microsoft.Network/applicationGateways', 'Application Gateways',\r\n\tstrcat(\"Not Translated: \", type))\r\n| summarize count() by type\r\n| where type !has \"Not Translated\"", + "size": 3, + "title": "Count of all network resources by resource type", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "tiles", + "tileSettings": { + "showBorder": false, + "titleContent": { + "columnMatch": "type", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + } + }, + "name": "query - Network resource" + }, + { + "type": 1, + "content": { + "json": "# Network security group" + }, + "name": "Network security group title" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "7763ba7f-6187-4448-a94c-890392ed31d0", + "version": "KqlParameterItem/1.0", + "name": "OrphanNSG", + "label": "Orphaned", + "type": 10, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[[\r\n { \"value\":\"and isnotnull(properties.networkInterfaces) or type =~ 'microsoft.network/networksecuritygroups' and isnotnull(properties.subnets)\", \"label\":\"No\" },\r\n { \"value\":\"and isnull(properties.networkInterfaces) and isnull(properties.subnets)\", \"label\":\"Yes\" }\r\n]", + "timeContext": { + "durationMs": 86400000 + }, + "value": "and isnotnull(properties.networkInterfaces) or type =~ 'microsoft.network/networksecuritygroups' and isnotnull(properties.subnets)" + } + ], + "style": "above", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "NSG" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources\r\n| where type =~ 'microsoft.network/networksecuritygroups' {OrphanNSG}\r\n| project Resource=id, resourceGroup, subscriptionId, location", + "size": 0, + "title": "NSGs", + "noDataMessage": "No NSGs Found", + "noDataMessageStyle": 3, + "exportedParameters": [ + { + "fieldName": "Resource", + "parameterName": "SelectedResourceId", + "parameterType": 5 + } + ], + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "filter": true + }, + "sortBy": [] + }, + "name": "NSGs" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources\r\n | where type =~ 'microsoft.network/networksecuritygroups'\r\n | where id == \"{SelectedResourceId}\"\r\n | project id, nsgRules = parse_json(parse_json(properties).securityRules), networksecurityGroupName = name, subscriptionId, resourceGroup , location\r\n | mvexpand nsgRule = nsgRules\r\n | project id, location, access=nsgRule.properties.access,protocol=nsgRule.properties.protocol ,direction=nsgRule.properties.direction,provisioningState= nsgRule.properties.provisioningState ,priority=nsgRule.properties.priority, \r\n sourceAddressPrefix = nsgRule.properties.sourceAddressPrefix, \r\n sourceAddressPrefixes = nsgRule.properties.sourceAddressPrefixes,\r\n destinationAddressPrefix = nsgRule.properties.destinationAddressPrefix, \r\n destinationAddressPrefixes = nsgRule.properties.destinationAddressPrefixes, \r\n networksecurityGroupName, networksecurityRuleName = tostring(nsgRule.name), \r\n subscriptionId, resourceGroup,\r\n destinationPortRanges = nsgRule.properties.destinationPortRanges,\r\n destinationPortRange = nsgRule.properties.destinationPortRange,\r\n sourcePortRanges = nsgRule.properties.sourcePortRanges,\r\n sourcePortRange = nsgRule.properties.sourcePortRange\r\n| extend Details = pack_all()\r\n| project id, location, access, direction, priority, sourceAddressPrefix, sourcePortRange, destinationPortRange, subscriptionId, resourceGroup, Details", + "size": 1, + "title": "NSG rules", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 5 + }, + { + "columnMatch": "resourceGroup", + "formatter": 5 + }, + { + "columnMatch": "Details", + "formatter": 7, + "formatOptions": { + "linkTarget": "CellDetails", + "linkLabel": "🔍 View details", + "linkIsContextBlade": true + } + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "SelectedResourceId", + "comparison": "isNotEqualTo" + }, + "name": "NSG rules" + }, + { + "type": 1, + "content": { + "json": "# Public IPs" + }, + "name": "Public IP title" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "37cdc20d-07c3-466c-84bb-4d8050932641", + "version": "KqlParameterItem/1.0", + "name": "OrphanIPs", + "label": "Orphaned", + "type": 10, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[[\r\n { \"value\":\"isnotnull\", \"label\":\"No\" },\r\n { \"value\":\"isnull\", \"label\":\"Yes\" }\r\n]", + "timeContext": { + "durationMs": 86400000 + }, + "value": "isnotnull" + } + ], + "style": "above", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "Public IPs" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources\r\n| where type =~ 'Microsoft.Network/publicIPAddresses' and {OrphanIPs}(properties.ipAddress)\r\n| extend ipAddress = properties.ipAddress\r\n| extend sku = sku.name\r\n| extend Details = pack_all()\r\n| project Resource=id, subscriptionId, resourceGroup, name, location,sku,Details", + "size": 0, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "table", + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 13, + "formatOptions": { + "linkTarget": "Resource", + "linkIsContextBlade": true, + "showIcon": true + } + }, + { + "columnMatch": "Resource", + "formatter": 5 + }, + { + "columnMatch": "subscriptionId", + "formatter": 5 + }, + { + "columnMatch": "Details", + "formatter": 7, + "formatOptions": { + "linkTarget": "CellDetails", + "linkLabel": "🔍 View details", + "linkIsContextBlade": true + } + } + ], + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "subscriptionId" + ], + "expandTopLevel": true, + "finalBy": "Resource" + } + } + }, + "name": "query - Networking Details - PiPs" + }, + { + "type": 1, + "content": { + "json": "# Application gateway" + }, + "name": "Application gateway title" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "007b8dbe-6bc6-40f9-b4bc-55f2ec14916c", + "version": "KqlParameterItem/1.0", + "name": "OrphanAppGW", + "label": "Orphaned", + "type": 10, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[[\r\n { \"value\":\"//\", \"label\":\"No\" },\r\n { \"value\":\"|\", \"label\":\"Yes\" }\r\n]", + "timeContext": { + "durationMs": 86400000 + }, + "value": "//" + } + ], + "style": "above", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "ApplicationGateway" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type =~ 'Microsoft.Network/applicationGateways'\r\n| extend backendPoolsCount = array_length(properties.backendAddressPools),SKUName= tostring(properties.sku.name), SKUTier= tostring(properties.sku.tier),SKUCapacity=properties.sku.capacity,backendPools=properties.backendAddressPools\r\n| project id, name, SKUName, SKUTier, SKUCapacity\r\n| join (\r\n resources\r\n | where type =~ 'Microsoft.Network/applicationGateways'\r\n | mvexpand backendPools = properties.backendAddressPools\r\n | extend backendIPCount = array_length(backendPools.properties.backendIPConfigurations)\r\n | extend backendAddressesCount = array_length(backendPools.properties.backendAddresses)\r\n | extend backendPoolName = backendPools.properties.backendAddressPools.name\r\n | summarize backendIPCount = sum(backendIPCount) ,backendAddressesCount=sum(backendAddressesCount) by id\r\n) on id\r\n| project-away id1\r\n{OrphanAppGW} where (backendIPCount == 0 or isempty(backendIPCount)) and (backendAddressesCount==0 or isempty(backendAddressesCount))\r\n| order by id asc", + "size": 0, + "noDataMessage": "No app gateways", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ] + }, + "name": "query - Application Gateways" + }, + { + "type": 1, + "content": { + "json": "# Load balancer" + }, + "name": "Load balancer title" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "8cffc283-1878-4035-a669-5d9697e9edc1", + "version": "KqlParameterItem/1.0", + "name": "OrphanLB", + "label": "Orphaned", + "type": 10, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[[\r\n { \"value\":\"!=\", \"label\":\"No\" },\r\n { \"value\":\"==\", \"label\":\"Yes\" }\r\n]", + "timeContext": { + "durationMs": 86400000 + }, + "value": "!=" + } + ], + "style": "above", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "LoadBalancers" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type == \"microsoft.network/loadbalancers\"\r\n| where properties.backendAddressPools {OrphanLB} \"[]\"\r\n| extend Details = pack_all()\r\n| project Resource=id, subscriptionId, resourceGroup, location, tostring(sku.name), Details", + "size": 0, + "noDataMessage": "No load balancers", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "location", + "formatter": 17 + }, + { + "columnMatch": "Details", + "formatter": 7, + "formatOptions": { + "linkTarget": "CellDetails", + "linkLabel": "🔍 View details", + "linkIsContextBlade": true + } + } + ], + "labelSettings": [ + { + "columnId": "Resource", + "label": "Resource Name" + }, + { + "columnId": "subscriptionId", + "label": "Subscription" + }, + { + "columnId": "resourceGroup", + "label": "Resource group" + }, + { + "columnId": "location", + "label": "Region" + }, + { + "columnId": "sku_name", + "label": "SKU" + } + ] + } + }, + "name": "query - Load Balancers" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RC_Network" + }, + { + "parameterName": "Subscription", + "comparison": "isNotEqualTo" + } + ], + "name": "RC_Network" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "## Stay informed and act quickly on service issues\r\nAzure Service Health notifies you about Azure service incidents and planned maintenance so you can take action to mitigate downtime. Configure customisable cloud alerts and use your personalised dashboard to analyse health issues, monitor the impact to your cloud resources, get guidance and support, and share details and updates." + }, + "name": "text - 4" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "servicehealthresources\r\n| where type =~ 'Microsoft.ResourceHealth/events'\r\n| extend eventType = properties.EventType, status = properties.Status, description = properties.Title, trackingId = properties.TrackingId, summary = properties.Summary, priority = properties.Priority, impactStartTime = properties.ImpactStartTime, impactMitigationTime = properties.ImpactMitigationTime\r\n| where properties.Status == 'Active' and tolong(impactStartTime) > 1\r\n\r\n| extend Details = pack_all()\r\n| project ServiceHealthID=id, Description=description, Region=location, eventType, Status=status, Details", + "size": 1, + "title": "All active Service Health events", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "ServiceHealthID", + "formatter": 13, + "formatOptions": { + "linkTarget": null, + "showIcon": false, + "customColumnWidthSetting": "20ch" + } + }, + { + "columnMatch": "Description", + "formatter": 1, + "formatOptions": { + "customColumnWidthSetting": "60ch" + } + }, + { + "columnMatch": "eventType", + "formatter": 1 + }, + { + "columnMatch": "Status", + "formatter": 1 + }, + { + "columnMatch": "Details", + "formatter": 7, + "formatOptions": { + "linkTarget": "CellDetails", + "linkLabel": "🔍 View details", + "linkIsContextBlade": true + } + } + ] + }, + "tileSettings": { + "showBorder": false + } + }, + "name": "query - 15" + }, + { + "type": 1, + "content": { + "json": "## Activity log monitoring" + }, + "name": "text - 15" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resourcechanges\r\n| extend changeTime = todatetime(properties.changeAttributes.timestamp), targetResourceId = tostring(properties.targetResourceId),\r\nchangeType = tostring(properties.changeType), correlationId = properties.changeAttributes.correlationId, \r\nchangedProperties = properties.changes, changeCount = properties.changeAttributes.changesCount\r\n| where changeTime > ago(1d)\r\n| order by changeTime desc\r\n| project changeTime, targetResourceId, changeType, correlationId, changeCount, changedProperties", + "size": 0, + "title": "All changes in the past one day", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "changeTime", + "formatter": 6, + "formatOptions": { + "customColumnWidthSetting": "24ch" + } + }, + { + "columnMatch": "targetResourceId", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "42.7143ch" + } + }, + { + "columnMatch": "changedProperties", + "formatter": 7, + "formatOptions": { + "linkTarget": "CellDetails", + "linkLabel": "🔍 View details", + "linkIsContextBlade": true + } + } + ] + } + }, + "name": "query - 12" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resourcechanges\r\n| extend changeTime = todatetime(properties.changeAttributes.timestamp), targetResourceId = tostring(properties.targetResourceId),\r\nchangeType = tostring(properties.changeType), correlationId = properties.changeAttributes.correlationId\r\n| where changeType == \"Delete\"\r\n| order by changeTime desc\r\n| project changeTime, resourceGroup, targetResourceId, changeType, correlationId", + "size": 0, + "title": "Resources deleted", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "changeTime", + "formatter": 6 + } + ] + } + }, + "name": "query - 13" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RC_Monitoring" + }, + { + "parameterName": "Subscription", + "comparison": "isNotEqualTo" + } + ], + "name": "RC_Monitoring" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "## Use tags to organize your Azure resources and management hierarchy\r\nTags are metadata elements that you apply to your Azure resources. They're key-value pairs that help you identify resources based on settings that are relevant to your organization. If you want to track the deployment environment for your resources, add a key named Environment. To identify the resources deployed to production, give them a value of Production. The fully formed key-value pair is Environment = Production.\r\n\r\nTo get more information about tags, see [Resource naming and tagging decision guide](https://learn.microsoft.com/azure/cloud-adoption-framework/ready/azure-best-practices/resource-naming-and-tagging-decision-guide?toc=%2Fazure%2Fazure-resource-manager%2Fmanagement%2Ftoc.json)" + }, + "name": "text - 9" + }, + { + "type": 1, + "content": { + "json": "Tag names with spaces, hyphens, and underscores are not supported.", + "style": "info" + }, + "name": "warning tag explorer" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "bae67738-90ef-4698-9020-5e1f91d67f82", + "version": "KqlParameterItem/1.0", + "name": "TagName", + "label": "Tag name", + "type": 2, + "isRequired": true, + "query": "Resources\r\n| where tags != '' and tags != '[]'\r\n| mvexpand tags\r\n| extend tagName = tostring(bag_keys(tags)[0])\r\n| distinct tagName\r\n| sort by tagName asc", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null + } + ], + "style": "formVertical", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "33", + "name": "parameters - 0" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "cb0ae78d-a49b-457b-baed-d83c97a2c934", + "version": "KqlParameterItem/1.0", + "name": "TagValue", + "label": "Tag value", + "type": 2, + "query": "Resources\r\n| extend TagValue = tostring(tags.{TagName})\r\n| project TagValue\r\n| distinct TagValue", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + } + ], + "style": "formVertical", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "customWidth": "33", + "conditionalVisibility": { + "parameterName": "TagName", + "comparison": "isNotEqualTo" + }, + "name": "parameters - 2" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "81756016-e942-4fa0-976e-06d8ce919f83", + "version": "KqlParameterItem/1.0", + "name": "ResourceType", + "label": "Resource type", + "type": 7, + "typeSettings": { + "additionalResourceOptions": [], + "includeAll": true, + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "value": null + } + ], + "style": "formVertical", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "customWidth": "33", + "conditionalVisibility": { + "parameterName": "TagName", + "comparison": "isNotEqualTo" + }, + "name": "ResourceType" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources\r\n| extend tag = tags.{TagName}\r\n| mv-expand bagexpansion=array tags\r\n| where isnotempty(tags)\r\n//| where tags[0] =~ '{TagName}' and tags[1] =~ '{TagValue}'\r\n| where tags[0] == '{TagName}' and tags[1] == '{TagValue}'\r\n| where type contains '{ResourceType}'\r\n| project id, tag", + "size": 0, + "title": "Resource with tag", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "filter": true, + "labelSettings": [ + { + "columnId": "id", + "label": "Resource name" + }, + { + "columnId": "tag", + "label": "Tag value" + } + ] + } + }, + "customWidth": "50", + "conditionalVisibility": { + "parameterName": "TagName", + "comparison": "isNotEqualTo" + }, + "name": "query - Resource with tag" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "Resources\r\n| extend tag = tags.{TagName}\r\n| mv-expand bagexpansion=array tags\r\n| where isnotempty(tags)\r\n| where tags[0] == '{TagName}' and tags[1] == ''\r\n| where type contains '{ResourceType}'\r\n| project id, tag", + "size": 0, + "title": "Tag with empty value", + "noDataMessage": "No tagged resources with empty value found", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "table", + "gridSettings": { + "filter": true, + "labelSettings": [ + { + "columnId": "id", + "label": "Resource name" + }, + { + "columnId": "tag", + "label": "Tag value" + } + ] + } + }, + "customWidth": "50", + "conditionalVisibility": { + "parameterName": "TagName", + "comparison": "isNotEqualTo" + }, + "name": "query - Empty value" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where tags =~ '' or tags =~ '{}'\r\n| where type contains '{ResourceType}'\r\n| project Name=id", + "size": 0, + "title": "Untagged resources", + "noDataMessage": "No untagged resources found", + "noDataMessageStyle": 3, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "rowLimit": 100, + "filter": true, + "labelSettings": [ + { + "columnId": "Name", + "label": "Resource name" + } + ] + } + }, + "name": "query - Untagged resources" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resourcecontainers\r\n| where type == \"microsoft.resources/subscriptions\"\r\n| extend tag = tags.{TagName}\r\n| mv-expand bagexpansion=array tags\r\n| where isnotempty(tags)\r\n| where tags[0] == '{TagName}' and tags[1] == '{TagValue}'\r\n| project id, tag", + "size": 0, + "title": "Subscription list", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "tag", + "formatter": 1 + } + ], + "labelSettings": [ + { + "columnId": "id", + "label": "Subscription" + }, + { + "columnId": "tag", + "label": "Tag value" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "TagName", + "comparison": "isNotEqualTo" + }, + "name": "query - Subscription list" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resourcecontainers\r\n| where type == \"microsoft.resources/subscriptions/resourcegroups\"\r\n| extend tag = tags.{TagName}\r\n| mv-expand bagexpansion=array tags\r\n| where isnotempty(tags)\r\n| where tags[0] == '{TagName}' and tags[1] == '{TagValue}'\r\n| project id, tag", + "size": 0, + "title": "Resource groups list", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "tag", + "formatter": 1 + } + ], + "labelSettings": [ + { + "columnId": "id", + "label": "Subscription" + }, + { + "columnId": "tag", + "label": "Tag value" + } + ] + } + }, + "conditionalVisibility": { + "parameterName": "TagName", + "comparison": "isNotEqualTo" + }, + "name": "query - ResourceGroup list" + } + ] + }, + "conditionalVisibility": { + "parameterName": "Subscription", + "comparison": "isNotEqualTo" + }, + "name": "group - TagQueries" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RC_Tag" + }, + "name": "RC_Tags" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "securityresources\r\n| where type == \"microsoft.security/securescores\"\r\n| extend subscriptionSecureScore = round(100 * bin((todouble(properties.score.current))/ todouble(properties.score.max), 0.001))\r\n| where subscriptionSecureScore > 0\r\n| project subscriptionId, subscriptionSecureScore\r\n| order by subscriptionSecureScore asc", + "size": 0, + "title": "Security Scores by Subscription", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "subscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": "Resource", + "showIcon": true, + "customColumnWidthSetting": "65ch" + } + }, + { + "columnMatch": "subscriptionSecureScore", + "formatter": 8, + "formatOptions": { + "min": 0, + "max": 100, + "palette": "redGreen", + "customColumnWidthSetting": "55ch" + }, + "numberFormat": { + "unit": 1, + "options": { + "style": "decimal", + "useGrouping": false + } + } + } + ], + "labelSettings": [ + { + "columnId": "subscriptionId", + "label": "Subscription Name" + }, + { + "columnId": "subscriptionSecureScore", + "label": "Subscription Secure Score" + } + ] + } + }, + "name": "query - Monitor & Security - Security Scores" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "SecurityResources \r\n| where type == 'microsoft.security/securescores/securescorecontrols' \r\n| extend SecureControl = properties.displayName, unhealthy = properties.unhealthyResourceCount, currentscore = properties.score.current, maxscore = properties.score.max, subscriptionId, details = properties\r\n| project SecureControl , unhealthy, currentscore, maxscore, subscriptionId, details", + "size": 0, + "title": "Security Scores by Control", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 13, + "formatOptions": { + "linkTarget": null, + "showIcon": true, + "customColumnWidthSetting": "65ch" + } + }, + { + "columnMatch": "Subscription", + "formatter": 13, + "formatOptions": { + "linkTarget": null, + "showIcon": true, + "customColumnWidthSetting": "65ch" + } + }, + { + "columnMatch": "SecureControl", + "formatter": 5, + "tooltipFormat": {} + }, + { + "columnMatch": "unhealthy", + "formatter": 8, + "formatOptions": { + "min": 0, + "palette": "greenRed", + "customColumnWidthSetting": "20ch" + } + }, + { + "columnMatch": "currentscore", + "formatter": 8, + "formatOptions": { + "palette": "redGreen", + "customColumnWidthSetting": "20ch" + }, + "numberFormat": { + "unit": 1, + "options": { + "style": "decimal" + } + } + }, + { + "columnMatch": "maxscore", + "formatter": 8, + "formatOptions": { + "palette": "blue", + "customColumnWidthSetting": "20ch" + }, + "numberFormat": { + "unit": 1, + "options": { + "style": "decimal" + } + } + }, + { + "columnMatch": "subscriptionId", + "formatter": 5 + }, + { + "columnMatch": "details", + "formatter": 7, + "formatOptions": { + "linkTarget": "CellDetails", + "linkLabel": "Details", + "linkIsContextBlade": true + } + }, + { + "columnMatch": "subscriptionSecureScore", + "formatter": 8, + "formatOptions": { + "min": 0, + "max": 100, + "palette": "redGreen", + "customColumnWidthSetting": "20" + }, + "numberFormat": { + "unit": 1, + "options": { + "style": "decimal", + "useGrouping": false + } + } + } + ], + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "subscriptionId" + ], + "expandTopLevel": true, + "finalBy": "SecureControl" + }, + "labelSettings": [ + { + "columnId": "unhealthy", + "label": "Unhealthy" + }, + { + "columnId": "currentscore", + "label": "Current Score" + }, + { + "columnId": "maxscore", + "label": "Max Score" + }, + { + "columnId": "subscriptionId", + "label": "Subscription" + } + ] + } + }, + "name": "query - Monitor & Security - Security Scores by Control" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "4f93ebba-a9d5-4e11-8de4-b605c2b4368f", + "version": "KqlParameterItem/1.0", + "name": "ResourceIdFilter", + "type": 1, + "isGlobal": true, + "isHiddenWhenLocked": true, + "label": "Resource ID" + }, + { + "id": "e505498f-d2eb-4dd6-928f-0f0f0e9cc371", + "version": "KqlParameterItem/1.0", + "name": "AlertDisplayNameFilter", + "type": 1, + "isGlobal": true, + "isHiddenWhenLocked": true, + "label": "Alert display name" + }, + { + "id": "39e382f9-4780-40fa-8595-15eda0f08ad4", + "version": "KqlParameterItem/1.0", + "name": "NewAlertFilter", + "type": 1, + "isGlobal": true, + "isHiddenWhenLocked": true, + "label": "New alert" + } + ], + "style": "above", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "parameters - 15" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": " securityresources\r\n | where type =~ 'microsoft.security/locations/alerts'\r\n | where properties.Status in ('Active')\r\n | where properties.Severity in ('Low', 'Medium', 'High')\r\n | extend SeverityRank = case(\r\n properties.Severity == 'High', 3,\r\n properties.Severity == 'Medium', 2,\r\n properties.Severity == 'Low', 1,\r\n 0\r\n )\r\n | project-away SeverityRank\r\n | extend Severity = properties.Severity\r\n | project Severity = tostring(Severity)\r\n | summarize Count = count() by Severity", + "size": 0, + "title": "Severity ", + "exportedParameters": [ + { + "fieldName": "Subscription", + "parameterName": "Subscription", + "parameterType": 1 + }, + { + "fieldName": "Severity", + "parameterName": "SeverityFilter", + "parameterType": 1 + } + ], + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Severity", + "formatter": 1 + } + ] + }, + "chartSettings": { + "yAxis": [ + "Count" + ], + "seriesLabelSettings": [ + { + "seriesName": "Medium", + "color": "orange" + }, + { + "seriesName": "High", + "color": "redDark" + }, + { + "seriesName": "Low", + "color": "yellow" + } + ] + } + }, + "customWidth": "33", + "name": "Severity" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": " securityresources\r\n | where type =~ 'microsoft.security/locations/alerts'\r\n | summarize Count =count() by resourceGroup", + "size": 0, + "title": "Resource Group", + "exportFieldName": "resourceGroup", + "exportParameterName": "resourceGroupFilter", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart" + }, + "customWidth": "33", + "name": "query - 9" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n | project id = tolower(id), tags\r\n | join kind=inner (securityresources\r\n | where type =~ \"microsoft.security/locations/alerts\"\r\n | extend isAzure = tostring(properties.ResourceIdentifiers) matches regex '\"Type\"\\\\s*:\\\\s*\"AzureResource\"'\r\n | extend affectedResourceId = extract('\"AzureResourceId\"\\\\s*:\\\\s*\"([^\"]*)\"', 1, tostring(properties.ResourceIdentifiers))\r\n | extend hostName = iff(isAzure, \"\", extract('\"HostName\"\\\\s*:\\\\s*\"([^\"]*)\"', 1, tostring(properties.Entities)))\r\n | extend splitAffectedResourceId = split(affectedResourceId, \"/\")\r\n | extend resourceNameIndex = iff(array_length(splitAffectedResourceId) > 1, array_length(splitAffectedResourceId) - 1, 0)\r\n | extend affectedResourceName = iff(isAzure, splitAffectedResourceId[resourceNameIndex], iff(isempty(hostName), \"Non-Azure\", hostName))| project-away resourceNameIndex, splitAffectedResourceId, hostName, isAzure\r\n | project alertId = id, subscriptionId, alertProperties = properties, affectedResourceId = tolower(affectedResourceId)\r\n ) on $left.id == $right.affectedResourceId\r\n | extend id = alertId, subscriptionId, properties = alertProperties\r\n | where properties.Status in ('Active')\r\n | where properties.Severity in ('Low', 'Medium', 'High')\r\n | extend SeverityRank = case(\r\n properties.Severity == 'High', 3,\r\n properties.Severity == 'Medium', 2,\r\n properties.Severity == 'Low', 1,\r\n 0\r\n )\r\n | sort by SeverityRank desc, tostring(properties.SystemAlertId) asc\r\n | extend Tag = parse_json(tags)\r\n | mv-expand Tag\r\n | parse Tag with * ':\"' TagValue '\"}'\r\n | project TagValue, alertId\r\n | summarize Count = count() by TagValue", + "size": 0, + "title": "Tag", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart" + }, + "customWidth": "30", + "name": "query - 7", + "styleSettings": { + "maxWidth": "100%" + } + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "datatable(ResourceId: string) [ \"All\"] | union (securityresources\r\n | where type =~ 'microsoft.security/locations/alerts'\r\n | extend Prop = parse_json(properties)\r\n | where Prop.Severity == \"High\"\r\n | extend ResourceIdentifiers = Prop.[\"ResourceIdentifiers\"]\r\n | project ResourceIdentifiers\r\n | mv-expand ResourceIdentifiers\r\n | extend ResourceId = parse_json(ResourceIdentifiers).[\"AzureResourceId\"]\r\n | where isnotempty(ResourceId )\r\n| summarize Count=count() by tostring(ResourceId)\r\n | top 5 by Count)", + "size": 1, + "title": "Top 5 attacked resources (with High Severity)", + "noDataMessage": "There are no Top 5 attacked resources found", + "exportedParameters": [ + { + "fieldName": "ResourceId", + "parameterName": "ResourceIdFilter", + "defaultValue": "All" + }, + { + "fieldName": "ResourceId", + "parameterName": "ShowTable", + "parameterType": 1 + } + ], + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "table", + "gridSettings": { + "formatters": [ + { + "columnMatch": "Resource ID", + "formatter": 13, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "sortBy": [ + { + "itemKey": "Count", + "sortOrder": 2 + } + ], + "labelSettings": [ + { + "columnId": "ResourceId", + "label": "Resource ID" + } + ] + }, + "sortBy": [ + { + "itemKey": "Count", + "sortOrder": 2 + } + ] + }, + "customWidth": "33", + "name": "Top 5 attacked resources (with High Severity)" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": " datatable(AlertDisplayName: string) [ \"All\"] | union(securityresources\r\n | where type =~ 'microsoft.security/locations/alerts'\r\n | extend Prop = parse_json(properties)\r\n | extend AlertDisplayName = Prop.[\"AlertDisplayName\"]\r\n | project tostring(AlertDisplayName)\r\n | summarize Count = count() by AlertDisplayName\r\n | top 5 by Count)", + "size": 1, + "title": "Top alert types ", + "exportedParameters": [ + { + "fieldName": "AlertDisplayName", + "parameterName": "AlertDisplayNameFilter", + "defaultValue": "All" + }, + { + "fieldName": "AlertDisplayName", + "parameterName": "ShowTable", + "parameterType": 1 + } + ], + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "table", + "gridSettings": { + "sortBy": [ + { + "itemKey": "Count", + "sortOrder": 2 + } + ], + "labelSettings": [ + { + "columnId": "AlertDisplayName", + "label": "Alert display name" + } + ] + }, + "sortBy": [ + { + "itemKey": "Count", + "sortOrder": 2 + } + ] + }, + "customWidth": "33", + "name": "Top alert types" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": " datatable(AlertDisplayName: string) [ \"All\"] | union(securityresources\r\n| where type =~ 'microsoft.security/locations/alerts'\r\n| extend Prop = parse_json(properties)\r\n| extend TimeGeneratedUtc = Prop.[\"TimeGeneratedUtc\"]\r\n| extend AlertDisplayName = Prop.[\"AlertDisplayName\"]\r\n| where TimeGeneratedUtc > ago(24h)\r\n| summarize Count=count() by tostring(AlertDisplayName))", + "size": 1, + "title": "New Alerts (Since last 24hrs)", + "noDataMessage": "No new alerts in Last 24 hours", + "noDataMessageStyle": 3, + "exportedParameters": [ + { + "fieldName": "AlertDisplayName", + "parameterName": "NewAlertFilter", + "defaultValue": "All" + }, + { + "fieldName": "AlertDisplayName", + "parameterName": "ShowTable", + "parameterType": 1 + } + ], + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "ClearOther", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "AlertDisplayName", + "label": "Alert display name" + } + ] + }, + "sortBy": [] + }, + "customWidth": "33", + "name": "New Alerts (Since last 24hrs)" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "", + "size": 0, + "title": "Parameters at this point", + "queryType": 2 + }, + "conditionalVisibility": { + "parameterName": "parameter1", + "comparison": "isEqualTo", + "value": "1" + }, + "name": "query - 23" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "securityresources\r\n| where type == \"microsoft.security/locations/alerts\"\r\n| project-rename P= properties\r\n| extend Details = parse_json(P)\r\n| extend IsIncident = Details.[\"IsIncident\"]\r\n| extend AlertDisplayName = Details.[\"AlertDisplayName\"]\r\n| extend SystemAlertId = Details.[\"SystemAlertId\"]\r\n| extend Severity = tostring(Details.[\"Severity\"])\r\n| where Severity == \"High\"\r\n| extend AlertUri = Details.[\"AlertUri\"]\r\n| extend Status = tostring(Details.[\"Status\"])\r\n| extend Tactics = tostring(Details.[\"Intent\"])\r\n| extend ResourceIdentifiers = Details.[\"ResourceIdentifiers\"]\r\n| mv-expand ResourceIdentifiers\r\n| extend ResourceId = parse_json(ResourceIdentifiers).[\"AzureResourceId\"]\r\n| where Status == \"Active\"\r\n| where (\"{ResourceIdFilter}\" == \"All\" or ResourceId == \"{ResourceIdFilter}\") \r\n // if either alert name or new alert are set, union those 2 together, if neither are set treat as all\r\n and ((\"{AlertDisplayNameFilter}\" == \"All\" and \"{NewAlertFilter}\" == \"All\") or AlertDisplayName == \"{AlertDisplayNameFilter}\" or AlertDisplayName == \"{NewAlertFilter}\")\r\n| extend SeverityRank = case(\r\n Severity == 'High', 3,\r\n Severity == 'Medium', 2,\r\n Severity == 'Low', 1,\r\n 0\r\n )\r\n| parse AlertUri with * '/subscriptionId/' SubscriptionId '/' *\r\n| parse AlertUri with * '/resourceGroup/' ResourceGroup '/' *\r\n| parse AlertUri with * '/location/' Location \r\n| project\r\n Severity,\r\n SystemAlertId,\r\n AlertDisplayName,\r\n IsIncident = iif(IsIncident == \"true\", \"Incident\", \"Alert\"),\r\n AlertUri,\r\n Tactics,\r\n SeverityRank,\r\n SubscriptionId,\r\n ResourceGroup,\r\n Location,\r\n ResourceId\r\n| sort by SeverityRank", + "size": 0, + "title": "{$rowCount} Active Alerts ", + "exportedParameters": [ + { + "fieldName": "ResourceId", + "parameterName": "Resource", + "parameterType": 1 + }, + { + "fieldName": "AlertUri", + "parameterName": "AlertUri", + "parameterType": 1 + }, + { + "fieldName": "SystemAlertId", + "parameterName": "SystemAlertId", + "parameterType": 1 + }, + { + "fieldName": "SubscriptionId", + "parameterName": "SubscriptionId", + "parameterType": 1 + }, + { + "fieldName": "ResourceGroup", + "parameterName": "ResourceGroup", + "parameterType": 1 + }, + { + "fieldName": "Location", + "parameterName": "Location", + "parameterType": 1 + } + ], + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "Severity", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "colors", + "thresholdsGrid": [ + { + "operator": "contains", + "thresholdValue": "High", + "representation": "redBright", + "text": "{0}{1}" + }, + { + "operator": "contains", + "thresholdValue": "Medium", + "representation": "orange", + "text": "{0}{1}" + }, + { + "operator": "contains", + "thresholdValue": "Low", + "representation": "yellow", + "text": "{0}{1}" + }, + { + "operator": "contains", + "thresholdValue": "Informational ", + "representation": "gray", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": null, + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "SystemAlertId", + "formatter": 5 + }, + { + "columnMatch": "AlertDisplayName", + "formatter": 1, + "formatOptions": { + "linkTarget": "OpenBlade", + "bladeOpenContext": { + "bladeName": "AlertBlade", + "extensionName": "Microsoft_Azure_Security", + "bladeParameters": [ + { + "name": "alertId", + "source": "column", + "value": "SystemAlertId" + }, + { + "name": "subscriptionId", + "source": "column", + "value": "SubscriptionId" + }, + { + "name": "resourceGroup", + "source": "column", + "value": "ResourceGroup" + }, + { + "name": "referencedFrom", + "source": "static", + "value": "activeAlertsWorkbook" + }, + { + "name": "location", + "source": "column", + "value": "Location" + } + ] + } + } + }, + { + "columnMatch": "IsIncident", + "formatter": 1 + }, + { + "columnMatch": "AlertUri", + "formatter": 5 + }, + { + "columnMatch": "Tactics", + "formatter": 1 + }, + { + "columnMatch": "SubscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": "Resource", + "showIcon": true + } + }, + { + "columnMatch": "Location", + "formatter": 17 + }, + { + "columnMatch": "ResourceId", + "formatter": 13, + "formatOptions": { + "linkTarget": "Resource", + "showIcon": true + } + }, + { + "columnMatch": "TenantId", + "formatter": 5 + }, + { + "columnMatch": "AlertName", + "formatter": 5 + }, + { + "columnMatch": "Description", + "formatter": 5 + }, + { + "columnMatch": "ProviderName", + "formatter": 5 + }, + { + "columnMatch": "VendorName", + "formatter": 5 + }, + { + "columnMatch": "VendorOriginalId", + "formatter": 5 + }, + { + "columnMatch": "SourceComputerId", + "formatter": 5 + }, + { + "columnMatch": "AlertType", + "formatter": 5 + }, + { + "columnMatch": "ConfidenceLevel", + "formatter": 5 + }, + { + "columnMatch": "ConfidenceScore", + "formatter": 5 + }, + { + "columnMatch": "StartTime", + "formatter": 5 + }, + { + "columnMatch": "EndTime", + "formatter": 5 + }, + { + "columnMatch": "ProcessingEndTime", + "formatter": 5 + }, + { + "columnMatch": "RemediationSteps", + "formatter": 5 + }, + { + "columnMatch": "ExtendedProperties", + "formatter": 5 + }, + { + "columnMatch": "Entities", + "formatter": 5 + }, + { + "columnMatch": "SourceSystem", + "formatter": 5 + }, + { + "columnMatch": "WorkspaceSubscriptionId", + "formatter": 5 + }, + { + "columnMatch": "WorkspaceResourceGroup", + "formatter": 5 + }, + { + "columnMatch": "ExtendedLinks", + "formatter": 5 + }, + { + "columnMatch": "ProductName", + "formatter": 5 + }, + { + "columnMatch": "ProductComponentName", + "formatter": 5 + }, + { + "columnMatch": "AlertLink", + "formatter": 7, + "formatOptions": { + "linkTarget": "Url" + } + }, + { + "columnMatch": "SystemIncidentId", + "formatter": 5 + }, + { + "columnMatch": "SystemAlertId1", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "SystemAlertId", + "label": "Alert ID" + }, + { + "columnId": "AlertDisplayName", + "label": "Alert name" + }, + { + "columnId": "IsIncident", + "label": "Incident/alert" + }, + { + "columnId": "SeverityRank", + "label": "Severity" + }, + { + "columnId": "SubscriptionId", + "label": "Subscription" + }, + { + "columnId": "ResourceGroup", + "label": "Resource group" + }, + { + "columnId": "ResourceId", + "label": "Resource" + } + ] + }, + "sortBy": [] + }, + "conditionalVisibility": { + "parameterName": "ShowTable", + "comparison": "isNotEqualTo" + }, + "showPin": true, + "name": "SecurityIncidents - FilterbyResourceId", + "styleSettings": { + "showBorder": true + } + }, + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "list", + "links": [ + { + "id": "2f6ff56b-9afb-46f6-968d-a59cb744ea14", + "linkTarget": "OpenBlade", + "linkLabel": "Open Alert View", + "style": "primary", + "bladeOpenContext": { + "bladeName": "AlertBlade", + "extensionName": "Microsoft_Azure_Security", + "bladeParameters": [ + { + "name": "alertId", + "source": "static", + "value": "{SystemAlertId}" + }, + { + "name": "subscriptionId", + "source": "static", + "value": "{SubscriptionId}" + }, + { + "name": "resourceGroup", + "source": "static", + "value": "{ResourceGroup}" + }, + { + "name": "referencedFrom", + "source": "static", + "value": "activeAlertsWorkbook" + }, + { + "name": "location", + "source": "static", + "value": "{Location}" + } + ] + } + } + ] + }, + "conditionalVisibility": { + "parameterName": "SystemAlertId", + "comparison": "isNotEqualTo" + }, + "name": "Alerts " + }, + { + "type": 1, + "content": { + "json": "### MITRE ATT&CK tactics                                 " + }, + "customWidth": "100", + "name": "text - 17" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "securityresources\r\n| where type == \"microsoft.security/locations/alerts\"\r\n| extend Details = parse_json(properties)\r\n| extend Tactics = Details.[\"Intent\"]\r\n| project Tactics\r\n| extend Tactic = split(Tactics,\",\")\r\n| mv-expand Tactic\r\n| extend Tactic = trim(\" \",tostring(Tactic))\r\n| summarize Count = count() by Tactic\r\n| sort by Count desc\r\n", + "size": 0, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "barchart", + "tileSettings": { + "showBorder": false, + "titleContent": { + "columnMatch": "Tactics", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + }, + "graphSettings": { + "type": 0, + "topContent": { + "columnMatch": "Tactics", + "formatter": 1 + }, + "centerContent": { + "columnMatch": "count_", + "formatter": 1, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + }, + "mapSettings": { + "locInfo": "LatLong", + "sizeSettings": "count_", + "sizeAggregation": "Sum", + "legendMetric": "count_", + "legendAggregation": "Sum", + "itemColorSettings": { + "type": "heatmap", + "colorAggregation": "Sum", + "nodeColorField": "count_", + "heatmapPalette": "greenRed" + } + } + }, + "name": "query - 17" + }, + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "tabs", + "links": [ + { + "id": "bd374a50-b240-4232-ad4a-77725f80bcf5", + "cellValue": "View", + "linkTarget": "parameter", + "linkLabel": "List View", + "subTarget": "List", + "preText": "", + "style": "link" + }, + { + "id": "588b7d9f-8ff1-4afa-8d3f-b0085ae6b148", + "cellValue": "View", + "linkTarget": "parameter", + "linkLabel": "Map View", + "subTarget": "Map", + "preText": "", + "style": "link" + } + ] + }, + "name": "links - 10" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "1ffc8fe9-a919-4c9e-8489-a92f0a7d79e1", + "version": "KqlParameterItem/1.0", + "name": "ResourceFilter", + "label": "Resource", + "type": 5, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": "securityresources\r\n | where type =~ 'microsoft.security/locations/alerts'\r\n | extend Prop = parse_json(properties)\r\n | extend ResourceIdentifiers = Prop.[\"ResourceIdentifiers\"]\r\n | project ResourceIdentifiers\r\n | mv-expand ResourceIdentifiers\r\n | extend ResourceId = parse_json(ResourceIdentifiers).[\"AzureResourceId\"]\r\n //| where isnotempty(ResourceId )\r\n | extend Resource = tolower(tostring(ResourceId))\r\n | summarize count() by Resource\r\n | project Resource\r\n //| order by Resource asc\r\n", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::all", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": [ + "value::all" + ] + }, + { + "id": "e9522d87-143f-408b-93ea-b8f07223995e", + "version": "KqlParameterItem/1.0", + "name": "SeverityFilter", + "label": "Severity", + "type": 2, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "value": [ + "value::all" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "showDefault": false + }, + "jsonData": "[\r\n\r\n{\"value\": \"High\", \"label\":\"High\"},\r\n{\"value\": \"Medium\", \"label\":\"Medium\"},\r\n{\"value\": \"Low\", \"label\":\"Low\"},\r\n{\"value\": \"Informational\", \"label\":\"Informational\"}\r\n]\r\n \r\n ", + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::all" + }, + { + "id": "664365b5-1fc4-4cfa-b99d-a72e3d35ab11", + "version": "KqlParameterItem/1.0", + "name": "ResourceGroupFilter", + "label": "Resource group", + "type": 2, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": " securityresources\r\n | where type =~ 'microsoft.security/locations/alerts'\r\n | extend resourceGroup = iif(isempty(resourceGroup),\" \",resourceGroup)\r\n| summarize Count =count() by resourceGroup\r\n | project resourceGroup", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "defaultValue": "value::all", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": [ + "value::all" + ] + }, + { + "id": "48a8dd7e-43ab-413e-88f8-a433100d92ce", + "version": "KqlParameterItem/1.0", + "name": "AlertNameFilter", + "label": "Alert name", + "type": 2, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": " securityresources\r\n | where type =~ 'microsoft.security/locations/alerts'\r\n | extend Prop = parse_json(properties)\r\n | extend AlertDisplayName = Prop.[\"AlertDisplayName\"]\r\n | distinct tostring(AlertDisplayName)\r\n | order by AlertDisplayName asc\r\n ", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "showDefault": false + }, + "defaultValue": "value::all", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + { + "id": "378aeb0c-9135-43fa-b46a-86f71baa0137", + "version": "KqlParameterItem/1.0", + "name": "TagFilter", + "label": "Tag", + "type": 2, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": "securityresources\r\n | where type =~ \"microsoft.security/locations/alerts\"\r\n | extend isAzure = tostring(properties.ResourceIdentifiers) matches regex '\"Type\"\\\\s*:\\\\s*\"AzureResource\"'\r\n | extend Details = parse_json(properties)\r\n| extend IsIncident = Details.[\"IsIncident\"]\r\n| extend AlertDisplayName = Details.[\"AlertDisplayName\"]\r\n| extend SystemAlertId = Details.[\"SystemAlertId\"]\r\n| extend Severity = Details.[\"Severity\"]\r\n| extend AlertUri = Details.[\"AlertUri\"]\r\n| extend Status = Details.[\"Status\"]\r\n| extend Tactics = Details.[\"Intent\"]\r\n| parse AlertUri with * '/subscriptionId/' SubscriptionId '/' *\r\n| parse AlertUri with * '/resourceGroup/' ResourceGroup '/' *\r\n| parse AlertUri with * '/location/' Location \r\n | extend affectedResourceId = extract('\"AzureResourceId\"\\\\s*:\\\\s*\"([^\"]*)\"', 1, tostring(properties.ResourceIdentifiers))\r\n | extend hostName = iff(isAzure, \"\", extract('\"HostName\"\\\\s*:\\\\s*\"([^\"]*)\"', 1, tostring(properties.Entities)))\r\n | extend splitAffectedResourceId = split(affectedResourceId, \"/\")\r\n | extend resourceNameIndex = iff(array_length(splitAffectedResourceId) > 1, array_length(splitAffectedResourceId) - 1, 0)\r\n | extend affectedResourceName = iff(isAzure, splitAffectedResourceId[resourceNameIndex], iff(isempty(hostName), \"Non-Azure\", hostName))| project-away resourceNameIndex, splitAffectedResourceId, hostName\r\n | extend ResourceIdentifiers = Details.[\"ResourceIdentifiers\"]\r\n | mv-expand ResourceIdentifiers\r\n | extend ResourceId = parse_json(ResourceIdentifiers).[\"AzureResourceId\"]\r\n | extend Resource = tolower(tostring(ResourceId))\r\n | project alertId = id, subscriptionId, alertProperties = properties, affectedResourceId = tolower(affectedResourceId),tostring(Severity), SystemAlertId, AlertDisplayName,IsIncident = iif(IsIncident==\"true\",\"Incident\",\"Alert\"),AlertUri,Status,Tactics,SubscriptionId,ResourceGroup,Location, ResourceIdentifier=Details.[\"ResourceIdentifiers\"],Resource\r\n | join kind=leftouter (\r\n resources\r\n | project id = tolower(id), tags\r\n ) on $left.affectedResourceId == $right.id\r\n | extend Tag = parse_json(tags)\r\n | mv-expand Tag\r\n | parse Tag with * ':\"' TagValue '\"}'\r\n | extend TagValue = iif(isempty(TagValue),\" \",TagValue)\r\n | project TagValue, alertId\r\n | distinct TagValue\r\n ", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "showDefault": false + }, + "defaultValue": "value::all", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "conditionalVisibility": { + "parameterName": "View", + "comparison": "isEqualTo", + "value": "List" + }, + "name": "parameters - 23" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "securityresources\r\n| where type =~ \"microsoft.security/locations/alerts\"\r\n| extend isAzure = tostring(properties.ResourceIdentifiers) matches regex '\"Type\"\\\\s*:\\\\s*\"AzureResource\"'\r\n| extend Details = parse_json(properties)\r\n| extend IsIncident = Details.[\"IsIncident\"]\r\n| extend AlertDisplayName = Details.[\"AlertDisplayName\"]\r\n| extend SystemAlertId = Details.[\"SystemAlertId\"]\r\n| extend Severity = Details.[\"Severity\"]\r\n| extend AlertUri = Details.[\"AlertUri\"]\r\n| extend Status = Details.[\"Status\"]\r\n| extend Tactics = Details.[\"Intent\"]\r\n| parse AlertUri with * '/subscriptionId/' SubscriptionId '/' *\r\n| parse AlertUri with * '/resourceGroup/' ResourceGroup '/' *\r\n| parse AlertUri with * '/location/' Location \r\n| extend affectedResourceId = extract('\"AzureResourceId\"\\\\s*:\\\\s*\"([^\"]*)\"', 1, tostring(properties.ResourceIdentifiers))\r\n| extend hostName = iff(isAzure, \"\", extract('\"HostName\"\\\\s*:\\\\s*\"([^\"]*)\"', 1, tostring(properties.Entities)))\r\n| extend splitAffectedResourceId = split(affectedResourceId, \"/\")\r\n| extend resourceNameIndex = iff(array_length(splitAffectedResourceId) > 1, array_length(splitAffectedResourceId) - 1, 0)\r\n| extend affectedResourceName = iff(isAzure, splitAffectedResourceId[resourceNameIndex], iff(isempty(hostName), \"Non-Azure\", hostName))\r\n| project-away resourceNameIndex, splitAffectedResourceId, hostName\r\n| extend ResourceIdentifiers = Details.[\"ResourceIdentifiers\"]\r\n| mv-expand ResourceIdentifiers\r\n| extend ResourceId = parse_json(ResourceIdentifiers).[\"AzureResourceId\"]\r\n| extend Resource = tolower(tostring(ResourceId))\r\n| project\r\n alertId = id,\r\n subscriptionId,\r\n alertProperties = properties,\r\n affectedResourceId = tolower(affectedResourceId),\r\n tostring(Severity),\r\n SystemAlertId,\r\n AlertDisplayName,\r\n IsIncident = iif(IsIncident == \"true\", \"Incident\", \"Alert\"),\r\n AlertUri,\r\n Status,\r\n Tactics,\r\n SubscriptionId,\r\n ResourceGroup,\r\n Location,\r\n ResourceId,\r\n ResourceIdentifier=Details.[\"ResourceIdentifiers\"],\r\n Resource\r\n| join kind=leftouter (\r\n resources\r\n | project id = tolower(id), tags\r\n )\r\n on $left.affectedResourceId == $right.id\r\n| extend id = alertId, subscriptionId, properties = alertProperties\r\n| extend ResourceFilter =\" {ResourceFilter}\"\r\n| where Resource in~ ({ResourceFilter})\r\n| where Severity in~ ({SeverityFilter})\r\n| where AlertDisplayName in~ ({AlertNameFilter})\r\n| where Status == \"Active\"\r\n| extend ResourceGroup = iif(isempty(ResourceGroup), \" \", ResourceGroup)\r\n| where ResourceGroup in~ ({ResourceGroupFilter})\r\n| extend tag = iff(isempty(tags), dynamic({\"tags\": \" \"}), parse_json(tags))\r\n| mv-expand tag\r\n| parse tag with * ':\"' TagValue '\"}'\r\n| extend TagValue = iif(isempty(TagValue), \" \", TagValue)\r\n| where TagValue in ({TagFilter})\r\n| where AlertDisplayName !startswith ('[SAMPLE ALERT]')\r\n| project\r\n (Severity),\r\n tostring(SystemAlertId),\r\n tostring(AlertDisplayName),\r\n IsIncident = iif(IsIncident == \"true\", \"Incident\", \"Alert\"),\r\n AlertURI = tostring(AlertUri),\r\n tostring(Status),\r\n tostring(Tactics),\r\n SubscriptionId,\r\n ResourceGroup,\r\n Location,\r\n TagValue,\r\n tostring(tags),\r\n tostring(ResourceId)\r\n| distinct\r\n Severity,\r\n SystemAlertId,\r\n AlertDisplayName,\r\n IsIncident,\r\n AlertURI,\r\n Status,\r\n Tactics,\r\n SubscriptionId,\r\n ResourceGroup,\r\n Location,\r\n tags,\r\n ResourceId\r\n| order by Severity asc", + "size": 0, + "title": "Active Alerts ", + "exportedParameters": [ + { + "fieldName": "Resource", + "parameterName": "Resource", + "parameterType": 1 + }, + { + "fieldName": "AlertUri", + "parameterName": "AlertUri", + "parameterType": 1 + }, + { + "fieldName": "SystemAlertId", + "parameterName": "SystemAlertId", + "parameterType": 1 + }, + { + "fieldName": "SubscriptionId", + "parameterName": "SubscriptionId", + "parameterType": 1 + }, + { + "fieldName": "ResourceGroup", + "parameterName": "ResourceGroup", + "parameterType": 1 + }, + { + "fieldName": "Location", + "parameterName": "Location", + "parameterType": 1 + } + ], + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "Severity", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "colors", + "thresholdsGrid": [ + { + "operator": "contains", + "thresholdValue": "High", + "representation": "redBright", + "text": "{0}{1}" + }, + { + "operator": "contains", + "thresholdValue": "Medium", + "representation": "orange", + "text": "{0}{1}" + }, + { + "operator": "contains", + "thresholdValue": "Low", + "representation": "yellow", + "text": "{0}{1}" + }, + { + "operator": "contains", + "thresholdValue": "Informational ", + "representation": "gray", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": null, + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "SystemAlertId", + "formatter": 5 + }, + { + "columnMatch": "IsIncident", + "formatter": 1 + }, + { + "columnMatch": "AlertURI", + "formatter": 5, + "formatOptions": { + "linkTarget": "Url" + } + }, + { + "columnMatch": "Status", + "formatter": 5 + }, + { + "columnMatch": "SubscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": "Resource", + "showIcon": true + } + }, + { + "columnMatch": "Location", + "formatter": 5 + }, + { + "columnMatch": "ResourceId", + "formatter": 13, + "formatOptions": { + "linkTarget": "Resource", + "showIcon": true + } + }, + { + "columnMatch": "Alert ID", + "formatter": 5 + }, + { + "columnMatch": "Alert URI", + "formatter": 5, + "formatOptions": { + "linkTarget": "Url" + } + }, + { + "columnMatch": "Resource ID", + "formatter": 5 + }, + { + "columnMatch": "AlertUri", + "formatter": 5 + }, + { + "columnMatch": "ResourceIdentifier", + "formatter": 5 + }, + { + "columnMatch": "TenantId", + "formatter": 5 + }, + { + "columnMatch": "AlertName", + "formatter": 5 + }, + { + "columnMatch": "Description", + "formatter": 5 + }, + { + "columnMatch": "ProviderName", + "formatter": 5 + }, + { + "columnMatch": "VendorName", + "formatter": 5 + }, + { + "columnMatch": "VendorOriginalId", + "formatter": 5 + }, + { + "columnMatch": "SourceComputerId", + "formatter": 5 + }, + { + "columnMatch": "AlertType", + "formatter": 5 + }, + { + "columnMatch": "ConfidenceLevel", + "formatter": 5 + }, + { + "columnMatch": "ConfidenceScore", + "formatter": 5 + }, + { + "columnMatch": "StartTime", + "formatter": 5 + }, + { + "columnMatch": "EndTime", + "formatter": 5 + }, + { + "columnMatch": "ProcessingEndTime", + "formatter": 5 + }, + { + "columnMatch": "RemediationSteps", + "formatter": 5 + }, + { + "columnMatch": "ExtendedProperties", + "formatter": 5 + }, + { + "columnMatch": "Entities", + "formatter": 5 + }, + { + "columnMatch": "SourceSystem", + "formatter": 5 + }, + { + "columnMatch": "WorkspaceSubscriptionId", + "formatter": 5 + }, + { + "columnMatch": "WorkspaceResourceGroup", + "formatter": 5 + }, + { + "columnMatch": "ExtendedLinks", + "formatter": 5 + }, + { + "columnMatch": "ProductName", + "formatter": 5 + }, + { + "columnMatch": "ProductComponentName", + "formatter": 5 + }, + { + "columnMatch": "AlertLink", + "formatter": 7, + "formatOptions": { + "linkTarget": "Url" + } + }, + { + "columnMatch": "SystemIncidentId", + "formatter": 5 + }, + { + "columnMatch": "SystemAlertId1", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "AlertDisplayName", + "label": "Alert name" + }, + { + "columnId": "IsIncident", + "label": "Incident/alert" + }, + { + "columnId": "AlertURI", + "label": "Alert URI" + }, + { + "columnId": "SubscriptionId", + "label": "Subscription" + }, + { + "columnId": "ResourceGroup", + "label": "Resource group" + }, + { + "columnId": "tags", + "label": "Tags" + }, + { + "columnId": "ResourceId", + "label": "Resource" + } + ] + }, + "sortBy": [] + }, + "conditionalVisibility": { + "parameterName": "View", + "comparison": "isEqualTo", + "value": "List" + }, + "showPin": true, + "name": "SecurityIncidents" + }, + { + "type": 11, + "content": { + "version": "LinkItem/1.0", + "style": "list", + "links": [ + { + "id": "8e6f9368-ccbe-4092-b898-8a27c77a06b3", + "linkTarget": "OpenBlade", + "linkLabel": "Open Alert View", + "preText": "", + "style": "primary", + "bladeOpenContext": { + "bladeName": "AlertBlade", + "extensionName": "Microsoft_Azure_Security", + "bladeParameters": [ + { + "name": "alertId", + "source": "static", + "value": "{SystemAlertId}" + }, + { + "name": "subscriptionId", + "source": "static", + "value": "{SubscriptionId}" + }, + { + "name": "resourceGroup", + "source": "static", + "value": "{ResourceGroup}" + }, + { + "name": "referencedFrom", + "source": "static", + "value": "activeAlertsWorkbook" + }, + { + "name": "location", + "source": "static", + "value": "{Location}" + } + ] + } + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "View", + "comparison": "isEqualTo", + "value": "List" + }, + { + "parameterName": "SystemAlertId", + "comparison": "isNotEqualTo" + } + ], + "name": "links - 19" + }, + { + "type": 1, + "content": { + "json": " To see more information about the alerts in the map view:

  1. Configure continuous export to export your security alerts to a Log Analytics workspace by following the instructions described \r\n[ here. ](https://docs.microsoft.com/azure/defender-for-cloud/continuous-export?tabs=azure-portal)\r\n
  2. In the \"Workspace\" filter below, choose the Log Analytics workspace your security alerts are exported to.\r\n\r\n" + }, + "conditionalVisibility": { + "parameterName": "View", + "comparison": "isEqualTo", + "value": "Map" + }, + "name": "text - 21" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "8724f927-b766-4814-a895-8c55565fb7f8", + "version": "KqlParameterItem/1.0", + "name": "Workspace", + "type": 5, + "multiSelect": true, + "quote": "'", + "delimiter": ",", + "query": "resources\r\n| where type contains \"solution\"\r\n| where name contains \"security\"\r\n| project id = tostring(properties.workspaceResourceId)\r\n| distinct id", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [ + "value::all" + ], + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + } + ], + "style": "pills", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "conditionalVisibility": { + "parameterName": "View", + "comparison": "isEqualTo", + "value": "Map" + }, + "name": "parameters - 15" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "securityresources\r\n| where type == \"microsoft.security/locations/alerts\"\r\n| project-rename P= properties\r\n| extend Details = parse_json(P)\r\n | extend ResourceIdentifiers = Details.[\"ResourceIdentifiers\"]\r\n| extend IsIncident = Details.[\"IsIncident\"]\r\n| extend AlertDisplayName = Details.[\"AlertDisplayName\"]\r\n| extend SystemAlertId = Details.[\"SystemAlertId\"]\r\n| extend Severity = Details.[\"Severity\"]\r\n| extend AlertLink = Details.[\"AlertUri\"]\r\n| extend Status = Details.[\"Status\"]\r\n| extend Tactics = Details.[\"Intent\"]\r\n| parse AlertLink with * '/subscriptionId/' SubscriptionId '/' *\r\n| parse AlertLink with * '/resourceGroup/' ResourceGroup '/' *\r\n| parse AlertLink with * '/location/' Location \r\n | mv-expand ResourceIdentifiers\r\n | extend ResourceId = parse_json(ResourceIdentifiers).[\"AzureResourceId\"]\r\n | where isnotempty(ResourceId )\r\n| project Severity, SystemAlertId, tostring(AlertDisplayName),IsIncident = iif(IsIncident==\"true\",\"Incident\",\"Alert\"),tostring(AlertLink),Status,Tactics,tostring(ResourceId),SubscriptionId,ResourceGroup,Location\r\n| distinct tostring(SystemAlertId),tostring(AlertDisplayName),tostring(AlertLink),tostring(ResourceId)\r\n| summarize count() by ResourceId, AlertLink, AlertDisplayName\r\n", + "size": 0, + "title": "AlertsMapView ", + "exportMultipleValues": true, + "exportAggregateParts": true, + "exportedParameters": [ + { + "fieldName": "ResourceId", + "parameterName": "Resource", + "parameterType": 1 + }, + { + "fieldName": "AlertLink", + "parameterName": "AlertLink", + "parameterType": 1 + }, + { + "fieldName": "AlertDisplayName", + "parameterName": "AlertDisplayName", + "parameterType": 1 + } + ], + "exportToExcelOptions": "all", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "map", + "mapSettings": { + "locInfo": "AzureResource", + "locInfoColumn": "ResourceId", + "sizeSettings": "count_", + "sizeAggregation": "Sum", + "legendMetric": "count_", + "legendAggregation": "Sum", + "itemColorSettings": { + "nodeColorField": "count_", + "colorAggregation": "Sum", + "type": "heatmap", + "heatmapPalette": "coldHot" + } + } + }, + "customWidth": "50", + "conditionalVisibility": { + "parameterName": "View", + "comparison": "isEqualTo", + "value": "Map" + }, + "name": "AlertsMapView ", + "styleSettings": { + "showBorder": true + } + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "let T = datatable ( AlertLink:string)\r\n[\r\n{AlertLink}\r\n];\r\nT\r\n| parse AlertLink with * '/alertId/' AlertId '/subscriptionId/' SubscriptionId '/resourceGroup/' ResourceGroup '/' * 'location/' Location \r\n| distinct AlertLink, AlertId, ResourceGroup,Location,SubscriptionId\r\n| join kind = inner (SecurityAlert\r\n| where isempty(ResourceId) == false\r\n| where TimeGenerated > ago(90d)\r\n| project SystemAlertId,ResourceId, DisplayName,StartTime) on $left.AlertId == $right.SystemAlertId\r\n| project ResourceId,DisplayName,AlertId, SubscriptionId, ResourceGroup, Location,StartTime\r\n| order by ResourceId,DisplayName, StartTime asc\r\n\r\n\r\n\r\n", + "size": 0, + "exportedParameters": [ + { + "fieldName": "AlertId", + "parameterName": "AlertId", + "parameterType": 1 + }, + { + "fieldName": "SubscriptionId", + "parameterName": "SubscriptionId", + "parameterType": 1 + }, + { + "fieldName": "Location", + "parameterName": "Location", + "parameterType": 1 + }, + { + "fieldName": "ResourceGroup", + "parameterName": "ResourceGroup", + "parameterType": 1 + } + ], + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces", + "crossComponentResources": [ + "{Workspace}" + ], + "visualization": "table", + "gridSettings": { + "formatters": [ + { + "columnMatch": "ResourceId", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "23ch" + } + }, + { + "columnMatch": "DisplayName", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "45ch" + } + }, + { + "columnMatch": "AlertId", + "formatter": 5 + }, + { + "columnMatch": "Location", + "formatter": 5 + }, + { + "columnMatch": "StartTime", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "23ch" + } + }, + { + "columnMatch": "TimeGenerated", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "22ch" + } + }, + { + "columnMatch": "AlertLink", + "formatter": 5 + } + ], + "labelSettings": [ + { + "columnId": "ResourceId", + "label": "Resource ID" + }, + { + "columnId": "DisplayName", + "label": "Alert name" + }, + { + "columnId": "AlertId", + "label": "Alert ID" + }, + { + "columnId": "SubscriptionId", + "label": "Subscription ID" + }, + { + "columnId": "ResourceGroup", + "label": "Resource group" + }, + { + "columnId": "StartTime", + "label": "Start time" + } + ] + }, + "sortBy": [], + "tileSettings": { + "showBorder": false + } + }, + "customWidth": "45", + "conditionalVisibilities": [ + { + "parameterName": "View", + "comparison": "isEqualTo", + "value": "Map" + }, + { + "parameterName": "AlertLink", + "comparison": "isNotEqualTo" + } + ], + "name": "AlertLink-Table" + } + ] + }, + "name": "Security Discipline" + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RC_Security" + }, + { + "parameterName": "Subscription", + "comparison": "isNotEqualTo" + } + ], + "name": "RC_Security" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "1e6e4cc7-5d76-48ef-8ce1-16f33f4f6dea", + "version": "KqlParameterItem/1.0", + "name": "SubscriptionAge", + "label": "Subscription", + "type": 6, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "includeAll": false, + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "value": null + } + ], + "style": "above", + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "parameters - SubscriptionAge" + }, + { + "type": 1, + "content": { + "json": "## Azure resource age\r\nAzure *resource age* is one of the metric to monitor as part of the \"resource consistency\" discipline of the Cloud Adoption Framework. This metric help you to identify old resources to be assessed and cleaned if they are not used anymore." + }, + "name": "text - ResourceAgeDescription" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "template", + "loadFromTemplateId": "community-Workbooks/Common/noSubscriptions", + "items": [] + }, + "conditionalVisibility": { + "parameterName": "SubscriptionAge", + "comparison": "isEqualTo", + "value": "" + }, + "name": "No Subscriptions group - Age" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"ARMEndpoint/1.0\",\"data\":null,\"headers\":[],\"method\":\"GET\",\"path\":\"/subscriptions/{SubscriptionAge:id}/resources?\",\"urlParams\":[{\"key\":\"api-version\",\"value\":\"2021-04-01\"},{\"key\":\"$expand\",\"value\":\"createdTime,changedTime,provisioningState\"}],\"batchDisabled\":true,\"transformers\":[{\"type\":\"jsonpath\",\"settings\":{\"tablePath\":\"$.value\",\"columns\":[{\"path\":\"$..id\",\"columnid\":\"id\",\"columnType\":\"string\"},{\"path\":\"$..type\",\"columnid\":\"type\",\"columnType\":\"string\"},{\"path\":\"$..location\",\"columnid\":\"location\",\"columnType\":\"string\"},{\"path\":\"$..createdTime\",\"columnid\":\"createdTime\",\"columnType\":\"datetime\"},{\"path\":\"$..changedTime\",\"columnid\":\"changedTime\",\"columnType\":\"datetime\"},{\"path\":\"$..provisioningState\",\"columnid\":\"provisioningState\",\"columnType\":\"string\"},{\"path\":\"$..tags\",\"columnid\":\"tags\"}]}}]}", + "size": 0, + "title": "Resource age", + "showExportToExcel": true, + "queryType": 12, + "gridSettings": { + "formatters": [ + { + "columnMatch": "type", + "formatter": 16, + "formatOptions": { + "showIcon": true + } + }, + { + "columnMatch": "location", + "formatter": 17 + }, + { + "columnMatch": "createdTime", + "formatter": 6 + }, + { + "columnMatch": "changedTime", + "formatter": 6 + }, + { + "columnMatch": "provisioningState", + "formatter": 1 + }, + { + "columnMatch": "tags", + "formatter": 1 + } + ], + "filter": true, + "sortBy": [ + { + "itemKey": "changedTime", + "sortOrder": 1 + } + ], + "labelSettings": [ + { + "columnId": "id", + "label": "Resource Name" + }, + { + "columnId": "type", + "label": "Resource Type" + }, + { + "columnId": "location", + "label": "Region" + }, + { + "columnId": "createdTime", + "label": "Created Time" + }, + { + "columnId": "changedTime", + "label": "Last Change" + }, + { + "columnId": "provisioningState", + "label": "Provisioning State" + }, + { + "columnId": "tags", + "label": "Tags" + } + ] + }, + "sortBy": [ + { + "itemKey": "changedTime", + "sortOrder": 1 + } + ] + }, + "conditionalVisibility": { + "parameterName": "SubscriptionAge", + "comparison": "isNotEqualTo" + }, + "name": "query - Resource age" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RC_Age" + }, + "name": "Resource Age" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "parameters": [ + { + "id": "e15ef842-dadb-4a7b-b5f6-5d1bbe35b7af", + "version": "KqlParameterItem/1.0", + "name": "Subscription", + "type": 6, + "description": "Cost information can only be displayed per subscription", + "typeSettings": { + "additionalResourceOptions": [], + "includeAll": true, + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "value": null + }, + { + "id": "b73ef334-95b2-4ead-8dd2-51a90a90ce6f", + "version": "KqlParameterItem/1.0", + "name": "Aggregation", + "type": 2, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "jsonData": "[[\r\n { \"value\": \"SubscriptionId\", \"label\": \"Subscription\", \"selected\":true},\r\n { \"value\": \"ResourceGroup\", \"label\": \"Resource Group\"},\r\n { \"value\": \"ResourceType\", \"label\": \"Resource Type\"}\r\n]", + "timeContext": { + "durationMs": 86400000 + } + }, + { + "id": "55ef4a45-0603-48cf-bb9b-a963e7a33be2", + "version": "KqlParameterItem/1.0", + "name": "TimeFrame", + "type": 2, + "typeSettings": { + "additionalResourceOptions": [] + }, + "jsonData": "[[\r\n { \"value\": \"BillingMonthToDate\", \"label\": \"Billing MonthToDate\"},\r\n { \"value\": \"MonthToDate\", \"label\": \"MonthToDate\", \"selected\":true },\r\n { \"value\": \"TheLastBillingMonth\", \"label\": \"Last Billing Month\"},\r\n { \"value\": \"TheLastMonth\", \"label\": \"Last Month\"},\r\n { \"value\": \"WeekToDate\", \"label\": \"WeekToDate\"}\r\n]", + "timeContext": { + "durationMs": 86400000 + }, + "label": "Timeframe" + } + ], + "style": "above", + "doNotRunWhenHidden": true, + "queryType": 0, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "parameters - Cost Subscription" + }, + { + "type": 1, + "content": { + "json": "## Microsoft Cost Management\r\n\r\nBefore you can control and optimize your costs, you first need to understand where they originated – from the underlying resources used to support your cloud projects to the environments they're deployed in and the owners who manage them. Full visibility backed by a thorough tagging strategy is critical to accurately understand your spending patterns and enforce cost control mechanisms.\r\n\r\n[Cost Management](https://portal.azure.com/#view/Microsoft_Azure_CostManagement/Menu) is a set of FinOps tools that enable you to analyze, manage, and optimize your costs.\r\n\r\nCalculate your estimated hourly or monthly costs for using Azure with the [Azure Calculator](https://azure.microsoft.com/pricing/calculator/).\r\n\r\nFor more advanced reporting options, build custom [Power BI reports in the FinOps toolkit](https://aka.ms/ftk/pbi)." + }, + "name": "text - AzureCostManagement" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"ARMEndpoint/1.0\",\"data\":\" {\\r\\n \\\"type\\\": \\\"Usage\\\",\\r\\n \\\"timeframe\\\": \\\"{TimeFrame}\\\",\\r\\n \\\"dataset\\\": {\\r\\n \\\"granularity\\\": \\\"None\\\",\\r\\n \\\"aggregation\\\": {\\r\\n \\\"totalCost\\\": {\\r\\n \\\"name\\\": \\\"PreTaxCost\\\",\\r\\n \\\"function\\\": \\\"Sum\\\"\\r\\n }\\r\\n },\\r\\n \\\"grouping\\\": [\\r\\n {\\r\\n \\\"type\\\": \\\"Dimension\\\",\\r\\n \\\"name\\\": \\\"{Aggregation}\\\"\\r\\n }\\r\\n ]\\r\\n }\\r\\n }\",\"headers\":[],\"method\":\"POST\",\"path\":\"/subscriptions/{Subscription:id}/providers/Microsoft.CostManagement/query?\",\"urlParams\":[{\"key\":\"api-version\",\"value\":\"2023-11-01\"}],\"batchDisabled\":true,\"transformers\":[{\"type\":\"jsonpath\",\"settings\":{\"tablePath\":\"$.properties\",\"columns\":[]}}]}", + "size": 0, + "title": "Overall cost", + "showExportToExcel": true, + "queryType": 12, + "gridSettings": { + "formatters": [ + { + "columnMatch": "PreTaxCost", + "formatter": 0, + "numberFormat": { + "unit": 0, + "options": { + "style": "decimal", + "useGrouping": true, + "maximumFractionDigits": 2 + }, + "emptyValCustomText": "\"0\"" + } + }, + { + "columnMatch": "SubscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "ResourceGroup", + "formatter": 14, + "formatOptions": { + "linkTarget": "Resource", + "showIcon": false + } + }, + { + "columnMatch": "ResourceType", + "formatter": 16, + "formatOptions": { + "showIcon": true + } + }, + { + "columnMatch": "resourceGroup", + "formatter": 14, + "formatOptions": { + "linkTarget": "Resource", + "linkIsContextBlade": false, + "showIcon": false + } + } + ], + "filter": true, + "sortBy": [ + { + "itemKey": "$gen_number_PreTaxCost_0", + "sortOrder": 2 + } + ], + "labelSettings": [ + { + "columnId": "PreTaxCost", + "label": "Cost" + }, + { + "columnId": "Currency", + "label": "Currency" + } + ] + }, + "sortBy": [ + { + "itemKey": "$gen_number_PreTaxCost_0", + "sortOrder": 2 + } + ], + "tileSettings": { + "showBorder": false + } + }, + "conditionalVisibility": { + "parameterName": "Subscription", + "comparison": "isNotEqualTo" + }, + "name": "query - Overall cost" + }, + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "6cc7fc26-1a56-41cb-ad43-301e0f9f8903", + "version": "KqlParameterItem/1.0", + "name": "TagName", + "label": "Tag name", + "type": 2, + "isRequired": true, + "query": "Resources\r\n| where tags != '' and tags != '[]'\r\n| mvexpand tags\r\n| extend tagName = tostring(bag_keys(tags)[0])\r\n| distinct tagName\r\n| sort by tagName asc", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": null + }, + { + "id": "2fc46f5d-ce69-42ea-8ebf-1c3d69c4e780", + "version": "KqlParameterItem/1.0", + "name": "TagValue", + "label": "Tag value", + "type": 2, + "isRequired": true, + "query": "Resources\r\n| extend TagValue = tostring(tags.{TagName})\r\n| project TagValue\r\n| distinct TagValue", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": "" + } + ], + "style": "above", + "doNotRunWhenHidden": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "conditionalVisibility": { + "parameterName": "Subscription", + "comparison": "isNotEqualTo" + }, + "name": "parameters - TagFilter" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"ARMEndpoint/1.0\",\"data\":\" {\\r\\n \\\"type\\\": \\\"Usage\\\",\\r\\n \\\"timeframe\\\": \\\"{TimeFrame}\\\",\\r\\n \\\"dataset\\\": {\\r\\n \\\"granularity\\\": \\\"None\\\",\\r\\n \\\"filter\\\": {\\r\\n \\\"tags\\\" : {\\r\\n \\\"name\\\" : \\\"{TagName}\\\",\\r\\n \\\"operator\\\" : \\\"In\\\",\\r\\n \\\"values\\\" : [\\r\\n \\\"{TagValue}\\\"\\r\\n ]\\r\\n }\\r\\n },\\r\\n \\\"aggregation\\\": {\\r\\n \\\"totalCost\\\": {\\r\\n \\\"name\\\": \\\"PreTaxCost\\\",\\r\\n \\\"function\\\": \\\"Sum\\\"\\r\\n }\\r\\n },\\r\\n \\\"grouping\\\": [\\r\\n {\\r\\n \\\"type\\\": \\\"Dimension\\\",\\r\\n \\\"name\\\": \\\"{Aggregation}\\\"\\r\\n }\\r\\n ]\\r\\n }\\r\\n }\",\"headers\":[],\"method\":\"POST\",\"path\":\"/subscriptions/{Subscription:id}/providers/Microsoft.CostManagement/query?\",\"urlParams\":[{\"key\":\"api-version\",\"value\":\"2023-11-01\"}],\"batchDisabled\":true,\"transformers\":[{\"type\":\"jsonpath\",\"settings\":{\"tablePath\":\"$.properties\",\"columns\":[]}}]}", + "size": 3, + "title": "Overall cost filtered by tag", + "showExportToExcel": true, + "queryType": 12, + "gridSettings": { + "formatters": [ + { + "columnMatch": "PreTaxCost", + "formatter": 0, + "numberFormat": { + "unit": 0, + "options": { + "style": "decimal", + "useGrouping": true, + "maximumFractionDigits": 2 + }, + "emptyValCustomText": "\"0\"" + } + }, + { + "columnMatch": "SubscriptionId", + "formatter": 15, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "ResourceType", + "formatter": 16, + "formatOptions": { + "showIcon": true + } + }, + { + "columnMatch": "resourceGroup", + "formatter": 14, + "formatOptions": { + "linkTarget": "Resource", + "linkIsContextBlade": false, + "showIcon": true + } + } + ], + "labelSettings": [ + { + "columnId": "PreTaxCost", + "label": "Cost" + } + ] + }, + "tileSettings": { + "showBorder": false + } + }, + "conditionalVisibility": { + "parameterName": "TagValue", + "comparison": "isNotEqualTo", + "value": "" + }, + "name": "query - Sub cost per tag" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RC_Cost" + }, + "name": "RC_Cost Management" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "loadType": "always", + "items": [ + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "value::tenant" + ], + "parameters": [ + { + "id": "476f61f4-2271-4e58-9b5e-7958d9a4ca3b", + "version": "KqlParameterItem/1.0", + "name": "Subscription", + "type": 6, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "includeAll": false, + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "value": null + } + ], + "style": "above", + "doNotRunWhenHidden": true, + "queryType": 1, + "resourceType": "microsoft.resources/tenants" + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RC_Compliance" + }, + "name": "parameters - Scope Filter For RC_Compliance" + }, + { + "type": 1, + "content": { + "json": "## Build and scale your applications quickly while maintaining control\r\nTake advantage of built-in and custom policies to set guardrails in your subscriptions. Easily deploy fully governed environments throughout your organization with Azure Blueprints. And, manage costs by gaining insights into your cloud spend so that you get the most from your cloud investments.
\r\n- Enforce and audit your policies for any Azure service
\r\n- Create compliant environments using Azure Blueprints, including resources, policies, and role-access controls
\r\n- Ensure that you’re compliant with external regulations by using built-in compliance controls
\r\n- Monitor cost and encourage accountability across your entire organization" + }, + "name": "text - 16" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "// Breakdown of compliance information for each assignment at subscription/MG/tenant scope\r\n// Gets aggregated compliance and policy definition information for each of the assignments in the selected scope as well as a few additional details, including: policySetDefinition or policyDefinition details for those assignments, the number of policies/groups within the policysetDefinitions listed, number of non-compliant policies within each policySetDefinition and the resource count breakdown per compliance state for those assignments.\r\n// Click the \"Run query\" command above to execute the query and see results.\r\npolicyResources\r\n| where type =~'Microsoft.Authorization/PolicyAssignments'\r\n| project policyAssignmentId = tolower(tostring(id)), policyAssignmentName = name, policyAssignmentDisplayName = tostring(properties.displayName), policyAssignmentScope = tostring(properties.scope), policyAssignmentDefinitionId = tolower(properties.policyDefinitionId), policyAssignmentNotScopes = tolower(properties.notScopes) \r\n| where policyAssignmentScope == \"{Subscription}\"\r\n| join kind=leftouter(\r\n policyResources\r\n | where type =~'Microsoft.Authorization/PolicySetDefinitions' or type =~'Microsoft.Authorization/PolicyDefinitions'\r\n | project definitionId = tolower(id), type, numberOfPolicies = array_length(properties.policyDefinitions), category = tostring(properties.metadata.category), numberOfGroups= array_length(properties.policyDefinitionGroups), mode = tostring(properties.mode)\r\n | extend isRegulatoryInitiative = iff(category =~ 'Regulatory Compliance', true, false)\r\n | extend definitionType = iff(type =~ 'Microsoft.Authorization/PolicysetDefinitions', 'initiative', 'policy')\r\n | extend isRPMode = iff(mode startswith 'Microsoft.', true, false)\r\n | project definitionId, numberOfPolicies, category, numberOfGroups, isRegulatoryInitiative, definitionType, isRPMode\r\n) on $left.policyAssignmentDefinitionId == $right.definitionId\r\n| join kind=leftouter(\r\n policyResources \r\n | where type =~ 'Microsoft.PolicyInsights/PolicyStates'\r\n | extend complianceState = tostring(properties.complianceState)\r\n | extend policyStateResourceId =id, resourceId = tostring(properties.resourceId), policyAssignmentId = tostring(properties.policyAssignmentId), policyDefinitionId = tostring(properties.policyDefinitionId), policySetDefinitionId = tostring(properties.policySetDefinitionId), policyDefinitionReferenceId = tostring(properties.policyDefinitionReferenceId), policyDefinitionAction = tostring(properties.policyDefinitionAction), policyDefinitionGroupNames = iff(isnotnull(properties.policyDefinitionGroupNames), properties.policyDefinitionGroupNames, dynamic([''])), stateWeight = toint(properties.stateWeight)\r\n | summarize max(stateWeight) by resourceId, policyAssignmentId, policySetDefinitionId\r\n | summarize resourceCounts = count() by policyAssignmentId, policySetDefinitionId, max_stateWeight\r\n| extend complianceState = case(\r\nmax_stateWeight == 300, 'noncompliant',\r\nmax_stateWeight == 200, 'compliant',\r\nmax_stateWeight == 100, 'conflict',\r\nmax_stateWeight == 50, 'exempt',\r\nmax_stateWeight == 10, 'unknown',\r\n'notapplicable')\r\n | extend pack = pack('complianceState', complianceState, 'resourceCounts', resourceCounts), numberOfNonCompliantResources = toint(iff(complianceState =~ 'NonCompliant', resourceCounts,0))\r\n | summarize numberOfNonCompliantResources = max(numberOfNonCompliantResources), details = makelist(pack) by policyAssignmentId, policySetDefinitionId\r\n | limit 5000\r\n) on $left.policyAssignmentId == $right.policyAssignmentId\r\n| sort by numberOfNonCompliantResources desc\r\n| project-away policyAssignmentId1", + "size": 0, + "title": "Resource compliance", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resources/tenants", + "crossComponentResources": [ + "value::tenant" + ], + "visualization": "table", + "gridSettings": { + "formatters": [ + { + "columnMatch": "policyAssignmentId", + "formatter": 5 + }, + { + "columnMatch": "policyAssignmentName", + "formatter": 5 + }, + { + "columnMatch": "policyAssignmentDisplayName", + "formatter": 7, + "formatOptions": { + "linkTarget": "GenericDetails", + "linkIsContextBlade": true + } + }, + { + "columnMatch": "policyAssignmentScope", + "formatter": 5 + }, + { + "columnMatch": "policyAssignmentDefinitionId", + "formatter": 5 + }, + { + "columnMatch": "policyAssignmentNotScopes", + "formatter": 5 + }, + { + "columnMatch": "definitionId", + "formatter": 5 + }, + { + "columnMatch": "numberOfPolicies", + "formatter": 5 + }, + { + "columnMatch": "numberOfGroups", + "formatter": 5 + }, + { + "columnMatch": "isRegulatoryInitiative", + "formatter": 5 + }, + { + "columnMatch": "isRPMode", + "formatter": 5 + }, + { + "columnMatch": "policySetDefinitionId", + "formatter": 5 + }, + { + "columnMatch": "details", + "formatter": 7, + "formatOptions": { + "linkTarget": "CellDetails", + "linkLabel": "🔍 View details", + "linkIsContextBlade": true + } + } + ], + "filter": true, + "sortBy": [ + { + "itemKey": "category", + "sortOrder": 2 + } + ], + "labelSettings": [ + { + "columnId": "policyAssignmentId", + "label": "Assignment ID" + }, + { + "columnId": "policyAssignmentName", + "label": "Assignment name" + }, + { + "columnId": "policyAssignmentDisplayName", + "label": "Assignment display name" + }, + { + "columnId": "policyAssignmentScope", + "label": "Assignment scope" + }, + { + "columnId": "policyAssignmentDefinitionId", + "label": "Assignment definition ID" + }, + { + "columnId": "definitionId", + "label": "Definition ID" + }, + { + "columnId": "numberOfPolicies", + "label": "Number of policies" + }, + { + "columnId": "category", + "label": "Category" + }, + { + "columnId": "definitionType", + "label": "Type" + }, + { + "columnId": "numberOfNonCompliantResources", + "label": "Non compliant resources" + }, + { + "columnId": "details", + "label": "Details" + } + ] + }, + "sortBy": [ + { + "itemKey": "category", + "sortOrder": 2 + } + ] + }, + "name": "query - ResourceCompliance" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "AzureActivity\r\n| where ActivityStatusValue has \"Failure\"\r\n| summarize AggregatedValue = count() by ResourceProviderValue\r\n| order by AggregatedValue desc", + "size": 3, + "showAnalytics": true, + "title": "Failures by resources", + "timeContext": { + "durationMs": 604800000 + }, + "queryType": 0, + "resourceType": "microsoft.resources/subscriptions", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart", + "tileSettings": { + "showBorder": false, + "titleContent": { + "columnMatch": "ResourceProviderValue", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "AggregatedValue", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + }, + "graphSettings": { + "type": 0, + "topContent": { + "columnMatch": "ResourceProviderValue", + "formatter": 1 + }, + "centerContent": { + "columnMatch": "AggregatedValue", + "formatter": 1, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + } + }, + "customWidth": "33", + "name": "query - Failures by resources" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "AzureActivity\r\n| where ActivityStatusValue has \"Failure\" or ActivityStatusValue has \"Failed\"\r\n| summarize AggregatedValue = count() by OperationNameValue\r\n| order by AggregatedValue desc", + "size": 3, + "showAnalytics": true, + "title": "Failures by operations", + "timeContext": { + "durationMs": 604800000 + }, + "queryType": 0, + "resourceType": "microsoft.resources/subscriptions", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart", + "tileSettings": { + "showBorder": false, + "titleContent": { + "columnMatch": "ResourceProviderValue", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "AggregatedValue", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + }, + "graphSettings": { + "type": 0, + "topContent": { + "columnMatch": "ResourceProviderValue", + "formatter": 1 + }, + "centerContent": { + "columnMatch": "AggregatedValue", + "formatter": 1, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + } + }, + "customWidth": "33", + "name": "query - Failures by operations" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "AzureActivity\r\n| where ActivityStatusValue has \"Failure\" or ActivityStatusValue has \"Failed\"\r\n| summarize AggregatedValue = count() by CategoryValue\r\n| order by AggregatedValue desc", + "size": 3, + "showAnalytics": true, + "title": "Failures by category", + "timeContext": { + "durationMs": 604800000 + }, + "queryType": 0, + "resourceType": "microsoft.resources/subscriptions", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart", + "tileSettings": { + "showBorder": false, + "titleContent": { + "columnMatch": "ResourceProviderValue", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "AggregatedValue", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + }, + "graphSettings": { + "type": 0, + "topContent": { + "columnMatch": "ResourceProviderValue", + "formatter": 1 + }, + "centerContent": { + "columnMatch": "AggregatedValue", + "formatter": 1, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + } + }, + "customWidth": "33", + "name": "query - Failures by category" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "AzureActivity\r\n| where ActivityStatusValue has \"Failure\" or ActivityStatusValue has \"Failed\"\r\n| order by CategoryValue\r\n", + "size": 0, + "title": "Failure by category details", + "timeContext": { + "durationMs": 604800000 + }, + "showExportToExcel": true, + "queryType": 0, + "resourceType": "microsoft.resources/subscriptions", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "Authorization", + "formatter": 5 + }, + { + "columnMatch": "Authorization_d", + "formatter": 5 + }, + { + "columnMatch": "Claims", + "formatter": 5 + }, + { + "columnMatch": "Claims_d", + "formatter": 5 + }, + { + "columnMatch": "Properties_d", + "formatter": 5 + }, + { + "columnMatch": "_ResourceId", + "formatter": 5 + } + ], + "filter": true + } + }, + "name": "query - Failure by category details" + } + ] + }, + "conditionalVisibility": { + "parameterName": "Subscription", + "comparison": "isNotEqualTo" + }, + "name": "group - ComplianceQueries" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RC_Compliance" + }, + "name": "RC_Compliance" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "4168a8b2-a522-4f0d-9575-893d70d9239d", + "version": "KqlParameterItem/1.0", + "name": "RulesCount", + "type": 1, + "description": "Count of the governance rule, when there is no rules, empty state will be shown", + "query": "securityresources\r\n| where type == \"microsoft.security/governancerules\"\r\n| where tostring(properties.isDisabled) == \"false\"\r\n| count", + "crossComponentResources": [ + "{Subscription}" + ], + "isHiddenWhenLocked": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + } + ], + "style": "above", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources" + }, + "name": "Tabs" + }, + { + "type": 1, + "content": { + "json": "## Security governance in Microsoft Defender for Cloud\r\n\r\n Microsoft Defender for Cloud continuously assesses your hybrid and multi-cloud workloads and provides you with recommendations to harden your assets and enhance your security posture.
Central security teams often experience challenges when driving the personnel within their organizations to implement recommendations. The organizations' security posture can suffer as a result.
\r\nWe're introducing a brand-new, built-in governance experience to set ownership and expected remediation timeframes to resolve recommendations.\r\n\r\nTo use this governance report, you need to create security governance rules.\r\n
[Learn more >](https://aka.ms/GovernanceDocumentation)\r\n" + }, + "conditionalVisibility": { + "parameterName": "RulesCount", + "comparison": "isEqualTo" + }, + "name": "text - 13" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "Select one or more governance rules from the list to see a list of affected recommendations", + "style": "info" + }, + "name": "RulesGridExplination" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "securityresources \r\n| where type == \"microsoft.security/assessments\"\r\n| where isnull(properties.resourceDetails.AwsResourceId) and isnull(properties.resourceDetails.GcpResourceId)\r\n| extend DisplayName = tostring(properties.displayName)\r\n| where isempty(DisplayName) == false\r\n| join kind=leftouter (securityresources \r\n| where type == \"microsoft.security/assessments/governanceassignments\"\r\n| extend assignedResourceId = tostring(todynamic(properties).assignedResourceId)\r\n| extend remediationDueDate = todatetime(properties.remediationDueDate)\r\n| project id = assignedResourceId, governanceassignmentsProperties = todynamic(properties), remediationDueDate) on id\r\n| extend hasAssignment = isempty( governanceassignmentsProperties) == false and isnull( governanceassignmentsProperties) == false\r\n| extend assignmentStatus = iif(tostring(properties.status.code) == \"Unhealthy\",iif(hasAssignment == true, iif(bin(remediationDueDate, 1d) < bin(now(), 1d), \"Overdue\", \"Ontime\"), \"Unassigned\") , \"Completed\")\r\n| summarize count() by assignmentStatus\r\n", + "size": 3, + "title": "Resource status", + "noDataMessage": "No unhealthy resources found", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart", + "tileSettings": { + "titleContent": { + "columnMatch": "OsType", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "style": "decimal", + "useGrouping": false, + "maximumFractionDigits": 2, + "maximumSignificantDigits": 3 + } + } + }, + "showBorder": true + }, + "graphSettings": { + "type": 0, + "topContent": { + "columnMatch": "OsType", + "formatter": 1 + }, + "centerContent": { + "columnMatch": "count_", + "formatter": 1, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + }, + "chartSettings": { + "seriesLabelSettings": [ + { + "seriesName": "Ontime", + "color": "blue" + }, + { + "seriesName": "Completed", + "color": "green" + }, + { + "seriesName": "Unassigned", + "color": "orange" + }, + { + "seriesName": "Overdue", + "color": "redBright" + } + ] + } + }, + "customWidth": "20", + "name": "statusePerAssessment" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "securityresources\r\n| where type == \"microsoft.security/governancerules\"\r\n| where tostring(properties.isDisabled) == \"false\"\r\n| extend ruleName = todynamic(name)\r\n| extend ownerType = iif(tostring(properties.ownerSource.type) == \"Manually\", \"Email\", \"ByTag\")\r\n| extend description = tostring(properties.description)\r\n| extend displayName = tostring(properties.displayName)\r\n| extend governanceEmailNotification = todynamic(properties.governanceEmailNotification)\r\n| extend isGracePeriod = todynamic(properties.isGracePeriod)\r\n| extend remediationTimeframe = todynamic(properties.remediationTimeframe)\r\n| extend Days = tolong(totimespan(remediationTimeframe)/1d)\r\n| extend Days = iff(Days > 0, iff(Days == 1, \"1 day\", strcat(Days,\" days\")), \"\")\r\n| extend sourceResourceType = todynamic(properties.sourceResourceType)\r\n| extend conditionSets = todynamic(properties.conditionSets)\r\n| extend rulePriority = todynamic(properties.rulePriority)\r\n| extend ownerSource = todynamic(properties.ownerSource)\r\n| extend isDisabled = todynamic(properties.isDisabled)\r\n| extend ruleType = todynamic(properties.ruleType)\r\n| extend RuleConditionSet = tostring(properties.conditionSets), property = properties.conditionSets[0].conditions[0].property, operator = properties.conditionSets[0].conditions[0].operator\r\n| project Subscription = tostring(subscriptionId), [\"Display name\"] = tostring(properties.displayName), Priority = toint(properties.rulePriority), [\"Remediation timeframe\"] = Days, [\"Owner type\"] = ownerType, Owner = tostring(properties.ownerSource.value), [\"Grace period enabled\"] = tostring(properties.isGracePeriod), Rule = id, properties, RuleConditionSet\r\n| sort by Subscription, Priority asc", + "size": 0, + "title": "Governance rules", + "noDataMessage": "No Rules found", + "exportMultipleValues": true, + "exportedParameters": [ + { + "fieldName": "Rule", + "parameterName": "Rule", + "parameterType": 1, + "quote": "" + }, + { + "fieldName": "RuleConditionSet", + "parameterName": "RuleConditionSet", + "parameterType": 1, + "quote": "" + }, + { + "fieldName": "Owner", + "parameterName": "Owner", + "parameterType": 1, + "quote": "" + } + ], + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "Subscription", + "formatter": 15, + "formatOptions": { + "linkTarget": "Resource", + "showIcon": true + } + }, + { + "columnMatch": "Display name", + "formatter": 1, + "formatOptions": { + "bladeOpenContext": { + "bladeName": "CreateGovernanceRuleContextBlade", + "extensionName": "Microsoft_Azure_Security", + "bladeParameters": [ + { + "name": "", + "source": "column", + "value": "properties" + }, + { + "name": "subscriptionId", + "source": "column", + "value": "subscriptionId" + }, + { + "name": "governanceRuleToEdit", + "source": "column", + "value": "properties" + } + ] + } + } + }, + { + "columnMatch": "Priority", + "formatter": 1 + }, + { + "columnMatch": "Remediation timeframe", + "formatter": 0, + "tooltipFormat": { + "tooltip": "DD.HH.MM.SS" + } + }, + { + "columnMatch": "Grace period enabled", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "true", + "representation": "success", + "text": "" + }, + { + "operator": "==", + "thresholdValue": "false", + "representation": "4", + "text": "" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "Rule", + "formatter": 5 + }, + { + "columnMatch": "subscriptionId", + "formatter": 5 + }, + { + "columnMatch": "name", + "formatter": 1 + }, + { + "columnMatch": "DisplayName", + "formatter": 1 + }, + { + "columnMatch": "ownerDetails", + "formatter": 1 + }, + { + "columnMatch": "isGracePeriod", + "formatter": 1 + }, + { + "columnMatch": "remediationTimeframe", + "formatter": 1 + } + ], + "rowLimit": 1000, + "filter": true, + "sortBy": [ + { + "itemKey": "$gen_link_Subscription_0", + "sortOrder": 2 + } + ], + "labelSettings": [ + { + "columnId": "Owner", + "label": "Owner details" + } + ] + }, + "sortBy": [ + { + "itemKey": "$gen_link_Subscription_0", + "sortOrder": 2 + } + ] + }, + "customWidth": "80", + "conditionalVisibility": { + "parameterName": "Subscription", + "comparison": "isNotEqualTo" + }, + "name": "Rules", + "styleSettings": { + "maxWidth": "100" + } + } + ], + "exportParameters": true + }, + "customWidth": "100", + "conditionalVisibility": { + "parameterName": "RulesCount", + "comparison": "isNotEqualTo" + }, + "name": "subscriptionOverView" + }, + { + "type": 1, + "content": { + "json": "---" + }, + "conditionalVisibility": { + "parameterName": "selectedTab", + "comparison": "isEqualTo", + "value": "resourceView" + }, + "name": "LineSeparator1" + }, + { + "type": 1, + "content": { + "json": "💡 Selected filter for **RuleConditionSet:** {RuleConditionSet}\r\n💡 Selected filter for **Rule:** {Rule}\r\n💡 Selected filter for **Owner:** {Owner}\r\n", + "style": "{selectedTab}" + }, + "conditionalVisibility": { + "parameterName": "parameter1", + "comparison": "isEqualTo", + "value": "1" + }, + "name": "ResourceFilter" + }, + { + "type": 1, + "content": { + "json": " \r\n---" + }, + "conditionalVisibility": { + "parameterName": "selectedTab", + "comparison": "isEqualTo", + "value": "resourceView" + }, + "name": "LineSeparator2" + }, + { + "type": 1, + "content": { + "json": "Select a recommendation from the list to see a list of affected resources", + "style": "info" + }, + "conditionalVisibilities": [ + { + "parameterName": "Rule", + "comparison": "isNotEqualTo" + }, + { + "parameterName": "DisplayName", + "comparison": "isEqualTo" + } + ], + "name": "assessmentsExplaination" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "securityresources \r\n| where type == \"microsoft.security/assessments\"\r\n| where isnull(properties.resourceDetails.AwsResourceId) and isnull(properties.resourceDetails.GcpResourceId)\r\n| extend DisplayName = tostring(properties.displayName)\r\n| where isempty(DisplayName) == false\r\n| extend RuleConditionSet = '{RuleConditionSet}'\r\n| where RuleConditionSet contains name or RuleConditionSet contains properties.metadata.severity\r\n| join kind=leftouter (securityresources \r\n| where type == \"microsoft.security/assessments/governanceassignments\"\r\n| extend assignedResourceId = tostring(todynamic(properties).assignedResourceId)\r\n| extend remediationDueDate = todatetime(properties.remediationDueDate)\r\n| project id = assignedResourceId, governanceassignmentsProperties = todynamic(properties), remediationDueDate) on id\r\n| extend hasAssignment = isempty( governanceassignmentsProperties) == false and isnull( governanceassignmentsProperties) == false\r\n| extend assignmentStatus = iif(tostring(properties.status.code) == \"Unhealthy\",iif(hasAssignment == true, iif(bin(remediationDueDate, 1d) < bin(now(), 1d), \"Overdue\", \"Ontime\"), \"Unassigned\") , \"Completed\")\r\n| summarize count() by assignmentStatus", + "size": 3, + "title": "Status per rule", + "noDataMessage": "No unhealthy resources found", + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "piechart", + "gridSettings": { + "rowLimit": 10000 + }, + "tileSettings": { + "titleContent": { + "columnMatch": "OsType", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "style": "decimal", + "useGrouping": false, + "maximumFractionDigits": 2, + "maximumSignificantDigits": 3 + } + } + }, + "showBorder": true + }, + "graphSettings": { + "type": 0, + "topContent": { + "columnMatch": "OsType", + "formatter": 1 + }, + "centerContent": { + "columnMatch": "count_", + "formatter": 1, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + }, + "chartSettings": { + "seriesLabelSettings": [ + { + "seriesName": "Ontime", + "color": "blue" + }, + { + "seriesName": "Completed", + "color": "green" + }, + { + "seriesName": "Unassigned", + "color": "orange" + }, + { + "seriesName": "Overdue", + "color": "redBright" + } + ] + } + }, + "customWidth": "20", + "conditionalVisibility": { + "parameterName": "Rule", + "comparison": "isNotEqualTo" + }, + "name": "statusPerRule" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "securityresources \r\n| where type == \"microsoft.security/assessments\"\r\n| where isnull(properties.resourceDetails.AwsResourceId) and isnull(properties.resourceDetails.GcpResourceId)\r\n| extend DisplayName = tostring(properties.displayName)\r\n| where isempty(DisplayName) == false\r\n| extend RuleConditionSet = '{RuleConditionSet}'\r\n| where RuleConditionSet contains name or RuleConditionSet contains properties.metadata.severity\r\n| join kind=leftouter (securityresources \r\n| where type == \"microsoft.security/assessments/governanceassignments\"\r\n| extend assignedResourceId = tostring(todynamic(properties).assignedResourceId)\r\n| extend remediationDueDate = todatetime(properties.remediationDueDate)\r\n| project id = assignedResourceId, governanceassignmentsProperties = todynamic(properties), remediationDueDate) on id\r\n| extend hasAssignment = isempty( governanceassignmentsProperties) == false and isnull( governanceassignmentsProperties) == false\r\n| where hasAssignment == true\r\n| extend owner = tostring(governanceassignmentsProperties.owner)\r\n| extend owner = iif(isnull(owner) == false and isempty(owner) == false, owner, \"Unspecified\")\r\n| extend assignmentStatus = iif(bin(remediationDueDate, 1d) < bin(now(), 1d), \"Overdue\", \"Ontime\")\r\n| summarize Ontime = countif(assignmentStatus == \"Ontime\"), Overdue = countif(assignmentStatus == \"Overdue\") by selectedOwner = owner\r\n| sort by Overdue desc", + "size": 0, + "title": "Status per owner", + "exportMultipleValues": true, + "exportedParameters": [ + { + "fieldName": "selectedOwner", + "parameterName": "selectedOwner", + "quote": "" + } + ], + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "Ontime", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "!=", + "thresholdValue": "0", + "representation": "info", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "Overdue", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "!=", + "thresholdValue": "0", + "representation": "3", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + } + } + ], + "labelSettings": [ + { + "columnId": "selectedOwner", + "label": "Owner" + } + ] + } + }, + "customWidth": "30", + "conditionalVisibilities": [ + { + "parameterName": "Owner", + "comparison": "isNotEqualTo" + }, + { + "parameterName": "RuleConditionSet", + "comparison": "isNotEqualTo" + } + ], + "name": "Owner status" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "securityresources \r\n| where type == \"microsoft.security/assessments\"\r\n| where isnull(properties.resourceDetails.AwsResourceId) and isnull(properties.resourceDetails.GcpResourceId)\r\n| extend DisplayName = tostring(properties.displayName)\r\n| where isempty(DisplayName) == false and isnull(DisplayName) == false\r\n| extend RuleConditionSet = '{RuleConditionSet}'\r\n| where RuleConditionSet contains name or RuleConditionSet contains properties.metadata.severity\r\n| join kind=leftouter (securityresources \r\n| where type == \"microsoft.security/assessments/governanceassignments\"\r\n| extend assignedResourceId = tostring(todynamic(properties).assignedResourceId)\r\n| extend remediationDueDate = todatetime(properties.remediationDueDate)\r\n| project id = assignedResourceId, governanceassignmentsProperties = todynamic(properties), remediationDueDate) on id\r\n| extend hasAssignment = isempty( governanceassignmentsProperties) == false and isnull( governanceassignmentsProperties) == false\r\n| extend assignmentStatus = iif(tostring(properties.status.code) == \"Unhealthy\",iif(hasAssignment == true, iif(bin(remediationDueDate, 1d) < bin(now(), 1d), \"Overdue\", \"Ontime\"), \"Unassigned\") , \"Completed\")\r\n| extend Status = assignmentStatus\r\n| summarize Completed = countif(Status == \"Completed\"), Ontime = countif(Status == \"Ontime\"), Overdue = countif(Status == \"Overdue\"),Unassigned = countif(Status == \"Unassigned\") by DisplayName = tostring(properties.displayName)\r\n| sort by Overdue desc", + "size": 0, + "title": "Recommendations", + "noDataMessage": "No Assessments found", + "exportedParameters": [ + { + "fieldName": "id", + "parameterName": "id", + "parameterType": 1 + }, + { + "fieldName": "DisplayName", + "parameterName": "DisplayName", + "parameterType": 1 + } + ], + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "DisplayName", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "75ch" + } + }, + { + "columnMatch": "Completed", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "!=", + "thresholdValue": "0", + "representation": "success", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Blank", + "text": "{0}{1}" + } + ], + "customColumnWidthSetting": "15ch" + } + }, + { + "columnMatch": "Ontime", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "!=", + "thresholdValue": "0", + "representation": "1", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Blank", + "text": "{0}{1}" + }, + { + "representation": "Unknown", + "text": "{0}{1}" + } + ], + "customColumnWidthSetting": "15ch" + } + }, + { + "columnMatch": "Overdue", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "!=", + "thresholdValue": "0", + "representation": "3", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Blank", + "text": "{0}{1}" + } + ], + "customColumnWidthSetting": "15ch" + } + }, + { + "columnMatch": "Unassigned", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "!=", + "thresholdValue": "0", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Blank", + "text": "0" + } + ], + "customColumnWidthSetting": "15ch" + } + }, + { + "columnMatch": "Status", + "formatter": 1 + }, + { + "columnMatch": "id", + "formatter": 1, + "formatOptions": { + "customColumnWidthSetting": "90ch" + } + }, + { + "columnMatch": "owner", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "40ch" + } + }, + { + "columnMatch": "DueDate", + "formatter": 6 + }, + { + "columnMatch": "Severity", + "formatter": 5 + }, + { + "columnMatch": "Resource", + "formatter": 13, + "formatOptions": { + "linkTarget": "OpenBlade", + "linkIsContextBlade": false, + "showIcon": true, + "bladeOpenContext": { + "bladeName": "GenericResourceHealthDetailsBlade", + "extensionName": "Microsoft_Azure_Security", + "bladeParameters": [ + { + "name": "resourceId", + "source": "cell", + "value": "%2Fsubscriptions%2F3b5bc982-20bc-4b59-b1ca-f8488bb86736%2FresourceGroups%2Fdemo%2Fproviders%2FMicrosoft.HybridCompute%2Fmachines%2FW2019" + } + ] + }, + "customColumnWidthSetting": "20ch" + } + }, + { + "columnMatch": "ResourceGroup", + "formatter": 14, + "formatOptions": { + "linkTarget": null, + "showIcon": true, + "customColumnWidthSetting": "25ch" + } + }, + { + "columnMatch": "Source", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "15ch" + } + }, + { + "columnMatch": "OperatingSystem", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "25ch" + } + }, + { + "columnMatch": "Category", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "20ch" + } + }, + { + "columnMatch": "Remediation", + "formatter": 5 + }, + { + "columnMatch": "RemediationSteps", + "formatter": 11, + "formatOptions": { + "linkColumn": "Remediation", + "linkTarget": "Url" + }, + "tooltipFormat": { + "tooltip": "Click to view remediation steps" + } + }, + { + "columnMatch": "Code", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "Healthy", + "representation": "success", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Unhealthy", + "representation": "3", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "resourceGroup", + "formatter": 14, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "rowLimit": 10000, + "filter": true, + "sortBy": [ + { + "itemKey": "$gen_thresholds_Ontime_2", + "sortOrder": 2 + } + ], + "labelSettings": [ + { + "columnId": "DisplayName", + "label": "Display name" + } + ] + }, + "sortBy": [ + { + "itemKey": "$gen_thresholds_Ontime_2", + "sortOrder": 2 + } + ] + }, + "customWidth": "50", + "conditionalVisibility": { + "parameterName": "Rule", + "comparison": "isNotEqualTo" + }, + "name": "Assessmetns" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 1, + "content": { + "json": "" + }, + "customWidth": "50", + "conditionalVisibility": { + "parameterName": "RuleConditionSet", + "comparison": "isNotEqualTo" + }, + "name": "empty text" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "securityresources \r\n| where type == \"microsoft.security/assessments\"\r\n| where isnull(properties.resourceDetails.AwsResourceId) and isnull(properties.resourceDetails.GcpResourceId)\r\n| extend displayNameFilter = tostring(\"{DisplayName}\")\r\n| extend selectedOwner = '{selectedOwner}'\r\n| where displayNameFilter == tostring(properties.displayName)\r\n| join kind=leftouter (securityresources \r\n| where type == \"microsoft.security/assessments/governanceassignments\"\r\n| extend assignedResourceId = tostring(todynamic(properties).assignedResourceId)\r\n| extend remediationDueDate = todatetime(properties.remediationDueDate)\r\n| project id = assignedResourceId, owner = properties.owner,governanceassignmentsProperties = todynamic(properties), remediationDueDate, isGrace = properties.isGracePeriod) on id\r\n| extend hasAssignment = isempty( governanceassignmentsProperties) == false and isnull( governanceassignmentsProperties) == false\r\n| extend assignmentStatus = iif(tostring(properties.status.code) == \"Unhealthy\",iif(hasAssignment == true, iif(bin(remediationDueDate, 1d) < bin(now(), 1d), \"Overdue\", \"Ontime\"), \"Unassigned\") , \"Completed\")\r\n| extend source = trim(' ', tolower(tostring(properties.resourceDetails.Source)))\r\n | extend resourceId = iff(source =~ \"azure\", properties.resourceDetails.Id, iff(source =~ \"aws\" and isnotempty(tostring(properties.resourceDetails.ConnectorId)), properties.resourceDetails.Id, iff(source =~ \"gcp\" and isnotempty(tostring(properties.resourceDetails.ConnectorId)), properties.resourceDetails.Id, iff(source =~ 'aws', properties.resourceDetails.AzureResourceId, iff(source =~ 'gcp', properties.resourceDetails.AzureResourceId, properties.resourceDetails.Id)))))\r\n| extend owner = tostring(governanceassignmentsProperties.owner)\r\n| extend owner = iif(isnull(owner) == false and isempty(owner) == false and hasAssignment == true , owner, iif(hasAssignment == false, owner, \"Unspecified\"))\r\n| where '{selectedOwner}' == '' or (selectedOwner contains owner and hasAssignment == true)\r\n| project [\"Resource\"] = resourceId, Subscription = subscriptionId ,Status = assignmentStatus, Owner = owner, [\"Due date\"] = remediationDueDate, [\"Grace period enabled\"] = isGrace\r\n| sort by Status desc", + "size": 0, + "title": "List of resources for: {DisplayName}", + "noDataMessage": "No Assessments found", + "exportFieldName": "id", + "exportParameterName": "id", + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "Resource id", + "formatter": 13, + "formatOptions": { + "linkTarget": null, + "showIcon": true, + "customColumnWidthSetting": "90ch" + } + }, + { + "columnMatch": "Subscription", + "formatter": 15, + "formatOptions": { + "linkTarget": "Resource", + "showIcon": true + } + }, + { + "columnMatch": "Status", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "Completed", + "representation": "success", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Unassigned", + "representation": "2", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Overdue", + "representation": "3", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Ontime", + "representation": "1", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": null, + "text": "{0}{1}" + } + ] + }, + "numberFormat": { + "unit": 0, + "options": { + "style": "decimal" + } + } + }, + { + "columnMatch": "Grace period enabled", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "true", + "representation": "success", + "text": "" + }, + { + "operator": "==", + "thresholdValue": "false", + "representation": "4", + "text": "" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "Blank", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "owner", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "40ch" + } + }, + { + "columnMatch": "DueDate", + "formatter": 6 + }, + { + "columnMatch": "id", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "90ch" + } + }, + { + "columnMatch": "DisplayName", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "100ch" + } + }, + { + "columnMatch": "Completed", + "formatter": 4, + "formatOptions": { + "palette": "green", + "customColumnWidthSetting": "15ch" + } + }, + { + "columnMatch": "Ontime", + "formatter": 4, + "formatOptions": { + "palette": "blue", + "customColumnWidthSetting": "15ch" + } + }, + { + "columnMatch": "Overdue", + "formatter": 4, + "formatOptions": { + "palette": "redBright", + "customColumnWidthSetting": "15ch" + } + }, + { + "columnMatch": "Unassigned", + "formatter": 4, + "formatOptions": { + "palette": "orange", + "customColumnWidthSetting": "15ch" + } + }, + { + "columnMatch": "Severity", + "formatter": 5 + }, + { + "columnMatch": "Resource", + "formatter": 13, + "formatOptions": { + "linkTarget": "OpenBlade", + "linkIsContextBlade": false, + "showIcon": true, + "bladeOpenContext": { + "bladeName": "GenericResourceHealthDetailsBlade", + "extensionName": "Microsoft_Azure_Security", + "bladeParameters": [ + { + "name": "resourceId", + "source": "cell", + "value": "%2Fsubscriptions%2F3b5bc982-20bc-4b59-b1ca-f8488bb86736%2FresourceGroups%2Fdemo%2Fproviders%2FMicrosoft.HybridCompute%2Fmachines%2FW2019" + } + ] + }, + "customColumnWidthSetting": "20ch" + } + }, + { + "columnMatch": "ResourceGroup", + "formatter": 14, + "formatOptions": { + "linkTarget": null, + "showIcon": true, + "customColumnWidthSetting": "25ch" + } + }, + { + "columnMatch": "Source", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "15ch" + } + }, + { + "columnMatch": "OperatingSystem", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "25ch" + } + }, + { + "columnMatch": "Category", + "formatter": 0, + "formatOptions": { + "customColumnWidthSetting": "20ch" + } + }, + { + "columnMatch": "Remediation", + "formatter": 5 + }, + { + "columnMatch": "RemediationSteps", + "formatter": 11, + "formatOptions": { + "linkColumn": "Remediation", + "linkTarget": "Url" + }, + "tooltipFormat": { + "tooltip": "Click to view remediation steps" + } + }, + { + "columnMatch": "Code", + "formatter": 18, + "formatOptions": { + "thresholdsOptions": "icons", + "thresholdsGrid": [ + { + "operator": "==", + "thresholdValue": "Healthy", + "representation": "success", + "text": "{0}{1}" + }, + { + "operator": "==", + "thresholdValue": "Unhealthy", + "representation": "3", + "text": "{0}{1}" + }, + { + "operator": "Default", + "thresholdValue": null, + "representation": "success", + "text": "{0}{1}" + } + ] + } + }, + { + "columnMatch": "resourceGroup", + "formatter": 14, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + } + ], + "rowLimit": 10000, + "filter": true + }, + "sortBy": [] + }, + "conditionalVisibility": { + "parameterName": "DisplayName", + "comparison": "isNotEqualTo" + }, + "name": "Assignments" + } + ] + }, + "name": "assessmentsWithExplaination" + }, + { + "type": 1, + "content": { + "json": "💡 Selected filter for **DisplayName:** {DisplayName}\r\n💡 Selected filter for **selectedOwner:** {selectedOwner}\r\n", + "style": "{selectedTab}" + }, + "conditionalVisibility": { + "parameterName": "parameter1", + "comparison": "isEqualTo", + "value": "1" + }, + "name": "ResourceFilter - Copy" + } + ] + }, + "name": "assessmentsGrid" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RC_Governance" + }, + "name": "RC_Governance" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 9, + "content": { + "version": "KqlParameterItem/1.0", + "crossComponentResources": [ + "{Subscription}" + ], + "parameters": [ + { + "id": "cc98cfec-0182-4887-854e-536e9f3857da", + "version": "KqlParameterItem/1.0", + "name": "Subscription", + "type": 6, + "isRequired": true, + "typeSettings": { + "additionalResourceOptions": [], + "includeAll": false, + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "value": null + }, + { + "id": "1c3411d9-e319-4d74-8e97-61e2f4c56a56", + "version": "KqlParameterItem/1.0", + "name": "Location", + "type": 2, + "isRequired": true, + "query": "resources\r\n| summarize by location\r\n| where location != \"global\"", + "crossComponentResources": [ + "{Subscription}" + ], + "typeSettings": { + "additionalResourceOptions": [], + "showDefault": false + }, + "timeContext": { + "durationMs": 86400000 + }, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "value": "westeurope" + } + ], + "style": "above", + "queryType": 1, + "resourceType": "microsoft.operationalinsights/workspaces" + }, + "name": "parameters - 0" + }, + { + "type": 1, + "content": { + "json": "## Azure subscription and service limits, quotas, and constraints
\r\nTo know more about Azure service limits & quotas, see [Azure subscription and service limits, quotas, and constraints](https://learn.microsoft.com/azure/azure-resource-manager/management/azure-subscription-service-limits?toc=%2Fazure%2Fnetworking%2Ftoc.json#networking-limits)." + }, + "name": "text - Limits" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "template", + "loadFromTemplateId": "community-Workbooks/Common/noSubscriptions", + "items": [] + }, + "conditionalVisibility": { + "parameterName": "Subscription", + "comparison": "isEqualTo", + "value": "" + }, + "name": "No Subscriptions group - RC_Quota" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"ARMEndpoint/1.0\",\"data\":null,\"headers\":[],\"method\":\"GET\",\"path\":\"/subscriptions/{Subscription:id}/providers/microsoft.compute/locations/{Location}/usages?api-version=2022-03-01\",\"urlParams\":[],\"batchDisabled\":false,\"transformers\":[{\"type\":\"jsonpath\",\"settings\":{\"tablePath\":\"$.value\",\"columns\":[{\"path\":\"currentValue\",\"columnid\":\"Used\",\"columnType\":\"long\"},{\"path\":\"limit\",\"columnid\":\"Limit\",\"columnType\":\"long\"},{\"path\":\"name.localizedValue\",\"columnid\":\"Resource\"}]}}]}", + "size": 0, + "title": "Compute resource limits", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 12, + "gridSettings": { + "filter": true, + "sortBy": [ + { + "itemKey": "Limit", + "sortOrder": 1 + } + ] + }, + "sortBy": [ + { + "itemKey": "Limit", + "sortOrder": 1 + } + ] + }, + "customWidth": "50", + "conditionalVisibilities": [ + { + "parameterName": "Subscription", + "comparison": "isNotEqualTo", + "value": "" + }, + { + "parameterName": "Location", + "comparison": "isNotEqualTo" + } + ], + "name": "query - ComputeLimits" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"ARMEndpoint/1.0\",\"data\":null,\"headers\":[],\"method\":\"GET\",\"path\":\"/subscriptions/{Subscription:id}/providers/microsoft.network/locations/{Location}/usages?api-version=2022-01-01\",\"urlParams\":[],\"batchDisabled\":false,\"transformers\":[{\"type\":\"jsonpath\",\"settings\":{\"tablePath\":\"$.value\",\"columns\":[{\"path\":\"currentValue\",\"columnid\":\"Used\",\"columnType\":\"long\"},{\"path\":\"limit\",\"columnid\":\"Limit\",\"columnType\":\"long\"},{\"path\":\"name.localizedValue\",\"columnid\":\"Resource\"}]}}]}", + "size": 0, + "title": "Network resource limits", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 12, + "gridSettings": { + "filter": true, + "sortBy": [ + { + "itemKey": "Limit", + "sortOrder": 1 + } + ] + }, + "sortBy": [ + { + "itemKey": "Limit", + "sortOrder": 1 + } + ] + }, + "customWidth": "50", + "conditionalVisibility": { + "parameterName": "Subscription", + "comparison": "isNotEqualTo" + }, + "name": "query - NetworkLimits" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "{\"version\":\"ARMEndpoint/1.0\",\"data\":null,\"headers\":[],\"method\":\"GET\",\"path\":\"/subscriptions/{Subscription:id}/providers/microsoft.storage/locations/{Location}/usages?api-version=2021-09-01\",\"urlParams\":[],\"batchDisabled\":false,\"transformers\":[{\"type\":\"jsonpath\",\"settings\":{\"tablePath\":\"$.value\",\"columns\":[{\"path\":\"currentValue\",\"columnid\":\"Used\",\"columnType\":\"long\"},{\"path\":\"limit\",\"columnid\":\"Limit\",\"columnType\":\"long\"},{\"path\":\"name.localizedValue\",\"columnid\":\"Resource\"}]}}]}", + "size": 4, + "title": "Storage account limits", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 12, + "gridSettings": { + "filter": true, + "sortBy": [ + { + "itemKey": "Limit", + "sortOrder": 1 + } + ] + }, + "sortBy": [ + { + "itemKey": "Limit", + "sortOrder": 1 + } + ] + }, + "customWidth": "100", + "conditionalVisibility": { + "parameterName": "Subscription", + "comparison": "isNotEqualTo" + }, + "name": "query - StorageLimits" + } + ] + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RC_Quota" + }, + "name": "Usage + limits" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "items": [ + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "title": "Automation", + "expandable": true, + "expanded": true, + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type has 'microsoft.automation'\r\n\tor type has 'microsoft.logic'\r\n\tor type has 'microsoft.web/customapis'\r\n| extend type = case(\r\n\ttype =~ 'microsoft.automation/automationaccounts', 'Automation Accounts',\r\n\ttype == 'microsoft.web/serverfarms', \"App Service Plans\",\r\n\tkind == 'functionapp', \"Azure Functions\", \r\n\tkind == \"api\", \"API Apps\", \r\n\ttype == 'microsoft.web/sites', \"App Services\",\r\n\ttype =~ 'microsoft.web/connections', 'LogicApp Connectors',\r\n\ttype =~ 'microsoft.web/customapis','LogicApp API Connectors',\r\n\ttype =~ 'microsoft.logic/workflows','LogicApps',\r\n\ttype =~ 'microsoft.automation/automationaccounts/runbooks', 'Automation Runbooks',\r\n type =~ 'microsoft.automation/automationaccounts/configurations', 'Automation Configurations',\r\nstrcat(\"Not Translated: \", type))\r\n| summarize count() by type\r\n| where type !has \"Not Translated\"", + "size": 3, + "title": "Count of all resource types", + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "tiles", + "tileSettings": { + "showBorder": false, + "titleContent": { + "columnMatch": "type", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + } + } + }, + "name": "Count of all resource types" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type has 'microsoft.automation'\r\n\t or type has 'microsoft.logic'\r\n\t or type has 'microsoft.web/customapis'\r\n| extend type = case(\r\n\ttype =~ 'microsoft.automation/automationaccounts', 'Automation Accounts',\r\n\ttype =~ 'microsoft.web/connections', 'LogicApp Connectors',\r\n\ttype =~ 'microsoft.web/customapis','LogicApp API Connectors',\r\n\ttype =~ 'microsoft.logic/workflows','LogicApps',\r\n\ttype =~ 'microsoft.automation/automationaccounts/runbooks', 'Automation Runbooks',\r\n\ttype =~ 'microsoft.automation/automationaccounts/configurations', 'Automation Configurations',\r\n\tstrcat(\"Not Translated: \", type))\r\n| extend RunbookType = tostring(properties.runbookType)\r\n| extend LogicAppTrigger = properties.definition.triggers\r\n| extend LogicAppTrigger = iif(type =~ 'LogicApps', case(\r\n\tLogicAppTrigger has 'manual', tostring(LogicAppTrigger.manual.type),\r\n\tLogicAppTrigger has 'Recurrence', tostring(LogicAppTrigger.Recurrence.type),\r\n\tstrcat(\"Unknown Trigger type\", LogicAppTrigger)), LogicAppTrigger)\r\n| extend State = case(\r\n\ttype =~ 'Automation Runbooks', properties.state, \r\n\ttype =~ 'LogicApps', properties.state,\r\n\ttype =~ 'Automation Accounts', properties.state,\r\n\ttype =~ 'Automation Configurations', properties.state,\r\n\t' ')\r\n| extend CreatedDate = case(\r\n\ttype =~ 'Automation Runbooks', properties.creationTime, \r\n\ttype =~ 'LogicApps', properties.createdTime,\r\n\ttype =~ 'Automation Accounts', properties.creationTime,\r\n\ttype =~ 'Automation Configurations', properties.creationTime,\r\n\t' ')\r\n| extend LastModified = case(\r\n\ttype =~ 'Automation Runbooks', properties.lastModifiedTime, \r\n\ttype =~ 'LogicApps', properties.changedTime,\r\n\ttype =~ 'Automation Accounts', properties.lastModifiedTime,\r\n\ttype =~ 'Automation Configurations', properties.lastModifiedTime,\r\n\t' ')\r\n| extend Details = pack_all()\r\n| project Resource=id, subscriptionId, type, resourceGroup, RunbookType, LogicAppTrigger, State, Details", + "size": 0, + "title": "Details", + "noDataMessage": "No resources found", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 13, + "formatOptions": { + "linkTarget": "Resource", + "showIcon": true + } + }, + { + "columnMatch": "Resource", + "formatter": 5 + }, + { + "columnMatch": "subscriptionId", + "formatter": 5 + }, + { + "columnMatch": "Details", + "formatter": 7, + "formatOptions": { + "linkTarget": "CellDetails", + "linkLabel": "🔍 View details", + "linkIsContextBlade": true + } + } + ], + "rowLimit": 1000, + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "subscriptionId" + ], + "expandTopLevel": true, + "finalBy": "Resource" + } + }, + "tileSettings": { + "titleContent": { + "columnMatch": "type", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + }, + "showBorder": true, + "sortCriteriaField": "type", + "sortOrderField": 1 + } + }, + "name": "query - PaaS - Automation Detailed" + } + ] + }, + "name": "Group - Automation", + "styleSettings": { + "showBorder": true + } + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "title": "App services", + "expandable": true, + "expanded": true, + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type has 'microsoft.web'\r\n\t or type =~ 'microsoft.apimanagement/service'\r\n\t or type =~ 'microsoft.network/frontdoors'\r\n\t or type =~ 'microsoft.network/applicationgateways'\r\n\t or type =~ 'microsoft.appconfiguration/configurationstores'\r\n| extend type = case(\r\n\ttype == 'microsoft.web/serverfarms', \"App Service Plans\",\r\n\tkind == 'functionapp', \"Azure Functions\", \r\n\tkind == \"api\", \"API Apps\", \r\n\ttype == 'microsoft.web/sites', \"App Services\",\r\n\ttype =~ 'microsoft.network/applicationgateways', 'App Gateways',\r\n\ttype =~ 'microsoft.network/frontdoors', 'Front Door',\r\n\ttype =~ 'microsoft.apimanagement/service', 'API Management',\r\n\ttype =~ 'microsoft.web/certificates', 'App Certificates',\r\n\ttype =~ 'microsoft.appconfiguration/configurationstores', 'App Config Stores',\r\n\tstrcat(\"Not Translated: \", type))\r\n| where type !has \"Not Translated\"\r\n| summarize count() by type", + "size": 3, + "title": "Count of all resource types", + "noDataMessage": "No resources found", + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "tiles", + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 13, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Resource", + "formatter": 5 + }, + { + "columnMatch": "subscriptionId", + "formatter": 5 + } + ], + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "subscriptionId" + ], + "expandTopLevel": true, + "finalBy": "Resource" + } + }, + "tileSettings": { + "titleContent": { + "columnMatch": "type", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + }, + "showBorder": false, + "sortCriteriaField": "type", + "sortOrderField": 1 + } + }, + "name": "query - PaaS - Apps Overview" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources\r\n| where type has 'microsoft.web'\r\n\t or type =~ 'microsoft.apimanagement/service'\r\n\t or type =~ 'microsoft.network/frontdoors'\r\n\t or type =~ 'microsoft.network/applicationgateways'\r\n\t or type =~ 'microsoft.appconfiguration/configurationstores'\r\n| extend type = case(\r\n\ttype == 'microsoft.web/serverfarms', \"App Service Plans\",\r\n\tkind == 'functionapp', \"Azure Functions\", \r\n\tkind == \"api\", \"API Apps\", \r\n\ttype == 'microsoft.web/sites', \"App Services\",\r\n\ttype =~ 'microsoft.network/applicationgateways', 'App Gateways',\r\n\ttype =~ 'microsoft.network/frontdoors', 'Front Door',\r\n\ttype =~ 'microsoft.apimanagement/service', 'API Management',\r\n\ttype =~ 'microsoft.web/certificates', 'App Certificates',\r\n\ttype =~ 'microsoft.appconfiguration/configurationstores', 'App Config Stores',\r\n\tstrcat(\"Not Translated: \", type))\r\n| where type !has \"Not Translated\"\r\n| extend Sku = case(\r\n\ttype =~ 'App Gateways', properties.sku.name, \r\n\ttype =~ 'Azure Functions', properties.sku,\r\n\ttype =~ 'API Management', sku.name,\r\n\ttype =~ 'App Service Plans', sku.name,\r\n\ttype =~ 'App Services', properties.sku,\r\n\ttype =~ 'App Config Stores', sku.name,\r\n\t' ')\r\n| extend State = case(\r\n\ttype =~ 'App Config Stores', properties.provisioningState,\r\n\ttype =~ 'App Service Plans', properties.status,\r\n\ttype =~ 'Azure Functions', properties.state,\r\n\ttype =~ 'App Services', properties.state,\r\n\ttype =~ 'API Management', properties.provisioningState,\r\n\ttype =~ 'App Gateways', properties.provisioningState,\r\n\ttype =~ 'Front Door', properties.provisioningState,\r\n\t' ')\r\n| mv-expand publicIpId = properties.frontendIPConfigurations\r\n| mv-expand publicIpId = publicIpId.properties.publicIPAddress.id\r\n| extend publicIpId = tostring(publicIpId)\r\n\t| join kind=leftouter(\r\n\t \tResources\r\n \t\t| where type =~ 'microsoft.network/publicipaddresses'\r\n \t\t| project publicIpId = id, publicIpAddress = tostring(properties.ipAddress)) on publicIpId\r\n| extend PublicIP = case(\r\n\ttype =~ 'API Management', properties.publicIPAddresses,\r\n\ttype =~ 'App Gateways', publicIpAddress,\r\n type =~ 'App Services', properties.inboundIpAddress,\r\n type =~ 'Azure Functions', properties.inboundIpAddress,\r\n\t' ')\r\n| extend Instances = case(\r\n\ttype =~ 'API Management', sku.capacity,\r\n type =~ 'App Services', properties.siteConfig.numberOfWorkers,\r\n type =~ 'Azure Functions', properties.siteConfig.numberOfWorkers,\r\n type =~ 'App Service Plans', properties.currentNumberOfWorkers,\r\n\t' ')\r\n| extend ServicePlan = case(\r\n type =~ 'App Services', properties.serverFarmId,\r\n type =~ 'Azure Functions', properties.serverFarmId,\r\n\t' ')\r\n| extend Details = pack_all()\r\n| project Resource=id, type, subscriptionId, Sku, State, PublicIP, Instances, ServicePlan, Details", + "size": 0, + "noDataMessage": "No resources found", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 13, + "formatOptions": { + "linkTarget": "Resource", + "showIcon": true + } + }, + { + "columnMatch": "Resource", + "formatter": 5 + }, + { + "columnMatch": "subscriptionId", + "formatter": 5 + }, + { + "columnMatch": "Details", + "formatter": 7, + "formatOptions": { + "linkTarget": "CellDetails", + "linkLabel": "🔍 View details", + "linkIsContextBlade": true + } + } + ], + "rowLimit": 1000, + "filter": true, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "subscriptionId" + ], + "expandTopLevel": true, + "finalBy": "Resource" + } + }, + "tileSettings": { + "titleContent": { + "columnMatch": "type", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + }, + "showBorder": true, + "sortCriteriaField": "type", + "sortOrderField": 1 + } + }, + "name": "query - PaaS - Apps Detailed" + } + ] + }, + "name": "Group - App Services", + "styleSettings": { + "showBorder": true + } + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "editable", + "title": "Data", + "expandable": true, + "expanded": true, + "items": [ + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "resources \r\n| where type =~ 'microsoft.documentdb/databaseaccounts'\r\n\tor type =~ 'microsoft.sql/servers/databases'\r\n\tor type =~ 'microsoft.dbformysql/servers'\r\n\tor type =~ 'microsoft.sql/servers'\r\n or type =~ 'Microsoft.DBforPostgreSQL/servers'\r\n or type =~ 'Microsoft.DBforMariaDB/servers'\r\n or type =~ 'microsoft.dbforpostgresql/flexibleservers'\r\n| extend type = case(\r\n\ttype =~ 'microsoft.documentdb/databaseaccounts', 'CosmosDB',\r\n\ttype =~ 'microsoft.sql/servers/databases', 'SQL DBs',\r\n\ttype =~ 'microsoft.dbformysql/servers', 'MySQL Servers',\r\n\ttype =~ 'microsoft.sql/servers', 'SQL Servers',\r\n type =~ 'Microsoft.DBforPostgreSQL/servers', 'PostgreSQL Servers',\r\n type =~ 'microsoft.dbforpostgresql/flexibleservers', 'PostgreSQL Flexi Servers',\r\n type =~ 'Microsoft.DBforMariaDB/servers', 'MariaDB Servers',\r\n\tstrcat(\"Not Translated: \", type))\r\n| where type !has \"Not Translated\"\r\n| summarize count() by type", + "size": 3, + "title": "Count of all resource types", + "noDataMessage": "No resources found", + "showRefreshButton": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "visualization": "tiles", + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 13, + "formatOptions": { + "linkTarget": null, + "showIcon": true + } + }, + { + "columnMatch": "Resource", + "formatter": 5 + }, + { + "columnMatch": "subscriptionId", + "formatter": 5 + } + ], + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "subscriptionId" + ], + "expandTopLevel": true, + "finalBy": "Resource" + } + }, + "tileSettings": { + "titleContent": { + "columnMatch": "type", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + }, + "showBorder": true, + "sortCriteriaField": "type", + "sortOrderField": 1 + } + }, + "name": "query - PaaS - Data Overview" + }, + { + "type": 3, + "content": { + "version": "KqlItem/1.0", + "query": "// data\r\n// Click the \"Run query\" command above to execute the query and see results.\r\nresources \r\n| where type =~ 'microsoft.documentdb/databaseaccounts'\r\n\tor type =~ 'microsoft.sql/servers/databases'\r\n\tor type =~ 'microsoft.dbformysql/servers'\r\n\tor type =~ 'microsoft.sql/servers'\r\n or type =~ 'Microsoft.DBforPostgreSQL/servers'\r\n or type =~ 'Microsoft.DBforMariaDB/servers'\r\n or type =~ 'microsoft.dbforpostgresql/flexibleservers'\r\n| extend type = case(\r\n\ttype =~ 'microsoft.documentdb/databaseaccounts', 'CosmosDB',\r\n\ttype =~ 'microsoft.sql/servers/databases', 'SQL DBs',\r\n\ttype =~ 'microsoft.dbformysql/servers', 'MySQL Servers',\r\n\ttype =~ 'microsoft.sql/servers', 'SQL Servers',\r\n type =~ 'Microsoft.DBforPostgreSQL/servers', 'PostgreSQL Servers',\r\n type =~ 'microsoft.dbforpostgresql/flexibleservers', 'PostgreSQL Flexi Servers',\r\n type =~ 'Microsoft.DBforMariaDB/servers', 'MariaDB Servers',\r\n\tstrcat(\"Not Translated: \", type))\r\n| extend Sku = case(\r\n\ttype =~ 'CosmosDB', properties.databaseAccountOfferType,\r\n\ttype =~ 'SQL DBs', sku.name,\r\n\ttype =~ 'MySQL Servers', sku.name,\r\n type =~ 'PostgreSQL Servers', sku.name,\r\n type =~ 'PostgreSQL Flexi Servers', sku.name,\r\n type =~ 'MariaDB Servers', sku.name,\r\n\t' ')\r\n| extend Status = case(\r\n\ttype =~ 'CosmosDB', properties.provisioningState,\r\n\ttype =~ 'SQL DBs', properties.status,\r\n type =~ 'SQL Servers', properties.state,\r\n\ttype =~ 'MySQL Servers', properties.userVisibleState,\r\n type =~ 'PostgreSQL Servers', properties.state,\r\n type =~ 'PostgreSQL Flexi Servers', properties.state,\r\n type =~ 'MariaDB Servers', properties.userVisibleState,\r\n\t' ')\r\n| extend Endpoint = case(\r\n\ttype =~ 'MySQL Servers', properties.fullyQualifiedDomainName,\r\n\ttype =~ 'SQL Servers', properties.fullyQualifiedDomainName,\r\n\ttype =~ 'CosmosDB', properties.documentEndpoint,\r\n type =~ 'PostgreSQL Servers', properties.fullyQualifiedDomainName,\r\n type =~ 'PostgreSQL Flexi Servers', properties.fullyQualifiedDomainName,\r\n type =~ 'MariaDB Servers', properties.fullyQualifiedDomainName,\r\n\t' ')\r\n| extend PublicNetworkAccess = case(\r\n\ttype =~ 'MySQL Servers', properties.publicNetworkAccess,\r\n\ttype =~ 'SQL Servers', properties.publicNetworkAccess,\r\n type =~ 'PostgreSQL Servers', properties.publicNetworkAccess,\r\n type =~ 'PostgreSQL Flexi Servers', properties.publicNetworkAccess,\r\n type =~ 'MariaDB Servers', properties.publicNetworkAccess,\r\n\t' ')\r\n| extend Version = case(\r\n\ttype =~ 'MySQL Servers', properties.version,\r\n\ttype =~ 'SQL Servers', properties.version,\r\n type =~ 'PostgreSQL Servers', properties.version,\r\n type =~ 'PostgreSQL Flexi Servers', properties.version,\r\n type =~ 'MariaDB Servers', properties.version,\r\n\t' ')\r\n| extend maxSizeGB = todouble(case(\r\n\ttype =~ 'SQL DBs', properties.maxSizeBytes,\r\n\ttype =~ 'MySQL Servers', properties.storageProfile.storageMB,\r\n type =~ 'PostgreSQL Servers', properties.storageProfile.storageMB,\r\n type =~ 'PostgreSQL Flexi Servers', properties.storageProfile.storageMB,\r\n type =~ 'MariaDB Servers', properties.storageProfile.storageMB,\r\n\t' '))\r\n| extend maxSizeGB = iif(type has 'SQL DBs', maxSizeGB /1000 /1000, maxSizeGB)\r\n| extend Details = pack_all()\r\n| project Resource=id, resourceGroup, subscriptionId, type, Sku, Status, Endpoint, Version, PublicNetworkAccess, maxSizeGB, Details\r\n\r\n", + "size": 0, + "title": "Details", + "noDataMessage": "No resources found", + "showRefreshButton": true, + "showExportToExcel": true, + "queryType": 1, + "resourceType": "microsoft.resourcegraph/resources", + "crossComponentResources": [ + "{Subscription}" + ], + "gridSettings": { + "formatters": [ + { + "columnMatch": "$gen_group", + "formatter": 13, + "formatOptions": { + "linkTarget": "Resource", + "showIcon": true + } + }, + { + "columnMatch": "Resource", + "formatter": 5 + }, + { + "columnMatch": "subscriptionId", + "formatter": 5 + }, + { + "columnMatch": "maxSizeGB", + "formatter": 0, + "numberFormat": { + "unit": 4, + "options": { + "style": "decimal", + "useGrouping": false + } + } + }, + { + "columnMatch": "Details", + "formatter": 7, + "formatOptions": { + "linkTarget": "CellDetails", + "linkLabel": "🔍 View details", + "linkIsContextBlade": true + } + } + ], + "rowLimit": 1000, + "hierarchySettings": { + "treeType": 1, + "groupBy": [ + "subscriptionId" + ], + "expandTopLevel": true, + "finalBy": "Resource" + } + }, + "tileSettings": { + "titleContent": { + "columnMatch": "type", + "formatter": 1 + }, + "leftContent": { + "columnMatch": "count_", + "formatter": 12, + "formatOptions": { + "palette": "auto" + }, + "numberFormat": { + "unit": 17, + "options": { + "maximumSignificantDigits": 3, + "maximumFractionDigits": 2 + } + } + }, + "showBorder": true, + "sortCriteriaField": "type", + "sortOrderField": 1 + } + }, + "name": "query - PaaS - Data Detailed" + } + ] + }, + "name": "Data", + "styleSettings": { + "showBorder": true + } + } + ] + }, + "conditionalVisibilities": [ + { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RC_PaaS" + }, + { + "parameterName": "Subscription", + "comparison": "isNotEqualTo" + } + ], + "name": "RC_PaaS" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "template", + "loadFromTemplateId": "community-Workbooks/Azure Advisor/AzureServiceRetirement", + "items": [] + }, + "conditionalVisibility": { + "parameterName": "SelectedTab", + "comparison": "isEqualTo", + "value": "RC_ServicesRetirement" + }, + "name": "group - Service retirement" + }, + { + "type": 12, + "content": { + "version": "NotebookGroup/1.0", + "groupType": "template", + "loadFromTemplateId": "community-Workbooks/Common/noSubscriptions", + "items": [] + }, + "conditionalVisibilities": [ + { + "parameterName": "Subscription", + "comparison": "isEqualTo", + "value": "" + }, + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "RC_Quota" + }, + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "RC_Age" + }, + { + "parameterName": "SelectedTab", + "comparison": "isNotEqualTo", + "value": "RC_ServicesRetirement" + } + ], + "name": "No Subscriptions group" + } + ] + }, + "name": "Azure Governance Workbook" + } + ], + "fallbackResourceIds": [ + "azure monitor" + ], + "$schema": "https://github.com/Microsoft/Application-Insights-Workbooks/blob/master/schema/workbook.json" + }, + "version": "", + "workbookJson": "[string(variables('$fxv#0'))]", + "workbookId": "907", + "telemetryId": "[format('00f120b5-2007-6120-0000-{0}30126b006', variables('workbookId'))]", + "finOpsToolkitVersion": "13.0", + "resourceTags": "[if(contains(parameters('tags'), 'ftk-tool'), parameters('tags'), union(parameters('tags'), createObject('ftk-version', variables('finOpsToolkitVersion'), 'ftk-tool', format('{0} workbook', parameters('displayName')))))]" + }, + "resources": [ + { + "condition": "[parameters('enableDefaultTelemetry')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2022-09-01", + "name": "[format('pid-{0}-{1}', variables('telemetryId'), uniqueString(deployment().name, parameters('location')))]", + "properties": { + "mode": "Incremental", + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "FinOps toolkit", + "version": "[variables('finOpsToolkitVersion')]" + } + }, + "resources": [] + } + } + }, + { + "type": "Microsoft.Insights/workbooks", + "apiVersion": "2022-04-01", + "name": "[guid(resourceGroup().id, 'Microsoft.Insights/workbooks', parameters('displayName'))]", + "location": "[parameters('location')]", + "tags": "[variables('resourceTags')]", + "kind": "shared", + "properties": { + "category": "workbook", + "description": "[parameters('description')]", + "displayName": "[parameters('displayName')]", + "serializedData": "[variables('workbookJson')]", + "sourceId": "Azure Monitor", + "version": "[variables('version')]" + } + } + ], + "outputs": { + "workbookId": { + "type": "string", + "metadata": { + "description": "The resource ID of the workbook." + }, + "value": "[resourceId('Microsoft.Insights/workbooks', guid(resourceGroup().id, 'Microsoft.Insights/workbooks', parameters('displayName')))]" + }, + "workbookUrl": { + "type": "string", + "metadata": { + "description": "Link to the workbook in the Azure portal." + }, + "value": "[format('{0}/#view/AppInsightsExtension/UsageNotebookBlade/ComponentId/Azure%20Monitor/ConfigurationId/{1}/Type/{2}/WorkbookTemplateName/{3}', environment().portal, uriComponent(resourceId('Microsoft.Insights/workbooks', guid(resourceGroup().id, 'Microsoft.Insights/workbooks', parameters('displayName')))), reference(resourceId('Microsoft.Insights/workbooks', guid(resourceGroup().id, 'Microsoft.Insights/workbooks', parameters('displayName'))), '2022-04-01').category, uriComponent(reference(resourceId('Microsoft.Insights/workbooks', guid(resourceGroup().id, 'Microsoft.Insights/workbooks', parameters('displayName'))), '2022-04-01').displayName))]" + } + } + } + } + } + ], + "outputs": { + "optimizationId": { + "type": "string", + "metadata": { + "description": "Optimization workbook resource ID." + }, + "value": "[if(parameters('includeOptimization'), reference(resourceId('Microsoft.Resources/deployments', format('{0}-Optimization', parameters('displayNamePrefix'))), '2025-04-01').outputs.workbookId.value, '')]" + }, + "optimizationUrl": { + "type": "string", + "metadata": { + "description": "Optimization workbook Azure portal link." + }, + "value": "[if(parameters('includeOptimization'), reference(resourceId('Microsoft.Resources/deployments', format('{0}-Optimization', parameters('displayNamePrefix'))), '2025-04-01').outputs.workbookUrl.value, '')]" + }, + "governanceId": { + "type": "string", + "metadata": { + "description": "Governance workbook resource ID." + }, + "value": "[if(parameters('includeGovernance'), reference(resourceId('Microsoft.Resources/deployments', format('{0}-Governance', parameters('displayNamePrefix'))), '2025-04-01').outputs.workbookId.value, '')]" + }, + "governanceUrl": { + "type": "string", + "metadata": { + "description": "Governance workbook Azure portal link." + }, + "value": "[if(parameters('includeGovernance'), reference(resourceId('Microsoft.Resources/deployments', format('{0}-Governance', parameters('displayNamePrefix'))), '2025-04-01').outputs.workbookUrl.value, '')]" + } + } +} \ No newline at end of file diff --git a/docs/deploy/finops-workbooks-13.0.ui.json b/docs/deploy/finops-workbooks-13.0.ui.json new file mode 100644 index 000000000..adf646b7a --- /dev/null +++ b/docs/deploy/finops-workbooks-13.0.ui.json @@ -0,0 +1,65 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/0.1.2-preview/CreateUIDefinition.MultiVm.json#", + "handler": "Microsoft.Azure.CreateUIDef", + "version": "0.1.2-preview", + "parameters": { + "config": { + "basics": { + "description": "FinOps workbooks facilitate FinOps tasks that help you build, manage, and optimize cloud resources. [Learn more](https://aka.ms/finops/workbooks)", + "location": { + "label": "Location", + "resourceTypes": ["Microsoft.Insights/workbooks"] + } + } + }, + "resourceTypes": ["Microsoft.Insights/workbooks"], + "basics": [ + { + "name": "displayNamePrefix", + "type": "Microsoft.Common.TextBox", + "label": "Name prefix", + "defaultValue": "FinOps", + "toolTip": "Prefix to use for each of the workbook names (for example, 'FinOps-Optimization', 'FinOps-Governance').", + "constraints": { + "required": true, + "regex": "^.{1,220}$", + "validationMessage": "Name prefix cannot be longer than 220 characters." + }, + "visible": true + } + ], + "steps": [ + { + "name": "workbooks", + "label": "Workbooks", + "elements": [ + { + "name": "workbookSelectionIntro", + "options": { "text": "Select the FinOps workbooks you would like to include in this deployment. You can select more later." }, + "type": "Microsoft.Common.TextBlock" + }, + { + "name": "includeOptimization", + "label": "Optimization", + "toolTip": "Select to include the optimization workbook for workload and rate optimization as well as sustainability, security, and more.", + "type": "Microsoft.Common.CheckBox", + "defaultValue": true + }, + { + "name": "includeGovernance", + "label": "Governance", + "toolTip": "Select to include the governance workbook for queries to support the Well-Architected Framework governance pillar.", + "type": "Microsoft.Common.CheckBox", + "defaultValue": true + } + ] + } + ], + "outputs": { + "displayNamePrefix": "[basics('displayNamePrefix')]", + "includeOptimization": "[steps('workbooks').includeOptimization]", + "includeGovernance": "[steps('workbooks').includeGovernance]", + "location": "[location()]" + } + } +} diff --git a/docs/deploy/finops-workbooks-latest.json b/docs/deploy/finops-workbooks-latest.json index 8bda7d068..cad19f780 100644 --- a/docs/deploy/finops-workbooks-latest.json +++ b/docs/deploy/finops-workbooks-latest.json @@ -4,8 +4,8 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "13074221599166400425" + "version": "0.40.2.10011", + "templateHash": "14432727458044791742" } }, "parameters": { @@ -61,7 +61,7 @@ }, "variables": { "telemetryId": "00f120b5-2007-6120-0000-a7730126b006", - "finOpsToolkitVersion": "12.0", + "finOpsToolkitVersion": "13.0", "resourceTags": "[union(parameters('tags'), coalesce(tryGet(parameters('tagsByResource'), 'Microsoft.Insights/workbooks'), createObject()), createObject('ftk-version', variables('finOpsToolkitVersion'), 'ftk-tool', 'FinOps workbooks'))]" }, "resources": [ @@ -88,7 +88,7 @@ { "condition": "[parameters('includeOptimization')]", "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", + "apiVersion": "2025-04-01", "name": "[format('{0}-Optimization', parameters('displayNamePrefix'))]", "properties": { "expressionEvaluationOptions": { @@ -115,8 +115,8 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "9310226075524657390" + "version": "0.40.2.10011", + "templateHash": "16879565610975211575" } }, "parameters": { @@ -9493,7 +9493,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and tostring(properties.['licenseType']) == 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=tostring(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/databases', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n )\r\n) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, CheckSQLDBAHB,SQLLocation, LicenseType, StorageAccountType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and tostring(properties.['licenseType']) == 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/databases', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n )\r\n) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, CheckSQLDBAHB,SQLLocation, LicenseType, StorageAccountType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n", "size": 0, "title": "AHB Disabled", "queryType": 1, @@ -9513,7 +9513,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and tostring(properties.['licenseType']) != 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=sku.name, SKUTier=sku.tier, SQLLocation = location, vCores=tostring(sku.capacity), LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, SQLLocation, LicenseType, StorageAccountType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n\r\n", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources | where type =~ 'Microsoft.Sql/servers/databases' and name != 'master' and tostring(properties.['licenseType']) != 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\" and tostring(sku.name) != \"ElasticPool\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=sku.name, SKUTier=sku.tier, SQLLocation = location, vCores=toint(sku.capacity), LicenseType = tostring(properties.['licenseType']), StorageAccountType=tostring(properties.['storageAccountType'])) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, SQLLocation, LicenseType, StorageAccountType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n\r\n", "size": 0, "title": "AHB Enabled", "queryType": 1, @@ -10140,7 +10140,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and tostring(properties.['licenseType']) == 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=tostring(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n )\r\n) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, CheckSQLDBAHB,SQLLocation, LicenseType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and tostring(properties.['licenseType']) == 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n )\r\n) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, CheckSQLDBAHB,SQLLocation, LicenseType, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n", "size": 0, "title": "AHB Disabled", "queryType": 1, @@ -10160,7 +10160,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and tostring(properties.['licenseType']) != 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=tostring(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, SQLLocation, LicenseType, CheckSQLDBAHB, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n\r\n", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n| extend SubscriptionName=name | join (resources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and tostring(properties.['licenseType']) != 'LicenseIncluded' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n| extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project SQLDBID,SQLName,SQLRG, SKUName, SKUTier, vCores, SQLLocation, LicenseType, CheckSQLDBAHB, SubscriptionName\r\n| join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n\r\n", "size": 0, "title": "AHB Enabled", "queryType": 1, @@ -10204,7 +10204,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=tostring(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by SubscriptionName, CheckSQLDBAHB", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by SubscriptionName, CheckSQLDBAHB", "size": 0, "title": "Summary of SQL Databases with or without AHB per Subscription", "showRefreshButton": true, @@ -10290,7 +10290,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=tostring(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by CheckSQLDBAHB", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" | extend SubscriptionName=name \r\n| join (\r\nresources\r\n| where type =~ 'Microsoft.Sql/servers/elasticPools' and kind contains 'vcore' and kind !contains \"serverless\"\r\n| extend SQLDBID=id,SQLName = name, SQLRG = resourceGroup, SKUName=tostring(sku.name), SKUTier=tostring(sku.tier), vCores=toint(sku.capacity), SQLLocation = location, LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), SQLDBID=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct SQLDBID\r\n )\r\n on SQLDBID\r\n | extend CheckSQLDBAHB = case(\r\n type =~ 'Microsoft.Sql/servers/elasticPools', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n \"Not SQL DB\"\r\n )\r\n ) on subscriptionId \r\n| summarize count() by CheckSQLDBAHB", "size": 0, "title": "Summary of SQL Databases with or without AHB", "showRefreshButton": true, @@ -10783,7 +10783,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' and tostring(properties.['licenseType']) == 'LicenseIncluded'\r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=tostring(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n | project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n | where SQLRG in ({ResourceGroup})\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' and tostring(properties.['licenseType']) == 'LicenseIncluded'\r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n | project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n | where SQLRG in ({ResourceGroup})\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance", "size": 0, "queryType": 1, "resourceType": "microsoft.resourcegraph/resources", @@ -10802,7 +10802,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' and tostring(properties.['licenseType']) != 'LicenseIncluded'\r\n | extend ManagedInstance=id, SQLName=name, SQLRG=resourceGroup, SQLLocation=location, vCores=tostring(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n | project ManagedInstance, SQLName, SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n | where SQLRG in ({ResourceGroup})\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n ", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' and tostring(properties.['licenseType']) != 'LicenseIncluded'\r\n | extend ManagedInstance=id, SQLName=name, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n | project ManagedInstance, SQLName, SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n | where SQLRG in ({ResourceGroup})\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n ", "size": 0, "queryType": 1, "resourceType": "microsoft.resourcegraph/resources", @@ -10852,7 +10852,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' \r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=tostring(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n| summarize count() by SubscriptionName, CheckSQLMIAHB", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\"\r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances' \r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n| summarize count() by SubscriptionName, CheckSQLMIAHB", "size": 0, "title": "Summary of SQL MI with or without AHB per Subscription", "showRefreshButton": true, @@ -10901,7 +10901,7 @@ "type": 3, "content": { "version": "KqlItem/1.0", - "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" \r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances'\r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=tostring(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n| summarize count() by SubscriptionName, CheckSQLMIAHB", + "query": "ResourceContainers | where type =~ 'Microsoft.Resources/subscriptions' | where tostring (properties.subscriptionPolicies.quotaId) !has \"MSDNDevTest_2014-09-01\" \r\n | extend SubscriptionName=name \r\n | join (resources | where type =~ 'Microsoft.Sql/managedInstances'\r\n | extend ManagedInstance=id, SQLRG=resourceGroup, SQLLocation=location, vCores=toint(sku.capacity),LicenseType = tostring(properties.['licenseType'])\r\n | join kind = innerunique(\r\n resources\r\n | extend replaced_tags = replace('{}', 'null', tostring(tags))\r\n | extend replaced_tags = parse_json(replaced_tags)\r\n | mv-expand replaced_tags\r\n | extend tagName = tostring(bag_keys(replaced_tags)[0])\r\n | extend tagValue = tostring(replaced_tags['{TagName}']), ManagedInstance=id\r\n | where tagName has '{TagName}' and tagValue has '{TagValue}'\r\n | distinct ManagedInstance\r\n )\r\n on ManagedInstance\r\n | extend CheckSQLMIAHB = case(\r\n type =~ 'Microsoft.Sql/managedInstances', iif((properties.['licenseType'])\r\n has 'LicenseIncluded', \"AHB Not Enabled\", \"AHB Enabled\"),\r\n\"Not SQL DB\"\r\n ) \r\n ) on subscriptionId \r\n| project ManagedInstance,SQLRG, SQLLocation, CheckSQLMIAHB, vCores, LicenseType, SubscriptionName\r\n| summarize count() by SubscriptionName, CheckSQLMIAHB", "size": 0, "title": "Summary of SQL Managed Instance with or without AHB", "showRefreshButton": true, @@ -12575,7 +12575,7 @@ "workbookJson": "[string(variables('$fxv#0'))]", "workbookId": "0b2", "telemetryId": "[format('00f120b5-2007-6120-0000-{0}30126b006', variables('workbookId'))]", - "finOpsToolkitVersion": "12.0", + "finOpsToolkitVersion": "13.0", "resourceTags": "[if(contains(parameters('tags'), 'ftk-tool'), parameters('tags'), union(parameters('tags'), createObject('ftk-version', variables('finOpsToolkitVersion'), 'ftk-tool', format('{0} workbook', parameters('displayName')))))]" }, "resources": [ @@ -12638,7 +12638,7 @@ { "condition": "[parameters('includeGovernance')]", "type": "Microsoft.Resources/deployments", - "apiVersion": "2022-09-01", + "apiVersion": "2025-04-01", "name": "[format('{0}-Governance', parameters('displayNamePrefix'))]", "properties": { "expressionEvaluationOptions": { @@ -12665,8 +12665,8 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.36.177.2456", - "templateHash": "10645867907621769482" + "version": "0.40.2.10011", + "templateHash": "10362789456967541266" } }, "parameters": { @@ -20839,7 +20839,7 @@ "workbookJson": "[string(variables('$fxv#0'))]", "workbookId": "907", "telemetryId": "[format('00f120b5-2007-6120-0000-{0}30126b006', variables('workbookId'))]", - "finOpsToolkitVersion": "12.0", + "finOpsToolkitVersion": "13.0", "resourceTags": "[if(contains(parameters('tags'), 'ftk-tool'), parameters('tags'), union(parameters('tags'), createObject('ftk-version', variables('finOpsToolkitVersion'), 'ftk-tool', format('{0} workbook', parameters('displayName')))))]" }, "resources": [ @@ -20906,28 +20906,28 @@ "metadata": { "description": "Optimization workbook resource ID." }, - "value": "[reference(resourceId('Microsoft.Resources/deployments', format('{0}-Optimization', parameters('displayNamePrefix'))), '2022-09-01').outputs.workbookId.value]" + "value": "[if(parameters('includeOptimization'), reference(resourceId('Microsoft.Resources/deployments', format('{0}-Optimization', parameters('displayNamePrefix'))), '2025-04-01').outputs.workbookId.value, '')]" }, "optimizationUrl": { "type": "string", "metadata": { "description": "Optimization workbook Azure portal link." }, - "value": "[reference(resourceId('Microsoft.Resources/deployments', format('{0}-Optimization', parameters('displayNamePrefix'))), '2022-09-01').outputs.workbookUrl.value]" + "value": "[if(parameters('includeOptimization'), reference(resourceId('Microsoft.Resources/deployments', format('{0}-Optimization', parameters('displayNamePrefix'))), '2025-04-01').outputs.workbookUrl.value, '')]" }, "governanceId": { "type": "string", "metadata": { "description": "Governance workbook resource ID." }, - "value": "[reference(resourceId('Microsoft.Resources/deployments', format('{0}-Governance', parameters('displayNamePrefix'))), '2022-09-01').outputs.workbookId.value]" + "value": "[if(parameters('includeGovernance'), reference(resourceId('Microsoft.Resources/deployments', format('{0}-Governance', parameters('displayNamePrefix'))), '2025-04-01').outputs.workbookId.value, '')]" }, "governanceUrl": { "type": "string", "metadata": { "description": "Governance workbook Azure portal link." }, - "value": "[reference(resourceId('Microsoft.Resources/deployments', format('{0}-Governance', parameters('displayNamePrefix'))), '2022-09-01').outputs.workbookUrl.value]" + "value": "[if(parameters('includeGovernance'), reference(resourceId('Microsoft.Resources/deployments', format('{0}-Governance', parameters('displayNamePrefix'))), '2025-04-01').outputs.workbookUrl.value, '')]" } } } \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/azuredeploy-nested.bicep b/docs/deploy/optimization-engine/13.0/azuredeploy-nested.bicep new file mode 100644 index 000000000..0e12a764d --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/azuredeploy-nested.bicep @@ -0,0 +1,2158 @@ +param projectLocation string +param templateLocation string + +param storageAccountName string +param automationAccountName string +param sqlServerName string +param sqlServerAlreadyExists bool +param sqlDatabaseName string +param logAnalyticsReuse bool +param logAnalyticsWorkspaceName string +param logAnalyticsWorkspaceRG string +param logAnalyticsRetentionDays int +param sqlBackupRetentionDays int +param userObjectId string +param userPrincipalName string +param sqlAdminPrincipalType string + +param cloudEnvironment string +param authenticationOption string +param baseTime string +param resourceTags object +param contributorRoleAssignmentGuid string + +param argDiskExportJobId string = newGuid() +param argVhdExportJobId string = newGuid() +param argVmExportJobId string = newGuid() +param argVmssExportJobId string = newGuid() +param argAvailSetExportJobId string = newGuid() +param advisorExportJobId string = newGuid() +param consumptionExportJobId string = newGuid() +param aadObjectsExportJobId string = newGuid() +param argLoadBalancersExportJobId string = newGuid() +param argAppGWsExportJobId string = newGuid() +param rbacExportJobId string = newGuid() +param argResContainersExportJobId string = newGuid() +param argNICExportJobId string = newGuid() +param argNSGExportJobId string = newGuid() +param argPublicIPExportJobId string = newGuid() +param argVNetExportJobId string = newGuid() +param argSqlDbExportJobId string = newGuid() +param policyStateExportJobId string = newGuid() +param monitorVmssCpuMaxExportJobId string = newGuid() +param monitorVmssCpuAvgExportJobId string = newGuid() +param monitorVmssMemoryMinExportJobId string = newGuid() +param monitorSqlDbDtuMaxExportJobId string = newGuid() +param monitorSqlDbDtuAvgExportJobId string = newGuid() +param monitorAppServiceCpuMaxExportJobId string = newGuid() +param monitorAppServiceCpuAvgExportJobId string = newGuid() +param monitorAppServiceMemoryMaxExportJobId string = newGuid() +param monitorAppServiceMemoryAvgExportJobId string = newGuid() +param monitorDiskIOPSAvgExportJobId string = newGuid() +param monitorDiskMBPsAvgExportJobId string = newGuid() +param argAppServicePlanExportJobId string = newGuid() +param pricesheetExportJobId string = newGuid() +param reservationPricesExportJobId string = newGuid() +param reservationUsageExportJobId string = newGuid() +param savingsPlansUsageExportJobId string = newGuid() +param argDiskIngestJobId string = newGuid() +param argVhdIngestJobId string = newGuid() +param argVmIngestJobId string = newGuid() +param argVmssIngestJobId string = newGuid() +param argAvailSetIngestJobId string = newGuid() +param advisorIngestJobId string = newGuid() +param remediationLogsIngestJobId string = newGuid() +param consumptionIngestJobId string = newGuid() +param aadObjectsIngestJobId string = newGuid() +param argLoadBalancersIngestJobId string = newGuid() +param argAppGWsIngestJobId string = newGuid() +param argResContainersIngestJobId string = newGuid() +param rbacIngestJobId string = newGuid() +param argNICIngestJobId string = newGuid() +param argNSGIngestJobId string = newGuid() +param argPublicIPIngestJobId string = newGuid() +param argVNetIngestJobId string = newGuid() +param argSqlDbIngestJobId string = newGuid() +param policyStateIngestJobId string = newGuid() +param monitorIngestJobId string = newGuid() +param argAppServicePlanIngestJobId string = newGuid() +param pricesheetIngestJobId string = newGuid() +param reservationPricesIngestJobId string = newGuid() +param reservationUsageIngestJobId string = newGuid() +param savingsPlansUsageIngestJobId string = newGuid() +param unattachedDisksRecommendationJobId string = newGuid() +param advisorCostAugmentedRecommendationJobId string = newGuid() +param advisorAsIsRecommendationJobId string = newGuid() +param vmsHaRecommendationJobId string = newGuid() +param vmOptimizationsRecommendationJobId string = newGuid() +param aadExpiringCredsRecommendationJobId string = newGuid() +param unusedLoadBalancersRecommendationJobId string = newGuid() +param unusedAppGWsRecommendationJobId string = newGuid() +param armOptimizationsRecommendationJobId string = newGuid() +param vnetOptimizationsRecommendationJobId string = newGuid() +param vmssOptimizationsRecommendationJobId string = newGuid() +param sqldbOptimizationsRecommendationJobId string = newGuid() +param storageOptimizationsRecommendationJobId string = newGuid() +param appServiceOptimizationsRecommendationJobId string = newGuid() +param diskOptimizationsRecommendationJobId string = newGuid() +param recommendationsIngestJobId string = newGuid() +param recommendationsLogAnalyticsIngestJobId string = newGuid() +param suppressionsLogAnalyticsIngestJobId string = newGuid() +param recommendationsCleanUpJobId string = newGuid() + +param roleContributor string = '/subscriptions/${subscription().subscriptionId}/providers/Microsoft.Authorization/roleDefinitions/ba92f5b4-2d11-453d-a403-e96b0029c9fe' + +@description('Optional. Enable telemetry to track anonymous module usage trends, monitor for bugs, and improve future releases.') +param enableDefaultTelemetry bool = true + +var telemetryId = '00f120b5-2007-6120-0000-000000000a0e' +var finOpsToolkitVersion = loadTextContent('ftkver.txt') +var advisorExportsRunbookName = 'Export-AdvisorRecommendationsToBlobStorage' +var argVmExportsRunbookName = 'Export-ARGVirtualMachinesPropertiesToBlobStorage' +var argVmssExportsRunbookName = 'Export-ARGVMSSPropertiesToBlobStorage' +var argDisksExportsRunbookName = 'Export-ARGManagedDisksPropertiesToBlobStorage' +var argVhdExportsRunbookName = 'Export-ARGUnmanagedDisksPropertiesToBlobStorage' +var argAvailSetExportsRunbookName = 'Export-ARGAvailabilitySetPropertiesToBlobStorage' +var consumptionExportsRunbookName = 'Export-ConsumptionToBlobStorage' +var aadObjectsExportsRunbookName = 'Export-AADObjectsToBlobStorage' +var argLoadBalancersExportsRunbookName = 'Export-ARGLoadBalancerPropertiesToBlobStorage' +var argAppGWsExportsRunbookName = 'Export-ARGAppGatewayPropertiesToBlobStorage' +var argResContainersExportsRunbookName = 'Export-ARGResourceContainersPropertiesToBlobStorage' +var rbacExportsRunbookName = 'Export-RBACAssignmentsToBlobStorage' +var argNICExportsRunbookName = 'Export-ARGNICPropertiesToBlobStorage' +var argNSGExportsRunbookName = 'Export-ARGNSGPropertiesToBlobStorage' +var argVNetExportsRunbookName = 'Export-ARGVNetPropertiesToBlobStorage' +var argPublicIpExportsRunbookName = 'Export-ARGPublicIpPropertiesToBlobStorage' +var argSqlDbExportsRunbookName = 'Export-ARGSqlDatabasePropertiesToBlobStorage' +var policyStateExportsRunbookName = 'Export-PolicyComplianceToBlobStorage' +var monitorExportsRunbookName = 'Export-AzMonitorMetricsToBlobStorage' +var argAppServicePlanExportsRunbookName = 'Export-ARGAppServicePlanPropertiesToBlobStorage' +var reservationsExportsRunbookName = 'Export-ReservationsUsageToBlobStorage' +var reservationsPriceExportsRunbookName = 'Export-ReservationsPriceToBlobStorage' +var priceSheetExportsRunbookName = 'Export-PriceSheetToBlobStorage' +var savingsPlansExportsRunbookName = 'Export-SavingsPlansUsageToBlobStorage' +var advisorExportsScheduleName = 'AzureOptimization_ExportAdvisorWeekly' +var argExportsScheduleName = 'AzureOptimization_ExportARGDaily' +var consumptionExportsScheduleName = 'AzureOptimization_ExportConsumptionDaily' +var aadObjectsExportsScheduleName = 'AzureOptimization_ExportAADObjectsDaily' +var rbacExportsScheduleName = 'AzureOptimization_ExportRBACDaily' +var policyStateExportsScheduleName = 'AzureOptimization_ExportPolicyStateDaily' +var monitorVmssCpuMaxExportsScheduleName = 'AzureOptimization_ExportMonitorVmssCpuMaxHourly' +var monitorVmssCpuAvgExportsScheduleName = 'AzureOptimization_ExportMonitorVmssCpuAvgHourly' +var monitorVmssMemoryMinExportsScheduleName = 'AzureOptimization_ExportMonitorVmssMemoryMinHourly' +var monitorSqlDbDtuMaxExportsScheduleName = 'AzureOptimization_ExportMonitorSqlDbDtuMaxHourly' +var monitorSqlDbDtuAvgExportsScheduleName = 'AzureOptimization_ExportMonitorSqlDbDtuAvgHourly' +var monitorAppServiceCpuMaxExportsScheduleName = 'AzureOptimization_ExportMonitorAppServiceCpuMaxHourly' +var monitorAppServiceCpuAvgExportsScheduleName = 'AzureOptimization_ExportMonitorAppServiceCpuAvgHourly' +var monitorAppServiceMemoryMaxExportsScheduleName = 'AzureOptimization_ExportMonitorAppServiceMemoryMaxHourly' +var monitorAppServiceMemoryAvgExportsScheduleName = 'AzureOptimization_ExportMonitorAppServiceMemoryAvgHourly' +var monitorDiskIOPSAvgExportsScheduleName = 'AzureOptimization_ExportMonitorDiskIOPSHourly' +var monitorDiskMBPsAvgExportsScheduleName = 'AzureOptimization_ExportMonitorDiskMBPsHourly' +var priceExportsScheduleName = 'AzureOptimization_ExportPricesWeekly' +var reservationsUsageExportsScheduleName = 'AzureOptimization_ExportReservationsDaily' +var savingsPlansUsageExportsScheduleName = 'AzureOptimization_ExportSavingsPlansDaily' +var csvExportsSchedules = [ + { + exportSchedule: argExportsScheduleName + exportDescription: 'Daily Azure Resource Graph exports' + exportTimeOffset: 'PT1H05M' + exportFrequency: 'Day' + } + { + exportSchedule: advisorExportsScheduleName + exportDescription: 'Weekly Azure Advisor exports' + exportTimeOffset: 'PT1H15M' + exportFrequency: 'Week' + } + { + exportSchedule: consumptionExportsScheduleName + exportDescription: 'Daily Azure Consumption exports' + exportTimeOffset: 'PT1H' + exportFrequency: 'Day' + } + { + exportSchedule: aadObjectsExportsScheduleName + exportDescription: 'Daily Microsoft Entra Objects exports' + exportTimeOffset: 'PT1H' + exportFrequency: 'Day' + } + { + exportSchedule: rbacExportsScheduleName + exportDescription: 'Daily Azure RBAC exports' + exportTimeOffset: 'PT1H02M' + exportFrequency: 'Day' + } + { + exportSchedule: policyStateExportsScheduleName + exportDescription: 'Daily Azure Policy State exports' + exportTimeOffset: 'PT1H' + exportFrequency: 'Day' + } + { + exportSchedule: monitorVmssCpuAvgExportsScheduleName + exportDescription: 'Hourly Azure Monitor metrics exports for VMSS Percentage CPU (Avg.)' + exportTimeOffset: 'PT1H15M' + exportFrequency: 'Hour' + } + { + exportSchedule: monitorVmssCpuMaxExportsScheduleName + exportDescription: 'Hourly Azure Monitor metrics exports for VMSS Percentage CPU (Max.)' + exportTimeOffset: 'PT1H15M' + exportFrequency: 'Hour' + } + { + exportSchedule: monitorVmssMemoryMinExportsScheduleName + exportDescription: 'Hourly Azure Monitor metrics exports for VMSS Available Memory (Min.)' + exportTimeOffset: 'PT1H15M' + exportFrequency: 'Hour' + } + { + exportSchedule: monitorSqlDbDtuMaxExportsScheduleName + exportDescription: 'Hourly Azure Monitor metrics exports for SQL Database Percentage DTU (Max.)' + exportTimeOffset: 'PT1H15M' + exportFrequency: 'Hour' + } + { + exportSchedule: monitorSqlDbDtuAvgExportsScheduleName + exportDescription: 'Hourly Azure Monitor metrics exports for SQL Database Percentage DTU (Avg.)' + exportTimeOffset: 'PT1H16M' + exportFrequency: 'Hour' + } + { + exportSchedule: monitorAppServiceCpuAvgExportsScheduleName + exportDescription: 'Hourly Azure Monitor metrics exports for App Service Percentage CPU (Avg.)' + exportTimeOffset: 'PT1H16M' + exportFrequency: 'Hour' + } + { + exportSchedule: monitorAppServiceCpuMaxExportsScheduleName + exportDescription: 'Hourly Azure Monitor metrics exports for App Service Percentage CPU (Max.)' + exportTimeOffset: 'PT1H16M' + exportFrequency: 'Hour' + } + { + exportSchedule: monitorAppServiceMemoryAvgExportsScheduleName + exportDescription: 'Hourly Azure Monitor metrics exports for App Service Percentage RAM (Avg.)' + exportTimeOffset: 'PT1H16M' + exportFrequency: 'Hour' + } + { + exportSchedule: monitorAppServiceMemoryMaxExportsScheduleName + exportDescription: 'Hourly Azure Monitor metrics exports for App Service Percentage RAM (Max.)' + exportTimeOffset: 'PT1H17M' + exportFrequency: 'Hour' + } + { + exportSchedule: monitorDiskIOPSAvgExportsScheduleName + exportDescription: 'Hourly Azure Monitor metrics exports for Disk IOPS (Avg.)' + exportTimeOffset: 'PT1H17M' + exportFrequency: 'Hour' + } + { + exportSchedule: monitorDiskMBPsAvgExportsScheduleName + exportDescription: 'Hourly Azure Monitor metrics exports for Disk MBPs (Avg.)' + exportTimeOffset: 'PT1H17M' + exportFrequency: 'Hour' + } + { + exportSchedule: priceExportsScheduleName + exportDescription: 'Weekly Pricesheet and Reservation Prices exports' + exportTimeOffset: 'PT1H35M' + exportFrequency: 'Week' + } + { + exportSchedule: reservationsUsageExportsScheduleName + exportDescription: 'Daily Reservation Usage exports' + exportTimeOffset: 'PT2H' + exportFrequency: 'Day' + } + { + exportSchedule: savingsPlansUsageExportsScheduleName + exportDescription: 'Daily Savings Plans Usage exports' + exportTimeOffset: 'PT2H05M' + exportFrequency: 'Day' + } +] +var csvExports = [ + { + runbookName: advisorExportsRunbookName + isOneToMany: false + containerName: 'advisorexports' + variableName: 'AzureOptimization_AdvisorContainer' + variableDescription: 'The Storage Account container where Azure Advisor exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestAdvisorWeekly' + ingestDescription: 'Weekly Azure Advisor recommendations ingests' + ingestTimeOffset: 'PT1H45M' + ingestFrequency: 'Week' + ingestJobId: advisorIngestJobId + exportSchedule: advisorExportsScheduleName + exportJobId: advisorExportJobId + } + { + runbookName: argVmExportsRunbookName + isOneToMany: false + containerName: 'argvmexports' + variableName: 'AzureOptimization_ARGVMContainer' + variableDescription: 'The Storage Account container where Azure Resource Graph Virtual Machine exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestARGVMsDaily' + ingestDescription: 'Daily Azure Resource Graph Virtual Machines ingests' + ingestTimeOffset: 'PT1H30M' + ingestFrequency: 'Day' + ingestJobId: argVmIngestJobId + exportSchedule: argExportsScheduleName + exportJobId: argVmExportJobId + } + { + runbookName: argVmssExportsRunbookName + isOneToMany: false + containerName: 'argvmssexports' + variableName: 'AzureOptimization_ARGVMSSContainer' + variableDescription: 'The Storage Account container where Azure Resource Graph VMSS exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestARGVMSSDaily' + ingestDescription: 'Daily Azure Resource Graph VMSS ingests' + ingestTimeOffset: 'PT1H30M' + ingestFrequency: 'Day' + ingestJobId: argVmssIngestJobId + exportSchedule: argExportsScheduleName + exportJobId: argVmssExportJobId + } + { + runbookName: argDisksExportsRunbookName + isOneToMany: false + containerName: 'argdiskexports' + variableName: 'AzureOptimization_ARGDiskContainer' + variableDescription: 'The Storage Account container where Azure Resource Graph Managed Disks exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestARGDisksDaily' + ingestDescription: 'Daily Azure Resource Graph Managed Disks ingests' + ingestTimeOffset: 'PT1H30M' + ingestFrequency: 'Day' + ingestJobId: argDiskIngestJobId + exportSchedule: argExportsScheduleName + exportJobId: argDiskExportJobId + } + { + runbookName: argVhdExportsRunbookName + isOneToMany: false + containerName: 'argvhdexports' + variableName: 'AzureOptimization_ARGVhdContainer' + variableDescription: 'The Storage Account container where Azure Resource Graph Unmanaged Disks exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestARGVHDsDaily' + ingestDescription: 'Daily Azure Resource Graph Unmanaged Disks ingests' + ingestTimeOffset: 'PT1H30M' + ingestFrequency: 'Day' + ingestJobId: argVhdIngestJobId + exportSchedule: argExportsScheduleName + exportJobId: argVhdExportJobId + } + { + runbookName: argAvailSetExportsRunbookName + isOneToMany: false + containerName: 'argavailsetexports' + variableName: 'AzureOptimization_ARGAvailabilitySetContainer' + variableDescription: 'The Storage Account container where Azure Resource Graph Availability Set exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestARGAvailSetsDaily' + ingestDescription: 'Daily Azure Resource Graph Availability Sets ingests' + ingestTimeOffset: 'PT1H31M' + ingestFrequency: 'Day' + ingestJobId: argAvailSetIngestJobId + exportSchedule: argExportsScheduleName + exportJobId: argAvailSetExportJobId + } + { + runbookName: consumptionExportsRunbookName + isOneToMany: false + containerName: 'consumptionexports' + variableName: 'AzureOptimization_ConsumptionContainer' + variableDescription: 'The Storage Account container where Azure Consumption exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestConsumptionDaily' + ingestDescription: 'Daily Azure Consumption ingests' + ingestTimeOffset: 'PT2H' + ingestFrequency: 'Day' + ingestJobId: consumptionIngestJobId + exportSchedule: consumptionExportsScheduleName + exportJobId: consumptionExportJobId + } + { + runbookName: aadObjectsExportsRunbookName + isOneToMany: false + containerName: 'aadobjectsexports' + variableName: 'AzureOptimization_AADObjectsContainer' + variableDescription: 'The Storage Account container where Microsoft Entra Objects exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestAADObjectsDaily' + ingestDescription: 'Daily Microsoft Entra Objects ingests' + ingestTimeOffset: 'PT2H' + ingestFrequency: 'Day' + ingestJobId: aadObjectsIngestJobId + exportSchedule: aadObjectsExportsScheduleName + exportJobId: aadObjectsExportJobId + } + { + runbookName: argLoadBalancersExportsRunbookName + isOneToMany: false + containerName: 'arglbexports' + variableName: 'AzureOptimization_ARGLoadBalancerContainer' + variableDescription: 'The Storage Account container where Azure Resource Graph Load Balancer exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestARGLoadBalancersDaily' + ingestDescription: 'Daily Azure Resource Graph Load Balancers ingests' + ingestTimeOffset: 'PT1H31M' + ingestFrequency: 'Day' + ingestJobId: argLoadBalancersIngestJobId + exportSchedule: argExportsScheduleName + exportJobId: argLoadBalancersExportJobId + } + { + runbookName: argAppGWsExportsRunbookName + isOneToMany: false + containerName: 'argappgwexports' + variableName: 'AzureOptimization_ARGAppGatewayContainer' + variableDescription: 'The Storage Account container where Azure Resource Graph Application Gateway exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestARGAppGWsDaily' + ingestDescription: 'Daily Azure Resource Graph Application Gateways ingests' + ingestTimeOffset: 'PT1H31M' + ingestFrequency: 'Day' + ingestJobId: argAppGWsIngestJobId + exportSchedule: argExportsScheduleName + exportJobId: argAppGWsExportJobId + } + { + runbookName: argResContainersExportsRunbookName + isOneToMany: false + containerName: 'argrescontainersexports' + variableName: 'AzureOptimization_ARGResourceContainersContainer' + variableDescription: 'The Storage Account container where Azure Resource Graph Resource Containers exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestARGResourceContainersDaily' + ingestDescription: 'Daily Azure Resource Graph Resource Containers ingests' + ingestTimeOffset: 'PT1H32M' + ingestFrequency: 'Day' + ingestJobId: argResContainersIngestJobId + exportSchedule: argExportsScheduleName + exportJobId: argResContainersExportJobId + } + { + runbookName: rbacExportsRunbookName + isOneToMany: false + containerName: 'rbacexports' + variableName: 'AzureOptimization_RBACAssignmentsContainer' + variableDescription: 'The Storage Account container where RBAC Assignments exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestRBACDaily' + ingestDescription: 'Daily Azure RBAC ingests' + ingestTimeOffset: 'PT1H32M' + ingestFrequency: 'Day' + ingestJobId: rbacIngestJobId + exportSchedule: rbacExportsScheduleName + exportJobId: rbacExportJobId + } + { + runbookName: argNICExportsRunbookName + isOneToMany: false + containerName: 'argnicexports' + variableName: 'AzureOptimization_ARGNICContainer' + variableDescription: 'The Storage Account container where Azure Resource Graph NIC exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestARGNICsDaily' + ingestDescription: 'Daily Azure Resource Graph NIC ingests' + ingestTimeOffset: 'PT1H32M' + ingestFrequency: 'Day' + ingestJobId: argNICIngestJobId + exportSchedule: argExportsScheduleName + exportJobId: argNICExportJobId + } + { + runbookName: argNSGExportsRunbookName + isOneToMany: false + containerName: 'argnsgexports' + variableName: 'AzureOptimization_ARGNSGContainer' + variableDescription: 'The Storage Account container where Azure Resource Graph NSG exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestARGNSGsDaily' + ingestDescription: 'Daily Azure Resource Graph NSG ingests' + ingestTimeOffset: 'PT1H32M' + ingestFrequency: 'Day' + ingestJobId: argNSGIngestJobId + exportSchedule: argExportsScheduleName + exportJobId: argNSGExportJobId + } + { + runbookName: argVNetExportsRunbookName + isOneToMany: false + containerName: 'argvnetexports' + variableName: 'AzureOptimization_ARGVNetContainer' + variableDescription: 'The Storage Account container where Azure Resource Graph VNet exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestARGVNetsDaily' + ingestDescription: 'Daily Azure Resource Graph Virtual Network ingests' + ingestTimeOffset: 'PT1H33M' + ingestFrequency: 'Day' + ingestJobId: argVNetIngestJobId + exportSchedule: argExportsScheduleName + exportJobId: argVNetExportJobId + } + { + runbookName: argPublicIpExportsRunbookName + isOneToMany: false + containerName: 'argpublicipexports' + variableName: 'AzureOptimization_ARGPublicIpContainer' + variableDescription: 'The Storage Account container where Azure Resource Graph Public IP exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestARGPublicIPsDaily' + ingestDescription: 'Daily Azure Resource Graph Public IP ingests' + ingestTimeOffset: 'PT1H33M' + ingestFrequency: 'Day' + ingestJobId: argPublicIPIngestJobId + exportSchedule: argExportsScheduleName + exportJobId: argPublicIPExportJobId + } + { + runbookName: argSqlDbExportsRunbookName + isOneToMany: false + containerName: 'argsqldbexports' + variableName: 'AzureOptimization_ARGSqlDatabaseContainer' + variableDescription: 'The Storage Account container where Azure Resource Graph SQL DB exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestARGSqlDbDaily' + ingestDescription: 'Daily Azure Resource Graph SQL DB ingests' + ingestTimeOffset: 'PT1H33M' + ingestFrequency: 'Day' + ingestJobId: argSqlDbIngestJobId + exportSchedule: argExportsScheduleName + exportJobId: argSqlDbExportJobId + } + { + runbookName: policyStateExportsRunbookName + isOneToMany: false + containerName: 'policystateexports' + variableName: 'AzureOptimization_PolicyStatesContainer' + variableDescription: 'The Storage Account container where Azure Policy State exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestPolicyStateDaily' + ingestDescription: 'Daily Azure Policy State ingests' + ingestTimeOffset: 'PT1H33M' + ingestFrequency: 'Day' + ingestJobId: policyStateIngestJobId + exportSchedule: policyStateExportsScheduleName + exportJobId: policyStateExportJobId + } + { + runbookName: monitorExportsRunbookName + isOneToMany: true + containerName: 'azmonitorexports' + variableName: 'AzureOptimization_AzMonitorContainer' + variableDescription: 'The Storage Account container where Azure Monitor metrics exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestAzMonitorMetricsHourly' + ingestDescription: 'Hourly Azure Monitor metrics ingests' + ingestTimeOffset: 'PT2H' + ingestFrequency: 'Hour' + ingestJobId: monitorIngestJobId + exportSchedule: null + exportJobId: 'dummy' + } + { + runbookName: argAppServicePlanExportsRunbookName + isOneToMany: false + containerName: 'argappserviceplanexports' + variableName: 'AzureOptimization_ARGAppServicePlanContainer' + variableDescription: 'The Storage Account container where Azure Resource Graph App Service Plan exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestARGAppServicePlanDaily' + ingestDescription: 'Daily Azure Resource Graph App Service Plan ingests' + ingestTimeOffset: 'PT1H34M' + ingestFrequency: 'Day' + ingestJobId: argAppServicePlanIngestJobId + exportSchedule: argExportsScheduleName + exportJobId: argAppServicePlanExportJobId + } + { + runbookName: priceSheetExportsRunbookName + isOneToMany: false + containerName: 'pricesheetexports' + variableName: 'AzureOptimization_PriceSheetContainer' + variableDescription: 'The Storage Account container where Pricesheet exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestPricesheetWeekly' + ingestDescription: 'Weekly Pricesheet ingests' + ingestTimeOffset: 'PT2H' + ingestFrequency: 'Week' + ingestJobId: pricesheetIngestJobId + exportSchedule: priceExportsScheduleName + exportJobId: pricesheetExportJobId + } + { + runbookName: reservationsPriceExportsRunbookName + isOneToMany: false + containerName: 'reservationspriceexports' + variableName: 'AzureOptimization_ReservationsPriceContainer' + variableDescription: 'The Storage Account container where Reservations Prices exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestReservationsPriceWeekly' + ingestDescription: 'Weekly Reservations Prices ingests' + ingestTimeOffset: 'PT2H' + ingestFrequency: 'Week' + ingestJobId: reservationPricesIngestJobId + exportSchedule: priceExportsScheduleName + exportJobId: reservationPricesExportJobId + } + { + runbookName: reservationsExportsRunbookName + isOneToMany: false + containerName: 'reservationsexports' + variableName: 'AzureOptimization_ReservationsContainer' + variableDescription: 'The Storage Account container where Reservations Usage exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestReservationsUsageDaily' + ingestDescription: 'Daily Reservations Usage ingests' + ingestTimeOffset: 'PT2H30M' + ingestFrequency: 'Day' + ingestJobId: reservationUsageIngestJobId + exportSchedule: reservationsUsageExportsScheduleName + exportJobId: reservationUsageExportJobId + } + { + runbookName: savingsPlansExportsRunbookName + isOneToMany: false + containerName: 'savingsplansexports' + variableName: 'AzureOptimization_SavingsPlansContainer' + variableDescription: 'The Storage Account container where Savings Plans Usage exports are dumped to' + ingestSchedule: 'AzureOptimization_IngestSavingsPlansUsageDaily' + ingestDescription: 'Daily Savings Plans Usage ingests' + ingestTimeOffset: 'PT2H35M' + ingestFrequency: 'Day' + ingestJobId: savingsPlansUsageIngestJobId + exportSchedule: savingsPlansUsageExportsScheduleName + exportJobId: savingsPlansUsageExportJobId + } +] +var csvParameterizedExports = [ + { + runbookName: monitorExportsRunbookName + exportSchedule: monitorVmssCpuMaxExportsScheduleName + exportJobId: monitorVmssCpuMaxExportJobId + parameters: { + ResourceType: 'microsoft.compute/virtualmachinescalesets' + TimeSpan: '01:00:00' + aggregationType: 'Maximum' + MetricNames: 'Percentage CPU' + TimeGrain: '01:00:00' + } + } + { + runbookName: monitorExportsRunbookName + exportSchedule: monitorVmssCpuAvgExportsScheduleName + exportJobId: monitorVmssCpuAvgExportJobId + parameters: { + ResourceType: 'microsoft.compute/virtualmachinescalesets' + TimeSpan: '01:00:00' + aggregationType: 'Average' + MetricNames: 'Percentage CPU' + TimeGrain: '01:00:00' + } + } + { + runbookName: monitorExportsRunbookName + exportSchedule: monitorVmssMemoryMinExportsScheduleName + exportJobId: monitorVmssMemoryMinExportJobId + parameters: { + ResourceType: 'microsoft.compute/virtualmachinescalesets' + TimeSpan: '01:00:00' + aggregationType: 'Minimum' + MetricNames: 'Available Memory Bytes' + TimeGrain: '01:00:00' + } + } + { + runbookName: monitorExportsRunbookName + exportSchedule: monitorSqlDbDtuMaxExportsScheduleName + exportJobId: monitorSqlDbDtuMaxExportJobId + parameters: { + ResourceType: 'microsoft.sql/servers/databases' + ARGFilter: 'sku.tier in (\'Standard\',\'Premium\')' + TimeSpan: '01:00:00' + aggregationType: 'Maximum' + MetricNames: 'dtu_consumption_percent' + TimeGrain: '01:00:00' + } + } + { + runbookName: monitorExportsRunbookName + exportSchedule: monitorSqlDbDtuAvgExportsScheduleName + exportJobId: monitorSqlDbDtuAvgExportJobId + parameters: { + ResourceType: 'microsoft.sql/servers/databases' + ARGFilter: 'sku.tier in (\'Standard\',\'Premium\')' + TimeSpan: '01:00:00' + aggregationType: 'Average' + AggregationOfType: 'Maximum' + MetricNames: 'dtu_consumption_percent' + TimeGrain: '00:01:00' + } + } + { + runbookName: monitorExportsRunbookName + exportSchedule: monitorAppServiceCpuMaxExportsScheduleName + exportJobId: monitorAppServiceCpuMaxExportJobId + parameters: { + ResourceType: 'microsoft.web/serverfarms' + ARGFilter: 'properties.computeMode == \'Dedicated\' and sku.tier != \'Free\'' + TimeSpan: '01:00:00' + aggregationType: 'Maximum' + MetricNames: 'CpuPercentage' + TimeGrain: '01:00:00' + } + } + { + runbookName: monitorExportsRunbookName + exportSchedule: monitorAppServiceCpuAvgExportsScheduleName + exportJobId: monitorAppServiceCpuAvgExportJobId + parameters: { + ResourceType: 'microsoft.web/serverfarms' + ARGFilter: 'properties.computeMode == \'Dedicated\' and sku.tier != \'Free\'' + TimeSpan: '01:00:00' + aggregationType: 'Average' + AggregationOfType: 'Maximum' + MetricNames: 'CpuPercentage' + TimeGrain: '00:01:00' + } + } + { + runbookName: monitorExportsRunbookName + exportSchedule: monitorAppServiceMemoryMaxExportsScheduleName + exportJobId: monitorAppServiceMemoryMaxExportJobId + parameters: { + ResourceType: 'microsoft.web/serverfarms' + ARGFilter: 'properties.computeMode == \'Dedicated\' and sku.tier != \'Free\'' + TimeSpan: '01:00:00' + aggregationType: 'Maximum' + MetricNames: 'MemoryPercentage' + TimeGrain: '01:00:00' + } + } + { + runbookName: monitorExportsRunbookName + exportSchedule: monitorAppServiceMemoryAvgExportsScheduleName + exportJobId: monitorAppServiceMemoryAvgExportJobId + parameters: { + ResourceType: 'microsoft.web/serverfarms' + ARGFilter: 'properties.computeMode == \'Dedicated\' and sku.tier != \'Free\'' + TimeSpan: '01:00:00' + aggregationType: 'Average' + AggregationOfType: 'Maximum' + MetricNames: 'MemoryPercentage' + TimeGrain: '00:01:00' + } + } + { + runbookName: monitorExportsRunbookName + exportSchedule: monitorDiskIOPSAvgExportsScheduleName + exportJobId: monitorDiskIOPSAvgExportJobId + parameters: { + ResourceType: 'microsoft.compute/disks' + ARGFilter: 'sku.name startswith \'Premium_\' and properties.diskState =~ \'Attached\'' + TimeSpan: '01:00:00' + aggregationType: 'Average' + AggregationOfType: 'Maximum' + MetricNames: 'Composite Disk Read Operations/sec,Composite Disk Write Operations/sec' + TimeGrain: '00:01:00' + } + } + { + runbookName: monitorExportsRunbookName + exportSchedule: monitorDiskMBPsAvgExportsScheduleName + exportJobId: monitorDiskMBPsAvgExportJobId + parameters: { + ResourceType: 'microsoft.compute/disks' + ARGFilter: 'sku.name startswith \'Premium_\' and properties.diskState =~ \'Attached\'' + TimeSpan: '01:00:00' + aggregationType: 'Average' + AggregationOfType: 'Maximum' + MetricNames: 'Composite Disk Read Bytes/sec,Composite Disk Write Bytes/sec' + TimeGrain: '00:01:00' + } + } +] +var unattachedDisksRecommendationsRunbookName = 'Recommend-UnattachedDisksToBlobStorage' +var advisorCostAugmentedRecommendationsRunbookName = 'Recommend-AdvisorCostAugmentedToBlobStorage' +var advisorAsIsRecommendationsRunbookName = 'Recommend-AdvisorAsIsToBlobStorage' +var vmsHARecommendationsRunbookName = 'Recommend-VMsHighAvailabilityToBlobStorage' +var vmOptimizationsRecommendationsRunbookName = 'Recommend-VMOptimizationsToBlobStorage' +var aadExpiringCredsRecommendationsRunbookName = 'Recommend-AADExpiringCredentialsToBlobStorage' +var unusedLBsRecommendationsRunbookName = 'Recommend-UnusedLoadBalancersToBlobStorage' +var unusedAppGWsRecommendationsRunbookName = 'Recommend-UnusedAppGWsToBlobStorage' +var armOptimizationsRecommendationsRunbookName = 'Recommend-ARMOptimizationsToBlobStorage' +var vnetOptimizationsRecommendationsRunbookName = 'Recommend-VNetOptimizationsToBlobStorage' +var vmssOptimizationsRecommendationsRunbookName = 'Recommend-VMSSOptimizationsToBlobStorage' +var sqldbOptimizationsRecommendationsRunbookName = 'Recommend-SqlDbOptimizationsToBlobStorage' +var storageOptimizationsRecommendationsRunbookName = 'Recommend-StorageAccountOptimizationsToBlobStorage' +var appServiceOptimizationsRecommendationsRunbookName = 'Recommend-AppServiceOptimizationsToBlobStorage' +var diskOptimizationsRecommendationsRunbookName = 'Recommend-DiskOptimizationsToBlobStorage' +var cleanUpOlderRecommendationsRunbookName = 'CleanUp-OlderRecommendationsFromSqlServer' +var recommendations = [ + { + recommendationJobId: unattachedDisksRecommendationJobId + runbookName: unattachedDisksRecommendationsRunbookName + } + { + recommendationJobId: advisorCostAugmentedRecommendationJobId + runbookName: advisorCostAugmentedRecommendationsRunbookName + } + { + recommendationJobId: advisorAsIsRecommendationJobId + runbookName: advisorAsIsRecommendationsRunbookName + } + { + recommendationJobId: vmsHaRecommendationJobId + runbookName: vmsHARecommendationsRunbookName + } + { + recommendationJobId: vmOptimizationsRecommendationJobId + runbookName: vmOptimizationsRecommendationsRunbookName + } + { + recommendationJobId: aadExpiringCredsRecommendationJobId + runbookName: aadExpiringCredsRecommendationsRunbookName + } + { + recommendationJobId: unusedLoadBalancersRecommendationJobId + runbookName: unusedLBsRecommendationsRunbookName + } + { + recommendationJobId: unusedAppGWsRecommendationJobId + runbookName: unusedAppGWsRecommendationsRunbookName + } + { + recommendationJobId: armOptimizationsRecommendationJobId + runbookName: armOptimizationsRecommendationsRunbookName + } + { + recommendationJobId: vnetOptimizationsRecommendationJobId + runbookName: vnetOptimizationsRecommendationsRunbookName + } + { + recommendationJobId: vmssOptimizationsRecommendationJobId + runbookName: vmssOptimizationsRecommendationsRunbookName + } + { + recommendationJobId: sqldbOptimizationsRecommendationJobId + runbookName: sqldbOptimizationsRecommendationsRunbookName + } + { + recommendationJobId: storageOptimizationsRecommendationJobId + runbookName: storageOptimizationsRecommendationsRunbookName + } + { + recommendationJobId: appServiceOptimizationsRecommendationJobId + runbookName: appServiceOptimizationsRecommendationsRunbookName + } + { + recommendationJobId: diskOptimizationsRecommendationJobId + runbookName: diskOptimizationsRecommendationsRunbookName + } +] +var remediationLogsContainerName = 'remediationlogs' +var recommendationsContainerName = 'recommendationsexports' +var csvIngestRunbookName = 'Ingest-OptimizationCSVExportsToLogAnalytics' +var recommendationsIngestRunbookName = 'Ingest-RecommendationsToSQLServer' +var recommendationsLogAnalyticsIngestRunbookName = 'Ingest-RecommendationsToLogAnalytics' +var suppressionsLogAnalyticsIngestRunbookName = 'Ingest-SuppressionsToLogAnalytics' +var advisorRightSizeFilteredRemediationRunbookName = 'Remediate-AdvisorRightSizeFiltered' +var longDeallocatedVMsFilteredRemediationRunbookName = 'Remediate-LongDeallocatedVMsFiltered' +var unattachedDisksFilteredRemediationRunbookName = 'Remediate-UnattachedDisksFiltered' +var remediationLogsIngestScheduleName = 'AzureOptimization_IngestRemediationLogsDaily' +var recommendationsScheduleName = 'AzureOptimization_RecommendationsWeekly' +var recommendationsIngestScheduleName = 'AzureOptimization_IngestRecommendationsWeekly' +var suppressionsIngestScheduleName = 'AzureOptimization_IngestSuppressionsWeekly' +var recommendationsCleanUpScheduleName = 'AzureOptimization_CleanUpRecommendationsWeekly' +var Az_Accounts = { + name: 'Az.Accounts' + url: 'https://www.powershellgallery.com/api/v2/package/Az.Accounts/2.12.1' +} +var Microsoft_Graph_Authentication = { + name: 'Microsoft.Graph.Authentication' + url: 'https://www.powershellgallery.com/api/v2/package/Microsoft.Graph.Authentication/2.4.0' +} +var psModules = [ + { + name: 'Az.Compute' + url: 'https://www.powershellgallery.com/api/v2/package/Az.Compute/5.7.0' + } + { + name: 'Az.OperationalInsights' + url: 'https://www.powershellgallery.com/api/v2/package/Az.OperationalInsights/3.2.0' + } + { + name: 'Az.ResourceGraph' + url: 'https://www.powershellgallery.com/api/v2/package/Az.ResourceGraph/0.13.0' + } + { + name: 'Az.Storage' + url: 'https://www.powershellgallery.com/api/v2/package/Az.Storage/5.5.0' + } + { + name: 'Az.Resources' + url: 'https://www.powershellgallery.com/api/v2/package/Az.Resources/6.6.0' + } + { + name: 'Az.Monitor' + url: 'https://www.powershellgallery.com/api/v2/package/Az.Monitor/4.4.1' + } + { + name: 'Az.PolicyInsights' + url: 'https://www.powershellgallery.com/api/v2/package/Az.PolicyInsights/1.6.0' + } + { + name: 'Microsoft.Graph.Users' + url: 'https://www.powershellgallery.com/api/v2/package/Microsoft.Graph.Users/2.4.0' + } + { + name: 'Microsoft.Graph.Groups' + url: 'https://www.powershellgallery.com/api/v2/package/Microsoft.Graph.Groups/2.4.0' + } + { + name: 'Microsoft.Graph.Applications' + url: 'https://www.powershellgallery.com/api/v2/package/Microsoft.Graph.Applications/2.4.0' + } + { + name: 'Microsoft.Graph.Identity.DirectoryManagement' + url: 'https://www.powershellgallery.com/api/v2/package/Microsoft.Graph.Identity.DirectoryManagement/2.4.0' + } +] +var runbooks = [ + { + name: advisorExportsRunbookName + version: '1.4.3.0' + description: 'Exports Azure Advisor recommendations to Blob Storage using the Advisor API' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${advisorExportsRunbookName}.ps1') + } + { + name: argDisksExportsRunbookName + version: '1.3.5.0' + description: 'Exports Managed Disks properties to Blob Storage using Azure Resource Graph' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${argDisksExportsRunbookName}.ps1') + } + { + name: argVhdExportsRunbookName + version: '1.1.5.0' + description: 'Exports Unmanaged Disks (owned by a VM) properties to Blob Storage using Azure Resource Graph' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${argVhdExportsRunbookName}.ps1') + } + { + name: argVmExportsRunbookName + version: '1.4.5.0' + description: 'Exports Virtual Machine properties to Blob Storage using Azure Resource Graph' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${argVmExportsRunbookName}.ps1') + } + { + name: argVmssExportsRunbookName + version: '1.0.3.0' + description: 'Exports VMSS properties to Blob Storage using Azure Resource Graph' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${argVmssExportsRunbookName}.ps1') + } + { + name: argAvailSetExportsRunbookName + version: '1.1.5.0' + description: 'Exports Availability Set properties to Blob Storage using Azure Resource Graph' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${argAvailSetExportsRunbookName}.ps1') + } + { + name: consumptionExportsRunbookName + version: '2.1.1.0' + description: 'Exports Azure Consumption events to Blob Storage using Azure Consumption API' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${consumptionExportsRunbookName}.ps1') + } + { + name: aadObjectsExportsRunbookName + version: '1.3.1.0' + description: 'Exports Azure AAD Objects to Blob Storage using Azure ARM API' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${aadObjectsExportsRunbookName}.ps1') + } + { + name: argLoadBalancersExportsRunbookName + version: '1.1.5.0' + description: 'Exports Load Balancer properties to Blob Storage using Azure Resource Graph' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${argLoadBalancersExportsRunbookName}.ps1') + } + { + name: argAppGWsExportsRunbookName + version: '1.1.5.0' + description: 'Exports Application Gateway properties to Blob Storage using Azure Resource Graph' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${argAppGWsExportsRunbookName}.ps1') + } + { + name: argResContainersExportsRunbookName + version: '1.0.6.0' + description: 'Exports Resource Containers properties to Blob Storage using Azure Resource Graph' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${argResContainersExportsRunbookName}.ps1') + } + { + name: rbacExportsRunbookName + version: '1.1.1.0' + description: 'Exports RBAC assignments to Blob Storage using ARM and Microsoft Entra' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${rbacExportsRunbookName}.ps1') + } + { + name: argNICExportsRunbookName + version: '1.0.3.0' + description: 'Exports NIC properties to Blob Storage using Azure Resource Graph' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${argNICExportsRunbookName}.ps1') + } + { + name: argNSGExportsRunbookName + version: '1.0.3.0' + description: 'Exports NSG properties to Blob Storage using Azure Resource Graph' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${argNSGExportsRunbookName}.ps1') + } + { + name: argPublicIpExportsRunbookName + version: '1.0.3.0' + description: 'Exports Public IP properties to Blob Storage using Azure Resource Graph' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${argPublicIpExportsRunbookName}.ps1') + } + { + name: argVNetExportsRunbookName + version: '1.0.3.0' + description: 'Exports VNet properties to Blob Storage using Azure Resource Graph' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${argVNetExportsRunbookName}.ps1') + } + { + name: argSqlDbExportsRunbookName + version: '1.0.3.0' + description: 'Exports SQL DB properties to Blob Storage using Azure Resource Graph' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${argSqlDbExportsRunbookName}.ps1') + } + { + name: policyStateExportsRunbookName + version: '1.0.4.0' + description: 'Exports Azure Policy State to Blob Storage' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${policyStateExportsRunbookName}.ps1') + } + { + name: monitorExportsRunbookName + version: '1.0.3.0' + description: 'Exports Azure Monitor metrics to Blob Storage' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${monitorExportsRunbookName}.ps1') + } + { + name: argAppServicePlanExportsRunbookName + version: '1.0.2.0' + description: 'Exports App Service Plan properties to Blob Storage using Azure Resource Graph' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${argAppServicePlanExportsRunbookName}.ps1') + } + { + name: reservationsExportsRunbookName + version: '1.1.3.0' + description: 'Exports Reservations Usage to Blob Storage using the EA or MCA APIs' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${reservationsExportsRunbookName}.ps1') + } + { + name: reservationsPriceExportsRunbookName + version: '1.0.2.0' + description: 'Exports Reservations Prices to Blob Storage using the Retail Prices API' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${reservationsPriceExportsRunbookName}.ps1') + } + { + name: priceSheetExportsRunbookName + version: '1.1.2.0' + description: 'Exports Price Sheet to Blob Storage using the EA or MCA APIs' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${priceSheetExportsRunbookName}.ps1') + } + { + name: savingsPlansExportsRunbookName + version: '1.0.2.0' + description: 'Exports Savings Plans Usage to Blob Storage using the EA or MCA APIs' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${savingsPlansExportsRunbookName}.ps1') + } + { + name: csvIngestRunbookName + version: '1.6.2.0' + description: 'Ingests CSV blobs as custom logs to Log Analytics' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/data-collection/${csvIngestRunbookName}.ps1') + } + { + name: unattachedDisksRecommendationsRunbookName + version: '2.5.1.0' + description: 'Generates unattached disks recommendations' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${unattachedDisksRecommendationsRunbookName}.ps1') + } + { + name: advisorCostAugmentedRecommendationsRunbookName + version: '2.10.1.0' + description: 'Generates augmented Advisor Cost recommendations' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${advisorCostAugmentedRecommendationsRunbookName}.ps1') + } + { + name: advisorAsIsRecommendationsRunbookName + version: '1.6.1.0' + description: 'Generates all types of Advisor recommendations' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${advisorAsIsRecommendationsRunbookName}.ps1') + } + { + name: vmsHARecommendationsRunbookName + version: '1.1.1.0' + description: 'Generates VMs High Availability recommendations' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${vmsHARecommendationsRunbookName}.ps1') + } + { + name: vmOptimizationsRecommendationsRunbookName + version: '1.1.1.0' + description: 'Generates VM optimizations recommendations' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${vmOptimizationsRecommendationsRunbookName}.ps1') + } + { + name: aadExpiringCredsRecommendationsRunbookName + version: '1.2.1.0' + description: 'Generates AAD Objects with expiring credentials recommendations' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${aadExpiringCredsRecommendationsRunbookName}.ps1') + } + { + name: unusedLBsRecommendationsRunbookName + version: '1.3.1.0' + description: 'Generates unused Load Balancers recommendations' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${unusedLBsRecommendationsRunbookName}.ps1') + } + { + name: unusedAppGWsRecommendationsRunbookName + version: '1.3.1.0' + description: 'Generates unused Application Gateways recommendations' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${unusedAppGWsRecommendationsRunbookName}.ps1') + } + { + name: armOptimizationsRecommendationsRunbookName + version: '1.1.1.0' + description: 'Generates ARM optimizations recommendations' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${armOptimizationsRecommendationsRunbookName}.ps1') + } + { + name: vnetOptimizationsRecommendationsRunbookName + version: '1.1.1.0' + description: 'Generates Virtual Network optimizations recommendations' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${vnetOptimizationsRecommendationsRunbookName}.ps1') + } + { + name: vmssOptimizationsRecommendationsRunbookName + version: '1.2.1.0' + description: 'Generates VM Scale Set optimizations recommendations' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${vmssOptimizationsRecommendationsRunbookName}.ps1') + } + { + name: sqldbOptimizationsRecommendationsRunbookName + version: '1.2.1.0' + description: 'Generates SQL DB optimizations recommendations' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${sqldbOptimizationsRecommendationsRunbookName}.ps1') + } + { + name: storageOptimizationsRecommendationsRunbookName + version: '1.1.1.0' + description: 'Generates Storage Account optimizations recommendations' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${storageOptimizationsRecommendationsRunbookName}.ps1') + } + { + name: appServiceOptimizationsRecommendationsRunbookName + version: '1.1.1.0' + description: 'Generates App Service optimizations recommendations' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${appServiceOptimizationsRecommendationsRunbookName}.ps1') + } + { + name: diskOptimizationsRecommendationsRunbookName + version: '1.2.1.0' + description: 'Generates Disk optimizations recommendations' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${diskOptimizationsRecommendationsRunbookName}.ps1') + } + { + name: recommendationsIngestRunbookName + version: '1.7.1.0' + description: 'Ingests JSON-based recommendations into an Azure SQL Database' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${recommendationsIngestRunbookName}.ps1') + } + { + name: recommendationsLogAnalyticsIngestRunbookName + version: '1.1.1.0' + description: 'Ingests JSON-based recommendations into Log Analytics' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${recommendationsLogAnalyticsIngestRunbookName}.ps1') + } + { + name: suppressionsLogAnalyticsIngestRunbookName + version: '1.1.0.0' + description: 'Ingests suppressions into Log Analytics' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/recommendations/${suppressionsLogAnalyticsIngestRunbookName}.ps1') + } + { + name: advisorRightSizeFilteredRemediationRunbookName + version: '1.3.1.0' + description: 'Remediates Azure Advisor right-size recommendations given fit and tag filters' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/remediations/${advisorRightSizeFilteredRemediationRunbookName}.ps1') + } + { + name: longDeallocatedVMsFilteredRemediationRunbookName + version: '1.1.1.0' + description: 'Remediates long-deallocated VMs recommendations given fit and tag filters' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/remediations/${longDeallocatedVMsFilteredRemediationRunbookName}.ps1') + } + { + name: unattachedDisksFilteredRemediationRunbookName + version: '1.1.1.0' + description: 'Remediates unattached disks recommendations given fit and tag filters' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/remediations/${unattachedDisksFilteredRemediationRunbookName}.ps1') + } + { + name: cleanUpOlderRecommendationsRunbookName + version: '1.1.0.0' + description: 'Cleans up older recommendations from SQL Database' + type: 'PowerShell' + scriptUri: uri(templateLocation, 'runbooks/maintenance/${cleanUpOlderRecommendationsRunbookName}.ps1') + } +] +var automationVariables = [ + { + name: 'AzureOptimization_CloudEnvironment' + description: 'Azure Cloud environment (for example, AzureCloud, AzureChinaCloud, etc.)' + value: '"${cloudEnvironment}"' + } + { + name: 'AzureOptimization_AuthenticationOption' + description: 'Runbook authentication type (RunAsAccount or ManagedIdentity)' + value: '"${authenticationOption}"' + } + { + name: 'AzureOptimization_StorageSink' + description: 'The Azure Storage Account where data source exports are dumped to' + value: '"${storageAccountName}"' + } + { + name: 'AzureOptimization_ConsumptionOffsetDays' + description: 'The offset (in days) for querying for consumption data' + value: 3 + } + { + name: 'AzureOptimization_AdvisorFilter' + description: 'The category filter to use for Azure Advisor (non-Cost) recommendations exports' + value: '"HighAvailability,Security,Performance,OperationalExcellence"' + } + { + name: 'AzureOptimization_ReferenceRegion' + description: 'The Azure region used as a reference for getting details about Azure VM sizes available' + value: '"${projectLocation}"' + } + { + name: 'AzureOptimization_SQLServerDatabase' + description: 'The Azure SQL Database name for the ingestion control and recommendations tables' + value: '"${sqlDatabaseName}"' + } + { + name: 'AzureOptimization_LogAnalyticsChunkSize' + description: 'The size (in rows) for each chunk of Log Analytics ingestion request' + value: 6000 + } + { + name: 'AzureOptimization_StorageBlobsPageSize' + description: 'The size (in blobs count) for each page of Storage Account container blob listing' + value: 1000 + } + { + name: 'AzureOptimization_SQLServerInsertSize' + description: 'The size (in inserted lines) for each page of recommendations ingestion into the SQL Database' + value: 900 + } + { + name: 'AzureOptimization_LogAnalyticsLogPrefix' + description: 'The prefix for all Azure Optimization custom log tables in Log Analytics' + value: '"AzureOptimization"' + } + { + name: 'AzureOptimization_LogAnalyticsWorkspaceName' + description: 'The Log Analytics Workspace Name where optimization data will be ingested' + value: '"${logAnalyticsWorkspaceName}"' + } + { + name: 'AzureOptimization_LogAnalyticsWorkspaceRG' + description: 'The resource group for the Log Analytics Workspace where optimization data will be ingested' + value: '"${((!logAnalyticsReuse) ? resourceGroup().name : logAnalyticsWorkspaceRG)}"' + } + { + name: 'AzureOptimization_LogAnalyticsWorkspaceSubId' + description: 'The Azure subscription for the Log Analytics Workspace where optimization data will be ingested' + value: '"${subscription().subscriptionId}"' + } + { + name: 'AzureOptimization_LogAnalyticsWorkspaceTenantId' + description: 'The Microsoft Entra tenant for the Log Analytics Workspace where optimization data will be ingested' + value: '"${subscription().tenantId}"' + } + { + name: 'AzureOptimization_PriceSheetMeterCategories' + description: 'Comma-separated meter categories to be included in the Price Sheet (remove variable to include all categories)' + value: '"Virtual Machines,Storage"' + } + { + name: 'AzureOptimization_RetailPricesCurrencyCode' + description: 'The currency code to be used for the retail prices exports (used for Reservations prices)' + value: '"EUR"' + } + { + name: 'AzureOptimization_RecommendAdvisorPeriodInDays' + description: 'The period (in days) to look back for Advisor exported recommendations' + value: 7 + } + { + name: 'AzureOptimization_RecommendationLongDeallocatedVmsIntervalDays' + description: 'The period (in days) for considering a VM long deallocated' + value: 30 + } + { + name: 'AzureOptimization_PerfPercentileCpu' + description: 'The percentile to be used for processor metrics' + value: 99 + } + { + name: 'AzureOptimization_PerfPercentileMemory' + description: 'The percentile to be used for memory metrics' + value: 99 + } + { + name: 'AzureOptimization_PerfPercentileNetwork' + description: 'The percentile to be used for network metrics' + value: 99 + } + { + name: 'AzureOptimization_PerfPercentileDisk' + description: 'The percentile to be used for disk metrics' + value: 99 + } + { + name: 'AzureOptimization_PerfPercentileSqlDtu' + description: 'The percentile to be used for SQL DB DTU metrics' + value: 99 + } + { + name: 'AzureOptimization_PerfThresholdCpuPercentage' + description: 'The processor usage percentage threshold above which the fit score is decreased or below which the instance is considered underutilized' + value: 30 + } + { + name: 'AzureOptimization_PerfThresholdMemoryPercentage' + description: 'The memory usage percentage threshold above which the fit score is decreased or below which the instance is considered underutilized' + value: 50 + } + { + name: 'AzureOptimization_PerfThresholdCpuDegradedMaxPercentage' + description: 'The maximum processor usage percentage threshold above which the instance is considered degraded' + value: 95 + } + { + name: 'AzureOptimization_PerfThresholdCpuDegradedAvgPercentage' + description: 'The average processor usage percentage threshold above which the instance is considered degraded' + value: 75 + } + { + name: 'AzureOptimization_PerfThresholdMemoryDegradedPercentage' + description: 'The memory usage percentage threshold above which the instance is considered degraded' + value: 90 + } + { + name: 'AzureOptimization_PerfThresholdNetworkMbps' + description: 'The network usage threshold (in Mbps) above which the fit score is decreased' + value: 750 + } + { + name: 'AzureOptimization_PerfThresholdCpuShutdownPercentage' + description: 'The processor usage percentage threshold above which the fit score is decreased (shutdown scenarios)' + value: 5 + } + { + name: 'AzureOptimization_PerfThresholdMemoryShutdownPercentage' + description: 'The memory usage percentage threshold above which the fit score is decreased (shutdown scenarios)' + value: 100 + } + { + name: 'AzureOptimization_PerfThresholdNetworkShutdownMbps' + description: 'The network usage threshold (in Mbps) above which the fit score is decreased (shutdown scenarios)' + value: 10 + } + { + name: 'AzureOptimization_PerfThresholdDtuPercentage' + description: 'The DTU usage percentage threshold below which a SQL Database instance is considered underutilized' + value: 40 + } + { + name: 'AzureOptimization_PerfThresholdDtuDegradedPercentage' + description: 'The DTU usage percentage threshold above which a SQL Database instance is considered performance degraded' + value: 75 + } + { + name: 'AzureOptimization_PerfThresholdDiskIOPSPercentage' + description: 'The IOPS usage percentage threshold below which a Disk is considered underutilized' + value: 5 + } + { + name: 'AzureOptimization_PerfThresholdDiskMBsPercentage' + description: 'The throughput (MBps) usage percentage threshold below which a Disk is considered underutilized' + value: 5 + } + { + name: 'AzureOptimization_RemediateRightSizeMinFitScore' + description: 'The minimum fit score for right-size remediation' + value: '"5.0"' + } + { + name: 'AzureOptimization_RemediateRightSizeMinWeeksInARow' + description: 'The minimum number of weeks in a row required for a right-size recommendation to be remediated' + value: 4 + } + { + name: 'AzureOptimization_RecommendationAdvisorCostRightSizeId' + description: 'The Azure Advisor VM right-size recommendation ID' + value: '"e10b1381-5f0a-47ff-8c7b-37bd13d7c974"' + } + { + name: 'AzureOptimization_RemediateLongDeallocatedVMsMinFitScore' + description: 'The minimum fit score for long-deallocated VM remediation' + value: '"5.0"' + } + { + name: 'AzureOptimization_RemediateLongDeallocatedVMsMinWeeksInARow' + description: 'The minimum number of weeks in a row required for a long-deallocated VM recommendation to be remediated' + value: 4 + } + { + name: 'AzureOptimization_RecommendationLongDeallocatedVMsId' + description: 'The long deallocated VM recommendation ID' + value: '"c320b790-2e58-452a-aa63-7b62c383ad8a"' + } + { + name: 'AzureOptimization_RemediateUnattachedDisksMinFitScore' + description: 'The minimum fit score for unattached disk remediation' + value: '"5.0"' + } + { + name: 'AzureOptimization_RemediateUnattachedDisksMinWeeksInARow' + description: 'The minimum number of weeks in a row required for a unattached disk recommendation to be remediated' + value: 4 + } + { + name: 'AzureOptimization_RemediateUnattachedDisksAction' + description: 'The action for the unattached disk recommendation to be remediated (Delete or Downsize)' + value: '"Delete"' + } + { + name: 'AzureOptimization_RecommendationUnattachedDisksId' + description: 'The unattached disk recommendation ID' + value: '"c84d5e86-e2d6-4d62-be7c-cecfbd73b0db"' + } + { + name: 'AzureOptimization_RecommendationAADMinCredValidityDays' + description: 'The minimum validity of an AAD Object credential in days' + value: 30 + } + { + name: 'AzureOptimization_RecommendationAADMaxCredValidityYears' + description: 'The maximum validity of an AAD Object credential in years' + value: 2 + } + { + name: 'AzureOptimization_AADObjectsFilter' + description: 'The Microsoft Entra object types to export' + value: '"Application,ServicePrincipal,User,Group"' + } + { + name: 'AzureOptimization_RecommendationRBACAssignmentsPercentageThreshold' + description: 'The percentage threshold (used to trigger recommendations) for total RBAC assignments limits' + value: 80 + } + { + name: 'AzureOptimization_RecommendationResourceGroupsPerSubPercentageThreshold' + description: 'The percentage threshold (used to trigger recommendations) for resource group count limits' + value: 80 + } + { + name: 'AzureOptimization_RecommendationVNetSubnetMaxUsedPercentageThreshold' + description: 'The percentage threshold (used to trigger recommendations) for maximum subnet address space usage' + value: 80 + } + { + name: 'AzureOptimization_RecommendationVNetSubnetMinUsedPercentageThreshold' + description: 'The percentage threshold (used to trigger recommendations) for minimum subnet address space usage' + value: 5 + } + { + name: 'AzureOptimization_RecommendationVNetSubnetEmptyMinAgeInDays' + description: 'The minimum age (in days) for an empty subnet to trigger an NSG rule recommendation' + value: 30 + } + { + name: 'AzureOptimization_RecommendationsMaxAgeInDays' + description: 'The maximum age (in days) for a recommendation to be kept in the SQL database' + value: 365 + } + { + name: 'AzureOptimization_RecommendationStorageAcountGrowthThresholdPercentage' + description: 'The minimum Storage Account growth percentage required to flag Storage as not having a retention policy in place' + value: 5 + } + { + name: 'AzureOptimization_RecommendationStorageAcountGrowthMonthlyCostThreshold' + description: 'The minimum monthly cost (in your EA/MCA currency) required to flag Storage as not having a retention policy in place' + value: 50 + } + { + name: 'AzureOptimization_RecommendationStorageAcountGrowthLookbackDays' + description: 'The lookback period (in days) for analyzing Storage Account growth' + value: 30 + } +] + +//------------------------------------------------------------------------------ +// Telemetry +// Used to anonymously count the number of times the template has been deployed +// and to track and fix deployment bugs to ensure the highest quality. +// No information about you or your cost data is collected. +//------------------------------------------------------------------------------ + +resource defaultTelemetry 'Microsoft.Resources/deployments@2022-09-01' = if (enableDefaultTelemetry) { + name: 'pid-${telemetryId}-${uniqueString(deployment().name, projectLocation)}' + properties: { + mode: 'Incremental' + template: { + '$schema': 'https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#' + contentVersion: '1.0.0.0' + metadata: { + _generator: { + name: 'FinOps toolkit' + version: finOpsToolkitVersion + } + } + resources: [] + } + } +} + +resource logAnalyticsWorkspace 'microsoft.operationalinsights/workspaces@2020-08-01' = if (!logAnalyticsReuse) { + name: logAnalyticsWorkspaceName + location: projectLocation + tags: resourceTags + properties: { + sku: { + name: 'pergb2018' + } + retentionInDays: logAnalyticsRetentionDays + } +} + +resource storageAccount 'Microsoft.Storage/storageAccounts@2022-09-01' = { + name: storageAccountName + location: projectLocation + tags: resourceTags + sku: { + name: 'Standard_LRS' + } + kind: 'StorageV2' + properties: { + allowBlobPublicAccess: false + networkAcls: { + bypass: 'AzureServices' + virtualNetworkRules: [] + ipRules: [] + defaultAction: 'Allow' + } + supportsHttpsTrafficOnly: true + encryption: { + services: { + file: { + enabled: true + } + blob: { + enabled: true + } + } + keySource: 'Microsoft.Storage' + } + minimumTlsVersion: 'TLS1_2' + accessTier: 'Cool' + } +} + +resource storageBlobServices 'Microsoft.Storage/storageAccounts/blobServices@2022-09-01' = { + parent: storageAccount + name: 'default' + properties: { + cors: { + corsRules: [] + } + deleteRetentionPolicy: { + enabled: false + } + } +} + +resource storageCsvExportsContainers 'Microsoft.Storage/storageAccounts/blobServices/containers@2022-09-01' = [for item in csvExports: { + name: '${storageAccountName}/default/${item.containerName}' + properties: { + publicAccess: 'None' + } + dependsOn: [ + storageBlobServices + storageAccount + ] +}] + +resource storageRecommendationsContainer 'Microsoft.Storage/storageAccounts/blobServices/containers@2022-09-01' = { + name: '${storageAccountName}/default/${recommendationsContainerName}' + properties: { + publicAccess: 'None' + } + dependsOn: [ + storageBlobServices + storageAccount + ] +} + +resource storageRemediationLogsContainer 'Microsoft.Storage/storageAccounts/blobServices/containers@2022-09-01' = { + name: '${storageAccountName}/default/${remediationLogsContainerName}' + properties: { + publicAccess: 'None' + } + dependsOn: [ + storageBlobServices + storageAccount + ] +} + +resource storageLifecycleManagementPolicy 'Microsoft.Storage/storageAccounts/managementPolicies@2021-02-01' = { + parent: storageAccount + name: 'default' + properties: { + policy: { + rules: [ + { + enabled: true + name: 'Clean6MonthsOldBlobs' + type: 'Lifecycle' + definition: { + actions: { + baseBlob: { + delete: { + daysAfterModificationGreaterThan: 180 + } + } + snapshot: { + delete: { + daysAfterCreationGreaterThan: 180 + } + } + version: { + delete: { + daysAfterCreationGreaterThan: 180 + } + } + } + filters: { + blobTypes: [ + 'blockBlob' + ] + } + } + } + ] + } + } + dependsOn: [ + storageBlobServices + ] +} + +resource existingSqlServer 'Microsoft.Sql/servers@2022-05-01-preview' existing = if (sqlServerAlreadyExists) { + name: sqlServerName +} + +resource sqlServer 'Microsoft.Sql/servers@2022-05-01-preview' = if (!sqlServerAlreadyExists) { + name: sqlServerName + location: projectLocation + tags: resourceTags + properties: { + version: '12.0' + publicNetworkAccess: 'Enabled' + minimalTlsVersion: '1.2' + administrators: { + administratorType: 'ActiveDirectory' + azureADOnlyAuthentication: true + login: userPrincipalName + principalType: sqlAdminPrincipalType + sid: userObjectId + tenantId: tenant().tenantId + } + } +} + +resource sqlAdminsResource 'Microsoft.Sql/servers/administrators@2022-05-01-preview' = if (sqlServerAlreadyExists) { + parent: existingSqlServer + name: 'ActiveDirectory' + properties: { + administratorType: 'ActiveDirectory' + login: userPrincipalName + sid: userObjectId + tenantId: tenant().tenantId + } +} + +resource sqlAzureAdOnly 'Microsoft.Sql/servers/azureADOnlyAuthentications@2022-05-01-preview' = if (sqlServerAlreadyExists) { + name: 'Default' + parent: existingSqlServer + properties: { + azureADOnlyAuthentication: true + } + dependsOn:[sqlAdminsResource] +} + +resource sqlServerFirewall 'Microsoft.Sql/servers/firewallRules@2022-05-01-preview' = { + parent: sqlServer + name: 'AllowAllWindowsAzureIps' + properties: { + endIpAddress: '0.0.0.0' + startIpAddress: '0.0.0.0' + } +} + +resource sqlDatabase 'Microsoft.Sql/servers/databases@2022-05-01-preview' = { + parent: sqlServer + name: sqlDatabaseName + location: projectLocation + tags: resourceTags + sku: { + name: 'Basic' + tier: 'Basic' + capacity: 5 + } + properties: { + collation: 'SQL_Latin1_General_CP1_CI_AS' + maxSizeBytes: 2147483648 + catalogCollation: 'SQL_Latin1_General_CP1_CI_AS' + zoneRedundant: false + readScale: 'Disabled' + autoPauseDelay: 60 + requestedBackupStorageRedundancy: 'Local' + } +} + +resource sqlServerName_sqlDatabaseName_default 'Microsoft.Sql/servers/databases/backupShortTermRetentionPolicies@2022-05-01-preview' = { + name: '${sqlServerName}/${sqlDatabaseName}/default' + properties: { + retentionDays: sqlBackupRetentionDays + } + dependsOn: [ + sqlDatabase + sqlServer + ] +} + +resource automationAccount 'Microsoft.Automation/automationAccounts@2020-01-13-preview' = { + name: automationAccountName + location: projectLocation + tags: resourceTags + identity: { + type: 'SystemAssigned' + } + properties: { + sku: { + name: 'Basic' + } + } +} + +resource automationModule_Az_Accounts 'Microsoft.Automation/automationAccounts/modules@2020-01-13-preview' = { + parent: automationAccount + name: Az_Accounts.name + tags: resourceTags + properties: { + contentLink: { + uri: Az_Accounts.url + } + } +} + +resource automationModule_Microsoft_Graph_Authentication 'Microsoft.Automation/automationAccounts/modules@2020-01-13-preview' = { + parent: automationAccount + name: Microsoft_Graph_Authentication.name + tags: resourceTags + properties: { + contentLink: { + uri: Microsoft_Graph_Authentication.url + } + } +} + +resource automationModule_All 'Microsoft.Automation/automationAccounts/modules@2020-01-13-preview' = [for item in psModules: { + parent: automationAccount + name: item.name + tags: resourceTags + properties: { + contentLink: { + uri: item.url + } + } + dependsOn: [ + automationModule_Az_Accounts + automationModule_Microsoft_Graph_Authentication + ] +}] + +resource automationRunbooks 'Microsoft.Automation/automationAccounts/runbooks@2020-01-13-preview' = [for item in runbooks: { + parent: automationAccount + name: item.name + tags: resourceTags + location: projectLocation + properties: { + runbookType: item.type + logProgress: false + logVerbose: false + description: item.description + publishContentLink: { + uri: item.scriptUri + version: item.version + } + } + dependsOn: [ + automationModule_All + ] +}] + +resource automationVariablesAll 'Microsoft.Automation/automationAccounts/variables@2020-01-13-preview' = [for item in automationVariables: { + parent: automationAccount + name: item.name + properties: { + description: item.description + value: item.value + } +}] + +resource automationVariables_csvExports 'Microsoft.Automation/automationAccounts/variables@2020-01-13-preview' = [for item in csvExports: { + parent: automationAccount + name: item.variableName + properties: { + description: item.variableDescription + value: '"${item.containerName}"' + } +}] + +resource automationVariables_SQLServerHostname 'Microsoft.Automation/automationAccounts/variables@2020-01-13-preview' = { + parent: automationAccount + name: 'AzureOptimization_SQLServerHostname' + properties: { + description: 'The Azure SQL Server hostname for the ingestion control and recommendations tables' + value: '"${sqlServer.properties.fullyQualifiedDomainName}"' + } +} + +resource automationVariables_LogAnalyticsWorkspaceId 'Microsoft.Automation/automationAccounts/variables@2020-01-13-preview' = { + parent: automationAccount + name: 'AzureOptimization_LogAnalyticsWorkspaceId' + properties: { + description: 'The Log Analytics Workspace ID where optimization data will be ingested' + value: '"${reference(((!logAnalyticsReuse) ? logAnalyticsWorkspace.id : resourceId(logAnalyticsWorkspaceRG, 'microsoft.operationalinsights/workspaces', logAnalyticsWorkspaceName)), '2020-08-01').customerId}"' + } +} + +resource automationVariables_LogAnalyticsWorkspaceKey 'Microsoft.Automation/automationAccounts/variables@2020-01-13-preview' = { + parent: automationAccount + name: 'AzureOptimization_LogAnalyticsWorkspaceKey' + properties: { + description: 'The shared key for the Log Analytics Workspace where optimization data will be ingested' + value: '"${listKeys(((!logAnalyticsReuse) ? logAnalyticsWorkspace.id : resourceId(logAnalyticsWorkspaceRG, 'microsoft.operationalinsights/workspaces', logAnalyticsWorkspaceName)), '2020-08-01').primarySharedKey}"' + isEncrypted: true + } +} + +resource automationSchedules_csvExports 'Microsoft.Automation/automationAccounts/schedules@2020-01-13-preview' = [for item in csvExportsSchedules: { + parent: automationAccount + name: item.exportSchedule + properties: { + description: item.exportDescription + expiryTime: '9999-12-31T17:59:00-06:00' + startTime: dateTimeAdd(baseTime, item.exportTimeOffset) + interval: 1 + frequency: item.exportFrequency + } +}] + +resource automationSchedules_csvIngests 'Microsoft.Automation/automationAccounts/schedules@2020-01-13-preview' = [for item in csvExports: { + parent: automationAccount + name: item.ingestSchedule + properties: { + description: item.ingestDescription + expiryTime: '9999-12-31T17:59:00-06:00' + startTime: dateTimeAdd(baseTime, item.ingestTimeOffset) + interval: 1 + frequency: item.ingestFrequency + } +}] + +resource automationSchedules_remediationCsvIngest 'Microsoft.Automation/automationAccounts/schedules@2020-01-13-preview' = { + parent: automationAccount + name: remediationLogsIngestScheduleName + properties: { + description: 'Starts the daily Remediation Logs ingests' + expiryTime: '9999-12-31T17:59:00-06:00' + startTime: dateTimeAdd(baseTime, 'PT1H30M') + interval: 1 + frequency: 'Day' + } +} + +resource automationSchedules_recommendationsExport 'Microsoft.Automation/automationAccounts/schedules@2020-01-13-preview' = { + parent: automationAccount + name: recommendationsScheduleName + properties: { + description: 'Starts the weekly Recommendations generation' + expiryTime: '9999-12-31T17:59:00-06:00' + startTime: dateTimeAdd(baseTime, 'PT2H30M') + interval: 1 + frequency: 'Week' + } +} + +resource automationSchedules_recommendationsIngest 'Microsoft.Automation/automationAccounts/schedules@2020-01-13-preview' = { + parent: automationAccount + name: recommendationsIngestScheduleName + properties: { + description: 'Starts the weekly Recommendations ingests' + expiryTime: '9999-12-31T17:59:00-06:00' + startTime: dateTimeAdd(baseTime, 'PT3H30M') + interval: 1 + frequency: 'Week' + } +} + +resource automationSchedules_suppressionsIngest 'Microsoft.Automation/automationAccounts/schedules@2020-01-13-preview' = { + parent: automationAccount + name: suppressionsIngestScheduleName + properties: { + description: 'Starts the weekly Suppressions ingests' + expiryTime: '9999-12-31T17:59:00-06:00' + startTime: dateTimeAdd(baseTime, 'PT3H00M') + interval: 1 + frequency: 'Week' + } +} + +resource automationSchedules_recommendationsCleanUp 'Microsoft.Automation/automationAccounts/schedules@2020-01-13-preview' = { + parent: automationAccount + name: recommendationsCleanUpScheduleName + properties: { + description: 'Starts the weekly Recommendations cleanup' + expiryTime: '9999-12-31T17:59:00-06:00' + startTime: dateTimeAdd(baseTime, 'P6D') + interval: 1 + frequency: 'Week' + } +} + +resource automationJobSchedules_csvExports 'Microsoft.Automation/automationAccounts/jobSchedules@2020-01-13-preview' = [for item in csvExports: if (!item.isOneToMany) { + parent: automationAccount + name: item.exportJobId + properties: { + schedule: { + name: item.exportSchedule + } + runbook: { + name: item.runbookName + } + } + dependsOn: [ + automationSchedules_csvExports + automationModule_All + automationRunbooks + ] +}] + +resource automationJobSchedules_csvParameterizedExports 'Microsoft.Automation/automationAccounts/jobSchedules@2020-01-13-preview' = [for item in csvParameterizedExports: { + parent: automationAccount + name: item.exportJobId + properties: { + schedule: { + name: item.exportSchedule + } + runbook: { + name: item.runbookName + } + parameters: item.parameters + } + dependsOn: [ + automationSchedules_csvExports + automationModule_All + automationRunbooks + ] +}] + +resource automationJobSchedules_csvIngests 'Microsoft.Automation/automationAccounts/jobSchedules@2020-01-13-preview' = [for item in csvExports: { + parent: automationAccount + name: item.ingestJobId + properties: { + schedule: { + name: item.ingestSchedule + } + runbook: { + name: csvIngestRunbookName + } + parameters: { + StorageSinkContainer: item.containerName + } + } + dependsOn: [ + automationSchedules_csvIngests + automationModule_All + automationRunbooks + ] +}] + +resource automationJobSchedules_remediationLogsIngests 'Microsoft.Automation/automationAccounts/jobSchedules@2020-01-13-preview' = { + parent: automationAccount + name: remediationLogsIngestJobId + properties: { + schedule: { + name: remediationLogsIngestScheduleName + } + runbook: { + name: csvIngestRunbookName + } + parameters: { + StorageSinkContainer: remediationLogsContainerName + } + } + dependsOn: [ + automationSchedules_remediationCsvIngest + automationModule_All + automationRunbooks + ] +} + +resource automationJobSchedules_recommendationsExports 'Microsoft.Automation/automationAccounts/jobSchedules@2020-01-13-preview' = [for item in recommendations: { + parent: automationAccount + name: item.recommendationJobId + properties: { + schedule: { + name: recommendationsScheduleName + } + runbook: { + name: item.runbookName + } + } + dependsOn: [ + automationSchedules_recommendationsExport + automationModule_All + automationRunbooks + ] +}] + +resource automationJobSchedules_recommendationsIngests 'Microsoft.Automation/automationAccounts/jobSchedules@2020-01-13-preview' = { + parent: automationAccount + name: recommendationsIngestJobId + properties: { + schedule: { + name: recommendationsIngestScheduleName + } + runbook: { + name: recommendationsIngestRunbookName + } + } + dependsOn: [ + automationSchedules_recommendationsIngest + automationModule_All + automationRunbooks + ] +} + +resource automationJobSchedules_recommendationsLogAnalyticsIngest 'Microsoft.Automation/automationAccounts/jobSchedules@2020-01-13-preview' = { + parent: automationAccount + name: recommendationsLogAnalyticsIngestJobId + properties: { + schedule: { + name: recommendationsIngestScheduleName + } + runbook: { + name: recommendationsLogAnalyticsIngestRunbookName + } + } + dependsOn: [ + automationSchedules_recommendationsIngest + automationModule_All + automationRunbooks + ] +} + +resource automationJobSchedules_suppressionsLogAnalyticsIngest 'Microsoft.Automation/automationAccounts/jobSchedules@2020-01-13-preview' = { + parent: automationAccount + name: suppressionsLogAnalyticsIngestJobId + properties: { + schedule: { + name: suppressionsIngestScheduleName + } + runbook: { + name: suppressionsLogAnalyticsIngestRunbookName + } + } + dependsOn: [ + automationSchedules_suppressionsIngest + automationModule_All + automationRunbooks + ] +} + +resource automationJobSchedules_recommendationsCleanUp 'Microsoft.Automation/automationAccounts/jobSchedules@2020-01-13-preview' = { + parent: automationAccount + name: recommendationsCleanUpJobId + properties: { + schedule: { + name: recommendationsCleanUpScheduleName + } + runbook: { + name: cleanUpOlderRecommendationsRunbookName + } + } + dependsOn: [ + automationSchedules_recommendationsCleanUp + automationModule_All + automationRunbooks + ] +} + +resource contributorRoleAssignmentGuid_resource 'Microsoft.Authorization/roleAssignments@2018-09-01-preview' = { + name: contributorRoleAssignmentGuid + properties: { + roleDefinitionId: roleContributor + principalId: reference(automationAccount.id, '2019-06-01', 'Full').identity.principalId + principalType: 'ServicePrincipal' + } +} + +output automationPrincipalId string = reference(automationAccount.id, '2019-06-01', 'Full').identity.principalId diff --git a/docs/deploy/optimization-engine/13.0/azuredeploy.bicep b/docs/deploy/optimization-engine/13.0/azuredeploy.bicep new file mode 100644 index 000000000..0a09fce3b --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/azuredeploy.bicep @@ -0,0 +1,82 @@ +targetScope = 'subscription' +param rgName string +param readerRoleAssignmentGuid string = guid(subscription().subscriptionId, rgName) +param contributorRoleAssignmentGuid string = guid(rgName) +param projectLocation string + +@description('The base URI where artifacts required by this template are located') +param templateLocation string + +param storageAccountName string +param automationAccountName string +param sqlServerName string +param sqlServerAlreadyExists bool = false +param sqlDatabaseName string = 'azureoptimization' +param logAnalyticsReuse bool +param logAnalyticsWorkspaceName string +param logAnalyticsWorkspaceRG string +param logAnalyticsRetentionDays int = 120 +param sqlBackupRetentionDays int = 7 +param userPrincipalName string +param userObjectId string +param sqlAdminPrincipalType string = 'User' +param cloudEnvironment string = 'AzureCloud' +param authenticationOption string = 'ManagedIdentity' + +@description('Base time for all automation runbook schedules.') +param baseTime string = utcNow('u') +param resourceTags object + +param roleReader string = '/subscriptions/${subscription().subscriptionId}/providers/Microsoft.Authorization/roleDefinitions/acdd72a7-3385-48ef-bd42-f606fba81ae7' + +@description('Optional. Enable telemetry to track anonymous module usage trends, monitor for bugs, and improve future releases.') +param enableDefaultTelemetry bool = true + +resource rg 'Microsoft.Resources/resourceGroups@2021-04-01' = { + name: rgName + location: projectLocation + tags: resourceTags + dependsOn: [] +} + +module resourcesDeployment './azuredeploy-nested.bicep' = { + name: 'resourcesDeployment' + scope: resourceGroup(rgName) + params: { + projectLocation: projectLocation + templateLocation: templateLocation + storageAccountName: storageAccountName + automationAccountName: automationAccountName + sqlServerName: sqlServerName + sqlServerAlreadyExists: sqlServerAlreadyExists + sqlDatabaseName: sqlDatabaseName + logAnalyticsReuse: logAnalyticsReuse + logAnalyticsWorkspaceName: logAnalyticsWorkspaceName + logAnalyticsWorkspaceRG: logAnalyticsWorkspaceRG + logAnalyticsRetentionDays: logAnalyticsRetentionDays + sqlBackupRetentionDays: sqlBackupRetentionDays + cloudEnvironment: cloudEnvironment + authenticationOption: authenticationOption + baseTime: baseTime + contributorRoleAssignmentGuid: contributorRoleAssignmentGuid + resourceTags: resourceTags + userPrincipalName: userPrincipalName + userObjectId: userObjectId + sqlAdminPrincipalType: sqlAdminPrincipalType + enableDefaultTelemetry: enableDefaultTelemetry + } + dependsOn: [ + rg + ] +} + +resource readerRoleAssignmentGuid_resource 'Microsoft.Authorization/roleAssignments@2018-09-01-preview' = { + name: readerRoleAssignmentGuid + properties: { + roleDefinitionId: roleReader + principalId: resourcesDeployment.outputs.automationPrincipalId + principalType: 'ServicePrincipal' + } +} + +output automationPrincipalId string = resourcesDeployment.outputs.automationPrincipalId diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-AADObjectsToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-AADObjectsToBlobStorage.ps1 new file mode 100644 index 000000000..8bc12d759 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-AADObjectsToBlobStorage.ps1 @@ -0,0 +1,519 @@ +param( + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName, + + [Parameter(Mandatory = $false)] + [string] $groupFilter, + + [Parameter(Mandatory = $false)] + [string] $userFilter +) + +$ErrorActionPreference = "Stop" + +function Build-CredObjectWithDates { + param ( + [object] $appObject + ) + + $credObjects = @() + + foreach ($obj in $appObject.KeyCredentials) + { + $credObject = New-Object PSObject -Property @{ + DisplayName = $obj.DisplayName + KeyId = $obj.KeyId + KeyType = $obj.Type + StartDate = (Get-Date($obj.StartDateTime)).ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:00.000Z") + EndDate = (Get-Date($obj.EndDateTime)).ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:00.000Z") + } + $credObjects += $credObject + } + + foreach ($obj in $appObject.PasswordCredentials) + { + $credObject = New-Object PSObject -Property @{ + DisplayName = $obj.DisplayName + KeyId = $obj.KeyId + KeyType = "Password" + StartDate = (Get-Date($obj.StartDateTime)).ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:00.000Z") + EndDate = (Get-Date($obj.EndDateTime)).ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:00.000Z") + } + $credObjects += $credObject + } + + return $credObjects +} + +function Build-PrincipalNames { + param ( + [object] $appObject + ) + + $principalNames = @() + + if ($appObject.Web.HomePageUrl) + { + $principalNames += $appObject.Web.HomePageUrl + } + + foreach ($obj in $appObject.IdentifierUris) + { + $principalNames += $obj + } + + foreach ($obj in $appObject.ServicePrincipalNames) + { + $principalNames += $obj + } + + foreach ($obj in $appObject.AlternativeNames) + { + $principalNames += $obj + } + + return $principalNames +} + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_AADObjectsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "aadobjectsexports" +} + +# Application,ServicePrincipal,User,Group +$aadObjectsFilter = Get-AutomationVariable -Name "AzureOptimization_AADObjectsFilter" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($aadObjectsFilter)) +{ + $aadObjectsFilter = "Application,ServicePrincipal" +} + +$groupFilterVariable = Get-AutomationVariable -Name "AzureOptimization_AADObjectsGroupFilter" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($groupFilter) -and -not([string]::IsNullOrEmpty($groupFilterVariable))) +{ + $groupFilter = $groupFilterVariable +} + +$userFilterVariable = Get-AutomationVariable -Name "AzureOptimization_AADObjectsUserFilter" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($userFilter) -and -not([string]::IsNullOrEmpty($userFilterVariable))) +{ + $userFilter = $userFilterVariable +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +#workaround for https://github.com/microsoftgraph/msgraph-sdk-powershell/issues/888 +$localPath = [System.Environment]::GetFolderPath([System.Environment+SpecialFolder]::UserProfile) +if (-not(get-item "$localPath\.graph\" -ErrorAction SilentlyContinue)) +{ + New-Item -Type Directory "$localPath\.graph" +} + +Import-Module Microsoft.Graph.Authentication +Import-Module Microsoft.Graph.Users +Import-Module Microsoft.Graph.Applications +Import-Module Microsoft.Graph.Groups + +switch ($cloudEnvironment) { + "AzureUSGovernment" { + $graphEnvironment = "USGov" + break + } + "AzureChinaCloud" { + $graphEnvironment = "China" + break + } + "AzureGermanCloud" { + $graphEnvironment = "Germany" + break + } + Default { + $graphEnvironment = "Global" + } +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Microsoft Graph with $externalCredentialName external credential..." + Connect-MgGraph -TenantId $externalTenantId -ClientSecretCredential $externalCredential -Environment $graphEnvironment -NoWelcome +} +else +{ + "Logging in to Microsoft Graph with $authenticationOption..." + + switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-MgGraph -Identity -ClientId $uamiClientID -Environment $graphEnvironment -NoWelcome + break + } + Default { #ManagedIdentity + Connect-MgGraph -Identity -Environment $graphEnvironment -NoWelcome + break + } + } +} + +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +$aadObjectsTypes = $aadObjectsFilter.Split(",") + +$fileDate = $datetime.ToString("yyyyMMdd") + +if ("Application" -in $aadObjectsTypes) +{ + $aadObjects = @() + + "Getting AAD applications..." + $apps = Get-MgApplication -All -ExpandProperty Owners -Property Id,AppId,CreatedDateTime,DeletedDateTime,DisplayName,KeyCredentials,PasswordCredentials,Owners,PublisherDomain,Web,IdentifierUris + "Found $($apps.Count) AAD applications" + + foreach ($app in $apps) + { + $owners = $null + if ($app.Owners.Count -gt 0) + { + $owners = ($app.Owners | Where-Object { [string]::IsNullOrEmpty($_.DeletedDateTime) }).Id | ConvertTo-Json -Compress + } + $createdDate = $null + if ($app.CreatedDateTime) + { + $createdDate = (Get-Date($app.CreatedDateTime)).ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:00.000Z") + } + $deletedDate = $null + if ($app.DeletedDateTime) + { + $deletedDate = (Get-Date($app.DeletedDateTime)).ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:00.000Z") + } + $aadObject = New-Object PSObject -Property @{ + Timestamp = $timestamp + TenantGuid = $tenantId + Cloud = $cloudEnvironment + ObjectId = $app.Id + ObjectType = "Application" + ObjectSubType = "N/A" + DisplayName = $app.DisplayName + SecurityEnabled = "N/A" + ApplicationId = $app.AppId + Keys = (Build-CredObjectWithDates -appObject $app) | ConvertTo-Json -Compress + PrincipalNames = (Build-PrincipalNames -appObject $app) | ConvertTo-Json -Compress + Owners = $owners + CreatedDate = $createdDate + DeletedDate = $deletedDate + } + $aadObjects += $aadObject + } + + $jsonExportPath = "$fileDate-$tenantId-aadobjects-apps.json" + $csvExportPath = "$fileDate-$tenantId-aadobjects-apps.csv" + + $aadObjects | ConvertTo-Json -Depth 3 -Compress | Out-File $jsonExportPath + "Exported to JSON: $($aadObjects.Count) lines" + $aadObjectsJson = Get-Content -Path $jsonExportPath | ConvertFrom-Json + "JSON Import: $($aadObjectsJson.Count) lines" + $aadObjectsJson | Export-Csv -NoTypeInformation -Path $csvExportPath + "Export to $csvExportPath" + + $csvBlobName = $csvExportPath + $csvProperties = @{"ContentType" = "text/csv"}; + + Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + "[$now] Uploaded $csvBlobName to Blob Storage..." + + Remove-Item -Path $csvExportPath -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + "[$now] Removed $csvExportPath from local disk..." + + Remove-Item -Path $jsonExportPath -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + "[$now] Removed $jsonExportPath from local disk..." +} + +if ("ServicePrincipal" -in $aadObjectsTypes) +{ + $aadObjects = @() + + "Getting AAD service principals..." + $spns = Get-MgServicePrincipal -All -ExpandProperty Owners -Property Id,AppId,DeletedDateTime,DisplayName,KeyCredentials,PasswordCredentials,Owners,ServicePrincipalNames,ServicePrincipalType,AccountEnabled,AlternativeNames + "Found $($spns.Count) AAD service principals" + + foreach ($spn in $spns) + { + $owners = $null + if ($spn.Owners.Count -gt 0) + { + $owners = ($spn.Owners | Where-Object { [string]::IsNullOrEmpty($_.DeletedDateTime) }).Id | ConvertTo-Json -Compress + } + $deletedDate = $null + if ($spn.DeletedDateTime) + { + $deletedDate = (Get-Date($spn.DeletedDateTime)).ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:00.000Z") + } + $aadObject = New-Object PSObject -Property @{ + Timestamp = $timestamp + TenantGuid = $tenantId + Cloud = $cloudEnvironment + ObjectId = $spn.Id + ObjectType = "ServicePrincipal" + ObjectSubType = $spn.ServicePrincipalType + DisplayName = $spn.DisplayName + SecurityEnabled = $spn.AccountEnabled + ApplicationId = $spn.AppId + Keys = (Build-CredObjectWithDates -appObject $spn) | ConvertTo-Json -Compress + PrincipalNames = (Build-PrincipalNames -appObject $spn) | ConvertTo-Json -Compress + Owners = $owners + DeletedDate = $deletedDate + } + $aadObjects += $aadObject + } + + $jsonExportPath = "$fileDate-$tenantId-aadobjects-spns.json" + $csvExportPath = "$fileDate-$tenantId-aadobjects-spns.csv" + + $aadObjects | ConvertTo-Json -Depth 3 -Compress | Out-File $jsonExportPath + "Exported to JSON: $($aadObjects.Count) lines" + $aadObjectsJson = Get-Content -Path $jsonExportPath | ConvertFrom-Json + "JSON Import: $($aadObjectsJson.Count) lines" + $aadObjectsJson | Export-Csv -NoTypeInformation -Path $csvExportPath + "Export to $csvExportPath" + + $csvBlobName = $csvExportPath + $csvProperties = @{"ContentType" = "text/csv"}; + + Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + "[$now] Uploaded $csvBlobName to Blob Storage..." + + Remove-Item -Path $csvExportPath -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + "[$now] Removed $csvExportPath from local disk..." + + Remove-Item -Path $jsonExportPath -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + "[$now] Removed $jsonExportPath from local disk..." +} + +if ("User" -in $aadObjectsTypes) +{ + $aadObjects = @() + + if ([string]::IsNullOrEmpty($userFilter)) + { + "Getting AAD users..." + $users = Get-MgUser -All -Property Id,AccountEnabled,DisplayName,UserPrincipalName,UserType,CreatedDateTime,DeletedDateTime + } + else + { + "Getting AAD users with filter $userFilter..." + $users = Get-MgUser -Filter $userFilter -All -Property Id,AccountEnabled,DisplayName,UserPrincipalName,UserType,CreatedDateTime,DeletedDateTime + } + "Found $($users.Count) AAD users" + + foreach ($user in $users) + { + $createdDate = $null + if ($user.CreatedDateTime) + { + $createdDate = (Get-Date($user.CreatedDateTime)).ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:00.000Z") + } + $deletedDate = $null + if ($user.DeletedDateTime) + { + $deletedDate = (Get-Date($user.DeletedDateTime)).ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:00.000Z") + } + $aadObject = New-Object PSObject -Property @{ + Timestamp = $timestamp + TenantGuid = $tenantId + Cloud = $cloudEnvironment + ObjectId = $user.Id + ObjectType = "User" + ObjectSubType = $user.UserType + DisplayName = $user.DisplayName + SecurityEnabled = $user.AccountEnabled + PrincipalNames = $user.UserPrincipalName + CreatedDate = $createdDate + DeletedDate = $deletedDate + } + $aadObjects += $aadObject + } + + $jsonExportPath = "$fileDate-$tenantId-aadobjects-users.json" + $csvExportPath = "$fileDate-$tenantId-aadobjects-users.csv" + + $aadObjects | Export-Csv -NoTypeInformation -Path $csvExportPath + "Export to $csvExportPath" + + $csvBlobName = $csvExportPath + $csvProperties = @{"ContentType" = "text/csv"}; + + Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + "[$now] Uploaded $csvBlobName to Blob Storage..." + + Remove-Item -Path $csvExportPath -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + "[$now] Removed $csvExportPath from local disk..." +} + +if ("Group" -in $aadObjectsTypes) +{ + $aadObjects = @() + + if ([string]::IsNullOrEmpty($groupFilter)) + { + "Getting AAD groups..." + $groups = Get-MgGroup -All -ExpandProperty Members -Property Id,SecurityEnabled,DisplayName,Members,CreatedDateTime,DeletedDateTime,GroupTypes + } + else + { + "Getting AAD groups with filter $groupFilter..." + $groups = Get-MgGroup -Filter $groupFilter -All -ExpandProperty Members -Property Id,SecurityEnabled,DisplayName,Members,CreatedDateTime,DeletedDateTime,GroupTypes + } + "Found $($groups.Count) AAD groups" + + foreach ($group in $groups) + { + $groupMembers = $null + if ($group.Members.Count -gt 0) + { + $groupMembers = $group.Members.Id | ConvertTo-Json -Compress + } + $createdDate = $null + if ($group.CreatedDateTime) + { + $createdDate = (Get-Date($group.CreatedDateTime)).ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:00.000Z") + } + $deletedDate = $null + if ($group.DeletedDateTime) + { + $deletedDate = (Get-Date($group.DeletedDateTime)).ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:00.000Z") + } + $aadObject = New-Object PSObject -Property @{ + Timestamp = $timestamp + TenantGuid = $tenantId + Cloud = $cloudEnvironment + ObjectId = $group.Id + ObjectType = "Group" + ObjectSubType = $group.GroupTypes | ConvertTo-Json -Compress + DisplayName = $group.DisplayName + SecurityEnabled = $group.SecurityEnabled + PrincipalNames = $groupMembers + CreatedDate = $createdDate + DeletedDate = $deletedDate + } + $aadObjects += $aadObject + } + + $jsonExportPath = "$fileDate-$tenantId-aadobjects-groups.json" + $csvExportPath = "$fileDate-$tenantId-aadobjects-groups.csv" + + $aadObjects | ConvertTo-Json -Depth 3 -Compress | Out-File $jsonExportPath + "Exported to JSON: $($aadObjects.Count) lines" + $aadObjectsJson = Get-Content -Path $jsonExportPath | ConvertFrom-Json + "JSON Import: $($aadObjectsJson.Count) lines" + $aadObjectsJson | Export-Csv -NoTypeInformation -Path $csvExportPath + "Export to $csvExportPath" + + $csvBlobName = $csvExportPath + $csvProperties = @{"ContentType" = "text/csv"}; + + Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + "[$now] Uploaded $csvBlobName to Blob Storage..." + + Remove-Item -Path $csvExportPath -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + "[$now] Removed $csvExportPath from local disk..." + + Remove-Item -Path $jsonExportPath -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + "[$now] Removed $jsonExportPath from local disk..." +} + +"DONE!" \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGAppGatewayPropertiesToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGAppGatewayPropertiesToBlobStorage.ps1 new file mode 100644 index 000000000..39e908398 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGAppGatewayPropertiesToBlobStorage.ps1 @@ -0,0 +1,231 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ARGAppGatewayContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "argappgwexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +$allAppGWs = @() + +Write-Output "Getting subscriptions target $TargetSubscription" +if (-not([string]::IsNullOrEmpty($TargetSubscription))) +{ + $subscriptions = $TargetSubscription + $subscriptionSuffix = $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = $cloudSuffix + "all-" + $tenantId +} + +$appGWsTotal = @() +$resultsSoFar = 0 + +Write-Output "Querying for Application Gateways properties" + +$argQuery = @" +resources +| where type =~ 'Microsoft.Network/applicationGateways' +| extend gatewayIPsCount = array_length(properties.gatewayIPConfigurations) +| extend frontendIPsCount = array_length(properties.frontendIPConfigurations) +| extend frontendPortsCount = array_length(properties.frontendPorts) +| extend backendPoolsCount = array_length(properties.backendAddressPools) +| extend httpSettingsCount = array_length(properties.backendHttpSettingsCollection) +| extend httpListenersCount = array_length(properties.httpListeners) +| extend urlPathMapsCount = array_length(properties.urlPathMaps) +| extend requestRoutingRulesCount = array_length(properties.requestRoutingRules) +| extend probesCount = array_length(properties.probes) +| extend rewriteRulesCount = array_length(properties.rewriteRuleSets) +| extend redirectConfsCount = array_length(properties.redirectConfigurations) +| project id, name, resourceGroup, subscriptionId, tenantId, location, zones, skuName = properties.sku.name, skuTier = properties.sku.tier, skuCapacity = properties.sku.capacity, enableHttp2 = properties.enableHttp2, gatewayIPsCount, frontendIPsCount, frontendPortsCount, httpSettingsCount, httpListenersCount, backendPoolsCount, urlPathMapsCount, requestRoutingRulesCount, probesCount, rewriteRulesCount, redirectConfsCount, tags +| join kind=leftouter ( + resources + | where type =~ 'Microsoft.Network/applicationGateways' + | mvexpand backendPools = properties.backendAddressPools + | extend backendIPCount = array_length(backendPools.properties.backendIPConfigurations) + | extend backendAddressesCount = array_length(backendPools.properties.backendAddresses) + | summarize backendIPCount = sum(backendIPCount), backendAddressesCount = sum(backendAddressesCount) by id +) on id +| project-away id1 +| order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $appGWs = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $appGWs = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($appGWs -and $appGWs.GetType().Name -eq "PSResourceGraphResponse") + { + $appGWs = $appGWs.Data + } + $resultsCount = $appGWs.Count + $resultsSoFar += $resultsCount + $appGWsTotal += $appGWs + +} while ($resultsCount -eq $ARGPageSize) + +Write-Output "Found $($appGWsTotal.Count) Application Gateway entries" + +<# + Building CSV entries +#> + +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +foreach ($appGW in $appGWsTotal) +{ + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $appGW.tenantId + SubscriptionGuid = $appGW.subscriptionId + ResourceGroupName = $appGW.resourceGroup.ToLower() + InstanceName = $appGW.name.ToLower() + InstanceId = $appGW.id.ToLower() + SkuName = $appGW.skuName + SkuTier = $appGW.skuTier + SkuCapacity = $appGW.skuCapacity + Location = $appGW.location + Zones = $appGW.zones + EnableHttp2 = $appGW.enableHttp2 + GatewayIPsCount = $appGW.gatewayIPsCount + FrontendIPsCount = $appGW.frontendIPsCount + FrontendPortsCount = $appGW.frontendPortsCount + BackendIPCount = $appGW.backendIPCount + BackendAddressesCount = $appGW.backendAddressesCount + HttpSettingsCount = $appGW.httpSettingsCount + HttpListenersCount = $appGW.httpListenersCount + BackendPoolsCount = $appGW.backendPoolsCount + ProbesCount = $appGW.probesCount + UrlPathMapsCount = $appGW.urlPathMapsCount + RequestRoutingRulesCount = $appGW.requestRoutingRulesCount + RewriteRulesCount = $appGW.rewriteRulesCount + RedirectConfsCount = $appGW.redirectConfsCount + StatusDate = $statusDate + Tags = $appGW.tags + } + + $allAppGWs += $logentry +} + +<# + Actually exporting CSV to Azure Storage +#> + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-appgws-$subscriptionSuffix.csv" + +$allAppGWs | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGAppServicePlanPropertiesToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGAppServicePlanPropertiesToBlobStorage.ps1 new file mode 100644 index 000000000..7e54b8420 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGAppServicePlanPropertiesToBlobStorage.ps1 @@ -0,0 +1,209 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$referenceRegion = Get-AutomationVariable -Name "AzureOptimization_ReferenceRegion" -ErrorAction SilentlyContinue # e.g., westeurope +if ([string]::IsNullOrEmpty($referenceRegion)) +{ + $referenceRegion = "westeurope" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ARGAppServicePlanContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "argappserviceplanexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +$allasp = @() + +Write-Output "Getting subscriptions target $TargetSubscription" +if (-not([string]::IsNullOrEmpty($TargetSubscription))) +{ + $subscriptions = $TargetSubscription + $subscriptionSuffix = $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = $cloudSuffix + "all-" + $tenantId +} + +$aspTotal = @() + +$resultsSoFar = 0 + +Write-Output "Querying for App Service Plan properties" + +$argQuery = @" + resources + | where type =~ 'microsoft.web/serverfarms' + | extend skuName = sku.name, skuTier = sku.tier, skuCapacity = sku.capacity, skuFamily = sku.family, skuSize = sku.size + | extend computeMode = properties.computeMode, zoneRedundant = properties.zoneRedundant + | extend numberOfWorkers = properties.numberOfWorkers, currentNumberOfWorkers = properties.currentNumberOfWorkers, maximumNumberOfWorkers = properties.maximumNumberOfWorkers + | extend numberOfSites = properties.numberOfSites, planName = properties.planName + | order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $asp = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $asp = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($asp -and $asp.GetType().Name -eq "PSResourceGraphResponse") + { + $asp = $asp.Data + } + $resultsCount = $asp.Count + $resultsSoFar += $resultsCount + $aspTotal += $asp + +} while ($resultsCount -eq $ARGPageSize) + +$datetime = (Get-Date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +Write-Output "Building $($aspTotal.Count) App Service Plan entries" + +foreach ($asplan in $aspTotal) +{ + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $asplan.tenantId + SubscriptionGuid = $asplan.subscriptionId + ResourceGroupName = $asplan.resourceGroup.ToLower() + ZoneRedundant = $asplan.zoneRedundant + Location = $asplan.location + AppServicePlanName = $asplan.name.ToLower() + InstanceId = $asplan.id.ToLower() + Kind = $asplan.kind + SkuName = $asplan.skuName + SkuTier = $asplan.skuTier + SkuCapacity = $asplan.skuCapacity + SkuFamily = $asplan.skuFamily + SkuSize = $asplan.skuSize + ComputeMode = $asplan.computeMode + NumberOfWorkers = $asplan.numberOfWorkers + CurrentNumberOfWorkers = $asplan.currentNumberOfWorkers + MaximumNumberOfWorkers = $asplan.maximumNumberOfWorkers + NumberOfSites = $asplan.numberOfSites + PlanName = $asplan.planName + Tags = $asplan.tags + StatusDate = $statusDate + } + + $allasp += $logentry +} + +Write-Output "Uploading CSV to Storage" + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-asp-$subscriptionSuffix.csv" + +$allasp | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGAvailabilitySetPropertiesToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGAvailabilitySetPropertiesToBlobStorage.ps1 new file mode 100644 index 000000000..b8d1cf9c9 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGAvailabilitySetPropertiesToBlobStorage.ps1 @@ -0,0 +1,198 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ARGAvailabilitySetContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "argavailsetexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +$allAvSets = @() + +Write-Output "Getting subscriptions target $TargetSubscription" +if (-not([string]::IsNullOrEmpty($TargetSubscription))) +{ + $subscriptions = $TargetSubscription + $subscriptionSuffix = $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = $cloudSuffix + "all-" + $tenantId +} + +$avSetsTotal = @() +$resultsSoFar = 0 + +Write-Output "Querying for Availability Set properties" + +$argQuery = @" +resources +| where type =~ 'Microsoft.Compute/availabilitySets' +| project id, name, location, resourceGroup, subscriptionId, tenantId, skuName = tostring(sku.name), faultDomains = tostring(properties.platformFaultDomainCount), updateDomains = tostring(properties.platformUpdateDomainCount), vmCount = array_length(properties.virtualMachines), tags, zones +| order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $avSets = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $avSets = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($avSets -and $avSets.GetType().Name -eq "PSResourceGraphResponse") + { + $avSets = $avSets.Data + } + $resultsCount = $avSets.Count + $resultsSoFar += $resultsCount + $avSetsTotal += $avSets + +} while ($resultsCount -eq $ARGPageSize) + +Write-Output "Found $($avSetsTotal.Count) Availability Set entries" + +<# + Building CSV entries +#> + +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +foreach ($avSet in $avSetsTotal) +{ + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $avSet.tenantId + SubscriptionGuid = $avSet.subscriptionId + ResourceGroupName = $avSet.resourceGroup.ToLower() + InstanceName = $avSet.name.ToLower() + InstanceId = $avSet.id.ToLower() + SkuName = $avSet.skuName + Location = $avSet.location + FaultDomains = $avSet.faultDomains + UpdateDomains = $avSet.updateDomains + VmCount = $avSet.vmCount + StatusDate = $statusDate + Tags = $avSet.tags + Zones = $avSet.zones + } + + $allAvSets += $logentry +} + +<# + Actually exporting CSV to Azure Storage +#> + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-availsets-$subscriptionSuffix.csv" + +$allAvSets | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGLoadBalancerPropertiesToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGLoadBalancerPropertiesToBlobStorage.ps1 new file mode 100644 index 000000000..54602655e --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGLoadBalancerPropertiesToBlobStorage.ps1 @@ -0,0 +1,222 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ARGLoadBalancerContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "arglbexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +$allLBs = @() + +Write-Output "Getting subscriptions target $TargetSubscription" +if (-not([string]::IsNullOrEmpty($TargetSubscription))) +{ + $subscriptions = $TargetSubscription + $subscriptionSuffix = $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = $cloudSuffix + "all-" + $tenantId +} + +$LBsTotal = @() +$resultsSoFar = 0 + +Write-Output "Querying for Load Balancer properties" + +$argQuery = @" +resources +| where type =~ 'Microsoft.Network/loadBalancers' +| extend lbType = iif(properties.frontendIPConfigurations contains 'publicIPAddress', 'Public', iif(properties.frontendIPConfigurations contains 'privateIPAddress', 'Internal', 'Unknown')) +| extend lbRulesCount = array_length(properties.loadBalancingRules) +| extend frontendIPsCount = array_length(properties.frontendIPConfigurations) +| extend inboundNatRulesCount = array_length(properties.inboundNatRules) +| extend outboundRulesCount = array_length(properties.outboundRules) +| extend inboundNatPoolsCount = array_length(properties.inboundNatPools) +| extend backendPoolsCount = array_length(properties.backendAddressPools) +| extend probesCount = array_length(properties.probes) +| project id, name, resourceGroup, subscriptionId, tenantId, location, skuName = sku.name, skuTier = sku.tier, lbType, lbRulesCount, frontendIPsCount, inboundNatRulesCount, outboundRulesCount, inboundNatPoolsCount, backendPoolsCount, probesCount, tags +| join kind=leftouter ( + resources + | where type =~ 'Microsoft.Network/loadBalancers' + | mvexpand backendPools = properties.backendAddressPools + | extend backendIPCount = array_length(backendPools.properties.backendIPConfigurations) + | extend backendAddressesCount = array_length(backendPools.properties.loadBalancerBackendAddresses) + | summarize backendIPCount = sum(backendIPCount), backendAddressesCount = sum(backendAddressesCount) by id +) on id +| project-away id1 +| order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $LBs = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $LBs = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($LBs -and $LBs.GetType().Name -eq "PSResourceGraphResponse") + { + $LBs = $LBs.Data + } + $resultsCount = $LBs.Count + $resultsSoFar += $resultsCount + $LBsTotal += $LBs + +} while ($resultsCount -eq $ARGPageSize) + +Write-Output "Found $($LBsTotal.Count) Load Balancer entries" + +<# + Building CSV entries +#> + +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +foreach ($lb in $LBsTotal) +{ + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $lb.tenantId + SubscriptionGuid = $lb.subscriptionId + ResourceGroupName = $lb.resourceGroup.ToLower() + InstanceName = $lb.name.ToLower() + InstanceId = $lb.id.ToLower() + SkuName = $lb.skuName + SkuTier = $lb.skuTier + Location = $lb.location + LbType = $lb.lbType + LbRulesCount = $lb.lbRulesCount + InboundNatRulesCount = $lb.inboundNatRulesCount + OutboundRulesCount = $lb.outboundRulesCount + FrontendIPsCount = $lb.frontendIPsCount + BackendIPCount = $lb.backendIPCount + BackendAddressesCount = $lb.backendAddressesCount + InboundNatPoolsCount = $lb.inboundNatPoolsCount + BackendPoolsCount = $lb.backendPoolsCount + ProbesCount = $lb.probesCount + StatusDate = $statusDate + Tags = $lb.tags + } + + $allLBs += $logentry +} + +<# + Actually exporting CSV to Azure Storage +#> + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-lbs-$subscriptionSuffix.csv" + +$allLBs | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGManagedDisksPropertiesToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGManagedDisksPropertiesToBlobStorage.ps1 new file mode 100644 index 000000000..d6bd5d086 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGManagedDisksPropertiesToBlobStorage.ps1 @@ -0,0 +1,232 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ARGDiskContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "argdiskexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +$alldisks = @() + +Write-Output "Getting subscriptions target $TargetSubscription" +if (-not([string]::IsNullOrEmpty($TargetSubscription))) +{ + $subscriptions = $TargetSubscription + $subscriptionSuffix = $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = $cloudSuffix + "all-" + $tenantId +} + +$mdisksTotal = @() +$resultsSoFar = 0 + +<# + Getting all Managed Disks properties with Azure Resource Graph query +#> + +Write-Output "Querying for ARM Managed Disks properties" + +$argQuery = @" + resources + | where type =~ 'Microsoft.Compute/disks' + | extend DiskId = tolower(id), OwnerVmId = tolower(managedBy) + | join kind=leftouter ( + resources + | where type =~ 'Microsoft.Compute/virtualMachines' and array_length(properties.storageProfile.dataDisks) > 0 + | extend OwnerVmId = tolower(id) + | mv-expand DataDisks = properties.storageProfile.dataDisks + | extend DiskId = tolower(DataDisks.managedDisk.id), diskCaching = tostring(DataDisks.caching), diskType = 'Data' + | project DiskId, OwnerVmId, diskCaching, diskType + | union ( + resources + | where type =~ 'Microsoft.Compute/virtualMachines' + | extend OwnerVmId = tolower(id) + | extend DiskId = tolower(properties.storageProfile.osDisk.managedDisk.id), diskCaching = tostring(properties.storageProfile.osDisk.caching), diskType = 'OS' + | project DiskId, OwnerVmId, diskCaching, diskType + ) + ) on OwnerVmId, DiskId + | project-away OwnerVmId, DiskId, OwnerVmId1, DiskId1 + | order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $mdisks = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $mdisks = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($mdisks -and $mdisks.GetType().Name -eq "PSResourceGraphResponse") + { + $mdisks = $mdisks.Data + } + $resultsCount = $mdisks.Count + $resultsSoFar += $resultsCount + $mdisksTotal += $mdisks + +} while ($resultsCount -eq $ARGPageSize) + +Write-Output "Found $($mdisksTotal.Count) Managed Disk entries" + +<# + Building CSV entries +#> + +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +foreach ($disk in $mdisksTotal) +{ + $ownerVmId = $null + if ($null -ne $disk.managedBy) + { + $ownerVmId = $disk.managedBy.ToLower() + } + + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $disk.tenantId + SubscriptionGuid = $disk.subscriptionId + ResourceGroupName = $disk.resourceGroup.ToLower() + DiskName = $disk.name.ToLower() + InstanceId = $disk.id.ToLower() + Location = $disk.location + OwnerVMId = $ownerVmId + DeploymentModel = "Managed" + DiskType = $disk.diskType + TimeCreated = $disk.properties.timeCreated + DiskIOPS = $disk.properties.diskIOPSReadWrite + DiskThroughput = $disk.properties.diskMBpsReadWrite + DiskTier = $disk.properties.tier + DiskState = $disk.properties.diskState + EncryptionType = $disk.properties.encryption.type + Zones = $disk.zones + Caching = $disk.diskCaching + DiskSizeGB = $disk.properties.diskSizeGB + SKU = $disk.sku.name + StatusDate = $statusDate + Tags = $disk.tags + } + + $alldisks += $logentry +} + +<# + Actually exporting CSV to Azure Storage +#> + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-disks-$subscriptionSuffix.csv" + +$alldisks | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGNICPropertiesToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGNICPropertiesToBlobStorage.ps1 new file mode 100644 index 000000000..547876357 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGNICPropertiesToBlobStorage.ps1 @@ -0,0 +1,235 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$referenceRegion = Get-AutomationVariable -Name "AzureOptimization_ReferenceRegion" -ErrorAction SilentlyContinue # e.g., westeurope +if ([string]::IsNullOrEmpty($referenceRegion)) +{ + $referenceRegion = "westeurope" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ARGNICContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "argnicexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +$allnics = @() + +Write-Output "Getting subscriptions target $TargetSubscription" +if (-not([string]::IsNullOrEmpty($TargetSubscription))) +{ + $subscriptions = $TargetSubscription + $subscriptionSuffix = $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = $cloudSuffix + "all-" + $tenantId +} + +$nicsTotal = @() + +$resultsSoFar = 0 + +Write-Output "Querying for NIC properties" + +$argQuery = @" + resources + | where type =~ 'microsoft.network/networkinterfaces' + | extend isPrimary = properties.primary + | extend enableAcceleratedNetworking = properties.enableAcceleratedNetworking + | extend enableIPForwarding = properties.enableIPForwarding + | extend tapConfigurationsCount = array_length(properties.tapConfigurations) + | extend hostedWorkloadsCount = array_length(properties.hostedWorkloads) + | extend internalDomainNameSuffix = properties.dnsSettings.internalDomainNameSuffix + | extend appliedDnsServers = properties.dnsSettings.appliedDnsServers + | extend dnsServers = properties.dnsSettings.dnsServers + | extend ownerVMId = tolower(properties.virtualMachine.id) + | extend ownerPEId = tolower(properties.privateEndpoint.id) + | extend macAddress = properties.macAddress + | extend nicType = properties.nicType + | extend nicNsgId = tolower(properties.networkSecurityGroup.id) + | mv-expand ipconfigs = properties.ipConfigurations + | project-away properties + | extend privateIPAddressVersion = tostring(ipconfigs.properties.privateIPAddressVersion) + | extend privateIPAllocationMethod = tostring(ipconfigs.properties.privateIPAllocationMethod) + | extend isIPConfigPrimary = tostring(ipconfigs.properties.primary) + | extend privateIPAddress = tostring(ipconfigs.properties.privateIPAddress) + | extend publicIPId = tolower(ipconfigs.properties.publicIPAddress.id) + | extend IPConfigName = tostring(ipconfigs.name) + | extend subnetId = tolower(ipconfigs.properties.subnet.id) + | project-away ipconfigs + | order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $nics = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $nics = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($nics -and $nics.GetType().Name -eq "PSResourceGraphResponse") + { + $nics = $nics.Data + } + $resultsCount = $nics.Count + $resultsSoFar += $resultsCount + $nicsTotal += $nics + +} while ($resultsCount -eq $ARGPageSize) + +$datetime = (Get-Date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +Write-Output "Building $($nicsTotal.Count) ARM VNet nic entries" + +foreach ($nic in $nicsTotal) +{ + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $nic.tenantId + SubscriptionGuid = $nic.subscriptionId + ResourceGroupName = $nic.resourceGroup.ToLower() + Location = $nic.location + Name = $nic.name.ToLower() + InstanceId = $nic.id.ToLower() + IsPrimary = $nic.isPrimary + EnableAcceleratedNetworking = $nic.enableAcceleratedNetworking + EnableIPForwarding = $nic.enableIPForwarding + TapConfigurationsCount = $nic.tapConfigurationsCount + HostedWorkloadsCount = $nic.hostedWorkloadsCount + InternalDomainNameSuffix = $nic.internalDomainNameSuffix + AppliedDnsServers = $nic.appliedDnsServers + DnsServers = $nic.dnsServers + OwnerVMId = $nic.ownerVMId + OwnerPEId = $nic.ownerPEId + MacAddress = $nic.macAddress + NicType = $nic.nicType + NicNSGId = $nic.nicNsgId + PrivateIPAddressVersion = $nic.privateIPAddressVersion + PrivateIPAllocationMethod = $nic.privateIPAllocationMethod + IsIPConfigPrimary = $nic.isIPConfigPrimary + PrivateIPAddress = $nic.privateIPAddress + PublicIPId = $nic.publicIPId + IPConfigName = $nic.IPConfigName + SubnetId = $nic.subnetId + Tags = $nic.tags + StatusDate = $statusDate + } + + $allnics += $logentry +} + +Write-Output "Uploading CSV to Storage" + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-nics-$subscriptionSuffix.csv" + +$allnics | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGNSGPropertiesToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGNSGPropertiesToBlobStorage.ps1 new file mode 100644 index 000000000..ea8b4e6c6 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGNSGPropertiesToBlobStorage.ps1 @@ -0,0 +1,217 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$referenceRegion = Get-AutomationVariable -Name "AzureOptimization_ReferenceRegion" -ErrorAction SilentlyContinue # e.g., westeurope +if ([string]::IsNullOrEmpty($referenceRegion)) +{ + $referenceRegion = "westeurope" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ARGNSGContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "argnsgexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +$allnsgRules = @() + +Write-Output "Getting subscriptions target $TargetSubscription" +if (-not([string]::IsNullOrEmpty($TargetSubscription))) +{ + $subscriptions = $TargetSubscription + $subscriptionSuffix = $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = $cloudSuffix + "all-" + $tenantId +} + +$nsgRulesTotal = @() + +$resultsSoFar = 0 + +Write-Output "Querying for NSG properties" + +$argQuery = @" +resources +| where type =~ 'Microsoft.Network/networkSecurityGroups' +| extend nicCount = iif(isnotempty(properties.networkInterfaces),array_length(properties.networkInterfaces),0) +| extend subnetCount = iif(isnotempty(properties.subnets),array_length(properties.subnets),0) +| mvexpand securityRules = properties.securityRules +| extend ruleName = tolower(securityRules.name) +| extend ruleProtocol = tolower(securityRules.properties.protocol) +| extend ruleDirection = tolower(securityRules.properties.direction) +| extend rulePriority = toint(securityRules.properties.priority) +| extend ruleAccess = tolower(securityRules.properties.access) +| extend ruleDestinationAddresses = tolower(iif(array_length(securityRules.properties.destinationAddressPrefixes) > 0,strcat_array(securityRules.properties.destinationAddressPrefixes, ','),securityRules.properties.destinationAddressPrefix)) +| extend ruleSourceAddresses = tolower(iif(array_length(securityRules.properties.sourceAddressPrefixes) > 0,strcat_array(securityRules.properties.sourceAddressPrefixes, ','),securityRules.properties.sourceAddressPrefix)) +| extend ruleDestinationPorts = iif(array_length(securityRules.properties.destinationPortRanges) > 0,strcat_array(securityRules.properties.destinationPortRanges, ','),securityRules.properties.destinationPortRange) +| extend ruleSourcePorts = iif(array_length(securityRules.properties.sourcePortRanges) > 0,strcat_array(securityRules.properties.sourcePortRanges, ','),securityRules.properties.sourcePortRange) +| extend ruleId = tolower(securityRules.id) +| project-away securityRules, properties +| order by ruleId asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $nsgRules = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $nsgRules = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($nsgRules -and $nsgRules.GetType().Name -eq "PSResourceGraphResponse") + { + $nsgRules = $nsgRules.Data + } + $resultsCount = $nsgRules.Count + $resultsSoFar += $resultsCount + $nsgRulesTotal += $nsgRules + +} while ($resultsCount -eq $ARGPageSize) + +$datetime = (Get-Date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +Write-Output "Building $($nsgRulesTotal.Count) ARM NSG entries" + +foreach ($nsgRule in $nsgRulesTotal) +{ + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $nsgRule.tenantId + SubscriptionGuid = $nsgRule.subscriptionId + ResourceGroupName = $nsgRule.resourceGroup.ToLower() + Location = $nsgRule.location + NSGName = $nsgRule.name.ToLower() + InstanceId = $nsgRule.id.ToLower() + NicCount = $nsgRule.nicCount + SubnetCount = $nsgRule.subnetCount + RuleName = $nsgRule.ruleName + RuleProtocol = $nsgRule.ruleProtocol + RuleDirection = $nsgRule.ruleDirection + RulePriority = $nsgRule.rulePriority + RuleAccess = $nsgRule.ruleAccess + RuleDestinationAddresses = $nsgRule.ruleDestinationAddresses + RuleSourceAddresses = $nsgRule.ruleSourceAddresses + RuleDestinationPorts = $nsgRule.ruleDestinationPorts + RuleSourcePorts = $nsgRule.ruleSourcePorts + Tags = $nsgRule.tags + StatusDate = $statusDate + } + + $allnsgRules += $logentry +} + +Write-Output "Uploading CSV to Storage" + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-nsgrules-$subscriptionSuffix.csv" + +$allnsgRules | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGPublicIpPropertiesToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGPublicIpPropertiesToBlobStorage.ps1 new file mode 100644 index 000000000..ebc543003 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGPublicIpPropertiesToBlobStorage.ps1 @@ -0,0 +1,275 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$referenceRegion = Get-AutomationVariable -Name "AzureOptimization_ReferenceRegion" -ErrorAction SilentlyContinue # e.g., westeurope +if ([string]::IsNullOrEmpty($referenceRegion)) +{ + $referenceRegion = "westeurope" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ARGPublicIpContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "argpublicipexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +$allpips = @() + +Write-Output "Getting subscriptions target $TargetSubscription" +if (-not([string]::IsNullOrEmpty($TargetSubscription))) +{ + $subscriptions = $TargetSubscription + $subscriptionSuffix = $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = $cloudSuffix + "all-" + $tenantId +} + +$pipsTotal = @() + +$resultsSoFar = 0 + +Write-Output "Querying for ARM Public IP properties" + +$argQuery = @" +resources +| where type =~ 'microsoft.network/publicipaddresses' +| extend skuName = tolower(sku.name) +| extend skuTier = tolower(sku.tier) +| extend allocationMethod = tolower(properties.publicIPAllocationMethod) +| extend addressVersion = tolower(properties.publicIPAddressVersion) +| extend associatedResourceId = iif(isnotempty(properties.ipConfiguration.id),tolower(properties.ipConfiguration.id),tolower(properties.natGateway.id)) +| extend ipAddress = tostring(properties.ipAddress) +| extend fqdn = tolower(properties.dnsSettings.fqdn) +| extend publicIpPrefixId = tostring(properties.publicIPPrefix.id) +| order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $pips = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $pips = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($pips -and $pips.GetType().Name -eq "PSResourceGraphResponse") + { + $pips = $pips.Data + } + $resultsCount = $pips.Count + $resultsSoFar += $resultsCount + $pipsTotal += $pips + +} while ($resultsCount -eq $ARGPageSize) + +$datetime = (Get-Date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +Write-Output "Building $($pipsTotal.Count) ARM Public IP entries" + +foreach ($pip in $pipsTotal) +{ + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $pip.tenantId + SubscriptionGuid = $pip.subscriptionId + ResourceGroupName = $pip.resourceGroup.ToLower() + Location = $pip.location + Name = $pip.name.ToLower() + InstanceId = $pip.id.ToLower() + Model = "ARM" + SkuName = $pip.skuName + SkuTier = $pip.skuTier + AllocationMethod = $pip.allocationMethod + AddressVersion = $pip.addressVersion + AssociatedResourceId = $pip.associatedResourceId + PublicIpPrefixId = $pip.publicIpPrefixId + IPAddress = $pip.ipAddress + FQDN = $pip.fqdn + Zones = $pip.zones + Tags = $pip.tags + StatusDate = $statusDate + } + + $allpips += $logentry +} + +$pipsTotal = @() + +$resultsSoFar = 0 + +Write-Output "Querying for Classic Reserved IP properties" + +$argQuery = @" +resources +| where type =~ 'microsoft.classicnetwork/reservedips' +| extend ipAddress = tostring(properties.ipAddress) +| extend allocationMethod = 'static' +| extend addressVersion = 'ipv4' +| extend associatedResourceId = tolower(properties.attachedTo.id) +| extend ipAddress = tostring(properties.ipAddress) +| order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $pips = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $pips = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($pips -and $pips.GetType().Name -eq "PSResourceGraphResponse") + { + $pips = $pips.Data + } + $resultsCount = $pips.Count + $resultsSoFar += $resultsCount + $pipsTotal += $pips + +} while ($resultsCount -eq $ARGPageSize) + +$datetime = (Get-Date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +Write-Output "Building $($pipsTotal.Count) Classic Reserved IP entries" + +foreach ($pip in $pipsTotal) +{ + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $pip.tenantId + SubscriptionGuid = $pip.subscriptionId + ResourceGroupName = $pip.resourceGroup.ToLower() + Location = $pip.location + Name = $pip.name.ToLower() + InstanceId = $pip.id.ToLower() + Model = "Classic" + AllocationMethod = $pip.allocationMethod + AddressVersion = $pip.addressVersion + AssociatedResourceId = $pip.associatedResourceId + IPAddress = $pip.ipAddress + StatusDate = $statusDate + } + + $allpips += $logentry +} + +Write-Output "Uploading CSV to Storage" + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-publicips-$subscriptionSuffix.csv" + +$allpips | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGResourceContainersPropertiesToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGResourceContainersPropertiesToBlobStorage.ps1 new file mode 100644 index 000000000..70be047cb --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGResourceContainersPropertiesToBlobStorage.ps1 @@ -0,0 +1,272 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$referenceRegion = Get-AutomationVariable -Name "AzureOptimization_ReferenceRegion" -ErrorAction SilentlyContinue # e.g., westeurope +if ([string]::IsNullOrEmpty($referenceRegion)) +{ + $referenceRegion = "westeurope" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ARGResourceContainersContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "argrescontainersexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +$allResourceContainers = @() + +Write-Output "Getting subscriptions target $TargetSubscription" +if (-not([string]::IsNullOrEmpty($TargetSubscription))) +{ + $subscriptions = $TargetSubscription + $subscriptionSuffix = $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = $cloudSuffix + "all-" + $tenantId +} + +$rgsTotal = @() +$subsTotal = @() + +$resultsSoFar = 0 + +Write-Output "Querying for resource groups..." + +$argQuery = @" + resourcecontainers + | where type == "microsoft.resources/subscriptions/resourcegroups" + | join kind=leftouter ( + resources + | summarize ResourceCount= count() by subscriptionId, resourceGroup + ) on subscriptionId, resourceGroup + | extend ResourceCount = iif(isempty(ResourceCount), 0, ResourceCount) + | project id, name, type, tenantId, location, subscriptionId, managedBy, tags, properties, ResourceCount + | order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $rgs = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $rgs = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($rgs -and $rgs.GetType().Name -eq "PSResourceGraphResponse") + { + $rgs = $rgs.Data + } + $resultsCount = $rgs.Count + $resultsSoFar += $resultsCount + $rgsTotal += $rgs + +} while ($resultsCount -eq $ARGPageSize) + +$resultsSoFar = 0 + +Write-Output "Querying for subscriptions" + +$argQuery = @" + resourcecontainers + | where type == "microsoft.resources/subscriptions" + | join kind=leftouter ( + resources + | summarize ResourceCount= count() by subscriptionId + ) on subscriptionId + | extend ResourceCount = iif(isempty(ResourceCount), 0, ResourceCount) + | project id, name, type, tenantId, subscriptionId, managedBy, tags, properties, ResourceCount + | order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $subs = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $subs = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($subs -and $subs.GetType().Name -eq "PSResourceGraphResponse") + { + $subs = $subs.Data + } + $resultsCount = $subs.Count + $resultsSoFar += $resultsCount + $subsTotal += $subs + +} while ($resultsCount -eq $ARGPageSize) + +$datetime = (Get-Date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +Write-Output "Building $($rgsTotal.Count) RG entries" + +foreach ($rg in $rgsTotal) +{ + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $rg.tenantId + SubscriptionGuid = $rg.subscriptionId + Location = $rg.location + ContainerType = $rg.type + ContainerName = $rg.name.ToLower() + InstanceId = $rg.id.ToLower() + ResourceCount = $rg.ResourceCount + ManagedBy = $rg.managedBy + ContainerProperties = $rg.properties | ConvertTo-Json -Compress + Tags = $rg.tags + StatusDate = $statusDate + } + + $allResourceContainers += $logentry +} + +Write-Output "Building $($subsTotal.Count) subscription entries" + +foreach ($sub in $subsTotal) +{ + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $sub.tenantId + SubscriptionGuid = $sub.subscriptionId + Location = $sub.location + ContainerType = $sub.type + ContainerName = $sub.name.ToLower() + InstanceId = $sub.id.ToLower() + ResourceCount = $sub.ResourceCount + ManagedBy = $sub.managedBy + ContainerProperties = $sub.properties | ConvertTo-Json -Compress + Tags = $sub.tags + StatusDate = $statusDate + } + + $allResourceContainers += $logentry +} + +Write-Output "Uploading CSV to Storage" + +$today = $datetime.ToString("yyyyMMdd") +$jsonExportPath = "$today-rescontainers-$subscriptionSuffix.json" +$csvExportPath = "$today-rescontainers-$subscriptionSuffix.csv" + +$allResourceContainers | ConvertTo-Json -Depth 3 -Compress | Out-File $jsonExportPath +Write-Output "Exported to JSON: $($allResourceContainers.Count) lines" +$allResourceContainersJson = Get-Content -Path $jsonExportPath | ConvertFrom-Json +Write-Output "JSON Import: $($allResourceContainersJson.Count) lines" +$allResourceContainersJson | Export-Csv -NoTypeInformation -Path $csvExportPath +Write-Output "Export to $csvExportPath" + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGSqlDatabasePropertiesToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGSqlDatabasePropertiesToBlobStorage.ps1 new file mode 100644 index 000000000..62b041de4 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGSqlDatabasePropertiesToBlobStorage.ps1 @@ -0,0 +1,204 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$referenceRegion = Get-AutomationVariable -Name "AzureOptimization_ReferenceRegion" -ErrorAction SilentlyContinue # e.g., westeurope +if ([string]::IsNullOrEmpty($referenceRegion)) +{ + $referenceRegion = "westeurope" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ARGSqlDatabaseContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "argsqldbexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +$alldbs = @() + +Write-Output "Getting subscriptions target $TargetSubscription" +if (-not([string]::IsNullOrEmpty($TargetSubscription))) +{ + $subscriptions = $TargetSubscription + $subscriptionSuffix = $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = $cloudSuffix + "all-" + $tenantId +} + +$dbsTotal = @() + +$resultsSoFar = 0 + +Write-Output "Querying for SQL Databases properties" + +$argQuery = @" + resources + | where type =~ 'microsoft.sql/servers/databases' and name != 'master' + | extend skuName = sku.name, skuTier = sku.tier, skuCapacity = sku.capacity + | extend storageAccountType = properties.storageAccountType, licenseType = properties.licenseType, serviceObjectiveName = properties.currentServiceObjectiveName + | extend zoneRedundant = properties.zoneRedundant, maxSizeBytes = properties.maxSizeBytes, maxLogSizeBytes = properties.maxLogSizeBytes + | order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $dbs = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $dbs = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($dbs -and $dbs.GetType().Name -eq "PSResourceGraphResponse") + { + $dbs = $dbs.Data + } + $resultsCount = $dbs.Count + $resultsSoFar += $resultsCount + $dbsTotal += $dbs + +} while ($resultsCount -eq $ARGPageSize) + +$datetime = (Get-Date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +Write-Output "Building $($dbsTotal.Count) SQL Database entries" + +foreach ($db in $dbsTotal) +{ + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $db.tenantId + SubscriptionGuid = $db.subscriptionId + ResourceGroupName = $db.resourceGroup.ToLower() + ZoneRedundant = $db.zoneRedundant + Location = $db.location + DBName = $db.name.ToLower() + InstanceId = $db.id.ToLower() + SkuName = $db.skuName + SkuTier = $db.skuTier + SkuCapacity = $db.skuCapacity + ServiceObjectiveName = $db.serviceObjectiveName + StorageAccountType = $db.storageAccountType + LicenseType = $db.licenseType + MaxSizeBytes = $db.maxSizeBytes + MaxLogSizeBytes = $db.maxLogSizeBytes + Tags = $db.tags + StatusDate = $statusDate + } + + $alldbs += $logentry +} + +Write-Output "Uploading CSV to Storage" + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-sqldbs-$subscriptionSuffix.csv" + +$alldbs | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGUnmanagedDisksPropertiesToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGUnmanagedDisksPropertiesToBlobStorage.ps1 new file mode 100644 index 000000000..92bf715bf --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGUnmanagedDisksPropertiesToBlobStorage.ps1 @@ -0,0 +1,236 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ARGVhdContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "argvhdexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +$alldisks = @() + +Write-Output "Getting subscriptions target $TargetSubscription" +if (-not([string]::IsNullOrEmpty($TargetSubscription))) +{ + $subscriptions = $TargetSubscription + $subscriptionSuffix = $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = $cloudSuffix + "all-" + $tenantId +} + +$mdisksTotal = @() +$resultsSoFar = 0 + +Write-Output "Querying for ARM Unmanaged OS Disks properties" + +$argQuery = @" +resources +| where type =~ 'Microsoft.Compute/virtualMachines' and isnull(properties.storageProfile.osDisk.managedDisk) +| extend diskType = 'OS', diskCaching = tostring(properties.storageProfile.osDisk.caching), diskSize = tostring(properties.storageProfile.osDisk.diskSizeGB) +| extend vhdUriParts = split(tostring(properties.storageProfile.osDisk.vhd.uri),'/') +| extend diskStorageAccountName = tostring(split(vhdUriParts[2],'.')[0]), diskContainerName = tostring(vhdUriParts[3]), diskVhdName = tostring(vhdUriParts[4]) +| order by id, diskStorageAccountName, diskContainerName, diskVhdName +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $mdisks = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $mdisks = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($mdisks -and $mdisks.GetType().Name -eq "PSResourceGraphResponse") + { + $mdisks = $mdisks.Data + } + $resultsCount = $mdisks.Count + $resultsSoFar += $resultsCount + $mdisksTotal += $mdisks + +} while ($resultsCount -eq $ARGPageSize) + +$resultsSoFar = 0 + +Write-Output "Found $($mdisksTotal.Count) Unmanaged OS Disk entries" + +Write-Output "Querying for ARM Unmanaged Data Disks properties" + +$argQuery = @" +resources +| where type =~ 'Microsoft.Compute/virtualMachines' and isnull(properties.storageProfile.osDisk.managedDisk) +| mvexpand dataDisks = properties.storageProfile.dataDisks +| extend diskType = 'Data', diskCaching = tostring(dataDisks.caching), diskSize = tostring(dataDisks.diskSizeGB) +| extend vhdUriParts = split(tostring(dataDisks.vhd.uri),'/') +| extend diskStorageAccountName = tostring(split(vhdUriParts[2],'.')[0]), diskContainerName = tostring(vhdUriParts[3]), diskVhdName = tostring(vhdUriParts[4]) +| order by id, diskStorageAccountName, diskContainerName, diskVhdName +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $mdisks = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $mdisks = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($mdisks -and $mdisks.GetType().Name -eq "PSResourceGraphResponse") + { + $mdisks = $mdisks.Data + } + $resultsCount = $mdisks.Count + $resultsSoFar += $resultsCount + $mdisksTotal += $mdisks + +} while ($resultsCount -eq $ARGPageSize) + +Write-Output "Found overall $($mdisksTotal.Count) Unmanaged Disk entries" + +<# + Building CSV entries +#> + +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +foreach ($disk in $mdisksTotal) +{ + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $disk.tenantId + SubscriptionGuid = $disk.subscriptionId + ResourceGroupName = $disk.resourceGroup.ToLower() + DiskName = $disk.diskVhdName.ToLower() + InstanceId = ($disk.diskStorageAccountName + "/" + $disk.diskContainerName + "/" + $disk.diskVhdName).ToLower() + OwnerVMId = $disk.id.ToLower() + Location = $disk.location + DeploymentModel = "Unmanaged" + DiskType = $disk.diskType + Caching = $disk.diskCaching + DiskSizeGB = $disk.diskSize + StatusDate = $statusDate + Tags = $disk.tags + } + + $alldisks += $logentry +} + +<# + Actually exporting CSV to Azure Storage +#> + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-vhds-$subscriptionSuffix.csv" + +$alldisks | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGVMSSPropertiesToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGVMSSPropertiesToBlobStorage.ps1 new file mode 100644 index 000000000..0fc85a46d --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGVMSSPropertiesToBlobStorage.ps1 @@ -0,0 +1,239 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$referenceRegion = Get-AutomationVariable -Name "AzureOptimization_ReferenceRegion" -ErrorAction SilentlyContinue # e.g., westeurope +if ([string]::IsNullOrEmpty($referenceRegion)) +{ + $referenceRegion = "westeurope" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ARGVMSSContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "argvmssexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +Write-Output "Getting VM sizes details for $referenceRegion" +$sizes = Get-AzVMSize -Location $referenceRegion + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +$allvmss = @() + +if ($TargetSubscription) +{ + $subscriptions = $TargetSubscription + $subscriptionSuffix = "-" + $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = $cloudSuffix + "all-" + $tenantId +} + +$armVmssTotal = @() + +$resultsSoFar = 0 + +$argQuery = @" +resources +| where type =~ 'microsoft.compute/virtualmachinescalesets' +| project id, tenantId, name, location, resourceGroup, subscriptionId, skUName = tostring(sku.name), + computerNamePrefix = tostring(properties.virtualMachineProfile.osProfile.computerNamePrefix), + usesManagedDisks = iif(isnull(properties.virtualMachineProfile.storageProfile.osDisk.managedDisk), 'false', 'true'), + capacity = tostring(sku.capacity), priority = tostring(properties.virtualMachineProfile.priority), tags, zones, + osType = iif(isnotnull(properties.virtualMachineProfile.osProfile.linuxConfiguration), "Linux", "Windows"), + osDiskSize = tostring(properties.virtualMachineProfile.storageProfile.osDisk.diskSizeGB), + osDiskCaching = tostring(properties.virtualMachineProfile.storageProfile.osDisk.caching), + osDiskSKU = tostring(properties.virtualMachineProfile.storageProfile.osDisk.managedDisk.storageAccountType), + dataDiskCount = iif(isnotnull(properties.virtualMachineProfile.storageProfile.dataDisks), array_length(properties.virtualMachineProfile.storageProfile.dataDisks), 0), + nicCount = array_length(properties.virtualMachineProfile.networkProfile.networkInterfaceConfigurations), + imagePublisher = iif(isnotempty(properties.virtualMachineProfile.storageProfile.imageReference.publisher),tostring(properties.virtualMachineProfile.storageProfile.imageReference.publisher),'Custom'), + imageOffer = iif(isnotempty(properties.virtualMachineProfile.storageProfile.imageReference.offer),tostring(properties.virtualMachineProfile.storageProfile.imageReference.offer),tostring(properties.virtualMachineProfile.storageProfile.imageReference.id)), + imageSku = tostring(properties.virtualMachineProfile.storageProfile.imageReference.sku), + imageVersion = tostring(properties.virtualMachineProfile.storageProfile.imageReference.version), + imageExactVersion = tostring(properties.virtualMachineProfile.storageProfile.imageReference.exactVersion), + singlePlacementGroup = tostring(properties.singlePlacementGroup), + upgradePolicy = tostring(properties.upgradePolicy.mode), + overProvision = tostring(properties.overprovision), + platformFaultDomainCount = tostring(properties.platformFaultDomainCount), + zoneBalance = tostring(properties.zoneBalance) +| order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $armVmss = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $armVmss = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + + if ($armVmss -and $armVmss.GetType().Name -eq "PSResourceGraphResponse") + { + $armVmss = $armVmss.Data + } + $resultsCount = $armVmss.Count + $resultsSoFar += $resultsCount + $armVmssTotal += $armVmss + +} while ($resultsCount -eq $ARGPageSize) + +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +Write-Output "Building $($armVmssTotal.Count) VMSS entries" + +foreach ($vmss in $armVmssTotal) +{ + $vmSize = $sizes | Where-Object {$_.name -eq $vmss.skUName} + + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $vmss.tenantId + SubscriptionGuid = $vmss.subscriptionId + ResourceGroupName = $vmss.resourceGroup.ToLower() + Zones = $vmss.zones + Location = $vmss.location + VMSSName = $vmss.name.ToLower() + ComputerNamePrefix = $vmss.computerNamePrefix.ToLower() + InstanceId = $vmss.id.ToLower() + VMSSSize = $vmSize.name.ToLower() + CoresCount = $vmSize.NumberOfCores + MemoryMB = $vmSize.MemoryInMB + OSType = $vmss.osType + DataDiskCount = $vmss.dataDiskCount + NicCount = $vmss.nicCount + StatusDate = $statusDate + Tags = $vmss.tags + Capacity = $vmss.capacity + Priority = $vmss.priority + OSDiskSize = $vmss.osDiskSize + OSDiskCaching = $vmss.osDiskCaching + OSDiskSKU = $vmss.osDiskSKU + SinglePlacementGroup = $vmss.singlePlacementGroup + UpgradePolicy = $vmss.upgradePolicy + OverProvision = $vmss.overProvision + PlatformFaultDomainCount = $vmss.platformFaultDomainCount + ZoneBalance = $vmss.zoneBalance + UsesManagedDisks = $vmss.usesManagedDisks + ImagePublisher = $vmss.imagePublisher + ImageOffer = $vmss.imageOffer + ImageSku = $vmss.imageSku + ImageVersion = $vmss.imageVersion + ImageExactVersion = $vmss.imageExactVersion + } + + $allvmss += $logentry +} + +Write-Output "Uploading CSV to Storage" + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-vmss-$subscriptionSuffix.csv" + +$allvmss | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGVNetPropertiesToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGVNetPropertiesToBlobStorage.ps1 new file mode 100644 index 000000000..2d905ce03 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGVNetPropertiesToBlobStorage.ps1 @@ -0,0 +1,308 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$referenceRegion = Get-AutomationVariable -Name "AzureOptimization_ReferenceRegion" -ErrorAction SilentlyContinue # e.g., westeurope +if ([string]::IsNullOrEmpty($referenceRegion)) +{ + $referenceRegion = "westeurope" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ARGVNetContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "argvnetexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +$allsubnets = @() + +Write-Output "Getting subscriptions target $TargetSubscription" +if (-not([string]::IsNullOrEmpty($TargetSubscription))) +{ + $subscriptions = $TargetSubscription + $subscriptionSuffix = $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = $cloudSuffix + "all-" + $tenantId +} + +$subnetsTotal = @() + +$resultsSoFar = 0 + +Write-Output "Querying for ARM VNet properties" + +$argQuery = @" + resources + | where type =~ 'microsoft.network/virtualnetworks' + | mv-expand subnets = properties.subnets limit 400 + | extend peeringsCount = array_length(properties.virtualNetworkPeerings) + | extend vnetPrefixes = properties.addressSpace.addressPrefixes + | extend dnsServers = properties.dhcpOptions.dnsServers + | extend enableDdosProtection = properties.enableDdosProtection + | project-away properties + | extend subnetPrefix = tostring(subnets.properties.addressPrefix) + | extend subnetDelegationsCount = array_length(subnets.properties.delegations) + | extend subnetUsedIPs = iif(isnotempty(subnets.properties.ipConfigurations), array_length(subnets.properties.ipConfigurations), 0) + | extend subnetTotalPrefixIPs = pow(2, 32 - toint(split(subnetPrefix,'/')[1])) - 5 + | extend subnetNsgId = tolower(subnets.properties.networkSecurityGroup.id) + | project id, vnetName = name, resourceGroup, subscriptionId, tenantId, location, vnetPrefixes, dnsServers, subnetName = tolower(tostring(subnets.name)), subnetPrefix, subnetDelegationsCount, subnetTotalPrefixIPs, subnetUsedIPs, subnetNsgId, peeringsCount, enableDdosProtection, tags + | order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $subnets = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $subnets = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($subnets -and $subnets.GetType().Name -eq "PSResourceGraphResponse") + { + $subnets = $subnets.Data + } + $resultsCount = $subnets.Count + $resultsSoFar += $resultsCount + $subnetsTotal += $subnets + +} while ($resultsCount -eq $ARGPageSize) + +$datetime = (Get-Date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +Write-Output "Building $($subnetsTotal.Count) ARM VNet subnet entries" + +foreach ($subnet in $subnetsTotal) +{ + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $subnet.tenantId + SubscriptionGuid = $subnet.subscriptionId + ResourceGroupName = $subnet.resourceGroup.ToLower() + Location = $subnet.location + VNetName = $subnet.vnetName.ToLower() + InstanceId = $subnet.id.ToLower() + Model = "ARM" + VNetPrefixes = $subnet.vnetPrefixes + DNSServers = $subnet.dnsServers + PeeringsCount = $subnet.peeringsCount + EnableDdosProtection = $subnet.enableDdosProtection + SubnetName = $subnet.subnetName + SubnetPrefix = $subnet.subnetPrefix + SubnetDelegationsCount = $subnet.subnetDelegationsCount + SubnetTotalPrefixIPs = $subnet.subnetTotalPrefixIPs + SubnetUsedIPs = $subnet.subnetUsedIPs + SubnetNSGId = $subnet.subnetNsgId + Tags = $subnet.tags + StatusDate = $statusDate + } + + $allsubnets += $logentry +} + +$subnetsTotal = @() + +$resultsSoFar = 0 + +Write-Output "Querying for Classic VNet properties" + +$argQuery = @" + resources + | where type =~ 'microsoft.classicnetwork/virtualnetworks' + | extend vNetId = tolower(id) + | mv-expand subnets = properties.subnets limit 400 + | extend subnetName = tolower(tostring(subnets.name)) + | join kind=leftouter ( + resources + | where type =~ 'microsoft.network/virtualnetworks' + | mvexpand peerings = properties.virtualNetworkPeerings limit 400 + | extend vNetId = tolower(tostring(peerings.properties.remoteVirtualNetwork.id)) + | where vNetId has "microsoft.classicnetwork" + | summarize vNetPeerings=count() by vNetId + ) on vNetId + | extend peeringsCount = iif(isnotempty(vNetPeerings), vNetPeerings, 0) + | extend vnetPrefixes = properties.addressSpace.addressPrefixes + | extend dnsServers = properties.dhcpOptions.dnsServers + | project-away properties + | extend subnetPrefix = tostring(subnets.addressPrefix) + | join kind=leftouter ( + resources + | where type =~ 'microsoft.classiccompute/virtualmachines' + | extend networkProfile = properties.networkProfile + | mvexpand subnets = networkProfile.virtualNetwork.subnetNames limit 400 + | extend subnetName = tolower(tostring(subnets)) + | project id, vNetId = tolower(tostring(networkProfile.virtualNetwork.id)), subnetName + | summarize subnetUsedIPs = count() by vNetId, subnetName + ) on vNetId and subnetName + | extend subnetUsedIPs = iif(isnotempty(subnetUsedIPs), subnetUsedIPs, 0) + | extend subnetTotalPrefixIPs = pow(2, 32 - toint(split(subnetPrefix,'/')[1])) - 5 + | extend enableDdosProtection = 'false' + | project vNetId, vnetName = name, resourceGroup, subscriptionId, tenantId, location, vnetPrefixes, dnsServers, subnetName, subnetPrefix, subnetTotalPrefixIPs, subnetUsedIPs, peeringsCount, enableDdosProtection + | order by vNetId asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $subnets = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $subnets = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($subnets -and $subnets.GetType().Name -eq "PSResourceGraphResponse") + { + $subnets = $subnets.Data + } + $resultsCount = $subnets.Count + $resultsSoFar += $resultsCount + $subnetsTotal += $subnets + +} while ($resultsCount -eq $ARGPageSize) + +$datetime = (Get-Date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +Write-Output "Building $($subnetsTotal.Count) Classic VNet subnet entries" + +foreach ($subnet in $subnetsTotal) +{ + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $subnet.tenantId + SubscriptionGuid = $subnet.subscriptionId + ResourceGroupName = $subnet.resourceGroup.ToLower() + Location = $subnet.location + VNetName = $subnet.vnetName.ToLower() + InstanceId = $subnet.vNetId.ToLower() + Model = "Classic" + VNetPrefixes = $subnet.vnetPrefixes + DNSServers = $subnet.dnsServers + PeeringsCount = $subnet.peeringsCount + EnableDdosProtection = $subnet.enableDdosProtection + SubnetName = $subnet.subnetName + SubnetPrefix = $subnet.subnetPrefix + SubnetTotalPrefixIPs = $subnet.subnetTotalPrefixIPs + SubnetUsedIPs = $subnet.subnetUsedIPs + StatusDate = $statusDate + } + + $allsubnets += $logentry +} + +Write-Output "Uploading CSV to Storage" + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-vnetsubnets-$subscriptionSuffix.csv" + +$allsubnets | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGVirtualMachinesPropertiesToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGVirtualMachinesPropertiesToBlobStorage.ps1 new file mode 100644 index 000000000..5ba0ddbfa --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ARGVirtualMachinesPropertiesToBlobStorage.ps1 @@ -0,0 +1,340 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$referenceRegion = Get-AutomationVariable -Name "AzureOptimization_ReferenceRegion" -ErrorAction SilentlyContinue # e.g., westeurope +if ([string]::IsNullOrEmpty($referenceRegion)) +{ + $referenceRegion = "westeurope" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ARGVMContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "argvmexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +# get list of all VM sizes +Write-Output "Getting VM sizes details for $referenceRegion" +$sizes = Get-AzVMSize -Location $referenceRegion + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +$allvms = @() + +Write-Output "Getting subscriptions target $TargetSubscription" +if (-not([string]::IsNullOrEmpty($TargetSubscription))) +{ + $subscriptions = $TargetSubscription + $subscriptionSuffix = $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = $cloudSuffix + "all-" + $tenantId +} + +$armVmsTotal = @() +$classicVmsTotal = @() + +$resultsSoFar = 0 + +<# + Getting all ARM VMs properties with Azure Resource Graph query +#> + +Write-Output "Querying for ARM VM properties" + +$argQuery = @" + resources + | where type =~ 'Microsoft.Compute/virtualMachines' + | extend dataDiskCount = array_length(properties.storageProfile.dataDisks), nicCount = array_length(properties.networkProfile.networkInterfaces) + | extend usesManagedDisks = iif(isnull(properties.storageProfile.osDisk.managedDisk), 'false', 'true') + | extend availabilitySetId = tostring(properties.availabilitySet.id) + | extend bootDiagnosticsEnabled = tostring(properties.diagnosticsProfile.bootDiagnostics.enabled) + | extend bootDiagnosticsStorageAccount = split(split(properties.diagnosticsProfile.bootDiagnostics.storageUri, '/')[2],'.')[0] + | extend powerState = tostring(properties.extended.instanceView.powerState.code) + | extend imagePublisher = iif(isnotempty(properties.storageProfile.imageReference.publisher),tostring(properties.storageProfile.imageReference.publisher),'Custom') + | extend imageOffer = iif(isnotempty(properties.storageProfile.imageReference.offer),tostring(properties.storageProfile.imageReference.offer),tostring(properties.storageProfile.imageReference.id)) + | extend imageSku = tostring(properties.storageProfile.imageReference.sku) + | extend imageVersion = tostring(properties.storageProfile.imageReference.version) + | extend imageExactVersion = tostring(properties.storageProfile.imageReference.exactVersion) + | extend osName = tostring(properties.extended.instanceView.osName) + | extend osVersion = tostring(properties.extended.instanceView.osVersion) + | order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $armVms = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $armVms = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($armVms -and $armVms.GetType().Name -eq "PSResourceGraphResponse") + { + $armVms = $armVms.Data + } + $resultsCount = $armVms.Count + $resultsSoFar += $resultsCount + $armVmsTotal += $armVms + +} while ($resultsCount -eq $ARGPageSize) + +$resultsSoFar = 0 + +<# + Getting all Classic VMs properties with Azure Resource Graph query +#> + +Write-Output "Querying for Classic VM properties" + +$argQuery = @" + resources + | where type =~ 'Microsoft.ClassicCompute/virtualMachines' + | extend dataDiskCount = iif(isnotnull(properties.storageProfile.dataDisks), array_length(properties.storageProfile.dataDisks), 0), nicCount = iif(isnotnull(properties.networkProfile.virtualNetwork.networkInterfaces), array_length(properties.networkProfile.virtualNetwork.networkInterfaces) + 1, 1) + | extend usesManagedDisks = 'false' + | extend availabilitySetId = tostring(properties.hardwareProfile.availabilitySet) + | extend bootDiagnosticsEnabled = tostring(properties.debugProfile.bootDiagnosticsEnabled) + | extend bootDiagnosticsStorageAccount = split(split(properties.debugProfile.serialOutputBlobUri, '/')[2],'.')[0] + | extend powerState = tostring(properties.instanceView.status) + | extend imageOffer = tostring(properties.storageProfile.operatingSystemDisk.sourceImageName) + | order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $classicVms = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $classicVms = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($classicVms -and $classicVms.GetType().Name -eq "PSResourceGraphResponse") + { + $classicVms = $classicVms.Data + } + $resultsCount = $classicVms.Count + $resultsSoFar += $resultsCount + $classicVmsTotal += $classicVms + +} while ($resultsCount -eq $ARGPageSize) + +<# + Merging ARM + Classic VMs, enriching VM size details and building CSV entries +#> + +$datetime = (Get-Date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +Write-Output "Building $($armVmsTotal.Count) ARM VM entries" + +foreach ($vm in $armVmsTotal) +{ + $vmSize = $sizes | Where-Object {$_.name -eq $vm.properties.hardwareProfile.vmSize} + + $avSetId = $null + if ($vm.availabilitySetId) + { + $avSetId = $vm.availabilitySetId.ToLower() + } + + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $vm.tenantId + SubscriptionGuid = $vm.subscriptionId + ResourceGroupName = $vm.resourceGroup.ToLower() + Zones = $vm.zones + Location = $vm.location + VMName = $vm.name.ToLower() + DeploymentModel = 'ARM' + InstanceId = $vm.id.ToLower() + VMSize = $vm.properties.hardwareProfile.vmSize + CoresCount = $vmSize.NumberOfCores + MemoryMB = $vmSize.MemoryInMB + OSType = $vm.properties.storageProfile.osDisk.osType + LicenseType = $vm.properties.licenseType + DataDiskCount = $vm.dataDiskCount + NicCount = $vm.nicCount + UsesManagedDisks = $vm.usesManagedDisks + AvailabilitySetId = $avSetId + BootDiagnosticsEnabled = $vm.bootDiagnosticsEnabled + BootDiagnosticsStorageAccount = $vm.bootDiagnosticsStorageAccount + StatusDate = $statusDate + PowerState = $vm.powerState + ImagePublisher = $vm.imagePublisher + ImageOffer = $vm.imageOffer + ImageSku = $vm.imageSku + ImageVersion = $vm.imageVersion + ImageExactVersion = $vm.imageExactVersion + OSName = $vm.osName + OSVersion = $vm.osVersion + Tags = $vm.tags + } + + $allvms += $logentry +} + +Write-Output "Building $($classicVmsTotal.Count) Classic VM entries" + +foreach ($vm in $classicVmsTotal) +{ + $vmSize = $sizes | Where-Object {$_.name -eq $vm.properties.hardwareProfile.size} + + $avSetId = $null + if ($vm.availabilitySetId) + { + $avSetId = $vm.availabilitySetId.ToLower() + } + + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $vm.tenantId + SubscriptionGuid = $vm.subscriptionId + ResourceGroupName = $vm.resourceGroup.ToLower() + VMName = $vm.name.ToLower() + DeploymentModel = 'Classic' + Location = $vm.location + InstanceId = $vm.id.ToLower() + VMSize = $vm.properties.hardwareProfile.size + CoresCount = $vmSize.NumberOfCores + MemoryMB = $vmSize.MemoryInMB + OSType = $vm.properties.storageProfile.operatingSystemDisk.operatingSystem + LicenseType = "N/A" + DataDiskCount = $vm.dataDiskCount + NicCount = $vm.nicCount + UsesManagedDisks = $vm.usesManagedDisks + AvailabilitySetId = $avSetId + BootDiagnosticsEnabled = $vm.bootDiagnosticsEnabled + BootDiagnosticsStorageAccount = $vm.bootDiagnosticsStorageAccount + PowerState = $vm.powerState + StatusDate = $statusDate + ImagePublisher = $vm.imagePublisher + ImageOffer = $vm.imageOffer + ImageSku = $vm.imageSku + ImageVersion = $vm.imageVersion + ImageExactVersion = $vm.imageExactVersion + OSName = $vm.osName + OSVersion = $vm.osVersion + Tags = $null + } + + $allvms += $logentry +} + +<# + Actually exporting CSV to Azure Storage +#> + +Write-Output "Uploading CSV to Storage" + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-vms-$subscriptionSuffix.csv" + +$allvms | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-AdvisorRecommendationsToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-AdvisorRecommendationsToBlobStorage.ps1 new file mode 100644 index 000000000..68acb0c85 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-AdvisorRecommendationsToBlobStorage.ps1 @@ -0,0 +1,247 @@ +param( + [Parameter(Mandatory = $false)] + [string] $targetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_AdvisorContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "advisorexports" +} + +$CategoryFilter = Get-AutomationVariable -Name "AzureOptimization_AdvisorFilter" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($CategoryFilter)) +{ + $CategoryFilter = "HighAvailability,Security,Performance,OperationalExcellence" # comma-separated list of categories +} +$CategoryFilter += ",Cost" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudEnvironment = $externalCloudEnvironment +} + +Write-Output "Getting subscriptions target $TargetSubscription" + +$tenantId = (Get-AzContext).Tenant.Id + +$ARGPageSize = 1000 + +if (-not([string]::IsNullOrEmpty($TargetSubscription))) +{ + $subscriptions = $TargetSubscription + $scope = $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" -and $_.SubscriptionPolicies.QuotaId -notlike "AAD*" } | ForEach-Object { "$($_.Id)"} + $scope = $tenantId +} + + +<# + Getting Advisor recommendations for each subscription and building CSV entries +#> + +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +$recommendationsARG = @() + +$resultsSoFar = 0 + +$FinalCategoryFilter = "" + +if (-not([string]::IsNullOrEmpty($CategoryFilter))) +{ + $categories = $CategoryFilter.Split(',') + for ($i = 0; $i -lt $categories.Count; $i++) + { + $categories[$i] = "'" + $categories[$i] + "'" + } + $FinalCategoryFilter = " and properties.category in (" + ($categories -join ",") + ")" +} + +$argQuery = @" +advisorresources +| where type == 'microsoft.advisor/recommendations' +| where isnull(properties.suppressionIds)$FinalCategoryFilter +| extend resourceId = tostring(split(tolower(id),'/providers/microsoft.advisor')[0]) +| join kind=leftouter (resources | project resourceId=tolower(id), resourceTags=tags) on resourceId +| project id, category = properties.category, impact = properties.impact, impactedArea = properties.impactedField, + description = properties.shortDescription.problem, recommendationText = properties.shortDescription.solution, + recommendationTypeId = properties.recommendationTypeId, instanceName = properties.impactedValue, + additionalInfo = properties.extendedProperties, tags=resourceTags +| order by id asc +"@ + +do +{ + if ($resultsSoFar -eq 0) + { + $recs = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $recs = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($recs -and $recs.GetType().Name -eq "PSResourceGraphResponse") + { + $recs = $recs.Data + } + $resultsCount = $recs.Count + $resultsSoFar += $resultsCount + $recommendationsARG += $recs + +} while ($resultsCount -eq $ARGPageSize) + +Write-Output "Building $($recommendationsARG.Count) recommendations entries" + +$recommendations = @() + +foreach ($advisorRecommendation in $recommendationsARG) +{ + $resourceIdParts = $advisorRecommendation.id.Split('/') + if ($resourceIdParts.Count -ge 9) + { + # if the Resource ID is made of 9 parts, then the recommendation is relative to a specific Azure resource + $realResourceIdParts = $resourceIdParts[0..8] + $instanceId = ($realResourceIdParts -join "/").ToLower() + $resourceGroup = $realResourceIdParts[4].ToLower() + $subscriptionId = $realResourceIdParts[2] + } + else + { + # otherwise it is not a resource-specific recommendation (for example, reservations) + $resourceGroup = "notavailable" + $instanceId = $advisorRecommendation.id.ToLower() + $subscriptionId = $resourceIdParts[2] + } + + if (-not([string]::IsNullOrEmpty($advisorRecommendation.additionalInfo))) + { + $additionalInfo = $advisorRecommendation.additionalInfo | ConvertTo-Json -Compress + } + else + { + $additionalInfo = $null + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + Category = $advisorRecommendation.category + Impact = $advisorRecommendation.impact + ImpactedArea = $advisorRecommendation.impactedArea + Description = $advisorRecommendation.description + RecommendationText = $advisorRecommendation.recommendationText + RecommendationTypeId = $advisorRecommendation.recommendationTypeId + InstanceId = $instanceId + InstanceName = $advisorRecommendation.instanceName + Tags = $advisorRecommendation.tags + AdditionalInfo = $additionalInfo + ResourceGroup = $resourceGroup + SubscriptionGuid = $subscriptionId + TenantGuid = $tenantId + } + + $recommendations += $recommendation +} + +Write-Output "Found $($recommendations.Count) ($CategoryFilter) recommendations..." + +$fileDate = $datetime.ToString("yyyyMMdd") +$advisorFilter = $CategoryFilter.Replace(',','').ToLower() +$csvExportPath = "$fileDate-$advisorFilter-$scope.csv" + +$recommendations | Export-Csv -NoTypeInformation -Path $csvExportPath +Write-Output "Export to $csvExportPath" + +$csvBlobName = $csvExportPath +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." + +Write-Output "DONE!" \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-AzMonitorMetricsToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-AzMonitorMetricsToBlobStorage.ps1 new file mode 100644 index 000000000..0297b5842 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-AzMonitorMetricsToBlobStorage.ps1 @@ -0,0 +1,296 @@ +Param ( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $true)] + [string] $ResourceType, # ARM resource type + + [Parameter(Mandatory = $false)] + [string] $ARGFilter, # e.g., name != 'master' and sku.tier in ('Basic','Standard','Premium') + + [Parameter(Mandatory = $true)] + [string] $MetricNames, # comma-separated metrics names (use Get-AzMetricDefinition for a list of supported metric names for a given resource) + + [Parameter(Mandatory = $true)] + [ValidateSet("Maximum", "Minimum", "Average", "Total")] + [string] $AggregationType, + + [Parameter(Mandatory = $false)] + [ValidateSet("Default", "Maximum", "Minimum", "Average", "Total")] + [string] $AggregationOfType = "Default", + + [Parameter(Mandatory = $true)] + [string] $TimeSpan, # [d.]hh:mm:ss + + [Parameter(Mandatory = $true)] + [string] $TimeGrain, # [d.]hh:mm:ss (00:01:00, 00:05:00, 00:15:00, 00:30:00, 01:00:00, 06:00:00, 12:00:00, 1.00:00:00, 7.00:00:00, 30.00:00:00) + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_AzMonitorContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "azmonitorexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +if (-not([string]::IsNullOrEmpty($TargetSubscription))) { + $subscriptions = $TargetSubscription + $subscriptionSuffix = "-" + $TargetSubscription +} +else { + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = "all-" + $tenantId +} + +[TimeSpan]::Parse($TimeGrain) | Out-Null +$TimeSpanObj = [TimeSpan]::Parse("-$TimeSpan") + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Querying for $ResourceType with page size $ARGPageSize and target subscription $TargetSubscription..." + +$allResources = @() + +$resultsSoFar = 0 + +$argWhere = "" +if (-not([string]::IsNullOrEmpty($ARGFilter))) +{ + $argWhere = " and $ARGFilter" +} + +$argQuery = @" +resources +| where type =~ '$ResourceType'$argWhere +| project id, name, subscriptionId, resourceGroup, tenantId +| order by id asc +"@ + +do { + if ($resultsSoFar -eq 0) { + $resources = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else { + $resources = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($resources -and $resources.GetType().Name -eq "PSResourceGraphResponse") + { + $resources = $resources.Data + } + $resultsCount = $resources.Count + $resultsSoFar += $resultsCount + $allResources += $resources + +} while ($resultsCount -eq $ARGPageSize) + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Found $($allResources.Count) resources." + +$metrics = $MetricNames.Split(',') + +$queryDate = Get-Date +$utcNow = $queryDate.ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +$utcAgo = $queryDate.Add($TimeSpanObj).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + +$customMetrics = @() + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Analyzing resources for $MetricNames metrics ($AggregationType with $TimeGrain time grain) since $utcAgo..." + +foreach ($resource in $allResources) { + $valuesAggregation = @() + $foundResource = $true + foreach ($metric in $metrics) { + $metricValues = Get-AzMetric -ResourceId $resource.id -MetricName $metric -TimeGrain $TimeGrain -AggregationType $AggregationType ` + -StartTime $utcAgo -EndTime $utcNow -WarningAction SilentlyContinue -ErrorAction Continue + if ($metricValues.Data) { + if ($valuesAggregation.Count -eq 0) { + $valuesAggregation = $metricValues.Data."$AggregationType" + } + else { + for ($i = 0; $i -lt $valuesAggregation.Count; $i++) { + if ($metricValues.Data.Count -gt 1) + { + $valuesAggregation[$i] += $metricValues.Data[$i]."$AggregationType" + } + else + { + $valuesAggregation += $metricValues.Data."$AggregationType" + } + } + } + } + + if (-not($metricValues.Id)) + { + $foundResource = $false + } + } + + if ($foundResource) + { + $aggregatedValue = $null + $finalAggregationType = $AggregationType + if ($AggregationOfType -ne "Default") + { + $finalAggregationType = $AggregationOfType + } + if ($valuesAggregation.Count -gt 0) { + switch ($finalAggregationType) { + "Maximum" { + $aggregatedValue = ($valuesAggregation | Measure-Object -Maximum).Maximum + } + "Minimum" { + $aggregatedValue = ($valuesAggregation | Measure-Object -Minimum).Minimum + } + "Average" { + $aggregatedValue = ($valuesAggregation | Measure-Object -Average).Average + } + "Total" { + $aggregatedValue = ($valuesAggregation | Measure-Object -Sum).Sum + } + } + } + + $customMetric = New-Object PSObject -Property @{ + Timestamp = $utcNow + Cloud = $cloudEnvironment + TenantGuid = $resource.tenantId + SubscriptionGuid = $resource.subscriptionId + ResourceGroupName = $resource.resourceGroup.ToLower() + ResourceName = $resource.name.ToLower() + ResourceId = $resource.id.ToLower() + MetricNames = $MetricNames + AggregationType = $AggregationType + AggregationOfType = $AggregationOfType + MetricValue = $aggregatedValue + TimeGrain = $TimeGrain + TimeSpan = $TimeSpan + } + + $customMetrics += $customMetric + } +} + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Found $($customMetrics.Count) resources to collect metrics from..." + +$metricMoment = $queryDate.Add($TimeSpanObj).ToUniversalTime().ToString("yyyyMMddHHmmss") +$ResourceTypeName = $ResourceType.Split('/')[1].ToLower() +$MetricName = $MetricNames.Replace(',','').Replace(' ','').Replace('/','').ToLower() +$AggregationOfTypeName = "" +if ($AggregationOfType -ne "Default") +{ + $AggregationOfTypeName = ("-$AggregationOfType").ToLower() +} +$AggregationTypeName = "$($AggregationType.ToLower())$AggregationOfTypeName" +$csvExportPath = "$metricMoment-metrics-$ResourceTypeName-$MetricName-$AggregationTypeName-$subscriptionSuffix.csv" + +$ci = [CultureInfo]::new([System.Threading.Thread]::CurrentThread.CurrentCulture.Name) +if ($ci.NumberFormat.NumberDecimalSeparator -ne '.') +{ + Write-Output "Current culture ($($ci.Name)) does not use . as decimal separator" + $ci.NumberFormat.NumberDecimalSeparator = '.' + [System.Threading.Thread]::CurrentThread.CurrentCulture = $ci +} + +$customMetrics | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." + diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ConsumptionToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ConsumptionToBlobStorage.ps1 new file mode 100644 index 000000000..11dabcb0d --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ConsumptionToBlobStorage.ps1 @@ -0,0 +1,875 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName, + + [Parameter(Mandatory = $false)] + [string] $targetStartDate, # YYYY-MM-DD format + + [Parameter(Mandatory = $false)] + [string] $targetEndDate # YYYY-MM-DD format +) + +$ErrorActionPreference = "Stop" +$global:hadErrors = $false +$global:scopesWithErrors = @() + +function Authenticate-AzureWithOption { + param ( + [string] $authOption = "ManagedIdentity", + [string] $cloudEnv = "AzureCloud", + [string] $clientID + ) + + switch ($authOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnv -AccountId $clientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnv + break + } + } +} + +function Generate-CostDetails { + param ( + [string] $ScopeId, + [string] $ScopeName + ) + + $MaxTries = 20 # The typical Retry-After is set to 20 seconds. We'll give ~6 minutes overall to download the cost details report + $hadErrors = $false + + $CostDetailsApiPath = "$ScopeId/providers/Microsoft.CostManagement/generateCostDetailsReport?api-version=2022-05-01" + $body = "{ `"metric`": `"$consumptionMetric`", `"timePeriod`": { `"start`": `"$targetStartDate`", `"end`": `"$targetEndDate`" } }" + $result = Invoke-AzRestMethod -Path $CostDetailsApiPath -Method POST -Payload $body + $requestResultPath = $result.Headers.Location.PathAndQuery + if ($result.StatusCode -in (200,202)) + { + $tries = 0 + $requestSuccess = $false + + Write-Output "Obtained cost detail results endpoint: $requestResultPath..." + + Write-Output "Was told to wait $($result.Headers.RetryAfter.Delta.TotalSeconds) seconds." + + $sleepSeconds = 60 + if ($result.Headers.RetryAfter.Delta.TotalSeconds -gt 0) + { + $sleepSeconds = $result.Headers.RetryAfter.Delta.TotalSeconds + } + + do + { + $tries++ + Write-Output "Checking whether export is ready (try $tries)..." + + Start-Sleep -Seconds $sleepSeconds + $downloadResult = Invoke-AzRestMethod -Method GET -Path $requestResultPath + + if ($downloadResult.StatusCode -eq 200) + { + + Write-Output "Export is ready. Proceeding with CSV download..." + + $downloadBlobJson = $downloadResult.Content | ConvertFrom-Json + + $blobCounter = 0 + foreach ($blob in $downloadBlobJson.manifest.blobs) + { + $blobCounter++ + + Write-Output "Downloading blob $blobCounter..." + + $csvExportPath = "$env:TEMP\$targetStartDate-$ScopeName-$consumptionMetric-$blobCounter.csv" + $finalCsvExportPath = "$env:TEMP\$targetStartDate-$ScopeName-$consumptionMetric-$blobCounter-final.csv" + + Invoke-WebRequest -Uri $blob.blobLink -OutFile $csvExportPath + + Write-Output "Blob downloaded to $csvExportPath successfully." + + $r = [IO.File]::OpenText($csvExportPath) + $w = [System.IO.StreamWriter]::new($finalCsvExportPath) + + # header normalization between MCA and EA + $headerConversion = @{ + additionalInfo = "AdditionalInfo"; + billingAccountId = "BillingAccountId"; + billingAccountName = "BillingAccountName"; + billingCurrency = "BillingCurrencyCode"; + billingPeriodEndDate = "BillingPeriodEndDate"; + billingPeriodStartDate = "BillingPeriodStartDate"; + billingProfileId = "BillingProfileId"; + billingProfileName = "BillingProfileName"; + chargeType = "ChargeType"; + consumedService = "ConsumedService"; + costAllocationRuleName = "CostAllocationRuleName"; + costCenter = "CostCenter"; + costInBillingCurrency = "CostInBillingCurrency"; + date = "Date"; + effectivePrice = "EffectivePrice"; + frequency = "Frequency"; + invoiceSectionId = "InvoiceSectionId"; + invoiceSectionName = "InvoiceSectionName"; + isAzureCreditEligible = "IsAzureCreditEligible"; + meterCategory = "MeterCategory"; + meterId = "MeterId"; + meterName = "MeterName"; + meterRegion = "MeterRegion"; + meterSubCategory = "MeterSubCategory"; + offerId = "OfferId"; + pricingModel = "PricingModel"; + productOrderId = "ProductOrderId"; + productOrderName = "ProductOrderName"; + publisherName = "PublisherName"; + publisherType = "PublisherType"; + quantity = "Quantity"; + reservationId = "ReservationId"; + reservationName = "ReservationName"; + resourceGroupName = "ResourceGroup"; + resourceLocation = "ResourceLocation"; + serviceFamily = "ServiceFamily"; + serviceInfo1 = "ServiceInfo1"; + serviceInfo2 = "ServiceInfo2"; + subscriptionName = "SubscriptionName"; + tags = "Tags"; + term = "Term"; + unitOfMeasure = "UnitOfMeasure"; + unitPrice = "UnitPrice" + } + + $lineCounter = 0 + while ($r.Peek() -ge 0) { + $line = $r.ReadLine() + $lineCounter++ + if ($lineCounter -eq 1) + { + $headers = $line.Split(",") + + for ($i = 0; $i -lt $headers.Length; $i++) + { + $header = $headers[$i] + if ($headerConversion.ContainsKey($header)) + { + $headers[$i] = $headerConversion[$header] + } + } + + $line = $headers -join "," + + $w.WriteLine($line) + } + else + { + $w.WriteLine($line) + } + } + $r.Dispose() + $w.Close() + + $csvBlobName = [System.IO.Path]::GetFileName($finalCsvExportPath) + $csvProperties = @{"ContentType" = "text/csv"}; + Set-AzStorageBlobContent -File $finalCsvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + + Remove-Item -Path $csvExportPath -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + Write-Output "[$now] Removed $csvExportPath from local disk..." + + Remove-Item -Path $finalCsvExportPath -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + Write-Output "[$now] Removed $finalCsvExportPath from local disk..." + } + + $requestSuccess = $true + } + elseif ($downloadResult.StatusCode -eq 202) + { + Write-Output "Was told to wait a bit more... $($downloadResult.Headers.RetryAfter.Delta.TotalSeconds) seconds." + + $sleepSeconds = 60 + if ($downloadResult.Headers.RetryAfter.Delta.TotalSeconds -gt 0) + { + $sleepSeconds = $downloadResult.Headers.RetryAfter.Delta.TotalSeconds + } + } + elseif ($downloadResult.StatusCode -eq 401) + { + Write-Output "Had an authentication issue. Will login again and sleep just a couple of seconds." + + if ($authenticationOption -eq "UserAssignedManagedIdentity") + { + Authenticate-AzureWithOption -authOption $authenticationOption -cloudEnv $cloudEnvironment -clientID $uamiClientID + } + else + { + Authenticate-AzureWithOption -authOption $authenticationOption -cloudEnv $cloudEnvironment + } + + $sleepSeconds = 2 + } + else + { + $global:hadErrors = $true + $global:scopesWithErrors += $ScopeName + Write-Warning "Got an unexpected response code: $($downloadResult.StatusCode)" + } + } + while (-not($requestSuccess) -and $tries -lt $MaxTries) + + if (-not($requestSuccess)) + { + $global:hadErrors = $true + $global:scopesWithErrors += $ScopeName + if ($tries -eq $MaxTries) + { + Write-Warning "Reached maximum number of tries. Aborting..." + } + else + { + Write-Warning "Error returned by the Download Cost Details API. Status Code: $($downloadResult.StatusCode). Message: $($downloadResult.Content)" + } + } + else + { + Write-Output "Export download processing complete." + } + } + else + { + if ($result.StatusCode -ne 204) + { + $global:hadErrors = $true + $global:scopesWithErrors += $ScopeName + Write-Warning "Error returned by the Generate Cost Details API. Status Code: $($result.StatusCode). Message: $($result.Content)" + } + else + { + Write-Output "Request returned 204 No Content" + } + } +} + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ConsumptionContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "consumptionexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$consumptionOffsetDays = [int] (Get-AutomationVariable -Name "AzureOptimization_ConsumptionOffsetDays") + +$consumptionMetric = Get-AutomationVariable -Name "AzureOptimization_ConsumptionMetric" -ErrorAction SilentlyContinue # AmortizedCost|ActualCost +if ([string]::IsNullOrEmpty($consumptionMetric)) +{ + $consumptionMetric = "AmortizedCost" +} + +$consumptionAPIOption = Get-AutomationVariable -Name "AzureOptimization_ConsumptionAPIOption" -ErrorAction SilentlyContinue # CostDetails|UsageDetails +if ([string]::IsNullOrEmpty($consumptionAPIOption)) +{ + $consumptionAPIOption = "CostDetails" +} + +$consumptionScope = Get-AutomationVariable -Name "AzureOptimization_ConsumptionScope" -ErrorAction SilentlyContinue # Subscription|BillingAccount +if ([string]::IsNullOrEmpty($consumptionScope)) +{ + "Consumption Scope not specified, defaulting to Subscription" + $consumptionScope = "Subscription" +} +else +{ + "Consumption Scope is $consumptionScope" + if ($consumptionScope -eq "BillingAccount") + { + $BillingAccountID = Get-AutomationVariable -Name "AzureOptimization_BillingAccountID" + } + else + { + if ($consumptionScope -eq "BillingProfile") + { + $BillingAccountID = Get-AutomationVariable -Name "AzureOptimization_BillingAccountID" + $BillingProfileID = Get-AutomationVariable -Name "AzureOptimization_BillingProfileID" + } + else + { + if ($consumptionScope -ne "Subscription") + { + throw "Invalid value for AzureOptimization_ConsumptionScope. Valid values are 'Subscription' or 'BillingAccount'." + } + } + } +} + +if ($cloudEnvironment -eq "AzureChinaCloud") +{ + $chinaEAEnrollment = Get-AutomationVariable -Name "AzureOptimization_AzureChinaEAEnrollment" -ErrorAction SilentlyContinue + $chinaEAKey = Get-AutomationVariable -Name "AzureOptimization_AzureChinaEAKey" -ErrorAction SilentlyContinue +} + +"Logging in to Azure with $authenticationOption..." + +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + Authenticate-AzureWithOption -authOption $authenticationOption -cloudEnv $cloudEnvironment -clientID $uamiClientID +} +else +{ + Authenticate-AzureWithOption -authOption $authenticationOption -cloudEnv $cloudEnvironment +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudEnvironment = $externalCloudEnvironment +} + +# compute start+end dates + +if ([string]::IsNullOrEmpty($targetStartDate) -or [string]::IsNullOrEmpty($targetEndDate)) +{ + $targetStartDate = (Get-Date).Date.AddDays($consumptionOffsetDays * -1).ToString("yyyy-MM-dd") + $targetEndDate = $targetStartDate +} + +if ($consumptionScope -eq "Subscription") +{ + if (-not([string]::IsNullOrEmpty($TargetSubscription))) + { + $subscriptions = Get-AzSubscription -SubscriptionId $TargetSubscription + } + else + { + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } + } + "Exporting consumption data from $targetStartDate to $targetEndDate for $($subscriptions.Count) subscriptions..." +} +else +{ + "Exporting consumption data from $targetStartDate to $targetEndDate for $consumptionScope..." +} + + +# for each subscription, get billing data + +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +if ($cloudEnvironment -eq "AzureChinaCloud" -and -not([string]::IsNullOrEmpty($chinaEAEnrollment)) -and -not([string]::IsNullOrEmpty($chinaEAKey))) +{ + $targetMonth = $targetStartDate.Substring(0,7) + $consumption = $null + $billingEntries = @() + + $BillingApiUri = "https://ea.azure.cn/rest/$chinaEAEnrollment/usage-report?month=$targetMonth&type=detail&fmt=Csv" + $PricesheetApiUri = "https://ea.azure.cn/rest/$chinaEAEnrollment/usage-report?month=$targetMonth&type=pricesheet&fmt=Csv" + + $Headers = @{} + $Headers.Add("Authorization","Bearer $chinaEAKey") + + Write-Output "Getting pricesheet for month $targetMonth (EA enrollment $chinaEAEnrollment)..." + + Invoke-RestMethod -Method Get -Uri $PricesheetApiUri -Headers $Headers -OutFile "pricesheet-$targetMonth.csv" + + Write-Output "Pricesheet data exported to disk as CSV." + + $csvFile = Get-Content -Path "pricesheet-$targetMonth.csv" + + Write-Output "Pricesheet data imported from disk as string." + + Remove-Item -Path "pricesheet-$targetMonth.csv" -Force + + Write-Output "Removed pricesheet-$targetMonth.csv from local disk..." + + $csvFile2 = $csvFile[2..($csvFile.Count-1)] + $headerLine = $csvFile2[0] + $columnHeaders = $headerLine.Split(",") + for ($i = 0; $i -lt $columnHeaders.Count; $i++) + { + if($columnHeaders[$i] -match '.+\((?.+)\)') + { + $columnHeaders[$i] = $Matches.ColumnName + } + } + $csvFile2[0] = $columnHeaders -join "," + + Write-Output "Removed first 2 lines and replaced header." + + $pricesheet = $csvFile2 | ConvertFrom-Csv + + Write-Output "Starting Azure China billing export process from $targetStartDate to $targetEndDate (month $targetMonth) for EA enrollment $chinaEAEnrollment..." + + $tries = 0 + $requestSuccess = $false + do + { + try { + $tries++ + Invoke-RestMethod -Method Get -Uri $BillingApiUri -Headers $Headers -OutFile "usagedetails-$targetStartDate.csv" + + Write-Output "Consumption data exported to disk as CSV." + + $csvFile = Get-Content -Path "usagedetails-$targetStartDate.csv" + + Write-Output "Consumption data imported from disk as string." + + Remove-Item -Path "usagedetails-$targetStartDate.csv" -Force + + Write-Output "Removed usagedetails-$targetStartDate.csv from local disk..." + + $csvFile2 = $csvFile[2..($csvFile.Count-1)] + $headerLine = $csvFile2[0] + $columnHeaders = $headerLine.Split(",") + for ($i = 0; $i -lt $columnHeaders.Count; $i++) + { + if($columnHeaders[$i] -match '.+\((?.+)\)') + { + $columnHeaders[$i] = $Matches.ColumnName + } + } + $csvFile2[0] = $columnHeaders -join "," + + Write-Output "Removed first 2 lines and replaced header." + + $consumption = $csvFile2 | ConvertFrom-Csv + $requestSuccess = $true + } + catch { + $ErrorMessage = $_.Exception.Message + Write-Warning "Error getting consumption data: $ErrorMessage. $tries of 3 tries. Waiting 60 seconds..." + Start-Sleep -s 60 + } + + } while ( -not($requestSuccess) -and $tries -lt 3 ) + + if (-not($requestSuccess)) + { + throw "Failed consumption export" + } + + Write-Output "Consumption data in memory as CSV. Processing lines..." + + foreach ($consumptionLine in $consumption) + { + $usageDate = [Datetime]::ParseExact($consumptionLine.Date, 'MM/dd/yyyy', $null).ToString("yyyy-MM-dd") + + if ($usageDate -ge $targetStartDate -and $usageDate -le $targetEndDate -and ($subscriptions.Count -gt 1 -or $subscriptions.Id -eq $consumptionLine.SubscriptionGuid)) + { + $instanceId = $null + $instanceName = $null + if ($null -ne $consumptionLine.'Instance ID') + { + $instanceId = $consumptionLine.'Instance ID'.ToLower() + $idParts = $consumptionLine.'Instance ID'.Split("/") + $instanceName = $idParts[$idParts.Count-1].ToLower() + } + + $rgName = $null + if ($null -ne $consumptionLine.'Resource Group') + { + $rgName = $consumptionLine.'Resource Group'.ToLower() + } + + $convertedCost = 0.0 + if ([double]$consumptionLine.ExtendedCost -ne 0) + { + $convertedCost = [double]$consumptionLine.ExtendedCost + } + $convertedPrice = 0.0 + if ([double]$consumptionLine.ResourceRate -ne 0) + { + $convertedPrice = [double]$consumptionLine.ResourceRate + } + + $unitPrice = 0.0 + $partNumber = "N/A" + foreach ($priceItem in $pricesheet) + { + if ($priceItem.Service -eq $consumptionLine.Product) + { + $partNumber = $priceItem.'Part Number' + if ($consumptionLine.'Meter Category' -eq "Virtual Machines") + { + $tempUnitPrice = [double] $priceItem.'Unit Price' + $uom = $priceItem.'Unit of Measure' + $currentUnitHours = [int] (Select-String -InputObject $uom -Pattern "^\d+").Matches[0].Value + if ($currentUnitHours -gt 0) + { + $unitPrice = [double] ($tempUnitPrice / $currentUnitHours) + } + } + else + { + $unitPrice = $convertedPrice + } + break + } + } + + $billingEntry = New-Object PSObject -Property @{ + Timestamp = $timestamp + SubscriptionId = $consumptionLine.SubscriptionGuid + ResourceGroup = $rgName + ResourceName = $instanceName + ResourceId = $instanceId + Date = $consumptionLine.Date + Tags = $consumptionLine.Tags + AdditionalInfo = $consumptionLine.AdditionalInfo + BillingCurrencyCode = "CNY" + ChargeType = "Usage" + ConsumedService = $consumptionLine.'Consumed Service' + CostInBillingCurrency = $convertedCost + EffectivePrice = $convertedPrice + Frequency = "UsageBased" + MeterCategory = $consumptionLine.'Meter Category' + MeterId = $consumptionLine.'Meter ID' + MeterName = $consumptionLine.'Meter Name' + MeterSubCategory = $consumptionLine.'Meter Sub-Category' + PartNumber = $partNumber + ProductName = $consumptionLine.Product + Quantity = $consumptionLine.'Consumed Quantity' + UnitOfMeasure = $consumptionLine.'Unit of Measure' + UnitPrice = $unitPrice + ResourceLocation = $consumptionLine.'Resource Location' + AccountOwnerId = $consumptionLine.AccountOwnerId + } + + $billingEntries += $billingEntry + } + } + + if ($targetStartDate -ne $targetEndDate) + { + $targetStartDate = "$targetStartDate-$targetEndDate" + } + + $csvExportPath = "$targetStartDate-eachina.csv" + + $billingEntries | Export-Csv -Path $csvExportPath -NoTypeInformation + + Write-Output "Exported $($billingEntries.Count) entries as CSV to $csvExportPath" + + $csvBlobName = $csvExportPath + $csvProperties = @{"ContentType" = "text/csv"}; + Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + + Write-Output "Uploaded to blob storage!" + + Remove-Item -Path $csvExportPath -Force + + Write-Output "Removed $csvExportPath from local disk..." +} +else +{ + if ($consumptionScope -eq "Subscription") + { + $CostDetailsSupportedQuotaIDs = @('EnterpriseAgreement_2014-09-01','Internal_2014-09-01','CSP_2015-05-01') + $ConsumptionSupportedQuotaIDs = @('PayAsYouGo_2014-09-01','MSDN_2014-09-01') + + foreach ($subscription in $subscriptions) + { + $subscriptionQuotaID = $subscription.SubscriptionPolicies.QuotaId + + if ($subscriptionQuotaID -in $ConsumptionSupportedQuotaIDs -or $consumptionAPIOption -eq "UsageDetails") + { + $consumption = $null + $billingEntries = @() + + $ConsumptionApiPath = "/subscriptions/$($subscription.Id)/providers/Microsoft.Consumption/usageDetails?api-version=2021-10-01&metric=$($consumptionMetric.ToLower())&%24expand=properties%2FmeterDetails%2Cproperties%2FadditionalInfo&%24filter=properties%2FusageStart%20ge%20%27$targetStartDate%27%20and%20properties%2FusageEnd%20le%20%27$targetEndDate%27" + + "Starting consumption export process from $targetStartDate to $targetEndDate for subscription $($subscription.Name)..." + + do + { + if (-not([string]::IsNullOrEmpty($consumption.nextLink))) + { + $ConsumptionApiPath = $consumption.nextLink.Substring($consumption.nextLink.IndexOf("/subscriptions/")) + } + $tries = 0 + $requestSuccess = $false + do + { + try { + $tries++ + $consumption = (Invoke-AzRestMethod -Path $ConsumptionApiPath -Method GET).Content | ConvertFrom-Json + $requestSuccess = $true + } + catch { + $ErrorMessage = $_.Exception.Message + Write-Warning "Error getting consumption data: $ErrorMessage. $tries of 3 tries. Waiting 60 seconds..." + Start-Sleep -s 60 + } + } while ( -not($requestSuccess) -and $tries -lt 3 ) + + foreach ($consumptionLine in $consumption.value) + { + if ((Get-Date $consumptionLine.properties.date).ToString("yyyy-MM-dd") -ge $targetStartDate -and (Get-Date $consumptionLine.properties.date).ToString("yyyy-MM-dd") -le $targetEndDate) + { + if ($consumptionLine.tags) + { + $tags = $consumptionLine.tags | ConvertTo-Json -Compress + } + else + { + $tags = $null + } + + if ([string]::IsNullOrEmpty($consumptionLine.properties.billingProfileId)) + { + # legacy consumption schema + + $billingEntry = New-Object PSObject -Property @{ + Timestamp = $timestamp + AccountName = $consumptionLine.properties.accountName + AccountOwnerId = $consumptionLine.properties.accountOwnerId + AdditionalInfo = $consumptionLine.properties.additionalInfo + benefitId = $consumptionLine.properties.benefitId + benefitName = $consumptionLine.properties.benefitName + BillingAccountId = $consumptionLine.properties.billingAccountId + BillingAccountName = $consumptionLine.properties.billingAccountName + BillingCurrencyCode = $consumptionLine.properties.billingCurrency + BillingPeriodEndDate= $consumptionLine.properties.billingPeriodEndDate + BillingPeriodStartDate= $consumptionLine.properties.billingPeriodStartDate + BillingProfileId = $consumptionLine.properties.billingProfileId + BillingProfileName= $consumptionLine.properties.billingProfileName + ChargeType = $consumptionLine.properties.chargeType + ConsumedService = $consumptionLine.properties.consumedService + CostAllocationRuleName = $consumptionLine.properties.costAllocationRuleName + CostCenter = $consumptionLine.properties.costCenter + CostInBillingCurrency = $consumptionLine.properties.cost + Date = (Get-Date $consumptionLine.properties.date).ToString("MM/dd/yyyy") + EffectivePrice = $consumptionLine.properties.effectivePrice + Frequency = $consumptionLine.properties.frequency + InvoiceSectionName = $consumptionLine.properties.invoiceSection + IsAzureCreditEligible = $consumptionLine.properties.isAzureCreditEligible + MeterCategory = $consumptionLine.properties.meterDetails.meterCategory + MeterId = $consumptionLine.properties.meterId + MeterName = $consumptionLine.properties.meterDetails.meterName + MeterRegion = $consumptionLine.properties.meterDetails.meterRegion + MeterSubCategory = $consumptionLine.properties.meterDetails.meterSubCategory + OfferId = $consumptionLine.properties.offerId + PartNumber = $consumptionLine.properties.partNumber + PayGPrice = $consumptionLine.properties.PayGPrice + PlanName = $consumptionLine.properties.planName + PricingModel = $consumptionLine.properties.pricingModel + ProductName = $consumptionLine.properties.product + PublisherName = $consumptionLine.properties.publisherName + PublisherType = $consumptionLine.properties.publisherType + Quantity = $consumptionLine.properties.quantity + ReservationId = $consumptionLine.properties.reservationId + ReservationName = $consumptionLine.properties.reservationName + ResourceGroup = $consumptionLine.properties.resourceGroup + ResourceId = $consumptionLine.properties.resourceId + ResourceLocation = $consumptionLine.properties.resourceLocation + ResourceName = $consumptionLine.properties.resourceName + ServiceFamily = $consumptionLine.properties.meterDetails.serviceFamily + SubscriptionId = $consumptionLine.properties.subscriptionId + SubscriptionName = $consumptionLine.properties.subscriptionName + Tags = $tags + Term = $consumptionLine.properties.term + UnitOfMeasure = $consumptionLine.properties.meterDetails.unitOfMeasure + UnitPrice = $consumptionLine.properties.unitPrice + } + } + else + { + # MCA consumption schema + $billingEntry = New-Object PSObject -Property @{ + Timestamp = $timestamp + AdditionalInfo = $consumptionLine.properties.additionalInfo + benefitId = $consumptionLine.properties.benefitId + benefitName = $consumptionLine.properties.benefitName + BillingAccountId = $consumptionLine.properties.billingAccountId + BillingAccountName = $consumptionLine.properties.billingAccountName + BillingCurrencyCode = $consumptionLine.properties.billingCurrencyCode + BillingPeriodEndDate= $consumptionLine.properties.billingPeriodEndDate + BillingPeriodStartDate= $consumptionLine.properties.billingPeriodStartDate + BillingProfileId = $consumptionLine.properties.billingProfileId + BillingProfileName= $consumptionLine.properties.billingProfileName + ChargeType = $consumptionLine.properties.chargeType + ConsumedService = $consumptionLine.properties.consumedService + CostAllocationRuleName = $consumptionLine.properties.costAllocationRuleName + CostCenter = $consumptionLine.properties.costCenter + CostInBillingCurrency = $consumptionLine.properties.costInBillingCurrency + costInPricingCurrency = $consumptionLine.properties.costInPricingCurrency + costInUSD = $consumptionLine.properties.costInUSD + customerName = $consumptionLine.properties.customerName + Date = (Get-Date $consumptionLine.properties.date).ToString("MM/dd/yyyy") + EffectivePrice = $consumptionLine.properties.effectivePrice + exchangeRate = $consumptionLine.properties.exchangeRate + exchangeRateDate = $consumptionLine.properties.exchangeRateDate + exchangeRatePricingToBilling = $consumptionLine.properties.exchangeRatePricingToBilling + Frequency = $consumptionLine.properties.frequency + invoiceSectionId = $consumptionLine.properties.invoiceSectionId + InvoiceSectionName = $consumptionLine.properties.invoiceSectionName + IsAzureCreditEligible = $consumptionLine.properties.isAzureCreditEligible + MeterCategory = $consumptionLine.properties.meterCategory + MeterId = $consumptionLine.properties.meterId + MeterName = $consumptionLine.properties.meterName + MeterRegion = $consumptionLine.properties.meterRegion + MeterSubCategory = $consumptionLine.properties.meterSubCategory + PartNumber = $consumptionLine.properties.partNumber + paygCostInBillingCurrency = $consumptionLine.properties.paygCostInBillingCurrency + paygCostInUSD = $consumptionLine.properties.paygCostInUSD + PayGPrice = $consumptionLine.properties.payGPrice + PlanName = $consumptionLine.properties.planName + pricingCurrencyCode = $consumptionLine.properties.pricingCurrencyCode + PricingModel = $consumptionLine.properties.pricingModel + ProductName = $consumptionLine.properties.product + productIdentifier = $consumptionLine.properties.productIdentifier + PublisherName = $consumptionLine.properties.publisherName + PublisherType = $consumptionLine.properties.publisherType + Quantity = $consumptionLine.properties.quantity + ReservationId = $consumptionLine.properties.reservationId + ReservationName = $consumptionLine.properties.reservationName + ResourceGroup = $consumptionLine.properties.resourceGroup + ResourceId = $consumptionLine.properties.instanceName + ResourceLocation = $consumptionLine.properties.resourceLocation + resourceLocationNormalized = $consumptionLine.properties.resourceLocationNormalized + ServiceFamily = $consumptionLine.properties.serviceFamily + SubscriptionId = $consumptionLine.properties.subscriptionGuid + SubscriptionName = $consumptionLine.properties.subscriptionName + Tags = $tags + Term = $consumptionLine.properties.term + UnitOfMeasure = $consumptionLine.properties.unitOfMeasure + UnitPrice = $consumptionLine.properties.unitPrice + } + } + $billingEntries += $billingEntry + } + } + } + while ($requestSuccess -and -not([string]::IsNullOrEmpty($consumption.nextLink))) + + if ($requestSuccess) + { + "Generated $($billingEntries.Count) entries..." + + "Uploading CSV to Storage" + + $ci = [CultureInfo]::new([System.Threading.Thread]::CurrentThread.CurrentCulture.Name) + if ($ci.NumberFormat.NumberDecimalSeparator -ne '.') + { + "Current culture ($($ci.Name)) does not use . as decimal separator" + $ci.NumberFormat.NumberDecimalSeparator = '.' + [System.Threading.Thread]::CurrentThread.CurrentCulture = $ci + } + + $csvExportPath = "$targetStartDate-$($subscription.Id)-$consumptionMetric.csv" + + $billingEntries | Export-Csv -Path $csvExportPath -NoTypeInformation + + $csvBlobName = $csvExportPath + $csvProperties = @{"ContentType" = "text/csv"}; + Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + "[$now] Uploaded $csvBlobName to Blob Storage..." + + Remove-Item -Path $csvExportPath -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + "[$now] Removed $csvExportPath from local disk..." + } + else + { + $global:hadErrors = $true + $global:scopesWithErrors += $ScopeName + Write-Warning "Failed to get consumption data for subscription $($subscription.Name)..." + } + } + elseif ($subscriptionQuotaID -in $CostDetailsSupportedQuotaIDs -or $consumptionAPIOption -eq "CostDetails") + { + "Starting cost details export process from $targetStartDate to $targetEndDate for subscription $($subscription.Name)..." + Generate-CostDetails -ScopeId "/subscriptions/$($subscription.Id)" -ScopeName $subscription.Id + } + else + { + $global:hadErrors = $true + $global:scopesWithErrors += $ScopeName + Write-Warning "Subscription quota $subscriptionQuotaID not supported" + } + } + } + else + { + if ($consumptionScope -eq "BillingAccount") + { + "Starting cost details export process from $targetStartDate to $targetEndDate for Billing Account ID $BillingAccountID..." + Generate-CostDetails -ScopeId "/providers/Microsoft.Billing/billingAccounts/$BillingAccountID" -ScopeName $BillingAccountID.Replace(":","_") + } + if ($consumptionScope -eq "BillingProfile") + { + "Starting cost details export process from $targetStartDate to $targetEndDate for Billing Account ID $BillingAccountID / Billing Profile ID $BillingProfileID ..." + Generate-CostDetails -ScopeId "/providers/Microsoft.Billing/billingAccounts/$BillingAccountID/billingProfiles/$BillingProfileID" -ScopeName $BillingProfileID + } + } +} + +if ($global:hadErrors) +{ + $scopesWithErrorsString = $global:scopesWithErrors -join "," + throw "There were errors during the export process with the following scopes: $scopesWithErrorsString. Please check the output for details." +} \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-PolicyComplianceToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-PolicyComplianceToBlobStorage.ps1 new file mode 100644 index 000000000..f8493abf5 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-PolicyComplianceToBlobStorage.ps1 @@ -0,0 +1,644 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetSubscription, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName, + + [Parameter(Mandatory = $false)] + [ValidateSet("ARG", "ARM")] + [string] $PolicyStatesEndpoint = "ARG" +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$referenceRegion = Get-AutomationVariable -Name "AzureOptimization_ReferenceRegion" -ErrorAction SilentlyContinue # e.g., westeurope +if ([string]::IsNullOrEmpty($referenceRegion)) +{ + $referenceRegion = "westeurope" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_PolicyStatesContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "policystateexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$ARGPageSize = 1000 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +$cloudSuffix = "" + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudSuffix = $externalCloudEnvironment.ToLower() + "-" + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +$allpolicyStates = @() + +Write-Output "Getting subscriptions target $TargetSubscription" +if (-not([string]::IsNullOrEmpty($TargetSubscription))) +{ + $subscriptions = $TargetSubscription + $subscriptionSuffix = $TargetSubscription +} +else +{ + $subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" -and $_.SubscriptionPolicies.QuotaId -notlike "AAD*" } | ForEach-Object { "$($_.Id)"} + $subscriptionSuffix = $cloudSuffix + "all-" + $tenantId +} + +Write-Output "Building Policy display names..." + +$policyAssignments = @{} +$policyInitiatives = @{} +$policyDefinitions = @{} +$excludedAssignmentScopes = @() +$allInitiatives = @() + +if ($PolicyStatesEndpoint -eq "ARG") +{ + $resultsSoFar = 0 + + $argQuery = @" + policyresources + | where type =~ 'microsoft.authorization/policyassignments' + | extend displayName = iif(isnotempty(properties.displayName), tostring(properties.displayName), 'N/A') + | distinct id, displayName + | order by id asc +"@ + + $argAssignmentsTotal = @() + + do + { + if ($resultsSoFar -eq 0) + { + $argAssignments = Search-AzGraph -Query $argQuery -First $ARGPageSize -UseTenantScope + } + else + { + $argAssignments = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -UseTenantScope + } + if ($argAssignments -and $argAssignments.GetType().Name -eq "PSResourceGraphResponse") + { + $argAssignments = $argAssignments.Data + } + $resultsCount = $argAssignments.Count + $resultsSoFar += $resultsCount + $argAssignmentsTotal += $argAssignments + + } while ($resultsCount -eq $ARGPageSize) + + Write-Output "Building $($argAssignmentsTotal.Count) assignment entries" + + foreach ($assignment in $argAssignmentsTotal) + { + $policyAssignments.Add($assignment.id, $assignment.displayName) + } + + $resultsSoFar = 0 + + $argQuery = @" + policyresources + | where type =~ 'microsoft.authorization/policysetdefinitions' + | extend displayName = iif(isnotempty(properties.displayName), tostring(properties.displayName), 'N/A') + | distinct id, displayName + | order by id asc +"@ + + $argInitiativesTotal = @() + + do + { + if ($resultsSoFar -eq 0) + { + $argInitiatives = Search-AzGraph -Query $argQuery -First $ARGPageSize -UseTenantScope + } + else + { + $argInitiatives = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -UseTenantScope + } + if ($argInitiatives -and $argInitiatives.GetType().Name -eq "PSResourceGraphResponse") + { + $argInitiatives = $argInitiatives.Data + } + $resultsCount = $argInitiatives.Count + $resultsSoFar += $resultsCount + $argInitiativesTotal += $argInitiatives + + } while ($resultsCount -eq $ARGPageSize) + + Write-Output "Building $($argInitiativesTotal.Count) initiative entries" + + foreach ($initiative in $argInitiativesTotal) + { + $policyInitiatives.Add($initiative.id, $initiative.displayName) + } + + $resultsSoFar = 0 + + $argQuery = @" + policyresources + | where type =~ 'microsoft.authorization/policydefinitions' + | extend displayName = iif(isnotempty(properties.displayName), tostring(properties.displayName), 'N/A') + | distinct id, displayName + | order by id asc +"@ + + $argDefinitionsTotal = @() + + do + { + if ($resultsSoFar -eq 0) + { + $argDefinitions = Search-AzGraph -Query $argQuery -First $ARGPageSize -UseTenantScope + } + else + { + $argDefinitions = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -UseTenantScope + } + if ($argDefinitions -and $argDefinitions.GetType().Name -eq "PSResourceGraphResponse") + { + $argDefinitions = $argDefinitions.Data + } + $resultsCount = $argDefinitions.Count + $resultsSoFar += $resultsCount + $argDefinitionsTotal += $argDefinitions + + } while ($resultsCount -eq $ARGPageSize) + + Write-Output "Building $($argDefinitionsTotal.Count) definition entries" + + foreach ($definition in $argDefinitionsTotal) + { + $policyDefinitions.Add($definition.id, $definition.displayName) + } +} +else +{ + foreach ($sub in $subscriptions) + { + Select-AzSubscription -SubscriptionId $sub | Out-Null + $assignments = Get-AzPolicyAssignment -IncludeDescendent + foreach ($assignment in $assignments) + { + if (-not($policyAssignments[$assignment.PolicyAssignmentId])) + { + $assignmentName = $assignment.Properties.DisplayName + if([string]::IsNullOrWhiteSpace($assignmentName)) { + $policyAssignments.Add($assignment.PolicyAssignmentId, 'N/A') + } + else { + $policyAssignments.Add($assignment.PolicyAssignmentId, $assignmentName) + } + } + if ($assignment.Properties.NotScopes -and -not($excludedAssignmentScopes | Where-Object { $_.PolicyAssignmentId -eq $assignment.PolicyAssignmentId })) + { + $excludedAssignmentScopes += $assignment + } + } + + $initiatives = Get-AzPolicySetDefinition + foreach ($initiative in $initiatives) + { + if (-not($policyInitiatives[$initiative.PolicySetDefinitionId])) + { + $setDefinitionName = $initiative.Properties.DisplayName + if([string]::IsNullOrWhiteSpace($setDefinitionName)) { + $policyInitiatives.Add($initiative.PolicySetDefinitionId, 'N/A') + } + else { + $policyInitiatives.Add($initiative.PolicySetDefinitionId, $setDefinitionName) + } + } + if (-not($allInitiatives | Where-Object { $_.PolicySetDefinitionId -eq $initiative.PolicySetDefinitionId })) + { + $allInitiatives += $initiative + } + } + + $definitions = Get-AzPolicyDefinition + foreach ($definition in $definitions) + { + if (-not($policyDefinitions[$definition.PolicyDefinitionId])) + { + $definitionName = $initiative.Properties.DisplayName + if([string]::IsNullOrWhiteSpace($definitionName)) { + $policyDefinitions.Add($definition.PolicyDefinitionId, 'N/A') + } + else { + $policyDefinitions.Add($definition.PolicyDefinitionId, $definitionName) + } + } + } + } +} + +$policyStatesTotal = @() + +Write-Output "Querying for Policy states using $PolicyStatesEndpoint endpoint..." + +if ($PolicyStatesEndpoint -eq "ARG") +{ + $resultsSoFar = 0 + + $argQuery = @" + policyresources + | where type =~ 'microsoft.policyinsights/policystates' + | extend complianceState = tostring(properties.complianceState) + | extend complianceReason = tostring(properties.complianceReasonCode) + | where complianceState != 'Compliant' and complianceReason !contains 'ResourceNotFound' + | extend effect = tostring(properties.policyDefinitionAction) + | extend assignmentId = tolower(properties.policyAssignmentId) + | extend definitionId = tolower(properties.policyDefinitionId) + | extend definitionReferenceId = tolower(properties.policyDefinitionReferenceId) + | extend initiativeId = tolower(properties.policySetDefinitionId) + | extend resourceId = tolower(properties.resourceId) + | extend resourceType = tostring(properties.resourceType) + | extend evaluatedOn = todatetime(properties.timestamp) + | summarize StatesCount = count() by id, tenantId, subscriptionId, resourceGroup, resourceId, resourceType, complianceState, complianceReason, effect, assignmentId, definitionReferenceId, definitionId, initiativeId, evaluatedOn + | union ( policyresources + | where type =~ 'microsoft.policyinsights/policystates' + | extend complianceState = tostring(properties.complianceState) + | where complianceState == 'Compliant' + | extend effect = tostring(properties.policyDefinitionAction) + | extend assignmentId = tolower(properties.policyAssignmentId) + | extend definitionId = tolower(properties.policyDefinitionId) + | extend definitionReferenceId = tolower(properties.policyDefinitionReferenceId) + | extend initiativeId = tolower(properties.policySetDefinitionId) + | summarize StatesCount = count() by tenantId, subscriptionId, complianceState, effect, assignmentId, definitionReferenceId, definitionId, initiativeId + ) + | join kind=leftouter ( + resources + | project resourceId=tolower(id), tags + ) on resourceId + | project-away resourceId1 + | order by id asc +"@ + + do + { + if ($resultsSoFar -eq 0) + { + $policyStates = Search-AzGraph -Query $argQuery -First $ARGPageSize -Subscription $subscriptions + } + else + { + $policyStates = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -Subscription $subscriptions + } + if ($policyStates -and $policyStates.GetType().Name -eq "PSResourceGraphResponse") + { + $policyStates = $policyStates.Data + } + $resultsCount = $policyStates.Count + $resultsSoFar += $resultsCount + $policyStatesTotal += $policyStates + + } while ($resultsCount -eq $ARGPageSize) + + Write-Output "Building $($policyStatesTotal.Count) policyState entries" +} +else +{ + foreach ($sub in $subscriptions) + { + Select-AzSubscription -SubscriptionId $sub | Out-Null + $policyStates = Get-AzPolicyState -All + + $nonCompliantStates = $policyStates | Where-Object { $_.ComplianceState -ne "Compliant" } + + foreach ($policyState in $nonCompliantStates) + { + $policyStateObject = New-Object PSObject -Property @{ + tenantId = $tenantId + subscriptionId = $sub + resourceGroup = $policyState.ResourceGroup + resourceId = $policyState.ResourceId + resourceType = $policyState.ResourceType + complianceState = $policyState.ComplianceState + complianceReason = $policyState.AdditionalProperties.complianceReasonCode + effect = $policyState.PolicyDefinitionAction + assignmentId = $policyState.PolicyAssignmentId + initiativeId = $policyState.PolicySetDefinitionId + definitionId = $policyState.PolicyDefinitionId + definitionReferenceId = $policyState.PolicyDefinitionReferenceId + evaluatedOn = $policyState.Timestamp + StatesCount = 1 + } + $policyStatesTotal += $policyStateObject + } + + $compliantStates = $policyStates | Where-Object { $_.ComplianceState -eq "Compliant" } ` + | Group-Object PolicyDefinitionAction, PolicyAssignmentId, PolicyDefinitionId, PolicyDefinitionReferenceId, PolicySetDefinitionId + + foreach ($policyState in $compliantStates) + { + $compliantStateProps = $policyState.Name.Split(',') + $definitionReferenceId = $null + if ($compliantStateProps[3]) + { + $definitionReferenceId = $compliantStateProps[3].Trim().ToLower() + } + $initiativeId = $null + if ($compliantStateProps[4]) + { + $initiativeId = $compliantStateProps[4].Trim().ToLower() + } + + $policyStateObject = New-Object PSObject -Property @{ + tenantId = $tenantId + subscriptionId = $sub + complianceState = "Compliant" + effect = $compliantStateProps[0] + assignmentId = $compliantStateProps[1].Trim().ToLower() + definitionId = $compliantStateProps[2].Trim().ToLower() + definitionReferenceId = $definitionReferenceId + initiativeId = $initiativeId + StatesCount = $policyState.Count + } + $policyStatesTotal += $policyStateObject + } + } + + Write-Output "Building $($policyStatesTotal.Count) policyState entries" +} + +$datetime = (Get-Date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") +$statusDate = $datetime.ToString("yyyy-MM-dd") + +foreach ($policyState in $policyStatesTotal) +{ + $resourceGroup = $null + if ($policyState.resourceGroup) + { + $resourceGroup = $policyState.resourceGroup.ToLower() + } + + if (-not([string]::IsNullOrEmpty($policyState.tags))) + { + $tags = $policyState.tags | ConvertTo-Json -Compress + } + else + { + $tags = $null + } + + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $policyState.tenantId + SubscriptionGuid = $policyState.subscriptionId + ResourceGroupName = $resourceGroup + ResourceId = $policyState.resourceId + ResourceType = $policyState.resourceType + ComplianceState = $policyState.complianceState + ComplianceReason = $policyState.complianceReason + Effect = $policyState.effect + AssignmentId = $policyState.assignmentId + AssignmentName = $policyAssignments[$policyState.assignmentId] + InitiativeId = $policyState.initiativeId + InitiativeName = $policyInitiatives[$policyState.initiativeId] + DefinitionId = $policyState.definitionId + DefinitionName = $policyDefinitions[$policyState.definitionId] + DefinitionReferenceId = $policyState.definitionReferenceId + EvaluatedOn = $policyState.evaluatedOn + StatesCount = $policyState.StatesCount + Tags = $tags + StatusDate = $statusDate + } + + $allpolicyStates += $logentry +} + +if ($PolicyStatesEndpoint -eq "ARG") +{ + $resultsSoFar = 0 + + $argQuery = @" + policyresources + | where type =~ 'microsoft.authorization/policyassignments' + | where array_length(properties.notScopes) > 0 + | mv-expand notScope = properties.notScopes + | extend policyAssignmentId = tolower(id) + | extend assignmentPolicyDefinitionId = tolower(properties.policyDefinitionId) + | join kind=leftouter ( + policyresources + | where type =~ 'microsoft.authorization/policysetdefinitions' + | mv-expand policyDefinition = properties.policyDefinitions + | project policySetDefinitionId = tolower(id), policyDefinitionId = tolower(policyDefinition.policyDefinitionId), policyDefinitionReferenceId = tolower(policyDefinition.policyDefinitionReferenceId) + ) on `$left.assignmentPolicyDefinitionId == `$right.policySetDefinitionId + | project policyAssignmentId, notScope, assignmentPolicyDefinitionId, policySetDefinitionId, policyDefinitionId, policyDefinitionReferenceId + | order by policyDefinitionReferenceId, tostring(notScope) +"@ + + do + { + if ($resultsSoFar -eq 0) + { + $argExcludedAssignments = Search-AzGraph -Query $argQuery -First $ARGPageSize -UseTenantScope + } + else + { + $argExcludedAssignments = Search-AzGraph -Query $argQuery -First $ARGPageSize -Skip $resultsSoFar -UseTenantScope + } + if ($argExcludedAssignments -and $argExcludedAssignments.GetType().Name -eq "PSResourceGraphResponse") + { + $argExcludedAssignments = $argExcludedAssignments.Data + } + $resultsCount = $argExcludedAssignments.Count + $resultsSoFar += $resultsCount + $excludedAssignmentScopes += $argExcludedAssignments + + } while ($resultsCount -eq $ARGPageSize) + + Write-Output "Adding excluded scopes from $($excludedAssignmentScopes.Count) assignments" + + foreach ($excludedAssignmentScope in $excludedAssignmentScopes) + { + if (-not([String]::IsNullOrEmpty($excludedAssignmentScope.policySetDefinitionId))) + { + $initiativeId = $excludedAssignmentScope.policySetDefinitionId + $initiativeName = $policyInitiatives[$initiativeId] + $definitionReferenceId = $excludedAssignmentScope.policyDefinitionReferenceId + $definitionId = $excludedAssignmentScope.policyDefinitionId + } + else + { + $initiativeId = $null + $initiativeName = $null + $definitionReferenceId = $null + $definitionId = $excludedAssignmentScope.assignmentPolicyDefinitionId + } + + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $tenantId + ResourceId = $excludedAssignmentScope.notScope + ComplianceState = 'Excluded' + AssignmentId = $excludedAssignmentScope.policyAssignmentId + AssignmentName = $policyAssignments[$excludedAssignmentScope.policyAssignmentId] + InitiativeId = $initiativeId + InitiativeName = $initiativeName + DefinitionId = $definitionId + DefinitionName = $policyDefinitions[$definitionId] + DefinitionReferenceId = $definitionReferenceId + StatusDate = $statusDate + } + + $allpolicyStates += $logentry + } +} +else +{ + Write-Output "Adding excluded scopes from $($excludedAssignmentScopes.Count) assignments" + + foreach ($excludedAssignment in $excludedAssignmentScopes) + { + $excludedIDs = @() + $excludedInitiative = $allInitiatives | Where-Object { $_.PolicySetDefinitionId -eq $excludedAssignment.Properties.PolicyDefinitionId } + if ($excludedInitiative) + { + $excludedDefinitions = $excludedInitiative.Properties.PolicyDefinitions + foreach ($excludedDefinition in $excludedDefinitions) + { + $excludedIDs += "$($excludedDefinition.policyDefinitionId)|$($excludedDefinition.policyDefinitionReferenceId)" + } + } + else + { + $excludedIDs += $excludedAssignment.Properties.PolicyDefinitionId + } + + foreach ($excludedID in $excludedIDs) + { + $excludedIDParts = $excludedID.Split('|') + $definitionId = $excludedIDParts[0].ToLower() + $definitionReferenceId = $null + if (-not([string]::IsNullOrEmpty($excludedIDParts[1]))) + { + $definitionReferenceId = $excludedIDParts[1].ToLower() + } + + $initiativeId = $null + $initiativeName = $null + if ($excludedInitiative) + { + $initiativeId = $excludedInitiative.PolicySetDefinitionId.ToLower() + $initiativeName = $policyInitiatives[$initiativeId] + } + + foreach ($notScope in $excludedAssignment.Properties.NotScopes) + { + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + TenantGuid = $tenantId + ResourceId = $notScope.ToLower() + ComplianceState = 'Excluded' + AssignmentId = $excludedAssignment.PolicyAssignmentId.ToLower() + AssignmentName = $policyAssignments[$excludedAssignment.PolicyAssignmentId] + InitiativeId = $initiativeId + InitiativeName = $initiativeName + DefinitionId = $definitionId + DefinitionName = $policyDefinitions[$definitionId] + DefinitionReferenceId = $definitionReferenceId + StatusDate = $statusDate + } + + $allpolicyStates += $logentry + } + } + } +} + +Write-Output "Uploading CSV to Storage" + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-policyStates-$subscriptionSuffix.csv" + +$allpolicyStates | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +Write-Output "Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +Write-Output "Removed $csvExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-PriceSheetToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-PriceSheetToBlobStorage.ps1 new file mode 100644 index 000000000..5d042a2f8 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-PriceSheetToBlobStorage.ps1 @@ -0,0 +1,452 @@ +param( + [Parameter(Mandatory = $false)] + [string] $BillingAccountID, + + [Parameter(Mandatory = $false)] + [string] $BillingProfileID, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName, + + [Parameter(Mandatory = $false)] + [string] $billingPeriod, # YYYYMM format + + [Parameter(Mandatory = $false)] + [string] $meterCategories, # comma-separated meter categories (for example, "Virtual Machines,Storage") + + [Parameter(Mandatory = $false)] + [string] $meterRegions # comma-separated billing meter regions (for example, "EU North,EU West") +) + +$ErrorActionPreference = "Stop" + +function Authenticate-AzureWithOption { + param ( + [string] $authOption = "ManagedIdentity", + [string] $cloudEnv = "AzureCloud", + [string] $clientID + ) + + switch ($authOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnv -AccountId $clientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnv + break + } + } +} + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_PriceSheetContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "pricesheetexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$consumptionOffsetDays = [int] (Get-AutomationVariable -Name "AzureOptimization_ConsumptionOffsetDays") +$meterCategoriesVar = Get-AutomationVariable -Name "AzureOptimization_PriceSheetMeterCategories" -ErrorAction SilentlyContinue +$meterRegionsVar = Get-AutomationVariable -Name "AzureOptimization_PriceSheetMeterRegions" -ErrorAction SilentlyContinue +$BillingAccountIDVar = Get-AutomationVariable -Name "AzureOptimization_BillingAccountID" -ErrorAction SilentlyContinue +$BillingProfileIDVar = Get-AutomationVariable -Name "AzureOptimization_BillingProfileID" -ErrorAction SilentlyContinue + +"Logging in to Azure with $authenticationOption..." + +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + Authenticate-AzureWithOption -authOption $authenticationOption -cloudEnv $cloudEnvironment -clientID $uamiClientID +} +else +{ + Authenticate-AzureWithOption -authOption $authenticationOption -cloudEnv $cloudEnvironment +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudEnvironment = $externalCloudEnvironment +} + +# compute billing period + +if ([string]::IsNullOrEmpty($billingPeriod)) +{ + $billingPeriod = (Get-Date).Date.AddDays($consumptionOffsetDays * -1).ToString("yyyyMM") +} + +$exportDate = (Get-Date).ToUniversalTime().ToString("yyyyMMdd") + +if ([string]::IsNullOrEmpty($BillingAccountID) -and -not([string]::IsNullOrEmpty($BillingAccountIDVar))) +{ + $BillingAccountID = $BillingAccountIDVar +} + +if ([string]::IsNullOrEmpty($BillingProfileID) -and -not([string]::IsNullOrEmpty($BillingProfileIDVar))) +{ + $BillingProfileID = $BillingProfileIDVar +} + +$mcaBillingAccountIdRegex = "([A-Za-z0-9]+(-[A-Za-z0-9]+)+):([A-Za-z0-9]+(-[A-Za-z0-9]+)+)_[0-9]{4}-[0-9]{2}-[0-9]{2}" +$mcaBillingProfileIdRegex = "([A-Za-z0-9]+(-[A-Za-z0-9]+)+)" + +if ([string]::IsNullOrEmpty($BillingAccountID)) +{ + throw "Billing Account ID undefined. Use either the AzureOptimization_BillingAccountID variable or the BillingAccountID parameter" +} +else { + if ($BillingAccountID -match $mcaBillingAccountIdRegex) + { + if ([string]::IsNullOrEmpty($BillingProfileID)) + { + throw "Billing Profile ID undefined for MCA. Use either the AzureOptimization_BillingProfileID variable or the BillingProfileID parameter" + } + if (-not($BillingProfileID -match $mcaBillingProfileIdRegex)) + { + throw "Billing Profile ID does not follow pattern for MCA: ([A-Za-z0-9]+(-[A-Za-z0-9]+)+)" + } + } +} + +if (-not([string]::IsNullOrEmpty($meterCategoriesVar))) +{ + $meterCategories = $meterCategoriesVar +} + +if (-not([string]::IsNullOrEmpty($meterRegionsVar))) +{ + $meterRegions = $meterRegionsVar +} + +$meterCategoryFilters = $null +$meterRegionFilters = $null + +if (-not([string]::IsNullOrEmpty($meterCategories))) +{ + $meterCategoryFilters = $meterCategories.Split(',') +} + +if (-not([string]::IsNullOrEmpty($meterRegions))) +{ + $meterRegionFilters = $meterRegions.Split(',') +} + +function Generate-Pricesheet { + param ( + [string] $InputCSVPath, + [string] $OutputCSVPath, + [string] $HeaderLine + ) + + # header normalization between MCA and EA + $headerConversion = @{ + 'Meter ID' = "MeterID"; + meterId = "MeterID"; + 'Meter name' = "MeterName"; + meterName = "MeterName"; + 'Meter category' = "MeterCategory"; + meterCategory = "MeterCategory"; + 'Meter sub-category' = "MeterSubCategory"; + meterSubCategory = "MeterSubCategory"; + 'Meter region' = "MeterRegion"; + meterRegion = "MeterRegion"; + 'Unit of measure' = "UnitOfMeasure"; + unitOfMeasure = "UnitOfMeasure"; + 'Part number' = "PartNumber"; + 'Unit price' = "UnitPrice"; + unitPrice = "UnitPrice"; + 'Currency code' = "CurrencyCode"; + currency = "CurrencyCode"; + 'Included quantity' = "IncludedQuantity"; + includedQuantity = "IncludedQuantity"; + 'Offer Id' = "OfferId"; + Term = "Term"; + 'Price type' = "PriceType"; + priceType = "PriceType" + } + + $r = [IO.File]::OpenText($InputCSVPath) + $w = [System.IO.StreamWriter]::new($OutputCSVPath) + $lineCounter = 0 + while ($r.Peek() -ge 0) { + $line = $r.ReadLine() + $lineCounter++ + if ($lineCounter -eq $HeaderLine) + { + $headers = $line.Split(",") + + for ($i = 0; $i -lt $headers.Length; $i++) + { + $header = $headers[$i] + if ($headerConversion.ContainsKey($header)) + { + $headers[$i] = $headerConversion[$header] + } + } + + $line = $headers -join "," + + if (-not($line -match "SubCategory")) + { + throw "Pricesheet format has changed at line $HeaderLine - $line" + } + + Write-Output "New headers: $line" + + $w.WriteLine($line) + } + else + { + if ($lineCounter -gt $HeaderLine) + { + $categoryWriteLine = $categoryWriteLineDefault + $regionWriteLine = $regionWriteLineDefault + + foreach ($meterCategory in $meterCategoryFilters) + { + if ($line -match ",$meterCategory,") + { + $categoryWriteLine = $true + break + } + } + + foreach ($meterRegion in $meterRegionFilters) + { + if ($line -match ",$meterRegion,") + { + $regionWriteLine = $true + break + } + } + + if ($categoryWriteLine -eq $true -and $regionWriteLine -eq $true) + { + $w.WriteLine($line) + } + } + } + } + $r.Dispose() + $w.Close() + + $csvBlobName = [System.IO.Path]::GetFileName($OutputCSVPath) + $csvProperties = @{"ContentType" = "text/csv"}; + Set-AzStorageBlobContent -File $OutputCSVPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + + Remove-Item -Path $InputCSVPath -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + Write-Output "[$now] Removed $InputCSVPath from local disk..." + + Remove-Item -Path $OutputCSVPath -Force + + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + Write-Output "[$now] Removed $OutputCSVPath from local disk..." +} + +Write-Output "Starting pricesheet export process for $billingPeriod billing period for Billing Account $BillingAccountID..." + +$MaxTries = 30 # The typical Retry-After is set to 20 seconds. We'll give 10 minutes overall to download the pricesheet report + +if ($BillingAccountID -match $mcaBillingAccountIdRegex) +{ + $PriceSheetApiPath = "/providers/Microsoft.Billing/billingAccounts/$BillingAccountID/billingProfiles/$BillingProfileID/providers/Microsoft.CostManagement/pricesheets/default/download?api-version=2023-03-01&format=csv" + $result = Invoke-AzRestMethod -Path $PriceSheetApiPath -Method POST +} +else +{ + $PriceSheetApiPath = "/providers/Microsoft.Billing/billingAccounts/$BillingAccountID/billingPeriods/$billingPeriod/providers/Microsoft.Consumption/pricesheets/download?api-version=2022-06-01&ln=en" + $result = Invoke-AzRestMethod -Path $PriceSheetApiPath -Method GET +} + +$requestResultPath = $result.Headers.Location.PathAndQuery +if ($result.StatusCode -in (200,202)) +{ + $tries = 0 + $requestSuccess = $false + + Write-Output "Obtained pricesheet results endpoint: $requestResultPath..." + + Write-Output "Was told to wait $($result.Headers.RetryAfter.Delta.TotalSeconds) seconds." + + $sleepSeconds = 60 + if ($result.Headers.RetryAfter.Delta.TotalSeconds -gt 0) + { + $sleepSeconds = $result.Headers.RetryAfter.Delta.TotalSeconds + } + + do + { + $tries++ + Write-Output "Checking whether export is ready (try $tries)..." + + Start-Sleep -Seconds $sleepSeconds + $downloadResult = Invoke-AzRestMethod -Method GET -Path $requestResultPath + + if ($downloadResult.StatusCode -eq 200) + { + Write-Output "Filtering data with meter categories $meterCategories and meter regions $meterRegions to $finalCsvExportPath..." + + $categoryWriteLineDefault = $true + if ($meterCategoryFilters.Count -gt 0) + { + $categoryWriteLineDefault = $false + } + $regionWriteLineDefault = $true + if ($meterRegionFilters.Count -gt 0) + { + $regionWriteLineDefault = $false + } + + Write-Output "Defaulting to meter categories writes $($categoryWriteLineDefault) and meter regions writes $($regionWriteLineDefault)..." + + if ($BillingAccountID -match $mcaBillingAccountIdRegex) + { + Write-Output "Export is ready. Proceeding with ZIP download..." + $downloadUrl = ($downloadResult.Content | ConvertFrom-Json).publishedEntity.properties.downloadUrl + $zipExportPath = "$env:TEMP\pricesheet-$BillingProfileID-$exportDate.zip" + $zipExpandPath = "$env:TEMP\pricesheet" + Invoke-WebRequest -Uri $downloadUrl -OutFile $zipExportPath + Write-Output "Blob downloaded to $zipExportPath successfully." + Expand-Archive -LiteralPath $zipExportPath -DestinationPath $zipExpandPath -Force + Write-Output "Zip expanded to $zipExpandPath successfully." + $csvFiles = Get-ChildItem -Path $zipExpandPath -Filter *.csv -Recurse + foreach ($csvFile in $csvFiles) + { + $csvExportPath = $csvFile.FullName + $finalCsvExportPath = "$env:TEMP\$($csvFile.Name)-final.csv" + Generate-Pricesheet -InputCSVPath $csvExportPath -OutputCSVPath $finalCsvExportPath -HeaderLine 1 + } + Remove-Item -Path $zipExportPath -Force + $now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + Write-Output "[$now] Removed $zipExportPath from local disk..." + } + else + { + Write-Output "Export is ready. Proceeding with CSV download..." + $downloadUrl = ($downloadResult.Content | ConvertFrom-Json).properties.downloadUrl + $csvExportPath = "$env:TEMP\pricesheet-$billingPeriod-$BillingAccountID.csv" + $finalCsvExportPath = "$env:TEMP\pricesheet-$billingPeriod-$BillingAccountID$($meterCategories.Replace(',',''))$($meterRegions.Replace(',',''))-$exportDate-final.csv" + Invoke-WebRequest -Uri $downloadUrl -OutFile $csvExportPath + Write-Output "Blob downloaded to $csvExportPath successfully." + Generate-Pricesheet -InputCSVPath $csvExportPath -OutputCSVPath $finalCsvExportPath -HeaderLine 3 + } + + $requestSuccess = $true + } + elseif ($downloadResult.StatusCode -eq 202) + { + Write-Output "Was told to wait a bit more... $($downloadResult.Headers.RetryAfter.Delta.TotalSeconds) seconds." + + $sleepSeconds = 60 + if ($downloadResult.Headers.RetryAfter.Delta.TotalSeconds -gt 0) + { + $sleepSeconds = $downloadResult.Headers.RetryAfter.Delta.TotalSeconds + } + } + elseif ($downloadResult.StatusCode -eq 401) + { + Write-Output "Had an authentication issue. Will login again and sleep just a couple of seconds." + + if ($authenticationOption -eq "UserAssignedManagedIdentity") + { + Authenticate-AzureWithOption -authOption $authenticationOption -cloudEnv $cloudEnvironment -clientID $uamiClientID + } + else + { + Authenticate-AzureWithOption -authOption $authenticationOption -cloudEnv $cloudEnvironment + } + + $sleepSeconds = 2 + } + else + { + Write-Output "Got an unexpected response code: $($downloadResult.StatusCode)" + } + } + while (-not($requestSuccess) -and $tries -lt $MaxTries) + + if ($tries -ge $MaxTries) + { + throw "Couldn't complete request before the alloted number of $MaxTries retries" + } + + if (-not($requestSuccess)) + { + throw "Error returned by the Download PriceSheet API. Status Code: $($downloadResult.StatusCode). Message: $($downloadResult.Content)" + } + else + { + Write-Output "Export download processing complete." + } +} +else +{ + if ($result.StatusCode -ne 204) + { + throw "Error returned by the Download PriceSheet API. Status Code: $($result.StatusCode). Message: $($result.Content)" + } + else + { + Write-Output "Request returned 204 No Content" + } +} \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-RBACAssignmentsToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-RBACAssignmentsToBlobStorage.ps1 new file mode 100644 index 000000000..e21b1b2c9 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-RBACAssignmentsToBlobStorage.ps1 @@ -0,0 +1,268 @@ +param( + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RBACAssignmentsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "rbacexports" +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +$subscriptions = Get-AzSubscription | Where-Object { $_.State -eq "Enabled" } + +$roleAssignments = @() + +"Iterating through all reachable subscriptions..." + +foreach ($subscription in $subscriptions) { + + Select-AzSubscription -SubscriptionId $subscription.Id -TenantId $tenantId | Out-Null + + $assignments = Get-AzRoleAssignment -IncludeClassicAdministrators -ErrorAction Continue + "Found $($assignments.Count) assignments for $($subscription.Name) subscription..." + + foreach ($assignment in $assignments) { + if ($null -eq $assignment.ObjectId -and $assignment.Scope.Contains($subscription.Id)) + { + $assignmentEntry = New-Object PSObject -Property @{ + Timestamp = $timestamp + TenantGuid = $tenantId + Cloud = $cloudEnvironment + Model = "AzureClassic" + PrincipalId = $assignment.SignInName + Scope = $assignment.Scope + RoleDefinition = $assignment.RoleDefinitionName + } + $roleAssignments += $assignmentEntry + } + else + { + $duplicateRoleAssignment = $roleAssignments | Where-Object { $_.PrincipalId -eq $assignment.ObjectId -and $_.Scope -eq $assignment.Scope -and $_.RoleDefinition -eq $assignment.RoleDefinitionName} + if (-not($duplicateRoleAssignment)) + { + $assignmentEntry = New-Object PSObject -Property @{ + Timestamp = $timestamp + TenantGuid = $tenantId + Cloud = $cloudEnvironment + Model = "AzureRM" + PrincipalId = $assignment.ObjectId + Scope = $assignment.Scope + RoleDefinition = $assignment.RoleDefinitionName + } + $roleAssignments += $assignmentEntry + } + } + } +} + +$fileDate = $datetime.ToString("yyyyMMdd") +$jsonExportPath = "$fileDate-$tenantId-rbacassignments.json" +$csvExportPath = "$fileDate-$tenantId-rbacassignments.csv" + +$roleAssignments | ConvertTo-Json -Depth 3 -Compress | Out-File $jsonExportPath +"Exported to JSON: $($roleAssignments.Count) lines" +$rbacObjectsJson = Get-Content -Path $jsonExportPath | ConvertFrom-Json +"JSON Import: $($rbacObjectsJson.Count) lines" +$rbacObjectsJson | Export-Csv -NoTypeInformation -Path $csvExportPath +"Export to $csvExportPath" + +$csvBlobName = $csvExportPath +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +"[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +"[$now] Removed $csvExportPath from local disk..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +"[$now] Removed $jsonExportPath from local disk..." + +$roleAssignments = @() + +"Getting Microsoft Entra ID roles..." + +#workaround for https://github.com/microsoftgraph/msgraph-sdk-powershell/issues/888 +$localPath = [System.Environment]::GetFolderPath([System.Environment+SpecialFolder]::UserProfile) +if (-not(get-item "$localPath\.graph\" -ErrorAction SilentlyContinue)) +{ + New-Item -Type Directory "$localPath\.graph" +} + +Import-Module Microsoft.Graph.Identity.DirectoryManagement + +switch ($cloudEnvironment) { + "AzureUSGovernment" { + $graphEnvironment = "USGov" + break + } + "AzureChinaCloud" { + $graphEnvironment = "China" + break + } + "AzureGermanCloud" { + $graphEnvironment = "Germany" + break + } + Default { + $graphEnvironment = "Global" + } +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Microsoft Graph with $externalCredentialName external credential..." + Connect-MgGraph -TenantId $externalTenantId -ClientSecretCredential $externalCredential -Environment $graphEnvironment -NoWelcome +} +else +{ + "Logging in to Microsoft Graph with $authenticationOption..." + + switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-MgGraph -Identity -ClientId $uamiClientID -Environment $graphEnvironment -NoWelcome + break + } + Default { #ManagedIdentity + Connect-MgGraph -Identity -Environment $graphEnvironment -NoWelcome + break + } + } +} + +$domainName = (Get-MgDomain | Where-Object { $_.IsVerified -and $_.IsDefault } | Select-Object -First 1).Id + +$roles = Get-MgDirectoryRole -ExpandProperty Members -Property DisplayName,Members +foreach ($role in $roles) +{ + $roleMembers = $role.Members | Where-Object { -not($_.DeletedDateTime) } + foreach ($roleMember in $roleMembers) + { + $assignmentEntry = New-Object PSObject -Property @{ + Timestamp = $timestamp + TenantGuid = $tenantId + Cloud = $cloudEnvironment + Model = "AzureAD" + PrincipalId = $roleMember.Id + Scope = $domainName + RoleDefinition = $role.DisplayName + } + $roleAssignments += $assignmentEntry + } +} + +$fileDate = $datetime.ToString("yyyyMMdd") +$jsonExportPath = "$fileDate-$tenantId-aadrbacassignments.json" +$csvExportPath = "$fileDate-$tenantId-aadrbacassignments.csv" + +$roleAssignments | ConvertTo-Json -Depth 3 -Compress | Out-File $jsonExportPath +"Exported to JSON: $($roleAssignments.Count) lines" +$rbacObjectsJson = Get-Content -Path $jsonExportPath | ConvertFrom-Json +"JSON Import: $($rbacObjectsJson.Count) lines" +$rbacObjectsJson | Export-Csv -NoTypeInformation -Path $csvExportPath +"Export to $csvExportPath" + +$csvBlobName = $csvExportPath +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +"[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +"[$now] Removed $csvExportPath from local disk..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +"[$now] Removed $jsonExportPath from local disk..." diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ReservationsPriceToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ReservationsPriceToBlobStorage.ps1 new file mode 100644 index 000000000..675c0b37f --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ReservationsPriceToBlobStorage.ps1 @@ -0,0 +1,146 @@ +param( + [Parameter(Mandatory = $false)] + [string] $Filter = "serviceName eq 'Virtual Machines' and priceType eq 'Reservation'" # e.g., serviceName eq 'Virtual Machines' and priceType eq 'Reservation' and armRegionName eq 'northeurope' +) + +$ErrorActionPreference = "Stop" + +function Authenticate-AzureWithOption { + param ( + [string] $authOption = "ManagedIdentity", + [string] $cloudEnv = "AzureCloud", + [string] $clientID + ) + + switch ($authOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnv -AccountId $clientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnv + break + } + } +} + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ReservationsPriceContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "reservationspriceexports" +} + +$filterVar = Get-AutomationVariable -Name "AzureOptimization_RetailPricesFilter" -ErrorAction SilentlyContinue +$currencyCode = Get-AutomationVariable -Name "AzureOptimization_RetailPricesCurrencyCode" + +"Logging in to Azure with $authenticationOption..." + +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + Authenticate-AzureWithOption -authOption $authenticationOption -cloudEnv $cloudEnvironment -clientID $uamiClientID +} +else +{ + Authenticate-AzureWithOption -authOption $authenticationOption -cloudEnv $cloudEnvironment +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudEnvironment = $externalCloudEnvironment +} + +if (-not([string]::IsNullOrEmpty($filterVar))) +{ + $Filter = $filterVar +} + +Write-Output "Starting retails prices export process with $currencyCode currency code and filter: $Filter ..." + +$RetailPricesApiPath = "https://prices.azure.com/api/retail/prices?currencyCode='$currencyCode'&`$filter=$Filter" + +$prices = @() + +do +{ + $Response = Invoke-RestMethod -Method Get -Uri $RetailPricesApiPath + if ($Response.Items.Count -gt 0) + { + $prices += $Response.Items + } + $RetailPricesApiPath = $Response.NextPageLink +} while ($Response.NextPageLink) + +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyyMMdd") + +$fileFriendlyFilter = $Filter.Replace(" ","").Replace("'","") +$csvExportPath = "reservationsprice-$timestamp-$fileFriendlyFilter.csv" + +$ci = [CultureInfo]::new([System.Threading.Thread]::CurrentThread.CurrentCulture.Name) +if ($ci.NumberFormat.NumberDecimalSeparator -ne '.') +{ + Write-Output "Current culture ($($ci.Name)) does not use . as decimal separator" + $ci.NumberFormat.NumberDecimalSeparator = '.' + [System.Threading.Thread]::CurrentThread.CurrentCulture = $ci +} + +$prices | Export-Csv -NoTypeInformation -Path $csvExportPath + +Write-Output "Reservations price CSV exported to $csvExportPath successfully." + +$csvBlobName = $csvExportPath +$csvProperties = @{"ContentType" = "text/csv"}; +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ReservationsUsageToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ReservationsUsageToBlobStorage.ps1 new file mode 100644 index 000000000..c51d78fc3 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-ReservationsUsageToBlobStorage.ps1 @@ -0,0 +1,304 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetScope, + + [Parameter(Mandatory = $false)] + [string] $BillingAccountID, + + [Parameter(Mandatory = $false)] + [string] $BillingProfileID, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName, + + [Parameter(Mandatory = $false)] + [string] $targetStartDate, # YYYY-MM-DD format + + [Parameter(Mandatory = $false)] + [string] $targetEndDate # YYYY-MM-DD format +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_ReservationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "reservationsexports" +} + +$BillingAccountIDVar = Get-AutomationVariable -Name "AzureOptimization_BillingAccountID" -ErrorAction SilentlyContinue +$BillingProfileIDVar = Get-AutomationVariable -Name "AzureOptimization_BillingProfileID" -ErrorAction SilentlyContinue + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +$consumptionOffsetDays = [int] (Get-AutomationVariable -Name "AzureOptimization_ConsumptionOffsetDays") + +if ([string]::IsNullOrEmpty($BillingAccountID) -and -not([string]::IsNullOrEmpty($BillingAccountIDVar))) +{ + $BillingAccountID = $BillingAccountIDVar +} + +if ([string]::IsNullOrEmpty($BillingProfileID) -and -not([string]::IsNullOrEmpty($BillingProfileIDVar))) +{ + $BillingProfileID = $BillingProfileIDVar +} + +$mcaBillingAccountIdRegex = "([A-Za-z0-9]+(-[A-Za-z0-9]+)+):([A-Za-z0-9]+(-[A-Za-z0-9]+)+)_[0-9]{4}-[0-9]{2}-[0-9]{2}" +$mcaBillingProfileIdRegex = "([A-Za-z0-9]+(-[A-Za-z0-9]+)+)" + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +# compute start+end dates + +if ([string]::IsNullOrEmpty($targetStartDate) -or [string]::IsNullOrEmpty($targetEndDate)) +{ + $targetStartDate = (Get-Date).Date.AddDays($consumptionOffsetDays * -1).ToString("yyyy-MM-dd") + $targetEndDate = $targetStartDate +} + +if (-not([string]::IsNullOrEmpty($TargetScope))) +{ + $scope = $TargetScope +} +else +{ + if ([string]::IsNullOrEmpty($BillingAccountID)) + { + throw "Billing Account ID undefined. Use either the AzureOptimization_BillingAccountID variable or the BillingAccountID parameter" + } + if ($BillingAccountID -match $mcaBillingAccountIdRegex) + { + if ([string]::IsNullOrEmpty($BillingProfileID)) + { + throw "Billing Profile ID undefined for MCA. Use either the AzureOptimization_BillingProfileID variable or the BillingProfileID parameter" + } + if (-not($BillingProfileID -match $mcaBillingProfileIdRegex)) + { + throw "Billing Profile ID does not follow pattern for MCA: ([A-Za-z0-9]+(-[A-Za-z0-9]+)+)" + } + $scope = "/providers/Microsoft.Billing/billingaccounts/$BillingAccountID/billingProfiles/$BillingProfileID" + } + else + { + $scope = "/providers/Microsoft.Billing/billingaccounts/$BillingAccountID" + } +} + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Starting reservations export process from $targetStartDate to $targetEndDate for scope $scope..." + +# get reservations details + +$reservationsDetailsResponse = $null +$reservationsDetails = @() +$reservationsDetailsPath = "$scope/reservations?api-version=2020-05-01&&refreshSummary=true" + +do +{ + if (-not([string]::IsNullOrEmpty($reservationsDetailsResponse.nextLink))) + { + $reservationsDetailsPath = $reservationsDetailsResponse.nextLink.Substring($reservationsDetailsResponse.nextLink.IndexOf("/providers/")) + } + + $result = Invoke-AzRestMethod -Path $reservationsDetailsPath -Method GET + + if (-not($result.StatusCode -in (200, 201, 202))) + { + throw "Error while getting reservations details: $($result.Content)" + } + + $reservationsDetailsResponse = $result.Content | ConvertFrom-Json + if ($reservationsDetailsResponse.value) + { + $reservationsDetails += $reservationsDetailsResponse.value + } +} +while (-not([string]::IsNullOrEmpty($reservationsDetailsResponse.nextLink))) + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Found $($reservationsDetails.Count) reservation details." + +# get reservations usage + +$reservationsUsage = @() +if ($BillingAccountID -match $mcaBillingAccountIdRegex) +{ + $reservationsUsagePath = "$scope/providers/Microsoft.Consumption/reservationSummaries?api-version=2023-05-01&startDate=$targetStartDate&endDate=$targetEndDate&grain=daily" +} +else +{ + $reservationsUsagePath = "$scope/providers/Microsoft.Consumption/reservationSummaries?api-version=2023-05-01&`$filter=properties/UsageDate ge $targetStartDate and properties/UsageDate le $targetEndDate&grain=daily" +} + +$result = Invoke-AzRestMethod -Path $reservationsUsagePath -Method GET + +if (-not($result.StatusCode -in (200, 201, 202))) +{ + throw "Error while getting reservations usage: $($result.Content)" +} + +$reservationsUsageResponse = $result.Content | ConvertFrom-Json +if ($reservationsUsageResponse.value) +{ + $reservationsUsage += $reservationsUsageResponse.value +} + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Found $($reservationsUsage.Count) reservation usages." + +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +$reservations = @() + +foreach ($usage in $reservationsUsage) +{ + $reservationResourceId = "/providers/microsoft.capacity/reservationorders/$($usage.properties.reservationOrderId)/reservations/$($usage.properties.reservationId)" + $reservationDetail = $reservationsDetails | Where-Object { $_.id -eq $reservationResourceId } + $reservationEntry = New-Object PSObject -Property @{ + ReservationResourceId = $reservationResourceId + ReservationOrderId = $usage.properties.reservationOrderId + ReservationId = $usage.properties.reservationId + DisplayName = $reservationDetail.properties.displayName + SKUName = $usage.properties.skuName + Location = $reservationDetail.location + ResourceType = $reservationDetail.properties.reservedResourceType + AppliedScopeType = $reservationDetail.properties.userFriendlyAppliedScopeType + Term = $reservationDetail.properties.term + ProvisioningState = $reservationDetail.properties.displayProvisioningState + RenewState = $reservationDetail.properties.userFriendlyRenewState + PurchaseDate = $reservationDetail.properties.purchaseDate + ExpiryDate = $reservationDetail.properties.expiryDate + Archived = $reservationDetail.properties.archived + ReservedHours = $usage.properties.reservedHours + UsedHours = $usage.properties.usedHours + UsageDate = $usage.properties.usageDate + MinUtilPercentage = $usage.properties.minUtilizationPercentage + AvgUtilPercentage = $usage.properties.avgUtilizationPercentage + MaxUtilPercentage = $usage.properties.maxUtilizationPercentage + PurchasedQuantity = $usage.properties.purchasedQuantity + RemainingQuantity = $usage.properties.remainingQuantity + TotalReservedQuantity = $usage.properties.totalReservedQuantity + UsedQuantity = $usage.properties.usedQuantity + UtilizedPercentage = $usage.properties.utilizedPercentage + UtilTrend = $reservationDetail.properties.utilization.trend + Util1Days = ($reservationDetail.properties.utilization.aggregates | Where-Object { $_.grain -eq 1 }).value + Util7Days = ($reservationDetail.properties.utilization.aggregates | Where-Object { $_.grain -eq 7 }).value + Util30Days = ($reservationDetail.properties.utilization.aggregates | Where-Object { $_.grain -eq 30 }).value + Scope = $scope + TenantGuid = $tenantId + Cloud = $cloudEnvironment + CollectedDate = $timestamp + Timestamp = $timestamp + } + $reservations += $reservationEntry +} + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Generated $($reservations.Count) entries..." + +if ($BillingAccountID -match $mcaBillingAccountIdRegex) +{ + $csvExportPath = "$targetStartDate-$BillingProfileID.csv" +} +else +{ + $csvExportPath = "$targetStartDate-$BillingAccountID-$($scope.Split('/')[-1]).csv" +} + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploading CSV to Storage" + +$ci = [CultureInfo]::new([System.Threading.Thread]::CurrentThread.CurrentCulture.Name) +if ($ci.NumberFormat.NumberDecimalSeparator -ne '.') +{ + Write-Output "Current culture ($($ci.Name)) does not use . as decimal separator" + $ci.NumberFormat.NumberDecimalSeparator = '.' + [System.Threading.Thread]::CurrentThread.CurrentCulture = $ci +} + +$reservations | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath +$csvProperties = @{"ContentType" = "text/csv"}; +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-SavingsPlansUsageToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-SavingsPlansUsageToBlobStorage.ps1 new file mode 100644 index 000000000..b48856f2c --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Export-SavingsPlansUsageToBlobStorage.ps1 @@ -0,0 +1,263 @@ +param( + [Parameter(Mandatory = $false)] + [string] $TargetScope, + + [Parameter(Mandatory = $false)] + [string] $BillingAccountID, + + [Parameter(Mandatory = $false)] + [string] $BillingProfileID, + + [Parameter(Mandatory = $false)] + [string] $externalCloudEnvironment, + + [Parameter(Mandatory = $false)] + [string] $externalTenantId, + + [Parameter(Mandatory = $false)] + [string] $externalCredentialName +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkEnv = Get-AutomationVariable -Name "AzureOptimization_StorageSinkEnvironment" -ErrorAction SilentlyContinue +if (-not($storageAccountSinkEnv)) +{ + $storageAccountSinkEnv = $cloudEnvironment +} +$storageAccountSinkKeyCred = Get-AutomationPSCredential -Name "AzureOptimization_StorageSinkKey" -ErrorAction SilentlyContinue +$storageAccountSinkKey = $null +if ($storageAccountSinkKeyCred) +{ + $storageAccountSink = $storageAccountSinkKeyCred.UserName + $storageAccountSinkKey = $storageAccountSinkKeyCred.GetNetworkCredential().Password +} + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_SavingsPlansContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ + $storageAccountSinkContainer = "savingsplansexports" +} + +$BillingAccountIDVar = Get-AutomationVariable -Name "AzureOptimization_BillingAccountID" -ErrorAction SilentlyContinue +$BillingProfileIDVar = Get-AutomationVariable -Name "AzureOptimization_BillingProfileID" -ErrorAction SilentlyContinue + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + $externalCredential = Get-AutomationPSCredential -Name $externalCredentialName +} + +if ([string]::IsNullOrEmpty($BillingAccountID) -and -not([string]::IsNullOrEmpty($BillingAccountIDVar))) +{ + $BillingAccountID = $BillingAccountIDVar +} + +if ([string]::IsNullOrEmpty($BillingProfileID) -and -not([string]::IsNullOrEmpty($BillingProfileIDVar))) +{ + $BillingProfileID = $BillingProfileIDVar +} + +$mcaBillingAccountIdRegex = "([A-Za-z0-9]+(-[A-Za-z0-9]+)+):([A-Za-z0-9]+(-[A-Za-z0-9]+)+)_[0-9]{4}-[0-9]{2}-[0-9]{2}" +$mcaBillingProfileIdRegex = "([A-Za-z0-9]+(-[A-Za-z0-9]+)+)" + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +if (-not($storageAccountSinkKey)) +{ + Write-Output "Getting Storage Account context with login" + + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment +} +else +{ + Write-Output "Getting Storage Account context with key" + $saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -StorageAccountKey $storageAccountSinkKey -Environment $storageAccountSinkEnv +} + +if (-not([string]::IsNullOrEmpty($externalCredentialName))) +{ + "Logging in to Azure with $externalCredentialName external credential..." + Connect-AzAccount -ServicePrincipal -EnvironmentName $externalCloudEnvironment -Tenant $externalTenantId -Credential $externalCredential + $cloudEnvironment = $externalCloudEnvironment +} + +$tenantId = (Get-AzContext).Tenant.Id + +if (-not([string]::IsNullOrEmpty($TargetScope))) +{ + $scope = $TargetScope +} +else +{ + if ([string]::IsNullOrEmpty($BillingAccountID)) + { + throw "Billing Account ID undefined. Use either the AzureOptimization_BillingAccountID variable or the BillingAccountID parameter" + } + if ($BillingAccountID -match $mcaBillingAccountIdRegex) + { + if ([string]::IsNullOrEmpty($BillingProfileID)) + { + throw "Billing Profile ID undefined for MCA. Use either the AzureOptimization_BillingProfileID variable or the BillingProfileID parameter" + } + if (-not($BillingProfileID -match $mcaBillingProfileIdRegex)) + { + throw "Billing Profile ID does not follow pattern for MCA: ([A-Za-z0-9]+(-[A-Za-z0-9]+)+)" + } + #$scope = "/providers/Microsoft.BillingBenefits" + $scope = "/providers/Microsoft.Billing/billingaccounts/$BillingAccountID" + } + else + { + $scope = "/providers/Microsoft.Billing/billingaccounts/$BillingAccountID" + } +} + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Starting savings plans export process for scope $scope..." + +$savingsPlansUsage = @() +if ($BillingAccountID -match $mcaBillingAccountIdRegex) +{ + #$savingsPlansUsagePath = "$scope/savingsPlans?api-version=2022-11-01&refreshsummary=true&take=100" + $savingsPlansUsagePath = "$scope/savingsPlans?api-version=2022-10-01-privatepreview&refreshsummary=true&take=100&`$filter=(properties/billingProfileId eq '/providers/Microsoft.Billing/billingAccounts/$BillingAccountID/billingProfiles/$BillingProfileID')" +} +else +{ + $savingsPlansUsagePath = "$scope/savingsPlans?api-version=2020-12-15-privatepreview&refreshsummary=true&take=100" +} + +$result = Invoke-AzRestMethod -Path $savingsPlansUsagePath -Method GET + +if (-not($result.StatusCode -in (200, 201, 202))) +{ + throw "Error while getting savings plans usage: $($result.Content)" +} + +$savingsPlansUsageResponse = $result.Content | ConvertFrom-Json +if ($savingsPlansUsageResponse.value) +{ + $savingsPlansUsage += $savingsPlansUsageResponse.value +} + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Found $($savingsPlansUsage.Count) savings plans usages." + +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +$savingsPlans = @() + +foreach ($usage in $savingsPlansUsage) +{ + $purchaseDate = $usage.properties.purchaseDateTime + if ([string]::IsNullOrEmpty($purchaseDate) -and -not([string]::IsNullOrEmpty($usage.properties.purchaseDate))) + { + $purchaseDate = (Get-Date -Date $usage.properties.purchaseDate).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + } + $expiryDate = $usage.properties.expiryDateTime + if ([string]::IsNullOrEmpty($expiryDate) -and -not([string]::IsNullOrEmpty($usage.properties.expiryDate))) + { + $expiryDate = (Get-Date -Date $usage.properties.expiryDate).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + } + + $savingsPlanEntry = New-Object PSObject -Property @{ + SavingsPlanResourceId = $usage.id + SavingsPlanOrderId = $usage.id.Substring(0,$usage.id.IndexOf("/savingsPlans/")) + SavingsPlanId = $usage.id.Split("/")[-1] + DisplayName = $usage.properties.displayName + SKUName = $usage.sku.name + Term = $usage.properties.term + ProvisioningState = $usage.properties.displayProvisioningState + AppliedScopeType = $usage.properties.userFriendlyAppliedScopeType + RenewState = $usage.properties.renew + PurchaseDate = $purchaseDate + BenefitStart = $usage.properties.benefitStartTime + ExpiryDate = $expiryDate + EffectiveDate = $usage.properties.effectiveDateTime + BillingScopeId = $usage.properties.billingScopeId + BillingAccountId = $usage.properties.billingAccountId + BillingProfileId = $usage.properties.billingProfileId + BillingPlan = $usage.properties.billingPlan + CommitmentGrain = $usage.properties.commitment.grain + CommitmentCurrencyCode = $usage.properties.commitment.currencyCode + CommitmentAmount = $usage.properties.commitment.amount + UtilTrend = $usage.properties.utilization.trend + Util1Days = ($usage.properties.utilization.aggregates | Where-Object { $_.grain -eq 1 }).value + Util7Days = ($usage.properties.utilization.aggregates | Where-Object { $_.grain -eq 7 }).value + Util30Days = ($usage.properties.utilization.aggregates | Where-Object { $_.grain -eq 30 }).value + Scope = $scope + TenantGuid = $tenantId + Cloud = $cloudEnvironment + CollectedDate = $timestamp + Timestamp = $timestamp + } + $savingsPlans += $savingsPlanEntry +} + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Generated $($savingsPlans.Count) entries..." + +$targetDate = $datetime.ToString("yyyy-MM-dd") + +if ($BillingAccountID -match $mcaBillingAccountIdRegex) +{ + $csvExportPath = "$targetDate-$BillingProfileID.csv" +} +else +{ + $csvExportPath = "$targetDate-$BillingAccountID.csv" +} + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploading CSV to Storage" + +$ci = [CultureInfo]::new([System.Threading.Thread]::CurrentThread.CurrentCulture.Name) +if ($ci.NumberFormat.NumberDecimalSeparator -ne '.') +{ + Write-Output "Current culture ($($ci.Name)) does not use . as decimal separator" + $ci.NumberFormat.NumberDecimalSeparator = '.' + [System.Threading.Thread]::CurrentThread.CurrentCulture = $ci +} + +$savingsPlans | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath +$csvProperties = @{"ContentType" = "text/csv"}; +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $csvBlobName to Blob Storage..." + +Remove-Item -Path $csvExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $csvExportPath from local disk..." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 new file mode 100644 index 000000000..b3be2d52f --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 @@ -0,0 +1,344 @@ +param( + [Parameter(Mandatory = $true)] + [string] $StorageSinkContainer +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$sharedKey = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceKey" +$LogAnalyticsChunkSize = [int] (Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsChunkSize" -ErrorAction SilentlyContinue) +if (-not($LogAnalyticsChunkSize -gt 0)) +{ + $LogAnalyticsChunkSize = 6000 +} +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = $StorageSinkContainer +$StorageBlobsPageSize = [int] (Get-AutomationVariable -Name "AzureOptimization_StorageBlobsPageSize" -ErrorAction SilentlyContinue) +if (-not($StorageBlobsPageSize -gt 0)) +{ + $StorageBlobsPageSize = 1000 +} + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +#region Functions + +# Function to create the authorization signature +Function Build-OMSSignature ($workspaceId, $sharedKey, $date, $contentLength, $method, $contentType, $resource) { + $xHeaders = "x-ms-date:" + $date + $stringToHash = $method + "`n" + $contentLength + "`n" + $contentType + "`n" + $xHeaders + "`n" + $resource + $bytesToHash = [Text.Encoding]::UTF8.GetBytes($stringToHash) + $keyBytes = [Convert]::FromBase64String($sharedKey) + $sha256 = New-Object System.Security.Cryptography.HMACSHA256 + $sha256.Key = $keyBytes + $calculatedHash = $sha256.ComputeHash($bytesToHash) + $encodedHash = [Convert]::ToBase64String($calculatedHash) + $authorization = 'SharedKey {0}:{1}' -f $workspaceId, $encodedHash + return $authorization +} + +# Function to create and post the request +Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField, $AzureEnvironment) { + $method = "POST" + $contentType = "application/json" + $resource = "/api/logs" + $rfc1123date = [DateTime]::UtcNow.ToString("r") + $contentLength = $body.Length + $signature = Build-OMSSignature ` + -workspaceId $workspaceId ` + -sharedKey $sharedKey ` + -date $rfc1123date ` + -contentLength $contentLength ` + -method $method ` + -contentType $contentType ` + -resource $resource + + $uri = "https://" + $workspaceId + ".ods.opinsights.azure.com" + $resource + "?api-version=2016-04-01" + if ($AzureEnvironment -eq "AzureChinaCloud") + { + $uri = "https://" + $workspaceId + ".ods.opinsights.azure.cn" + $resource + "?api-version=2016-04-01" + } + if ($AzureEnvironment -eq "AzureUSGovernment") + { + $uri = "https://" + $workspaceId + ".ods.opinsights.azure.us" + $resource + "?api-version=2016-04-01" + } + if ($AzureEnvironment -eq "AzureGermanCloud") + { + throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + } + + $OMSheaders = @{ + "Authorization" = $signature; + "Log-Type" = $logType; + "x-ms-date" = $rfc1123date; + "time-generated-field" = $TimeStampField; + } + + Try { + + $response = Invoke-WebRequest -Uri $uri -Method POST -ContentType $contentType -Headers $OMSheaders -Body $body -UseBasicParsing -TimeoutSec 1000 + } + catch { + if ($_.Exception.Response.StatusCode.Value__ -eq 401) { + "REAUTHENTICATING" + + $response = Invoke-WebRequest -Uri $uri -Method POST -ContentType $contentType -Headers $OMSheaders -Body $body -UseBasicParsing -TimeoutSec 1000 + } + else + { + return $_.Exception.Response.StatusCode.Value__ + } + } + + return $response.StatusCode +} +#endregion Functions + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +# get reference to storage sink +Write-Output "Getting blobs list from $storageAccountSink storage account ($storageAccountSinkContainer container)..." + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +$allblobs = @() + +$continuationToken = $null +do +{ + $blobs = Get-AzStorageBlob -Container $storageAccountSinkContainer -MaxCount $StorageBlobsPageSize -ContinuationToken $continuationToken -Context $saCtx | Sort-Object -Property LastModified + if ($blobs.Count -le 0) { break } + $allblobs += $blobs + $continuationToken = $blobs[$blobs.Count -1].ContinuationToken; +} +While ($null -ne $continuationToken) + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE StorageContainerName = '$storageAccountSinkContainer'" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$Conn.Close() +$Conn.Dispose() + +if ($controlRows.Count -eq 0 -or -not($controlRows[0].LastProcessedDateTime)) +{ + throw "Could not find a valid ingestion control row for $storageAccountSinkContainer" +} + +$controlRow = $controlRows[0] +$lastProcessedLine = $controlRow.LastProcessedLine +$lastProcessedDateTime = $controlRow.LastProcessedDateTime.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +$LogAnalyticsSuffix = $controlRow.LogAnalyticsSuffix +$logname = $lognamePrefix + $LogAnalyticsSuffix + +Write-Output "Processing blobs modified after $lastProcessedDateTime (line $lastProcessedLine) and ingesting them into the $($logname)_CL table..." + +$newProcessedTime = $null + +$unprocessedBlobs = @() + +foreach ($blob in $allblobs) { + $blobLastModified = $blob.LastModified.UtcDateTime.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + if ($lastProcessedDateTime -lt $blobLastModified -or ` + ($lastProcessedDateTime -eq $blobLastModified -and $lastProcessedLine -gt 0)) { + Write-Output "$($blob.Name) found (modified on $blobLastModified)" + $unprocessedBlobs += $blob + } +} + +$unprocessedBlobs = $unprocessedBlobs | Sort-Object -Property LastModified + +Write-Output "Found $($unprocessedBlobs.Count) new blobs to process..." + +foreach ($blob in $unprocessedBlobs) { + $newProcessedTime = $blob.LastModified.UtcDateTime.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + Write-Output "About to process $($blob.Name) ($($blob.Length) bytes)..." + $blobFilePath = "$env:TEMP\$($blob.Name)" + Get-AzStorageBlobContent -CloudBlob $blob.ICloudBlob -Context $saCtx -Force -Destination $blobFilePath | Out-Null + + $r = [IO.File]::OpenText($blobFilePath) + + $linesProcessed = 0 + $lineCounter = 0 + $chunkLines = @() + + while ($r.Peek() -ge 0) + { + $line = $r.ReadLine() + if ($lineCounter -eq 0) + { + $header = $line + $chunkLines += $line + } + else + { + $linesProcessed++ + } + if ($lastProcessedLine -lt $linesProcessed -and $lineCounter -gt 0) + { + $chunkLines += $line + } + if (($lineCounter -eq $LogAnalyticsChunkSize -or $r.Peek() -lt 0) -and $linesProcessed -gt 0) + { + $csvObject = $chunkLines | ConvertFrom-Csv + $jsonObject = ConvertTo-Json -InputObject $csvObject + + if ($null -ne $jsonObject) + { + $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment + + if ($res -ge 200 -and $res -lt 300) + { + Write-Output "Succesfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" + } + else + { + Write-Warning "Failed to upload $lineCounter $LogAnalyticsSuffix rows. Error code: $res" + $r.Dispose() + Remove-Item -Path $blobFilePath -Force + throw + } + } + else + { + Write-Warning "Skipped uploading $lineCounter $LogAnalyticsSuffix rows. Null JSON object." + } + + if ($r.Peek() -lt 0) { + $lastProcessedLine = -1 + } + else { + $lastProcessedLine = $linesProcessed - 1 + } + + $updatedLastProcessedLine = $lastProcessedLine + $updatedLastProcessedDateTime = $lastProcessedDateTime + if ($r.Peek() -lt 0) { + $updatedLastProcessedDateTime = $newProcessedTime + } + $lastProcessedDateTime = $updatedLastProcessedDateTime + Write-Output "Updating last processed time / line to $($updatedLastProcessedDateTime) / $updatedLastProcessedLine" + $sqlStatement = "UPDATE [$LogAnalyticsIngestControlTable] SET LastProcessedLine = $updatedLastProcessedLine, LastProcessedDateTime = '$updatedLastProcessedDateTime' WHERE StorageContainerName = '$storageAccountSinkContainer'" + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandText = $sqlStatement + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.ExecuteReader() + $Conn.Close() + $Conn.Dispose() + + $chunkLines = @() + $chunkLines += $header + $lineCounter = 1 + } + else + { + $lineCounter++ + } + } + $r.Dispose() + + if ($linesProcessed -eq 0) + { + Write-Output "No rows found" + $updatedLastProcessedLine = -1 + $updatedLastProcessedDateTime = $newProcessedTime + Write-Output "Updating last processed time / line to $($updatedLastProcessedDateTime) / $updatedLastProcessedLine" + $sqlStatement = "UPDATE [$LogAnalyticsIngestControlTable] SET LastProcessedLine = $updatedLastProcessedLine, LastProcessedDateTime = '$updatedLastProcessedDateTime' WHERE StorageContainerName = '$storageAccountSinkContainer'" + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandText = $sqlStatement + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.ExecuteReader() + $Conn.Close() + $Conn.Dispose() + } + else + { + Write-Output "Processed $linesProcessed row(s) in total." + } + + Remove-Item -Path $blobFilePath -Force +} + +Write-Output "DONE" \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/maintenance/CleanUp-OlderRecommendationsFromSqlServer.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/maintenance/CleanUp-OlderRecommendationsFromSqlServer.ps1 new file mode 100644 index 000000000..e86f49753 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/maintenance/CleanUp-OlderRecommendationsFromSqlServer.ps1 @@ -0,0 +1,84 @@ +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} +$RecommendationsMaxAge = [int] (Get-AutomationVariable -Name "AzureOptimization_RecommendationsMaxAgeInDays" -ErrorAction SilentlyContinue) +if (-not($RecommendationsMaxAge -gt 0)) +{ + $RecommendationsMaxAge = 365 +} + +$recommendationsTable = "Recommendations" +$SqlTimeout = 120 + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +$tries = 0 +$connectionSuccess = $false + +Write-Output "Cleaning up recommendations older than $RecommendationsMaxAge days..." + +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = 0 + $Cmd.CommandText = "DELETE FROM [dbo].[$recommendationsTable] WHERE GeneratedDate < GETDATE()-$RecommendationsMaxAge" + $DeletedRows = $Cmd.ExecuteNonQuery() + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } + finally { + $Conn.Close() + $Conn.Dispose() + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +Write-Output "Cleaned up $DeletedRows recommendations." \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 new file mode 100644 index 000000000..f6ff02274 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 @@ -0,0 +1,325 @@ +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$sharedKey = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceKey" +$LogAnalyticsChunkSize = [int] (Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsChunkSize" -ErrorAction SilentlyContinue) +if (-not($LogAnalyticsChunkSize -gt 0)) +{ + $LogAnalyticsChunkSize = 6000 +} +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} +$StorageBlobsPageSize = [int] (Get-AutomationVariable -Name "AzureOptimization_StorageBlobsPageSize" -ErrorAction SilentlyContinue) +if (-not($StorageBlobsPageSize -gt 0)) +{ + $StorageBlobsPageSize = 1000 +} + +#region Functions + +# Function to create the authorization signature +Function Build-OMSSignature ($workspaceId, $sharedKey, $date, $contentLength, $method, $contentType, $resource) { + $xHeaders = "x-ms-date:" + $date + $stringToHash = $method + "`n" + $contentLength + "`n" + $contentType + "`n" + $xHeaders + "`n" + $resource + $bytesToHash = [Text.Encoding]::UTF8.GetBytes($stringToHash) + $keyBytes = [Convert]::FromBase64String($sharedKey) + $sha256 = New-Object System.Security.Cryptography.HMACSHA256 + $sha256.Key = $keyBytes + $calculatedHash = $sha256.ComputeHash($bytesToHash) + $encodedHash = [Convert]::ToBase64String($calculatedHash) + $authorization = 'SharedKey {0}:{1}' -f $workspaceId, $encodedHash + return $authorization +} + +# Function to create and post the request +Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField, $AzureEnvironment) { + $method = "POST" + $contentType = "application/json" + $resource = "/api/logs" + $rfc1123date = [DateTime]::UtcNow.ToString("r") + $contentLength = $body.Length + $signature = Build-OMSSignature ` + -workspaceId $workspaceId ` + -sharedKey $sharedKey ` + -date $rfc1123date ` + -contentLength $contentLength ` + -method $method ` + -contentType $contentType ` + -resource $resource + + $uri = "https://" + $workspaceId + ".ods.opinsights.azure.com" + $resource + "?api-version=2016-04-01" + if ($AzureEnvironment -eq "AzureChinaCloud") + { + $uri = "https://" + $workspaceId + ".ods.opinsights.azure.cn" + $resource + "?api-version=2016-04-01" + } + if ($AzureEnvironment -eq "AzureUSGovernment") + { + $uri = "https://" + $workspaceId + ".ods.opinsights.azure.us" + $resource + "?api-version=2016-04-01" + } + if ($AzureEnvironment -eq "AzureGermanCloud") + { + throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + } + + $OMSheaders = @{ + "Authorization" = $signature; + "Log-Type" = $logType; + "x-ms-date" = $rfc1123date; + "time-generated-field" = $TimeStampField; + } + + Try { + + $response = Invoke-WebRequest -Uri $uri -Method POST -ContentType $contentType -Headers $OMSheaders -Body $body -UseBasicParsing -TimeoutSec 1000 + } + catch { + if ($_.Exception.Response.StatusCode.Value__ -eq 401) { + "REAUTHENTICATING" + + $response = Invoke-WebRequest -Uri $uri -Method POST -ContentType $contentType -Headers $OMSheaders -Body $body -UseBasicParsing -TimeoutSec 1000 + } + else + { + return $_.Exception.Response.StatusCode.Value__ + } + } + + return $response.StatusCode +} +#endregion Functions + + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +# get reference to storage sink +Write-Output "Getting reference to $storageAccountSink storage account (recommendations exports sink)" + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +$allblobs = @() + +Write-Output "Getting blobs list..." +$continuationToken = $null +do +{ + $blobs = Get-AzStorageBlob -Container $storageAccountSinkContainer -MaxCount $StorageBlobsPageSize -ContinuationToken $continuationToken -Context $saCtx | Sort-Object -Property LastModified + if ($blobs.Count -le 0) { break } + $allblobs += $blobs + $continuationToken = $blobs[$blobs.Count -1].ContinuationToken; +} +While ($null -ne $continuationToken) + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE StorageContainerName = '$storageAccountSinkContainer'" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$Conn.Close() +$Conn.Dispose() + +if ($controlRows.Count -eq 0 -or -not($controlRows[0].LastProcessedDateTime)) +{ + throw "Could not find a valid ingestion control row for $storageAccountSinkContainer" +} + +$controlRow = $controlRows[0] +$lastProcessedLine = $controlRow.LastProcessedLine +$lastProcessedDateTime = $controlRow.LastProcessedDateTime.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +$LogAnalyticsSuffix = $controlRow.LogAnalyticsSuffix +$logname = $lognamePrefix + $LogAnalyticsSuffix + +Write-Output "Processing blobs modified after $lastProcessedDateTime (line $lastProcessedLine) and ingesting them into the $($logname)_CL table..." + +$newProcessedTime = $null + +$unprocessedBlobs = @() + +foreach ($blob in $allblobs) { + $blobLastModified = $blob.LastModified.UtcDateTime.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + if ($lastProcessedDateTime -lt $blobLastModified -or ` + ($lastProcessedDateTime -eq $blobLastModified -and $lastProcessedLine -gt 0)) { + Write-Output "$($blob.Name) found (modified on $blobLastModified)" + $unprocessedBlobs += $blob + } +} + +$unprocessedBlobs = $unprocessedBlobs | Sort-Object -Property LastModified + +Write-Output "Found $($unprocessedBlobs.Count) new blobs to process..." + +foreach ($blob in $unprocessedBlobs) { + $newProcessedTime = $blob.LastModified.UtcDateTime.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + Write-Output "About to process $($blob.Name)..." + Get-AzStorageBlobContent -CloudBlob $blob.ICloudBlob -Context $saCtx -Force + $jsonObject = Get-Content -Path $blob.Name | ConvertFrom-Json + Write-Output "Blob contains $($jsonObject.Count) results..." + + if ($null -eq $jsonObject) + { + $recCount = 0 + } + elseif ($null -eq $jsonObject.Count) + { + $recCount = 1 + } + else + { + $recCount = $jsonObject.Count + } + + $linesProcessed = 0 + $jsonObjectSplitted = @() + + if ($recCount -gt 1) + { + for ($i = 0; $i -lt $recCount; $i += $LogAnalyticsChunkSize) { + $jsonObjectSplitted += , @($jsonObject[$i..($i + ($LogAnalyticsChunkSize - 1))]); + } + } + else + { + $jsonObjectArray = @() + $jsonObjectArray += $jsonObject + $jsonObjectSplitted += , $jsonObjectArray + } + + for ($j = 0; $j -lt $jsonObjectSplitted.Count; $j++) + { + if ($jsonObjectSplitted[$j]) + { + $currentObjectLines = $jsonObjectSplitted[$j].Count + if ($lastProcessedLine -lt $linesProcessed) + { + for ($i = 0; $i -lt $jsonObjectSplitted[$j].Count; $i++) + { + $jsonObjectSplitted[$j][$i].RecommendationDescription = $jsonObjectSplitted[$j][$i].RecommendationDescription.Replace("'", "") + $jsonObjectSplitted[$j][$i].RecommendationAction = $jsonObjectSplitted[$j][$i].RecommendationAction.Replace("'", "") + $jsonObjectSplitted[$j][$i].AdditionalInfo = $jsonObjectSplitted[$j][$i].AdditionalInfo | ConvertTo-Json -Compress + $jsonObjectSplitted[$j][$i].Tags = $jsonObjectSplitted[$j][$i].Tags | ConvertTo-Json -Compress + } + + $jsonObject = ConvertTo-Json -InputObject $jsonObjectSplitted[$j] + $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment + If ($res -ge 200 -and $res -lt 300) { + Write-Output "Succesfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" + $linesProcessed += $currentObjectLines + if ($j -eq ($jsonObjectSplitted.Count - 1)) { + $lastProcessedLine = -1 + } + else { + $lastProcessedLine = $linesProcessed - 1 + } + + $updatedLastProcessedLine = $lastProcessedLine + $updatedLastProcessedDateTime = $lastProcessedDateTime + if ($j -eq ($jsonObjectSplitted.Count - 1)) { + $updatedLastProcessedDateTime = $newProcessedTime + } + $lastProcessedDateTime = $updatedLastProcessedDateTime + Write-Output "Updating last processed time / line to $($updatedLastProcessedDateTime) / $updatedLastProcessedLine" + $sqlStatement = "UPDATE [$LogAnalyticsIngestControlTable] SET LastProcessedLine = $updatedLastProcessedLine, LastProcessedDateTime = '$updatedLastProcessedDateTime' WHERE StorageContainerName = '$storageAccountSinkContainer'" + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandText = $sqlStatement + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.ExecuteReader() + $Conn.Close() + $Conn.Dispose() + } + Else { + $linesProcessed += $currentObjectLines + Write-Warning "Failed to upload $currentObjectLines $LogAnalyticsSuffix rows. Error code: $res" + throw + } + } + else + { + $linesProcessed += $currentObjectLines + } + } + } + + Remove-Item -Path $blob.Name -Force +} + +Write-Output "DONE" \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-RecommendationsToSQLServer.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-RecommendationsToSQLServer.ps1 new file mode 100644 index 000000000..eb9d9e963 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-RecommendationsToSQLServer.ps1 @@ -0,0 +1,291 @@ +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} +$ChunkSize = [int] (Get-AutomationVariable -Name "AzureOptimization_SQLServerInsertSize" -ErrorAction SilentlyContinue) +if (-not($ChunkSize -gt 0)) +{ + $ChunkSize = 900 +} +$SqlTimeout = 120 + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} +$StorageBlobsPageSize = [int] (Get-AutomationVariable -Name "AzureOptimization_StorageBlobsPageSize" -ErrorAction SilentlyContinue) +if (-not($StorageBlobsPageSize -gt 0)) +{ + $StorageBlobsPageSize = 1000 +} + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +# get reference to storage sink +Write-Output "Getting reference to $storageAccountSink storage account (recommendations exports sink)" + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +$allblobs = @() + +Write-Output "Getting blobs list..." +$continuationToken = $null +do +{ + $blobs = Get-AzStorageBlob -Container $storageAccountSinkContainer -MaxCount $StorageBlobsPageSize -ContinuationToken $continuationToken -Context $saCtx | Sort-Object -Property LastModified + if ($blobs.Count -le 0) { break } + $allblobs += $blobs + $continuationToken = $blobs[$blobs.Count -1].ContinuationToken; +} +While ($null -ne $continuationToken) + +$SqlServerIngestControlTable = "SqlServerIngestControl" +$recommendationsTable = "Recommendations" + +$tries = 0 +$connectionSuccess = $false + +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$SqlServerIngestControlTable] WHERE StorageContainerName = '$storageAccountSinkContainer' and SqlTableName = '$recommendationsTable'" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +if ($controlRows.Count -eq 0) +{ + throw "Could not find a control row for $storageAccountSinkContainer container and $recommendationsTable table." +} + +$controlRow = $controlRows[0] +$lastProcessedLine = $controlRow.LastProcessedLine +$lastProcessedDateTime = $controlRow.LastProcessedDateTime.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + +$Conn.Close() +$Conn.Dispose() + +Write-Output "Processing blobs modified after $lastProcessedDateTime (line $lastProcessedLine) and ingesting them into the Recommendations SQL table..." + +$newProcessedTime = $null + +$unprocessedBlobs = @() + +foreach ($blob in $allblobs) { + $blobLastModified = $blob.LastModified.UtcDateTime.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + if ($lastProcessedDateTime -lt $blobLastModified -or ` + ($lastProcessedDateTime -eq $blobLastModified -and $lastProcessedLine -gt 0)) { + Write-Output "$($blob.Name) found (modified on $blobLastModified)" + $unprocessedBlobs += $blob + } +} + +$unprocessedBlobs = $unprocessedBlobs | Sort-Object -Property LastModified + +Write-Output "Found $($unprocessedBlobs.Count) new blobs to process..." + +foreach ($blob in $unprocessedBlobs) { + $newProcessedTime = $blob.LastModified.UtcDateTime.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") + Write-Output "About to process $($blob.Name)..." + Get-AzStorageBlobContent -CloudBlob $blob.ICloudBlob -Context $saCtx -Force + $jsonObject = Get-Content -Path $blob.Name | ConvertFrom-Json + Write-Output "Blob contains $($jsonObject.Count) results..." + + if ($null -eq $jsonObject) + { + $recCount = 0 + } + elseif ($null -eq $jsonObject.Count) + { + $recCount = 1 + } + else + { + $recCount = $jsonObject.Count + } + + $linesProcessed = 0 + $jsonObjectSplitted = @() + + if ($recCount -gt 1) + { + for ($i = 0; $i -lt $recCount; $i += $ChunkSize) { + $jsonObjectSplitted += , @($jsonObject[$i..($i + ($ChunkSize - 1))]); + } + } + else + { + $jsonObjectArray = @() + $jsonObjectArray += $jsonObject + $jsonObjectSplitted += , $jsonObjectArray + } + + for ($j = 0; $j -lt $jsonObjectSplitted.Count; $j++) + { + if ($jsonObjectSplitted[$j]) + { + $currentObjectLines = $jsonObjectSplitted[$j].Count + if ($lastProcessedLine -lt $linesProcessed) + { + $sqlStatement = "INSERT INTO [$recommendationsTable]" + $sqlStatement += " (RecommendationId, GeneratedDate, Cloud, Category, ImpactedArea, Impact, RecommendationType, RecommendationSubType," + $sqlStatement += " RecommendationSubTypeId, RecommendationDescription, RecommendationAction, InstanceId, InstanceName, AdditionalInfo," + $sqlStatement += " ResourceGroup, SubscriptionGuid, SubscriptionName, TenantGuid, FitScore, Tags, DetailsUrl) VALUES" + for ($i = 0; $i -lt $jsonObjectSplitted[$j].Count; $i++) + { + $jsonObjectSplitted[$j][$i].RecommendationDescription = $jsonObjectSplitted[$j][$i].RecommendationDescription.Replace("'", "") + $jsonObjectSplitted[$j][$i].RecommendationAction = $jsonObjectSplitted[$j][$i].RecommendationAction.Replace("'", "") + if ($null -ne $jsonObjectSplitted[$j][$i].InstanceName) + { + $jsonObjectSplitted[$j][$i].InstanceName = $jsonObjectSplitted[$j][$i].InstanceName.Replace("'", "") + } + $additionalInfoString = $jsonObjectSplitted[$j][$i].AdditionalInfo | ConvertTo-Json -Compress + $tagsString = $jsonObjectSplitted[$j][$i].Tags | ConvertTo-Json -Compress + $subscriptionGuid = "NULL" + if ($jsonObjectSplitted[$j][$i].SubscriptionGuid) + { + $subscriptionGuid = "'$($jsonObjectSplitted[$j][$i].SubscriptionGuid)'" + } + $subscriptionName = "NULL" + if ($jsonObjectSplitted[$j][$i].SubscriptionName) + { + $subscriptionName = $jsonObjectSplitted[$j][$i].SubscriptionName.Replace("'", "") + $subscriptionName = "'$subscriptionName'" + } + $resourceGroup = "NULL" + if ($jsonObjectSplitted[$j][$i].ResourceGroup) + { + $resourceGroup = "'$($jsonObjectSplitted[$j][$i].ResourceGroup)'" + } + $sqlStatement += " (NEWID(), CONVERT(DATETIME, '$($jsonObjectSplitted[$j][$i].Timestamp)'), '$($jsonObjectSplitted[$j][$i].Cloud)'" + $sqlStatement += ", '$($jsonObjectSplitted[$j][$i].Category)', '$($jsonObjectSplitted[$j][$i].ImpactedArea)'" + $sqlStatement += ", '$($jsonObjectSplitted[$j][$i].Impact)', '$($jsonObjectSplitted[$j][$i].RecommendationType)'" + $sqlStatement += ", '$($jsonObjectSplitted[$j][$i].RecommendationSubType)', '$($jsonObjectSplitted[$j][$i].RecommendationSubTypeId)'" + $sqlStatement += ", '$($jsonObjectSplitted[$j][$i].RecommendationDescription)', '$($jsonObjectSplitted[$j][$i].RecommendationAction)'" + $sqlStatement += ", '$($jsonObjectSplitted[$j][$i].InstanceId)', '$($jsonObjectSplitted[$j][$i].InstanceName)', '$additionalInfoString'" + $sqlStatement += ", $resourceGroup, $subscriptionGuid, $subscriptionName, '$($jsonObjectSplitted[$j][$i].TenantGuid)'" + $sqlStatement += ", $($jsonObjectSplitted[$j][$i].FitScore), '$tagsString', '$($jsonObjectSplitted[$j][$i].DetailsURL)')" + if ($i -ne ($jsonObjectSplitted[$j].Count-1)) + { + $sqlStatement += "," + } + } + + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn2 = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn2.AccessToken = $dbToken.Token + $Conn2.Open() + + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn2 + $Cmd.CommandText = $sqlStatement + $Cmd.CommandTimeout = $SqlTimeout + try + { + $Cmd.ExecuteReader() + } + catch + { + Write-Output "Failed statement: $sqlStatement" + throw + } + + $Conn2.Close() + + $linesProcessed += $currentObjectLines + Write-Output "Processed $linesProcessed lines..." + if ($j -eq ($jsonObjectSplitted.Count - 1)) { + $lastProcessedLine = -1 + } + else { + $lastProcessedLine = $linesProcessed - 1 + } + + $updatedLastProcessedLine = $lastProcessedLine + $updatedLastProcessedDateTime = $lastProcessedDateTime + if ($j -eq ($jsonObjectSplitted.Count - 1)) { + $updatedLastProcessedDateTime = $newProcessedTime + } + $lastProcessedDateTime = $updatedLastProcessedDateTime + Write-Output "Updating last processed time / line to $($updatedLastProcessedDateTime) / $updatedLastProcessedLine" + $sqlStatement = "UPDATE [$SqlServerIngestControlTable] SET LastProcessedLine = $updatedLastProcessedLine, LastProcessedDateTime = '$updatedLastProcessedDateTime' WHERE StorageContainerName = '$storageAccountSinkContainer'" + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandText = $sqlStatement + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.ExecuteReader() + $Conn.Close() + } + else + { + $linesProcessed += $currentObjectLines + } + } + } + + Remove-Item -Path $blob.Name -Force +} + +Write-Output "DONE" \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 new file mode 100644 index 000000000..b37ea56da --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 @@ -0,0 +1,249 @@ +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$sharedKey = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceKey" +$LogAnalyticsChunkSize = [int] (Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsChunkSize" -ErrorAction SilentlyContinue) +if (-not($LogAnalyticsChunkSize -gt 0)) +{ + $LogAnalyticsChunkSize = 6000 +} +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$SqlTimeout = 300 +$FiltersTable = "Filters" + +#region Functions + +# Function to create the authorization signature +Function Build-OMSSignature ($workspaceId, $sharedKey, $date, $contentLength, $method, $contentType, $resource) { + $xHeaders = "x-ms-date:" + $date + $stringToHash = $method + "`n" + $contentLength + "`n" + $contentType + "`n" + $xHeaders + "`n" + $resource + $bytesToHash = [Text.Encoding]::UTF8.GetBytes($stringToHash) + $keyBytes = [Convert]::FromBase64String($sharedKey) + $sha256 = New-Object System.Security.Cryptography.HMACSHA256 + $sha256.Key = $keyBytes + $calculatedHash = $sha256.ComputeHash($bytesToHash) + $encodedHash = [Convert]::ToBase64String($calculatedHash) + $authorization = 'SharedKey {0}:{1}' -f $workspaceId, $encodedHash + return $authorization +} + +# Function to create and post the request +Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField, $AzureEnvironment) { + $method = "POST" + $contentType = "application/json" + $resource = "/api/logs" + $rfc1123date = [DateTime]::UtcNow.ToString("r") + $contentLength = $body.Length + $signature = Build-OMSSignature ` + -workspaceId $workspaceId ` + -sharedKey $sharedKey ` + -date $rfc1123date ` + -contentLength $contentLength ` + -method $method ` + -contentType $contentType ` + -resource $resource + + $uri = "https://" + $workspaceId + ".ods.opinsights.azure.com" + $resource + "?api-version=2016-04-01" + if ($AzureEnvironment -eq "AzureChinaCloud") + { + $uri = "https://" + $workspaceId + ".ods.opinsights.azure.cn" + $resource + "?api-version=2016-04-01" + } + if ($AzureEnvironment -eq "AzureUSGovernment") + { + $uri = "https://" + $workspaceId + ".ods.opinsights.azure.us" + $resource + "?api-version=2016-04-01" + } + if ($AzureEnvironment -eq "AzureGermanCloud") + { + throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + } + + $OMSheaders = @{ + "Authorization" = $signature; + "Log-Type" = $logType; + "x-ms-date" = $rfc1123date; + "time-generated-field" = $TimeStampField; + } + + Try { + + $response = Invoke-WebRequest -Uri $uri -Method POST -ContentType $contentType -Headers $OMSheaders -Body $body -UseBasicParsing -TimeoutSec 1000 + } + catch { + if ($_.Exception.Response.StatusCode.Value__ -eq 401) { + "REAUTHENTICATING" + + $response = Invoke-WebRequest -Uri $uri -Method POST -ContentType $contentType -Headers $OMSheaders -Body $body -UseBasicParsing -TimeoutSec 1000 + } + else + { + return $_.Exception.Response.StatusCode.Value__ + } + } + + return $response.StatusCode +} +#endregion Functions + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +Write-Output "Getting excluded recommendation sub-type IDs..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$FiltersTable] WHERE IsEnabled = 1 AND (FilterEndDate IS NULL OR FilterEndDate > GETDATE())" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $filters = New-Object System.Data.DataTable + $sqlAdapter.Fill($filters) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$Conn.Close() +$Conn.Dispose() + +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +$filterObjects = @() + +$filterObject = New-Object PSObject -Property @{ + Timestamp = $timestamp + FilterId = (New-Guid).Guid + RecommendationSubTypeId = [System.Guid]::empty.Guid + FilterType = "Dummy" + InstanceId = [System.Guid]::empty.Guid + InstanceName = "Dummy" + FilterStartDate = "2019-01-01T00:00:00.000Z" + FilterEndDate = "2199-12-31T23:59:59.000Z" + Author = "AOE" + Notes = "This is a dummy suppression required to build the full suppressions schema in Log Analytics" +} +$filterObjects += $filterObject + +foreach ($filter in $filters) +{ + $filterEndDate = $null + if (-not([string]::IsNullOrEmpty($filter.FilterEndDate))) + { + Write-Output $filter.FilterEndDate + $filterEndDate = $filter.FilterEndDate.ToString("yyyy-MM-ddTHH:mm:00.000Z") + } + else + { + $filterEndDate = "2199-12-31T23:59:59.000Z" + } + + $filterStartDate = $null + if (-not([string]::IsNullOrEmpty($filter.FilterStartDate))) + { + $filterStartDate = $filter.FilterStartDate.ToString("yyyy-MM-ddTHH:mm:00.000Z") + } + else + { + $filterStartDate = "2019-01-01T00:00:00.000Z" + } + + $instanceId = $null + $instanceName = $null + $ObjectGuid = [System.Guid]::empty + if ([System.Guid]::TryParse($filter.InstanceId, [System.Management.Automation.PSReference]$ObjectGuid)) + { + $instanceId = $filter.InstanceId + } + else + { + $instanceName = $filter.InstanceId + } + + $filterObject = New-Object PSObject -Property @{ + Timestamp = $timestamp + FilterId = $filter.FilterId + RecommendationSubTypeId = $filter.RecommendationSubTypeId + FilterType = $filter.FilterType + InstanceId = $instanceId + InstanceName = $instanceName + FilterStartDate = $filterStartDate + FilterEndDate = $filterEndDate + Author = $filter.Author + Notes = $filter.Notes + } + $filterObjects += $filterObject +} + +$filtersJson = $filterObjects | ConvertTo-Json + +$LogAnalyticsSuffix = "SuppressionsV1" +$logname = $lognamePrefix + $LogAnalyticsSuffix + +$res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($filtersJson)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment +If ($res -ge 200 -and $res -lt 300) { + Write-Output "Succesfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" +} +Else { + Write-Warning "Failed to upload $($filterObjects.Count) $LogAnalyticsSuffix rows. Error code: $res" + throw +} diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AADExpiringCredentialsToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AADExpiringCredentialsToBlobStorage.ps1 new file mode 100644 index 000000000..8873f6f81 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AADExpiringCredentialsToBlobStorage.ps1 @@ -0,0 +1,371 @@ +$ErrorActionPreference = "Stop" + +# Collect generic and recommendation-specific variables + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$workspaceSubscriptionId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceSubId" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} + +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +$expiringCredsDays = [int] (Get-AutomationVariable -Name "AzureOptimization_RecommendationAADMinCredValidityDays") +$notExpiringCredsDays = ([int] (Get-AutomationVariable -Name "AzureOptimization_RecommendationAADMaxCredValidityYears")) * 365 + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" + +# Authenticate against Azure + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +Write-Output "Finding tables where recommendations will be generated from..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE CollectedType IN ('AADObjects')" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$aadObjectsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'AADObjects' }).LogAnalyticsSuffix + "_CL" + +Write-Output "Will run query against tables $aadObjectsTableName" + +$Conn.Close() +$Conn.Dispose() + +$recommendationSearchTimeSpan = 1 + +# Grab a context reference to the Storage Account where the recommendations file will be stored + + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +if ($workspaceSubscriptionId -ne $storageAccountSinkSubscriptionId) +{ + Select-AzSubscription -SubscriptionId $workspaceSubscriptionId +} + +$recommendationsErrors = 0 + +# Execute the expiring creds recommendation query against Log Analytics + +$baseQuery = @" + let expiryInterval = $($expiringCredsDays)d; + let AppsAndKeys = materialize ($aadObjectsTableName + | where TimeGenerated > ago(1d) + | where ObjectType_s in ('Application','ServicePrincipal') + | where ObjectSubType_s != 'ManagedIdentity' + | where Keys_s startswith '[' + | extend Keys = parse_json(Keys_s) + | project-away Keys_s + | mv-expand Keys + | evaluate bag_unpack(Keys) + | union ( + $aadObjectsTableName + | where TimeGenerated > ago(1d) + | where ObjectType_s in ('Application','ServicePrincipal') + | where ObjectSubType_s != 'ManagedIdentity' + | where isnotempty(Keys_s) and Keys_s !startswith '[' + | extend Keys = parse_json(Keys_s) + | project-away Keys_s + | evaluate bag_unpack(Keys) + ) + ); + let ExpirationInRisk = AppsAndKeys + | where EndDate < now()+expiryInterval + | project ApplicationId_g, KeyId, RiskDate = EndDate; + let NotInRisk = AppsAndKeys + | where EndDate > now()+expiryInterval + | project ApplicationId_g, KeyId, ComfortDate = EndDate; + let ApplicationsInRisk = ExpirationInRisk + | join kind=leftouter ( NotInRisk ) on ApplicationId_g + | where isempty(ComfortDate) + | summarize ExpiresOn = max(RiskDate) by ApplicationId_g; + AppsAndKeys + | join kind=inner (ApplicationsInRisk) on ApplicationId_g + | summarize ExpiresOn = max(EndDate) by ApplicationId_g, ObjectType_s, DisplayName_s, Cloud_s, KeyType, TenantGuid_g + | order by ExpiresOn desc +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.ApplicationId_g + $detailsURL = "https://portal.azure.$azureTld/#blade/Microsoft_AAD_RegisteredApps/ApplicationMenuBlade/Credentials/appId/$queryInstanceId" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["ObjectType"] = $result.ObjectType_s + $additionalInfoDictionary["KeyType"] = $result.KeyType + $additionalInfoDictionary["ExpiresOn"] = $result.ExpiresOn + + $fitScore = 5 + + $tags = @{} + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "OperationalExcellence" + ImpactedArea = "Microsoft.AzureActiveDirectory/objects" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "AADExpiringCredentials" + RecommendationSubTypeId = "3292c489-2782-498b-aad0-a4cef50f6ca2" + RecommendationDescription = "Microsoft Entra application with credentials expired or about to expire" + RecommendationAction = "Update the Microsoft Entra application credential before the expiration date" + InstanceId = $result.ApplicationId_g + InstanceName = $result.DisplayName_s + AdditionalInfo = $additionalInfoDictionary + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "aadexpiringcerts-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +# Execute the not expiring in less than X years creds recommendation query against Log Analytics + +$baseQuery = @" + let expiryInterval = $($notExpiringCredsDays)d; + let AppsAndKeys = materialize ($aadObjectsTableName + | where TimeGenerated > ago(1d) + | where ObjectSubType_s != 'ManagedIdentity' + | where Keys_s startswith '[' + | extend Keys = parse_json(Keys_s) + | project-away Keys_s + | mv-expand Keys + | evaluate bag_unpack(Keys) + | union ( + $aadObjectsTableName + | where TimeGenerated > ago(1d) + | where ObjectSubType_s != 'ManagedIdentity' + | where isnotempty(Keys_s) and Keys_s !startswith '[' + | extend Keys = parse_json(Keys_s) + | project-away Keys_s + | evaluate bag_unpack(Keys) + ) + ); + AppsAndKeys + | where EndDate > now()+expiryInterval + | project ApplicationId_g, ObjectType_s, DisplayName_s, Cloud_s, KeyType, TenantGuid_g, EndDate +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.ApplicationId_g + $detailsURL = "https://portal.azure.$azureTld/#blade/Microsoft_AAD_RegisteredApps/ApplicationMenuBlade/Credentials/appId/$queryInstanceId" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["ObjectType"] = $result.ObjectType_s + $additionalInfoDictionary["KeyType"] = $result.KeyType + $additionalInfoDictionary["ExpiresOn"] = $result.EndDate + + $fitScore = 5 + + $tags = @{} + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Security" + ImpactedArea = "Microsoft.AzureActiveDirectory/objects" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "AADNotExpiringCredentials" + RecommendationSubTypeId = "ecd969c8-3f16-481a-9577-5ed32e5e1a1d" + RecommendationDescription = "Microsoft Entra application with credentials expiration not set or too far in time" + RecommendationAction = "Update the Microsoft Entra application credential with a shorter expiration date" + InstanceId = $result.ApplicationId_g + InstanceName = $result.DisplayName_s + AdditionalInfo = $additionalInfoDictionary + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "aadnotexpiringcerts-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +if ($recommendationsErrors -gt 0) +{ + throw "Some of the recommendations queries failed. Please, review the job logs for additional information." +} \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-ARMOptimizationsToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-ARMOptimizationsToBlobStorage.ps1 new file mode 100644 index 000000000..31d803150 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-ARMOptimizationsToBlobStorage.ps1 @@ -0,0 +1,517 @@ +$ErrorActionPreference = "Stop" + +# Collect generic and recommendation-specific variables + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$workspaceSubscriptionId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceSubId" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} + +$deploymentDate = Get-AutomationVariable -Name "AzureOptimization_DeploymentDate" # yyyy-MM-dd format +$deploymentDate = $deploymentDate.Replace('"', "") + +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +$assignmentsPercentageThresholdVar = Get-AutomationVariable -Name "AzureOptimization_RecommendationRBACAssignmentsPercentageThreshold" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($assignmentsPercentageThresholdVar) -or $assignmentsPercentageThresholdVar -eq 0) +{ + $assignmentsPercentageThreshold = 80 +} +else +{ + $assignmentsPercentageThreshold = [int] $assignmentsPercentageThresholdVar +} + +$assignmentsSubscriptionsLimitVar = Get-AutomationVariable -Name "AzureOptimization_RecommendationRBACSubscriptionsAssignmentsLimit" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($assignmentsSubscriptionsLimitVar) -or $assignmentsSubscriptionsLimitVar -eq 0) +{ + $assignmentsSubscriptionsLimit = 4000 +} +else +{ + $assignmentsSubscriptionsLimit = [int] $assignmentsSubscriptionsLimitVar +} + +$assignmentsMgmtGroupsLimitVar = Get-AutomationVariable -Name "AzureOptimization_RecommendationRBACMgmtGroupsAssignmentsLimit" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($assignmentsMgmtGroupsLimitVar) -or $assignmentsMgmtGroupsLimitVar -eq 0) +{ + $assignmentsMgmtGroupsLimit = 500 +} +else +{ + $assignmentsMgmtGroupsLimit = [int] $assignmentsMgmtGroupsLimitVar +} + +$rgPercentageThresholdVar = Get-AutomationVariable -Name "AzureOptimization_RecommendationResourceGroupsPerSubPercentageThreshold" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($rgPercentageThresholdVar) -or $rgPercentageThresholdVar -eq 0) +{ + $rgPercentageThreshold = 80 +} +else +{ + $rgPercentageThreshold = [int] $rgPercentageThresholdVar +} + +$rgLimitVar = Get-AutomationVariable -Name "AzureOptimization_RecommendationResourceGroupsPerSubLimit" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($rgLimitVar) -or $rgLimitVar -eq 0) +{ + $rgLimit = 980 +} +else +{ + $rgLimit = [int] $rgLimitVar +} + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" + +# Authenticate against Azure + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +Write-Output "Finding tables where recommendations will be generated from..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE CollectedType IN ('RBACAssignments','ARGResourceContainers')" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$rbacTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'RBACAssignments' }).LogAnalyticsSuffix + "_CL" +$subscriptionsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGResourceContainers' }).LogAnalyticsSuffix + "_CL" + +Write-Output "Will run query against tables $rbacTableName and $subscriptionsTableName" + +$Conn.Close() +$Conn.Dispose() + +$recommendationSearchTimeSpan = 30 + $consumptionOffsetDaysStart + +# Grab a context reference to the Storage Account where the recommendations file will be stored + + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +if ($workspaceSubscriptionId -ne $storageAccountSinkSubscriptionId) +{ + Select-AzSubscription -SubscriptionId $workspaceSubscriptionId +} + +$recommendationsErrors = 0 + +$assignmentsThreshold = $assignmentsSubscriptionsLimit * ($assignmentsPercentageThreshold / 100) + +Write-Output "Looking for subscriptions with more than $assignmentsPercentageThreshold% of the $assignmentsSubscriptionsLimit RBAC assignments limit..." + +$baseQuery = @" + $rbacTableName + | where TimeGenerated > ago(1d) and Model_s == 'AzureRM' and Scope_s startswith '/subscriptions/' + | extend SubscriptionGuid_g = tostring(split(Scope_s, '/')[2]) + | summarize AssignmentsCount=count() by SubscriptionGuid_g, TenantGuid_g, Cloud_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s, Tags_s, InstanceId_s + ) on SubscriptionGuid_g + | where AssignmentsCount >= $assignmentsThreshold +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId_s + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/users" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["assignmentsCount"] = $result.AssignmentsCount + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "OperationalExcellence" + ImpactedArea = "Microsoft.Resources/subscriptions" + Impact = "High" + RecommendationType = "BestPractices" + RecommendationSubType = "HighRBACAssignmentsSubscriptions" + RecommendationSubTypeId = "c6a88d8c-3242-44b0-9793-c91897ef68bc" + RecommendationDescription = "Subscriptions close to the maximum limit of RBAC assignments" + RecommendationAction = "Remove unneeded RBAC assignments or use group-based (or nested group-based) assignments" + InstanceId = $result.InstanceId_s + InstanceName = $result.SubscriptionName + AdditionalInfo = $additionalInfoDictionary + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "subscriptionsrbaclimits-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +$assignmentsThreshold = $assignmentsMgmtGroupsLimit * ($assignmentsPercentageThreshold / 100) + +Write-Output "Looking for management groups with more than $assignmentsPercentageThreshold% of the $assignmentsMgmtGroupsLimit RBAC assignments limit..." + +$baseQuery = @" + $rbacTableName + | where TimeGenerated > ago(1d) and Model_s == 'AzureRM' and Scope_s has 'managementGroups' + | extend ManagementGroupId = tostring(split(Scope_s, '/')[4]) + | summarize AssignmentsCount=count() by ManagementGroupId, TenantGuid_g, Scope_s, Cloud_s + | where AssignmentsCount >= $assignmentsThreshold +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/blade/Microsoft_Azure_ManagementGroups/ManagementGroupBrowseBlade/MGBrowse_overview" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["assignmentsCount"] = $result.AssignmentsCount + + $fitScore = 5 + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "OperationalExcellence" + ImpactedArea = "Microsoft.Management/managementGroups" + Impact = "High" + RecommendationType = "BestPractices" + RecommendationSubType = "HighRBACAssignmentsManagementGroups" + RecommendationSubTypeId = "b36dea3e-ef21-45a9-a704-6f629fab236d" + RecommendationDescription = "Management Groups close to the maximum limit of RBAC assignments" + RecommendationAction = "Remove unneeded RBAC assignments or use group-based (or nested group-based) assignments" + InstanceId = $result.Scope_s + InstanceName = $result.ManagementGroupId + AdditionalInfo = $additionalInfoDictionary + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "mgmtgroupsrbaclimits-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +$rgThreshold = $rgLimit * ($rgPercentageThreshold / 100) + +Write-Output "Looking for subscriptions with more than $rgPercentageThreshold% of the $rgLimit Resource Groups limit..." + +$baseQuery = @" + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions/resourceGroups' + | summarize RGCount=count() by SubscriptionGuid_g, TenantGuid_g, Cloud_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s, Tags_s, InstanceId_s + ) on SubscriptionGuid_g + | where RGCount >= $rgThreshold +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId_s + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/resourceGroups" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["resourceGroupsCount"] = $result.RGCount + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "OperationalExcellence" + ImpactedArea = "Microsoft.Resources/subscriptions" + Impact = "High" + RecommendationType = "BestPractices" + RecommendationSubType = "HighResourceGroupCountSubscriptions" + RecommendationSubTypeId = "4468da8d-1e72-4998-b6d2-3bc38ddd9330" + RecommendationDescription = "Subscriptions close to the maximum limit of resource groups" + RecommendationAction = "Remove unneeded resource groups or split your resource groups across multiple subscriptions" + InstanceId = $result.InstanceId_s + InstanceName = $result.SubscriptionName + AdditionalInfo = $additionalInfoDictionary + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "subscriptionsrglimits-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +if ($recommendationsErrors -gt 0) +{ + throw "Some of the recommendations queries failed. Please, review the job logs for additional information." +} \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AdvisorAsIsToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AdvisorAsIsToBlobStorage.ps1 new file mode 100644 index 000000000..c47bb3b39 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AdvisorAsIsToBlobStorage.ps1 @@ -0,0 +1,315 @@ +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$workspaceSubscriptionId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceSubId" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} + +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +# must be less than or equal to the advisor exports frequency +$daysBackwards = [int] (Get-AutomationVariable -Name "AzureOptimization_RecommendAdvisorPeriodInDays" -ErrorAction SilentlyContinue) +if (-not($daysBackwards -gt 0)) { + $daysBackwards = 7 +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +$CategoryFilter = Get-AutomationVariable -Name "AzureOptimization_AdvisorFilter" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($CategoryFilter)) +{ + $CategoryFilter = "HighAvailability,Security,Performance,OperationalExcellence" # comma-separated list of categories +} + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" +$FiltersTable = "Filters" + +# Authenticate against Azure + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +Write-Output "Finding tables where recommendations will be generated from..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE CollectedType IN ('ARGVirtualMachine','AzureAdvisor','ARGResourceContainers')" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$advisorTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'AzureAdvisor' }).LogAnalyticsSuffix + "_CL" +$subscriptionsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGResourceContainers' }).LogAnalyticsSuffix + "_CL" + +Write-Output "Will run query against tables $subscriptionsTableName and $advisorTableName" + +$Conn.Close() +$Conn.Dispose() + +Write-Output "Getting excluded recommendation sub-type IDs..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$FiltersTable] WHERE FilterType = 'Exclude' AND IsEnabled = 1 AND (FilterEndDate IS NULL OR FilterEndDate > GETDATE())" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $filters = New-Object System.Data.DataTable + $sqlAdapter.Fill($filters) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$Conn.Close() +$Conn.Dispose() + +# Grab a context reference to the Storage Account where the recommendations file will be stored + + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +if ($workspaceSubscriptionId -ne $storageAccountSinkSubscriptionId) +{ + Select-AzSubscription -SubscriptionId $workspaceSubscriptionId +} + +# Execute the recommendation query against Log Analytics + +$FinalCategoryFilter = "" + +if (-not([string]::IsNullOrEmpty($CategoryFilter))) +{ + $categories = $CategoryFilter.Split(',') + for ($i = 0; $i -lt $categories.Count; $i++) + { + $categories[$i] = "'" + $categories[$i] + "'" + } + $FinalCategoryFilter = " and Category in (" + ($categories -join ",") + ")" +} + +$baseQuery = @" +let advisorInterval = $($daysBackwards)d; +$advisorTableName +| where todatetime(TimeGenerated) > ago(advisorInterval)$FinalCategoryFilter +| extend AdvisorRecIdIndex = indexof(InstanceId_s, '/providers/microsoft.advisor/recommendations') +| extend InstanceName_s = iif(isnotempty(InstanceName_s),InstanceName_s,iif(AdvisorRecIdIndex > 0, split(substring(InstanceId_s, 0, AdvisorRecIdIndex),'/')[-1], split(InstanceId_s,'/')[-1])) +| summarize by InstanceId_s, InstanceName_s, Category, Description_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroup, Cloud_s, AdditionalInfo_s, RecommendationText_s, ImpactedArea_s, Impact_s, RecommendationTypeId_g, Tags_s +| join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s +) on SubscriptionGuid_g +"@ + +Write-Output "Getting $CategoryFilter recommendations for $($daysBackwards)d Advisor..." + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $daysBackwards) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + throw "Execution aborted" +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +Write-Output "Generating fit score..." + +foreach ($result in $results) { + + if ($filters | Where-Object { $_.RecommendationSubTypeId -eq $result.RecommendationTypeId_g}) + { + continue + } + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $additionalInfoDictionary = @{} + if (-not([string]::IsNullOrEmpty($result.AdditionalInfo_s))) + { + ($result.AdditionalInfo_s | ConvertFrom-Json).PsObject.Properties | ForEach-Object { $additionalInfoDictionary[$_.Name] = $_.Value } + } + + $fitScore = 5 + + $queryInstanceId = $result.InstanceId_s + + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $recommendationSubType = "Advisor" + $result.Category + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = $result.Category + ImpactedArea = $result.ImpactedArea_s + Impact = $result.Impact_s + RecommendationType = "BestPractices" + RecommendationSubType = $recommendationSubType + RecommendationSubTypeId = $result.RecommendationTypeId_g + RecommendationDescription = $result.Description_s.Replace("'","") + RecommendationAction = $result.RecommendationText_s.Replace("'","") + InstanceId = $result.InstanceId_s + InstanceName = $result.InstanceName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroup + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +Write-Output "Exporting final $($recommendations.Count) results as a JSON file..." + +$fileDate = $datetime.ToString("yyyyMMdd") +$jsonExportPath = "advisor-asis-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +Write-Output "Uploading $jsonExportPath to blob storage..." + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json" }; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AdvisorCostAugmentedToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AdvisorCostAugmentedToBlobStorage.ps1 new file mode 100644 index 000000000..eada80510 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AdvisorCostAugmentedToBlobStorage.ps1 @@ -0,0 +1,903 @@ +$ErrorActionPreference = "Stop" + +function Find-SkuHourlyPrice { + param ( + [object[]] $SKUPriceSheet, + [string] $SKUName + ) + + $skuPriceObject = $null + + if ($SKUPriceSheet) + { + $skuNameParts = $SKUName.Split('_') + + if ($skuNameParts.Count -eq 3) # e.g., Standard_D1_v2 + { + $skuNameFilter = "*" + $skuNameParts[1] + " *" + $skuVersionFilter = "*" + $skuNameParts[2] + $skuPrices = $SKUPriceSheet | Where-Object { $_.MeterName_s -like $skuNameFilter ` + -and $_.MeterName_s -notlike '*Low Priority' -and $_.MeterName_s -notlike '*Expired' ` + -and $_.MeterName_s -like $skuVersionFilter -and $_.MeterSubCategory_s -notlike '*Windows' -and $_.UnitPrice_s -ne 0 } + + if (($skuPrices -or $skuPrices.Count -ge 1) -and $skuPrices.Count -le 2) + { + $skuPriceObject = $skuPrices[0] + } + if ($skuPrices.Count -gt 2) # D1-like scenarios + { + $skuFilter = "*" + $skuNameParts[1] + " " + $skuNameParts[2] + "*" + $skuPrices = $skuPrices | Where-Object { $_.MeterName_s -like $skuFilter } + + if (($skuPrices -or $skuPrices.Count -ge 1) -and $skuPrices.Count -le 2) + { + $skuPriceObject = $skuPrices[0] + } + } + } + + if ($skuNameParts.Count -eq 2) # e.g., Standard_D1 + { + $skuNameFilter = "*" + $skuNameParts[1] + "*" + + $skuPrices = $SKUPriceSheet | Where-Object { $_.MeterName_s -like $skuNameFilter ` + -and $_.MeterName_s -notlike '*Low Priority' -and $_.MeterName_s -notlike '*Expired' ` + -and $_.MeterName_s -notlike '* v*' -and $_.MeterSubCategory_s -notlike '*Windows' -and $_.UnitPrice_s -ne 0 } + + if (($skuPrices -or $skuPrices.Count -ge 1) -and $skuPrices.Count -le 2) + { + $skuPriceObject = $skuPrices[0] + } + if ($skuPrices.Count -gt 2) # D1-like scenarios + { + $skuFilterLeft = "*" + $skuNameParts[1] + "/*" + $skuFilterRight = "*/" + $skuNameParts[1] + "*" + $skuPrices = $skuPrices | Where-Object { $_.MeterName_s -like $skuFilterLeft -or $_.MeterName_s -like $skuFilterRight } + + if (($skuPrices -or $skuPrices.Count -ge 1) -and $skuPrices.Count -le 2) + { + $skuPriceObject = $skuPrices[0] + } + } + } + } + + $targetHourlyPrice = [double]::MaxValue + if ($null -ne $skuPriceObject) + { + $targetUnitHours = [int] (Select-String -InputObject $skuPriceObject.UnitOfMeasure_s -Pattern "^\d+").Matches[0].Value + if ($targetUnitHours -gt 0) + { + $targetHourlyPrice = [double] ($skuPriceObject.UnitPrice_s / $targetUnitHours) + } + } + + return $targetHourlyPrice +} + +# Collect generic and recommendation-specific variables + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$workspaceName = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceName" +$workspaceRG = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceRG" +$workspaceSubscriptionId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceSubId" +$workspaceTenantId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceTenantId" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} + +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$referenceRegion = Get-AutomationVariable -Name "AzureOptimization_ReferenceRegion" + +# must be less than or equal to the advisor exports frequency +$daysBackwards = [int] (Get-AutomationVariable -Name "AzureOptimization_RecommendAdvisorPeriodInDays" -ErrorAction SilentlyContinue) +if (-not($daysBackwards -gt 0)) { + $daysBackwards = 7 +} + +$perfDaysBackwards = [int] (Get-AutomationVariable -Name "AzureOptimization_RecommendPerfPeriodInDays" -ErrorAction SilentlyContinue) +if (-not($perfDaysBackwards -gt 0)) { + $perfDaysBackwards = 7 +} + +$perfTimeGrain = Get-AutomationVariable -Name "AzureOptimization_RecommendPerfTimeGrain" -ErrorAction SilentlyContinue +if (-not($perfTimeGrain)) { + $perfTimeGrain = "1h" +} + +# percentiles variables +$cpuPercentile = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfPercentileCpu" -ErrorAction SilentlyContinue) +if (-not($cpuPercentile -gt 0)) { + $cpuPercentile = 99 +} +$memoryPercentile = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfPercentileMemory" -ErrorAction SilentlyContinue) +if (-not($memoryPercentile -gt 0)) { + $memoryPercentile = 99 +} +$networkPercentile = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfPercentileNetwork" -ErrorAction SilentlyContinue) +if (-not($networkPercentile -gt 0)) { + $networkPercentile = 99 +} +$diskPercentile = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfPercentileDisk" -ErrorAction SilentlyContinue) +if (-not($diskPercentile -gt 0)) { + $diskPercentile = 99 +} + +# perf thresholds variables +$cpuPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdCpuPercentage" -ErrorAction SilentlyContinue) +if (-not($cpuPercentageThreshold -gt 0)) { + $cpuPercentageThreshold = 30 +} +$memoryPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdMemoryPercentage" -ErrorAction SilentlyContinue) +if (-not($memoryPercentageThreshold -gt 0)) { + $memoryPercentageThreshold = 50 +} +$networkMpbsThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdNetworkMbps" -ErrorAction SilentlyContinue) +if (-not($networkMpbsThreshold -gt 0)) { + $networkMpbsThreshold = 750 +} + +# perf thresholds variables (shutdown) +$cpuPercentageShutdownThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdCpuShutdownPercentage" -ErrorAction SilentlyContinue) +if (-not($cpuPercentageShutdownThreshold -gt 0)) { + $cpuPercentageShutdownThreshold = 5 +} +$memoryPercentageShutdownThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdMemoryShutdownPercentage" -ErrorAction SilentlyContinue) +if (-not($memoryPercentageShutdownThreshold -gt 0)) { + $memoryPercentageShutdownThreshold = 100 +} +$networkMpbsShutdownThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdNetworkShutdownMbps" -ErrorAction SilentlyContinue ) +if (-not($networkMpbsShutdownThreshold -gt 0)) { + $networkMpbsShutdownThreshold = 10 +} + +$rightSizeRecommendationId = Get-AutomationVariable -Name "AzureOptimization_RecommendationAdvisorCostRightSizeId" -ErrorAction SilentlyContinue +if (-not($rightSizeRecommendationId)) { + $rightSizeRecommendationId = 'e10b1381-5f0a-47ff-8c7b-37bd13d7c974' +} + +$additionalPerfWorkspaces = Get-AutomationVariable -Name "AzureOptimization_RightSizeAdditionalPerfWorkspaces" -ErrorAction SilentlyContinue + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +$consumptionOffsetDays = [int] (Get-AutomationVariable -Name "AzureOptimization_ConsumptionOffsetDays") +$consumptionOffsetDaysStart = $consumptionOffsetDays + 1 + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" +$FiltersTable = "Filters" + +# Authenticate against Azure + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +Write-Output "Finding tables where recommendations will be generated from..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE CollectedType IN ('ARGVirtualMachine','AzureAdvisor','AzureConsumption','ARGResourceContainers','Pricesheet')" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$vmsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGVirtualMachine' }).LogAnalyticsSuffix + "_CL" +$advisorTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'AzureAdvisor' }).LogAnalyticsSuffix + "_CL" +$consumptionTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'AzureConsumption' }).LogAnalyticsSuffix + "_CL" +$subscriptionsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGResourceContainers' }).LogAnalyticsSuffix + "_CL" +$pricesheetTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'Pricesheet' }).LogAnalyticsSuffix + "_CL" + +Write-Output "Will run query against tables $vmsTableName, $subscriptionsTableName, $advisorTableName, $pricesheetTableName and $consumptionTableName" + +$Conn.Close() +$Conn.Dispose() + +Write-Output "Getting excluded recommendation sub-type IDs..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$FiltersTable] WHERE FilterType = 'Exclude' AND IsEnabled = 1 AND (FilterEndDate IS NULL OR FilterEndDate > GETDATE())" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $filters = New-Object System.Data.DataTable + $sqlAdapter.Fill($filters) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$Conn.Close() +$Conn.Dispose() + +$recommendationSearchTimeSpan = 30 + $consumptionOffsetDaysStart + +# Grab a context reference to the Storage Account where the recommendations file will be stored + + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +if ($workspaceSubscriptionId -ne $storageAccountSinkSubscriptionId) +{ + Select-AzSubscription -SubscriptionId $workspaceSubscriptionId +} + + +Write-Output "Getting Virtual Machine SKUs for the $referenceRegion region..." +# Get all the VM SKUs information for the reference Azure region +$skus = Get-AzComputeResourceSku -Location $referenceRegion | Where-Object { $_.ResourceType -eq "virtualMachines" } + +Write-Output "Getting the current Pricesheet..." + +if ($cloudEnvironment -eq "AzureCloud") +{ + $pricesheetRegion = "EU West" +} + +try +{ + $pricesheetEntries = @() + + $baseQuery = @" + $pricesheetTableName + | where TimeGenerated > ago(14d) + | where MeterCategory_s == 'Virtual Machines' and MeterRegion_s == '$pricesheetRegion' and PriceType_s == 'Consumption' + | distinct MeterName_s, MeterSubCategory_s, MeterCategory_s, MeterRegion_s, UnitPrice_s, UnitOfMeasure_s +"@ + + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days 14) -Wait 600 -IncludeStatistics + $pricesheetEntries = [System.Linq.Enumerable]::ToArray($queryResults.Results) + + Write-Output "Query finished with $($pricesheetEntries.Count) results." + Write-Output "Query statistics: $($queryResults.Statistics.query)" +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + Write-Output "Consumption pricesheet not available, will estimate savings based in cores count..." +} + +$linuxMemoryPerfAdditionalWorkspaces = "" +$windowsMemoryPerfAdditionalWorkspaces = "" +$processorPerfAdditionalWorkspaces = "" +$windowsNetworkPerfAdditionalWorkspaces = "" +$diskPerfAdditionalWorkspaces = "" +if ($additionalPerfWorkspaces) +{ + $additionalWorkspaces = $additionalPerfWorkspaces.Split(",") + foreach ($additionalWorkspace in $additionalWorkspaces) { + $additionalWorkspace = $additionalWorkspace.Trim() + $linuxMemoryPerfAdditionalWorkspaces += @" + | union ( workspace('$additionalWorkspace').Perf + | where TimeGenerated > ago(perfInterval) and _ResourceId in (RightSizeInstanceIds) + | where CounterName == '% Used Memory' + | extend WorkspaceId = TenantId + | summarize hint.strategy=shuffle PMemoryPercentage = percentile(CounterValue, memoryPercentileValue) by _ResourceId, WorkspaceId) +"@ + $windowsMemoryPerfAdditionalWorkspaces += @" + | union ( workspace('$additionalWorkspace').Perf + | where TimeGenerated > ago(perfInterval) and _ResourceId in (RightSizeInstanceIds) + | where CounterName == 'Available MBytes' + | extend WorkspaceId = TenantId + | project TimeGenerated, MemoryAvailableMBs = CounterValue, _ResourceId, WorkspaceId) +"@ + $processorPerfAdditionalWorkspaces += @" + | union ( workspace('$additionalWorkspace').Perf + | where TimeGenerated > ago(perfInterval) and _ResourceId in (RightSizeInstanceIds) + | where ObjectName == 'Processor' and CounterName == '% Processor Time' and InstanceName == '_Total' + | extend WorkspaceId = TenantId + | summarize hint.strategy=shuffle PCPUPercentage = percentile(CounterValue, cpuPercentileValue) by _ResourceId, WorkspaceId) +"@ + $windowsNetworkPerfAdditionalWorkspaces += @" + | union ( workspace('$additionalWorkspace').Perf + | where TimeGenerated > ago(perfInterval) and _ResourceId in (RightSizeInstanceIds) + | where CounterName == 'Bytes Total/sec' + | extend WorkspaceId = TenantId + | summarize hint.strategy=shuffle PCounter = percentile(CounterValue, networkPercentileValue) by InstanceName, _ResourceId, WorkspaceId + | summarize PNetwork = sum(PCounter) by _ResourceId, WorkspaceId) +"@ + $diskPerfAdditionalWorkspaces += @" + | union ( workspace('$additionalWorkspace').Perf + | where TimeGenerated > ago(perfInterval) and _ResourceId in (RightSizeInstanceIds) + | where CounterName in ('Disk Reads/sec', 'Disk Writes/sec', 'Disk Read Bytes/sec', 'Disk Write Bytes/sec') and InstanceName !in ('_Total', 'D:', '/mnt/resource', '/mnt') + | extend WorkspaceId = TenantId + | summarize hint.strategy=shuffle PCounter = percentile(CounterValue, diskPercentileValue) by bin(TimeGenerated, perfTimeGrain), CounterName, InstanceName, _ResourceId, WorkspaceId + | summarize SumPCounter = sum(PCounter) by CounterName, TimeGenerated, _ResourceId, WorkspaceId + | summarize MaxPReadIOPS = maxif(SumPCounter, CounterName == 'Disk Reads/sec'), + MaxPWriteIOPS = maxif(SumPCounter, CounterName == 'Disk Writes/sec'), + MaxPReadMiBps = (maxif(SumPCounter, CounterName == 'Disk Read Bytes/sec') / 1024 / 1024), + MaxPWriteMiBps = (maxif(SumPCounter, CounterName == 'Disk Write Bytes/sec') / 1024 / 1024) by _ResourceId, WorkspaceId) +"@ + } +} + +# Execute the recommendation query against Log Analytics + +$baseQuery = @" +let advisorInterval = $($daysBackwards)d; +let perfInterval = $($perfDaysBackwards)d; +let perfTimeGrain = $perfTimeGrain; +let cpuPercentileValue = $cpuPercentile; +let memoryPercentileValue = $memoryPercentile; +let networkPercentileValue = $networkPercentile; +let diskPercentileValue = $diskPercentile; +let rightSizeRecommendationId = '$rightSizeRecommendationId'; +let billingInterval = 30d; +let etime = todatetime(toscalar($consumptionTableName | where todatetime(Date_s) < now() and todatetime(Date_s) > ago(30d) | summarize max(todatetime(Date_s)))); +let stime = etime-billingInterval; +let RightSizeInstanceIds = materialize($advisorTableName +| where todatetime(TimeGenerated) > ago(advisorInterval) and Category == 'Cost' and RecommendationTypeId_g == rightSizeRecommendationId +| distinct InstanceId_s); +let LinuxMemoryPerf = Perf +| where TimeGenerated > ago(perfInterval) and _ResourceId in (RightSizeInstanceIds) +| where CounterName == '% Used Memory' +| extend WorkspaceId = TenantId +| summarize hint.strategy=shuffle PMemoryPercentage = percentile(CounterValue, memoryPercentileValue) by _ResourceId, WorkspaceId$linuxMemoryPerfAdditionalWorkspaces; +let WindowsMemoryPerf = Perf +| where TimeGenerated > ago(perfInterval) and _ResourceId in (RightSizeInstanceIds) +| where CounterName == 'Available MBytes' +| extend WorkspaceId = TenantId +| project TimeGenerated, MemoryAvailableMBs = CounterValue, _ResourceId, WorkspaceId$windowsMemoryPerfAdditionalWorkspaces; +let MemoryPerf = $vmsTableName +| where TimeGenerated > ago(1d) +| distinct InstanceId_s, MemoryMB_s +| join kind=inner hint.strategy=broadcast ( + WindowsMemoryPerf +) on `$left.InstanceId_s == `$right._ResourceId +| extend MemoryPercentage = todouble(toint(MemoryMB_s) - toint(MemoryAvailableMBs)) / todouble(MemoryMB_s) * 100 +| summarize hint.strategy=shuffle PMemoryPercentage = percentile(MemoryPercentage, memoryPercentileValue) by _ResourceId, WorkspaceId +| union LinuxMemoryPerf; +let ProcessorPerf = Perf +| where TimeGenerated > ago(perfInterval) and _ResourceId in (RightSizeInstanceIds) +| where ObjectName == 'Processor' and CounterName == '% Processor Time' and InstanceName == '_Total' +| extend WorkspaceId = TenantId +| summarize hint.strategy=shuffle PCPUPercentage = percentile(CounterValue, cpuPercentileValue) by _ResourceId, WorkspaceId$processorPerfAdditionalWorkspaces; +let WindowsNetworkPerf = Perf +| where TimeGenerated > ago(perfInterval) and _ResourceId in (RightSizeInstanceIds) +| where CounterName == 'Bytes Total/sec' +| extend WorkspaceId = TenantId +| summarize hint.strategy=shuffle PCounter = percentile(CounterValue, networkPercentileValue) by InstanceName, _ResourceId, WorkspaceId +| summarize PNetwork = sum(PCounter) by _ResourceId, WorkspaceId$windowsNetworkPerfAdditionalWorkspaces; +let DiskPerf = Perf +| where TimeGenerated > ago(perfInterval) and _ResourceId in (RightSizeInstanceIds) +| where CounterName in ('Disk Reads/sec', 'Disk Writes/sec', 'Disk Read Bytes/sec', 'Disk Write Bytes/sec') and InstanceName !in ('_Total', 'D:', '/mnt/resource', '/mnt') +| extend WorkspaceId = TenantId +| summarize hint.strategy=shuffle PCounter = percentile(CounterValue, diskPercentileValue) by bin(TimeGenerated, perfTimeGrain), CounterName, InstanceName, _ResourceId, WorkspaceId +| summarize SumPCounter = sum(PCounter) by CounterName, TimeGenerated, _ResourceId, WorkspaceId +| summarize MaxPReadIOPS = maxif(SumPCounter, CounterName == 'Disk Reads/sec'), + MaxPWriteIOPS = maxif(SumPCounter, CounterName == 'Disk Writes/sec'), + MaxPReadMiBps = (maxif(SumPCounter, CounterName == 'Disk Read Bytes/sec') / 1024 / 1024), + MaxPWriteMiBps = (maxif(SumPCounter, CounterName == 'Disk Write Bytes/sec') / 1024 / 1024) by _ResourceId, WorkspaceId$diskPerfAdditionalWorkspaces; +$advisorTableName +| where todatetime(TimeGenerated) > ago(advisorInterval) and Category == 'Cost' +| extend AdvisorRecIdIndex = indexof(InstanceId_s, '/providers/microsoft.advisor/recommendations') +| extend InstanceName_s = iif(isnotempty(InstanceName_s),InstanceName_s,iif(AdvisorRecIdIndex > 0, split(substring(InstanceId_s, 0, AdvisorRecIdIndex),'/')[-1], split(InstanceId_s,'/')[-1])) +| distinct InstanceId_s, InstanceName_s, Description_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroup, Cloud_s, AdditionalInfo_s, RecommendationText_s, ImpactedArea_s, Impact_s, RecommendationTypeId_g, Tags_s +| join kind=leftouter ( + $consumptionTableName + | where todatetime(Date_s) between (stime..etime) + | extend VMConsumedQuantity = iif(ResourceId contains 'virtualmachines' and MeterCategory_s == 'Virtual Machines', todouble(Quantity_s), 0.0) + | extend VMPrice = iif(ResourceId contains 'virtualmachines' and MeterCategory_s == 'Virtual Machines', todouble(EffectivePrice_s), 0.0) + | extend FinalCost = iif(ResourceId contains 'virtualmachines', VMPrice * VMConsumedQuantity, todouble(CostInBillingCurrency_s)) + | extend InstanceId_s = tolower(ResourceId) + | summarize Last30DaysCost = sum(FinalCost), Last30DaysQuantity = sum(VMConsumedQuantity) by InstanceId_s +) on InstanceId_s +| join kind=leftouter ( + $vmsTableName + | where TimeGenerated > ago(1d) + | distinct InstanceId_s, NicCount_s, DataDiskCount_s +) on InstanceId_s +| where RecommendationTypeId_g != rightSizeRecommendationId or (RecommendationTypeId_g == rightSizeRecommendationId and toint(NicCount_s) >= 0 and toint(DataDiskCount_s) >= 0) +| join kind=leftouter hint.strategy=broadcast ( MemoryPerf ) on `$left.InstanceId_s == `$right._ResourceId +| join kind=leftouter hint.strategy=broadcast ( ProcessorPerf ) on `$left.InstanceId_s == `$right._ResourceId +| join kind=leftouter hint.strategy=broadcast ( WindowsNetworkPerf ) on `$left.InstanceId_s == `$right._ResourceId +| join kind=leftouter hint.strategy=broadcast ( DiskPerf ) on `$left.InstanceId_s == `$right._ResourceId +| extend MaxPIOPS = MaxPReadIOPS + MaxPWriteIOPS, MaxPMiBps = MaxPReadMiBps + MaxPWriteMiBps +| extend PNetworkMbps = PNetwork * 8 / 1000 / 1000 +| distinct Last30DaysCost, Last30DaysQuantity, InstanceId_s, InstanceName_s, Description_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroup, Cloud_s, AdditionalInfo_s, RecommendationText_s, ImpactedArea_s, Impact_s, RecommendationTypeId_g, NicCount_s, DataDiskCount_s, PMemoryPercentage, PCPUPercentage, PNetworkMbps, MaxPIOPS, MaxPMiBps, Tags_s, WorkspaceId +| join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s +) on SubscriptionGuid_g +"@ + +Write-Output "Will run the following query (use this query against the LA workspace for troubleshooting): $baseQuery" + +Write-Output "Getting cost recommendations for $($daysBackwards)d Advisor and $($perfDaysBackwards)d Perf history and a $perfTimeGrain time grain..." + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + throw "Execution aborted" +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +$skuPricesFound = @{} + +Write-Output "Generating fit score..." + +foreach ($result in $results) { + + if ($filters | Where-Object { $_.RecommendationSubTypeId -eq $result.RecommendationTypeId_g}) + { + continue + } + + $queryInstanceId = $result.InstanceId_s + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $additionalInfoDictionary = @{} + if (-not([string]::IsNullOrEmpty($result.AdditionalInfo_s))) + { + ($result.AdditionalInfo_s | ConvertFrom-Json).PsObject.Properties | ForEach-Object { $additionalInfoDictionary[$_.Name] = $_.Value } + } + + # Fixing reservation model inconsistencies + if (-not([string]::IsNullOrEmpty($additionalInfoDictionary["location"]))) + { + $additionalInfoDictionary["region"] = $additionalInfoDictionary["location"] + } + if (-not([string]::IsNullOrEmpty($additionalInfoDictionary["targetResourceCount"]))) + { + $additionalInfoDictionary["qty"] = $additionalInfoDictionary["targetResourceCount"] + } + if (-not([string]::IsNullOrEmpty($additionalInfoDictionary["vmSize"]))) + { + $additionalInfoDictionary["displaySKU"] = $additionalInfoDictionary["vmSize"] + } + + $additionalInfoDictionary["CostsAmount"] = [double] $result.Last30DaysCost + + $fitScore = 5 + $hasCpuRamPerfMetrics = $false + + if ($additionalInfoDictionary.targetSku -and $result.RecommendationTypeId_g -eq $rightSizeRecommendationId) { + $additionalInfoDictionary["SupportsDataDisksCount"] = "true" + $additionalInfoDictionary["DataDiskCount"] = "$($result.DataDiskCount_s)" + $additionalInfoDictionary["SupportsNICCount"] = "true" + $additionalInfoDictionary["NicCount"] = "$($result.NicCount_s)" + $additionalInfoDictionary["SupportsIOPS"] = "true" + $additionalInfoDictionary["MetricIOPS"] = "$($result.MaxPIOPS)" + $additionalInfoDictionary["SupportsMiBps"] = "true" + $additionalInfoDictionary["MetricMiBps"] = "$($result.MaxPMiBps)" + $additionalInfoDictionary["BelowCPUThreshold"] = "true" + $additionalInfoDictionary["MetricCPUPercentage"] = "$($result.PCPUPercentage)" + $additionalInfoDictionary["BelowMemoryThreshold"] = "true" + $additionalInfoDictionary["MetricMemoryPercentage"] = "$($result.PMemoryPercentage)" + $additionalInfoDictionary["BelowNetworkThreshold"] = "true" + $additionalInfoDictionary["MetricNetworkMbps"] = "$($result.PNetworkMbps)" + + $targetSku = $null + if ($additionalInfoDictionary.targetSku -ne "Shutdown") { + $currentSku = $skus | Where-Object { $_.Name -eq $additionalInfoDictionary.currentSku } + $currentSkuvCPUs = [int]($currentSku.Capabilities | Where-Object { $_.Name -eq 'vCPUsAvailable' }).Value + $targetSku = $skus | Where-Object { $_.Name -eq $additionalInfoDictionary.targetSku } + $targetSkuvCPUs = [int]($targetSku.Capabilities | Where-Object { $_.Name -eq 'vCPUsAvailable' }).Value + $targetMaxDataDiskCount = [int]($targetSku.Capabilities | Where-Object { $_.Name -eq 'MaxDataDiskCount' }).Value + if ($targetMaxDataDiskCount -gt 0) { + if (-not([string]::isNullOrEmpty($result.DataDiskCount_s))) { + if ([int]$result.DataDiskCount_s -gt $targetMaxDataDiskCount) { + $fitScore = 1 + $additionalInfoDictionary["SupportsDataDisksCount"] = "false:needs$($result.DataDiskCount_s)-max$targetMaxDataDiskCount" + } + } + else { + $fitScore -= 1 + $additionalInfoDictionary["SupportsDataDisksCount"] = "unknown:max$targetMaxDataDiskCount" + } + } + else { + $fitScore -= 1 + $additionalInfoDictionary["SupportsDataDisksCount"] = "unknown:needs$($result.DataDiskCount_s)" + } + $targetMaxNICCount = [int]($targetSku.Capabilities | Where-Object { $_.Name -eq 'MaxNetworkInterfaces' }).Value + if ($targetMaxNICCount -gt 0) { + if (-not([string]::isNullOrEmpty($result.NicCount_s))) { + if ([int]$result.NicCount_s -gt $targetMaxNICCount) { + $fitScore = 1 + $additionalInfoDictionary["SupportsNICCount"] = "false:needs$($result.NicCount_s)-max$targetMaxNICCount" + } + } + else { + $fitScore -= 1 + $additionalInfoDictionary["SupportsNICCount"] = "unknown:max$targetMaxNICCount" + } + } + else { + $fitScore -= 1 + $additionalInfoDictionary["SupportsNICCount"] = "unknown:needs$($result.NicCount_s)" + } + $targetUncachedDiskIOPS = [int]($targetSku.Capabilities | Where-Object { $_.Name -eq 'UncachedDiskIOPS' }).Value + if ($targetUncachedDiskIOPS -gt 0) { + if (-not([string]::isNullOrEmpty($result.MaxPIOPS))) { + if ([double]$result.MaxPIOPS -ge [double]$targetUncachedDiskIOPS) { + $fitScore -= 1 + $additionalInfoDictionary["SupportsIOPS"] = "false:needs$($result.MaxPIOPS)-max$targetUncachedDiskIOPS" + } + } + else { + $fitScore -= 0.5 + $additionalInfoDictionary["SupportsIOPS"] = "unknown:max$targetUncachedDiskIOPS" + } + } + else { + $fitScore -= 1 + $additionalInfoDictionary["SupportsIOPS"] = "unknown:needs$($result.MaxPIOPS)" + } + $targetUncachedDiskMiBps = [double]([long]($targetSku.Capabilities | Where-Object { $_.Name -eq 'UncachedDiskBytesPerSecond' }).Value) / 1024 / 1024 + if ($targetUncachedDiskMiBps -gt 0) { + if (-not([string]::isNullOrEmpty($result.MaxPMiBps))) { + if ([double]$result.MaxPMiBps -ge $targetUncachedDiskMiBps) { + $fitScore -= 1 + $additionalInfoDictionary["SupportsMiBps"] = "false:needs$($result.MaxPMiBps)-max$targetUncachedDiskMiBps" + } + } + else { + $fitScore -= 0.5 + $additionalInfoDictionary["SupportsMiBps"] = "unknown:max$targetUncachedDiskMiBps" + } + } + else { + $additionalInfoDictionary["SupportsMiBps"] = "unknown:needs$($result.MaxPMiBps)" + } + + $savingCoefficient = [double] $currentSkuvCPUs / $targetSkuvCPUs + + if ($savingCoefficient -gt 1) + { + $targetSkuSavingsMonthly = [double]$result.Last30DaysCost - ([double]$result.Last30DaysCost / $savingCoefficient) + } + else + { + $targetSkuSavingsMonthly = [double]$result.Last30DaysCost / 2 + } + + if ($targetSku -and $null -eq $skuPricesFound[$targetSku.Name]) + { + $skuPricesFound[$targetSku.Name] = Find-SkuHourlyPrice -SKUName $targetSku.Name -SKUPriceSheet $pricesheetEntries + } + + $tentativeTargetSkuSavingsMonthly = -1 + + if ($targetSku -and $skuPricesFound[$targetSku.Name] -gt 0 -and $skuPricesFound[$targetSku.Name] -lt [double]::MaxValue) + { + $targetSkuPrice = $skuPricesFound[$targetSku.Name] + + if ($null -eq $skuPricesFound[$currentSku.Name]) + { + $skuPricesFound[$currentSku.Name] = Find-SkuHourlyPrice -SKUName $currentSku.Name -SKUPriceSheet $pricesheetEntries + } + + if ($skuPricesFound[$currentSku.Name] -gt 0) + { + $currentSkuPrice = $skuPricesFound[$currentSku.Name] + $tentativeTargetSkuSavingsMonthly = ($currentSkuPrice * [double] $result.Last30DaysQuantity) - ($targetSkuPrice * [double] $result.Last30DaysQuantity) + } + else + { + $tentativeTargetSkuSavingsMonthly = [double]$result.Last30DaysCost - ($targetSkuPrice * [double] $result.Last30DaysQuantity) + } + } + + if ($tentativeTargetSkuSavingsMonthly -ge 0) + { + $targetSkuSavingsMonthly = $tentativeTargetSkuSavingsMonthly + } + + if ($targetSkuSavingsMonthly -eq [double]::PositiveInfinity) + { + $targetSkuSavingsMonthly = [double] $result.Last30DaysCost / 2 + } + + $savingsMonthly = $targetSkuSavingsMonthly + + } + else + { + $savingsMonthly = [double]$result.Last30DaysCost + } + + $cpuThreshold = $cpuPercentageThreshold + $memoryThreshold = $memoryPercentageThreshold + $networkThreshold = $networkMpbsThreshold + if ($additionalInfoDictionary.targetSku -eq "Shutdown") { + $cpuThreshold = $cpuPercentageShutdownThreshold + $memoryThreshold = $memoryPercentageShutdownThreshold + $networkThreshold = $networkMpbsShutdownThreshold + } + + if (-not([string]::isNullOrEmpty($result.PCPUPercentage))) { + if ([double]$result.PCPUPercentage -ge [double]$cpuThreshold) { + $fitScore -= 0.5 + $additionalInfoDictionary["BelowCPUThreshold"] = "false:needs$($result.PCPUPercentage)-max$cpuThreshold" + } + $hasCpuRamPerfMetrics = $true + } + else { + $fitScore -= 0.5 + $additionalInfoDictionary["BelowCPUThreshold"] = "unknown:max$cpuThreshold" + } + if (-not([string]::isNullOrEmpty($result.PMemoryPercentage))) { + if ([double]$result.PMemoryPercentage -ge [double]$memoryThreshold) { + $fitScore -= 0.5 + $additionalInfoDictionary["BelowMemoryThreshold"] = "false:needs$($result.PMemoryPercentage)-max$memoryThreshold" + } + $hasCpuRamPerfMetrics = $true + } + else { + $fitScore -= 0.5 + $additionalInfoDictionary["BelowMemoryThreshold"] = "unknown:max$memoryThreshold" + } + if (-not([string]::isNullOrEmpty($result.PNetworkMbps))) { + if ([double]$result.PNetworkMbps -ge [double]$networkThreshold) { + $fitScore -= 0.1 + $additionalInfoDictionary["BelowNetworkThreshold"] = "false:needs$($result.PNetworkMbps)-max$networkThreshold" + } + } + else { + $fitScore -= 0.1 + $additionalInfoDictionary["BelowNetworkThreshold"] = "unknown:max$networkThreshold" + } + + $fitScore = [Math]::max(0.0, $fitScore) + } + else + { + if (-not([string]::IsNullOrEmpty($additionalInfoDictionary["annualSavingsAmount"]))) + { + $savingsMonthly = [double] $additionalInfoDictionary["annualSavingsAmount"] / 12 + } + else + { + if ($result.RecommendationTypeId_g -eq $rightSizeRecommendationId) + { + $savingsMonthly = [double] $result.Last30DaysCost + } + else + { + $savingsMonthly = 0.0 # unknown + } + } + } + + $additionalInfoDictionary["savingsAmount"] = [double] $savingsMonthly + + $queryInstanceId = $result.InstanceId_s + if (-not($hasCpuRamPerfMetrics)) + { + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + } + else + { + $queryWorkspace = "" + if (-not([string]::IsNullOrEmpty($result.WorkspaceId)) -and $result.WorkspaceId -ne $workspaceId) + { + $queryWorkspace = "workspace('$($result.WorkspaceId)')." + } + + $queryText = @" + let perfInterval = $($perfDaysBackwards)d; + let armId = tolower(`'$queryInstanceId`'); + let gInt = $perfTimeGrain; + let LinuxMemoryPerf = $($queryWorkspace)Perf + | where TimeGenerated > ago(perfInterval) + | where CounterName == '% Used Memory' and _ResourceId =~ armId + | project TimeGenerated, MemoryPercentage = CounterValue; + let WindowsMemoryPerf = $($queryWorkspace)Perf + | where TimeGenerated > ago(perfInterval) + | where CounterName == 'Available MBytes' and _ResourceId =~ armId + | extend MemoryAvailableMBs = CounterValue, InstanceId = tolower(_ResourceId) + | project TimeGenerated, MemoryAvailableMBs, InstanceId; + let MemoryPerf = WindowsMemoryPerf + | join kind=inner ( + $vmsTableName + | where TimeGenerated > ago(1d) + | extend InstanceId = tolower(InstanceId_s) + | distinct InstanceId, MemoryMB_s + ) on InstanceId + | extend MemoryPercentage = todouble(toint(MemoryMB_s) - toint(MemoryAvailableMBs)) / todouble(MemoryMB_s) * 100 + | project TimeGenerated, MemoryPercentage + | union LinuxMemoryPerf + | summarize P$($memoryPercentile)MemoryPercentage = percentile(MemoryPercentage, $memoryPercentile) by bin(TimeGenerated, gInt); + let ProcessorPerf = $($queryWorkspace)Perf + | where TimeGenerated > ago(perfInterval) + | where CounterName == '% Processor Time' and InstanceName == '_Total' and _ResourceId =~ armId + | summarize P$($cpuPercentile)CPUPercentage = percentile(CounterValue, $cpuPercentile) by bin(TimeGenerated, gInt); + MemoryPerf + | join kind=inner (ProcessorPerf) on TimeGenerated + | render timechart +"@ + + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $datetime.AddDays(-30).ToString("yyyy-MM-dd") + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Cost" + ImpactedArea = $result.ImpactedArea_s + Impact = $result.Impact_s + RecommendationType = "Saving" + RecommendationSubType = "AdvisorCost" + RecommendationSubTypeId = $result.RecommendationTypeId_g + RecommendationDescription = $result.Description_s + RecommendationAction = $result.RecommendationText_s + InstanceId = $result.InstanceId_s + InstanceName = $result.InstanceName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroup + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +Write-Output "Exporting final $($recommendations.Count) results as a JSON file..." + +$fileDate = $datetime.ToString("yyyyMMdd") +$jsonExportPath = "advisor-cost-augmented-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +Write-Output "Uploading $jsonExportPath to blob storage..." + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json" }; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AppServiceOptimizationsToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AppServiceOptimizationsToBlobStorage.ps1 new file mode 100644 index 000000000..ed8e1fe5d --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-AppServiceOptimizationsToBlobStorage.ps1 @@ -0,0 +1,695 @@ +$ErrorActionPreference = "Stop" + +# Collect generic and recommendation-specific variables + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$workspaceName = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceName" +$workspaceRG = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceRG" +$workspaceSubscriptionId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceSubId" +$workspaceTenantId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceTenantId" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} + +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +$deploymentDate = Get-AutomationVariable -Name "AzureOptimization_DeploymentDate" # yyyy-MM-dd format +$deploymentDate = $deploymentDate.Replace('"', "") + +$perfDaysBackwards = [int] (Get-AutomationVariable -Name "AzureOptimization_RecommendPerfPeriodInDays" -ErrorAction SilentlyContinue) +if (-not($perfDaysBackwards -gt 0)) { + $perfDaysBackwards = 7 +} + +$perfTimeGrain = Get-AutomationVariable -Name "AzureOptimization_RecommendPerfTimeGrain" -ErrorAction SilentlyContinue +if (-not($perfTimeGrain)) { + $perfTimeGrain = "1h" +} + +# percentiles variables +$cpuPercentile = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfPercentileCpu" -ErrorAction SilentlyContinue) +if (-not($cpuPercentile -gt 0)) { + $cpuPercentile = 99 +} +$memoryPercentile = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfPercentileMemory" -ErrorAction SilentlyContinue) +if (-not($memoryPercentile -gt 0)) { + $memoryPercentile = 99 +} + +# perf thresholds variables +$cpuPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdCpuPercentage" -ErrorAction SilentlyContinue) +if (-not($cpuPercentageThreshold -gt 0)) { + $cpuPercentageThreshold = 30 +} +$memoryPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdMemoryPercentage" -ErrorAction SilentlyContinue) +if (-not($memoryPercentageThreshold -gt 0)) { + $memoryPercentageThreshold = 50 +} +$cpuDegradedMaxPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdCpuDegradedMaxPercentage" -ErrorAction SilentlyContinue) +if (-not($cpuDegradedMaxPercentageThreshold -gt 0)) { + $cpuDegradedMaxPercentageThreshold = 95 +} +$cpuDegradedAvgPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdCpuDegradedAvgPercentage" -ErrorAction SilentlyContinue) +if (-not($cpuDegradedAvgPercentageThreshold -gt 0)) { + $cpuDegradedAvgPercentageThreshold = 75 +} +$memoryDegradedPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdMemoryDegradedPercentage" -ErrorAction SilentlyContinue) +if (-not($memoryDegradedPercentageThreshold -gt 0)) { + $memoryDegradedPercentageThreshold = 90 +} + +$consumptionOffsetDays = [int] (Get-AutomationVariable -Name "AzureOptimization_ConsumptionOffsetDays") +$consumptionOffsetDaysStart = $consumptionOffsetDays + 1 + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" + +# Authenticate against Azure + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +Write-Output "Finding tables where recommendations will be generated from..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE CollectedType IN ('AppServicePlans','MonitorMetrics','AzureConsumption','ARGResourceContainers')" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$appServicePlansTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'AppServicePlans' }).LogAnalyticsSuffix + "_CL" +$metricsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'MonitorMetrics' }).LogAnalyticsSuffix + "_CL" +$consumptionTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'AzureConsumption' }).LogAnalyticsSuffix + "_CL" +$subscriptionsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGResourceContainers' }).LogAnalyticsSuffix + "_CL" + +Write-Output "Will run query against tables $appServicePlansTableName, $subscriptionsTableName, $metricsTableName and $consumptionTableName" + +$Conn.Close() +$Conn.Dispose() + +$recommendationSearchTimeSpan = 30 + $consumptionOffsetDaysStart + +# Grab a context reference to the Storage Account where the recommendations file will be stored + + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +if ($workspaceSubscriptionId -ne $storageAccountSinkSubscriptionId) +{ + Select-AzSubscription -SubscriptionId $workspaceSubscriptionId +} + +$recommendationsErrors = 0 + +# Execute the recommendation query against Log Analytics +Write-Output "Looking for underused App Service Plans, with less than $cpuPercentageThreshold% CPU and $memoryPercentageThreshold% RAM usage..." + +$baseQuery = @" + let billingInterval = 30d; + let perfInterval = $($perfDaysBackwards)d; + let cpuPercentileValue = $cpuPercentile; + let memoryPercentileValue = $memoryPercentile; + let etime = todatetime(toscalar($consumptionTableName | where todatetime(Date_s) < now() and todatetime(Date_s) > ago(30d) | summarize max(todatetime(Date_s)))); + let stime = etime-billingInterval; + + let BilledPlans = $consumptionTableName + | where todatetime(Date_s) between (stime..etime) and ResourceId has 'microsoft.web/serverfarms' + | extend ConsumedQuantity = todouble(Quantity_s) + | extend FinalCost = todouble(EffectivePrice_s) * ConsumedQuantity + | extend InstanceId_s = tolower(ResourceId) + | summarize Last30DaysCost = sum(FinalCost), Last30DaysQuantity = sum(ConsumedQuantity) by InstanceId_s; + + let ProcessorPerf = $metricsTableName + | where TimeGenerated > ago(perfInterval) + | where ResourceId has 'microsoft.web/serverfarms' + | where MetricNames_s == "CpuPercentage" and AggregationType_s == 'Maximum' + | extend InstanceId_s = ResourceId + | summarize PCPUPercentage = percentile(todouble(MetricValue_s), cpuPercentileValue) by InstanceId_s; + + let MemoryPerf = $metricsTableName + | where TimeGenerated > ago(perfInterval) + | where ResourceId has 'microsoft.web/serverfarms' + | where MetricNames_s == "MemoryPercentage" and AggregationType_s == 'Maximum' + | extend InstanceId_s = ResourceId + | summarize PMemoryPercentage = percentile(todouble(MetricValue_s), memoryPercentileValue) by InstanceId_s; + + $appServicePlansTableName + | where TimeGenerated > ago(1d) and ComputeMode_s == 'Dedicated' and SkuTier_s != 'Free' + | distinct InstanceId_s, AppServicePlanName_s, ResourceGroupName_s, SubscriptionGuid_g, Cloud_s, TenantGuid_g, SkuSize_s, NumberOfWorkers_s, Tags_s + | join kind=inner ( BilledPlans ) on InstanceId_s + | join kind=leftouter ( MemoryPerf ) on InstanceId_s + | join kind=leftouter ( ProcessorPerf ) on InstanceId_s + | project InstanceId_s, AppServicePlan = AppServicePlanName_s, ResourceGroup = ResourceGroupName_s, SubscriptionId = SubscriptionGuid_g, Cloud_s, TenantGuid_g, SkuSize_s, NumberOfWorkers_s, PMemoryPercentage, PCPUPercentage, Tags_s, Last30DaysCost, Last30DaysQuantity + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionId = SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionId + | where isnotempty(PMemoryPercentage) and isnotempty(PCPUPercentage) and PMemoryPercentage < $memoryPercentageThreshold and PCPUPercentage < $cpuPercentageThreshold +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + $queryInstanceId = $result.InstanceId_s + $queryText = @" +let perfInterval = $($perfDaysBackwards)d; +let armId = `'$queryInstanceId`'; +let gInt = $perfTimeGrain; +let MemoryPerf = $metricsTableName +| where TimeGenerated > ago(perfInterval) +| extend CollectedDate = todatetime(strcat(format_datetime(TimeGenerated, 'yyyy-MM-dd'),'T',format_datetime(TimeGenerated, 'HH'),':00:00Z')) +| where ResourceId == armId +| where MetricNames_s == 'MemoryPercentage' and AggregationType_s == 'Maximum' +| extend MemoryPercentage = todouble(MetricValue_s) +| summarize percentile(MemoryPercentage, $memoryPercentile) by bin(CollectedDate, gInt); +let ProcessorPerf = $metricsTableName +| where TimeGenerated > ago(perfInterval) +| extend CollectedDate = todatetime(strcat(format_datetime(TimeGenerated, 'yyyy-MM-dd'),'T',format_datetime(TimeGenerated, 'HH'),':00:00Z')) +| where ResourceId == armId +| where MetricNames_s == 'CpuPercentage' and AggregationType_s == 'Maximum' +| extend ProcessorPercentage = todouble(MetricValue_s) +| summarize percentile(ProcessorPercentage, $cpuPercentile) by bin(CollectedDate, gInt); +MemoryPerf +| join kind=inner (ProcessorPerf) on CollectedDate +| render timechart +"@ + + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $datetime.AddDays(-30).ToString("yyyy-MM-dd") + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["currentSku"] = "$($result.SkuSize_s)" + $additionalInfoDictionary["InstanceCount"] = [int] $result.NumberOfWorkers_s + $additionalInfoDictionary["MetricCPUPercentage"] = "$($result.PCPUPercentage)" + $additionalInfoDictionary["MetricMemoryPercentage"] = "$($result.PMemoryPercentage)" + $additionalInfoDictionary["CostsAmount"] = [double] $result.Last30DaysCost + $additionalInfoDictionary["savingsAmount"] = ([double] $result.Last30DaysCost / 2) + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Cost" + ImpactedArea = "Microsoft.Web/serverFarms" + Impact = "High" + RecommendationType = "Saving" + RecommendationSubType = "UnderusedAppServicePlans" + RecommendationSubTypeId = "042adaca-ebdf-49b4-bc1b-2800b6e40fea" + RecommendationDescription = "Underused App Service Plans (performance capacity waste)" + RecommendationAction = "Right-size underused App Service Plans or scale it in" + InstanceId = $result.InstanceId_s + InstanceName = $result.AppServicePlan + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroup + SubscriptionGuid = $result.SubscriptionId + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "appserviceplans-underused-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for performance constrained App Service Plans, with more than $cpuDegradedMaxPercentageThreshold% Max. CPU, $cpuDegradedAvgPercentageThreshold% Avg. CPU and $memoryDegradedPercentageThreshold% RAM usage..." + +$baseQuery = @" + let perfInterval = $($perfDaysBackwards)d; + + let MemoryPerf = $metricsTableName + | where TimeGenerated > ago(perfInterval) + | where ResourceId has 'microsoft.web/serverfarms' + | where MetricNames_s == "MemoryPercentage" and AggregationType_s == 'Average' and AggregationOfType_s == 'Maximum' + | extend InstanceId_s = ResourceId + | summarize PMemoryPercentage = avg(todouble(MetricValue_s)) by InstanceId_s; + + let ProcessorMaxPerf = $metricsTableName + | where TimeGenerated > ago(perfInterval) + | where ResourceId has 'microsoft.web/serverfarms' + | where MetricNames_s == "CpuPercentage" and AggregationType_s == 'Maximum' + | extend InstanceId_s = ResourceId + | summarize PCPUMaxPercentage = avg(todouble(MetricValue_s)) by InstanceId_s; + + let ProcessorAvgPerf = $metricsTableName + | where TimeGenerated > ago(perfInterval) + | where ResourceId has 'microsoft.web/serverfarms' + | where MetricNames_s == "CpuPercentage" and AggregationType_s == 'Average' and AggregationOfType_s == 'Maximum' + | extend InstanceId_s = ResourceId + | summarize PCPUAvgPercentage = avg(todouble(MetricValue_s)) by InstanceId_s; + + $appServicePlansTableName + | where TimeGenerated > ago(1d) and ComputeMode_s == 'Dedicated' and SkuTier_s != 'Free' + | distinct InstanceId_s, AppServicePlanName_s, ResourceGroupName_s, SubscriptionGuid_g, Cloud_s, TenantGuid_g, SkuSize_s, NumberOfWorkers_s, Tags_s + | join kind=leftouter ( MemoryPerf ) on InstanceId_s + | join kind=leftouter ( ProcessorMaxPerf ) on InstanceId_s + | join kind=leftouter ( ProcessorAvgPerf ) on InstanceId_s + | project InstanceId_s, AppServicePlan = AppServicePlanName_s, ResourceGroup = ResourceGroupName_s, SubscriptionId = SubscriptionGuid_g, Cloud_s, TenantGuid_g, SkuSize_s, NumberOfWorkers_s, PMemoryPercentage, PCPUMaxPercentage, PCPUAvgPercentage, Tags_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionId = SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionId + | where isnotempty(PMemoryPercentage) and isnotempty(PCPUAvgPercentage) and isnotempty(PCPUMaxPercentage) and (PMemoryPercentage > $memoryDegradedPercentageThreshold or (PCPUMaxPercentage > $cpuDegradedMaxPercentageThreshold and PCPUAvgPercentage > $cpuDegradedAvgPercentageThreshold)) +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + $queryInstanceId = $result.InstanceId_s + $queryText = @" +let perfInterval = $($perfDaysBackwards)d; +let armId = `'$queryInstanceId`'; +let gInt = $perfTimeGrain; +let MemoryPerf = $metricsTableName +| where TimeGenerated > ago(perfInterval) +| extend CollectedDate = todatetime(strcat(format_datetime(TimeGenerated, 'yyyy-MM-dd'),'T',format_datetime(TimeGenerated, 'HH'),':00:00Z')) +| where ResourceId == armId +| where MetricNames_s == 'MemoryPercentage' and AggregationType_s == 'Average' and AggregationOfType_s == 'Maximum' +| extend MemoryPercentage = todouble(MetricValue_s) +| summarize percentile(MemoryPercentage, $memoryPercentile) by bin(CollectedDate, gInt); +let ProcessorMaxPerf = $metricsTableName +| where TimeGenerated > ago(perfInterval) +| extend CollectedDate = todatetime(strcat(format_datetime(TimeGenerated, 'yyyy-MM-dd'),'T',format_datetime(TimeGenerated, 'HH'),':00:00Z')) +| where ResourceId == armId +| where MetricNames_s == 'CpuPercentage' and AggregationType_s == 'Maximum' +| extend ProcessorMaxPercentage = todouble(MetricValue_s) +| summarize percentile(ProcessorMaxPercentage, $cpuPercentile) by bin(CollectedDate, gInt); +let ProcessorAvgPerf = $metricsTableName +| where TimeGenerated > ago(perfInterval) +| extend CollectedDate = todatetime(strcat(format_datetime(TimeGenerated, 'yyyy-MM-dd'),'T',format_datetime(TimeGenerated, 'HH'),':00:00Z')) +| where ResourceId == armId +| where MetricNames_s == 'CpuPercentage' and AggregationType_s == 'Average' and AggregationOfType_s == 'Maximum' +| extend ProcessorAvgPercentage = todouble(MetricValue_s) +| summarize percentile(ProcessorAvgPercentage, $cpuPercentile) by bin(CollectedDate, gInt); +MemoryPerf +| join kind=inner (ProcessorMaxPerf) on CollectedDate +| join kind=inner (ProcessorAvgPerf) on CollectedDate +| render timechart +"@ + + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $datetime.AddDays(-30).ToString("yyyy-MM-dd") + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["currentSku"] = "$($result.SkuSize_s)" + $additionalInfoDictionary["InstanceCount"] = [int] $result.NumberOfWorkers_s + $additionalInfoDictionary["MetricCPUAvgPercentage"] = "$($result.PCPUAvgPercentage)" + $additionalInfoDictionary["MetricCPUMaxPercentage"] = "$($result.PCPUMaxPercentage)" + $additionalInfoDictionary["MetricMemoryPercentage"] = "$($result.PMemoryPercentage)" + + $fitScore = 3 # needs a more complete analysis to improve score + + if ([double] $result.PCPUMaxPercentage -gt [double] $cpuDegradedMaxPercentageThreshold -and [double] $result.PCPUAvgPercentage -gt [double] $cpuDegradedAvgPercentageThreshold) + { + $fitScore = 4 + } + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Performance" + ImpactedArea = "Microsoft.Web/serverFarms" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "PerfConstrainedAppServicePlans" + RecommendationSubTypeId = "351574cb-c105-4538-a778-11dfbe4857bf" + RecommendationDescription = "App Service Plan performance has been constrained by lack of resources" + RecommendationAction = "Resize App Service Plan to higher SKU or scale it out" + InstanceId = $result.InstanceId_s + InstanceName = $result.AppServicePlan + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroup + SubscriptionGuid = $result.SubscriptionId + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "appserviceplans-perfconstrained-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for empty App Service Plans..." + +$baseQuery = @" +let interval = 30d; +let etime = todatetime(toscalar($consumptionTableName | where todatetime(Date_s) < now() and todatetime(Date_s) > ago(interval) | summarize max(todatetime(Date_s)))); +let stime = etime-interval; +$appServicePlansTableName +| where TimeGenerated > ago(1d) and ComputeMode_s == 'Dedicated' and SkuTier_s != 'Free' and toint(NumberOfSites_s) == 0 +| distinct AppServicePlanName_s, InstanceId_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroupName_s, SkuSize_s, NumberOfWorkers_s, Tags_s, Cloud_s +| join kind=leftouter ( + $consumptionTableName + | where todatetime(Date_s) between (stime..etime) + | project InstanceId_s=tolower(ResourceId), CostInBillingCurrency_s, Date_s +) on InstanceId_s +| summarize Last30DaysCost=sum(todouble(CostInBillingCurrency_s)) by AppServicePlanName_s, InstanceId_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroupName_s, SkuSize_s, NumberOfWorkers_s, Tags_s, Cloud_s +| join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s +) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + $queryInstanceId = $result.InstanceId_s + $queryText = @" + $appServicePlansTableName + | where InstanceId_s == '$queryInstanceId' + | where toint(NumberOfSites_s) == 0 + | distinct InstanceId_s, AppServicePlanName_s, TimeGenerated + | summarize FirstUnusedDate = min(TimeGenerated) by InstanceId_s, AppServicePlanName_s + | join kind=leftouter ( + $consumptionTableName + | project InstanceId_s=tolower(ResourceId), CostInBillingCurrency_s, Date_s + ) on InstanceId_s + | summarize CostsSinceUnused = sumif(todouble(CostInBillingCurrency_s), todatetime(Date_s) > FirstUnusedDate) by AppServicePlanName_s, FirstUnusedDate +"@ + + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $deploymentDate + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["currentSku"] = $result.SkuSize_s + $additionalInfoDictionary["InstanceCount"] = $result.NumberOfWorkers_s + $additionalInfoDictionary["CostsAmount"] = [double] $result.Last30DaysCost + $additionalInfoDictionary["savingsAmount"] = [double] $result.Last30DaysCost + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Cost" + ImpactedArea = "Microsoft.Web/serverFarms" + Impact = "High" + RecommendationType = "Saving" + RecommendationSubType = "EmptyAppServicePlans" + RecommendationSubTypeId = "ef525225-8b91-47a3-81f3-e674e94564b6" + RecommendationDescription = "App Service Plans without any application incur in unnecessary costs" + RecommendationAction = "Delete the App Service Plan" + InstanceId = $result.InstanceId_s + InstanceName = $result.AppServicePlanName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "appserviceplans-empty-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +if ($recommendationsErrors -gt 0) +{ + throw "Some of the recommendations queries failed. Please, review the job logs for additional information." +} \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-DiskOptimizationsToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-DiskOptimizationsToBlobStorage.ps1 new file mode 100644 index 000000000..364b3b303 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-DiskOptimizationsToBlobStorage.ps1 @@ -0,0 +1,539 @@ +$ErrorActionPreference = "Stop" + +function Find-DiskMonthlyPrice { + param ( + [object[]] $SKUPriceSheet, + [string] $DiskSizeTier + ) + + $diskSkus = $SKUPriceSheet | Where-Object { $_.MeterName_s.Replace(" Disks","").Replace(" Disk","") -eq $DiskSizeTier } + $targetMonthlyPrice = [double]::MaxValue + if ($diskSkus) + { + $targetMonthlyPrice = [double] ($diskSkus | Sort-Object -Property UnitPrice_s | Select-Object -First 1).UnitPrice_s + } + return $targetMonthlyPrice +} + +# Collect generic and recommendation-specific variables + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$workspaceName = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceName" +$workspaceRG = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceRG" +$workspaceSubscriptionId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceSubId" +$workspaceTenantId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceTenantId" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} + +$deploymentDate = Get-AutomationVariable -Name "AzureOptimization_DeploymentDate" # yyyy-MM-dd format +$deploymentDate = $deploymentDate.Replace('"', "") + +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +# perf thresholds variables +$iopsPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdDiskIOPSPercentage" -ErrorAction SilentlyContinue) +if (-not($iopsPercentageThreshold -gt 0)) { + $iopsPercentageThreshold = 5 +} +$mbsPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdDiskMBsPercentage" -ErrorAction SilentlyContinue) +if (-not($mbsPercentageThreshold -gt 0)) { + $mbsPercentageThreshold = 5 +} + +$consumptionOffsetDays = [int] (Get-AutomationVariable -Name "AzureOptimization_ConsumptionOffsetDays") +$consumptionOffsetDaysStart = $consumptionOffsetDays + 1 + +$perfDaysBackwards = [int] (Get-AutomationVariable -Name "AzureOptimization_RecommendPerfPeriodInDays" -ErrorAction SilentlyContinue) +if (-not($perfDaysBackwards -gt 0)) { + $perfDaysBackwards = 7 +} + +$perfTimeGrain = Get-AutomationVariable -Name "AzureOptimization_RecommendPerfTimeGrain" -ErrorAction SilentlyContinue +if (-not($perfTimeGrain)) { + $perfTimeGrain = "1h" +} + +$referenceRegion = Get-AutomationVariable -Name "AzureOptimization_ReferenceRegion" + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" + +# Authenticate against Azure + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +Write-Output "Finding tables where recommendations will be generated from..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE CollectedType IN ('ARGManagedDisk','MonitorMetrics','ARGResourceContainers','AzureConsumption','Pricesheet')" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$disksTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGManagedDisk' }).LogAnalyticsSuffix + "_CL" +$metricsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'MonitorMetrics' }).LogAnalyticsSuffix + "_CL" +$subscriptionsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGResourceContainers' }).LogAnalyticsSuffix + "_CL" +$consumptionTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'AzureConsumption' }).LogAnalyticsSuffix + "_CL" +$pricesheetTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'Pricesheet' }).LogAnalyticsSuffix + "_CL" + + +Write-Output "Will run query against tables $disksTableName, $metricsTableName, $subscriptionsTableName, $pricesheetTableName and $consumptionTableName" + +$Conn.Close() +$Conn.Dispose() + +$recommendationSearchTimeSpan = 30 + $consumptionOffsetDaysStart + +# Grab a context reference to the Storage Account where the recommendations file will be stored + + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +if ($workspaceSubscriptionId -ne $storageAccountSinkSubscriptionId) +{ + Select-AzSubscription -SubscriptionId $workspaceSubscriptionId +} + +Write-Output "Getting Disks SKUs for the $referenceRegion region..." + +$skus = Get-AzComputeResourceSku -Location $referenceRegion | Where-Object { $_.ResourceType -eq "disks" } + +Write-Output "Getting the current Pricesheet..." + +if ($cloudEnvironment -eq "AzureCloud") +{ + $pricesheetRegion = "EU West" +} + +try +{ + $pricesheetEntries = @() + + $baseQuery = @" + $pricesheetTableName + | where TimeGenerated > ago(14d) + | where MeterCategory_s == 'Storage' and MeterSubCategory_s contains "Managed Disk" and (MeterName_s endswith "Disk" or MeterName_s endswith "Disks") and MeterName_s !has 'Special' and MeterRegion_s == '$pricesheetRegion' and PriceType_s == 'Consumption' + | distinct MeterName_s, MeterSubCategory_s, MeterCategory_s, MeterRegion_s, UnitPrice_s, UnitOfMeasure_s +"@ + + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days 14) -Wait 600 -IncludeStatistics + $pricesheetEntries = [System.Linq.Enumerable]::ToArray($queryResults.Results) + + Write-Output "Query finished with $($pricesheetEntries.Count) results." + Write-Output "Query statistics: $($queryResults.Statistics.query)" +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + Write-Output "Consumption pricesheet not available, will estimate savings based in price difference ratio..." +} + +$skuPricesFound = @{} + +Write-Output "Looking for underutilized Disks, with less than $iopsPercentageThreshold% IOPS and $mbsPercentageThreshold% MB/s usage..." + +$baseQuery = @" + let billingInterval = 30d; + let perfInterval = $($perfDaysBackwards)d; + let etime = todatetime(toscalar($consumptionTableName | where todatetime(Date_s) < now() and todatetime(Date_s) > ago(billingInterval) | summarize max(todatetime(Date_s)))); + let stime = etime-billingInterval; + + let BilledDisks = $consumptionTableName + | where todatetime(Date_s) between (stime..etime) and ResourceId contains '/disks/' and MeterCategory_s == 'Storage' and MeterSubCategory_s has 'Premium' and MeterName_s has 'Disk' + | extend DiskConsumedQuantity = todouble(Quantity_s) + | extend DiskPrice = todouble(EffectivePrice_s) + | extend FinalCost = DiskPrice * DiskConsumedQuantity + | extend ResourceId = tolower(ResourceId) + | summarize Last30DaysCost = sum(FinalCost), Last30DaysQuantity = sum(DiskConsumedQuantity) by ResourceId; + + $metricsTableName + | where MetricNames_s == 'Composite Disk Read Operations/sec,Composite Disk Write Operations/sec' and TimeGenerated > ago(perfInterval) and isnotempty(MetricValue_s) + | summarize MaxIOPSMetric = max(todouble(MetricValue_s)) by ResourceId + | join kind=inner ( + $disksTableName + | where TimeGenerated > ago(1d) and DiskState_s =~ 'Attached' and SKU_s startswith 'Premium' and SKU_s !contains "V2" + | extend DiskTier_s = strcat(DiskTier_s, ' ', tostring(split(SKU_s, '_')[1])) + | project ResourceId=InstanceId_s, DiskName_s, ResourceGroup = ResourceGroupName_s, SubscriptionId = SubscriptionGuid_g, Cloud_s, TenantGuid_g, Tags_s, MaxIOPSDisk=toint(DiskIOPS_s), DiskSizeGB_s, SKU_s, DiskTier_s, DiskType_s + ) on ResourceId + | project-away ResourceId1 + | extend IOPSPercentage = MaxIOPSMetric/MaxIOPSDisk*100 + | where IOPSPercentage < $iopsPercentageThreshold + | join kind=inner ( + $metricsTableName + | where MetricNames_s == 'Composite Disk Read Bytes/sec,Composite Disk Write Bytes/sec' and TimeGenerated > ago(perfInterval) and isnotempty(MetricValue_s) + | summarize MaxMBsMetric = max(todouble(MetricValue_s)/1024/1024) by ResourceId + | join kind=inner ( + $disksTableName + | where TimeGenerated > ago(1d) and DiskState_s =~ 'Attached' and SKU_s startswith 'Premium' and SKU_s !contains "V2" + | extend DiskTier_s = strcat(DiskTier_s, ' ', tostring(split(SKU_s, '_')[1])) + | project ResourceId=InstanceId_s, DiskName_s, ResourceGroup = ResourceGroupName_s, SubscriptionId = SubscriptionGuid_g, Cloud_s, TenantGuid_g, Tags_s, MaxMBsDisk=toint(DiskThroughput_s), DiskSizeGB_s, SKU_s, DiskTier_s, DiskType_s + ) on ResourceId + | project-away ResourceId1 + | extend MBsPercentage = MaxMBsMetric/MaxMBsDisk*100 + | where MBsPercentage < $mbsPercentageThreshold + ) on ResourceId + | join kind=inner ( BilledDisks ) on ResourceId + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionId = SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionId +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + throw "Execution aborted" +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + $targetSku = $null + $currentDiskTier = $null + + if ([string]::IsNullOrEmpty($result.DiskTier_s) -or $result.DiskTier_s.Trim().Length -le 3) # older disks do not have Tier info in their properties + { + $currentSkuCandidates = @() + foreach ($sku in $skus) + { + $currentSkuCandidate = $null + $skuMinSizeGB = [int] ($sku.Capabilities | Where-Object { $_.Name -eq 'MinSizeGiB' }).Value + $skuMaxSizeGB = [int] ($sku.Capabilities | Where-Object { $_.Name -eq 'MaxSizeGiB' }).Value + $skuMaxIOps = [int] ($sku.Capabilities | Where-Object { $_.Name -eq 'MaxIOps' }).Value + $skuMaxBandwidthMBps = [int] ($sku.Capabilities | Where-Object { $_.Name -eq 'MaxBandwidthMBps' }).Value + + if ($sku.Name -eq $result.SKU_s -and $skuMinSizeGB -lt [int]$result.DiskSizeGB_s -and $skuMaxSizeGB -ge [int]$result.DiskSizeGB_s ` + -and [int]$skuMaxIOps -eq [int]$result.MaxIOPSDisk -and [int]$skuMaxBandwidthMBps -eq [int]$result.MaxMBsDisk) + { + $skuSize = $sku.Size + " " + $result.SKU_s.Split("_")[1] + if ($null -eq $skuPricesFound[$skuSize]) + { + $skuPricesFound[$sku.Size] = Find-DiskMonthlyPrice -DiskSizeTier $skuSize -SKUPriceSheet $pricesheetEntries + } + + $currentSkuCandidate = New-Object PSObject -Property @{ + Name = $skuSize + MaxSizeGB = $skuMaxSizeGB + } + + $currentSkuCandidates += $currentSkuCandidate + } + } + $currentDiskTier = ($currentSkuCandidates | Sort-Object -Property MaxSizeGB | Select-Object -First 1).Name + } + else + { + $currentDiskTier = $result.DiskTier_s + } + + if ($null -eq $skuPricesFound[$currentDiskTier]) + { + $skuPricesFound[$currentDiskTier] = Find-DiskMonthlyPrice -DiskSizeTier $currentDiskTier -SKUPriceSheet $pricesheetEntries + } + + $targetSkuPerfTier = $result.SKU_s.Replace("Premium", "StandardSSD") + $targetSkuCandidates = @() + + foreach ($sku in $skus) + { + $targetSkuCandidate = $null + + $skuMinSizeGB = [int] ($sku.Capabilities | Where-Object { $_.Name -eq 'MinSizeGiB' }).Value + $skuMaxSizeGB = [int] ($sku.Capabilities | Where-Object { $_.Name -eq 'MaxSizeGiB' }).Value + $skuMaxIOps = [int] ($sku.Capabilities | Where-Object { $_.Name -eq 'MaxIOps' }).Value + $skuMaxBandwidthMBps = [int] ($sku.Capabilities | Where-Object { $_.Name -eq 'MaxBandwidthMBps' }).Value + + if ($sku.Name -eq $targetSkuPerfTier -and $skuMinSizeGB -lt [int]$result.DiskSizeGB_s -and $skuMaxSizeGB -ge [int]$result.DiskSizeGB_s ` + -and [double]$skuMaxIOps -ge [double]$result.MaxIOPSMetric -and [double]$skuMaxBandwidthMBps -ge [double]$result.MaxMBsMetric) + { + $skuSize = $sku.Size + " " + $targetSkuPerfTier.Split("_")[1] + if ($null -eq $skuPricesFound[$sku.Size]) + { + $skuPricesFound[$skuSize] = Find-DiskMonthlyPrice -DiskSizeTier $skuSize -SKUPriceSheet $pricesheetEntries + } + + if ($skuPricesFound[$skuSize] -lt [double]::MaxValue -and $skuPricesFound[$skuSize] -lt $skuPricesFound[$currentDiskTier]) + { + $targetSkuCandidate = New-Object PSObject -Property @{ + Name = $skuSize + MonthlyPrice = $skuPricesFound[$skuSize] + MaxSizeGB = $skuMaxSizeGB + MaxIOPS = $skuMaxIOps + MaxMBps = $skuMaxBandwidthMBps + } + + $targetSkuCandidates += $targetSkuCandidate + } + } + } + + $targetSku = $targetSkuCandidates | Sort-Object -Property MonthlyPrice | Select-Object -First 1 + + if ($null -ne $targetSku) + { + $queryInstanceId = $result.ResourceId + $queryText = @" + let billingInterval = 30d; + let armId = `'$queryInstanceId`'; + let gInt = $perfTimeGrain; + let ThroughputMBsPerf = $metricsTableName + | where TimeGenerated > ago(billingInterval) + | extend CollectedDate = todatetime(strcat(format_datetime(TimeGenerated, 'yyyy-MM-dd'),'T',format_datetime(TimeGenerated, 'HH'),':00:00Z')) + | where ResourceId == armId + | where MetricNames_s == 'Composite Disk Read Bytes/sec,Composite Disk Write Bytes/sec' and AggregationType_s == 'Average' and AggregationOfType_s == 'Maximum' + | extend ThroughputMBs = todouble(MetricValue_s)/1024/1024 + | project CollectedDate, ThroughputMBs, InstanceId_s=ResourceId + | join kind=inner ( + $disksTableName + | where TimeGenerated > ago(1d) + | distinct InstanceId_s, DiskThroughput_s + ) on InstanceId_s + | extend MBsPercentage = ThroughputMBs / todouble(DiskThroughput_s) * 100 + | summarize max(MBsPercentage) by bin(CollectedDate, gInt); + let IOPSPerf = $metricsTableName + | where TimeGenerated > ago(billingInterval) + | extend CollectedDate = todatetime(strcat(format_datetime(TimeGenerated, 'yyyy-MM-dd'),'T',format_datetime(TimeGenerated, 'HH'),':00:00Z')) + | where ResourceId == armId + | where MetricNames_s == 'Composite Disk Read Operations/sec,Composite Disk Write Operations/sec' and AggregationType_s == 'Average' and AggregationOfType_s == 'Maximum' + | extend IOPS = todouble(MetricValue_s) + | project CollectedDate, IOPS, InstanceId_s=ResourceId + | join kind=inner ( + $disksTableName + | where TimeGenerated > ago(1d) + | distinct InstanceId_s, DiskIOPS_s + ) on InstanceId_s + | extend IOPSPercentage = IOPS / todouble(DiskIOPS_s) * 100 + | summarize max(IOPSPercentage) by bin(CollectedDate, gInt); + ThroughputMBsPerf + | join kind=inner (IOPSPerf) on CollectedDate + | render timechart +"@ + + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $datetime.AddDays(-30).ToString("yyyy-MM-dd") + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["DiskType"] = "Managed" + $additionalInfoDictionary["currentSku"] = $result.SKU_s + $additionalInfoDictionary["targetSku"] = $targetSkuPerfTier + $additionalInfoDictionary["DiskSizeGB"] = [int] $result.DiskSizeGB_s + $additionalInfoDictionary["currentTier"] = $currentDiskTier + $additionalInfoDictionary["targetTier"] = $targetSku.Name + $additionalInfoDictionary["MaxIOPSMetric"] = [double] $($result.MaxIOPSMetric) + $additionalInfoDictionary["MaxMBpsMetric"] = [double] $($result.MaxMBsMetric) + $additionalInfoDictionary["MetricIOPSPercentage"] = [double] $($result.IOPSPercentage) + $additionalInfoDictionary["MetricMBpsPercentage"] = [double] $($result.MBsPercentage) + $additionalInfoDictionary["targetMaxSizeGB"] = [int] $targetSku.MaxSizeGB + $additionalInfoDictionary["targetMaxIOPS"] = [int] $targetSku.MaxIOPS + $additionalInfoDictionary["targetMaxMBps"] =[int] $targetSku.MaxMBps + + $fitScore = 4 # needs Maximum of Maximum for metrics to have higher fit score + if ([int] $result.DiskSizeGB_s -gt 512) + { + $fitScore = 3.5 #disk will not support credit-based bursting, therefore the recommendation risk increases a bit + } + + $fitScore = [Math]::max(0.0, $fitScore) + + $savingCoefficient = 2 # Standard SSD is generally close to half the price of Premium SSD + + $targetSkuSavingsMonthly = $result.Last30DaysCost - ($result.Last30DaysCost / $savingCoefficient) + + $tentativeTargetSkuSavingsMonthly = -1 + + if ($targetSku -and $skuPricesFound[$targetSku.Name] -lt [double]::MaxValue) + { + $targetSkuPrice = $skuPricesFound[$targetSku.Name] + + if ($skuPricesFound[$currentDiskTier] -lt [double]::MaxValue) + { + $currentSkuPrice = $skuPricesFound[$currentDiskTier] + $tentativeTargetSkuSavingsMonthly = ($currentSkuPrice * [double] $result.Last30DaysQuantity) - ($targetSkuPrice * [double] $result.Last30DaysQuantity) + } + else + { + $tentativeTargetSkuSavingsMonthly = $result.Last30DaysCost - ($targetSkuPrice * [double] $result.Last30DaysQuantity) + } + } + + if ($tentativeTargetSkuSavingsMonthly -ge 0) + { + $targetSkuSavingsMonthly = $tentativeTargetSkuSavingsMonthly + } + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + if ($targetSkuSavingsMonthly -eq [double]::PositiveInfinity) + { + $targetSkuSavingsMonthly = [double] $result.Last30DaysCost / 2 + } + + $additionalInfoDictionary["savingsAmount"] = [double] $targetSkuSavingsMonthly + $additionalInfoDictionary["CostsAmount"] = [double] $result.Last30DaysCost + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Cost" + ImpactedArea = "Microsoft.Compute/disks" + Impact = "High" + RecommendationType = "Saving" + RecommendationSubType = "UnderusedPremiumSSDDisks" + RecommendationSubTypeId = "4854b5dc-4124-4ade-879e-6a7bb65350ab" + RecommendationDescription = "Premium SSD disk has been underutilized" + RecommendationAction = "Change disk tier at least to the equivalent for Standard SSD" + InstanceId = $result.ResourceId + InstanceName = $result.DiskName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroup + SubscriptionGuid = $result.SubscriptionId + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation + } +} + +# Export the recommendations as JSON to blob storage + +Write-Output "Exporting final $($recommendations.Count) results as a JSON file..." + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "disks-underutilized-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-SqlDbOptimizationsToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-SqlDbOptimizationsToBlobStorage.ps1 new file mode 100644 index 000000000..dcb3aae36 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-SqlDbOptimizationsToBlobStorage.ps1 @@ -0,0 +1,447 @@ +$ErrorActionPreference = "Stop" + +# Collect generic and recommendation-specific variables + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$workspaceName = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceName" +$workspaceRG = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceRG" +$workspaceSubscriptionId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceSubId" +$workspaceTenantId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceTenantId" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} + +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +$perfDaysBackwards = [int] (Get-AutomationVariable -Name "AzureOptimization_RecommendPerfPeriodInDays" -ErrorAction SilentlyContinue) +if (-not($perfDaysBackwards -gt 0)) { + $perfDaysBackwards = 7 +} + +$dtuPercentile = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfPercentileSqlDtu" -ErrorAction SilentlyContinue) +if (-not($dtuPercentile -gt 0)) { + $dtuPercentile = 99 +} +$dtuPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdDtuPercentage" -ErrorAction SilentlyContinue) +if (-not($dtuPercentageThreshold -gt 0)) { + $dtuPercentageThreshold = 40 +} +$dtuDegradedPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdDtuDegradedPercentage" -ErrorAction SilentlyContinue) +if (-not($dtuDegradedPercentageThreshold -gt 0)) { + $dtuDegradedPercentageThreshold = 75 +} + +$consumptionOffsetDays = [int] (Get-AutomationVariable -Name "AzureOptimization_ConsumptionOffsetDays") +$consumptionOffsetDaysStart = $consumptionOffsetDays + 1 + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" + +# Authenticate against Azure + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +Write-Output "Finding tables where recommendations will be generated from..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE CollectedType IN ('ARGSqlDb','MonitorMetrics','AzureConsumption','ARGResourceContainers')" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$sqlDbsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGSqlDb' }).LogAnalyticsSuffix + "_CL" +$metricsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'MonitorMetrics' }).LogAnalyticsSuffix + "_CL" +$consumptionTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'AzureConsumption' }).LogAnalyticsSuffix + "_CL" +$subscriptionsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGResourceContainers' }).LogAnalyticsSuffix + "_CL" + +Write-Output "Will run query against tables $sqlDbsTableName, $subscriptionsTableName, $metricsTableName and $consumptionTableName" + +$Conn.Close() +$Conn.Dispose() + +$recommendationSearchTimeSpan = 30 + $consumptionOffsetDaysStart + +# Grab a context reference to the Storage Account where the recommendations file will be stored + + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +if ($workspaceSubscriptionId -ne $storageAccountSinkSubscriptionId) +{ + Select-AzSubscription -SubscriptionId $workspaceSubscriptionId +} + +$recommendationsErrors = 0 + +# Execute the recommendation query against Log Analytics +Write-Output "Looking for underused SQL Databases, with less than $dtuPercentageThreshold % Max. DTU usage..." + +$baseQuery = @" + let DTUPercentageThreshold = $dtuPercentageThreshold; + let MetricsInterval = $($perfDaysBackwards)d; + let BillingInterval = 30d; + let dtuPercentPercentile = $dtuPercentile; + let etime = todatetime(toscalar($consumptionTableName | where todatetime(Date_s) < now() and todatetime(Date_s) > ago(BillingInterval) | summarize max(todatetime(Date_s)))); + let stime = etime-BillingInterval; + let CandidateDatabaseIds = $sqlDbsTableName + | where TimeGenerated > ago(1d) and SkuName_s in ('Standard','Premium') + | distinct InstanceId_s; + $metricsTableName + | where TimeGenerated > ago(MetricsInterval) + | where ResourceId in (CandidateDatabaseIds) and MetricNames_s == 'dtu_consumption_percent' and AggregationType_s == 'Maximum' + | summarize P99DTUPercentage = percentile(todouble(MetricValue_s), dtuPercentPercentile) by ResourceId + | where P99DTUPercentage < DTUPercentageThreshold + | join ( + $sqlDbsTableName + | where TimeGenerated > ago(1d) + | project ResourceId = InstanceId_s, DBName_s, ResourceGroupName_s, SubscriptionGuid_g, TenantGuid_g, SkuName_s, ServiceObjectiveName_s, Tags_s, Cloud_s + ) on ResourceId + | join kind=leftouter ( + $consumptionTableName + | where todatetime(Date_s) between (stime..etime) + | project ResourceId=tolower(ResourceId), CostInBillingCurrency_s, Date_s + ) on ResourceId + | summarize Last30DaysCost=sum(todouble(CostInBillingCurrency_s)) by DBName_s, ResourceId, TenantGuid_g, SubscriptionGuid_g, ResourceGroupName_s, SkuName_s, ServiceObjectiveName_s, Tags_s, Cloud_s, P99DTUPercentage + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + $queryInstanceId = $result.ResourceId + $queryText = @" + $metricsTableName + | where ResourceId == '$queryInstanceId' + | where MetricNames_s == 'dtu_consumption_percent' and AggregationType_s == 'Maximum' + | project TimeGenerated, DTUPercentage = toint(MetricValue_s) + | render timechart +"@ + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $datetime.AddDays(-30).ToString("yyyy-MM-dd") + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["currentSku"] = "$($result.SkuName_s) $($result.ServiceObjectiveName_s)" + $additionalInfoDictionary["DTUPercentage"] = [int] $result.P99DTUPercentage + $additionalInfoDictionary["CostsAmount"] = [double] $result.Last30DaysCost + $additionalInfoDictionary["savingsAmount"] = ([double] $result.Last30DaysCost / 2) + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Cost" + ImpactedArea = "Microsoft.Sql/servers/databases" + Impact = "High" + RecommendationType = "Saving" + RecommendationSubType = "UnderusedSqlDatabases" + RecommendationSubTypeId = "ff68f4e5-1197-4be9-8e5f-8760d7863cb4" + RecommendationDescription = "Underused SQL Databases (performance capacity waste)" + RecommendationAction = "Right-size underused SQL Databases" + InstanceId = $result.ResourceId + InstanceName = $result.DBName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "sqldbs-underused-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for performance constrained SQL Databases, with more than $dtuDegradedPercentageThreshold % Avg. DTU usage..." + +$baseQuery = @" + let DTUPercentageThreshold = $dtuDegradedPercentageThreshold; + let MetricsInterval = $($perfDaysBackwards)d; + let CandidateDatabaseIds = $sqlDbsTableName + | where TimeGenerated > ago(1d) and SkuName_s in ('Basic','Standard','Premium') + | distinct InstanceId_s; + $metricsTableName + | where TimeGenerated > ago(MetricsInterval) + | where ResourceId in (CandidateDatabaseIds) and MetricNames_s == 'dtu_consumption_percent' and AggregationType_s == 'Average' and AggregationOfType_s == 'Maximum' + | summarize AvgDTUPercentage = avg(todouble(MetricValue_s)) by ResourceId + | where AvgDTUPercentage > DTUPercentageThreshold + | join ( + $sqlDbsTableName + | where TimeGenerated > ago(1d) + | project ResourceId = InstanceId_s, DBName_s, ResourceGroupName_s, SubscriptionGuid_g, TenantGuid_g, SkuName_s, ServiceObjectiveName_s, Tags_s, Cloud_s + ) on ResourceId + | project DBName_s, ResourceId, TenantGuid_g, SubscriptionGuid_g, ResourceGroupName_s, SkuName_s, ServiceObjectiveName_s, Tags_s, Cloud_s, AvgDTUPercentage + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + $queryInstanceId = $result.ResourceId + $queryText = @" + $metricsTableName + | where ResourceId == '$queryInstanceId' + | where MetricNames_s == 'dtu_consumption_percent' and AggregationType_s == 'Average' + | project TimeGenerated, DTUPercentage = toint(MetricValue_s) + | render timechart +"@ + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $datetime.AddDays(-30).ToString("yyyy-MM-dd") + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["currentSku"] = "$($result.SkuName_s) $($result.ServiceObjectiveName_s)" + $additionalInfoDictionary["DTUPercentage"] = [int] $result.AvgDTUPercentage + + $fitScore = 4 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Performance" + ImpactedArea = "Microsoft.Sql/servers/databases" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "PerfConstrainedSqlDatabases" + RecommendationSubTypeId = "724ff2f5-8c83-4105-b00d-029c4560d774" + RecommendationDescription = "SQL Database performance has been constrained by lack of resources" + RecommendationAction = "Resize SQL Database to higher SKU or scale it out" + InstanceId = $result.ResourceId + InstanceName = $result.DBName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "sqldbs-perfconstrained-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + + +if ($recommendationsErrors -gt 0) +{ + throw "Some of the recommendations queries failed. Please, review the job logs for additional information." +} \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-StorageAccountOptimizationsToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-StorageAccountOptimizationsToBlobStorage.ps1 new file mode 100644 index 000000000..431ab4b49 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-StorageAccountOptimizationsToBlobStorage.ps1 @@ -0,0 +1,330 @@ +function ConvertTo-Hashtable { + [CmdletBinding()] + [OutputType('hashtable')] + param ( + [Parameter(ValueFromPipeline)] + $InputObject + ) + + process { + if ($null -eq $InputObject) { + return $null + } + + if ($InputObject -is [System.Collections.IEnumerable] -and $InputObject -isnot [string]) { + $collection = @( + foreach ($object in $InputObject) { + ConvertTo-Hashtable -InputObject $object + } + ) + Write-Output -NoEnumerate $collection + } elseif ($InputObject -is [psobject]) { + $hash = @{} + foreach ($property in $InputObject.PSObject.Properties) { + $hash[$property.Name] = ConvertTo-Hashtable -InputObject $property.Value + } + $hash + } else { + $InputObject + } + } +} + +$ErrorActionPreference = "Stop" + +# Collect generic and recommendation-specific variables + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$workspaceName = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceName" +$workspaceRG = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceRG" +$workspaceSubscriptionId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceSubId" +$workspaceTenantId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceTenantId" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} + +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +# storage account thresholds variables +$growthPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_RecommendationStorageAcountGrowthThresholdPercentage" -ErrorAction SilentlyContinue) +if (-not($growthPercentageThreshold -gt 0)) { + $growthPercentageThreshold = 5 +} +$monthlyCostThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_RecommendationStorageAcountGrowthMonthlyCostThreshold" -ErrorAction SilentlyContinue) +if (-not($monthlyCostThreshold -gt 0)) { + $monthlyCostThreshold = 50 +} +$growthLookbackDays = [int] (Get-AutomationVariable -Name "AzureOptimization_RecommendationStorageAcountGrowthLookbackDays" -ErrorAction SilentlyContinue) +if (-not($growthLookbackDays -gt 0)) { + $growthLookbackDays = 30 +} + +$consumptionOffsetDays = [int] (Get-AutomationVariable -Name "AzureOptimization_ConsumptionOffsetDays") +$consumptionOffsetDaysStart = $consumptionOffsetDays + 1 + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" + +# Authenticate against Azure + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +$tenantId = (Get-AzContext).Tenant.Id + +Write-Output "Finding tables where recommendations will be generated from..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE CollectedType IN ('ARGResourceContainers','AzureConsumption')" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$subscriptionsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGResourceContainers' }).LogAnalyticsSuffix + "_CL" +$consumptionTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'AzureConsumption' }).LogAnalyticsSuffix + "_CL" + +Write-Output "Will run query against tables $subscriptionsTableName and $consumptionTableName" + +$Conn.Close() +$Conn.Dispose() + +$recommendationSearchTimeSpan = $growthLookbackDays + $consumptionOffsetDaysStart + +# Grab a context reference to the Storage Account where the recommendations file will be stored + + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +if ($workspaceSubscriptionId -ne $storageAccountSinkSubscriptionId) +{ + Select-AzSubscription -SubscriptionId $workspaceSubscriptionId +} + +Write-Output "Looking for ever growing Storage Accounts, with more than $monthlyCostThreshold/month costs, growing more than $growthPercentageThreshold% over the last $growthLookbackDays days..." + +$dailyCostThreshold = [Math]::Round($monthlyCostThreshold / 30) + +$baseQuery = @" +let interval = $($growthLookbackDays)d; +let etime = endofday(todatetime(toscalar($consumptionTableName | where todatetime(Date_s) > ago(interval) and todatetime(Date_s) < now() | summarize max(todatetime(Date_s))))); +let etime_subs = endofday(todatetime(toscalar($subscriptionsTableName | where TimeGenerated > ago(interval) | summarize max(TimeGenerated)))); +let stime = endofday(etime-interval); +let lastday_stime = endofday(etime-1d); +let lastday_stime_subs = endofday(etime_subs-1d); +let costThreshold = $dailyCostThreshold; +let growthPercentageThreshold = $growthPercentageThreshold; +let StorageAccountsWithLastTags = $consumptionTableName +| where todatetime(Date_s) between (lastday_stime..etime) +| where MeterCategory_s == 'Storage' and ConsumedService_s == 'Microsoft.Storage' and MeterName_s endswith 'Data Stored' and ChargeType_s == 'Usage' +| extend ResourceId = tolower(ResourceId) +| summarize arg_max(todatetime(Date_s), Tags_s) by ResourceId; +$consumptionTableName +| where todatetime(Date_s) between (stime..etime) +| where MeterCategory_s == 'Storage' and ConsumedService_s == 'Microsoft.Storage' and MeterName_s endswith 'Data Stored' and ChargeType_s == 'Usage' +| extend ResourceId = tolower(ResourceId) +| make-series CostSum=sum(todouble(CostInBillingCurrency_s)) default=0.0 on todatetime(Date_s) from stime to etime step 1d by ResourceId, ResourceGroup, SubscriptionId +| extend InitialDailyCost = todouble(CostSum[0]), CurrentDailyCost = todouble(CostSum[array_length(CostSum)-1]) +| extend GrowthPercentage = round((CurrentDailyCost-InitialDailyCost)/InitialDailyCost*100) +| where InitialDailyCost > 0 and CurrentDailyCost > costThreshold and GrowthPercentage > growthPercentageThreshold +| project ResourceId, InitialDailyCost, CurrentDailyCost, GrowthPercentage, ResourceGroup, SubscriptionId +| join kind=leftouter (StorageAccountsWithLastTags) on ResourceId +| join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > lastday_stime_subs + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionId=SubscriptionGuid_g, SubscriptionName = ContainerName_s +) on SubscriptionId +| extend Tags_s = iif(Tags_s !startswith "{", strcat('{', Tags_s, '}'), Tags_s) +| extend Tags_s = parse_json(tolower(Tags_s)) +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + throw "Execution aborted" +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + $queryInstanceId = $result.ResourceId + $queryText = @" + $consumptionTableName + | where MeterCategory_s == 'Storage' and ConsumedService_s == 'Microsoft.Storage' and MeterName_s endswith 'Data Stored' and ChargeType_s == 'Usage' + | extend ResourceId = tolower(ResourceId) + | where ResourceId =~ '$queryInstanceId' + | summarize DailyCosts = sum(todouble(CostInBillingCurrency_s)) by bin(todatetime(Date_s), 1d) + | render timechart +"@ + + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $datetime.AddDays(-1 * $recommendationSearchTimeSpan).ToString("yyyy-MM-dd") + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + + $additionalInfoDictionary = @{} + + $costsAmount = ([double] $result.InitialDailyCost + [double] $result.CurrentDailyCost) / 2 * 30 + + $additionalInfoDictionary["InitialDailyCost"] = $result.InitialDailyCost + $additionalInfoDictionary["CurrentDailyCost"] = $result.CurrentDailyCost + $additionalInfoDictionary["GrowthPercentage"] = $result.GrowthPercentage + $additionalInfoDictionary["CostsAmount"] = $costsAmount + $additionalInfoDictionary["savingsAmount"] = $costsAmount * 0.25 # estimated 25% savings + + $fitScore = 4 # savings are estimated with a significant error margin + + $fitScore = [Math]::max(0.0, $fitScore) + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + if (-not($result.Tags_s -like "{*")) + { + $result.Tags_s = '{' + $result.Tags_s + '}' + } + $tags = ConvertFrom-Json $result.Tags_s | ConvertTo-Hashtable + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $cloudEnvironment + Category = "Cost" + ImpactedArea = "Microsoft.Storage/storageAccounts" + Impact = "Medium" + RecommendationType = "Saving" + RecommendationSubType = "StorageAccountsGrowing" + RecommendationSubTypeId = "08e049ca-18b0-4d22-b174-131a91d0381c" + RecommendationDescription = "Storage Account without retention policy in place" + RecommendationAction = "Review whether the Storage Account has a retention policy for example via Lifecycle Management" + InstanceId = $result.ResourceId + InstanceName = $result.ResourceId.Split('/')[-1] + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroup + SubscriptionGuid = $result.SubscriptionId + SubscriptionName = $result.SubscriptionName + TenantGuid = $tenantId + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +Write-Output "Exporting final $($recommendations.Count) results as a JSON file..." + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "storageaccounts-costsgrowing-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-UnattachedDisksToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-UnattachedDisksToBlobStorage.ps1 new file mode 100644 index 000000000..b2de201b2 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-UnattachedDisksToBlobStorage.ps1 @@ -0,0 +1,272 @@ +$ErrorActionPreference = "Stop" + +# Collect generic and recommendation-specific variables + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$workspaceName = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceName" +$workspaceRG = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceRG" +$workspaceSubscriptionId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceSubId" +$workspaceTenantId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceTenantId" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} + +$deploymentDate = Get-AutomationVariable -Name "AzureOptimization_DeploymentDate" # yyyy-MM-dd format +$deploymentDate = $deploymentDate.Replace('"', "") + +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +$consumptionOffsetDays = [int] (Get-AutomationVariable -Name "AzureOptimization_ConsumptionOffsetDays") +$consumptionOffsetDaysStart = $consumptionOffsetDays + 1 + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" + +# Authenticate against Azure + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +Write-Output "Finding tables where recommendations will be generated from..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE CollectedType IN ('ARGManagedDisk','AzureConsumption','ARGResourceContainers')" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$disksTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGManagedDisk' }).LogAnalyticsSuffix + "_CL" +$consumptionTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'AzureConsumption' }).LogAnalyticsSuffix + "_CL" +$subscriptionsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGResourceContainers' }).LogAnalyticsSuffix + "_CL" + +Write-Output "Will run query against tables $disksTableName, $subscriptionsTableName and $consumptionTableName" + +$Conn.Close() +$Conn.Dispose() + +$recommendationSearchTimeSpan = 30 + $consumptionOffsetDaysStart + +# Grab a context reference to the Storage Account where the recommendations file will be stored + + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +if ($workspaceSubscriptionId -ne $storageAccountSinkSubscriptionId) +{ + Select-AzSubscription -SubscriptionId $workspaceSubscriptionId +} + +# Execute the recommendation query against Log Analytics + +$baseQuery = @" + let interval = 30d; + let etime = todatetime(toscalar($consumptionTableName | where todatetime(Date_s) < now() and todatetime(Date_s) > ago(interval) | summarize max(todatetime(Date_s)))); + let stime = etime-interval; + $disksTableName + | where TimeGenerated > ago(1d) and isempty(OwnerVMId_s) and Tags_s !has 'ASR-ReplicaDisk' and Tags_s !has 'asrseeddisk' + | distinct DiskName_s, InstanceId_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroupName_s, SKU_s, DiskSizeGB_s, Tags_s, Cloud_s + | join kind=leftouter ( + $consumptionTableName + | where todatetime(Date_s) between (stime..etime) + | project InstanceId_s=tolower(ResourceId), CostInBillingCurrency_s, Date_s + ) on InstanceId_s + | summarize Last30DaysCost=sum(todouble(CostInBillingCurrency_s)) by DiskName_s, InstanceId_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroupName_s, SKU_s, DiskSizeGB_s, Tags_s, Cloud_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + throw "Execution aborted" +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + $queryInstanceId = $result.InstanceId_s + $queryText = @" + $disksTableName + | where InstanceId_s == '$queryInstanceId' and isempty(OwnerVMId_s) + | distinct InstanceId_s, DiskName_s, DiskSizeGB_s, SKU_s, TimeGenerated + | summarize LastAttachedDate = min(TimeGenerated) by InstanceId_s, DiskName_s, DiskSizeGB_s, SKU_s + | join kind=leftouter ( + $consumptionTableName + | project InstanceId_s=tolower(ResourceId), CostInBillingCurrency_s, Date_s + ) on InstanceId_s + | summarize CostsSinceDetached = sumif(todouble(CostInBillingCurrency_s), todatetime(Date_s) > LastAttachedDate) by DiskName_s, LastAttachedDate, DiskSizeGB_s, SKU_s +"@ + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $deploymentDate + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["DiskType"] = "Managed" + $additionalInfoDictionary["currentSku"] = $result.SKU_s + $additionalInfoDictionary["DiskSizeGB"] = [int] $result.DiskSizeGB_s + $additionalInfoDictionary["CostsAmount"] = [double] $result.Last30DaysCost + $additionalInfoDictionary["savingsAmount"] = [double] $result.Last30DaysCost + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Cost" + ImpactedArea = "Microsoft.Compute/disks" + Impact = "Medium" + RecommendationType = "Saving" + RecommendationSubType = "UnattachedDisks" + RecommendationSubTypeId = "c84d5e86-e2d6-4d62-be7c-cecfbd73b0db" + RecommendationDescription = "Unattached disks (without owner VM) incur in unnecessary costs" + RecommendationAction = "Delete or downgrade disk to Standard SKU" + InstanceId = $result.InstanceId_s + InstanceName = $result.DiskName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "unattacheddisks-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-UnusedAppGWsToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-UnusedAppGWsToBlobStorage.ps1 new file mode 100644 index 000000000..a1708c117 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-UnusedAppGWsToBlobStorage.ps1 @@ -0,0 +1,273 @@ +$ErrorActionPreference = "Stop" + +# Collect generic and recommendation-specific variables + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$workspaceName = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceName" +$workspaceRG = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceRG" +$workspaceSubscriptionId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceSubId" +$workspaceTenantId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceTenantId" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} + +$deploymentDate = Get-AutomationVariable -Name "AzureOptimization_DeploymentDate" # yyyy-MM-dd format +$deploymentDate = $deploymentDate.Replace('"', "") + +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +$consumptionOffsetDays = [int] (Get-AutomationVariable -Name "AzureOptimization_ConsumptionOffsetDays") +$consumptionOffsetDaysStart = $consumptionOffsetDays + 1 + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" + +# Authenticate against Azure + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +Write-Output "Finding tables where recommendations will be generated from..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE CollectedType IN ('ARGAppGateway','AzureConsumption','ARGResourceContainers')" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$appGWsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGAppGateway' }).LogAnalyticsSuffix + "_CL" +$consumptionTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'AzureConsumption' }).LogAnalyticsSuffix + "_CL" +$subscriptionsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGResourceContainers' }).LogAnalyticsSuffix + "_CL" + +Write-Output "Will run query against tables $appGWsTableName, $subscriptionsTableName and $consumptionTableName" + +$Conn.Close() +$Conn.Dispose() + +$recommendationSearchTimeSpan = 30 + $consumptionOffsetDaysStart + +# Grab a context reference to the Storage Account where the recommendations file will be stored + + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +if ($workspaceSubscriptionId -ne $storageAccountSinkSubscriptionId) +{ + Select-AzSubscription -SubscriptionId $workspaceSubscriptionId +} + +# Execute the Cost recommendation query against Log Analytics + +$baseQuery = @" + let interval = 30d; + let etime = todatetime(toscalar($consumptionTableName | where todatetime(Date_s) < now() and todatetime(Date_s) > ago(interval) | summarize max(todatetime(Date_s)))); + let stime = etime-interval; + $appGWsTableName + | where TimeGenerated > ago(1d) + | where toint(BackendPoolsCount_s) == 0 or ((BackendIPCount_s == 0 or isempty(BackendIPCount_s)) and (BackendAddressesCount_s == 0 or isempty(BackendAddressesCount_s))) + | distinct InstanceName_s, InstanceId_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroupName_s, SkuName_s, SkuCapacity_s, Tags_s, Cloud_s + | join kind=leftouter ( + $consumptionTableName + | where todatetime(Date_s) between (stime..etime) + | project InstanceId_s=tolower(ResourceId), CostInBillingCurrency_s, Date_s + ) on InstanceId_s + | summarize Last30DaysCost=sum(todouble(CostInBillingCurrency_s)) by InstanceName_s, InstanceId_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroupName_s, SkuName_s, SkuCapacity_s, Tags_s, Cloud_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + throw "Execution aborted" +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + $queryInstanceId = $result.InstanceId_s + $queryText = @" + $appGWsTableName + | where InstanceId_s == '$queryInstanceId' + | where toint(BackendPoolsCount_s) == 0 or ((toint(BackendIPCount_s) == 0 or isempty(BackendIPCount_s)) and (toint(BackendAddressesCount_s) == 0 or isempty(BackendAddressesCount_s))) + | distinct InstanceId_s, InstanceName_s, TimeGenerated + | summarize FirstUnusedDate = min(TimeGenerated) by InstanceId_s, InstanceName_s + | join kind=leftouter ( + $consumptionTableName + | project InstanceId_s=tolower(ResourceId), CostInBillingCurrency_s, Date_s + ) on InstanceId_s + | summarize CostsSinceUnused = sumif(todouble(CostInBillingCurrency_s), todatetime(Date_s) > FirstUnusedDate) by InstanceName_s, FirstUnusedDate +"@ + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $deploymentDate + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["currentSku"] = $result.SkuName_s + $additionalInfoDictionary["InstanceCount"] = $result.SkuCapacity_s + $additionalInfoDictionary["CostsAmount"] = [double] $result.Last30DaysCost + $additionalInfoDictionary["savingsAmount"] = [double] $result.Last30DaysCost + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Cost" + ImpactedArea = "Microsoft.Network/applicationGateways" + Impact = "High" + RecommendationType = "Saving" + RecommendationSubType = "UnusedAppGateways" + RecommendationSubTypeId = "dc3d2baa-26c8-435e-aa9d-edb2bfd6fff6" + RecommendationDescription = "Application Gateways without a backend pool incur in unnecessary costs" + RecommendationAction = "Delete the Application Gateway" + InstanceId = $result.InstanceId_s + InstanceName = $result.InstanceName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "unusedappgateways-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-UnusedLoadBalancersToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-UnusedLoadBalancersToBlobStorage.ps1 new file mode 100644 index 000000000..ccd2ebe37 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-UnusedLoadBalancersToBlobStorage.ps1 @@ -0,0 +1,404 @@ +$ErrorActionPreference = "Stop" + +# Collect generic and recommendation-specific variables + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$workspaceName = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceName" +$workspaceRG = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceRG" +$workspaceSubscriptionId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceSubId" +$workspaceTenantId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceTenantId" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} + +$deploymentDate = Get-AutomationVariable -Name "AzureOptimization_DeploymentDate" # yyyy-MM-dd format +$deploymentDate = $deploymentDate.Replace('"', "") + +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +$consumptionOffsetDays = [int] (Get-AutomationVariable -Name "AzureOptimization_ConsumptionOffsetDays") +$consumptionOffsetDaysStart = $consumptionOffsetDays + 1 + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" + +# Authenticate against Azure + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +Write-Output "Finding tables where recommendations will be generated from..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE CollectedType IN ('ARGLoadBalancer','AzureConsumption','ARGResourceContainers')" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$lbsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGLoadBalancer' }).LogAnalyticsSuffix + "_CL" +$consumptionTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'AzureConsumption' }).LogAnalyticsSuffix + "_CL" +$subscriptionsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGResourceContainers' }).LogAnalyticsSuffix + "_CL" + +Write-Output "Will run query against tables $lbsTableName, $subscriptionsTableName and $consumptionTableName" + +$Conn.Close() +$Conn.Dispose() + +$recommendationSearchTimeSpan = 30 + $consumptionOffsetDaysStart + +# Grab a context reference to the Storage Account where the recommendations file will be stored + + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +if ($workspaceSubscriptionId -ne $storageAccountSinkSubscriptionId) +{ + Select-AzSubscription -SubscriptionId $workspaceSubscriptionId +} + +$recommendationsErrors = 0 + +# Execute the Cost recommendation query against Log Analytics + +$baseQuery = @" + let interval = 30d; + let etime = todatetime(toscalar($consumptionTableName | where todatetime(Date_s) < now() and todatetime(Date_s) > ago(interval) | summarize max(todatetime(Date_s)))); + let stime = etime-interval; + $lbsTableName + | where TimeGenerated > ago(1d) + | where SkuName_s == 'Standard' + | where (toint(BackendPoolsCount_s) == 0 or ((toint(BackendIPCount_s) == 0 or isempty(BackendIPCount_s)) and (toint(BackendAddressesCount_s) == 0 or isempty(BackendAddressesCount_s)))) and toint(InboundNatPoolsCount_s) == 0 + | where toint(LbRulesCount_s) != 0 or toint(InboundNatRulesCount_s) != 0 or toint(OutboundRulesCount_s) != 0 + | distinct InstanceName_s, InstanceId_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroupName_s, SkuName_s, Tags_s, Cloud_s + | join kind=leftouter ( + $consumptionTableName + | where todatetime(Date_s) between (stime..etime) + | project InstanceId_s=tolower(ResourceId), CostInBillingCurrency_s, Date_s + ) on InstanceId_s + | summarize Last30DaysCost=sum(todouble(CostInBillingCurrency_s)) by InstanceName_s, InstanceId_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroupName_s, SkuName_s, Tags_s, Cloud_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Costs query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + $queryInstanceId = $result.InstanceId_s + $queryText = @" + $lbsTableName + | where InstanceId_s == '$queryInstanceId' + | where SkuName_s == 'Standard' + | where (toint(BackendPoolsCount_s) == 0 or ((BackendIPCount_s == 0 or isempty(BackendIPCount_s)) and (BackendAddressesCount_s == 0 or isempty(BackendAddressesCount_s)))) and toint(InboundNatPoolsCount_s) == 0 + | where toint(LbRulesCount_s) != 0 or toint(InboundNatRulesCount_s) != 0 or toint(OutboundRulesCount_s) != 0 + | distinct InstanceId_s, InstanceName_s, SkuName_s, TimeGenerated + | summarize FirstUnusedDate = min(TimeGenerated) by InstanceId_s, InstanceName_s, SkuName_s + | join kind=leftouter ( + $consumptionTableName + | project InstanceId_s=tolower(ResourceId), CostInBillingCurrency_s, Date_s + ) on InstanceId_s + | summarize CostsSinceUnused = sumif(todouble(CostInBillingCurrency_s), todatetime(Date_s) > FirstUnusedDate) by InstanceName_s, FirstUnusedDate, SkuName_s +"@ + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $deploymentDate + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["currentSku"] = $result.SkuName_s + $additionalInfoDictionary["CostsAmount"] = [double] $result.Last30DaysCost + $additionalInfoDictionary["savingsAmount"] = [double] $result.Last30DaysCost + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Cost" + ImpactedArea = "Microsoft.Network/loadBalancers" + Impact = "Medium" + RecommendationType = "Saving" + RecommendationSubType = "UnusedStandardLoadBalancers" + RecommendationSubTypeId = "f1ed3bb2-3cb5-41e6-ba38-7001d5ff87f5" + RecommendationDescription = "Standard Load Balancers with rules defined and without a backend pool incur in unnecessary costs" + RecommendationAction = "Delete the Load Balancer" + InstanceId = $result.InstanceId_s + InstanceName = $result.InstanceName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "unusedstdloadbalancers-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + + +# Execute the Operational Excellence recommendation query against Log Analytics + +$baseQuery = @" + $lbsTableName + | where TimeGenerated > ago(1d) + | where (toint(BackendPoolsCount_s) == 0 or BackendIPCount_s == 0 or isempty(BackendIPCount_s)) and toint(InboundNatPoolsCount_s) == 0 + | distinct InstanceName_s, InstanceId_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroupName_s, SkuName_s, Tags_s, Cloud_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days 2) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Operational Excellence query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId_s + $detailsURL = "https://portal.azure.$azureTld/#@$workspaceTenantId/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["currentSku"] = $result.SkuName_s + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "OperationalExcellence" + ImpactedArea = "Microsoft.Network/loadBalancers" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "UnusedLoadBalancers" + RecommendationSubTypeId = "48619512-f4e6-4241-9c85-16f7c987950c" + RecommendationDescription = "Load Balancers without a backend pool are useless" + RecommendationAction = "Delete the Load Balancer" + InstanceId = $result.InstanceId_s + InstanceName = $result.InstanceName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "unusedloadbalancers-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +if ($recommendationsErrors -gt 0) +{ + throw "Some of the recommendations queries failed. Please, review the job logs for additional information." +} \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VMOptimizationsToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VMOptimizationsToBlobStorage.ps1 new file mode 100644 index 000000000..b09e8490c --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VMOptimizationsToBlobStorage.ps1 @@ -0,0 +1,461 @@ +$ErrorActionPreference = "Stop" + +# Collect generic and recommendation-specific variables + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$workspaceName = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceName" +$workspaceRG = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceRG" +$workspaceSubscriptionId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceSubId" +$workspaceTenantId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceTenantId" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} + +$deploymentDate = Get-AutomationVariable -Name "AzureOptimization_DeploymentDate" # yyyy-MM-dd format +$deploymentDate = $deploymentDate.Replace('"', "") + +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +$deallocatedIntervalDays = [int] (Get-AutomationVariable -Name "AzureOptimization_RecommendationLongDeallocatedVmsIntervalDays") +$consumptionOffsetDays = [int] (Get-AutomationVariable -Name "AzureOptimization_ConsumptionOffsetDays") +$consumptionOffsetDaysStart = $consumptionOffsetDays + 1 + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" + +# Authenticate against Azure + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +Write-Output "Finding tables where recommendations will be generated from..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE CollectedType IN ('ARGManagedDisk','ARGVirtualMachine','AzureConsumption','ARGResourceContainers')" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$vmsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGVirtualMachine' }).LogAnalyticsSuffix + "_CL" +$disksTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGManagedDisk' }).LogAnalyticsSuffix + "_CL" +$consumptionTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'AzureConsumption' }).LogAnalyticsSuffix + "_CL" +$subscriptionsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGResourceContainers' }).LogAnalyticsSuffix + "_CL" + +Write-Output "Will run query against tables $vmsTableName, $disksTableName, $subscriptionsTableName and $consumptionTableName" + +$Conn.Close() +$Conn.Dispose() + +$recommendationSearchTimeSpan = $deallocatedIntervalDays + $consumptionOffsetDaysStart +$offlineInterval = $deallocatedIntervalDays + $consumptionOffsetDays +$billingInterval = 30 + $consumptionOffsetDays + +# Grab a context reference to the Storage Account where the recommendations file will be stored + + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +if ($workspaceSubscriptionId -ne $storageAccountSinkSubscriptionId) +{ + Select-AzSubscription -SubscriptionId $workspaceSubscriptionId +} + +$recommendationsErrors = 0 + +Write-Output "Looking for VMs that have been deallocated for more than 30 days..." + +# Execute the recommendation query against Log Analytics + +$baseQuery = @" + let offlineInterval = $($offlineInterval)d; + let billingInterval = $($billingInterval)d; + let billingWindowIntervalEnd = $($consumptionOffsetDays)d; + let billingWindowIntervalStart = $($consumptionOffsetDaysStart)d; + let etime = todatetime(toscalar($consumptionTableName | where todatetime(Date_s) < now() and todatetime(Date_s) > ago(billingInterval) | summarize max(todatetime(Date_s)))); + let stime = etime-offlineInterval; + let BilledVMs = $consumptionTableName + | where todatetime(Date_s) between (stime..etime) + | where ResourceId like 'microsoft.compute/virtualmachines/' or ResourceId like 'microsoft.classiccompute/virtualmachines/' + | extend InstanceId_s = tolower(ResourceId) + | distinct InstanceId_s; + let RunningVMs = $vmsTableName + | where TimeGenerated > ago(billingWindowIntervalStart) and TimeGenerated < ago(billingWindowIntervalEnd) + | where PowerState_s has_any ('running','starting','readyrole') + | distinct InstanceId_s; + let BilledDisks = $consumptionTableName + | where todatetime(Date_s) between (stime..etime) + | where ResourceId like 'microsoft.compute/disks/' + | extend BillingInstanceId = tolower(ResourceId) + | summarize DisksCosts = sum(todouble(CostInBillingCurrency_s)) by BillingInstanceId; + $vmsTableName + | where TimeGenerated > ago(billingWindowIntervalStart) and TimeGenerated < ago(billingWindowIntervalEnd) + | where InstanceId_s !in (RunningVMs) + | join kind=leftouter (BilledVMs) on InstanceId_s + | where isempty(InstanceId_s1) + | project InstanceId_s, VMName_s, ResourceGroupName_s, SubscriptionGuid_g, TenantGuid_g, Cloud_s, Tags_s + | join kind=leftouter ( + $disksTableName + | where TimeGenerated > ago(1d) + | project DiskInstanceId = InstanceId_s, SKU_s, OwnerVMId_s + ) on `$left.InstanceId_s == `$right.OwnerVMId_s + | join kind=leftouter ( + BilledDisks + ) on `$left.DiskInstanceId == `$right.BillingInstanceId + | summarize TotalDisksCosts = sum(DisksCosts) by InstanceId_s, VMName_s, ResourceGroupName_s, SubscriptionGuid_g, TenantGuid_g, Cloud_s, Tags_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + $queryInstanceId = $result.InstanceId_s + $queryText = @" + let offlineInterval = $($offlineInterval)d; + $consumptionTableName + | extend ResourceId = tolower(ResourceId) + | where ResourceId =~ '$queryInstanceId' + | where todatetime(Date_s) < now() + | join kind=inner ( + $disksTableName + | extend DiskInstanceId = InstanceId_s + ) + on `$left.ResourceId == `$right.OwnerVMId_s + | summarize DeallocatedSince = max(todatetime(Date_s)) by DiskName_s, DiskSizeGB_s, SKU_s, DiskInstanceId + | join kind=inner + ( + $consumptionTableName + | where todatetime(Date_s) > ago(offlineInterval) + | extend DiskInstanceId = tolower(ResourceId) + | summarize DiskCosts = sum(todouble(CostInBillingCurrency_s)) by DiskInstanceId + ) + on DiskInstanceId + | project DeallocatedSince, DiskName_s, DiskSizeGB_s, SKU_s, MonthlyCosts = DiskCosts +"@ + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $deploymentDate + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["LongDeallocatedThreshold"] = $deallocatedIntervalDays + $additionalInfoDictionary["CostsAmount"] = [double] $result.TotalDisksCosts + $additionalInfoDictionary["savingsAmount"] = [double] $result.TotalDisksCosts + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Cost" + ImpactedArea = "Microsoft.Compute/virtualMachines" + Impact = "Medium" + RecommendationType = "Saving" + RecommendationSubType = "LongDeallocatedVms" + RecommendationSubTypeId = "c320b790-2e58-452a-aa63-7b62c383ad8a" + RecommendationDescription = "Virtual Machine has been deallocated for long with disks still incurring costs" + RecommendationAction = "Delete Virtual Machine or downgrade its disks to Standard HDD SKU" + InstanceId = $result.InstanceId_s + InstanceName = $result.VMName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "longdeallocatedvms-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for VMs that are stopped (not deallocated)..." + +# Execute the recommendation query against Log Analytics + +$baseQuery = @" + $vmsTableName + | where TimeGenerated > ago(1d) + | where PowerState_s has 'stopped' + | project InstanceId_s, VMName_s, ResourceGroupName_s, SubscriptionGuid_g, TenantGuid_g, Cloud_s, Tags_s + | join kind=leftouter ( + $consumptionTableName + | where TimeGenerated > ago(1d) and MeterCategory_s == 'Virtual Machines' + | project InstanceId_s=tolower(ResourceId), UnitPrice_s, EffectivePrice_s + | summarize arg_max(todouble(EffectivePrice_s), *) by InstanceId_s + | project InstanceId_s, MonthlyCost=24*todouble(iif(todouble(UnitPrice_s) > 0, todouble(UnitPrice_s), todouble(EffectivePrice_s)))*30 + ) on InstanceId_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + $queryInstanceId = $result.InstanceId_s + $queryText = @" + let LastNonStopped = toscalar($vmsTableName + | where InstanceId_s =~ '$queryInstanceId' + | where TimeGenerated < now() + | where PowerState_s !has 'stopped' + | summarize max(todatetime(StatusDate_s))); + $consumptionTableName + | where ResourceId =~ '$queryInstanceId' + | where todatetime(Date_s) >= LastNonStopped + | where MeterCategory_s == 'Virtual Machines' + | summarize ComputeCostsSinceStopped = sum(todouble(Quantity_s)*todouble(UnitPrice_s)) by MeterSubCategory_s, StoppedSince=LastNonStopped +"@ + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $deploymentDate + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["CostsAmount"] = [double] $result.MonthlyCost + $additionalInfoDictionary["savingsAmount"] = [double] $result.MonthlyCost + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Cost" + ImpactedArea = "Microsoft.Compute/virtualMachines" + Impact = "High" + RecommendationType = "Saving" + RecommendationSubType = "StoppedVms" + RecommendationSubTypeId = "110fea55-a9c3-480d-8248-116f61e139a8" + RecommendationDescription = "Virtual Machine is stopped (not deallocated) and still incurring costs" + RecommendationAction = "Deallocate Virtual Machine" + InstanceId = $result.InstanceId_s + InstanceName = $result.VMName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "stoppedvms-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +if ($recommendationsErrors -gt 0) +{ + throw "Some of the recommendations queries failed. Please, review the job logs for additional information." +} \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VMSSOptimizationsToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VMSSOptimizationsToBlobStorage.ps1 new file mode 100644 index 000000000..2b478d142 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VMSSOptimizationsToBlobStorage.ps1 @@ -0,0 +1,797 @@ +$ErrorActionPreference = "Stop" + +function Find-SkuHourlyPrice { + param ( + [object[]] $SKUPriceSheet, + [string] $SKUName + ) + + $skuPriceObject = $null + + if ($SKUPriceSheet) + { + $skuNameParts = $SKUName.Split('_') + + if ($skuNameParts.Count -eq 3) # e.g., Standard_D1_v2 + { + $skuNameFilter = "*" + $skuNameParts[1] + " *" + $skuVersionFilter = "*" + $skuNameParts[2] + $skuPrices = $SKUPriceSheet | Where-Object { $_.MeterName_s -like $skuNameFilter ` + -and $_.MeterName_s -notlike '*Low Priority' -and $_.MeterName_s -notlike '*Expired' ` + -and $_.MeterName_s -like $skuVersionFilter -and $_.MeterSubCategory_s -notlike '*Windows' -and $_.UnitPrice_s -ne 0 } + + if (($skuPrices -or $skuPrices.Count -ge 1) -and $skuPrices.Count -le 2) + { + $skuPriceObject = $skuPrices[0] + } + if ($skuPrices.Count -gt 2) # D1-like scenarios + { + $skuFilter = "*" + $skuNameParts[1] + " " + $skuNameParts[2] + "*" + $skuPrices = $skuPrices | Where-Object { $_.MeterName_s -like $skuFilter } + + if (($skuPrices -or $skuPrices.Count -ge 1) -and $skuPrices.Count -le 2) + { + $skuPriceObject = $skuPrices[0] + } + } + } + + if ($skuNameParts.Count -eq 2) # e.g., Standard_D1 + { + $skuNameFilter = "*" + $skuNameParts[1] + "*" + + $skuPrices = $SKUPriceSheet | Where-Object { $_.MeterName_s -like $skuNameFilter ` + -and $_.MeterName_s -notlike '*Low Priority' -and $_.MeterName_s -notlike '*Expired' ` + -and $_.MeterName_s -notlike '* v*' -and $_.MeterSubCategory_s -notlike '*Windows' -and $_.UnitPrice_s -ne 0 } + + if (($skuPrices -or $skuPrices.Count -ge 1) -and $skuPrices.Count -le 2) + { + $skuPriceObject = $skuPrices[0] + } + if ($skuPrices.Count -gt 2) # D1-like scenarios + { + $skuFilterLeft = "*" + $skuNameParts[1] + "/*" + $skuFilterRight = "*/" + $skuNameParts[1] + "*" + $skuPrices = $skuPrices | Where-Object { $_.MeterName_s -like $skuFilterLeft -or $_.MeterName_s -like $skuFilterRight } + + if (($skuPrices -or $skuPrices.Count -ge 1) -and $skuPrices.Count -le 2) + { + $skuPriceObject = $skuPrices[0] + } + } + } + } + + $targetHourlyPrice = [double]::MaxValue + if ($null -ne $skuPriceObject) + { + $targetUnitHours = [int] (Select-String -InputObject $skuPriceObject.UnitOfMeasure_s -Pattern "^\d+").Matches[0].Value + if ($targetUnitHours -gt 0) + { + $targetHourlyPrice = [double] ($skuPriceObject.UnitPrice_s / $targetUnitHours) + } + } + + return $targetHourlyPrice +} + +# Collect generic and recommendation-specific variables + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$workspaceName = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceName" +$workspaceRG = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceRG" +$workspaceSubscriptionId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceSubId" +$workspaceTenantId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceTenantId" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} + +$deploymentDate = Get-AutomationVariable -Name "AzureOptimization_DeploymentDate" # yyyy-MM-dd format +$deploymentDate = $deploymentDate.Replace('"', "") + +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +# percentiles variables +$cpuPercentile = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfPercentileCpu" -ErrorAction SilentlyContinue) +if (-not($cpuPercentile -gt 0)) { + $cpuPercentile = 99 +} +$memoryPercentile = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfPercentileMemory" -ErrorAction SilentlyContinue) +if (-not($memoryPercentile -gt 0)) { + $memoryPercentile = 99 +} + +# perf thresholds variables +$cpuPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdCpuPercentage" -ErrorAction SilentlyContinue) +if (-not($cpuPercentageThreshold -gt 0)) { + $cpuPercentageThreshold = 30 +} +$memoryPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdMemoryPercentage" -ErrorAction SilentlyContinue) +if (-not($memoryPercentageThreshold -gt 0)) { + $memoryPercentageThreshold = 50 +} +$cpuDegradedMaxPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdCpuDegradedMaxPercentage" -ErrorAction SilentlyContinue) +if (-not($cpuDegradedMaxPercentageThreshold -gt 0)) { + $cpuDegradedMaxPercentageThreshold = 95 +} +$cpuDegradedAvgPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdCpuDegradedAvgPercentage" -ErrorAction SilentlyContinue) +if (-not($cpuDegradedAvgPercentageThreshold -gt 0)) { + $cpuDegradedAvgPercentageThreshold = 75 +} +$memoryDegradedPercentageThreshold = [int] (Get-AutomationVariable -Name "AzureOptimization_PerfThresholdMemoryDegradedPercentage" -ErrorAction SilentlyContinue) +if (-not($memoryDegradedPercentageThreshold -gt 0)) { + $memoryDegradedPercentageThreshold = 90 +} + +$consumptionOffsetDays = [int] (Get-AutomationVariable -Name "AzureOptimization_ConsumptionOffsetDays") +$consumptionOffsetDaysStart = $consumptionOffsetDays + 1 + +$perfDaysBackwards = [int] (Get-AutomationVariable -Name "AzureOptimization_RecommendPerfPeriodInDays" -ErrorAction SilentlyContinue) +if (-not($perfDaysBackwards -gt 0)) { + $perfDaysBackwards = 7 +} + +$perfTimeGrain = Get-AutomationVariable -Name "AzureOptimization_RecommendPerfTimeGrain" -ErrorAction SilentlyContinue +if (-not($perfTimeGrain)) { + $perfTimeGrain = "1h" +} + +$referenceRegion = Get-AutomationVariable -Name "AzureOptimization_ReferenceRegion" + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" + +# Authenticate against Azure + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +Write-Output "Finding tables where recommendations will be generated from..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE CollectedType IN ('ARGVMSS','MonitorMetrics','ARGResourceContainers','AzureConsumption','Pricesheet')" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$vmssTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGVMSS' }).LogAnalyticsSuffix + "_CL" +$metricsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'MonitorMetrics' }).LogAnalyticsSuffix + "_CL" +$subscriptionsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGResourceContainers' }).LogAnalyticsSuffix + "_CL" +$consumptionTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'AzureConsumption' }).LogAnalyticsSuffix + "_CL" +$pricesheetTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'Pricesheet' }).LogAnalyticsSuffix + "_CL" + +Write-Output "Will run query against tables $vmssTableName, $metricsTableName, $subscriptionsTableName, $pricesheetTableName and $consumptionTableName" + +$Conn.Close() +$Conn.Dispose() + +$recommendationSearchTimeSpan = 30 + $consumptionOffsetDaysStart + +# Grab a context reference to the Storage Account where the recommendations file will be stored + + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +if ($workspaceSubscriptionId -ne $storageAccountSinkSubscriptionId) +{ + Select-AzSubscription -SubscriptionId $workspaceSubscriptionId +} + +Write-Output "Getting Virtual Machine SKUs for the $referenceRegion region..." + +$skus = Get-AzComputeResourceSku -Location $referenceRegion | Where-Object { $_.ResourceType -eq "virtualMachines" } + +Write-Output "Getting the current Pricesheet..." + +if ($cloudEnvironment -eq "AzureCloud") +{ + $pricesheetRegion = "EU West" +} + +try +{ + $pricesheetEntries = @() + + $baseQuery = @" + $pricesheetTableName + | where TimeGenerated > ago(14d) + | where MeterCategory_s == 'Virtual Machines' and MeterRegion_s == '$pricesheetRegion' and PriceType_s == 'Consumption' + | distinct MeterName_s, MeterSubCategory_s, MeterCategory_s, MeterRegion_s, UnitPrice_s, UnitOfMeasure_s +"@ + + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days 14) -Wait 600 -IncludeStatistics + $pricesheetEntries = [System.Linq.Enumerable]::ToArray($queryResults.Results) + + Write-Output "Query finished with $($pricesheetEntries.Count) results." + Write-Output "Query statistics: $($queryResults.Statistics.query)" +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + Write-Output "Consumption pricesheet not available, will estimate savings based in cores count..." +} + +$skuPricesFound = @{} + +$recommendationsErrors = 0 + +Write-Output "Looking for underutilized Scale Sets, with less than $cpuPercentageThreshold% CPU and $memoryPercentageThreshold% RAM usage..." + +$baseQuery = @" + let billingInterval = 30d; + let perfInterval = $($perfDaysBackwards)d; + let cpuPercentileValue = $cpuPercentile; + let memoryPercentileValue = $memoryPercentile; + let etime = todatetime(toscalar($consumptionTableName | where todatetime(Date_s) < now() and todatetime(Date_s) > ago(30d) | summarize max(todatetime(Date_s)))); + let stime = etime-billingInterval; + + let BilledVMs = $consumptionTableName + | where todatetime(Date_s) between (stime..etime) and ResourceId contains 'virtualmachinescalesets' + | extend VMConsumedQuantity = iif(ResourceId contains 'virtualmachinescalesets' and MeterCategory_s == 'Virtual Machines', todouble(Quantity_s), 0.0) + | extend VMPrice = iif(ResourceId contains 'virtualmachinescalesets' and MeterCategory_s == 'Virtual Machines', todouble(EffectivePrice_s), 0.0) + | extend FinalCost = VMPrice * VMConsumedQuantity + | extend InstanceId_s = tolower(ResourceId) + | summarize Last30DaysCost = sum(FinalCost), Last30DaysQuantity = sum(VMConsumedQuantity) by InstanceId_s; + + let MemoryPerf = $metricsTableName + | where TimeGenerated > ago(perfInterval) + | where MetricNames_s == "Available Memory Bytes" and AggregationType_s == "Minimum" + | extend MemoryAvailableMBs = todouble(MetricValue_s)/1024/1024 + | project TimeGenerated, MemoryAvailableMBs, InstanceId_s=ResourceId + | join kind=inner ( + $vmssTableName + | where TimeGenerated > ago(1d) + | distinct InstanceId_s, MemoryMB_s + ) on InstanceId_s + | extend MemoryPercentage = todouble(toint(MemoryMB_s) - toint(MemoryAvailableMBs)) / todouble(MemoryMB_s) * 100 + | summarize PMemoryPercentage = percentile(MemoryPercentage, memoryPercentileValue) by InstanceId_s; + + let ProcessorPerf = $metricsTableName + | where TimeGenerated > ago(perfInterval) + | where MetricNames_s == "Percentage CPU" and AggregationType_s == 'Maximum' + | extend InstanceId_s = ResourceId + | summarize PCPUPercentage = percentile(todouble(MetricValue_s), cpuPercentileValue) by InstanceId_s; + + $vmssTableName + | where TimeGenerated > ago(1d) + | distinct InstanceId_s, VMSSName_s, ResourceGroupName_s, SubscriptionGuid_g, Cloud_s, TenantGuid_g, VMSSSize_s, NicCount_s, DataDiskCount_s, Capacity_s, Tags_s + | join kind=inner ( BilledVMs ) on InstanceId_s + | join kind=leftouter ( MemoryPerf ) on InstanceId_s + | join kind=leftouter ( ProcessorPerf ) on InstanceId_s + | project InstanceId_s, VMSSName = VMSSName_s, ResourceGroup = ResourceGroupName_s, SubscriptionId = SubscriptionGuid_g, Cloud_s, TenantGuid_g, VMSSSize_s, NicCount_s, DataDiskCount_s, Capacity_s, PMemoryPercentage, PCPUPercentage, Tags_s, Last30DaysCost, Last30DaysQuantity + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionId = SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionId + | where isnotempty(PMemoryPercentage) and isnotempty(PCPUPercentage) and PMemoryPercentage < $memoryPercentageThreshold and PCPUPercentage < $cpuPercentageThreshold +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + + $targetSku = $null + $currentSku = $skus | Where-Object { $_.Name -eq $result.VMSSSize_s } + + $currentSkuvCPUs = [int]($currentSku.Capabilities | Where-Object { $_.Name -eq 'vCPUsAvailable' }).Value + + $memoryNeeded = [double]($currentSku.Capabilities | Where-Object { $_.Name -eq 'MemoryGB' }).Value * ([double] $result.PMemoryPercentage / 100) + $cpuNeeded = [double]$currentSkuvCPUs * ([double] $result.PCPUPercentage / 100) + $currentPremiumIO = [bool] ($currentSku.Capabilities | Where-Object { $_.Name -eq 'PremiumIO' }).Value + $currentCpuArch = ($currentSku.Capabilities | Where-Object { $_.Name -eq 'CpuArchitectureType' }).Value + + if ($null -eq $skuPricesFound[$currentSku.Name]) + { + $skuPricesFound[$currentSku.Name] = Find-SkuHourlyPrice -SKUName $currentSku.Name -SKUPriceSheet $pricesheetEntries + } + + $targetSkuCandidates = @() + + foreach ($sku in $skus) + { + $targetSkuCandidate = $null + + $skuCPUs = [int] ($sku.Capabilities | Where-Object { $_.Name -eq 'vCPUsAvailable' }).Value + $skuMemory = [int] ($sku.Capabilities | Where-Object { $_.Name -eq 'MemoryGB' }).Value + $skuMaxDataDisks = [int] ($sku.Capabilities | Where-Object { $_.Name -eq 'MaxDataDiskCount' }).Value + $skuMaxNICs = [int] ($sku.Capabilities | Where-Object { $_.Name -eq 'MaxNetworkInterfaces' }).Value + $skuPremiumIO = [bool] ($sku.Capabilities | Where-Object { $_.Name -eq 'PremiumIO' }).Value + $skuCpuArch = ($sku.Capabilities | Where-Object { $_.Name -eq 'CpuArchitectureType' }).Value + + if ($currentSku.Name -ne $sku.Name -and -not($sku.Name -like "*Promo*") -and [double]$skuCPUs -ge $cpuNeeded -and [double]$skuMemory -ge $memoryNeeded ` + -and $skuMaxDataDisks -ge [int] $result.DataDiskCount_s -and $skuMaxNICs -ge [int] $result.NicCount_s ` + -and ($currentPremiumIO -eq $false -or $skuPremiumIO -eq $currentPremiumIO) -and $skuCpuArch -eq $currentCpuArch) + { + if ($null -eq $skuPricesFound[$sku.Name]) + { + $skuPricesFound[$sku.Name] = Find-SkuHourlyPrice -SKUName $sku.Name -SKUPriceSheet $pricesheetEntries + } + + if ($skuPricesFound[$currentSku.Name] -eq 0 -or $skuPricesFound[$sku.Name] -lt $skuPricesFound[$currentSku.Name]) + { + $targetSkuCandidate = New-Object PSObject -Property @{ + Name = $sku.Name + HourlyPrice = $skuPricesFound[$sku.Name] + vCPUsAvailable = $skuCPUs + MemoryGB = $skuMemory + } + + $targetSkuCandidates += $targetSkuCandidate + } + } + } + + $targetSku = $targetSkuCandidates | Sort-Object -Property HourlyPrice,MemoryGB,vCPUsAvailable | Select-Object -First 1 + + if ($null -ne $targetSku) + { + $queryInstanceId = $result.InstanceId_s + $queryText = @" + let billingInterval = 30d; + let armId = `'$queryInstanceId`'; + let gInt = $perfTimeGrain; + let MemoryPerf = $metricsTableName + | where TimeGenerated > ago(billingInterval) + | extend CollectedDate = todatetime(strcat(format_datetime(TimeGenerated, 'yyyy-MM-dd'),'T',format_datetime(TimeGenerated, 'HH'),':00:00Z')) + | where ResourceId == armId + | where MetricNames_s == 'Available Memory Bytes' and AggregationType_s == 'Minimum' + | extend MemoryAvailableMBs = todouble(MetricValue_s)/1024/1024 + | project CollectedDate, MemoryAvailableMBs, InstanceId_s=ResourceId + | join kind=inner ( + $vmssTableName + | where TimeGenerated > ago(1d) + | distinct InstanceId_s, MemoryMB_s + ) on InstanceId_s + | extend MemoryPercentage = todouble(toint(MemoryMB_s) - toint(MemoryAvailableMBs)) / todouble(MemoryMB_s) * 100 + | summarize percentile(MemoryPercentage, $memoryPercentile) by bin(CollectedDate, gInt); + let ProcessorPerf = $metricsTableName + | where TimeGenerated > ago(billingInterval) + | extend CollectedDate = todatetime(strcat(format_datetime(TimeGenerated, 'yyyy-MM-dd'),'T',format_datetime(TimeGenerated, 'HH'),':00:00Z')) + | where ResourceId == armId + | where MetricNames_s == 'Percentage CPU' and AggregationType_s == 'Maximum' + | extend ProcessorPercentage = todouble(MetricValue_s) + | summarize percentile(ProcessorPercentage, $cpuPercentile) by bin(CollectedDate, gInt); + MemoryPerf + | join kind=inner (ProcessorPerf) on CollectedDate + | render timechart +"@ + + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $datetime.AddDays(-30).ToString("yyyy-MM-dd") + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["SupportsDataDisksCount"] = "true" + $additionalInfoDictionary["SupportsNICCount"] = "true" + $additionalInfoDictionary["BelowCPUThreshold"] = "true" + $additionalInfoDictionary["BelowMemoryThreshold"] = "true" + $additionalInfoDictionary["currentSku"] = "$($result.VMSSSize_s)" + $additionalInfoDictionary["InstanceCount"] = [int] $result.Capacity_s + $additionalInfoDictionary["targetSku"] = "$($targetSku.Name)" + $additionalInfoDictionary["DataDiskCount"] = "$($result.DataDiskCount_s)" + $additionalInfoDictionary["NicCount"] = "$($result.NicCount_s)" + $additionalInfoDictionary["MetricCPUPercentage"] = "$($result.PCPUPercentage)" + $additionalInfoDictionary["MetricMemoryPercentage"] = "$($result.PMemoryPercentage)" + + $fitScore = 4 # needs disk IOPS and throughput analysis to improve score + + $fitScore = [Math]::max(0.0, $fitScore) + + $savingCoefficient = [double] $currentSkuvCPUs / [double] $targetSku.vCPUsAvailable + + if ($targetSku -and $null -eq $skuPricesFound[$targetSku.Name]) + { + $skuPricesFound[$targetSku.Name] = Find-SkuHourlyPrice -SKUName $targetSku.Name -SKUPriceSheet $pricesheetEntries + } + + $targetSkuSavingsMonthly = $result.Last30DaysCost - ($result.Last30DaysCost / $savingCoefficient) + + $tentativeTargetSkuSavingsMonthly = -1 + + if ($targetSku -and $skuPricesFound[$targetSku.Name] -lt [double]::MaxValue) + { + $targetSkuPrice = $skuPricesFound[$targetSku.Name] + + if ($null -eq $skuPricesFound[$currentSku.Name]) + { + $skuPricesFound[$currentSku.Name] = Find-SkuHourlyPrice -SKUName $currentSku.Name -SKUPriceSheet $pricesheetEntries + } + + if ($skuPricesFound[$currentSku.Name] -lt [double]::MaxValue) + { + $currentSkuPrice = $skuPricesFound[$currentSku.Name] + $tentativeTargetSkuSavingsMonthly = ($currentSkuPrice * [double] $result.Last30DaysQuantity) - ($targetSkuPrice * [double] $result.Last30DaysQuantity) + } + else + { + $tentativeTargetSkuSavingsMonthly = $result.Last30DaysCost - ($targetSkuPrice * [double] $result.Last30DaysQuantity) + } + } + + if ($tentativeTargetSkuSavingsMonthly -ge 0) + { + $targetSkuSavingsMonthly = $tentativeTargetSkuSavingsMonthly + } + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + if ($targetSkuSavingsMonthly -eq [double]::PositiveInfinity) + { + $targetSkuSavingsMonthly = [double] $result.Last30DaysCost / 2 + } + + $additionalInfoDictionary["savingsAmount"] = [double] $targetSkuSavingsMonthly + $additionalInfoDictionary["CostsAmount"] = [double] $result.Last30DaysCost + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Cost" + ImpactedArea = "Microsoft.Compute/virtualMachineScaleSets" + Impact = "High" + RecommendationType = "Saving" + RecommendationSubType = "UnderusedVMSS" + RecommendationSubTypeId = "a4955cc9-533d-46a2-8625-5c4ebd1c30d5" + RecommendationDescription = "VM Scale Set has been underutilized" + RecommendationAction = "Resize VM Scale Set to lower SKU or scale it in" + InstanceId = $result.InstanceId_s + InstanceName = $result.VMSSName + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroup + SubscriptionGuid = $result.SubscriptionId + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation + } +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "vmss-underutilized-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for performance constrained Scale Sets, with more than $cpuDegradedMaxPercentageThreshold% Max. CPU, $cpuDegradedAvgPercentageThreshold% Avg. CPU and $memoryDegradedPercentageThreshold% RAM usage..." + +$baseQuery = @" + let perfInterval = $($perfDaysBackwards)d; + + let MemoryPerf = $metricsTableName + | where TimeGenerated > ago(perfInterval) + | where MetricNames_s == "Available Memory Bytes" and AggregationType_s == "Minimum" + | extend MemoryAvailableMBs = todouble(MetricValue_s)/1024/1024 + | project TimeGenerated, MemoryAvailableMBs, InstanceId_s=ResourceId + | join kind=inner ( + $vmssTableName + | where TimeGenerated > ago(1d) + | distinct InstanceId_s, MemoryMB_s + ) on InstanceId_s + | extend MemoryPercentage = todouble(toint(MemoryMB_s) - toint(MemoryAvailableMBs)) / todouble(MemoryMB_s) * 100 + | summarize PMemoryPercentage = avg(MemoryPercentage) by InstanceId_s; + + let ProcessorMaxPerf = $metricsTableName + | where TimeGenerated > ago(perfInterval) + | where MetricNames_s == "Percentage CPU" and AggregationType_s == 'Maximum' + | extend InstanceId_s = ResourceId + | summarize PCPUMaxPercentage = avg(todouble(MetricValue_s)) by InstanceId_s; + + let ProcessorAvgPerf = $metricsTableName + | where TimeGenerated > ago(perfInterval) + | where MetricNames_s == "Percentage CPU" and AggregationType_s == 'Average' + | extend InstanceId_s = ResourceId + | summarize PCPUAvgPercentage = avg(todouble(MetricValue_s)) by InstanceId_s; + + $vmssTableName + | where TimeGenerated > ago(1d) + | distinct InstanceId_s, VMSSName_s, ResourceGroupName_s, SubscriptionGuid_g, Cloud_s, TenantGuid_g, VMSSSize_s, NicCount_s, DataDiskCount_s, Capacity_s, Tags_s + | join kind=leftouter ( MemoryPerf ) on InstanceId_s + | join kind=leftouter ( ProcessorMaxPerf ) on InstanceId_s + | join kind=leftouter ( ProcessorAvgPerf ) on InstanceId_s + | project InstanceId_s, VMSSName = VMSSName_s, ResourceGroup = ResourceGroupName_s, SubscriptionId = SubscriptionGuid_g, Cloud_s, TenantGuid_g, VMSSSize_s, NicCount_s, DataDiskCount_s, Capacity_s, PMemoryPercentage, PCPUMaxPercentage, PCPUAvgPercentage, Tags_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionId = SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionId + | where isnotempty(PMemoryPercentage) and isnotempty(PCPUAvgPercentage) and isnotempty(PCPUMaxPercentage) and (PMemoryPercentage > $memoryDegradedPercentageThreshold or (PCPUMaxPercentage > $cpuDegradedMaxPercentageThreshold and PCPUAvgPercentage > $cpuDegradedAvgPercentageThreshold)) +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + $queryInstanceId = $result.InstanceId_s + $queryText = @" + let perfInterval = $($perfDaysBackwards)d; + let armId = `'$queryInstanceId`'; + let gInt = $perfTimeGrain; + let MemoryPerf = $metricsTableName + | where TimeGenerated > ago(perfInterval) + | extend CollectedDate = todatetime(strcat(format_datetime(TimeGenerated, 'yyyy-MM-dd'),'T',format_datetime(TimeGenerated, 'HH'),':00:00Z')) + | where ResourceId == armId + | where MetricNames_s == 'Available Memory Bytes' and AggregationType_s == 'Minimum' + | extend MemoryAvailableMBs = todouble(MetricValue_s)/1024/1024 + | project CollectedDate, MemoryAvailableMBs, InstanceId_s=ResourceId + | join kind=inner ( + $vmssTableName + | where TimeGenerated > ago(1d) + | distinct InstanceId_s, MemoryMB_s + ) on InstanceId_s + | extend MemoryPercentage = todouble(toint(MemoryMB_s) - toint(MemoryAvailableMBs)) / todouble(MemoryMB_s) * 100 + | summarize avg(MemoryPercentage) by bin(CollectedDate, gInt); + let ProcessorMaxPerf = $metricsTableName + | where TimeGenerated > ago(perfInterval) + | extend CollectedDate = todatetime(strcat(format_datetime(TimeGenerated, 'yyyy-MM-dd'),'T',format_datetime(TimeGenerated, 'HH'),':00:00Z')) + | where ResourceId == armId + | where MetricNames_s == 'Percentage CPU' and AggregationType_s == 'Maximum' + | extend ProcessorMaxPercentage = todouble(MetricValue_s) + | summarize percentile(ProcessorMaxPercentage, $cpuPercentile) by bin(CollectedDate, gInt); + let ProcessorAvgPerf = $metricsTableName + | where TimeGenerated > ago(perfInterval) + | extend CollectedDate = todatetime(strcat(format_datetime(TimeGenerated, 'yyyy-MM-dd'),'T',format_datetime(TimeGenerated, 'HH'),':00:00Z')) + | where ResourceId == armId + | where MetricNames_s == 'Percentage CPU' and AggregationType_s == 'Average' + | extend ProcessorAvgPercentage = todouble(MetricValue_s) + | summarize percentile(ProcessorAvgPercentage, $cpuPercentile) by bin(CollectedDate, gInt); + MemoryPerf + | join kind=inner (ProcessorMaxPerf) on CollectedDate + | join kind=inner (ProcessorAvgPerf) on CollectedDate + | render timechart +"@ + + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $datetime.AddDays(-30).ToString("yyyy-MM-dd") + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["currentSku"] = "$($result.VMSSSize_s)" + $additionalInfoDictionary["InstanceCount"] = [int] $result.Capacity_s + $additionalInfoDictionary["MetricCPUAvgPercentage"] = "$($result.PCPUAvgPercentage)" + $additionalInfoDictionary["MetricCPUMaxPercentage"] = "$($result.PCPUMaxPercentage)" + $additionalInfoDictionary["MetricMemoryPercentage"] = "$($result.PMemoryPercentage)" + + $fitScore = 3 # needs disk IOPS and throughput analysis to improve score + + if ([double] $result.PCPUMaxPercentage -gt [double] $cpuDegradedMaxPercentageThreshold -and [double] $result.PCPUAvgPercentage -gt [double] $cpuDegradedAvgPercentageThreshold) + { + $fitScore = 4 + } + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Performance" + ImpactedArea = "Microsoft.Compute/virtualMachineScaleSets" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "PerfConstrainedVMSS" + RecommendationSubTypeId = "20a40c62-e5c8-4cc3-9fc2-f4ac75013182" + RecommendationDescription = "VM Scale Set performance has been constrained by lack of resources" + RecommendationAction = "Resize VM Scale Set to higher SKU or scale it out" + InstanceId = $result.InstanceId_s + InstanceName = $result.VMSSName + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroup + SubscriptionGuid = $result.SubscriptionId + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "vmss-perfconstrained-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +if ($recommendationsErrors -gt 0) +{ + throw "Some of the recommendations queries failed. Please, review the job logs for additional information." +} \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VMsHighAvailabilityToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VMsHighAvailabilityToBlobStorage.ps1 new file mode 100644 index 000000000..8716e24a8 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VMsHighAvailabilityToBlobStorage.ps1 @@ -0,0 +1,1476 @@ +$ErrorActionPreference = "Stop" + +# Collect generic and recommendation-specific variables + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$workspaceSubscriptionId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceSubId" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} + +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" + +# Authenticate against Azure + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +Write-Output "Finding tables where recommendations will be generated from..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE CollectedType IN ('ARGVirtualMachine','ARGUnmanagedDisk','ARGAvailabilitySet','ARGResourceContainers','ARGVMSS')" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$availSetTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGAvailabilitySet' }).LogAnalyticsSuffix + "_CL" +$subscriptionsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGResourceContainers' }).LogAnalyticsSuffix + "_CL" +$vmsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGVirtualMachine' }).LogAnalyticsSuffix + "_CL" +$vhdsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGUnmanagedDisk' }).LogAnalyticsSuffix + "_CL" +$vmssTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGVMSS' }).LogAnalyticsSuffix + "_CL" + +Write-Output "Will run query against tables $availSetTableName, $vmsTableName, $vmssTableName, $vhdsTableName and $subscriptionsTableName" + +$Conn.Close() +$Conn.Dispose() + +$recommendationSearchTimeSpan = 1 + +# Grab a context reference to the Storage Account where the recommendations file will be stored + + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +if ($workspaceSubscriptionId -ne $storageAccountSinkSubscriptionId) +{ + Select-AzSubscription -SubscriptionId $workspaceSubscriptionId +} + +$recommendationsErrors = 0 + +Write-Output "Looking for Availability Sets with a low fault domain count..." + +# Execute the recommendation query against Log Analytics + +$baseQuery = @" + $availSetTableName + | where TimeGenerated > ago(1d) and toint(FaultDomains_s) < 3 and toint(FaultDomains_s) < todouble(VmCount_s)/2 + | project TimeGenerated, InstanceId_s, InstanceName_s, ResourceGroupName_s, SubscriptionGuid_g, TenantGuid_g, Cloud_s, Tags_s, FaultDomains_s, VmCount_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId_s + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["FaultDomainCount"] = $result.FaultDomains_s + $additionalInfoDictionary["VMCount"] = $result.VmCount_s + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "HighAvailability" + ImpactedArea = "Microsoft.Compute/virtualMachines" + Impact = "High" + RecommendationType = "BestPractices" + RecommendationSubType = "AvailSetLowFaultDomainCount" + RecommendationSubTypeId = "255de20b-d5e4-4be5-9695-620b4a905774" + RecommendationDescription = "Availability Sets should have a fault domain count of 3 or equal or greater than half of the Virtual Machines count" + RecommendationAction = "Increase the fault domain count of your Availability Set" + InstanceId = $result.InstanceId_s + InstanceName = $result.InstanceName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "availsetsfaultdomaincount-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for Availability Sets with a low update domain count..." + +$baseQuery = @" + $availSetTableName + | where TimeGenerated > ago(1d) and toint(UpdateDomains_s) < todouble(VmCount_s)/2 + | project TimeGenerated, InstanceId_s, InstanceName_s, ResourceGroupName_s, SubscriptionGuid_g, TenantGuid_g, Cloud_s, Tags_s, UpdateDomains_s, VmCount_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId_s + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["UpdateDomainCount"] = $result.UpdateDomains_s + $additionalInfoDictionary["VMCount"] = $result.VmCount_s + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "HighAvailability" + ImpactedArea = "Microsoft.Compute/virtualMachines" + Impact = "High" + RecommendationType = "BestPractices" + RecommendationSubType = "AvailSetLowUpdateDomainCount" + RecommendationSubTypeId = "9764e285-2eca-46c5-b49e-649c039cf0cf" + RecommendationDescription = "Availability Sets should have an update domain count equal or greater than half of the Virtual Machines count" + RecommendationAction = "Increase the update domain count of your Availability Set" + InstanceId = $result.InstanceId_s + InstanceName = $result.InstanceName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "availsetsupdatedomaincount-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for Availability Sets with VMs sharing storage accounts..." + +$baseQuery = @" + $vhdsTableName + | where TimeGenerated > ago(1d) + | extend StorageAccountName = tostring(split(InstanceId_s, '/')[0]) + | distinct TimeGenerated, StorageAccountName, OwnerVMId_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroupName_s + | join kind=inner ( + $vmsTableName + | where TimeGenerated > ago(1d) and isnotempty(AvailabilitySetId_s) + | distinct VMName_s, InstanceId_s, AvailabilitySetId_s, Cloud_s, Tags_s + ) on `$left.OwnerVMId_s == `$right.InstanceId_s + | extend AvailabilitySetName = tostring(split(AvailabilitySetId_s,'/')[8]) + | summarize TimeGenerated = any(TimeGenerated), Tags_s=any(Tags_s), VMCount = count() by AvailabilitySetName, AvailabilitySetId_s, StorageAccountName, ResourceGroupName_s, SubscriptionGuid_g, TenantGuid_g, Cloud_s + | where VMCount > 1 + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics -ErrorAction Continue + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.AvailabilitySetId_s + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["SharedStorageAccountName"] = $result.StorageAccountName + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "HighAvailability" + ImpactedArea = "Microsoft.Compute/virtualMachines" + Impact = "High" + RecommendationType = "BestPractices" + RecommendationSubType = "AvailSetSharedStorageAccount" + RecommendationSubTypeId = "e530029f-9b6a-413a-99ed-81af54502bb9" + RecommendationDescription = "Virtual Machines in unmanaged Availability Sets should not share the same Storage Account" + RecommendationAction = "Migrate Virtual Machines disks to Managed Disks or keep the disks in a dedicated Storage Account per VM" + InstanceId = $result.AvailabilitySetId_s + InstanceName = $result.AvailabilitySetName + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "availsetsharedsa-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for Storage Accounts with multiple VMs..." + +$baseQuery = @" + $vhdsTableName + | where TimeGenerated > ago(1d) + | extend StorageAccountName = tostring(split(InstanceId_s, '/')[0]) + | distinct TimeGenerated, StorageAccountName, OwnerVMId_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroupName_s, Cloud_s + | join kind=inner ( + $vmsTableName + | where TimeGenerated > ago(1d) + | distinct InstanceId_s, Tags_s + ) on `$left.OwnerVMId_s == `$right.InstanceId_s + | summarize TimeGenerated = any(TimeGenerated), Tags_s=any(Tags_s), VMCount = count() by StorageAccountName, SubscriptionGuid_g, TenantGuid_g, ResourceGroupName_s, Cloud_s + | where VMCount > 1 + | extend StorageAccountId = strcat('/subscriptions/', SubscriptionGuid_g, '/resourcegroups/', ResourceGroupName_s, '/providers/microsoft.storage/storageaccounts/', StorageAccountName) + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics -ErrorAction Continue + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.StorageAccountId + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["VirtualMachineCount"] = $result.VMCount + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "HighAvailability" + ImpactedArea = "Microsoft.Compute/virtualMachines" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "StorageAccountsMultipleVMs" + RecommendationSubTypeId = "b70f44fa-5ef9-4180-b2f9-9cc6be07ab3e" + RecommendationDescription = "Virtual Machines with unmanaged disks should not share the same Storage Account" + RecommendationAction = "Migrate Virtual Machines disks to Managed Disks or keep the disks in a dedicated Storage Account per VM" + InstanceId = $result.StorageAccountId + InstanceName = $result.StorageAccountName + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "storageaccountsmultiplevms-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for VMs with no Availability Set..." + +$baseQuery = @" + $vmsTableName + | where TimeGenerated > ago(1d) and isempty(AvailabilitySetId_s) and isempty(Zones_s) and Tags_s !has 'databricks-instance-name' + | project TimeGenerated, VMName_s, InstanceId_s, Tags_s, TenantGuid_g, SubscriptionGuid_g, ResourceGroupName_s, Cloud_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId_s + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "HighAvailability" + ImpactedArea = "Microsoft.Compute/virtualMachines" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "VMsNoAvailSet" + RecommendationSubTypeId = "998b50d8-e654-417b-ab20-a31cb11629c0" + RecommendationDescription = "Virtual Machines should be placed in an Availability Set together with other instances with the same role" + RecommendationAction = "Add VM to an Availability Set together with other VMs of the same role" + InstanceId = $result.InstanceId_s + InstanceName = $result.VMName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "vmsnoavailset-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for VMs alone in an Availability Set..." + +$baseQuery = @" + $vmsTableName + | where TimeGenerated > ago(1d) and isnotempty(AvailabilitySetId_s) and isempty(Zones_s) + | distinct TimeGenerated, VMName_s, InstanceId_s, AvailabilitySetId_s, TenantGuid_g, SubscriptionGuid_g, ResourceGroupName_s, Cloud_s, Tags_s + | summarize any(TimeGenerated, VMName_s, InstanceId_s, Tags_s), VMCount = count() by AvailabilitySetId_s, TenantGuid_g, SubscriptionGuid_g, ResourceGroupName_s, Cloud_s + | where VMCount == 1 + | project TimeGenerated = any_TimeGenerated, VMName_s = any_VMName_s, InstanceId_s = any_InstanceId_s, Tags_s = any_Tags_s, TenantGuid_g, SubscriptionGuid_g, ResourceGroupName_s, Cloud_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId_s + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "HighAvailability" + ImpactedArea = "Microsoft.Compute/virtualMachines" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "VMsSingleInAvailSet" + RecommendationSubTypeId = "fe577af5-dfa2-413a-82a9-f183196c1f49" + RecommendationDescription = "Virtual Machines should not be the only instance in an Availability Set" + RecommendationAction = "Add more VMs of the same role to the Availability Set" + InstanceId = $result.InstanceId_s + InstanceName = $result.VMName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "vmssingleinavailset-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for VMs with disks in multiple Storage Accounts..." + +$baseQuery = @" + $vhdsTableName + | where TimeGenerated > ago(1d) + | extend StorageAccountName = tostring(split(InstanceId_s, '/')[0]) + | distinct TimeGenerated, StorageAccountName, OwnerVMId_s + | summarize TimeGenerated = any(TimeGenerated), StorageAcccountCount = count() by OwnerVMId_s + | where StorageAcccountCount > 1 + | join kind=inner ( + $vmsTableName + | where TimeGenerated > ago(1d) + | distinct VMName_s, InstanceId_s, Cloud_s, TenantGuid_g, SubscriptionGuid_g, ResourceGroupName_s, Tags_s + ) on `$left.OwnerVMId_s == `$right.InstanceId_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics -ErrorAction Continue + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId_s + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["StorageAccountsUsed"] = $result.StorageAcccountCount + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "HighAvailability" + ImpactedArea = "Microsoft.Compute/virtualMachines" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "DisksMultipleStorageAccounts" + RecommendationSubTypeId = "024049e7-f63a-4e1c-b620-f011aafbc576" + RecommendationDescription = "Each Virtual Machine should have its unmanaged disks stored in a single Storage Account for higher availability and manageability" + RecommendationAction = "Migrate Virtual Machines disks to Managed Disks or move VHDs to the same Storage Account" + InstanceId = $result.InstanceId_s + InstanceName = $result.VMName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "disksmultiplesa-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for VMs using unmanaged disks..." + +$baseQuery = @" + $vmsTableName + | where TimeGenerated > ago(1d) and UsesManagedDisks_s == 'false' + | distinct InstanceId_s, VMName_s, ResourceGroupName_s, SubscriptionGuid_g, TenantGuid_g, DeploymentModel_s, Tags_s, Cloud_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId_s + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["DeploymentModel"] = $result.DeploymentModel_s + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "HighAvailability" + ImpactedArea = "Microsoft.Compute/virtualMachines" + Impact = "High" + RecommendationType = "BestPractices" + RecommendationSubType = "UnmanagedDisks" + RecommendationSubTypeId = "b576a069-b1f2-43a6-9134-5ee75376402a" + RecommendationDescription = "Virtual Machines should use Managed Disks for higher availability and manageability" + RecommendationAction = "Migrate Virtual Machines disks to Managed Disks" + InstanceId = $result.InstanceId_s + InstanceName = $result.VMName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "unmanageddisks-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for Resource Groups with VMs not in multiple AZs..." + +$baseQuery = @" + let VMsInZones = materialize($vmsTableName + | where TimeGenerated > ago(1d) and isempty(AvailabilitySetId_s) and isnotempty(Zones_s)); + VMsInZones + | distinct ResourceGroupName_s, Zones_s, SubscriptionGuid_g, TenantGuid_g, Cloud_s + | summarize ZonesCount=count() by ResourceGroupName_s, SubscriptionGuid_g, TenantGuid_g, Cloud_s + | where ZonesCount < 3 + | join kind=inner ( + VMsInZones + | where PowerState_s has 'running' + | distinct VMName_s, ResourceGroupName_s, SubscriptionGuid_g + | summarize VMCount=count() by ResourceGroupName_s, SubscriptionGuid_g + ) on ResourceGroupName_s and SubscriptionGuid_g + | where VMCount == 1 or VMCount > ZonesCount + | project-away SubscriptionGuid_g1, ResourceGroupName_s1 + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g + | extend InstanceId = strcat('/subscriptions/', SubscriptionGuid_g, '/resourcegroups/', ResourceGroupName_s) +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["ZonesCount"] = $result.ZonesCount + $additionalInfoDictionary["VMsCount"] = $result.VMCount + + $fitScore = 4 # a resource group may contain VMs from multiple applications which may lead to false negatives + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "HighAvailability" + ImpactedArea = "Microsoft.Compute/virtualMachines" + Impact = "High" + RecommendationType = "BestPractices" + RecommendationSubType = "VMsMultipleAZs" + RecommendationSubTypeId = "1a77887c-7375-434e-af19-c2543171e0b8" + RecommendationDescription = "Virtual Machines should be placed in multiple Availability Zones" + RecommendationAction = "Distribute Virtual Machines instances of the same role in multiple Availability Zones" + InstanceId = $result.InstanceId + InstanceName = $result.ResourceGroupName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "vmsmultipleazs-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for VMSS not in multiple AZs..." + +$baseQuery = @" + $vmssTableName + | where TimeGenerated > ago(1d) + | where (isempty(Zones_s) and toint(Capacity_s) > 1) or (array_length(split(Zones_s, ' ')) != 3 and toint(Capacity_s) > 2) + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId_s + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["Zones"] = $result.Zones_s + $additionalInfoDictionary["VMSSCapacity"] = $result.Capacity_s + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "HighAvailability" + ImpactedArea = "Microsoft.Compute/virtualMachineScaleSets" + Impact = "High" + RecommendationType = "BestPractices" + RecommendationSubType = "VMSSMultipleAZs" + RecommendationSubTypeId = "47e5457c-b345-4372-b536-8887fa8f0298" + RecommendationDescription = "Virtual Machine Scale Sets should be placed in multiple Availability Zones" + RecommendationAction = "Reprovision the Scale Set leveraging enough Availability Zones" + InstanceId = $result.InstanceId_s + InstanceName = $result.VMSSName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "vmssmultipleazs-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for VMSS using unmanaged disks..." + +$baseQuery = @" + $vmssTableName + | where TimeGenerated > ago(1d) and UsesManagedDisks_s == 'false' + | distinct InstanceId_s, VMSSName_s, ResourceGroupName_s, SubscriptionGuid_g, TenantGuid_g, Tags_s, Cloud_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId_s + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "HighAvailability" + ImpactedArea = "Microsoft.Compute/virtualMachineScaleSets" + Impact = "High" + RecommendationType = "BestPractices" + RecommendationSubType = "UnmanagedDisksVMSS" + RecommendationSubTypeId = "1bf03c4a-c402-4e6c-bf20-051b18af30e2" + RecommendationDescription = "Virtual Machine Scale Sets should use Managed Disks for higher availability and manageability" + RecommendationAction = "Migrate Virtual Machine Scale Sets disks to Managed Disks" + InstanceId = $result.InstanceId_s + InstanceName = $result.VMSSName_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "unmanageddisksvmss-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +if ($recommendationsErrors -gt 0) +{ + throw "Some of the recommendations queries failed. Please, review the job logs for additional information." +} \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VNetOptimizationsToBlobStorage.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VNetOptimizationsToBlobStorage.ps1 new file mode 100644 index 000000000..52ead6cb6 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Recommend-VNetOptimizationsToBlobStorage.ps1 @@ -0,0 +1,1329 @@ +$ErrorActionPreference = "Stop" + +# Collect generic and recommendation-specific variables + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$workspaceId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceId" +$workspaceName = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceName" +$workspaceRG = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceRG" +$workspaceSubscriptionId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceSubId" +$workspaceTenantId = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsWorkspaceTenantId" + +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "recommendationsexports" +} + +$deploymentDate = Get-AutomationVariable -Name "AzureOptimization_DeploymentDate" # yyyy-MM-dd format +$deploymentDate = $deploymentDate.Replace('"', "") + +$lognamePrefix = Get-AutomationVariable -Name "AzureOptimization_LogAnalyticsLogPrefix" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($lognamePrefix)) +{ + $lognamePrefix = "AzureOptimization" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} + +$subnetMaxUsedThresholdVar = Get-AutomationVariable -Name "AzureOptimization_RecommendationVNetSubnetMaxUsedPercentageThreshold" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($subnetMaxUsedThresholdVar) -or $subnetMaxUsedThresholdVar -eq 0) +{ + $subnetMaxUsedThreshold = 80 +} +else +{ + $subnetMaxUsedThreshold = [int] $subnetMaxUsedThresholdVar +} + +$subnetMinUsedThresholdVar = Get-AutomationVariable -Name "AzureOptimization_RecommendationVNetSubnetMinUsedPercentageThreshold" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($subnetMinUsedThresholdVar) -or $subnetMinUsedThresholdVar -eq 0) +{ + $subnetMinUsedThreshold = 5 +} +else +{ + $subnetMinUsedThreshold = [int] $subnetMinUsedThresholdVar +} + +# must be a comma-separated, single-quote enclosed list of subnet names, e.g., 'gatewaysubnet','azurebastionsubnet' +$subnetFreeExclusions = Get-AutomationVariable -Name "AzureOptimization_RecommendationVNetSubnetUsedPercentageExclusions" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($subnetFreeExclusions)) +{ + $subnetFreeExclusions = "'gatewaysubnet'" +} + +$subnetMinAgeVar = Get-AutomationVariable -Name "AzureOptimization_RecommendationVNetSubnetEmptyMinAgeInDays" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($subnetMinAgeVar) -or $subnetMinAgeVar -eq 0) +{ + $subnetMinAge = 30 +} +else +{ + $subnetMinAge = [int] $subnetMinAgeVar +} + +$consumptionOffsetDays = [int] (Get-AutomationVariable -Name "AzureOptimization_ConsumptionOffsetDays") +$consumptionOffsetDaysStart = $consumptionOffsetDays + 1 + +$SqlTimeout = 120 +$LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" + +# Authenticate against Azure + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +Write-Output "Finding tables where recommendations will be generated from..." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE CollectedType IN ('ARGNetworkInterface','ARGVirtualNetwork','ARGResourceContainers', 'ARGNSGRule', 'ARGPublicIP','AzureConsumption')" + + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $controlRows = New-Object System.Data.DataTable + $sqlAdapter.Fill($controlRows) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +$nicsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGNetworkInterface' }).LogAnalyticsSuffix + "_CL" +$vNetsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGVirtualNetwork' }).LogAnalyticsSuffix + "_CL" +$subscriptionsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGResourceContainers' }).LogAnalyticsSuffix + "_CL" +$nsgRulesTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGNSGRule' }).LogAnalyticsSuffix + "_CL" +$publicIpsTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'ARGPublicIP' }).LogAnalyticsSuffix + "_CL" +$consumptionTableName = $lognamePrefix + ($controlRows | Where-Object { $_.CollectedType -eq 'AzureConsumption' }).LogAnalyticsSuffix + "_CL" + +Write-Output "Will run query against tables $nicsTableName, $nsgRulesTableName, $publicIpsTableName, $subscriptionsTableName, $consumptionTableName and $vNetsTableName" + +$Conn.Close() +$Conn.Dispose() + +$recommendationSearchTimeSpan = 30 + $consumptionOffsetDaysStart + +# Grab a context reference to the Storage Account where the recommendations file will be stored + + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +if ($workspaceSubscriptionId -ne $storageAccountSinkSubscriptionId) +{ + Select-AzSubscription -SubscriptionId $workspaceSubscriptionId +} + +$recommendationsErrors = 0 + +Write-Output "Looking for subnets with free IP space less than $subnetMaxUsedThreshold%, excluding $subnetFreeExclusions..." + +$baseQuery = @" + $vNetsTableName + | where TimeGenerated > ago(1d) + | where SubnetName_s !in ($subnetFreeExclusions) + | extend FreeIPs = toint(SubnetTotalPrefixIPs_s) - toint(SubnetUsedIPs_s) + | extend UsedIPPercentage = (todouble(SubnetUsedIPs_s) / todouble(SubnetTotalPrefixIPs_s)) * 100 + | where UsedIPPercentage >= $subnetMaxUsedThreshold + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId_s + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/subnets" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["subnetName"] = $result.SubnetName_s + $additionalInfoDictionary["subnetPrefix"] = $result.SubnetPrefix_s + $additionalInfoDictionary["subnetTotalIPs"] = $result.SubnetTotalPrefixIPs_s + $additionalInfoDictionary["subnetFreeIPs"] = $result.FreeIPs + $additionalInfoDictionary["subnetUsedIPPercentage"] = $result.UsedIPPercentage + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "OperationalExcellence" + ImpactedArea = "Microsoft.Network/virtualNetworks" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "HighSubnetIPSpaceUsage" + RecommendationSubTypeId = "5292525b-5095-4e52-803e-e17192f1d099" + RecommendationDescription = "Subnets with a high IP space usage may constrain operations" + RecommendationAction = "Move network devices to a subnet with a larger address space" + InstanceId = $result.InstanceId_s + InstanceName = "$($result.VNetName_s)/$($result.SubnetName_s)" + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "subnetshighspaceusage-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for subnets with used IP space less than $subnetMinUsedThreshold%..." + +$baseQuery = @" + $vNetsTableName + | where TimeGenerated > ago(1d) + | where SubnetName_s !in ($subnetFreeExclusions) + | extend FreeIPs = toint(SubnetTotalPrefixIPs_s) - toint(SubnetUsedIPs_s) + | extend UsedIPPercentage = (todouble(SubnetUsedIPs_s) / todouble(SubnetTotalPrefixIPs_s)) * 100 + | where UsedIPPercentage > 0 and UsedIPPercentage <= $subnetMinUsedThreshold + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId_s + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/subnets" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["subnetName"] = $result.SubnetName_s + $additionalInfoDictionary["subnetPrefix"] = $result.SubnetPrefix_s + $additionalInfoDictionary["subnetTotalIPs"] = $result.SubnetTotalPrefixIPs_s + $additionalInfoDictionary["subnetUsedIPs_s"] = $result.SubnetUsedIPs_s + $additionalInfoDictionary["subnetUsedIPPercentage"] = $result.UsedIPPercentage + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "OperationalExcellence" + ImpactedArea = "Microsoft.Network/virtualNetworks" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "LowSubnetIPSpaceUsage" + RecommendationSubTypeId = "0f27b41c-869a-4563-86e9-d1c94232ba81" + RecommendationDescription = "Subnets with a low IP space usage are a waste of virtual network address space" + RecommendationAction = "Move network devices to a subnet with a smaller address space" + InstanceId = $result.InstanceId_s + InstanceName = "$($result.VNetName_s)/$($result.SubnetName_s)" + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "subnetslowspaceusage-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for subnets without any device..." + +$baseQuery = @" + $vNetsTableName + | where TimeGenerated > ago(1d) + | where toint(SubnetUsedIPs_s) == 0 and toint(SubnetDelegationsCount_s) == 0 + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId_s + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/subnets" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["subnetName"] = $result.SubnetName_s + $additionalInfoDictionary["subnetPrefix"] = $result.SubnetPrefix_s + $additionalInfoDictionary["subnetTotalIPs"] = $result.SubnetTotalPrefixIPs_s + $additionalInfoDictionary["subnetUsedIPs_s"] = $result.SubnetUsedIPs_s + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "OperationalExcellence" + ImpactedArea = "Microsoft.Network/virtualNetworks" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "NoSubnetIPSpaceUsage" + RecommendationSubTypeId = "343bbfb7-5bec-4711-8353-398454d42b7b" + RecommendationDescription = "Subnets without any IP usage are a waste of virtual network address space" + RecommendationAction = "Delete the subnet to reclaim address space" + InstanceId = $result.InstanceId_s + InstanceName = "$($result.VNetName_s)/$($result.SubnetName_s)" + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "subnetsnospaceusage-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for orphaned NICs..." + +$baseQuery = @" + $nicsTableName + | where TimeGenerated > ago(1d) + | where isempty(OwnerVMId_s) and isempty(OwnerPEId_s) + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.InstanceId_s + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["privateIpAddress"] = $result.PrivateIPAddress_s + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "OperationalExcellence" + ImpactedArea = "Microsoft.Network/networkInterfaces" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "OrphanedNIC" + RecommendationSubTypeId = "4c5c2d0c-b6a4-4c59-bc18-6fff6c1f5b23" + RecommendationDescription = "Orphaned Network Interfaces (without owner VM or PE) unnecessarily consume IP address space" + RecommendationAction = "Delete the NIC to reclaim address space" + InstanceId = $result.InstanceId_s + InstanceName = $result.Name_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "orphanednics-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for NSG rules referring empty or removed subnets..." + +$baseQuery = @" + let MinimumSubnetAge = $($subnetMinAge)d; + let SubnetsToday = materialize( $vNetsTableName + | where TimeGenerated > ago(1d) + | extend SubnetId = tolower(strcat(InstanceId_s, '/subnets/', SubnetName_s)) + | distinct SubnetId, SubnetPrefix_s, SubnetUsedIPs_s, SubnetDelegationsCount_s ); + let SubnetsBefore = materialize( $vNetsTableName + | where TimeGenerated < ago(1d) + | extend SubnetId = tolower(strcat(InstanceId_s, '/subnets/', SubnetName_s)) + | summarize ExistsSince = min(todatetime(StatusDate_s)) by SubnetId, SubnetPrefix_s ); + let SubnetsExistingLongEnoughIds = SubnetsBefore | where ExistsSince < ago(MinimumSubnetAge) | distinct SubnetId; + let EmptySubnets = SubnetsToday | where SubnetId in (SubnetsExistingLongEnoughIds) and toint(SubnetUsedIPs_s) == 0 and toint(SubnetDelegationsCount_s) == 0; + let SubnetsTodayIds = SubnetsToday | distinct SubnetId; + let SubnetsTodayPrefixes = SubnetsToday | distinct SubnetPrefix_s; + let RemovedSubnets = SubnetsBefore | where SubnetId !in (SubnetsTodayIds) and SubnetPrefix_s !in (SubnetsTodayPrefixes); + let NSGRules = materialize($nsgRulesTableName + | where TimeGenerated > ago(1d) + | extend SourceAddresses = split(RuleSourceAddresses_s,',') + | mvexpand SourceAddresses + | extend SourceAddress = tostring(SourceAddresses) + | extend DestinationAddresses = split(RuleDestinationAddresses_s,',') + | mvexpand DestinationAddresses + | extend DestinationAddress = tostring(DestinationAddresses) + | project NSGId = InstanceId_s, RuleName_s, DestinationAddress, SourceAddress, SubscriptionGuid_g, Cloud_s, TenantGuid_g, ResourceGroupName_s, NSGName = NSGName_s, Tags_s); + let EmptySubnetsAsSource = EmptySubnets + | join kind=inner ( NSGRules ) on `$left.SubnetPrefix_s == `$right.SourceAddress + | extend SubnetState = 'empty'; + let EmptySubnetsAsDestination = EmptySubnets + | join kind=inner ( NSGRules ) on `$left.SubnetPrefix_s == `$right.DestinationAddress + | extend SubnetState = 'empty'; + let RemovedSubnetsAsSource = RemovedSubnets + | join kind=inner ( NSGRules ) on `$left.SubnetPrefix_s == `$right.SourceAddress + | extend SubnetState = 'unexisting'; + let RemovedSubnetsAsDestination = RemovedSubnets + | join kind=inner ( NSGRules ) on `$left.SubnetPrefix_s == `$right.DestinationAddress + | extend SubnetState = 'unexisting'; + EmptySubnetsAsSource + | union EmptySubnetsAsDestination + | union RemovedSubnetsAsSource + | union RemovedSubnetsAsDestination + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g + | where isnotempty(SubnetPrefix_s) + | distinct NSGId, NSGName, RuleName_s, SubscriptionGuid_g, SubscriptionName, ResourceGroupName_s, TenantGuid_g, Cloud_s, SubnetId, SubnetPrefix_s, SubnetState, Tags_s +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.NSGId + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["subnetId"] = $result.SubnetId + $additionalInfoDictionary["subnetPrefix"] = $result.SubnetPrefix_s + $additionalInfoDictionary["subnetState"] = $result.SubnetState + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Security" + ImpactedArea = "Microsoft.Network/networkSecurityGroups" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "NSGRuleForEmptyOrUnexistingSubnet" + RecommendationSubTypeId = "b5491cde-f76c-4423-8c4c-89e3558ff2f2" + RecommendationDescription = "NSG rules referring to empty or unexisting subnets" + RecommendationAction = "Update or remove the NSG rule to improve your network security posture" + InstanceId = $result.NSGId + InstanceName = "$($result.NSGName)/$($result.RuleName_s)" + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "nsgrules-emptyunexistingsubnets-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for NSG rules referring orphan or removed NICs..." + +$baseQuery = @" + let NICsToday = materialize( $nicsTableName + | where TimeGenerated > ago(1d) + | extend NICId = tolower(InstanceId_s) + | distinct NICId, PrivateIPAddress_s, PublicIPId_s, OwnerVMId_s, OwnerPEId_s ); + let NICsBefore = $nicsTableName + | where TimeGenerated < ago(1d) + | extend NICId = tolower(InstanceId_s) + | distinct NICId, PrivateIPAddress_s, PublicIPId_s; + let OrphanNICs = NICsToday + | where isempty(OwnerVMId_s) and isempty(OwnerPEId_s) + | extend PublicIPId_s = tolower(PublicIPId_s) + | join kind=leftouter ( + $publicIpsTableName + | where TimeGenerated > ago(1d) + | project PublicIPId_s = tolower(InstanceId_s), PublicIPAddress = IPAddress + ) on PublicIPId_s; + let NICsTodayIds = NICsToday | distinct NICId; + let NICsTodayIPs = NICsToday | distinct PrivateIPAddress_s; + let RemovedNICs = NICsBefore + | where NICId !in (NICsTodayIds) and PrivateIPAddress_s !in (NICsTodayIPs) + | extend PublicIPId_s = tolower(PublicIPId_s) + | join kind=leftouter ( + $publicIpsTableName + | where TimeGenerated < ago(1d) + | project PublicIPId_s = tolower(InstanceId_s), PublicIPAddress = IPAddress + ) on PublicIPId_s; + let NSGRules = materialize($nsgRulesTableName + | where TimeGenerated > ago(1d) + | extend SourceAddresses = split(RuleSourceAddresses_s,',') + | mvexpand SourceAddresses + | extend SourceAddress = replace('/32','',tostring(SourceAddresses)) + | extend DestinationAddresses = split(RuleDestinationAddresses_s,',') + | mvexpand DestinationAddresses + | extend DestinationAddress = replace('/32','',tostring(DestinationAddresses)) + | project NSGId = InstanceId_s, RuleName_s, DestinationAddress, SourceAddress, SubscriptionGuid_g, Cloud_s, TenantGuid_g, ResourceGroupName_s, NSGName = NSGName_s, Tags_s); + let OrphanNICsAsPrivateSource = OrphanNICs + | join kind=inner ( NSGRules ) on `$left.PrivateIPAddress_s == `$right.SourceAddress + | extend NICState = 'orphan', IPAddress = PrivateIPAddress_s; + let OrphanNICsAsPublicSource = OrphanNICs + | join kind=inner ( NSGRules ) on `$left.PublicIPAddress == `$right.SourceAddress + | extend NICState = 'orphan', IPAddress = PublicIPAddress; + let OrphanNICsAsPrivateDestination = OrphanNICs + | join kind=inner ( NSGRules ) on `$left.PrivateIPAddress_s == `$right.DestinationAddress + | extend NICState = 'orphan', IPAddress = PrivateIPAddress_s; + let OrphanNICsAsPublicDestination = OrphanNICs + | join kind=inner ( NSGRules ) on `$left.PublicIPAddress == `$right.DestinationAddress + | extend NICState = 'orphan', IPAddress = PublicIPAddress; + let RemovedNICsAsPrivateSource = RemovedNICs + | join kind=inner ( NSGRules ) on `$left.PrivateIPAddress_s == `$right.SourceAddress + | extend NICState = 'unexisting', IPAddress = PrivateIPAddress_s; + let RemovedNICsAsPublicSource = RemovedNICs + | join kind=inner ( NSGRules ) on `$left.PublicIPAddress == `$right.SourceAddress + | extend NICState = 'unexisting', IPAddress = PublicIPAddress; + let RemovedNICsAsPrivateDestination = RemovedNICs + | join kind=inner ( NSGRules ) on `$left.PrivateIPAddress_s == `$right.DestinationAddress + | extend NICState = 'unexisting', IPAddress = PrivateIPAddress_s; + let RemovedNICsAsPublicDestination = RemovedNICs + | join kind=inner ( NSGRules ) on `$left.PublicIPAddress == `$right.DestinationAddress + | extend NICState = 'unexisting', IPAddress = PublicIPAddress; + OrphanNICsAsPrivateSource + | union OrphanNICsAsPublicSource + | union OrphanNICsAsPrivateDestination + | union OrphanNICsAsPublicDestination + | union RemovedNICsAsPrivateSource + | union RemovedNICsAsPublicSource + | union RemovedNICsAsPrivateDestination + | union RemovedNICsAsPublicDestination + | where isnotempty(IPAddress) + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g + | distinct NSGId, NSGName, RuleName_s, SubscriptionGuid_g, SubscriptionName, ResourceGroupName_s, TenantGuid_g, Cloud_s, NICId, IPAddress, NICState, Tags_s +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.NSGId + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["nicId"] = $result.NICId + $additionalInfoDictionary["ipAddress"] = $result.IPAddress + $additionalInfoDictionary["nicState"] = $result.NICState + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Security" + ImpactedArea = "Microsoft.Network/networkSecurityGroups" + Impact = "Medium" + RecommendationType = "BestPractices" + RecommendationSubType = "NSGRuleForOrphanOrUnexistingNIC" + RecommendationSubTypeId = "3dc1d1f8-19ef-4572-9c9d-78d62831f55a" + RecommendationDescription = "NSG rules referring to orphan or unexisting NICs" + RecommendationAction = "Update or remove the NSG rule to improve your network security posture" + InstanceId = $result.NSGId + InstanceName = "$($result.NSGName)/$($result.RuleName_s)" + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "nsgrules-orphanunexistingnics-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for NSG rules referring orphan or removed Public IPs..." + +$baseQuery = @" + let PIPsToday = materialize( $publicIpsTableName + | where TimeGenerated > ago(1d) + | extend PublicIPId = tolower(InstanceId_s) + | distinct PublicIPId, AssociatedResourceId_s, AllocationMethod_s, IPAddress ); + let PIPsBefore = materialize( $publicIpsTableName + | where TimeGenerated < ago(1d) + | extend PublicIPId = tolower(InstanceId_s) + | distinct PublicIPId, IPAddress ); + let OrphanStaticPIPs = PIPsToday + | where isempty(AssociatedResourceId_s) and AllocationMethod_s == 'static'; + let OrphanDynamicPIPIDs = PIPsToday + | where isempty(AssociatedResourceId_s) and AllocationMethod_s == 'dynamic' + | distinct PublicIPId; + let PIPsTodayIds = PIPsToday | distinct PublicIPId; + let PIPsTodayIPs = PIPsToday | distinct IPAddress; + let OrphanDynamicPIPs = PIPsBefore + | where PublicIPId in (OrphanDynamicPIPIDs) and isnotempty(IPAddress) and IPAddress !in (PIPsTodayIPs); + let RemovedPIPs = PIPsBefore + | where PublicIPId !in (PIPsTodayIds) and isnotempty(IPAddress) and IPAddress !in (PIPsTodayIPs); + let NSGRules = materialize( $nsgRulesTableName + | where TimeGenerated > ago(1d) + | extend SourceAddresses = split(RuleSourceAddresses_s,',') + | mvexpand SourceAddresses + | extend SourceAddress = replace('/32','',tostring(SourceAddresses)) + | extend DestinationAddresses = split(RuleDestinationAddresses_s,',') + | mvexpand DestinationAddresses + | extend DestinationAddress = replace('/32','',tostring(DestinationAddresses)) + | project NSGId = InstanceId_s, RuleName_s, DestinationAddress, SourceAddress, SubscriptionGuid_g, Cloud_s, TenantGuid_g, ResourceGroupName_s, NSGName = NSGName_s, Tags_s); + let OrphanStaticPIPsAsSource = OrphanStaticPIPs + | join kind=inner ( NSGRules ) on `$left.IPAddress == `$right.SourceAddress + | extend PIPState = 'orphan'; + let OrphanStaticPIPsAsDestination = OrphanStaticPIPs + | join kind=inner ( NSGRules ) on `$left.IPAddress == `$right.DestinationAddress + | extend PIPState = 'orphan'; + let OrphanDynamicPIPsAsSource = OrphanDynamicPIPs + | join kind=inner ( NSGRules ) on `$left.IPAddress == `$right.SourceAddress + | extend PIPState = 'orphan'; + let OrphanDynamicPIPsAsDestination = OrphanDynamicPIPs + | join kind=inner ( NSGRules ) on `$left.IPAddress == `$right.DestinationAddress + | extend PIPState = 'orphan'; + let RemovedPIPsAsSource = RemovedPIPs + | join kind=inner ( NSGRules ) on `$left.IPAddress == `$right.SourceAddress + | extend PIPState = 'unexisting'; + let RemovedPIPsAsDestination = RemovedPIPs + | join kind=inner ( NSGRules ) on `$left.IPAddress == `$right.DestinationAddress + | extend PIPState = 'unexisting'; + OrphanStaticPIPsAsSource + | union OrphanDynamicPIPsAsSource + | union OrphanStaticPIPsAsDestination + | union OrphanDynamicPIPsAsDestination + | union RemovedPIPsAsSource + | union RemovedPIPsAsDestination + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g + | distinct NSGId, NSGName, RuleName_s, SubscriptionGuid_g, SubscriptionName, ResourceGroupName_s, TenantGuid_g, Cloud_s, PublicIPId, IPAddress, PIPState, AllocationMethod_s, Tags_s +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + switch ($result.Cloud_s) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + + $queryInstanceId = $result.NSGId + $detailsURL = "https://portal.azure.$azureTld/#@$($result.TenantGuid_g)/resource/$queryInstanceId/overview" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["publicIPId"] = $result.PublicIPId + $additionalInfoDictionary["ipAddress"] = $result.IPAddress + $additionalInfoDictionary["publicIPState"] = $result.PIPState + $additionalInfoDictionary["allocationMethod"] = $result.AllocationMethod_s + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Security" + ImpactedArea = "Microsoft.Network/networkSecurityGroups" + Impact = "High" + RecommendationType = "BestPractices" + RecommendationSubType = "NSGRuleForOrphanOrUnexistingPublicIP" + RecommendationSubTypeId = "fe40cbe7-bdee-4cce-b072-cf25e1247b7a" + RecommendationDescription = "NSG rules referring to orphan or unexisting Public IPs" + RecommendationAction = "Update or remove the NSG rule to improve your network security posture" + InstanceId = $result.NSGId + InstanceName = "$($result.NSGName)/$($result.RuleName_s)" + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "nsgrules-orphanunexistingpublicips-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +Write-Output "Looking for orphaned Public IPs..." + +$baseQuery = @" + let interval = 30d; + let etime = todatetime(toscalar($consumptionTableName | where todatetime(Date_s) < now() and todatetime(Date_s) > ago(30d) | summarize max(todatetime(Date_s)))); + let stime = etime-interval; + $publicIpsTableName + | where TimeGenerated > ago(1d) and isempty(AssociatedResourceId_s) + | distinct Name_s, InstanceId_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroupName_s, SkuName_s, AllocationMethod_s, Tags_s, Cloud_s + | join kind=leftouter ( + $consumptionTableName + | where todatetime(Date_s) between (stime..etime) + | project InstanceId_s=tolower(ResourceId), CostInBillingCurrency_s, Date_s + ) on InstanceId_s + | summarize Last30DaysCost=sum(todouble(CostInBillingCurrency_s)) by Name_s, InstanceId_s, SubscriptionGuid_g, TenantGuid_g, ResourceGroupName_s, SkuName_s, AllocationMethod_s, Tags_s, Cloud_s + | join kind=leftouter ( + $subscriptionsTableName + | where TimeGenerated > ago(1d) + | where ContainerType_s =~ 'microsoft.resources/subscriptions' + | project SubscriptionGuid_g, SubscriptionName = ContainerName_s + ) on SubscriptionGuid_g +"@ + +try +{ + $queryResults = Invoke-AzOperationalInsightsQuery -WorkspaceId $workspaceId -Query $baseQuery -Timespan (New-TimeSpan -Days $recommendationSearchTimeSpan) -Wait 600 -IncludeStatistics + if ($queryResults) + { + $results = [System.Linq.Enumerable]::ToArray($queryResults.Results) + } +} +catch +{ + Write-Warning -Message "Query failed. Debug the following query in the AOE Log Analytics workspace: $baseQuery" + Write-Warning -Message $error[0] + $recommendationsErrors++ +} + +Write-Output "Query finished with $($results.Count) results." + +Write-Output "Query statistics: $($queryResults.Statistics.query)" + +# Build the recommendations objects + +$recommendations = @() +$datetime = (get-date).ToUniversalTime() +$timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") + +foreach ($result in $results) +{ + $queryInstanceId = $result.InstanceId_s + $queryText = @" + $publicIpsTableName + | where InstanceId_s == '$queryInstanceId' and isempty(AssociatedResourceId_s) + | distinct InstanceId_s, Name_s, AllocationMethod_s, SkuName_s, TimeGenerated + | summarize LastAttachedDate = min(TimeGenerated) by InstanceId_s, Name_s, AllocationMethod_s, SkuName_s + | join kind=leftouter ( + $consumptionTableName + | project InstanceId_s=tolower(ResourceId), CostInBillingCurrency_s, Date_s + ) on InstanceId_s + | summarize CostsSinceDetached = sumif(todouble(CostInBillingCurrency_s), todatetime(Date_s) > LastAttachedDate) by Name_s, LastAttachedDate, AllocationMethod_s, SkuName_s +"@ + $encodedQuery = [System.Uri]::EscapeDataString($queryText) + $detailsQueryStart = $deploymentDate + $detailsQueryEnd = $datetime.AddDays(8).ToString("yyyy-MM-dd") + switch ($cloudEnvironment) + { + "AzureCloud" { $azureTld = "com" } + "AzureChinaCloud" { $azureTld = "cn" } + "AzureUSGovernment" { $azureTld = "us" } + default { $azureTld = "com" } + } + $detailsURL = "https://portal.azure.$azureTld#@$workspaceTenantId/blade/Microsoft_Azure_Monitoring_Logs/LogsBlade/resourceId/%2Fsubscriptions%2F$workspaceSubscriptionId%2Fresourcegroups%2F$workspaceRG%2Fproviders%2Fmicrosoft.operationalinsights%2Fworkspaces%2F$workspaceName/source/LogsBlade.AnalyticsShareLinkToQuery/query/$encodedQuery/timespan/$($detailsQueryStart)T00%3A00%3A00.000Z%2F$($detailsQueryEnd)T00%3A00%3A00.000Z" + + $additionalInfoDictionary = @{} + + $additionalInfoDictionary["currentSku"] = $result.SkuName_s + $additionalInfoDictionary["allocationMethod"] = $result.AllocationMethod_s + $additionalInfoDictionary["CostsAmount"] = [double] $result.Last30DaysCost + $additionalInfoDictionary["savingsAmount"] = [double] $result.Last30DaysCost + + $fitScore = 5 + + $tags = @{} + + if (-not([string]::IsNullOrEmpty($result.Tags_s))) + { + $tagPairs = $result.Tags_s.Substring(2, $result.Tags_s.Length - 3).Split(';') + foreach ($tagPairString in $tagPairs) + { + $tagPair = $tagPairString.Split('=') + if (-not([string]::IsNullOrEmpty($tagPair[0])) -and -not([string]::IsNullOrEmpty($tagPair[1]))) + { + $tagName = $tagPair[0].Trim() + $tagValue = $tagPair[1].Trim() + $tags[$tagName] = $tagValue + } + } + } + + $recommendation = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $result.Cloud_s + Category = "Cost" + ImpactedArea = "Microsoft.Network/publicIPAddresses" + Impact = "Low" + RecommendationType = "Saving" + RecommendationSubType = "OrphanedPublicIP" + RecommendationSubTypeId = "3125883f-8b9f-4bde-a0ff-6c739858c6e1" + RecommendationDescription = "Orphaned Public IP (without owner resource) incur in unnecessary costs" + RecommendationAction = "Delete the Public IP or change its configuration to dynamic allocation" + InstanceId = $result.InstanceId_s + InstanceName = $result.Name_s + AdditionalInfo = $additionalInfoDictionary + ResourceGroup = $result.ResourceGroupName_s + SubscriptionGuid = $result.SubscriptionGuid_g + SubscriptionName = $result.SubscriptionName + TenantGuid = $result.TenantGuid_g + FitScore = $fitScore + Tags = $tags + DetailsURL = $detailsURL + } + + $recommendations += $recommendation +} + +# Export the recommendations as JSON to blob storage + +$fileDate = $datetime.ToString("yyyy-MM-dd") +$jsonExportPath = "orphanedpublicips-$fileDate.json" +$recommendations | ConvertTo-Json | Out-File $jsonExportPath + +$jsonBlobName = $jsonExportPath +$jsonProperties = @{"ContentType" = "application/json"}; +Set-AzStorageBlobContent -File $jsonExportPath -Container $storageAccountSinkContainer -Properties $jsonProperties -Blob $jsonBlobName -Context $saCtx -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Uploaded $jsonBlobName to Blob Storage..." + +Remove-Item -Path $jsonExportPath -Force + +$now = (Get-Date).ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +Write-Output "[$now] Removed $jsonExportPath from local disk..." + +if ($recommendationsErrors -gt 0) +{ + throw "Some of the recommendations queries failed. Please, review the job logs for additional information." +} \ No newline at end of file diff --git a/docs/deploy/optimization-engine/13.0/runbooks/remediations/Remediate-AdvisorRightSizeFiltered.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/remediations/Remediate-AdvisorRightSizeFiltered.ps1 new file mode 100644 index 000000000..314dc522c --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/remediations/Remediate-AdvisorRightSizeFiltered.ps1 @@ -0,0 +1,225 @@ +param( + [Parameter(Mandatory = $false)] + [bool] $Simulate = $true +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RemediationLogsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "remediationlogs" +} + +$minFitScore = [double] (Get-AutomationVariable -Name "AzureOptimization_RemediateRightSizeMinFitScore" -ErrorAction SilentlyContinue) +if (-not($minFitScore -gt 0.0)) { + $minFitScore = 5.0 +} + +$minWeeksInARow = [int] (Get-AutomationVariable -Name "AzureOptimization_RemediateRightSizeMinWeeksInARow" -ErrorAction SilentlyContinue) +if (-not($minWeeksInARow -gt 0)) { + $minWeeksInARow = 4 +} + +$tagsFilter = Get-AutomationVariable -Name "AzureOptimization_RemediateRightSizeTagsFilter" -ErrorAction SilentlyContinue +# example: '[ { "tagName": "a", "tagValue": "b" }, { "tagName": "c", "tagValue": "d" } ]' +if (-not($tagsFilter)) { + $tagsFilter = '{}' +} +$tagsFilter = $tagsFilter | ConvertFrom-Json + +$rightSizeRecommendationId = Get-AutomationVariable -Name "AzureOptimization_RecommendationAdvisorCostRightSizeId" -ErrorAction SilentlyContinue +if (-not($rightSizeRecommendationId)) { + $rightSizeRecommendationId = 'e10b1381-5f0a-47ff-8c7b-37bd13d7c974' +} + +$SqlTimeout = 0 +$recommendationsTable = "Recommendations" + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +# get reference to storage sink + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +Write-Output "Querying for right-size recommendations with fit score >= $minFitScore made consecutively for the last $minWeeksInARow weeks." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = @" + SELECT InstanceId, Cloud, TenantGuid, JSON_VALUE(AdditionalInfo, '`$.currentSku') AS CurrentSKU, JSON_VALUE(AdditionalInfo, '`$.targetSku') AS TargetSKU, COUNT(InstanceId) + FROM [dbo].[$recommendationsTable] + WHERE RecommendationSubTypeId = '$rightSizeRecommendationId' AND FitScore >= $minFitScore AND GeneratedDate >= GETDATE()-(7*$minWeeksInARow) + GROUP BY InstanceId, Cloud, TenantGuid, JSON_VALUE(AdditionalInfo, '`$.currentSku'), JSON_VALUE(AdditionalInfo, '`$.targetSku') + HAVING COUNT(InstanceId) >= $minWeeksInARow +"@ + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $vmsToRightSize = New-Object System.Data.DataTable + $sqlAdapter.Fill($vmsToRightSize) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +Write-Output "Found $($vmsToRightSize.Rows.Count) remediation opportunities." + +$Conn.Close() +$Conn.Dispose() + +$logEntries = @() + +$datetime = (get-date).ToUniversalTime() +$hour = $datetime.Hour +$min = $datetime.Minute +$timestamp = $datetime.ToString("yyyy-MM-ddT$($hour):$($min):00.000Z") + +$ctx = Get-AzContext + +foreach ($vm in $vmsToRightSize.Rows) +{ + $isEligible = $false + $logDetails = $null + if ([string]::IsNullOrEmpty($tagsFilter)) + { + $isEligible = $true + } + else + { + $vmTags = Get-AzTag -ResourceId $vm.InstanceId -ErrorAction SilentlyContinue + if ($vmTags) + { + foreach ($tagFilter in $tagsFilter) + { + if ($vmTags.Properties.TagsProperty.($tagFilter.tagName) -eq $tagFilter.tagValue) + { + $isEligible = $true + } + else + { + $isEligible = $false + break + } + } + } + } + + $subscriptionId = $vm.InstanceId.Split("/")[2] + $resourceGroup = $vm.InstanceId.Split("/")[4] + $instanceName = $vm.InstanceId.Split("/")[8] + + if ($isEligible) + { + Write-Output "Downsizing (SIMULATE=$Simulate) $($vm.InstanceId) to $($vm.TargetSKU)..." + if (-not($Simulate) -and $ctx.Environment.Name -eq $vm.Cloud -and $ctx.Tenant.Id -eq $vm.TenantGuid) + { + if ($ctx.Subscription.Id -ne $subscriptionId) + { + Select-AzSubscription -SubscriptionId $subscriptionId | Out-Null + $ctx = Get-AzContext + } + $vmObj = Get-AzVM -ResourceGroupName $resourceGroup -VMName $instanceName -ErrorAction SilentlyContinue + if ($vmObj) + { + $vmObj.HardwareProfile.VmSize = $vm.TargetSKU + Update-AzVM -VM $vmObj -ResourceGroupName $resourceGroup + } + else + { + Write-Output "Skipping as VM was already removed." + } + } + else + { + Write-Output "Did not apply remediation." + } + } + + $logDetails = @{ + IsEligible = $isEligible + CurrentSku = $vm.CurrentSKU + TargetSku = $vm.TargetSKU + } + + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $vm.Cloud + TenantGuid = $vm.TenantGuid + SubscriptionGuid = $subscriptionId + ResourceGroupName = $resourceGroup.ToLower() + InstanceName = $instanceName.ToLower() + InstanceId = $vm.InstanceId.ToLower() + Simulate = $Simulate + LogDetails = $logDetails | ConvertTo-Json -Compress + RecommendationSubTypeId = $rightSizeRecommendationId + } + + $logEntries += $logentry +} + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-rightsizefiltered.csv" + +$logEntries | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force diff --git a/docs/deploy/optimization-engine/13.0/runbooks/remediations/Remediate-LongDeallocatedVMsFiltered.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/remediations/Remediate-LongDeallocatedVMsFiltered.ps1 new file mode 100644 index 000000000..978ecb9b2 --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/remediations/Remediate-LongDeallocatedVMsFiltered.ps1 @@ -0,0 +1,306 @@ +param( + [Parameter(Mandatory = $false)] + [bool] $Simulate = $true +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RemediationLogsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "remediationlogs" +} + +$minFitScore = [double] (Get-AutomationVariable -Name "AzureOptimization_RemediateLongDeallocatedVMsMinFitScore" -ErrorAction SilentlyContinue) +if (-not($minFitScore -gt 0.0)) { + $minFitScore = 5.0 +} + +$minWeeksInARow = [int] (Get-AutomationVariable -Name "AzureOptimization_RemediateLongDeallocatedVMsMinWeeksInARow" -ErrorAction SilentlyContinue) +if (-not($minWeeksInARow -gt 0)) { + $minWeeksInARow = 4 +} + +$tagsFilter = Get-AutomationVariable -Name "AzureOptimization_RemediateLongDeallocatedVMsTagsFilter" -ErrorAction SilentlyContinue +# example: '[ { "tagName": "a", "tagValue": "b" }, { "tagName": "c", "tagValue": "d" } ]' +if (-not($tagsFilter)) { + $tagsFilter = '{}' +} +$tagsFilter = $tagsFilter | ConvertFrom-Json + +$recommendationId = Get-AutomationVariable -Name "AzureOptimization_RecommendationLongDeallocatedVMsId" -ErrorAction SilentlyContinue +if (-not($recommendationId)) { + $recommendationId = 'c320b790-2e58-452a-aa63-7b62c383ad8a' +} + +$SqlTimeout = 0 +$recommendationsTable = "Recommendations" + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +# get reference to storage sink + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +Write-Output "Querying for long-deallocated recommendations with fit score >= $minFitScore made consecutively for the last $minWeeksInARow weeks." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = @" + SELECT InstanceId, Cloud, TenantGuid, COUNT(InstanceId) + FROM [dbo].[$recommendationsTable] + WHERE RecommendationSubTypeId = '$recommendationId' AND FitScore >= $minFitScore AND GeneratedDate >= GETDATE()-(7*$minWeeksInARow) + GROUP BY InstanceId, Cloud, TenantGuid + HAVING COUNT(InstanceId) >= $minWeeksInARow +"@ + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $deallocatedVMs = New-Object System.Data.DataTable + $sqlAdapter.Fill($deallocatedVMs) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +Write-Output "Found $($deallocatedVMs.Rows.Count) remediation opportunities." + +$Conn.Close() +$Conn.Dispose() + +$logEntries = @() + +$datetime = (get-date).ToUniversalTime() +$hour = $datetime.Hour +$min = $datetime.Minute +$timestamp = $datetime.ToString("yyyy-MM-ddT$($hour):$($min):00.000Z") + +$ctx = Get-AzContext + +foreach ($vm in $deallocatedVMs.Rows) +{ + $isEligible = $false + $logDetails = $null + if ([string]::IsNullOrEmpty($tagsFilter)) + { + $isEligible = $true + } + else + { + $vmTags = Get-AzTag -ResourceId $vm.InstanceId -ErrorAction SilentlyContinue + if ($vmTags) + { + foreach ($tagFilter in $tagsFilter) + { + if ($vmTags.Properties.TagsProperty.($tagFilter.tagName) -eq $tagFilter.tagValue) + { + $isEligible = $true + } + else + { + $isEligible = $false + break + } + } + } + } + + $subscriptionId = $vm.InstanceId.Split("/")[2] + $resourceGroup = $vm.InstanceId.Split("/")[4] + $instanceName = $vm.InstanceId.Split("/")[8] + + if ($isEligible) + { + $vmState = "Unknown" + $hasManagedDisks = $false + $osDiskSkuName = "Unknown" + $dataDisksSkuNames = "Unknown" + + Write-Output "Downsizing (SIMULATE=$Simulate) $($vm.InstanceId) disks to Standard_LRS..." + if ($ctx.Environment.Name -eq $vm.Cloud -and $ctx.Tenant.Id -eq $vm.TenantGuid) + { + if ($ctx.Subscription.Id -ne $subscriptionId) + { + Select-AzSubscription -SubscriptionId $subscriptionId | Out-Null + $ctx = Get-AzContext + } + $vmObj = Get-AzVM -ResourceGroupName $resourceGroup -VMName $instanceName -Status -ErrorAction SilentlyContinue + if (($vmObj.Statuses | Where-Object { $_.Code -like "PowerState/*" }).Code -eq "PowerState/deallocated") + { + $vmState = "Deallocated" + $vmObj = Get-AzVM -ResourceGroupName $resourceGroup -VMName $instanceName + if ($vmObj.StorageProfile.OsDisk.ManagedDisk.Id) + { + $hasManagedDisks = $true + $disks = @($vmObj.StorageProfile.OsDisk.ManagedDisk.Id) + if ($vmObj.StorageProfile.DataDisks.ManagedDisk.Id) + { + $disks = $disks + $vmObj.StorageProfile.DataDisks.ManagedDisk.Id + } + foreach ($disk in $disks) + { + $diskObj = Get-AzDisk -ResourceGroupName $disk.Split("/")[4] -DiskName $disk.Split("/")[8] + if ($diskObj.OsType) + { + $osDiskSkuName = $diskObj.Sku.Name + } + else + { + if ($dataDisksSkuNames -eq 'Unknown') + { + $dataDisksSkuNames = $diskObj.Sku.Name + } + else + { + if ($dataDisksSkuNames -notlike "*$($diskObj.Sku.Name)*") + { + $dataDisksSkuNames += ",$($diskObj.Sku.Name)" + } + } + } + if ($diskObj.Sku.Name -notin ('Standard_LRS','StandardSSD_ZRS')) + { + if ($diskObj.Sku.Name -like "*_LRS" -and $diskObj.Sku.Name -notlike "*V2*") + { + Write-Output "Downgrading $($diskObj.Name) to Standard_LRS..." + if (-not($Simulate)) + { + $diskObj.Sku = [Microsoft.Azure.Management.Compute.Models.DiskSku]::new('Standard_LRS', 'Standard') + $diskObj | Update-AzDisk | Out-Null + } + } + elseif ($diskObj.Sku.Name -like "*_ZRS" -and $diskObj.Sku.Name -notlike "*V2*") + { + Write-Output "Downgrading $($diskObj.Name) to StandardSSD_ZRS..." + if (-not($Simulate)) + { + $diskObj.Sku = [Microsoft.Azure.Management.Compute.Models.DiskSku]::new('StandardSSD_ZRS', 'Standard') + $diskObj | Update-AzDisk | Out-Null + } + } + else + { + Write-Output "Skipping as $($diskObj.Name) disk is in an unsupported SKU ($($diskObj.Sku.Name))..." + } + } + else + { + Write-Output "Skipping as $($diskObj.Name) disk is already in the lowest SKU ($($diskObj.Sku.Name))." + } + } + } + else + { + Write-Output "Skipping as disks are not Managed Disks." + $hasManagedDisks = $false + } + } + else + { + if ($vmObj) + { + Write-Output "Skipping as VM is not deallocated." + $vmState = "Running" + } + else + { + Write-Output "Skipping as VM was already removed." + $vmState = "Removed" + } + } + } + else + { + Write-Output "Could not apply remediation as VM is in another cloud/tenant." + } + } + + $logDetails = @{ + IsEligible = $isEligible + VMState = $vmState + HasManagedDisks = $hasManagedDisks + OsDiskSkuName = $osDiskSkuName + DataDisksSkuName = $dataDisksSkuNames + } + + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $vm.Cloud + TenantGuid = $vm.TenantGuid + SubscriptionGuid = $subscriptionId + ResourceGroupName = $resourceGroup.ToLower() + InstanceName = $instanceName.ToLower() + InstanceId = $vm.InstanceId.ToLower() + Simulate = $Simulate + LogDetails = $logDetails | ConvertTo-Json -Compress + RecommendationSubTypeId = $recommendationId + } + + $logEntries += $logentry +} + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-longdeallocatedvmsfiltered.csv" + +$logEntries | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force diff --git a/docs/deploy/optimization-engine/13.0/runbooks/remediations/Remediate-UnattachedDisksFiltered.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/remediations/Remediate-UnattachedDisksFiltered.ps1 new file mode 100644 index 000000000..b048a815b --- /dev/null +++ b/docs/deploy/optimization-engine/13.0/runbooks/remediations/Remediate-UnattachedDisksFiltered.ps1 @@ -0,0 +1,286 @@ +param( + [Parameter(Mandatory = $false)] + [bool] $Simulate = $true +) + +$ErrorActionPreference = "Stop" + +$cloudEnvironment = Get-AutomationVariable -Name "AzureOptimization_CloudEnvironment" -ErrorAction SilentlyContinue # AzureCloud|AzureChinaCloud +if ([string]::IsNullOrEmpty($cloudEnvironment)) +{ + $cloudEnvironment = "AzureCloud" +} +$authenticationOption = Get-AutomationVariable -Name "AzureOptimization_AuthenticationOption" -ErrorAction SilentlyContinue # ManagedIdentity|UserAssignedManagedIdentity +if ([string]::IsNullOrEmpty($authenticationOption)) +{ + $authenticationOption = "ManagedIdentity" +} +if ($authenticationOption -eq "UserAssignedManagedIdentity") +{ + $uamiClientID = Get-AutomationVariable -Name "AzureOptimization_UAMIClientID" +} + +$sqlserver = Get-AutomationVariable -Name "AzureOptimization_SQLServerHostname" +$sqldatabase = Get-AutomationVariable -Name "AzureOptimization_SQLServerDatabase" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($sqldatabase)) +{ + $sqldatabase = "azureoptimization" +} +$storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSink" + + +$storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RemediationLogsContainer" -ErrorAction SilentlyContinue +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { + $storageAccountSinkContainer = "remediationlogs" +} + +$minFitScore = [double] (Get-AutomationVariable -Name "AzureOptimization_RemediateUnattachedDisksMinFitScore" -ErrorAction SilentlyContinue) +if (-not($minFitScore -gt 0.0)) { + $minFitScore = 5.0 +} + +$minWeeksInARow = [int] (Get-AutomationVariable -Name "AzureOptimization_RemediateUnattachedDisksMinWeeksInARow" -ErrorAction SilentlyContinue) +if (-not($minWeeksInARow -gt 0)) { + $minWeeksInARow = 4 +} + +$tagsFilter = Get-AutomationVariable -Name "AzureOptimization_RemediateUnattachedDisksTagsFilter" -ErrorAction SilentlyContinue +# example: '[ { "tagName": "a", "tagValue": "b" }, { "tagName": "c", "tagValue": "d" } ]' +if (-not($tagsFilter)) { + $tagsFilter = '{}' +} +$tagsFilter = $tagsFilter | ConvertFrom-Json + +$remediationAction = Get-AutomationVariable -Name "AzureOptimization_RemediateUnattachedDisksAction" -ErrorAction SilentlyContinue # Delete / Downsize +if (-not($remediationAction)) { + $remediationAction = "Delete" +} + +$recommendationId = Get-AutomationVariable -Name "AzureOptimization_RecommendationUnattachedDisksId" -ErrorAction SilentlyContinue +if (-not($recommendationId)) { + $recommendationId = 'c84d5e86-e2d6-4d62-be7c-cecfbd73b0db' +} + +$SqlTimeout = 0 +$recommendationsTable = "Recommendations" + +"Logging in to Azure with $authenticationOption..." + +switch ($authenticationOption) { + "UserAssignedManagedIdentity" { + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID + break + } + Default { #ManagedIdentity + Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment + break + } +} + +$cloudDetails = Get-AzEnvironment -Name $CloudEnvironment +$azureSqlDomain = $cloudDetails.SqlDatabaseDnsSuffix.Substring(1) + +# get reference to storage sink + +$saCtx = New-AzStorageContext -StorageAccountName $storageAccountSink -UseConnectedAccount -Environment $cloudEnvironment + +Write-Output "Querying for unattached disks recommendations with fit score >= $minFitScore made consecutively for the last $minWeeksInARow weeks." + +$tries = 0 +$connectionSuccess = $false +do { + $tries++ + try { + $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" + $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") + $Conn.AccessToken = $dbToken.Token + $Conn.Open() + $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd.Connection = $Conn + $Cmd.CommandTimeout = $SqlTimeout + $Cmd.CommandText = @" + SELECT InstanceId, Cloud, TenantGuid, COUNT(InstanceId) + FROM [dbo].[$recommendationsTable] + WHERE RecommendationSubTypeId = '$recommendationId' AND FitScore >= $minFitScore AND GeneratedDate >= GETDATE()-(7*$minWeeksInARow) + GROUP BY InstanceId, Cloud, TenantGuid + HAVING COUNT(InstanceId) >= $minWeeksInARow +"@ + $sqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter + $sqlAdapter.SelectCommand = $Cmd + $unattachedDisks = New-Object System.Data.DataTable + $sqlAdapter.Fill($unattachedDisks) | Out-Null + $connectionSuccess = $true + } + catch { + Write-Output "Failed to contact SQL at try $tries." + Write-Output $Error[0] + Start-Sleep -Seconds ($tries * 20) + } +} while (-not($connectionSuccess) -and $tries -lt 3) + +if (-not($connectionSuccess)) +{ + throw "Could not establish connection to SQL." +} + +Write-Output "Found $($unattachedDisks.Rows.Count) remediation opportunities." + +$Conn.Close() +$Conn.Dispose() + +$logEntries = @() + +$datetime = (get-date).ToUniversalTime() +$hour = $datetime.Hour +$min = $datetime.Minute +$timestamp = $datetime.ToString("yyyy-MM-ddT$($hour):$($min):00.000Z") + +$ctx = Get-AzContext + +foreach ($disk in $unattachedDisks.Rows) +{ + $isEligible = $false + $logDetails = $null + if ([string]::IsNullOrEmpty($tagsFilter)) + { + $isEligible = $true + } + else + { + $diskTags = Get-AzTag -ResourceId $disk.InstanceId -ErrorAction SilentlyContinue + if ($diskTags) + { + foreach ($tagFilter in $tagsFilter) + { + if ($diskTags.Properties.TagsProperty.($tagFilter.tagName) -eq $tagFilter.tagValue) + { + $isEligible = $true + } + else + { + $isEligible = $false + break + } + } + } + } + + $subscriptionId = $disk.InstanceId.Split("/")[2] + $resourceGroup = $disk.InstanceId.Split("/")[4] + $instanceName = $disk.InstanceId.Split("/")[8] + + if ($isEligible) + { + $diskState = "Unknown" + $currentSku = "Unknown" + + Write-Output "Performing $remediationAction action (SIMULATE=$Simulate) on $($disk.InstanceId) disk..." + if ($ctx.Environment.Name -eq $disk.Cloud -and $ctx.Tenant.Id -eq $disk.TenantGuid) + { + if ($ctx.Subscription.Id -ne $subscriptionId) + { + Select-AzSubscription -SubscriptionId $subscriptionId | Out-Null + $ctx = Get-AzContext + } + $diskObj = Get-AzDisk -ResourceGroupName $resourceGroup -DiskName $instanceName -ErrorAction SilentlyContinue + if (-not($diskObj.ManagedBy)) + { + $diskState = "Unattached" + $currentSku = $diskObj.Sku.Name + if ($remediationAction -eq "Downsize") + { + if ($diskObj.Sku.Name -notin ('Standard_LRS','StandardSSD_ZRS')) + { + if ($diskObj.Sku.Name -like "*_LRS" -and $diskObj.Sku.Name -notlike "*V2*") + { + Write-Output "Downgrading $($diskObj.Name) to Standard_LRS..." + if (-not($Simulate)) + { + $diskObj.Sku = [Microsoft.Azure.Management.Compute.Models.DiskSku]::new('Standard_LRS', 'Standard') + $diskObj | Update-AzDisk | Out-Null + } + } + elseif ($diskObj.Sku.Name -like "*_ZRS" -and $diskObj.Sku.Name -notlike "*V2*") + { + Write-Output "Downgrading $($diskObj.Name) to StandardSSD_ZRS..." + if (-not($Simulate)) + { + $diskObj.Sku = [Microsoft.Azure.Management.Compute.Models.DiskSku]::new('StandardSSD_ZRS', 'Standard') + $diskObj | Update-AzDisk | Out-Null + } + } + else + { + Write-Output "Skipping as $($diskObj.Name) disk is in an unsupported SKU ($($diskObj.Sku.Name))..." + } + } + else + { + Write-Output "Skipping as $($diskObj.Name) disk is already in the lowest SKU ($($diskObj.Sku.Name))." + } + } + elseif ($remediationAction -eq "Delete") + { + if (-not($Simulate)) + { + Remove-AzDisk -ResourceGroupName $resourceGroup -DiskName $instanceName -Force | Out-Null + } + } + else + { + Write-Output "Skipping as action is not supported." + } + } + else + { + if ($diskObj) + { + Write-Output "Skipping as disk is not unattached." + $diskState = "Attached" + } + else + { + Write-Output "Skipping as disk was already removed." + $diskState = "Removed" + } + } + } + else + { + Write-Output "Could not apply remediation as disk is in another cloud/tenant." + } + } + + $logDetails = @{ + IsEligible = $isEligible + RemediationAction = $remediationAction + DiskState = $diskState + CurrentSku = $currentSku + } + + $logentry = New-Object PSObject -Property @{ + Timestamp = $timestamp + Cloud = $disk.Cloud + TenantGuid = $disk.TenantGuid + SubscriptionGuid = $subscriptionId + ResourceGroupName = $resourceGroup.ToLower() + InstanceName = $instanceName.ToLower() + InstanceId = $disk.InstanceId.ToLower() + Simulate = $Simulate + LogDetails = $logDetails | ConvertTo-Json -Compress + RecommendationSubTypeId = $recommendationId + } + + $logEntries += $logentry +} + +$today = $datetime.ToString("yyyyMMdd") +$csvExportPath = "$today-unattacheddisksfiltered.csv" + +$logEntries | Export-Csv -Path $csvExportPath -NoTypeInformation + +$csvBlobName = $csvExportPath + +$csvProperties = @{"ContentType" = "text/csv"}; + +Set-AzStorageBlobContent -File $csvExportPath -Container $storageAccountSinkContainer -Properties $csvProperties -Blob $csvBlobName -Context $saCtx -Force diff --git a/docs/deploy/optimization-engine/latest/azuredeploy-nested.bicep b/docs/deploy/optimization-engine/latest/azuredeploy-nested.bicep index d05e2408a..0e12a764d 100644 --- a/docs/deploy/optimization-engine/latest/azuredeploy-nested.bicep +++ b/docs/deploy/optimization-engine/latest/azuredeploy-nested.bicep @@ -1761,7 +1761,7 @@ resource sqlDatabase 'Microsoft.Sql/servers/databases@2022-05-01-preview' = { zoneRedundant: false readScale: 'Disabled' autoPauseDelay: 60 - requestedBackupStorageRedundancy: 'Geo' + requestedBackupStorageRedundancy: 'Local' } } diff --git a/docs/deploy/optimization-engine/latest/runbooks/recommendations/Recommend-DiskOptimizationsToBlobStorage.ps1 b/docs/deploy/optimization-engine/latest/runbooks/recommendations/Recommend-DiskOptimizationsToBlobStorage.ps1 index 86a484745..364b3b303 100644 --- a/docs/deploy/optimization-engine/latest/runbooks/recommendations/Recommend-DiskOptimizationsToBlobStorage.ps1 +++ b/docs/deploy/optimization-engine/latest/runbooks/recommendations/Recommend-DiskOptimizationsToBlobStorage.ps1 @@ -224,7 +224,7 @@ $baseQuery = @" | summarize MaxIOPSMetric = max(todouble(MetricValue_s)) by ResourceId | join kind=inner ( $disksTableName - | where TimeGenerated > ago(1d) and DiskState_s =~ 'Attached' and SKU_s startswith 'Premium' + | where TimeGenerated > ago(1d) and DiskState_s =~ 'Attached' and SKU_s startswith 'Premium' and SKU_s !contains "V2" | extend DiskTier_s = strcat(DiskTier_s, ' ', tostring(split(SKU_s, '_')[1])) | project ResourceId=InstanceId_s, DiskName_s, ResourceGroup = ResourceGroupName_s, SubscriptionId = SubscriptionGuid_g, Cloud_s, TenantGuid_g, Tags_s, MaxIOPSDisk=toint(DiskIOPS_s), DiskSizeGB_s, SKU_s, DiskTier_s, DiskType_s ) on ResourceId @@ -237,7 +237,7 @@ $baseQuery = @" | summarize MaxMBsMetric = max(todouble(MetricValue_s)/1024/1024) by ResourceId | join kind=inner ( $disksTableName - | where TimeGenerated > ago(1d) and DiskState_s =~ 'Attached' and SKU_s startswith 'Premium' + | where TimeGenerated > ago(1d) and DiskState_s =~ 'Attached' and SKU_s startswith 'Premium' and SKU_s !contains "V2" | extend DiskTier_s = strcat(DiskTier_s, ' ', tostring(split(SKU_s, '_')[1])) | project ResourceId=InstanceId_s, DiskName_s, ResourceGroup = ResourceGroupName_s, SubscriptionId = SubscriptionGuid_g, Cloud_s, TenantGuid_g, Tags_s, MaxMBsDisk=toint(DiskThroughput_s), DiskSizeGB_s, SKU_s, DiskTier_s, DiskType_s ) on ResourceId diff --git a/docs/guide.md b/docs/guide.md index e270e297b..844874d02 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -15,15 +15,13 @@ Your guide to implementing FinOps in the Microsoft Cloud. FinOps is an expansive area with branches that extend out into many cloud management and governance activities. Cost especially is a part of many cloud and business initiatives, like security, resiliency, and adopting cloud-native solutions like AI. And to streamline cloud adoption in a way that prioritizes business value, you need the right guidance to help you make the right data-driven decisions. The Implementing FinOps guide is a collection of resources that help you facilitate your FinOps goals. - ## Getting started with FinOps diff --git a/docs/hubs.md b/docs/hubs.md index 08e0f6d56..a4b61a09c 100644 --- a/docs/hubs.md +++ b/docs/hubs.md @@ -22,7 +22,7 @@ FinOps hubs are a reliable, trustworthy platform for cost analytics, insights, a

What's new in January 2026v13

- In January, FinOps hubs reorganized Bicep modules into separate apps, enhanced scope configuration documentation to clarify multi-account and cross-cloud support, added comprehensive troubleshooting for Data Explorer ingestion errors, improved KQL function reliability, fixed all Bicep compilation warnings, and added documentation for removing private networking. + In January, FinOps hubs reorganized Bicep modules into separate apps, enhanced scope configuration documentation to clarify multi-account and cross-cloud support, added documentation for removing private networking and comprehensive troubleshooting, improved KQL function reliability, and fixed numerous pipeline and data processing bugs.

See all changes

diff --git a/docs/open-data.md b/docs/open-data.md index 53aef0e13..3fc7a5da2 100644 --- a/docs/open-data.md +++ b/docs/open-data.md @@ -19,13 +19,15 @@ Leverage open data to normalize and enhance your FinOps reporting. FinOps toolkit open data is used to transform Cost Management actual and amortized data into FOCUS. Use the same mappings to clean your FinOps datasets. + diff --git a/docs/powershell.md b/docs/powershell.md index ba37b1279..8e3c3ab6e 100644 --- a/docs/powershell.md +++ b/docs/powershell.md @@ -19,15 +19,13 @@ Automate and scale your FinOps efforts with PowerShell commands that streamline The FinOps toolkit PowerShell module helps you automate and scale common Cost Management and FinOps toolkit management operations and work with FinOps toolkit open data. - diff --git a/docs/workbooks.md b/docs/workbooks.md index 296cad816..8b794d83a 100644 --- a/docs/workbooks.md +++ b/docs/workbooks.md @@ -19,15 +19,13 @@ Engineering hub to maximize cloud ROI through FinOps. FinOps workbooks are Azure workbooks that provide a series of tools to help engineers perform targeted FinOps tasks, modeled after the Well-Architected Framework guidance. - diff --git a/package-lock.json b/package-lock.json index 410c1a19e..7278824e4 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "ftk", - "version": "12.0.0", + "version": "13.0.0", "lockfileVersion": 2, "requires": true, "packages": { "": { "name": "ftk", - "version": "12.0.0", + "version": "13.0.0", "license": "MIT", "devDependencies": { "all-contributors-cli": "^6.26.1" diff --git a/package.json b/package.json index 2dd215d4c..ee8ca6348 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "ftk", - "version": "12.0.0", + "version": "13.0.0", "description": "Starter kits, scripts, and advanced solutions to accelerate your FinOps journey in the Microsoft Cloud.", "main": "index.js", "directories": { diff --git a/src/optimization-engine/ftkver.txt b/src/optimization-engine/ftkver.txt index 8bafbd775..7f27d6b1d 100644 --- a/src/optimization-engine/ftkver.txt +++ b/src/optimization-engine/ftkver.txt @@ -1 +1 @@ -12.0 \ No newline at end of file +13.0 \ No newline at end of file diff --git a/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl b/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl index 904fd4447..4786bd206 100644 --- a/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl +++ b/src/power-bi/storage/Shared.Dataset/definition/tables/Costs.tmdl @@ -613,6 +613,14 @@ table Costs annotation SummarizationSetBy = Automatic + column x_SkuInstanceType + dataType: string + lineageTag: 34616157-7ddc-4ed4-8bda-0afea596e583 + summarizeBy: none + sourceColumn: x_SkuInstanceType + + annotation SummarizationSetBy = Automatic + column x_SkuLicenseStatus dataType: string lineageTag: 86d87994-b011-4838-a8e9-97ddb856d6fa @@ -1648,13 +1656,13 @@ table Costs // AHB columns // TODO: Add SQL AHB handling AHB = Table.RemoveColumns( - Table.AddColumn(Table.AddColumn(Table.AddColumn(Table.AddColumn(Table.AddColumn(Table.AddColumn( + Table.AddColumn(Table.AddColumn(Table.AddColumn(Table.AddColumn(Table.AddColumn(Table.AddColumn(Table.AddColumn( Table.TransformColumnTypes( Table.ExpandRecordColumn( Table.ReplaceErrorValues(Table.AddColumn(ExtractedTags, "tmp_SkuDetails", each [x_SkuDetailsDictionary]), {{"tmp_SkuDetails", null}}), "tmp_SkuDetails", - {"UsageType", "ImageType", "ServiceType", "VMName", "VMProperties", "VCPUs", "ReservationOrderId", "ReservationId", "VMCapacityReservationId", "AHB", "vCores"}, - {"x_SkuUsageType", "x_SkuImageType", "x_SkuType", "tmp_VMName", "x_SkuVMProperties", "tmp_VMvCPUs", "tmp_AddlReservationOrderId", "tmp_AddlReservationId", "x_CapacityCommitmentId", "tmp_SQLAHB", "tmp_SQLvCores"} + {"UsageType", "ImageType", "ServiceType", "VMName", "VMProperties", "VCPUs", "ReservationOrderId", "ReservationId", "VMCapacityReservationId", "AHB", "vCores", "ServerSku"}, + {"x_SkuUsageType", "x_SkuImageType", "x_SkuType", "tmp_VMName", "x_SkuVMProperties", "tmp_VMvCPUs", "tmp_AddlReservationOrderId", "tmp_AddlReservationId", "x_CapacityCommitmentId", "tmp_SQLAHB", "tmp_SQLvCores", "tmp_ServerSku"} ), {{"tmp_VMvCPUs", Int64.Type}, {"tmp_SQLvCores", Int64.Type}} ), @@ -1669,8 +1677,9 @@ table Costs type number ), "x_SkuLicenseStatus", each if _isNotBlank([x_SkuMeterSubcategory]) and Text.Contains([x_SkuMeterSubcategory], "Windows") or [tmp_SQLAHB] = "False" then "Not enabled" else if (_isNotBlank([x_SkuImageType]) and Text.Contains([x_SkuImageType], "Windows Server BYOL")) or [tmp_SQLAHB] = "True" or (_isNotBlank([x_SkuMeterSubcategory]) and Text.Contains([x_SkuMeterSubcategory], "Azure Hybrid Benefit")) then "Enabled" else "Not supported"), - "x_CommitmentDiscountKey", each [x_SkuType] & [x_SkuMeterId]), - { "tmp_AddlReservationOrderId", "tmp_AddlReservationId", "tmp_SQLAHB", "tmp_SQLvCores", "tmp_VMName", "tmp_VMvCPUs" } + "x_SkuInstanceType", each if [x_SkuType] <> null then [x_SkuType] else if [tmp_ServerSku] <> null then [tmp_ServerSku] else null), + "x_CommitmentDiscountKey", each if [x_SkuInstanceType] <> null then [x_SkuInstanceType] & [x_SkuMeterId] else null), + { "tmp_AddlReservationOrderId", "tmp_AddlReservationId", "tmp_SQLAHB", "tmp_ServerSku", "tmp_SQLvCores", "tmp_VMName", "tmp_VMvCPUs" } ), // Unique key for the record -- WARNING: This increases the data size significantly. diff --git a/src/powershell/Private/Get-VersionNumber.ps1 b/src/powershell/Private/Get-VersionNumber.ps1 index ea542c956..95daac0d7 100644 --- a/src/powershell/Private/Get-VersionNumber.ps1 +++ b/src/powershell/Private/Get-VersionNumber.ps1 @@ -4,5 +4,5 @@ function Get-VersionNumber { param() - return '12.0' + return '13.0' } diff --git a/src/powershell/Private/Save-FinOpsHubTemplate.ps1 b/src/powershell/Private/Save-FinOpsHubTemplate.ps1 index b7aed581a..3d37e3487 100644 --- a/src/powershell/Private/Save-FinOpsHubTemplate.ps1 +++ b/src/powershell/Private/Save-FinOpsHubTemplate.ps1 @@ -36,7 +36,7 @@ function Save-FinOpsHubTemplate [Parameter()] [string] - $Destination = $env:temp + $Destination = [System.IO.Path]::GetTempPath() ) $progress = $ProgressPreference diff --git a/src/powershell/Public/Deploy-FinOpsHub.ps1 b/src/powershell/Public/Deploy-FinOpsHub.ps1 index 5f8c55ce8..9abc9b2a6 100644 --- a/src/powershell/Public/Deploy-FinOpsHub.ps1 +++ b/src/powershell/Public/Deploy-FinOpsHub.ps1 @@ -176,7 +176,7 @@ function Deploy-FinOpsHub } # Create folder for download - $toolkitPath = Join-Path $env:temp -ChildPath 'FinOpsToolkit' + $toolkitPath = Join-Path ([System.IO.Path]::GetTempPath()) -ChildPath 'FinOpsToolkit' if (Test-ShouldProcess $PSCmdlet $toolkitPath 'CreateTempDirectory') { New-Directory -Path $toolkitPath diff --git a/src/powershell/Tests/Integration/CostExports.Tests.ps1 b/src/powershell/Tests/Integration/CostExports.Tests.ps1 index 21aaffb99..6af8a6bc6 100644 --- a/src/powershell/Tests/Integration/CostExports.Tests.ps1 +++ b/src/powershell/Tests/Integration/CostExports.Tests.ps1 @@ -305,14 +305,14 @@ Describe 'CostExports' { # NOTE: This is a unit test that mocks dependencies. It's run with integration tests due to how long it takes to run. # Arrange - $waited = $false + $script:waited = $false $testStartTime = Get-Date function CheckDate($date) { $monthToThrottle = (Get-Date -Day 1 -Hour 0 -Minute 0 -Second 0 -Millisecond 0 -AsUTC).AddMonths(-3).ToUniversalTime().Date.ToString("yyyy-MM-dd'T'HH:mm:ss'Z'") - if ($date -eq $monthToThrottle -and -not $waited) + if ($date -eq $monthToThrottle -and -not $script:waited) { - $waited = $true + $script:waited = $true return $true } return $false diff --git a/src/powershell/Tests/Integration/Hubs.Tests.ps1 b/src/powershell/Tests/Integration/Hubs.Tests.ps1 index 19bb673d3..370b03859 100644 --- a/src/powershell/Tests/Integration/Hubs.Tests.ps1 +++ b/src/powershell/Tests/Integration/Hubs.Tests.ps1 @@ -42,7 +42,7 @@ Describe 'Hubs' { [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseDeclaredVarsMoreThanAssignments', '')] $ftk_ResourceGroup = "ftk-test-integration" [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseDeclaredVarsMoreThanAssignments', '')] - $ftk_HubName = "ftk-test-DeployHub_$($env:USERNAME)" + $ftk_HubName = "ftk-test-DeployHub_$($env:USER ?? $env:USERNAME ?? 'unknown')" [Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseDeclaredVarsMoreThanAssignments', '')] $ftk_HubRG = "ftk-test-integration" # TODO: Confirm lowercase/space requirement and add handling to avoid the limitation diff --git a/src/powershell/Tests/Integration/New-Directory.Tests.ps1 b/src/powershell/Tests/Integration/New-Directory.Tests.ps1 index ae41914ab..31ad07e96 100644 --- a/src/powershell/Tests/Integration/New-Directory.Tests.ps1 +++ b/src/powershell/Tests/Integration/New-Directory.Tests.ps1 @@ -7,7 +7,7 @@ InModuleScope 'FinOpsToolkit' { Describe 'New-Directory' { It 'Should create a directory' { # Arrange - $path = "$env:temp/ftk-test/New-Directory" + $path = Join-Path ([System.IO.Path]::GetTempPath()) 'ftk-test/New-Directory' if (Test-Path $path) { Remove-Item -Path $path -Recurse -Force diff --git a/src/powershell/Tests/Integration/Toolkit.Tests.ps1 b/src/powershell/Tests/Integration/Toolkit.Tests.ps1 index fc9b57229..2d1b7ad51 100644 --- a/src/powershell/Tests/Integration/Toolkit.Tests.ps1 +++ b/src/powershell/Tests/Integration/Toolkit.Tests.ps1 @@ -6,8 +6,15 @@ Describe 'Get-FinOpsToolkitVersion' { It 'Should return all known releases' { # Arrange - $plannedRelease = '12.0' - $expected = @('0.11', '0.10', '0.9', '0.8', '0.7', '0.6', '0.5', '0.4', '0.3', '0.2', '0.1.1', '0.1', '0.0.1') + $plannedRelease = '13' + $expected = @('12', '0.11', '0.10', '0.9', '0.8', '0.7', '0.6', '0.5', '0.4', '0.3', '0.2', '0.1.1', '0.1', '0.0.1') + + # Helper function to normalize version strings for [version] parsing + # Single-part versions like "12" need to become "12.0" for [version] to parse them + function NormalizeVersion($ver) { + if ($ver -notmatch '\.') { return "$ver.0" } + return $ver + } # Act $result = Get-FinOpsToolkitVersion @@ -18,7 +25,7 @@ Describe 'Get-FinOpsToolkitVersion' { $result.Count | Should -BeLessOrEqual ($expected.Count + 1) $result | ForEach-Object { $verStr = $_.Version - $verObj = [version]$verStr + $verObj = [version](NormalizeVersion $verStr) $fileCount = 0 function CheckFile($file, $minVer, $maxVer) @@ -47,6 +54,7 @@ Describe 'Get-FinOpsToolkitVersion' { CheckFile "optimization-workbook-v$verStr.zip" $null '0.5' # Power BI + CheckFile "FinOpsToolkitData.pbix" '12.0' $null CheckFile "PowerBI-demo.zip" '0.7' $null CheckFile "PowerBI-kql.zip" '0.7' $null CheckFile "PowerBI-storage.zip" '0.7' $null diff --git a/src/powershell/Tests/Unit/Deploy-FinOpsHub.Tests.ps1 b/src/powershell/Tests/Unit/Deploy-FinOpsHub.Tests.ps1 index 445bb8689..dce1982bd 100644 --- a/src/powershell/Tests/Unit/Deploy-FinOpsHub.Tests.ps1 +++ b/src/powershell/Tests/Unit/Deploy-FinOpsHub.Tests.ps1 @@ -124,7 +124,7 @@ InModuleScope 'FinOpsToolkit' { Context 'More' { BeforeAll { - $templateFile = Join-Path -Path $env:temp -ChildPath 'FinOps\finops-hub-v1.0.0\main.bicep' + $templateFile = Join-Path -Path ([System.IO.Path]::GetTempPath()) -ChildPath 'FinOps/finops-hub-v1.0.0/main.bicep' Mock -CommandName 'Get-ChildItem' -MockWith { return @{ FullName = $templateFile } } Mock -CommandName 'New-AzResourceGroupDeployment' } diff --git a/src/scripts/Build-OpenData.ps1 b/src/scripts/Build-OpenData.ps1 index c0a961a45..d1ac137d4 100644 --- a/src/scripts/Build-OpenData.ps1 +++ b/src/scripts/Build-OpenData.ps1 @@ -588,50 +588,74 @@ if ($Hubs) } } +# Load the generic cube SVG for comparison (normalize whitespace) +$genericCubeSvg = (Get-Content "$svgDir/microsoft.resources/resources.svg" -Raw).Trim() + (git diff --name-only) ` | Where-Object { $_ -match '^docs/svg/([^/]+/)+[^\.]+\.svg$' } ` | ForEach-Object { $file = "$PSScriptRoot/../../$_" - $diff = git diff -- $file - $changes = $diff -split "`n" ` - | Where-Object { $_ -match '^\+|^\-' } ` # Remove lines that are not changes - | Where-Object { $_ -notmatch '^\+\+\+|^\-\-\-' } # Remove the diff metadata lines - - # Match added/removed lines - $added = @() - $removed = @() - foreach ($line in $changes) + $shouldRevert = $false + $revertReason = "" + + # Check if changed TO the generic cube (regression from unique icon) + $newContent = (Get-Content $file -Raw -ErrorAction SilentlyContinue)?.Trim() + $oldContent = (git show HEAD:$_ 2>$null)?.Trim() + if ($newContent -eq $genericCubeSvg -and $oldContent -and $oldContent -ne $genericCubeSvg) { - if ($line.StartsWith('+')) - { - $added += $line.Substring(1) - } - elseif ($line.StartsWith('-')) - { - $removed += $line.Substring(1) - } + $shouldRevert = $true + $revertReason = "regression to generic cube" } - # Filter out files that only changed GUIDs - $onlyGuidChanges = $false - $maxPairs = [Math]::Max($added.Count, $removed.Count) - for ($i = 0; $i -lt $maxPairs; $i++) + # Check if only non-visual changes (GUIDs, formatting, attribute order, minor precision) + if (-not $shouldRevert -and $oldContent -and $newContent) { - $before = if ($i -lt $removed.Count) { $removed[$i] } else { '' } - $after = if ($i -lt $added.Count) { $added[$i] } else { '' } + # Normalize SVG for comparison (remove non-visual differences) + # Note: This catches common reformatting but not all SVG optimizations (path simplification, etc.) + function Get-NormalizedSvg($svg) { + $n = $svg + # Normalize generated IDs (GUIDs and short hashes like 'p8eaoyhoh__b') + $n = $n -replace '[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}', '__ID__' + $n = $n -replace "id='[a-z0-9_]+__[a-z]'", "id='__ID__'" + $n = $n -replace "url\(#[a-z0-9_]+__[a-z]\)", "url(#__ID__)" + $n = $n -replace "url\(#__ID__\)", "url(#__ID__)" + # Remove non-visual attributes + $n = $n -replace " class='[^']*'", '' + $n = $n -replace ' class="[^"]*"', '' + $n = $n -replace " fill='none'", '' + $n = $n -replace ' fill="none"', '' + # Normalize hex colors (#ffffff -> #fff, etc.) + $n = $n -replace '#([0-9a-fA-F])\1([0-9a-fA-F])\2([0-9a-fA-F])\3', '#$1$2$3' + # Remove wrappers (they don't affect rendering) + $n = $n -replace '', '' + $n = $n -replace '', '' + # Normalize decimal precision - truncate to 1 decimal place + $n = $n -replace '(\d+\.\d)\d+', '$1' + # Extract and sort attributes within each tag to handle reordering + $n = [regex]::Replace($n, '<(\w+)\s+([^>]+)(/?)>', { + param($match) + $tagName = $match.Groups[1].Value + $attrs = $match.Groups[2].Value + $selfClose = $match.Groups[3].Value + # Parse attributes and sort them + $attrList = [regex]::Matches($attrs, "(\w+[-\w]*)='[^']*'") | ForEach-Object { $_.Value } | Sort-Object + return "<$tagName $($attrList -join ' ')$selfClose>" + }) + return $n + } - # Replace all GUIDs in both lines with a placeholder and compare - $beforeClean = $before -replace '[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}', '__GUID__' - $afterClean = $after -replace '[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}', '__GUID__' - if ($beforeClean -eq $afterClean) + $oldNormalized = Get-NormalizedSvg $oldContent + $newNormalized = Get-NormalizedSvg $newContent + if ($oldNormalized -eq $newNormalized) { - $onlyGuidChanges = $true - break + $shouldRevert = $true + $revertReason = "only non-visual changes" } } - if ($onlyGuidChanges) + + if ($shouldRevert) { - Write-Host "Reverting $file" + Write-Host "Reverting $file ($revertReason)" git checkout -- $file } } diff --git a/src/scripts/Get-Version.ps1 b/src/scripts/Get-Version.ps1 index fd4cac5e3..39dd69b2b 100644 --- a/src/scripts/Get-Version.ps1 +++ b/src/scripts/Get-Version.ps1 @@ -11,7 +11,7 @@ After getting the version from NPM, we also do the following to clean up the value: 1. Remove quotes 2. Strip control characters - 3. Remove trailing 0s (keep major/minor/label) + 3. Remove patch version when 0 (keep major.minor and label) .PARAMETER AsDotNetVersion Optional. Indicates that the returned version should be in the format "x.x.x.x". Otherwise, semantic versioning is used. Deafult = false. @@ -28,9 +28,9 @@ param( $ver = (Get-Content (Join-Path $PSScriptRoot ../../package.json) | ConvertFrom-Json).version ` -replace '^[^\d]*((\d+\.\d+)(\.\d+)?(-[a-z]+)?(\.\d+)?)[^\d]*$', '$1' ` - -replace '^(\d+\.\d+)(\.\d+)?(-[a-z]+)?(\.0)?$', '$1$2$3' ` + -replace '^(\d+\.\d+)(\.0)(-[a-z]+)?$', '$1$3' ` -replace '^(\d+\.\d+)(\.0)?(-[a-z]+)?(\.\d+)?$', '$1$3$4' ` - -replace '^(\d+)(\.0)$', '$1' + -replace '(-[a-z]+)\.0$', '$1' if ($AsDotNetVersion -and $ver.Contains('-')) { diff --git a/src/scripts/Update-Version.ps1 b/src/scripts/Update-Version.ps1 index cff513bcc..1fea7d516 100644 --- a/src/scripts/Update-Version.ps1 +++ b/src/scripts/Update-Version.ps1 @@ -100,12 +100,17 @@ if ($update -or $Version) # Update version in secondary files, if needed if ($update -or $Version) { - # Update version in ftkver.txt files (templates, modules, docs) - Write-Verbose "Updating ftkver.txt files..." - Get-ChildItem ../.. -Include ftkver.txt -Recurse ` - | ForEach-Object { - Write-Verbose "- $($_.FullName.Replace((Get-Item ../..).FullName + [IO.Path]::DirectorySeparatorChar, ''))" - $ver | Out-File $_ -NoNewline + # Update version files: ftkver.txt (major.minor) and ftktag.txt (major only, for git tags) + $repoRoot = (Resolve-Path "$PSScriptRoot/../..").Path + $tag = $ver -replace '\.0$', '' + foreach ($entry in @{ 'ftkver.txt' = $ver; 'ftktag.txt' = $tag }.GetEnumerator()) + { + Write-Verbose "Updating $($entry.Key) files..." + Get-ChildItem $repoRoot -Include $entry.Key -Recurse -Force ` + | ForEach-Object { + Write-Verbose "- $($_.FullName.Replace($repoRoot + [IO.Path]::DirectorySeparatorChar, ''))" + $entry.Value | Out-File $_ -NoNewline + } } # Update version in PowerShell diff --git a/src/templates/finops-alerts/modules/ftkver.txt b/src/templates/finops-alerts/modules/ftkver.txt index 8bafbd775..7f27d6b1d 100644 --- a/src/templates/finops-alerts/modules/ftkver.txt +++ b/src/templates/finops-alerts/modules/ftkver.txt @@ -1 +1 @@ -12.0 \ No newline at end of file +13.0 \ No newline at end of file diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep index 521731d37..6274ea88e 100644 --- a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep @@ -124,9 +124,10 @@ var INGESTION = 'ingestion' var INGESTION_DB = 'Ingestion' var INGESTION_ID_SEPARATOR = '__' -var ftkReleaseUri = endsWith(finOpsToolkitVersion, '-dev') +var ftkGitTag = loadTextContent('../../fx/ftktag.txt') // cSpell:ignore ftktag +var ftkReleaseUri = indexOf(finOpsToolkitVersion, '-dev') != -1 ? 'https://raw.githubusercontent.com/microsoft/finops-toolkit/refs/heads/dev/src/open-data' - : 'https://raw.githubusercontent.com/microsoft/finops-toolkit/refs/tags/v${finOpsToolkitVersion}/src/open-data' + : 'https://raw.githubusercontent.com/microsoft/finops-toolkit/refs/tags/v${ftkGitTag}/src/open-data' var useFabric = !empty(fabricQueryUri) var useAzure = !useFabric && !empty(clusterName) @@ -645,7 +646,7 @@ resource dataset_ftkReleaseFile 'Microsoft.DataFactory/factories/datasets@2018-0 } version: { type: 'string' - defaultValue: finOpsToolkitVersion + defaultValue: ftkGitTag // Must match the tag, not a major.minor version (e.g., 13, not 13.0) } } annotations: [] diff --git a/src/templates/finops-hub/modules/fx/ftktag.txt b/src/templates/finops-hub/modules/fx/ftktag.txt new file mode 100644 index 000000000..ca7bf83ac --- /dev/null +++ b/src/templates/finops-hub/modules/fx/ftktag.txt @@ -0,0 +1 @@ +13 \ No newline at end of file diff --git a/src/templates/finops-hub/modules/fx/ftkver.txt b/src/templates/finops-hub/modules/fx/ftkver.txt index 8bafbd775..7f27d6b1d 100644 --- a/src/templates/finops-hub/modules/fx/ftkver.txt +++ b/src/templates/finops-hub/modules/fx/ftkver.txt @@ -1 +1 @@ -12.0 \ No newline at end of file +13.0 \ No newline at end of file diff --git a/src/templates/finops-workbooks/ftkver.txt b/src/templates/finops-workbooks/ftkver.txt index 8bafbd775..7f27d6b1d 100644 --- a/src/templates/finops-workbooks/ftkver.txt +++ b/src/templates/finops-workbooks/ftkver.txt @@ -1 +1 @@ -12.0 \ No newline at end of file +13.0 \ No newline at end of file diff --git a/src/workbooks/.scaffold/ftkver.txt b/src/workbooks/.scaffold/ftkver.txt index 8bafbd775..7f27d6b1d 100644 --- a/src/workbooks/.scaffold/ftkver.txt +++ b/src/workbooks/.scaffold/ftkver.txt @@ -1 +1 @@ -12.0 \ No newline at end of file +13.0 \ No newline at end of file From d4370b49d3c9f836d9e6310dde559c49135ad4ef Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Sat, 31 Jan 2026 09:26:31 +0000 Subject: [PATCH 65/69] Fix ftktag.txt to contain full git tag (#1959) Co-authored-by: Claude Opus 4.5 --- docs-mslearn/toolkit/changelog.md | 115 +++++++++++++++--- .../workbooks/finops-workbooks-overview.md | 9 +- docs/README.md | 2 +- docs/_includes/ftktag.txt | 2 +- ...t-OptimizationCSVExportsToLogAnalytics.ps1 | 4 +- .../Ingest-RecommendationsToLogAnalytics.ps1 | 4 +- .../Ingest-SuppressionsToLogAnalytics.ps1 | 4 +- ...t-OptimizationCSVExportsToLogAnalytics.ps1 | 4 +- .../Ingest-RecommendationsToLogAnalytics.ps1 | 4 +- .../Ingest-SuppressionsToLogAnalytics.ps1 | 4 +- ...t-OptimizationCSVExportsToLogAnalytics.ps1 | 4 +- .../Ingest-RecommendationsToLogAnalytics.ps1 | 4 +- .../Ingest-SuppressionsToLogAnalytics.ps1 | 4 +- ...t-OptimizationCSVExportsToLogAnalytics.ps1 | 4 +- .../Ingest-RecommendationsToLogAnalytics.ps1 | 4 +- .../Ingest-SuppressionsToLogAnalytics.ps1 | 4 +- ...t-OptimizationCSVExportsToLogAnalytics.ps1 | 4 +- .../Ingest-RecommendationsToLogAnalytics.ps1 | 4 +- .../Ingest-SuppressionsToLogAnalytics.ps1 | 4 +- ...t-OptimizationCSVExportsToLogAnalytics.ps1 | 4 +- .../Ingest-RecommendationsToLogAnalytics.ps1 | 4 +- .../Ingest-SuppressionsToLogAnalytics.ps1 | 4 +- ...t-OptimizationCSVExportsToLogAnalytics.ps1 | 4 +- .../Ingest-RecommendationsToLogAnalytics.ps1 | 4 +- .../Ingest-SuppressionsToLogAnalytics.ps1 | 4 +- ...t-OptimizationCSVExportsToLogAnalytics.ps1 | 4 +- .../Ingest-RecommendationsToLogAnalytics.ps1 | 4 +- .../Ingest-SuppressionsToLogAnalytics.ps1 | 4 +- ...t-OptimizationCSVExportsToLogAnalytics.ps1 | 4 +- .../Ingest-RecommendationsToLogAnalytics.ps1 | 4 +- .../Ingest-SuppressionsToLogAnalytics.ps1 | 4 +- ...t-OptimizationCSVExportsToLogAnalytics.ps1 | 4 +- .../Ingest-RecommendationsToLogAnalytics.ps1 | 4 +- .../Ingest-SuppressionsToLogAnalytics.ps1 | 4 +- ...t-OptimizationCSVExportsToLogAnalytics.ps1 | 4 +- .../Ingest-RecommendationsToLogAnalytics.ps1 | 4 +- .../Ingest-SuppressionsToLogAnalytics.ps1 | 4 +- ...t-OptimizationCSVExportsToLogAnalytics.ps1 | 4 +- .../Ingest-RecommendationsToLogAnalytics.ps1 | 4 +- .../Ingest-SuppressionsToLogAnalytics.ps1 | 4 +- ...t-OptimizationCSVExportsToLogAnalytics.ps1 | 4 +- .../Ingest-RecommendationsToLogAnalytics.ps1 | 4 +- .../Ingest-SuppressionsToLogAnalytics.ps1 | 4 +- ...t-OptimizationCSVExportsToLogAnalytics.ps1 | 78 +++++++----- .../Ingest-RecommendationsToLogAnalytics.ps1 | 86 ++++++++----- .../Ingest-SuppressionsToLogAnalytics.ps1 | 92 ++++++++------ src/scripts/Update-Version.ps1 | 4 +- .../Microsoft.FinOpsHubs/Analytics/app.bicep | 2 +- .../finops-hub/modules/fx/ftktag.txt | 2 +- 49 files changed, 344 insertions(+), 204 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 838f33028..4b26c927b 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -97,7 +97,12 @@ _Released January 2026_ - **Added** - Added `-Format` and `-CompressionMode` parameters to [New-FinOpsCostExport](powershell/cost/New-FinOpsCostExport.md) to support Parquet format and gzip/snappy compression ([#1074](https://github.com/microsoft/finops-toolkit/issues/1074)). -> [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v13) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v12...v13) + +> [!div class="nextstepaction"] +> [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v13) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v12...v13) +
@@ -201,7 +206,12 @@ _Released July 16, 2025_ - microsoft.durabletask/schedulers - microsoft.edge/contexts -> [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v12) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.11...v12) + +> [!div class="nextstepaction"] +> [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v12) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.11...v12) +
@@ -335,7 +345,12 @@ _Released June 2, 2025_ - microsoft.synapse/workspaces/sqlpools - microsoft.web/sites/slots -> [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.11) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.10...v0.11) + +> [!div class="nextstepaction"] +> [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.11) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.10...v0.11) +
@@ -452,7 +467,12 @@ _Released May 4, 2025_ - microsoft.liftrpilot/organizations - mongodb.atlas/organizations -> [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.10) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.9...v0.10) + +> [!div class="nextstepaction"] +> [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.10) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.9...v0.10) +
@@ -635,7 +655,12 @@ _Released April 4, 2025_ - **Added** - Added sample data for MCA reservation exports. -> [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.9) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.8...v0.9) + +> [!div class="nextstepaction"] +> [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.9) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.8...v0.9) +
@@ -847,7 +872,12 @@ _Released February 12, 2025_ - microsoft.iotoperations/instances - microsoft.networkcloud/baremetalmachines -> [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.8) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.7...v0.8) + +> [!div class="nextstepaction"] +> [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.8) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.7...v0.8) +
@@ -1002,7 +1032,12 @@ _**Breaking change**_ - microsoft.healthdataaiservices/deidservices - microsoft.insights/datacollectionrules -> [!div class="nextstepaction"] > [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.7) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.6...v0.7) + +> [!div class="nextstepaction"] +> [Download](https://github.com/microsoft/finops-toolkit/releases/tag/v0.7) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.6...v0.7) +
@@ -1152,7 +1187,12 @@ _Released October 2, 2024_ - microsoft.sql/longtermretentionservers - microsoft.verifiedid/authorities -> [!div class="nextstepaction"] > [Download v0.6](https://github.com/microsoft/finops-toolkit/releases/tag/v0.6) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.5...v0.6) + +> [!div class="nextstepaction"] +> [Download v0.6](https://github.com/microsoft/finops-toolkit/releases/tag/v0.6) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.5...v0.6) +
@@ -1421,7 +1461,12 @@ _Released September 1, 2024_ - Move Microsoft Defender for Endpoint from the **Multicloud** service category to **Security**. - Move StorSimple from the **Multicloud** service category to **Storage**. -> [!div class="nextstepaction"] > [Download v0.5](https://github.com/microsoft/finops-toolkit/releases/tag/v0.5) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.4...v0.5) + +> [!div class="nextstepaction"] +> [Download v0.5](https://github.com/microsoft/finops-toolkit/releases/tag/v0.5) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.4...v0.5) +
@@ -1543,7 +1588,12 @@ _Released July 12, 2024_ - Changed the primary columns in the [Regions](open-data.md#regions) and [Services](open-data.md#services) open data files to be lowercase. - Updated all [sample exports](open-data.md#dataset-examples) to use the same date range as the FOCUS 1.0 dataset. -> [!div class="nextstepaction"] > [Download v0.4](https://github.com/microsoft/finops-toolkit/releases/tag/v0.4) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.3...v0.4) + +> [!div class="nextstepaction"] +> [Download v0.4](https://github.com/microsoft/finops-toolkit/releases/tag/v0.4) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.3...v0.4) +
@@ -1631,7 +1681,12 @@ _Released March 28, 2024_ - Added ServiceModel and Environment columns to the [services](open-data.md#services) data ([#585](https://github.com/microsoft/finops-toolkit/issues/585)). - New and updated [resource types](open-data.md#resource-types) and icons. -> [!div class="nextstepaction"] > [Download v0.3](https://github.com/microsoft/finops-toolkit/releases/tag/v0.3) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.2...v0.3) + +> [!div class="nextstepaction"] +> [Download v0.3](https://github.com/microsoft/finops-toolkit/releases/tag/v0.3) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.2...v0.3) +
@@ -1720,7 +1775,12 @@ _**Breaking change**_ - **Added** - [FinOps Open Cost and Usage Specification (FOCUS) details](../focus/what-is-focus.md). -> [!div class="nextstepaction"] > [Download v0.2](https://github.com/microsoft/finops-toolkit/releases/tag/v0.2) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.1.1...v0.2) + +> [!div class="nextstepaction"] +> [Download v0.2](https://github.com/microsoft/finops-toolkit/releases/tag/v0.2) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.1.1...v0.2) +
@@ -1755,7 +1815,12 @@ _Released October 26, 2023_ - [Register-FinOpsHubProviders](powershell/hubs/Register-FinOpsHubProviders.md) - [Remove-FinOpsHub](powershell/hubs/Remove-FinOpsHub.md) -> [!div class="nextstepaction"] > [Download v0.1.1](https://github.com/microsoft/finops-toolkit/releases/tag/v0.1.1) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.1...v0.1.1) + +> [!div class="nextstepaction"] +> [Download v0.1.1](https://github.com/microsoft/finops-toolkit/releases/tag/v0.1.1) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.1...v0.1.1) +
@@ -1803,7 +1868,12 @@ _Released October 22, 2023_ - [Regions](open-data.md#regions) to map historical resource location values in Microsoft Cost Management to standard Azure regions. - [Services](open-data.md#services) to map all resource types to FOCUS service names and categories. -> [!div class="nextstepaction"] > [Download v0.1](https://github.com/microsoft/finops-toolkit/releases/tag/v0.1) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.0.1...v0.1) + +> [!div class="nextstepaction"] +> [Download v0.1](https://github.com/microsoft/finops-toolkit/releases/tag/v0.1) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/v0.0.1...v0.1) +
@@ -1828,7 +1898,12 @@ _Released May 27, 2023_ - **Added** - [Cost optimization workbook](workbooks/optimization.md) to centralize cost optimization. -> [!div class="nextstepaction"] > [Download v0.0.1](https://github.com/microsoft/finops-toolkit/releases/tag/v0.0.1) > [!div class="nextstepaction"] > [Full changelog](https://github.com/microsoft/finops-toolkit/compare/878e4864ca785db4fc13bdd2ec3a6a00058688c3...v0.0.1) + +> [!div class="nextstepaction"] +> [Download v0.0.1](https://github.com/microsoft/finops-toolkit/releases/tag/v0.0.1) +> [!div class="nextstepaction"] +> [Full changelog](https://github.com/microsoft/finops-toolkit/compare/878e4864ca785db4fc13bdd2ec3a6a00058688c3...v0.0.1) +
@@ -1836,10 +1911,16 @@ _Released May 27, 2023_ Let us know how we're doing with a quick review. We use these reviews to improve and expand FinOps tools and resources. -> [!div class="nextstepaction"] > [Give feedback](https://portal.azure.com/#view/HubsExtension/InProductFeedbackBlade/extensionName/FinOpsToolkit/cesQuestion/How%20easy%20or%20hard%20is%20it%20to%20use%20FinOps%20toolkit%20tools%20and%20resources%3F/cvaQuestion/How%20valuable%20is%20the%20FinOps%20toolkit%3F/surveyId/FTK/bladeName/Toolkit/featureName/Changelog) + +> [!div class="nextstepaction"] +> [Give feedback](https://portal.azure.com/#view/HubsExtension/InProductFeedbackBlade/extensionName/FinOpsToolkit/cesQuestion/How%20easy%20or%20hard%20is%20it%20to%20use%20FinOps%20toolkit%20tools%20and%20resources%3F/cvaQuestion/How%20valuable%20is%20the%20FinOps%20toolkit%3F/surveyId/FTK/bladeName/Toolkit/featureName/Changelog) + If you're looking for something specific, vote for an existing or create a new idea. Share ideas with others to get more votes. We focus on ideas with the most votes. -> [!div class="nextstepaction"] > [Vote on or suggest ideas](https://github.com/microsoft/finops-toolkit/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%252B1-desc) + +> [!div class="nextstepaction"] +> [Vote on or suggest ideas](https://github.com/microsoft/finops-toolkit/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%252B1-desc) +
diff --git a/docs-mslearn/toolkit/workbooks/finops-workbooks-overview.md b/docs-mslearn/toolkit/workbooks/finops-workbooks-overview.md index 7d675f71b..3bc2522f4 100644 --- a/docs-mslearn/toolkit/workbooks/finops-workbooks-overview.md +++ b/docs-mslearn/toolkit/workbooks/finops-workbooks-overview.md @@ -12,6 +12,7 @@ ms.reviewer: arclares --- + # FinOps workbooks FinOps workbooks are Azure Monitor workbooks that provide a series of tools to help engineers perform targeted FinOps capabilities, modeled after the Well-Architected Framework guidance. @@ -47,10 +48,10 @@ This template includes the following workbooks: --> **Option 2:** Import JSON files directly (works with Reader access) - + 1. Download FinOps workbooks from the [latest release](https://aka.ms/ftk/latest). 2. Navigate to [Azure Monitor Workbooks](https://portal.azure.com/#view/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/~/workbooks) in the Azure portal - 3. Click on **+ New** and select **Advanced editory** + 3. Click on **+ New** and select **Advanced editor** 4. Copy the text from the desired workbook.json from the downloaded ZIP file, paste it into the editor, and select **Apply**. 5. Select **Done editing** to view the workbook. 6. Repeat steps 3-5 for each workbook. @@ -61,13 +62,17 @@ This template includes the following workbooks: Let us know how we're doing with a quick review. We use these reviews to improve and expand FinOps tools and resources. + > [!div class="nextstepaction"] > [Give feedback](https://portal.azure.com/#view/HubsExtension/InProductFeedbackBlade/extensionName/FinOpsToolkit/cesQuestion/How%20easy%20or%20hard%20is%20it%20to%20use%20FinOps%20workbooks%3F/cvaQuestion/How%20valuable%20are%20FinOps%20workbooks%3F/surveyId/FTK/bladeName/Workbooks/featureName/Overview) + If you're looking for something specific, vote for an existing or create a new idea. Share ideas with others to get more votes. We focus on ideas with the most votes. + > [!div class="nextstepaction"] > [Vote on or suggest ideas](https://github.com/microsoft/finops-toolkit/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22Tool%3A%20Workbooks%22%20sort%3A"reactions-%2B1-desc") +
diff --git a/docs/README.md b/docs/README.md index 0a847bb04..067d07d7e 100644 --- a/docs/README.md +++ b/docs/README.md @@ -76,7 +76,7 @@ Automate and extend the Microsoft Cloud with starter kits, scripts, and advanced -Download FinOps toolkit {% include ftkver.txt %} +Download FinOps toolkit {% include ftktag.txt %} 💜 Give feedback
diff --git a/docs/_includes/ftktag.txt b/docs/_includes/ftktag.txt index ca7bf83ac..817ba4bff 100644 --- a/docs/_includes/ftktag.txt +++ b/docs/_includes/ftktag.txt @@ -1 +1 @@ -13 \ No newline at end of file +v13 \ No newline at end of file diff --git a/docs/deploy/optimization-engine/0.10/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.10/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 index b3be2d52f..14faedd88 100644 --- a/docs/deploy/optimization-engine/0.10/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.10/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 @@ -107,7 +107,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -261,7 +261,7 @@ foreach ($blob in $unprocessedBlobs) { if ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" } else { diff --git a/docs/deploy/optimization-engine/0.10/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.10/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 index f6ff02274..81a0b5751 100644 --- a/docs/deploy/optimization-engine/0.10/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.10/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 @@ -93,7 +93,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -277,7 +277,7 @@ foreach ($blob in $unprocessedBlobs) { $jsonObject = ConvertTo-Json -InputObject $jsonObjectSplitted[$j] $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" $linesProcessed += $currentObjectLines if ($j -eq ($jsonObjectSplitted.Count - 1)) { $lastProcessedLine = -1 diff --git a/docs/deploy/optimization-engine/0.10/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.10/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 index b37ea56da..93405ffe8 100644 --- a/docs/deploy/optimization-engine/0.10/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.10/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 @@ -82,7 +82,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -241,7 +241,7 @@ $logname = $lognamePrefix + $LogAnalyticsSuffix $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($filtersJson)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" } Else { Write-Warning "Failed to upload $($filterObjects.Count) $LogAnalyticsSuffix rows. Error code: $res" diff --git a/docs/deploy/optimization-engine/0.11/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.11/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 index b3be2d52f..14faedd88 100644 --- a/docs/deploy/optimization-engine/0.11/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.11/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 @@ -107,7 +107,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -261,7 +261,7 @@ foreach ($blob in $unprocessedBlobs) { if ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" } else { diff --git a/docs/deploy/optimization-engine/0.11/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.11/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 index f6ff02274..81a0b5751 100644 --- a/docs/deploy/optimization-engine/0.11/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.11/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 @@ -93,7 +93,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -277,7 +277,7 @@ foreach ($blob in $unprocessedBlobs) { $jsonObject = ConvertTo-Json -InputObject $jsonObjectSplitted[$j] $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" $linesProcessed += $currentObjectLines if ($j -eq ($jsonObjectSplitted.Count - 1)) { $lastProcessedLine = -1 diff --git a/docs/deploy/optimization-engine/0.11/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.11/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 index b37ea56da..93405ffe8 100644 --- a/docs/deploy/optimization-engine/0.11/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.11/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 @@ -82,7 +82,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -241,7 +241,7 @@ $logname = $lognamePrefix + $LogAnalyticsSuffix $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($filtersJson)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" } Else { Write-Warning "Failed to upload $($filterObjects.Count) $LogAnalyticsSuffix rows. Error code: $res" diff --git a/docs/deploy/optimization-engine/0.12/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.12/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 index b3be2d52f..14faedd88 100644 --- a/docs/deploy/optimization-engine/0.12/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.12/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 @@ -107,7 +107,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -261,7 +261,7 @@ foreach ($blob in $unprocessedBlobs) { if ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" } else { diff --git a/docs/deploy/optimization-engine/0.12/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.12/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 index f6ff02274..81a0b5751 100644 --- a/docs/deploy/optimization-engine/0.12/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.12/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 @@ -93,7 +93,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -277,7 +277,7 @@ foreach ($blob in $unprocessedBlobs) { $jsonObject = ConvertTo-Json -InputObject $jsonObjectSplitted[$j] $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" $linesProcessed += $currentObjectLines if ($j -eq ($jsonObjectSplitted.Count - 1)) { $lastProcessedLine = -1 diff --git a/docs/deploy/optimization-engine/0.12/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.12/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 index b37ea56da..93405ffe8 100644 --- a/docs/deploy/optimization-engine/0.12/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.12/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 @@ -82,7 +82,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -241,7 +241,7 @@ $logname = $lognamePrefix + $LogAnalyticsSuffix $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($filtersJson)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" } Else { Write-Warning "Failed to upload $($filterObjects.Count) $LogAnalyticsSuffix rows. Error code: $res" diff --git a/docs/deploy/optimization-engine/0.2.1-rc.2/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.2.1-rc.2/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 index e45dac978..d27fa0d10 100644 --- a/docs/deploy/optimization-engine/0.2.1-rc.2/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.2.1-rc.2/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 @@ -110,7 +110,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -256,7 +256,7 @@ foreach ($blob in $unprocessedBlobs) { $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment if ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" if ($r.Peek() -lt 0) { $lastProcessedLine = -1 } diff --git a/docs/deploy/optimization-engine/0.2.1-rc.2/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.2.1-rc.2/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 index 391ed24c9..a0a6b3bb9 100644 --- a/docs/deploy/optimization-engine/0.2.1-rc.2/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.2.1-rc.2/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 @@ -96,7 +96,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -275,7 +275,7 @@ foreach ($blob in $unprocessedBlobs) { $jsonObject = ConvertTo-Json -InputObject $jsonObjectSplitted[$j] $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" $linesProcessed += $currentObjectLines if ($j -eq ($jsonObjectSplitted.Count - 1)) { $lastProcessedLine = -1 diff --git a/docs/deploy/optimization-engine/0.2.1-rc.2/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.2.1-rc.2/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 index 2ebbe91fb..684bbd3d5 100644 --- a/docs/deploy/optimization-engine/0.2.1-rc.2/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.2.1-rc.2/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 @@ -75,7 +75,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -216,7 +216,7 @@ $logname = $lognamePrefix + $LogAnalyticsSuffix $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($filtersJson)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" } Else { Write-Warning "Failed to upload $($filterObjects.Count) $LogAnalyticsSuffix rows. Error code: $res" diff --git a/docs/deploy/optimization-engine/0.4/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.4/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 index e45dac978..d27fa0d10 100644 --- a/docs/deploy/optimization-engine/0.4/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.4/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 @@ -110,7 +110,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -256,7 +256,7 @@ foreach ($blob in $unprocessedBlobs) { $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment if ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" if ($r.Peek() -lt 0) { $lastProcessedLine = -1 } diff --git a/docs/deploy/optimization-engine/0.4/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.4/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 index 391ed24c9..a0a6b3bb9 100644 --- a/docs/deploy/optimization-engine/0.4/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.4/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 @@ -96,7 +96,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -275,7 +275,7 @@ foreach ($blob in $unprocessedBlobs) { $jsonObject = ConvertTo-Json -InputObject $jsonObjectSplitted[$j] $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" $linesProcessed += $currentObjectLines if ($j -eq ($jsonObjectSplitted.Count - 1)) { $lastProcessedLine = -1 diff --git a/docs/deploy/optimization-engine/0.4/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.4/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 index 2ebbe91fb..684bbd3d5 100644 --- a/docs/deploy/optimization-engine/0.4/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.4/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 @@ -75,7 +75,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -216,7 +216,7 @@ $logname = $lognamePrefix + $LogAnalyticsSuffix $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($filtersJson)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" } Else { Write-Warning "Failed to upload $($filterObjects.Count) $LogAnalyticsSuffix rows. Error code: $res" diff --git a/docs/deploy/optimization-engine/0.5/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.5/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 index bab858192..af6ad3bd6 100644 --- a/docs/deploy/optimization-engine/0.5/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.5/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 @@ -107,7 +107,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -258,7 +258,7 @@ foreach ($blob in $unprocessedBlobs) { $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment if ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" if ($r.Peek() -lt 0) { $lastProcessedLine = -1 } diff --git a/docs/deploy/optimization-engine/0.5/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.5/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 index 075c57b85..54dc332f3 100644 --- a/docs/deploy/optimization-engine/0.5/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.5/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 @@ -93,7 +93,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -277,7 +277,7 @@ foreach ($blob in $unprocessedBlobs) { $jsonObject = ConvertTo-Json -InputObject $jsonObjectSplitted[$j] $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" $linesProcessed += $currentObjectLines if ($j -eq ($jsonObjectSplitted.Count - 1)) { $lastProcessedLine = -1 diff --git a/docs/deploy/optimization-engine/0.5/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.5/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 index 84725c4fc..4f1a25d7c 100644 --- a/docs/deploy/optimization-engine/0.5/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.5/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 @@ -82,7 +82,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -241,7 +241,7 @@ $logname = $lognamePrefix + $LogAnalyticsSuffix $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($filtersJson)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" } Else { Write-Warning "Failed to upload $($filterObjects.Count) $LogAnalyticsSuffix rows. Error code: $res" diff --git a/docs/deploy/optimization-engine/0.6/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.6/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 index dc6a6206f..b1c9ba5be 100644 --- a/docs/deploy/optimization-engine/0.6/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.6/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 @@ -107,7 +107,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -258,7 +258,7 @@ foreach ($blob in $unprocessedBlobs) { $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment if ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" if ($r.Peek() -lt 0) { $lastProcessedLine = -1 } diff --git a/docs/deploy/optimization-engine/0.6/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.6/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 index 4580d5902..94dae2480 100644 --- a/docs/deploy/optimization-engine/0.6/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.6/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 @@ -93,7 +93,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -277,7 +277,7 @@ foreach ($blob in $unprocessedBlobs) { $jsonObject = ConvertTo-Json -InputObject $jsonObjectSplitted[$j] $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" $linesProcessed += $currentObjectLines if ($j -eq ($jsonObjectSplitted.Count - 1)) { $lastProcessedLine = -1 diff --git a/docs/deploy/optimization-engine/0.6/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.6/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 index 84725c4fc..4f1a25d7c 100644 --- a/docs/deploy/optimization-engine/0.6/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.6/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 @@ -82,7 +82,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -241,7 +241,7 @@ $logname = $lognamePrefix + $LogAnalyticsSuffix $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($filtersJson)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" } Else { Write-Warning "Failed to upload $($filterObjects.Count) $LogAnalyticsSuffix rows. Error code: $res" diff --git a/docs/deploy/optimization-engine/0.7/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.7/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 index a162e9931..e4ddbfc05 100644 --- a/docs/deploy/optimization-engine/0.7/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.7/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 @@ -107,7 +107,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -261,7 +261,7 @@ foreach ($blob in $unprocessedBlobs) { if ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" } else { diff --git a/docs/deploy/optimization-engine/0.7/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.7/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 index 4580d5902..94dae2480 100644 --- a/docs/deploy/optimization-engine/0.7/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.7/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 @@ -93,7 +93,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -277,7 +277,7 @@ foreach ($blob in $unprocessedBlobs) { $jsonObject = ConvertTo-Json -InputObject $jsonObjectSplitted[$j] $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" $linesProcessed += $currentObjectLines if ($j -eq ($jsonObjectSplitted.Count - 1)) { $lastProcessedLine = -1 diff --git a/docs/deploy/optimization-engine/0.7/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.7/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 index 84725c4fc..4f1a25d7c 100644 --- a/docs/deploy/optimization-engine/0.7/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.7/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 @@ -82,7 +82,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -241,7 +241,7 @@ $logname = $lognamePrefix + $LogAnalyticsSuffix $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($filtersJson)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" } Else { Write-Warning "Failed to upload $($filterObjects.Count) $LogAnalyticsSuffix rows. Error code: $res" diff --git a/docs/deploy/optimization-engine/0.8/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.8/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 index a162e9931..e4ddbfc05 100644 --- a/docs/deploy/optimization-engine/0.8/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.8/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 @@ -107,7 +107,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -261,7 +261,7 @@ foreach ($blob in $unprocessedBlobs) { if ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" } else { diff --git a/docs/deploy/optimization-engine/0.8/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.8/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 index 4580d5902..94dae2480 100644 --- a/docs/deploy/optimization-engine/0.8/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.8/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 @@ -93,7 +93,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -277,7 +277,7 @@ foreach ($blob in $unprocessedBlobs) { $jsonObject = ConvertTo-Json -InputObject $jsonObjectSplitted[$j] $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" $linesProcessed += $currentObjectLines if ($j -eq ($jsonObjectSplitted.Count - 1)) { $lastProcessedLine = -1 diff --git a/docs/deploy/optimization-engine/0.8/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.8/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 index 84725c4fc..4f1a25d7c 100644 --- a/docs/deploy/optimization-engine/0.8/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.8/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 @@ -82,7 +82,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -241,7 +241,7 @@ $logname = $lognamePrefix + $LogAnalyticsSuffix $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($filtersJson)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" } Else { Write-Warning "Failed to upload $($filterObjects.Count) $LogAnalyticsSuffix rows. Error code: $res" diff --git a/docs/deploy/optimization-engine/0.9/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.9/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 index b3be2d52f..14faedd88 100644 --- a/docs/deploy/optimization-engine/0.9/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.9/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 @@ -107,7 +107,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -261,7 +261,7 @@ foreach ($blob in $unprocessedBlobs) { if ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" } else { diff --git a/docs/deploy/optimization-engine/0.9/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.9/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 index f6ff02274..81a0b5751 100644 --- a/docs/deploy/optimization-engine/0.9/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.9/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 @@ -93,7 +93,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -277,7 +277,7 @@ foreach ($blob in $unprocessedBlobs) { $jsonObject = ConvertTo-Json -InputObject $jsonObjectSplitted[$j] $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" $linesProcessed += $currentObjectLines if ($j -eq ($jsonObjectSplitted.Count - 1)) { $lastProcessedLine = -1 diff --git a/docs/deploy/optimization-engine/0.9/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/0.9/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 index b37ea56da..93405ffe8 100644 --- a/docs/deploy/optimization-engine/0.9/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/0.9/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 @@ -82,7 +82,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -241,7 +241,7 @@ $logname = $lognamePrefix + $LogAnalyticsSuffix $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($filtersJson)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" } Else { Write-Warning "Failed to upload $($filterObjects.Count) $LogAnalyticsSuffix rows. Error code: $res" diff --git a/docs/deploy/optimization-engine/12.0/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/12.0/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 index b3be2d52f..14faedd88 100644 --- a/docs/deploy/optimization-engine/12.0/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/12.0/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 @@ -107,7 +107,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -261,7 +261,7 @@ foreach ($blob in $unprocessedBlobs) { if ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" } else { diff --git a/docs/deploy/optimization-engine/12.0/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/12.0/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 index f6ff02274..81a0b5751 100644 --- a/docs/deploy/optimization-engine/12.0/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/12.0/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 @@ -93,7 +93,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -277,7 +277,7 @@ foreach ($blob in $unprocessedBlobs) { $jsonObject = ConvertTo-Json -InputObject $jsonObjectSplitted[$j] $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" $linesProcessed += $currentObjectLines if ($j -eq ($jsonObjectSplitted.Count - 1)) { $lastProcessedLine = -1 diff --git a/docs/deploy/optimization-engine/12.0/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/12.0/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 index b37ea56da..93405ffe8 100644 --- a/docs/deploy/optimization-engine/12.0/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/12.0/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 @@ -82,7 +82,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -241,7 +241,7 @@ $logname = $lognamePrefix + $LogAnalyticsSuffix $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($filtersJson)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" } Else { Write-Warning "Failed to upload $($filterObjects.Count) $LogAnalyticsSuffix rows. Error code: $res" diff --git a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 index b3be2d52f..14faedd88 100644 --- a/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/13.0/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 @@ -107,7 +107,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -261,7 +261,7 @@ foreach ($blob in $unprocessedBlobs) { if ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" } else { diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 index f6ff02274..81a0b5751 100644 --- a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 @@ -93,7 +93,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -277,7 +277,7 @@ foreach ($blob in $unprocessedBlobs) { $jsonObject = ConvertTo-Json -InputObject $jsonObjectSplitted[$j] $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" $linesProcessed += $currentObjectLines if ($j -eq ($jsonObjectSplitted.Count - 1)) { $lastProcessedLine = -1 diff --git a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 index b37ea56da..93405ffe8 100644 --- a/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/13.0/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 @@ -82,7 +82,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -241,7 +241,7 @@ $logname = $lognamePrefix + $LogAnalyticsSuffix $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($filtersJson)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" } Else { Write-Warning "Failed to upload $($filterObjects.Count) $LogAnalyticsSuffix rows. Error code: $res" diff --git a/docs/deploy/optimization-engine/latest/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/latest/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 index b3be2d52f..14faedd88 100644 --- a/docs/deploy/optimization-engine/latest/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/latest/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 @@ -107,7 +107,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -261,7 +261,7 @@ foreach ($blob in $unprocessedBlobs) { if ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" } else { diff --git a/docs/deploy/optimization-engine/latest/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/latest/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 index f6ff02274..81a0b5751 100644 --- a/docs/deploy/optimization-engine/latest/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/latest/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 @@ -93,7 +93,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -277,7 +277,7 @@ foreach ($blob in $unprocessedBlobs) { $jsonObject = ConvertTo-Json -InputObject $jsonObjectSplitted[$j] $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" $linesProcessed += $currentObjectLines if ($j -eq ($jsonObjectSplitted.Count - 1)) { $lastProcessedLine = -1 diff --git a/docs/deploy/optimization-engine/latest/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 b/docs/deploy/optimization-engine/latest/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 index b37ea56da..93405ffe8 100644 --- a/docs/deploy/optimization-engine/latest/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 +++ b/docs/deploy/optimization-engine/latest/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 @@ -82,7 +82,7 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ @@ -241,7 +241,7 @@ $logname = $lognamePrefix + $LogAnalyticsSuffix $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($filtersJson)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" } Else { Write-Warning "Failed to upload $($filterObjects.Count) $LogAnalyticsSuffix rows. Error code: $res" diff --git a/src/optimization-engine/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 b/src/optimization-engine/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 index b3be2d52f..2fdcb1bdf 100644 --- a/src/optimization-engine/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 +++ b/src/optimization-engine/runbooks/data-collection/Ingest-OptimizationCSVExportsToLogAnalytics.ps1 @@ -53,12 +53,16 @@ $LogAnalyticsIngestControlTable = "LogAnalyticsIngestControl" "Logging in to Azure with $authenticationOption..." -switch ($authenticationOption) { - "UserAssignedManagedIdentity" { +switch ($authenticationOption) +{ + "UserAssignedManagedIdentity" + { Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID break } - Default { #ManagedIdentity + default + { + #ManagedIdentity Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment break } @@ -67,7 +71,8 @@ switch ($authenticationOption) { #region Functions # Function to create the authorization signature -Function Build-OMSSignature ($workspaceId, $sharedKey, $date, $contentLength, $method, $contentType, $resource) { +function Build-OMSSignature ($workspaceId, $sharedKey, $date, $contentLength, $method, $contentType, $resource) +{ $xHeaders = "x-ms-date:" + $date $stringToHash = $method + "`n" + $contentLength + "`n" + $contentType + "`n" + $xHeaders + "`n" + $resource $bytesToHash = [Text.Encoding]::UTF8.GetBytes($stringToHash) @@ -81,7 +86,8 @@ Function Build-OMSSignature ($workspaceId, $sharedKey, $date, $contentLength, $m } # Function to create and post the request -Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField, $AzureEnvironment) { +function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField, $AzureEnvironment) +{ $method = "POST" $contentType = "application/json" $resource = "/api/logs" @@ -107,22 +113,25 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ - "Authorization" = $signature; - "Log-Type" = $logType; - "x-ms-date" = $rfc1123date; - "time-generated-field" = $TimeStampField; + "Authorization" = $signature + "Log-Type" = $logType + "x-ms-date" = $rfc1123date + "time-generated-field" = $TimeStampField } - Try { + try + { $response = Invoke-WebRequest -Uri $uri -Method POST -ContentType $contentType -Headers $OMSheaders -Body $body -UseBasicParsing -TimeoutSec 1000 } - catch { - if ($_.Exception.Response.StatusCode.Value__ -eq 401) { + catch + { + if ($_.Exception.Response.StatusCode.Value__ -eq 401) + { "REAUTHENTICATING" $response = Invoke-WebRequest -Uri $uri -Method POST -ContentType $contentType -Headers $OMSheaders -Body $body -UseBasicParsing -TimeoutSec 1000 @@ -153,20 +162,22 @@ do $blobs = Get-AzStorageBlob -Container $storageAccountSinkContainer -MaxCount $StorageBlobsPageSize -ContinuationToken $continuationToken -Context $saCtx | Sort-Object -Property LastModified if ($blobs.Count -le 0) { break } $allblobs += $blobs - $continuationToken = $blobs[$blobs.Count -1].ContinuationToken; + $continuationToken = $blobs[$blobs.Count - 1].ContinuationToken } -While ($null -ne $continuationToken) +while ($null -ne $continuationToken) $tries = 0 $connectionSuccess = $false -do { +do +{ $tries++ - try { + try + { $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") $Conn.AccessToken = $dbToken.Token $Conn.Open() - $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd = New-Object system.Data.SqlClient.SqlCommand $Cmd.Connection = $Conn $Cmd.CommandTimeout = $SqlTimeout $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE StorageContainerName = '$storageAccountSinkContainer'" @@ -177,7 +188,8 @@ do { $sqlAdapter.Fill($controlRows) | Out-Null $connectionSuccess = $true } - catch { + catch + { Write-Output "Failed to contact SQL at try $tries." Write-Output $Error[0] Start-Sleep -Seconds ($tries * 20) @@ -209,11 +221,13 @@ $newProcessedTime = $null $unprocessedBlobs = @() -foreach ($blob in $allblobs) { - $blobLastModified = $blob.LastModified.UtcDateTime.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") +foreach ($blob in $allblobs) +{ + $blobLastModified = $blob.LastModified.UtcDateTime.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") if ($lastProcessedDateTime -lt $blobLastModified -or ` - ($lastProcessedDateTime -eq $blobLastModified -and $lastProcessedLine -gt 0)) { - Write-Output "$($blob.Name) found (modified on $blobLastModified)" + ($lastProcessedDateTime -eq $blobLastModified -and $lastProcessedLine -gt 0)) + { + Write-Output "$($blob.Name) found (modified on $blobLastModified)" $unprocessedBlobs += $blob } } @@ -222,7 +236,8 @@ $unprocessedBlobs = $unprocessedBlobs | Sort-Object -Property LastModified Write-Output "Found $($unprocessedBlobs.Count) new blobs to process..." -foreach ($blob in $unprocessedBlobs) { +foreach ($blob in $unprocessedBlobs) +{ $newProcessedTime = $blob.LastModified.UtcDateTime.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") Write-Output "About to process $($blob.Name) ($($blob.Length) bytes)..." $blobFilePath = "$env:TEMP\$($blob.Name)" @@ -261,7 +276,7 @@ foreach ($blob in $unprocessedBlobs) { if ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" + Write-Output "Successfully uploaded $lineCounter $LogAnalyticsSuffix rows to Log Analytics" } else { @@ -276,16 +291,19 @@ foreach ($blob in $unprocessedBlobs) { Write-Warning "Skipped uploading $lineCounter $LogAnalyticsSuffix rows. Null JSON object." } - if ($r.Peek() -lt 0) { + if ($r.Peek() -lt 0) + { $lastProcessedLine = -1 } - else { + else + { $lastProcessedLine = $linesProcessed - 1 } $updatedLastProcessedLine = $lastProcessedLine $updatedLastProcessedDateTime = $lastProcessedDateTime - if ($r.Peek() -lt 0) { + if ($r.Peek() -lt 0) + { $updatedLastProcessedDateTime = $newProcessedTime } $lastProcessedDateTime = $updatedLastProcessedDateTime @@ -295,7 +313,7 @@ foreach ($blob in $unprocessedBlobs) { $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") $Conn.AccessToken = $dbToken.Token $Conn.Open() - $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd = New-Object system.Data.SqlClient.SqlCommand $Cmd.Connection = $Conn $Cmd.CommandText = $sqlStatement $Cmd.CommandTimeout = $SqlTimeout @@ -325,7 +343,7 @@ foreach ($blob in $unprocessedBlobs) { $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") $Conn.AccessToken = $dbToken.Token $Conn.Open() - $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd = New-Object system.Data.SqlClient.SqlCommand $Cmd.Connection = $Conn $Cmd.CommandText = $sqlStatement $Cmd.CommandTimeout = $SqlTimeout diff --git a/src/optimization-engine/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 b/src/optimization-engine/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 index f6ff02274..a28957e0c 100644 --- a/src/optimization-engine/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 +++ b/src/optimization-engine/runbooks/recommendations/Ingest-RecommendationsToLogAnalytics.ps1 @@ -41,7 +41,8 @@ $storageAccountSink = Get-AutomationVariable -Name "AzureOptimization_StorageSi $storageAccountSinkContainer = Get-AutomationVariable -Name "AzureOptimization_RecommendationsContainer" -ErrorAction SilentlyContinue -if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) { +if ([string]::IsNullOrEmpty($storageAccountSinkContainer)) +{ $storageAccountSinkContainer = "recommendationsexports" } $StorageBlobsPageSize = [int] (Get-AutomationVariable -Name "AzureOptimization_StorageBlobsPageSize" -ErrorAction SilentlyContinue) @@ -53,7 +54,8 @@ if (-not($StorageBlobsPageSize -gt 0)) #region Functions # Function to create the authorization signature -Function Build-OMSSignature ($workspaceId, $sharedKey, $date, $contentLength, $method, $contentType, $resource) { +function Build-OMSSignature ($workspaceId, $sharedKey, $date, $contentLength, $method, $contentType, $resource) +{ $xHeaders = "x-ms-date:" + $date $stringToHash = $method + "`n" + $contentLength + "`n" + $contentType + "`n" + $xHeaders + "`n" + $resource $bytesToHash = [Text.Encoding]::UTF8.GetBytes($stringToHash) @@ -67,7 +69,8 @@ Function Build-OMSSignature ($workspaceId, $sharedKey, $date, $contentLength, $m } # Function to create and post the request -Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField, $AzureEnvironment) { +function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField, $AzureEnvironment) +{ $method = "POST" $contentType = "application/json" $resource = "/api/logs" @@ -93,22 +96,25 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ - "Authorization" = $signature; - "Log-Type" = $logType; - "x-ms-date" = $rfc1123date; - "time-generated-field" = $TimeStampField; + "Authorization" = $signature + "Log-Type" = $logType + "x-ms-date" = $rfc1123date + "time-generated-field" = $TimeStampField } - Try { + try + { $response = Invoke-WebRequest -Uri $uri -Method POST -ContentType $contentType -Headers $OMSheaders -Body $body -UseBasicParsing -TimeoutSec 1000 } - catch { - if ($_.Exception.Response.StatusCode.Value__ -eq 401) { + catch + { + if ($_.Exception.Response.StatusCode.Value__ -eq 401) + { "REAUTHENTICATING" $response = Invoke-WebRequest -Uri $uri -Method POST -ContentType $contentType -Headers $OMSheaders -Body $body -UseBasicParsing -TimeoutSec 1000 @@ -126,12 +132,16 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField "Logging in to Azure with $authenticationOption..." -switch ($authenticationOption) { - "UserAssignedManagedIdentity" { +switch ($authenticationOption) +{ + "UserAssignedManagedIdentity" + { Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID break } - Default { #ManagedIdentity + default + { + #ManagedIdentity Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment break } @@ -154,20 +164,22 @@ do $blobs = Get-AzStorageBlob -Container $storageAccountSinkContainer -MaxCount $StorageBlobsPageSize -ContinuationToken $continuationToken -Context $saCtx | Sort-Object -Property LastModified if ($blobs.Count -le 0) { break } $allblobs += $blobs - $continuationToken = $blobs[$blobs.Count -1].ContinuationToken; + $continuationToken = $blobs[$blobs.Count - 1].ContinuationToken } -While ($null -ne $continuationToken) +while ($null -ne $continuationToken) $tries = 0 $connectionSuccess = $false -do { +do +{ $tries++ - try { + try + { $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") $Conn.AccessToken = $dbToken.Token $Conn.Open() - $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd = New-Object system.Data.SqlClient.SqlCommand $Cmd.Connection = $Conn $Cmd.CommandTimeout = $SqlTimeout $Cmd.CommandText = "SELECT * FROM [dbo].[$LogAnalyticsIngestControlTable] WHERE StorageContainerName = '$storageAccountSinkContainer'" @@ -178,7 +190,8 @@ do { $sqlAdapter.Fill($controlRows) | Out-Null $connectionSuccess = $true } - catch { + catch + { Write-Output "Failed to contact SQL at try $tries." Write-Output $Error[0] Start-Sleep -Seconds ($tries * 20) @@ -210,10 +223,12 @@ $newProcessedTime = $null $unprocessedBlobs = @() -foreach ($blob in $allblobs) { +foreach ($blob in $allblobs) +{ $blobLastModified = $blob.LastModified.UtcDateTime.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") if ($lastProcessedDateTime -lt $blobLastModified -or ` - ($lastProcessedDateTime -eq $blobLastModified -and $lastProcessedLine -gt 0)) { + ($lastProcessedDateTime -eq $blobLastModified -and $lastProcessedLine -gt 0)) + { Write-Output "$($blob.Name) found (modified on $blobLastModified)" $unprocessedBlobs += $blob } @@ -223,7 +238,8 @@ $unprocessedBlobs = $unprocessedBlobs | Sort-Object -Property LastModified Write-Output "Found $($unprocessedBlobs.Count) new blobs to process..." -foreach ($blob in $unprocessedBlobs) { +foreach ($blob in $unprocessedBlobs) +{ $newProcessedTime = $blob.LastModified.UtcDateTime.ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'") Write-Output "About to process $($blob.Name)..." Get-AzStorageBlobContent -CloudBlob $blob.ICloudBlob -Context $saCtx -Force @@ -248,8 +264,9 @@ foreach ($blob in $unprocessedBlobs) { if ($recCount -gt 1) { - for ($i = 0; $i -lt $recCount; $i += $LogAnalyticsChunkSize) { - $jsonObjectSplitted += , @($jsonObject[$i..($i + ($LogAnalyticsChunkSize - 1))]); + for ($i = 0; $i -lt $recCount; $i += $LogAnalyticsChunkSize) + { + $jsonObjectSplitted += , @($jsonObject[$i..($i + ($LogAnalyticsChunkSize - 1))]) } } else @@ -276,19 +293,23 @@ foreach ($blob in $unprocessedBlobs) { $jsonObject = ConvertTo-Json -InputObject $jsonObjectSplitted[$j] $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($jsonObject)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment - If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" + if ($res -ge 200 -and $res -lt 300) + { + Write-Output "Successfully uploaded $currentObjectLines $LogAnalyticsSuffix rows to Log Analytics" $linesProcessed += $currentObjectLines - if ($j -eq ($jsonObjectSplitted.Count - 1)) { + if ($j -eq ($jsonObjectSplitted.Count - 1)) + { $lastProcessedLine = -1 } - else { + else + { $lastProcessedLine = $linesProcessed - 1 } $updatedLastProcessedLine = $lastProcessedLine $updatedLastProcessedDateTime = $lastProcessedDateTime - if ($j -eq ($jsonObjectSplitted.Count - 1)) { + if ($j -eq ($jsonObjectSplitted.Count - 1)) + { $updatedLastProcessedDateTime = $newProcessedTime } $lastProcessedDateTime = $updatedLastProcessedDateTime @@ -298,7 +319,7 @@ foreach ($blob in $unprocessedBlobs) { $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") $Conn.AccessToken = $dbToken.Token $Conn.Open() - $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd = New-Object system.Data.SqlClient.SqlCommand $Cmd.Connection = $Conn $Cmd.CommandText = $sqlStatement $Cmd.CommandTimeout = $SqlTimeout @@ -306,7 +327,8 @@ foreach ($blob in $unprocessedBlobs) { $Conn.Close() $Conn.Dispose() } - Else { + else + { $linesProcessed += $currentObjectLines Write-Warning "Failed to upload $currentObjectLines $LogAnalyticsSuffix rows. Error code: $res" throw diff --git a/src/optimization-engine/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 b/src/optimization-engine/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 index b37ea56da..db867ee93 100644 --- a/src/optimization-engine/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 +++ b/src/optimization-engine/runbooks/recommendations/Ingest-SuppressionsToLogAnalytics.ps1 @@ -42,7 +42,8 @@ $FiltersTable = "Filters" #region Functions # Function to create the authorization signature -Function Build-OMSSignature ($workspaceId, $sharedKey, $date, $contentLength, $method, $contentType, $resource) { +function Build-OMSSignature ($workspaceId, $sharedKey, $date, $contentLength, $method, $contentType, $resource) +{ $xHeaders = "x-ms-date:" + $date $stringToHash = $method + "`n" + $contentLength + "`n" + $contentType + "`n" + $xHeaders + "`n" + $resource $bytesToHash = [Text.Encoding]::UTF8.GetBytes($stringToHash) @@ -56,7 +57,8 @@ Function Build-OMSSignature ($workspaceId, $sharedKey, $date, $contentLength, $m } # Function to create and post the request -Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField, $AzureEnvironment) { +function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField, $AzureEnvironment) +{ $method = "POST" $contentType = "application/json" $resource = "/api/logs" @@ -82,22 +84,25 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField } if ($AzureEnvironment -eq "AzureGermanCloud") { - throw "Azure Germany isn't suported for the Log Analytics Data Collector API" + throw "Azure Germany isn't supported for the Log Analytics Data Collector API" } $OMSheaders = @{ - "Authorization" = $signature; - "Log-Type" = $logType; - "x-ms-date" = $rfc1123date; - "time-generated-field" = $TimeStampField; + "Authorization" = $signature + "Log-Type" = $logType + "x-ms-date" = $rfc1123date + "time-generated-field" = $TimeStampField } - Try { + try + { $response = Invoke-WebRequest -Uri $uri -Method POST -ContentType $contentType -Headers $OMSheaders -Body $body -UseBasicParsing -TimeoutSec 1000 } - catch { - if ($_.Exception.Response.StatusCode.Value__ -eq 401) { + catch + { + if ($_.Exception.Response.StatusCode.Value__ -eq 401) + { "REAUTHENTICATING" $response = Invoke-WebRequest -Uri $uri -Method POST -ContentType $contentType -Headers $OMSheaders -Body $body -UseBasicParsing -TimeoutSec 1000 @@ -114,12 +119,16 @@ Function Post-OMSData($workspaceId, $sharedKey, $body, $logType, $TimeStampField "Logging in to Azure with $authenticationOption..." -switch ($authenticationOption) { - "UserAssignedManagedIdentity" { +switch ($authenticationOption) +{ + "UserAssignedManagedIdentity" + { Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment -AccountId $uamiClientID break } - Default { #ManagedIdentity + default + { + #ManagedIdentity Connect-AzAccount -Identity -EnvironmentName $cloudEnvironment break } @@ -132,14 +141,16 @@ Write-Output "Getting excluded recommendation sub-type IDs..." $tries = 0 $connectionSuccess = $false -do { +do +{ $tries++ - try { + try + { $dbToken = Get-AzAccessToken -ResourceUrl "https://$azureSqlDomain/" $Conn = New-Object System.Data.SqlClient.SqlConnection("Server=tcp:$sqlserver,1433;Database=$sqldatabase;Encrypt=True;Connection Timeout=$SqlTimeout;") $Conn.AccessToken = $dbToken.Token $Conn.Open() - $Cmd=new-object system.Data.SqlClient.SqlCommand + $Cmd = New-Object system.Data.SqlClient.SqlCommand $Cmd.Connection = $Conn $Cmd.CommandTimeout = $SqlTimeout $Cmd.CommandText = "SELECT * FROM [dbo].[$FiltersTable] WHERE IsEnabled = 1 AND (FilterEndDate IS NULL OR FilterEndDate > GETDATE())" @@ -150,7 +161,8 @@ do { $sqlAdapter.Fill($filters) | Out-Null $connectionSuccess = $true } - catch { + catch + { Write-Output "Failed to contact SQL at try $tries." Write-Output $Error[0] Start-Sleep -Seconds ($tries * 20) @@ -165,22 +177,22 @@ if (-not($connectionSuccess)) $Conn.Close() $Conn.Dispose() -$datetime = (get-date).ToUniversalTime() +$datetime = (Get-Date).ToUniversalTime() $timestamp = $datetime.ToString("yyyy-MM-ddTHH:mm:00.000Z") $filterObjects = @() $filterObject = New-Object PSObject -Property @{ - Timestamp = $timestamp - FilterId = (New-Guid).Guid + Timestamp = $timestamp + FilterId = (New-Guid).Guid RecommendationSubTypeId = [System.Guid]::empty.Guid - FilterType = "Dummy" - InstanceId = [System.Guid]::empty.Guid - InstanceName = "Dummy" - FilterStartDate = "2019-01-01T00:00:00.000Z" - FilterEndDate = "2199-12-31T23:59:59.000Z" - Author = "AOE" - Notes = "This is a dummy suppression required to build the full suppressions schema in Log Analytics" + FilterType = "Dummy" + InstanceId = [System.Guid]::empty.Guid + InstanceName = "Dummy" + FilterStartDate = "2019-01-01T00:00:00.000Z" + FilterEndDate = "2199-12-31T23:59:59.000Z" + Author = "AOE" + Notes = "This is a dummy suppression required to build the full suppressions schema in Log Analytics" } $filterObjects += $filterObject @@ -220,16 +232,16 @@ foreach ($filter in $filters) } $filterObject = New-Object PSObject -Property @{ - Timestamp = $timestamp - FilterId = $filter.FilterId + Timestamp = $timestamp + FilterId = $filter.FilterId RecommendationSubTypeId = $filter.RecommendationSubTypeId - FilterType = $filter.FilterType - InstanceId = $instanceId - InstanceName = $instanceName - FilterStartDate = $filterStartDate - FilterEndDate = $filterEndDate - Author = $filter.Author - Notes = $filter.Notes + FilterType = $filter.FilterType + InstanceId = $instanceId + InstanceName = $instanceName + FilterStartDate = $filterStartDate + FilterEndDate = $filterEndDate + Author = $filter.Author + Notes = $filter.Notes } $filterObjects += $filterObject } @@ -240,10 +252,12 @@ $LogAnalyticsSuffix = "SuppressionsV1" $logname = $lognamePrefix + $LogAnalyticsSuffix $res = Post-OMSData -workspaceId $workspaceId -sharedKey $sharedKey -body ([System.Text.Encoding]::UTF8.GetBytes($filtersJson)) -logType $logname -TimeStampField "Timestamp" -AzureEnvironment $cloudEnvironment -If ($res -ge 200 -and $res -lt 300) { - Write-Output "Succesfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" +if ($res -ge 200 -and $res -lt 300) +{ + Write-Output "Successfully uploaded $($filterObjects.Count) $LogAnalyticsSuffix rows to Log Analytics" } -Else { +else +{ Write-Warning "Failed to upload $($filterObjects.Count) $LogAnalyticsSuffix rows. Error code: $res" throw } diff --git a/src/scripts/Update-Version.ps1 b/src/scripts/Update-Version.ps1 index 1fea7d516..0f4bf0d21 100644 --- a/src/scripts/Update-Version.ps1 +++ b/src/scripts/Update-Version.ps1 @@ -100,9 +100,9 @@ if ($update -or $Version) # Update version in secondary files, if needed if ($update -or $Version) { - # Update version files: ftkver.txt (major.minor) and ftktag.txt (major only, for git tags) + # Update version files: ftkver.txt (major.minor) and ftktag.txt (git tag, e.g., "v13") $repoRoot = (Resolve-Path "$PSScriptRoot/../..").Path - $tag = $ver -replace '\.0$', '' + $tag = 'v' + ($ver -replace '\.0$', '') foreach ($entry in @{ 'ftkver.txt' = $ver; 'ftktag.txt' = $tag }.GetEnumerator()) { Write-Verbose "Updating $($entry.Key) files..." diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep index 6274ea88e..8b35ddec8 100644 --- a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep @@ -127,7 +127,7 @@ var INGESTION_ID_SEPARATOR = '__' var ftkGitTag = loadTextContent('../../fx/ftktag.txt') // cSpell:ignore ftktag var ftkReleaseUri = indexOf(finOpsToolkitVersion, '-dev') != -1 ? 'https://raw.githubusercontent.com/microsoft/finops-toolkit/refs/heads/dev/src/open-data' - : 'https://raw.githubusercontent.com/microsoft/finops-toolkit/refs/tags/v${ftkGitTag}/src/open-data' + : 'https://raw.githubusercontent.com/microsoft/finops-toolkit/refs/tags/${ftkGitTag}/src/open-data' var useFabric = !empty(fabricQueryUri) var useAzure = !useFabric && !empty(clusterName) diff --git a/src/templates/finops-hub/modules/fx/ftktag.txt b/src/templates/finops-hub/modules/fx/ftktag.txt index ca7bf83ac..817ba4bff 100644 --- a/src/templates/finops-hub/modules/fx/ftktag.txt +++ b/src/templates/finops-hub/modules/fx/ftktag.txt @@ -1 +1 @@ -13 \ No newline at end of file +v13 \ No newline at end of file From a070513c1bb6011839674c5e8df4cadb03b93fe9 Mon Sep 17 00:00:00 2001 From: "allcontributors[bot]" <46447321+allcontributors[bot]@users.noreply.github.com> Date: Sat, 31 Jan 2026 13:41:00 -0800 Subject: [PATCH 66/69] add francesco1119 as a contributor for bug (#1962) Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com> --- .all-contributorsrc | 9 +++++++++ README.md | 3 +++ docs/README.md | 3 +++ 3 files changed, 15 insertions(+) diff --git a/.all-contributorsrc b/.all-contributorsrc index 65afc192c..41f24ce4e 100644 --- a/.all-contributorsrc +++ b/.all-contributorsrc @@ -871,6 +871,15 @@ "contributions": [ "code" ] + }, + { + "login": "francesco1119", + "name": "Francesco Mantovani", + "avatar_url": "https://avatars.githubusercontent.com/u/3397477?v=4", + "profile": "http://www.jeeja.biz", + "contributions": [ + "bug" + ] } ], "commitType": "docs", diff --git a/README.md b/README.md index 6cc99c362..e40b9dc31 100644 --- a/README.md +++ b/README.md @@ -144,6 +144,9 @@ There are many ways to participate. From reporting bugs and requesting features + + +
NameResource GroupSubscription
Sander Naus
Sander Naus
💻		 gorkomikus
gorkomikus
💻
Francesco Mantovani
Francesco Mantovani
🐛
diff --git a/docs/README.md b/docs/README.md index 067d07d7e..13fe216d3 100644 --- a/docs/README.md +++ b/docs/README.md @@ -214,6 +214,9 @@ Whether you're looking for a little assistance or are interested in contributing Sander Naus
Sander Naus
💻 gorkomikus
gorkomikus
💻 + + Francesco Mantovani
Francesco Mantovani
🐛 + From f4fd8719eb1d1ba5c0e45bfe450757bcd4459a1e Mon Sep 17 00:00:00 2001 From: "allcontributors[bot]" <46447321+allcontributors[bot]@users.noreply.github.com> Date: Sat, 31 Jan 2026 13:44:29 -0800 Subject: [PATCH 67/69] add gajadv as a contributor for bug (#1963) Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com> Co-authored-by: Michael Flanakin --- .all-contributorsrc | 9 +++++++++ README.md | 1 + docs/README.md | 1 + 3 files changed, 11 insertions(+) diff --git a/.all-contributorsrc b/.all-contributorsrc index 41f24ce4e..281e50e53 100644 --- a/.all-contributorsrc +++ b/.all-contributorsrc @@ -872,6 +872,15 @@ "code" ] }, + { + "login": "gajadv", + "name": "gajadv", + "avatar_url": "https://avatars.githubusercontent.com/u/125584305?v=4", + "profile": "https://github.com/gajadv", + "contributions": [ + "bug" + ] + }, { "login": "francesco1119", "name": "Francesco Mantovani", diff --git a/README.md b/README.md index e40b9dc31..e36512d91 100644 --- a/README.md +++ b/README.md @@ -145,6 +145,7 @@ There are many ways to participate. From reporting bugs and requesting features gorkomikus
gorkomikus
💻 + gajadv
gajadv
🐛 Francesco Mantovani
Francesco Mantovani
🐛 diff --git a/docs/README.md b/docs/README.md index 13fe216d3..d94e0ee4a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -215,6 +215,7 @@ Whether you're looking for a little assistance or are interested in contributing gorkomikus
gorkomikus
💻 + gajadv
gajadv
🐛 Francesco Mantovani
Francesco Mantovani
🐛 From 613cf18d919bf386f485d00e4785a809c48d79b0 Mon Sep 17 00:00:00 2001 From: "allcontributors[bot]" <46447321+allcontributors[bot]@users.noreply.github.com> Date: Sat, 31 Jan 2026 13:47:19 -0800 Subject: [PATCH 68/69] add ahpandit as a contributor for bug (#1964) Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com> --- .all-contributorsrc | 9 +++++++++ README.md | 1 + docs/README.md | 1 + 3 files changed, 11 insertions(+) diff --git a/.all-contributorsrc b/.all-contributorsrc index 281e50e53..ead80a0da 100644 --- a/.all-contributorsrc +++ b/.all-contributorsrc @@ -889,6 +889,15 @@ "contributions": [ "bug" ] + }, + { + "login": "ahpandit", + "name": "ahpandit", + "avatar_url": "https://avatars.githubusercontent.com/u/98794500?v=4", + "profile": "https://github.com/ahpandit", + "contributions": [ + "bug" + ] } ], "commitType": "docs", diff --git a/README.md b/README.md index e36512d91..069898326 100644 --- a/README.md +++ b/README.md @@ -147,6 +147,7 @@ There are many ways to participate. From reporting bugs and requesting features gajadv
gajadv
🐛 Francesco Mantovani
Francesco Mantovani
🐛 + ahpandit
ahpandit
🐛 diff --git a/docs/README.md b/docs/README.md index d94e0ee4a..487ceecba 100644 --- a/docs/README.md +++ b/docs/README.md @@ -217,6 +217,7 @@ Whether you're looking for a little assistance or are interested in contributing gajadv
gajadv
🐛 Francesco Mantovani
Francesco Mantovani
🐛 + ahpandit
ahpandit
🐛 From 998ce6199f8bb87c2f28376cade73237d7334776 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Sat, 31 Jan 2026 22:41:00 +0000 Subject: [PATCH 69/69] Update Just The Docs remote theme file references (#1961) Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/_sass/color_schemes/dark.scss | 6 +++--- docs/_sass/color_schemes/light.scss | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/_sass/color_schemes/dark.scss b/docs/_sass/color_schemes/dark.scss index 57a0f0790..da6f052f2 100644 --- a/docs/_sass/color_schemes/dark.scss +++ b/docs/_sass/color_schemes/dark.scss @@ -7,12 +7,12 @@ $nav-child-link-color: $grey-dk-000; $sidebar-color: $grey-dk-300; $base-button-color: $grey-dk-250; $btn-primary-color: $purple-200; -$code-background-color: #31343f; // OneDarkJekyll default for syntax-one-dark-vivid -$code-linenumber-color: #dee2f7; // OneDarkJekyll .nf for syntax-one-dark-vivid +$code-background-color: #31343f; +$code-linenumber-color: #dee2f7; $feedback-color: darken($sidebar-color, 3%); $table-background-color: $grey-dk-250; $search-background-color: $grey-dk-250; $search-result-preview-color: $grey-dk-000; $border-color: $grey-dk-200; -@import './vendor/OneDarkJekyll/syntax'; // this is the one-dark-vivid atom syntax theme +@import 'vendor/accessible-pygments/github-dark'; // WCAG 2.1 AA compliant syntax theme diff --git a/docs/_sass/color_schemes/light.scss b/docs/_sass/color_schemes/light.scss index a5f60c4d9..45c95c97d 100644 --- a/docs/_sass/color_schemes/light.scss +++ b/docs/_sass/color_schemes/light.scss @@ -13,4 +13,4 @@ $table-background-color: $white !default; $search-background-color: $white !default; $search-result-preview-color: $grey-dk-000 !default; -@import "./vendor/OneLightJekyll/syntax"; \ No newline at end of file +@import 'vendor/accessible-pygments/github-light'; // WCAG 2.1 AA compliant syntax theme