From 62ea5f089cf779567cd125f26f8b70416c23071f Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 25 Mar 2026 07:27:52 +0000 Subject: [PATCH 1/5] Initial plan From df3ef97a04f2036e213ed88d623e56e0882d6da6 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 25 Mar 2026 07:36:30 +0000 Subject: [PATCH 2/5] Add high-level flow diagram and code references to expansion internals docs Co-authored-by: BernieWhite <13513058+BernieWhite@users.noreply.github.com> Agent-Logs-Url: https://github.com/Azure/PSRule.Rules.Azure/sessions/053574ba-c576-4701-aa85-d257e2ffdd86 --- docs/changelog.md | 2 + .../expansion-internals.md | 61 +++++++++++++++++++ docs/license-contributing/index.md | 1 + mkdocs.yml | 6 +- 4 files changed, 69 insertions(+), 1 deletion(-) diff --git a/docs/changelog.md b/docs/changelog.md index 953a25e7049..eefd732c6b3 100644 --- a/docs/changelog.md +++ b/docs/changelog.md @@ -34,6 +34,8 @@ See [upgrade notes][1] for helpful information when upgrading from previous vers - Azure Kubernetes Service: - Updated `Azure.AKS.Version` to use `1.33.7` as the minimum version by @BernieWhite. [#3708](https://github.com/Azure/PSRule.Rules.Azure/issues/3708) +- Improved documentation for expansion internals with a high-level flow diagram and code references by @Copilot. + [#3715](https://github.com/Azure/PSRule.Rules.Azure/issues/3715) ## v1.47.0 diff --git a/docs/license-contributing/expansion-internals.md b/docs/license-contributing/expansion-internals.md index af847d0285e..3bbc7b6c4ad 100644 --- a/docs/license-contributing/expansion-internals.md +++ b/docs/license-contributing/expansion-internals.md @@ -44,6 +44,46 @@ The main components of the expansion process are: - **Materialized properties** — Some resources types have properties that are affected by multiple deployments or child resources. These must be calculated to determine the final state value of the property. +### High-level flow + +The following diagram shows the high-level flow of the expansion process from source file to expanded resources. + +```mermaid +flowchart TD + A["Input file (.bicep / .bicepparam / .json)"] --> B{Is Bicep source?} + B -->|Yes| C["Build with Bicep CLI\n(bicep build / bicep build-params)"] + B -->|No| D[Read ARM template JSON] + C --> D + D --> E["Load parameters\n(parameter file + AZURE_PARAMETER_DEFAULTS)"] + E --> F["Visit deployment\n(DeploymentVisitor)"] + F --> G["Process definitions\n(parameters, variables, functions, types)"] + G --> H["Build dependency graph\n(ResourceDependencyGraph)"] + H --> I[Visit each resource in dependency order] + I --> J{Is nested deployment?} + J -->|Yes| K["Visit nested deployment recursively\n(inner / outer scope)"] + J -->|No| L["Evaluate condition\nExpand copy loops"] + L --> M["Resolve property expressions\n(ExpressionBuilder)"] + M --> N[Emit resource to context] + K --> N + N --> O{More resources?} + O -->|Yes| I + O -->|No| P["Post-process\n(MaterializedDeploymentVisitor)\nNest children, materialize properties"] + P --> Q[Output expanded resources] +``` + +### Key components + +The key source code components involved in the expansion process are: + +| Component | Source file | Description | +|-----------|-------------|-------------| +| `BicepHelper` | `src/PSRule.Rules.Azure/Data/Bicep/BicepHelper.cs` | Invokes the Bicep CLI and coordinates expansion of Bicep and ARM files. | +| `DeploymentVisitor` | `src/PSRule.Rules.Azure/Arm/Deployments/DeploymentVisitor.cs` | The core visitor that walks the ARM deployment structure. | +| `MaterializedDeploymentVisitor` | `src/PSRule.Rules.Azure/Arm/Deployments/MaterializedDeploymentVisitor.cs` | Extends `DeploymentVisitor` to handle post-processing of emitted resources. | +| `ResourceDependencyGraph` | `src/PSRule.Rules.Azure/Arm/Deployments/ResourceDependencyGraph.cs` | Builds and resolves the dependency graph for resources in a deployment. | +| `ExpressionBuilder` | `src/PSRule.Rules.Azure/Arm/Expressions/ExpressionBuilder.cs` | Parses and evaluates ARM template expressions. | +| `Functions` | `src/PSRule.Rules.Azure/Arm/Expressions/Functions.cs` | Implementations of ARM template built-in functions used during expression evaluation. | + ## Building Bicep Azure Bicep code syntax is a domain specific language provides a higher level of abstraction over ARM deployments. @@ -55,6 +95,13 @@ As a result, the Bicep CLI must be installed and available prior to running the To build a Bicep file, the Bicep CLI is invoked with `bicep build` or `bicep build-params` command. +The `BicepHelper` class (`src/PSRule.Rules.Azure/Data/Bicep/BicepHelper.cs`) is responsible for: + +- Discovering the Bicep CLI. +- Spawning the Bicep CLI process. +- Calling `ProcessFile` for a `.bicep` file or `ProcessParamFile` for a `.bicepparam` file. +- Passing the resulting ARM template JSON to the deployment visitor for expansion. + ### CLI discovery To find an instance of the Bicep CLI, PSRule for Azure probes several paths, and uses the first instance found. @@ -93,6 +140,8 @@ Secrets are a good example of this, as they should not be specified in the param Definitions are the building blocks of the ARM deployment and may be reference by resources or other definitions. For most cases, definitions are lazy loaded into the context of the deployment during expansion. +The `LazyParameter`, `LazyVariable`, and `LazyOutput` classes (in `src/PSRule.Rules.Azure/Arm/Deployments/`) implement this lazy loading pattern, +deferring evaluation of each definition until it is first referenced. Exceptions to this are when copy loops are used to define variables and parameters. Otherwise the definitions are not resolved until they are referenced by a resource. @@ -107,6 +156,9 @@ Similarly, a deployment may return outputs that are used in the parent deploymen As a result, each resource must be visited based on a dependency graph so that dependencies are resolved before dependant resources. +The `ResourceDependencyGraph` class (`src/PSRule.Rules.Azure/Arm/Deployments/ResourceDependencyGraph.cs`) builds this graph +from the `dependsOn` properties declared in the template, and performs a topological sort to produce the correct visit order. + ## Visiting each resource When a resource is visited: @@ -146,6 +198,15 @@ For each function to be understood by the expansion process, it must be implemen When an expression is called, context about the deployment is passed into the root function of the expression. +The key classes for expression evaluation are: + +- `ExpressionParser` (`src/PSRule.Rules.Azure/Arm/Expressions/ExpressionParser.cs`) — + Tokenizes and parses ARM expression strings into an `ExpressionStream`. +- `ExpressionBuilder` (`src/PSRule.Rules.Azure/Arm/Expressions/ExpressionBuilder.cs`) — + Builds a callable expression tree from the parsed tokens. +- `Functions` (`src/PSRule.Rules.Azure/Arm/Expressions/Functions.cs`) — + Contains implementations of all supported ARM template built-in functions. + ### Context properties During deployment to Azure, ARM maintains several context objects that are used to evaluate expressions. diff --git a/docs/license-contributing/index.md b/docs/license-contributing/index.md index e4350e82565..b81ae61ecee 100755 --- a/docs/license-contributing/index.md +++ b/docs/license-contributing/index.md @@ -27,6 +27,7 @@ Please read our [contributing guidelines][2] and [code of conduct][3] to learn h - [Writing documentation](writing-documentation.md) — Guidelines for writing and improving documentation. - [Getting started with documentation](getting-started-with-documentation.md) — How to get started contributing to documentation. - [Get started contributing](get-started-contributing.md) — How to get started contributing to the project. +- [Expansion internals](expansion-internals.md) — Internals of the expansion process for developers and contributors. [1]: https://github.com/Azure/PSRule.Rules.Azure/blob/main/LICENSE [2]: https://github.com/Azure/PSRule.Rules.Azure/blob/main/CONTRIBUTING.md diff --git a/mkdocs.yml b/mkdocs.yml index 54442f96a68..c9ddead7b6c 100755 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -137,7 +137,11 @@ markdown_extensions: anchor_linenums: true line_spans: __span pygments_lang_class: true - - pymdownx.superfences + - pymdownx.superfences: + custom_fences: + - name: mermaid + class: mermaid + format: !!python/name:pymdownx.superfences.fence_code_format - pymdownx.pathconverter - pymdownx.tabbed: alternate_style: true From 97de4e2ae00b6dc6327de89f5b60bda8ce2fd63c Mon Sep 17 00:00:00 2001 From: Bernie White Date: Wed, 25 Mar 2026 14:10:19 +0100 Subject: [PATCH 3/5] Add implementation reference --- docs/assets/stylesheets/extra.css | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/docs/assets/stylesheets/extra.css b/docs/assets/stylesheets/extra.css index 3c66ab76a07..68ced5d6083 100644 --- a/docs/assets/stylesheets/extra.css +++ b/docs/assets/stylesheets/extra.css @@ -32,6 +32,7 @@ --md-admonition-icon--learn: url('data:image/svg+xml;charset=utf-8,'); --md-admonition-icon--message: url('data:image/svg+xml;charset=utf-8,'); --md-admonition-icon--security: url('data:image/svg+xml;charset=utf-8,'); + --md-admonition-icon--diamond: url('data:image/svg+xml;charset=utf-8,'); } /* experimental */ @@ -110,6 +111,25 @@ mask-image: var(--md-admonition-icon--security); } +/* implementation reference */ +.md-typeset .admonition.implementation, +.md-typeset details.implementation { + border-color: rgb(124, 77, 255) +} + +.md-typeset .implementation>.admonition-title, +.md-typeset .implementation>summary { + background-color: rgba(124, 77, 255, .1); + border-color: rgb(124, 77, 255); +} + +.md-typeset .implementation>.admonition-title::before, +.md-typeset .implementation>summary::before { + background-color: rgb(124, 77, 255); + -webkit-mask-image: var(--md-admonition-icon--diamond); + mask-image: var(--md-admonition-icon--diamond); +} + /* badges */ .badge { From c0032a215b0207ae601704cf07d41e3976cb406f Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 25 Mar 2026 13:17:24 +0000 Subject: [PATCH 4/5] Convert code/class reference sections to use Implementation admonition Co-authored-by: BernieWhite <13513058+BernieWhite@users.noreply.github.com> Agent-Logs-Url: https://github.com/Azure/PSRule.Rules.Azure/sessions/7158ab17-3381-4706-a24e-d6d068adf843 --- .../expansion-internals.md | 54 ++++++++++--------- 1 file changed, 30 insertions(+), 24 deletions(-) diff --git a/docs/license-contributing/expansion-internals.md b/docs/license-contributing/expansion-internals.md index 3bbc7b6c4ad..3dbcf32d796 100644 --- a/docs/license-contributing/expansion-internals.md +++ b/docs/license-contributing/expansion-internals.md @@ -75,14 +75,15 @@ flowchart TD The key source code components involved in the expansion process are: -| Component | Source file | Description | -|-----------|-------------|-------------| -| `BicepHelper` | `src/PSRule.Rules.Azure/Data/Bicep/BicepHelper.cs` | Invokes the Bicep CLI and coordinates expansion of Bicep and ARM files. | -| `DeploymentVisitor` | `src/PSRule.Rules.Azure/Arm/Deployments/DeploymentVisitor.cs` | The core visitor that walks the ARM deployment structure. | -| `MaterializedDeploymentVisitor` | `src/PSRule.Rules.Azure/Arm/Deployments/MaterializedDeploymentVisitor.cs` | Extends `DeploymentVisitor` to handle post-processing of emitted resources. | -| `ResourceDependencyGraph` | `src/PSRule.Rules.Azure/Arm/Deployments/ResourceDependencyGraph.cs` | Builds and resolves the dependency graph for resources in a deployment. | -| `ExpressionBuilder` | `src/PSRule.Rules.Azure/Arm/Expressions/ExpressionBuilder.cs` | Parses and evaluates ARM template expressions. | -| `Functions` | `src/PSRule.Rules.Azure/Arm/Expressions/Functions.cs` | Implementations of ARM template built-in functions used during expression evaluation. | +!!! Implementation + | Component | Source file | Description | + |-----------|-------------|-------------| + | `BicepHelper` | `src/PSRule.Rules.Azure/Data/Bicep/BicepHelper.cs` | Invokes the Bicep CLI and coordinates expansion of Bicep and ARM files. | + | `DeploymentVisitor` | `src/PSRule.Rules.Azure/Arm/Deployments/DeploymentVisitor.cs` | The core visitor that walks the ARM deployment structure. | + | `MaterializedDeploymentVisitor` | `src/PSRule.Rules.Azure/Arm/Deployments/MaterializedDeploymentVisitor.cs` | Extends `DeploymentVisitor` to handle post-processing of emitted resources. | + | `ResourceDependencyGraph` | `src/PSRule.Rules.Azure/Arm/Deployments/ResourceDependencyGraph.cs` | Builds and resolves the dependency graph for resources in a deployment. | + | `ExpressionBuilder` | `src/PSRule.Rules.Azure/Arm/Expressions/ExpressionBuilder.cs` | Parses and evaluates ARM template expressions. | + | `Functions` | `src/PSRule.Rules.Azure/Arm/Expressions/Functions.cs` | Implementations of ARM template built-in functions used during expression evaluation. | ## Building Bicep @@ -95,12 +96,13 @@ As a result, the Bicep CLI must be installed and available prior to running the To build a Bicep file, the Bicep CLI is invoked with `bicep build` or `bicep build-params` command. -The `BicepHelper` class (`src/PSRule.Rules.Azure/Data/Bicep/BicepHelper.cs`) is responsible for: +!!! Implementation + The `BicepHelper` class (`src/PSRule.Rules.Azure/Data/Bicep/BicepHelper.cs`) is responsible for: -- Discovering the Bicep CLI. -- Spawning the Bicep CLI process. -- Calling `ProcessFile` for a `.bicep` file or `ProcessParamFile` for a `.bicepparam` file. -- Passing the resulting ARM template JSON to the deployment visitor for expansion. + - Discovering the Bicep CLI. + - Spawning the Bicep CLI process. + - Calling `ProcessFile` for a `.bicep` file or `ProcessParamFile` for a `.bicepparam` file. + - Passing the resulting ARM template JSON to the deployment visitor for expansion. ### CLI discovery @@ -140,8 +142,10 @@ Secrets are a good example of this, as they should not be specified in the param Definitions are the building blocks of the ARM deployment and may be reference by resources or other definitions. For most cases, definitions are lazy loaded into the context of the deployment during expansion. -The `LazyParameter`, `LazyVariable`, and `LazyOutput` classes (in `src/PSRule.Rules.Azure/Arm/Deployments/`) implement this lazy loading pattern, -deferring evaluation of each definition until it is first referenced. + +!!! Implementation + The `LazyParameter`, `LazyVariable`, and `LazyOutput` classes (in `src/PSRule.Rules.Azure/Arm/Deployments/`) implement this lazy loading pattern, + deferring evaluation of each definition until it is first referenced. Exceptions to this are when copy loops are used to define variables and parameters. Otherwise the definitions are not resolved until they are referenced by a resource. @@ -156,8 +160,9 @@ Similarly, a deployment may return outputs that are used in the parent deploymen As a result, each resource must be visited based on a dependency graph so that dependencies are resolved before dependant resources. -The `ResourceDependencyGraph` class (`src/PSRule.Rules.Azure/Arm/Deployments/ResourceDependencyGraph.cs`) builds this graph -from the `dependsOn` properties declared in the template, and performs a topological sort to produce the correct visit order. +!!! Implementation + The `ResourceDependencyGraph` class (`src/PSRule.Rules.Azure/Arm/Deployments/ResourceDependencyGraph.cs`) builds this graph + from the `dependsOn` properties declared in the template, and performs a topological sort to produce the correct visit order. ## Visiting each resource @@ -198,14 +203,15 @@ For each function to be understood by the expansion process, it must be implemen When an expression is called, context about the deployment is passed into the root function of the expression. -The key classes for expression evaluation are: +!!! Implementation + The key classes for expression evaluation are: -- `ExpressionParser` (`src/PSRule.Rules.Azure/Arm/Expressions/ExpressionParser.cs`) — - Tokenizes and parses ARM expression strings into an `ExpressionStream`. -- `ExpressionBuilder` (`src/PSRule.Rules.Azure/Arm/Expressions/ExpressionBuilder.cs`) — - Builds a callable expression tree from the parsed tokens. -- `Functions` (`src/PSRule.Rules.Azure/Arm/Expressions/Functions.cs`) — - Contains implementations of all supported ARM template built-in functions. + - `ExpressionParser` (`src/PSRule.Rules.Azure/Arm/Expressions/ExpressionParser.cs`) — + Tokenizes and parses ARM expression strings into an `ExpressionStream`. + - `ExpressionBuilder` (`src/PSRule.Rules.Azure/Arm/Expressions/ExpressionBuilder.cs`) — + Builds a callable expression tree from the parsed tokens. + - `Functions` (`src/PSRule.Rules.Azure/Arm/Expressions/Functions.cs`) — + Contains implementations of all supported ARM template built-in functions. ### Context properties From fd5daea1f121d124282c7393a915dd759d17a6ff Mon Sep 17 00:00:00 2001 From: Bernie White Date: Wed, 25 Mar 2026 18:02:15 +0000 Subject: [PATCH 5/5] Additional formatting updates --- docs/en/rules/module.md | 2 +- docs/es/rules/module.md | 2 +- .../license-contributing/expansion-internals.md | 17 ++++++++--------- 3 files changed, 10 insertions(+), 11 deletions(-) diff --git a/docs/en/rules/module.md b/docs/en/rules/module.md index d929c5a4cfb..af352e6dfef 100644 --- a/docs/en/rules/module.md +++ b/docs/en/rules/module.md @@ -641,7 +641,7 @@ Name | Synopsis | Severity | Level ---- | -------- | -------- | ----- [Azure.VNET.FirewallSubnet](Azure.VNET.FirewallSubnet.md) | Use Azure Firewall to filter network traffic to and from Azure resources. | Important | Error -### SE:01 Security baseline +### SE:01 Security Baseline Name | Synopsis | Severity | Level ---- | -------- | -------- | ----- diff --git a/docs/es/rules/module.md b/docs/es/rules/module.md index d929c5a4cfb..af352e6dfef 100644 --- a/docs/es/rules/module.md +++ b/docs/es/rules/module.md @@ -641,7 +641,7 @@ Name | Synopsis | Severity | Level ---- | -------- | -------- | ----- [Azure.VNET.FirewallSubnet](Azure.VNET.FirewallSubnet.md) | Use Azure Firewall to filter network traffic to and from Azure resources. | Important | Error -### SE:01 Security baseline +### SE:01 Security Baseline Name | Synopsis | Severity | Level ---- | -------- | -------- | ----- diff --git a/docs/license-contributing/expansion-internals.md b/docs/license-contributing/expansion-internals.md index 3dbcf32d796..db8bdf04bf9 100644 --- a/docs/license-contributing/expansion-internals.md +++ b/docs/license-contributing/expansion-internals.md @@ -75,15 +75,14 @@ flowchart TD The key source code components involved in the expansion process are: -!!! Implementation - | Component | Source file | Description | - |-----------|-------------|-------------| - | `BicepHelper` | `src/PSRule.Rules.Azure/Data/Bicep/BicepHelper.cs` | Invokes the Bicep CLI and coordinates expansion of Bicep and ARM files. | - | `DeploymentVisitor` | `src/PSRule.Rules.Azure/Arm/Deployments/DeploymentVisitor.cs` | The core visitor that walks the ARM deployment structure. | - | `MaterializedDeploymentVisitor` | `src/PSRule.Rules.Azure/Arm/Deployments/MaterializedDeploymentVisitor.cs` | Extends `DeploymentVisitor` to handle post-processing of emitted resources. | - | `ResourceDependencyGraph` | `src/PSRule.Rules.Azure/Arm/Deployments/ResourceDependencyGraph.cs` | Builds and resolves the dependency graph for resources in a deployment. | - | `ExpressionBuilder` | `src/PSRule.Rules.Azure/Arm/Expressions/ExpressionBuilder.cs` | Parses and evaluates ARM template expressions. | - | `Functions` | `src/PSRule.Rules.Azure/Arm/Expressions/Functions.cs` | Implementations of ARM template built-in functions used during expression evaluation. | + Component | Source file | Description +---------- | ----------- | ----------- +`BicepHelper` | `src/PSRule.Rules.Azure/Data/Bicep/BicepHelper.cs` | Invokes the Bicep CLI and coordinates expansion of Bicep and ARM files. +`DeploymentVisitor` | `src/PSRule.Rules.Azure/Arm/Deployments/DeploymentVisitor.cs` | The core visitor that walks the ARM deployment structure. +`MaterializedDeploymentVisitor` | `src/PSRule.Rules.Azure/Arm/Deployments/MaterializedDeploymentVisitor.cs` | Extends `DeploymentVisitor` to handle post-processing of emitted resources. +`ResourceDependencyGraph` | `src/PSRule.Rules.Azure/Arm/Deployments/ResourceDependencyGraph.cs` | Builds and resolves the dependency graph for resources in a deployment. +`ExpressionBuilder` | `src/PSRule.Rules.Azure/Arm/Expressions/ExpressionBuilder.cs` | Parses and evaluates ARM template expressions. +`Functions` | `src/PSRule.Rules.Azure/Arm/Expressions/Functions.cs` | Implementations of ARM template built-in functions used during expression evaluation. ## Building Bicep