diff --git a/docs/assets/stylesheets/extra.css b/docs/assets/stylesheets/extra.css index 3c66ab76a07..68ced5d6083 100644 --- a/docs/assets/stylesheets/extra.css +++ b/docs/assets/stylesheets/extra.css @@ -32,6 +32,7 @@ --md-admonition-icon--learn: url('data:image/svg+xml;charset=utf-8,'); --md-admonition-icon--message: url('data:image/svg+xml;charset=utf-8,'); --md-admonition-icon--security: url('data:image/svg+xml;charset=utf-8,'); + --md-admonition-icon--diamond: url('data:image/svg+xml;charset=utf-8,'); } /* experimental */ @@ -110,6 +111,25 @@ mask-image: var(--md-admonition-icon--security); } +/* implementation reference */ +.md-typeset .admonition.implementation, +.md-typeset details.implementation { + border-color: rgb(124, 77, 255) +} + +.md-typeset .implementation>.admonition-title, +.md-typeset .implementation>summary { + background-color: rgba(124, 77, 255, .1); + border-color: rgb(124, 77, 255); +} + +.md-typeset .implementation>.admonition-title::before, +.md-typeset .implementation>summary::before { + background-color: rgb(124, 77, 255); + -webkit-mask-image: var(--md-admonition-icon--diamond); + mask-image: var(--md-admonition-icon--diamond); +} + /* badges */ .badge { diff --git a/docs/changelog.md b/docs/changelog.md index 12c766929e6..546b99b63ee 100644 --- a/docs/changelog.md +++ b/docs/changelog.md @@ -48,6 +48,8 @@ What's changed since v1.47.0: - Azure Kubernetes Service: - Updated `Azure.AKS.Version` to use `1.33.7` as the minimum version by @BernieWhite. [#3708](https://github.com/Azure/PSRule.Rules.Azure/issues/3708) +- Improved documentation for expansion internals with a high-level flow diagram and code references by @Copilot. + [#3715](https://github.com/Azure/PSRule.Rules.Azure/issues/3715) ## v1.47.0 diff --git a/docs/en/rules/module.md b/docs/en/rules/module.md index d929c5a4cfb..af352e6dfef 100644 --- a/docs/en/rules/module.md +++ b/docs/en/rules/module.md @@ -641,7 +641,7 @@ Name | Synopsis | Severity | Level ---- | -------- | -------- | ----- [Azure.VNET.FirewallSubnet](Azure.VNET.FirewallSubnet.md) | Use Azure Firewall to filter network traffic to and from Azure resources. | Important | Error -### SE:01 Security baseline +### SE:01 Security Baseline Name | Synopsis | Severity | Level ---- | -------- | -------- | ----- diff --git a/docs/es/rules/module.md b/docs/es/rules/module.md index d929c5a4cfb..af352e6dfef 100644 --- a/docs/es/rules/module.md +++ b/docs/es/rules/module.md @@ -641,7 +641,7 @@ Name | Synopsis | Severity | Level ---- | -------- | -------- | ----- [Azure.VNET.FirewallSubnet](Azure.VNET.FirewallSubnet.md) | Use Azure Firewall to filter network traffic to and from Azure resources. | Important | Error -### SE:01 Security baseline +### SE:01 Security Baseline Name | Synopsis | Severity | Level ---- | -------- | -------- | ----- diff --git a/docs/license-contributing/expansion-internals.md b/docs/license-contributing/expansion-internals.md index af847d0285e..db8bdf04bf9 100644 --- a/docs/license-contributing/expansion-internals.md +++ b/docs/license-contributing/expansion-internals.md @@ -44,6 +44,46 @@ The main components of the expansion process are: - **Materialized properties** — Some resources types have properties that are affected by multiple deployments or child resources. These must be calculated to determine the final state value of the property. +### High-level flow + +The following diagram shows the high-level flow of the expansion process from source file to expanded resources. + +```mermaid +flowchart TD + A["Input file (.bicep / .bicepparam / .json)"] --> B{Is Bicep source?} + B -->|Yes| C["Build with Bicep CLI\n(bicep build / bicep build-params)"] + B -->|No| D[Read ARM template JSON] + C --> D + D --> E["Load parameters\n(parameter file + AZURE_PARAMETER_DEFAULTS)"] + E --> F["Visit deployment\n(DeploymentVisitor)"] + F --> G["Process definitions\n(parameters, variables, functions, types)"] + G --> H["Build dependency graph\n(ResourceDependencyGraph)"] + H --> I[Visit each resource in dependency order] + I --> J{Is nested deployment?} + J -->|Yes| K["Visit nested deployment recursively\n(inner / outer scope)"] + J -->|No| L["Evaluate condition\nExpand copy loops"] + L --> M["Resolve property expressions\n(ExpressionBuilder)"] + M --> N[Emit resource to context] + K --> N + N --> O{More resources?} + O -->|Yes| I + O -->|No| P["Post-process\n(MaterializedDeploymentVisitor)\nNest children, materialize properties"] + P --> Q[Output expanded resources] +``` + +### Key components + +The key source code components involved in the expansion process are: + + Component | Source file | Description +---------- | ----------- | ----------- +`BicepHelper` | `src/PSRule.Rules.Azure/Data/Bicep/BicepHelper.cs` | Invokes the Bicep CLI and coordinates expansion of Bicep and ARM files. +`DeploymentVisitor` | `src/PSRule.Rules.Azure/Arm/Deployments/DeploymentVisitor.cs` | The core visitor that walks the ARM deployment structure. +`MaterializedDeploymentVisitor` | `src/PSRule.Rules.Azure/Arm/Deployments/MaterializedDeploymentVisitor.cs` | Extends `DeploymentVisitor` to handle post-processing of emitted resources. +`ResourceDependencyGraph` | `src/PSRule.Rules.Azure/Arm/Deployments/ResourceDependencyGraph.cs` | Builds and resolves the dependency graph for resources in a deployment. +`ExpressionBuilder` | `src/PSRule.Rules.Azure/Arm/Expressions/ExpressionBuilder.cs` | Parses and evaluates ARM template expressions. +`Functions` | `src/PSRule.Rules.Azure/Arm/Expressions/Functions.cs` | Implementations of ARM template built-in functions used during expression evaluation. + ## Building Bicep Azure Bicep code syntax is a domain specific language provides a higher level of abstraction over ARM deployments. @@ -55,6 +95,14 @@ As a result, the Bicep CLI must be installed and available prior to running the To build a Bicep file, the Bicep CLI is invoked with `bicep build` or `bicep build-params` command. +!!! Implementation + The `BicepHelper` class (`src/PSRule.Rules.Azure/Data/Bicep/BicepHelper.cs`) is responsible for: + + - Discovering the Bicep CLI. + - Spawning the Bicep CLI process. + - Calling `ProcessFile` for a `.bicep` file or `ProcessParamFile` for a `.bicepparam` file. + - Passing the resulting ARM template JSON to the deployment visitor for expansion. + ### CLI discovery To find an instance of the Bicep CLI, PSRule for Azure probes several paths, and uses the first instance found. @@ -94,6 +142,10 @@ Secrets are a good example of this, as they should not be specified in the param Definitions are the building blocks of the ARM deployment and may be reference by resources or other definitions. For most cases, definitions are lazy loaded into the context of the deployment during expansion. +!!! Implementation + The `LazyParameter`, `LazyVariable`, and `LazyOutput` classes (in `src/PSRule.Rules.Azure/Arm/Deployments/`) implement this lazy loading pattern, + deferring evaluation of each definition until it is first referenced. + Exceptions to this are when copy loops are used to define variables and parameters. Otherwise the definitions are not resolved until they are referenced by a resource. @@ -107,6 +159,10 @@ Similarly, a deployment may return outputs that are used in the parent deploymen As a result, each resource must be visited based on a dependency graph so that dependencies are resolved before dependant resources. +!!! Implementation + The `ResourceDependencyGraph` class (`src/PSRule.Rules.Azure/Arm/Deployments/ResourceDependencyGraph.cs`) builds this graph + from the `dependsOn` properties declared in the template, and performs a topological sort to produce the correct visit order. + ## Visiting each resource When a resource is visited: @@ -146,6 +202,16 @@ For each function to be understood by the expansion process, it must be implemen When an expression is called, context about the deployment is passed into the root function of the expression. +!!! Implementation + The key classes for expression evaluation are: + + - `ExpressionParser` (`src/PSRule.Rules.Azure/Arm/Expressions/ExpressionParser.cs`) — + Tokenizes and parses ARM expression strings into an `ExpressionStream`. + - `ExpressionBuilder` (`src/PSRule.Rules.Azure/Arm/Expressions/ExpressionBuilder.cs`) — + Builds a callable expression tree from the parsed tokens. + - `Functions` (`src/PSRule.Rules.Azure/Arm/Expressions/Functions.cs`) — + Contains implementations of all supported ARM template built-in functions. + ### Context properties During deployment to Azure, ARM maintains several context objects that are used to evaluate expressions. diff --git a/docs/license-contributing/index.md b/docs/license-contributing/index.md index e4350e82565..b81ae61ecee 100755 --- a/docs/license-contributing/index.md +++ b/docs/license-contributing/index.md @@ -27,6 +27,7 @@ Please read our [contributing guidelines][2] and [code of conduct][3] to learn h - [Writing documentation](writing-documentation.md) — Guidelines for writing and improving documentation. - [Getting started with documentation](getting-started-with-documentation.md) — How to get started contributing to documentation. - [Get started contributing](get-started-contributing.md) — How to get started contributing to the project. +- [Expansion internals](expansion-internals.md) — Internals of the expansion process for developers and contributors. [1]: https://github.com/Azure/PSRule.Rules.Azure/blob/main/LICENSE [2]: https://github.com/Azure/PSRule.Rules.Azure/blob/main/CONTRIBUTING.md diff --git a/mkdocs.yml b/mkdocs.yml index 54442f96a68..c9ddead7b6c 100755 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -137,7 +137,11 @@ markdown_extensions: anchor_linenums: true line_spans: __span pygments_lang_class: true - - pymdownx.superfences + - pymdownx.superfences: + custom_fences: + - name: mermaid + class: mermaid + format: !!python/name:pymdownx.superfences.fence_code_format - pymdownx.pathconverter - pymdownx.tabbed: alternate_style: true