Conversation
Deploying localstack-docs with
|
| Latest commit: |
0924625
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://0a4241df.localstack-docs.pages.dev |
| Branch Preview URL: | https://harshmishra-doc-81.localstack-docs.pages.dev |
| - filter by type: `where type =~ 'Microsoft.Web/sites'` | ||
| - filter by type and name: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81'` | ||
| - filter by name: `where name =~ 'ls-app-doc81'` | ||
| - project only IDs: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81' | project id` |
There was a problem hiding this comment.
Is it worth pointing out the known limitations here?
For example:
| - project only IDs: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81' | project id` | |
| The following queries are all supported: | |
| - filter by type: `where type =~ 'Microsoft.Web/sites'` | |
| - filter by type and name: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81'` | |
| - filter by name: `where name =~ 'ls-app-doc81'` | |
| - project only IDs: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81' | project id` | |
| - | |
| Known limitations: the Localstack for Azure emulator currently does not support the following constructs: | |
| - nested queries | |
| - the `count` keyword | |
| - the `extend` keyword | |
| - the `join` keyword | |
| - the `limit` keyword | |
| - the `mv-expand` keyword | |
| - the `order` keyword | |
| - the `parse` keyword | |
| - the `project-away` keyword | |
| - the `sort` keyword | |
| - the `summarize` keyword | |
| - the `take` keyword | |
| - the `top` keyword | |
| - the `union` keyword |
For reference, the full list of operators can be found here:
https://learn.microsoft.com/en-us/azure/governance/resource-graph/concepts/query-language#supported-tabulartop-level-operators
Considering we only support a small subset of this list, it could also be easier to only show the constructs that we do support!
There was a problem hiding this comment.
Thanks @bblommers, I documented features and limitations below. The new version of our Azure Resource Graph emulator supports more operators. See below.
There was a problem hiding this comment.
@HarshCasper / @bblommers, while using az rest to call the Azure Resource Graph is perfectly doable, it's very unusual. Users prefer to install to the az extension resource-graph as follows:
az extension add --name resource-graphand use it as shown in Quickstart: Run Resource Graph query using Azure CLI
az graph query --graph-query 'Resources | project name, type | limit 5'
az graph query --graph-query 'Resources | where type =~ 'Microsoft.Storage/storageAccounts' | project name, location`
...I strongly suggest to refactor the article to use az graph. Said that, I would also keep the az rest you and Bert created to show readers an additional method to invoke the Azure Resource Graph API in Azure as well as our emulator.
I realized that the current implementation throws exceptions with some licit queries, so we need to fix the codebase. I'll take this offline with Bert.
cc: @quetzalliwrites
paolosalvatori
left a comment
There was a problem hiding this comment.
Thanks @HarshCasper, see my comments, amendments, suggestions, and additions.
| Azure Resource Graph is a service for querying Azure resources at scale using a structured query language. | ||
| It helps you search, filter, and project resource metadata across subscriptions. | ||
| Resource Graph is useful for inventory, governance checks, and automated analysis workflows. |
There was a problem hiding this comment.
| Azure Resource Graph is a service for querying Azure resources at scale using a structured query language. | |
| It helps you search, filter, and project resource metadata across subscriptions. | |
| Resource Graph is useful for inventory, governance checks, and automated analysis workflows. | |
| Azure Resource Graph is a service for querying Azure resources at scale using a structured query language. | |
| It helps you search, filter, and project resource metadata across subscriptions. | |
| Resource Graph is useful for inventory, governance checks, and automated analysis workflows. | |
| For more information, see [Azure Resource Graph overview](https://learn.microsoft.com/en-us/azure/governance/resource-graph/overview). |
| It helps you search, filter, and project resource metadata across subscriptions. | ||
| Resource Graph is useful for inventory, governance checks, and automated analysis workflows. | ||
|
|
||
| LocalStack for Azure allows you to build and test Resource Graph workflows in your local environment. |
There was a problem hiding this comment.
| LocalStack for Azure allows you to build and test Resource Graph workflows in your local environment. | |
| LocalStack for Azure enables users to explore resources deployed within the local environment using [Kusto Query Language (KQL)](https://learn.microsoft.com/en-us/azure/governance/resource-graph/concepts/query-language#supported-tabulartop-level-operators). |
| Start your LocalStack container using your preferred method. | ||
| Then start CLI interception: | ||
|
|
||
| ```bash | ||
| azlocal start_interception | ||
| ``` |
There was a problem hiding this comment.
| Start your LocalStack container using your preferred method. | |
| Then start CLI interception: | |
| ```bash | |
| azlocal start_interception | |
| ``` | |
| Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: | |
| ```bash | |
| azlocal start-interception | |
| ``` | |
| :::note | |
| As an alternative to using the `azlocal` CLI, users can run: | |
| `azlocal start-interception` | |
| This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. | |
| To revert this configuration, run: | |
| `azlocal stop-interception` | |
| This reconfigures the `az` CLI to send commands to the official Azure management REST API. | |
| ::: |
| The following queries mirror the same patterns used in our validated tests: | ||
|
|
||
| - filter by type: `where type =~ 'Microsoft.Web/sites'` | ||
| - filter by type and name: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81'` | ||
| - filter by name: `where name =~ 'ls-app-doc81'` | ||
| - project only IDs: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81' | project id` | ||
|
|
||
| For example, run a query for all web sites: | ||
|
|
||
| ```bash | ||
| az rest --method post \ | ||
| --url "http://management.localhost.localstack.cloud:4566/providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01" \ | ||
| --headers "Content-Type=application/json" \ | ||
| --body "{\"subscriptions\":[\"00000000-0000-0000-0000-000000000000\"],\"query\":\"where type =~ 'Microsoft.Web/sites'\"}" | ||
| ``` | ||
|
|
||
| ```bash title="Output" | ||
| { | ||
| "count": 1, | ||
| "data": { | ||
| "columns": [ | ||
| { | ||
| "name": "id", | ||
| "type": "string" | ||
| }, | ||
| { | ||
| "name": "name", | ||
| "type": "string" | ||
| }, | ||
| ... | ||
| ], | ||
| "rows": [ | ||
| [ | ||
| "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resourcegraph-demo/providers/Microsoft.Web/sites/ls-app-doc81", | ||
| "ls-app-doc81", | ||
| "westeurope", | ||
| ... | ||
| ] | ||
| ] | ||
| }, | ||
| ... | ||
| "totalRecords": 1 | ||
| } | ||
| ``` | ||
|
|
||
| Or, run a query for an unknown resource name: | ||
|
|
||
| ```bash | ||
| az rest --method post \ | ||
| --url "http://management.localhost.localstack.cloud:4566/providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01" \ | ||
| --headers "Content-Type=application/json" \ | ||
| --body "{\"subscriptions\":[\"00000000-0000-0000-0000-000000000000\"],\"query\":\"where type =~ 'Microsoft.Web/sites' and name =~ 'doesnotexist'\"}" | ||
| ``` | ||
|
|
||
| ```bash title="Output" | ||
| { | ||
| "count": 0, | ||
| "facets": [], | ||
| "resultTruncated": "false", | ||
| "totalRecords": 0 | ||
| } | ||
| ``` |
There was a problem hiding this comment.
| The following queries mirror the same patterns used in our validated tests: | |
| - filter by type: `where type =~ 'Microsoft.Web/sites'` | |
| - filter by type and name: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81'` | |
| - filter by name: `where name =~ 'ls-app-doc81'` | |
| - project only IDs: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81' | project id` | |
| For example, run a query for all web sites: | |
| ```bash | |
| az rest --method post \ | |
| --url "http://management.localhost.localstack.cloud:4566/providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01" \ | |
| --headers "Content-Type=application/json" \ | |
| --body "{\"subscriptions\":[\"00000000-0000-0000-0000-000000000000\"],\"query\":\"where type =~ 'Microsoft.Web/sites'\"}" | |
| ``` | |
| ```bash title="Output" | |
| { | |
| "count": 1, | |
| "data": { | |
| "columns": [ | |
| { | |
| "name": "id", | |
| "type": "string" | |
| }, | |
| { | |
| "name": "name", | |
| "type": "string" | |
| }, | |
| ... | |
| ], | |
| "rows": [ | |
| [ | |
| "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resourcegraph-demo/providers/Microsoft.Web/sites/ls-app-doc81", | |
| "ls-app-doc81", | |
| "westeurope", | |
| ... | |
| ] | |
| ] | |
| }, | |
| ... | |
| "totalRecords": 1 | |
| } | |
| ``` | |
| Or, run a query for an unknown resource name: | |
| ```bash | |
| az rest --method post \ | |
| --url "http://management.localhost.localstack.cloud:4566/providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01" \ | |
| --headers "Content-Type=application/json" \ | |
| --body "{\"subscriptions\":[\"00000000-0000-0000-0000-000000000000\"],\"query\":\"where type =~ 'Microsoft.Web/sites' and name =~ 'doesnotexist'\"}" | |
| ``` | |
| ```bash title="Output" | |
| { | |
| "count": 0, | |
| "facets": [], | |
| "resultTruncated": "false", | |
| "totalRecords": 0 | |
| } | |
| ``` | |
| The Resource Graph extension must be installed to use `az graph` commands. Refer to the [official installation instructions](https://learn.microsoft.com/en-us/azure/governance/resource-graph/first-query-azurecli#install-the-extension). Use the [az graph query](https://learn.microsoft.com/en-us/cli/azure/graph?view=azure-cli-latest#az-graph-query) command to run Kusto Query language (KQL) queries against the emulator. | |
| The following examples demonstrate common query patterns: | |
| ```bash | |
| az graph query \ | |
| --graph-query "Resources | where type =~ 'Microsoft.Web/sites'" | |
| ``` | |
| ```bash title="Output" | |
| { | |
| "count": 1, | |
| "data": [ | |
| { | |
| "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resourcegraph-demo/providers/Microsoft.Web/sites/ls-app-doc81", | |
| "location": "westeurope", | |
| "name": "ls-app-doc81", | |
| "properties": {}, | |
| "resourceGroup": "rg-resourcegraph-demo", | |
| "subscriptionId": "00000000-0000-0000-0000-000000000000", | |
| "type": "Microsoft.Web/sites" | |
| } | |
| ], | |
| "skip_token": null, | |
| "total_records": 1 | |
| } | |
| ``` | |
| Query a web site by type and name, projecting only the `id` column: | |
| ```bash | |
| az graph query \ | |
| --graph-query "Resources | where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81' | project id" | |
| ``` | |
| ```bash title="Output" | |
| { | |
| "count": 1, | |
| "data": [ | |
| { | |
| "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resourcegraph-demo/providers/Microsoft.Web/sites/ls-app-doc81", | |
| "resourceGroup": "rg-resourcegraph-demo" | |
| } | |
| ], | |
| "skip_token": null, | |
| "total_records": 1 | |
| } | |
| ``` | |
| Count all resources in a resource group: | |
| ```bash | |
| az graph query \ | |
| --graph-query "Resources | where resourceGroup =~ 'rg-resourcegraph-demo' | count" | |
| ``` | |
| ```bash title="Output" | |
| { | |
| "count": 1, | |
| "data": [ | |
| { | |
| "Count": 2 | |
| } | |
| ], | |
| "skip_token": null, | |
| "total_records": 1 | |
| } | |
| ``` | |
| List resources sorted by name with a row limit: | |
| ```bash | |
| az graph query \ | |
| --graph-query "Resources | where resourceGroup =~ 'rg-resourcegraph-demo' | project name, type | order by name asc | limit 5" | |
| ``` | |
| ```bash title="Output" | |
| "count": 2, | |
| "data": [ | |
| { | |
| "name": "asp-doc81", | |
| "type": "Microsoft.Web/serverfarms" | |
| }, | |
| { | |
| "name": "ls-app-doc81", | |
| "type": "Microsoft.Web/sites" | |
| } | |
| ], | |
| "skip_token": null, | |
| "total_records": 2 | |
| } | |
| ``` | |
| Query a resource that does not exist: | |
| ```bash | |
| az graph query \ | |
| --graph-query "Resources | where type =~ 'Microsoft.Web/sites' and name =~ 'doesnotexist'" | |
| ``` | |
| ```bash title="Output" | |
| { | |
| "count": 0, | |
| "data": [], | |
| "skip_token": null, | |
| "total_records": 0 | |
| } | |
| ``` | |
| ### Query resources with the REST API | |
| An alternative way to invoke Azure Resource Graph is to call its REST API directly using the [`az rest`](https://learn.microsoft.com/en-us/cli/azure/reference-index?view=azure-cli-latest#az-rest) command: | |
| ```bash | |
| az rest --method post \ | |
| --url "http://azure.localhost.localstack.cloud:4566/providers/Microsoft.ResourceGraph/resources?api-version=2024-04-01" \ | |
| --headers "Content-Type=application/json" \ | |
| --body "{\"subscriptions\":[\"00000000-0000-0000-0000-000000000000\"],\"query\":\"Resources | where type=~'Microsoft.Web/sites'\", \"options\":{\"resultFormat\":\"objectArray\"}}" | |
| ``` | |
| ```bash title="Output" | |
| { | |
| "count": 1, | |
| "data": [ | |
| { | |
| "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resourcegraph-demo/providers/Microsoft.Web/sites/ls-app-doc81", | |
| "location": "westeurope", | |
| "name": "ls-app-doc81", | |
| "properties": {}, | |
| "resourceGroup": "rg-resourcegraph-demo", | |
| "subscriptionId": "00000000-0000-0000-0000-000000000000", | |
| "type": "Microsoft.Web/sites" | |
| } | |
| ], | |
| "facets": [], | |
| "resultTruncated": "false", | |
| "totalRecords": 1 | |
| } | |
| ``` | |
| ## Features | |
| The Resource Graph emulator supports the following features: | |
| - **KQL query engine**: A built-in parser and executor for the Kusto Query Language (KQL) subset used by Azure Resource Graph. | |
| - **Tabular and object result formats**: The `resultFormat` option controls whether results are returned as a column/row table or as an array of objects. | |
| - **Scalar functions**: Built-in functions including `tolower`, `toupper`, `strlen`, `trim`, `substring`, `strcat`, `isnull`, `isnotnull`, `isempty`, `isnotempty`, `tostring`, `toint`, `tolong`, `todouble`, and `coalesce`. | |
| - **Comparison operators**: Full support for `==`, `!=`, `=~`, `!~`, `contains`, `!contains`, `contains_cs`, `!contains_cs`, `startswith`, `!startswith`, `endswith`, `!endswith`, `has`, `!has`, `in`, `!in`, and `matches regex`. | |
| - **Aggregate functions**: `count()`, `dcount()`, `countif()`, `sum()`, `sumif()`, `avg()`, `min()`, and `max()` for use in `summarize` stages. | |
| ## Limitations | |
| - **Single table only**: The emulator queries the `Resources` table. Other Resource Graph tables (such as `ResourceContainers`, `AdvisorResources`, and `SecurityResources`) are not available. | |
| - **No data persistence across restarts**: Resource metadata is not persisted and is lost when the LocalStack emulator is stopped or restarted. | |
| ### Supported tabular operators | |
| The table below lists the KQL tabular operators supported by Azure Resource Graph and their availability in the LocalStack emulator. | |
| For the full reference, see [Supported tabular/top-level operators](https://learn.microsoft.com/en-us/azure/governance/resource-graph/concepts/query-language#supported-tabulartop-level-operators). | |
| | Operator | Supported | Notes | | |
| |---|---|---| | |
| | `count` | Yes | Returns a single row with the total number of input rows. | | |
| | `distinct` | Yes | Deduplicates rows by the specified columns. | | |
| | `extend` | Yes | Adds computed columns to the result set. | | |
| | `join` | No | Cross-table joins are not supported. The emulator does not implement `ResourceContainers` or other secondary tables. | | |
| | `limit` | Yes | Synonym of `take`. | | |
| | `mv-expand` | No | Array expansion into multiple rows is not supported. | | |
| | `order` | Yes | Synonym of `sort`. Supports `asc` and `desc` directions. | | |
| | `parse` | No | String parsing with pattern matching is not supported. | | |
| | `project` | Yes | Supports column selection and aliased expressions. | | |
| | `project-away` | Yes | Removes specified columns from the result set. | | |
| | `sort` | Yes | Synonym of `order`. | | |
| | `summarize` | Yes | Supports aggregate functions with an optional `by` clause. | | |
| | `take` | Yes | Synonym of `limit`. | | |
| | `top` | Yes | Returns the first N rows sorted by specified columns. | | |
| | `union` | No | Combining results from multiple tables is not supported. | | |
| | `where` | Yes | Filters rows using comparison, logical, and string operators. | | |
| ``` |
| - filter by type: `where type =~ 'Microsoft.Web/sites'` | ||
| - filter by type and name: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81'` | ||
| - filter by name: `where name =~ 'ls-app-doc81'` | ||
| - project only IDs: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81' | project id` |
There was a problem hiding this comment.
Thanks @bblommers, I documented features and limitations below. The new version of our Azure Resource Graph emulator supports more operators. See below.
Fixes DOC-81