Skip to content

add Azure Resource Graph service doc#463

Open
HarshCasper wants to merge 1 commit intoazure-docsfrom
harshmishra/doc-81
Open

add Azure Resource Graph service doc#463
HarshCasper wants to merge 1 commit intoazure-docsfrom
harshmishra/doc-81

Conversation

@HarshCasper
Copy link
Copy Markdown
Member

Fixes DOC-81

@cloudflare-workers-and-pages
Copy link
Copy Markdown

Deploying localstack-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 0924625
Status: ✅  Deploy successful!
Preview URL: https://0a4241df.localstack-docs.pages.dev
Branch Preview URL: https://harshmishra-doc-81.localstack-docs.pages.dev

View logs

- filter by type: `where type =~ 'Microsoft.Web/sites'`
- filter by type and name: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81'`
- filter by name: `where name =~ 'ls-app-doc81'`
- project only IDs: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81' | project id`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth pointing out the known limitations here?

For example:

Suggested change
- project only IDs: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81' | project id`
The following queries are all supported:
- filter by type: `where type =~ 'Microsoft.Web/sites'`
- filter by type and name: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81'`
- filter by name: `where name =~ 'ls-app-doc81'`
- project only IDs: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81' | project id`
-
Known limitations: the Localstack for Azure emulator currently does not support the following constructs:
- nested queries
- the `count` keyword
- the `extend` keyword
- the `join` keyword
- the `limit` keyword
- the `mv-expand` keyword
- the `order` keyword
- the `parse` keyword
- the `project-away` keyword
- the `sort` keyword
- the `summarize` keyword
- the `take` keyword
- the `top` keyword
- the `union` keyword

For reference, the full list of operators can be found here:
https://learn.microsoft.com/en-us/azure/governance/resource-graph/concepts/query-language#supported-tabulartop-level-operators

Considering we only support a small subset of this list, it could also be easier to only show the constructs that we do support!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bblommers, I documented features and limitations below. The new version of our Azure Resource Graph emulator supports more operators. See below.

Copy link
Copy Markdown
Contributor

@paolosalvatori paolosalvatori left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@HarshCasper / @bblommers, while using az rest to call the Azure Resource Graph is perfectly doable, it's very unusual. Users prefer to install to the az extension resource-graph as follows:

az extension add --name resource-graph

and use it as shown in Quickstart: Run Resource Graph query using Azure CLI

az graph query --graph-query 'Resources | project name, type | limit 5'
az graph query --graph-query 'Resources | where type =~ 'Microsoft.Storage/storageAccounts' | project name, location`
...

I strongly suggest to refactor the article to use az graph. Said that, I would also keep the az rest you and Bert created to show readers an additional method to invoke the Azure Resource Graph API in Azure as well as our emulator.

I realized that the current implementation throws exceptions with some licit queries, so we need to fix the codebase. I'll take this offline with Bert.

cc: @quetzalliwrites

Copy link
Copy Markdown
Contributor

@paolosalvatori paolosalvatori left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @HarshCasper, see my comments, amendments, suggestions, and additions.

Comment on lines +11 to +13
Azure Resource Graph is a service for querying Azure resources at scale using a structured query language.
It helps you search, filter, and project resource metadata across subscriptions.
Resource Graph is useful for inventory, governance checks, and automated analysis workflows.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Azure Resource Graph is a service for querying Azure resources at scale using a structured query language.
It helps you search, filter, and project resource metadata across subscriptions.
Resource Graph is useful for inventory, governance checks, and automated analysis workflows.
Azure Resource Graph is a service for querying Azure resources at scale using a structured query language.
It helps you search, filter, and project resource metadata across subscriptions.
Resource Graph is useful for inventory, governance checks, and automated analysis workflows.
For more information, see [Azure Resource Graph overview](https://learn.microsoft.com/en-us/azure/governance/resource-graph/overview).

It helps you search, filter, and project resource metadata across subscriptions.
Resource Graph is useful for inventory, governance checks, and automated analysis workflows.

LocalStack for Azure allows you to build and test Resource Graph workflows in your local environment.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
LocalStack for Azure allows you to build and test Resource Graph workflows in your local environment.
LocalStack for Azure enables users to explore resources deployed within the local environment using [Kusto Query Language (KQL)](https://learn.microsoft.com/en-us/azure/governance/resource-graph/concepts/query-language#supported-tabulartop-level-operators).

Comment on lines +22 to +27
Start your LocalStack container using your preferred method.
Then start CLI interception:

```bash
azlocal start_interception
```
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Start your LocalStack container using your preferred method.
Then start CLI interception:
```bash
azlocal start_interception
```
Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running:
```bash
azlocal start-interception
```
:::note
As an alternative to using the `azlocal` CLI, users can run:
`azlocal start-interception`
This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API.
To revert this configuration, run:
`azlocal stop-interception`
This reconfigures the `az` CLI to send commands to the official Azure management REST API.
:::

Comment on lines +112 to +173
The following queries mirror the same patterns used in our validated tests:

- filter by type: `where type =~ 'Microsoft.Web/sites'`
- filter by type and name: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81'`
- filter by name: `where name =~ 'ls-app-doc81'`
- project only IDs: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81' | project id`

For example, run a query for all web sites:

```bash
az rest --method post \
--url "http://management.localhost.localstack.cloud:4566/providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01" \
--headers "Content-Type=application/json" \
--body "{\"subscriptions\":[\"00000000-0000-0000-0000-000000000000\"],\"query\":\"where type =~ 'Microsoft.Web/sites'\"}"
```

```bash title="Output"
{
"count": 1,
"data": {
"columns": [
{
"name": "id",
"type": "string"
},
{
"name": "name",
"type": "string"
},
...
],
"rows": [
[
"/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resourcegraph-demo/providers/Microsoft.Web/sites/ls-app-doc81",
"ls-app-doc81",
"westeurope",
...
]
]
},
...
"totalRecords": 1
}
```

Or, run a query for an unknown resource name:

```bash
az rest --method post \
--url "http://management.localhost.localstack.cloud:4566/providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01" \
--headers "Content-Type=application/json" \
--body "{\"subscriptions\":[\"00000000-0000-0000-0000-000000000000\"],\"query\":\"where type =~ 'Microsoft.Web/sites' and name =~ 'doesnotexist'\"}"
```

```bash title="Output"
{
"count": 0,
"facets": [],
"resultTruncated": "false",
"totalRecords": 0
}
```
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The following queries mirror the same patterns used in our validated tests:
- filter by type: `where type =~ 'Microsoft.Web/sites'`
- filter by type and name: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81'`
- filter by name: `where name =~ 'ls-app-doc81'`
- project only IDs: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81' | project id`
For example, run a query for all web sites:
```bash
az rest --method post \
--url "http://management.localhost.localstack.cloud:4566/providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01" \
--headers "Content-Type=application/json" \
--body "{\"subscriptions\":[\"00000000-0000-0000-0000-000000000000\"],\"query\":\"where type =~ 'Microsoft.Web/sites'\"}"
```
```bash title="Output"
{
"count": 1,
"data": {
"columns": [
{
"name": "id",
"type": "string"
},
{
"name": "name",
"type": "string"
},
...
],
"rows": [
[
"/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resourcegraph-demo/providers/Microsoft.Web/sites/ls-app-doc81",
"ls-app-doc81",
"westeurope",
...
]
]
},
...
"totalRecords": 1
}
```
Or, run a query for an unknown resource name:
```bash
az rest --method post \
--url "http://management.localhost.localstack.cloud:4566/providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01" \
--headers "Content-Type=application/json" \
--body "{\"subscriptions\":[\"00000000-0000-0000-0000-000000000000\"],\"query\":\"where type =~ 'Microsoft.Web/sites' and name =~ 'doesnotexist'\"}"
```
```bash title="Output"
{
"count": 0,
"facets": [],
"resultTruncated": "false",
"totalRecords": 0
}
```
The Resource Graph extension must be installed to use `az graph` commands. Refer to the [official installation instructions](https://learn.microsoft.com/en-us/azure/governance/resource-graph/first-query-azurecli#install-the-extension). Use the [az graph query](https://learn.microsoft.com/en-us/cli/azure/graph?view=azure-cli-latest#az-graph-query) command to run Kusto Query language (KQL) queries against the emulator.
The following examples demonstrate common query patterns:
```bash
az graph query \
--graph-query "Resources | where type =~ 'Microsoft.Web/sites'"
```
```bash title="Output"
{
"count": 1,
"data": [
{
"id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resourcegraph-demo/providers/Microsoft.Web/sites/ls-app-doc81",
"location": "westeurope",
"name": "ls-app-doc81",
"properties": {},
"resourceGroup": "rg-resourcegraph-demo",
"subscriptionId": "00000000-0000-0000-0000-000000000000",
"type": "Microsoft.Web/sites"
}
],
"skip_token": null,
"total_records": 1
}
```
Query a web site by type and name, projecting only the `id` column:
```bash
az graph query \
--graph-query "Resources | where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81' | project id"
```
```bash title="Output"
{
"count": 1,
"data": [
{
"id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resourcegraph-demo/providers/Microsoft.Web/sites/ls-app-doc81",
"resourceGroup": "rg-resourcegraph-demo"
}
],
"skip_token": null,
"total_records": 1
}
```
Count all resources in a resource group:
```bash
az graph query \
--graph-query "Resources | where resourceGroup =~ 'rg-resourcegraph-demo' | count"
```
```bash title="Output"
{
"count": 1,
"data": [
{
"Count": 2
}
],
"skip_token": null,
"total_records": 1
}
```
List resources sorted by name with a row limit:
```bash
az graph query \
--graph-query "Resources | where resourceGroup =~ 'rg-resourcegraph-demo' | project name, type | order by name asc | limit 5"
```
```bash title="Output"
"count": 2,
"data": [
{
"name": "asp-doc81",
"type": "Microsoft.Web/serverfarms"
},
{
"name": "ls-app-doc81",
"type": "Microsoft.Web/sites"
}
],
"skip_token": null,
"total_records": 2
}
```
Query a resource that does not exist:
```bash
az graph query \
--graph-query "Resources | where type =~ 'Microsoft.Web/sites' and name =~ 'doesnotexist'"
```
```bash title="Output"
{
"count": 0,
"data": [],
"skip_token": null,
"total_records": 0
}
```
### Query resources with the REST API
An alternative way to invoke Azure Resource Graph is to call its REST API directly using the [`az rest`](https://learn.microsoft.com/en-us/cli/azure/reference-index?view=azure-cli-latest#az-rest) command:
```bash
az rest --method post \
--url "http://azure.localhost.localstack.cloud:4566/providers/Microsoft.ResourceGraph/resources?api-version=2024-04-01" \
--headers "Content-Type=application/json" \
--body "{\"subscriptions\":[\"00000000-0000-0000-0000-000000000000\"],\"query\":\"Resources | where type=~'Microsoft.Web/sites'\", \"options\":{\"resultFormat\":\"objectArray\"}}"
```
```bash title="Output"
{
"count": 1,
"data": [
{
"id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resourcegraph-demo/providers/Microsoft.Web/sites/ls-app-doc81",
"location": "westeurope",
"name": "ls-app-doc81",
"properties": {},
"resourceGroup": "rg-resourcegraph-demo",
"subscriptionId": "00000000-0000-0000-0000-000000000000",
"type": "Microsoft.Web/sites"
}
],
"facets": [],
"resultTruncated": "false",
"totalRecords": 1
}
```
## Features
The Resource Graph emulator supports the following features:
- **KQL query engine**: A built-in parser and executor for the Kusto Query Language (KQL) subset used by Azure Resource Graph.
- **Tabular and object result formats**: The `resultFormat` option controls whether results are returned as a column/row table or as an array of objects.
- **Scalar functions**: Built-in functions including `tolower`, `toupper`, `strlen`, `trim`, `substring`, `strcat`, `isnull`, `isnotnull`, `isempty`, `isnotempty`, `tostring`, `toint`, `tolong`, `todouble`, and `coalesce`.
- **Comparison operators**: Full support for `==`, `!=`, `=~`, `!~`, `contains`, `!contains`, `contains_cs`, `!contains_cs`, `startswith`, `!startswith`, `endswith`, `!endswith`, `has`, `!has`, `in`, `!in`, and `matches regex`.
- **Aggregate functions**: `count()`, `dcount()`, `countif()`, `sum()`, `sumif()`, `avg()`, `min()`, and `max()` for use in `summarize` stages.
## Limitations
- **Single table only**: The emulator queries the `Resources` table. Other Resource Graph tables (such as `ResourceContainers`, `AdvisorResources`, and `SecurityResources`) are not available.
- **No data persistence across restarts**: Resource metadata is not persisted and is lost when the LocalStack emulator is stopped or restarted.
### Supported tabular operators
The table below lists the KQL tabular operators supported by Azure Resource Graph and their availability in the LocalStack emulator.
For the full reference, see [Supported tabular/top-level operators](https://learn.microsoft.com/en-us/azure/governance/resource-graph/concepts/query-language#supported-tabulartop-level-operators).
| Operator | Supported | Notes |
|---|---|---|
| `count` | Yes | Returns a single row with the total number of input rows. |
| `distinct` | Yes | Deduplicates rows by the specified columns. |
| `extend` | Yes | Adds computed columns to the result set. |
| `join` | No | Cross-table joins are not supported. The emulator does not implement `ResourceContainers` or other secondary tables. |
| `limit` | Yes | Synonym of `take`. |
| `mv-expand` | No | Array expansion into multiple rows is not supported. |
| `order` | Yes | Synonym of `sort`. Supports `asc` and `desc` directions. |
| `parse` | No | String parsing with pattern matching is not supported. |
| `project` | Yes | Supports column selection and aliased expressions. |
| `project-away` | Yes | Removes specified columns from the result set. |
| `sort` | Yes | Synonym of `order`. |
| `summarize` | Yes | Supports aggregate functions with an optional `by` clause. |
| `take` | Yes | Synonym of `limit`. |
| `top` | Yes | Returns the first N rows sorted by specified columns. |
| `union` | No | Combining results from multiple tables is not supported. |
| `where` | Yes | Filters rows using comparison, logical, and string operators. |
```

- filter by type: `where type =~ 'Microsoft.Web/sites'`
- filter by type and name: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81'`
- filter by name: `where name =~ 'ls-app-doc81'`
- project only IDs: `where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81' | project id`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bblommers, I documented features and limitations below. The new version of our Azure Resource Graph emulator supports more operators. See below.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants