Skip to content

Docs for app quality config profiles and comment field #6333

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
marktnoonan wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Docs for app quality config profiles and comment field #6333

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 43 additions & 9 deletions docs/accessibility/configuration/overview.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,45 +22,79 @@ To add or modify the configuration for your project, navigate to the "App Qualit
alt="The Cypress Cloud UI showing the configuration editor"
/>

You can use the provided editor to write configuration in JSON format. A complete configuration with all available options looks as follows:
You can use the provided editor to write configuration in JSON format.

```typescript
### Comments

All configuration objects support an optional `comment` property that you can use to provide context and explanations for why certain values are set. This helps make your configuration easier to understand and maintain, especially when working in teams or revisiting configuration after some time.

```json
{
"elementFilters": [
{
"selector": "[data-testid*='temp']",
"include": false,
"comment": "Exclude temporary test elements from accessibility reports"
}
]
}
```

### Profiles

The `profiles` property allows you to use different configuration settings for different runs based on [run tags](/app/references/command-line#cypress-run-tag-lt-tag-gt). See the [Profiles](/accessibility/configuration/profiles) guide for more details.

### Complete configuration example

A complete configuration with all available options looks as follows:

```json
{
"views": [
{
"pattern": string,
"groupBy": [
string
]
],
"comment": string
}
],
"viewFilters": [
{
"pattern": string,
"include": boolean
"include": boolean,
"comment": string
}
],
"elementFilters": [
{
"selector": string,
"include": boolean
"include": boolean,
"comment": string
}
],
"significantAttributes": [
string
]
],
"attributeFilters": [
{
"attribute": string,
"value": string,
"include": boolean
"include": boolean,
"comment": string
}
],
"profiles": [
{
"name": string,
"config": {
// Any App Quality configuration options
}
}
]
}
```

Note that these root-level App Quality configuration properties (`elementFilters`, `views`, and `viewFilters`) impact both UI Coverage and Accessibility.

### Viewing Configuration for a Run

You can view configuration information for each run in the Properties tab, as shown below. This is the configuration set for the project at the start of the run.
Expand Down
12 changes: 12 additions & 0 deletions docs/accessibility/configuration/profiles.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
sidebar_label: profiles
title: 'Profiles | Cypress Accessibility'
description: 'The `profiles` configuration property allows you to create configuration overrides that are applied based on run tags.'
sidebar_position: 100
---

<ProductHeading product="accessibility" />

# profiles

<Profiles />
4 changes: 3 additions & 1 deletion docs/partials/_attributefilters.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ supported, if you need to split them up.
{
"attribute": string,
"value": string,
"include": boolean
"include": boolean,
"comment": string
}
]
}
Expand All @@ -39,6 +40,7 @@ For every attribute that an element has, the first `attributeFilters` rule for w
| `attribute` | Required | | A regex string to match attribute names |
| `value` | Optional | `.*` | A regex string to match attribute values |
| `include` | Optional | `true` | A boolean to specify whether the matched attribute should be included. |
| `comment` | Optional | | A comment describing the purpose of this filter rule. |

## Examples

Expand Down
7 changes: 5 additions & 2 deletions docs/partials/_elementfilters.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ supported, if you need to split them up.
"elementFilters": [
{
"selector": string,
"include": boolean
"include": boolean,
"comment": string
}
]
}
Expand All @@ -32,6 +33,7 @@ The first `elementFilters` rule for which the selector property matches the elem
| ---------- | -------- | ------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `selector` | Required | | A CSS selector to identify elements. Supports standard CSS selector syntax, including IDs, classes, attributes, and combinators. |
| `include` | Optional | `true` | A boolean indicating whether the matched elements should be included in the report. |
| `comment` | Optional | | A comment describing the purpose of this filter rule. |

## Examples

Expand All @@ -44,7 +46,8 @@ The first `elementFilters` rule for which the selector property matches the elem
"elementFilters": [
{
"selector": "#button-2",
"include": false
"include": false,
"comment": "Exclude test-only button from reports"
}
]
}
Expand Down
249 changes: 249 additions & 0 deletions docs/partials/_profiles.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,249 @@
The `profiles` property allows you to create configuration overrides that are applied based on run tags. This enables you to use different configuration settings for different types of runs, such as regression tests compared to smoke tests environments, or providing scoped results relevant to specific teams.

## Why use profiles?

- **Team specific reporting**: If multiple teams use the same Cypress Cloud project, they may have completely different areas of concern for which pages or DOM elements are included in reports about accessibility or UI Coverage. Profiles allow each team to see and track against only the results that matter to them, and remove all other findings.
- **Run-type customization**: Use different filters or settings for regression runs versus pull requests. It can be useful to have a narrow config for blocking a merge, optimized for the most critical areas of your app -- while still running a wide config on a full regression suite to manage on a difference cadence.
- **Skip runs when needed**: If you know that certain kinds of runs are not going to be valuable for App Quality reporting, you can ignore all view on these runs so that no report is created. In some situations this can improve clarity about when to look at a report and which reports are considered significant.

## How profiles work

Profiles are selected by matching run tags to profile names. When you run Cypress with the [`--tag`](/app/references/command-line#cypress-run-tag-lt-tag-gt) flag, Cypress Cloud looks for a profile whose `name` matches one of the tags. If a match is found, properties defined in profile's `config` properties override the root configuration. Properties defined in the root that are not referenced in the profile will be inherited, meaning you do not need to repeat config values that you want to keep the same.

If more than one tag provided to a run matches a profile in your App Quality profiles array, the first matching profile in the array will be used. The order in which the tags are passed to the run doesn't matter.

## Best practices

Use a naming convention like `aq-config-x` (e.g., `aq-config-regression`, `aq-config-staging`) to make it clear that a tag is used for configuration lookup purposes.

While relying on existing tags works just fine, but being explicit will help avoid accidental removal or changes of important tags.

## Syntax

```json
{
"profiles": [
{
"name": string,
"config": {
// Any App Quality configuration options
},
"comment": string
}
]
}
```

### Options

| Option | Required | Description |
| --------- | -------- | --------------------------------------------------------------------------------------------------------- |
| `name` | Required | The profile name that must match a run tag to activate this profile. |
| `config` | Required | An object containing any App Quality configuration options. These values override the root configuration. |
| `comment` | Optional | A comment describing the purpose of this profile. |

## Examples

### Basic profile structure

#### Config

```json
{
"elementFilters": [
{
"selector": "[data-testid*='temp']",
"include": false,
"comment": "Exclude temporary test elements"
}
],
"profiles": [
{
"name": "aq-config-regression",
"config": {
"elementFilters": [
{
"selector": "[data-testid*='temp']",
"include": false,
"comment": "Exclude temporary test elements in regression runs"
},
{
"selector": "[data-role='debug']",
"include": false,
"comment": "Exclude debug elements in regression runs"
}
]
}
}
]
}
```

#### Usage

To use this profile, run Cypress with a matching tag:

```shell
cypress run --record --tag "aq-config-regression"
```

When this tag is used, the profile's `elementFilters` configuration will override the base `elementFilters`, excluding both temporary test elements and debug elements.

---

### Multiple profiles

You can define multiple profiles for different scenarios:

#### Config

```json
{
"elementFilters": [
{
"selector": "[data-testid*='temp']",
"include": false
}
],
"profiles": [
{
"name": "aq-config-regression",
"config": {
"elementFilters": [
{
"selector": "[data-testid*='temp']",
"include": false
},
{
"selector": "[data-role='debug']",
"include": false,
"comment": "Exclude debug elements in regression runs"
}
]
}
},
{
"name": "aq-config-staging",
"config": {
"viewFilters": [
{
"pattern": "https://staging.example.com/admin/*",
"include": false,
"comment": "Exclude admin pages in staging runs"
}
]
}
}
]
}
```

#### Usage

Use different tags to activate different profiles:

```shell
# For regression runs
cypress run --record --tag "aq-config-regression"

# For staging runs
cypress run --record --tag "aq-config-staging"
```

---

### Profile with nested configuration

Profiles can override configuration at any level, including nested configuration specific to Cypress Accessibility or UI Coverage, if your project has both projects enabled. For example:

#### Config

```json
{
"elementFilters": [
{
"selector": "[data-testid*='temp']",
"include": false
}
],
"uiCoverage": {
"attributeFilters": [
{
"attribute": "id",
"value": ":r.*:",
"include": false,
"comment": "Filter out React auto-generated IDs"
}
]
},
"profiles": [
{
"name": "aq-config-feature-branch",
"config": {
"uiCoverage": {
"elementGroups": [
{
"selector": "[data-feature='new-checkout']",
"name": "New Checkout Flow",
"comment": "Group new checkout elements for feature branch testing"
}
]
}
}
}
]
}
```

#### Usage

```shell
cypress run --record --tag "aq-config-feature-branch"
```

This profile adds element grouping for the new checkout flow while keeping the base configuration for element filters and attribute filters.

---

### Profile selection with multiple matching tags

If you pass multiple tags and more than one matches a profile name, the first matching profile in the `profiles` array is used:

#### Config

```json
{
"profiles": [
{
"name": "aq-config-regression",
"config": {
"elementFilters": [
{
"selector": "[data-role='debug']",
"include": false
}
]
}
},
{
"name": "aq-config-staging",
"config": {
"viewFilters": [
{
"pattern": "https://staging.example.com/admin/*",
"include": false
}
]
}
}
]
}
```

#### Usage

```shell
cypress run --record --tag "aq-config-regression,aq-config-staging"
```

In this case, the `aq-config-regression` profile will be used because it appears first in the `profiles` array, even though both tags match profile names.
Loading