Skip to content

Conversation

@TovRudyy
Copy link
Contributor

@TovRudyy TovRudyy commented Dec 5, 2025

What does this PR do? What is the motivation?

Updating the docs of Flaky Tests Management documenting the new "Failure rate" automatic rule.

Updating mentions of "Flaky Test Management" to "Flaky Tests Management" for coherence. In our DataDog UI, the product is called "Flaky Tests Management".
image

Merge instructions

Merge readiness:

Do not merge after approving.

  • Ready for merge

For Datadog employees:

Your branch name MUST follow the <name>/<description> convention and include the forward slash (/). Without this format, your pull request will not pass CI, the GitLab pipeline will not run, and you won't get a branch preview. Getting a branch preview makes it easier for us to check any issues with your PR, such as broken links.

If your branch doesn't follow this format, rename it or create a new branch and PR.

[6/5/2025] Merge queue has been disabled on the documentation repo. If you have write access to the repo, the PR has been reviewed by a Documentation team member, and all of the required checks have passed, you can use the Squash and Merge button to merge the PR. If you don't have write access, or you need help, reach out in the #documentation channel in Slack.

Additional notes

@TovRudyy TovRudyy requested review from a team as code owners December 5, 2025 10:47
@github-actions
Copy link
Contributor

github-actions bot commented Dec 5, 2025

Preview links (active after the build_preview check completes)

Modified Files

@estherk15 estherk15 self-assigned this Dec 5, 2025
Copy link
Contributor

@estherk15 estherk15 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a suggestion to fix the list formatting, but approved!

| ---- | ---- |
| **Quarantine** | Toggle to allow flaky tests to be quarantined for this repository. Customize automation rules based on: <li>Time: Quarantine a test if its status is `Active` for a specified number of days. <li>Branch: Quarantine an `Active` test if it flakes in one or more specified branches.|
| **Disable** | Toggle to allow flaky tests to be disabled for this repository. You may want to do this after quarantining or to protect specific branches from flakiness. Customize automation rules based on: <li>Status and time: Disable a test if it has a specified status for a specified number of days. <li>Branch: Disable an `Active` or `Quarantined` test if it flakes in one or more specified branches. |
| **Quarantine** | Toggle to allow flaky tests to be quarantined for this repository. Customize automation rules based on: <li>Time: Quarantine a test if its status is `Active` for a specified number of days. The rule is triggered every day at 12:15 UTC. <li>Branch: Quarantine an `Active` test if it flakes in one or more specified branches. <li>Failure rate: Quarantine an `Active` test if its failure rate over the last 7 days is greater or equal to the specified threshold. The rule is triggered every 15 minutes. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| **Quarantine** | Toggle to allow flaky tests to be quarantined for this repository. Customize automation rules based on: <li>Time: Quarantine a test if its status is `Active` for a specified number of days. The rule is triggered every day at 12:15 UTC. <li>Branch: Quarantine an `Active` test if it flakes in one or more specified branches. <li>Failure rate: Quarantine an `Active` test if its failure rate over the last 7 days is greater or equal to the specified threshold. The rule is triggered every 15 minutes. |
| **Quarantine** | Toggle to allow flaky tests to be quarantined for this repository. Customize automation rules based on: <ul><li>Time: Quarantine a test if its status is `Active` for a specified number of days. The rule is triggered every day at 12:15 UTC. <li>Branch: Quarantine an `Active` test if it flakes in one or more specified branches. <li>Failure rate: Quarantine an `Active` test if its failure rate over the last 7 days is greater or equal to the specified threshold. The rule is triggered every 15 minutes.</ul> |

| **Quarantine** | Toggle to allow flaky tests to be quarantined for this repository. Customize automation rules based on: <li>Time: Quarantine a test if its status is `Active` for a specified number of days. <li>Branch: Quarantine an `Active` test if it flakes in one or more specified branches.|
| **Disable** | Toggle to allow flaky tests to be disabled for this repository. You may want to do this after quarantining or to protect specific branches from flakiness. Customize automation rules based on: <li>Status and time: Disable a test if it has a specified status for a specified number of days. <li>Branch: Disable an `Active` or `Quarantined` test if it flakes in one or more specified branches. |
| **Quarantine** | Toggle to allow flaky tests to be quarantined for this repository. Customize automation rules based on: <li>Time: Quarantine a test if its status is `Active` for a specified number of days. The rule is triggered every day at 12:15 UTC. <li>Branch: Quarantine an `Active` test if it flakes in one or more specified branches. <li>Failure rate: Quarantine an `Active` test if its failure rate over the last 7 days is greater or equal to the specified threshold. The rule is triggered every 15 minutes. |
| **Disable** | Toggle to allow flaky tests to be disabled for this repository. You may want to do this after quarantining or to protect specific branches from flakiness. Customize automation rules based on: <li>Status and time: Disable a test if it has a specified status for a specified number of days. The rule is triggered every day at 12:30 UTC. <li>Branch: Disable an `Active` or `Quarantined` test if it flakes in one or more specified branches. <li>Failure rate: Disable an `Active` or `Quarantined` test if its failure rate over the last 7 days is greater or equal to the specified threshold. The rule is triggered every 15 minutes. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| **Disable** | Toggle to allow flaky tests to be disabled for this repository. You may want to do this after quarantining or to protect specific branches from flakiness. Customize automation rules based on: <li>Status and time: Disable a test if it has a specified status for a specified number of days. The rule is triggered every day at 12:30 UTC. <li>Branch: Disable an `Active` or `Quarantined` test if it flakes in one or more specified branches. <li>Failure rate: Disable an `Active` or `Quarantined` test if its failure rate over the last 7 days is greater or equal to the specified threshold. The rule is triggered every 15 minutes. |
| **Disable** | Toggle to allow flaky tests to be disabled for this repository. You may want to do this after quarantining or to protect specific branches from flakiness. Customize automation rules based on: <li>Status and time: Disable a test if it has a specified status for a specified number of days. The rule is triggered every day at 12:30 UTC. <ul><li>Branch: Disable an `Active` or `Quarantined` test if it flakes in one or more specified branches. <li>Failure rate: Disable an `Active` or `Quarantined` test if its failure rate over the last 7 days is greater or equal to the specified threshold. The rule is triggered every 15 minutes. </ul>|

@TovRudyy
Copy link
Contributor Author

TovRudyy commented Dec 9, 2025

Left a suggestion to fix the list formatting, but approved!

I opted to refactor the table in HTML. It actually looks better:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants