diff --git a/content/en/monitors/types/data_observability.md b/content/en/monitors/types/data_observability.md index 4f13abba145..1a98fe7a6b1 100644 --- a/content/en/monitors/types/data_observability.md +++ b/content/en/monitors/types/data_observability.md @@ -1,6 +1,6 @@ --- title: Data Observability Monitor -description: "Monitor freshness, row count, column-level metrics, and custom SQL queries across your data warehouses." +description: "Monitor freshness, row count, column-level metrics, and custom SQL queries across your data warehouses, and monitor data job run failures." further_reading: - link: "/data_observability/" tag: "Documentation" @@ -8,6 +8,9 @@ further_reading: - link: "/data_observability/quality_monitoring/" tag: "Documentation" text: "Quality Monitoring" + - link: "/data_observability/jobs_monitoring/" + tag: "Documentation" + text: "Jobs Monitoring" - link: "/monitors/notify/" tag: "Documentation" text: "Configure your monitor notifications" @@ -271,6 +274,88 @@ Detect when a column's null percentage exceeds normal levels, which may indicate {{% /tab %}} {{< /tabs >}} +## Job monitors + +
Job monitors are in preview. Contact your Datadog representative or support to request access.
+ +Job monitors alert you when a data job reaches a threshold of consecutive failed runs. Job monitors require [Jobs Monitoring][11] to be set up with at least one supported job source. + +### Monitor creation + +To create a job monitor, navigate to {{< ui >}}Monitors{{< /ui >}} > {{< ui >}}New Monitor{{< /ui >}} > {{< ui >}}Job{{< /ui >}}. + +### Choose jobs to monitor + +1. Select a technology from the {{< ui >}}Select a technology{{< /ui >}} menu: {{< ui >}}Airflow{{< /ui >}}, {{< ui >}}AWS Glue{{< /ui >}}, {{< ui >}}Azure Data Factory{{< /ui >}}, {{< ui >}}Databricks{{< /ui >}}, {{< ui >}}dbt{{< /ui >}} (Job, Model, or Test), {{< ui >}}Spark{{< /ui >}}, or {{< ui >}}Custom Job{{< /ui >}} for jobs that emit [custom OpenLineage events][12]. + - If you select {{< ui >}}Custom Job{{< /ui >}}, also select the specific operation to monitor from the {{< ui >}}Select an operation{{< /ui >}} menu. +1. Optionally, narrow the jobs matched by this monitor using the {{< ui >}}Filter by tags{{< /ui >}} field (for example, `env:prod`). + +The preview panel shows the job names and recent run statistics that currently match your selection. + + + +### Configure alert conditions + +1. Under {{< ui >}}Define alerting context{{< /ui >}}, optionally add {{< ui >}}Group by{{< /ui >}} tags. Datadog automatically groups by `job_name`, plus additional tags specific to the selected technology (for example, `workspace_name` for Databricks, `project` for dbt). +1. Under {{< ui >}}Detection Method - Threshold{{< /ui >}}, set {{< ui >}}Trigger when{{< /ui >}} to the number of consecutive failed runs that should trigger an alert. + +The monitor automatically recovers after the next run succeeds. + +**Note**: Job monitors alert on consecutive run failures. They do not currently support alerting on job run duration. + +### Example notification + +{{< code-block lang="text" >}} +{{#is_alert}} +Databricks job **{{job_name.name}} ({{workspace_name.name}})** has reached {{value}} consecutive failures. + +Error: {{job.run_error}} + +[View in Databricks]({{job.run_url}}) +{{/is_alert}} +{{#is_recovery}} +Databricks job **{{job_name.name}}** recovered. +{{/is_recovery}} +{{< /code-block >}} + +## Schema change monitors + +
Schema change monitors are in preview. Contact your Datadog representative or support to request access.
+ +Schema change monitors alert you when a database, schema, table, or column is added, removed, renamed, or has its type changed. + +### Monitor creation + +To create a schema change monitor, navigate to {{< ui >}}Monitors{{< /ui >}} > {{< ui >}}New Monitor{{< /ui >}} > {{< ui >}}Schema Change{{< /ui >}}. + +### Choose data to monitor + +1. Select a warehouse from the {{< ui >}}Select warehouse{{< /ui >}} menu. +1. Use {{< ui >}}Choose entities within the warehouse to monitor{{< /ui >}} to filter by `account`, `database`, `schema`, `table`, or `column`. + +### Alert conditions + +Schema change monitors alert on any matching change (there is currently no configurable threshold). Each alert reports the affected entity's fully-qualified name, the change type (`added`, `removed`, `renamed`, or `type_changed`), and — for renames and type changes — the old and new values. + +**Note**: You cannot yet configure a custom alert message template for schema change monitors; alerts use a standard format grouped by change type and entity. + +## Source-to-target monitors + +
Source-to-target monitors are in preview. Contact your Datadog representative or support to request access.
+ +Source-to-target monitors compare a metric computed on a source table or column against the same metric computed on a target table or column — for example, to confirm that a row count in a source system matches the row count after it lands in a warehouse. Use them to catch data loss or drift introduced during a transformation or copy step. + +### Monitor creation + +1. Navigate to {{< ui >}}Monitors{{< /ui >}} > {{< ui >}}New Monitor{{< /ui >}} > {{< ui >}}Data Quality{{< /ui >}}, then select {{< ui >}}Source to Target{{< /ui >}}. +1. Choose a warehouse and a source table or column. +1. Choose a second warehouse and a target table or column. The target must be the same entity type as the source (table-to-table or column-to-column). +1. Select a metric type, following the same options as [Quality monitors](#select-your-metric-type). +1. Choose a comparison format: {{< ui >}}Difference{{< /ui >}} (`target - source`) or {{< ui >}}Percent Difference{{< /ui >}} (`(target - source) / source`). +1. Set an alert threshold using {{< ui >}}more than{{< /ui >}} or {{< ui >}}less than{{< /ui >}}. + +**Note**: Source and target metrics must be computed within 30 minutes of each other, or the comparison for that evaluation is skipped. + ## Annotate bounds For monitors using the **Anomaly** detection method, you can annotate bound ranges to provide feedback and improve the model over time. Unlike infrastructure metrics, data quality metrics are often business-specific, so use annotations to teach the model what behavior is normal for your data. @@ -300,3 +385,5 @@ On a monitor's status page, click {{< ui >}}Annotate Bounds{{< /ui >}}, select a [8]: /monitors/configuration/?tab=thresholdalert#thresholds [9]: /help/ [10]: /api/latest/data-observability/ +[11]: /data_observability/jobs_monitoring/ +[12]: /data_observability/jobs_monitoring/openlineage/