Skip to content

feat: add anomaly detection section to data profiling docs #289

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kevinnguyen-db wants to merge 4 commits into
base: main
Choose a base branch
from
Open

feat: add anomaly detection section to data profiling docs #289

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3dc1621
feat: add anomaly detection section to data profiling docs
kevinnguyen-db Mar 10, 2026
8f499ce
style: fix ruff formatting for auth.py and test_sql.py
kevinnguyen-db Mar 10, 2026
e998212
Revert "style: fix ruff formatting for auth.py and test_sql.py"
kevinnguyen-db Mar 10, 2026
6ae5ac5
docs: remove redundant anomaly detection link
kevinnguyen-db Mar 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions databricks-skills/databricks-unity-catalog/7-data-profiling.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,28 @@ w.data_quality.delete_monitor(

---

## Anomaly Detection

Anomaly detection is enabled at the **schema level**, not per table. Once enabled, Databricks automatically scans all tables in the schema at the same frequency they are updated.

```python
from databricks.sdk.service.dataquality import Monitor, AnomalyDetectionConfig

schema_info = w.schemas.get("catalog.schema")

monitor = w.data_quality.create_monitor(
monitor=Monitor(
object_type="schema",
object_id=schema_info.schema_id,
anomaly_detection_config=AnomalyDetectionConfig(),
),
)
```

> **Note:** Anomaly detection requires `MANAGE SCHEMA` or `MANAGE CATALOG` privileges and serverless compute enabled on the workspace.

---

## Output Tables

When a monitor is created, two metric tables are generated in the specified output schema:
Expand Down
Loading