Skip to content

Add Quality Monitors reference to Unity Catalog skill#226

Open
CheeYuTan wants to merge 1 commit intodatabricks-solutions:mainfrom
CheeYuTan:feat/quality-monitors-reference
Open

Add Quality Monitors reference to Unity Catalog skill#226
CheeYuTan wants to merge 1 commit intodatabricks-solutions:mainfrom
CheeYuTan:feat/quality-monitors-reference

Conversation

@CheeYuTan
Copy link
Contributor

Summary

Documents the manage_uc_monitors MCP tool which had zero skill coverage. Adds 8-quality-monitors.md as a new reference file under the Unity Catalog skill.

What's in the file (384 lines)

  • MCP tool reference — all 5 actions (create, get, run_refresh, list_refreshes, delete) with parameters
  • Verified patterns — create with/without schedule, get, refresh, list refreshes, delete
  • Output table schemas — all 28 profile metrics columns and 22 drift metrics columns
  • SQL queries — discovering monitors, querying profiles, detecting drift, checking freshness
  • Troubleshooting — 7 real error patterns with solutions

Test evidence — full lifecycle on live workspace

E2E monitor lifecycle

Step Action Input Result Status
1 Create monitor table_name="main.pizza_llc.menu_items" MONITOR_STATUS_ACTIVE, dashboard_id assigned PASS
2 Get monitor same table Full config returned with output table names PASS
3 Run refresh same table refresh_id returned, state=PENDING, trigger=MANUAL PASS
4 List refreshes same table refreshes array with correct shape PASS
5 Query profile metrics SELECT * FROM ...menu_items_profile_metrics All 28 columns present and queryable PASS
6 Run 2nd refresh same table Triggered successfully PASS
7 Query drift metrics SELECT * FROM ...menu_items_drift_metrics 8 rows after 2nd refresh (0 after 1st — confirmed 2+ needed) PASS
8 Delete monitor same table Returns {} (empty dict) PASS

What testing found and fixed

Fix Before (wrong) After (correct) How discovered
Delete response {"status": "deleted", "table_name": "..."} {} (empty dict) Live delete returned {}
Profile metrics schema 19 columns listed 28 columns (added 9 missing) DESCRIBE TABLE on real output table
Drift metrics schema 14 columns listed 22 columns (added 8 missing) DESCRIBE TABLE on real output table
assets_dir behavior "auto-generated" Clarified MCP vs REST API difference Tested both paths

Key verified claims

  • No list action exists — GET /unity-catalog/monitors returns "No API found"
  • Only snapshot monitors via MCP (TimeSeries requires Python SDK)
  • Drift metrics require 2+ refreshes to populate (confirmed: 0 rows after 1st, 8 after 2nd)
  • Output tables follow {table}_profile_metrics / {table}_drift_metrics naming

Test plan

  • CI validation passes
  • MCP tool schema verified against JSON descriptor
  • Full lifecycle tested: create → get → refresh → query → delete
  • All 28 profile metrics columns verified against real output
  • All 22 drift metrics columns verified against real output
  • All SQL query examples executed successfully
  • Test monitors cleaned up after testing

Documents the manage_uc_monitors MCP tool which had zero skill coverage.
Covers the full monitor lifecycle: create → get → refresh → query → delete.

Tested end-to-end against a live workspace with real monitors:
- Created monitors on real Delta tables
- Triggered manual refreshes and verified output tables
- Verified all 28 profile metrics columns and 22 drift metrics columns
- Confirmed drift metrics require 2+ refreshes to populate
- Verified delete response is {} (not a status object)
- All test monitors cleaned up after testing

Key findings from testing:
- Profile metrics schema had 9 undocumented columns (now included)
- Drift metrics schema had 8 undocumented columns (now included)
- Delete returns empty dict, not status object
- No "list" action exists — must use information_schema to discover
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant