Skip to content

Add Resource Tracking reference for demo/POC cleanup#227

Open
CheeYuTan wants to merge 1 commit intodatabricks-solutions:mainfrom
CheeYuTan:feat/resource-tracking-reference
Open

Add Resource Tracking reference for demo/POC cleanup#227
CheeYuTan wants to merge 1 commit intodatabricks-solutions:mainfrom
CheeYuTan:feat/resource-tracking-reference

Conversation

@CheeYuTan
Copy link
Contributor

Summary

Documents list_tracked_resources and delete_tracked_resource MCP tools which had zero skill coverage. These tools automatically track every resource created via MCP (dashboards, jobs, pipelines, catalogs, volumes, etc.) for cleanup after demos and POCs.

What's in the file (258 lines)

  • Overview — what resource tracking is and why it matters
  • 15 tracked resource types — dashboard, job, pipeline, genie_space, catalog, schema, volume, app, lakebase_instance, vs_endpoint, vs_index, metric_view, knowledge_assistant, multi_agent_supervisor, lakebase_project
  • MCP tools referencelist_tracked_resources (type filter) and delete_tracked_resource (manifest-only vs Databricks deletion)
  • 4 workflows — post-demo cleanup, targeted cleanup by type, manifest-only removal, audit before workspace handoff
  • Behavior notes — safe defaults, cross-session persistence, upsert on recreate, deletion support matrix
  • Troubleshooting — 5 real error patterns

Test evidence — 10 iterations

Verification method

MCP server was intermittently unavailable during testing. Verification done via:

  1. MCP tool JSON schemas — parameter names, types, defaults all confirmed
  2. MCP server source codetrack_resource(), list_tracked_resources(), delete_tracked_resource() implementations reviewed
  3. Prior live tests — 7 MCP calls from earlier session (list all, filter by dashboard/volume, delete non-existent)

What testing found and fixed (6 inaccuracies)

Fix Before (wrong) After (correct) Evidence
Resource types count 9 types listed 15 types (added app, lakebase_instance, lakebase_project, vs_endpoint, vs_index, metric_view) Source code: 15 modules call track_resource()
Schema ID format "Three-level name" Two-level name (catalog.schema) Source: resource_id=f"{catalog}.{schema}"
URL field presence "dashboards and some other types" Dashboards only Source: only aibi_dashboards.py passes url=
Overview resource list Missing catalogs, MAS, apps, Lakebase, VS All 15 types listed Source code audit
Volume vs schema ID Both described as "three-level" Schema=two-level, Volume=three-level Source code
Deletion support Not documented 10 types support Databricks deletion, 5 are manifest-only Source: _RESOURCE_DELETERS dict has 10 entries

Prior live test results (from earlier session)

Tool Input Result Status
list_tracked_resources {} 5 resources (1 catalog, 2 volumes, 1 dashboard, 1 genie_space) PASS
list_tracked_resources {type: "dashboard"} 1 dashboard with url field PASS
list_tracked_resources {type: "volume"} 2 volumes, no url field PASS
list_tracked_resources {type: "job"} {"resources": [], "count": 0} PASS
delete_tracked_resource {type: "job", resource_id: "nonexistent"} {"success": false, "error": "Resource job/nonexistent not found"} PASS
Re-list after failed delete {} All 5 resources intact PASS

Test plan

  • CI validation passes
  • Both MCP tool schemas verified against JSON descriptors
  • All 15 resource types verified against source code
  • Response shapes verified against source code
  • Safe-by-default (delete_from_databricks: false) confirmed in schema
  • 6 inaccuracies found and fixed
  • install_skills.sh updated with new extra file

Documents list_tracked_resources and delete_tracked_resource MCP tools
which had zero skill coverage. These tools track resources created
during MCP sessions for cleanup after demos/POCs.

Verified against MCP tool schemas and source code. Key findings:
- 15 resource types tracked (not 9 as initially documented)
- URL field only present on dashboards (not other types)
- Schema IDs are two-level (catalog.schema), not three-level
- 10 types support Databricks deletion, 5 are manifest-only
- Manifest stored in .databricks-resources.json, persists across sessions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant