Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions databricks-skills/databricks-agent-bricks/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,13 @@ manage_mas(
- **[databricks-model-serving](../databricks-model-serving/SKILL.md)** - Deploy custom agent endpoints used as MAS agents
- **[databricks-vector-search](../databricks-vector-search/SKILL.md)** - Build vector indexes for RAG applications paired with KAs

## Resources

- [Databricks Agent Bricks](https://docs.databricks.com/generative-ai/agent-bricks/index.html)
- [Knowledge Assistants](https://docs.databricks.com/generative-ai/agent-bricks/knowledge-assistants.html)
- [Supervisor Agents](https://docs.databricks.com/generative-ai/agent-bricks/supervisor-agents.html)
- [Genie Spaces](https://docs.databricks.com/genie/index.html)

## See Also

- `1-knowledge-assistants.md` - Detailed KA patterns and examples
Expand Down
6 changes: 6 additions & 0 deletions databricks-skills/databricks-aibi-dashboards/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -914,6 +914,12 @@ print(result["url"])
- Use **separate text widgets** for title and subtitle at different y positions
- Example: title at y=0 with height=1, subtitle at y=1 with height=1

## Resources

- [AI/BI Dashboards](https://docs.databricks.com/dashboards/index.html)
- [Lakeview API Reference](https://docs.databricks.com/api/workspace/lakeview)
- [Dashboard Embedding](https://docs.databricks.com/dashboards/embed.html)

## Related Skills

- **[databricks-unity-catalog](../databricks-unity-catalog/SKILL.md)** - for querying the underlying data and system tables
Expand Down
7 changes: 7 additions & 0 deletions databricks-skills/databricks-app-python/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,13 @@ class EntityIn(BaseModel):
- **[app.yaml Reference](https://docs.databricks.com/aws/en/dev-tools/databricks-apps/app-runtime)** — command and env config
- **[System Environment](https://docs.databricks.com/aws/en/dev-tools/databricks-apps/system-env)** — pre-installed packages, runtime details

## Resources

- [Databricks Apps Overview](https://docs.databricks.com/dev-tools/databricks-apps/index.html)
- [App Configuration (app.yaml)](https://docs.databricks.com/dev-tools/databricks-apps/configuration.html)
- [App Resources and Permissions](https://docs.databricks.com/dev-tools/databricks-apps/app-resources.html)
- [Streamlit on Databricks](https://docs.databricks.com/dev-tools/databricks-apps/streamlit.html)

## Related Skills

- **[databricks-app-apx](../databricks-app-apx/SKILL.md)** - full-stack apps with FastAPI + React
Expand Down
11 changes: 11 additions & 0 deletions databricks-skills/databricks-config/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,14 @@ Use the `manage_workspace` MCP tool for all workspace operations. Do NOT edit `~
4. Present the result. For `status`/`switch`/`login`: show host, profile, username. For `list`: formatted table with the active profile marked.

> **Note:** The switch is session-scoped — it resets on MCP server restart. For permanent profile setup, use `databricks auth login -p <profile>` and update `~/.databrickscfg` with `cluster_id` or `serverless_compute_id = auto`.

## Related Skills

- **[databricks-python-sdk](../databricks-python-sdk/SKILL.md)** - SDK patterns that depend on workspace configuration
- **[databricks-docs](../databricks-docs/SKILL.md)** - Documentation reference for authentication methods

## Resources

- [Databricks CLI Authentication](https://docs.databricks.com/dev-tools/cli/authentication.html)
- [Unified Auth Configuration](https://docs.databricks.com/dev-tools/auth/unified-auth.html)
- [Environment Variables](https://docs.databricks.com/dev-tools/auth/env-variables.html)
17 changes: 17 additions & 0 deletions databricks-skills/databricks-dbsql/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -298,3 +298,20 @@ Load these for detailed syntax, full parameter lists, and advanced patterns:
- **Define PK/FK constraints** on dimensional models for query optimization
- **Use `COLLATE UTF8_LCASE`** for user-facing string columns that need case-insensitive search
- **Use MCP tools** (`execute_sql`, `execute_sql_multi`) to test and validate all SQL before deploying

## Related Skills

- **[databricks-unity-catalog](../databricks-unity-catalog/SKILL.md)** - Governance, grants, and catalog management for SQL objects
- **[databricks-aibi-dashboards](../databricks-aibi-dashboards/SKILL.md)** - Building dashboards that consume DBSQL queries
- **[databricks-metric-views](../databricks-metric-views/SKILL.md)** - Defining reusable business metrics in SQL
- **[databricks-spark-declarative-pipelines](../databricks-spark-declarative-pipelines/SKILL.md)** - ETL pipelines using SQL and materialized views
- **[databricks-genie](../databricks-genie/SKILL.md)** - Natural language SQL exploration via Genie Spaces

## Resources

- [Databricks SQL Language Reference](https://docs.databricks.com/sql/language-manual/index.html)
- [AI Functions](https://docs.databricks.com/large-language-models/ai-functions.html)
- [Pipe Syntax](https://docs.databricks.com/sql/language-manual/sql-ref-pipe-syntax.html)
- [Materialized Views](https://docs.databricks.com/sql/language-manual/sql-ref-materialized-views.html)
- [SQL Scripting](https://docs.databricks.com/sql/language-manual/sql-ref-sql-scripting.html)
- [Geospatial H3 Functions](https://docs.databricks.com/sql/language-manual/sql-ref-h3-geospatial-functions.html)
7 changes: 7 additions & 0 deletions databricks-skills/databricks-docs/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,13 @@ The llms.txt file is organized by category:
2. Read the specific docs to understand the feature
3. Determine which skill/tools apply, then use them

## Resources

- [Databricks Documentation](https://docs.databricks.com/)
- [Databricks llms.txt](https://docs.databricks.com/llms.txt)
- [Release Notes](https://docs.databricks.com/release-notes/index.html)
- [REST API Reference](https://docs.databricks.com/api/workspace/introduction)

## Related Skills

- **[databricks-python-sdk](../databricks-python-sdk/SKILL.md)** - SDK patterns for programmatic Databricks access
Expand Down
6 changes: 6 additions & 0 deletions databricks-skills/databricks-genie/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,3 +124,9 @@ Use these skills in sequence:
- **[databricks-synthetic-data-gen](../databricks-synthetic-data-gen/SKILL.md)** - Generate raw parquet data to populate tables for Genie
- **[databricks-spark-declarative-pipelines](../databricks-spark-declarative-pipelines/SKILL.md)** - Build bronze/silver/gold tables consumed by Genie Spaces
- **[databricks-unity-catalog](../databricks-unity-catalog/SKILL.md)** - Manage the catalogs, schemas, and tables Genie queries

## Resources

- [Genie Spaces](https://docs.databricks.com/genie/index.html)
- [Genie Conversation API](https://docs.databricks.com/api/workspace/genie)
- [Genie Best Practices](https://docs.databricks.com/genie/best-practices.html)
6 changes: 6 additions & 0 deletions databricks-skills/databricks-lakebase-autoscale/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -292,3 +292,9 @@ These features are NOT yet supported in Lakebase Autoscaling:
- **[databricks-python-sdk](../databricks-python-sdk/SKILL.md)** - SDK used for project management and token generation
- **[databricks-asset-bundles](../databricks-asset-bundles/SKILL.md)** - deploying apps with Lakebase resources
- **[databricks-jobs](../databricks-jobs/SKILL.md)** - scheduling reverse ETL sync jobs

## Resources

- [Lakebase Overview](https://docs.databricks.com/database/lakebase/index.html)
- [Lakebase Autoscale](https://docs.databricks.com/database/lakebase/autoscale.html)
- [Lakebase Branching](https://docs.databricks.com/database/lakebase/branching.html)
6 changes: 6 additions & 0 deletions databricks-skills/databricks-lakebase-provisioned/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -306,3 +306,9 @@ databricks database start-database-instance --name my-lakebase-instance
- **[databricks-python-sdk](../databricks-python-sdk/SKILL.md)** - SDK used for instance management and token generation
- **[databricks-asset-bundles](../databricks-asset-bundles/SKILL.md)** - deploying apps with Lakebase resources
- **[databricks-jobs](../databricks-jobs/SKILL.md)** - scheduling reverse ETL sync jobs

## Resources

- [Lakebase Overview](https://docs.databricks.com/database/lakebase/index.html)
- [Lakebase Provisioned](https://docs.databricks.com/database/lakebase/provisioned.html)
- [Lakebase Sync (Reverse ETL)](https://docs.databricks.com/database/lakebase/sync.html)
6 changes: 6 additions & 0 deletions databricks-skills/databricks-metric-views/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,12 @@ Metric views work natively with:
- **SQL Editor** - Direct SQL querying with MEASURE()
- **Catalog Explorer UI** - Visual creation and browsing

## Related Skills

- **[databricks-dbsql](../databricks-dbsql/SKILL.md)** - SQL features used in metric view definitions
- **[databricks-unity-catalog](../databricks-unity-catalog/SKILL.md)** - Catalog governance for metric view objects
- **[databricks-aibi-dashboards](../databricks-aibi-dashboards/SKILL.md)** - Dashboards that consume metric views

## Resources

- [Metric Views Documentation](https://docs.databricks.com/en/metric-views/)
Expand Down
7 changes: 7 additions & 0 deletions databricks-skills/databricks-mlflow-evaluation/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,3 +146,10 @@ See `GOTCHAS.md` for complete list.
- **[databricks-agent-bricks](../databricks-agent-bricks/SKILL.md)** - Building agents that can be evaluated with this skill
- **[databricks-python-sdk](../databricks-python-sdk/SKILL.md)** - SDK patterns used alongside MLflow APIs
- **[databricks-unity-catalog](../databricks-unity-catalog/SKILL.md)** - Unity Catalog tables for managed evaluation datasets

## Resources

- [MLflow GenAI Evaluation](https://mlflow.org/docs/latest/llms/llm-evaluate/index.html)
- [MLflow Scorers](https://mlflow.org/docs/latest/llms/llm-evaluate/llm-as-judge.html)
- [MLflow Tracing](https://mlflow.org/docs/latest/llms/tracing/index.html)
- [Databricks Agent Evaluation](https://docs.databricks.com/generative-ai/agent-evaluation/index.html)
6 changes: 6 additions & 0 deletions databricks-skills/databricks-parsing/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,3 +273,9 @@ error_status[] -- errors per page (if any)
- **[databricks-agent-bricks](../databricks-agent-bricks/SKILL.md)** — Pre-built Knowledge Assistants (out-of-the-box RAG without custom parsing)
- **[databricks-spark-declarative-pipelines](../databricks-spark-declarative-pipelines/SKILL.md)** — Production pipelines for batch document processing
- **[databricks-dbsql](../databricks-dbsql/SKILL.md)** — Full AI functions reference including `ai_query`, `ai_extract`, `ai_classify`

## Resources

- [Document Parsing with Databricks](https://docs.databricks.com/generative-ai/tutorials/ai-cookbook/nbs/1-pdf-chunk-extract.html)
- [Unstructured Data in Unity Catalog Volumes](https://docs.databricks.com/ingestion/file-upload/upload-to-volume.html)
- [Foundation Model APIs](https://docs.databricks.com/machine-learning/foundation-models/index.html)
7 changes: 7 additions & 0 deletions databricks-skills/databricks-python-sdk/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -623,3 +623,10 @@ If I'm unsure about a method, I should:
- **[databricks-model-serving](../databricks-model-serving/SKILL.md)** - serving endpoint management
- **[databricks-vector-search](../databricks-vector-search/SKILL.md)** - vector index operations
- **[databricks-lakebase-provisioned](../databricks-lakebase-provisioned/SKILL.md)** - managed PostgreSQL via SDK

## Resources

- [Databricks SDK for Python](https://docs.databricks.com/dev-tools/sdk-python.html)
- [SDK API Reference](https://databricks-sdk-py.readthedocs.io/en/latest/)
- [Databricks Connect](https://docs.databricks.com/dev-tools/databricks-connect/index.html)
- [REST API Reference](https://docs.databricks.com/api/workspace/introduction)
Original file line number Diff line number Diff line change
Expand Up @@ -575,3 +575,11 @@ For advanced configuration options (development mode, continuous pipelines, cust
- **[databricks-asset-bundles](../databricks-asset-bundles/SKILL.md)** - for multi-environment deployment of pipeline projects
- **[databricks-synthetic-data-gen](../databricks-synthetic-data-gen/SKILL.md)** - for generating test data to feed into pipelines
- **[databricks-unity-catalog](../databricks-unity-catalog/SKILL.md)** - for catalog/schema/volume management and governance

## Resources

- [Lakeflow Declarative Pipelines](https://docs.databricks.com/delta-live-tables/index.html)
- [Streaming Tables](https://docs.databricks.com/sql/language-manual/sql-ref-streaming-tables.html)
- [Materialized Views](https://docs.databricks.com/sql/language-manual/sql-ref-materialized-views.html)
- [Change Data Capture](https://docs.databricks.com/delta-live-tables/cdc.html)
- [Auto Loader](https://docs.databricks.com/ingestion/auto-loader/index.html)
15 changes: 15 additions & 0 deletions databricks-skills/databricks-spark-structured-streaming/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,21 @@ df.writeStream \
|-------|-------------|-----------|
| **Production Checklist** | Comprehensive best practices | See [streaming-best-practices.md](streaming-best-practices.md) |

## Related Skills

- **[databricks-spark-declarative-pipelines](../databricks-spark-declarative-pipelines/SKILL.md)** - Declarative streaming tables and DLT pipelines
- **[databricks-unity-catalog](../databricks-unity-catalog/SKILL.md)** - Catalog governance for streaming tables
- **[databricks-zerobus-ingest](../databricks-zerobus-ingest/SKILL.md)** - Near real-time ingestion into Delta tables via gRPC
- **[databricks-dbsql](../databricks-dbsql/SKILL.md)** - SQL queries on streaming table outputs

## Resources

- [Structured Streaming Guide](https://docs.databricks.com/structured-streaming/index.html)
- [Auto Loader](https://docs.databricks.com/ingestion/auto-loader/index.html)
- [Streaming Tables](https://docs.databricks.com/sql/language-manual/sql-ref-streaming-tables.html)
- [RocksDB State Store](https://docs.databricks.com/structured-streaming/rocksdb-state-store.html)
- [Stream-Stream Joins](https://docs.databricks.com/structured-streaming/stream-stream-joins.html)

## Production Checklist

- [ ] Checkpoint location is persistent (UC volumes, not DBFS)
Expand Down
6 changes: 6 additions & 0 deletions databricks-skills/databricks-synthetic-data-gen/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -256,3 +256,9 @@ See [references/5-output-formats.md](references/5-output-formats.md) for detaile
| `F.window` vs `Window` confusion | Use `from pyspark.sql.window import Window` for `row_number()`, `rank()`, etc. `F.window` is for streaming only. |

See [references/6-troubleshooting.md](references/6-troubleshooting.md) for full troubleshooting guide.

## Resources

- [Faker Library](https://faker.readthedocs.io/en/master/)
- [PySpark DataFrame API](https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/dataframe.html)
- [Unity Catalog Volumes](https://docs.databricks.com/connect/unity-catalog/volumes.html)
Original file line number Diff line number Diff line change
Expand Up @@ -192,3 +192,9 @@ AZURE_OPENAI_DEPLOYMENT=gpt-4o
- **[databricks-vector-search](../databricks-vector-search/SKILL.md)** - Index generated documents for semantic search and RAG
- **[databricks-synthetic-data-gen](../databricks-synthetic-data-gen/SKILL.md)** - Generate structured tabular data (complement to unstructured PDFs)
- **[databricks-mlflow-evaluation](../databricks-mlflow-evaluation/SKILL.md)** - Evaluate RAG systems using the generated question/guideline pairs

## Resources

- [Unity Catalog Volumes](https://docs.databricks.com/connect/unity-catalog/volumes.html)
- [ReportLab Documentation](https://docs.reportlab.com/)
- [Document Parsing with Databricks](https://docs.databricks.com/generative-ai/tutorials/ai-cookbook/nbs/1-pdf-chunk-extract.html)
7 changes: 7 additions & 0 deletions databricks-skills/databricks-vector-search/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -360,3 +360,10 @@ The following MCP tools are available for managing Vector Search infrastructure.
- **[databricks-unstructured-pdf-generation](../databricks-unstructured-pdf-generation/SKILL.md)** - Generate documents to index in Vector Search
- **[databricks-unity-catalog](../databricks-unity-catalog/SKILL.md)** - Manage the catalogs and tables that back Delta Sync indexes
- **[databricks-spark-declarative-pipelines](../databricks-spark-declarative-pipelines/SKILL.md)** - Build Delta tables used as Vector Search sources

## Resources

- [Vector Search Overview](https://docs.databricks.com/generative-ai/vector-search/index.html)
- [Vector Search API Reference](https://docs.databricks.com/api/workspace/vectorsearchindexes)
- [Embedding Models](https://docs.databricks.com/machine-learning/foundation-models/index.html)
- [RAG Tutorial](https://docs.databricks.com/generative-ai/tutorials/ai-cookbook/index.html)
8 changes: 2 additions & 6 deletions databricks-tools-core/databricks_tools_core/auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -160,9 +160,7 @@ def get_workspace_client() -> WorkspaceClient:
# Cross-workspace: explicit token overrides env OAuth so tool operations
# target the caller-specified workspace instead of the app's own workspace
if force and host and token:
return tag_client(
WorkspaceClient(host=host, token=token, auth_type="pat", **product_kwargs)
)
return tag_client(WorkspaceClient(host=host, token=token, auth_type="pat", **product_kwargs))

# In Databricks Apps (OAuth credentials in env), explicitly use OAuth M2M.
# Setting auth_type="oauth-m2m" prevents the SDK from also reading
Expand All @@ -185,9 +183,7 @@ def get_workspace_client() -> WorkspaceClient:

# Development mode: use explicit token if provided
if host and token:
return tag_client(
WorkspaceClient(host=host, token=token, auth_type="pat", **product_kwargs)
)
return tag_client(WorkspaceClient(host=host, token=token, auth_type="pat", **product_kwargs))

if host:
return tag_client(WorkspaceClient(host=host, **product_kwargs))
Expand Down
Loading