Skip to content

Add Claude Skill for AI Functions#220

Open
CEDipEngineering wants to merge 1 commit intodatabricks-solutions:mainfrom
CEDipEngineering:main
Open

Add Claude Skill for AI Functions#220
CEDipEngineering wants to merge 1 commit intodatabricks-solutions:mainfrom
CEDipEngineering:main

Conversation

@CEDipEngineering
Copy link

Summary

Adds a new databricks-ai-functions skill covering all Databricks built-in AI Functions
with SQL and PySpark examples, function selection guidance, and an end-to-end document
processing pipeline pattern.

Why This Is Useful

Databricks AI Functions (ai_classify, ai_extract, ai_query, ai_forecast, etc.)
are often underused because developers default to ai_query for everything or reach for
external LLM libraries when a simpler built-in function would suffice. This skill gives
Claude Code the context to:

  • Pick the right function for the task (task-specific functions first, ai_query only as
    a last resort for nested JSON / custom endpoints / multimodal input)
  • Write correct SQL and PySpark for each function without hallucinating parameter names
    or endpoint names
  • Build end-to-end document processing pipelines using the recommended stage order:
    ai_parse_documentai_classifyai_extractai_query (nested only) →
    ai_similarity
  • Use ai_forecast correctly (it's a table-valued function, not a scalar — a common
    mistake)
  • Apply failOnError => false and sidecar error tables in batch pipelines

Files Changed

New

File Contents
databricks-skills/databricks-ai-functions/SKILL.md Master skill file with YAML front matter, overview, prerequisites, quick start, common patterns, reference index, and common issues table
databricks-skills/databricks-ai-functions/1-task-functions.md Full reference for all 10 task-specific functions (ai_analyze_sentiment, ai_classify, ai_extract, ai_fix_grammar, ai_gen, ai_mask, ai_similarity, ai_summarize, ai_translate, ai_parse_document) with SQL + PySpark examples
databricks-skills/databricks-ai-functions/2-ai-query.md Full ai_query reference: when to use it, all parameters with runtime requirements, Foundation Model name table, patterns (structured output, multimodal, SQL UDF, batch error handling)
databricks-skills/databricks-ai-functions/3-ai-forecast.md Full ai_forecast reference: Pro/Serverless requirement, syntax, output columns, single/multi-group/multi-metric patterns, PySpark via spark.sql()
databricks-skills/databricks-ai-functions/4-document-processing-pipeline.md End-to-end DLT pipeline example (5 stages), config.yml centralization pattern, near-real-time variant with DSPy + MLflow ChatAgent

Modified

File Change
databricks-skills/README.md Added databricks-ai-functions entry to the AI & Agents section
README.md Updated skill count: 19 → 20

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant