Skip to content

feat: declare CATALOG_V2_CONFIGS for catalogs.yml v2 support#1440

Draft
aahel wants to merge 5 commits intodatabricks:mainfrom
aahel:feat/catalogs-v2-configs
Draft

feat: declare CATALOG_V2_CONFIGS for catalogs.yml v2 support#1440
aahel wants to merge 5 commits intodatabricks:mainfrom
aahel:feat/catalogs-v2-configs

Conversation

@aahel
Copy link
Copy Markdown

@aahel aahel commented May 4, 2026

Summary

Adds UnityDatabricksConfig and HiveMetastoreDatabricksConfig schema dataclasses and declares them on DatabricksAdapter.CATALOG_V2_CONFIGS to support catalogs.yml v2 parse-time validation for Databricks catalog types.

What changed

New file: dbt/adapters/databricks/catalogs/_v2.py

  • UnityDatabricksConfig — validates the config.databricks block for type: unity. Enforces file_format='parquet' when use_uniform is false/unset, 'delta' when use_uniform=true.
  • HiveMetastoreDatabricksConfig — validates config.databricks for type: hive_metastore. Accepts delta, parquet, or hudi.

dbt/adapters/databricks/impl.py

CATALOG_V2_CONFIGS = {
    "unity": UnityDatabricksConfig,
    "hive_metastore": HiveMetastoreDatabricksConfig,
}

Tests

15 unit tests covering registration, valid inputs, use_uniform/file_format cross-field constraint, and invalid inputs.

Adds UnityDatabricksConfig and HiveMetastoreDatabricksConfig dataclasses
and declares them on DatabricksAdapter.CATALOG_V2_CONFIGS. This enables
parse-time validation of catalogs.yml v2 when the use_catalogs_v2
behavior flag is set in dbt-core.

UnityDatabricksConfig enforces file_format='parquet' when use_uniform is
false/unset, and 'delta' when use_uniform=true. HiveMetastoreDatabricksConfig
accepts delta, parquet, or hudi.

Both schemas are dbtClassMixin dataclasses — structural validation
(unknown keys, required fields) via jsonschema; semantic constraints
in __post_init__.
@aahel aahel marked this pull request as draft May 4, 2026 11:20
@aahel aahel marked this pull request as ready for review May 4, 2026 11:59
@aahel aahel marked this pull request as draft May 6, 2026 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant