As we all know, it is possible to use Databricks Column Tags in both schema.yml and as config. The documentation gives this example for apply the tag data_classification with the value public:
models:
- name: customers
columns:
- name: customer_id
databricks_tags:
data_classification: "public"However in version 1.11.3 and at least in 1.11.2, a new warning is raised
Custom key `databricks_tags` found at `models[1].columns[1]` in file
`models/customers/schema.yml`. This may mean the key is a typo, or
is simply not a key supported by the object.
It still works, but the warning seems to indicate that it won't work for much longer. What is the proper way to run going forward?
Debug information
13:49:44 Running with dbt=1.11.0-rc3
WARNING:thrift.transport.sslcompat:using legacy validation callback
13:49:47 Registered adapter: databricks=1.11.3
When running the task with version 1.10.12, we instead get this debug information
05:05:56 Running with dbt=1.10.11
WARNING:thrift.transport.sslcompat:using legacy validation callback
/databricks/python/lib/python3.12/site-packages/pydantic/_internal/_config.py:345: UserWarning: Valid config keys have changed in V2:
* 'allow_population_by_field_name' has been renamed to 'populate_by_name'
warnings.warn(message, UserWarning)
05:05:58 Registered adapter: databricks=1.10.12
As we all know, it is possible to use Databricks Column Tags in both schema.yml and as config. The documentation gives this example for apply the tag data_classification with the value public:
However in version 1.11.3 and at least in 1.11.2, a new warning is raised
It still works, but the warning seems to indicate that it won't work for much longer. What is the proper way to run going forward?
Debug information
When running the task with version 1.10.12, we instead get this debug information