Feat: Add config flag to infer the state schema per dbt target

[erindru]

Currently, when storing state in the warehouse, by default SQLMesh assumes that you only have a single project accessing the warehouse and creates a schema called sqlmesh to store state in. This is because it also assumes that you want to use Virtual Data Environments.

However, this creates a problem in dbt projects with existing workflows that are not built around VDE's. A common pattern is to use different targets to point to the same warehouse and just override the default schema.

For example, a dev target may populate models into a dev schema while a prod target may put them in a prod schema, but these schemas exist side by side in the same warehouse. Creating models using --target dev allows analysts to test things out and then deploy by running the models against --target prod.

This creates a problem for SQLMesh because it assumes a single state schema but the state for these targets should not overlap.

So this PR:

• Adds a new section to the root config called dbt with, for now, a single flag infer_state_schema_name
• Uses this flag to, if set, infer a default state schema name based on the dbt profile name and target schema
• Enables this flag by default for sqlmesh init -t dbt so that dbt projects by default have isolated state between targets

Note that users can still override the inferred schema by setting the state schema manually in sqlmesh.yaml as documented here. Existing projects will not have this new property set so will continue to behave as usual.

[izeigerman]

Should we check whether target_schema is empty?

[erindru]

Man, this was a rabbit hole.

So in dbt's Credentials object that defines this field, schema is required. If you don't define it, the base validator fails with something like:

Runtime Error
  Credentials in profile "jaffle_shop", target "postgres" invalid: 'schema' is a required property

(and some adapters, like the duckdb adapter, have a hardcoded default to pass this check).

dbt will, however, happily allow you to specify an empty string for the schema. If you do this, you push the failure to runtime:

23:11:33  Encountered an error:
Database Error
  zero-length delimited identifier at or near """"
  LINE 2: create schema if not exists ""

So i've changed our side to raise an exception if the schema is an empty string. If someone encounters this and their project works fine on dbt core then hopefully theyre willing to work with us to understand the correct behaviour here (and the workaround is of course to define the state schema name manually)