[WIP] Tech/dbt add marts#31
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a dbt “marts” layer and updates the Streamlit BigQuery access layer to read from a new mart model.
Changes:
- Removes unused/placeholder query functions from
src/data_layer/bigquery.pyand switches one query to read from a mart table. - Introduces two new dbt mart models as passthrough views on existing intermediate models.
- Adds a marts schema YAML and bumps
dbt_project.ymlversion (plus a newmarts:config key).
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/data_layer/bigquery.py | Points get_nb_moy_nuits_sup_20deg() to the new mart table; removes unused query helpers. |
| models/marts/mart_mensq_temperatures_sup_20deg.sql | New mart model referencing int_mensq_temperatures_sup_20deg. |
| models/marts/mart_mensq_pluviometrie_sup_100mm.sql | New mart model referencing int_mensq_pluviometrie_sup_100mm. |
| models/marts/_marts_models.yml | Adds docs/tests for the new mart models. |
| dbt_project.yml | Version bump and adds a marts: section under models.projet_cc. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| models: | ||
| projet_cc: | ||
| +materialized: view | ||
| marts: No newline at end of file |
There was a problem hiding this comment.
marts: is added under models.projet_cc but has no value/config. In YAML this becomes null, and dbt model configs expect a mapping for subdirectories; this can lead to dbt parsing/runtime errors or silently not applying intended marts configuration. Either remove this key or give it an explicit config block (e.g., set +schema / +materialized, or at least {} if it’s meant as a placeholder).
| marts: | |
| marts: {} |
| Contient les mois avec température moyenne supérieure à 20°C. | ||
|
|
||
| columns: | ||
| - name: annee | ||
| tests: | ||
| - not_null | ||
|
|
||
| - name: mois | ||
| tests: | ||
| - not_null |
There was a problem hiding this comment.
The schema tests/documentation for mart_mensq_temperatures_sup_20deg don’t match the model it references. mart_mensq_temperatures_sup_20deg is a passthrough of int_mensq_temperatures_sup_20deg, which outputs ANNEE, Z_GEO, nb_stations, moy_nuits_ge_20 (no mois). As written, the mois not_null test will error because the column doesn’t exist, and the description claims the table contains “mois” data. Update the column list/tests/description to reflect the actual columns (or adjust the underlying model to produce mois if that’s the intent).
| Contient les mois avec température moyenne supérieure à 20°C. | |
| columns: | |
| - name: annee | |
| tests: | |
| - not_null | |
| - name: mois | |
| tests: | |
| - not_null | |
| Contient, par année et zone géographique, des indicateurs sur les nuits avec température minimale supérieure ou égale à 20°C. | |
| columns: | |
| - name: annee | |
| tests: | |
| - not_null | |
| - name: z_geo | |
| tests: | |
| - not_null | |
| - name: nb_stations | |
| tests: | |
| - not_null | |
| - name: moy_nuits_ge_20 | |
| tests: | |
| - not_null |
…pect dbt process and that should be brought into compliance with dbt
…s' names are located in one place only, and sql queries make use of aliases
No description provided.