TurboVault4dbt is an open-source tool that automatically generates dbt models according to our datavault4dbt templates. It uses a metadata input of your Data Vault 2.0 from one of the supported databases and creates ready-to-process dbt-models.
- TurboVault4dbt requires a metadata analysis done by hand and stored in supported metadata storage.
- Python must be installed as TurboVault4dbt is a software written in Python.
- A dbt project is required to use the generated models.
- Additionally, our dbt package datavault4dbt must be used with version 1.9.0 or newer since the dbt models are using datavault4dbt macros.
You can find DDL scripts and templates for the metadata tables and the Excel sheet here]((https://github.com/ScalefreeCOM/turbovault4dbt/tree/main/metadata_ddl)).
Your metadata needs to be stored in the following tables/worksheets:
- Source Data
- Standard Hubs
- Standard Links
- Non-Historized Links
- Standard Satellites
- Non-Historized Satellites
- Multi-Active Satellites
- Point-In-Time Tables
- Reference Tables
Currently, TurboVault4dbt supports metadata input from
- Snowflake
- BigQuery
- Google Sheets
- Excel
- SQLite DB Files
While our development team is eager to answer all questions and fix bugs directly here on Github, this sometimes isn't enough for organizations looking for a guaranteed level of support. To meet this requirement, Scalefree has created the DataVault4dbt Premium Package! Follow the link to learn about all benefits included.
To install TurboVault4dbt, follow the instructions on this page.
You can configure the connection to your metadata storage in the config.ini located in turbovault4dbt/backend /config/ Whichever section is found to be relevant to your project needs to be configured, and the rest could be deleted. In the next run, the section configured in the config.ini will appear in Turbo Vault's UI.
To run TurboVault4dbt, you need to install Python, as well as the required packages. Then, simply execute main.py, which will open a GUI that looks like this:
First, choose your metadata input platform, where you have your metadata stored, from the dropdown menu.
From the Sources section, choose the source objects that you would like to use. At least one source object should be selected.
From the Entities section, you can select which types of entities you want to generate. These are:
- Stage
- Standard Hub
- Standard Link
- Satellite (both version 0 and version 1)
- Multi-Active Satellite (both version 0 and version 1)
- Non-Historized Link
- Non-Historized Satellite
- Dependent Child Link
- Point-In-Time
- Reference Table
By using the checkboxes:
- a sources.yml can be generated,
- properties .yml files can be generated for each source object,
- and a Data Vault Model Visualization can be created using the external tool DBDocs.
Now you can click on the START button and TurboVault4dbt will generate all necessary dbt models that work with datavault4dbt!
v2.0 (14.11.2024) - Current Version
v1.2 (18.09.2024)
v1.1.2 (23.01.2024)
v1.1.1 (24.05.2023)
v1.1.0 (22.05.2023)
v1.0.3 (16.02.2023)
v1.0.2 (13.02.2023)
v1.0.1 (30.01.2023)
v1.0.0 (26.01.2023)
