dataform CLI replaces the incremental table when the project id is not supplied #2053
Description
As a developer working with Git, I prefer not to commit workflow_settings.yaml with a hardcoded defaultProject, since different environments typically use different GCP projects.
In the Dataform UI, this works correctly when the project_id is not in the workflow_settings, but provided via compilation variables.
However, when an incremental table is executed via the Dataform CLI, the behavior differs: the table is always gets replaced instead of performing an incremental append. It appears that incremental() evaluates to false when workflow_settings.yaml does not contain a defaultProject.
What is confusing is that Dataform is still able to resolve the target dataset and table correctly, as evidenced by the CREATE OR REPLACE operation being executed.
Could you please clarify whether this is expected behavior or a known limitation of the Dataform CLI when defaultProject is omitted from workflow_settings.yaml?
Steps to reproduce :
- SQLx:
config {
type:"incremental",
name: "third"
}
select current_timestamp as col1, ${incremental()} as col2
- workflow_settings.yaml
defaultLocation: europe-west2
defaultDataset: dataform
defaultAssertionDataset: dataform_assertions
dataformCoreVersion: 3.0.0
- Run the SQLx using the dataform CLI 2 times .
Expected behavior :
2 rows after the second run with first record having col2=false and second record having col2=true
Actual Behavior
Only one record with col2=false