Skip to content

Feat!: Skip model evaluation if upstream external model(s) have not changed#5277

Merged
VaggelisD merged 12 commits intomainfrom
vaggelisd/external_model_freshness
Sep 29, 2025
Merged

Feat!: Skip model evaluation if upstream external model(s) have not changed#5277
VaggelisD merged 12 commits intomainfrom
vaggelisd/external_model_freshness

Conversation

@VaggelisD
Copy link
Contributor

@VaggelisD VaggelisD commented Sep 2, 2025

This PR enables SQLMesh to skip re-evaluating models that depend on external model(s) which have not been updated since the last run.

This is done by:

  1. Adding the last_altered_ts field in the _intervals table which tracks the last modification to the physical table of that snapshot
  2. Querying the INFORMATION_SCHEMA for the external model's "last altered" timestamp wherever applicable (BQ & Snowflake for the time being)

Edit n.1: The freshness check was refactored into a built-in signal to (1) make the feature opt-in and (2) matches conceptually and the rest of the UX is given to us "for free".

@VaggelisD VaggelisD force-pushed the vaggelisd/external_model_freshness branch from 64db71b to f627f57 Compare September 10, 2025 14:51
@VaggelisD VaggelisD force-pushed the vaggelisd/external_model_freshness branch 2 times, most recently from 7d1e4c2 to d50f087 Compare September 22, 2025 16:14
@VaggelisD VaggelisD force-pushed the vaggelisd/external_model_freshness branch from 701000f to 1958d17 Compare September 25, 2025 17:36
@VaggelisD VaggelisD merged commit b29e71b into main Sep 29, 2025
35 of 36 checks passed
@VaggelisD VaggelisD deleted the vaggelisd/external_model_freshness branch September 29, 2025 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants