Skip to content

fix tascomi create daily snapshot#2159

Merged
Tian-2017 merged 2 commits intomainfrom
fix-tascomi_create_daily_snapshot_planning
Mar 11, 2025
Merged

fix tascomi create daily snapshot#2159
Tian-2017 merged 2 commits intomainfrom
fix-tascomi_create_daily_snapshot_planning

Conversation

@Tian-2017
Copy link
Contributor

@Tian-2017 Tian-2017 commented Mar 10, 2025

I made two main changes; the rest are just formatting.
The changes are:

  • Function get_latest_snapshot: If snapshot_date does not exist, add the column.
  • Function loadIncrementsSinceDate: Added a default date — 20210101 (the one used multiple times as the date in the original script).

It’s working fine now. The data has been loaded to the latest partition, for example

s3://dataplatform-stg-refined-zone/planning/tascomi/snapshot/appeal_decision/snapshot_year=2025/snapshot_month=03/snapshot_day=10/snapshot_date=20250310/

@Tian-2017 Tian-2017 requested review from a team as code owners March 10, 2025 20:26
@sonarqubecloud
Copy link

@Tian-2017 Tian-2017 merged commit 88a5a44 into main Mar 11, 2025
6 checks passed
@Tian-2017 Tian-2017 deleted the fix-tascomi_create_daily_snapshot_planning branch March 11, 2025 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants