v1.13.4

What's Changed

• Move catalog utils in a separate utility object for reuse in #708
• Add support for RedShift driver in 7445150
• Make backoff times for reconnecting to bookeeping databases configurable in 7445150
• Tirm column names of leading and trailing spaces as part of column name sanitization in 1bbe96f

Full Changelog: v1.13.3...v1.13.4

v1.13.3

What's Changed

• #703 Fix the default value for backfill.days to be compatible with the previous Pramen scheduling behavior in #704
• #706 Add the ability to delete bookkeeping tables from the database via Bookeeper trait in #707

Full Changelog: v1.13.2...v1.13.3

v1.13.2

What's Changed

• #699 Add retries for Bookkeeping database, and make sure only 1 connection is active in #700
• #698 Fix the backfill logic for late jobs in #701

Full Changelog: v1.13.1...v1.13.2

v1.13.1

What's Changed

• #685 Fix when transformers and sinks are marked as not allowed to run in parallel have the behavior of transformations with self-dependencies in #688
• #686 Fix self-dependencies by making sure optional self-dependencies do not cause skips for informations dates if previous information dates had failed such dependencies in #689
• #690 Fix transient persistent metatable cache cleanup in #694
• #687 Add support for backfill days in #691
• #690 Add support for SQLite as the option for storing bookeeping data in #696
• #695 Add support for marking operations as 'critical' failure of which causes the pipeline to fail in #697
• Do not include stack traces of a job if a dependent lazy job failed because that stack trace is already included in #697
• Use final version tags for standard GH actions, and SHA1 commit references to community GH actions in #684

Full Changelog: v1.13.0...v1.13.1

v1.13.0

What's Changed

• #677 Add access to ticket locks and incremental information from inside transformers and sinks in #681
• #677 Add batch Id and the number of appended record to the bookeeping in #682
• #674 Allow specifying to ignore schema changes for selected operations in #683

Full Changelog: v1.12.14...v1.13.0

v1.12.14

What's Changed

• #669 Add created_at field to the lock database so that old locks can be cleaned up in #676
• #675 Add country as a configuration parameter of Pramen pipelines in #678
• #679 Add Iceberg and Delta table names to the catalog table name column in notifications in #680

Full Changelog: v1.12.13...v1.12.14

v1.12.13

What's Changed

• #651 Remove a redundant method and redundant parameters from the bookkeeping interface in #671
• #672 Do not fail pipeline if no data arrives for incremental dependencies because it is expected in #673

Full Changelog: v1.12.12...v1.12.13

v1.12.12

What's Changed

• #667 Fix Denodo filter pushdown for timestamp-based queries in #668

Full Changelog: v1.12.11...v1.12.12

v1.12.11

What's Changed

• #664 Fix the passivness of auto-added sink dependencies. Fix incremental sink and transfer jobs in #665
• #654 Fix db_version not created when PostgreSQL autocommit=false in #666

Full Changelog: v1.12.10...v1.12.11

v1.12.10

What's Changed

• #661 Implement incremental offset commits for re-runs so normal runs can be ran after reruns in #662
• #660 Make spark-avro dependency provided because it is tightly coupled with Spark version in #663

Full Changelog: v1.12.9...v1.12.10