25.12 public release

What's Changed

• feat: config changes for 25.09-ppp run by @project-defiant in #144
• feat: add make rule to wipe airflow vm by @ireneisdoomed in #142
• feat(fingen meta): add dag for finnGen UK biobank MVP meta analysis ingestion by @project-defiant in #145
• feat: updating colocalisation steps to use colocpip_and_ecaviar by @Daniel-Considine in #147
• feat: 25.09 reference update by @project-defiant in #148
• feat: expose gencode version to up config by @project-defiant in #150
• feat: update gentropy and staging config by @project-defiant in #151
• Run PTS steps in dataproc when there are pyspark tasks by @javfg in #152
• feat(l2g predictions) batch job by @project-defiant in #155
• feat: prettify unified pipeline graph in Airflow by @d0choa in #156
• feat(staging): 25.12 genetics staging data update by @project-defiant in #149
• feat: new unified_pipeline dag after pis, pts, etl changes by @ireneisdoomed in #153

Full Changelog: v25.09.1...v25.12

25.09 public release

What's Changed

• feat: changes to the genetics etl test run for 2025.03 release by @project-defiant in #104
• fix: staging genetics_etl test run fixes by @project-defiant in #106
• feat: ingest Loss-of-Function variant data from OTAR2075 by @vivienho in #105
• feat: changes for the unified pipeline run in gentropy part by @project-defiant in #107
• chore: update l2g train and predict configs by @ireneisdoomed in #109
• feat: google batch dag for l2g predictions by @project-defiant in #110
• feat: 2503 ppp pipeline run changes by @javfg in #121
• chore: reflect updated paths by @project-defiant in #122
• chore: update documentation after google cloud storage data cleanup by @project-defiant in #123
• chore: update gnomad datasource and vep plugin docs by @project-defiant in #125
• Big cleanup by @javfg in #126
• fix: change latest image to 2.10.5 by @project-defiant in #128
• feat: cloud deployment by @javfg in #127
• fix: ensure the correct project is used to fetch ssh config by @project-defiant in #129
• chore: added docs describing interval dataset locations by @project-defiant in #124
• chore: update curation path by @project-defiant in #130
• feat: ensure 1 partition for studyIndex and susie finemapping by @project-defiant in #132
• chore: allow specifying ADC from makefile by @project-defiant in #131
• chore: update gentropy version in genetic dags by @project-defiant in #133
• feat(ppp): per step config overrides by @project-defiant in #134
• 25.09 staging data release by @project-defiant in #138
• feat: move intervals with pis by @project-defiant in #139
• feat: (3317) gnomad genetic constrains update by @remo87 in #137
• feat: add literature_dating input to etl evidence config by @javfg in #140
• 25.09-testrun-1 configuration by @ireneisdoomed in #141

New Contributors

• @remo87 made their first contribution in #137

Full Changelog: v1.1.0...v25.09.1

v25.06-pub.2

What's Changed

• feat: ingesting FoldX mutation data from OTAR2081 by @DSuveges in #87
• feat: update to finngen release 12 by @Daniel-Considine in #90
• chore: allow poetry 2.0.X by @project-defiant in #91
• refactor: remove gene_index from pipeline by @vivienho in #89
• chore(docker): update github cli version in airflow docker image by @project-defiant in #93
• feat(gwas_catalog): sumstat harmonisation dag by @project-defiant in #55
• fix: do not run action on branch and tag release by @project-defiant in #97
• feat: sane path structure for releases by @javfg in #95
• chore: removing CADD from VEP annotation by @DSuveges in #98
• chore: bump dataproc image version by @project-defiant in #94
• fix: gnomad annotation path update by @DSuveges in #99
• feat: sum pips to 0.95 in susie finemapper by @project-defiant in #100
• chore(PIS): Updating evidence sources by @DSuveges in #101
• fix: finngen r12 curation url updated from R11 to R12 by @Daniel-Considine in #92
• fix: finngen r12 curation url by @Daniel-Considine in #103
• feat(staging):updates for staging data release 2025.03 by @project-defiant in #102
• feat: changes to the genetics etl test run for 2025.03 release by @project-defiant in #104
• fix: staging genetics_etl test run fixes by @project-defiant in #106
• feat: ingest Loss-of-Function variant data from OTAR2075 by @vivienho in #105
• feat: changes for the unified pipeline run in gentropy part by @project-defiant in #107
• chore: update l2g train and predict configs by @ireneisdoomed in #109
• feat: google batch dag for l2g predictions by @project-defiant in #110
• feat: 2503 ppp pipeline run changes by @javfg in #121
• chore: reflect updated paths by @project-defiant in #122
• chore: update documentation after google cloud storage data cleanup by @project-defiant in #123
• chore: update gnomad datasource and vep plugin docs by @project-defiant in #125
• Big cleanup by @javfg in #126
• fix: change latest image to 2.10.5 by @project-defiant in #128
• feat: cloud deployment by @javfg in #127
• fix: ensure the correct project is used to fetch ssh config by @project-defiant in #129
• chore: added docs describing interval dataset locations by @project-defiant in #124
• chore: update curation path by @project-defiant in #130
• feat: ensure 1 partition for studyIndex and susie finemapping by @project-defiant in #132
• chore: allow specifying ADC from makefile by @project-defiant in #131
• chore: update gentropy version in genetic dags by @project-defiant in #133
• feat(ppp): per step config overrides by @project-defiant in #134

Full Changelog: v1.0.0...v25.06-pub.2

v1.0.0

First release of orchestration pipeline

This release contains the logic to run the unified pipeline with that was used for running the 24.12 data release for the Open Targets Platform

What's Changed

• feat: support for dependabot by @d0choa in #3
• feat: initial config and airflow layer from gentropy by @project-defiant in #1
• feat: fetch manifest from github by @project-defiant in #4
• feat: add interface with batch operators and refactor the utils by @ireneisdoomed in #7
• chore: start Poetry shell at the end of environment configuration by @tskir in #6
• docs: update and migrate from Gentropy by @tskir in #5
• refactor: code clenaup by @project-defiant in #9
• fix: pre-commit syntax in Makefile by @tskir in #8
• feat: platform pis stage first prototype by @javfg in #11
• feat: finngen ingestion by @project-defiant in #13
• feat: pis stage for the platform orchestration dag by @javfg in #14
• feat: extract config for finngen ingestion to yaml file by @project-defiant in #15
• feat: Allow gcs paths without prefix by @project-defiant in #16
• feat: move ukb_ppp_eur datasource ingestion to orchestration by @project-defiant in #17
• docs: fix README formatting by @tskir in #19
• docs: describe gwas catalog data bucket content by @project-defiant in #20
• feat: synced gentropy dags and latest configs to orchestration by @project-defiant in #21
• feat: add FinnGen UKB meta-analysis ingestion DAG by @tskir in #22
• feat: add validation to genetics etl by @project-defiant in #18
• feat: include LD_CLUMPED QC flag in credible set validation by @d0choa in #23
• feat: finemapping template and DAG for UKB PPP by @tskir in #10
• feat: add ETL stage to the platform pipeline by @javfg in #24
• fix: platform becomes pipeline dag by @javfg in #25
• refactor: genetic dags by @project-defiant in #26
• feat: allow for force reinstall dependencies on cluster when set up by @project-defiant in #32
• feat: GWAS catalog top-hit DAG by @d0choa in #33
• feat: gwas_catalog_sumstats_pics DAG and harmonised paths by @d0choa in #35
• feat: variant-index refactoring by @project-defiant in #29
• refactor: clarify paths by @d0choa in #36
• fix(vep_operator): change dtype for pos to int for correct sorting by @project-defiant in #38
• feat(ukb_ppp_eur) dags refactoring by @project-defiant in #39
• chore(finngen): params and documentation update by @project-defiant in #44
• feat: working genetics_etl including L2G by @d0choa in #45
• chore(eqtl_catalogue): params and documentation update by @project-defiant in #41
• fix(variant_to_vcf): increase batch job size by @project-defiant in #46
• fix(yarn): add dataproc parameter to hydra comamnd parsing function in gentropy step by @project-defiant in #40
• chore(finngen_ingestion): update configuration by @project-defiant in #47
• feat(susie_finemapper): ensure proper output paths by @project-defiant in #48
• feat(add_efo_biosample): add input to orchestration by @Tobi1kenobi in #52
• feat: l2g evidence generation step added to genetics etl by @DSuveges in #53
• fix(vep): change vep outputs extension to .json by @ireneisdoomed in #56
• feat: locus to gene evidence step - polished by @DSuveges in #58
• feat(gwas catalog sumstats): finemapping by @project-defiant in #51
• feat(credible set qc dag): added dag and docs by @project-defiant in #59
• feat(freeze_5): november data release freeze5 by @project-defiant in #60
• refactor: reorder tasks in genetics_etl by @project-defiant in #62
• fix: prevent code from listing buckets at compile time by @project-defiant in #61
• perf(efm): enhanced flexibility mode in genetics etl cluster by @project-defiant in #63
• chore: name l2g run with release id and move download_from_hub param by @ireneisdoomed in #65
• feat: add task to genetics etl to generate association data by @vivienho in #64
• feat: updating invalid qc reasons for validation by @DSuveges in #66
• feat: refactoring vep annotation step by @project-defiant in #67
• chore: configure hf repo in l2g predicting step by @ireneisdoomed in #68
• feat: freeze 6 improvements by @project-defiant in #69
• fix: changing to the 99% file and using a different column for fine mapping ingestion by @Daniel-Considine in #70
• chore: improve versioning of l2g model in training step by @ireneisdoomed in #72
• feat: configuration for etl run with gentropy by @project-defiant in #71
• chore: set l2g_threshold to 0.05 by @ireneisdoomed in #74
• chore: drop old dags by @project-defiant in #75
• chore: request additional vep fields for variant index by @DSuveges in #77
• fix: gnomad variant index and ld preprocess by @DSuveges in #78
• feat(curation update): genetics curation update 24.11.20 by @project-defiant in #80
• feat(docs): add command to update the documentation in buckets by @project-defiant in #76
• feat: changing to 95% file for finngen ingestion by @Daniel-Considine in #83
• feat: adding conservation VEP plugin for variant annotation by @DSuveges in #85
• fix: small fixes for the unified pipeline by @javfg in #86

New Contributors

• @project-defiant made their first contribution in #1
• @tskir made their first contribution in #6
• @Tobi1kenobi made their first contribution in #52
• @DSuveges made their first contribution in #53
• @vivienho made their first contribution in #64
• @Daniel-Considine made their first contribution in #70

Full Changelog: https://github.com/opentargets/orchestration/commits/v1.0.0