diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000..8b7dd42 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,28 @@ +## Description + + +## Dependencies (Issues/PRs) + + +## Type of Change +- [ ] Bug fix +- [ ] New feature +- [ ] Documentation + +## Task Checklist +- [ ] documentation updated +- [ ] `miniwdl check` passed +- [ ] `womtool validate` passed + + +## Testing +### Workflow Engine Tested On +- [ ] HealthOmics, Amazon Web Services +- [ ] Google Cloud Platform, Cromwell +- [ ] Google Cloud Platform +- [ ] Microsoft Azure, Cromwell +- [ ] GA4GH Workflow Execution Service, On Premises and Multi Cloud +- [ ] Other + +### Successful Workflow IDs +* diff --git a/.github/workflows/lint-test-workflows.yml b/.github/workflows/lint-test-workflows.yml index a27bec4..c5e3acf 100644 --- a/.github/workflows/lint-test-workflows.yml +++ b/.github/workflows/lint-test-workflows.yml @@ -21,5 +21,5 @@ jobs: workbench-workflow-service-url: ${{ secrets.WORKBENCH_WORKFLOW_SERVICE_URL }} workbench-ewes-refresh-token: ${{ secrets.WORKBENCH_EWES_REFRESH_TOKEN }} workbench-workflow-service-refresh-token: ${{ secrets.WORKBENCH_WORKFLOW_SERVICE_REFRESH_TOKEN }} - wdl-ci-custom-test-wdl-dir: pmdbs-spatial-transcriptomics-wdl-ci-custom-test-dir + wdl-ci-custom-test-wdl-dir: spatial-transcriptomics-wdl-ci-custom-test-dir diff --git a/README.md b/README.md index a17d8f7..7809206 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ -# pmdbs-spatial-transcriptomics-wf -Repo for testing and developing a common postmortem-derived brain sequencing (PMDBS) workflow harmonized across ASAP with human and mouse spatial transcriptomics data for both Nanostring GeoMx and 10x Visium platforms. The main goal is to uncover spatially distinct gene expression profiles across tissue samples, enabling insights into tissue architecture, cell type composition, and disease-related molecular patterns. +# spatial-transcriptomics-wf +Repo for testing and developing a common postmortem-derived brain sequencing (PMDBS) and non-human workflow harmonized across ASAP with human and mouse spatial transcriptomics data for both Nanostring GeoMx and 10x Visium platforms. The main goal is to uncover spatially distinct gene expression profiles across tissue samples, enabling insights into tissue architecture, cell type composition, and disease-related molecular patterns. Common workflows, tasks, utility scripts, and docker images reused across harmonized ASAP workflows are defined in [the wf-common repository](https://github.com/ASAP-CRN/wf-common). @@ -15,7 +15,7 @@ Common workflows, tasks, utility scripts, and docker images reused across harmon # Workflows -Worfklows are defined in [the `workflows` directory](workflows). There is [the `pmdbs_spatial_geomx` workflow directory](workflows/pmdbs_spatial_geomx) and [the `pmdbs_spatial_visium` workflow directory](workflows/pmdbs_spatial_visium). +Worfklows are defined in [the `workflows` directory](workflows). There is [the `spatial_geomx` workflow directory](workflows/spatial_geomx) and [the `spatial_visium` workflow directory](workflows/spatial_visium). These workflows are set up to analyze spatial transcriptomics data: Nanostring GeoMx in WDL using command line, R, and Python scripts and 10x Visium in WDL using command line and Python scripts. @@ -26,11 +26,11 @@ _Note: Unlike our other workflows (e.g., [pmdbs-sc-rnaseq-wf](https://github.com For the Nanostring GeoMx workflow, we start with raw output files from the instrument and convert them into counts. Then, we clean the data by removing unreliable segments and genes, adjust for technical noise, and combine data from different slides. Finally, we cluster based on segment (which may contain many cells) and visualize their transcriptional profiles in a UMAP space. **Nanostring GeoMx workflow diagram:** -![Nanostring GeoMx workflow diagram](workflows/pmdbs_spatial_geomx/workflow_diagram.svg "Workflow diagram") +![Nanostring GeoMx workflow diagram](workflows/spatial_geomx/workflow_diagram.svg "Workflow diagram") -**Nanostring GeoMx entrypoint**: [workflows/pmdbs_spatial_geomx/main.wdl](workflows/pmdbs_spatial_geomx/main.wdl) +**Nanostring GeoMx entrypoint**: [workflows/spatial_geomx/main.wdl](workflows/spatial_geomx/main.wdl) -**Nanostring GeoMx input template**: [workflows/pmdbs_spatial_geomx/inputs.json](workflows/pmdbs_spatial_geomx/inputs.json) +**Nanostring GeoMx input template**: [workflows/spatial_geomx/inputs.json](workflows/spatial_geomx/inputs.json) The Nanostring GeoMx workflow is broken up into three main chunks: @@ -54,11 +54,11 @@ Run once per team (all slide from a single team) if `project.run_project_cohort_ For the 10x Visium workflow, we start with raw output files from the instrument and convert them into counts. Then, we clean the data by removing unreliable spots and genes, adjust for technical noise, and combine data from different samples. We cluster based on spots (which may contain many cells) and visualize their transcriptional profiles in a UMAP space. We also identify spatially variable genes to uncover expression patterns that are location-specific within the tissue. **10x Visium workflow diagram:** -![10x Visium workflow diagram](workflows/pmdbs_spatial_visium/workflow_diagram.svg "Workflow diagram") +![10x Visium workflow diagram](workflows/spatial_visium/workflow_diagram.svg "Workflow diagram") -**10x Visium entrypoint**: [workflows/pmdbs_spatial_visium/main.wdl](workflows/pmdbs_spatial_visium/main.wdl) +**10x Visium entrypoint**: [workflows/spatial_visium/main.wdl](workflows/spatial_visium/main.wdl) -**10x Visium input template**: [workflows/pmdbs_spatial_visium/inputs.json](workflows/pmdbs_spatial_visium/inputs.json) +**10x Visium input template**: [workflows/spatial_visium/inputs.json](workflows/spatial_visium/inputs.json) The 10x Visium workflow is broken up into two main chunks: @@ -78,7 +78,7 @@ Run once per team (all samples from a single team) if `project.run_project_cohor ## Nanostring GeoMx inputs -An input template file can be found at [workflows/pmdbs_spatial_geomx/inputs.json](workflows/pmdbs_spatial_geomx/inputs.json). +An input template file can be found at [workflows/spatial_geomx/inputs.json](workflows/spatial_geomx/inputs.json). | Type | Name | Description | | :- | :- | :- | @@ -103,7 +103,7 @@ An input template file can be found at [workflows/pmdbs_spatial_geomx/inputs.jso ## 10x Visium inputs -An input template file can be found at [workflows/pmdbs_spatial_visium/inputs.json](workflows/pmdbs_spatial_visium/inputs.json). +An input template file can be found at [workflows/spatial_visium/inputs.json](workflows/spatial_visium/inputs.json). | Type | Name | Description | | :- | :- | :- | @@ -189,7 +189,7 @@ An input template file can be found at [workflows/pmdbs_spatial_visium/inputs.js ## Generating the inputs JSON -The inputs JSON may be generated manually, however when running a large number of samples, this can become unwieldly. The [`generate_inputs` utility script](https://github.com/ASAP-CRN/wf-common/blob/main/util/generate_inputs) may be used to automatically generate the inputs JSON (`inputs.{staging_env}.{source}-{cohort_dataset}.{date}.json` and a sample list TSV (`{team_id}.{source}-{cohort_dataset}.sample_list.{date}.tsv`; same as the one generated in [the write_cohort_sample_list task](https://github.com/ASAP-CRN/wf-common/wdl/tasks/write_cohort_sample_list.wdl)). The script requires the libraries outlined in [the requirements.txt file](https://github.com/ASAP-CRN/wf-common/util/requirements.txt) and the following inputs: +The inputs JSON may be generated manually, however when running a large number of samples, this can become unwieldly. The [`generate_inputs` utility script](https://github.com/ASAP-CRN/wf-common/blob/main/util/generate_inputs) may be used to automatically generate the inputs JSON (`inputs.{staging_env}.{source}-{cohort_dataset}.{date}.json`) and a sample list TSV (`{team_id}.{source}-{cohort_dataset}.sample_list.{date}.tsv`); same as the one generated in [the write_cohort_sample_list task](https://github.com/ASAP-CRN/wf-common/wdl/tasks/write_cohort_sample_list.wdl)). The script requires the libraries outlined in [the requirements.txt file](https://github.com/ASAP-CRN/wf-common/util/requirements.txt) and the following inputs: - `project-tsv`: One or more project TSVs with one row per sample and columns team_id, ASAP_dataset_id, ASAP_sample_id, batch, fastq_R1s, fastq_R2s, fastq_I1s, fastq_I2s, embargoed, source, dataset, dataset_DOI_url, and SPATIAL columns if applicable: geomx_config, geomx_dsp_config, geomx_annotation_file, visium_cytassist, visium_probe_set, visium_slide_ref, and visium_capture_area. All samples from all projects may be included in the same project TSV, or multiple project TSVs may be provided. - `team_id`: A unique identifier for the team from which the sample(s) arose. @@ -220,16 +220,16 @@ Example usage: ```bash ./wf-common/util/generate_inputs \ --project-tsv metadata.tsv \ - --inputs-template workflows/pmdbs_spatial_geomx/inputs.json \ + --inputs-template workflows/spatial_geomx/inputs.json \ --run-project-cohort-analysis \ - --workflow-name pmdbs_spatial_geomx_analysis \ + --workflow-name spatial_geomx_analysis \ --cohort-dataset spatial-geomx ./wf-common/util/generate_inputs \ --project-tsv metadata.tsv \ - --inputs-template workflows/pmdbs_spatial_visium/inputs.json \ + --inputs-template workflows/spatial_visium/inputs.json \ --run-project-cohort-analysis \ - --workflow-name pmdbs_spatial_visium_analysis \ + --workflow-name spatial_visium_analysis \ --cohort-dataset spatial-visium ``` @@ -251,7 +251,7 @@ In the workflow, task outputs are either specified as `String` (final outputs, w ```bash asap-raw-{cohort,team-xxyy}-{source}-{dataset} └── workflow_execution - └── pmdbs_spatial_geomx + └── spatial_geomx ├── cohort_analysis │ └──${cohort_analysis_workflow_version} │ └── ${workflow_run_timestamp} @@ -273,7 +273,7 @@ asap-raw-{cohort,team-xxyy}-{source}-{dataset} asap-raw-{cohort,team-xxyy}-{source}-{dataset} └── workflow_execution - └── pmdbs_spatial_visium + └── spatial_visium ├── cohort_analysis │ └──${cohort_analysis_workflow_version} │ └── ${workflow_run_timestamp} @@ -298,7 +298,7 @@ Data may be synced using [the `promote_staging_data` script](#promoting-staging- ```bash asap-dev-{team-xxyy}-{source}-{dataset} -└── pmdbs_spatial_geomx +└── spatial_geomx ├── cohort_analysis │ ├── ${team_id}.sample_list.tsv │ ├── ${team_id}.merged_metadata.csv @@ -336,7 +336,7 @@ asap-dev-{team-xxyy}-{source}-{dataset} └── MANIFEST.tsv asap-dev-{team-xxyy}-{source}-{dataset} -└── pmdbs_spatial_visium +└── spatial_visium ├── cohort_analysis │ ├── ${team_id}.sample_list.tsv │ ├── ${team_id}.merged_cleaned_unfiltered.h5ad @@ -415,14 +415,14 @@ The script defaults to a dry run, printing out the files that would be copied or ```bash # List available teams -./wf-common/util/promote_staging_data -t cohort -l -s pmdbs -d spatial-geomx -w pmdbs_spatial_geomx -./wf-common/util/promote_staging_data -t cohort -l -s pmdbs -d spatial-visium -w pmdbs_spatial_visium +./wf-common/util/promote_staging_data -t cohort -l -s pmdbs -d spatial-geomx -w spatial_geomx +./wf-common/util/promote_staging_data -t cohort -l -s mouse -d spatial-visium -w spatial_visium # Print out the files that would be copied or deleted from the staging bucket to the curated bucket for teams team-edwards and team-vila -./wf-common/util/promote_staging_data -t team-edwards team-vila -s pmdbs -d spatial-geomx-th spatial-geomx-thlc -w pmdbs_spatial_geomx +./wf-common/util/promote_staging_data -t team-edwards team-vila -s pmdbs -d spatial-geomx-th spatial-geomx-thlc -w spatial_geomx # Promote data for team-edwards and team-vila -./wf-common/util/promote_staging_data -t team-edwards team-vila -s pmdbs -d spatial-geomx-th spatial-geomx-thlc -w pmdbs_spatial_geomx -p +./wf-common/util/promote_staging_data -t team-edwards team-vila -s pmdbs -d spatial-geomx-th spatial-geomx-thlc -w spatial_geomx -p ``` # Docker images @@ -481,16 +481,16 @@ Docker images can be build using the [`build_docker_images`](https://github.com/ | Image | Major tool versions | Links | Workflow | | :- | :- | :- | :- | -| geomxngs | | [Dockerfile](https://github.com/ASAP-CRN/pmdbs-spatial-transcriptomics-wf/tree/main/docker/geomxngs) | pmdbs_spatial_geomx | -| spaceranger | | [Dockerfile](https://github.com/ASAP-CRN/pmdbs-spatial-transcriptomics-wf/tree/main/docker/spaceranger) | pmdbs_spatial_visium | +| geomxngs | | [Dockerfile](https://github.com/ASAP-CRN/spatial-transcriptomics-wf/tree/main/docker/geomxngs) | spatial_geomx | +| spaceranger | | [Dockerfile](https://github.com/ASAP-CRN/spatial-transcriptomics-wf/tree/main/docker/spaceranger) | spatial_visium | | util | | [Dockerfile](https://github.com/ASAP-CRN/wf-common/tree/main/docker/util) | both | -| spatial_r | R (v4.4.2) packages: | [Dockerfile](https://github.com/ASAP-CRN/pmdbs-spatial-transcriptomics-wf/tree/main/docker/spatial_r) | pmdbs_spatial_geomx | -| spatial_py | Python (v3.12.5) libraries: | [Dockerfile](https://github.com/ASAP-CRN/pmdbs-spatial-transcriptomics-wf/tree/main/docker/spatial_py) | both | +| spatial_r | R (v4.4.2) packages: | [Dockerfile](https://github.com/ASAP-CRN/spatial-transcriptomics-wf/tree/main/docker/spatial_r) | spatial_geomx | +| spatial_py | Python (v3.12.5) libraries: | [Dockerfile](https://github.com/ASAP-CRN/spatial-transcriptomics-wf/tree/main/docker/spatial_py) | both | # wdl-ci -[`wdl-ci`](https://github.com/DNAstack/wdl-ci) provides tools to validate and test workflows and tasks written in [Workflow Description Language (WDL)](https://github.com/openwdl/wdl). In addition to the tests packaged in `wdl-ci`, the [pmdbs-spatial-transcriptomics-wdl-ci-custom-test-dir](./pmdbs-spatial-transcriptomics-wdl-ci-custom-test-dir) is a directory containing custom WDL-based tests that are used to test workflow tasks. `wdl-ci` in this repository is set up to run on pull request. +[`wdl-ci`](https://github.com/DNAstack/wdl-ci) provides tools to validate and test workflows and tasks written in [Workflow Description Language (WDL)](https://github.com/openwdl/wdl). In addition to the tests packaged in `wdl-ci`, the [spatial-transcriptomics-wdl-ci-custom-test-dir](./spatial-transcriptomics-wdl-ci-custom-test-dir) is a directory containing custom WDL-based tests that are used to test workflow tasks. `wdl-ci` in this repository is set up to run on pull request. In general, `wdl-ci` will use inputs provided in the [wdl-ci.config.json](./wdl-ci.config.json) and compare current outputs and validated outputs based on changed tasks/workflows to ensure outputs are still valid by meeting the critera in the specified tests. For example, if the rds to adata task in our workflow was changed, then this task would be submitted and that output would be considered the "current output". When inspecting the converted adata object, there is a test specified in the [wdl-ci.config.json](./wdl-ci.config.json) called, "check_hdf5". The test will compare the "current output" and "validated output" (provided in the [wdl-ci.config.json](./wdl-ci.config.json)) to make sure that the .h5ad file is still a valid HDF5 file. @@ -499,7 +499,7 @@ In general, `wdl-ci` will use inputs provided in the [wdl-ci.config.json](./wdl- ## Nanostring GeoMx notes -The [GeoMx DSP Instrument User Manual](https://nanostring.com/wp-content/uploads/2022/06/MAN-10152-01-GeoMx-DSP-Instrument-User-Manual.pdf) and [GeoMx DSP Data Analysis User Manual](https://nanostring.com/wp-content/uploads/2022/06/MAN-10154-01-GeoMx-DSP-Data-Analysis-User-Manual.pdf) provide a comprehensive overview of the GeoMx DSP workflow- from sample preparation and slide scanning to sequencing and data analysis. The required input files listed below for our `pmdbs_spatial_geomx` pipeline are described in detail in these manuals. These files are included as part of the GeoMx Readout Package, which are used during the experimental runs. +The [GeoMx DSP Instrument User Manual](https://nanostring.com/wp-content/uploads/2022/06/MAN-10152-01-GeoMx-DSP-Instrument-User-Manual.pdf) and [GeoMx DSP Data Analysis User Manual](https://nanostring.com/wp-content/uploads/2022/06/MAN-10154-01-GeoMx-DSP-Data-Analysis-User-Manual.pdf) provide a comprehensive overview of the GeoMx DSP workflow- from sample preparation and slide scanning to sequencing and data analysis. The required input files listed below for our `spatial_geomx` pipeline are described in detail in these manuals. These files are included as part of the GeoMx Readout Package, which are used during the experimental runs. ### GeoMx Lab Worksheet diff --git a/docker/spatial_py/scripts/geomx_merge_and_prep b/docker/spatial_py/scripts/geomx_merge_and_prep index 385d61e..711894d 100755 --- a/docker/spatial_py/scripts/geomx_merge_and_prep +++ b/docker/spatial_py/scripts/geomx_merge_and_prep @@ -48,7 +48,7 @@ def main(args): hvg_features = hvg_adata.var.copy() sc.pl.highly_variable_genes( - hvg_adata, + merged_adata, ) fig = plt.gcf() fig.suptitle(f"Highly variable genes dispersion plot - {args.output_prefix}", va="center", ha="center", fontsize=16) diff --git a/docker/spatial_py/scripts/integrate_harmony b/docker/spatial_py/scripts/integrate_harmony index 0312d62..081c806 100755 --- a/docker/spatial_py/scripts/integrate_harmony +++ b/docker/spatial_py/scripts/integrate_harmony @@ -15,10 +15,21 @@ def main(args): ################################## adata = sc.read_h5ad(args.adata_input) - sc.external.pp.harmony_integrate( - adata, - key=args.batch_key, - ) + n_cells = adata.n_obs + if n_cells < 100: + nclust = min(10, max(3, n_cells // 5)) + print(f"[INFO] Detected small dataset (n={n_cells}). Setting nclust={nclust}.") + sc.external.pp.harmony_integrate( + adata, + key=args.batch_key, + nclust=nclust, + ) + else: + print(f"[INFO] Running Harmony with default settings (n={n_cells}).") + sc.external.pp.harmony_integrate( + adata, + key=args.batch_key, + ) # Save outputs adata.write_h5ad(filename=args.adata_output, compression="gzip") diff --git a/docker/spatial_py/scripts/visium_process b/docker/spatial_py/scripts/visium_process index e06f2bb..a9d8246 100755 --- a/docker/spatial_py/scripts/visium_process +++ b/docker/spatial_py/scripts/visium_process @@ -62,7 +62,7 @@ def main(args): hvg_features = hvg_adata.var.copy() sc.pl.highly_variable_genes( - hvg_adata, + adata, ) fig = plt.gcf() fig.suptitle(f"Highly variable genes dispersion plot - {args.output_prefix}", va="center", ha="center", fontsize=16) diff --git a/docker/spatial_r/scripts/geomx_counts_to_rds b/docker/spatial_r/scripts/geomx_counts_to_rds index 4520a95..a2919e2 100755 --- a/docker/spatial_r/scripts/geomx_counts_to_rds +++ b/docker/spatial_r/scripts/geomx_counts_to_rds @@ -85,10 +85,12 @@ geomxdata <- readNanoStringGeoMxSet(dccFiles = dcc_files, ## ADD METADATA ## ################## metadata <- read.csv(args$sample_metadata, header = TRUE, stringsAsFactors = FALSE) -slide_filename_parts <- strsplit(args$slide_id, "_")[[1]] -slide_filename_batch <- tail(slide_filename_parts, 1) +slide_filename_batch <- sub(".*BATCH_", "", args$slide_id) # Per slide by batch/run filtered_metadata <- metadata[metadata$batch == slide_filename_batch, ] +if (nrow(filtered_metadata) == 0) { + stop("No metadata found for batch: ", slide_filename_batch) +} filtered_metadata$sample <- paste0(filtered_metadata$ASAP_sample_id, "_", filtered_metadata$replicate) filtered_metadata <- filtered_metadata[, c("sample_id", "batch", "sample")] # Some batches contain multiple replicates for the same ROI which will be in one DCC file, need to group @@ -107,8 +109,8 @@ merged_pdata <- merge( sort = FALSE ) merged_pdata$batch_id <- paste0(args$team_id, "_", args$dataset_id, "_", merged_pdata$batch) -num_pdata_rows = length(rownames(pData(geomxdata))) -num_metadata_rows = length(rownames(merged_pdata)) +num_pdata_rows = nrow(pData(geomxdata)) +num_metadata_rows = nrow(merged_pdata) if (num_pdata_rows != num_metadata_rows) { message(glue( "DCC files AKA RDS object row names:\n", diff --git a/pmdbs-spatial-transcriptomics-wdl-ci-custom-test-dir/check_zip.wdl b/spatial-transcriptomics-wdl-ci-custom-test-dir/check_zip.wdl similarity index 100% rename from pmdbs-spatial-transcriptomics-wdl-ci-custom-test-dir/check_zip.wdl rename to spatial-transcriptomics-wdl-ci-custom-test-dir/check_zip.wdl diff --git a/pmdbs-spatial-transcriptomics-wdl-ci-custom-test-dir/image_validator.wdl b/spatial-transcriptomics-wdl-ci-custom-test-dir/image_validator.wdl similarity index 100% rename from pmdbs-spatial-transcriptomics-wdl-ci-custom-test-dir/image_validator.wdl rename to spatial-transcriptomics-wdl-ci-custom-test-dir/image_validator.wdl diff --git a/wdl-ci.config.json b/wdl-ci.config.json index 93f805e..704c465 100644 --- a/wdl-ci.config.json +++ b/wdl-ci.config.json @@ -1,13 +1,13 @@ { "workflows": { - "workflows/pmdbs_spatial_geomx/cohort_analysis/cohort_analysis.wdl": { - "key": "workflows/pmdbs_spatial_geomx/cohort_analysis/cohort_analysis.wdl", + "workflows/spatial_geomx/cohort_analysis/cohort_analysis.wdl": { + "key": "workflows/spatial_geomx/cohort_analysis/cohort_analysis.wdl", "name": "", "description": "", "tasks": { "merge_and_prep": { "key": "merge_and_prep", - "digest": "cgyktioycaor6b3vkycc5qmefyw65lmo", + "digest": "hk4w6imkncqpkyjrw2pn3srhzv6n7bme", "tests": [ { "inputs": { @@ -74,7 +74,7 @@ }, "export_final_artifacts": { "key": "export_final_artifacts", - "digest": "i6i56p6qrv5iew6tthkzl7p6mlmkyvsl", + "digest": "ozzdvj7t2bw2gxlizkge2wfg4uthjmn3", "tests": [ { "inputs": { @@ -109,14 +109,14 @@ } } }, - "workflows/pmdbs_spatial_geomx/preprocess/preprocess.wdl": { - "key": "workflows/pmdbs_spatial_geomx/preprocess/preprocess.wdl", + "workflows/spatial_geomx/preprocess/preprocess.wdl": { + "key": "workflows/spatial_geomx/preprocess/preprocess.wdl", "name": "", "description": "", "tasks": { "fastq_to_dcc": { "key": "fastq_to_dcc", - "digest": "zj5w4vfxuwy4nz4s5inmeq6jup3ml5v3", + "digest": "pfk4rrdfy7iez3u5qkid6fkfd7u24utl", "tests": [ { "inputs": { @@ -157,15 +157,15 @@ }, "dcc_to_rds": { "key": "dcc_to_rds", - "digest": "qcfvjcqoup3damuk2elqrt7dhrefknds", + "digest": "oosp5zheppmriogec7fyrznw5xrlbe65", "tests": [ { "inputs": { "team_id": "${geomx_team_id}", "dataset_id": "${geomx_dataset_id}", - "slide_id": "${geomx_slide_id}", + "slide_id": "${geomx_slide_id_new}", "project_sample_metadata_csv": "gs://asap-raw-team-edwards-pmdbs-spatial-geomx-th/metadata/release/SAMPLE.csv", - "geomxngs_dcc_zip": "${geomx_input_file_path}/${preprocess_workflow_name}/${geomx_slide_id}.DCC.zip", + "geomxngs_dcc_zip": "${geomx_input_file_path}/${preprocess_workflow_name}/${geomx_slide_id_new}.DCC.zip", "geomx_lab_annotation_xlsx": "gs://asap-raw-team-edwards-pmdbs-spatial-geomx-th/spatial/annotation_files/cleaned_DNAstack_GeoMx_round_1_YH_20230719T2320_LabWorksheet.xlsx", "geomxngs_config_pkc": "${input_resources_file_path}/Hs_R_NGS_WTA_v1.0.pkc", "raw_data_path": "${dcc_to_rds_raw_data_path}", @@ -176,7 +176,7 @@ }, "output_tests": { "initial_rds_object": { - "value": "${geomx_input_file_path}/${preprocess_workflow_name}/${geomx_slide_id}.NanoStringGeoMxSet.rds", + "value": "${geomx_input_file_path}/${preprocess_workflow_name}/${geomx_slide_id_new}.NanoStringGeoMxSet.rds", "test_tasks": [ "compare_file_basename", "check_rds" @@ -188,7 +188,7 @@ }, "qc": { "key": "qc", - "digest": "vc72sxvdhbvf5yttwe7iiwk6jvend2uq", + "digest": "4elbse4c5saojyrv4mplwwvzykixdinz", "tests": [ { "inputs": { @@ -262,7 +262,7 @@ "tasks": { "integrate_sample_data": { "key": "integrate_sample_data", - "digest": "rcbwbwgpehbd4bez5gbwzmofudi22up6", + "digest": "qjngtpgf2tzriftjt4hfwgo2i2z2uucg", "tests": [ { "inputs": { @@ -286,7 +286,7 @@ }, "cluster": { "key": "cluster", - "digest": "qouf6flpm3rprnfz4fztlh3ybrumqcfb", + "digest": "ibhtiot7sakpfyfuuvpxnqiqmsgzh4hj", "tests": [ { "inputs": { @@ -322,14 +322,14 @@ } } }, - "workflows/pmdbs_spatial_visium/cohort_analysis/cohort_analysis.wdl": { - "key": "workflows/pmdbs_spatial_visium/cohort_analysis/cohort_analysis.wdl", + "workflows/spatial_visium/cohort_analysis/cohort_analysis.wdl": { + "key": "workflows/spatial_visium/cohort_analysis/cohort_analysis.wdl", "name": "", "description": "", "tasks": { "merge_and_plot_qc_metrics": { "key": "merge_and_plot_qc_metrics", - "digest": "kppsqzekmaf2vvut6ijgchkdof56io7u", + "digest": "tfku5xa5owqltkkjxr2ibq36iw5fh6hx", "tests": [ { "inputs": { @@ -369,7 +369,7 @@ }, "filter_and_normalize": { "key": "filter_and_normalize", - "digest": "lujek6d7l3dinbjtu23rvf2bplcbdkl5", + "digest": "526uk3dkuil6gmxfyxiltlyvegxbxpki", "tests": [ { "inputs": { @@ -429,7 +429,7 @@ }, "plot_spatial": { "key": "plot_spatial", - "digest": "mra4pyglrzr5eym4pht6d3e5dtxhks7a", + "digest": "ufcwh7kuftyt6nu74cyoawgjvro7lsfv", "tests": [ { "inputs": { @@ -456,14 +456,14 @@ } } }, - "workflows/pmdbs_spatial_visium/cohort_analysis/spatial_statistics/spatial_statistics.wdl": { - "key": "workflows/pmdbs_spatial_visium/cohort_analysis/spatial_statistics/spatial_statistics.wdl", + "workflows/spatial_visium/cohort_analysis/spatial_statistics/spatial_statistics.wdl": { + "key": "workflows/spatial_visium/cohort_analysis/spatial_statistics/spatial_statistics.wdl", "name": "", "description": "", "tasks": { "spatially_variable_gene_analysis": { "key": "spatially_variable_gene_analysis", - "digest": "swz5i2qx4fxynsb77wgp53zvqraibph3", + "digest": "in5bndajrnrb4kupxqv7xydhlqpnmghy", "tests": [ { "inputs": { @@ -515,14 +515,14 @@ } } }, - "workflows/pmdbs_spatial_visium/preprocess/preprocess.wdl": { - "key": "workflows/pmdbs_spatial_visium/preprocess/preprocess.wdl", + "workflows/spatial_visium/preprocess/preprocess.wdl": { + "key": "workflows/spatial_visium/preprocess/preprocess.wdl", "name": "", "description": "", "tasks": { "spaceranger_count": { "key": "spaceranger_count", - "digest": "u5qwsbzibn5jfeyr5jelgd6kxzcydo2o", + "digest": "vzxtqwz6w4yusua47lbceqc5smvqkvsf", "tests": [ { "inputs": { @@ -627,7 +627,7 @@ }, "counts_to_adata": { "key": "counts_to_adata", - "digest": "uty3rh5gqsqcymnegbiacpejn2mhklcm", + "digest": "zh45hobarbnoa2z2b6d3i66fdyneqj3w", "tests": [ { "inputs": { @@ -658,7 +658,7 @@ }, "qc": { "key": "qc", - "digest": "5fkcmlo3542jluxz73qg376vhtbfsv3b", + "digest": "2jfnfowy32wp57i7ctsx7ni5pzzeadbi", "tests": [ { "inputs": { @@ -755,32 +755,32 @@ } } }, - "workflows/pmdbs_spatial_geomx/structs.wdl": { - "key": "workflows/pmdbs_spatial_geomx/structs.wdl", + "workflows/spatial_geomx/structs.wdl": { + "key": "workflows/spatial_geomx/structs.wdl", "name": "", "description": "", "tasks": {} }, - "workflows/pmdbs_spatial_geomx/main.wdl": { - "key": "workflows/pmdbs_spatial_geomx/main.wdl", + "workflows/spatial_geomx/main.wdl": { + "key": "workflows/spatial_geomx/main.wdl", "name": "", "description": "", "tasks": {} }, - "workflows/pmdbs_spatial_visium/structs.wdl": { - "key": "workflows/pmdbs_spatial_visium/structs.wdl", + "workflows/spatial_visium/structs.wdl": { + "key": "workflows/spatial_visium/structs.wdl", "name": "", "description": "", "tasks": {} }, - "workflows/pmdbs_spatial_visium/main.wdl": { - "key": "workflows/pmdbs_spatial_visium/main.wdl", + "workflows/spatial_visium/main.wdl": { + "key": "workflows/spatial_visium/main.wdl", "name": "", "description": "", "tasks": {} }, - "pmdbs-spatial-transcriptomics-wdl-ci-custom-test-dir/check_zip.wdl": { - "key": "pmdbs-spatial-transcriptomics-wdl-ci-custom-test-dir/check_zip.wdl", + "spatial-transcriptomics-wdl-ci-custom-test-dir/check_zip.wdl": { + "key": "spatial-transcriptomics-wdl-ci-custom-test-dir/check_zip.wdl", "name": "", "description": "", "tasks": { @@ -791,8 +791,8 @@ } } }, - "pmdbs-spatial-transcriptomics-wdl-ci-custom-test-dir/image_validator.wdl": { - "key": "pmdbs-spatial-transcriptomics-wdl-ci-custom-test-dir/image_validator.wdl", + "spatial-transcriptomics-wdl-ci-custom-test-dir/image_validator.wdl": { + "key": "spatial-transcriptomics-wdl-ci-custom-test-dir/image_validator.wdl", "name": "", "description": "", "tasks": { @@ -803,14 +803,14 @@ } } }, - "workflows/pmdbs_spatial_geomx/process_to_adata/process_to_adata.wdl": { - "key": "workflows/pmdbs_spatial_geomx/process_to_adata/process_to_adata.wdl", + "workflows/spatial_geomx/process_to_adata/process_to_adata.wdl": { + "key": "workflows/spatial_geomx/process_to_adata/process_to_adata.wdl", "name": "", "description": "", "tasks": { "filter_and_normalize": { "key": "filter_and_normalize", - "digest": "3aorzvkdrznalhtf4arsxwmurayzkuyp", + "digest": "o6cbfdsmcpcuqvxfvqxmeuqkf6tonvu2", "tests": [ { "inputs": { @@ -870,20 +870,20 @@ }, "rds_to_adata": { "key": "rds_to_adata", - "digest": "2wxyiftqdrsemi7mtgomssz4iytrwo2x", + "digest": "t5xfpog3ckjgqho5vpw3ekyjp65aulgy", "tests": [ { "inputs": { - "processed_rds_object": "${geomx_input_file_path}/${process_to_adata_workflow_name}/${geomx_slide_id}.processed.rds", + "processed_rds_object": "${geomx_input_file_path}/${process_to_adata_workflow_name}/${geomx_slide_id_new}.processed.rds", "container_registry": "${container_registry}", "zones": "${zones}" }, "output_tests": { "processed_adata_object": { - "value": "${geomx_input_file_path}/${process_to_adata_workflow_name}/${geomx_slide_id}.processed.h5ad", + "value": "${geomx_input_file_path}/${process_to_adata_workflow_name}/${geomx_slide_id_new}.processed.h5ad", "test_tasks": [ "compare_file_basename", - "check_rds" + "check_hdf5" ] } } @@ -905,11 +905,12 @@ "geomx_team_id": "team-edwards", "geomx_dataset_id": "DS_PMDBS_0013", "geomx_slide_id": "ASAP_DS_PMDBS_0013_SLIDE_0001_BATCH_1", + "geomx_slide_id_new": "ASAP_DS_PMDBS_0013_SLIDE_0001_BATCH_r1", "geomx_slide_id_2": "ASAP_DS_PMDBS_0013_SLIDE_0001_BATCH_2", "cohort_id": "asap-cohort", - "geomx_workflow_name": "pmdbs_spatial_geomx", + "geomx_workflow_name": "spatial_geomx", "geomx_workflow_version": "v1.0.0", - "geomx_workflow_release": "https://github.com/ASAP-CRN/pmdbs-spatial-transcriptomics-wf/releases/tag/pmdbs_spatial_geomx_analysis-${visium_workflow_version}", + "geomx_workflow_release": "https://github.com/ASAP-CRN/spatial-transcriptomics-wf/releases/tag/spatial_geomx_analysis-${visium_workflow_version}", "geomx_workflow_info": [ [ "${run_timestamp}", @@ -935,9 +936,9 @@ "visium_dataset_id": "DS_MOUSE_0014", "visium_sample_id": "ASAP_MOUSE_000001_s001_Rep2", "visium_sample_id_2": "ASAP_MOUSE_000002_s001_Rep1", - "visium_workflow_name": "pmdbs_spatial_visium", + "visium_workflow_name": "spatial_visium", "visium_workflow_version": "v1.0.0", - "visium_workflow_release": "https://github.com/ASAP-CRN/pmdbs-spatial-transcriptomics-wf/releases/tag/pmdbs_spatial_visium_analysis-${visium_workflow_version}", + "visium_workflow_release": "https://github.com/ASAP-CRN/spatial-transcriptomics-wf/releases/tag/spatial_visium_analysis-${visium_workflow_version}", "visium_workflow_info": [ [ "${run_timestamp}", @@ -962,8 +963,8 @@ "engine_params": { "asap-cromwell": { "input_resources_file_path": "gs://asap-workflow-dev/workflow-resources", - "geomx_input_file_path": "gs://asap-wdl-ci/pmdbs_spatial_geomx", - "visium_input_file_path": "gs://asap-wdl-ci/pmdbs_spatial_visium" + "geomx_input_file_path": "gs://asap-wdl-ci/spatial_geomx", + "visium_input_file_path": "gs://asap-wdl-ci/spatial_visium" } } } diff --git a/wf-common b/wf-common index 4bf08b0..dd31646 160000 --- a/wf-common +++ b/wf-common @@ -1 +1 @@ -Subproject commit 4bf08b09cbf8f7cc0c6ee61ad31cf6e2e230696b +Subproject commit dd31646b060593e1424b31588b3f845ca2853c74 diff --git a/workflows/integrate_data/integrate_data.wdl b/workflows/integrate_data/integrate_data.wdl index 1241820..19235d9 100644 --- a/workflows/integrate_data/integrate_data.wdl +++ b/workflows/integrate_data/integrate_data.wdl @@ -97,7 +97,6 @@ task integrate_sample_data { memory: "~{mem_gb} GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 10 zones: zones } @@ -161,7 +160,6 @@ task cluster { memory: "~{mem_gb} GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 10 zones: zones } diff --git a/workflows/pmdbs_spatial_geomx/inputs.json b/workflows/pmdbs_spatial_geomx/inputs.json deleted file mode 100644 index 89a7f40..0000000 --- a/workflows/pmdbs_spatial_geomx/inputs.json +++ /dev/null @@ -1,21 +0,0 @@ -{ - "pmdbs_spatial_geomx_analysis.projects": "Array[WomCompositeType {\n slides -> Array[WomCompositeType {\n slide_id -> String\ngeomx_lab_annotation_xlsx -> File\nsamples -> Array[WomCompositeType {\n fastq_I1s -> Array[File]\nsample_id -> String\nfastq_R1s -> Array[File]+\nbatch -> String?\nfastq_I2s -> Array[File]\nfastq_R2s -> Array[File]+ \n}] \n}]\nproject_sample_metadata_csv -> File\nteam_id -> String\ndataset_id -> String\ngeomx_config_ini -> File\nstaging_data_buckets -> Array[String]\nrun_project_cohort_analysis -> Boolean\ndataset_doi_url -> String\nraw_data_bucket -> String \n}]", - "pmdbs_spatial_geomx_analysis.geomxngs_config_pkc": "File", - "pmdbs_spatial_geomx_analysis.min_segment_reads": "Int (optional, default = 1000)", - "pmdbs_spatial_geomx_analysis.min_percent_reads_trimmed": "Int (optional, default = 80)", - "pmdbs_spatial_geomx_analysis.min_percent_reads_stitched": "Int (optional, default = 80)", - "pmdbs_spatial_geomx_analysis.min_percent_reads_aligned": "Int (optional, default = 80)", - "pmdbs_spatial_geomx_analysis.min_saturation": "Int (optional, default = 50)", - "pmdbs_spatial_geomx_analysis.min_neg_ctrl_count": "Int (optional, default = 1)", - "pmdbs_spatial_geomx_analysis.max_ntc_count": "Int (optional, default = 1000)", - "pmdbs_spatial_geomx_analysis.min_nuclei": "Int (optional, default = 100)", - "pmdbs_spatial_geomx_analysis.min_segment_area": "Int (optional, default = 5000)", - "pmdbs_spatial_geomx_analysis.cell_type_markers_list": "File", - "pmdbs_spatial_geomx_analysis.min_genes_detected_in_percent_segment": "Float (optional, default = 0.01)", - "pmdbs_spatial_geomx_analysis.n_top_genes": "Int (optional, default = 3000)", - "pmdbs_spatial_geomx_analysis.n_comps": "Int (optional, default = 30)", - "pmdbs_spatial_geomx_analysis.batch_key": "String (optional, default = \"batch_id\")", - "pmdbs_spatial_geomx_analysis.leiden_resolution": "Float (optional, default = 0.4)", - "pmdbs_spatial_geomx_analysis.container_registry": "String", - "pmdbs_spatial_geomx_analysis.zones": "String (optional, default = \"us-central1-c us-central1-f\")" -} diff --git a/workflows/pmdbs_spatial_visium/inputs.json b/workflows/pmdbs_spatial_visium/inputs.json deleted file mode 100644 index d165e2c..0000000 --- a/workflows/pmdbs_spatial_visium/inputs.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "pmdbs_spatial_visium_analysis.projects": "Array[WomCompositeType {\n team_id -> String\ndataset_id -> String\nsamples -> Array[WomCompositeType {\n fastq_I1s -> Array[File]\nsample_id -> String\nfastq_R1s -> Array[File]+\nbatch -> String?\nvisium_capture_area -> String\nvisium_brightfield_image -> File\nfastq_I2s -> Array[File]\nvisium_slide_serial_number -> String\nfastq_R2s -> Array[File]+ \n}]\nstaging_data_buckets -> Array[String]\nrun_project_cohort_analysis -> Boolean\ndataset_doi_url -> String\nraw_data_bucket -> String \n}]", - "pmdbs_spatial_visium_analysis.spaceranger_reference_data": "File", - "pmdbs_spatial_visium_analysis.visium_probe_set_csv": "File? (optional)", - "pmdbs_spatial_visium_analysis.filter_cells_min_counts": "Int (optional, default = 5000)", - "pmdbs_spatial_visium_analysis.filter_cells_min_genes": "Int (optional, default = 3000)", - "pmdbs_spatial_visium_analysis.filter_genes_min_cells": "Int (optional, default = 10)", - "pmdbs_spatial_visium_analysis.filter_mt_max_percent": "Float (optional, default = 0.2)", - "pmdbs_spatial_visium_analysis.normalize_target_sum": "Float (optional, default = 10000)", - "pmdbs_spatial_visium_analysis.n_top_genes": "Int (optional, default = 3000)", - "pmdbs_spatial_visium_analysis.n_comps": "Int (optional, default = 30)", - "pmdbs_spatial_visium_analysis.batch_key": "String (optional, default = \"batch_id\")", - "pmdbs_spatial_visium_analysis.leiden_resolution": "Float (optional, default = 0.4)", - "pmdbs_spatial_visium_analysis.container_registry": "String", - "pmdbs_spatial_visium_analysis.zones": "String (optional, default = \"us-central1-c us-central1-f\")" -} diff --git a/workflows/pmdbs_spatial_geomx/cohort_analysis/cohort_analysis.wdl b/workflows/spatial_geomx/cohort_analysis/cohort_analysis.wdl similarity index 99% rename from workflows/pmdbs_spatial_geomx/cohort_analysis/cohort_analysis.wdl rename to workflows/spatial_geomx/cohort_analysis/cohort_analysis.wdl index 34a3bde..ff9086d 100644 --- a/workflows/pmdbs_spatial_geomx/cohort_analysis/cohort_analysis.wdl +++ b/workflows/spatial_geomx/cohort_analysis/cohort_analysis.wdl @@ -243,7 +243,6 @@ task merge_and_prep { memory: "~{mem_gb} GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 15 zones: zones } @@ -309,7 +308,6 @@ task export_final_artifacts { memory: "~{mem_gb} GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 15 zones: zones } diff --git a/workflows/spatial_geomx/inputs.json b/workflows/spatial_geomx/inputs.json new file mode 100644 index 0000000..9eefe5d --- /dev/null +++ b/workflows/spatial_geomx/inputs.json @@ -0,0 +1,21 @@ +{ + "spatial_geomx_analysis.projects": "Array[WomCompositeType {\n slides -> Array[WomCompositeType {\n slide_id -> String\ngeomx_lab_annotation_xlsx -> File\nsamples -> Array[WomCompositeType {\n fastq_I1s -> Array[File]\nsample_id -> String\nfastq_R1s -> Array[File]+\nbatch -> String?\nfastq_I2s -> Array[File]\nfastq_R2s -> Array[File]+ \n}] \n}]\nproject_sample_metadata_csv -> File\nteam_id -> String\ndataset_id -> String\ngeomx_config_ini -> File\nstaging_data_buckets -> Array[String]\nrun_project_cohort_analysis -> Boolean\ndataset_doi_url -> String\nraw_data_bucket -> String \n}]", + "spatial_geomx_analysis.geomxngs_config_pkc": "File", + "spatial_geomx_analysis.min_segment_reads": "Int (optional, default = 1000)", + "spatial_geomx_analysis.min_percent_reads_trimmed": "Int (optional, default = 80)", + "spatial_geomx_analysis.min_percent_reads_stitched": "Int (optional, default = 80)", + "spatial_geomx_analysis.min_percent_reads_aligned": "Int (optional, default = 80)", + "spatial_geomx_analysis.min_saturation": "Int (optional, default = 50)", + "spatial_geomx_analysis.min_neg_ctrl_count": "Int (optional, default = 1)", + "spatial_geomx_analysis.max_ntc_count": "Int (optional, default = 1000)", + "spatial_geomx_analysis.min_nuclei": "Int (optional, default = 100)", + "spatial_geomx_analysis.min_segment_area": "Int (optional, default = 5000)", + "spatial_geomx_analysis.cell_type_markers_list": "File", + "spatial_geomx_analysis.min_genes_detected_in_percent_segment": "Float (optional, default = 0.01)", + "spatial_geomx_analysis.n_top_genes": "Int (optional, default = 3000)", + "spatial_geomx_analysis.n_comps": "Int (optional, default = 30)", + "spatial_geomx_analysis.batch_key": "String (optional, default = \"batch_id\")", + "spatial_geomx_analysis.leiden_resolution": "Float (optional, default = 0.4)", + "spatial_geomx_analysis.container_registry": "String", + "spatial_geomx_analysis.zones": "String (optional, default = \"us-central1-c us-central1-f\")" +} diff --git a/workflows/pmdbs_spatial_geomx/main.wdl b/workflows/spatial_geomx/main.wdl similarity index 95% rename from workflows/pmdbs_spatial_geomx/main.wdl rename to workflows/spatial_geomx/main.wdl index 8fa6423..168e0e6 100644 --- a/workflows/pmdbs_spatial_geomx/main.wdl +++ b/workflows/spatial_geomx/main.wdl @@ -1,6 +1,6 @@ version 1.0 -# Harmonized human and non-human PMDBS spatial transcriptomics workflow entrypoint for Nanostring GeoMx data +# Harmonized human PMDBS and non-human spatial transcriptomics workflow entrypoint for Nanostring GeoMx data import "structs.wdl" import "../../wf-common/wdl/tasks/get_workflow_metadata.wdl" as GetWorkflowMetadata @@ -8,7 +8,7 @@ import "preprocess/preprocess.wdl" as Preprocess import "process_to_adata/process_to_adata.wdl" as ProcessToAdata import "cohort_analysis/cohort_analysis.wdl" as CohortAnalysis -workflow pmdbs_spatial_geomx_analysis { +workflow spatial_geomx_analysis { input { Array[Project] projects @@ -40,9 +40,9 @@ workflow pmdbs_spatial_geomx_analysis { } String workflow_execution_path = "workflow_execution" - String workflow_name = "pmdbs_spatial_geomx" + String workflow_name = "spatial_geomx" String workflow_version = "v1.0.0" - String workflow_release = "https://github.com/ASAP-CRN/pmdbs-spatial-transcriptomics-wf/releases/tag/pmdbs_spatial_geomx_analysis-~{workflow_version}" + String workflow_release = "https://github.com/ASAP-CRN/spatial-transcriptomics-wf/releases/tag/spatial_geomx_analysis-~{workflow_version}" call GetWorkflowMetadata.get_workflow_metadata { input: @@ -182,7 +182,7 @@ workflow pmdbs_spatial_geomx_analysis { } meta { - description: "Harmonized human and non-human postmortem-derived brain sequencing (PMDBS) spatial transcriptomics workflow for Nanostring GeoMx data" + description: "Harmonized human postmortem-derived brain sequencing (PMDBS) and non-human spatial transcriptomics workflow for Nanostring GeoMx data" } parameter_meta { diff --git a/workflows/pmdbs_spatial_geomx/preprocess/preprocess.wdl b/workflows/spatial_geomx/preprocess/preprocess.wdl similarity index 99% rename from workflows/pmdbs_spatial_geomx/preprocess/preprocess.wdl rename to workflows/spatial_geomx/preprocess/preprocess.wdl index 54c3407..b07c72a 100644 --- a/workflows/pmdbs_spatial_geomx/preprocess/preprocess.wdl +++ b/workflows/spatial_geomx/preprocess/preprocess.wdl @@ -252,7 +252,6 @@ task check_output_files_exist { memory: "4 GB" disks: "local-disk 20 HDD" preemptible: 3 - maxRetries: 2 zones: zones } @@ -360,7 +359,6 @@ task fastq_to_dcc { memory: "~{mem_gb} GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 15 zones: zones } @@ -437,7 +435,6 @@ task dcc_to_rds { memory: "~{mem_gb} GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 15 zones: zones } @@ -529,7 +526,6 @@ task qc { memory: "~{mem_gb} GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 15 zones: zones } diff --git a/workflows/pmdbs_spatial_geomx/process_to_adata/process_to_adata.wdl b/workflows/spatial_geomx/process_to_adata/process_to_adata.wdl similarity index 99% rename from workflows/pmdbs_spatial_geomx/process_to_adata/process_to_adata.wdl rename to workflows/spatial_geomx/process_to_adata/process_to_adata.wdl index caab706..2f53e3c 100644 --- a/workflows/pmdbs_spatial_geomx/process_to_adata/process_to_adata.wdl +++ b/workflows/spatial_geomx/process_to_adata/process_to_adata.wdl @@ -134,7 +134,6 @@ task filter_and_normalize { memory: "~{mem_gb} GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 15 zones: zones } @@ -186,7 +185,6 @@ task rds_to_adata { memory: "~{mem_gb} GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 15 zones: zones } diff --git a/workflows/pmdbs_spatial_geomx/structs.wdl b/workflows/spatial_geomx/structs.wdl similarity index 100% rename from workflows/pmdbs_spatial_geomx/structs.wdl rename to workflows/spatial_geomx/structs.wdl diff --git a/workflows/pmdbs_spatial_geomx/workflow_diagram.svg b/workflows/spatial_geomx/workflow_diagram.svg similarity index 100% rename from workflows/pmdbs_spatial_geomx/workflow_diagram.svg rename to workflows/spatial_geomx/workflow_diagram.svg diff --git a/workflows/pmdbs_spatial_visium/cohort_analysis/cohort_analysis.wdl b/workflows/spatial_visium/cohort_analysis/cohort_analysis.wdl similarity index 99% rename from workflows/pmdbs_spatial_visium/cohort_analysis/cohort_analysis.wdl rename to workflows/spatial_visium/cohort_analysis/cohort_analysis.wdl index 0cadcfd..c7f902c 100644 --- a/workflows/pmdbs_spatial_visium/cohort_analysis/cohort_analysis.wdl +++ b/workflows/spatial_visium/cohort_analysis/cohort_analysis.wdl @@ -278,7 +278,6 @@ task merge_and_plot_qc_metrics { memory: "~{mem_gb} GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 15 zones: zones } @@ -360,7 +359,6 @@ task filter_and_normalize { memory: "~{mem_gb} GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 15 zones: zones } @@ -427,7 +425,6 @@ task plot_spatial { memory: "~{mem_gb} GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 15 zones: zones } diff --git a/workflows/pmdbs_spatial_visium/cohort_analysis/spatial_statistics/spatial_statistics.wdl b/workflows/spatial_visium/cohort_analysis/spatial_statistics/spatial_statistics.wdl similarity index 99% rename from workflows/pmdbs_spatial_visium/cohort_analysis/spatial_statistics/spatial_statistics.wdl rename to workflows/spatial_visium/cohort_analysis/spatial_statistics/spatial_statistics.wdl index d5c8060..9bd2403 100644 --- a/workflows/pmdbs_spatial_visium/cohort_analysis/spatial_statistics/spatial_statistics.wdl +++ b/workflows/spatial_visium/cohort_analysis/spatial_statistics/spatial_statistics.wdl @@ -94,7 +94,6 @@ task spatially_variable_gene_analysis { memory: "~{mem_gb} GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 15 zones: zones } diff --git a/workflows/spatial_visium/inputs.json b/workflows/spatial_visium/inputs.json new file mode 100644 index 0000000..4bdf084 --- /dev/null +++ b/workflows/spatial_visium/inputs.json @@ -0,0 +1,16 @@ +{ + "spatial_visium_analysis.projects": "Array[WomCompositeType {\n team_id -> String\ndataset_id -> String\nsamples -> Array[WomCompositeType {\n fastq_I1s -> Array[File]\nsample_id -> String\nfastq_R1s -> Array[File]+\nbatch -> String?\nvisium_capture_area -> String\nvisium_brightfield_image -> File\nfastq_I2s -> Array[File]\nvisium_slide_serial_number -> String\nfastq_R2s -> Array[File]+ \n}]\nstaging_data_buckets -> Array[String]\nrun_project_cohort_analysis -> Boolean\ndataset_doi_url -> String\nraw_data_bucket -> String \n}]", + "spatial_visium_analysis.spaceranger_reference_data": "File", + "spatial_visium_analysis.visium_probe_set_csv": "File? (optional)", + "spatial_visium_analysis.filter_cells_min_counts": "Int (optional, default = 5000)", + "spatial_visium_analysis.filter_cells_min_genes": "Int (optional, default = 3000)", + "spatial_visium_analysis.filter_genes_min_cells": "Int (optional, default = 10)", + "spatial_visium_analysis.filter_mt_max_percent": "Float (optional, default = 0.2)", + "spatial_visium_analysis.normalize_target_sum": "Float (optional, default = 10000)", + "spatial_visium_analysis.n_top_genes": "Int (optional, default = 3000)", + "spatial_visium_analysis.n_comps": "Int (optional, default = 30)", + "spatial_visium_analysis.batch_key": "String (optional, default = \"batch_id\")", + "spatial_visium_analysis.leiden_resolution": "Float (optional, default = 0.4)", + "spatial_visium_analysis.container_registry": "String", + "spatial_visium_analysis.zones": "String (optional, default = \"us-central1-c us-central1-f\")" +} diff --git a/workflows/pmdbs_spatial_visium/main.wdl b/workflows/spatial_visium/main.wdl similarity index 94% rename from workflows/pmdbs_spatial_visium/main.wdl rename to workflows/spatial_visium/main.wdl index 8f8a6b0..190a37e 100644 --- a/workflows/pmdbs_spatial_visium/main.wdl +++ b/workflows/spatial_visium/main.wdl @@ -1,13 +1,13 @@ version 1.0 -# Harmonized human and non-human PMDBS spatial transcriptomics workflow entrypoint for 10x Visium data +# Harmonized human PMDBS and non-human spatial transcriptomics workflow entrypoint for 10x Visium data import "structs.wdl" import "../../wf-common/wdl/tasks/get_workflow_metadata.wdl" as GetWorkflowMetadata import "preprocess/preprocess.wdl" as Preprocess import "cohort_analysis/cohort_analysis.wdl" as CohortAnalysis -workflow pmdbs_spatial_visium_analysis { +workflow spatial_visium_analysis { input { Array[Project] projects @@ -30,9 +30,9 @@ workflow pmdbs_spatial_visium_analysis { } String workflow_execution_path = "workflow_execution" - String workflow_name = "pmdbs_spatial_visium" + String workflow_name = "spatial_visium" String workflow_version = "v1.0.0" - String workflow_release = "https://github.com/ASAP-CRN/pmdbs-spatial-transcriptomics-wf/releases/tag/pmdbs_spatial_visium_analysis-~{workflow_version}" + String workflow_release = "https://github.com/ASAP-CRN/spatial-transcriptomics-wf/releases/tag/spatial_visium_analysis-~{workflow_version}" call GetWorkflowMetadata.get_workflow_metadata { input: @@ -151,7 +151,7 @@ workflow pmdbs_spatial_visium_analysis { } meta { - description: "Harmonized human and non-human postmortem-derived brain sequencing (PMDBS) spatial transcriptomics workflow for 10x Visium data" + description: "Harmonized human postmortem-derived brain sequencing (PMDBS) and non-human spatial transcriptomics workflow for 10x Visium data" } parameter_meta { diff --git a/workflows/pmdbs_spatial_visium/preprocess/preprocess.wdl b/workflows/spatial_visium/preprocess/preprocess.wdl similarity index 99% rename from workflows/pmdbs_spatial_visium/preprocess/preprocess.wdl rename to workflows/spatial_visium/preprocess/preprocess.wdl index f282c8b..4af4294 100644 --- a/workflows/pmdbs_spatial_visium/preprocess/preprocess.wdl +++ b/workflows/spatial_visium/preprocess/preprocess.wdl @@ -240,7 +240,6 @@ task check_output_files_exist { memory: "4 GB" disks: "local-disk 20 HDD" preemptible: 3 - maxRetries: 2 zones: zones } @@ -388,7 +387,6 @@ task spaceranger_count { memory: "~{mem_gb} GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 15 zones: zones } @@ -468,7 +466,6 @@ task counts_to_adata { memory: "4 GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 15 zones: zones } @@ -534,7 +531,6 @@ task qc { memory: "~{mem_gb} GB" disks: "local-disk ~{disk_size} HDD" preemptible: 3 - maxRetries: 2 bootDiskSizeGb: 15 zones: zones } diff --git a/workflows/pmdbs_spatial_visium/structs.wdl b/workflows/spatial_visium/structs.wdl similarity index 100% rename from workflows/pmdbs_spatial_visium/structs.wdl rename to workflows/spatial_visium/structs.wdl diff --git a/workflows/pmdbs_spatial_visium/workflow_diagram.svg b/workflows/spatial_visium/workflow_diagram.svg similarity index 100% rename from workflows/pmdbs_spatial_visium/workflow_diagram.svg rename to workflows/spatial_visium/workflow_diagram.svg