Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion conf/modules.config
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ process {
}
withName: FMHFUNPROFILER {
ext.args = { "${meta.db_params}" }
ext.prefix = params.perform_runmerging ? { "${meta.id}_${meta.db_name}.fmhfunprofiler" } : { "${meta.id}_${meta.run_accession}_${meta.db_name}.fmhfunprofiler" }
ext.prefix = { params.perform_runmerging ? "${meta.id}_${meta.db_name}.fmhfunprofiler" : "${meta.id}_${meta.run_accession}_${meta.db_name}.fmhfunprofiler" }
publishDir = [
path: { "${params.outdir}/fmhfunprofiler/${meta.db_name}/" },
mode: params.publish_dir_mode,
Expand Down
79 changes: 16 additions & 63 deletions docs/output.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ The directories listed below will be created in the results directory after the

## Pipeline overview

The pipeline processes data using the following steps:
The pipeline is built using [Nextflow](https://www.nextflow.io/) and processes data using the following steps:

- [FastQC](#fastqc) - Raw read QC and preprocessing
- [HUMANn v3 / v4](#humann-v3--v4) - Functional profiling via MetaPhlAn + HUMANn
Expand All @@ -20,21 +20,21 @@ The pipeline processes data using the following steps:
- [MultiQC](#multiqc) - Aggregate report describing results and QC from the whole pipeline
- [Pipeline information](#pipeline-information) - Report metrics generated during the workflow execution

## Other than FastQC, MultiQC and pipeline information, all other steps (the profilers) are off by default, and must be switched on manually.
:::note
Other than FastQC, MultiQC and pipeline information, all other steps (the profilers) are off by default, and must be switched on manually.
:::

### FastQC

- [Short reads QC and preprocessing](https://nf-co.re/subworkflows/fastq_shortreads_preprocess_qc/), see [Output section](https://nf-co.re/subworkflows/fastq_shortreads_preprocess_qc/#output) for details.
- Long reads QC and preprocessing (work in progress)
[FastQC](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/) gives general quality metrics about your sequenced reads. It provides information about the quality score distribution across your reads, per base sequence content, adapter contamination, and overrepresented sequences. For further reading and documentation see the [FastQC help pages](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/Help/).

---
- `fastqc/`
- `*_fastqc.html`: FastQC report containing quality metrics for each sample.
- `*_fastqc.zip`: Zip archive containing the FastQC report and data files.

### HUMANn v3 / v4

Enabled with `--run_humann_v3` or `--run_humann_v4`. Each sample is first run through MetaPhlAn to generate a taxonomic profile, which guides HUMANn functional profiling.

<details markdown="1">
<summary>Output files</summary>
[HUMANn](https://huttenhower.sph.harvard.edu/humann/) (HMP Unified Metabolic Analysis Network) is a method for efficiently and accurately profiling the presence/absence and abundance of microbial pathways in a community from metagenomic or metatranscriptomic sequencing data. Enabled with `--run_humann_v3` or `--run_humann_v4`. Each sample is first run through MetaPhlAn to generate a taxonomic profile, which guides HUMANn functional profiling.

- `humann_v3/<sample>/` or `humann_v4/<sample>/`
- `*_genefamilies.tsv`: Gene family abundances in reads per kilobase (RPK), stratified by contributing species.
Expand All @@ -43,39 +43,21 @@ Enabled with `--run_humann_v3` or `--run_humann_v4`. Each sample is first run th
- `metaphlan/<sample>/`
- `*_profile.txt`: Species-level taxonomic abundance profile used as input to HUMANn.

</details>

---

### FMH FunProfiler

Enabled with `--run_fmhfunprofiler`. Uses FracMinHash sketching to rapidly assign reads to functional categories.

<details markdown="1">
<summary>Output files</summary>
[FMH FunProfiler](https://github.com/vdblab/fmhfunprofiler) uses FracMinHash sketching to rapidly assign reads to KEGG Orthology (KO) functional categories. Enabled with `--run_fmhfunprofiler`.

- `fmhfunprofiler/<sample>/`
- `*.ko.txt`: KO (KEGG Orthology) abundance table for the sample.

</details>

---
- `*.fmhfunprofiler.ko`: KO (KEGG Orthology) abundance table for the sample.

### mifaser

Enabled with `--run_mifaser`. Maps reads to functional databases at the protein level to produce enzyme function profiles.

<details markdown="1">
<summary>Output files</summary>
[mifaser](https://bromberglab.org/project/mifaser/) maps reads to functional databases at the protein level to produce enzyme function profiles. Enabled with `--run_mifaser`.

- `mifaser/<db_name>/<sample>/`
- `analysis.tsv`: Tab-separated table of functional assignments with read counts per enzyme function (EC number).
- `analysis.log`: Log file with run statistics including number of reads processed and assigned.

</details>

---

### DIAMOND blastx

Enabled with `--run_diamond`. Performs fast translated alignment of metagenomic reads against a protein reference database. Each read is aligned in all six reading frames and only significant hits are reported.
Expand All @@ -90,12 +72,6 @@ Enabled with `--run_diamond`. Performs fast translated alignment of metagenomic
- `*.tsv`: Tabular alignment results (BLAST tabular format 6) with one row per query-subject hit.
- `*.log`: DIAMOND run log containing alignment statistics (query count, alignment rate, etc.).

</details>

Requires a pre-built `.dmnd` database (see [usage docs](usage.md#diamond-blastx)).

---

### EggNOG-mapper

Enabled with `--run_eggnogmapper`. Assigns functional annotations to sequences by mapping them to orthologous groups in the EggNOG database.
Expand All @@ -109,56 +85,33 @@ Enabled with `--run_eggnogmapper`. Assigns functional annotations to sequences b
- `eggnogmapper/<db_name>/`
- `*.emapper.annotations`: TSV file with functional annotations per query sequence, including GO terms, KEGG pathways, COG categories, and more.
- `*.emapper.seed_orthologs`: TSV linking query sequences to their best seed orthologs _(optional, produced when search is performed)_.
- `*.emapper.hits`: TSV with raw search hits from the search phase .

</details>

---
- `*.emapper.hits`: TSV with raw search hits from the search phase.

### RGI BWT

Enabled with `--run_rgi`. Aligns reads against the CARD database using Bowtie2/BWA to identify antimicrobial resistance genes.

<details markdown="1">
<summary>Output files</summary>
[RGI](https://github.com/arpcard/rgi) (Resistance Gene Identifier) aligns reads against the CARD database using Bowtie2/BWA to identify antimicrobial resistance genes. Enabled with `--run_rgi`.

- `rgi/<db_name>/`
- `*.txt`: Tab-separated AMR gene hit table with gene family, resistance mechanism, drug class, and read counts.
- `*.json`: Full RGI output in JSON format with detailed per-hit annotations.

</details>

---

### MultiQC

<details markdown="1">
<summary>Output files</summary>
[MultiQC](http://multiqc.info) is a visualization tool that generates a single HTML report summarising all samples in your project. Most of the pipeline QC results are visualised in the report and further statistics are available in the report data directory.

- `multiqc/`
- `multiqc_report.html`: a standalone HTML file that can be viewed in your web browser.
- `multiqc_data/`: directory containing parsed statistics from the different tools used in the pipeline.
- `multiqc_plots/`: directory containing static images from the report in various formats.

</details>

[MultiQC](http://multiqc.info) is a visualization tool that generates a single HTML report summarising all samples in your project. Most of the pipeline QC results are visualised in the report and further statistics are available in the report data directory.

Results generated by MultiQC collate pipeline QC from supported tools e.g. FastQC. The pipeline has special steps which also allow the software versions to be reported in the MultiQC output for future traceability. For more information about how to use MultiQC reports, see <http://multiqc.info>.

---

### Pipeline information

<details markdown="1">
<summary>Output files</summary>
[Nextflow](https://www.nextflow.io/docs/latest/tracing.html) provides excellent functionality for generating various reports relevant to the running and execution of the pipeline. This will allow you to troubleshoot errors with the running of the pipeline, and also provide you with other information such as launch commands, run times and resource usage.

- `pipeline_info/`
- Reports generated by Nextflow: `execution_report.html`, `execution_timeline.html`, `execution_trace.txt` and `pipeline_dag.dot`/`pipeline_dag.svg`.
- Reports generated by the pipeline: `pipeline_report.html`, `pipeline_report.txt` and `software_versions.yml`. The `pipeline_report*` files will only be present if the `--email` / `--email_on_fail` parameter's are used when running the pipeline.
- Reformatted samplesheet files used as input to the pipeline: `samplesheet.valid.csv`.
- Parameters used by the pipeline run: `params.json`.

</details>

[Nextflow](https://www.nextflow.io/docs/latest/tracing.html) provides excellent functionality for generating various reports relevant to the running and execution of the pipeline. This will allow you to troubleshoot errors with the running of the pipeline, and also provide you with other information such as launch commands, run times and resource usage.
39 changes: 25 additions & 14 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,10 @@ The samplesheet is a comma-separated file with the following columns:
| `fastq_2` | No | Full path to gzipped FASTQ file for read 2 (paired-end only). Leave empty for single-end or Nanopore reads. |
| `fasta` | No\* | Full path to gzipped FASTA file. Provide instead of FASTQ if your data is already assembled. Must end in `.fa.gz`, `.fna.gz`, or `.fasta.gz`. |

:::note

> \*Either `fastq_1` or `fasta` must be provided for each row.
> :::

### Example samplesheet

Expand Down Expand Up @@ -111,9 +114,11 @@ eggnogmapper,eggnog_v5,eggnogmapper_db,diamond,,/data/databases/eggnog_mapper/eg
eggnogmapper,eggnog_v5,eggnogmapper_data_dir,,,/data/databases/eggnog_mapper/data
```

> The EggNOG data directory can be downloaded with `download_eggnog_data.py` from the eggnog-mapper package. See the [EggNOG-mapper documentation](https://github.com/eggnogdb/eggnog-mapper/wiki) for details.
:::note
The EggNOG data directory can be downloaded with `download_eggnog_data.py` from the eggnog-mapper package. See the [EggNOG-mapper documentation](https://github.com/eggnogdb/eggnog-mapper/wiki) for details.
:::

### mifaser
### Mifaser

[mifaser](https://bromberglab.org/project/mifaser/) performs functional profiling by mapping reads to functional databases at the protein level. It supports both short-read and long-read data. Enable with `--run_mifaser`.

Expand Down Expand Up @@ -164,8 +169,9 @@ tool,db_name,db_entity,db_params,db_type,db_path
rgi,card_v3,,,,/data/databases/card
```

> [!NOTE]
> Wildcard variant databases are not currently supported by the pipeline. Only the core CARD database is used.
:::note
Wildcard variant databases are not currently supported by the pipeline. Only the core CARD database is used.
:::

### DIAMOND blastx

Expand All @@ -185,8 +191,9 @@ diamond makedb --in proteins.faa --db proteins

See the [DIAMOND makedb documentation](https://github.com/bbuchfink/diamond/wiki/3.-Command-line-options#makedb-options) for all available options (e.g. adding taxonomy, setting block size).

> [!IMPORTANT]
> The path should point to the **directory** containing the `.dmnd` file, not the file itself. The pipeline will automatically locate the `.dmnd` file within that directory.
:::warning
The path should point to the **directory** containing the `.dmnd` file, not the file itself. The pipeline will automatically locate the `.dmnd` file within that directory.
:::

## Running the pipeline

Expand Down Expand Up @@ -219,8 +226,9 @@ If you wish to repeatedly use the same parameters for multiple runs, rather than

Pipeline settings can be provided in a `yaml` or `json` file via `-params-file <file>`.

> [!WARNING]
> Do not use `-c <file>` to specify parameters as this will result in errors. Custom config files specified with `-c` must only be used for [tuning process resource specifications](https://nf-co.re/docs/usage/configuration#tuning-workflow-resources), other infrastructural tweaks (such as output directories), or module arguments (args).
:::warning
Do not use `-c <file>` to specify parameters as this will result in errors. Custom config files specified with `-c` must only be used for [tuning process resource specifications](https://nf-co.re/docs/usage/configuration#tuning-workflow-resources), other infrastructural tweaks (such as output directories), or module arguments (args).
:::

The above pipeline run specified with a params file in yaml format:

Expand Down Expand Up @@ -257,22 +265,25 @@ This version number will be logged in reports when you run the pipeline, so that

To further assist in reproducibility, you can use share and reuse [parameter files](#running-the-pipeline) to repeat pipeline runs with the same settings without having to write out a command with every single parameter.

> [!TIP]
> If you wish to share such profile (such as upload as supplementary material for academic publications), make sure to NOT include cluster specific paths to files, nor institutional specific profiles.
:::tip
If you wish to share such profile (such as upload as supplementary material for academic publications), make sure to NOT include cluster specific paths to files, nor institutional specific profiles.
:::

## Core Nextflow arguments

> [!NOTE]
> These options are part of Nextflow and use a _single_ hyphen (pipeline parameters use a double-hyphen)
:::note
These options are part of Nextflow and use a _single_ hyphen (pipeline parameters use a double-hyphen)
:::

### `-profile`

Use this parameter to choose a configuration profile. Profiles can give configuration presets for different compute environments.

Several generic profiles are bundled with the pipeline which instruct the pipeline to use software packaged using different methods (Docker, Singularity, Podman, Shifter, Charliecloud, Apptainer, Conda) - see below.

> [!IMPORTANT]
> We highly recommend the use of Docker or Singularity containers for full pipeline reproducibility, however when this is not possible, Conda is also supported.
:::warning
We highly recommend the use of Docker or Singularity containers for full pipeline reproducibility, however when this is not possible, Conda is also supported.
:::

The pipeline also dynamically loads configurations from [https://github.com/nf-core/configs](https://github.com/nf-core/configs) when it runs, making multiple config profiles for various institutional clusters available at run time. For more information and to check if your system is supported, please see the [nf-core/configs documentation](https://github.com/nf-core/configs#documentation).

Expand Down
10 changes: 4 additions & 6 deletions modules/local/fmhfunprofiler/main.nf
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,8 @@ process FMHFUNPROFILER {
tuple val(dbmeta), path(fmhfunprofiler_db)

output:
// TODO nf-core: Named file extensions MUST be emitted for ALL output channels
tuple val(meta), path("*.fmhfuncprofiler.ko"), emit: ko
// TODO nf-core: List additional required output channels/values here
tuple val("${task.process}"), val('fmh-funprofiler'), val("ghcr.io/vdblab/fmhfunprofiler:20250930a"), emit: versions_fmhfunprofiler, topic: versions
tuple val(meta), path("*.fmhfunprofiler.ko"), emit: ko
tuple val("${task.process}"), val('fmh-funprofiler'), val("20250930a"), emit: versions_fmhfunprofiler, topic: versions

when:
task.ext.when == null || task.ext.when
Expand All @@ -29,15 +27,15 @@ process FMHFUNPROFILER {
${fastqs} \\
${fmhfunprofiler_db} \\
${args} \\
${prefix}.fmhfuncprofiler.ko
${prefix}.fmhfunprofiler.ko

"""

stub:
def args = dbmeta.db_params
def prefix = task.ext.prefix ?: "${meta.id}"
"""
echo ${args} > ${prefix}.fmhfuncprofiler.ko
echo ${args} > ${prefix}.fmhfunprofiler.ko

"""
}
65 changes: 28 additions & 37 deletions modules/local/fmhfunprofiler/meta.yml
Original file line number Diff line number Diff line change
@@ -1,61 +1,52 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/meta-schema.json
# # TODO nf-core: Add a description of the module and list keywords
name: "fmhfunprofiler"
description: write your description here
description: FracMinHash-based functional profiler that assigns reads to KEGG Orthology (KO) categories using sketch databases.
keywords:
- sort
- example
- genomics
- functional profiling
- sketch
- KEGG
- metagenomics
tools:
## TODO nf-core: Add a description and other details for the software below
- "fmhfunprofiler":
description: ""
homepage: ""
documentation: ""
tool_dev_url: ""
description: "Rapid sketch-based functional profiling of metagenomic reads against KEGG Orthology databases using FracMinHash sketching."
homepage: "https://github.com/vdblab/fmhfunprofiler"
documentation: "https://github.com/vdblab/fmhfunprofiler"
tool_dev_url: "https://github.com/vdblab/fmhfunprofiler"
doi: ""
licence: null
licence: ["MIT"]
identifier: null

input:
### TODO nf-core: Add a description of all of the variables used as input
- - meta:
type: map
description: |
Groovy Map containing sample information
e.g. `[ id:'sample1' ]`
- bam:
e.g. `[ id:'sample1', single_end:false ]`
- fastqs:
type: file
description: Sorted BAM/CRAM/SAM file
pattern: "*.{bam,cram,sam}"
ontologies:
- edam: "http://edamontology.org/format_2572" # BAM
- edam: "http://edamontology.org/format_2573" # CRAM
- edam: "http://edamontology.org/format_3462" # SAM
description: One or more gzipped FASTQ (or FASTA) files for the sample.
pattern: "*.{fastq.gz,fq.gz,fa.gz,fasta.gz}"
- - dbmeta:
type: map
description: |
Groovy Map containing database metadata, including db_params (space-separated kmer and sketch scale integers)
e.g. `[ db_name:'kegg_v1', db_params:'11 1000' ]`
- fmhfunprofiler_db:
type: file
description: FracMinHash sketch database (.sig.zip) built against KEGG Orthology entries.
pattern: "*.sig.zip"

output:
### TODO nf-core: Add a description of all of the variables used as output
bam:
- - meta:
- ko:
- meta:
type: map
description: |
Groovy Map containing sample information
e.g. `[ id:'sample1' ]`
- "*.bam":
- "*.fmhfunprofiler.ko":
type: file
description: Sorted BAM/CRAM/SAM file
pattern: "*.{bam,cram,sam}"
ontologies:
- edam: "http://edamontology.org/format_2572" # BAM
- edam: "http://edamontology.org/format_2573" # CRAM
- edam: "http://edamontology.org/format_3462" # SAM
versions:
- "versions.yml":
type: file
description: File containing software versions
pattern: "versions.yml"
ontologies:
- edam: "http://edamontology.org/format_3750" # YAML
description: Tab-separated KO abundance table for the sample.
pattern: "*.fmhfunprofiler.ko"

authors:
- "@nickp60"
Expand Down
2 changes: 0 additions & 2 deletions modules/local/fmhfunprofiler/tests/main.nf.test
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ nextflow_process {
when {
process {
"""
// TODO nf-core: define inputs of the process here. Example:
input[0]= CONCAT_ALL.out[0]
input[1] = Channel.of([[db_params:"11 1000"],file(params.pipelines_testdata_base_path + 'funcprofiler/data/database/fmhfunprofiler/KOs_sketched_scaled_1000_demo.sig.zip')])

Expand Down Expand Up @@ -64,7 +63,6 @@ nextflow_process {
assertAll(
{ assert process.success },
{ assert snapshot(process.out).match() }
//TODO nf-core: Add all required assertions to verify the test output.
)
}

Expand Down
Loading
Loading