Bug: input file name collision

[samarth8392]

Description of the bug

ERROR ~ Error executing process > 'VC:combineVariants_octopus (1)'

Caused by:
Process VC:combineVariants_octopus input file name collision -- There are multiple input files for each of the following file names: WGS_NC_T_vs_WGS_NC_N_11.octopus.vcf.gz, WGS_NC_T_vs_WGS_NC_N_11.octopus.vcf.gz.tbi

Container:
/data/CCBR_Pipeliner/SIFS/dnousome-ccbr_logan_base-v0.3.9.img

Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out

-- Check '.nextflow.log' file for details

Command used and terminal output

sinteractive --mem=8g -N 1 -n 4

MAINDIR="/data/CCBR/projects/techDev"
DATADIR="/data/CCBR/projects/techDev/data/wgs"

module use /data/CCBR_Pipeliner/Pipelines/ccbrpipeliner/r8/modules && module load ccbrpipeliner/8

# logan --version
# logan, version 0.3.1

# Step1: Initilalization
/vf/users/CCBR/projects/techDev/LOGAN/bin/logan init \
--output $MAINDIR/runs/logan_test2


# Step2: stub-run

/vf/users/CCBR/projects/techDev/LOGAN/bin/logan run /vf/users/CCBR/projects/techDev/LOGAN/main.nf \
--output $MAINDIR/runs/logan_test2 \
--mode local -profile ci_stub \
--genome hg38 \
--sample_sheet /data/CCBR/projects/techDev/logan_pairs.txt \
--fastq_file_input /data/CCBR/projects/techDev/logan_fastq_input.txt \
-stub --vc --sv --cnv

Relevant files

No response

System information

No response

[kelly-sovacool]

running with these parameters while testing to get the problem code faster:

./bin/logan run \
  --mode local -profile ci_stub --genome hg38 \
  --sample_sheet /data/CCBR/projects/techDev/logan_pairs.txt \
  --fastq_file_input /data/CCBR/projects/techDev/logan_fastq_input.txt \
  --vc --qc false --callers octopus -stub

[kelly-sovacool]

the channel is fine before bcftools_index_octopus but repeats the bed interval afterward, can't figure out why so far. trying to run without -stub to see if that makes a difference.

[kelly-sovacool]

the main slurm job timed out after 24 hrs and wasn't quite finished with octopus. we're going to need a smaller test dataset that finishes in a reasonable amount of time.

executor >  slurm (568)
[28/f4f487] ALIGN:matchbed (1)             | 1 of 1 ✔
[b6/29d515] ALIGN:splitinterval (1)        | 1 of 1 ✔
[09/442406] ALIGN:fastp (null)             | 2 of 2 ✔
[d2/5adda8] ALIGN:bwamem2 (null)           | 2 of 2 ✔
[c8/dce4c9] ALIGN:bqsr (140)               | 140 of 140 ✔
[bc/5cc45b] ALIGN:gatherbqsr (2)           | 2 of 2 ✔
[8d/56481d] ALIGN:applybqsr (1)            | 2 of 2 ✔
[9b/67fa41] VC:octopus_tn (68)             | 69 of 70
[a7/3d3771] VC:…s_WGS_NC_N_70.octopus.vcf) | 69 of 69
[-        ] VC:combineVariants_octopus     -
[-        ] VC:annotvep_tn_octopus         -
[-        ] VC:octopus_convertvcf          -
[4b/191c21] VC:octopus_tonly (93)          | 139 of 140
[4e/6fc230] VC:…_NC_N_3.tonly.octopus.vcf) | 139 of 139
[-        ] VC:…bineVariants_octopus_tonly -
[-        ] VC:annotvep_tonly_octopus      -
[-        ] VC:octopus_convertvcf_tonly    -
slurmstepd: error: *** JOB 60903019 ON cn4325 CANCELLED AT 2025-06-24T16:07:16 DUE TO TIME LIMIT ***

[kopardev]

@samarth8392 how was this test dataset created?

[kelly-sovacool]

@samarth8392 my test run (without stub) completed successfully. It seems like this bug is related to -stub, probably not something we can fix. I would recommend using -preview if you need to check that the code syntax is correct during development.

command:

./bin/logan run --genome hg38 --sample_sheet /data/CCBR/projects/techDev/logan_pairs.txt --fastq_file_input /data/CCBR/projects/techDev/logan_fastq_input.txt --vc --qc false --no_tonly --callers octopus

end of the slurm log:

executor >  slurm (143)
[28/f4f487] ALIGN:matchbed (1)             | 1 of 1, cached: 1 ✔
[b6/29d515] ALIGN:splitinterval (1)        | 1 of 1, cached: 1 ✔
[91/2ffc2a] ALIGN:fastp (null)             | 2 of 2, cached: 2 ✔
[4c/de4040] ALIGN:bwamem2 (null)           | 2 of 2, cached: 2 ✔
[96/cdb541] ALIGN:bqsr (140)               | 140 of 140, cached: 140 ✔
[bc/5cc45b] ALIGN:gatherbqsr (2)           | 2 of 2, cached: 2 ✔
[8d/56481d] ALIGN:applybqsr (2)            | 2 of 2, cached: 2 ✔
[9e/5c9632] VC:octopus_tn (23)             | 70 of 70 ✔
[13/e2cb6e] VC:…vs_WGS_NC_N_3.octopus.vcf) | 70 of 70 ✔
[9d/a37621] VC:combineVariants_octopus (1) | 1 of 1 ✔
[19/a66556] VC:annotvep_tn_octopus (1)     | 1 of 1 ✔
[9c/a64857] VC:octopus_convertvcf (1)      | 1 of 1 ✔
Completed at: 27-Jun-2025 07:31:04
Duration    : 20h 44m 40s
CPU hours   : 3'343.4 (3.5% cached)
Succeeded   : 143
Cached      : 150

[kelly-sovacool]

This seems to be a -stub bug and outside of our control