Skip to content

Fix krakenuniq build input deletion and tests#11607

Open
mahesh-panchal wants to merge 14 commits into
nf-core:masterfrom
mahesh-panchal:fix/krakenuniq-build-input-deletion
Open

Fix krakenuniq build input deletion and tests#11607
mahesh-panchal wants to merge 14 commits into
nf-core:masterfrom
mahesh-panchal:fix/krakenuniq-build-input-deletion

Conversation

@mahesh-panchal
Copy link
Copy Markdown
Member

@mahesh-panchal mahesh-panchal commented May 12, 2026

Description

  • Fixes a bug that deleted inputs when keep_intermediate was false.
  • Updates tests to stage taxonomy files rather than the directory.

PR checklist

Closes #11557

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the module conventions in the contribution docs
  • If necessary, include test data in your PR.
  • Remove all TODO statements.
  • Broadcast software version numbers to topic: versions - See version_topics
  • Follow the naming conventions.
  • Follow the parameters requirements.
  • Follow the input/output options guidelines.
  • Add a resource label
  • Use BioConda and BioContainers if possible to fulfil software requirements.
  • Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
    • For modules:
      • nf-core modules test <MODULE> --profile docker
      • nf-core modules test <MODULE> --profile singularity
      • nf-core modules test <MODULE> --profile conda
    • For subworkflows:
      • nf-core subworkflows test <SUBWORKFLOW> --profile docker
      • nf-core subworkflows test <SUBWORKFLOW> --profile singularity
      • nf-core subworkflows test <SUBWORKFLOW> --profile conda

Comment thread modules/nf-core/krakenuniq/build/tests/main.nf.test
Comment thread modules/nf-core/krakenuniq/build/tests/main.nf.test
Comment thread modules/nf-core/krakenuniq/build/tests/main.nf.test
Comment thread modules/nf-core/krakenuniq/build/main.nf

"""
${custom_db}
mkdir ${prefix} && cp -rL library taxonomy ${custom_seqid2taxid} ${prefix}
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the custom_db as it seems to be required. The nf-test doesn't describe an alternate way of using this module.
The updated meta.yml from #11587 also removes "Optional".

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the one bit I'm unsure about this PR - if you are copying the entireity of the library directory, which includes 100,000s of genomes, you will massively explode the harddrive usage of the process (e.g. 200 GB of input FASTAs to 400 GB by having two times the same set of FASTA files)

This is something I was trying to avoid with the original mv operation...

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed the dereference -L. cp -r will just copy the symlinks as is, and then the rm cleans those ups. The test with keep_intermediate = true shows the files as being correctly removed. The original symlinks remain and the input files remain intact. The copied symlinks are removed.

Comment thread modules/nf-core/krakenuniq/build/main.nf
@mahesh-panchal mahesh-panchal marked this pull request as ready for review May 12, 2026 10:54
@mahesh-panchal mahesh-panchal changed the title Fix/krakenuniq build input deletion Fix krakenuniq build input deletion May 12, 2026
@mahesh-panchal mahesh-panchal changed the title Fix krakenuniq build input deletion Fix krakenuniq build input deletion and tests May 12, 2026
@mahesh-panchal
Copy link
Copy Markdown
Member Author

mahesh-panchal commented May 12, 2026

Now why would the hash change for taxDB halfway through testing?

Edit: This is worrying. taxDB doesn't appear to be a stable artifact.

@mahesh-panchal mahesh-panchal requested a review from jfy133 May 13, 2026 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KRAKENUNIQ_BUILD is deleting the symlinked inputs when keep_intermediates is false.

2 participants