`pipelines download` not working as expected?

[fellen31]

Description of the bug

After looking at #3430 and this Slack thread, I thought the offline usage documentation needed an update, but found some issues and questions:

• With the current behaviour of the template, should --download-configuration be yes by default?
• Currently, nextflow.config is only updated when --container-system singularity and --container-cache-utilisation is set to copy but not amend, is this expected?
• Currently, with --container-system singularity and --container-cache-utilisation copy, nextflow.config is updated with the wrong paths if you've defined a shared Singularity image download folder.
• Should pipelines download really update the nextflow.config of the pipeline, or is there a better way to do this?

Command used and terminal output

System information

No response

[MatthiasZepper]

With the current behaviour of the template, should --download-configuration be yes by default?

What do you consider to be the current behavior, and which gain to you see there?
I fail to see any other effect than downloading an additional repository with every pipeline by changing the default?

Currently, nextflow.config is only updated when --container-system singularity and --container-cache-utilisation is set to copy but not amend, is this expected?

Yes, because you are amending to the $NXF_SINGULARITY_CACHEDIR. It is expected that Nextflow will take care of using the images from there without any specific configuration updates.

Currently, with --container-system singularity and --container-cache-utilisation copy, nextflow.config is updated with the wrong paths if you've defined a shared Singularity image download folder.

It can only update the path to the relative location the tool will download the images to. There is no way of knowing in advance how you will be organizing your downloaded images later.

Should pipelines download really update the nextflow.config of the pipeline, or is there a better way to do this?

I am open to suggestions. The intention was, that users using nf-core pipelines download end up with a single folder for that pipeline that includes configs and containers and just runs. It is assumed that people who download multiple pipelines and change the code organization know what they are doing and can provide their own custom config.

[fellen31]

Thanks for the explanation! I had these questions when trying to update the documentation, which was at the time I believe a bit outdated. It was not clear to me why some of the flags behaved the way they did. Perhaps we could clarify the documentation a bit.

[MatthiasZepper]

At the moment, Erik is working hard on #3634 , which will soon bring many changes to the behavior and API, so at the moment there is no use of changing the documentation. But I am aware of your open PR to the website with some documentation update and we are happing to have you reviewing the manual once the changes have been incorporated.