RFC Stage 3: Restructure pipeline resource labels into orthogonal CPU / memory / time axes#140
Open
pinin4fjords wants to merge 4 commits into
Open
RFC Stage 3: Restructure pipeline resource labels into orthogonal CPU / memory / time axes#140pinin4fjords wants to merge 4 commits into
pinin4fjords wants to merge 4 commits into
Conversation
… / memory / time axes Implements stage 3 of nf-core#139 (approved 2026-05-13). Companion implementation: - nf-core/tools#4265 (template change) - nf-core/website#4212 (documentation) [skip ci]
The /docs/contributing/migrating-resource-labels URL 404s until nf-core/website#4212 is merged, so point reviewers at the draft PR instead. [skip ci]
muffato
reviewed
May 13, 2026
|
|
||
| The current label set was designed to be simple and to "just work" out of the box, with users expected to override values in their own configs. In practice: | ||
|
|
||
| - There is no way to express "many CPUs but little memory". Authors either pick `process_high` (which over-allocates memory by a large factor) or stack `process_high_memory` on top of `process_medium`, which is undocumented and counter-intuitive. |
Member
There was a problem hiding this comment.
The second half is weird. Why would someone stick the process_high_memory label onto a process that needs "many CPUS but little memory" ?
Did you mean "little CPUs but high memory ?
…rd cases [skip ci] Address @muffato review feedback: the original bullet conflated "many CPUs, little memory" (no clean expression) with the workaround for the reverse case (stacking process_high_memory on process_medium). Now phrased as two bullets. Also tidy the ad-hoc-labels examples: scnanoseq's process_high_memory matches the template default, so that case is stacking, not inventing. Only genomeassembler's process_high_cpu is a true invention. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…kip ci]
The previous parenthetical ("later labels win, definition order matters") was
ambiguous - it could be read as "the last label on the process wins". Spell
out that precedence is controlled by the order of withLabel: blocks in
base.config, not by the order labels are declared on the process.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stage 3 RFC for the resource-label restructure approved at #139 (10/10, 2026-05-13).
Replaces the bundled labels (
process_low,process_medium,process_high,process_long,process_high_memory) with three orthogonal axes (process_cpus_*,process_mem_*,process_time_*) that are mixed and matched per process, with a documented deprecation path for the existing labels.Companion implementation PRs
The RFC is intended to be merged once both companion PRs are ready for review and the open questions in the RFC body have been resolved.
Test plan
process_low → process_mem_mediumrow is the most opinionated call).base.configare acceptable, or propose alternatives.