Skip to content

fix(qpx): emit dataset as qpx_output/* files instead of a directory#39

Merged
ypriverol merged 1 commit into
mainfrom
fix/qpx-flatten-dataset-output
Jun 2, 2026
Merged

fix(qpx): emit dataset as qpx_output/* files instead of a directory#39
ypriverol merged 1 commit into
mainfrom
fix/qpx-flatten-dataset-output

Conversation

@ypriverol

@ypriverol ypriverol commented Jun 2, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Member

What

Change the qpx_dataset output of the bigbio/qpx module from the qpx_output/ directory to the files it contains (qpx_output/*).

Why

Emitting a directory forces downstream pipelines to publish the QPX Parquet dataset under an intermediate qpx_output/ subfolder. With a Nextflow publishDir, a directory output and a sibling file output (the .h5mu) targeting the same results folder race against each other and one is silently dropped — so the dataset cannot be flattened into a single results folder reliably.

Emitting qpx_output/* lets consumers publish the dataset files flat (e.g. directly into qpx/ next to <prefix>.h5mu) via a saveAs that strips the qpx_output/ prefix.

Compatibility

  • mudata and versions outputs are unchanged.
  • The stub still emits a single qpx_output/stub.parquet, so the stub test (qpx_dataset.size() == 1) still holds (a glob with multiple matches is emitted as one list).
  • meta.yml updated to describe the output as files.

Consumed by bigbio/quantmsdiann to flatten QPX export results into a single qpx/ folder.

Summary by CodeRabbit

  • Bug Fixes
    • Improved QPX export process to properly declare and emit individual output files instead of treating the entire output directory as a single unit, enhancing downstream pipeline compatibility and output accuracy.
    • Updated output specification documentation to reflect the refined file-based handling approach.

@ypriverol
fix(qpx): emit dataset as qpx_output/* files instead of a directory
7587959 
Emit the QPX Parquet dataset as individual files (`qpx_output/*`) rather
than the `qpx_output/` directory. This lets downstream pipelines publish the
dataset files flat (e.g. into a single `qpx/` results folder alongside the
`.h5mu`) via a `publishDir` saveAs, instead of being forced to keep an
intermediate `qpx_output/` subdirectory.

The stub still produces a single `qpx_output/stub.parquet`, so the existing
stub test (qpx_dataset.size() == 1) is unaffected.
@qodo-code-review

qodo-code-review Bot commented Jun 2, 2026

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@coderabbitai

coderabbitai Bot commented Jun 2, 2026 •
edited
Loading

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 269337de-c6ec-4290-8184-8a160f5d3a31

📥 Commits

Reviewing files that changed from the base of the PR and between 17a895b and 7587959.

📒 Files selected for processing (2)
  • modules/bigbio/qpx/main.nf
  • modules/bigbio/qpx/meta.yml
📝 Walkthrough

Walkthrough

This PR refines the QPX module's output specification by changing the qpx_dataset emission from a directory glob to individual file globs. The Nextflow process declaration and module metadata are updated in parallel to reflect this change.

Changes

QPX output specification refinement

Layer / File(s) Summary
QPX output specification - files instead of directory
modules/bigbio/qpx/main.nf, modules/bigbio/qpx/meta.yml		 The qpx_dataset output is refactored to emit individual files matching qpx_output/* instead of the entire qpx_output/ directory. Both the Nextflow process output declaration and the module metadata specification are updated together to reflect this change from type: directory to type: file.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~5 minutes

Possibly related PRs

  • bigbio/nf-modules#38: Adds the QPX_EXPORT module with initial main.nf and meta.yml definitions; this PR refines the qpx_dataset output specification in that same module.

Poem

A rabbit hops through directories bright,
Trading folders for files in the light,
Each wildcard * now dances free,
From qpx_output/ to qpx_output/* spree! 🐰✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and specifically describes the main change: switching the qpx_dataset output from a directory to individual files, which is the primary modification across both the Nextflow process and metadata files.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/qpx-flatten-dataset-output

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codacy-production

codacy-production Bot commented Jun 2, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@ypriverol ypriverol merged commit dc3dcca into main Jun 2, 2026
7 checks passed
@ypriverol ypriverol mentioned this pull request Jun 2, 2026
Merged
yueqixuan pushed a commit to yueqixuan/quantmsdiann that referenced this pull request Jun 4, 2026
@ypriverol
feat(qpx): flatten QPX export output into qpx/ (drop qpx_output/ subf…
e2c31a7 
…older)

Publish the QPX Parquet dataset files and the .h5mu MuData file directly
under results/qpx/, removing the intermediate qpx/qpx_output/ level.

- modules/bigbio/qpx: emit the dataset as qpx_output/* files instead of the
  qpx_output/ directory (mirrors bigbio/nf-modules#39), so publishDir can
  flatten the files. A directory output and the sibling .h5mu race in
  publishDir and one is dropped, so flattening config-only is not reliable.
- conf/modules/shared.config: QPX_EXPORT publishDir now strips the
  qpx_output/ prefix via saveAs (tokenize('/').last()) and publishes to qpx/.

Pin bump for modules.json follows once bigbio/nf-modules#39 lands on main.
Docs (output.md, usage.md) already describe the flat results/qpx/ layout.
yueqixuan pushed a commit to yueqixuan/quantmsdiann that referenced this pull request Jun 4, 2026
@ypriverol
chore(qpx): bump pin to bigbio/nf-modules main (qpx_output/* output)
07006e7 
Update modules.json qpx git_sha to the merge commit of
bigbio/nf-modules#39, which emits the QPX dataset as qpx_output/* files.
Keeps nf-core check_local_copy green for the flattened output change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ypriverol