Skip to content

Add parallel I/O output system#713

Open
SeanBryan51 wants to merge 12 commits into
mainfrom
add-parallelio-output-module
Open

Add parallel I/O output system#713
SeanBryan51 wants to merge 12 commits into
mainfrom
add-parallelio-output-module

Conversation

@SeanBryan51
Copy link
Copy Markdown
Collaborator

@SeanBryan51 SeanBryan51 commented Apr 7, 2026

This change brings in a new output system based on the parallel I/O infrastructure introduced in #706, and is a direct replacement of the previous output module used for offline CABLE. The main motivation behind this is to add MPI support to the serial offline driver, and eventually, to replace the legacy MPI implementation (#358). This also makes progress towards the proposed output redesign (#715) by introducing the underlying data structures needed for its implementation.

The new output system brings in the following enhancements:

  • Optional parallel I/O support via PIO
  • A new "aggregator" class for calculating time aggregations throughout a simulation
  • New data structures for describing output variables which allows for easily adding new diagnostics and/or restarts variables

This change should be brought in after #712.

Type of change

Please delete options that are not relevant.

  • Enhancement
  • New or updated documentation

Checklist

  • The new content is accessible and located in the appropriate section
  • I have checked that links are valid and point to the intended content
  • I have checked my code/text and corrected any misspellings

Testing

  • Are the changes bitwise-compatible with the main branch? If working on an optional feature, are the results bitwise-compatible when this feature is off? If yes, copy benchcab output showing successful completion of the bitwise compatibility tests or equivalent results below this line.

CABLE benchcab runs tested using ifort 2021.10.0.

2026-04-28 06:43:59,566 - INFO - benchcab.benchcab.py:380 - Running comparison tasks...
2026-04-28 06:43:59,593 - INFO - benchcab.benchcab.py:381 - tasks: 168 (models: 2, sites: 42, science configurations: 4)
2026-04-28 06:46:47,366 - INFO - benchcab.benchcab.py:391 - 0 failed, 168 passed

Please add a reviewer when ready for review.


📚 Documentation preview 📚: https://cable--713.org.readthedocs.build/en/713/

@SeanBryan51 SeanBryan51 changed the title Add parallelio output module Add new parallel I/O output system Apr 7, 2026
@SeanBryan51 SeanBryan51 changed the title Add new parallel I/O output system Add parallel I/O output system Apr 7, 2026
@SeanBryan51 SeanBryan51 force-pushed the add-parallelio-output-module branch from 4c009fd to b226a95 Compare April 8, 2026 15:29
@SeanBryan51 SeanBryan51 force-pushed the add-parallelio-output-module branch 8 times, most recently from beefeab to 0d53f58 Compare April 13, 2026 19:05
@SeanBryan51 SeanBryan51 force-pushed the add-parallelio-output-module branch from 458d67d to df516f5 Compare April 27, 2026 19:31
@SeanBryan51 SeanBryan51 marked this pull request as ready for review April 27, 2026 19:49
@SeanBryan51
Copy link
Copy Markdown
Collaborator Author

Hi @abhaasgoyal @Whyborn, thank you for your time in reviewing these PRs! This one is now ready for review, and no rush of course, it's the heaviest one so far!

As an introduction, it might be useful to read these developer documentation pages:

Benchcab is running currently - I had to make a small change to achieve bitwise compatibility, I will update the status soon.

Thank you again!

Copy link
Copy Markdown
Contributor

@Whyborn Whyborn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done a first pass. My main issue at the moment is with the structure of the output_variable_t definition. The current structure is amenable with what already existed within CABLE, but I don't see it being amenable to the planned user output API. The current structure will lead to a combinatory explosion of the number of output variables defined, e.g. a variable with 2 allowed reductions would require 15 output_variable_t, for each combination of reduction (3 including no reduction) and aggregation method (5). These would be allocated regardless of whether the variable is written or not.

I think it would make more sense to have a variable definition, which has only one instance per internal variable. It would contain a reference to the target variable, dimensionality and attributes for the variable. Then we use this and the information supplied by the user's output definition to only define the output variables that are actually going to be written.

Comment thread src/util/aggregator.F90
Comment thread src/offline/cable_diagnostics.F90
Comment thread src/util/output/cable_output.F90
Comment thread src/util/output/cable_output.F90
Comment thread src/util/output/cable_output.F90
… new dimension in top level output module docs
@SeanBryan51
Copy link
Copy Markdown
Collaborator Author

Hi @abhaasgoyal and @Whyborn, thanks again for taking a look at this! As per our meetings, I removed the range_native component from cable_output_variable_t (fcec7c3), and I updated the documentation to better highlight the instructions on adding a new variable dimension (cd26ac0).

As I mentioned, there are some changes that should follow after this PR. These were left out either because they broke bitwise compatibility or because they were too outside the scope of the PR. These are:

  • Remove #ifdef ENFORCE_SINGLE_PRECISION to allow for writing double precision data
  • Replace normalised time averaging with incremental time averaging
  • Add time bounds (Time bounds in CABLE offline output #420)
  • Replace divide_by with scale_by in cable_output_variable_t
  • Harmonise output and restart file differences by removing grid_type == "restart"
  • Built-in documentation generation for the output and restart variable table
  • Move diagnostics definitions (src/offline/cable_diagnostics.F90 and src/offline/cable_diagnostics_casa.F90) to science modules - to coordinate with CABLE library development

Let me know if you any further questions or comments on this! Many thanks

@SeanBryan51 SeanBryan51 requested a review from Whyborn May 20, 2026 18:04
@Whyborn
Copy link
Copy Markdown
Contributor

Whyborn commented May 20, 2026

Which configurations have you used to check bitwise repro? And have you tested with the various current output options available in the output_inclusion_type:

  • The variable groupings here
  • The different frequencies and grids available here

Since this PR commit removes the old system, just want to make sure we don't break existing setups before introducing the new API.

@SeanBryan51
Copy link
Copy Markdown
Collaborator Author

Which configurations have you used to check bitwise repro? And have you tested with the various current output options available in the output_inclusion_type:

  • The variable groupings here
  • The different frequencies and grids available here

Since this PR commit removes the old system, just want to make sure we don't break existing setups before introducing the new API.

@Whyborn thanks for reminding me about this! I'm now checking the grid and frequency settings for both flux tower and spatial using benchcab:

science_configurations:
- cable:
    output:
      grid: land
      averaging: all
- cable:
    output:
      grid: land
      averaging: user6
- cable:
    output:
      grid: land
      averaging: daily
- cable:
    output:
      grid: land
      averaging: monthly
- cable:
    output:
      grid: mask
      averaging: all
- cable:
    output:
      grid: mask
      averaging: user6
- cable:
    output:
      grid: mask
      averaging: daily
- cable:
    output:
      grid: mask
      averaging: monthly

I'll try out a similar method for testing combinations of output variable groups. More to come.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants