XIOS: Separate out testing of restarts and diagnostics by joewallwork · Pull Request #1010 · nextsimhub/nextsimdg

joewallwork · 2026-01-08T18:54:29Z

XIOS: Separate out testing of restarts and diagnostics

Fixes #977
Fixes #1003
Refinement of #1002

Task List

Defined the tests that specify a complete and functioning change (It may help to create a design specification & test specification)
Implemented the source code change that satisfies the tests
Documented the feature by providing worked example
Updated the README or other documentation
Completed the pre-Request checklist below

Test Description

This PR separates the XIOS reading and writing tests out by type. In addition, we add a read test for the diagnostic outputs and check that they are time-averaged.

Change Description

Code changes are made to account for the case where only one XIOS domain is used when writing. This is required due to an unfortunate consequence of the way XIOS appends the domain name only if there are two or more domains.

In detail: if only one domain type is used then the output file will use x and y dimensions, rather than the versions appended with _<dimName>. We account for this by assuming that x and y imply x_dim and y_dim if encountered in an input file.

Documentation Impact

n/a

Pre-Request Checklist

The requirements of this pull request are fully captured in an issue or design specification and are linked and summarised in the description of this PR
No new warnings are generated
Methods and Tests are commented such that they can be understood without having to obtain additional context
This PR/Issue is labelled as a bug/feature/enhancement/breaking change
This change conforms to the conventions described in the README

joewallwork · 2026-01-13T15:51:25Z

+#ifdef USE_XIOS
+            // Account for the fact that XIOS writes dimensions differently if only one
+            // discretisation is written out. If the dimension is still null at this point then we
+            // assume that the only discretisation used is HDomain-based, i.e., HField, DGField, or
+            // DGSField.
+            if (dim.isNull()) {
+                if (dimensionSpec.name == "x_dim") {
+                    dim = ncFile.getDim("x");
+                } else if (dimensionSpec.name == "y_dim") {
+                    dim = ncFile.getDim("y");
+                } else {
+                    continue;
+                }
+            }
+#endif


This is fixed by #1015, which ensures that XIOS never writes out files with x or y as dimensions.

joewallwork · 2026-01-13T16:19:04Z

+            { "yx", ModelArray::Type::H },
            { "ydimxdim", ModelArray::Type::H },
            { "y_dimx_dim", ModelArray::Type::H },
+            { "yxdg_comp", ModelArray::Type::DG },
            { "ydimxdimdg_comp", ModelArray::Type::DG },
            { "y_dimx_dimdg_comp", ModelArray::Type::DG },
+            { "yxdgstress_comp", ModelArray::Type::DGSTRESS },


These changes will also be reverted in #1015.

timspainNERSC

As usual with the XIOS tests, I haven't done more than a check that the they look reasonable and logical. Everything else looks good.

joewallwork · 2026-01-21T08:46:50Z

As usual with the XIOS tests, I haven't done more than a check that the they look reasonable and logical. Everything else looks good.

Thanks @timspainNERSC. I'm going to wait for the current CI jobs on develop to finish before merging because the Docker one should 🤞 update the Docker image following #1024.

Merges into #1010. Currently in #1010 we have the following issue with writing fields with XIOS: * If there are multiple fields that are (between them) based on two or more XIOS domains then the dimension names get appended with those domain names. By naming the domains `dim`, `vertex`, and `cg`, this gives rise to dimensions `x_dim`, `y_dim`, `x_vertex`, `y_vertex`, `x_cg`, and `y_cg`. * If the fields being written out are only based on one XIOS domain then the appendment doesn't happen, meaning we just have `x` and `y`. The workaround in that PR handles dimensions `x` or `y` in a file being read by assuming they correspond to `x_dim` and `y_dim`. Ideally, we would avoid having `x` or `y` alone in any files and instead make a post-processing of the output file after it is written with XIOS. That's achieved in this PR via a post-processing step. Upon finalising the XIOS context, we: 1. Count how many different XIOS domains were used for writing out. If there was exactly one then we proceed to tackle the issue. 2. Loop overall output files that might have been written by XIOS and check they exist. 3. For any such files that exist, rename the dimensions and variables as appropriate and write out. To achieve this, I had to implement subtraction of a `Duration` from a `TimePoint` because of how XIOS presents the time window suffices.

# XIOS: properly handle filename patterns Fixes #935 Fixes #997 Fixes #898 Fixes #933 Makes #967 redundant Refinement of #1002 ~~Merges into #1010~~ Merges into #1015 ### Task List - [x] Defined the tests that specify a complete and functioning change (*It may help to create a [design specification & test specification](../../../wiki/Specification-Template)*) - [x] Implemented the source code change that satisfies the tests - [x] Documented the feature by providing worked example - [x] Completed the pre-Request checklist below --- # Change Description This PR fixes several things related to how restarts and diagnostics are configured with XIOS: 1. Properly handle file patterns in filenames. 2. Restart file output frequency and split frequency are aligned so that each output file dump only includes a single timestep's worth of data. 3. Restart files are only read once. 4. Ensure XIOS writes the first timestep. Whilst file patterns are made consistent with what's used elsewhere in nextSIM-DG, the output filenames show the range of datetimes separated by a hyphen. --- # Test Description Tests are updated to account for the new behaviour. I decided to drop `XiosFile_test` because it is no longer needed. This meant that we could drop the annoying `unittest` special case. This is what gives the large number of lines removed. --- # Documentation Impact The XIOS documentation page has been updated accordingly. --- ### Pre-Request Checklist - [x] The requirements of this pull request are fully captured in an issue or design specification and are linked and summarised in the description of this PR - [x] No new warnings are generated - [x] The documentation has been updated (or an issue has been created to track the corresponding change) - [x] Methods and Tests are commented such that they can be understood without having to obtain additional context - [x] This PR/Issue is labelled as a bug/feature/enhancement/breaking change - [x] This change conforms to the conventions described in the README

joewallwork added 6 commits January 8, 2026 18:47

Write diagnostics with averaging. Fixes #977.

a5947b2

Make XiosWrite_test output fields time-dependent

f6f6530

Use Unix epoch as origin for forcing file

364b64f

Separate out testing of reading for restarts, forcings, and diagnostics

153a7d0

Separate out testing of writing for restarts and diagnostics

a43288e

Account for XIOS with only one domain. Fixes #1003.

bba40c2

joewallwork self-assigned this Jan 8, 2026

joewallwork added enhancement New feature or request ICCS Tasks or reviews for the ICCS team labels Jan 8, 2026

This was referenced Jan 8, 2026

File behaviour for restarts and diagnostics (take 2) #1002

Closed

XIOS: properly handle filename patterns #1011

Merged

joewallwork marked this pull request as ready for review January 9, 2026 09:31

joewallwork requested a review from timspainNERSC January 9, 2026 09:32

Use proper names for write tests

d55f358

joewallwork mentioned this pull request Jan 13, 2026

Post-process dims in XIOS output files #1015

Merged

joewallwork commented Jan 13, 2026

View reviewed changes

joewallwork added a commit that referenced this pull request Jan 13, 2026

Revert another change from #1010

7908858

joewallwork commented Jan 13, 2026

View reviewed changes

timspainNERSC approved these changes Jan 21, 2026

View reviewed changes

joewallwork merged commit 52ed982 into develop Jan 21, 2026
9 checks passed

joewallwork deleted the issue977_xios-diagnostic-averaging branch January 21, 2026 09:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

XIOS: Separate out testing of restarts and diagnostics#1010

XIOS: Separate out testing of restarts and diagnostics#1010
joewallwork merged 7 commits intodevelopfrom
issue977_xios-diagnostic-averaging

joewallwork commented Jan 8, 2026 •

edited

Loading

Uh oh!

joewallwork Jan 13, 2026

Uh oh!

joewallwork Jan 13, 2026

Uh oh!

timspainNERSC left a comment

Uh oh!

joewallwork commented Jan 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

joewallwork commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

XIOS: Separate out testing of restarts and diagnostics

Task List

Test Description

Change Description

Documentation Impact

Pre-Request Checklist

Uh oh!

joewallwork Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

joewallwork Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

timspainNERSC left a comment

Choose a reason for hiding this comment

Uh oh!

joewallwork commented Jan 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

joewallwork commented Jan 8, 2026 •

edited

Loading