WriterADIOS2: write fields in separate components

[germasch]

No description provided.

[JamesMcClung]

Thanks! This and psc-code/pscpy#42 works perfectly in psc-plot, but they have the effect of changing "t" to "time". I'd personally prefer "t" as the dimension and scalar value, but I can manually rename downstream if you disagree.

Also, what are the reasons for storing dimensions in a single space-separated string? Just seems odd.

[JamesMcClung]

Also, shouldn't it be "step/dimensions", not "step_dimension"?

[germasch]

Thanks! This and psc-code/pscpy#42 works perfectly in psc-plot, but they have the effect of changing "t" to "time". I'd personally prefer "t" as the dimension and scalar value, but I can manually rename downstream if you disagree.

I think that should be easy enough to change (by changing step_dimension to t). Since we just use short symbols (x, y, z) otherwise, I guess it makes sense to do that here, too. To stay consistent, it means the actual time variable (coordinate) also needs to change, though, but it can have a long_name attribute set to time.

Also, what are the reasons for storing dimensions in a single space-separated string? Just seems odd.

Well, I was going to say that I copied this from how netcdf4 does it, but actually that may not be true. I'm also not totally positive that adios2 can handle an attribute that's an array of strings, but I'll take a look.

[germasch]

Also, shouldn't it be "step/dimensions", not "step_dimension"?

Well, what I had in mind is that adios2's concept of steps should translate into an additional dimension in xarray, and that attributes to serve to give it a name (other than step, which I think is default). So it's not quite the same as the usual var/dimensions, since it's purely a dimension, not specific to any particular variable. I somewhat agree that your suggestion reads better, though, but this is also already used in openggcm, so anything that changes here needs changes there, too, plus potential compatibilities woes (though I guess I might not actually care enough).

[JamesMcClung]

Well, what I had in mind is that adios2's concept of steps should translate into an additional dimension in xarray, and that attributes to serve to give it a name (other than step, which I think is default). So it's not quite the same as the usual var/dimensions, since it's purely a dimension, not specific to any particular variable.

That kinda makes sense to me, but the actual implementation doesn't seem to match that. Here's a snippet of the current output:

double   jz_ec               {16, 128, 1} = 0 / 0
string   jz_ec/dimensions    attr   = "z y x"
int32_t  step                attr   = 12
string   step_dimension      attr   = "time"
double   time                scalar = 0.397748
double   x                   {1} = 0.03125 / 0.03125
string   x/dimensions        attr   = "x"

To match how x handles its own dimensions, I think step should be like this:

int32_t  step                attr   = 12
string   step/dimensions     attr   = "step"

And similarly, time would be:

int32_t  t                scalar  = 0.397748
string   t/dimensions     attr    = "t"

I don't know about scalar vs. attr vs. length-1 1D array, or if maybe step/dimensions should be "t" like it currently is.

this is also already used in openggcm, so anything that changes here needs changes there, too, plus potential compatibilities woes (though I guess I might not actually care enough).

Assuming this impacts xarray_adios2, can ggcmpy handle the discrepancy?

[germasch]

Well, so if one were to use step/dimension = step, then data would become associated with (x,y,z,step), and step the default coordinate array for it, but not a particularly useful one. I guess one would also get a time array with step dimensions, but I think it wouldn't even be considered a coordinate by default. I guess one could make it so, and xarray can have multiple coordinates associated with the same dimension, but to me this all seems more confusing than need be.

Also, step could end up kinda overloaded (well, it already is): It could refer to the adios2 step we're in, or to the PSC timestep number. I know step_dimension is an exception / special treatment, but that kinda corresponds to adios2's special handling of steps.

I guess I didn't even remember that psc writes out a step variable (well, actually, it doesn't, it writes it as an attribute, which means it won't turn into a xarray variable, anyway). It could write a step variable, and that would become a coordinate, and one could even turn that into the default coordinate (by having step_dimension == "step" or not present). The existence of step_dimension actually makes step not special in this output format in general. It allows the writer (like PSC) to specify the additional dim/coordinate -- and I'd still be in favor of choosing t for PSC.