Some clarifying questions about using pythie

[zion0210]

Hello,

We are working on correction the monthly average T2m value.

Our data cube only contained reforecast for one month for each year (for example for Jan).
The analysis data cube also contained one month data for each year.

We have some clarifying questions about used pythie.

1. What variables of the input numpy array passed to the Data class? Do I understand correctly that these are variables predictor, start_time, ens, variable, lead_time, lat, lon?
2. Is it possible to train models for each lead time, or does this work automatically in pythie?
3. The preparation step worked without errors

reforecasts_data = Data(...)
analysis_data = Data(...)
forecasts_data = Data(...)
ebc = MBM.BiasCorrection()
ebc.train(analysis_data, reforecasts_data)

but the forecast function sends an error message.

forecast = ebc(forestart_data)

pythie/postprocessors/[MBM.py](https://mbm.py/)", line 611, in get_interpolated_parameters
    f = interp1d(interp_time_timestamp, par1d, kind='cubic')
  File "pythie/lib/python3.7/site-packages/scipy/interpolate/[interpolate.py](https://interpolate.py/)", line 554, in init
    check_finite=False)
  File "pythie/lib/python3.7/site-packages/scipy/interpolate/_[bsplines.py](https://bsplines.py/)", line 1099, in make_interp_spline
    "match: expected %s, got %s+%s" % (nt-n, nleft, nright))
ValueError: The number of derivatives at boundaries does not match: expected 3, got 0+0

If we run the "ebc" function for reforecast_data, it sends identical message.

Best regards,
zion0210

[jodemaey]

Hi,

I do not understand how you can apply Pythie to monthly averaged T2m values as forecasts. Do you have one value per month ? or hourly values over the course of a month ?

I would need more details about the data shape and organization to understand the problem.

I answer your questions below:

1. You can create the Dataarrays in various ways, for example just using the constructor Data(a) where a is a numpy array, or using the advanced functions load_scalars and load_timestamps. So what variables are passed depends on the way you created the numpy arrays and loaded them into Data objects. From what I see, the order of the indices (i.e. the data model) of the Data object that you provide is correct, as explained here.
2. By definition, Pythie will train one model (or "regression") per lead time, along the lead_time axis.
3. This errors is probably due to a problem with the preparation of the Data objects. Basically, if timestamps are provided, Pythie can do interpolation to make post-processing between the lead times provided in the training dataset. It does this interpolation by default. To use it properly, you have to specify the timestamps correctly, and that's where I guess your code is failing. I would suggest not to use this feature for now as this is a bit advanced usage. To disable it, one simply has to not provide timestamps in the Data objects. You can restore them afterward if needed. If you did not set up the timestamps, then it is probably another issue with the Data objects shape or something else.

The bottom line is that without you telling me more about the structure of your data and how you construct your Dataobjects, I cannot help you more on that.

Best regards,

Jonathan