I have a suggestion for an enhancement to the weather-dl tool. Currently, when downloading data for a month range that spans multiple years, the process requires manual splitting of the month range into separate configurations and executing multiple jobs for each part. This process can be tedious and time-consuming.
For instance, if we wish to download data from April 1994 to August 2012, and the target path follows this format: (gs://xyz/{year}/{year}{month:02d}_hres_var.grb2), we need to split the dates into three parts as follows:
1st part :
year=1994
month=04/to/12
day=all
For the 2nd part:
year=1995/to/2011
month=01/to/12
day=all
And for the last part:
year=2012
month=01/to/08
day=all
I propose an enhancement in the weather-dl parser by introducing a new keyword, such as month_year, in the configuration.
The value of this keyword would look like 08-2012 or 01-2022. Additionally, we can add a script to the parser of weather-dl that can correctly interpret this keyword.
P.S.: A similar use-case can be found in the Arco-era5 dataset. Please refer to any file containing sfc in the filename for more context.
I have a suggestion for an enhancement to the
weather-dltool. Currently, when downloading data for amonth rangethat spans multiple years, the process requires manual splitting of themonth rangeinto separate configurations and executing multiple jobs for each part. This process can be tedious and time-consuming.For instance, if we wish to download data from
April 1994toAugust 2012, and thetarget pathfollows this format:(gs://xyz/{year}/{year}{month:02d}_hres_var.grb2), we need to split the dates into three parts as follows:1st part :
year=1994month=04/to/12day=allFor the 2nd part:
year=1995/to/2011month=01/to/12day=allAnd for the last part:
year=2012month=01/to/08day=allI propose an enhancement in the
weather-dl parserby introducing a new keyword, such asmonth_year, in the configuration.The value of this keyword would look like
08-2012or01-2022. Additionally, we can add a script to theparserofweather-dlthat can correctly interpret this keyword.P.S.: A similar use-case can be found in the Arco-era5 dataset. Please refer to any file containing
sfcin the filename for more context.