Bring in land diags from clm-diags branch

LAND-DIAGS_README.md

@@ -0,0 +1,81 @@
+### Here's a quick start that you can use for Land Diagnostics
+#### Download the adf repo
+On casper:
+1. Navigate to the directory where you want the adf (e.g., `cd ~`)
+2. Clone the ADF
+`git clone https://github.com/NCAR/ADF.git`
+3. Set your personal repository as the upstream repo
+```
+cd ADF
+git remote add upstream https://github.com/<git_user_name>/ADF.git
+```
+4. Switch to the clm-diags branch
+`git switch -c clm-diags origin/clm-diags`
+
+#### Set up your computing environment
+5. Create a conda environment. On NCAR's CISL machines (derecho and casper), these can be loaded by running the following on the command line:
+```
+module load conda
+conda env create -f env/ldf_v0.0.yaml
+conda activate ldf_v0.0
+```
+
+**Note** This is somewhat redundant, as it's a clone of cupid-analysis, but land diagnostics need the latest version of uxarray (25.3.0), and this will prevent overwriting your other conda environments.
+
+Also, along with these python requirements, the `ncrcat` NetCDF Operator (NCO) is also needed.  On the CISL machines this can be loaded by simply running the following on the command line:
+
+```
+module load nco
+```
+
+## Running ADF diagnostics
+
+Detailed instructions for users and developers are availabe on this repository's [wiki](https://github.com/NCAR/ADF/wiki). 
+
+You'll have to add your username to the appropriate config file, but after that, for a quick try of land diagnostics
+
+`./run_adf_diag config_clm_unstructured_plots.yaml`
+
+This should generate a collection of time series files, climatology (climo) files, re-gridded climo files, and example ADF diagnostic figures, all in their respective directories.
+
+**NOTE:** If you get NCO failures at the generate timeseries stage that end up causing LDF to fail, see issue [#365](https://github.com/NCAR/ADF/issues/365) 
+
+When additional memory is needed sometimes need to run interactive session on casper:
+`execcasper -A P93300041 -l select=1:ncpus=4:mem=64GB`
+
+## TEST for Land Diags:
+
+For this branch there are (3) ways to run the ADF:
+
+1) On native grid with unstructured plotting via Uxarray
+2) On native grid but gridded to lat/lon
+3) On already lat/lon gridded input files (hist, ts, or climo)
+
+For (1), the config yaml file will be essentially the same, but with a couple of additional arguments:
+  - in `diag_basic_info` set the `unstructured_plotting` argument to `true`
+  - in each of the test and baseline section supply a mesh file in the `mesh_file` argument
+
+  Example yaml file: `config_clm_unstructured_plots.yaml`
+
+For (2), the config yaml file will need some additional arguments:
+  - in each of the test and baseline sections, supply the following arguments:
+
+    Weights file:
+
+    `weights_file: /glade/work/wwieder/map_ne30pg3_to_fv0.9x1.25_scripgrids_conserve_nomask_c250108.nc`
+
+    Regridding method:
+
+    `regrid_method: 'coservative'`
+    (Yes, spelled incorectly for a bug in xESMF)
+
+    Lat/lon file:
+
+    `latlon_file: /glade/derecho/scratch/wwieder/ctsm5.3.018_SP_f09_t232_mask/run/ctsm5.3.018_SP_f09_t232_mask.clm2.h0.0001-01.nc`
+
+  NOTE: The regridding method set in `regrid_method` MUST match the method in the weights file
+
+  Example yaml file: `config_clm_native_grid_to_latlon.yaml`
+
+
+:

config_clm_BvsI.yml

@@ -0,0 +1,299 @@
+#==============================
+#config_cam_baseline_example.yaml
+
+#This is the main CAM diagnostics config file
+#for doing comparisons of a CAM run against
+#another CAM run, or a CAM baseline simulation.
+
+#Currently, if one is on NCAR's Casper or
+#Cheyenne machine, then only the diagnostic output
+#paths are needed, at least to perform a quick test
+#run (these are indicated with "MUST EDIT" comments).
+#Running these diagnostics on a different machine,
+#or with a different, non-example simulation, will
+#require additional modifications.
+#
+#Config file Keywords:
+#--------------------
+#
+#1.  Using ${xxx} will substitute that text with the
+#    variable referenced by xxx. For example:
+#
+#    cam_case_name: cool_run
+#    cam_climo_loc: /some/where/${cam_case_name}
+#
+#    will set "cam_climo_loc" in the diagnostics package to:
+#    /some/where/cool_run
+#
+#    Please note that currently this will only work if the
+#    variable only exists in one location in the file.
+#
+#2.  Using ${<top_level_section>.xxx} will do the same as
+#    keyword 1 above, but specifies which sub-section the
+#    variable is coming from, which is necessary for variables
+#    that are repeated in different subsections.  For example:
+#
+#    diag_basic_info:
+#      cam_climo_loc:  /some/where/${diag_cam_climo.start_year}
+#
+#    diag_cam_climo:
+#      start_year: 1850
+#
+#    will set "cam_climo_loc" in the diagnostics package to:
+#    /some/where/1850
+#
+#Finally, please note that for both 1 and 2 the keywords must be lowercase.
+#This is because future developments will hopefully use other keywords
+#that are uppercase. Also please avoid using periods (".") in variable
+#names, as this will likely cause issues with the current file parsing
+#system.
+#--------------------
+#
+##==============================
+#
+# This file doesn't (yet) read environment variables, so the user must
+# set this themselves. It is also a good idea to search the doc for 'user'
+# to see what default paths are being set for output/working files.
+#
+# Note that the string 'USER-NAME-NOT-SET' is used in the jupyter script
+# to check for a failure to customize
+#
+user: wwieder #'USER-NAME-NOT-SET'
+
+#This first set of variables specify basic info used by all diagnostic runs:
+diag_basic_info:
+
+    #Does the user want plotting of unstructured (native) grid?
+    #If "false" or missing, then the ADF expects ALL cases to be on lat/lon grids:
+    unstructured_plotting: false
+
+    #Is this a model vs observations comparison?
+    #If "false" or missing, then a model-model comparison is assumed:
+    compare_obs: false
+
+    #Generate HTML website (assumed false if missing):
+    #Note:  The website files themselves will be located in the path
+    #specified by "cam_diag_plot_loc", under the "<diag_run>/website" subdirectory,
+    #where "<diag_run>" is the subdirectory created for this particular diagnostics run
+    #(usually "case_vs_obs_XXX" or "case_vs_baseline_XXX").
+    create_html: true
+
+    #Location of observational datasets:
+    #Note: this only matters if "compare_obs" is true and the path
+    #isn't specified in the variable defaults file.
+    obs_data_loc: /glade/campaign/cgd/amp/amwg/ADF_obs
+
+    #Location where re-gridded and interpolated CAM climatology files are stored:
+    cam_regrid_loc: /glade/derecho/scratch/${user}/ADF_LatLon/regrid
+
+    #Overwrite CAM re-gridded files?
+    #If false, or missing, then regridding will be skipped for regridded variables
+    #that already exist in "cam_regrid_loc":
+    cam_overwrite_regrid: false
+
+    #Location where diagnostic plots are stored:
+    cam_diag_plot_loc: /glade/derecho/scratch/${user}/ADF_LatLon/plots
+
+    #Location of ADF variable plotting defaults YAML file:
+    #If left blank or missing, ADF/lib/adf_variable_defaults.yaml will be used
+    #Uncomment and change path for custom variable defaults file
+    defaults_file: lib/ldf_variable_defaults.yaml
+
+    #Longitude line on which to center all lat/lon maps.
+    #If this config option is missing then the central
+    #longitude will default to 180 degrees E.
+    central_longitude: 0
+
+    #Number of processors on which to run the ADF.
+    #If this config variable isn't present then
+    #the ADF defaults to one processor.  Also, if
+    #you set it to "*" then it will default
+    #to all of the processors available on a
+    #single node/machine:
+    num_procs: 1
+
+    #If set to true, then redo all plots even if they already exist.
+    #If set to false, then if a plot is found it will be skipped:
+    redo_plot: true
+
+#This second set of variables provides info for the CAM simulation(s) being diagnosed:
+diag_cam_climo:
+
+    # History file list of strings to match
+    # eg. cam.h0 or ocn.pop.h.ecosys.nday1 or hist_str: [cam.h2,cam.h0]  
+    # Only affects timeseries as everything else uses the created timeseries 
+    # Default: 
+    hist_str: clm2.h0
+
+    #Calculate climatologies?
+    #If false, the climatology files will not be created:
+    calc_cam_climo: true
+
+    #Overwrite CAM climatology files?
+    #If false, or not prsent, then already existing climatology files will be skipped:
+    cam_overwrite_climo: false
+
+    #Location of CAM climatologies (to be created and then used by this script)
+    cam_climo_loc: /glade/derecho/scratch/${user}/ADF_LatLon/${diag_cam_climo.cam_case_name}/climo
+
+    #Name of CAM case (or CAM run name):
+    cam_case_name: b.e30_alpha06b.B1850C_LTso.ne30_t232_wgx3.143
+
+    #Case nickname
+    #NOTE: if nickname starts with '0' - nickname must be in quotes!
+    # ie '026a' as opposed to 026a
+    #If missing or left blank, will default to cam_case_name
+    case_nickname: '143'
+
+    #Location of CAM history (h0) files:
+    cam_hist_loc: /glade/derecho/scratch/hannay/archive/${diag_cam_climo.cam_case_name}/lnd/hist/
+
+    # SE to FV regridding options
+    # Leave these blank if not on the native grid
+    #-----------------------------
+    # Weights file:
+    weights_file: /glade/work/wwieder/map_ne30pg3_to_fv0.9x1.25_scripgrids_conserve_nomask_c250108.nc
+    # Regridding method:
+    regrid_method: 'coservative'
+    # Lat/lon file:
+    latlon_file: /glade/derecho/scratch/wwieder/ctsm5.3.018_SP_f09_t232_mask/run/ctsm5.3.018_SP_f09_t232_mask.clm2.h0.0001-01.nc
+
+    #model year when time series files should start:
+    #Note:  Leaving this entry blank will make time series
+    #       start at earliest available year.
+    start_year: 30
+
+    #model year when time series files should end:
+    #Note:  Leaving this entry blank will make time series
+    #       end at latest available year.
+    end_year: 40
+
+    #Do time series files exist?
+    #If True, then diagnostics assumes that model files are already time series.
+    #If False, or if simply not present, then diagnostics will attempt to create
+    #time series files from history (time-slice) files:
+    cam_ts_done: false
+
+    #Save interim time series files?
+    #WARNING:  This can take up a significant amount of space,
+       #          but will save processing time the next time
+    cam_ts_save: false
+
+    #Overwrite time series files, if found?
+    #If set to false, then time series creation will be skipped if files are found:
+    cam_overwrite_ts: false
+
+    #Location where time series files are (or will be) stored:
+    cam_ts_loc: /glade/derecho/scratch/${user}/ADF_LatLon/${diag_cam_climo.cam_case_name}/ts
+
+
+#This third set of variables provide info for the CAM baseline climatologies.
+#This only matters if "compare_obs" is false:
+diag_cam_baseline_climo:
+
+    # History file list of strings to match
+    # eg. cam.h0 or ocn.pop.h.ecosys.nday1 or hist_str: [cam.h2,cam.h0]  
+    # Only affects timeseries as everything else uses the created timeseries 
+    # Default: 
+    hist_str: clm2.h0
+
+    #Calculate cam baseline climatologies?
+    #If false, the climatology files will not be created:
+    calc_cam_climo: true
+
+    #Overwrite CAM climatology files?
+    #If false, or not present, then already existing climatology files will be skipped:
+    cam_overwrite_climo: false
+
+    #Name of CAM baseline case:
+    cam_case_name: ctsm53019_f09_BNF_hist
+
+    #Baseline case nickname
+    #NOTE: if nickname starts with '0' - nickname must be in quotes!
+    # ie '026a' as opposed to 026a
+    #If missing or left blank, will default to cam_case_name
+    case_nickname: '5.3.019_BNF'
+
+    #Location of CAM baseline history (h0) files:
+    #Example test files
+    cam_hist_loc: /glade/derecho/scratch/slevis/archive/${diag_cam_baseline_climo.cam_case_name}/lnd/hist/
+
+    #Location of CAM climatologies (to be created and then used by this script)
+    cam_climo_loc: /glade/derecho/scratch/${user}/ADF_LatLon/${diag_cam_baseline_climo.cam_case_name}/climo
+
+    #model year when time series files should start:
+    #Note:  Leaving this entry blank will make time series
+    #       start at earliest available year.
+    start_year: 1850
+
+    #model year when time series files should end:
+    #Note:  Leaving this entry blank will make time series
+    #       end at latest available year.
+    end_year: 1860
+
+    #Do time series files need to be generated?
+    #If True, then diagnostics assumes that model files are already time series.
+    #If False, or if simply not present, then diagnostics will attempt to create
+    #time series files from history (time-slice) files:
+    cam_ts_done: false
+
+    #Save interim time series files for baseline run?
+    #WARNING:  This can take up a significant amount of space:
+    cam_ts_save: true
+
+    #Overwrite baseline time series files, if found?
+    #If set to false, then time series creation will be skipped if files are found:
+    cam_overwrite_ts: false
+
+    #Location where time series files are (or will be) stored:
+    cam_ts_loc: /glade/derecho/scratch/${user}/ADF_LatLon/${diag_cam_baseline_climo.cam_case_name}/ts
+
+#+++++++++++++++++++++++++++++++++++++++++++++++++++
+#These variables below only matter if you are using
+#a non-standard method, or are adding your own
+#diagnostic scripts.
+#+++++++++++++++++++++++++++++++++++++++++++++++++++
+
+#Note:  If you want to pass arguments to a particular script, you can
+#do it like so (using the "averaging_example" script in this case):
+# - {create_climo_files: {kwargs: {clobber: true}}}
+
+#Name of time-averaging scripts being used to generate climatologies.
+#These scripts must be located in "scripts/averaging":
+time_averaging_scripts:
+    - create_climo_files
+
+#Name of regridding scripts being used.
+#These scripts must be located in "scripts/regridding":
+regridding_scripts:
+    - regrid_and_vert_interp
+
+#List of analysis scripts being used.
+#These scripts must be located in "scripts/analysis":
+analysis_scripts:
+    - lmwg_table
+
+#List of plotting scripts being used.
+#These scripts must be located in "scripts/plotting":
+plotting_scripts:
+    - global_latlon_map
+    - global_mean_timeseries_lnd
+    - polar_map
+
+#List of CAM variables that will be processesd:
+#If CVDP is to be run PSL, TREFHT, TS and PRECT (or PRECC and PRECL) should be listed
+diag_var_list:
+    - TSA
+    - PREC
+    - ELAI
+    - GPP
+    - NPP
+    - FSDS
+    - ALTMAX
+    - ET
+    - GRAINC_TO_FOOD
+    - TOTRUNOFF
+    - DSTFLXT
+    - MEG_isoprene 
+
+#END OF FILE