README.notes

**  2025-05-13 15:49:51 by Ashley Pera <ashley.pera@noaa.gov> **

  Downloaded 2024 data, symlinked older data. Older data hashes don't match.

**  Thu 14 Nov 2024 02:40:38 PM UTC by Andy Jacobson <andy.jacobson@noaa.gov> **

  Note that we use daily_1x1 (created by ingest_byyear.r) for
  fire/fuel inputs, and the monthly_1x1 -> ERA5 pipeline via
  diurnalize.r for GPP and Resp.

**  Mon 01 Apr 2024 23:07:32 UTC by Andy Jacobson <andy.jacobson@noaa.gov> **

  Use fire (and fuel) emissions from daily files, but use monthly
  files to create downscaled NEE.



**  Fri 05 Apr 2024 14:33:02 UTC by Andy Jacobson <andy.jacobson@noaa.gov> **

  Need to rerun diurnalize for 2023 and 2024 when ERA5 meteo becomes
  available.



**  Sun 19 May 2024 23:15:21 UTC by Andy Jacobson <andy.jacobson@noaa.gov> **

  v1 pre-release daily and monthly downloaded. Note change of file
  names,

  MiCASAv1_flux.x3600_y1800.monthly.%d%02d.nc to
  MiCASA_v1_flux_x3600_y1800_monthly_%d%02d.nc

  Re-ran ingest-monthly.r

  Created check_daily_downloads.r since there were connection-refused
  error messages. All files apparently present, can proceed with
  ingest_daily.r.
  

**  Mon 20 May 2024 13:07:16 UTC by Andy Jacobson <andy.jacobson@noaa.gov> **

  Need to re-run 2023 and 2024 when we have meteo past 20231031.


  

**  Thu 13 Feb 2025 09:40:40 PM UTC by Andy Jacobson <andy.jacobson@noaa.gov> **

  Bug in 2018-01 has caused all kinds of unexpected trouble. At some
  point, Brad Weir changed the units of fluxes in daily and monthly
  files from Kg C m-2 day-1 to gC m-2 s-1. This was not noticed, and
  needed to be corrected in the ingest_byyear.r (for fire and fuel)
  and ingest_monthly.r (for NPP and Rh, thereafter PIQS, thereafter
  diurnalize).

  Just caught the monthly file units change today, fixing, and will
  rewind the p8 CT2024 runs.

  Monthly procedure:
  ingest_monthly.r
  cat_monthly.sh (new)
  compute_clim.sh
  write_piqs.r
  Rscript diurnalize-ERA5.r
  daysplitter.sh
  
**  2026-04-26 by Ash Pera <ashley.pera@colorado.edu> **

  Latent-bug sweep after the Tier-1 refactor. Six fixes; the headline
  is a numerical bug in the area-weighted aggregator
  (lib/ingest_common.r:aggregate.to.1x1) where lat-area weights were
  being recycled along the longitude axis instead of latitude, AND the
  inner per-longitude loop was dead. Fix builds an explicit length-100
  weight vector that mirrors weighted.mean's column-major flatten of
  the 10x10 sub-block. Added lib/test_aggregate.r as a self-contained
  regression test (passes; the old buggy formula is verified to fail
  on a sin(lat) test field).

  Other fixes: run_year.sh trailing comma in --export, write_piqs.r
  hardcoded v1 filename, check_hashes.py 202[4-5] year glob,
  link_old_micasa_raw.sh layout-detection (handles both 2024 from_weir
  and 2025+ flat layouts), check_unchanged.sh self-blessing reference
  chain. See README.ash "Latent-bug sweep" section for the full list.

  Also documented the v1+vNRT hybrid stream model: download.sh now
  supports MICASA_VERSION=both, and link_vNRT_to_v1.sh is the post-
  ingest step that exposes vNRT-tagged outputs as v1 for downstream
  consumers, while v1 (when present) always wins via -e checks.

  MICASA_CLIM_YEARS default changed from "2000 ${MICASA_YEAR}" to
  "2000 $(date +%Y)" so backfilling an earlier year doesn't
  accidentally clim a fully-published year.