feat(weather): tier-2 VIIRS products — AOD, CloudHeight, LST, SnowCover#104
Merged
Conversation
…, SnowCover Live-verified against real n20 granules (2026-07-06; per-satellite year availability probed across all three NODD buckets): - VIIRS-JRR-AOD / AOD550 — aerosol optical depth, (768,3200) M-band, fill -999.999, archive snpp/n20 2022+ n21 2023+. Smoke/haze days measurably suppress Tmax. - VIIRS-JRR-CloudHeight / CldTopHght+CldTopTemp+CldTopPres — float32 (768,3200), fill -999, ~67 MB granules (well under the 1 GiB cap), archive 2023+. Deep-convection vs stratus discrimination. - JPSSRR_LST / LST — land surface temperature swath, SCALED int16 (raw*0.005+200 -> Kelvin), validated end-to-end on a real granule (285-323 K over the tropical Pacific — physically exact). Polar-orbit LST covers the EU/Asia stations GOES cannot see. Archive ~Dec 2024+ (young; forward value). Filename prefix is LST_v2r2 (not JRR-*) — cosmetic, the transport lists day prefixes. - JPSSRR_SnowCover / SnowFraction+SnowMask — I-BAND grid (1536,6400), DOUBLE the M-band resolution of every other VIIRS product (per-entry registry shapes handle it). Archive ~Dec 2024+. Snow albedo suppresses Tmax; snowpack enables radiative-cooling cold NLOW nights. Floors restructured: the 2026-07-05 per-satellite floors were CloudMask product dates masquerading as platform floors — correct while CloudMask was the only product, but they would have silently clipped AOD's 2022+ archive on snpp/n20. Platform floors are now earliest-any-product; _PRODUCT_AVAILABLE_SINCE carries per-(satellite, product) specifics (CloudMask keeps its exact verified dates); the clamp uses _available_since_for() = max(platform, product), mirroring _goes_s3 (PR #103). Also syncs the no-extra VIIRS fallback product set + extends the AST drift guard to it (same class as the Codex P2 on the DSRC registration). Transport untouched: the flat {product}/{YYYY}/{MM}/{DD}/ listing is product-agnostic and all new granules fit the existing 1 GiB cap. SKIPPED with rationale: ISatSS (display imagery), DSRM (roaming mesoscale sector), SDR/RDR instrument products, L1b radiances. Himawari AHI-L2-FLDK-Winds deferred to its own PR: channel-partitioned multi-file scans under an extra path level ({YYYY}/{MM}/{DD}/{HHMM}/) need transport + a vector-extraction mode, not a registry entry. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codex P2 on the tier-2 VIIRS registrations: JPSSRR_LST / JPSSRR_SnowCover contain "_" — the progress key's FIELD SEPARATOR — so a resume run would write viirs-n20_JPSSRR_LST_... and then raise ProgressCorrupt loading its OWN file (the validator's product segment is deliberately underscore-free). _progress_key() now maps "_" -> "-" in key-space only: keys are opaque full-string identities (never parsed back), build and skip-check share the function, and no registered products differ only by underscore-vs-hyphen, so the transform is loss-free. Cache paths and scope digests keep the raw product names. Tests: sanitized key validates + round-trips _validate_progress; key identity stable for skip-checks and distinct across sibling products. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codex P2 (round 3, tier-2 branch): the VIIRS extractor applied only _FillValue + scale/offset — never the CF-1.8 raw-unit valid_range the GOES path enforces. With JPSSRR_LST (the first SCALED VIIRS product), an invalid raw like 30000 (> valid_range max 28600) scaled into a plausible-looking 350 K inside the broad registry bounds and entered rows as CLEAN data. _apply_scale_offset gains valid_range_filter (default False), masking raw values outside the attr range to NaN BEFORE scaling — mirroring _goes_extract._apply_valid_range semantics exactly, including the physical-units exemption (AOD550's valid_range is already physical, so its entry keeps the filter off, like GOES Cloud_Probabilities). The extract loop wires pv.valid_range_filter through; LST's entry uses the dataclass default True. Tests: out-of-range raw masks to NaN while in-range scales (float32 scale-precision tolerance), disabled filter passes physical-range products untouched, LST registry entry pins the filter on. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
📝 Docs-required check: REMINDER API-surface change without docs and no opt-out — surfacing reminder. API-surface files changed: Docs files changed: Docs surfaces to consider
How to silence this reminder
This check is advisory — it never blocks the merge. |
|
Parity ticket gate: PASSED See |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
"Backfill fully" tier-2: the VIIRS buckets carry far more than CloudMask. This registers the four highest-value families for temperature markets — including polar-orbit land-surface temperature, the only satellite LST available for EU/Asia stations (GOES can't see them).
What (3 commits, all live-verified on real n20 granules)
_PRODUCT_AVAILABLE_SINCEcarries specifics; effective = max (mirrors fix(weather): constellation product parity + probe-measured ingest speed #103's GOES pattern). No-extra fallback synced + AST drift guard extended.Review loop (codex ×4)
JPSSRR_*underscores collide with the progress-key field separator → resume wouldProgressCorruptits own file → keys sanitize_→-in key-space (opaque identities, loss-free)valid_range→ invalid LST raw 30000 scaled to a plausible 350 K as clean →_apply_scale_offsetnow mirrors_goes_extract._apply_valid_rangeexactly, incl. the physical-units exemptionDeferred with rationale: Himawari Winds (channel-partitioned multi-file layout → own transport PR), ISatSS (display imagery), DSRM (roaming sector).
After merge
Image rebuild (
main-<merge>) → VIIRS top-up dispatches for KNYC + RJTT + EGLC (polar = all three gain LST/AOD/cloud-props/snow).TS Parity
python_only: true — satellite ingest internals; no public API surface.
🤖 Generated with Claude Code