fix: research_by_source window trim + wire research(source=) pin — dual-SDK, v1.12.0#108
Merged
Merged
Conversation
mode2.research_by_source returned whole calendar months (the month-granular parquet cache flows through _fetch_observations_range unfiltered): a KNYC Jan 6-12 query returned all 865 January rows; a Jan 28-Feb 3 query returned two full months. Silent out-of-window leakage into backtest frames. Rows are now trimmed to [from_date, to_date] by UTC date-part, inclusive, matching the TS port's shipped filter (packages-ts/meta/src/mode2.ts) for cross-SDK lockstep. The +1-day fetch extension remains fetch-width only. Fixes every return shape (DataFrame, as_dataframe=False, wrapper/backends). TS side adds the missing window-boundary tests for iem.archive + awc.live. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The singular source= kwarg is now functional (was NotImplementedError since Phase 10): validated pre-fetch against the Mode 2 vocabulary (7 names, byte-identical alias tables across SDKs — Python _SOURCE_ALIASES, TS RESEARCH_SOURCE_ALIASES), then applied as a post-merge row subset before daily obs_* aggregation. cli_* settlement columns unaffected; pinned DataFrames carry df.attrs["source"]. sources=[...] (plural subset) stays v0.3 with a corrected error message. source=None default path is byte-identical (parity gate green). TS source: null follows the Phase 21 null-as-absent wire contract (codex P2, regression-tested). Inherited v0.1 limitation documented: the AWC > IEM > GHCNh merge runs before the pin filter, so pinning a lower-priority source yields only the rows the merge did not supersede. Review: codex GATE PASS (1 P2, fixed) + architect merge-ready (no P1/P2). P3 follow-ups: #107 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… 26 KB All Python dists (mostlyrightmd, -weather, -markets) and TS packages (@mostlyrightmd/core|weather|markets, mostlyright meta) 1.11.0 -> 1.12.0 in lockstep. Weather TS size-limit 25 -> 26 KB: the bundle was already 25.88 KB on main (877 B over, pre-existing — this PR touches only meta); growth investigation tracked in #107. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
✅ Docs-required check: PASS API-surface change includes docs updates — no reminder needed. API-surface files changed: Docs files changed: |
|
Parity ticket gate: PASSED See |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two coupled fixes shipped in lockstep across both SDKs, plus the 1.12.0 release bump:
mode2.research_by_sourcewindow leak (bug). It returned whole calendar months instead of the requested window — a KNYC Jan 6–12 query returned all 865 January rows; Jan 28–Feb 3 returned two full months. Triple-verified empirically (fresh-cache live run, sockets-blocked seeded-cache run, static path audit; 6-agent adversarial pass found zero evidence the behavior was intended — the TS port already shipped the correct strict filter). Rows are now trimmed to[from_date, to_date]by UTC date-part, inclusive, matching TS exactly. Every return shape fixed.research(source=...)single-source pin (feature). Functional in both SDKs (wasNotImplementedError): pre-fetch validation against the 7-name Mode 2 vocabulary, post-merge alias-set filter before daily aggregation,cli_*untouched,df.attrs["source"]provenance,sources=(plural) stays v0.3 with a corrected message.TS Parity
Identical API, shipped in this same PR (no parity ticket needed).
packages-ts/meta:research({source})wired with byte-identical vocabulary/alias tables (RESEARCH_SOURCE_ALIASES≡ Python_SOURCE_ALIASES, verified entry-by-entry), same pre-fetch error shape, same post-merge filter placement;source: nullfollows the Phase 21 null-as-absent wire contract. mode2 window-boundary tests added foriem.archive+awc.live.Parity firewall
source=Nonedefault path is byte-identical: every new statement is gated behindsource is not None; full fast suite incl.tests/test_parity.pygreen (×3 runs: implementer, architect, pre-push hook); goldenassert_frame_equalno-op test included. No merge-logic files touched.Review loop (two-reviewer discipline)
codex review, high effort): GATE PASS — 1 P2 (TSsource: nulltreated as present, violating the Phase 21present()wire contract) → fixed in this PR + regression test.observed_atstring-schema safety. 6 P3 nits deferred.Deferred follow-ups (P3s + stale
__version__+ weather bundle growth): #107Release
All 7 packages bumped 1.11.0 → 1.12.0 in lockstep (3 PyPI dists, 4 npm packages). Weather TS size-limit 25 → 26 KB — bundle was already 25.88 KB on main (pre-existing growth, this PR touches only meta; tracked in #107).
pnpm run sizegreen. Post-merge: tagv1.12.0(PyPI trusted publishing) +vts-1.12.0(npm OIDC).Gates
tsc --noEmit✅,pnpm run build✅, biome ✅ (no new)🤖 Generated with Claude Code