iddata refactoring#40
Open
matthewcornell wants to merge 7 commits into
Open
Conversation
lshandross
requested changes
May 4, 2026
Contributor
lshandross
left a comment
lshandross
left a comment
There was a problem hiding this comment.
I put all my comments on the code in this review. We can go over them together and elaborate/resolve as we see fit, then perhaps ask Claude to address the remaining ones
matthewcornell
commented
May 5, 2026
- Move _load_us_census() from nhsn.py to ancillary/population.py, its natural home - Fix PopulationData.load() to return season-indexed (location, season, pop, log_pop) instead of most-recent-only; DiseaseDataLoader now merges ancillary by ["location", "season"] when the ancillary has a season column - Restore original two-error logic in NHSNDataSource.load() for as_of < 2024-11-15 (separate errors for drop_pandemic_seasons=False and disease != FLU); add else branch for _load_from_nhsn; extract shared postprocessing into _postprocess() - Add load_fips_mappings() helper to utils.py; update all sources to use it - ILINet: move source-specific scaling to load() (before _aggregate_to_fips), set source before aggregation; replace deprecated groupby.apply(pd.DataFrame) with .agg(); add Region 10 comment - FluSurvNet: fix _load_base age_labels default to ["Overall"]; inline _load_us_census_for_flusurv(); fix deprecated fillna(method="ffill") → .ffill(); fix deprecated W-sat → W-SAT; replace .apply(pd.DataFrame) with .agg() - NSSP: fix error message to use "NSSPDataSource"; move source assignment before _fill_missing_states; fix .apply(pd.DataFrame) with .agg() in _fill_missing_states - Improve base.py DataSource docstring (location can be HSA NCI ID, agg_level examples, season/season_week wording) - Remove unused S3_BUCKET from constants.py - Bump version to 0.2.0 - Tests: remove stale transform columns from mock source df; add NSSPDataSource custom-disease test; add NSSP case to test_load_data_sources; simplify FluSurvNet parametrize (no spurious conditional) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Only called once; no other sources use this split pattern. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Member
Author
|
DEC: Bump version to "2.0.0" |
lshandross
approved these changes
May 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
implementation of design4.md