Skip to content

Comments

Data registry and caching#3

Draft
karimbahgat wants to merge 19 commits intomainfrom
data-registry-and-caching
Draft

Data registry and caching#3
karimbahgat wants to merge 19 commits intomainfrom
data-registry-and-caching

Conversation

@karimbahgat
Copy link
Collaborator

@karimbahgat karimbahgat commented Feb 24, 2026

  • datasets route which handles data registry via yaml files and via dhis2eo can trigger cache builds from api as well (though maybe not ideal for prod). Everything is generic and nothing is hardcoded for each data source, so it's easy to add new sources as long as they are supported by dhis2eo (in future should be possible to support non-dhis2eo sources too).
  • also includes a test concept of aggregate route for requesting aggregates from cached datasets and custom aggregation statistics.

Quick and dirty, many improvements needed.

Karim Bahgat added 19 commits February 23, 2026 13:02
…al code so we always work with an xarray dataset, dask compute after time aggregation to avoid very slow dask graphs later
…s zarr file if it exists which is much faster
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant