Roadmap

Description:

This page outlines the development principles & roadmap for the OceanDataStore library.

OceanDataStore CLI is designed to write and update archival files on local disk to Analysis Ready Cloud Optimised (ARCO) formats, such as Zarr stores and Icechunk repositories, in S3 compatible cloud object storage.
OceanDataStore also includes the functionality to produce Spatio-Temporal Access Catalogs to describe ocean model and observational data.
OceanDataCatalog is a Python API included in the OceanDataStore library enabling users to search, subset and access ocean model and observational datasets stored in cloud object storage.

OceanDataStore does not support cataloguing and accessing data via OpenDAP, THREDDS servers or third-party APIs (e.g., Copernicus Marine Service).
OceanDataStore is not a catalog of all ocean datasets produced by the National Oceanography Centre (i.e., not a general registry).
OceanDataCatalog is a Python API and is not interoperable with other programming languages (e.g., R or MATLAB).

Update OceanDataStore CLI to use Typer for command-line interface [Medium].
Add send_to_virtual_icechunk and update_virtual_icechunk functions to the CLI for virtualising archival netCDF files stored in S3-compatible object stores using VirtualiZarr and Icechunk [High].
Support for Dask Job Queues in addition to LocalClusters [Low].

Formal definition of NOC STAC schema (i.e., metadata requirements) [High].
Automation of NOC STAC catalog creation using GitHub Actions [High].
Refactor & improve automation of non-model-stac catalog creation & validation, enabling easy updates from Ocean Modellers via Pull Requests (PRs) [Medium].

Writing ocean observations and analyses to JASMIN Cloud Object Store, and including scripts in OceanDataStore repo [Medium].
Create new non-obs-stac for ocean observations and analyses, using automation pipelines [Medium].
Integrating into NOC STAC catalog alongside ocean model outputs [Medium].