Skip to content

Conversation

@melonora
Copy link

@melonora melonora commented Dec 3, 2025

[Description of PR]
This PR adds a convenience method for copying the store of a Group to a destination store.

TODO:

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/user-guide/*.md
  • Changes documented as a new file in changes/
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

@d-v-b

@github-actions github-actions bot added the needs release notes Automatically applied to PRs which haven't added release notes label Dec 3, 2025
@codecov
Copy link

codecov bot commented Dec 3, 2025

Codecov Report

❌ Patch coverage is 91.30435% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.21%. Comparing base (ee0e69a) to head (2cbb9b9).

Files with missing lines Patch % Lines
src/zarr/core/group.py 90.90% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3612      +/-   ##
==========================================
+ Coverage   59.14%   59.21%   +0.06%     
==========================================
  Files          86       86              
  Lines       10172    10194      +22     
==========================================
+ Hits         6016     6036      +20     
- Misses       4156     4158       +2     
Files with missing lines Coverage Δ
src/zarr/testing/strategies.py 97.82% <100.00%> (ø)
src/zarr/core/group.py 70.75% <90.90%> (+0.48%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@melonora
Copy link
Author

melonora commented Dec 3, 2025

@d-v-b I know right now this won't work when zarr v2 is used. I can add that functionality as well or we say the convenience function is not supported for zarr v2.

@melonora
Copy link
Author

melonora commented Dec 3, 2025

added zarr v2 support. If in general the code is approved, I will add documentation.

store: StoreLike,
*,
overwrite: bool = False,
consolidate_metadata: bool | None = None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this here? IMO it's simpler if copying is just copying. If people want to add consolidated metadata, they should do that after copying.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adjusted it. It now consolidates if the source store was consolidated, but this does not require an argument. One more thing, I only take into account when consolidation happened at the root level. It could be (although the user should not do this) that consolidation happened at the subgroup level. This could be accounted for by checking if member. metadata.consolidated_metadata when looping through members. WDYT should I add this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would copy the original group and its children exactly, including their consolidated metadata.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would be the purpose of copying over stale consolidated metadata?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it keeps the scope of this function simple -- it just copies the original group exactly. If this tool produces a different group, then it's not really copying.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

regarding the from_store method, I would open a separate PR for this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consolidated metadata is already in the group metadata document for zarr v3 if "kind" is set to "inline".

👍 but doing something other (more or less) than just copying would require interpreting that which is too far IMO.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @joshmoore the method under discussion here is a method on the Group class, not on stores. maybe this is a point of confusion.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO the only consolidated metadata-related parameter we need is whether iterating over the child groups uses their consolidated metadata (if present). See the relevant kwarg in the signature of members.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added both, only thing remaining is probably adding consolidated_metadata in the async group call.

@github-actions github-actions bot removed the needs release notes Automatically applied to PRs which haven't added release notes label Dec 5, 2025
@melonora
Copy link
Author

melonora commented Dec 5, 2025

@d-v-b @joshmoore I think I have now addressed all comments and I also added documentation, so I think this would be ready for a final round of review. @d-v-b thanks for your patience.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants