Skip to content

Restore parallel makeMovie execution and add deterministic Dask token for containers#12

Merged
haykh merged 3 commits into1.4.0rcfrom
codex/fix-serialization-issue-in-makemovie
Feb 10, 2026
Merged

Restore parallel makeMovie execution and add deterministic Dask token for containers#12
haykh merged 3 commits into1.4.0rcfrom
codex/fix-serialization-issue-in-makemovie

Conversation

@haykh
Copy link
Contributor

@haykh haykh commented Feb 10, 2026

Motivation

  • Ensure makeFrames continues to run via the executor (parallel) even when a data object is provided, as requested to avoid forcing a serial fallback while preserving laziness/xarray structures.
  • Fix nondeterministic hashing/tokenization failures when container objects are captured by plotting workflows so parallel workers can reliably accept container handles.

Description

  • Removed the in-process serial fallback so makeFrames submits all frames through the executor path and retained safe directory creation with os.makedirs(fpath, exist_ok=True) in nt2/plotters/export.py.
  • Added BaseContainer.__dask_tokenize__ to nt2/containers/container.py to return a deterministic token tuple of (class name, path, reader.format.value) for stable hashing by Dask.
  • Replaced/added tests: nt2/tests/test_export.py now asserts executor dispatch is used with data by injecting a fake executor, and nt2/tests/test_tokenize.py asserts deterministic container tokenization via dask.base.tokenize.

Testing

  • Ran python -m compileall -q nt2, which succeeded.
  • Ran pytest -q nt2/tests/test_export.py nt2/tests/test_tokenize.py, which failed before tests executed due to a ModuleNotFoundError for numpy during package import.
  • Attempted python -m pip install numpy -q to unblock tests, but installation is blocked in this environment due to restricted network/package index access.

Codex Task

@haykh haykh merged commit f824aa5 into 1.4.0rc Feb 10, 2026
1 check failed
@haykh haykh deleted the codex/fix-serialization-issue-in-makemovie branch February 10, 2026 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant