feat: add built-in frozendict support for Python 3.15+#274
Open
XuehaiPan wants to merge 24 commits into
Open
Conversation
XuehaiPan
added
enhancement
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #274 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 15 15
Lines 1582 1614 +32
Branches 210 218 +8
=========================================
+ Hits 1582 1614 +32 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. |
XuehaiPan
force-pushed
the
frozendict
branch
4 times, most recently
from
3426921 to
12a3276
Compare
XuehaiPan
force-pushed
the
frozendict
branch
4 times, most recently
from
fe42e10 to
0faccef
Compare
XuehaiPan
force-pushed
the
frozendict
branch
5 times, most recently
from
19952e3 to
7656326
Compare
Register `frozendict` (Python 3.15.0a7+) as a built-in PyTree node type with key-sorted traversal, matching the behavior of `dict` and `collections.defaultdict`. The `dict_insertion_ordered` context manager also affects `frozendict`. Changes across 29 files: - C++ enum `PyTreeKind::FrozenDict` and `PyFrozenDictTypeObject` macro (guarded by `PY_VERSION_HEX >= 0x030F00A7`) - All treespec switch statements updated (flatten, unflatten, serialize, hash, compare, traverse, construct, paths, accessors, entries) - Python registry with sorted and insertion-ordered flatten/unflatten - `treespec_frozendict()` constructor, `STANDARD_DICT_TYPES` expansion, `PyTree[T]` type union update - `optree.treespec.frozendict` namespace alias - README, Sphinx docs, and spelling wordlist - Test data extended (3 frozendict entries in TREES fixture), subinterpreter test tree includes frozendict, registry size checks - Minor: empty deque repr improved (`deque()` instead of `deque([])`)
Sphinx autodoc/autosummary imports symbols during the reading phase regardless of ``.. only::`` directives, causing import errors on Python < 3.15 where ``treespec_frozendict`` does not exist. Remove the entries for now with TODO comments to add them back when building with Python 3.15+.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 33 out of 34 changed files in this pull request and generated 14 comments.
Comments suppressed due to low confidence (2)
tests/test_prefix_errors.py:1
- The dots in
collections.OrderedDictandcollections.defaultdictare unescaped regex metacharacters. They will match any character. Escape them ascollections\.OrderedDict/collections\.defaultdictto avoid accidental matches and for consistency with the escaping used elsewhere in this file.
tests/helpers.py:1 - Hard-coding
9 if … else 8and repeating the same expression intests/concurrent/test_subinterpreters.py(twice) couples three test sites to the same magic constant. Consider exposing the expected size fromhelpers.py(e.g.,EXPECTED_REGISTRY_SIZE) and importing it where needed, so future builtin additions only need to update one place.
…all time Address Copilot PR review on `docs/source/ops.rst` and `docs/source/treespec.rst`: the previous `TODO(frozendict): Add ... when building docs with Python 3.15+` comments meant the new API was perpetually undocumented even on toolchains that could support it, and conditional Sphinx directives (`only::`, `ifconfig::`) don't actually suppress the autodoc/autosummary import warnings emitted when the symbol is missing — those extensions resolve symbols at directive-parse / pre-scan time, before any rendering-side filter applies. Move the version gating from definition-time to call-time so the symbol always exists. `autodoc` and `autosummary` can introspect it on every supported Python; the docstring's `.. note::` block explicitly states the runtime requirement. Calling the function on a build without `frozendict` support raises a clear `RuntimeError` naming the version requirement, rather than the previous `AttributeError` users got when the symbol was conditionally defined. API changes: - `optree.treespec_frozendict` is now defined unconditionally (always callable, always in `optree.__all__`), with a runtime guard that raises `RuntimeError` if `_C.OPTREE_HAS_FROZENDICT` is unset. - `optree.treespec.frozendict` mirrors the same: unconditional re-export of `treespec_frozendict`, always in `optree.treespec.__all__`. - Dropped now-unused `import sys` / `import optree._C as _C` / `del sys` from `optree/__init__.py` and `optree/treespec.py`. - Moved `import builtins` from `TYPE_CHECKING` to module-top in `optree/ops.py` since it's accessed at runtime via `builtins.frozendict`. Docs: - `docs/source/ops.rst`: replace the TODO with the actual `.. autofunction:: treespec_frozendict` directive and add the symbol to the surrounding autosummary block. - `docs/source/treespec.rst`: replace the TODO with a `frozendict` entry in the autosummary list. - Result: `make docs` is warning-free on every supported Python — the symbol is always importable so `autodoc`/`autosummary` succeed. Docstring polish: rewrite the opening line of every `treespec_*` constructor's docstring to the consistent form `Make a treespec representing a :class:`X` node from child treespecs.` and use `:class:` markup uniformly in the `Returns:` block (tuple, list, dict, OrderedDict, defaultdict, deque). Tests: - New `tests/test_treespec.py::test_treespec_frozendict_runtime_error_on_unsupported_interpreter`: on Python < 3.15 (or wheels without `frozendict` support), both `optree.treespec_frozendict()` and `optree.treespec.frozendict(...)` raise a `RuntimeError` whose message contains the exact version requirement. Drive-by: add `# pylint: disable=redefined-builtin` to two `sentinel` references in `tree_replace_nones` (parameter) and `tree_broadcast_common` (local). Python 3.15 added `builtins.sentinel` (PEP 661), which makes pylint flag prior uses of the name; surfaced now that the lint venv runs on 3.15t.
XuehaiPan
force-pushed
the
frozendict
branch
3 times, most recently
from
0795f05 to
2792b40
Compare
PyPy/Windows jobs intermittently fail in pip steps with transient network errors (e.g. `OSError: Connection broken: IncompleteRead`) while a nested build-isolation pip downloads build dependencies such as `scikit-build-core` for nightly `pybind11`. Set `PIP_RETRIES=10` (default 5) and `PIP_TIMEOUT=120` (default 15s) in the `env:` block of all test/build/lint workflows. As environment variables these propagate into nested build-isolation pip subprocesses (unlike CLI flags on the outer `pip install`), covering connection setup, DNS, and metadata-fetch hiccups that account for most CI pip failures. Also add both vars to cibuildwheel's `environment-pass` so they cross into the isolated (manylinux) wheel-build containers, which would otherwise not inherit the host environment. Note: a mid-download `IncompleteRead` can still escape pip's internal retry; a command-level retry would be the next lever if it recurs.
The pyenv-win bundled `.versions_cache.xml` lags behind python.org, and the `pyenv update` command is unusable on GitHub runners: its VBScript updater parses release listings via the Internet Explorer `htmlfile` COM object, which modern Windows images reject with "This command is not supported". Run the pyenv-win stdlib updater (`.github/scripts/update_versions_cache.py`) right after cloning so `pyenv install` can resolve the newest CPython release (3.15.0b3), which ships the gh-149786 fix for the free-threaded venv launcher that previously broke the 3.15 free-threaded debug build venv/ensurepip.
…urface Extend three existing tests with `frozendict` cases (gated on Python 3.15+) to exercise paths that were previously untested: - `test_tree_broadcast_common`: the `PyTreeKind::FrozenDict` arm in `BroadcastToCommonSuffix`. - `test_different_metadata`: the `frozendict:` key-mismatch message emitted by `FlattenUpTo`. - `test_pytree_typing`: the `optree.typing.FrozenDict` alias, its `__all__` position, and the `frozendict[Any, ...]` member of the `PyTree` generic union.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Register
frozendict(PEP 814, Python 3.15.0a7+) as a built-in PyTree node type with key-sorted traversal, matching the behavior ofdictandcollections.defaultdict. Thedict_insertion_orderedcontext manager also affectsfrozendict.Also improves the empty deque representation:
deque()instead ofdeque([]).Motivation and Context
Python 3.15 introduces
frozendictas a built-in immutable mapping type (PEP 814). As a fundamental container type, it should be supported as a built-in PyTree node — just likedict,OrderedDict, anddefaultdict.Types of changes
Implemented Tasks
PyTreeKind::FrozenDictenum andPyFrozenDictTypeObjectmacro (guarded byPY_VERSION_HEX >= 0x030F00A7)treespec_frozendict()constructor,STANDARD_DICT_TYPESexpansion,PyTree[T]type unionoptree.treespec.frozendictnamespace aliasChecklist
make format. (required)make lint. (required)make testpass. (required)