From ae8f0d308b971f602b160307013bf0dbfd19808b Mon Sep 17 00:00:00 2001 From: "Michael J. Sullivan" Date: Fri, 23 Jan 2026 15:08:54 -0800 Subject: [PATCH] Churning on the PEP, add a script for building it --- pre-pep.rst | 275 +++++++++++++++++++++++++++++++++++++++--- scripts/build-peps.sh | 15 +++ spec-draft.rst | 144 +--------------------- 3 files changed, 277 insertions(+), 157 deletions(-) create mode 100755 scripts/build-peps.sh diff --git a/pre-pep.rst b/pre-pep.rst index 64acac2..2f442b5 100644 --- a/pre-pep.rst +++ b/pre-pep.rst @@ -1,13 +1,13 @@ -PEP: -Title: Type-level Computation -Author: Michael J. Sullivan , Daniel Park , Yury Selivanov +PEP: 9999 +Title: Type Manipulation! +Author: Michael J. Sullivan , Daniel W. Park , Yury Selivanov Sponsor: PEP-Delegate: Discussions-To: Pending -Status: DRAFT +Status: Draft Type: Standards Track Topic: Typing -Requires: +Requires: 0000 Created: Python-Version: 3.15 or 3.16 Post-History: Pending @@ -19,15 +19,17 @@ Abstract We propose to add powerful type-level type introspection and type construction facilities to the type system, inspired in large part by -TypeScript's conditional and mapping types, but adapted to the quite +TypeScript's conditional and mapped types, but adapted to the quite different conditions of Python typing. Motivation ========== Python has a gradual type system, but at the heart of it is a fairly -conventional and tame static type system. In Python as a language, on -the other hand, it is not unusual to perform complex metaprogramming, +conventional and tame static type system (apart from untagged union +types and type narrowing, which are common in gradual type systems but +not in traditional static ones). In Python as a language, on the +other hand, it is not unusual to perform complex metaprogramming, especially at the library layer. Typically, type safety is lost when doing these sorts of things. Some @@ -422,16 +424,203 @@ Implementation ] -Rationale -========= +Specification of Needed Preliminaries +===================================== + +(Some content is still in `spec-draft.rst `_). + +We have two subproposals that are necessary to get mileage out of the +main part of this proposal. + + +Unpack of typevars for ``**kwargs`` +----------------------------------- + +A minor proposal that could be split out maybe: + +Supporting ``Unpack`` of typevars for ``**kwargs``:: + + def f[K: BaseTypedDict](**kwargs: Unpack[K]) -> K: + return kwargs -[Describe why particular design decisions were made.] +Here ``BaseTypedDict`` is defined as:: + + class BaseTypedDict(typing.TypedDict): + pass + +But any typeddict would be allowed there. (Or, maybe we should allow ``dict``?) + +This is basically a combination of +"PEP 692 – Using TypedDict for more precise ``**kwargs`` typing" +and the behavior of ``Unpack`` for ``*args`` +from "PEP 646 – Variadic Generics". + +This is potentially moderately useful on its own but is being done to +support processing ``**kwargs`` with type level computation. + +--- + +Extended Callables, take 2 +-------------------------- + +We introduce a ``Param`` type the contains all the information about a function param:: + + class Param[N: str | None, T, Q: ParamQuals = typing.Never]: + pass + + ParamQuals = typing.Literal["*", "**", "default", "keyword"] + + type PosParam[N: str | None, T] = Param[N, T, Literal["positional"]] + type PosDefaultParam[N: str | None, T] = Param[N, T, Literal["positional", "default"]] + type DefaultParam[N: str, T] = Param[N, T, Literal["default"]] + type NamedParam[N: str, T] = Param[N, T, Literal["keyword"]] + type NamedDefaultParam[N: str, T] = Param[N, T, Literal["keyword", "default"]] + type ArgsParam[T] = Param[Literal[None], T, Literal["*"]] + type KwargsParam[T] = Param[Literal[None], T, Literal["**"]] + +And then, we can represent the type of a function like:: + + def func( + a: int, + /, + b: int, + c: int = 0, + *args: int, + d: int, + e: int = 0, + **kwargs: int + ) -> int: + ... + +as (we are omiting the ``Literal`` in places):: + + Callable[ + [ + Param["a", int, "positional"], + Param["b", int], + Param["c", int, "default"], + Param[None, int, "*"], + Param["d", int, "keyword"], + Param["e", int, Literal["default", "keyword"]], + Param[None, int, "**"], + ], + int, + ] + + +or, using the type abbreviations we provide:: + + Callable[ + [ + PosParam["a", int], + Param["b", int], + DefaultParam["c", int, + ArgsParam[int, "*"], + NamedParam["d", int], + NamedDefaultParam["e", int], + KwargsParam[int], + ], + int, + ] + +(Rationale discussed :ref:`below `.) Specification ============= -See `spec-draft.rst `_ for the current draft specification. +As was visible in the examples above, we introduce a few new syntactic +forms of valid types, but much of the power comes from type level +**operators** that will be defined in the ``typing`` module. + + +Grammar specification of the extensions to the type language +------------------------------------------------------------ + +Note first that no changes to the **Python** grammar are being +proposed, only to the grammar of what Python expressions are +considered as valid types. + +(It's also slightly imprecise to call this a grammar: where operator +names are mentioned directly, like ``IsSub``, they require that name +to be imported, and it could also be used qualified as +``typing.IsSub`` or imported as a different name.) + +:: + + = ... + # Type booleans are all valid types too + | + + # Conditional types + | if else + + # Types with variadic arguments can have + # *[... for t in ...] arguments + | [ +] + + | # Only accepted in arguments to new functions? + + # Type conditional checks are boolean compositions of + # "subtype checking" and boolean Literal type checking. + = + IsSub[, ] + | Bool[] + | not + | and + | or + + # Do we want these next two? Maybe not. + | Any[ +] + | All[ +] + + = + , + | * , + + + = [ + * ] + = + # Iterate over a tuple type + for in Iter[] + = + if + + +.. _rt-support: + + +Runtime evaluation support +-------------------------- + +Rationale +========= + +.. _callable-rationale: + +Extended Callables +------------------ + +We need extended callable support, in order to inspect and produce +callables via type-level computation. mypy supports `extended +callables +`__ +but they are deprecated in favor of callback protocols. + +Unfortunately callback protocols don't work well for type level +computation. (They probably could be made to work, but it would +require a separate facility for creating and introspecting *methods*, +which wouldn't be any simpler.) + +I am proposing a fully new extended callable syntax because: + 1. The ``mypy_extensions`` functions are full no-ops, and we need + real runtime objects + 2. They use parentheses and not brackets, which really goes against + the philosophy here. + 3. We can make an API that more nicely matches what we are going to + do for inspecting members (We could introduce extended callables that + closely mimic the ``mypy_extensions`` version though, if something new + is a non starter) Backwards Compatibility @@ -449,7 +638,12 @@ None are expected. How to Teach This ================= -Honestly this seems very hard! +I think some inspiration can be taken from how TypeScript teaches +their equivalent features. + +(Though not complete inspiration---some important subtleties of things +like mapped types are unmentioned in current documentation +("homomorphic mappings").) Reference Implementation @@ -461,9 +655,60 @@ Reference Implementation Rejected Ideas ============== -* Don't attempt to support runtime evaluation, make +Renounce all cares of runtime evaluation +---------------------------------------- + +This would have a lot of simplifying features. + +We wouldn't need to worry about making ``IsSub`` be checkable at +runtime, + +XXX + + +Support TypeScript style pattern matching in subtype checking +------------------------------------------------------------- + +This would almost certainly only be possible if we also decide not to +care about runtime evaluation, as above. + +.. _less_syntax: + + +Use type operators for conditional and iteration +------------------------------------------------ + +Instead of writing: + * ``tt if tb else tf`` + * ``*[tres for T in Iter[ttuple]]`` + +we could use type operator forms like: + * ``Cond[tb, tt, tf]`` + * ``UnpackMap[ttuple, lambda T: tres]`` + * or ``UnpackMap[ttuple, T, tres]`` where ``T`` must be a declared + ``TypeVar`` + +Boolean operations would likewise become operators (``Not``, ``And``, +etc). + +The advantage of this is that constructing a type annotation never +needs to do non-trivial computation, and thus we don't need +:ref:`runtime hooks ` to support evaluating them. + +It would also mean that it would be much easier to extract the raw +type annotation. (The lambda form would still be somewhat fiddly. +The non-lambda form would be trivial to extract, but requiring the +declaration of a ``TypeVar`` goes against the grain of recent +changes.) + +Another advantage is not needing any notion of a special +```` class of types. + +The disadvantage is that is that the syntax seems a *lot* +worse. Supporting filtering while mapping would make it even more bad +(maybe an extra argument for a filter?). -[Why certain ideas that were brought while discussing this PEP were not ultimately pursued.] +We can explore other options too if needed. Open Issues diff --git a/scripts/build-peps.sh b/scripts/build-peps.sh new file mode 100755 index 0000000..47c379b --- /dev/null +++ b/scripts/build-peps.sh @@ -0,0 +1,15 @@ +#!/bin/sh -ex + +mkdir -p build +cd build +if [ ! -d peps ]; then + git clone --depth=1 https://github.com/python/peps/ +fi +cd peps/peps +if [ ! -s pep-9999.rst ]; then + ln -s ../../../pre-pep.rst pep-9999.rst +fi +cd .. +make html +rm -rf ../html +cp -r build ../html diff --git a/spec-draft.rst b/spec-draft.rst index 1b18941..a921196 100644 --- a/spec-draft.rst +++ b/spec-draft.rst @@ -1,152 +1,12 @@ -Unpack of typevars for ``**kwargs`` ------------------------------------ +I AM INCREMENTALLY SHIFTING THINGS TO ``pre-pep.rst`` -A minor proposal that could be split out maybe: - -Supporting ``Unpack`` of typevars for ``**kwargs``:: - - def f[K: BaseTypedDict](**kwargs: Unpack[K]) -> K: - return kwargs - -Here ``BaseTypedDict`` is defined as:: - - class BaseTypedDict(typing.TypedDict): - pass - -But any typeddict would be allowed there. (Or, maybe we should allow ``dict``?) - -This is basically a combination of "PEP 692 – Using TypedDict for more precise **kwargs typing" and the behavior of ``Unpack`` for ``*args`` from "PEP 646 – Variadic Generics". - -This is potentially moderately useful on its own but is being done to support processing **kwargs with type level computation. - -Extended Callables, take 2 --------------------------- - -We introduce a ``Param`` type the contains all the information about a function param:: - - class Param[N: str | None, T, Q: ParamQuals = typing.Never]: - pass - - ParamQuals = typing.Literal["*", "**", "default", "keyword"] - - type PosParam[N: str | None, T] = Param[N, T, Literal["positional"]] - type PosDefaultParam[N: str | None, T] = Param[N, T, Literal["positional", "default"]] - type DefaultParam[N: str, T] = Param[N, T, Literal["default"]] - type NamedParam[N: str, T] = Param[N, T, Literal["keyword"]] - type NamedDefaultParam[N: str, T] = Param[N, T, Literal["keyword", "default"]] - type ArgsParam[T] = Param[Literal[None], T, Literal["*"]] - type KwargsParam[T] = Param[Literal[None], T, Literal["**"]] - -And then, we can represent the type of a function like:: - - def func( - a: int, - /, - b: int, - c: int = 0, - *args: int, - d: int, - e: int = 0, - **kwargs: int - ) -> int: - ... - -as (we are omiting the ``Literal`` in places):: - - Callable[ - [ - Param["a", int, "positional"], - Param["b", int], - Param["c", int, "default"], - Param[None, int, "*"], - Param["d", int, "keyword"], - Param["e", int, Literal["default", "keyword"]], - Param[None, int, "**"], - ], - int, - ] - - -or, using the type abbreviations we provide:: - - Callable[ - [ - PosParam["a", int], - Param["b", int], - DefaultParam["c", int, - ArgsParam[int, "*"], - NamedParam["d", int], - NamedDefaultParam["e", int], - KwargsParam[int], - ], - int, - ] - -Rationale -''''''''' -We need extended callable support, in order to inspect and produce callables via type-level computation. mypy supports `extended callables `__ but they are deprecated in favor of callback protocols. - - -Unfortunately callback protocols don't work well for type level computation. (They probably could be made to work, but it would require a separate facility for creating and introspecting *methods*, which wouldn't be any simpler.) - -I am proposing a fully new extended callable syntax because: - 1. The ``mypy_extensions`` functions are full no-ops, and we need real runtime objects - 2. They use parentheses and not brackets, which really goes against the philosophy here - 3. We can make an API that more nicely matches what we are going to do for inspecting members -(We could introduce extended callables that closely mimic the ``mypy_extensions`` version though, if something new is a non starter) - 4. I thought they were missing support for something but may have been wrong. They can handle positional-only. - - -Grammar specification of the extensions to the type language ------------------------------------------------------------- - -It's important that there be a clearly specified type language for the type-level computation---we can't just be using some poorly specified subset of all Python. - -:: - - = ... - | if else - - # Types with variadic arguments can have - # *[... for t in ...] arguments - | [)> +] - - | # Only accepted in arguments to new functions? - - - # Type conditional checks are just boolean compositions of - # subtype checking. - = - IsSub[, ] - | not - | and - | or - - # Do we want these next two? Probably not. - | Any[)>] - | All[)>] - - = - T , - | * , - - - = [ T + * ] - = - # Iterate over a tuple type - for in Iter[] - = - if - - -``type-for(T)`` is a parameterized grammar rule, which can take different types. Not sure if we actually need this though---now it is only used for Any/All. ----- Type operators -------------- -* ``GetArg[T, Base, Idx: Literal[str]]`` - returns the type argument number ``Idx`` to ``T`` when interpreted as ``Base``, or ``Never`` if it cannot be. (That is, if we have ``class A(B[C]): ...``, then ``GetArg[A, B, 0] == C`` while ``GetArg[A, A, 0] == Never``). +* ``GetArg[T, Base, Idx: Literal[int]]`` - returns the type argument number ``Idx`` to ``T`` when interpreted as ``Base``, or ``Never`` if it cannot be. (That is, if we have ``class A(B[C]): ...``, then ``GetArg[A, B, 0] == C`` while ``GetArg[A, A, 0] == Never``). N.B: *Unfortunately* ``Base`` must be a proper class, *not* a protocol. So, for example, ``GetArg[Ty, Iterable, 0]]`` to get the type of something iterable *won't* work. This is because we can't do protocol checks at runtime in general. Special forms unfortunately require some special handling: the arguments list of a ``Callable`` will be packed in a tuple, and a ``...`` will become ``SpecialFormEllipsis``.