Rework the design a bunch

spec-draft.rst

@@ -3,45 +3,37 @@ Grammar specification of the extensions to the type language.
 
 It's important that there be a clearly specified type language for the type-level computation---we can't just be using some poorly specified subset of all Python.
 
+TODO:
+- Look into TupleTypeVar stuff for iteration
+
+Big Q: what should be an error and what should return Never?
+
 
 ::
 
    <type> = ...
         | <type> if <type-bool> else <type>
 
-        # Create NewProtocols and Unions using for loops.
-        # They can take either a single list comprehension as an
-        # argument, or starred list comprehensions can be included
-        # in the argument list.
-
-        # TODO: NewProtocol needs a way of doing bases also...
-        # TODO: Should probably support Callable, TypedDict, etc
-        | NewProtocol[<type-for(<prop-spec>)>]
-        | NewProtocol[<variadic-type-arg(<prop-spec>)> +]
+        # Types with variadic arguments can have
+        # *[... for t in ...] arguments
+        | <ident>[<variadic-type-arg(<type>)> +]
 
-        | Union[<type-for(<type>)>]
-        | Union[<variadic-type-arg(<type-for>)> +]
-
-        | GetAttr[<type>, <type>]
+        # This is syntax because taking an int literal makes it a
+        # special form.
         | GetArg[<type>, <int-literal>]
 
-        # String manipulation operations for string Literal types.
-        # We can put more in, but this is what typescript has.
-        | Uppercase[<type>] | Lowercase[<type>]
-        | Capitalize[<type>] | Uncapitalize[<type>]
 
    # Type conditional checks are just boolean compositions of
    # subtype checking.
    <type-bool> =
-         IsSubtype[<type>, <type>]
+         Is[<type>, <type>]
        | not <type-bool>
        | <type-bool> and <type-bool>
        | <type-bool> or <type-bool>
-       # Do we want these next two?
-       | any(<type-for(<type-bool>)>)
-       | all(<type-for(<type-bool>)>)
 
-   <prop-spec> = Property[<type>, <type>]
+       # Do we want these next two? Probably not.
+       | Any[<type-for(<type-bool>)>]
+       | All[<type-for(<type-bool>)>]
 
    <variadic-type-arg(T)> =
          T ,
@@ -50,14 +42,117 @@ It's important that there be a clearly specified type language for the type-leve
 
    <type-for(T)> = [ T <type-for-iter>+ <type-for-if>* ]
    <type-for-iter> =
-         for <var> in IterUnion<type>
-       | for <var>, <var> in DirProperties<type>
-       # TODO: callspecs
-       # TODO: variadic args (tuples, callables)
+         # Iterate over a tuple type
+         for <var> in Iter[<type>]
    <type-for-if> =
          if <type-bool>
 
 
+``type-for(T)`` is a parameterized grammar rule, which can take
+different types. Not sure if we actually need this though---now it is
+only used for Any/All.
+
+---
+
+# TODO: NewProtocol needs a way of doing bases also...
+# TODO: New TypedDict setup
+* ``NewProtocol[*Ps: Member]``
+
+* ``Members[T]`` produces a ``tuple`` of ``Member`` types.
+* ``Member[N: Literal[str], T, Q: Quals, D]``
+# These names are too long -- but we can't do ``Type`` !!
+* ``GetName[T: Member]``
+* ``GetType[T: Member]``
+* ``GetQuals[T: Member]``
+* ``GetDefiner[T: Member]``
+* Could we also put the defining type there??
+
+---
+
+* ``GetAttr[T, S: Literal[str]]``
+
+# TODO: how to deal with special forms like Callable and tuple[T, ...]
+* ``GetArgs[T]`` - returns a tuple containing all of the type arguments
+* ``FromUnion[T]`` - returns a tuple containing all of the union
+  elements, or a 1-ary tuple containing T if it is not a union.
+
+# TODO: How to do IsUnion?
+
+
+String manipulation operations for string Literal types.
+We can put more in, but this is what typescript has.
+``Slice`` and ``Concat`` are a poor man's literal template.
+We can actually implement the case functions in terms of them and a
+bunch of conditionals.
+
+
+* ``Slice[S: Literal[str], Start: Literal[int | None], End: Literal[int | None]]``
+* ``Concat[S1: Literal[str], S2: Literal[str]]``
+
+* ``Uppercase[S: Literal[str]]``
+* ``Lowercase[S: Literal[str]]``
+* ``Capitalize[S: Literal[str]]``
+* ``Uncapitalize[S: Literal[str]]``
+
+
+
+-------------------------------------------------------------------------
+
+
+Big open questions?
+
+1.
+Can we actually implement Is (IsSubtype) at runtime in a satisfactory way?
+ - Could we slightly dodge the question by *not* adding the evaluation library to the standard library, and letting the operations be opaque.
+
+   Then we would promise to have a third-party library, which would need to be "fit for purpose" for people to want to use, but would be free of the burden of being canonical?
+
+  There is a lot that needs to happen, like protocols and variance inference and
+callable subtyping (which might require matching against type vars...)
+
+ - I think we probably *can't* try to put it in the standard library. I think it would by nature bless the implementation with some degree of canonicity that I'm not sure we can back up. Different typecheckers don't always match on subtyping behavior, *and* it sometimes depends on config flags (like strict_optional in mypy). *And* we could imagine a bunch of other config flags: whether to be strict about argument names in protocols, for example.
+
+ - We can instead have something simpler, which I will call ``Matches``. ``Matches`` would do *simple* checking of the *head* of types, essentially, without looking at type parameters. It would still lift over unions and would check literals.
+   Honestly this is basically what is currently implemented for the examples, so it is probably good enough.
+
+   It's unsatisfying, though.
+
+2.
+How do we deal with modifiers? ClassVar, Final, Required, ReadOnly
+ - One option is to treat them not as types by as *modifiers* and have them
+   in a separate field where they are a union of Literals.
+   So ``x: Final[ClassVar[int]]`` would appear in ``Attrs`` as
+   ``Member[Literal['x'], int, Literal['Final' | 'ClassVar']]``
+
+   This is kind of unsatisfying but I think it's probably right.
+   We could also have a ``MemberUpdate[M: Member, T]`` that updates
+   the type of a member but preserves its name and modifiers.
+
+ -
+
+
+3.
+How do we deal with Callables? We need to support extended callable syntax basically.
+Or something like it.
+
+4.
+What do we do about ``Members`` on built-in types? ``typing.get_type_hints(int)`` returns ``{}`` but mypy will not agree!
+
+An object of an empty user-defined class has 29 entries in ``dir`` (all dunders), and ``object()`` has 24. (In 3.14. In 3.12, it was 27 for the user-defined object).
+
+5.
+Polymorphic callables? How do we represent their type and how do we construct their type?
+
+What does TS do here? - TS has full impredactive polymorphic functions. You can do System F stuff.
+
+=====
+
+This proposal is less "well-typed" than typescript... (Well-kinded, maybe?)
+Typescript has better typechecking at the alias definition site:
+For ``P[K]``, ``K`` needs to have ``keyof P``...
 
-``type-for(T)`` and ``variadic-type-arg(T)`` are parameterized grammar
-rules, which can take different
+Oh, we could maybe do better but it would require some new machinery.
+* ``KeyOf[T]`` - literal keys of ``T``
+* ``Member[T]``, when statically checking a type alias, could be treated as having some type like ``tuple[Member[KeyOf[T], object???, str], ...]``
+* ``GetAttr[T, S: KeyOf[T]]`` - but this isn't supported yet. TS supports it.
+* We would also need to do context sensitive type bound inference

tests/test_call.py

@@ -1,15 +1,22 @@
 import textwrap
 
 from typemap.type_eval import eval_call
-from typemap import typing as next
+from typemap.typing import (
+    CallSpec,
+    NewProtocol,
+    Member,
+    GetName,
+    Iter,
+    CallSpecKwargs,
+)
 
 from . import format_helper
 
 
-def func[C: next.CallSpec](
+def func[C: CallSpec](
     *args: C.args, **kwargs: C.kwargs
-) -> next.NewProtocol[
-    *[next.Property[c.name, int] for c in next.CallSpecKwargs[C]]
+) -> NewProtocol[
+    *[Member[GetName[c], int] for c in Iter[CallSpecKwargs[C]]]
 ]: ...
 
 

tests/test_qblike.py

@@ -1,7 +1,19 @@
 import textwrap
 
 from typemap.type_eval import eval_call, eval_typing
-from typemap import typing as next
+from typemap.typing import (
+    NewProtocol,
+    Iter,
+    Attrs,
+    Is,
+    GetType,
+    CallSpec,
+    Member,
+    GetName,
+    GetAttr,
+    CallSpecKwargs,
+    GetArg,
+)
 
 from . import format_helper
 
@@ -14,18 +26,12 @@ class Link[T]:
     pass
 
 
-type PropsOnly[T] = next.NewProtocol[
-    [
-        next.Property[p.name, p.type]
-        for p in next.DirProperties[T]
-        if next.IsSubtype[p.type, Property]
-    ]
+type PropsOnly[T] = NewProtocol[
+    *[p for p in Iter[Attrs[T]] if Is[GetType[p], Property]]
 ]
 
 # Conditional type alias!
-type FilterLinks[T] = (
-    Link[PropsOnly[next.GetArg[T, 0]]] if next.IsSubtype[T, Link] else T
-)
+type FilterLinks[T] = Link[PropsOnly[GetArg[T, 0]]] if Is[T, Link] else T
 
 
 # Basic filtering
@@ -45,15 +51,15 @@ class A:
     w: Property[list[str]]
 
 
-def select[C: next.CallSpec](
+def select[C: CallSpec](
     __rcv: A, *args: C.args, **kwargs: C.kwargs
-) -> next.NewProtocol[
-    [
-        next.Property[
-            c.name,
-            FilterLinks[next.GetAttr[A, c.name]],
+) -> NewProtocol[
+    *[
+        Member[
+            GetName[c],
+            FilterLinks[GetAttr[A, GetName[c]]],
         ]
-        for c in next.CallSpecKwargs[C]
+        for c in Iter[CallSpecKwargs[C]]
     ]
 ]: ...
 
@@ -103,7 +109,7 @@ class select[...]:
             z: tests.test_qblike.Link[PropsOnly[tests.test_qblike.Tgt]]
         """)
 
-    tgt = eval_typing(next.GetAttr[ret, "z"].__args__[0])
+    tgt = eval_typing(GetAttr[ret, "z"].__args__[0])
     fmt = format_helper.format_class(tgt)
 
     assert fmt == textwrap.dedent("""\