Assume unannotated __new__ returns Self#3139
Assume unannotated __new__ returns Self#3139grievejia wants to merge 2 commits intofacebook:mainfrom
__new__ returns Self#3139Conversation
Summary: Issue 3121 demonstrates bad behavior of recursive return type inference in `__new__` for networkx code, which is using highly dynamic logic to return a "`_dispatch`-like" partial function from `functools.partial`. Inferring something like this is clearly well out of the scope of a type checker; I don't think the static type is even denotable in any useful way, it's relying heavily on duck typing in ways the static type system does not allow. The best we can reasonably do is just pretend constructor returns `_dispatch`, which would avoid many thousands of downstream errors. This specific issue is very important because getting an inferred `__new__` type wrong is likely to cause exceptionally noisy downstream behavior, since almost every instance winds up with a bogus type. This test case demonstrates the issue (in a simpler example where there's no nonconvergence); I'll stack a fix to assume, in accordance with the typing spec, that an unannotated `__new__` means the constructor behaves normally returning an instance of `Self`. Differential Revision: D100705390
Summary: This fixes some pretty bad behaviors when dealing with return type inference on complex code that is really dynamically typed, but can be approximated by normal static typing (specifically, the networkx `_dispatch` class). In general, getting bad return type inference on constructors can fan out to a lot of errors, so this is a nice way to reduce FPs against untyped dependencies that might do complicated, dynamic things. It is consistent with the typing spec to do this. Note that `networkx` isn't in mypy primer, but we actually get a significant error reduction on `sympy` which is a good sign that this is a real improvement in practice on untyped code beyond just the one motivating example. Fixes facebook#3120 Differential Revision: D100686394
|
@grievejia has exported this pull request. If you are a Meta employee, you can view the originating Diff in D100686394. |
|
Diff from mypy_primer, showing the effect of this PR on open source code: ppb-vector (https://github.com/ppb/ppb-vector)
- ERROR ppb_vector/__init__.py:244:16-58: Returned type `None` is not assignable to declared return type `Vector` [bad-return]
- ERROR ppb_vector/__init__.py:263:16-58: Returned type `None` is not assignable to declared return type `Vector` [bad-return]
- ERROR ppb_vector/__init__.py:292:16-56: Returned type `None` is not assignable to declared return type `Vector` [bad-return]
- ERROR ppb_vector/__init__.py:352:16-54: Returned type `None` is not assignable to declared return type `Vector` [bad-return]
- ERROR ppb_vector/__init__.py:463:17-30: `None` is not assignable to variable `other` with type `Sequence[SupportsFloat] | Vector | VectorLikeDict | tuple[SupportsFloat, SupportsFloat]` [bad-assignment]
- ERROR ppb_vector/__init__.py:467:13-25: Object of class `Sequence` has no attribute `length`
- Object of class `VectorLikeDict` has no attribute `length`
- Object of class `tuple` has no attribute `length` [missing-attribute]
- ERROR ppb_vector/__init__.py:468:28-44: Object of class `NoneType` has no attribute `length` [missing-attribute]
- ERROR ppb_vector/__init__.py:506:16-28: Returned type `None` is not assignable to declared return type `Vector` [bad-return]
- ERROR ppb_vector/__init__.py:583:17-30: `None` is not assignable to variable `basis` with type `Sequence[SupportsFloat] | Vector | VectorLikeDict | tuple[SupportsFloat, SupportsFloat]` [bad-assignment]
- ERROR ppb_vector/__init__.py:584:24-36: Object of class `Sequence` has no attribute `length`
- Object of class `VectorLikeDict` has no attribute `length`
- Object of class `tuple` has no attribute `length` [missing-attribute]
- ERROR ppb_vector/__init__.py:587:13-35: `*` is not supported between `float` and `Sequence[SupportsFloat]` [unsupported-operation]
- ERROR ppb_vector/__init__.py:587:13-35: `*` is not supported between `float` and `VectorLikeDict` [unsupported-operation]
- ERROR ppb_vector/__init__.py:587:13-35: `*` is not supported between `float` and `tuple[SupportsFloat, SupportsFloat]` [unsupported-operation]
- ERROR ppb_vector/__init__.py:588:16-27: Returned type `tuple[Vector | float, Vector]` is not assignable to declared return type `tuple[Vector, Vector]` [bad-return]
- ERROR ppb_vector/__init__.py:588:19-27: `-` is not supported between `Self@Vector` and `float` [unsupported-operation]
- ERROR ppb_vector/__init__.py:606:26-48: `None` is not assignable to variable `surface_normal` with type `Sequence[SupportsFloat] | Vector | VectorLikeDict | tuple[SupportsFloat, SupportsFloat]` [bad-assignment]
- ERROR ppb_vector/__init__.py:607:24-45: Object of class `Sequence` has no attribute `length`
- Object of class `VectorLikeDict` has no attribute `length`
- Object of class `tuple` has no attribute `length` [missing-attribute]
- ERROR ppb_vector/__init__.py:610:16-69: `-` is not supported between `Self@Vector` and `float` [unsupported-operation]
- ERROR ppb_vector/__init__.py:610:24-68: `*` is not supported between `float` and `Sequence[SupportsFloat]` [unsupported-operation]
- ERROR ppb_vector/__init__.py:610:24-68: `*` is not supported between `float` and `VectorLikeDict` [unsupported-operation]
- ERROR ppb_vector/__init__.py:610:24-68: `*` is not supported between `float` and `tuple[SupportsFloat, SupportsFloat]` [unsupported-operation]
- ERROR ppb_vector/__init__.py:617:17-44: Object of class `NoneType` has no attribute `normalize` [missing-attribute]
- ERROR ppb_vector/__init__.py:622:17-29: `None` is not assignable to attribute `x_unit` with type `Vector` [bad-assignment]
- ERROR ppb_vector/__init__.py:623:17-29: `None` is not assignable to attribute `y_unit` with type `Vector` [bad-assignment]
- ERROR ppb_vector/__init__.py:624:15-27: `None` is not assignable to attribute `zero` with type `Vector` [bad-assignment]
- ERROR tests/test_angle.py:24:10-20: Object of class `NoneType` has no attribute `angle` [missing-attribute]
- ERROR tests/test_angle.py:25:10-21: Object of class `NoneType` has no attribute `angle` [missing-attribute]
- ERROR tests/test_decompose.py:35:72-76: Argument `None` is not assignable to parameter `other` with type `Sequence[SupportsFloat] | Vector | VectorLikeDict | tuple[SupportsFloat, SupportsFloat]` in function `ppb_vector.Vector.isclose` [bad-argument-type]
- ERROR tests/test_decompose.py:36:72-76: Argument `None` is not assignable to parameter `other` with type `Sequence[SupportsFloat] | Vector | VectorLikeDict | tuple[SupportsFloat, SupportsFloat]` in function `ppb_vector.Vector.isclose` [bad-argument-type]
- ERROR tests/test_length.py:23:12-25: Object of class `NoneType` has no attribute `length` [missing-attribute]
- ERROR tests/test_member_access.py:10:12-15: Object of class `NoneType` has no attribute `x` [missing-attribute]
- ERROR tests/test_member_access.py:11:12-15: Object of class `NoneType` has no attribute `y` [missing-attribute]
- ERROR tests/test_pattern_matching.py:30:21-22: Cannot match positional sub-patterns in `Never` [bad-match]
- ERROR tests/test_pattern_matching.py:30:24-25: Cannot match positional sub-patterns in `Never` [bad-match]
- ERROR tests/test_project.py:30:88-92: Argument `None` is not assignable to parameter `other` with type `Sequence[SupportsFloat] | Vector | VectorLikeDict | tuple[SupportsFloat, SupportsFloat]` in function `ppb_vector.Vector.isclose` [bad-argument-type]
- ERROR tests/test_project.py:50:29-33: Argument `None` is not assignable to parameter `other` with type `Sequence[SupportsFloat] | Vector | VectorLikeDict | tuple[SupportsFloat, SupportsFloat]` in function `ppb_vector.Vector.isclose` [bad-argument-type]
- ERROR tests/test_reflect.py:22:12-35: Object of class `NoneType` has no attribute `reflect` [missing-attribute]
- ERROR tests/test_rotate.py:87:35-38: Object of class `NoneType` has no attribute `x` [missing-attribute]
- ERROR tests/test_rotate.py:87:41-44: Object of class `NoneType` has no attribute `y` [missing-attribute]
- ERROR tests/test_rotate.py:147:17-31: Object of class `NoneType` has no attribute `rotate` [missing-attribute]
- ERROR tests/test_rotate.py:152:20-35: Object of class `NoneType` has no attribute `rotate` [missing-attribute]
- ERROR tests/test_rotate.py:156:43-57: Object of class `NoneType` has no attribute `length` [missing-attribute]
|
Primer Diff Classification✅ 1 improvement(s) | 1 project(s) total | -35 errors 1 improvement(s) across ppb-vector.
Detailed analysis✅ Improvement (1)ppb-vector (-35)
Was this helpful? React with 👍 or 👎 Classification by primer-classifier (1 LLM) |
Summary:
This fixes some pretty bad behaviors when dealing with return type
inference on complex code that is really dynamically typed, but can
be approximated by normal static typing (specifically, the
networkx
_dispatchclass).In general, getting bad return type inference on constructors can
fan out to a lot of errors, so this is a nice way to reduce FPs
against untyped dependencies that might do complicated, dynamic things.
It is consistent with the typing spec to do this. Note that
networkxisn'tin mypy primer, but we actually get a significant error reduction on
sympywhichis a good sign that this is a real improvement in practice on untyped code
beyond just the one motivating example.
Fixes #3120
Differential Revision: D100686394