Remove list.__add__ overloads.

[randolf-scholz]

See #14283.

The overloads on list.__add__ lead to divergence of mypy and pyright. Code sample in pyright playground, https://mypy-play.net/?mypy=latest&python=3.12&gist=abf6a8834020af17a16bd8cfb44b2f10

from typing import Any, overload

class ListA[T]:  # emulates builtins list
    @overload
    def __add__(self, other: "ListA[T]", /) -> "ListA[T]": return ListA()
    @overload
    def __add__[S](self, other: "ListA[S]", /) -> "ListA[T | S]": return ListA()
    
    
class ListB[T]:  # without overloads
    def __add__[S](self, other: "ListB[S]", /) -> "ListB[T | S]": return ListB()

                                            # mypy              | pyright                             
reveal_type( list[str]() + list[str]() )    # list[str]         | list[str]          ✅
reveal_type( list[str]() + list[int]() )    # list[str | int]   | list[str | int]    ✅
reveal_type( list[str]() + list[Any]() )    # list[Any]         | list[str]          ❌

reveal_type( ListA[str]() + ListA[str]() )  # ListA[str]        | ListA[str]         ✅
reveal_type( ListA[str]() + ListA[int]() )  # ListA[str | int]  | ListA[str | int]   ✅
reveal_type( ListA[str]() + ListA[Any]() )  # ListA[Any]        | ListA[str]         ❌

reveal_type( ListB[str]() + ListB[str]() )  # ListB[str]        | ListB[str]         ✅
reveal_type( ListB[str]() + ListB[int]() )  # ListB[str | int]  | ListB[str | int]   ✅
reveal_type( ListB[str]() + ListB[Any]() )  # ListB[str | Any]  | ListB[str | Any]   ✅

A comment from 3 years added in #8293 states that

# Overloading looks unnecessary, but is needed to work around complex mypy problems

I want to see the impact on mypy primer, and whether this comment is still valid given that mypy has had several major releases since.

[randolf-scholz]

There must be a strange bug in mypy. Take this example from the mypy primer above

If I change the offending line (Unsupported operand types for + ("list[FrameSummary]" and "list[FrameSummary]")) from

for s in traceback.format_list(tb + tb2):

to

dummy = tb + tb2
for s in traceback.format_list(dummy):

Then the file passes without issue. (using python -m mypy.stubtest --custom-typeshed-dir=../typeshed/ tmp)

[jorenham]

maybe something like this could work?

Suggested change

	def __add__(self, value: list[_S], /) -> list[_S | _T]: ...
	def __add__(self: list[_S], value: list[_S], /) -> list[_S]: ...

[randolf-scholz]

With this, wouldn't we get list[str] + list[int] = list[object] when checking with mypy, rather then list[str | int]?

I'd prefer python/mypy#19304 be fixed; then I think we would see most mypy-primer issues here go away. If you have seen this before and know and easier way to reproduce, please comment it.

[jorenham]

With this, wouldn't we get list[str] + list[int] = list[object] when checking with mypy, rather then list[str | int]?

With this, wouldn't we get list[str] + list[int] = list[object] when checking with mypy, rather then list[str | int]?

I'd prefer python/mypy#19304 be fixed; then I think we would see most mypy-primer issues here go away. If you have seen this before and know and easier way to reproduce, please comment it.

That's what I indeed expect here, given mypy's join vs union behavior. But if that's how they decide to behave, then that's up to them, afaik. At least it won't be rejected this way (is the hope)

[randolf-scholz]

Well, my goal with this PR was to reduce type-checker divergence, not to cement it.

[jorenham]

fair enough

[randolf-scholz]

This is probably gonna sit here until python/mypy#19304 is fixed. A minimal repro of this is

class Vec[T]:
    def get(self) -> T: ...   # just here to insure invariance
    def set(self, index: int, value: T) -> None: ...  # just here to insure invariance

    def __add__[S](self, other: "Vec[S]", /) -> "Vec[T | S]": return Vec()
        
def pprint(arg: Vec[object]) -> None: ...

def demo(ints: Vec[int], strs: Vec[str]):
    pprint(ints + strs)  # ❌️ mypy false positive

Last year I spend quite a bit of time trying to fix this: randolf-scholz/mypy#4

But gave up after a while because there were other blockers:

• RFC: Rework *args parsing mypy#20472
• Improved Literal? types mypy#20444

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

mypy (https://github.com/python/mypy)
+ mypyc/crash.py:29: error: Unsupported operand types for + ("list[FrameSummary]" and "list[FrameSummary]")  [operator]
+ mypyc/crash.py:29: note: See https://mypy.rtfd.io/en/stable/_refs.html#code-operator for more info
+ mypy/strconv.py:467: error: Unsupported operand types for + ("list[Any]" and "list[str | tuple[str, list[Any]]]")  [operator]
+ mypy/errors.py:1396: error: Unsupported operand types for + ("list[FrameSummary]" and "StackSummary")  [operator]
+ mypy/fastparse.py:1603: error: Unsupported operand types for + ("list[expr]" and "list[expr]")  [operator]
+ mypy/test/testcmdline.py:80: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]

freqtrade (https://github.com/freqtrade/freqtrade)
+ freqtrade/data/entryexitanalysis.py:285: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ freqtrade/data/entryexitanalysis.py:290: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ freqtrade/data/entryexitanalysis.py:296: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ freqtrade/data/entryexitanalysis.py:297: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]

spark (https://github.com/apache/spark)
+ python/pyspark/pandas/groupby.py:3797: error: Unsupported operand types for + ("list[Series[Any]]" and "list[Series[Any]]")  [operator]

graphql-core (https://github.com/graphql-python/graphql-core)
+ src/graphql/type/validate.py:417: error: Unsupported operand types for + ("list[NamedTypeNode]" and "list[NamedTypeNode]")  [operator]
+ src/graphql/validation/rules/overlapping_fields_can_be_merged.py:86: error: Unsupported operand types for + ("list[FieldNode]" and "list[FieldNode]")  [operator]

tornado (https://github.com/tornadoweb/tornado)
+ tornado/netutil.py:158: error: Incompatible types in assignment (expression has type "tuple[int | Any, ...]", variable has type "tuple[str, int] | tuple[str, int, int, int] | tuple[int, bytes]")  [assignment]
+ tornado/autoreload.py:234: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]

rotki (https://github.com/rotki/rotki)
+ rotkehlchen/chain/evm/decoding/uniswap/utils.py:318: error: Unsupported operand types for + ("list[EvmEvent]" and "list[EvmEvent]")  [operator]
+ rotkehlchen/chain/evm/decoding/pendle/decoder.py:496: error: Unsupported operand types for + ("list[EvmEvent]" and "list[EvmEvent]")  [operator]
+ rotkehlchen/chain/evm/decoding/beefy_finance/decoder.py:270: error: Unsupported operand types for + ("list[EvmEvent]" and "list[EvmEvent]")  [operator]
+ rotkehlchen/chain/evm/decoding/balancer/v3/decoder.py:238: error: Unsupported operand types for + ("list[EvmEvent]" and "list[EvmEvent]")  [operator]
+ rotkehlchen/chain/evm/decoding/odos/common.py:177: error: Unsupported operand types for + ("list[EvmEvent]" and "list[EvmEvent]")  [operator]
+ rotkehlchen/chain/evm/decoding/curve/decoder.py:438: error: Unsupported operand types for + ("list[EvmEvent]" and "list[EvmEvent]")  [operator]
+ rotkehlchen/chain/evm/decoding/balancer/v2/decoder.py:118: error: Unsupported operand types for + ("list[EvmEvent]" and "list[EvmEvent]")  [operator]
+ rotkehlchen/chain/evm/decoding/uniswap/v2/utils.py:393: error: Unsupported operand types for + ("list[EvmEvent]" and "list[EvmEvent]")  [operator]
+ rotkehlchen/chain/evm/decoding/uniswap/v2/utils.py:395: error: Unsupported operand types for + ("list[EvmEvent]" and "list[EvmEvent]")  [operator]

spack (https://github.com/spack/spack)
+ lib/spack/spack/environment/environment.py:2702: error: Incompatible return value type (got "list[tuple[Any, Any] | tuple[Spec, None]]", expected "list[tuple[Spec, Spec]]")  [return-value]

sympy (https://github.com/sympy/sympy)
+ sympy/polys/galoistools.py:1755: error: Need type annotation for "Q"  [var-annotated]

materialize (https://github.com/MaterializeInc/materialize)
+ misc/python/materialize/cli/ci_upload_heap_profiles.py:42: error: Unsupported operand types for + ("list[str | Any]" and "list[str]")  [operator]
+ misc/python/materialize/cargo.py:279: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ misc/python/materialize/cargo.py:283: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ misc/python/materialize/mzbuild.py:1568: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ misc/python/materialize/mzbuild.py:1572: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/io/stata.py:1403: error: List comprehension has incompatible type List[object]; expected List[str | dtype[Any]]  [misc]
+ pandas/tests/groupby/conftest.py:130: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ pandas/tests/frame/test_constructors.py:2357: error: Unsupported operand types for + ("list[ExtensionDtype | str | dtype[Any] | type[object] | type[str] | type[complex] | type[bool]]" and "list[ExtensionDtype | str | dtype[Any] | type[str] | type[complex] | type[bool] | type[object]]")  [operator]
+ pandas/tests/frame/test_constructors.py:2367: error: Unsupported operand types for + ("list[ExtensionDtype | str | dtype[Any] | type[object] | type[str] | type[complex] | type[bool]]" and "list[ExtensionDtype | str | dtype[Any] | type[str] | type[complex] | type[bool] | type[object]]")  [operator]
+ pandas/tests/arrays/test_datetimes.py:70: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ pandas/conftest.py:1661: error: Unsupported operand types for + ("list[ExtensionDtype | str | dtype[Any] | type[object] | type[str] | type[complex] | type[bool]]" and "list[ExtensionDtype | str | dtype[Any] | type[str] | type[complex] | type[bool] | type[object]]")  [operator]

prefect (https://github.com/PrefectHQ/prefect)
- src/prefect/utilities/callables.py:579: error: Incompatible types in assignment (expression has type "list[None]", variable has type "list[expr | None]")  [assignment]
- src/prefect/utilities/callables.py:579: note: "list" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
- src/prefect/utilities/callables.py:579: note: Consider using "Sequence" instead, which is covariant

pydantic (https://github.com/pydantic/pydantic)
- pydantic/aliases.py:29: error: Incompatible types in assignment (expression has type "list[str]", variable has type "list[int | str]")  [assignment]
- pydantic/aliases.py:29: note: "list" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
- pydantic/aliases.py:29: note: Consider using "Sequence" instead, which is covariant
- pydantic/aliases.py:29: error: Argument 1 to "list" has incompatible type "tuple[str | int, ...]"; expected "Iterable[str]"  [arg-type]
+ pydantic/aliases.py:29: error: Argument 1 to "list" has incompatible type "tuple[str | int, ...]"; expected "Iterable[int]"  [arg-type]

scipy (https://github.com/scipy/scipy)
+ scipy/fft/_pocketfft/tests/test_basic.py:473: error: Unsupported operand types for + ("list[int]" and "list[int]")  [operator]
+ scipy/fft/_pocketfft/tests/test_basic.py:483: error: Unsupported operand types for + ("list[int]" and "list[int]")  [operator]
+ scipy/fft/_pocketfft/tests/test_basic.py:502: error: Unsupported operand types for + ("list[int]" and "list[int]")  [operator]
+ scipy/fft/_pocketfft/tests/test_basic.py:512: error: Unsupported operand types for + ("list[int]" and "list[int]")  [operator]
+ scipy/fftpack/tests/test_basic.py:429: error: Unsupported operand types for + ("list[int]" and "list[int]")  [operator]
+ scipy/fftpack/tests/test_basic.py:439: error: Unsupported operand types for + ("list[int]" and "list[int]")  [operator]
+ scipy/fftpack/tests/test_basic.py:458: error: Unsupported operand types for + ("list[int]" and "list[int]")  [operator]
+ scipy/fftpack/tests/test_basic.py:468: error: Unsupported operand types for + ("list[int]" and "list[int]")  [operator]
+ scipy/optimize/tests/test_optimize.py:1295: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]

jax (https://github.com/google/jax)
+ jax/_src/internal_test_util/test_harnesses.py:422: error: Unsupported operand types for + ("list[str | type[Any] | dtype[Any] | SupportsDType]" and "list[str | type[Any] | dtype[Any] | SupportsDType]")  [operator]

meson (https://github.com/mesonbuild/meson)
+ mesonbuild/scripts/gettext.py:54:70: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/utils/universal.py:2382:87: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/cmake/interpreter.py:1080:69: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/cmake/interpreter.py:1225:60: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/modules/gnome.py:1558:23: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/modules/gnome.py:1575:77: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/scripts/depfixer.py:515:66: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/scripts/coverage.py:62:40: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/scripts/coverage.py:73:40: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/scripts/coverage.py:83:35: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/scripts/coverage.py:120:35: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/scripts/coverage.py:128:35: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/scripts/coverage.py:134:59: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/scripts/coverage.py:140:64: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/scripts/coverage.py:147:64: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/scripts/coverage.py:156:47: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/scripts/coverage.py:172:40: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]
+ mesonbuild/mtest.py:2263:47: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]

cloud-init (https://github.com/canonical/cloud-init)
+ cloudinit/config/cc_mounts.py:420: error: Incompatible return value type (got "list[list[str | Any | None]]", expected "list[list[str]]")  [return-value]

paasta (https://github.com/yelp/paasta)
+ paasta_tools/config_utils.py:100: error: Unsupported operand types for + ("list[str]" and "list[str]")  [operator]