Extend special case for context-based typevar inference to typevar unions in return position #18976

sterliakov · 2025-04-26T17:11:55Z

Fixes wrongly reported typing error in min with key lambda function and default value when result is Optional #17221.
Fixes [arg-type] wrong argument type in generic function with union #17654.
Fixes (🐞) generics not resolving correctly when multiple TypeVars don't meet the expected type #17553.
Fixes min()/max() with default value cause issues with key function type #17536.
Fixes Function returning union of type variables has inference/unification behaviour change when assigned to explicitly-typed location #16659.
Fixes Type inference for max() "tainted" by default #16267.
Fixes Inference failure in return statement with next and or #15755.
Fixes itertools' all_equal recipe causes false arg-type failures #15150.
Fixes False "Invalid index type" for min derived from return type #14664.
Incidentally improves error message in Misleading warning for generic argument type on wrong return type #12156.
Fixes Unneeded type widening of inner function call when using explicit return type #12092.
Fixes Typechecking differs between one-line def of "return expr" vs. "r = expr; return r" #11985.
Improves generics inferred differently when assigning to variable even though the variable's type is the same #11455 (but the problem with union TypeVar | SomeFixedType reported in comments there remains).
Fixes False positive for lambda within a function #10426.

When using context, we can perform some overly optimistic inference when return type is T1 | T2. This breaks in important case of builtins.min when default and key are passed, essentially making them always incompatible. This is not the most principled approach, but let's see the primer results.

This resolves quite a few issues (some of them duplicates, but some - substantially different), min problem was a very popular one... Diff run: sterliakov/mypy-issues#30

…ition to typevar unions

…str` is no less problematic

…m to be helpful

Sometimes our constraints builder needs to express "at least one of these constraints must be true". Sometimes it's trivial, but sometimes they have nothing in common - in that case we fall back to forgetting all of them and (usually) inferring `Never`. This PR extends the fallback handling logic by one more rule: before giving up, we try restricting the constraints to meta variables only. This means that when we try to express `T :> str || U :> str` in the following setup: ``` from typing import TypeVar T = TypeVar("T") U = TypeVar("U") class Foo(Generic[T]): def func(self, arg: T | U) -> None: ... ``` we won't ignore both and fall back to `Never` but will pick `U :> str` instead. The reason for this heuristic is that function-scoped typevars are definitely something we're trying to infer, while everything else is usually out of our control, any other type variable should stay intact. There are other places where constraint builder may emit restrictions on type variables it does not control, handling them consistently everywhere is left as an exercise to the reader. This isn't safe in general case - it might be that another typevar satisfies the constraints but the chosen one doesn't. However, this shouldn't make any existing inference worse: if we used to infer `Never` and it worked, then anything else should almost definitely work as well. See the added testcase for motivation: currently `mypy` fails to handle `Mapping.get` with default without return type context when `Mapping` has a type variable as the second argument. https://mypy-play.net/?mypy=1.15.0&python=3.12&flags=strict&gist=2f9493548082e66b77750655d3a90218 This is a prerequisite of #18976 - that inference change makes the problem solved here occur more often.

sterliakov · 2025-05-06T23:34:43Z

Okay, this is clearly an improvement - xarray case is a regression, but all other hits are either fixed false positives or improved error messages.

github-actions · 2025-05-14T15:26:41Z

Diff from mypy_primer, showing the effect of this PR on open source code:

xarray (https://github.com/pydata/xarray)
+ xarray/conventions.py: note: In function "decode_cf_variables":
+ xarray/conventions.py:410: error: Argument "decode_times" to "decode_cf_variable" has incompatible type "object"; expected "bool | CFDatetimeCoder"  [arg-type]

antidote (https://github.com/Finistere/antidote)
- src/antidote/lib/interface_ext/_provider.py:313: error: Generator has incompatible item type "NewWeight"; expected "Weight"  [misc]
- src/antidote/lib/interface_ext/_provider.py:314: error: Argument 2 to "sum" has incompatible type "NewWeight"; expected "Weight"  [arg-type]
+ src/antidote/lib/interface_ext/_provider.py:312: error: Argument "weight" to "replace" of "CandidateImplementation[Weight]" has incompatible type "NewWeight"; expected "Weight"  [arg-type]

operator (https://github.com/canonical/operator)
- ops/_main.py:97: error: Argument 2 to "next" has incompatible type "None"; expected "Storage"  [arg-type]
+ ops/_main.py:97: error: Incompatible types in assignment (expression has type "Storage | None", variable has type "Storage")  [assignment]

setuptools (https://github.com/pypa/setuptools)
+ setuptools/msvc.py:1442: error: Unused "type: ignore" comment  [unused-ignore]

Tanjun (https://github.com/FasterSpeeding/Tanjun)
- tanjun/clients.py:2428: error: Argument 1 to "get_type_dependency" of "Client" has incompatible type "type[SingleStoreCache[AuthorizationApplication]]"; expected "type[SingleStoreCache[Application]] | type[None]"  [arg-type]

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/datastructures/mixins.py:280: error: Unused "type: ignore" comment  [unused-ignore]
+ src/werkzeug/datastructures/mixins.py:280: error: Incompatible types in assignment (expression has type "Union[V, T]", variable has type "V")  [assignment]
+ src/werkzeug/datastructures/mixins.py:280: note: Error code "assignment" not covered by "type: ignore" comment

archinstall (https://github.com/archlinux/archinstall)
+ archinstall/lib/profile/profiles_handler.py:171: error: Unused "type: ignore[arg-type, union-attr]" comment  [unused-ignore]

hauntsaninja

Nice, many common functions with this shape!

ilevkivskyi

Yeah, this is a net improvement.

ilevkivskyi · 2025-05-15T09:02:40Z

Btw @hauntsaninja what does "upnext" label mean? Can I merge this PR now?

hauntsaninja · 2025-05-15T09:03:58Z

Yeah we can merge! It means "I want to make sure this PR is not forgotten forever in case no one else gets to it"

sterliakov added 2 commits April 26, 2025 19:09

Extend special-case for context-based typevar inference in return pos…

c4baf4e

…ition to typevar unions

Add missing get_proper_type

8331546