8000 Overloads with type variables and generic types are incorrectly rejected · Issue #9420 · python/mypy · GitHub
[go: up one dir, main page]
Skip to content

Overloads with type variables and generic types are incorrectly rejected #9420

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
catern opened this issue Sep 5, 2020 · 6 comments
Closed

Overloads with type variables and generic types are incorrectly rejected #9420

catern opened this issue Sep 5, 2020 · 6 comments
Labels
bug mypy got something wrong topic-overloads

Comments

@catern
Copy link
catern commented Sep 5, 2020

🐛 Bug Report

If there are multiple overloads for a function, each of which take different type variables, and which return a generic type parameterized on the type variable, mypy will incorrectly report that the overload is incorrect.

To Reproduce

from typing import TypeVar, overload, Type, Tuple, Union

class Foo:
    pass

class Bar:
    pass

T1 = TypeVar('T1', bound=Foo)
T2 = TypeVar('T2', bound=Bar)

@overload
def f(cls: Type[T1]) -> Tuple[T1]:
    pass
@overload
def f(cls: Type[T2]) -> Tuple[T2]:
    pass

def f(cls: Union[
        Type[T1],
        Type[T2],
]) -> Union[
    Tuple[T1],
    Tuple[T2],
]:
    pass

Expected Behavior

mypy succeeds with no errors.

Actual Behavior

mypy fails with

overload.py:19: error: Overloaded function implementation does not accept all possible arguments of signature 1
overload.py:19: error: Overloaded function implementation cannot produce return type of signature 1
overload.py:19: error: Overloaded function implementation does not accept all possible arguments of signature 2
overload.py:19: error: Overloaded function implementation cannot produce return type of signature 2

The same error message is printed also with the Foo/Bar bounds removed, plus an error about overlapping overloads. So I don't think the bounds are an essential part.

If I remove the Tuple wrapping the return values, it succeeds.

Your Environment

  • Mypy version used: 0.782
  • Mypy command-line flags: None
  • Mypy configuration options from mypy.ini (and other config files): None
  • Python version used: Python 3.7.8
  • Operating system and version: NixOS 19.09
@catern catern added the bug mypy got something wrong label Sep 5, 2020
@gvanrossum
Copy link
Member

Methinks (and mypy agrees :-) that the return value should be declared as Tuple[Union[T2, t2]].

If you put something in the function that returns an appropriate value (e.g. return (cls(),)) you'll see that the type of that value is a tuple of a union, not a union of tuples.

But perhaps your real-world use case was different?

@catern
Copy link
Author
catern commented Sep 5, 2020 •
edited
Loading

Yes, my real use case was with an invariant generic - in that case Generic[Union[T1, T2]] doesn't make sense (I think?). Sorry, was trying to get a minimal reproduction.

So, changing from Tuple to List, and using List[Union[T1, T2]] as the return type, the example becomes:

from typing import TypeVar, overload, Type, Tuple, Union, List

class Foo:
    pass

class Bar:
    pass

T1 = TypeVar('T1', bound=Foo)
T2 = TypeVar('T2', bound=Bar)

@overload
def f(cls: Type[T1]) -> List[T1]:
    pass
@overload
def f(cls: Type[T2]) -> List[T2]:
    pass

def f(cls: Union[
        Type[T1],
        Type[T2],
]) -> List[
    Union[T1, T2],
]:
    if isinstance(cls, Foo):
        return [cls()]
    elif isinstance(cls, Bar):
        return [cls()]
    else:
        raise Exception()

and the error is:

overload.py:19: error: Overloaded function implementation cannot satisfy signature 1 due to inconsistencies in how they use type variables
overload.py:19: error: Overloaded function implementation cannot satisfy signature 2 due to inconsistencies in how they use type variables

This code seems wrong to me though, and the error seems right, List[Union[T1, T2]] is the wrong type for variance reasons (I think?)

@catern
Copy link
Author
catern commented Sep 5, 2020

Also I should note that if I only change Tuple to List in the original reproducer, with the return type being Union[List[T1], List[T2]], the error is the same as in the original post.

@catern catern closed this as completed Sep 5, 2020
@catern catern reopened this Sep 5, 2020
@gvanrossum
Copy link
Member

Indeed, in your second example the problem is that List[Union[T1, T2]] is not a valid supertype of List[T1] or List[T2], due to invariance for mutable generic types.

So I agree there is something to your first example. What to do about it? I would just return Any from the implementation. Mypy in general does not check that an overload implementation is correct.

8000 @catern
Copy link
Author
catern commented Sep 5, 2020

Interesting, I would have assumed that mypy checks an overload implementation like a normal function, ignoring the overloads.

In any case, if I just return Any, then I get:

overload.py:19: error: Overloaded function implementation does not accept all possible arguments of signature 1
overload.py:19: error: Overloaded function implementation does not accept all possible arguments of signature 2

But if I take Any as well, ending up with

def f(cls: Any) -> Any:

Then it works fine. It seems a little unfortunate to have to do that, but at least the overloads work for the user. I might just tag it # type: ignore instead - that at least still has the type variables in scope in the body.

@mrbean-bremen mrbean-bremen mentioned this issue Jun 7, 2021
Merged
@9999years 9999years mentioned this issue Oct 26, 2021
Open
@ilevkivskyi
Copy link
Member

The original example now type-checks cleanly on master. While the second example gives:

19: error: Overloaded function implementation cannot satisfy signature 1 due to inconsistencies in how they use type variables
19: error: Overloaded function implementation cannot satisfy signature 2 due to inconsistencies in how they use type variables

I would propose to close this, we can track the second example (but also TBH it's in a gray area) in #11391

@Asaurus1 Asaurus1 mentioned this issue Jul 31, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug mypy got something wrong topic-overloads
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@JelleZijlstra @gvanrossum @catern @ilevkivskyi
0