[question] Overload with return type depending on an optional argument #5621

zero323 · 2018-09-15T15:28:24Z

i am working on annotation for a 3rd package. For simplicity let's say we have a function like this one:

def foo(x=0, f=None):
    return f(x) if f else x + 1

I would like to express, that the return type of the function is

int if f is missing.
The same as f return type otherwise.

So conceptually I would like to see something like this:

from typing import Callable, TypeVar, overload

T = TypeVar("T")

@overload
def foo(x: int = ..., f: Callable[[int], T]) -> T: ...
@overload
def foo(x: int = ...) -> int: ...

Of course, it is not valid, as non-default argument follows default argument. Similarly (as imprecise as it would be) we cannot use the following

from typing import Callable, TypeVar
from typing import overload

T = TypeVar("T")

@overload
def foo(x: int = ..., f: Callable[[int], T] = ...) -> T: ...
@overload
def foo(x: int = ...) -> int: ...

because the optional argument, makes the first signature broader than the second one.

A workaround is to split the first signature into two separate cases:

from typing import Callable, TypeVar
from typing import overload

T = TypeVar("T")

@overload
def foo(x: int, f: Callable[[int], T] = ...) -> T: ...
@overload
def foo(f: Callable[[int], T] = ...) -> T: ...
@overload
def foo(x: int = ...) -> int: ...

but it doesn't scale when the number of preceding optional arguments grows:

def bar(x=0,  y=0, z=0, f=None):
     return f(x + y + z) if f else x + y + z + 1

Is there any other solution here?

The text was updated successfully, but these errors were encountered:

ilevkivskyi · 2018-09-15T15:48:31Z

Is there any other solution here?

I think no. If it would be your code I would just recommend to move f to the first position. Unfortunately some existing APIs were designed without any thinking about types and are now hard to change (backwards compatibility etc.)

In certain cases it is possible to use *args, but it looks like it will be imprecise in this situation.

Michael0x2a · 2018-09-15T16:05:42Z

One option might be to use three overloads instead of two:

from typing import Callable, TypeVar, overload

T = TypeVar("T")

@overload
def foo(f: Callable[[int], T]) -> T: ...
@overload
def foo(x: int, f: Callable[[int], T]) -> T: ...
@overload
def foo(x: int = 0) -> int: ...
def foo(x, f):
    pass

reveal_type(foo(3))                     # int
reveal_type(foo(3, lambda x: "asdf"))   # str
reveal_type(foo(lambda x: "asdf"))      # str

In short, you split up the overload variant handling the callable into two -- one to handle the case where x is missing and the other for when x is present.

zero323 · 2018-09-15T17:28:34Z

@ilevkivskyi Than you for your prompt response. I was afraid this is the case.

@Michael0x2a I thought of that, but the number of required signatures grows exponentially in terms of the number of preceding optional args. In reality I deal with ~4 other args - feasible but hard to maintain.

AlexWaygood · 2022-04-02T16:31:12Z

I'm closing this issue, as I think the question has been answered.

I'm sorry that the type system isn't as expressive as we'd like it to be in this case. If you'd like to make a feature request to expand the typing system, you should post in the https://github.com/python/typing issue tracker or the typing-sig mailing list.

ilevkivskyi added the question label Sep 15, 2018

pmhahn mentioned this issue Oct 28, 2018

yaml.dump(..., stream) is optional python/typeshed#2289

Merged

AlexWaygood added the topic-overloads label Apr 2, 2022

AlexWaygood closed this as completed Apr 2, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[question] Overload with return type depending on an optional argument #5621

[question] Overload with return type depending on an optional argument #5621

[question] Overload with return type depending on an optional argument #5621

[question] Overload with return type depending on an optional argument #5621

Comments