When specifying a TypeVar to be a Callable, or union of Callables, ParamSpec cannot be used to indicate parameter types #14777

AndrewGibb · 2023-02-24T16:23:48Z

Bug Report

The declarations of F, G and H all result in the following error:
The first argument to Callable must be a list of types, parameter specification, or "..." [valid-type]
Despite the first argument to the callable being a parameter specification.

from typing import Callable, ParamSpec, TypeVar, Coroutine, Any, Union

P = ParamSpec('P')
F = TypeVar('F', bound=Callable[P, Any])
G = TypeVar('G', bound=Callable[P, Coroutine[Any, Any, Any]])
H = TypeVar('H', bound=Union[Callable[P, Any], Callable[P, Coroutine[Any, Any, Any]]])

To Reproduce
https://mypy-play.net/?mypy=latest&python=3.11&gist=191e47b4afd74e307a3042dfedac77d4

Expected Behavior

Either these should pass type checking, or the error message should be updated to indicate what the problem actually is.

Your Environment

Mypy version used: 1.0.0
Python version used: 3.11

The text was updated successfully, but these errors were encountered:

erictraut · 2023-02-24T17:06:27Z

The bounds of a TypeVar are not allowed to be generic, so they cannot include TypeVars, TypeVarTuples, or ParamSpecs. Mypy's error message could perhaps be clearer in this case.

AndrewGibb · 2023-02-27T09:51:29Z

The motivating problem behind the bug report looks like this:

from typing import Callable, TypeVar, Coroutine, Any, Union, cast, ParamSpec
import time
import asyncio
import functools

P = ParamSpec('P')
R = TypeVar('R')
F = TypeVar('F', bound=Union[Callable[P, R], Callable[P, Coroutine[Any, Any, R]]])

def retry_forever() -> Callable[[F], F]:
  def __decorator(f: F) -> F:
    if asyncio.iscoroutinefunction(f):
      @functools.wraps(f)
      async def __inner(*args: P.args, **kwargs: P.kwargs) -> R:
        while True:
          try:
            return await f(*args, **kwargs)
          except Exception:
            await asyncio.sleep(1)
      return __inner
    else:
      @functools.wraps(f)
      def __inner(*args: P.args, **kwargs: P.kwargs) -> R:
        while True:
          try:
            return f(*args, **kwargs)
          except Exception:
            time.sleep(1)
      return __inner
  return __decorator

Obviously this is a trivial example, but the structure is real and is taken from our codebase. This Bug report is notthe only problem MyPy has with this code at the moment. But I hope you'd agree this is more specific (and useful, to a sufficiently powerful static analyser) than omitting P (and R) as one must do at the moment.

Fixes #14832, fixes #13966, fixes #14622. Still does not report error in #14777, I'll work separately on that. Move all `ParamSpec` validity checking to `typeanal.py`. Stop treating `P.args` and `P.kwargs` as binding - only bare typevar makes it available in scope. Reject keyword arguments following `P.args`. This also makes one more conformance test pass.

sterliakov · 2025-03-30T01:46:16Z

This is still wrong - there is no way to "carry on" some type variables bound to another TypeVar. HKTs do not exist in python (yet?). This F definition is explicitly prohibited by typing spec. Furthermore, F -> F signature is not strictly correct: I could retry_forever an instance of some callable class but receive a function instead, so it does not preserve input type.

Yes, typing a decorator that supports both sync and async interfaces is challenging (impossible to achieve without ignore/cast AFAIC). You can't unwrap Awaitable[R] when in union, and asyncio.iscoroutinefunction is not even a TypeIs. The most readable (IMO) is a simple solution with two casts both baked by manual verification that you didn't swap sync and async branches:

import time
import asyncio
import functools
from typing import Any, Awaitable, Callable, ParamSpec, Protocol, TypeVar, cast

P = ParamSpec('P')
R = TypeVar('R')

class Retrier(Protocol):
    def __call__(self, fn: Callable[P, R], /) -> Callable[P, R]: ...
    
def retry_forever() -> Retrier:
    
    def __decorator(f: Callable[P, R]) -> Callable[P, R]:
        if asyncio.iscoroutinefunction(f):
            
            @functools.wraps(f)
            async def __ainner(*args: P.args, **kwargs: P.kwargs) -> R:
                while True:
                    try:
                        return await cast("Callable[P, Awaitable[R]]", f)(*args, **kwargs)
                    except Exception:
                        await asyncio.sleep(1)
                        
            return cast("Callable[P, R]", __ainner)
        else:
            
            @functools.wraps(f)
            def __inner(*args: P.args, **kwargs: P.kwargs) -> R:
                while True:
                    try:
                        return f(*args, **kwargs)
                    except Exception:
                        time.sleep(1)
                        
            return __inner
            
    return __decorator

But this isn't a mypy bug by any means.

AndrewGibb added the bug mypy got something wrong label Feb 24, 2023

AlexWaygood added topic-usability topic-paramspec PEP 612, ParamSpec, Concatenate labels Feb 24, 2023

sterliakov mentioned this issue Dec 11, 2024

Reject invalid ParamSpec locations #18278

Merged

sterliakov closed this as completed Mar 30, 2025

sterliakov closed this as not planned Won't fix, can't repro, duplicate, stale Mar 30, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

When specifying a TypeVar to be a Callable, or union of Callables, ParamSpec cannot be used to indicate parameter types #14777

When specifying a TypeVar to be a Callable, or union of Callables, ParamSpec cannot be used to indicate parameter types #14777

When specifying a TypeVar to be a Callable, or union of Callables, ParamSpec cannot be used to indicate parameter types #14777

When specifying a TypeVar to be a Callable, or union of Callables, ParamSpec cannot be used to indicate parameter types #14777

Comments