Base concurrent.futures.Executor on ContextManager #1711
Conversation
Update the concurrent.futures so that Executor, ThreadPoolExecutor, and ProcessPoolExecutor inherit from the typing.ContextManager[T] generic introduced in Python 3.6. Specifically, this solves a problem in code such as: with ThreadPoolExecutor() as p: ... Where the p variable would be typed as an abstract `Executor`, rather than the specific `ThreadPoolExecutor` as expected.
| def submit(self, fn: Callable[..., _T], *args: Any, **kwargs: Any) -> Future[_T]: ... | ||
| def map(self, func: Callable[..., _T], *iterables: Iterable[Any], timeout: Optional[float] = ..., chunksize: int = ...) -> Iterator[_T]: ... | ||
| def shutdown(self, wait: bool = ...) -> None: ... | ||
| def __enter__(self) -> Executor: ... |
There was a problem hiding this comment.
Why can't you achieve what you want with the simpler solution of typing __enter__ as def __enter__(self: _T) -> _T: ..., where _T is a TypeVar bound to Executor? That would make the rest of this PR unnecessary and remove the version_info checks.
There was a problem hiding this comment.
That would work, too.
I didn't do it that way because of the note here about generic selfs being experimental and not fully implemented.
But if you think that's the way forward I can rework it like that.
There was a problem hiding this comment.
But self is not generic here, is it? Executor itself is not a generic class.
There was a problem hiding this comment.
I just meant "generic self" to mean that self has a TypeVar type (which I believe is what the note is describing).
According to the note, I don't think it matters if the class (Executor) that gets bound to the TypeVar is itself a 'generic class'.
See the example, none of the Shape, Circle, or Square are generic classes either.
There was a problem hiding this comment.
My understanding is that self-types are fairly stable for non-generic classes (like the ones we're using here). The troubles arise when the class we're using a self type for is itself generic: see python/mypy#2354, python/mypy#3363, python/mypy#3631.
There was a problem hiding this comment.
In any case, self types are already widely used in typeshed for non-generic classes (grep finds about 130 usages). I'm confident that using them here too is safe.
There was a problem hiding this comment.
Okay, then! I'll rework this that way when I get a moment.
Make the Python 2 and 3 concurrent.futures stubs identical so fixes get applied to both. For example, python#1305 and python#2233 fixed the same problem at differnt times, as did python#1078 and python#1911. By making the stubs identical, we apply the fix from python#1711 to Python 2. Fixes python#2234.
Make the Python 2 and 3 concurrent.futures stubs identical so fixes get applied to both. For example, python#1305 and python#2233 fixed the same problem at different times, as did python#1078 and python#1911. By making the stubs identical, we apply the fix from python#1711 to Python 2. Fixes python#2234.
Make the Python 2 and 3 concurrent.futures stubs identical so fixes get applied to both. For example, python#1305 and python#2233 fixed the same problem at different times, as did python#1078 and python#1911. By making the stubs identical, we apply the fix from python#1711 to Python 2. Fixes python#2234.
Update the
concurrent.futuresso thatExecutor,ThreadPoolExecutor, andProcessPoolExecutorinherit from thetyping.ContextManager[T]generic introduced in Python 3.6.Specifically, this solves a problem in code such as:
Where the p variable would be typed as an abstract
concurrent.futures._base.Executor, rather than the specificThreadPoolExecutoras expected.