Add better types for asyncio.gather #9678

Gobot1234 · 2023-02-04T19:35:36Z

Before:

After:

Gobot1234 · 2023-02-04T21:59:38Z

Looking at #9105 I don't think the case presented where you have vargs of different types actually happens that much whereas unpacking as shown in the screenshots above is much more common and helps catch errors that would have gone amiss.

AlexWaygood · 2023-02-05T16:23:08Z

This looks promising.

The homeassistant hit is here: https://github.com/home-assistant/core/blob/04e05af44cbea2156d904d0d34e4e0d41c9725c1/homeassistant/components/amcrest/camera.py#L404-L418. They just miss out on the precise overloads, since they're passing in 6 coroutines. Previously the return type of the call to asyncio.gather was inferred as list[Any]; with this change, it is inferred as list[object]. That's probably not too terrible; and we could probably get rid of the hit by just adding an overload for if you pass in six coroutines.

Some of the test cases in https://github.com/python/typeshed/blob/main/test_cases/stdlib/asyncio/check_gather.py are failing, and will need to be updated.

Lastly, as well as adding the new overload, we should probably also update the existing fallback overloads here to be more precise as well:

typeshed/stdlib/asyncio/tasks.pyi

Lines 142 to 143 in cd5f976

    
           @overload 
        
           def gather(*coros_or_futures: _FutureLike[Any], return_exceptions: bool = False) -> Future[list[Any]]: ...  # type: ignore[misc]

typeshed/stdlib/asyncio/tasks.pyi

Lines 232 to 235 in cd5f976

    
           @overload 
        
           def gather(  # type: ignore[misc] 
        
               *coros_or_futures: _FutureLike[Any], loop: AbstractEventLoop | None = None, return_exceptions: bool = False 
        
           ) -> Future[list[Any]]: ...

They should probably be changed to:

@overload
def gather(*coros_or_futures: _FutureLike[_T], return_exceptions: bool) -> Future[list[_T | BaseException]]: ...  # type: ignore[misc]

Gobot1234 · 2023-02-05T20:07:03Z

home-assistant/core@04e05af/homeassistant/components/amcrest/camera.py#L404-L418 would be inferred as list[str | bool] but I guess that can be solved as you said by adding another overload.

Gobot1234 · 2023-02-11T21:09:58Z

I don't think this is gonna work (with pyright at least) cause adding a special case for the 0 arg call (which is required to avoid the return of await asyncio.gather() being list[BaseException], which thinking about it isn't technically wrong because it will always be empty) is preferred by asyncio.gather(*iterable) making the return for that list[BaseException].

Gobot1234 · 2023-04-22T10:11:40Z

All of the failures look like mypy bugs and the failure in pyright is IMO acceptable I just don't know what the test needs to be changed to to pass

Gobot1234 · 2023-10-02T12:02:03Z

Hmm the paasta things seem like they do now break which is annoying. I'm not entirely sure what's wrong there though.

Gobot1234 · 2023-10-02T17:54:45Z

This seems to me like it should now work, ~~I'm slightly confused by the behaviour I'm seeing by mypy and pyright here, unpacking a list seems to only give one element?~~

srittau

LGTM

AlexWaygood

This looks great, thanks for persevering with this! Pushed one more test just to be safe.

github-actions · 2023-10-04T09:12:51Z

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉