8000 support for Awaitable objects in `asyncio.wait` removed in 3.11 · Issue #95601 · python/cpython · GitHub
[go: up one dir, main page]
Skip to content

support for Awaitable objects in asyncio.wait removed in 3.11 #95601

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
graingert opened this issue Aug 3, 2022 · 8 comments
Closed

support for Awaitable objects in asyncio.wait removed in 3.11 #95601

graingert opened this issue Aug 3, 2022 · 8 comments
Assignees
@graingert
Labels
3.10 only security fixes 3.11 only security fixes 3.12 only security fixes topic-asyncio type-bug An unexpected behavior, bug, or error

Comments

@graingert
Copy link
Contributor
graingert commented Aug 3, 2022 •
edited
Loading

on python3.10 -W error this passes:

class ExampleAwaitable:
    def __await__(self):
        async def _():
            return await asyncio.sleep(0)

        return _().__await__()


import asyncio

print(f"{asyncio.iscoroutine(ExampleAwaitable())=}")

async def amain():
    await asyncio.wait([ExampleAwaitable()])
    print("waited!")

asyncio.run(amain())

and in 3.11 this fails:

asyncio.iscoroutine(ExampleAwaitable())=False
Traceback (most recent call last):
  File "/home/graingert/projects/foo.py", line 17, in <module>
    asyncio.run(amain())
  File "/usr/lib/python3.11/asyncio/runners.py", line 187, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 120, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 650, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/graingert/projects/foo.py", line 14, in amain
    await asyncio.wait([ExampleAwaitable()])
  File "/usr/lib/python3.11/asyncio/tasks.py", line 427, in wait
    return await _wait(fs, timeout, return_when, loop)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/tasks.py", line 531, in _wait
    f.add_done_callback(_on_completion)
    ^^^^^^^^^^^^^^^^^^^
AttributeError: 'ExampleAwaitable' object has no attribute 'add_done_callback'

this broke distributed and we missed the DeprecationWarning because ensure_future accepts a wider variety of objects than just futures tasks and coroutines

dask/distributed#6785

the removal happened here 903f0a0
@graingert graingert added the type-bug An unexpected behavior, bug, or error label Aug 3, 2022
@AlexWaygood AlexWaygood added topic-asyncio 3.11 only security fixes 3.12 only security fixes labels Aug 3, 2022
@graingert
Copy link
Contributor Author
graingert commented Aug 3, 2022 •
edited
Loading

I think using something like this might help?

def _insist_future(f, loop):
    f2 = ensure_future(f, loop=loop)
    if f2 is not f:
        warnings.warn("passing non-futures is deprecated")
    return f2
    fs = set(fs)
    loop = events.get_running_loop()
    if any(coroutines.iscoroutine(f) for f in fs):
        raise TypeError("Passing coroutines is forbidden, use tasks explicitly.")
    fs = {_insist_future(f, loop=loop) for f in fs}
    return await _wait(fs, timeout, return_when, loop)

@graingert graingert changed the title support for Awaitable objects in asyncio.wait_for removed in 3.11 support for Awaitable objects in asyncio.wait removed in 3.11 Aug 3, 2022
graingert added a commit to graingert/cpython that referenced this issue Aug 5, 2022
@graingert
pythongh-95601: forbid non-futures in asyncio.wait
88580c9
@bedevere-bot bedevere-bot mentioned this issue Aug 5, 2022
Closed
@graingert graingert added the 3.10 only security fixes label Aug 9, 2022
@graingert
Copy link
Contributor Author

This was intended to be deprecated in 3.10 so I'm tentatively adding the 3.10 label

@gvanrossum
Copy link
Member

So what is the bug exactly? Was the deprecation schedule not followed properly? Or do you just disagree with the deprecation and subsequent removal? In that case, tough luck. The release manager isn't going to allow the removal to be reversed.

@graingert
Copy link
Contributor Author

So what is the bug exactly? Was the deprecation schedule not followed properly? Or do you just disagree with the deprecation and subsequent removal? In that case, tough luck. The release manager isn't going to allow the removal to be reversed.

The deprecation schedule was not followed properly. The removal didn't match the deprecation, and support for iterables was removed. There's more discussion on the PR

@gvanrossum
Copy link
Member

Now that the PR is closed, can you come up with a better description? It should describe a desired course of action.

graingert added a commit to graingert/distributed that referenced this issue Nov 3, 2022
@graingert
daskGH-6785: asyncio.wait no longer calls ensure_future
f0388ef 
python/cpython#95601
graingert added a commit to graingert/distributed that referenced this issue Nov 3, 2022
@graingert
daskGH-6785: asyncio.wait no longer calls ensure_future
ed73547 
python/cpython#95601
@consideRatio consideRatio mentioned this issue Nov 9, 2022
Closed
graingert added a commit to graingert/distributed that referenced this issue Nov 9, 2022
@graingert
daskGH-6785: asyncio.wait no longer calls ensure_future
2386513 
python/cpython#95601
graingert added a commit to graingert/distributed that referenced this issue Nov 17, 2022
@graingert
daskGH-6785: asyncio.wait no longer calls ensure_future
c4ad1ff 
python/cpython#95601
graingert added a commit to graingert/distributed that referenced this issue Nov 23, 2022
@graingert
daskGH-6785: asyncio.wait no longer calls ensure_future
81a2ed6 
python/cpython#95601
@sileht
Copy link
Contributor
sileht commented Nov 23, 2022

I upgrade some projects to 3.11 and got the same issue.

We usually fix python deprecation warning the day python released a version, but we missed that one.

The real root cause issue, I guess, is that the deprecation warning of python < 3.11 was not working for this kind of object:

class ExampleAwaitable:
    def __await__(self):
          yield from asyncio.sleep(0)

graingert added a commit to graingert/distributed that referenced this issue Nov 24, 2022
@graingert
daskGH-6785: asyncio.wait no longer calls ensure_future
17b2577 
python/cpython#95601
graingert added a commit to graingert/distributed that referenced this issue Nov 30, 2022
@graingert
daskGH-6785: asyncio.wait no longer calls ensure_future
896f770 
python/cpython#95601
graingert added a commit to dask/distributed that referenced this issue Dec 14, 2022
@graingert
Add support for Python 3.11 (#7249)
64a7cae 
* use sys.executable for python binary

* pass the desired frame to traceback.walk_stack for 3.11

see python/cpython#96092
and it's not really intended for use outside of StackSummary.extract,
see python/cpython#99015 (comment)

* the GC overhead now depends on _PyType_PreHeaderSize

faster-cpython/ideas#125
python/cpython@8319114#diff-a3a5c73931235f7f344c072dc755d6508e13923db3f5d581c5e88652075871cbR1684

* GH-6785: asyncio.wait no longer calls ensure_future

python/cpython#95601

* test on python 3.11

* exclude macos py 3.10
@kumaraditya303
Copy link
Contributor

@graingert are you going to follow up on this? If not I am inclined to just close it, 3.11 is far ahead in release cycle now anyways.

@kumaraditya303
Copy link
Contributor

Closing because of no response to my and Guido's comment.

@kumaraditya303 kumaraditya303 closed this as not planned Won't fix, can't repro, duplicate, stale Apr 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@graingert graingert

Labels
3.10 only security fixes 3.11 only security fixes 3.12 only security fixes topic-asyncio type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@sileht @graingert @gvanrossum @kumaraditya303 @AlexWaygood
0