8000 shielded task exception never retrieved when outer task cancelled · Issue #86802 · python/cpython · GitHub
[go: up one dir, main page]
Skip to content

shielded task exception never retrieved when outer task cancelled #86802

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
natano mannequin opened this issue Dec 14, 2020 · 4 comments
Closed

shielded task exception never retrieved when outer task cancelled #86802

natano mannequin opened this issue Dec 14, 2020 · 4 comments
Labels
stdlib Python modules in the Lib dir topic-asyncio type-bug An unexpected behavior, bug, or error

Comments

@natano
Copy link
Mannequin
natano mannequin commented Dec 14, 2020 •
edited by bedevere-app bot
Loading
BPO 42636
Nosy @asvetlov, @1st1, @natano
Files
  • trigger-warning.py: Short script for showing the issue
  • regression-test.patch: Regression test
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2020-12-14.10:36:21.291>
labels = ['3.7', '3.8', '3.9', '3.10', 'expert-asyncio']
title = 'shielded task exception never retrieved when outer task cancelled'
updated_at = <Date 2020-12-15.09:44:41.561>
user = 'https://github.com/natano'

    bugs.python.org fields:

    activity = <Date 2020-12-15.09:44:41.561>
actor = 'natano'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2020-12-14.10:36:21.291>
creator = 'natano'
dependencies = []
files = ['49679', '49680']
hgrepos = []
issue_num = 42636
keywords = ['patch']
message_count = 4.0
messages = ['382975', '382976', '382979', '383039']
nosy_count = 3.0
nosy_names = ['asvetlov', 'yselivanov', 'natano']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue42636'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

    Linked PRs
    @natano
    Copy link
    Mannequin Author
    natano mannequin commented Dec 14, 2020

    A task created with asyncio.shield() never retrieves the task exception, which results in a log message being generated. See the attached script for a minimal example. Output looks something like this:

    <snip>
Task exception was never retrieved
future: <Task finished name='Task-2' coro=<some_coroutine() done, defined at /Users/natano/python-bug/trigger-warning.py:4> exception=Exception('foo')>
Traceback (most recent call last):
  File "/Users/natano/python-bug/trigger-warning.py", line 6, in some_coroutine
    raise Exception('foo')
Exception: foo
</snip>

    I believe this behaviour is not intended and is an unintended side-effect of commit b35acc5 (b35acc5). The _inner_done_callback has has code to retrieve the exception when the outer task was cancelled, but since that commit the _inner_done_callback is not called anymore when the outer task was cancelled.

    (Tested with 3.8, 3.9 and 3.10. I don't have a python 3.7 version available for testing, but I think this issue also affects 3.7.)

    @natano natano mannequin added 3.7 (EOL) end of life 3.8 (EOL) end of life 3.9 only security fixes 3.10 only security fixes topic-asyncio labels Dec 14, 2020
    @natano
    Copy link
    Mannequin Author
    natano mannequin commented Dec 14, 2020

    I'm not sure how to fix this, but here is a regression test for the issue. (failing on the current master branch)

    @asvetlov
    Copy link
    Contributor

    Thanks for the report.

    I didn't test but looks like outer.exception() call can suppress the message about not retrieved exception.

    @natano
    Copy link
    Mannequin Author
    natano mannequin commented Dec 15, 2020

    How would calling outer.exception() suppress the message? This happens when the outer task is cancelled while the inner task is still running. In that case the exception is never set on outer.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @ezio-melotti ezio-melotti moved this to Todo in asyncio Jul 17, 2022
    @kumaraditya303 kumaraditya303 removed 3.10 only security fixes 3.9 only security fixes 3.8 (EOL) end of life 3.7 (EOL) end of life labels Oct 18, 2022
    ChristianHrs added a commit to ChristianHrs/cpython that referenced this issue May 20, 2025
    @ChristianHrs
    pythongh-86802: Fix asyncio memory leak; shielded tasks where cancell…
    ab8786a 
    …ed log once through the exception handler
    @bedevere-app bedevere-app bot mentioned this issue May 20, 2025
    Merged
    ambv added a commit that referenced this issue May 20, 2025
    @ChristianHrs @ambv
    gh-86802: Fix asyncio memory leak; shielded task exceptions log once …
    f695eca 
    …through the exception handler (gh-134331)

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
    miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 20, 2025
    @ChristianHrs @ambv @miss-islington
    pythongh-86802: Fix asyncio memory leak; shielded task exceptions log…
    999bad0 
    … once through the exception handler (pythongh-134331)

(cherry picked from commit f695eca)

Co-authored-by: Christian Harries <68507104+ChristianHrs@users.noreply.github.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
    @bedevere-app bedevere-app bot mentioned this issue May 20, 2025
    Merged
    @picnixz picnixz added type-bug An unexpected behavior, bug, or error stdlib Python modules in the Lib dir labels May 20, 2025
    ambv added a commit that referenced this issue May 20, 2025
    @miss-islington @ChristianHrs @ambv
    [3.14] gh-86802: Fix asyncio memory leak; shielded task exceptions lo…
    efdaae5 
    …g once through the exception handler (gh-134331) (gh-134343)

(cherry picked from commit f695eca)

Co-authored-by: Christian Harries <68507104+ChristianHrs@users.noreply.github.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
    @ambv ambv closed this as completed May 20, 2025
    @github-project-automation github-project-automation bot moved this from Todo to Done in asyncio May 20, 2025
    ChristianHrs added a commit to ChristianHrs/cpython that referenced this issue May 20, 2025
    @ChristianHrs
    Revert "pythongh-86802: Fix asyncio memory leak; shielded task except…
    5893d43 
    …ions log once through the exception handler (pythongh-134331)"

This reverts commit f695eca.
    ChristianHrs added a commit to ChristianHrs/cpython that referenced this issue May 20, 2025
    @ChristianHrs
    Merge branch 'pythongh-86802' into pythongh-90871
    e871568
    ChristianHrs added a commit to ChristianHrs/cpython that referenced this issue May 21, 2025
    @ChristianHrs
    Revert "pythongh-86802: Fix asyncio memory leak; shielded task except…
    8c86da6 
    …ions log once through the exception handler (pythongh-134331)"

This reverts commit f695eca.
    7013
    lkollar pushed a commit to lkollar/cpython that referenced this issue May 26, 2025
    @ChristianHrs @ambv @lkollar
    pythongh-86802: Fix asyncio memory leak; shielded task exceptions log…
    ef96a80 
    … once through the exception handler (pythongh-134331)

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    stdlib Python modules in the Lib dir topic-asyncio type-bug An unexpected behavior, bug, or error
    Projects
    asyncio
    Status: Done
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @ambv @asvetlov @picnixz @kumaraditya303
    0