8000 Suppress CancelledError in zwave-js unload by epenet · Pull Request #91222 · home-assistant/core · GitHub
[go: up one dir, main page]

Skip to content

Suppress CancelledError in zwave-js unload #91222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 12, 2023
Merged

Conversation

epenet
Copy link
Contributor
@epenet epenet commented Apr 11, 2023

Proposed change

Spotted in #90850

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Black (black --fast homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

@home-assistant
Copy link

Hey there @home-assistant/z-wave, mind taking a look at this pull request as it has been labeled with an integration (zwave_js) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of zwave_js can trigger bot actions by commenting:

  • @home-assistant close Closes the pull request.
  • @home-assistant rename Awesome new title Renames the pull request.
  • @home-assistant reopen Reopen the pull request.
  • @home-assistant unassign zwave_js Removes the current integration label and assignees on the pull request, add the integration domain after the command.

@@ -792,7 +793,11 @@ async def disconnect_client(hass: HomeAssistant, entry: ConfigEntry) -> None:
for task in platform_setup_tasks:
task.cancel()

await asyncio.gather(listen_task, start_client_task, *platform_setup_tasks)
tasks = (listen_task, start_client_task, *platform_setup_tasks)
await asyncio.gather(*tasks, return_exceptions=True)
Copy link
Contributor Author
@epenet epenet Apr 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If return_exceptions is not set to True, then asyncio.CancelledError is raised and other tasks are not awaited.

The change here ensures that:

  • all tasks are awaited, even if one raises an exception
  • asyncio.CancelledError exception is ignored
  • other exception are still raised if they occur - but only after all tasks have been awaited

Copy link
Member
@MartinHjelmare MartinHjelmare Apr 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs says that a task that is canceled won't stop the other tasks from being awaited in gather.

https://docs.python.org/3/library/asyncio-task.html#asyncio.gather

Copy link
Contributor Author
@epenet epenet Apr 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But whilst the other tasks can still be awaited, they are no longer tracked by the gather, won't be cancelled nor awaited by default, will continue to run and may fail in the future.

the first raised exception is immediately propagated to the task that awaits on gather(). Other awaitables in the aws sequence won’t be cancelled and will continue to run.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's more effective to store the gather task and await it in a loop while suppressing CancelledError, until the gather task is done? That way we don't need to loop over all submitted tasks again after they have been awaited with gather.

Copy link
Member
@MartinHjelmare MartinHjelmare Apr 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm interpreting the docs to say that the gather task will still track the submitted tasks if one of them raises CancelledError, since the gather task is not cancelled. Ie, we can await the gather task again if needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I just ran some tests with a couple of failing task (one after 1s, one after 2s). On first gather, it will wait 1s and then raise the exception:

  • task1 is done
  • task 2 is still running
  • gather is done

Subsequent calls to gather do not wait for the second task, but simply raise again the failure from task1

I couldn't find a way to wait for the "remaining tasks"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you want to do asyncio.wait with ALL_COMPLETED?

https://docs.python.org/3/library/asyncio-task.html#asyncio.wait

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using await asyncio.wait(tasks) is the same as asyncio.gather(*tasks, return_exceptions=True)

It comes down to error tracking:

  • do we want to ignore only CancelledError?, and raise if there was a different error?
  • or do we want to ignore all errors?

If we want to ignore all errors, then we can do away with the loop.
If we want to check which errors are not CancelledError, then we need the loop anyway.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep it like this for now. We can improve it with TaskGroup when 3.11 is minimum Python version.

https://docs.python.org/3/library/asyncio-task.html#asyncio.TaskGroup

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably log any exception with a trace.

We could log CancelledError at debug level with a trace as well in case we find a case where cancelation is happening unexpectedly... but that might be do to the cancellation semantics of asyncio.wait_for if its used anywhere in the libs which is thankfully fixed in cpython 3.12 python/cpython#98518

@epenet epenet added the smash Indicator this PR is close to finish for merging or closing label Apr 12, 2023
@epenet epenet merged commit 4e78bcb into home-assistant:dev Apr 12, 2023
@epenet epenet deleted the zwave-js branch April 12, 2023 06:09
tkhadimullin pushed a commit to tkhadimullin/ha-core that referenced this pull request Apr 13, 2023
@github-actions github-actions bot locked and limited conversation to collaborators Apr 13, 2023
Sign up for free to subscribe to this co 70D4 nversation on GitHub. Already have an account? Sign in.
Labels
bugfix cla-signed integration: zwave_js Quality Scale: No score small-pr PRs with less than 30 lines. smash Indicator this PR is close to finish for merging or closing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0