8000 Trio102: await inside finally needs shielding and timeout by jakkdl · Pull Request #5 · python-trio/flake8-async · GitHub
[go: up one dir, main page]

Skip to content

Trio102: await inside finally needs shielding and timeout #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jul 27, 2022

Conversation

jakkdl
Copy link
Member
@jakkdl jakkdl commented Jul 19, 2022

A first draft of trio102. check_for_102 is kind of a mess, with having to inspect a lot of places to find cancel scopes, if they have a timeout specified, and if shielding is set. (It appears there was a shield parameter in the past that now is gone?). It might be a bit defensive and checking the parameter of the trio call is probably silly since they require one.

It also fails on fuzzing site code, since they do have await's in finally's and this doesn't care if they're related much to trio at all. Maybe add a global check that the file has to import or mention trio?

trio102.py has a fun case on line 39 (that will fail on python3.8 atm) that idk if it's worth caring about?

And it adds duplicates and is generally kinda ugly, but thought I'd push a first draft to see if I've understood the check correctly and so.

@jakkdl jakkdl marked this pull request as draft July 19, 2022 17:21
@jakkdl
Copy link
Member Author
jakkdl commented Jul 19, 2022

I'll merge in main to this when you've unbroken it :P
Upon seeing trio.CancelScope I'll have to modify this one a bit

@Zac-HD
Copy link
Member
Zac-HD commented Jul 19, 2022

Fixed main, and left you a review on the other PR 😁

@jakkdl
Copy link
Member Author
jakkdl commented Jul 20, 2022

for the shield value, I'll parse constant bools - but do you want to assume True or False for anything else? E.g.

        with trio.CancelScope(deadline=30, shield=(1==0)):
            await foo()
        myvar = False
        with trio.open_nursery(10) as s:
            s.shield = myvar
            await foo()

@Zac-HD
Copy link
Member
Zac-HD commented Jul 20, 2022

for the shield value, I'll parse constant bools - but do you want to assume True or False for anything else?

If it's inside a finally: block, then it must always be shielded, so I'd want an error on anything that isn't literally True. No need for a variable when you should always have exactly the same value!

I'd even be happy to go a step further and say that should be the first statement of the fail/move_on blocks (or passed as an argument to CancelScope), and should not otherwise be assigned to at all.

@jakkdl
Copy link
Member Author
jakkdl commented Jul 22, 2022

for the shield value, I'll parse constant bools - but do you want to assume True or False for anything else?

If it's inside a finally: block, then it must always be shielded, so I'd want an error on anything that isn't literally True. No need for a variable when you should always have exactly the same value!

I'd even be happy to go a step further and say that should be the first statement of the fail/move_on blocks (or passed as an argument to CancelScope), and should not otherwise be assigned to at all.

I think I'll pass on enforcing it to be the first statement unless you insist, I think current solution is pretty clean.

@jakkdl
Copy link
Member Author
jakkdl commented Jul 22, 2022

Big rewrite, I suspect you might dislike the added class but I I did not see a clean simpler solution.
All tests fails due to coverage ... which I don't understand and failed to alleviate - would love a tip on what's going on there.

I'm not loving the four separate class variables for tracking state, and them working differently: yield_is_error has the logic in the with&def, while the other three are facts about the state with logic at the point of adding the problem. I think I prefer the latter way with added reuse of the variables between checks, and if you agree I might convert the former.

Will add lineno reference to the finally (and probs fix trio101 at the same time). Could possibly give another reference to the scope if it e.g. has a timeout but no shield.

@jakkdl
Copy link
Member Author
jakkdl commented Jul 22, 2022

Oh, is this fine?

finally:
  with trio.CancelScope(deadline=10, shield=True):
    with trio.fail_after(5):
      await foo()

@Zac-HD
Copy link
Member
Zac-HD commented Jul 22, 2022

Oh, is this fine?

finally:
  with trio.CancelScope(deadline=10, shield=True):
    with trio.fail_after(5):
      await foo()

Yep, that's fine - the outer shield protects it from the Cancelled exception that (might have) triggered the finally clause, and there's a finite deadline 👍

Copy link
Member
@Zac-HD Zac-HD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're right that there's no clean and simple solution for this - and I think the solution you've got looks pretty good. Some detail-level comments below but at a high level I like it.

For the yield_is_error thing, sure, convert it if you wish, but let's do that in a subsequent PR? I've seen too many new features get delayed by scope-creep 😁

@jakkdl
Copy link
Member Author
jakkdl commented Jul 25, 2022

This should error, right?

with trio.CancelScope(deadline=10, shield=True):
  try:
    pass
  finally:
    await foo()

such that I should reset the scope whenever entering the finally

@Zac-HD
Copy link
Member
Zac-HD commented Jul 25, 2022

This should error, right?

with trio.CancelScope(deadline=10, shield=True):
  try:
    pass
  finally:
    await foo()

Correct, it should error, because if pass happened to raise Cancelled (😨) then await foo() would fail.

@jakkdl
Copy link
Member Author
jakkdl commented Jul 26, 2022

Oh, is this fine?

finally:
  with trio.CancelScope(deadline=10, shield=True):
    with trio.fail_after(5):
      await foo()

Yep, that's fine - the outer shield protects it from the Cancelled exception that (might have) triggered the finally clause, and there's a finite deadline +1

Hmmm, I think that means I need to make scope a stack (which is saved away upon entering finally) and when await'ing search through the stack to see if any of them have shield+timeout.

This check certainly isn't getting simpler!

jakkdl added 2 commits July 26, 2022 14:12
…at later), raise error on async for and async with, self._scope[s] is now a stack, scopes are reset on entering the finally, fixed coverage issue (and related bug). And added tests for all of the above, as well as separating out py39+ tests into it's own file
@jakkdl
Copy link
Member Author
jakkdl commented Jul 26, 2022

Fixed all the comments, and I broke out 102 on an impulse. As you said we can revisit that later.
I also got rid of visit_generic_with and visit_generic_functiondef in favor of visit_AsyncWith calling visit_With. felt cleaner /shrug

The finally doesn't have it's own lineno, do you want me to check the lineno of the first statement in the finalbody and subtract 1 ... or maybe just skip referring to the line number of the finally.

@jakkdl jakkdl marked this pull request as ready for review July 26, 2022 12:51
@Zac-HD
Copy link
Member
Zac-HD commented Jul 27, 2022

The finally doesn't have it's own lineno, do you want me to check the lineno of the first statement in the finalbody and subtract 1 ... or maybe just skip referring to the line number of the finally.

Let's just skip referring to the lineno of finally:, it should be pretty obvious from context.

Copy link
Member
@Zac-HD Zac-HD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

Couple of copyedit-level nitpicks below, but I'm planning to fix those and release later this evening so you can move on to the next meaningful check 🎉

@Zac-HD Zac-HD mentioned this pull request Jul 27, 2022
12 tasks
@Zac-HD Zac-HD merged commit b10621e into python-trio:main Jul 27, 2022
@jakkdl jakkdl deleted the trio102 branch December 9, 2022 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
0