10000 Support PEP 612 in typing_extensions (Python 3) by Fidget-Spinner · Pull Request #774 · python/typing · GitHub
[go: up one dir, main page]

Skip to content

Support PEP 612 in typing_extensions (Python 3) #774

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Apr 10, 2021

Conversation

Fidget-Spinner
Copy link
Member

Runtime correctness is zero due to wacky list workarounds. Unfortunately, I'm not enough of an MRO/metaclass wizard to know how to trick isinstance(arg, list) in Callable while still returning a typing._GenericAlias.

The first patch is for Python3. I'm mildly reluctant to write one for Python2 since it's EOL-ed for a year already, but we'll see ;).

@Fidget-Spinner Fidget-Spinner changed the title Support PEP 612 for typing_extensions in Python 3 Support PEP 612 in typing_extensions (Python 3) Jan 2, 2021
@Fidget-Spinner
Copy link
Member Author

@gvanrossum Would you like to see Python 2 supported as well? It's possible to have runtime correctness for Python 2 since I can edit the typing PyPI module as well, though that would require more work.

@gvanrossum
Copy link
Member

Let’s skip Python 2. Sorry for the slow review, will get to it soon.

@Fidget-Spinner
Copy link
Member Author
Fidget-Spinner commented Jan 11, 2021

Let’s skip Python 2. Sorry for the slow review, will get to it soon.

No worries, I didn't mean to rush and I hope it didn't come off that way. Please take your time :).

@gvanrossum
Copy link
Member
gvanrossum commented Jan 11, 2021 via email

@gvanrossum
Copy link
Member

@JelleZijlstra Have you reviewed this? I have sadly dropped the ball (again). If you approve I will land it.

Copy link
Member
@JelleZijlstra JelleZijlstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for letting this linger! I took a look at the code and I have some suggested improvements.

@JelleZijlstra
Copy link
Member

Also, I noticed we're not running 3.9 in CI, so I opened #796 to add it.

@Fidget-Spinner Fidget-Spinner marked this pull request as draft April 6, 2021 15:23
@Fidget-Spinner
Copy link
Member Author

@JelleZijlstra - Thanks! I really appreciate the reviews. I marked the PR as draft because you gave me some other ideas to improve the runtime information. So I'm working on them now.

This PR was originally just to implement all the basic stuff required for type checkers, which is why it's so barren in terms of runtime information.

@Fidget-Spinner Fidget-Spinner marked this pull request as ready for review April 6, 2021 16:54
@Fidget-Spinner
Copy link
Member Author

@JelleZijlstra this should be ready now. Please take a look when you have the time. Thanks!

Copy link
Member
@JelleZijlstra JelleZijlstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I checked out the code and played around with it some more and I have two suggestions to make the new types a bit more usable at runtime.

Note that only parameter specification variables defined in global scope can
be pickled.
"""
args = object()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit unhappy with this because it makes the runtime annotations just look like this:

In [11]: takes_int_str.__annotations__
Out[11]: 
{'args': <object at 0x7ff1831d5f30>,
 'kwargs': <object at 0x7ff1831d5f60>,
 'return': ~R}

However, it's the exact same in CPython, so it makes sense to follow the same principle here. I'll plan to submit a change to CPython to make this instead return some helper object pointing back to the ParamSpec; if that gets accepted we can apply the same change here.

For now no change is required here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was definitely something I overlooked when I implemented the CPython version :(. How I'd do it in typing.py:

  1. Create a _ParamSpecGenericAlias.
  2. __origin__ is the ParamSpec it belongs to, __args__ is just a tuple with 1 element - "args" or "kwargs".
  3. Override the __repr__ to print ~T.(args|kwargs) instead of the default ~T[args] found in genericaliases.

I look forward to seeing your PR!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I was thinking something similar, except with different classes for args and kwargs, but either way should work.

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
@JelleZijlstra
Copy link
Member

Looks like this broke CI because ParamSpec objects aren't hashable. I'm guessing we could make them hashable by overriding __hash__ and hoping that nobody will actually modify the underlying list.

@Fidget-Spinner
Copy link
Member Author

Hmm the maximum recursion depth exceeded errors due to _tp_cache in _concatenate_getitem are particularly hairy.

@JelleZijlstra
Copy link
Member

Hmm the maximum recursion depth exceeded errors due to _tp_cache in _concatenate_getitem are particularly hairy.

It reproduces only if some other test runs before it, not if you just run the line that fails on its own. I can spend some time later today trying to figure out where it's going wrong.

@JelleZijlstra
Copy link
Member

I tracked it down to ParamSpec("P") == ParamSpec("P2") failing. It works if you add def __eq__(self, other): return self is other. Comparing by identity seems reasonable to me because that's also how TypeVars work: TypeVar("T") == TypeVar("T") returns False.

@Fidget-Spinner
Copy link
Member Author

I tracked it down to ParamSpec("P") == ParamSpec("P2") failing. It works if you add def __eq__(self, other): return self is other. Comparing by identity seems reasonable to me because that's also how TypeVars work: TypeVar("T") == TypeVar("T") returns False.

Thank you for digging out the cause. I had forgotten that dictionary keys need to implement both __hash__ and __eq__ to work properly. This was a good refresher.

Copy link
Member
@JelleZijlstra JelleZijlstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Could you also add some unit tests doing == and hash() on ParamSpec and Concatenate? We should make sure they keep working if we ever refactor these classes here. With those changes this should be good to go.

@Fidget-Spinner
Copy link
Member Author

Thanks! Could you also add some unit tests doing == and hash() on ParamSpec and Concatenate? We should make sure they keep working if we ever refactor these classes here. With those changes this should be good to go.

Thanks for the reminder, that slipped my mind.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
0