8000 Avoid eagerly evaluating annotations · Issue #132493 · python/cpython · GitHub
[go: up one dir, main page]
Skip to content

Avoid eagerly evaluating annotations #132493

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
JelleZijlstra opened this issue Apr 14, 2025 · 1 comment
Open

Avoid eagerly evaluating annotations #132493

JelleZijlstra opened this issue Apr 14, 2025 · 1 comment
Labels
3.14 bugs and security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@JelleZijlstra
Copy link
Member
JelleZijlstra commented Apr 14, 2025 •
edited by bedevere-app bot
Loading

There are various places in the stdlib where we access the .__annotations__ attribute. Under PEP-649 / PEP-749, this now forces evaluation of lazy annotations. We should avoid evaluating annotations where it is not necessary.

For example, currently typing.Protocol evaluates its annotations at protocol creation time.

Linked PRs
@JelleZijlstra JelleZijlstra added the 3.14 bugs and security fixes label Apr 14, 2025
JelleZijlstra added a commit to JelleZijlstra/cpython that referenced this issue Apr 14, 2025
@JelleZijlstra
pythongh-132493: Support deferred annotations in Protocols
758fdb0
@bedevere-app bedevere-app bot mentioned this issue Apr 14, 2025
Merged
@JelleZijlstra
Copy link
Member Author

Other cases from a quick audit:

  • typing._proto_hook eagerly evaluates annotations. Looks like this could trigger on code like isinstance(X, SomeProtocol) where X has deferred annotations. It doesn't need to know the value of the annotations, only what keys exist.
  • inspect._signature_is_functionlike looks for the presence of annotations to decide that something is functionlike. Maybe it should look for __annotate__ instead?
  • pydoc in two places looks for the presence of annotations for something with lambdas that I haven't fully deciphered
  • reprlib.recursive_repr creates a wrapper function that inherits __annotations__. It should probably use __annotate__ instead.

@picnixz picnixz added type-feature A feature request or enhancement performance Performance or resource usage stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error and removed type-feature A feature request or enhancement labels Apr 14, 2025
@JelleZijlstra JelleZijlstra removed the performance Performance or resource usage label Apr 14, 2025
felixscherz added a commit to felixscherz/cpython that referenced this issue Apr 14, 2025
@felixscherz
pythongh-132493: enable protocol isinstance checks against classes wi…
5e91b7d 
…th deferred annotations
felixscherz added a commit to felixscherz/cpython that referenced this issue Apr 14, 2025
@felixscherz
pythongh-132493: enable protocol isinstance checks against classes wi…
34c6039 
…th deferred annotations
@felixscherz felixscherz mentioned this issue Apr 14, 2025
Merged
JelleZijlstra added a commit that referenced this issue Apr 15, 2025
@JelleZijlstra
gh-132493: Support deferred annotations in Protocols (#132494)
666eeda
JelleZijlstra added a commit to JelleZijlstra/cpython that referenced this issue Apr 16, 2025
@JelleZijlstra
pythongh-132493: Lazily determine protocol attributes
1a970b3 
pythongh-132494 made typing.py eagerly import annotationlib again because
typing contains several protocols. Avoid this by determining annotations
lazily. This should also make protocol creation faster:

Unpatched:

$ ./python.exe -m timeit -s 'from typing import Protocol, runtime_checkable' '''@runtime_checkable
class MyProtocol(Protocol):
    def meth(self): pass
'''
50000 loops, best of 5: 9.28 usec per loop
$ ./python.exe -m timeit -s 'from typing import Protocol, runtime_checkable' '''class MyProtocol(Protocol):
    def meth(self): pass
'''
50000 loops, best of 5: 9.05 usec per loop

Patched:

$ ./python.exe -m timeit -s 'from typing import Protocol, runtime_checkable' '''@runtime_checkable
class MyProtocol(Protocol):
    def meth(self): pass
'''
50000 loops, best of 5: 7.69 usec per loop
$ ./python.exe -m timeit -s 'from typing import Protocol, runtime_checkable' '''class MyProtocol(Protocol):
    def meth(self): pass
'''
50000 loops, best of 5: 7.78 usec per loop

This was on a debug build though and I haven't compared it with versions where Protocol just accessed
`.__annotations__` directly, and it's not a huge difference, so I don't think it's worth calling out the
optimization too much.

A downside of this change is that any errors that happen during the determination of attributes now
happen only the first time isinstance() is called. This seems OK since these errors happen only in
fairly exotic circumstances.

Another downside is that any attributes added after class initialization now get picked up as protocol
members. This came up in the typing test suite due to `@final`, but may cause issues elsewhere too.
@bedevere-app bedevere-app bot mentioned this issue Apr 16, 2025
Merged
JelleZijlstra added a commit that referenced this issue Apr 16, 2025
@felixscherz @JelleZijlstra
@sobolevn @AlexWaygood
gh-132493: lazy evaluation of annotations in typing._proto_hook (#1…
71af090 
…32534)

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: sobolevn <mail@sobolevn.me>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
JelleZijlstra added a commit that referenced this issue Apr 17, 2025
@JelleZijlstra @AlexWaygood
gh-132493: Avoid eager import of annotationlib in typing (again) (#13…
5707837 
…2596)

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
JelleZijlstra added a commit to JelleZijlstra/cpython that referenced this issue May 4, 2025
@JelleZijlstra
pythongh-132493: Avoid eager evaluation of annotations in `@reprlib.r 8000 …
dd208ed 
…ecursive_repr()`
@bedevere-app bedevere-app bot mentioned this issue May 4, 2025
Merged
JelleZijlstra added a commit to JelleZijlstra/cpython that referenced this issue May 5, 2025
@JelleZijlstra
pythongh-132493: Remove __annotations__ usage in inspect._signature_i…
22dbf9f 
…s_functionlike

This check is potentially problematic because it could force evaluation of
annotations unnecessarily. This doesn't trigger for builtin objects (functions,
classes, or modules) with annotations, but it could trigger for third-party objects.

The check was not particularly useful anyway, because it succeeds if ``__annotations__``
is a dict or None, so the only thing this did was guard against objects that have an
``__annotations__`` attribute that is of some other type. That doesn't seem particularly
useful, so I just removed the check.
@bedevere-app bedevere-app bot mentioned this issue May 5, 2025
Merged
JelleZijlstra added a commit that referenced this issue May 6, 2025
@JelleZijlstra
gh-132493: Avoid eager evaluation of annotations in `@reprlib.recursi…
dbee142 
…ve_repr()` (#133411)
JelleZijlstra added a commit that referenced this issue May 10, 2025
@JelleZijlstra
gh-132493: Remove __annotations__ usage in inspect._signature_is_func…
cb6596c 
…tionlike (#133415)

This check is potentially problematic because it could force evaluation of
annotations unnecessarily. This doesn't trigger for builtin objects (functions,
classes, or modules) with annotations, but it could trigger for third-party objects.

The check was not particularly useful anyway, because it succeeds if ``__annotations__``
is a dict or None, so the only thing this did was guard against objects that have an
``__annotations__`` attribute that is of some other type. That doesn't seem particularly
useful, so I just removed the check.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 10, 2025
@JelleZijlstra @miss-islington
pythongh-132493: Remove __annotations__ usage in inspect._signature_i…
775435e 
…s_functionlike (pythonGH-133415)

This check is potentially problematic because it could force evaluation of
annotations unnecessarily. This d
5EEB
oesn't trigger for builtin objects (functions,
classes, or modules) with annotations, but it could trigger for third-party objects.

The check was not particularly useful anyway, because it succeeds if ``__annotations__``
is a dict or None, so the only thing this did was guard against objects that have an
``__annotations__`` attribute that is of some other type. That doesn't seem particularly
useful, so I just removed the check.
(cherry picked from commit cb6596c)

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
@bedevere-app bedevere-app bot mentioned this issue May 10, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.14 bugs and security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@JelleZijlstra @picnixz
0