8000 PEP 563: Postponed Evaluation of Annotations · python/peps@454c889 · GitHub
[go: up one dir, main page]
Skip to content

Commit 454c889

Browse files
author
Lukasz Langa
committed
PEP 563: Postponed Evaluation of Annotations
Draft 0. PEP number assigned by GvR in ambv/static-annotations@77557db#commitcomment-24210467
1 parent 6fe4224 commit 454c889

File tree

1 file changed

+264
-0
lines changed

1 file changed

+264
-0
lines changed

‎pep-0563.rst

Lines changed: 264 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,264 @@
1+
PEP: 563
2+
Title: Postponed Evaluation of Annotations
3+
Version: $Revision$
4+
Last-Modified: $Date$
5+
Author: Łukasz Langa <lukasz@langa.pl>
6+
Discussions-To: Python-Dev <python-dev@python.org>
7+
Status: Draft
8+
Type: Standards Track
9+
Content-Type: text/x-rst
10+
Created: 8-Sep-2017
11+
Python-Version: 3.7
12+
Post-History:
13+
Resolution:
14+
15+
16+
Abstract
17+
========
18+
19+
PEP 3107 introduced syntax for function annotations, but the semantics
20+
were deliberately left undefined. PEP 484 introduced a standard meaning
21+
to annotations: type hints. PEP 526 defined variable annotations,
22+
explicitly tying them with the type hinting use case.
23+
24+
This PEP proposes changing function annotations and variable annotations
25+
so that they are no longer evaluated at function definition time.
26+
Instead, they are preserved in ``__annotations__`` in string form.
27+
28+
This change is going to be introduced gradually, starting with a new
29+
``__future__`` import in Python 3.7.
30+
31+
32+
Rationale and Goals
33+
===================
34+
35+
PEP 3107 added support for arbitrary annotations on parts of a function
36+
definition. Just like default values, annotations are evaluated at
37+
function definition time. This creates a number of issues for the type
38+
hinting use case:
39+
40+
* forward references: when a type hint contains names that have not been
41+
defined yet, that definition needs to be expressed as a string
42+
literal;
43+
44+
* type hints are executed at module import time, which is not
45+
computationally free.
46+
47+
Postponing the evaluation of annotations solves both problems.
48+
49+
Non-goals
50+
---------
51+
52+
Just like in PEP 484 and PEP 526, it should be emphasized that **Python
53+
will remain a dynamically typed language, and the authors have no desire
54+
to ever make type hints mandatory, even by convention.**
55+
56+
Annotations are still available for arbitrary use besides type checking.
57+
Using ``@typing.no_type_hints`` in this case is recommended to
58+
disambiguate the use case.
59+
60+
61+
Implementation
62+
==============
63+
64+
In a future version of Python, function and variable annotations will no
65+
longer be evaluated at definition time. Instead, a string form will be
66+
preserved in the respective ``__annotations__`` dictionary. Static type
67+
checkers will see no difference in behavior, whereas tools using
68+
annotations at runtime will have to perform postponed evaluation.
69+
70+
If an annotation was already a string, this string is preserved
71+
verbatim. In other cases, the string form is obtained from the AST
72+
during the compilation step, which means that the string form preserved
73+
might not preserve the exact formatting of the source.
74+
75+
Annotations need to be syntactically valid Python expressions, also when
76+
passed as literal strings (i.e. ``compile(literal, '', 'eval')``).
77+
Annotations can only use names present in the module scope as postponed
78+
evaluation using local names is not reliable.
79+
80+
Note that as per PEP 526, local variable annotations are not evaluated
81+
at all since they are not accessible outside of the function's closure.
82+
83+
Enabling the future behavior in Python 3.7
84+
------------------------------------------
85+
86+
The functionality described above can be enabled starting from Python
87+
3.7 using the following special import::
88+
89+
from __future__ import annotations
90+
91+
92+
Resolving Type Hints at Runtime
93+
===============================
94+
95+
To resolve an annotation at runtime from its string form to the result
96+
of the enclosed expression, user code needs to evaluate the string.
97+
98+
For code that uses type hints, the ``typing.get_type_hints()`` function
99+
correctly evaluates expressions back from its string form. Note that
100+
all valid code currently using ``__annotations__`` should already be
101+
doing that since a type annotation can be expressed as a string literal.
102+
103+
For code which uses annotations for other purposes, a regular
104+
``eval(ann, globals, locals)`` call is enough to resolve the
105+
annotation. The trick here is to get the correct value for globals.
106+
Fortunately, in the case of functions, they hold a reference to globals
107+
in an attribute called ``__globals__``. To get the correct module-level
108+
context to resolve class variables, use::
109+
110+
cls_globals = sys.modules[SomeClass.__module__].__dict__
111+
112+
Runtime annotation resolution and class decorators
113+
--------------------------------------------------
114+
115+
Metaclasses and class decorators that need to resolve annotations for
116+
the current class will fail for annotations that use the name of the
117+
current class. Example::
118+
119+
def class_decorator(cls):
120+
annotations = get_type_hints(cls) # raises NameError on 'C'
121+
print(f'Annotations for {cls}: {annotations}')
122+
return cls
123+
124+
@class_decorator
125+
class C:
126+
singleton: 'C' = None
127+
128+
This was already true before this PEP. The class decorator acts on
129+
the class before it's assigned a name in the current definition scope.
130+
131+
The situation is made somewhat stricter when class-level variables are
132+
considered. Previously, when the string form wasn't used in annotations,
133+
a class decorator would be able to cover situations like::
134+
135+
@class_decorator
136+
class Restaurant:
137+
class MenuOption(Enum):
138+
SPAM = 1
139+
EGGS = 2
140+
141+
default_menu: List[MenuOption] = []
142+
143+
This is no longer possible.
144+
145+
Runtime annotation resolution and ``TYPE_CHECKING``
146+
---------------------------------------------------
147+
148+
Sometimes there's code that must be seen by a type checker but should
149+
not be executed. For such situations the ``typing`` module defines a
150+
constant, ``TYPE_CHECKING``, that is considered ``True`` during type
151+
checking but ``False`` at runtime. Example::
152+
153+
import typing
154+
155+
if typing.TYPE_CHECKING:
156+
import expensive_mod
157+
158+
def a_func(arg: expensive_mod.SomeClass) -> None:
159+
a_var: expensive_mod.SomeClass = arg
160+
...
161+
162+
This approach is also useful when handling import cycles.
163+
164+
Trying to resolve annotations of ``a_func`` at runtime using
165+
``typing.get_type_hints()`` will fail since the name ``expensive_mod``
166+
is not defined (``TYPE_CHECKING`` variable being ``False`` at runtime).
167+
This was already true before this PEP.
168+
169+
170+
Backwards Compatibility
171+
=======================
172+
173+
This is a backwards incompatible change. Applications depending on
174+
arbitrary objects to be directly present in annotations will break
175+
if they are not using ``typing.get_type_hints()`` or ``eval()``.
176+
177+
Annotations that depend on locals at the time of the function/class
178+
definition are now invalid. Example::
179+
180+
def generate_class():
181+
some_local = datetime.datetime.now()
182+
class C:
183+
field: some_local = 1 # NOTE: INVALID ANNOTATION
184+
def method(self, arg: some_local.day) -> None: # NOTE: INVALID ANNOTATION
185+
...
186+
187+
Annotations using nested classes and their respective state are still
188+
valid, provided they use the fully qualified name. Example::
189+
190+
class C:
191+
field = 'c_field'
192+
def method(self, arg: C.field) -> None: # this is OK
193+
...
194+
195+
class D:
< 10000 /td>196+
field2 = 'd_field'
197+
def method(self, arg: C.field -> C.D.field2: # this is OK
198+
...
199+
200+
In the presence of an annotation that cannot be resolved using the
201+
current module's globals, a NameError is raised at compile time.
202+
203+
204+
Deprecation policy
205+
------------------
206+
207+
In Python 3.7, a ``__future__`` import is required to use the described
208+
functionality and a ``PendingDeprecationWarning`` is raised by the
209+
compiler in the presence of type annotations in modules without the
210+
``__future__`` import. In Python 3.8 the warning becomes a
211+
``DeprecationWarning``. In the next version this will become the
212+
default behavior.
213+
214+
215+
Rejected Ideas
216+
==============
217+
218+
Keep the ability to use local state when defining annotations
219+
-------------------------------------------------------------
220+
221+
With postponed evaluation, this is impossible for function locals. For
222+
classes, it would be possible to keep the ability to define annotations
223+
using the local scope. However, when using ``eval()`` to perform the
224+
postponed evaluation, we need to provide the correct globals and locals
225+
to the ``eval()`` call. In the face of nested classes, the routine to
226+
get the effective "globals" at definition time would have to look
227+
something like this::
228+
229+
def get_class_globals(cls):
230+
result = {}
231+
result.update(sys.modules[cls.__module__].__dict__)
232+
for child in cls.__qualname__.split('.'):
233+
result.update(result[child].__dict__)
234+
return result
235+
236+
This is brittle and doesn't even cover slots. Requiring the use of
237+
module-level names simplifies runtime evaluation and provides the
238+
"one obvious way" to read annotations. It's the equivalent of absolute
239+
imports.
240+
241+
242+
Acknowledgements
243+
================
244+
245+
This document could not be completed without valuable input,
246+
encouragement and advice from Guido van Rossum, Jukka Lehtosalo, and
247+
Ivan Levkivskyi.
248+
249+
250+
Copyright
251+
=========
252+
253+
This document has been placed in the public domain.
254+
255+
256+
257+
..
258+
Local Variables:
259+
mode: indented-text
260+
indent-tabs-mode: nil
261+
sentence-end-double-space: t
262+
fill-column: 70
263+
coding: utf-8
264+
End:

0 commit comments

Comments
 (0)
0