| PEP: | 526 | +
|---|---|
| Title: | Syntax for Variable Annotations | +
| Version: | $Revision$ | +
| Last-Modified: | $Date$ | +
| Author: | Ryan Gonzalez <rymg19 at gmail.com>, Philip House <phouse512 at gmail.com>, Ivan Levkivskyi <levkivskyi at gmail.com>, Lisa Roach <lisaroach14 at gmail.com>, Guido van Rossum <guido at python.org> | +
| Status: | Final | +
| Type: | Standards Track | +
| Content-Type: | text/x-rst | +
| Created: | 09-Aug-2016 | +
| Python-Version: | 3.6 | +
| Post-History: | 30-Aug-2016, 02-Sep-2016 | +
| Resolution: | https://mail.python.org/pipermail/python-dev/2016-September/146282.html | +
Contents
+ +This PEP has been provisionally accepted by the BDFL. +See the acceptance message for more color: +https://mail.python.org/pipermail/python-dev/2016-September/146282.html
+This PEP was drafted in a separate repo: +https://github.com/phouse512/peps/tree/pep-0526.
+There was preliminary discussion on python-ideas and at +https://github.com/python/typing/issues/258.
+Before you bring up an objection in a public forum please at least +read the summary of rejected ideas listed at the end of this PEP.
+PEP 484 introduced type hints, a.k.a. type annotations. While its +main focus was function annotations, it also introduced the notion of +type comments to annotate variables:
+
+# 'primes' is a list of integers
+primes = [] # type: List[int]
+
+# 'captain' is a string (Note: initial value is a problem)
+captain = ... # type: str
+
+class Starship:
+ # 'stats' is a class variable
+ stats = {} # type: Dict[str, int]
+
+This PEP aims at adding syntax to Python for annotating the types of variables +(including class variables and instance variables), +instead of expressing them through comments:
+
+primes: List[int] = []
+
+captain: str # Note: no initial value!
+
+class Starship:
+ stats: ClassVar[Dict[str, int]] = {}
+
+PEP 484 explicitly states that type comments are intended to help with +type inference in complex cases, and this PEP does not change this +intention. However, since in practice type comments have also been +adopted for class variables and instance variables, this PEP also +discusses the use of type annotations for those variables.
+Although type comments work well enough, the fact that they're +expressed through comments has some downsides:
+Text editors often highlight comments differently from type annotations.
+There's no way to annotate the type of an undefined variable; one needs to +initialize it to None (e.g. a = None # type: int).
+Variables annotated in a conditional branch are difficult to read:
++if some_value: + my_var = function() # type: Logger +else: + my_var = another_function() # Why isn't there a type here? ++
Since type comments aren't actually part of the language, if a Python script +wants to parse them, it requires a custom parser instead of just using +ast.
+Type comments are used a lot in typeshed. Migrating typeshed to use +the variable annotation syntax instead of type comments would improve +readability of stubs.
+In situations where normal comments and type comments are used together, it is +difficult to distinguish them:
++path = None # type: Optional[str] # Path to module source ++
It's impossible to retrieve the annotations at runtime outside of +attempting to find the module's source code and parse it at runtime, +which is inelegant, to say the least.
+The majority of these issues can be alleviated by making the syntax +a core part of the language. Moreover, having a dedicated annotation syntax +for class and instance variables (in addition to method annotations) will +pave the way to static duck-typing as a complement to nominal typing defined +by PEP 484.
+While the proposal is accompanied by an extension of the typing.get_type_hints +standard library function for runtime retrieval of annotations, variable +annotations are not designed for runtime type checking. Third party packages +will have to be developed to implement such functionality.
+It should also be emphasized that Python will remain a dynamically typed +language, and the authors have no desire to ever make type hints mandatory, +even by convention. Type annotations should not be confused with variable +declarations in statically typed languages. The goal of annotation syntax is +to provide an easy way to specify structured type metadata +for third party tools.
+This PEP does not require type checkers to change their type checking +rules. It merely provides a more readable syntax to replace type +comments.
+Type annotation can be added to an assignment statement or to a single +expression indicating the desired type of the annotation target to a third +party type checker:
++my_var: int +my_var = 5 # Passes type check. +other_var: int = 'a' # Flagged as error by type checker, + # but OK at runtime. ++
This syntax does not introduce any new semantics beyond PEP 484, so that +the following three statements are equivalent:
++var = value # type: annotation +var: annotation; var = value +var: annotation = value ++
Below we specify the syntax of type annotations +in different contexts and their runtime effects.
+We also suggest how type checkers might interpret annotations, but +compliance to these suggestions is not mandatory. (This is in line +with the attitude towards compliance in PEP 484.)
+The types of locals and globals can be annotated as follows:
++some_number: int # variable without initial value +some_list: List[int] = [] # variable with initial value ++
Being able to omit the initial value allows for easier typing of variables +assigned in conditional branches:
++sane_world: bool +if 2+2 == 4: + sane_world = True +else: + sane_world = False ++
Note that, although the syntax does allow tuple packing, it does not allow +one to annotate the types of variables when tuple unpacking is used:
++# Tuple packing with variable annotation syntax +t: Tuple[int, ...] = (1, 2, 3) + +# Tuple unpacking with variable annotation syntax +header: str +kind: int +body: Optional[List[str]] +header, kind, body = message ++
Omitting the initial value leaves the variable uninitialized:
++a: int +print(a) # raises NameError ++
However, annotating a local variable will cause the interpreter to always make +it a local:
++def f(): + a: int + print(a) # raises UnboundLocalError + # Commenting out the a: int makes it a NameError. ++
as if the code were:
++def f(): + if False: a = 0 + print(a) # raises UnboundLocalError ++
Duplicate type annotations will be ignored. However, static type +checkers may issue a warning for annotations of the same variable +by a different type:
++a: int +a: str # Static type checker may or may not warn about this. ++
Type annotations can also be used to annotate class and instance variables +in class bodies and methods. In particular, the value-less notation a: int +allows one to annotate instance variables that should be initialized +in __init__ or __new__. The proposed syntax is as follows:
+
+class BasicStarship:
+ captain: str = 'Picard' # instance variable with default
+ damage: int # instance variable without default
+ stats: ClassVar[Dict[str, int]] = {} # class variable
+
+Here ClassVar is a special class defined by the typing module that +indicates to the static type checker that this variable should not be +set on instances.
+This could be illustrated with a more detailed example. In this class:
+
+class Starship:
+ captain = 'Picard'
+ stats = {}
+
+ def __init__(self, damage, captain=None):
+ self.damage = damage
+ if captain:
+ self.captain = captain # Else keep the default
+
+ def hit(self):
+ Starship.stats['hits'] = Starship.stats.get('hits', 0) + 1
+
+stats is intended to be a class variable (keeping track of many different +per-game statistics), while captain is an instance variable with a default +value set in the class. This difference might not be seen by a type +checker: both get initialized in the class, but captain serves only +as a convenient default value for the instance variable, while stats +is truly a class variable -- it is intended to be shared by all instances.
+Since both variables happen to be initialized at the class level, it is +useful to distinguish them by marking class variables as annotated with +types wrapped in ClassVar[...]. In this way a type checker may flag +accidental assignments to attributes with the same name on instances.
+For example, annotating the discussed class:
+
+class Starship:
+ captain: str = 'Picard'
+ damage: int
+ stats: ClassVar[Dict[str, int]] = {}
+
+ def __init__(self, damage: int, captain: str = None):
+ self.damage = damage
+ if captain:
+ self.captain = captain # Else keep the default
+
+ def hit(self):
+ Starship.stats['hits'] = Starship.stats.get('hits', 0) + 1
+
+enterprise_d = Starship(3000)
+enterprise_d.stats = {} # Flagged as error by a type checker
+Starship.stats = {} # This is OK
+
+As a matter of convenience (and convention), instance variables can be +annotated in __init__ or other methods, rather than in the class:
++from typing import Generic, TypeVar +T = TypeVar(âTâ) + +class Box(Generic[T]): + def __init__(self, content): + self.content: T = content ++
The target of the annotation can be any valid single assignment +target, at least syntactically (it is up to the type checker what to +do with this):
+
+class Cls:
+ pass
+
+c = Cls()
+c.x: int = 0 # Annotates c.x with int.
+c.y: int # Annotates c.y with int.
+
+d = {}
+d['a']: int = 0 # Annotates d['a'] with int.
+d['b']: int # Annotates d['b'] with int.
+
+Note that even a parenthesized name is considered an expression, +not a simple name:
++(x): int # Annotates x with int, (x) treated as expression by compiler. +(y): int = 0 # Same situation here. ++
It is illegal to attempt to annotate variables subject to global +or nonlocal in the same function scope:
++def f(): + global x: int # SyntaxError + +def g(): + x: int # Also a SyntaxError + global x ++
The reason is that global and nonlocal don't own variables; +therefore, the type annotations belong in the scope owning the variable.
+Only single assignment targets and single right hand side values are allowed. +In addition, one cannot annotate variables used in a for or with +statement; they can be annotated ahead of time, in a similar manner to tuple +unpacking:
++a: int +for a in my_iter: + ... + +f: MyFile +with myfunc() as f: + ... ++
Annotating a local variable will cause +the interpreter to treat it as a local, even if it was never assigned to. +Annotations for local variables will not be evaluated:
++def f(): + x: NonexistentName # No error. ++
However, if it is at a module or class level, then the type will be +evaluated:
++x: NonexistentName # Error! +class X: + var: NonexistentName # Error! ++
In addition, at the module or class level, if the item being annotated is a +simple name, then it and the annotation will be stored in the +__annotations__ attribute of that module or class (mangled if private) +as an ordered mapping from names to evaluated annotations. +Here is an example:
+
+from typing import Dict
+class Player:
+ ...
+players: Dict[str, Player]
+__points: int
+
+print(__annotations__)
+# prints: {'players': typing.Dict[str, __main__.Player],
+# '_Player__points': <class 'int'>}
+
+__annotations__ is writable, so this is permitted:
++__annotations__['s'] = str ++
But attempting to update __annotations__ to something other than an +ordered mapping may result in a TypeError:
++class C: + __annotations__ = 42 + x: int = 5 # raises TypeError ++
(Note that the assignment to __annotations__, which is the +culprit, is accepted by the Python interpreter without questioning it +-- but the subsequent type annotation expects it to be a +MutableMapping and will fail.)
+The recommended way of getting annotations at runtime is by using +typing.get_type_hints function; as with all dunder attributes, +any undocummented use of __annotations__ is subject to breakage +without warning:
+
+from typing import Dict, ClassVar, get_type_hints
+class Starship:
+ hitpoints: int = 50
+ stats: ClassVar[Dict[str, int]] = {}
+ shield: int = 100
+ captain: str
+ def __init__(self, captain: str) -> None:
+ ...
+
+assert get_type_hints(Starship) == {'hitpoints': int,
+ 'stats': ClassVar[Dict[str, int]],
+ 'shield': int,
+ 'captain': str}
+
+assert get_type_hints(Starship.__init__) == {'captain': str,
+ 'return': None}
+
+Note that if annotations are not found statically, then the +__annotations__ dictionary is not created at all. Also the +value of having annotations available locally does not offset +the cost of having to create and populate the annotations dictionary +on every function call. Therefore annotations at function level are +not evaluated and not stored.
+While Python with this PEP will not object to:
++alice: 'well done' = 'A+' +bob: 'what a shame' = 'F-' ++
since it will not care about the type annotation beyond "it evaluates +without raising", a type checker that encounters it will flag it, +unless disabled with # type: ignore or @no_type_check.
+However, since Python won't care what the "type" is, +if the above snippet is at the global level or in a class, __annotations__ +will include {'alice': 'well done', 'bob': 'what a shame'}.
+These stored annotations might be used for other purposes, +but with this PEP we explicitly recommend type hinting as the +preferred use of annotations.
+Should we introduce variable annotations at all? +Variable annotations have already been around for almost two years +in the form of type comments, sanctioned by PEP 484. They are +extensively used by third party type checkers (mypy, pytype, +PyCharm, etc.) and by projects using the type checkers. However, the +comment syntax has many downsides listed in Rationale. This PEP is +not about the need for type annotations, it is about what should be +the syntax for such annotations.
+Introduce a new keyword: +The choice of a good keyword is hard, +e.g. it can't be var because that is way too common a variable name, +and it can't be local if we want to use it for class variables or +globals. Second, no matter what we choose, we'd still need +a __future__ import.
+Use 'def' as a keyword: +The proposal would be:
++def primes: List[int] = [] +def captain: str ++
The problem with this is that def means "define a function" to +generations of Python programmers (and tools!), and using it also to +define variables does not increase clarity. (Though this is of +course subjective.)
+Use function based syntax: +It was proposed to annotate types of variables using +var = cast(annotation[, value]). Although this syntax +alleviates some problems with type comments like absence of tne annotation +in AST, it does not solve other problems such as readability +and it introduces possible runtime overhead.
+Allow type annotations for tuple unpacking: +This causes ambiguity: it's not clear what this statement means:
++x, y: T ++
Are x and y both of type T, or do we expect T to be +a tuple type of two items that are distributed over x and y, +or perhaps x has type Any and y has type T? (The +latter is what this would mean if this occurred in a function +signature.) Rather than leave the (human) reader guessing, we +forbid this, at least for now.
+Parenthesized form ``(var: type)`` for annotations: +It was brought up on python-ideas as a remedy for the above-mentioned +ambiguity, but it was rejected since such syntax would be hairy, +the benefits are slight, and the readability would be poor.
+Allow annotations in chained assignments: +This has problems of ambiguity and readability similar to tuple +unpacking, for example in:
++x: int = y = 1 +z = w: int = 1 ++
it is ambiguous, what should the types of y and z be? +Also the second line is difficult to parse.
+Allow annotations in ``with`` and ``for`` statement: +This was rejected because in for it would make it hard to spot the actual +iterable, and in with it would confuse the CPython's LL(1) parser.
+Evaluate local annotations at function definition time: +This has been rejected by Guido because the placement of the annotation +strongly suggests that it's in the same scope as the surrounding code.
+Store variable annotations also in function scope: +The value of having the annotations available locally is just not enough +to significantly offset the cost of creating and populating the dictionary +on each function call.
+Initialize variables annotated without assignment: +It was proposed on python-ideas to initialize x in x: int to +None or to an additional special constant like Javascript's +undefined. However, adding yet another singleton value to the language +would needed to be checked for everywhere in the code. Therefore, +Guido just said plain "No" to this.
+Add also InstanceVar to the typing module: +This is redundant because instance variables are way more common than +class variables. The more common usage deserves to be the default.
+Allow instance variable annotations only in methods: +The problem is that many __init__ methods do a lot of things besides +initializing instance variables, and it would be harder (for a human) +to find all the instance variable annotations. +And sometimes __init__ is factored into more helper methods +so it's even harder to chase them down. Putting the instance variable +annotations together in the class makes it easier to find them, +and helps a first-time reader of the code.
+Use syntax x: class t = v for class variables: +This would require a more complicated parser and the class +keyword would confuse simple-minded syntax highlighters. Anyway we +need to have ClassVar store class variables to +__annotations__, so a simpler syntax was chosen.
+Forget about ClassVar altogether: +This was proposed since mypy seems to be getting along fine without a way +to distinguish between class and instance variables. But a type checker +can do useful things with the extra information, for example flag +accidental assignments to a class variable via the instance +(which would create an instance variable shadowing the class variable). +It could also flag instance variables with mutable defaults, +a well-known hazard.
+Do not evaluate annotations, treat them as strings: +This would be inconsistent with the behavior of function annotations that +are always evaluated. Although this might be reconsidered in future, +it was decided in PEP 484 that this would have to be a separate PEP.
+Annotate variable types in class docstring: +Many projects already use various docstring conventions, often without +much consistency and generally without conforming to the PEP 484 annotation +syntax yet. Also this would require a special sophisticated parser. +This, in turn, would defeat the purpose of the PEP -- +collaborating with the third party type checking tools.
+Implement ``__annotations__`` as a descriptor: +This was proposed to prohibit setting __annotations__ to something +non-dictionary or non-None. Guido has rejected this idea as unnecessary; +instead a TypeError will be raised if an attempt is made to update +__annotations__ when it is anything other than a mapping.
+Treating bare annotations the same as global or nonlocal: +The rejected proposal would prefer that the presence of an +annotation without assignment in a function body should not involve +any evaluation. In contrast, the PEP implies that if the target +is more complex than a single name, its "left-hand part" should be +evaluated at the point where it occurs in the function body, just to +enforce that it is defined. For example, in this example:
++def foo(self): + slef.name: str ++
the name slef should be evaluated, just so that if it is not +defined (as is likely in this example :-), the error will be caught +at runtime. This is more in line with what happens when there is +an initial value, and thus is expected to lead to fewer surprises. +(Also note that if the target was self.name (this time correctly +spelled :-), an optimizing compiler has no obligation to evaluate +self as long as it can prove that it will definitely be +defined.)
+This PEP is fully backwards compatible.
+An implementation for Python 3.6 is found on GitHub repo at +https://github.com/ilevkivskyi/cpython/tree/pep-526
+This document has been placed in the public domain.
+