8000 Optionally allow redefinition of variable with different type (#6197) · python/mypy@d2f1d34 · GitHub
[go: up one dir, main page]
Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit d2f1d34

Browse files
JukkaLJukkaL
authored
Optionally allow redefinition of variable with different type (#6197)
If the flag `--allow-redefinition` is used, it's now possible to redefine a variable with a different type. For example, this code will be okay: ``` x = 0 # Define 'x' with type 'int' print(x) x = 'a' # Define another 'x' with type 'str' print(x) ``` The variable needs to be read before redefinition is possible. This is mainly to allow these idioms: ``` x: List[int] x = [] # Not a redefinition y = None # type: List[int] # Strict optional disabled y = [] # Not a redefinition ``` It's also okay to use a type annotation in any of the assignments. This is fine: ``` x: int = 0 print(x) x: str = 'a' print(x) ``` Redefinition is only allowed to happen in the same block and the nesting level as the original definition. This lets us keep the implementation relatively simple, and there will hopefully be less of a readability impact, since it's less likely that users accidentally redefine a variable or use redefinition in confusing ways. We may want to lift the restriction in the future, however. Function arguments can be redefined in the body but only in non-nested blocks. The implementation does a simple static analysis to rename variables to make them distinct when we detect redefinition. This happens before semantic analysis pass 1. The new behavior is not fully backward compatible with the old behavior. For example, here we lose type context in the second assignment: ``` x: List[int] = [] f(x) x = [] # Need type annotation since this creates an independent variable ``` Internally we use a sequence of single quotes as a variable suffix to represent renamed variants. The implementation provides an `unmangle()` function to get the original name from a potentially mangled name. It's still impossible to redefine final names, classes and certain other things. The main use case is simple variables, either in single or multiple assignments. I did some benchmarking (not with the latest version though) and the performance impact of the new pass was minor (under 2% IIRC). Things to do in additional PRs: * Add documentation. * Make sure we unmangle names everywhere. * Consider allowing redefinition to happen sometimes in another block. * Consider propagating type context from original definition.
1 parent d5cf72b commit d2f1d34

20 files changed

+1315
-170
lines changed

‎mypy/checkmember.py

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,8 @@
99
)
1010
from mypy.nodes import (
1111
TypeInfo, FuncBase, Var, FuncDef, SymbolNode, Context, MypyFile, TypeVarExpr,
12-
ARG_POS, ARG_STAR, ARG_STAR2, Decorator, OverloadedFuncDef, TypeAlias, TempNode
12+
ARG_POS, ARG_STAR, ARG_STAR2, Decorator, OverloadedFuncDef, TypeAlias, TempNode,
13+
is_final_node
1314
)
1415
from mypy.messages import MessageBuilder
1516
from mypy.maptype import map_instance_to_supertype
@@ -895,8 +896,3 @@ def erase_to_bound(t: Type) -> Type:
895896
if isinstance(t.item, TypeVarType):
896897
return TypeType.make_normalized(t.item.upper_bound)
897898
return t
898-
899-
900-
def is_final_node(node: Optional[SymbolNode]) -> bool:
901-
"""Check whether `node` corresponds to a final attribute."""
902-
return isinstance(node, (Var, FuncDef, OverloadedFuncDef, Decorator)) and node.is_final

‎mypy/main.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -523,6 +523,10 @@ def add_invertible_flag(flag: str,
523523
help="Suppress toplevel errors caused by missing annotations",
524524
group=strictness_group)
525525

526+
add_invertible_flag('--allow-redefinition', default=False, strict_flag=False,
527+
help="Allow unconditional variable redefinition with a new type",
528+
group=strictness_group)
529+
526530
incremental_group = parser.add_argument_group(
527531
title='Incremental mode',
528532
description="Adjust how mypy incrementally type checks and caches modules. "

‎mypy/messages.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@
2929
ReturnStmt, NameExpr, Var, CONTRAVARIANT, COVARIANT, SymbolNode,
3030
CallExpr
3131
)
32+
from mypy.util import unmangle
3233
from mypy import message_registry
3334

3435
MYPY = False
@@ -952,7 +953,7 @@ def cant_assign_to_final(self, name: str, attr_assign: bool, ctx: Context) -> No
952953
Pass `attr_assign=True` if the assignment assigns to an attribute.
953954
"""
954955
kind = "attribute" if attr_assign else "name"
955-
self.fail('Cannot assign to final {} "{}"'.format(kind, name), ctx)
956+
self.fail('Cannot assign to final {} "{}"'.format(kind, unmangle(name)), ctx)
956957

957958
def protocol_members_cant_be_final(self, ctx: Context) -> None:
958959
self.fail("Protocol member cannot be final", ctx)
@@ -1064,7 +1065,7 @@ def unimported_type_becomes_any(self, prefix: str, typ: Type, ctx: Context) -> N
10641065
ctx)
10651066

10661067
def need_annotation_for_var(self, node: SymbolNode, context: Context) -> None:
1067-
self.fail("Need type annotation for '{}'".format(node.name()), context)
1068+
self.fail("Need type annotation for '{}'".format(unmangle(node.name())), context)
10681069

10691070
def explicit_any(self, ctx: Context) -> None:
10701071
self.fail('Explicit "Any" is not allowed', ctx)

‎mypy/nodes.py

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -924,15 +924,18 @@ def accept(self, visitor: StatementVisitor[T]) -> T:
924924

925925

926926
class AssignmentStmt(Statement):
927-
"""Assignment statement
927+
"""Assignment statement.
928+
928929
The same node class is used for single assignment, multiple assignment
929930
(e.g. x, y = z) and chained assignment (e.g. x = y = z), assignments
930-
that define new names, and assignments with explicit types (# type).
931+
that define new names, and assignments with explicit types ("# type: t"
932+
or "x: t [= ...]").
931933
932-
An lvalue can be NameExpr, TupleExpr, ListExpr, MemberExpr, IndexExpr.
934+
An lvalue can be NameExpr, TupleExpr, ListExpr, MemberExpr, or IndexExpr.
933935
"""
934936

935937
lvalues = None # type: List[Lvalue]
938+
# This is a TempNode if and only if no rvalue (x: t).
936939
rvalue = None # type: Expression
937940
# Declared type in a comment, may be None.
938941
type = None # type: Optional[mypy.types.Type]
@@ -2968,3 +2971,8 @@ def is_class_var(expr: NameExpr) -> bool:
29682971
if isinstance(expr.node, Var):
29692972
return expr.node.is_classvar
29702973
return False
2974+
2975+
2976+
def is_final_node(node: Optional[SymbolNode]) -> bool:
2977+
"""Check whether `node` corresponds to a final attribute."""
2978+
return isinstance(node, (Var, FuncDef, OverloadedFuncDef, Decorator)) and node.is_final

‎mypy/options.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ class BuildType:
2121
PER_MODULE_OPTIONS = {
2222
# Please keep this list sorted
2323
"allow_untyped_globals",
24+
"allow_redefinition",
2425
"always_false",
2526
"always_true",
2627
"check_untyped_defs",
@@ -149,6 +150,10 @@ def __init__(self) -> None:
149150
# Suppress toplevel errors caused by missing annotations
150151
self.allow_untyped_globals = False
151152

153+
# Allow variable to be redefined with an arbitrary type in the same block
154+
# and the same nesting level as the initialization
155+
self.allow_redefinition = False
156+
152157
# Variable names considered True
153158
self.always_true = [] # type: List[str]
154159

‎mypy/plugins/attrs.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
Overloaded, UnionType, FunctionLike
2020
)
2121
from mypy.typevars import fill_typevars
22+
from mypy.util import unmangle
2223
from mypy.server.trigger import make_wildcard_trigger
2324

2425
MYPY = False
@@ -376,9 +377,10 @@ def _attribute_from_auto_attrib(ctx: 'mypy.plugin.ClassDefContext',
376377
rvalue: Expression,
377378
stmt: AssignmentStmt) -> Attribute:
378379
"""Return an Attribute for a new type assignment."""
380+
name = unmangle(lhs.name)
379381
# `x: int` (without equal sign) assigns rvalue to TempNode(AnyType())
380382
has_rhs = not isinstance(rvalue, TempNode)
381-
return Attribute(lhs.name, ctx.cls.info, has_rhs, True, kw_only, Converter(), stmt)
383+
return Attribute(name, ctx.cls.info, has_rhs, True, kw_only, Converter(), stmt)
382384

383385

384386
def _attribute_from_attrib_maker(ctx: 'mypy.plugin.ClassDefContext',
@@ -443,7 +445,8 @@ def _attribute_from_attrib_maker(ctx: 'mypy.plugin.ClassDefContext',
443445
converter = convert
444446
converter_info = _parse_converter(ctx, converter)
445447

446-
return Attribute(lhs.name, ctx.cls.info, attr_has_default, init, kw_only, converter_info, stmt)
448+
name = unmangle(lhs.name)
449+
return Attribute(name, ctx.cls.info, attr_has_default, init, kw_only, converter_info, stmt)
< 465D code>447450

448451

449452
def _parse_converter(ctx: 'mypy.plugin.ClassDefContext',

0 commit comments

Comments
 (0)
0