8000 [conflict] Fix #1855: Multiassign from Union by elazarg · Pull Request #2154 · python/mypy · GitHub
[go: up one dir, main page]

Skip to content

[conflict] Fix #1855: Multiassign from Union #2154

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 54 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
7e043d0
handle union, add test
elazarg Sep 18, 2016
9a50d73
kill blank
elazarg Sep 18, 2016
5c4d86e
more tests
elazarg Sep 18, 2016
60cfbbb
handle binding
elazarg Sep 19, 2016
5e9e0f2
try to minimize visual difference
elazarg Sep 19, 2016
71f8475
(cont.)
elazarg Sep 19, 2016
42b6e73
add tests
elazarg Sep 21, 2016
0560bd8
no binder yet
elazarg Sep 23, 2016
7683ee2
Sync typeshed
Sep 23, 2016
8f0316f
Docs: Replace "if False" trick with "if TYPE_CHECKING". (#2181)
davidfstr Sep 25, 2016
fde83d5
Add column number support (#2163)
bavardage Sep 25, 2016
2fbf387
Fix #2070 - bytes formatting incorrect in python 3 (#2168)
buckbaskin Sep 26, 2016
da3a516
Support rebinding on multiassignment from union
elazarg Sep 27, 2016
ab60317
Merge remote-tracking branch 'upstream/master' into multiassign_union
elazarg Sep 27, 2016
6bb5519
more tests
elazarg Sep 27, 2016
0d2c594
Bare `Tuple` should mean `Tuple[Any, ...]`. (#2185)
gvanrossum Sep 27, 2016
2c73db5
Add --recursive and --ignore-errors flags to stubgen (#2183)
rtpg Sep 27, 2016
d2c6e8b
Fix lint
Sep 27, 2016
5c03ac9
Make runtests.py output less verbose
Sep 27, 2016
c3e1b35
Small module name translation fix for fast parser (#2187)
ddfisher Sep 27, 2016
87f7d90
Fix overzealous new submodule check (#2189)
Michael0x2a Sep 28, 2016
c752b42
Sync typeshed
Sep 28, 2016
5f0d02c
Implement PEP 526 Variable Annotations Syntax (#2131)
ilevkivskyi Sep 28, 2016
bec9454
Fix a fixup bug for nested class references.
Sep 28, 2016
a9fac0b
Add gitter chat room
gvanrossum Sep 29, 2016
ea23ac0
Rename --suppress-error-context to --hide-error-context (#2192)
gvanrossum Sep 29, 2016
4a34623
Add gitter badge for the new chat room (#2202)
gvanrossum Sep 30, 2016
2db6a11
Don't crash in fast parser on complex numbers in Python 2 mode
Oct 1, 2016
6151d20
Add more precise types to fastparse{,2}.py (#2204)
elazarg Oct 1, 2016
4a457fc
Tighter types for binder, checker and checkexpr (#2205)
elazarg Oct 1, 2016
c8a9b52
Tighten types for semanal (#2207)
elazarg Oct 2, 2016
af0b24a
Tighten types for parse.py and some more (#2208)
elazarg Oct 2, 2016
d5c5c02
Make Expression and Statement separable (#2209)
elazarg Oct 3, 2016
640921b
Make Expression, Statement true subclasses of Node. See #1783.
Oct 3, 2016
20aea69
Add mypy.ini. Fix strict none errors in the test subpackage. (#2200)
gvanrossum Oct 3, 2016
a3b6546
Sync typeshed
Oct 3, 2016
91a2275
Document support for Python 3.6 features.
Oct 3, 2016
bc4a390
Document --show-column-numbers and --scripts-are-modules
Oct 3, 2016
2e6cb95
Shorten "inherently stale" log message if all nodes in scc are stale
Oct 3, 2016
d6a7069
Fix lint error
Oct 3, 2016
eb16d1d
Remove weird implementation of --show_traceback and --pdb. (#2216)
gvanrossum Oct 4, 2016
dd71ddc
RIP weak mode. (#2217)
gvanrossum Oct 4, 2016
38651c4
handle union, add test
elazarg Sep 18, 2016
9000099
kill blank
elazarg Sep 18, 2016
f20f3d6
more tests
elazarg Sep 18, 2016
61be4e9
handle binding
elazarg Sep 19, 2016
9830cb4
try to minimize visual difference
elazarg Sep 19, 2016
59dc8b7
(cont.)
elazarg Sep 19, 2016
7f304e4
add tests
elazarg Sep 21, 2016
ff1ca80
no binder yet
elazarg Sep 23, 2016
168087e
Support rebinding on multiassignment from union
elazarg Sep 27, 2016
3f198cf
more tests
elazarg Sep 27, 2016
4fa059f
Rebase
elazarg Oct 5, 2016
ab35a4c
Merge
elazarg Oct 5, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
handle binding
  • Loading branch information
elazarg committed Oct 5, 2016
commit 61be4e961ffe8e9ffe53b59caaea8d6c8ac02b1b
123 changes: 64 additions & 59 deletions mypy/checker.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
from mypy.erasetype import erase_typevars
from mypy.expandtype import expand_type
from mypy.visitor import NodeVisitor
from mypy.join import join_types
from mypy.join import join_types, join_type_list
from mypy.treetransform import TransformVisitor
from mypy.meet import meet_simple, nearest_builtin_ancestor, is_overlapping_types
from mypy.binder import ConditionalTypeBinder
Expand Down Expand Up @@ -1037,8 +1037,13 @@ def check_assignment(self, lvalue: Lvalue, rvalue: Expression, infer_lvalue_type
new_syntax: bool = False) -> None:
"""Type check a single assignment: lvalue = rvalue."""
if isinstance(lvalue, TupleExpr) or isinstance(lvalue, ListExpr):
self.check_assignment_to_multiple_lvalues(lvalue.items, rvalue, lvalue,
infer_lvalue_type)
if isinstance(rvalue, TupleExpr) or isinstance(rvalue, ListExpr):
self.check_multi_assign_literal(lvalue.items, rvalue, lvalue, infer_lvalue_type)
return
# Infer the type of an ordinary rvalue expression.
# TODO maybe elsewhere; redundant
rvalue_type = self.accept(rvalue)
self.check_multi_assign(lvalue.items, rvalue, rvalue_type, lvalue, infer_lvalue_type)
else:
lvalue_type, index_lvalue, inferred = self.check_lvalue(lvalue)
if lvalue_type:
Expand Down Expand Up @@ -1089,44 +1094,6 @@ def check_assignment(self, lvalue: Lvalue, rvalue: Expression, infer_lvalue_type
if inferred:
self.infer_variable_type(inferred, lvalue, self.accept(rvalue),
rvalue)

def check_assignment_to_multiple_lvalues(self, lvalues: List[Lvalue], rvalue: Expression,
context: Context,
infer_lvalue_type: bool = True) -> None:
if isinstance(rvalue, TupleExpr) or isinstance(rvalue, ListExpr):
# Recursively go into Tuple or List expression rhs instead of
# using the type of rhs, because this allowed more fine grained
# control in cases like: a, b = [int, str] where rhs would get
# type List[object]
# Tuple is also special cased to handle mutually nested lists and tuples

rvalues = rvalue.items

if self.check_rvalue_count_in_assignment(lvalues, len(rvalues), context):
star_index = next((i for i, lv in enumerate(lvalues) if
isinstance(lv, StarExpr)), len(lvalues))

left_lvs = lvalues[:star_index]
star_lv = cast(StarExpr,
lvalues[star_index]) if star_index != len(lvalues) else None
right_lvs = lvalues[star_index + 1:]

left_rvs, star_rvs, right_rvs = self.split_around_star(
rvalues, star_index, len(lvalues))

lr_pairs = list(zip(left_lvs, left_rvs))
if star_lv:
rv_list = ListExpr(star_rvs)
rv_list.set_line(rvalue.get_line())
lr_pairs.append((star_lv.expr, rv_list))
lr_pairs.extend(zip(right_lvs, right_rvs))

for lv, rv in lr_pairs:
self.check_assignment(lv, rv, infer_lvalue_type)
else:
rvalue_type = self.accept(rvalue) # TODO maybe elsewhere; redundant
self.check_multi_assignment(lvalues, rvalue, rvalue_type, context, infer_lvalue_type)

def check_rvalue_count_in_assignment(self, lvalues: List[Lvalue], rvalue_count: int,
context: Context) -> bool:
if any(isinstance(lvalue, StarExpr) for lvalue in lvalues):
Expand All @@ -1140,39 +1107,77 @@ def check_rvalue_count_in_assignment(self, lvalues: List[Lvalue], rvalue_count:
return False
return True

def check_multi_assignment(self, lvalues: List[Lvalue],
rvalue: Expression,
rvalue_type: Type,
context: Context,
infer_lvalue_type: bool = True) -> None:
"""Check the assignment of one rvalue to a number of lvalues."""
# Infer the type of an ordinary rvalue expression.
undefined_rvalue = False

def check_multi_assign_literal(self, lvalues: List[Lvalue],
rvalue: Union[ListExpr, TupleExpr],
context: Context, infer_lvalue_type: bool = True) -> None:
# Recursively go into Tuple or List expression rhs instead of
# using the type of rhs, because this allowed more fine grained
# control in cases like: a, b = [int, str] where rhs would get
# type List[object]
# Tuple is also special cased to handle mutually nested lists and tuples
rvalues = rvalue.items
if self.check_rvalue_count_in_assignment(lvalues, len(rvalues), context):
star_index = next((i for (i, lv) in enumerate(lvalues) if isinstance(lv, StarExpr)),
len(lvalues))
left_lvs = lvalues[:star_index]
star_lv = cast(StarExpr, lvalues[star_index]) if star_index != len(lvalues) else None
right_lvs = lvalues[star_index + 1:]
left_rvs, star_rvs, right_rvs = self.split_around_star(
rvalues, star_index, len(lvalues))
lr_pairs = list(zip(left_lvs, left_rvs))
if star_lv:
rv_list = ListExpr(star_rvs)
rv_list.set_line(rvalue.get_line())
lr_pairs.append((star_lv.expr, rv_list))
lr_pairs.extend(zip(right_lvs, right_rvs))
for lv, rv in lr_pairs:
self.check_assignment(lv, rv, infer_lvalue_type)

def check_multi_assign(self, lvalues: List[Lvalue],
rvalue: Expression, rvalue_type: Type,
context: Context, infer_lvalue_type: bool = True,
undefined_rvalue = False) -> None:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No annotation for undefined_rvalue.

if isinstance(rvalue_type, AnyType):
for lv in lvalues:
if isinstance(lv, StarExpr):
lv = lv.expr
self.check_assignment(lv, self.temp_node(AnyType(), context), infer_lvalue_type)
elif isinstance(rvalue_type, TupleType):
self.check_multi_assignment_from_tuple(lvalues, rvalue, rvalue_type,
context, undefined_rvalue, infer_lvalue_type)
self.check_multi_assign_from_tuple(lvalues, rvalue, rvalue_type,
context, undefined_rvalue, infer_lvalue_type)
elif isinstance(rvalue_type, UnionType):
for item in rvalue_type.items:
self.check_multi_assignment(lvalues, rvalue, item, context, infer_lvalue_type)
self.check_multi_assign_from_union(lvalues, rvalue, rvalue_type,
context, infer_lvalue_type)
elif isinstance(rvalue_type, Instance) and self.type_is_iterable(rvalue_type):
self.check_multi_assignment_from_iterable(lvalues, rvalue_type,
context, infer_lvalue_type)
self.check_multi_assign_from_iterable(lvalues, rvalue_type,
context, infer_lvalue_type)
else:
self.msg.type_not_iterable(rvalue_type, context)

def type_is_iterable(self, rvalue_type: Type) -> bool:
return is_subtype(rvalue_type, self.named_generic_type('typing.Iterable',
[AnyType()]))

def check_multi_assignment_from_iterable(self, lvalues: List[Node], rvalue_type: Instance,
context: Context,
infer_lvalue_type: bool = True) -> None:
def check_multi_assign_from_union(self, lvalues: List[Expression], rvalue: Expression,
rvalue_type: UnionType, context: Context,
infer_lvalue_type: bool) -> None:
union_types = tuple([] for _ in lvalues) # type: Tuple[List[Type], ...]
for item in rvalue_type.items:
self.check_multi_assign(lvalues, rvalue, item, context, infer_lvalue_type,
undefined_rvalue=True)
for t, lv in zip(union_types, lvalues):
t.append(self.type_map[lv])
for ut, lv in zip(union_types, lvalues):
_1, _2, inferred = self.check_lvalue(lv)
union = join_type_list(ut)
if inferred:
self.set_inferred_type(inferred, lv, union)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll probably need to put the new type to the binder (maybe self.binder.assign_type(...)). Also add a test case for this. Example where this may make a difference:

class A: pass
class B(A): pass
class C(A): pass
x = None # type: object
a = [] # type: Union[List[B], List[C]]
x, y = a
# now type of x should be A
x = 1  # should be ok
# now type of x should be int

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The assignment to a is not accepted by mypy. I opened #2164

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simple assign_type did not work; I needed something more intrusive than that. I don't like this solution, to be honest. I want to ask the type to split itself, without side effects, but that's not how things works.

else:
self.store_type(lv, union)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The binding of the joined type is "retrofitted". I am not sure this is the best way to do it - maybe inferred vars should simply accumulate through joins?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retrofitting like this may not be a problem, but I suspect it might be. Accumulating through joins might not be any better. The most correct approach would be to infer the entire type for an lvalue before setting it, so that intermediate values can't leak. I don't know, but it may be possible that the intermediate lvalue types leak to the rvalue in some weird corner cases.

I'd recommend not worrying about this too much, but I'd suggest adding a TODO comment about the potential issues.


def check_multi_assign_from_iterable(self, lvalues: List[Expression], rvalue_type: Instance,
context: Context,
infer_lvalue_type: bool = True) -> None:
item_type = self.iterable_item_type(rvalue_type)
for lv in lvalues:
if isinstance(lv, StarExpr):
Expand Down
13 changes: 11 additions & 2 deletions test-data/unit/check-unions.test
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ a = C() # type: C[int, int]
b = a.f('a')
a.f(b) # E: Argument 1 to "f" of "C" has incompatible type "int"; expected "str"

[case testUnionMultissign]
[case testUnionMultiassign1]
from typing import Union, Tuple, Any

a = None # type: Tuple[int]
Expand All @@ -139,8 +139,17 @@ a = None # type: Tuple[int]
b = None # type: Union[Tuple[int], Tuple[float]]
(b1,) = b
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reveal type of b1.


c = None # type: Union[Tuple[int, int], Tuple[float, float]]
[case testUnionMultiassign2]
from typing import Union, Tuple

c = None # type: Union[Tuple[int, int], Tuple[int, str]]
q = None # type: Tuple[int, float]
(c1, c2) = c
reveal_type(c1) # E: Revealed type is 'builtins.int'
reveal_type(c2) # E: Revealed type is 'builtins.object'

[case testUnionMultiassign3]
from typing import Union, Tuple, Any

d = None # type: Union[Any, Tuple[float, float]]
(d1, d2) = d
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reveal types of d1 and d2.

Expand Down
0