8000 Explicit types for bindables ("Tighten Types" continued) by elazarg · Pull Request #2238 · python/mypy · GitHub
[go: up one dir, main page]

Skip to content

Explicit types for bindables ("Tighten Types" continued) #2238

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Mar 27, 2017
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
rename binder api
  • Loading branch information
elazarg committed Oct 6, 2016
commit d01d2f1cd9ac9175767be25a739e92bdffe04eb1
72 changes: 41 additions & 31 deletions mypy/binder.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,24 @@
from typing import (Any, Dict, List, Set, Iterator, Union)
from typing import (Dict, List, Set, Iterator, Union)
from contextlib import contextmanager

from mypy.types import Type, AnyType, PartialType
from mypy.nodes import (Node, Expression, Var, RefExpr, SymbolTableNode)
from mypy.nodes import (Node as Expression, Var, RefExpr, SymbolTableNode)

from mypy.subtypes import is_subtype
from mypy.join import join_simple
from mypy.sametypes import is_same_type

from mypy.nodes import IndexExpr, MemberExpr, NameExpr


BindableTypes = (IndexExpr, MemberExpr, NameExpr)
BindableExpression = Union[IndexExpr, MemberExpr, NameExpr]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the point of introducing this BindableExpression. It seems to duplicate information with which Expressions have (non-None) literals. It should be fine to just use Expression everywhere.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lists of literals have non-None literals.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the branch with literal() and literal_hash() functions:
https://github.com/elazarg/mypy/blob/literal_hash/mypy/nodes.py#L2351-L2421

class Frame(Dict[Any, Type]):
pass

Key = object

class Key(AnyType):

class Frame(Dict[object, Type]):
pass


Expand Down Expand Up @@ -85,7 +90,7 @@ def push_frame(self) -> Frame:
self.options_on_return.append([])
return f

def _push(self, key: Key, type: Type, index: int=-1) -> None:
def _put(self, key: Key, type: Type, index: int=-1) -> None:
self.frames[index][key] = type

def _get(self, key: Key, index: int=-1) -> Type:
Expand All @@ -96,20 +101,24 @@ def _get(self, key: Key, index: int=-1) -> Type:
return self.frames[i][key]
return None

def push(self, node: Node, typ: Type) -> None:
if not node.literal:
def put_if_bindable(self, expr: Expression, typ: Type) -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the change to put, but _if_bindable doesn't add any information (there's no contrasting put_even_if_not_bindable) and is pretty long.

Basically, put is just the way for the rest of the type checker to pass information into the binder, and I think of it as the binder's business whether it bothers to record a piece of information it is given or not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it adds no information only if you know exactly how the binder works. When I read this code I had no idea that some expressions can be simply ignored. I do not like long names, but shorter names feels misleading.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course it is the binder's business, but the reader should know that it is possible.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A docstring (or general documentation on the binder) seems like a better vehicle for that.

if not isinstance(expr, BindableTypes):
return
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not really sure about the need for this.

if not expr.literal:
return
key = node.literal_hash
key = expr.literal_hash
if key not in self.declarations:
self.declarations[key] = self.get_declaration(node)
self.declarations[key] = self.get_declaration(expr)
self._add_dependencies(key)
self._push(key, typ)
self._put(key, typ)

def get(self, expr: Union[Expression, Var]) -> Type:
def get(self, expr: Expression) -> Type:
if not isinstance(expr, BindableTypes):
return None
return self._get(expr.literal_hash)

def cleanse(self, expr: Expression) -> None:
"""Remove all references to a Node from the binder."""
def cleanse(self, expr: BindableExpression) -> None:
"""Remove all references to an Expression from the binder."""
self._cleanse_key(expr.literal_hash)

def _cleanse_key(self, key: Key) -> None:
Expand Down Expand Up @@ -144,7 +153,7 @@ def update_from_options(self, frames: List[Frame]) -> bool:
for other in resulting_values[1:]:
type = join_simple(self.declarations[key], type, other)
if not is_same_type(type, current_value):
self._push(key, type)
self._put(key, type)
changed = True

return changed
Expand All @@ -165,19 +174,20 @@ def pop_frame(self, fall_through: int = 0) -> Frame:

return result

def get_declaration(self, node: Node) -> Type:
if isinstance(node, (RefExpr, SymbolTableNode)) and isinstance(node.node, Var):
type = node.node.type
if isinstance(type, PartialType):
return None
return type
else:
return None
def get_declaration(self, expr: BindableExpression) -> Type:
assert not isinstance(expr, SymbolTableNode)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assert can't trigger given the definition of BindableExpression right? Or am I misunderstanding?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a leftover from an old check,
if isinstance(node, (RefExpr, SymbolTableNode)) and isinstance(node.node, Var):
I should remove it.

if isinstance(expr, RefExpr) and isinstance(expr.node, Var):
type = expr.node.type
if not isinstance(type, PartialType):
return type
return None

def assign_type(self, expr: Expression,
type: Type,
declared_type: Type,
restrict_any: bool = False) -> None:
def assign_type_if_bindable(self, expr: Expression,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, _if_bindable seems extraneous.

type: Type,
declared_type: Type,
restrict_any: bool = False) -> None:
if not isinstance(expr, BindableTypes):
return None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check doesn't seem necessary.

if not expr.literal:
return
self.invalidate_dependencies(expr)
Expand All @@ -202,16 +212,16 @@ def assign_type(self, expr: Expression,
and not restrict_any):
pass
elif isinstance(type, AnyType):
self.push(expr, declared_type)
self.put_if_bindable(expr, declared_type)
else:
self.push(expr, type)
self.put_if_bindable(expr, type)

for i in self.try_frames:
# XXX This should probably not copy the entire frame, but
# just copy this variable into a single stored frame.
self.allow_jump(i)

def invalidate_dependencies(self, expr: Expression) -> None:
def invalidate_dependencies(self, expr: BindableExpression) -> None:
"""Invalidate knowledge of types that include expr, but not expr itself.

For example, when expr is foo.bar, invalidate foo.bar.baz.
Expand All @@ -222,7 +232,7 @@ def invalidate_dependencies(self, expr: Expression) -> None:
for dep in self.dependencies.get(expr.literal_hash, set()):
self._cleanse_key(dep)

def most_recent_enclosing_type(self, expr: Expression, type: Type) -> Type:
def most_recent_enclosing_type(self, expr: BindableExpression, type: Type) -> Type:
if isinstance(type, AnyType):
return self.get_declaration(expr)
key = expr.literal_hash
Expand Down
22 changes: 11 additions & 11 deletions mypy/checker.py
Original file line number Diff line number Diff line change
Expand Up @@ -1079,10 +1079,10 @@ def check_assignment(self, lvalue: Lvalue, rvalue: Expression, infer_lvalue_type
rvalue_type = self.check_simple_assignment(lvalue_type, rvalue, lvalue)

if rvalue_type and infer_lvalue_type:
self.binder.assign_type(lvalue,
rvalue_type,
lvalue_type,
False)
self.binder.assign_type_if_bindable(lvalue,
rvalue_type,
lvalue_type,
False)
elif index_lvalue:
self.check_indexed_assignment(index_lvalue, rvalue, rvalue)

Expand Down Expand Up @@ -1535,14 +1535,14 @@ def visit_if_stmt(self, s: IfStmt) -> Type:
with self.binder.frame_context(2):
if if_map:
for var, type in if_map.items():
self.binder.push(var, type)
self.binder.put_if_bindable(var, type)

self.accept(b)
breaking_out = breaking_out and self.binder.last_pop_breaking_out

if else_map:
for var, type in else_map.items():
self.binder.push(var, type)
self.binder.put_if_bindable(var, type)
if else_map is None:
# The condition is always true => remaining elif/else blocks
# can never be reached.
Expand Down Expand Up @@ -1585,7 +1585,7 @@ def visit_assert_stmt(self, s: AssertStmt) -> Type:

if true_map:
for var, type in true_map.items():
self.binder.push(var, type)
self.binder.put_if_bindable(var, type)

def visit_raise_stmt(self, s: RaiseStmt) -> Type:
"""Type check a raise statement."""
Expand Down Expand Up @@ -1800,10 +1800,10 @@ def flatten(t: Expression) -> List[Expression]:
s.expr.accept(self)
for elt in flatten(s.expr):
if isinstance(elt, NameExpr):
self.binder.assign_type(elt,
DeletedType(source=elt.name),
self.binder.get_declaration(elt),
False)
self.binder.assign_type_if_bindable(elt,
DeletedType(source=elt.name),
self.binder.get_declaration(elt),
False)
return None

def visit_decorator(self, e: Decorator) -> Type:
Expand Down
17 changes: 6 additions & 11 deletions mypy/checkexpr.py
Original file line number Diff line number Diff line change
Expand Up @@ -150,18 +150,13 @@ def analyze_ref_expr(self, e: RefExpr, lvalue: bool = False) -> Type:
return result

def analyze_var_ref(self, var: Var, context: Context) -> Type:
if not var.type:
if var.type:
return var.type
else:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just double-checking that what you changed here isn't an accidental merge conflict? You've done more here than just revers the sense of the check and swap the then/else clauses.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part is related to #2222 - binder.get() does not accept Var; val is always None.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, I would like to extract literal-handling to a couple of functions, and avoid putting them in the Expression classes. I already have the branch ready. Will you consider it? It does not solve anything, but I think it's nice to have this information in a single place, instead of scattered around.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be worth calling out this "change" explicitly in the commit message, since it does not obviously look like a mere refactoring.

if not var.is_ready and self.chk.in_checked_function():
self.chk.handle_cannot_determine_type(var.name(), context)
# Implicit 'Any' type.
return AnyType()
else:
# Look up local type of variable with type (inferred or explicit).
val = self.chk.binder.get(var)
if val is None:
return var.type
else:
return val

def visit_call_expr(self, e: CallExpr) -> Type:
"""Type check a call expression."""
Expand Down Expand Up @@ -1201,7 +1196,7 @@ def check_boolean_op(self, e: OpExpr, context: Context) -> Type:
with self.chk.binder.frame_context():
if right_map:
for var, type in right_map.items():
self.chk.binder.push(var, type)
self.chk.binder.put_if_bindable(var, type)

right_type = self.accept(e.right, left_type)

Expand Down Expand Up @@ -1700,7 +1695,7 @@ def check_for_comp(self, e: Union[GeneratorExpr, DictionaryComprehension]) -> No

if true_map:
for var, type in true_map.items():
self.chk.binder.push(var, type)
self.chk.binder.put_if_bindable(var, type)

def visit_conditional_expr(self, e: ConditionalExpr) -> Type:
cond_type = self.accept(e.cond)
Expand Down Expand Up @@ -1739,7 +1734,7 @@ def analyze_cond_branch(self, map: Optional[Dict[Node, Type]],
with self.chk.binder.frame_context():
if map:
for var, type in map.items():
self.chk.binder.push(var, type)
self.chk.binder.put_if_bindable(var, type)
return self.accept(node, context=context)

def visit_backquote_expr(self, e: BackquoteExpr) -> Type:
Expand Down
0