8000 Plugin to typecheck attrs-generated classes by euresti · Pull Request #4397 · python/mypy · GitHub
[go: up one dir, main page]

Skip to content

Plugin to typecheck attrs-generated classes #4397

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 84 commits into from
Feb 13, 2018
Merged
Changes from 1 commit
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
005547a
WIP: attrs_plugin_real
euresti Jan 11, 2018
ecd0164
Handle untyped attrs
euresti Jan 11, 2018
8f4dbd7
Handle auto_attribs and refactor
euresti Jan 11, 2018
2b55334
WIP: attrs_plugin
euresti Jan 11, 2018
83b008c
WIP: attrs_plugin
euresti Jan 11, 2018
3bc5c11
WIP: attrs_plugin
euresti Jan 11, 2018
aa55fe5
WIP: attrs_plugin
euresti Jan 11, 2018
5f897b7
WIP: attrs_plugin
euresti Jan 11, 2018
8867278
Support cmp
euresti Jan 11, 2018
cd2aded
WIP: attrs_plugin
euresti Jan 11, 2018
104f9f4
WIP: attrs_plugin
euresti Jan 11, 2018
6856500
WIP: attrs_plugin
euresti Jan 11, 2018
0e359a8
WIP: attr_pyi
euresti Jan 11, 2018
ebb9834
WIP: attr_pyi
euresti Jan 11, 2018
001a16a
WIP: attr_pyi
euresti Jan 11, 2018
453c8ae
WIP: attrs_plugin
euresti Jan 11, 2018
d26bdb6
WIP: attrs_plugin
euresti Jan 11, 2018
e78c040
WIP: attr_pyi
euresti Jan 11, 2018
fd1a24b
WIP: attr_pyi
euresti Jan 11, 2018
1680e8f
Support inheritance
euresti Jan 11, 2018
521d215
Fix lint and mypy
euresti Jan 11, 2018
071c0fd
Optionals
euresti Jan 11, 2018
6986547
Never forget Python 3.5
euresti Jan 11, 2018
5d60d99
Sigh
euresti Jan 11, 2018
2284797
Support nested classes
euresti Jan 11, 2018
89f3537
WIP: attrs_plugin
euresti Jan 11, 2018
353319c
Simplify stubs
euresti Jan 11, 2018
72cc0a7
Cleanup tests
euresti Jan 11, 2018
e48091f
Get rid of self type.
euresti Jan 11, 2018
2d0f5dd
Comments
euresti Jan 11, 2018
f7f6033
Remove magic numbers
euresti Jan 11, 2018
2ff3a19
Cleanup argument pulling
euresti Jan 11, 2018
d06bd74
Fix multiple inheritance override
euresti Jan 11, 2018
6fbe81f
Support type=
euresti Jan 11, 2018
ccfd6c0
Support frozen=True
euresti Jan 11, 2018
4d9c182
Support @attr.dataclass
euresti Jan 11, 2018
2fc6f5f
Add more tests
euresti Jan 11, 2018
d11c138
Cleanup
euresti Jan 11, 2018
dd947d2
Support x.default decorator
euresti Jan 11, 2018
ce2cac6
Fix issue with classmethods calling cls
euresti Jan 11, 2018
d05c364
Get rid of accept. It was causing weird issues
euresti Jan 11, 2018
695b422
Fix Forward Reference Resolution issue
euresti Jan 11, 2018
76fa908
Better types for argument lookups
euresti Jan 11, 2018
405204d
Use type= to set the annotation
euresti Jan 11, 2018
536117f
Add Python2 annotation test
euresti Jan 11, 2018
0fa72a9
Support convert= (Happy New Year!)
euresti Jan 11, 2018
6506d7f
Make cmp methods work with superclasses
euresti Jan 11, 2018
5d16792
Do the correct MRO handling
euresti Jan 11, 2018
9fb4e30
Don't traverse superclasses a 2nd time
euresti Jan 11, 2018
256b446
Support converter
euresti Jan 11, 2018
d566806
Remove LastUpdatedOrderedDict
euresti Jan 11, 2018
5b7b290
Move Attribute list into plugin, to avoid touching TypeInfo
euresti Jan 11, 2018
b2f075e
Get the argument ordering from the stub instead of hardcoded
euresti Jan 11, 2018
f6f2f41
De-flake
euresti Jan 11, 2018
6f60854
Some CR
euresti Jan 17, 2018
7492278
Get rid of called_function helper
euresti Jan 18, 2018
28c7b8d
Add some more tests
euresti Jan 18, 2018
aeb01be
Small cleanups
euresti Jan 19, 2018
4fcb5dd
abstract
euresti Jan 19, 2018
1ddb89e
Add more tests.
euresti Jan 19, 2018
5155480
Split get_bool_argument
euresti Jan 19, 2018
ff00d73
Fix auto_attribs=True
euresti Jan 20, 2018
bc9db4b
Add some python 2 checks
euresti Jan 20, 2018
a4f34a8
Warn against sharing one attrib
euresti Jan 21, 2018
d7a8e2e
Move helper methods out
euresti Jan 22, 2018
6b86aa2
Merge branch 'master' into attrs_plugin
euresti Jan 22, 2018
8eae273
Fix for new messages
euresti Jan 22, 2018
93f80d6
Cleanup imports
euresti Jan 22, 2018
8cdcbfa
De-flake
euresti Jan 22, 2018
70e25dd
Cleanup comment
euresti Jan 22, 2018
fc2b22b
Add warning for using convert
euresti Jan 24, 2018
95e3ac0
Fix bug found when using real stubs
euresti Jan 30, 2018
a076957
Move all attrs code to attrs_plugin.py
euresti Feb 2, 2018
3fa9e4f
Refactor code, split into helper methods
euresti Feb 3, 2018
cad44bb
Work around circular import
euresti Feb 4, 2018
0240092
Support incremental mode with attrs
euresti Feb 5, 2018
c37147d
A little more refactoring
euresti Feb 5, 2018
cb795cd
Handle converter in one place
euresti Feb 5, 2018
b2b538c
Merge branch 'master' into attrs_plugin
euresti Feb 5, 2018
d1e3e0b
Sync typeshed
euresti Feb 8, 2018
6ef4a63
Merge branch 'master' into attrs_plugin
euresti Feb 10, 2018
feab654
Fix test
euresti Feb 10, 2018
641230d
Move attrs_plugin to plugins/attrs
euresti Feb 11, 2018
bf49a87
Add more incremental tests
euresti Feb 13, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
8000
Diff view
Prev Previous commit
Next Next commit
Handle auto_attribs and refactor
  • Loading branch information
euresti committed Jan 11, 2018
commit 8f4dbd7c6de3f3017d844dfe765f18e8f2e2142b
95 changes: 53 additions & 42 deletions mypy/plugin.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@

from collections import OrderedDict
from abc import abstractmethod
from typing import Callable, List, Tuple, Optional, NamedTuple, TypeVar
from typing import Callable, List, Tuple, Optional, NamedTuple, TypeVar, Set, \
cast

from mypy import messages
from mypy.nodes import Expression, StrExpr, IntExpr, UnaryExpr, Context, \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our style is to use (...) for multiline imports instead of \, see for example right below from mypy.types import etc.

DictExpr, ClassDef, Argument, Var, TypeInfo, FuncDef, Block, \
SymbolTableNode, MDEF, CallExpr, RefExpr, AssignmentStmt, TempNode, \
ARG_POS, ARG_OPT, EllipsisExpr
ARG_POS, ARG_OPT, EllipsisExpr, NameExpr
from mypy.types import (
Type, Instance, CallableType, TypedDictType, UnionType, NoneTyp, FunctionLike, TypeVarType,
AnyType, TypeList, UnboundType, TypeOfAny
Expand Down Expand Up @@ -269,6 +271,7 @@ def get_class_decorator_hook(self, fullname: str
) -> Optional[Callable[[ClassDefContext], None]]:
if fullname == 'attr.s':
return attr_s_callback
return None


def open_callback(ctx: FunctionContext) -> Type:
Expand Down Expand Up @@ -405,7 +408,7 @@ def add_method(
arg_names = [arg.variable.name() for arg in args]
arg_kinds = [arg.kind for arg in args]
assert None not in arg_types
signature = CallableType(arg_types, arg_kinds, arg_names,
signature = CallableType(cast(List[Type], arg_types), arg_kinds, arg_names,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to avoid the cast, why it is necessary? We try to avoid using casts.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The named tuple code does the same thing. arg_types is technically a List[Optional[Type]] but note the assert None not in arg_types right above to make sure it's right.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the cast is not required with that assert. Have you actually tried removing it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I get:

mypy/plugin.py:580: error: Argument 1 to "CallableType" has incompatible type "List[Optional[Type]]"; expected "List[Type]"

When running mypy mypy/plugin.py --strict

ret_type, function_type)
func = FuncDef(method_name, args, Block([]))
func.info = info
Expand All @@ -419,24 +422,25 @@ def attr_s_callback(ctx: ClassDefContext) -> None:
"""Add an __init__ method to classes decorated with attr.s."""
# TODO: Add __cmp__ methods.

def get_bool_argument(call: CallExpr, name: str, default: bool):
def get_bool_argument(call: CallExpr, name: str, default: Optional[bool]) -> Optional[bool]:
for arg_name, arg_value in zip(call.arg_names, call.args):
if arg_name == name:
# TODO: Handle None being returned here.
return ctx.api.parse_bool(arg_value)
return default

def get_argument(call: CallExpr, name: Optional[str], num: Optional[int]):
def get_argument(call: CallExpr, name: Optional[str], num: Optional[int]) -> Optional[Expression]:
for i, (attr_name, attr_value) in enumerate(zip(call.arg_names, call.args)):
if num is not None and i == num:
if num is not None and not attr_name and i == num:
return attr_value
if name and attr_name == name:
return attr_value
return None

def called_function(expr: Expression):
def called_function(expr: Expression) -> Optional[str]:
if isinstance(expr, CallExpr) and isinstance(expr.callee, RefExpr):
return expr.callee.fullname
return None

decorator = ctx.reason
if isinstance(decorator, CallExpr):
Expand All @@ -449,7 +453,6 @@ def called_function(expr: Expression):
auto_attribs = False

if not init:
print("Nothing to do", init)
return

print(f"{ctx.cls.info.fullname()} init={init} auto={auto_attribs}")
Expand All @@ -460,48 +463,56 @@ def called_function(expr: Expression):
names = [] # type: List[str]
types = [] # type: List[Type]
has_default = set() # type: Set[str]

def add_init_argument(name: str, typ:Optional[Type], default: bool, context:Context) -> None:
if not default and has_default:
ctx.api.fail(
"Non-default attributes not allowed after default attributes.",
context)
if not typ:
ctx.api.fail(messages.NEED_ANNOTATION_FOR_VAR, context)
typ = AnyType(TypeOfAny.unannotated)

names.append(name)
assert typ is not None
types.append(typ)
if default:
has_default.add(name)

def is_class_var(expr: NameExpr) -> bool:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move this function to the outer scope (or maybe actually move it to nodes.py and import it from there). It can be useful on its own.

# import pdb; pdb.set_trace()
if isinstance(expr.node, Var):
return expr.node.is_classvar
return False

for stmt in ctx.cls.defs.body:
if isinstance(stmt, AssignmentStmt):
name = stmt.lvalues[0].name.lstrip("_")
typ = (AnyType(TypeOfAny.unannotated) if stmt.type is None
else ctx.api.anal_type(stmt.type))

if isinstance(stmt.rvalue, TempNode):
print(f"{name}: {typ}")
# `x: int` (without equal sign) assigns rvalue to TempNode(AnyType())
if has_default:
print("DEFAULT ISSUE")
elif called_function(stmt.rvalue) == 'attr.ib':
if isinstance(stmt, AssignmentStmt) and isinstance(stmt.lvalues[0], NameExpr):
lhs = stmt.lvalues[0]
name = lhs.name.lstrip("_")
typ = stmt.type
print(name, typ, is_class_var(lhs))

if called_function(stmt.rvalue) == 'attr.ib':
# Look for a default value in the call.
if get_argument(stmt.rvalue, "default", 0):
has_default.add(name)
print(f"{name} = attr.ib(default=...)")
else:
if has_default:
ctx.api.fail("Non-default attributes not allowed after default attributes.", stmt.rvalue)
print(f"{name} = attr.ib()")

names.append(name)
types.append(typ)
assert isinstance(stmt.rvalue, CallExpr)
add_init_argument(name, typ, bool(get_argument(stmt.rvalue, "default", 0)), stmt)
else:
print(f"{name} = {stmt.rvalue}")
# rhs[name] = stmt.rvalue

any_type = AnyType(TypeOfAny.unannotated)

print(names, types, has_default)
if auto_attribs and not is_class_var(lhs):
# `x: int` (without equal sign) assigns rvalue to TempNode(AnyType())
has_rhs = not isinstance(stmt.rvalue, TempNode)
add_init_argument(name, typ, has_rhs, stmt)

args = []
for (name, typ) in zip(names, types):
var = Var(name, typ)
kind = ARG_OPT if name in has_default else ARG_POS
args.append(Argument(var, var.type, EllipsisExpr(), kind))
init_args = [
Argument(Var(name, typ), typ, EllipsisExpr(),
ARG_OPT if name in has_default else ARG_POS)
for (name, typ) in zip(names, types)
]

add_method(
info=info,
method_name='__init__',
args=args,
args=init_args,
ret_type=NoneTyp(),
self_type=ctx.api.named_type(info.name()),
function_type=ctx.api.named_type('__builtins__.function'),
)
)
0