8000 Fix crashes and fails in forward references by ilevkivskyi · Pull Request #3952 · python/mypy · GitHub
[go: up one dir, main page]

Skip to content

Fix crashes and fails in forward references #3952

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 43 commits into from
Sep 27, 2017
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
45e5931
Add basic tests, more details will be added when they will not crash
ilevkivskyi Aug 31, 2017
cb4caa5
Correct tests
ilevkivskyi Sep 1, 2017
1cdc980
Implement ForwardRef type, wrap UnboundType, pass SecondPass to third…
ilevkivskyi Sep 11, 2017
260ef02
Add ForwardRefRemover
ilevkivskyi Sep 11, 2017
a58a217
Add elimination patches
ilevkivskyi Sep 11, 2017
950a022
Fix replacement logic; fix newtype error formatting
ilevkivskyi Sep 11, 2017
411b24d
Fix third pass (need to go deeper)
ilevkivskyi Sep 11, 2017
b9b8528
Implement syntethic replacer
ilevkivskyi Sep 11, 2017
48d6de4
Need to go deeper (as usual)
ilevkivskyi Sep 11, 2017
ec45441
Fix postponed fallback join
ilevkivskyi Sep 11, 2017
ac32ed4
Simplify some code and add annotations
ilevkivskyi Sep 11, 2017
3fb3019
Simplify traversal logic; add loads of tests
ilevkivskyi Sep 12, 2017
f9b1320
Take care about one more special case; add few tests and dcostrings
ilevkivskyi Sep 12, 2017
cf014b8
Unify visitors
ilevkivskyi Sep 12, 2017
665236b
Add some more comments and docstrings
ilevkivskyi Sep 12, 2017
9a318aa
Add recursive type warnings
ilevkivskyi Sep 12, 2017
757fbd9
Fix lint
ilevkivskyi Sep 12, 2017
4502ce2
Also clean-up bases; add more tests and allow some previously skipped
ilevkivskyi Sep 13, 2017
3b39d40
One more TypedDict test
ilevkivskyi Sep 13, 2017
c8b28fe
Add another simple self-referrential NamedTuple test
ilevkivskyi Sep 13, 2017
9f92b0f
Fix type_override; add tests for recursive aliases; fix Callable TODO…
Sep 13, 2017
9779103
Merge branch 'master' into fix-synthetic-crashes
ilevkivskyi Sep 14, 2017
b914bdb
Merge remote-tracking branch 'upstream/master' into fix-synthetic-cra…
ilevkivskyi Sep 19, 2017
3568fdb
Skip the whole ForwardRef dance in unchecked functions
ilevkivskyi Sep 19, 2017
54d9331
Simplify test
ilevkivskyi Sep 19, 2017
b9ddacc
Fix self-check
ilevkivskyi Sep 19, 2017
5bfe9ca
Fix cross-file forward references (+test)
ilevkivskyi Sep 19, 2017
a2912e9
More tests
ilevkivskyi Sep 19, 2017
10c65b8
Merge branch 'master' into fix-synthetic-crashes
ilevkivskyi Sep 20, 2017
21dfbfe
Fix situation when recursive namedtuple appears directly in base clas…
ilevkivskyi Sep 20, 2017
f2ddbcd
Merge branch 'fix-synthetic-crashes' of https://github.com/ilevkivsky…
ilevkivskyi Sep 20, 2017
03597ee
Clean-up PR: Remove unnecesary imports, outdated comment, unnecessary…
ilevkivskyi Sep 20, 2017
649ef32
Add tests for generic classes, enums, with statements and for statements
ilevkivskyi Sep 20, 2017
83f8907
Add processing for for and with statements (+more tests)
ilevkivskyi Sep 20, 2017
13c7176
Add support for generic types with forward references
ilevkivskyi Sep 20, 2017
79b10d6
Prohibit forward refs to type vars and subscripted forward refs to al…
ilevkivskyi Sep 21, 2017
321a809
Refactor code to avoid passing semantic analyzer to type analyzer, on…
ilevkivskyi Sep 21, 2017
076c909
Address the rest of the review comments
ilevkivskyi Sep 22, 2017
c1a63ec
Improve two tests
ilevkivskyi Sep 22, 2017
97e6f47
Add one more test as suggested in #3990
ilevkivskyi Sep 23, 2017
8f52654
Address latest review comments
ilevkivskyi Sep 26, 2017
6edd078
Improve tests; Fix one more crash on NewType MRO
ilevkivskyi Sep 27, 2017
514b8bd
Fix formatting in tests
ilevkivskyi Sep 27, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Prohibit forward refs to type vars and subscripted forward refs to al…
…iases
  • Loading branch information
ilevkivskyi committed Sep 21, 2017
commit 79b10d6cf47dc606bce3f928b42c435ed9c8a87f
7 changes: 6 additions & 1 deletion mypy/semanal.py
10000
Original file line number Diff line number Diff line change
Expand Up @@ -1630,6 +1630,7 @@ def type_analyzer(self, *,
self.lookup_fully_qualified,
tvar_scope,
self.fail,
self.note,
self.plugin,
self.options,
self.is_typeshed_stub_file,
Expand All @@ -1638,6 +1639,7 @@ def type_analyzer(self, *,
allow_unnormalized=self.is_stub_file,
third_pass=third_pass)
tpan.in_dynamic_func = bool(self.function_stack and self.function_stack[-1].is_dynamic())
tpan.global_scope = not self.type and not self.function_stack
return tpan

def anal_type(self, t: Type, *,
Expand Down Expand Up @@ -1737,16 +1739,19 @@ def analyze_alias(self, rvalue: Expression,
If 'allow_unnormalized' is True, allow types like builtins.list[T].
"""
dynamic = bool(self.function_stack and self.function_stack[-1].is_dynamic())
global_scope = not self.type and not self.function_stack
res = analyze_type_alias(rvalue,
self.lookup_qualified,
self.lookup_fully_qualified,
self.tvar_scope,
self.fail,
self.note,
self.plugin,
self.options,
self.is_typeshed_stub_file,
allow_unnormalized=True,
in_dynamic_func=dynamic)
in_dynamic_func=dynamic,
global_scope=global_scope)
if res:
alias_tvars = [name for (name, _) in
res.accept(TypeVariableQuery(self.lookup_qualified, self.tvar_scope))]
Expand Down
31 changes: 23 additions & 8 deletions mypy/typeanal.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@
from mypy.nodes import (
TVAR, TYPE_ALIAS, UNBOUND_IMPORTED, TypeInfo, Context, SymbolTableNode, Var, Expression,
IndexExpr, RefExpr, nongen_builtins, check_arg_names, check_arg_kinds, ARG_POS, ARG_NAMED,
ARG_OPT, ARG_NAMED_OPT, ARG_STAR, ARG_STAR2, TypeVarExpr, FuncDef, CallExpr, NameExpr
ARG_OPT, ARG_NAMED_OPT, ARG_STAR, ARG_STAR2, TypeVarExpr, FuncDef, CallExpr, NameExpr,
Decorator
)
from mypy.tvar_scope import TypeVarScope
from mypy.sametypes import is_same_type
Expand Down Expand Up @@ -59,11 +60,13 @@ def analyze_type_alias(node: Expression,
lookup_fqn_func: Callable[[str], SymbolTableNode],
tvar_scope: TypeVarScope,
fail_func: Callable[[str, Context], None],
note_func: Callable[[str, Context], None],
plugin: Plugin,
options: Options,
is_typeshed_stub: bool,
allow_unnormalized: bool = False,
in_dynamic_func: bool = False) -> Optional[Type]:
in_dynamic_func: bool = False,
global_scope: bool = True) -> Optional[Type]:
"""Return type if node is valid as a type alias rvalue.

Return None otherwise. 'node' must have been semantically analyzed.
Expand Down Expand Up @@ -115,9 +118,11 @@ def analyze_type_alias(node: Expression,
except TypeTranslationError:
fail_func('Invalid type alias', node)
return None
analyzer = TypeAnalyser(lookup_func, lookup_fqn_func, tvar_scope, fail_func, plugin, options,
is_typeshed_stub, aliasing=True, allow_unnormalized=allow_unnormalized)
analyzer = TypeAnalyser(lookup_func, lookup_fqn_func, tvar_scope, fail_func, note_func,
plugin, options, is_typeshed_stub, aliasing=True,
allow_unnormalized=allow_unnormalized)
analyzer.in_dynamic_func = in_dynamic_func
analyzer.global_scope = global_scope
return type.accept(analyzer)


Expand All @@ -135,14 +140,17 @@ class TypeAnalyser(SyntheticTypeVisitor[Type], AnalyzerPluginInterface):
Converts unbound types into bound types.
"""

# Is this called from an untyped function definition
# Is this called from an untyped function definition?
in_dynamic_func = False # type: bool
# Is this called from global scope?
global_scope = True # type: bool

def __init__(self,
lookup_func: Callable[[str, Context], SymbolTableNode],
lookup_fqn_func: Callable[[str], SymbolTableNode],
tvar_scope: TypeVarScope,
fail_func: Callable[[str, Context], None],
note_func: Callable[[str, Context], None],
plugin: Plugin,
options: Options,
is_typeshed_stub: bool, *,
Expand All @@ -153,6 +161,7 @@ def __init__(self,
self.lookup = lookup_func
self.lookup_fqn_func = lookup_fqn_func
self.fail_func = fail_func
self.note_func = note_func
self.tvar_scope = tvar_scope
self.aliasing = aliasing
self.allow_tuple_literal = allow_tuple_literal
Expand Down Expand Up @@ -280,10 +289,16 @@ def visit_unbound_type(self, t: UnboundType) -> Type:
if not (self.aliasing and sym.kind == TVAR and
self.tvar_scope.get_binding(sym) is None):
if (not self.third_pass and not self.in_dynamic_func and
not (isinstance(sym.node, FuncDef) or
isinstance(sym.node, Var) and sym.node.is_ready)):
not (isinstance(sym.node, (FuncDef, Decorator)) or
isinstance(sym.node, Var) and sym.node.is_ready) and
not (sym.kind == TVAR and tvar_def is None)):
if t.args and not self.global_scope:
self.fail('Unsupported forward reference to "{}"'.format(t.name), t)
return AnyType(TypeOfAny.from_error)
return ForwardRef(t)
self.fail('Invalid type "{}"'.format(name), t)
if self.third_pass and sym.kind == TVAR:
self.note_func("Forward references to type variables are prohibited", t)
return t
info = sym.node # type: TypeInfo
if len(t.args) > 0 and info.fullname() == 'builtins.tuple':
Expand Down Expand Up @@ -319,7 +334,7 @@ def visit_unbound_type(self, t: UnboundType) -> Type:
return instance
else:
if self.third_pass:
self.fail('Invalid type {}'.format(t), t)
self.fail('Invalid type "{}"'.format(t.name), t)
return AnyType(TypeOfAny.from_error)
return AnyType(TypeOfAny.special_form)

Expand Down
35 changes: 35 additions & 0 deletions test-data/unit/check-type-aliases.test
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,41 @@ if isinstance(x, list):
[builtins fixtures/isinstancelist.pyi]
[out]

[case testProhibitedForwardRefToTypeVar]
from typing import TypeVar, List

a: List[T]

T = TypeVar('T')
[builtins fixtures/list.pyi]
[out]
main:3: error: Invalid type "__main__.T"
main:3: note: Forward references to type variables are prohibited

[case testUnsupportedForwardRef]
from typing import List, TypeVar

T = TypeVar('T')

def f(x: T) -> None:
y: A[T] # E: Unsupported forward reference to "A"

A = List[T]
[builtins fixtures/list.pyi]
[out]

[case testUnsupportedForwardRef2]
from typing import List, TypeVar

def f() -> None:
X = List[int]
x: A[X] # E: Unsupported forward reference to "A"

T = TypeVar('T')
A = List[T]
[builtins fixtures/list.pyi]
[out]

[case testNoneAlias]
from typing import Union
void = type(None)
Expand Down
0