8000 SEGV from endless loop in check_call in case of isinstance(callee, Instance) · Issue #7370 · python/mypy · GitHub
[go: up one dir, main page]
Skip to content
SEGV from endless loop in check_call in case of isinstance(callee, Instance) #7370
New issue
New issue
Closed
Closed
SEGV from endless loop in check_call in case of isinstance(callee, Instance)#7370
@dckc

Description

@dckc
dckc
opened on Aug 19, 2019

mypy dumped core while checking some code using spark.sql.Column.
The situation is similar to #6730, but using using PYTHONFAULTHANDLER=1 I got a different stack trace:

Current thread 0x00007fa5cceff740 (most recent call first):
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/types.py", line 113 in __init__
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/types.py", line 751 in __init__
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/types.py", line 825 in __init__
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/types.py", line 893 in copy_modified
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkmember.py", line 958 in bind_self
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkmember.py", line 344 in analyze_member_var_access
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkmember.py", line 193 in analyze_instance_member_access
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkmember.py", line 117 in _analyze_member_access
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkmember.py", line 103 in analyze_member_access
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 710 in check_call
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717 in check_call
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717 in check_call
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717 in check_call
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717 in check_call
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717 in check_call
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717 in check_call
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717 in check_call
  ...

I added assert call_function is not callee before the recursive call

mypy/mypy/checkexpr.py

Line 715 in 7aaa435

return self.check_call(call_function, args, arg_kinds, context, arg_names,

and now I get more useful diagnostics, at least:

tumor_reg_data.py:245: error: "object" has no attribute "toPandas"
...
tumor_reg_data.py:693: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues version: 0.701
tumor_reg_data.py:693: : note: please use --show-traceback to print a traceback when reporting a bug

Line 693 is the items = ... assignment:

def itemNumOfPath(bc_var: DataFrame,
                  item: str = 'item'):
    digits = func.regexp_extract('concept_path',
                                 r'\\i2b2\\naaccr\\S:[^\\]+\\(\d+)', 1)
    items = bc_var.select(digits.cast('int').alias(item)).dropna().distinct()
    return items.sort(item)

IOU the rest of the code, but it's not straightforward to share just now.

$ mypy --version
mypy 0.701

using mypy --show-traceback I get:

export PATH=$HOME/opt/miniconda3/envs/py2tr/bin:$PATH && python -m doctest tumor_reg_data.py && mypy --strict --show-traceback tumor_reg_data.py
tumor_reg_data.py:249: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues version: 0.701
Traceback (most recent call last):
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/bin/mypy", line 10, in <module>
    sys.exit(console_entry())
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/__main__.py", line 7, in console_entry
    main(None)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/main.py", line 84, in main
    res = build.build(sources, options, None, flush_errors, fscache)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/build.py", line 163, in build
    result = _build(sources, options, alt_lib_path, flush_errors, fscache)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/build.py", line 218, in _build
    graph = dispatch(sources, manager)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/build.py", line 2461, in dispatch
    process_graph(graph, manager)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/build.py", line 2761, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/build.py", line 2868, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/build.py", line 2020, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checker.py", line 284, in check_first_pass
    self.accept(d)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checker.py", line 395, in accept
    stmt.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 976, in accept
    return visitor.visit_expression_stmt(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checker.py", line 2650, in visit_expression_stmt
    self.expr_checker.accept(s.expr, allow_none_return=True, always_allow_any=True)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1679, in accept
    return visitor.visit_op_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1870, in visit_op_expr
    return self.check_boolean_op(e, e)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 2385, in check_boolean_op
    right_type = self.analyze_cond_branch(right_map, e.right, left_type)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3223, in analyze_cond_branch
    return self.accept(node, type_context=context)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1450, in accept
    return visitor.visit_member_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1766, in visit_member_expr
    result = self.analyze_ordinary_member_access(e, is_lvalue)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1777, in analyze_ordinary_member_access
    original_type = self.accept(e.expr)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1450, in accept
    return visitor.visit_member_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1766, in visit_member_expr
    result = self.analyze_ordinary_member_access(e, is_lvalue)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1777, in analyze_ordinary_member_access
    original_type = self.accept(e.expr)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1498, in accept
    return visitor.visit_call_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 247, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 292, in visit_call_expr_inner
    callee_type = self.accept(e.callee, type_context, always_allow_any=True)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1450, in accept
    return visitor.visit_member_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1766, in visit_member_expr
    result = self.analyze_ordinary_member_access(e, is_lvalue)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1777, in analyze_ordinary_member_access
    original_type = self.accept(e.expr)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1498, in accept
    return visitor.visit_call_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 247, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 292, in visit_call_expr_inner
    callee_type = self.accept(e.callee, type_context, always_allow_any=True)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1450, in accept
    return visitor.visit_member_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1766, in visit_member_expr
    result = self.analyze_ordinary_member_access(e, is_lvalue)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1777, in analyze_ordinary_member_access
    original_type = self.accept(e.expr)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1498, in accept
    return visitor.visit_call_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 247, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 317, in visit_call_expr_inner
    ret_type = self.check_call_expr_with_callee_type(callee_type, e, fullname, object_type)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 661, in check_call_expr_with_callee_type
    object_type=object_type)[0]
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717, in check_call
    callable_node, arg_messages, callable_name, callee)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 715, in check_call
    assert call_function is not callee, callee
AssertionError: pyspark.sql.column.Column
tumor_reg_data.py:249: : note: use --pdb to drop into pdb

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions
      0