Inject a test that checks the mypy exit status #79

dmtucker · 2020-03-01T18:25:25Z

Mypy is designed to doggedly follow all imports, even if the imported module is not a file you explicitly wanted mypy to check.

This is problematic because, if mypy reports errors in a file pytest didn't collect, pytest-mypy has no MypyItem (now MypyFileItem) to match the errors to, and, if there are no other errors to report, the plugin may not force the correct pytest exit status!

To remedy that, this injects a new MypyStatusItem which just checks the mypy exit status and fails if it's nonzero. This means that an extra test will run for every --mypy invocation. Also, both MypyStatusItem and the new MypyFileItem inherit from MypyItem, so some existing checks that currently check for MypyItems may need to be updated to check for MypyFileItems.

dmtucker · 2020-03-01T20:09:18Z

There are a couple other nuances to this that I'll highlight:

--mypy now affects collection only.
- CLI options are not checked in pytest_collection_modifyitems, which means that mypy will still run on MypyFileItems injected in any pytest_colleciton_modifyitems implementations.
  - To avoid this, check the CLI options manually before injecting, or invoke pytest_collect_file directly:
```
# conftest.py
import py

def pytest_collection_modifyitems(session, config, items):
    plugin = config.pluginmanager.getplugin('mypy')
    item = plugin.pytest_collect_file(py.path.local('foo.py'), session)
    if item:
        items.append(item)
```
    Remember, though, that pytest_collect_file will only collect .py.
That ☝️ enables a recipe for using the Mypy files directive (related: Running mypy only on certain files #67, cc: @brianmaissy):
```
[mypy]
files = foo.py  # not in tests/
```
```
# conftest.py
def pytest_collection_modifyitems(session, config, items):
    plugin = config.pluginmanager.getplugin('mypy')
    items.append(
        plugin.MypyStatusItem(plugin.nodeid_name, session, config, session),
    )
```
```
pytest tests/  # no --mypy
```
The conftest.py causes mypy to be run on all MypyFileItems, but, since --mypy wasn't passed, there are no MypyFileItems! That means mypy is called without any arguments (by the injected MypyStatusItem) which activates the files directive.

dmtucker added 7 commits February 29, 2020 19:25

Adjust the default tox invocation

bd09e7a

Never import pytest_mypy

040975d

Use pytest-style names in the tests

32a27f6

Add a missing test docstring

0791070

Split MypyFileItem out of MypyItem

2120ff1

Add _mypy_results

6cd5304

Create MypyStatusItem

0b66e30

dmtucker added this to the 0.6.0 milestone Mar 1, 2020

dmtucker merged commit 26c4cbf into realpython:master Mar 8, 2020

dmtucker deleted the status branch March 8, 2020 23:03

dmtucker mentioned this pull request Apr 12, 2020

False positives using when using pytest-xdist -f (--looponfail) #85

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Inject a test that checks the mypy exit status #79

Inject a test that checks the mypy exit status #79

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Inject a test that checks the mypy exit status #79

Inject a test that checks the mypy exit status #79

Uh oh!

Conversation

Uh oh!

Uh oh!

Uh oh!