8000 Document stubtest (#11636) · DeepSourceCorp/mypy@ec82adb · GitHub
[go: up one dir, main page]
8000
Skip to content

Commit ec82adb

Browse files
hauntsaninjatushar-deepsource
hauntsaninja
authored and
tushar-deepsource
committed
Document stubtest (python#11636)
Co-authored-by: hauntsaninja <>
1 parent 05b2065 commit ec82adb

File tree

5 files changed

+162
-20
lines changed

5 files changed

+162
-20
lines changed

‎docs/source/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,7 @@ Mypy is a static type checker for Python 3 and Python 2.7.
6060
installed_packages
6161
extending_mypy
6262
stubgen
63+
stubtest
6364

6465
.. toctree::
6566
:maxdepth: 2

‎docs/source/stubgen.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.. _stugen:
1+
.. _stubgen:
22

33
.. program:: stubgen
44

‎docs/source/stubs.rst

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,11 +36,16 @@ the source code. This can be useful, for example, if you use 3rd party
3636
open source libraries in your program (and there are no stubs in
3737
typeshed yet).
3838

39-
That's it! Now you can access the module in mypy programs and type check
39+
That's it!
40+
41+
Now you can access the module in mypy programs and type check
4042
code that uses the library. If you write a stub for a library module,
4143
consider making it available for other programmers that use mypy
4244
by contributing it back to the typeshed repo.
4345

46+
Mypy also ships with two tools for making it easier to create and maintain
47+
stubs: :ref:`stubgen` and :ref:`stubtest`.
48+
4449
The following sections explain the kinds of type annotations you can use
4550
in your programs and stub files.
4651

‎docs/source/stubtest.rst

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
.. _stubtest:
2+
3+
.. program:: stubtest
4+
5+
Automatic stub testing (stubtest)
6+
=================================
7+
8+
Stub files are files containing type annotations. See
9+
`PEP 484 <https://www.python.org/dev/peps/pep-0484/#stub-files>`_
10+
for more motivation and details.
11+
12+
A common problem with stub files is that they tend to diverge from the
13+
actual implementation. Mypy includes the ``stubtest`` tool that can
14+
automatically check for discrepancies between the stubs and the
15+
implementation at runtime.
16+
17+
What stubtest does and does not do
18+
**********************************
19+
20+
stubtest will import your code and introspect your code objects at runtime, for
21+
example, by using the capabilities of the :py:mod:`inspect` module. stubtest
22+
will then analyse the stub files, and compare the two, pointing out things that
23+
differ between stubs and the implementation at runtime.
24+
25+
It's important to be aware of the limitations of this comparison. stubtest will
26+
not make any attempt to statically analyse your actual code and relies only on
27+
dynamic runtime introspection (in particular, this approach means stubtest works
28+
well with extension modules). However, this means that stubtest has limited
29+
visibility; for instance, it cannot tell if a return type of a function is
30+
accurately typed in the stubs.
31+
32+
For clarity, here are some more things stubtest does not do:
33+
34+
* Type check your code, use ``mypy``
35+
* Generate stubs, use ``stubgen`` or ``pyright --createstub``
36+
* Generate stubs based on running your application or test suite, use ``monkeytype``
37+
* Apply stubs to code to produce inline types, use ``retype`` or ``libcst``
38+
39+
In summary, stubtest works very well for ensuring basic consistency between
40+
stubs and implementation or to check for stub completeness. It's used to
41+
test Python's official collection of library stubs,
42+
`typeshed <https://github.com/python/typeshed>`_.
43+
44+
Example
45+
*******
46+
47+
Here's a quick example of what stubtest can do:
48+
49+
.. code-block:: shell
50+
51+
$ python3 -m pip install mypy
52+
53+
$ cat library.py
54+
x = "hello, stubtest"
55+
56+
def foo(x=None):
57+
print(x)
58+
59+
$ cat library.pyi
60+
x: int
61+
62+
def foo(x: int) -> None: ...
63+
64+
$ python3 -m mypy.stubtest library
65+
error: library.foo is inconsistent, runtime argument "x" has a default value but stub argument does not
66+
Stub: at line 3
67+
def (x: builtins.int)
68+
Runtime: at line 3 in file ~/library.py
69+
def (x=None)
70+
71+
error: library.x variable differs from runtime type Literal['hello, stubtest']
72+
Stub: at line 1
73+
builtins.int
74+
Runtime:
75+
hello, stubtest
76+
77+
78+
Usage
79+
*****
80+
81+
Running stubtest can be as simple as ``stubtest module_to_check``.
82+
Run :option:`stubtest --help` for a quick summary of options.
83+
84+
stubtest must be able to import the code to be checked, so make sure that mypy
85+
is installed in the same environment as the library to be tested. In some
86+
cases, setting ``PYTHONPATH`` can help stubtest find the code to import.
87+
88+
Similarly, stubtest must be able to find the stubs to be checked. stubtest
89+
respects the ``MYPYPATH`` environment variable.
90+
91+
If you wish to ignore some of stubtest's complaints, stubtest supports a
92+
pretty handy allowlist system.
93+
94+
The rest of this section documents the command line interface of stubtest.
95+
96+
.. option:: --concise
97+
98+
Makes stubtest's output more concise, one line per error
99+
100+
.. option:: --ignore-missing-stub
101+
102+
Ignore errors for stub missing things that are present at runtime
103+
104+
.. option:: --ignore-positional-only
105+
106+
Ignore errors for whether an argument should or shouldn't be positional-only
107+
108+
.. option:: --allowlist FILE
109+
110+
Use file as an allowlist. Can be passed multiple times to combine multiple
111+
allowlists. Allowlists can be created with --generate-allowlist. Allowlists
112+
support regular expressions.
113+
114+
.. option:: --generate-allowlist
115+
116+
Print an allowlist (to stdout) to be used with --allowlist
117+
118+
.. option:: --ignore-unused-allowlist
119+
120+
Ignore unused allowlist entries
121+
122+
.. option:: --mypy-config-file FILE
123+
124+
Use specified mypy config file to determine mypy plugins and mypy path
125+
126+
.. option:: --custom-typeshed-dir DIR
127+
128+
Use the custom typeshed in DIR
129+
130+
.. option:: --check-typeshed
131+
132+
Check all stdlib modules in typeshed
133+
134+
.. option:: --help
135+
136+
Show a help message :-)

‎mypy/stubtest.py

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1229,7 +1229,11 @@ def parse_options(args: List[str]) -> argparse.Namespace:
12291229
description="Compares stubs to objects introspected from the runtime."
12301230
)
12311231
parser.add_argument("modules", nargs="*", help="Modules to test")
1232-
parser.add_argument("--concise", action="store_true", help="Make output concise")
1232+
parser.add_argument(
1233+
"--concise",
1234+
action="store_true",
1235+
help="Makes stubtest's output more concise, one line per error",
1236+
)
12331237
parser.add_argument(
12341238
"--ignore-missing-stub",
12351239
action="store_true",
@@ -1240,12 +1244,6 @@ def parse_options(args: List[str]) -> argparse.Namespace:
12401244
action="store_true",
12411245
help="Ignore errors for whether an argument should or shouldn't be positional-only",
12421246
)
1243-
parser.add_argument(
1244-
"--custom-typeshed-dir", metavar="DIR", help="Use the custom typeshed in DIR"
1245-
)
1246-
parser.add_argument(
1247-
"--check-typeshed", action="store_true", help="Check all stdlib modules in typeshed"
1248-
)
12491247
parser.add_argument(
12501248
"--allowlist",
12511249
"--whitelist",
@@ -1254,7 +1252,8 @@ def parse_options(args: List[str]) -> argparse.Namespace:
12541252
default=[],
12551253
help=(
12561254
"Use file as an allowlist. Can be passed multiple times to combine multiple "
1257-
"allowlists. Allowlists can be created with --generate-allowlist"
1255+
"allowlists. Allowlists can be created with --generate-allowlist. Allowlists "
1256+
"support regular expressions."
12581257
),
12591258
)
12601259
parser.add_argument(
@@ -1269,19 +1268,20 @@ def parse_options(args: List[str]) -> argparse.Namespace:
12691268
action="store_true",
12701269
help="Ignore unused allowlist entries",
12711270
)
1272-
config_group = parser.add_argument_group(
1273-
title='mypy config file',
1274-
description="Use a config file instead of command line arguments. "
1275-
"Plugins and mypy path are the only supported "
1276-
"configurations.",
1277-
)
1278-
config_group.add_argument(
1279-
'--mypy-config-file',
1271+
parser.add_argument(
1272+
"--mypy-config-file",
1273+
metavar="FILE",
12801274
help=(
1281-
"An existing mypy configuration file, currently used by stubtest to help "
1282-
"determine mypy path and plugins"
1275+
"Use specified mypy config file to determine mypy plugins "
1276+
"and mypy path"
12831277
),
12841278
)
1279+
parser.add_argument(
1280+
"--custom-typeshed-dir", metavar="DIR", help="Use the custom typeshed in DIR"
1281+
)
1282+
parser.add_argument(
1283+
"--check-typeshed", action="store_true", help="Check all stdlib modules in typeshed"
1284+
)
12851285

12861286
return parser.parse_args(args)
12871287

0 commit comments

Comments
 (0)
0