8000 Reorder sections in running_mypy by hauntsaninja · Pull Request #13738 · python/mypy · GitHub
[go: up one dir, main page]

Skip to content

Reorder sections in running_mypy #13738

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 27, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
286 changes: 136 additions & 150 deletions docs/source/running_mypy.rst
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,82 @@ flags, the recommended approach is to use a
:ref:`configuration file <config-file>` instead.


.. _mapping-paths-to-modules:

Mapping file paths to modules
*****************************

One of the main ways you can tell mypy what to type check
is by providing mypy a list of paths. For example::

$ mypy file_1.py foo/file_2.py file_3.pyi some/directory

This section describes how exactly mypy maps the provided paths
to modules to type check.

- Mypy will check all paths provided that correspond to files.

- Mypy will recursively discover and check all files ending in ``.py`` or
``.pyi`` in directory paths provided, after accounting for
:option:`--exclude <mypy --exclude>`.

- For each file to be checked, mypy will attempt to associate the file (e.g.
``project/foo/bar/baz.py``) with a fully qualified module name (e.g.
``foo.bar.baz``). The directory the package is in (``project``) is then
added to mypy's module search paths.

How mypy determines fully qualified module names depends on if the options
:option:`--no-namespace-packages <mypy --no-namespace-packages>` and
:option:`--explicit-package-bases <mypy --explicit-package-bases>` are set.

1. If :option:`--no-namespace-packages <mypy --no-namespace-packages>` is set,
mypy will rely solely upon the presence of ``__init__.py[i]`` files to
determine the fully qualified module name. That is, mypy will crawl up the
directory tree for as long as it continues to find ``__init__.py`` (or
``__init__.pyi``) files.

For example, if your directory tree consists of ``pkg/subpkg/mod.py``, mypy
would require ``pkg/__init__.py`` and ``pkg/subpkg/__init__.py`` to exist in
order correctly associate ``mod.py`` with ``pkg.subpkg.mod``

2. The default case. If :option:`--namespace-packages <mypy
--no-namespace-packages>` is on, but :option:`--explicit-package-bases <mypy
--explicit-package-bases>` is off, mypy will allow for the possibility that
directories without ``__init__.py[i]`` are packages. Specifically, mypy will
look at all parent directories of the file and use the location of the
highest ``__init__.py[i]`` in the directory tree to determine the top-level
package.

For example, say your directory tree consists solely of ``pkg/__init__.py``
and ``pkg/a/b/c/d/mod.py``. When determining ``mod.py``'s fully qualified
module name, mypy will look at ``pkg/__init__.py`` and conclude that the
associated module name is ``pkg.a.b.c.d.mod``.

3. You'll notice that the above case still relies on ``__init__.py``. If
you can't put an ``__init__.py`` in your top-level package, but still wish to
pass paths (as opposed to packages or modules using the ``-p`` or ``-m``
flags), :option:`--explicit-package-bases <mypy --explicit-package-bases>`
provides a solution.

With :option:`--explicit-package-bases <mypy --explicit-package-bases>`, mypy
will locate the nearest parent directory that is a member of the ``MYPYPATH``
environment variable, the :confval:`mypy_path` config or is the current
working directory. Mypy will then use the relative path to determine the
fully qualified module name.

For example, say your directory tree consists solely of
``src/namespace_pkg/mod.py``. If you run the following command, mypy
will correctly associate ``mod.py`` with ``namespace_pkg.mod``::

$ MYPYPATH=src mypy --namespace-packages --explicit-package-bases .

If you pass a file not ending in ``.py[i]``, the module name assumed is
``__main__`` (matching the behavior of the Python interpreter), unless
:option:`--scripts-are-modules <mypy --scripts-are-modules>` is passed.

Passing :option:`-v <mypy -v>` will show you the files and associated module
names that mypy will check.


How mypy handles imports
************************
Expand Down Expand Up @@ -326,6 +402,66 @@ on your system in an unconventional way.
In this case, follow the steps above on how to handle
:ref:`missing type hints in third party libraries <missing-type-hints-for-third-party-library>`.


.. _finding-imports:

How imports are found
*********************

When mypy encounters an ``import`` statement or receives module
names from the command line via the :option:`--module <mypy --module>` or :option:`--package <mypy --package>`
flags, mypy tries to find the module on the file system similar
to the way Python finds it. However, there are some differences.

First, mypy has its own search path.
This is computed from the following items:

- The ``MYPYPATH`` environment variable
(a list of directories, colon-separated on UNIX systems, semicolon-separated on Windows).
- The :confval:`mypy_path` config file option.
- The directories containing the sources given on the command line
(see :ref:`Mapping file paths to modules <mapping-paths-to-modules>`).
- The installed packages marked as safe for type checking (see
:ref:`PEP 561 support <installed-packages>`)
- The relevant directories of the
`typeshed <https://github.com/python/typeshed>`_ repo.

.. note::

You cannot point to a stub-only package (:pep:`561`) via the ``MYPYPATH``, it must be
installed (see :ref:`PEP 561 support <installed-packages>`)

Second, mypy searches for stub files in addition to regular Python files
and packages.
The rules for searching for a module ``foo`` are as follows:

- The search looks in each of the directories in the search path
(see above) until a match is found.
- If a package named ``foo`` is found (i.e. a directory
``foo`` containing an ``__init__.py`` or ``__init__.pyi`` file)
that's a match.
- If a stub file named ``foo.pyi`` is found, that's a match.
- If a Python module named ``foo.py`` is found, that's a match.

These matches are tried in order, so that if multiple matches are found
in the same directory on the search path
(e.g. a package and a Python file, or a stub file and a Python file)
the first one in the above list wins.

In particular, if a Python file and a stub file are both present in the
same directory on the search path, only the stub file is used.
(However, if the files are in different directories, the one found
in the earlier directory is used.)

Setting :confval:`mypy_path`/``MYPYPATH`` is mostly useful in the case
where you want to try running mypy against multiple distinct
sets of files that happen to share some common dependencies.

For example, if you have multiple projects that happen to be
using the same set of work-in-progress stubs, it could be
convenient to just have your ``MYPYPATH`` point to a single
directory containing the stubs.

.. _follow-imports:

Following imports
Expand Down Expand Up @@ -387,153 +523,3 @@ hard-to-debug errors.
Adjusting import following behaviour is often most useful when restricted to
specific modules. This can be accomplished by setting a per-module
:confval:`follow_imports` config option.


.. _mapping-paths-to-modules:

Mapping file paths to modules
*****************************

One of the main ways you can tell mypy what to type check
is by providing mypy a list of paths. For example::

$ mypy file_1.py foo/file_2.py file_3.pyi some/directory

This section describes how exactly mypy maps the provided paths
to modules to type check.

- Mypy will check all paths provided that correspond to files.

- Mypy will recursively discover and check all files ending in ``.py`` or
``.pyi`` in directory paths provided, after accounting for
:option:`--exclude <mypy --exclude>`.

- For each file to be checked, mypy will attempt to associate the file (e.g.
``project/foo/bar/baz.py``) with a fully qualified module name (e.g.
``foo.bar.baz``). The directory the package is in (``project``) is then
added to mypy's module search paths.

How mypy determines fully qualified module names depends on if the options
:option:`--no-namespace-packages <mypy --no-namespace-packages>` and
:option:`--explicit-package-bases <mypy --explicit-package-bases>` are set.

1. If :option:`--no-namespace-packages <mypy --no-namespace-packages>` is set,
mypy will rely solely upon the presence of ``__init__.py[i]`` files to
determine the fully qualified module name. That is, mypy will crawl up the
directory tree for as long as it continues to find ``__init__.py`` (or
``__init__.pyi``) files.

For example, if your directory tree consists of ``pkg/subpkg/mod.py``, mypy
would require ``pkg/__init__.py`` and ``pkg/subpkg/__init__.py`` to exist in
order correctly associate ``mod.py`` with ``pkg.subpkg.mod``

2. The default case. If :option:`--namespace-packages <mypy
--no-namespace-packages>` is on, but :option:`--explicit-package-bases <mypy
--explicit-package-bases>` is off, mypy will allow for the possibility that
directories without ``__init__.py[i]`` are packages. Specifically, mypy will
look at all parent directories of the file and use the location of the
highest ``__init__.py[i]`` in the directory tree to determine the top-level
package.

For example, say your directory tree consists solely of ``pkg/__init__.py``
and ``pkg/a/b/c/d/mod.py``. When determining ``mod.py``'s fully qualified
module name, mypy will look at ``pkg/__init__.py`` and conclude that the
associated module name is ``pkg.a.b.c.d.mod``.

3. You'll notice that the above case still relies on ``__init__.py``. If
you can't put an ``__init__.py`` in your top-level package, but still wish to
pass paths (as opposed to packages or modules using the ``-p`` or ``-m``
flags), :option:`--explicit-package-bases <mypy --explicit-package-bases>`
provides a solution.

With :option:`--explicit-package-bases <mypy --explicit-package-bases>`, mypy
will locate the nearest parent directory that is a member of the ``MYPYPATH``
environment variable, the :confval:`mypy_path` config or is the current
working directory. Mypy will then use the relative path to determine the
fully qualified module name.

8000
For example, say your directory tree consists solely of
``src/namespace_pkg/mod.py``. If you run the following command, mypy
will correctly associate ``mod.py`` with ``namespace_pkg.mod``::

$ MYPYPATH=src mypy --namespace-packages --explicit-package-bases .

If you pass a file not ending in ``.py[i]``, the module name assumed is
``__main__`` (matching the behavior of the Python interpreter), unless
:option:`--scripts-are-modules <mypy --scripts-are-modules>` is passed.

Passing :option:`-v <mypy -v>` will show you the files and associated module
names that mypy will check.


.. _finding-imports:

How imports are found
*********************

When mypy encounters an ``import`` statement or receives module
names from the command line via the :option:`--module <mypy --module>` or :option:`--package <mypy --package>`
flags, mypy tries to find the module on the file system similar
to the way Python finds it. However, there are some differences.

First, mypy has its own search path.
This is computed from the following items:

- The ``MYPYPATH`` environment variable
(a list of directories, colon-separated on UNIX systems, semicolon-separated on Windows).
- The :confval:`mypy_path` config file option.
- The directories containing the sources given on the command line
(see :ref:`Mapping file paths to modules <mapping-paths-to-modules>`).
- The installed packages marked as safe for type checking (see
:ref:`PEP 561 support <installed-packages>`)
- The relevant directories of the
`typeshed <https://github.com/python/typeshed>`_ repo.

.. note::

You cannot point to a stub-only package (:pep:`561`) via the ``MYPYPATH``, it must be
installed (see :ref:`PEP 561 support <installed-packages>`)

Second, mypy searches for stub files in addition to regular Python files
and packages.
The rules for searching for a module ``foo`` are as follows:

- The search looks in each of the directories in the search path
(see above) until a match is found.
- If a package named ``foo`` is found (i.e. a directory
``foo`` containing an ``__init__.py`` or ``__init__.pyi`` file)
that's a match.
- If a stub file named ``foo.pyi`` is found, that's a match.
- If a Python module named ``foo.py`` is found, that's a match.

These matches are tried in order, so that if multiple matches are found
in the same directory on the search path
(e.g. a package and a Python file, or a stub file and a Python file)
the first one in the above list wins.

In particular, if a Python file and a stub file are both present in the
same directory on the search path, only the stub file is used.
(However, if the files are in different directories, the one found
in the earlier directory is used.)


Other advice and best practices
*******************************

There are multiple ways of telling mypy what files to type check, ranging
from passing in command line arguments to using the :confval:`files` or :confval:`mypy_path`
config file options to setting the
``MYPYPATH`` environment variable.

However, in practice, it is usually sufficient to just use either
command line arguments or the :confval:`files` config file option (the two
are largely interchangeable).

Setting :confval:`mypy_path`/``MYPYPATH`` is mostly useful in the case
where you want to try running mypy against multiple distinct
sets of files that happen to share some common dependencies.

For example, if you have multiple projects that happen to be
using the same set of work-in-progress stubs, it could be
convenient to just have your ``MYPYPATH`` point to a single
directory containing the stubs.
0