-
-
Notifications
You must be signed in to change notification settings - Fork 11.1k
ENH: make typing module available at runtime #16558
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
8e8a8f1
70130f8
4a120f0
c88f5a2
c63f233
347a368
d985e8c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
``numpy.typing`` is accessible at runtime | ||
----------------------------------------- | ||
The types in ``numpy.typing`` can now be imported at runtime. Code | ||
like the following will now work: | ||
|
||
.. code:: python | ||
|
||
from numpy.typing import ArrayLike | ||
x: ArrayLike = [1, 2, 3, 4] |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
.. _typing: | ||
.. automodule:: numpy.typing |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,81 @@ | ||
""" | ||
============================ | ||
Typing (:mod:`numpy.typing`) | ||
============================ | ||
|
||
.. warning:: | ||
|
||
Some of the types in this module rely on features only present in | ||
the standard library in Python 3.8 and greater. If you want to use | ||
these types in earlier versions of Python, you should install the | ||
typing-extensions_ package. | ||
|
||
Large parts of the NumPy API have PEP-484-style type annotations. In | ||
addition, the following type aliases are available for users. | ||
|
||
- ``typing.ArrayLike``: objects that can be converted to arrays | ||
- ``typing.DtypeLike``: objects that can be converted to dtypes | ||
|
||
Roughly speaking, ``typing.ArrayLike`` is "objects that can be used as | ||
inputs to ``np.array``" and ``typing.DtypeLike`` is "objects that can | ||
be used as inputs to ``np.dtype``". | ||
|
||
.. _typing-extensions: https://pypi.org/project/typing-extensions/ | ||
|
||
Differences from the runtime NumPy API | ||
-------------------------------------- | ||
|
||
NumPy is very flexible. Trying to describe the full range of | ||
possibilities statically would result in types that are not very | ||
helpful. For that reason, the typed NumPy API is often stricter than | ||
the runtime NumPy API. This section describes some notable | ||
differences. | ||
|
||
ArrayLike | ||
~~~~~~~~~ | ||
|
||
The ``ArrayLike`` type tries to avoid creating object arrays. For | ||
example, | ||
|
||
.. code-block:: python | ||
|
||
>>> np.array(x**2 for x in range(10)) | ||
array(<generator object <genexpr> at 0x10c004cd0>, dtype=object) | ||
|
||
is valid NumPy code which will create a 0-dimensional object | ||
array. Type checkers will complain about the above example when using | ||
the NumPy types however. If you really intended to do the above, then | ||
you can either use a ``# type: ignore`` comment: | ||
|
||
.. code-block:: python | ||
|
||
>>> np.array(x**2 for x in range(10)) # type: ignore | ||
|
||
or explicitly type the array like object as ``Any``: | ||
|
||
.. code-block:: python | ||
|
||
>>> from typing import Any | ||
>>> array_like: Any = (x**2 for x in range(10)) | ||
>>> np.array(array_like) | ||
array(<generator object <genexpr> at 0x1192741d0>, dtype=object) | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we add a brief note on the suggested work-around? The obvious way would be to add a comment disabling typing:
Are there other recommended options? I think we've also discussed making checks less strict if
I don't know if that works yet. If it does, perhaps we should mention it, too. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The other way we test for: https://github.com/numpy/numpy/blob/master/numpy/tests/typing/pass/array_like.py#L43) is adding an explicit
Seems like @seberg would probably know the answer to that? |
||
ndarray | ||
~~~~~~~ | ||
|
||
It's possible to mutate the dtype of an array at runtime. For example, | ||
the following code is valid: | ||
|
||
.. code-block:: python | ||
|
||
x = np.array([1, 2]) | ||
x.dtype = np.bool_ | ||
|
||
This sort of mutation is not allowed by the types. Users who want to | ||
write statically typed code should insted use the `numpy.ndarray.view` | ||
method to create a view of the array with a different dtype. | ||
|
||
""" | ||
from ._array_like import _SupportsArray, ArrayLike | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There isn't much code in this package, but since typing is so verbose it would be a little painful to keep |
||
from ._shape import _Shape, _ShapeLike | ||
from ._dtype_like import DtypeLike |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
import sys | ||
from typing import Any, overload, Sequence, TYPE_CHECKING, Union | ||
|
||
from numpy import ndarray | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is an example of what I was trying to describe in the PR description where if we do |
||
from ._dtype_like import DtypeLike | ||
|
||
if sys.version_info >= (3, 8): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is the "no hard dependency on There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would it be an idea to issue an There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd worry about that because it's pretty common for projects to run their tests suites with warnings turned into errors, and I could see something like e.g. a SciPy test run that doesn't have mypy installed and errors out because of the warning. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmm, that might be a problem, yes. If not, then it should be mentioned in the documentation. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No logger, when most of the action is happening in C code (potentially with the GIL released) logging doesn't work great. I added a big warning to to the top of the documentation in 4a120f0. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ah, so be it. |
||
from typing import Protocol | ||
HAVE_PROTOCOL = True | ||
else: | ||
try: | ||
from typing_extensions import Protocol | ||
except ImportError: | ||
HAVE_PROTOCOL = False | ||
else: | ||
HAVE_PROTOCOL = True | ||
|
||
if TYPE_CHECKING or HAVE_PROTOCOL: | ||
class _SupportsArray(Protocol): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Note for the future: |
||
@overload | ||
def __array__(self, __dtype: DtypeLike = ...) -> ndarray: ... | ||
@overload | ||
def __array__(self, dtype: DtypeLike = ...) -> ndarray: ... | ||
else: | ||
_SupportsArray = Any | ||
|
||
# TODO: support buffer protocols once | ||
# | ||
# https://bugs.python.org/issue27501 | ||
# | ||
# is resolved. See also the mypy issue: | ||
# | ||
# https://github.com/python/typing/issues/593 | ||
ArrayLike = Union[bool, int, float, complex, _SupportsArray, Sequence] | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It would be nice to support buffer protocols, but Python's typing doesn't support that yet. In the meantime I would suggest adding There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since
passes mypy. I will add the comment though. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should add a test explicitly for memoryviews though; I'll open an issue. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd highly recommend commenting on this B.P.O issue asking for support for a Buffer protocol in typing, as that is the better place than the typing issue: https://bugs.python.org/issue27501 |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
from typing import Sequence, Tuple, Union | ||
|
||
_Shape = Tuple[int, ...] | ||
|
||
# Anything that can be coerced to a shape tuple | ||
_ShapeLike = Union[int, Sequence[int]] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We also execute all the code in
pass
, so we're also testing here that you can really import these things at runtime.