10000 TYP: compatibility with array API standard's typing · Issue #28665 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

TYP: compatibility with array API standard's typing #28665

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Saransh-cpp opened this issue Apr 7, 2025 · 3 comments
Closed

TYP: compatibility with array API standard's typing #28665

Saransh-cpp opened this issue Apr 7, 2025 · 3 comments
Labels
40 - array API standard PRs and issues related to support for the array API standard 41 - Static typing

Comments

@Saransh-cpp
Copy link
Contributor

Describe the issue:

It looks like NumPy does not follow the array API standard's typing, or maybe I am doing something wrong here :(

For instance, the array.__add__ function should have the following type signature -

array.__add__(other: int | float | complex | array, /) → array

by mypy cannot find one in the overloads. See MWE below.

Reproduce the code example:

import numpy as np
import typing

Array = typing.TypeVar("Array", bound="SuperArray")

class SuperArray(typing.Protocol):
    def __add__(self: Array, other: int | float | complex | Array, /) -> Array: ...

def ident(array: SuperArray) -> SuperArray:
    return array

ident(np.eye(3))

Error message:

(.env) saransh@Saranshs-MacBook-Pro glass % python3 -m mypy test_mypy.py --show-error-end
test_mypy.py:12:7:12:15: error: Argument 1 to "ident" has incompatible type "ndarray[tuple[int, ...], dtype[float64]]"; expected "SuperArray"  [arg-type]
test_mypy.py:12:7:12:15: note: Following member(s) of "ndarray[tuple[int, ...], dtype[float64]]" have conflicts:
test_mypy.py:12:7:12:15: note:     Expected:
test_mypy.py:12:7:12:15: note:         def __add__(self, int | float | complex | ndarray[tuple[int, ...], dtype[float64]], /) -> ndarray[tuple[int, ...], dtype[float64]]
test_mypy.py:12:7:12:15: note:     Got:
test_mypy.py:12:7:12:15: note:         @overload
test_mypy.py:12:7:12:15: note:         def __add__(self, int | numpy.bool[builtins.bool], /) -> ndarray[tuple[int, ...], dtype[float64]]
test_mypy.py:12:7:12:15: note:         @overload
test_mypy.py:12:7:12:15: note:         def __add__(self, _SupportsArray[dtype[numpy.bool[builtins.bool]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool]]]] | builtins.bool | _NestedSequence[builtins.bool], /) -> ndarray[tuple[int, ...], dtype[float64]]
test_mypy.py:12:7:12:15: note:         @overload
test_mypy.py:12:7:12:15: note:         def __add__(self, _SupportsArray[dtype[floating[_64Bit] | floating[_32Bit] | floating[_16Bit] | integer[Any] | numpy.bool[builtins.bool]]] | _NestedSequence[_SupportsArray[dtype[floating[_64Bit] | floating[_32Bit] | floating[_16Bit] | integer[Any] | numpy.bool[builtins.bool]]]] | float | int | _NestedSequence[float | int], /) -> ndarray[tuple[int, ...], dtype[float64]]
test_mypy.py:12:7:12:15: note:         @overload
test_mypy.py:12:7:12:15: note:         def __add__(self, _SupportsArray[dtype[floating[_64Bit]]] | _NestedSequence[_SupportsArray[dtype[floating[_64Bit]]]], /) -> ndarray[tuple[int, ...], dtype[float64]]
test_mypy.py:12:7:12:15: note:         @overload
test_mypy.py:12:7:12:15: note:         def __add__(self, _SupportsArray[dtype[complexfloating[_64Bit, _64Bit]]] | _NestedSequence[_SupportsArray[dtype[complexfloating[_64Bit, _64Bit]]]], /) -> ndarray[tuple[int, ...], dtype[complex128]]
test_mypy.py:12:7:12:15: note:         @overload
test_mypy.py:12:7:12:15: note:         def __add__(self, _SupportsArray[dtype[numpy.bool[builtins.bool]] | dtype[integer[Any]] | dtype[floating[Any]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool]] | dtype[integer[Any]] | dtype[floating[Any]]]] | builtins.bool | int | float | _NestedSequence[builtins.bool | int | float], /) -> ndarray[tuple[int, ...], dtype[floating[Any]]]
test_mypy.py:12:7:12:15: note:         @overload
test_mypy.py:12:7:12:15: note:         def __add__(self, _SupportsArray[dtype[numpy.bool[builtins.bool]] | dtype[integer[Any]] | dtype[floating[Any]] | dtype[complexfloating[Any, Any]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool]] | dtype[integer[Any]] | dtype[floating[Any]] | dtype[complexfloating[Any, Any]]]] | builtins.bool | int | float | complex | _NestedSequence[builtins.bool | int | float | complex], /) -> ndarray[tuple[int, ...], dtype[complexfloating[Any, Any]]]
test_mypy.py:12:7:12:15: note:         @overload
test_mypy.py:12:7:12:15: note:         def __add__(self, _SupportsArray[dtype[numpy.bool[builtins.bool]] | dtype[number[Any, int | float | complex]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool]] | dtype[number[Any, int | float | complex]]]] | builtins.bool | int | float | complex | _NestedSequence[builtins.bool | int | float | complex], /) -> ndarray[tuple[int, ...], dtype[number[Any, int | float | complex]]]
test_mypy.py:12:7:12:15: note:         @overload
test_mypy.py:12:7:12:15: note:         def __add__(self, _SupportsArray[dtype[object_]] | _NestedSequence[_SupportsArray[dtype[object_]]], /) -> Any
Found 1 error in 1 file (checked 1 source file)

Python and NumPy Versions:

2.2.4
3.13.2 (main, Feb 4 2025, 14:51:09) [Clang 16.0.0 (clang-1600.0.26.6)]

Type-checker version and settings:

mypy 1.15.0 (compiled: yes)

Additional typing packages.

No response
@jorenham
Copy link
Member
jorenham commented Apr 7, 2025 •
edited
Loading

For instance, the array.__add__ function should have the following type signature -

array.__add__(other: int | float | complex | array, /) → array

Let's say that ndarray would indeed have this as an overload, and consider this example:

np.array([True]) + 1j

Type-checkers infer np.array([True]) as npt.NDArray[np.bool] (where npt = numpy.typing). In this overload, self: Array causes the Array type-argument to resolve to npt.NDArray[np.bool]. The return type will therefore be inferred as npt.NDArray[np.bool].

But at runtime, this would result in a ndarray instance with a dtype of complex128. But npt.NDArray[np.complex128] is not compatible with npt.NDArray[np.bool].

This shows that these array-api annotations fundamentally forbid array types from including important static typing information such as its dtype and the shape-type.

There has been previous discussion on this, and the other (many) problems that the current array-api "stubs" have. See e.g. data-apis/array-api#863 and data-apis/array-api#857 (comment). That is why we have decided to build a separate typing package for the array-api: https://github.com/data-apis/array-api-typing/. So far, I haven't had the time to work on this. But I expect that I'll be able to work on in in a couple of months.

@jorenham jorenham closed this as not planned Won't fix, can't repro, duplicate, stale Apr 7, 2025
@jorenham
Copy link
Member
jorenham commented Apr 7, 2025 •
edited
Loading

Additionally, using a Protocol like this to match on overloaded method will only work if you're matching on the first overload; even if the overloads are completely independent. I hate that this is the case, but it's the unfortunate truth. Pyright recently introduced some support for it (see microsoft/pyright#9835), but if your overloads are overlapping (in a valid way), then you'll get an unexpected return type. Unfortunately the pyright isn't willing to address this bug: microsoft/pyright#9896

This is also the reason why most of the operator stdlib cannot be annotated: python/typeshed#13800

@jorenham jorenham added the 40 - array API standard PRs and issues related to support for the array API standard label Apr 7, 2025
@Saransh-cpp
Copy link
Contributor Author

Hi @jorenham, thanks for the details! 😄

I'm aware of https://github.com/data-apis/array-api-typing/, and one of the reasons I started experimenting with custom Protocols was because there has been no recent progress on array-api-typing (maintainers not having the availability is very understandable, not complaining at all). I got JAX working with the custom Protocol`s, but NumPy constantly errored out, which prompted me to open this issue.

I understand some of the internals now and that it would not be possible to get hacks to work before array-api-typing is up. Thank you!

@Saransh-cpp Saransh-cpp mentioned this issue Apr 25, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
40 - array API standard PRs and issues related to support for the array API standard 41 - Static typing
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jorenham @Saransh-cpp
0