8000 Deprecating in-place operations where the out-of-place equivalent would upcast? · Issue #25621 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

Deprecating in-place operations where the out-of-place equivalent would upcast? #25621

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
rgommers opened this issue Jan 18, 2024 · 1 comment
Open

Deprecating in-place operations where the out-of-place equivalent would upcast? #25621

rgommers opened this issue Jan 18, 2024 · 1 comment
Labels
07 - Deprecation component: numpy._core

Comments

@rgommers
Copy link
Member

In-place operators typically behave the same as their out-of-place equivalents (excluding what happens with views). For example, if x and y are float64 arrays, then x += y and x = x + y are equivalent statements. However, something unexpected may happen when the right-hand operand has a dtype that would normally cause upcasting when combined with the left-hand operand according to NumPy's casting rules. In such cases, the in-place version doesn't upcast the left-hand operand, but downcasts the right-hand operand. To illustrate the difference:

>>> # The in-place version:
>>> a = np.array(1, dtype=np.int8)
>>> a += np.array(2**12, dtype=np.int16)
>>> a
array(1, dtype=int8)

>>> # The out-of-place version:
>>> a + np.array(2**12, dtype=np.int16)
4097
>>> (a + np.array(2**12, dtype=np.int16)).dtype
dtype('int16')

This yields silently incorrect results, and seems undesirable.

It is already forbidden to do this with dtypes that aren't the same kind:

>>> a += np.array(2**12, dtype=np.float16)
...
UFuncTypeError: Cannot cast ufunc 'add' output from dtype('float16') to dtype('int8') with casting rule 'same_kind'

It seems like it would be safer if the behavior would only allow 'safe' rather than 'same_kind' casting. Making that change would avoid the silently incorrect results, and recover the property that in-place and out-of-place statements act the same.

This came up in the review of NEP 56 (gh-25542). We decided to punt on it there, since it's a little unclear how impactful deprecating unsafe casts are, and there is no hurry in making this change. It does seem desirable though.

The right course of action here is probably to implement this change in a branch, and then seeing what fails in test suites of downstream projects to assess the impact.
@rgommers rgommers mentioned this issue Jan 18, 2024
Merged
@seberg seberg mentioned this issue Jan 18, 2024
Closed
@rgommers rgommers mentioned this issue Apr 1, 2024
Closed
@mhvk
Copy link
Contributor
mhvk commented Apr 1, 2024

Note the related report about the performance impact of no longer automatically casting scalars in #26183. While that is not going to change, perhaps worth mentioning here was the suggestion for in-place operations to use the dtype of the in-place array by default (instead of erroring with casting='safe'), i.e., assume that for an in-place operation one effectively has the equivalent of np.ufunc(in, other, out=in, dtype=in.dtype) - this could be just the implementation for the __i*__ operators, not a default that happens automatically for doing np.ufunc(in, other, out=in).

@ngoldbaum ngoldbaum mentioned this issue May 24, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
07 - Deprecation component: numpy._core
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rgommers @mhvk
0