8000 ENH: Add casting option to `astype` that would raise in case of overflow · Issue #25396 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content
8000

ENH: Add casting option to astype that would raise in case of overflow #25396

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
lysnikolaou opened this issue Dec 14, 2023 · 1 comment
Open

ENH: Add casting option to astype that would raise in case of overflow #25396

lysnikolaou opened this issue Dec 14, 2023 · 1 comment

Comments

@lysnikolaou
Copy link
Member

Proposed new feature or change:

Calling astype on an ndarray with values that will overflow under the new dtype currently succeeds and the values do overflow.

In [1]: max_int32 = np.iinfo(np.int32).max

In [2]: np.array([max_int32 + 1]).astype(np.int32)
Out[2]: array([-2147483648], dtype=int32)

Do you think we could add a new casting option to astype that would check values for overflow? Something like the following (the exact option name and the resulting behavior - excpetion vs warning, etc. - is of course open to discussion):

In [3]: np.array([max_int32 + 1]).astype(np.int32, casting="checkoverflow")
---------------------------------------------------------------------------
OverflowError                                Traceback (most recent call last)
Cell In[3], line 1
----> 1 np.array([max_int32 + 1]).astype(np.int32, casting="checkoverflow")

OverflowError: Cannot cast array data from dtype('int64') to dtype('int32') because some values would overflow

This is already done for operations on scalars, but not on arrays, which I assume is due to the performance bottleneck it'd create, if we'd have to check integer overflow by hand. The same does not apply to astype though, since only the users that would be willing to sacrifice performance for checks would do so.

Thoughts?
@seberg
Copy link
Member
seberg commented Dec 22, 2023

There should be some old issues about this. This needs some focus to attack though.

  1. You need dedicated loops when things could fail.
  2. You need a new cast level:
    • But this casting level works very different from normal ones!
    • If the user requests it, but it isn't available, what do you do?
    • Actually, I am not sure it fits the current API, because the API is to return the cast safety. Maybe it works as a flag on unsafe or same-kind casting...
  3. You need to pass in the cast safety info into the inner-loop selection. (probably not hard, worst case, needs a nw function, which is OK).

Then, I don't think it should be checkoverflow IMO, it should be some wider definition of "is possible", although of course I have no idea if that applies to floating point casts (when they lose precision? Or just when they overflow to infinity? Or not at all?).

There is of course a different approach to this. I could imagine a ufunc as_index(). Mainly for casting to intp of course, but I suppose it could have loops for other integers.

@ngoldbaum ngoldbaum mentioned this issue Mar 28, 2024
Open
@crusaderky crusaderky mentioned this issue Oct 5, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@seberg @lysnikolaou
0