8000 ENH: Create obvious way of getting a single 1-D list with all array elements · Issue #24989 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content

ENH: Create obvious way of getting a single 1-D list with all array elements #24989

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
vadimkantorov opened this issue Oct 23, 2023 · 8 comments
Labels
01 - Enhancement 62 - Python API Changes or additions to the Python API. Mailing list should usually be notified.

Comments

@vadimkantorov
Copy link
vadimkantorov commented Oct 23, 2023

Proposed new feature or change:

@rgommers This has been discussed several times:

I propose to review it once again to potentially introduce a breaking change or some solution for this in NumPy 2.0 or Array API. It's most of time inconvenient to have the return type change as a function of numel() - usually we do some array processing after and having conditions complicates the consuming code.

Maybe a new argument force=True argument can be introduced with existing behavior force=False as default, and atleast1d().tolist() behavior if force=True is passed. OR maybe a new method aslist() can be introduced and tolist() be deprecated/produce-warnings and removed in a few years.

@rgommers
Copy link
Member

I browsed gh-16243 and I don't find the reasoning for not returning a list compelling:

They need to be different because of the intent that array(x.tolist()) == x (note there are exceptions to this intent already, in cases like np.zeros((0, 2)) where it is impossible).

It seems pretty obvious to me that a method named tolist must return a list. It's not really a problem if a 0-D array and 1-D array with 1 element return the same thing - which they don't right now:

>>> np.array([42]).tolist()
[42]
>>> np.array(42).tolist()
42
>>> np.int64(42).tolist()
42

There are also several TODO's in numpy/__init__.pyi, because the current behavior is a problem for static typing. And we decided to, in principle, fix up that type of non-type-stable behavior in the discussions for 2.0.

This wasn't really on my radar, but I think it'd be a reasonable change to make for 2.0 (always returning a list that is, not force=True), that won't have a significant backwards compatibility impact.

I wasn't involved in the previous decision though, and I may be missing something. So pinging @eric-wieser and @seberg for input here.

Also, we are pretty short on time for all the plans/wishes for 2.0. If the outcome is that we want to do this, would you be willing to implement it @vadimkantorov?

@rgommers rgommers added the 62 - Python API Changes or additions to the Python API. Mailing list should usually be notified. label Oct 24, 2023
@seberg
Copy link
Member
seberg commented Oct 24, 2023

Yes, it is mild misnomer, but no, I am absolutely not in favor of changing this.

It's not really a problem if a 0-D array and 1-D array with 1 element return the same thing

And what about N-D arrays? I suspect this request comes from users mixing 1-D or 0-D arrays for whatever reason, if they were having 2-D arrays in there we would have another round of discussion and frustration.
So changing this makes one use case (mixed 1-D and 0-D arrays) clearer, while breaking an unknown amount of use-cases where dimensionality matters.

I would say a tolits() that implicitly ravels makes sense and probably would be the better function to begin with, but fuzzying a correct dimensionality generalization isn't helpful IMO.

You can use arr.ravel().tolist() or list(arr.flat), or... (the .tolist() path might be slightly faster).

@rgommers
Copy link
Member

And what about N-D arrays? I suspect this request comes from users mixing 1-D or 0-D arrays for whatever reason, if they were having 2-D arrays in there we would have another round of discussion and frustration.

Why? That returns a list just fine, nothing changes:

>>> x = np.ones((3, 2))
>>> l = x.tolist()
>>> type(l)
<class 'list'>
>>> l
[[1.0, 1.0], [1.0, 1.0], [1.0, 1.0]]

@seberg
Copy link
Member
seberg commented Oct 24, 2023

The point is that we return a nested list of depth N, the reason that N=0 is special there, is because the user has only 1-D arrays otherwise, I am sure. Not because 0 is really all that special.

@eric-wieser
Copy link
Member

I'm not at python right now, but I think our tolist is consistent with memoryview.tolist on 0D arrays (and I definitely still think that array(1).tolist() should not be [1]!)

@rgommers
Copy link
Member

Okay, I interpreted the request as type(x.tolist() should be list, nothing beyond that. @vadimkantorov can you clarify? What will you do with it, and do you use >=2-D arrays?

@seberg
Copy link
Member
seberg commented Oct 24, 2023

The simple reason is, that returning a list breaks consistency quite badly, and that is bound to bite someone else. Most likely worse than then the mild convenience added for the original person here.

There is a simple solution IMO, and I have already said that, add one or both of:

  • arr.flat.tolist()
  • arr.tolist(raveled=True)

(It is the job of the user to know the want to ravel higher dimensional arrays then, but OK.)

@seberg seberg removed the 62 - Python API Changes or additions to the Python API. Mailing list should usually be notified. label Oct 26, 2023
@seberg seberg changed the title ENH: review once again tolist() return type, potentially for NumPy 2.0 or Array API ENH: Create obvious way of getting a single 1-D list with all array elements Oct 26, 2023
@seberg seberg added 01 - Enhancement 62 - Python API Changes or additions to the Python API. Mailing list should usually be notified. labels Oct 26, 2023
@seberg
Copy link
Member
seberg commented Oct 26, 2023

Also it seems like this was trigger of the current iteration: pytorch/pytorch#104908 (comment) where you are dealing with N dimensions anyway and already must flatten. All additionally added checks are unnecessary (unless torch gets flatten() wrong).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
01 - Enhancement 62 - Python API Changes or additions to the Python API. Mailing list should usually be notified.
Projects
None yet
Development

No branches or pull requests

4 participants
0