8000 ENH: support no-copy pickling for non-contiguous arrays, e.g., recarray · Issue #24226 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

ENH: support no-copy pickling for non-contiguous arrays, e.g., recarray #24226

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
dlee992 opened this issue Jul 21, 2023 · 2 comments
Closed

ENH: support no-copy pickling for non-contiguous arrays, e.g., recarray #24226

dlee992 opened this issue Jul 21, 2023 · 2 comments

Comments

@dlee992
Copy link
dlee992 commented Jul 21, 2023 •
edited
Loading

Proposed new feature or change:

Following this issue #11161 and related PR, can we support no-copy pickling for recarray, which is a subclass of ndarray? I think right now we don't support this feature, since recarray is non-contiguous arrays.

If it's possible in your experienced view, I guess we need to do some changes in numpy/core/src/multiarray/methods.c and related tests.

Example is here: ray-project/ray#37573 (comment). I can make up a version without Ray later.

And I saw a demo for how to handle pd DataFrame #12011 (comment), don't know if we can have a local trick for recarray too.
@seberg
Copy link
Member
seberg commented Jul 25, 2023 •
edited
Loading

Yes, you can probably do that, not sure what the rules are around non-contiguous buffers for pickling. But your examples are contiguous, the problem is just that it is an array subclass and that doesn't go down the buffer pickling path. Not sure there is actually a good reason for that.

EDIT: I.e. if you use a normal array with structured dtypes and not a recarray and it should work fine.

@dlee992
Copy link
Author
dlee992 commented Mar 1, 2024

close it for now I think.

@dlee992 dlee992 closed this as completed Mar 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@seberg @dlee992 and others
0