8000 Use Python buffer protocol when converting Python objects to .NET arrays · Issue #1838 · pythonnet/pythonnet · GitHub
[go: up one dir, main page]
Skip to content

Use Python buffer protocol when converting Python objects to .NET arrays #1838

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
lostmsu opened this issue Jun 27, 2022 · 9 comments
Open

Use Python buffer protocol when converting Python objects to .NET arrays #1838

lostmsu opened this issue Jun 27, 2022 · 9 comments
Labels
hacktoberfest performance

Comments

@lostmsu
Copy link
Member
lostmsu commented Jun 27, 2022 •
edited
Loading

Copying data from NumPy array to C# is a common task. Python.NET implements a simple sequence-to-array copying mechanism, which uses iterators. It is extremely inefficient for large arrays of primitive types.

Now we have a better way - using PyBuffer and buffer protocol. We could detect, that Python object implements the buffer protocol, use it to copy data when possible, and fallback to iteration otherwise.
@lostmsu
Copy link
Member Author
lostmsu commented Jun 27, 2022

If somebody wants to take on it, this test shows simple use of PyBuffer: TestBufferRead

And this is the method that needs to be changed: Converter.ToArray

@filmor
Copy link
Member
filmor commented Jun 27, 2022

This would not break API/ABI, so we can integrate it in a normal point release after 3.0, right?

@lostmsu
Copy link
Member Author
lostmsu commented Jun 27, 2022

Yep.

@lostmsu lostmsu added this to the PRs welcome milestone Jun 28, 2022
@bruceYangxianglin
Copy link

Copying data from C# array to Python list or Numpy array is also a common task. Is there a more efficient way?

@filmor filmor mentioned this issue Sep 15, 2022
Closed
@filmor filmor removed this from the PRs welcome milestone Sep 30, 2022
@filmor filmor mentioned this issue Jan 4, 2023
Closed
@bjudkewitz
Copy link

Agreed, converting a python array to a C# array is a very common task, and it is currently extremely slow. It would be fantastic to make it faster. Is it currently possibly with Python (and Python.NET) only, without any C# programming?

@yahav6893
Copy link

Hi! I’m looking for some guidance on how to speed up copying from Python to C#.
Would anyone be able to point me in the right way to do that?
Thanks ahead.

@chickenservice
Copy link

Hi, is someone currently working on this? If not I'd be happy to try. I've been using pythonnet at work successfully for years and would be glad to help. I'm familiar with the topic because I have already dealt with converting numpy arrays from/to C# arrays in an efficient manner. It'd be great to have this natively in pythonnet.

@filmor
Copy link
Member
filmor commented Apr 15, 2024

@chickenservice To my knowledge, no one is currently actively working on this, so give it a shot! An implementation would be greatly appreciated!

@chickenservice
Copy link

@filmor Perfect, is there anything besides the steps in CONTRIBUTING.md that I need to be aware of?

This was referenced May 5, 2024
Open
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
hacktoberfest performance
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@filmor @lostmsu @bjudkewitz @chickenservice @yahav6893 @bruceYangxianglin
0