8000 Providing Public Cython API within NumPy · Issue #11803 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

Providing Public Cython API within NumPy #11803

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jakirkham opened this issue Aug 22, 2018 · 7 comments · Fixed by #12284
Closed

Providing Public Cython API within NumPy #11803

jakirkham opened this issue Aug 22, 2018 · 7 comments · Fixed by #12284
Labels
15 - Discussion

Comments

@jakirkham
Copy link
Contributor

Would it make sense for NumPy to provide numpy.pxd and numpy/math.pxd (currently included in Cython)?

Given NumPy already ships SWIG bindings, providing Cython bindings seems reasonable and is not too different in concept. Plus there are already internal consumers of Cython bindings, which would likely benefit from having access to a single public API. Would add that other projects like SciPy have been adding a fair number of things in *.pxd files included in their public API making this a bit of a trend. This would certainly help build a framework for NumPy to expose other things in the public API. Finally would note that having the Cython bindings live in Cython complicates the process of keeping them up-to-date as they must provide support for various versions of NumPy, which should hopefully be less of a problem if Cython bindings were included in NumPy's public API.

Thoughts on this would be welcome. :)
@shoyer
Copy link
Member
shoyer commented Aug 22, 2018

I think this would probably be a good idea.

@rgommers
Copy link
Member

Makes sense I think. The Cython wrappers in SciPy have been popular and unproblematic.

@mattip
Copy link
Member
mattip commented Aug 23, 2018

Sounds reasonable, and there is even a start in numpy/random/mtrand/numpy.pxd, which is used as include "numpy.pxd" in mtrand.pyx. I see that most code does not use the include statement rather the cython cimport: cimport numpy as np. How we would get the latter to use our wrapper over cython's wrapper? Would it be enough to encourage use of from numpy cimport numpy as np?

Should we pin tests to a certain version of cython, or would it be better to test against a range of cython versions (with the associated complications)?

@bashtage
Copy link
Contributor

Doesn't Cython already provide a lot of the NumPy API?

https://github.com/cython/cython/blob/master/Cython/Includes/numpy/__init__.pxd

@mattip
Copy link
Member
mattip commented Aug 23, 2018

@bashtage I think this discussion began as cython/cython#2498, and now we need to decide what is easier:

  • depend on cython to ship the wrapper and stay consistent with NumPy API changes
  • ship our own wrapper, and make sure we do not break with changes in cython

@scoder
Copy link
Contributor
scoder commented Sep 1, 2018

It seems totally right to me to have NumPy maintain and ship its own Cython declarations, rather than letting Cython do it. Above all, the C-API that they describe depends on the NumPy version and not on the Cython version, and it's not Cython's decision when and how to deal with additions or incompatible changes.

For a migration, Cython already searches the include path before its own standard include directories, so as long as users pass numpy.get_includes() into Cython's build, and NumPy keeps its .pxd files in that path, Cython should pick up NumPy's own .pxd files first. We'd still have to keep the files on our side for a while so that users can Cython-translate their code with older NumPy versions, but at least there'd be a layed-out path towards removing them somewhere in the 2020's.

@mattip mattip mentioned this issue Sep 16, 2018
Open
2 tasks
@jakirkham jakirkham mentioned this issue Oct 16, 2018
Closed
@jakirkham
Copy link
Contributor Author

This came up again recently in regards to whether certain NumPy functions require the GIL and whether that is reflected correctly in the Cython API. ( cython/cython#2663 )

Side note: Some questions were raised regarding documentation of GIL use in NumPy's C API as well.

This was referenced Oct 29, 2018
Merged
Merged
This was referenced May 5, 2020
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
15 - Discussion
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ENH: supply our version of numpy.pxd, requires cython>=0.29 mattip/numpy
7 participants
@rgommers @scoder @mattip @shoyer @jakirkham @bashtage @tylerjereddy
0