8000 MNT: Adopt pythoncapi-compat? · Issue #26158 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content

MNT: Adopt pythoncapi-compat? #26158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ngoldbaum opened this issue Mar 28, 2024 · 5 comments
Closed

MNT: Adopt pythoncapi-compat? #26158

ngoldbaum opened this issue Mar 28, 2024 · 5 comments
Assignees
Labels
03 - Maintenance 39 - free-threading PRs and issues related to support for free-threading CPython (a.k.a. no-GIL, PEP 703)

Comments

@ngoldbaum
Copy link
Member

As noted in PEP 703, C API functions returning borrowed references are problematic in the nogil build of Python, and new functions are available in Python 3.13 that return new references.

Three years ago @vstinner proposed vendoring pythoncapi-compat to replace npy3k_compat.h in #18713 and that was rejected.

Unlike that proposal, I don't want to expose pythoncapi-compat.h in the public numpy C API, instead I'd only like to use it internally, replacing the internal npy_pycompat.h header. That means concerns raised in that PR about requiring C99 or adding potentially problematic constructs to the numpy C API are moot.

We could also just copy/paste the definitions of the functions I need from the pythoncapi-compat.h header, but I thought it would be worth taking the project's temperature on adding it as a build-time dependency via a submodule like highway and pocketfft, since there will inevitably be other things we want in the future.

@rgommers
Copy link
Member

This proposal seems like the right way to go to me. We now have other git submodules, and these compat functions are probably going to be updated semi-regularly during the 3.13/3.14 release cycles. An extra git submodule is preferable over copy-pasting (LoC count difference as discussed in gh-18713 isn't too relevant here).

@vstinner
Copy link
Contributor
vstinner commented Apr 2, 2024

You might also look at my experimental https://github.com/vstinner/deadparrot/ project which is similar to pythoncapi-compat, but it's implemented as a shared library, rather than a header file. It might be harder to integrate in numpy.

@rgommers
Copy link
Member
rgommers commented Apr 2, 2024

It's possible to integrate a shared library, but we don't have any yet. It'd be slightly annoying because we'd have to deal with settings RPATHs and things like os.add_dll_directory or preloading with ctypes.WinDLL on Windows. And it's not clear to me that it'll work when several packages each preload the shared library - this probably requires name mangling.

Is there an advantage to using deadparrot?

@vstinner
Copy link
Contributor
vstinner commented Apr 3, 2024

Is there an advantage to using deadparrot?

My plan for the future is to backport/forward port way more complicated functions which don't fit into pythoncapi-compat.h. But numpy should not really need them, so pythoncapi-compat.h should be better fit for your practical constraints.

@ngoldbaum
Copy link
Member Author
ngoldbaum 8239 commented Apr 9, 2024

We added the submodule in #26189

@rgommers rgommers added the 39 - free-threading PRs and issues related to support for free-threading CPython (a.k.a. no-GIL, PEP 703) label Apr 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
03 - Maintenance 39 - free-threading PRs and issues related to support for free-threading CPython (a.k.a. no-GIL, PEP 703)
Projects
None yet
Development

No branches or pull requests

3 participants
0