8000 MAINT: add freethreading_compatible directive to cython build by ngoldbaum · Pull Request #26950 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

MAINT: add freethreading_compatible directive to cython build #26950

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 16, 2024
Merged

MAINT: add freethreading_compatible directive to cython build #26950

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0858ae0
MAINT: add freethreading_compatible directive to cython build
ngoldbaum Jul 15, 2024
1ee327c
MAINT: re-add environment variable, add import test
ngoldbaum Jul 15, 2024
9d1d254
MAINT: verify importing numpy does not re-enable the GIL in CI
ngoldbaum Jul 15, 2024
072bd3c
TST: install cython nightly using a wheel
ngoldbaum Jul 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
MAINT: add freethreading_compatible directive to cython build
  • Loading branch information
@ngoldbaum
ngoldbaum committed Jul 15, 2024
commit 0858ae07b6941e40582aec4244b7b60fb826d6b6
2 changes: 0 additions & 2 deletions .github/workflows/linux.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -311,5 +311,3 @@ jobs:
run: |
pip install git+https://github.com/cython/cython
- uses: ./.github/meson_actions
env:
PYTHON_GIL: 0
6 changes: 6 additions & 0 deletions numpy/_core/tests/examples/cython/meson.build
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@ if not cy.version().version_compare('>=3.0.6')
error('tests requires Cython >= 3.0.6')
endif

cython_args = []
if cy.version().version_compare('>=3.1.0')
cython_args += ['-Xfreethreading_compatible=True']
endif

npy_include_path = run_command(py, [
'-c',
'import os; os.chdir(".."); import numpy; print(os.path.abspath(numpy.get_include()))'
Expand All @@ -34,4 +39,5 @@ py.extension_module(
'-DNPY_TARGET_VERSION=NPY_2_0_API_VERSION',
],
include_directories: [npy_include_path],
cython_args: cython_args,
)
10 changes: 9 additions & 1 deletion numpy/_core/tests/examples/cython/setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
for testing.
"""

import Cython
import numpy as np
from numpy._utils import _pep440
from distutils.core import setup
from Cython.Build import cythonize
from setuptools.extension import Extension
Expand All @@ -24,6 +26,12 @@

extensions = [checks]

compiler_directives = {}
if _pep440.parse(Cython.__version__) >= _pep440.parse("3.1.0a0"):
compiler_directives['freethreading_compatible'] = True

setup(
ext_modules=cythonize(extensions)
ext_modules=cythonize(
extensions,
compiler_directives=compiler_directives)
)
2 changes: 1 addition & 1 deletion numpy/meson.build
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -369,7 +369,7 @@ install_subdir('tests', install_dir: np_dir, install_tag: 'tests')
compilers = {
'C': cc,
'CPP': cpp,
'CYTHON': meson.get_compiler('cython')
'CYTHON': cy,
}

machines = {
Expand Down
9 changes: 8 additions & 1 deletion numpy/random/_examples/cython/meson.build
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@ if not cy.version().version_compare('>=3.0.6')
error('tests requires Cython >= 3.0.6')
endif

base_cython_args = []
if cy.version().version_compare('>=3.1.0a0')
base_cython_args += ['-Xfreethreading_compatible=True']
endif

_numpy_abs = run_command(py3, ['-c',
'import os; os.chdir(".."); import numpy; print(os.path.abspath(numpy.get_include() + "../../.."))'],
check: true).stdout().strip()
Expand All @@ -27,20 +32,22 @@ py3.extension_module(
install: false,
include_directories: [npy_include_path],
dependencies: [npyrandom_lib, npymath_lib],
cython_args: base_cython_args,
)
py3.extension_module(
'extending',
'extending.pyx',
install: false,
include_directories: [npy_include_path],
dependencies: [npyrandom_lib, npymath_lib],
cython_args: base_cython_args,
)
py3.extension_module(
'extending_cpp',
'extending_distributions.pyx',
install: false,
override_options : ['cython_language=cpp'],
cython_args: ['--module-name', 'extending_cpp'],
cython_args: base_cython_args + ['--module-name', 'extending_cpp'],
include_directories: [npy_include_path],
dependencies: [npyrandom_lib, npymath_lib],
)
6 changes: 6 additions & 0 deletions numpy/random/meson.build
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,11 @@ if host_machine.system() == 'cygwin'
c_args_random += ['-Wl,--export-all-symbols']
endif

cython_args = []
if cy.version().version_compare('>=3.1.0')
cython_args += ['-Xfreethreading_compatible=True']
endif

# name, sources, extra c_args, extra static libs to link
random_pyx_sources = [
['_bounded_integers', _bounded_integers_pyx, [], [npyrandom_lib, npymath_lib]],
Expand Down Expand Up @@ -83,6 +88,7 @@ foreach gen: random_pyx_sources
link_with: gen[3],
install: true,
subdir: 'numpy/random',
cython_args: cython_args,
)
endforeach

Expand Down
4 changes: 0 additions & 4 deletions tools/wheels/cibw_test_command.sh

View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,6 @@ if [[ $FREE_THREADED_BUILD == "True" ]]; then
# with a released version of cython
python -m pip uninstall -y cython
python -m pip install -i https://pypi.anaconda.org/scientific-python-nightly-wheels/simple cython
# TODO: delete when importing numpy no longer enables the GIL
# setting to zero ensures the GIL is disabled while running the
# tests under free-threaded python
export PYTHON_GIL=0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One problem here is that the f2py test still need it. So half-way along the tests suite we actually would re-enable this.

So I think this should stay for now but change the comment to say that it is for f2py?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, good point, let me check if the gil is being re-enabled in the test process or just in a subprocess.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

numpy/f2py/tests/test_assumed_shape.py::TestAssumedShapeSumExample::test_all
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /Users/goldbaum/Documents/numpy/build-install/usr/lib/python3.13/site-packages/numpy/f2py/tests/test_
8B07
assumed_shape.py(19)test_all()
-> breakpoint()
(Pdb) import sys
(Pdb) sys._is_gil_enabled()
True

It looks like the f2py tests do import the test modules in-process, so you're right we still need to set the environment variables. I'll update the comments around those.

If there was a way to enable and disable the GIL at runtime we could hack around this but there isn't an API for that yet (if ever).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added it back and also added an additional test to manually check that import numpy doesn't print the warning about the GIL being re-enabled.

fi

# Run full tests with -n=auto. This makes pytest-xdist distribute tests across
Expand Down
Loading
0