8000 BUG: Race in computing initial value for reductions under free-threading · Issue #28041 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

BUG: Race in computing initial value for reductions under free-threading #28041

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking &l 8000 dquo;Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
hawkinsp opened this issue Dec 19, 2024 · 5 comments · Fixed by #28123
Closed

BUG: Race in computing initial value for reductions under free-threading #28041

hawkinsp opened this issue Dec 19, 2024 · 5 comments · Fixed by #28123
Labels
00 - Bug 39 - free-threading PRs and issues related to support for free-threading CPython (a.k.a. no-GIL, PEP 703)
Milestone
2.2.2 release

Comments

@hawkinsp
Copy link
Contributor

Describe the issue:

With:

  • CPython 3.13 built with the GIL disabled, using thread-sanitizer under clang,
  • NumPy 2.3.0dev commit 35b2c4a built with thread-sanitizer and clang

thread sanitizer reports a race constructing the cached initial value for a reducer.

You may need to run the example a few times, but it reproduced first time for me.

Reproduce the code example:

import concurrent.futures
import threading
import numpy as np

num_threads = 8


x = np.arange(1000)
b = threading.Barrier(num_threads)
def closure():
  b.wait()
  np.sum(x)

with concurrent.futures.ThreadPoolExecutor(max_workers=num_threads) as executor:
  for _ in range(num_threads):
    executor.submit(closure)

Error message:

WARNING: ThreadSanitizer: data race (pid=4069590)
  Write of size 8 at 0x7f6412728ef8 by thread T135:
    #0 get_initial_from_ufunc legacy_array_method.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3de246) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #1 PyUFunc_ReduceWrapper <null> (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3e0426) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #2 PyUFunc_Reduce ufunc_object.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3f0349) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #3 PyUFunc_GenericReduction ufunc_object.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3efffa) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #4 ufunc_reduce ufunc_object.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3ed841) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #5 cfunction_vectorcall_FASTCALL_KEYWORDS /usr/local/google/home/phawkins/p/cpython/Objects/methodobject.c:441:24 (python3.13+0x28a010) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #6 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #7 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #8 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #9 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #10 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #11 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #12 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #13 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eafaa) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #14 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eafaa)
    #15 dispatcher_vectorcall arrayfunction_override.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x22d22b) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #16 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eafaa) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #17 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eafaa)
    #18 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e24fb) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #19 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #20 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #21 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #22 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #23 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
    #24 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #25 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #26 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #27 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #28 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)

  Previous write of size 8 at 0x7f6412728ef8 by thread T131:
    #0 get_initial_from_ufunc legacy_array_method.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3de246) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #1 PyUFunc_ReduceWrapper <null> (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3e0426) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #2 PyUFunc_Reduce ufunc_object.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3f0349) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #3 PyUFunc_GenericReduction ufunc_object.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3efffa) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #4 ufunc_reduce ufunc_object.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3ed841) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #5 cfunction_vectorcall_FASTCALL_KEYWORDS /usr/local/google/home/phawkins/p/cpython/Objects/methodobject.c:441:24 (python3.13+0x28a010) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #6 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #7 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #8 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #9 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #10 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #11 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #12 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #13 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eafaa) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #14 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eafaa)
    #15 dispatcher_vectorcall arrayfunction_override.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x22d22b) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #16 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eafaa) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #17 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eafaa)
    #18 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e24fb) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #19 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #20 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #21 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #22 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #23 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
    #24 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #25 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #26 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #27 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #28 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)

Python and NumPy Versions:

2.3.0.dev0+git20241219.35b2c4a
3.13.1 experimental free-threading build (tags/v3.13.1:06714517797, Dec 15 2024, 15:38:01) [Clang 18.1.8 (11)]

Runtime Environment:

[{'numpy_version': '2.3.0.dev0+git20241219.35b2c4a',
'python': '3.13.1 experimental free-threading build '
'(tags/v3.13.1:06714517797, Dec 15 2024, 15:38:01) [Clang 18.1.8 '
'(11)]',
'uname': uname_result(system='Linux', node='', release='', version='#1 SMP PREEMPT_DYNAMIC Debian 6.redacted (2024-10-16)', machine='x86_64')},
{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C',
'FMA3',
'AVX2'],
'not_found': ['AVX512F',
'AVX512CD',
'AVX512_KNL',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL']}},
{'architecture': 'Zen',
'filepath': '/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.27.so',
'internal_api': 'openblas',
'num_threads': 128,
'prefix': 'libopenblas',
'threading_layer': 'pthreads',
'user_api': 'blas',
'version': '0.3.27'}]

Context for the issue:

Found when working on free-threading support in JAX.
@charris charris added this to the 2.2.2 release milestone Dec 19, 2024
@seberg
Copy link
Member
seberg commented Dec 20, 2024

Doesn't seem too worrying in practice. Anyway, without diving in, my guess is that we can move this to ArrayMethod creation time where it is already guarded (either at import or during promotion).

@charris charris added the 39 - free-threading PRs and issues related to support for free-threading CPython (a.k.a. no-GIL, PEP 703) label Dec 21, 2024
@ngoldbaum
Copy link
Member

Unfortunately I'm not able to build NumPy with TSAN support on my Mac because of #28087. I'm guessing Linux doesn't hit that for whatever reason? Will try again on my Linux machine...

@ngoldbaum
8000 Copy link
Member

Thanks to some help from @Mousius I was able to fix the build issue and reproduce this. Thanks for the detailed reproducer and thanks also to @vfdev-5 for the detailed instructions on how to setup CPython and NumPy to reproduce this.

I'm planning to look through all the issues @hawkinsp opened using tsan and fix the most pressing issues. I'd also like to add new tests and a thread sanitizer CI run if there's appetite to do that. It seems pretty straightforward to do this on an ARM Mac and I think it would be nice to run thread sanitizer tests on a machine with weak memory ordering.

@ngoldbaum
Copy link
Member

Here's the same warning with line numbers in NumPy, as reported by clang 19:

WARNING: ThreadSanitizer: data race (pid=29241)
  Write of size 8 at 0x000108657318 by thread T50:
    #0 get_initial_from_ufunc legacy_array_method.c:330 (_multiarray_umath.cpython-313t-darwin.so:arm64+0x4899f0)
    #1 PyUFunc_ReduceWrapper reduction.c:328 (_multiarray_umath.cpython-313t-darwin.so:arm64+0x48cec0)
    #2 PyUFunc_Reduce ufunc_object.c:2557 (_multiarray_umath.cpython-313t-darwin.so:arm64+0x4a1aa0)
    #3 PyUFunc_GenericReduction ufunc_object.c:3680 (_multiarray_umath.cpython-313t-darwin.so:arm64+0x4a143c)
    #4 ufunc_reduce ufunc_object.c:5357 (_multiarray_umath.cpython-313t-darwin.so:arm64+0x49ed14)
    #5 cfunction_vectorcall_FASTCALL_KEYWORDS methodobject.c:441 (libpython3.13t.dylib:arm64+0x11491c)
    #6 _PyObject_Call call.c:348 (libpython3.13t.dylib:arm64+0x806ec)
    #7 PyObject_Call call.c:373 (libpython3.13t.dylib:arm64+0x80784)
    #8 _PyEval_EvalFrameDefault generated_cases.c.h:1355 (libpython3.13t.dylib:arm64+0x25fe5c)
    #9 _PyEval_Vector ceval.c:1807 (libpython3.13t.dylib:arm64+0x2552cc)
    #10 _PyFunction_Vectorcall call.c (libpython3.13t.dylib:arm64+0x80aa8)
    #11 PyObject_Vectorcall call.c:327 (libpython3.13t.dylib:arm64+0x8042c)
    #12 dispatcher_vectorcall arrayfunction_override.c:580 (_multiarray_umath.cpython-313t-darwin.so:arm64+0x241654)
    #13 PyObject_Vectorcall call.c:327 (libpython3.13t.dylib:arm64+0x8042c)
    #14 _PyEval_EvalFrameDefault generated_cases.c.h:813 (libpython3.13t.dylib:arm64+0x25e624)
    #15 _PyEval_Vector ceval.c:1807 (libpython3.13t.dylib:arm64+0x2552cc)
    #16 _PyFunction_Vectorcall call.c (libpython3.13t.dylib:arm64+0x80aa8)
    #17 method_vectorcall classobject.c:70 (libpython3.13t.dylib:arm64+0x84714)
    #18 _PyObject_Call call.c:348 (libpython3.13t.dylib:arm64+0x806ec)
    #19 PyObject_Call call.c:373 (libpython3.13t.dylib:arm64+0x80784)
    #20 thread_run _threadmodule.c:337 (libpython3.13t.dylib:arm64+0x3b6e40)
    #21 pythread_wrapper thread_pthread.h:243 (libpython3.13t.dylib:arm64+0x320c88)

  Previous write of size 8 at 0x000108657318 by thread T1:
    #0 get_initial_from_ufunc legacy_array_method.c:330 (_multiarray_umath.cpython-313t-darwin.so:arm64+0x4899f0)
    #1 PyUFunc_ReduceWrapper reduction.c:328 (_multiarray_umath.cpython-313t-darwin.so:arm64+0x48cec0)
    #2 PyUFunc_Reduce ufunc_object.c:2557 (_multiarray_umath.cpython-313t-darwin.so:arm64+0x4a1aa0)
    #3 PyUFunc_GenericReduction ufunc_object.c:3680 (_multiarray_umath.cpython-313t-darwin.so:arm64+0x4a143c)
    #4 ufunc_reduce ufunc_object.c:5357 (_multiarray_umath.cpython-313t-darwin.so:arm64+0x49ed14)
    #5 cfunction_vectorcall_FASTCALL_KEYWORDS methodobject.c:441 (libpython3.13t.dylib:arm64+0x11491c)
    #6 _PyObject_Call call.c:348 (libpython3.13t.dylib:arm64+0x806ec)
    #7 PyObject_Call call.c:373 (libpython3.13t.dylib:arm64+0x80784)
    #8 _PyEval_EvalFrameDefault generated_cases.c.h:1355 (libpython3.13t.dylib:arm64+0x25fe5c)
    #9 _PyEval_Vector ceval.c:1807 (libpython3.13t.dylib:arm64+0x2552cc)
    #10 _PyFunction_Vectorcall call.c (libpython3.13t.dylib:arm64+0x80aa8)
    #11 PyObject_Vectorcall call.c:327 (libpython3.13t.dylib:arm64+0x8042c)
    #12 dispatcher_vectorcall arrayfunction_override.c:580 (_multiarray_umath.cpython-313t-darwin.so:arm64+0x241654)
    #13 PyObject_Vectorcall call.c:327 (libpython3.13t.dylib:arm64+0x8042c)
    #14 _PyEval_EvalFrameDefault generated_cases.c.h:813 (libpython3.13t.dylib:arm64+0x25e624)
    #15 _PyEval_Vector ceval.c:1807 (libpython3.13t.dylib:arm64+0x2552cc)
    #16 _PyFunction_Vectorcall call.c (libpython3.13t.dylib:arm64+0x80aa8)
    #17 method_vectorcall classobject.c:70 (libpython3.13t.dylib:arm64+0x84714)
    #18 _PyObject_Call call.c:348 (libpython3.13t.dylib:arm64+0x806ec)
    #19 PyObject_Call call.c:373 (libpython3.13t.dylib:arm64+0x80784)
    #20 thread_run _threadmodule.c:337 (libpython3.13t.dylib:arm64+0x3b6e40)
    #21 pythread_wrapper thread_pthread.h:243 (libpython3.13t.dylib:arm64+0x320c88)

  Thread T50 (tid=32310110, running) created by main thread at:
    #0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31050)
    #1 do_start_joinable_thread thread_pthread.h:290 (libpython3.13t.dylib:arm64+0x31f8f4)
    #2 PyThread_start_joinable_thread thread_pthread.h:314 (libpython3.13t.dylib:arm64+0x31f738)
    #3 do_start_new_thread _threadmodule.c:1849 (libpython3.13t.dylib:arm64+0x3b69ec)
    #4 thread_PyThread_start_joinable_thread _threadmodule.c:1972 (libpython3.13t.dylib:arm64+0x3b5aa8)
    #5 cfunction_call methodobject.c:540 (libpython3.13t.dylib:arm64+0x115600)
    #6 _PyObject_MakeTpCall call.c:242 (libpython3.13t.dylib:arm64+0x7f870)
    #7 PyObject_Vectorcall call.c:327 (libpython3.13t.dylib:arm64+0x804c8)
    #8 _PyEval_EvalFrameDefault generated_cases.c.h:1502 (libpython3.13t.dylib:arm64+0x25f41c)
    #9 PyEval_EvalCode ceval.c:597 (libpython3.13t.dylib:arm64+0x254fbc)
    #10 run_eval_code_obj pythonrun.c:1337 (libpython3.13t.dylib:arm64+0x305250)
    #11 run_mod pythonrun.c:1422 (libpython3.13t.dylib:arm64+0x3049dc)
    #12 _PyRun_SimpleFileObject pythonrun.c:490 (libpython3.13t.dylib:arm64+0x300710)
    #13 _PyRun_AnyFileObject pythonrun.c:77 (libpython3.13t.dylib:arm64+0x2fff7c)
    #14 pymain_run_file main.c:429 (libpython3.13t.dylib:arm64+0x3369d4)
    #15 Py_RunMain main.c:776 (libpython3.13t.dylib:arm64+0x335c9c)
    #16 pymain_main main.c:806 (libpython3.13t.dylib:arm64+0x336040)
    #17 Py_BytesMain main.c:830 (libpython3.13t.dylib:arm64+0x336118)
    #18 main <null> (python3.13:arm64+0x100003ec4)

  Thread T1 (tid=32310061, running) created by main thread at:
    #0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31050)
    #1 do_start_joinable_thread thread_pthread.h:290 (libpython3.13t.dylib:arm64+0x31f8f4)
    #2 PyThread_start_joinable_thread thread_pthread.h:314 (libpython3.13t.dylib:arm64+0x31f738)
    #3 do_start_new_thread _threadmodule.c:1849 (libpython3.13t.dylib:arm64+0x3b69ec)
    #4 thread_PyThread_start_joinable_thread _threadmodule.c:1972 (libpython3.13t.dylib:arm64+0x3b5aa8)
    #5 cfunction_call methodobject.c:540 (libpython3.13t.dylib:arm64+0x115600)
    #6 _PyObject_MakeTpCall call.c:242 (libpython3.13t.dylib:arm64+0x7f870)
    #7 PyObject_Vectorcall call.c:327 (libpython3.13t.dylib:arm64+0x804c8)
    #8 _PyEval_EvalFrameDefault generated_cases.c.h:1502 (libpython3.13t.dylib:arm64+0x25f41c)
    #9 PyEval_EvalCode ceval.c:597 (libpython3.13t.dylib:arm64+0x254fbc)
    #10 run_eval_code_obj pythonrun.c:1337 (libpython3.13t.dylib:arm64+0x305250)
    #11 run_mod pythonrun.c:1422 (libpython3.13t.dylib:arm64+0x3049dc)
    #12 _PyRun_SimpleFileObject pythonrun.c:490 (libpython3.13t.dylib:arm64+0x300710)
    #13 _PyRun_AnyFileObject pythonrun.c:77 (libpython3.13t.dylib:arm64+0x2fff7c)
    #14 pymain_run_file main.c:429 (libpython3.13t.dylib:arm64+0x3369d4)
    #15 Py_RunMain main.c:776 (libpython3.13t.dylib:arm64+0x335c9c)
    #16 pymain_main main.c:806 (libpython3.13t.dylib:arm64+0x336040)
    #17 Py_BytesMain main.c:830 (libpython3.13t.dylib:arm64+0x336118)
    #18 main <null> (python3.13:arm64+0x100003ec4)

SUMMARY: ThreadSanitizer: data race legacy_array_method.c:330 in get_initial_from_ufunc

And indeed we're writing to global state on the ArrayMethod here:

numpy/numpy/_core/src/umath/legacy_array_method.c

Lines 328 to 330 in 206f697

memcpy(context->method->legacy_initial, initial,
context->descriptors[0]->elsize);
context->method->get_reduction_initial = &copy_cached_initial;

I agree it's probably best to do this when you create the ArrayMethod.

@ngoldbaum ngoldbaum mentioned this 8D7C issue Jan 7, 2025
Merged
@ngoldbaum
Copy link
Member

I opened #28123

@charris charris mentioned this issue Jan 9, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug 39 - free-threading PRs and issues related to support for free-threading CPython (a.k.a. no-GIL, PEP 703)
Projects
None yet
Milestone
2.2.2 release
Development

Successfully merging a pull request may close this issue.

BUG: move reduction initialization to ufunc initialization ngoldbaum/numpy
4 participants
@seberg @charris @hawkinsp @ngoldbaum
0