Description
Describe the issue:
I get an EXC_BAD_ACCESS Error when calling import_array() in an c++ program if Python runs in an different thread.
In detail I have written a module that uses Numpy. If I initialize Python in my main thread and import my module everything works fine. But if I initialize Python in an different thread and import the module in this thread, numpy crashes when calling import_array(). More precise the crash happens wenn PyImport_ImportModule("numpy.core._multiarray_umath") is called in _import_array() (_multiarray_api.h).
I'm using macOS Monterey 12.3.1 on an Apple Silicon (M1) processor. My Python version is 3.9.6 and the NumPy I'm using is 1.21.4. Python and Numpy are for ARM architecture. As compiler I'm using Clang 12.0.5.
The issue can also be reproduced on Mac machines with intel processors
On Windows no problem occurs.
Reproduce the code example:
#define PY_SSIZE_T_CLEAN
#include <Python.h>
#include "numpy/arrayobject.h"
#include <iostream>
static PyObject *
spam_func(PyObject *self, PyObject *args)
{
PyObject* rhsNpArray = NULL;
if (!PyArg_ParseTuple(args, "O!", &PyArray_Type, &rhsNpArray))
return NULL;
int number = PyArray_NDIM(rhsNpArray);
return PyLong_FromSize_t(number);
}
static PyMethodDef SpamMethods[] = {
{"func", spam_func, METH_VARARGS,
"Execute spam_func."},
{NULL, NULL, 0, NULL} /* Sentinel */
};
static struct PyModuleDef spammodule = {
PyModuleDef_HEAD_INIT,
"spam",
NULL,
-1,
SpamMethods
};
PyMODINIT_FUNC
PyInit_spam(void)
{
import_array();// here the crash happens
if (PyErr_Occurred()) {
std::cerr << "Failed to import numpy Python module(s)." << std::endl;
return NULL;
}
assert(! PyErr_Occurred());
return PyModule_Create(&spammodule);
}
void doPythonThing()
{
if (PyImport_AppendInittab("spam", PyInit_spam) == -1) {
fprintf(stderr, "Error: could not extend in-built modules table\n");
exit(1);
}
Py_Initialize();
PyObject *pmodule = PyImport_ImportModule("spam");
}
int
main(int argc, char *argv[])
{
//calling doPythonThing without a new thread works fine
std::thread thread(doPythonThing);
thread.join();
return 0;
}Error message:
Process 5308 launched: '/Users/robin/Desktop/build/pythonTest/spam' (arm64)
Process 5308 stopped
* thread #4, stop reason = EXC_BAD_ACCESS (code=2, address=0x16ff1bff8)
frame #0: 0x000000018ec250cc libsystem_pthread.dylib`___chkstk_darwin + 60
libsystem_pthread.dylib`___chkstk_darwin:
-> 0x18ec250cc <+60>: ldur x11, [x11, #-0x8]
0x18ec250d0 <+64>: mov x10, sp
0x18ec250d4 <+68>: cmp x9, #0x1, lsl #12 ; =0x1000
0x18ec250d8 <+72>: b.lo 0x18ec250f0 ; <+96>
Target 0: (spam) stopped.NumPy/Python version information:
1.21.4 3.9.6 (v3.9.6:db3ff76da1, Jun 28 2021, 11:14:58)
[Clang 12.0.5 (clang-1205.0.22.9)]