Record arrays with fields that contain multiple objects will segfault on array.resize #4857

fgregg · 2014-07-09T21:04:53Z

This short script will always segfault for me.

import numpy
dtype = [('multiple objects', object, 2)]
a = numpy.ones(10, dtype=dtype)
a.resize(15,)
print a

juliantaylor · 2014-07-09T21:39:18Z

is this even the expected result?

In [1]: numpy.zeros(10, dtype=[('multiple objects', object, 2)])
Out[1]: 
array([([0, None],), ([0, None],), ([0, None],), ([0, None],),
       ([0, None],), ([0, None],), ([0, None],), ([0, None],),
       ([0, None],), ([0, None],)], 
      dtype=[('multiple objects', 'O', (2,))])

shouldn't it be all zero and not first zero rest none?

the issue appears as the resize path only sets the first entry and the rest is not memset to 0 to get the None

juliantaylor · 2014-07-09T21:43:57Z

fix would be something along the lines of this, assuming this behavior is indeed intended:

--- a/numpy/core/src/multiarray/shape.c
+++ b/numpy/core/src/multiarray/shape.c
@@ -328,6 +328,8 @@ _putzero(char *optr, PyObject *zero, PyArray_Descr *dtype)
     else {
         Py_INCREF(zero);
         NPY_COPY_PYOBJECT_PTR(optr, &zero);
+        npy_intp elsize = dtype->elsize;
+        memset(optr + sizeof(PyObject*), 0, elsize - sizeof(PyObject*));
     }
     return;
 }

np.zeros(2, dtype=[('k', object, 2)]) did only initialize the first element to zero while the rest stayed None. In [1]: numpy.zeros(2, dtype=[('k', object, 2)]) Out[1]: array([([0, None],), ([0, None],)], dtype=[('k', 'O', (2,))]) This is a surprising and likely not intended behavior which is fixed here. The changed function PyArray_FillObjectArray is only used with None or zero inputs from numpy, though as its part of the API it could affect third parties but this is not very likely. Additionally the memory after the first element was not initialized when the object was resized. Closes numpygh-4857

juliantaylor mentioned this issue Jul 18, 2014

BUG: initialize object array of array on resize and zeros #4889

Merged

juliantaylor closed this as completed in #4889 Jul 23, 2014

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Record arrays with fields that contain multiple objects will segfault on array.resize #4857

Record arrays with fields that contain multiple objects will segfault on array.resize #4857

Uh oh!

Uh oh!

Uh oh!

Record arrays with fields that contain multiple objects will segfault on array.resize #4857

Record arrays with fields that contain multiple objects will segfault on array.resize #4857

Comments

Uh oh!

Uh oh!