8000 make warning conditional on NUMPY_WARN_IF_NO_MEM_POLICY · numpy/numpy@27106b0 · GitHub
[go: up one dir, main page]
Skip to content

Commit 27106b0

Browse files
mattipmattip
committed
make warning conditional on NUMPY_WARN_IF_NO_MEM_POLICY
1 parent 7465931 commit 27106b0

File tree

4 files changed

+62
-11
lines changed

4 files changed

+62
-11
lines changed

‎doc/release/upcoming_changes/17582.new_feature.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@ NEP 49 configurable allocators
33
As detailed in `NEP 49`_, the function used for allocation of the data segment
44
of a ndarray can be changed. The policy can be set globally or in a context.
55
For more information see the NEP and the :ref:`data_memory` reference docs.
6+
Also add a ``NUMPY_WARN_IF_NO_MEM_POLICY`` override to warn on dangerous use
7+
of transfering ownership by setting ``NPY_ARRAY_OWNDATA``.
68

79
.. _`NEP 49`: https://numpy.org/neps/nep-0049.html
810

‎doc/source/reference/c-api/data_memory.rst

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,3 +119,36 @@ For an example of setting up and using the PyDataMem_Handler, see the test in
119119
operations that might cause new allocation events (such as the
120120
creation/destruction numpy objects, or creating/destroying Python
121121
objects which might cause a gc)
122+
123+
What happens when deallocating if there is no policy set
124+
--------------------------------------------------------
125+
126+
A rare but useful technique is to allocate a buffer outside NumPy, use
127+
:c:function:`PyArray_NewFromDescr` to wrap the buffer in a ``ndarray``, then switch
128+
the ``OWNDATA`` flag to true. When the ``ndarray`` is released, the
129+
appropriate function from the ``ndarray``'s ``PyDataMem_Handler`` should be
130+
called to free the buffer. But the ``PyDataMem_Handler`` field was never set,
131+
it will be ``NULL``. For backward compatibility, NumPy will call ``free()`` to
132+
release the buffer. If ``NUMPY_WARN_IF_NO_MEM_POLICY`` is set to ``1``, a
133+
warning will be emitted. The current default is not to emit a warning, this may
134+
change in a future version of NumPy.
135+
136+
A better technique would be to use a ``PyCapsule`` as a base object:
137+
138+
.. code-block:: c
139+
140+
/* define a PyCapsule_Destructor, using the correct deallocator for buff */
141+
void free_wrap(PyObject *obj){ free(obj); };
142+
143+
/* then inside the function that creates arr from buff */
144+
...
145+
arr = PyArray_NewFromDescr(... buf, ...);
146+
if (arr == NULL) {
147+
return NULL;
148+
}
149+
capsule = PyCapsule_New(buf, "my_wrapped_buffer", free_wrap);
150+
if (PyArray_SetBaseObject(arr, capsule) == -1) {
151+
Py_DECREF(arr);
152+
return NULL;
153+
}
154+
...

‎numpy/core/src/multiarray/arrayobject.c

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -502,10 +502,17 @@ array_dealloc(PyArrayObject *self)
502502
nbytes = fa->descr->elsize ? fa->descr->elsize : 1;
503503
}
504504
if (fa->mem_handler == NULL) {
505-
char const * msg = "Trying to dealloc data, but a memory policy "
506-
"is not set. If you take ownership of the data, you must "
507-
"also set a memory policy.";
508-
WARN_IN_DEALLOC(PyExc_RuntimeWarning, msg);
505+
char *env = getenv("NUMPY_WARN_IF_NO_MEM_POLICY");
506+
if (env == NULL) {
507+
env = "0";
508+
}
509+
if (strncmp(env, "1", 1) == 0)
510+
{
511+
char const * msg = "Trying to dealloc data, but a memory policy "
512+
"is not set. If you take ownership of the data, you must "
513+
"also set a memory policy.";
514+
WARN_IN_DEALLOC(PyExc_RuntimeWarning, msg);
515+
}
509516
// Guess at malloc/free ???
510517
free(fa->data);
511518
} else {

‎numpy/core/tests/test_mem_policy.py

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
import asyncio
22
import gc
3+
import os
34
import pytest
45
import numpy as np
56
import threading
@@ -331,10 +332,18 @@ def test_switch_owner(get_module):
331332
a = get_module.get_array()
332333
assert np.core.multiarray.get_handler_name(a) is None
333334
get_module.set_own(a)
334-
with warnings.catch_warnings():
335-
warnings.filterwarnings('always')
336-
# The policy should be NULL, so we have to assume we can call "free"
337-
with assert_warns(RuntimeWarning) as w:
338-
del a
339-
gc.collect()
340-
print(w)
335+
oldval = os.environ.get('NUMPY_WARN_IF_NO_MEM_POLICY', None)
336+
os.environ['NUMPY_WARN_IF_NO_MEM_POLICY'] = "1"
337+
try:
338+
with warnings.catch_warnings():
339+
warnings.filterwarnings('always')
340+
# The policy should be NULL, so we have to assume we can call
341+
# "free"
342+
with assert_warns(RuntimeWarning) as w:
343+
del a
344+
gc.collect()
345+
finally:
346+
if oldval is None:
347+
os.environ.pop('NUMPY_WARN_IF_NO_MEM_POLICY')
348+
else:
349+
os.environ['NUMPY_WARN_IF_NO_MEM_POLICY'] = oldval

0 commit comments

Comments
 (0)
0