8000 ENH: Add `__array_ufunc__` by charris · Pull Request #8247 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content

ENH: Add __array_ufunc__ #8247

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 43 commits into from
Apr 27, 2017
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
4fd7e84
ENH: Revert "Temporarily disable __numpy_ufunc__"
charris Nov 6, 2016
fcd11d2
ENH: Rename __numpy_ufunc__ to __array_ufunc__.
charris Nov 9, 2016
c7b25e2
ENH: Remove position arg from __array_ufunc__.
charris Nov 12, 2016
8a9e790
MAINT: Put PyArray_GetAttrString_SuppressException in get_attr_string.h
njsmith Jun 24, 2015
4dd5380
MAINT: dike out a bunch of weird old code implementing scalar power
njsmith Jun 24, 2015
7d9bc2f
BUG/ENH: Switch to simplified __array_ufunc__/binop interaction
njsmith Jun 22, 2015
e4b5163
MAINT: allow __array_ufunc__ = None to force binops to defer.
mhvk Mar 12, 2017
2e6d8c0
MAINT: Split out C code in ufunc_override.h to .c file.
mhvk Mar 15, 2017
d5c5ac1
MAINT: Add NPY_NO_EXPORT modifier to PyUFunc_CheckOverride.
charris Mar 16, 2017
3124e96
MAINT: for __array_ufunc__ pass inputs as *args, ensure out is tuple.
mhvk Mar 14, 2017
6a3ca31
DOC: describe current implementation of __array_ufunc__.
mhvk Mar 14, 2017
79bb733
DOC: Style and sphinx fixes for arrays.classes.rst.
charris Mar 23, 2017
7c3dc5a
TST: test that gufuncs are also overridden by __array_ufunc__.
mhvk Mar 25, 2017
71201d2
DOC: Describe __array_func__ in subclassing
mhvk Mar 15, 2017
3041710
ENH: implement ndarray.__array_ufunc__
mhvk Mar 13, 2017
5fe6fc6
DOC Update NEP to reflect actual implementation.
mhvk Mar 31, 2017
e092823
MAINT: let ndarray.__array_ufunc__ bail if any overrides are in place.
mhvk Apr 2, 2017
1147894
MAINT: Update array_ufunc NEP.
pv Apr 1, 2017
e325a10
DOC: Document behavior of ufuncs with default ndarray.__array_ufunc__
pv Apr 1, 2017
39c2273
DOC: Update ndarray.__array_ufunc__ documentation vs. review comments
pv Apr 2, 2017
6b41d11
DOC: clarify use of super and getattr
mhvk Apr 2, 2017
0ede0e9
DOC: update NEP again.
mhvk Apr 2, 2017
5f9252c
DOC: implement many smaller and bigger changes suggested in review.
mhvk Apr 4, 2017
8cc2f71
BUG,MAINT: ensure out=None is never passed on to __array_ufunc__.
mhvk Apr 4, 2017
856da73
DOC: remove left-over piece discussing binops
mhvk Apr 5, 2017
2b6c7fd
REVERT: remove __array_ufunc__ override for np.dot and ndarray.dot.
mhvk Apr 6, 2017
36e8494
REVERT: remove __array_ufunc__ override for np.matmul.
mhvk Apr 6, 2017
55500b9
MAINT: simplify now that __array_ufunc__ overrides ufuncs only.
mhvk Apr 6, 2017
25e973d
MAINT: split out umath-specific part of ufunc_override.
mhvk Apr 6, 2017
b1fa10a
BUG: ensure subclass of override class doesn't segfault.
mhvk Apr 8, 2017
1de8f5a
DOC: Mention `__array_ufunc__` in the 1.13.0 release notes.
charris Apr 8, 2017
a460015
DOC: ufunc-overrides: sync the discussion vs. current implementation
pv Apr 9, 2017
cd2e42c
DOC: ufunc-overrides: revise hierarchy discussion
pv Apr 9, 2017
ff628f1
BUG: Add back removed elision code.
charris Apr 9, 2017
1fc6e63
DOC,TST: clarify example of ndarray subclass using __array_ufunc__
mhvk Apr 10, 2017
a431743
BUG: Support nout == 0 and at method
eric-wieser Apr 12, 2017
1e460b7
DOC,MAINT: small corrections to NEP following Stephan's comments.
mhvk Apr 12, 2017
02600d3
ENH: Add NDArrayOperatorsMixin mixin class.
shoyer Apr 21, 2017
d3ff023
DOC: clarify recommendations for subclasses, deprecations.
mhvk Apr 21, 2017
b9359f1
MAINT: remove unnecessary checks, wrong code for 'outer', cleanup.
mhvk Apr 21, 2017
256a8ae
BUG: Fix ArrayLike(NDArrayOperatorsMixin) operations with object()
shoyer Apr 23, 2017
3272a86
ENH: Better error message for __array_ufunc__ not implemented
shoyer Apr 24, 2017
32221df
ENH: NDArrayOperatorsMixin calls ufuncs directly, like ndarray
shoyer Apr 27, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
ENH: Rename __numpy_ufunc__ to __array_ufunc__.
The first commit in changing __numpy_ufunc__ by removing the index
argument and making the out argument value always a tuple. These changes
were proposed in gh-5986 and have been accepted. Renaming before further
changes avoids triggering tests in scipy and astropy while keeping the
numpy tests working.
  • Loading branch information
charris committed Apr 27, 2017
commit fcd11d2b0e30a3eb1ca8d391ee965431c4a1fdfd
22 changes: 11 additions & 11 deletions doc/source/reference/arrays.classes.rst
< 8000 td id="diff-254be9cb812126fe17236cc0e90593b77ec3ff54ad493af7104fcfb8de5836c8L103" data-line-number="103" class="blob-num blob-num-deletion js-linkable-line-number">
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Special attributes and methods

NumPy provides several hooks that classes can customize:

.. method:: class.__numpy_ufunc__(ufunc, method, i, inputs, **kwargs)
.. method:: class.__array_ufunc__(ufunc, method, i, inputs, **kwargs)

.. versionadded:: 1.11
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is definitely not right :-)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Expand All @@ -62,51 +62,51 @@ NumPy provides several hooks that classes can customize:
:obj:`NotImplemented` if the operation requested is not
implemented.

If one of the arguments has a :func:`__numpy_ufunc__` method, it is
If one of the arguments has a :func:`__array_ufunc__` method, it is
executed *instead* of the ufunc. If more than one of the input
arguments implements :func:`__numpy_ufunc__`, they are tried in the
arguments implements :func:`__array_ufunc__`, they are tried in the
order: subclasses before superclasses, otherwise left to right. The
first routine returning something else than :obj:`NotImplemented`
determines the result. If all of the :func:`__numpy_ufunc__`
determines the result. If all of the :func:`__array_ufunc__`
operations return :obj:`NotImplemented`, a :exc:`TypeError` is
raised.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we only check for __array_ufunc__ on input arrays, not output arrays or where= arguments?


If an :class:`ndarray` subclass defines the :func:`__numpy_ufunc__`
If an :class:`ndarray` subclass defines the :func:`__array_ufunc__`
method, this disables the :func:`__array_wrap__`,
:func:`__array_prepare__`, :data:`__array_priority__` mechanism
described below.

.. note:: In addition to ufuncs, :func:`__numpy_ufunc__` also
.. note:: In addition to ufuncs, :func:`__array_ufunc__` also
overrides the behavior of :func:`numpy.dot` even though it is
not an Ufunc.

.. note:: If you also define right-hand binary operator override
methods (such as ``__rmul__``) or comparison operations (such as
``__gt__``) in your class, they take precedence over the
:func:`__numpy_ufunc__` mechanism when resolving results of
:func:`__array_ufunc__` mechanism when resolving results of
binary operations (such as ``ndarray_obj * your_obj``).

The technical special case is: ``ndarray.__mul__`` returns
``NotImplemented`` if the other object is *not* a subclass of
:class:`ndarray`, and defines both ``__numpy_ufunc__`` and
:class:`ndarray`, and defines both ``__array_ufunc__`` and
``__rmul__``. Similar exception applies for the other operations
than multiplication.

In such a case, when computing a binary operation such as
``ndarray_obj * your_obj``, your ``__numpy_ufunc__`` method
``ndarray_obj * your_obj``, your ``__array_ufunc__`` method
*will not* be called. Instead, the execution passes on to your
right-hand ``__rmul__`` operation, as per standard Python
operator override rules.

Similar special case applies to *in-place operations*: If you
define ``__rmul__``, then ``ndarray_obj *= your_obj`` *will not*
call your ``__numpy_ufunc__`` implementation. Instead, the
call your ``__array_ufunc__`` implementation. Instead, the
default Python behavior ``ndarray_obj = ndarray_obj * your_obj``
occurs.

Note that the above discussion applies only to Python's builtin
binary operation mechanism. ``np.multiply(ndarray_obj,
your_obj)`` always calls only your ``__numpy_ufunc__``, as
your_obj)`` always calls only your ``__array_ufunc__``, as
expected.

.. method:: class.__array_finalize__(obj)
Expand Down
2 changes: 1 addition & 1 deletion numpy/core/src/multiarray/cblasfuncs.c
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,7 @@ _bad_strides(PyArrayObject *ap)
* This is for use by PyArray_MatrixProduct2. It is assumed on entry that
* the arrays ap1 and ap2 have a common data type given by typenum that is
* float, double, cfloat, or cdouble and have dimension <= 2. The
* __numpy_ufunc__ nonsense is also assumed to have been taken care of.
* __array_ufunc__ nonsense is also assumed to have been taken care of.
*/
NPY_NO_EXPORT PyObject *
cblas_matrixproduct(int typenum, PyArrayObject *ap1, PyArrayObject *ap2,
Expand Down
2 changes: 1 addition & 1 deletion numpy/core/src/multiarray/multiarraymodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -4530,7 +4530,7 @@ intern_strings(void)
npy_ma_str_array_wrap = PyUString_InternFromString("__array_wrap__");
npy_ma_str_array_finalize = PyUString_InternFromString("__array_finalize__");
npy_ma_str_buffer = PyUString_InternFromString("__buffer__");
npy_ma_str_ufunc = PyUString_InternFromString("__numpy_ufunc__");
npy_ma_str_ufunc = PyUString_InternFromString("__array_ufunc__");
npy_ma_str_order = PyUString_InternFromString("order");
npy_ma_str_copy = PyUString_InternFromString("copy");
npy_ma_str_dtype = PyUString_InternFromString("dtype");
Expand Down
6 changes: 3 additions & 3 deletions numpy/core/src/multiarray/number.c
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ has_ufunc_attr(PyObject * obj) {
return 0;
}
else {
return PyObject_HasAttrString(obj, "__numpy_ufunc__");
return PyObject_HasAttrString(obj, "__array_ufunc__");
}
}

Expand All @@ -105,7 +105,7 @@ has_ufunc_attr(PyObject * obj) {
*
* This is the case when all of the following conditions apply:
*
* (i) the other object defines __numpy_ufunc__
* (i) the other object defines __array_ufunc__
* (ii) the other object defines the right-hand operation __r*__
* (iii) Python hasn't already called the right-hand operation
* [occurs if the other object is a strict subclass provided
Expand All @@ -118,7 +118,7 @@ has_ufunc_attr(PyObject * obj) {
* This is needed, because CPython does not call __rmul__ if
* the tp_number slots of the two objects are the same.
*
* This always prioritizes the __r*__ routines over __numpy_ufunc__, independent
* This always prioritizes the __r*__ routines over __array_ufunc__, independent
* of whether the other object is an ndarray subclass or not.
*/

Expand Down
14 changes: 7 additions & 7 deletions numpy/core/src/private/ufunc_override.h
Original file line number Diff line number Diff line change
Expand Up @@ -160,11 +160,11 @@ normalize_at_args(PyUFuncObject *ufunc, PyObject *args,
}

/*
* Check a set of args for the `__numpy_ufunc__` method. If more than one of
* the input arguments implements `__numpy_ufunc__`, they are tried in the
* Check a set of args for the `__array_ufunc__` method. If more than one of
* the input arguments implements `__array_ufunc__`, they are tried in the
* order: subclasses before superclasses, otherwise left to right. The first
* routine returning something other than `NotImplemented` determines the
* result. If all of the `__numpy_ufunc__` operations returns `NotImplemented`,
* result. If all of the `__array_ufunc__` operations returns `NotImplemented`,
* a `TypeError` is raised.
*
* Returns 0 on success and 1 on exception. On success, *result contains the
Expand Down Expand Up @@ -249,7 +249,7 @@ PyUFunc_CheckOverride(PyUFuncObject *ufunc, char *method,
_is_basic_python_type(obj)) {
continue;
}
if (PyObject_HasAttrString(obj, "__numpy_ufunc__")) {
if (PyObject_HasAttrString(obj, "__array_ufunc__")) {
with_override[noa] = obj;
with_override_pos[noa] = i;
++noa;
Expand Down Expand Up @@ -318,7 +318,7 @@ PyUFunc_CheckOverride(PyUFuncObject *ufunc, char *method,
}

/*
* Call __numpy_ufunc__ functions in correct order
* Call __array_ufunc__ functions in correct order
*/
while (1) {
PyObject *numpy_ufunc;
Expand Down Expand Up @@ -361,13 +361,13 @@ PyUFunc_CheckOverride(PyUFuncObject *ufunc, char *method,
if (!override_obj) {
/* No acceptable override found. */
PyErr_SetString(PyExc_TypeError,
"__numpy_ufunc__ not implemented for this type.");
"__array_ufunc__ not implemented for this type.");
goto fail;
}

/* Call the override */
numpy_ufunc = PyObject_GetAttrString(override_obj,
"__numpy_ufunc__");
"__array_ufunc__");
if (numpy_ufunc == NULL) {
goto fail;
}
Expand Down
2 changes: 1 addition & 1 deletion numpy/core/src/umath/umathmodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -266,7 +266,7 @@ intern_strings(void)
npy_um_str_array_prepare = PyUString_InternFromString("__array_prepare__");
npy_um_str_array_wrap = PyUString_InternFromString("__array_wrap__");
npy_um_str_array_finalize = PyUString_InternFromString("__array_finalize__");
npy_um_str_ufunc = PyUString_InternFromString("__numpy_ufunc__");
npy_um_str_ufunc = PyUString_InternFromString("__array_ufunc__");
npy_um_str_pyvals_name = PyUString_InternFromString(UFUNC_PYVALS_NAME);

return npy_um_str_out && npy_um_str_subok && npy_um_str_array_prepare &&
Expand Down
50 changes: 25 additions & 25 deletions numpy/core/tests/test_multiarray.py
Original file line number Diff line number Diff line change
Expand Up @@ -2408,11 +2408,11 @@ def test_dot(self):

def test_dot_override(self):
class A(object):
def __numpy_ufunc__(self, ufunc, method, pos, inputs, **kwargs):
def __array_ufunc__(self, ufunc, method, pos, inputs, **kwargs):
return "A"

class B(object):
def __numpy_ufunc__(self, ufunc, method, pos, inputs, **kwargs):
def __array_ufunc__(self, ufunc, method, pos, inputs, **kwargs):
return NotImplemented

a = A()
Expand Down Expand Up @@ -2885,7 +2885,7 @@ def test_elide_scalar(self):

def test_ufunc_override_rop_precedence(self):
# Check that __rmul__ and other right-hand operations have
# precedence over __numpy_ufunc__
# precedence over __array_ufunc__

ops = {
'__add__': ('__radd__', np.add, True),
Expand Down Expand Up @@ -2913,8 +2913,8 @@ class OtherNdarraySubclass(np.ndarray):
pass

class OtherNdarraySubclassWithOverride(np.ndarray):
def __numpy_ufunc__(self, *a, **kw):
raise AssertionError(("__numpy_ufunc__ %r %r shouldn't have "
def __array_ufunc__(self, *a, **kw):
raise AssertionError(("__array_ufunc__ %r %r shouldn't have "
"been called!") % (a, kw))

def check(op_name, ndsubclass):
Expand All @@ -2933,8 +2933,8 @@ def check(op_name, ndsubclass):
def __init__(self, *a, **kw):
pass

def __numpy_ufunc__(self, *a, **kw):
raise AssertionError(("__numpy_ufunc__ %r %r shouldn't have "
def __array_ufunc__(self, *a, **kw):
raise AssertionError(("__array_ufunc__ %r %r shouldn't have "
"been called!") % (a, kw))

def __op__(self, *other):
Expand All @@ -2949,7 +2949,7 @@ def __rop__(self, *other):
bases = (object,)

dct = {'__init__': __init__,
'__numpy_ufunc__': __numpy_ufunc__,
'__array_ufunc__': __array_ufunc__,
op_name: __op__}
if op_name != rop_name:
dct[rop_name] = __rop__
Expand Down Expand Up @@ -2989,7 +2989,7 @@ def __rop__(self, *other):
# integer-like?
assert_equal(iop(arr, obj), "rop", err_msg=err_msg)

# Check that ufunc call __numpy_ufunc__ normally
# Check that ufunc call __array_ufunc__ normally
if np_op is not None:
assert_raises(AssertionError, np_op, arr, obj,
err_msg=err_msg)
Expand All @@ -3005,7 +3005,7 @@ def test_ufunc_override_rop_simple(self):
# Check parts of the binary op overriding behavior in an
# explicit test case that is easier to understand.
class SomeClass(object):
def __numpy_ufunc__(self, *a, **kw):
def __array_ufunc__(self, *a, **kw):
return "ufunc"

def __mul__(self, other):
Expand All @@ -3024,7 +3024,7 @@ def __lt__(self, other):
return "nope"

class SomeClass2(SomeClass, np.ndarray):
def __numpy_ufunc__(self, ufunc, method, i, inputs, **kw):
def __array_ufunc__(self, ufunc, method, i, inputs, **kw):
if ufunc is np.multiply or ufunc is np.bitwise_and:
return "ufunc"
else:
Expand Down Expand Up @@ -3052,17 +3052,17 @@ def __rsub__(self, other):

# obj is first, so should get to define outcome.
assert_equal(obj * arr, 123)
# obj is second, but has __numpy_ufunc__ and defines __rmul__.
# obj is second, but has __array_ufunc__ and defines __rmul__.
assert_equal(arr * obj, 321)
# obj is second, but has __numpy_ufunc__ and defines __rsub__.
# obj is second, but has __array_ufunc__ and defines __rsub__.
assert_equal(arr - obj, "no subs for me")
# obj is second, but has __numpy_ufunc__ and defines __lt__.
# obj is second, but has __array_ufunc__ and defines __lt__.
assert_equal(arr > obj, "nope")
# obj is second, but has __numpy_ufunc__ and defines __gt__.
# obj is second, but has __array_ufunc__ and defines __gt__.
assert_equal(arr < obj, "yep")
# Called as a ufunc, obj.__numpy_ufunc__ is used.
# Called as a ufunc, obj.__array_ufunc__ is used.
assert_equal(np.multiply(arr, obj), "ufunc")
# obj is second, but has __numpy_ufunc__ and defines __rmul__.
# obj is second, but has __array_ufunc__ and defines __rmul__.
arr *= obj
assert_equal(arr, 321)

Expand All @@ -3072,7 +3072,7 @@ def __rsub__(self, other):
assert_equal(arr - obj2, "no subs for me")
assert_equal(arr > obj2, "nope")
assert_equal(arr < obj2, "yep")
# Called as a ufunc, obj2.__numpy_ufunc__ is called.
# Called as a ufunc, obj2.__array_ufunc__ is called.
assert_equal(np.multiply(arr, obj2), "ufunc")
# Also when the method is not overridden.
assert_equal(arr & obj2, "ufunc")
Expand All @@ -3093,13 +3093,13 @@ def __rsub__(self, other):
assert_equal(obj2 * obj3, 123)
# And of course, here obj3.__mul__ should be called.
assert_equal(obj3 * obj2, 123)
# obj3 defines __numpy_ufunc__ but obj3.__radd__ is obj2.__radd__.
# obj3 defines __array_ufunc__ but obj3.__radd__ is obj2.__radd__.
# (and both are just ndarray.__radd__); see #4815.
res = obj2 + obj3
assert_equal(res, 46)
assert_(isinstance(res, SomeClass2))
# Since obj3 is a subclass, it should have precedence, like CPython
# would give, even though obj2 has __numpy_ufunc__ and __radd__.
# would give, even though obj2 has __array_ufunc__ and __radd__.
# See gh-4815 and gh-5747.
res = obj3 + obj2
assert_equal(res, 46)
Expand All @@ -3108,7 +3108,7 @@ def __rsub__(self, other):
def test_ufunc_override_normalize_signature(self):
# gh-5674
class SomeClass(object):
def __numpy_ufunc__(self, ufunc, method, i, inputs, **kw):
def __array_ufunc__(self, ufunc, method, i, inputs, **kw):
return kw

a = SomeClass()
Expand All @@ -3125,7 +3125,7 @@ def test_numpy_ufunc_index(self):
# Check that index is set appropriately, also if only an output
# is passed on (latter is another regression tests for github bug 4753)
class CheckIndex(object):
def __numpy_ufunc__(self, ufunc, method, i, inputs, **kw):
def __array_ufunc__(self, ufunc, method, i, inputs, **kw):
return i

a = CheckIndex()
Expand Down Expand Up @@ -3161,7 +3161,7 @@ def __numpy_ufunc__(self, ufunc, method, i, inputs, **kw):
def test_out_override(self):
# regression test for github bug 4753
class OutClass(np.ndarray):
def __numpy_ufunc__(self, ufunc, method, i, inputs, **kw):
def __array_ufunc__(self, ufunc, method, i, inputs, **kw):
if 'out' in kw:
tmp_kw = kw.copy()
tmp_kw.pop('out')
Expand Down Expand Up @@ -5307,14 +5307,14 @@ class A(np.ndarray):
def __new__(cls, *args, **kwargs):
return np.array(*args, **kwargs).view(cls)

def __numpy_ufunc__(self, ufunc, method, pos, inputs, **kwargs):
def __array_ufunc__(self, ufunc, method, pos, inputs, **kwargs):
return "A"

class B(np.ndarray):
def __new__(cls, *args, **kwargs):
return np.array(*args, **kwargs).view(cls)

def __numpy_ufunc__(self, ufunc, method, pos, inputs, **kwargs):
def __array_ufunc__(self, ufunc, method, pos, inputs, **kwargs):
return NotImplemented

a = A([1, 2])
Expand Down
2 changes: 1 addition & 1 deletion numpy/core/tests/test_ufunc.py
Original file line number Diff line number Diff line change
Expand Up @@ -1225,7 +1225,7 @@ def test_structured_equal(self):
# https://github.com/numpy/numpy/issues/4855

class MyA(np.ndarray):
def __numpy_ufunc__(self, ufunc, method, i, inputs, **kwargs):
def __array_ufunc__(self, ufunc, method, i, inputs, **kwargs):
return getattr(ufunc, method)(*(input.view(np.ndarray)
for input in inputs), **kwargs)
a = np.arange(12.).reshape(4,3)
Expand Down
Loading
0