8000 ENH: Add NDArrayOperatorsMixin by shoyer · Pull Request #19 · charris/numpy · GitHub
[go: up one dir, main page]

Skip to content

ENH: Add NDArrayOperatorsMixin #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 0 commits into from
Apr 21, 2017

Conversation

shoyer
Copy link
@shoyer shoyer commented Apr 12, 2017

This mixin class provides an easy way to implement arithmetic operators that defer to __array_ufunc__ like numpy.ndarray in non-ndarray subclasses.


Caveats: the rarely used `divmod` (``__divmod__``) and unary `+`
(``__pos__``) operators do not have corresponding ufuncs. Hence this mixin
passes the builtin functions ``divmod`` and ``operator.pos`` to

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ENH: add a np.divmod ufunc? Is there an issue for this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kwargs['out'] = tuple(x.value if isinstance(self, type(x)) else x
for x in out)
result = getattr(ufunc, method)(*inputs, **kwargs)
if isinstance(result, tuple):
Copy link
@eric-wieser eric-wieser Apr 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs an early exit path here for method == 'at', else we wrap None by accident.

Also, although ufunc.nout == 1 can't be used for divmod here, we can be a bit stricter by making this check be type(result) is tuple.

__rshift__, __rrshift__, __irshift__ = _numeric_methods(np.right_shift)
__and__, __rand__, __iand__ = _numeric_methods(np.logical_and)
__xor__, __rxor__, __ixor__ = _numeric_methods(np.logical_xor)
__or__, __ror__, __ior__ = _numeric_methods(np.logical_or)
Copy link
@eric-wieser eric-wieser Apr 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all super nice!

I do wonder whether we can do this base class in C though, so that this logic isn't duplicated inside the numpy C code.

@shoyer
Copy link
Author
shoyer commented Apr 12, 2017

At least one Travis-CI test is still failing on this branch:
https://travis-ci.org/shoyer/numpy/jobs/221427567


with warnings.catch_warnings():
# ignore warnings from operator.div when run with python -3
warnings.filterwarnings('ignore', 'classic int division',
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This didn't work:
https://travis-ci.org/shoyer/numpy/jobs/221437434
https://travis-ci.org/shoyer/numpy/jobs/221437437

Any other ideas for how to ignore these warnings on Python 2?

# one return value
return type(self)(result)
else:
# no return return value, e.g., ufunc.at
Copy link
@eric-wieser eric-wieser Apr 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The word return is repeated

This doesn't successfully distinguish between "the result is a scalar from an object array and it is None" and "this ufunc method never has a result". For instance, np.add.reduce([None]) returns None, but that should probably be wrapped for consistency with np.add.reduce([object()]).

I don't think there is any sensible implementation other than if method == 'at'

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it's worth handling this at all, given that this helper function currently only exists for the benefit of testing arithmetic. But I guess we'll probably be linking to this in the docs as recommended practices, so it's probably better to be on the safe side...

__add__, __radd__, __iadd__ = _numeric_methods(um.add)
__sub__, __rsub__, __isub__ = _numeric_methods(um.subtract)
__mul__, __rmul__, __imul__ = _numeric_methods(um.multiply)
__div__, __rdiv__, __idiv__ = _numeric_methods(um.divide) # Python 2 only
Copy link
@eric-wieser eric-wieser Apr 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be in a if sys.version_info.major < 2, or whatever name we give that in numpy? I don't think these methods have any place in code for 3

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -91,7 +91,7 @@ def check(result):
check(np.array(0) + ArrayLike(np.array(0)))

def test_divmod(self):
# divmod is subtle: it's returns a tuple
# divmod is subtle: its returns a tuple
Copy link
@eric-wieser eric-wieser Apr 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This correction is still wrong

@charris
Copy link
Owner
charris commented Apr 17, 2017

@shoyer What is the status of this?

@shoyer
Copy link
Author
shoyer commented Apr 17, 2017

Here's my TODO list:

  • figure out how to catch the warning for integer division -- see inline comments above Instead, I no longer test for div explicitly
  • remove the divmod and unary + implementations, given that we don't have ufuncs that can handle these yet
  • add a few tests for general ufunc overrides, i.e., methods other than '__call__' (even though they aren't used for binary operations)
  • add references to this mixin into the documentation

@shoyer shoyer changed the title WIP: NDArrayOperatorsMixin NDArrayOperatorsMixin Apr 20, 2017
@charris charris changed the title NDArrayOperatorsMixin ENH: Add NDArrayOperatorsMixin Apr 21, 2017
@charris charris merged this pull request into charris:enable-numpy_ufunc Apr 21, 2017
@charris
Copy link
Owner
charris commented Apr 21, 2017

Thanks @shoyer .

@shoyer
Copy link
Author
shoyer commented Apr 21, 2017

@charris I trust you'll squash this into a single commit before merging numpy#8247 :)

@shoyer
Copy link
Author
shoyer commented Apr 21, 2017

Nevermind, I see you already did that :)

@shoyer shoyer deleted the NDArrayOperatorsMixin branch April 21, 2017 16:43
except AttributeError:
pass
return self.__array_ufunc__(ufunc, '__call__', other, self)
return func

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice if func.__name__ could be set appropriately. So something like _reflected_binary_method(np.multiply, 'mul'), and internally it produces the name '__r{}__'.format('mul')

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mixin has been updated in the latest version, might want to review the current PR.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I will merge the current state in any case as the process is getting unwieldy at this point. You can make a new PR after that which should be easy to review and merge.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0