Do not warn when invalid values are masked in a MaskedArray #4959
Comments
|
This could probably be fixed using a combination of On Tue, Aug 12, 2014 at 8:22 PM, gerritholl notifications@github.com
Nathaniel J. Smith |
|
A little experimentation showed that np.ma.log10() worked on the example above. Is the intention for the default math functions to call the masked array function? In a closely related issue, supplying nan values in masked data causes an error in the safe domain calculations. Here the array's The test system is Debian 7.8 with the latest numpy 1.9.2 installed in ~/.local Test codeimport sys
import numpy as np
print(sys.version)
print("np.__version__ {0}".format(np.__version__))
A = np.arange(-2, 5)/2
Am = np.ma.masked_less_equal(A, 0)
print('np.ma.log10(Am) works OK.')
tm = np.ma.log10(Am)
print(tm.data)
print(tm.mask)
print('np.log10(Am) raises warning')
t = np.log10(Am)
print(t.data)
print(t.mask)
# Now my similar issue
np.seterr(all='raise') # provides full traceback
x = np.ma.array([1.0, np.nan, 3.0], mask=[False, True, False])
print(x)
print(1/x) # generates an errorOutputNotesI had a quick look at passing the mask (OR of the input values masks) to the domain function and only testing non-masked values. Something like for line 790 ret = np.oneslike(m, dtype=np.bool_)
ret[~m] = umath.absolute(a[~m]) * self.tolerance >= umath.absolute(b[~m])
return ret |
|
Just to summarize, there are two issues. One is that >>> import numpy as np
>>> np.log(np.ma.array([1, -1], mask=[0, 1]))
__main__:1: RuntimeWarning: invalid value encountered in log
masked_array(data = [0.0 --],
mask = [False True],
fill_value = 1e+20)
>>> np.ma.log(np.ma.array([1, np.nan], mask=[0, 1]))
/numpy/ma/core.py:805: RuntimeWarning: invalid value encountered in less_equal
return umath.less_equal(x, self.critical_value)
masked_array(data = [0.0 --],
mask = [False True],
fill_value = 1e+20) |
|
On the second issue, it is not just np.ma.log10 but a number of other functions that don't handle np.nan values in masked elements. |
|
This issue still exist for Numpy version 1.11.2 |
|
The problem also appears in simple comparisons: |
|
How would I do the following: This currently, creates a I love numpy and think masked arrays are a great thing to have, but not having this fixed makes them quite useless in many cases. More than happy to help, if anybody provides me with some pointers. Edit Lines 4045 to 4061 in efa7bad Looks like a fix has been implemented for structured arrays, maybe it is enough to default to the if statement?
Edit 2 Edit 3 |
|
@cosama: please open a new issue, that seems unrelated |
|
@eric-wieser. I can do that, but if I change Edit: I opened a new issue concerning the comparision operations here: #15978. So that the discussion here can focus on the |
|
I've been running into this warning when passing values to a log-scaled axis in Matplotlib, where I've masked the bad values before passing in, so I expect everything to be OK. The problem is that @efiring, I started using your simple example, but it is actually fixed in master as a side-effect of #15230. I think the action item would be that if a ufunc gets a MaskedArray as input, that it dispatches over to the Masked Implementation of that ufunc if it exists. I'm not sure if that was an explicit design decision or not though. |
|
It looks like def __array_ufunc__(self, ufunc, method, *inputs, **kwargs):
HANDLED_FUNCTIONS = {'log10': log10}
func = HANDLED_FUNCTIONS.get(ufunc.__name__, None)
if func:
return getattr(func, method)(*inputs, **kwargs)
return NotImplementedIs this something that is worth implementing on the I tried to see if anyone had worked on this before, but it doesn't look like it. This idea was brought up in #15200 suggesting this may be a good thing to do. |
|
@greglucas that sounds great, a PR would be nice.
Yes, I agree with those comments. |
|
Reopening, since the PR to use |
When doing a mathematical operation on a masked array, and some elements are invalid for the particular mathematical operation, numpy should not issue a warning if all invalid elements are masked.
This cannot be handled by
numpy.seterr: I want numpy to warn or raise when performing an invalid operation on any non-masked value, but to be silent when performing an invalid operation on a masked value.The text was updated successfully, but these errors were encountered: