8000 Unable to test equality of structured dtypes with no fields · Issue #13438 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

Unable to test equality of structured dtypes with no fields #13438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mcara opened this issue Apr 30, 2019 · 8 comments
Closed

Unable to test equality of structured dtypes with no fields #13438

mcara opened this issue Apr 30, 2019 · 8 comments
Labels
00 - Bug component: numpy.dtype

Comments

@mcara
Copy link
mcara commented Apr 30, 2019 •
edited by eric-wieser
Loading

If a structure type has no fields, equality throws

>>> a = np.zeros(10, dtype=[])
>>> a == a
ValueError: No fields found.

Original report:

Test for equality of empty masked arrays raises an exception.

Reproducing code example:

In [1]: import sys, numpy; print(numpy.__version__, sys.version)
1.16.3 3.6.1 (default, May 18 2017, 16:32:13) 
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)]

In [2]: a = numpy.ma.masked_array([], dtype=[], masked=False)  # or masked=True

In [3]: a == None

Error message:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-991e7ea0a64c> in <module>
----> 1 a == None

~/.../lib/python3.6/site-packages/numpy/ma/core.py in __eq__(self, other)
   4043         and other considered equal only if both were fully masked.
   4044         """
-> 4045         return self._comparison(other, operator.eq)
   4046 
   4047     def __ne__(self, other):

~/.../lib/python3.6/site-packages/numpy/ma/core.py in _comparison(self, other, compare)
   3996             # Now take care of the mask; the merged mask should have an item
   3997             # masked if all fields were masked (in one and/or other).
-> 3998             mask = (mask == np.ones((), mask.dtype))
   3999 
   4000         else:

ValueError: No fields found.

Numpy/Python version information:

1.16.3 3.6.1 (default, May 18 2017, 16:32:13)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)]
@mcara mcara mentioned this issue Apr 30, 2019
Merged
@seberg
Copy link
Member
seberg commented May 1, 2019

Uggh, should structured data types with no fields even be allowed?

In any case, the error is somewhat correct, at least not specific to masked arrays. While it does not happen for arrays, that is only because we replace the error with False currently (and we may at some point change that, its been long deprecated, but I think there have been opinions against actually going through with it).

Frankly, I am not sure we should take action here. Out of curiosity do you actually have a use case for this dtype?

@eric-wieser eric-wieser changed the title Unable to test equality of empty masked arrays Unable to test equality of structured dtypes with no fields May 1, 2019
@eric-wieser
Copy link
Member

Updated the top-post and title to reflect that masked arrays are irrelevant here.

I'd argue this should just return True, in the same way that () == () is True.

@seberg
Copy link
Member
seberg commented May 1, 2019

@eric-wieser yes, sure (or an array of True). What I am not sure about is, what the point of a no field structured type is. On the other hand, I guess there is also no real reason against just allowing it.

@mcara
Copy link
Author
mcara commented May 1, 2019

should structured data types with no fields even be allowed?

I honestly do not know, but currently you do and they crash, occasionally. This issue arose from astropy/astropy#8647 where I am trying to make Table.as_array() to return an empty array when a table is empty - for consistency purpose. When the table is masked (btw, sorry for carrying over masking from tables to the title of this issue), it creates an empty structured array. For this type of array testing for equality to something crashes.

I do not disagree with you that this is a rare situation but I think we should try to be consistent in how we handle edge cases. There are a couple of places in astropy or affiliated packages when a function may return an empty table (based on my memory - I may be wrong). Maybe we can do things differently if this is not a supported usage of arrays. But my first impression was that if an empty array is allowed to be created then why couldn't I use it for comparison.

@seberg
Copy link
Member
seberg commented May 1, 2019

Yeah, I was a bit misguided, there is definitely no point in not allowing this (especially going backward!), so the void comparison code really should just be capable of handling this.

@eric-wieser
Copy link
Member

or an array of True

This unfortunately is difficult to do correctly - not only does the shape of this array have to be correct, but it should probably inherit the right type through the array_ufunc mechanism.

I think this issue might be unfixable until void comparison is done via a ufunc.

@seberg
Copy link
Member
seberg commented Jun 9, 2019

Well, doing void comparison via a ufunc is very much doable though. And . while string ufuncs would be a bit annoying, a string comparison ufunc should be very much doable. Right now I would prefer not to put it on the agenda though probably. 1.17 is almost there, and I have some hope that we have some changes until 1.18... OTOH, the loops itself would still all work fine I suppose.

@eric-wieser eric-wieser mentioned this issue Aug 19, 2019
Merged
@eric-wieser eric-wieser mentioned this issue Sep 4, 2019
Merged
@mattip mattip closed this as completed in d6f7524 Sep 8, 2019
@mattip
Copy link
Member
mattip commented Sep 8, 2019

Confirm that #14393 closed this:

>>> a = np.zeros(10, dtype=[])
>>> a == a
array([ True,  True,  True,  True,  True,  True,  True,  True,  True,
        True])

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug component: numpy.dtype
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@seberg @eric-wieser @mattip @mcara
0