10000 support for scalars in `logical_(and|not|or|xor)` · Issue #58740 · pytorch/pytorch · GitHub
[go: up one dir, main page]
Skip to content

support for scalars in logical_(and|not|or|xor) #58740

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Tracked by #58743
pmeier opened this issue May 21, 2021 · 6 comments
Closed
Tracked by #58743

support for scalars in logical_(and|not|or|xor) #58740

pmeier opened this issue May 21, 2021 · 6 comments
Labels
triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@pmeier
Copy link
Collaborator
pmeier commented May 21, 2021 •
edited
Loading

The array API specification stipulates that the operators logical_(and|not|or|xor) should accept one scalar the same way arithmetic functions do.

PyTorch currently does not support that, although the magic methods seem to work just fine:

import torch

a = torch.tensor(True)
b = True

assert a & b
assert torch.logical_and(a, b)
tensor(True)
TypeError: logical_and(): argument 'other' (position 2) must be Tensor, not bool

cc @mruberry @rgommers @pmeier
@pmeier pmeier added the module: python array api Issues related to the Python Array API label May 21, 2021
@pmeier pmeier mentioned this issue May 21, 2021
Open
29 tasks
@zou3519 zou3519 added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label May 24, 2021
@mruberry
Copy link
Collaborator

We would accept a PR implementing this functionality.

@AnirudhDagar
Copy link
Contributor

This can be closed (or remove the module: python array api label) because the Array API spec doesn't really mention the support for scalar booleans (True, False). See logical_and, logical_not, logical_or and logical_xor for reference in the spec. (discussed offline with @rgommers)

Zeros should be considered the equivalent of False, while non-zeros should be considered the equivalent of True.

If this is the line that you are referring to, then it probably means that all elements in the tensor should follow this and not that a scalar should be accepted replacing an array/tensor.

Maybe @pmeier has more insight?

@pmeier
Copy link
Collaborator Author
pmeier commented Aug 25, 2021 •
edited
Loading

@AnirudhDagar @rgommers IIRC, I got this from here:

Mixing arrays with Python scalars

Using Python scalars (i.e., instances of bool , int , float) together with arrays must be supported for:

  • array <op> scalar

  • scalar <op> array

Plus, I think the array API test suite tests this, which is how I discovered it. Test suite seems fine now.

@rgommers
Copy link
Collaborator

<op> in that section is a Python builtin operator. There's normally an operator, a dunder method and a regular function in the API that all match (e.g., +, __add__, add).

For logical_and there's & and __and__ - but those are not the exact same I believe. For example, & on integer arrays does a bitwise thing while logical_and is only defined for boolean dtypes (may be coerced, or raise):

>>> x = np.array([True, False, True])
>>> x2 = np.arange(3)
>>> x & False
array([False, False, False])
>>> x & True
array([ True, False,  True])
>>> x2 & 0
array([0, 0, 0])
>>> x2 & 1
array([0, 1, 0])
>>> x2 & 2
array([0, 0, 2])
>>> x2 & 3
array([0, 1, 2])
>>> x2 & 4
array([0, 0, 0])

It's not impossible that's a NumPy bug, but I do have a vague memory about a decision that & and logical_and are not the same. Some digging in the history for both the array API standard and the NumPy issue tracker may confirm.

@AnirudhDagar
Copy link
Contributor
AnirudhDagar commented Aug 25, 2021 •
edited
Loading

This is from the NumPy docs here stating the difference between the two.

LOGICOPS: & or | in NumPy is bitwise AND/OR, while in MATLAB & and | are logical AND/OR. The two can appear to work the same, but there are important differences. If you would have used MATLAB’s & or | operators, you should use the NumPy ufuncs logical_and/logical_or. The notable differences between MATLAB’s and NumPy’s & and | operators are:

Non-logical {0,1} inputs: NumPy’s output is the bitwise AND of the inputs. MATLAB treats any non-zero value as 1 and returns the logical AND. For example (3 & 4) in NumPy is 0, while in MATLAB both 3 and 4 are considered logical true and (3 & 4) returns 1.

Precedence: NumPy’s & operator is higher precedence than logical operators like < and >; MATLAB’s is the reverse.

If you know you have boolean arguments, you can get away with using NumPy’s bitwise operators, but be careful with parentheses, like this: z = (x > 1) & (x < 2). The absence of NumPy operator forms of logical_and and logical_or is an unfortunate consequence of Python’s design.

Also, I tried to confirm and compare the behaviour for the recently merged array API PR into NumPy main branch.
It is nice that we can use that as a reference now. See here. The spec ingests and checks for boolean array inputs.

#### Extending @rgommers code snippet ####

>>> import numpy as np
>>> np.__version__
'1.22.0.dev0+930.ge778c0cf7'
>>> from numpy import array_api as xp
<stdin>:1: UserWarning: The numpy.array_api submodule is still experimental. See NEP 47.

#### Boolean Inputs Behaviour ####

# Array API
>>> x = xp.asarray([True, False, True])
>>> xp.logical_and(x, True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/gollum/Desktop/Work/numpy/numpy/array_api/_elementwise_functions.py", line 492, in logical_and
    if x1.dtype not in _boolean_dtypes or x2.dtype not in _boolean_dtypes:
AttributeError: 'bool' object has no attribute 'dtype'
>>> x & True
Array([ True, False,  True], dtype=bool)

### NumPy
>>> x = np.asarray([True, False, True])
>>> np.logical_and(x, True)
array([True, False, True])
>>> x & True
array([True, False, True])

#### Non-logical {0,1} Inputs Behaviour ####

### NumPy
>>> x2 = np.arange(3)
>>> x2 & 0
array([0, 0, 0])
>>> x2 & 1
array([0, 1, 0])
>>> x2 & 2
array([0, 0, 2])
>>> np.logical_and(x2, 0)
array([False, False, False])
>>> np.logical_and(x2, 1)
array([False,  True,  True])
>>> np.logical_and(x2, 2)
array([False,  True,  True])

### Array API
>>> x2 = xp.arange(3)
>>> x2 & 0
Array([0, 0, 0], dtype=int64)
>>> x2 & 1
Array([0, 1, 0], dtype=int64)
>>> x2 & 2
Array([0, 0, 2], dtype=int64)

### Raises when using scalars
>>> xp.logical_and(x2, 0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/gollum/Desktop/Work/numpy/numpy/array_api/_elementwise_functions.py", line 493, in logical_and
    raise TypeError("Only boolean dtypes are allowed in logical_and")
TypeError: Only boolean dtypes are allowed in logical_and
>>> xp.logical_and(x2, 1)  # same as above
>>> xp.logical_and(x2, 2)  # same as above

In conclusion

  • NumPy and Array API are divergent for logical_{and/or/not/xor}, since Array API (See here)
    • raises TypeError when passing non-boolean inputs.
    • AttributeError is encountered when scalars are passed.

but for NumPy, all of that works.

  • NumPy and Array API follow the same behaviour for &.
  • (MIGHT NOT BE RELEVANT, but) Since we are bringing it up, it is probably important to mention and which is a completely different story altogether. Unlike the above two, there is no single way to evaluate this hence it raises a value error when doing something like x2 and 2.

Notably, the behaviour in NumPy for logical_{and/or/not/xor}, assume zeros are False and all non-zeros are True which is similar to what is mentioned on the Array API spec page but in case of NumPy they will also work for scalar inputs.

@mruberry
Copy link
Collaborator

Thank you for the thorough review, @AnirudhDagar. We should also look at C++ and Python we we consider consistency, and in both C++ and Python a single ampersand is a bitwise and. Matlab just seems to be wrong about this.

@pmeier pmeier removed the module: python array api Issues related to the Python Array API label Jan 6, 2022
@pmeier pmeier closed this as completed Jan 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@rgommers @zou3519 @pmeier @AnirudhDagar @mruberry
0