You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We've had numerous issues about float32 precision bugs, but the inconsistency in comparisons found out in #9189 (comment) seems a bit much to bear: comparing 1 and something that is slightly larger but equal to 1 at float32 precision is admittedly not that well-defined, but it does seem simplest to just convert the float32 to float64 and do the comparison there. And this is done in some cases, but not all. It seems particularly troubling that array scalars and arrays with a shape are treated differently...
# not good: float32 array with float64 array scalar (or regular scalar or python scalar)np.array([1.], dtype=np.float32)>=np.array(1.00000001, dtype=np.float64)
# array([ True], dtype=bool)# OK float32 array with float64 arraynp.array([1.], dtype=np.float32)>=np.array([1.00000001], dtype=np.float64)
# array([False], dtype=bool)# also OK: (array) scalar with (array) scalarnp.array(1., dtype=np.float32)>=np.array(1.00000001, dtype=np.float64)
# False
The text was updated successfully, but these errors were encountered:
Uh oh!
There was an error while loading. Please reload this page.
We've had numerous issues about float32 precision bugs, but the inconsistency in comparisons found out in #9189 (comment) seems a bit much to bear: comparing 1 and something that is slightly larger but equal to 1 at float32 precision is admittedly not that well-defined, but it does seem simplest to just convert the float32 to float64 and do the comparison there. And this is done in some cases, but not all. It seems particularly troubling that array scalars and arrays with a shape are treated differently...
The text was updated successfully, but these errors were encountered: