8000 ENH: add isinf, isnan, fmin, fmax loops for datetime64, timedelta64 by mattip · Pull Request #14841 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

ENH: add isinf, isnan, fmin, fmax loops for datetime64, timedelta64 #14841

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Nov 22, 2019
Merged

ENH: add isinf, isnan, fmin, fmax loops for datetime64, timedelta64 #14841

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
9c1005d
ENH: add isinf, isnan for datetine64, timedelta64, adjust tests
mattip Nov 6, 2019
d593826
DOC: add release note
mattip Nov 6, 2019
259b6e3
MAINT: changes from review
mattip Nov 7, 2019
d6ad700
ENH: test, add fmin, fmax loops for datetime64, teimedelta64
mattip Nov 8, 2019
166895a
MAINT: improve message in assert_equal (from review)
mattip Nov 8, 2019
a7122fe
BUG, MAINT: fixes from review
mattip Nov 12, 2019
3483f80
BUG: timedelta64 also raises DeprecationWarning
mattip Nov 12, 2019
2e25313
DOC: fixes from review
mattip Nov 15, 2019
31c0971
Update numpy/testing/_private/utils.py
mattip Nov 21, 2019
0a056de
Update numpy/testing/_private/utils.py
mattip Nov 21, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions doc/release/upcoming_changes/14841.compatibility.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
Add more ufunc loops for ``datetime64``, ``timedelta64``
--------------------------------------------------------
``np.datetime('NaT')`` should behave more like ``float('Nan')``. Add needed
infrastructure so ``np.isinf(a)`` and ``np.isnan(a)`` will run on
``datetime64`` and ``timedelta64`` dtypes. Also added specific loops for
`numpy.fmin` and `numpy.fmax` that mask ``NaT``. This may require adjustment to user-
facing code. Specifically, code that either disallowed the calls to
`numpy.isinf` or `numpy.isnan` or checked that they raised an exception will
require adaptation, and code that mistakenly called `numpy.fmax` and
`numpy.fmin` instead of `numpy.maximum` or `numpy.minimum` respectively will
requre adjustment. This also affects `numpy.nanmax` and `numpy.nanmin`.
8 changes: 4 additions & 4 deletions numpy/core/code_generators/generate_umath.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -858,8 +858,8 @@ def english_upper(s):
'isnan':
Ufunc(1, 1, None,
docstrings.get('numpy.core.umath.isnan'),
None,
TD(nodatetime_or_obj, out='?'),
'PyUFunc_IsFiniteTypeResolver',
TD(noobj, out='?'),
),
'isnat':
Ufunc(1, 1, None,
Expand All @@ -870,8 +870,8 @@ def english_upper(s):
'isinf':
Ufunc(1, 1, None,
docstrings.get('numpy.core.umath.isinf'),
None,
TD(nodatetime_or_obj, out='?'),
'PyUFunc_IsFiniteTypeResolver',
TD(noobj, out='?'),
),
'isfinite':
Ufunc(1, 1, None,
Expand Down
29 changes: 29 additions & 0 deletions numpy/core/src/umath/loops.c.src
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1245,6 +1245,12 @@ NPY_NO_EXPORT void
}
}

NPY_NO_EXPORT void
@TYPE@_isinf(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func))
{
UNARY_LOOP_FAST(npy_bool, npy_bool, (void)in; *out = NPY_FALSE);
}

NPY_NO_EXPORT void
@TYPE@__ones_like(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data))
{
Expand Down Expand Up @@ -1306,6 +1312,29 @@ NPY_NO_EXPORT void
}
/**end repeat1**/

/**begin repeat1
* #kind = fmax, fmin#
* #OP = >=, <=#
**/
NPY_NO_EXPORT void
@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func))
{
BINARY_LOOP {
const @type@ in1 = *(@type@ *)ip1;
const @type@ in2 = *(@type@ *)ip2;
if (in1 == NPY_DATETIME_NAT) {
*((@type@ *)op1) = in2;
}
else if (in2 == NPY_DATETIME_NAT) {
*((@type@ *)op1) = in1;
}
else {
*((@type@ *)op1) = in1 @OP@ in2 ? in1 : in2;
}
}
}
/**end repeat1**/

/**end repeat**/

NPY_NO_EXPORT void
Expand Down
12 changes: 6 additions & 6 deletions numpy/core/src/umath/loops.h.src
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -477,6 +477,11 @@ NPY_NO_EXPORT void
NPY_NO_EXPORT void
@TYPE@_isfinite(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func));

NPY_NO_EXPORT void
@TYPE@_isinf(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func));

#define @TYPE@_isnan @TYPE@_isnat

NPY_NO_EXPORT void
@TYPE@__ones_like(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data));

Expand All @@ -489,8 +494,7 @@ NPY_NO_EXPORT void
/**end repeat1**/

/**begin repeat1
* #kind = maximum, minimum#
* #OP = >, <#
* #kind = maximum, minimum, fmin, fmax#
**/
NPY_NO_EXPORT void
@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func));
Expand Down Expand Up @@ -554,10 +558,6 @@ TIMEDELTA_mm_qm_divmod(char **args, npy_intp *dimensions, npy_intp *steps, void
#define TIMEDELTA_mq_m_floor_divide TIMEDELTA_mq_m_divide
#define TIMEDELTA_md_m_floor_divide TIMEDELTA_md_m_divide
/* #define TIMEDELTA_mm_d_floor_divide TIMEDELTA_mm_d_divide */
#define TIMEDELTA_fmin TIMEDELTA_minimum
#define TIMEDELTA_fmax TIMEDELTA_maximum
#define DATETIME_fmin DATETIME_minimum
#define DATETIME_fmax DATETIME_maximum

/*
*****************************************************************************
Expand Down
37 changes: 24 additions & 13 deletions numpy/core/tests/test_datetime.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1361,6 +1361,10 @@ def test_datetime_minmax(self):
assert_equal(np.minimum(dtnat, a), dtnat)
assert_equal(np.maximum(a, dtnat), dtnat)
assert_equal(np.maximum(dtnat, a), dtnat)
assert_equal(np.fmin(dtnat, a), a)
assert_equal(np.fmin(a, dtnat), a)
assert_equal(np.fmax(dtnat, a), a)
assert_equal(np.fmax(a, dtnat), a)

# Also do timedelta
a = np.array(3, dtype='m8[h]')
Expand Down Expand Up @@ -2232,26 +2236,33 @@ def test_isnat_error(self):
continue
assert_raises(TypeError, np.isnat, np.zeros(10, t))

def test_isfinite(self):
def test_isfinite_scalar(self):
assert_(not np.isfinite(np.datetime64('NaT', 'ms')))
assert_(not np.isfinite(np.datetime64('NaT', 'ns')))
assert_(np.isfinite(np.datetime64('2038-01-19T03:14:07')))

assert_(not np.isfinite(np.timedelta64('NaT', "ms")))
assert_(np.isfinite(np.timedelta64(34, "ms")))

res = np.array([True, True, False])
for unit in ['Y', 'M', 'W', 'D',
'h', 'm', 's', 'ms', 'us',
'ns', 'ps', 'fs', 'as']:
arr = np.array([123, -321, "NaT"], dtype='<datetime64[%s]' % unit)
assert_equal(np.isfinite(arr), res)
arr = np.array([123, -321, "NaT"], dtype='>datetime64[%s]' % unit)
assert_equal(np.isfinite(arr), res)
arr = np.array([123, -321, "NaT"], dtype='<timedelta64[%s]' % unit)
assert_equal(np.isfinite(arr), res)
arr = np.array([123, -321, "NaT"], dtype='>timedelta64[%s]' % unit)
assert_equal(np.isfinite(arr), res)
@pytest.mark.parametrize('unit', ['Y', 'M', 'W', 'D', 'h', 'm', 's', 'ms',
'us', 'ns', 'ps', 'fs', 'as'])
@pytest.mark.parametrize('dstr', ['<datetime64[%s]', '>datetime64[%s]',
'<timedelta64[%s]', '>timedelta64[%s]'])
def test_isfinite_isinf_isnan_units(self, unit, dstr):
'''check isfinite, isinf, isnan for all units of <M, >M, <m, >m dtypes
'''
arr_val = [123, -321, "NaT"]
arr = np.array(arr_val, dtype= dstr % unit)
pos = np.array([True, True, False])
neg = np.array([False, False, True])
false = np.array([False, False, False])
assert_equal(np.isfinite(arr), pos)
assert_equal(np.isinf(arr), false)
assert_equal(np.isnan(arr), neg)

def test_assert_equal(self):
assert_raises(AssertionError, assert_equal,
np.datetime64('nat'), np.timedelta64('nat'))

def test_corecursive_input(self):
# construct a co-recursive list
Expand Down
42 changes: 27 additions & 15 deletions numpy/testing/_private/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -374,21 +374,6 @@ def assert_equal(actual, desired, err_msg='', verbose=True):
if isscalar(desired) != isscalar(actual):
raise AssertionError(msg)

# Inf/nan/negative zero handling
try:
isdesnan = gisnan(desired)
isactnan = gisnan(actual)
if isdesnan and isactnan:
return # both nan, so equal

# handle signed zero specially for floats
if desired == 0 and actual == 0:
if not signbit(desired) == signbit(actual):
raise AssertionError(msg)

except (TypeError, ValueError, NotImplementedError):
pass

try:
isdesnat = isnat(desired)
isactnat = isnat(actual)
Expand All @@ -404,6 +389,33 @@ def assert_equal(actual, desired, err_msg='', verbose=True):
except (TypeError, ValueError, NotImplementedError):
pass

# Inf/nan/negative zero handling
try:
isdesnan = gisnan(desired)
isactnan = gisnan(actual)
if isdesnan and isactnan:
return # both nan, so equal

# handle signed zero specially for floats
array_actual = array(actual)
array_desired = array(desired)
if (array_actual.dtype.char in 'Mm' or
array_desired.dtype.char in 'Mm'):
Comment on lines +402 to +403
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You only want M here, not m

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't really matter, the test just below this is only for things that will have a signbit on a 0 value, so timedelta64 might as well never go to the next clause

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well at that point you may as well add integers here too - at any rate, the comment is saying "this won't work", not "this is a shortcut to save time".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, fixing

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, np.timedelta64(7, 's') == 0 emits a DeprecationWarning, so we need both 'Mm', and the comment is wrong

Copy link
Member
@eric-wieser eric-wieser Nov 15, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems weird that td[s] == td is deprecated yet td[s] + td is not - perhaps worth opening a new issue about

# version 1.18
# until this version, gisnan failed for datetime64 and timedelta64.
# Now it succeeds but comparison to scalar with a different type
# emits a DeprecationWarning.
# Avoid that by skipping the next check
raise NotImplementedError('cannot compare to a scalar '
'with a different type')

if desired == 0 and actual == 0:
if not signbit(desired) == signbit(actual):
raise AssertionError(msg)

except (TypeError, ValueError, NotImplementedError):
pass

try:
# Explicitly use __eq__ for comparison, gh-2552
if not (desired == actual):
Expand Down
0