8000 implement truediv, rtruediv directly in TimedeltaArray; tests by jbrockmendel · Pull Request #23829 · pandas-dev/pandas · GitHub
[go: up one dir, main page]

Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
6ec1f08
implement truediv, rtruediv directly in TimedeltaArray; tests
jbrockmendel Nov 21, 2018
4c2cc59
test tdi/tdi specifically
jbrockmendel Nov 21, 2018
bd2ee96
more checks and test cases
jbrockmendel Nov 21, 2018
3275dd9
Merge branch 'master' of https://github.com/pandas-dev/pandas into gt…
jbrockmendel Nov 21, 2018
79901f5
dont define _override_div_mod_methods, matches for pytest.raises
jbrockmendel Nov 21, 2018
adea273
change comment
jbrockmendel Nov 21, 2018
da9f743
whatsnew, GH references
jbrockmendel Nov 21, 2018
ba9e490
error msg py3 compat
jbrockmendel Nov 21, 2018
10bb49b
Merge branch 'master' of https://github.com/pandas-dev/pandas into gt…
jbrockmendel Nov 21, 2018
8f276ae
flake8 fixup, raise directly
jbrockmendel Nov 21, 2018
7d56da9
Merge branch 'master' of https://github.com/pandas-dev/pandas into gt…
jbrockmendel Nov 24, 2018
6097789
Merge branch 'master' of https://github.com/pandas-dev/pandas into gt…
jbrockmendel Nov 26, 2018
2037be8
sidestep object conversion
jbrockmendel Nov 26, 2018
ffedf35
Merge branch 'master' of https://github.com/pandas-dev/pandas into gt…
jbrockmendel Nov 27, 2018
2fc44aa
dont case result when operating against object dtype
jbrockmendel Nov 27, 2018
cd4ff57
Merge branch 'master' of https://github.com/pandas-dev/pandas into gt…
jbrockmendel Nov 28, 2018
641ad20
another GH reference
jbrockmendel Nov 28, 2018
e0d696f
Merge branch 'master' of https://github.com/pandas-dev/pandas into gt…
jbrockmendel Nov 28, 2018
7d9e677
comment
jbrockmendel Nov 28, 2018
dfc7af4
Merge branch 'master' of https://github.com/pandas-dev/pandas into gt…
jbrockmendel Nov 28, 2018
55cad6b
Fixup rebase mixup, un-skip part of a test that isnt broken after all
jbrockmendel Nov 29, 2018
d21ae78
Merge branch 'master' of https://github.com/pandas-dev/pandas into gt…
jbrockmendel Nov 29, 2018
d72bf90
flake8 fixup
jbrockmendel Nov 29, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
more checks and test cases
  • Loading branch information
jbrockmendel committed Nov 21, 2018
commit bd2ee969627a93ef32c8462ace74ff8539f24330
26 changes: 25 additions & 1 deletion pandas/core/arrays/timedeltas.py
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,7 @@ def _simple_new(cls, values, freq=None, dtype=_TD_DTYPE):
def __new__(cls, values, freq=None, dtype=_TD_DTYPE):

freq, freq_infer = dtl.maybe_infer_freq(freq)
values, inferred_freq = sequence_to_td64ns(values)

values = np.array(values, copy=False)
if values.dtype == np.object_:
Expand Down Expand Up @@ -328,7 +329,9 @@ def _evaluate_with_timedelta_like(self, other, op):
__rfloordiv__ = _wrap_tdi_op(ops.rfloordiv)

def __truediv__(self, other):
# TODO: should we unbox zero_dim?
# TODO: Decimals?
other = lib.item_from_zerodim(other)

if isinstance(other, (ABCSeries, ABCDataFrame, ABCIndexClass)):
return NotImplemented

Expand All @@ -352,6 +355,13 @@ def __truediv__(self, other):
freq = self.freq.delta / other
return type(self)(result, freq=freq)

if not hasattr(other, "dtype"):
# e.g. list, tuple
other = np.array(other)

if len(other) != len(self):
raise ValueError("Cannot divide vectors with unequal lengths")

elif is_timedelta64_dtype(other):
# let numpy handle it
return self._data / other
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to ensure other is a np.ndarray and not TimedeltaArray here? (meaning, extract the numpy array out of the TimedeltaArray)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. self._data.__div__(other) would return NotImplemented if other were a TimedeltaArray. This PR includes a test that covers TDA/TDA.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but that is another level of redirection, while we know this will happen and can directly do the correct thing here?
(I suppose is_timedelta64_dtype only passes through those two cases of ndarray or TimedeltaArray?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose is_timedelta64_dtype only passes through those two cases of ndarray or TimedeltaArray?

Yes, since we exclude Series and Index at the start.

Yes, but that is another level of redirection, while we know this will happen and can directly do the correct thing here?

I guess we could replace other with getattr(other, "_data", other) (or if/when TimedeltaArray gets an __array__ method, just np.array(other), which would be prettier)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the (hopefully not too distant future), TimedeltaArray will no longer be an Index. In this case we would want to explicitly grab the ._data out of it and proceed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what's the return type here? Does this need to be wrapped in a a type(self) so that we return a TimedeltaArray?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what's the return type here?

float-dtyped ndarray

Expand All @@ -371,6 +381,8 @@ def __truediv__(self, other):
return type(self)(result)

def __rtruediv__(self, other):
other = lib.item_from_zerodim(other)

if isinstance(other, (ABCSeries, ABCDataFrame, ABCIndexClass)):
return NotImplemented

Expand All @@ -385,6 +397,18 @@ def __rtruediv__(self, other):
# otherwise, dispatch to Timedelta implementation
return other / self._data

elif lib.is_scalar(other):
raise TypeError("Cannot divide {typ} by {cls}"
.format(typ=type(other).__name__,
cls=type(self).__name__))

if not hasattr(other, "dtype"):
# e.g. list, tuple
other = np.array(other)

if len(other) != len(self):
raise ValueError("Cannot divide vectors with unequal lengths")

elif is_timedelta64_dtype(other):
# let numpy handle it
return other / self._data
Expand Down
28 changes: 24 additions & 4 deletions pandas/tests/arithmetic/test_timedelta64.py
Original file line number Diff line number Diff line change
Expand Up @@ -1183,8 +1183,8 @@ def test_td64arr_div_tdlike_scalar_with_nat(self, two_hours,

def test_td64arr_div_td64_ndarray(self, box_with_array):
# GH#22631
rng = TimedeltaIndex(['1 days', pd.NaT, '2 days'], name='foo')
expected = pd.Float64Index([12, np.nan, 24], name='foo')
rng = TimedeltaIndex(['1 days', pd.NaT, '2 days'])
expected = pd.Float64Index([12, np.nan, 24])

rng = tm.box_expected(rng, box_with_array)
expected = tm.box_expected(expected, box_with_array)
Expand All @@ -1194,21 +1194,41 @@ def test_td64arr_div_td64_ndarray(self, box_with_array):
tm.assert_equal(result, expected)

result = rng / tm.box_expected(other, box_with_array)
tm.assert_equal(result, expected, check_names=False)
tm.assert_equal(result, expected)

result = rng / other.astype(object)
tm.assert_equal(result, expected)

result = rng / list(other)
tm.assert_equal(result, expected)

# reversed op
expected = 1 / expected
result = other / rng
tm.assert_equal(result, expected)

result = tm.box_expected(other, box_with_array) / rng
tm.assert_equal(result, expected, check_names=False)
tm.assert_equal(result, expected)

result = other.astype(object) / rng
tm.assert_equal(result, expected)

result = list(other) / rng
tm.assert_equal(result, expected)

def test_tdarr_div_length_mismatch(self, box_with_array):
rng = TimedeltaIndex(['1 days', pd.NaT, '2 days'])
mismatched = [1, 2, 3, 4]

rng = tm.box_expected(rng, box_with_array)
for obj in [mismatched, mismatched[:2]]:
# one shorter, one longer
for other in [obj, np.array(obj), pd.Index(obj)]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to parameterize this? Potentially via flags to indicate what transformation to perform on obj before testing for the error?

Definitely the outer-loop can be parameterized (just use [1, 2, 3, 4] and [1, 2]), unless there's a really strong reason for using mismatched and mismatched[:2].

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible, but I'm trying to push back a little bit against over-parametrization. The pytest setup cost is non-trivial

with pytest.raises(ValueError):
rng / other
with pytest.raises(ValueError):
other / rng

# ------------------------------------------------------------------
# __floordiv__, __rfloordiv__

Expand Down
0