10000 REF/TST: Fix remaining DatetimeArray with DateOffset arithmetic ops by jbrockmendel · Pull Request #23789 · pandas-dev/pandas · GitHub
[go: up one dir, main page]

Skip to content

REF/TST: Fix remaining DatetimeArray with DateOffset arithmetic ops #23789

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 25 commits into from
Nov 28, 2018
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
986fdbc
unrelated, change _window->libwindow
jbrockmendel Nov 19, 2018
fd75931
revert non-central
jbrockmendel Nov 19, 2018
66c866b
implement remaining methods to fix dateoffset arithmetic with DTA/TDA
jbrockmendel Nov 19, 2018
4dc17e2
xfail dataframe case
jbrockmendel Nov 19, 2018
da3459c
Merge branch 'master' of https://github.com/pandas-dev/pandas into et…
jbrockmendel Nov 20, 2018
348a8b2
Merge branch 'master' of https://github.com/pandas-dev/pandas into et…
jbrockmendel Nov 20, 2018
dd7e873
dont use verify_integrity, push one more level of test into parametrize
jbrockmendel Nov 20, 2018
c8351bc
Merge branch 'master' of https://github.com/pandas-dev/pandas into et…
jbrockmendel Nov 20, 2018
23a25d1
fix broken tests
jbrockmendel Nov 20, 2018
b4ae288
comment clarification
jbrockmendel Nov 22, 2018
d1ebdbf
Merge branch 'master' of https://github.com/pandas-dev/pandas into et…
jbrockmendel Nov 22, 2018
711ee61
dummy commit to force CI
jbrockmendel Nov 22, 2018
9338b5b
Merge branch 'master' of https://github.com/pandas-dev/pandas into et…
jbrockmendel Nov 25, 2018
5fbe9c8
comments about cached and non-cached implementations
jbrockmendel Nov 25, 2018
c7db0e4
Merge branch 'master' of https://github.com/pandas-dev/pandas into et…
jbrockmendel Nov 26, 2018
a4f9733
coverage
jbrockmendel Nov 26, 2018
b50fedf
special casing in freq_infer
jbrockmendel Nov 26, 2018
7e951e4
special casing followup
jbrockmendel Nov 26, 2018
317e1e7
fix simple_new args
jbrockmendel Nov 26, 2018
5de2d42
Merge branch 'master' of https://github.com/pandas-dev/pandas into et…
jbrockmendel Nov 27, 2018
5433a71
privatize
jbrockmendel Nov 27, 2018
dc137f3
Merge branch 'master' of https://github.com/pandas-dev/pandas into et…
jbrockmendel Nov 27, 2018
2c65f3b
more kludges
jbrockmendel Nov 27, 2018
31c5c0b
Merge branch 'master' of https://github.com/pandas-dev/pandas into et…
jbrockmendel Nov 27, 2018
c3d775e
typo fixup
jbrockmendel Nov 27, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
dont use verify_integrity, push one more level of test into parametrize
  • Loading branch information
jbrockmendel committed Nov 20, 2018
commit dd7e87396fef2a30c7d6018c0fd305c1bdd5e0a0
8 changes: 3 additions & 5 deletions pandas/core/arrays/timedeltas.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,12 +134,11 @@ def _simple_new(cls, values, freq=None, dtype=_TD_DTYPE):
return result

def __new__(cls, values, freq=None, dtype=_TD_DTYPE, copy=False):
verify_integrity = True

freq, freq_infer = dtl.maybe_infer_freq(freq)

values, inferred_freq = sequence_to_td64ns(values, copy=copy,
unit=None)
values, inferred_freq = sequence_to_td64ns(
values, copy=copy, unit=None)
if inferred_freq is not None:
if freq is not None and freq != inferred_freq:
raise ValueError('Inferred frequency {inferred} from passed '
Expand All @@ -150,11 +149,10 @@ def __new__(cls, values, freq=None, dtype=_TD_DTYPE, copy=False):
elif freq_infer:
freq = inferred_freq
freq_infer = False
verify_integrity = False

result = cls._simple_new(values, freq=freq)
# check that we are matching freqs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you could simplify a lot of these checks if also pass inferred_freq (optionally to ._validate_frequencey, and for example handle the fact that result could be a scalar (just ignore the validation) / sequence (do the validation)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ill take a look at this

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're right that we can change this from a 3-4 liner into a 1-2 liner. Since this pattern shows up in all four of TDA/DTA/TDI/DTI constructors (actually, future tense for DTA), I'd like to do change them all at once in a dedicated follow-up

if verify_integrity and len(result) > 0:
if inferred_freq is None and len(result) > 0:
if freq is not None and not freq_infer:
cls._validate_frequency(result, freq)

Expand Down
56 changes: 29 additions & 27 deletions pandas/tests/arithmetic/test_datetime64.py
Original file line number Diff line number Diff line change
Expand Up @@ -1125,10 +1125,25 @@ def test_dt64arr_add_sub_relativedelta_offsets(self, box_with_array):
'Easter', ('DateOffset', {'day': 4}),
('DateOffset', {'month': 5})])
@pytest.mark.parametrize('normalize', [True, False])
@pytest.mark.parametrize('n', [0, 5])
def test_dt64arr_add_sub_DateOffsets(self, box_with_array,
normalize, cls_and_kwargs):
n, normalize, cls_and_kwargs):
# GH#10699
# assert these are equal on a piecewise basis
# assert vectorized operation matches pointwise operations

if isinstance(cls_and_kwargs, tuple):
# If cls_name param is a tuple, then 2nd entry is kwargs for
# the offset constructor
cls_name, kwargs = cls_and_kwargs
else:
cls_name = cls_and_kwargs
kwargs = {}

if n == 0 and cls_name in ['WeekOfMonth', 'LastWeekOfMonth',
'FY5253Quarter', 'FY5253']:
# passing n = 0 is invalid for these offset classes
return

vec = DatetimeIndex([Timestamp('2000-01-05 00:15:00'),
Timestamp('2000-01-31 00:23:00'),
Timestamp('2000-01-01'),
Expand All @@ -1140,43 +1155,30 @@ def test_dt64arr_add_sub_DateOffsets(self, box_with_array,
vec = tm.box_expected(vec, box_with_array)
vec_items = vec.squeeze() if box_with_array is pd.DataFrame else vec

if isinstance(cls_and_kwargs, tuple):
# If cls_name param is a tuple, then 2nd entry is kwargs for
# the offset constructor
cls_name, kwargs = cls_and_kwargs
else:
cls_name = cls_and_kwargs
kwargs = {}

offset_cls = getattr(pd.offsets, cls_name)

with warnings.catch_warnings(record=True):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the record=True is needed? (as you are not checking the captured warnings)

# pandas.errors.PerformanceWarning: Non-vectorized DateOffset being
# applied to Series or DatetimeIndex
# we aren't testing that here, so ignore.
warnings.simplefilter("ignore", PerformanceWarning)
for n in [0, 5]:
if (cls_name in ['WeekOfMonth', 'LastWeekOfMonth',
'FY5253Quarter', 'FY5253'] and n == 0):
# passing n = 0 is invalid for these offset classes
continue

offset = offset_cls(n, normalize=normalize, **kwargs)
offset = offset_cls(n, normalize=normalize, **kwargs)

expected = DatetimeIndex([x + offset for x in vec_items])
expected = tm.box_expected(expected, box_with_array)
tm.assert_equal(expected, vec + offset)
expected = DatetimeIndex([x + offset for x in vec_items])
expected = tm.box_expected(expected, box_with_array)
tm.assert_equal(expected, vec + offset)

expected = DatetimeIndex([x - offset for x in vec_items])
expected = tm.box_expected(expected, box_with_array)
tm.assert_equal(expected, vec - offset)
expected = DatetimeIndex([x - offset for x in vec_items])
expected = tm.box_expected(expected, box_with_array)
tm.assert_equal(expected, vec - offset)

expected = DatetimeIndex([offset + x for x in vec_items])
expected = tm.box_expected(expected, box_with_array)
tm.assert_equal(expected, offset + vec)
expected = DatetimeIndex([offset + x for x in vec_items])
expected = tm.box_expected(expected, box_with_array)
tm.assert_equal(expected, offset + vec)

with pytest.raises(TypeError):
offset - vec
with pytest.raises(TypeError):
offset - vec

def test_dt64arr_add_sub_DateOffset(self, box_with_array):
# GH#10699
Expand Down
0