8000 REF: boilerplate for ops internal consistency by jbrockmendel · Pull Request #28037 · pandas-dev/pandas · GitHub
[go: up one dir, main page]

Skip to content

REF: boilerplate for ops internal consistency #28037

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 57 commits into from
Nov 14, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
1426061
OK
jbrockmendel Aug 9, 2019
c9e99f3
OK
jbrockmendel Aug 9, 2019
f109154
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 9, 2019
2462d9b
OK
jbrockmendel Aug 9, 2019
2289a85
OK
jbrockmendel Aug 10, 2019
68bab4b
OK
jbrockmendel Aug 10, 2019
7b00b02
OK
jbrockmendel Aug 12, 2019
c776dd7
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 12, 2019
c2e9fc6
cln
jbrockmendel Aug 12, 2019
afbd04a
remove validate_for_numeric_binop
jbrockmendel Aug 12, 2019
8af29a8
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 12, 2019
f3c9c20
OK
jbrockmendel Aug 12, 2019
2c72628
delete commented-out
jbrockmendel Aug 12, 2019
d19282d
OK
jbrockmendel Aug 12, 2019
7f337d2
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 12, 2019
242d7a0
cleanup
jbrockmendel Aug 12, 2019
fc498c0
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 12, 2019
9582d43
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 13, 2019
77cbdff
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 13, 2019
d3dafed
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 13, 2019
c539b2e
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 14, 2019
9d19cdb
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 15, 2019
e65ae3b
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 15, 2019
52c8ad3
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 15, 2019
2800a9b
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 16, 2019
d33ca40
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 16, 2019
2875b3a
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 19, 2019
f84e737
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 19, 2019
6149859
revert unnecessary
jbrockmendel Aug 19, 2019
05920a1
revert
jbrockmendel Aug 19, 2019
d6adfb0
docstring
jbrockmendel Aug 19, 2019
837d728
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 19, 2019
9eb8fbc
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 19, 2019
eb583c7
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Aug 20, 2019
9e30806
import fixup
jbrockmendel Aug 20, 2019
63d8f9c
blackify
jbrockmendel Aug 20, 2019
98f02b9
isort fixup
jbrockmendel Aug 20, 2019
f84258f
type ignore
jbrockmendel Aug 20, 2019
e3c07dd
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Sep 3, 2019
cc36fd7
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Sep 5, 2019
3d96ed6
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Sep 9, 2019
66fd7cc
add str type
jbrockmendel Sep 9, 2019
f994a3b
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Sep 15, 2019
96d4adb
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Sep 16, 2019
85fb1c1
lint fixup
jbrockmendel Sep 16, 2019
ddc056e
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Sep 18, 2019
6947bc0
blackify
jbrockmendel Sep 18, 2019
31e1ed2
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Sep 23, 2019
7e7064c
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Sep 26, 2019
69e349e
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Sep 26, 2019
23ceb47
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Sep 28, 2019
434686c
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Oct 6, 2019
7e98119
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Oct 6, 2019
d5caf5e
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Oct 8, 2019
109699e
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
jbrockmendel Oct 24, 2019
6136486
Merge branch 'master' of https://github.com/pandas-dev/pandas into opcom
8000 jbrockmendel Nov 14, 2019
bc2c0d5
rename unpack_and_defer -> unpack_zerodim_and_defer
jbrockmendel Nov 14, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
OK
  • Loading branch information
jbrockmendel committed Aug 10, 2019
commit 68bab4b8f57f1774a8b93f94554fb998366f0efe
32 changes: 14 additions & 18 deletions pandas/core/arrays/categorical.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

from pandas._config import get_option

from pandas._libs import algos as libalgos, hashtable as htable, lib
from pandas._libs import algos as libalgos, hashtable as htable
from pandas.compat.numpy import function as nv
from pandas.util._decorators import (
Appender,
Expand Down Expand Up @@ -38,7 +38,7 @@
is_timedelta64_dtype,
)
from pandas.core.dtypes.dtypes import CategoricalDtype
from pandas.core.dtypes.generic import ABCDataFrame, ABCIndexClass, ABCSeries
from pandas.core.dtypes.generic import ABCIndexClass, ABCSeries
from pandas.core.dtypes.inference import is_hashable
from pandas.core.dtypes.missing import isna, notna

Expand Down Expand Up @@ -78,22 +78,18 @@
)


def _cat_compare_op(op): # TODO: op-->opname
def _cat_compare_op(opname): # TODO: pass op instead of opname

@unpack_and_defer(op)
@unpack_and_defer(opname)
def f(self, other):
# On python2, you can usually compare any type to any type, and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not urgent but if you modify this again would be nice to clean up comment below

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea

# Categoricals can be seen as a custom type, but having different
# results depending whether categories are the same or not is kind of
# insane, so be a bit stricter here and use the python3 idea of
# comparing only things of equal type.
#if isinstance(other, (ABCDataFrame, ABCSeries, ABCIndexClass)):
# return NotImplemented

#other = lib.item_from_zerodim(other)

if not self.ordered:
if op in ["__lt__", "__gt__", "__le__", "__ge__"]:
if opname in ["__lt__", "__gt__", "__le__", "__ge__"]:
raise TypeError(
"Unordered Categoricals can only compare equality or not"
)
Expand All @@ -120,7 +116,7 @@ def f(self, other):
other_codes = other._codes

mask = (self._codes == -1) | (other_codes == -1)
f = getattr(self._codes, op)
f = getattr(self._codes, opname)
ret = f(other_codes)
if mask.any():
# In other series, the leads to False, so do that here too
Expand All @@ -130,38 +126,38 @@ def f(self, other):
if is_scalar(other):
if other in self.categories:
i = self.categories.get_loc(other)
ret = getattr(self._codes, op)(i)
ret = getattr(self._codes, opname)(i)

# check for NaN in self
mask = self._codes == -1
ret[mask] = False
return ret
else:
if op == "__eq__":
if opname == "__eq__":
return np.repeat(False, len(self))
elif op == "__ne__":
elif opname == "__ne__":
return np.repeat(True, len(self))
else:
msg = (
"Cannot compare a Categorical for op {op} with a "
"scalar, which is not a category."
)
raise TypeError(msg.format(op=op))
raise TypeError(msg.format(op=opname))
else:

# allow categorical vs object dtype array comparisons for equality
# these are only positional comparisons
if op in ["__eq__", "__ne__"]:
return getattr(np.array(self), op)(np.array(other))
if opname in ["__eq__", "__ne__"]:
return getattr(np.array(self), opname)(np.array(other))

msg = (
"Cannot compare a Categorical for op {op} with type {typ}."
"\nIf you want to compare values, use 'np.asarray(cat) "
"<op> other'."
)
raise TypeError(msg.format(op=op, typ=type(other)))
raise TypeError(msg.format(op=opname, typ=type(other)))

f.__name__ = op
f.__name__ = opname

return f

Expand Down
7 changes: 0 additions & 7 deletions pandas/core/arrays/datetimelike.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@
pandas_dtype,
)
from pandas.core.dtypes.generic import (
ABCDataFrame,
ABCIndexClass,
ABCPeriodArray,
ABCSeries,
Expand Down Expand Up @@ -1197,9 +1196,6 @@ def _time_shift(self, periods, freq=None):

@unpack_and_defer("__add__")
def __add__(self, other):
other = lib.item_from_zerodim(other)
#if isinstance(other, (ABCSeries, ABCDataFrame, ABCIndexClass)):
# return NotImplemented

# scalar others
if other is NaT:
Expand Down Expand Up @@ -1264,9 +1260,6 @@ def __radd__(self, other):

@unpack_and_defer("__sub__")
def __sub__(self, other):
other = lib.item_from_zerodim(other)
#if isinstance(other, (ABCSeries, ABCDataFrame, ABCIndexClass)):
# return NotImplemented

# scalar others
if other is NaT:
Expand Down
5 changes: 0 additions & 5 deletions pandas/core/arrays/datetimes.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@
)
from pandas.core.dtypes.dtypes import DatetimeTZDtype
from pandas.core.dtypes.generic import (
ABCDataFrame,
ABCIndexClass,
ABCPandasArray,
ABCSeries,
Expand Down Expand Up @@ -160,10 +159,6 @@ def _dt_array_cmp(cls, op):

@unpack_and_defer(opname)
def wrapper(self, other):
#if isinstance(other, (ABCDataFrame, ABCSeries, ABCIndexClass)):
# return NotImplemented

#other = lib.item_from_zerodim(other)

if isinstance(other, (datetime, np.datetime64, str)):
if isinstance(other, (datetime, np.datetime64)):
Expand Down
14 changes: 0 additions & 14 deletions pandas/core/arrays/integer.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@
is_scalar,
)
from pandas.core.dtypes.dtypes import register_extension_dtype
from pandas.core.dtypes.generic import ABCIndexClass, ABCSeries
from pandas.core.dtypes.missing import isna, notna

from pandas.core import nanops, ops
Expand Down Expand Up @@ -599,10 +598,6 @@ def _create_comparison_method(cls, op):
def cmp_method(self, other):
mask = None

#if isinstance(other, (ABCSeries, ABCIndexClass)):
# # Rely on pandas to unbox and dispatch to us.
# return NotImplemented

if isinstance(other, IntegerArray):
other, mask = other._data, other._mask

Expand All @@ -611,8 +606,6 @@ def cmp_method(self, other):
if other.ndim > 0 and len(self) != len(other):
raise ValueError("Lengths must match to compare")

#other = lib.item_from_zerodim(other)

# numpy will show a DeprecationWarning on invalid elementwise
# comparisons, this will raise in the future
with warnings.catch_warnings():
Expand Down Expand Up @@ -692,19 +685,12 @@ def integer_arithmetic_method(self, other):

mask = None

#if isinstance(other, (ABCSeries, ABCIndexClass)):
# # Rely on pandas to unbox and dispatch to us.
# return NotImplemented

if getattr(other, "ndim", 0) > 1:
raise NotImplementedError("can only perform ops with 1-d structures")

if isinstance(other, IntegerArray):
other, mask = other._data, other._mask

#elif getattr(other, "ndim", None) == 0:
# other = other.item()

elif is_list_like(other):
other = np.asarray(other)
if not other.ndim:
Expand Down
6 changes: 0 additions & 6 deletions pandas/core/arrays/period.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@

import numpy as np

from pandas._libs import lib
from pandas._libs.tslibs import (
NaT,
NaTType,
Expand Down Expand Up @@ -35,7 +34,6 @@
)
from pandas.core.dtypes.dtypes import PeriodDtype
from pandas.core.dtypes.generic import (
ABCDataFrame,
ABCIndexClass,
ABCPeriodArray,
ABCPeriodIndex,
Expand Down Expand Up @@ -74,10 +72,6 @@ def _period_array_cmp(cls, op):
def wrapper(self, other):
op = getattr(self.asi8, opname)

other = lib.item_from_zerodim(other)
#if isinstance(other, (ABCDataFrame, ABCSeries, ABCIndexClass)):
# return NotImplemented

if is_list_like(other) and len(other) != len(self):
raise ValueError("Lengths must match")

Expand Down
14 changes: 1 addition & 13 deletions pandas/core/arrays/sparse.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@
)
from pandas.core.dtypes.dtypes import register_extension_dtype
from pandas.core.dtypes.generic import (
ABCDataFrame,
ABCIndexClass,
ABCSeries,
ABCSparseArray,
Expand Down Expand Up @@ -1739,12 +1738,6 @@ def _create_arithmetic_method(cls, op):
@unpack_and_defer(op_name)
def sparse_arithmetic_method(self, other):

#if isinstance(other, (ABCDataFrame, ABCSeries, ABCIndexClass)):
# # Rely on pandas to dispatch to us.
# return NotImplemented

#other = lib.item_from_zerodim(other)

if isinstance(other, SparseArray):
return _sparse_array_op(self, other, op, op_name)

Expand Down Expand Up @@ -1791,14 +1784,9 @@ def _create_comparison_method(cls, op):
if op_name in {"and_", "or_"}:
op_name = op_name[:-1]

@unpack_and_defer(op_name)
def cmp_method(self, other):

if isinstance(other, (ABCDataFrame, ABCSeries, ABCIndexClass)):
# Rely on pandas to unbox and dispatch to us.
return NotImplemented

other = lib.item_from_zerodim(other)

if not is_scalar(other) and not isinstance(other, type(self)):
# convert list-like to ndarray
other = np.asarray(other)
Expand Down
3 changes: 0 additions & 3 deletions pandas/core/arrays/timedeltas.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,9 +83,6 @@ def _td_array_cmp(cls, op):

@unpack_and_defer(opname)
def wrapper(self, other):
other = lib.item_from_zerodim(other)
#if isinstance(other, (ABCDataFrame, ABCSeries, ABCIndexClass)):
# return NotImplemented

if _is_convertible_to_td(other) or other is NaT:
try:
Expand Down
6 changes: 1 addition & 5 deletions pandas/core/indexes/range.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
is_scalar,
is_timedelta64_dtype,
)
from pandas.core.dtypes.generic import ABCDataFrame, ABCSeries, ABCTimedeltaIndex
from pandas.core.dtypes.generic import ABCTimedeltaIndex

from pandas.core import ops
from pandas.core.ops.common import unpack_and_defer
Expand Down Expand Up @@ -733,8 +733,6 @@ def __getitem__(self, key):

@unpack_and_defer("__floordiv__")
def __floordiv__(self, other):
#if isinstance(other, (ABCSeries, ABCDataFrame)):
# return NotImplemented

if is_integer(other) and other != 0:
if len(self) == 0 or self.start % other == 0 and self.step % other == 0:
Expand Down Expand Up @@ -771,8 +769,6 @@ def _make_evaluate_binop(op, step=False):
"""
@unpack_and_defer(op.__name__)
def _evaluate_numeric_binop(self, other):
#if isinstance(other, (ABCSeries, ABCDataFrame)):
# return NotImplemented
if isinstance(other, ABCTimedeltaIndex):
# Defer to TimedeltaIndex implementation
return NotImplemented
Expand Down
15 changes: 0 additions & 15 deletions pandas/core/ops/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -680,12 +680,9 @@ def na_op(x, y):

@unpack_and_defer(op_name)
def wrapper(left, right):
#if isinstance(right, ABCDataFrame):
# return NotImplemented

left, right = _align_method_SERIES(left, right)
res_name = get_op_result_name(left, right)
right = lib.item_from_zerodim(right)
right = maybe_upcast_for_op(right, left.shape)

if is_categorical_dtype(left):
Expand Down Expand Up @@ -819,7 +816,6 @@ def wrapper(self, other, axis=None):
self._get_axis_number(axis)

res_name = get_op_result_name(self, other)
other = lib.item_from_zerodim(other)

# TODO: shouldn't we be applying finalize whenever
# not isinstance(other, ABCSeries)?
Expand All @@ -833,10 +829,6 @@ def wrapper(self, other, axis=None):
# TODO: same for tuples?
other = np.asarray(other)

#if isinstance(other, ABCDataFrame): # pragma: no cover
# # Defer to DataFrame implementation; fail early
# return NotImplemented

if isinstance(other, ABCSeries) and not self._indexed_same(other):
raise ValueError("Can only compare identically-labeled Series objects")

Expand Down Expand Up @@ -949,11 +941,6 @@ def wrapper(self, other):

self, other = _align_method_SERIES(self, other, align_asobject=True)
res_name = get_op_result_name(self, other)
other = lib.item_from_zerodim(other)

#if isinstance(other, ABCDataFrame):
# # Defer to DataFrame implementation; fail early
# return NotImplemented

if isinstance(other, (ABCSeries, ABCIndexClass)):
is_other_int_dtype = is_integer_dtype(other.dtype)
Expand Down Expand Up @@ -1287,8 +1274,6 @@ def _arith_method_SPARSE_SERIES(cls, op, special):

@unpack_and_defer(op_name)
def wrapper(self, other):
#if isinstance(other, ABCDataFrame):
# return NotImplemented
if isinstance(other, ABCSeries):
if not isinstance(other, ABCSparseSeries):
other = other.to_sparse(fill_value=self.fill_value)
Expand Down
0