8000 BUG: _convert_and_box_cache raise ValueError: Tz-aware datetime.datetime cannot be converted to datetime64 unless utc=True by anmyachev · Pull Request #26097 · pandas-dev/pandas · GitHub
[go: up one dir, main page]
Skip to content

BUG: _convert_and_box_cache raise ValueError: Tz-aware datetime.datetime cannot be converted to datetime64 unless utc=True #26097

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Jul 3, 2019
Merged

BUG: _convert_and_box_cache raise ValueError: Tz-aware datetime.datetime cannot be converted to datetime64 unless utc=True #26097

Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
655ec31
fixed _convert_and_box_cache that raised ValueError: Tz-aware datetim…
anmyachev Apr 15, 2019
6a9856e
Revert bandaid workaround
vnlitvinov Apr 16, 2019
ffd9ecf
Add test that fails only when to_datetime gets cache=True
vnlitvinov Apr 16, 2019
d6c584e
Fix to_datetime caching logic so test_to_datetime_offset passes
vnlitvinov Apr 16, 2019
00f72e0
Fix flake8 issues
vnlitvinov Apr 16, 2019
5ad9911
removed debugging stuff; 'name' is default argument now
anmyachev Apr 17, 2019
428cae0
added 'whatsnew'
anmyachev Apr 17, 2019
7ed05f2
Test that to_datetime produces equal result for cache on and off
vnlitvinov Apr 18, 2019
b60f1d5
fixed isort errors
anmyachev Apr 18, 2019
3e2df79
Rework _box_if_needed into _box_as_indexlike
vnlitvinov Apr 19, 2019
3f0285e
Clarify added tests intention
vnlitvinov Apr 19, 2019
d19c2cf
first using notations
anmyachev Apr 22, 2019
b1cf140
changed wildcard import in '/pandas/core/index.py' to explicit import
anmyachev Apr 22, 2019
56db677
changed 'notations' -> 'annotations'
anmyachev Apr 22, 2019
4f9ea36
fixed isort errors
anmyachev Apr 22, 2019
c72a561
rollback of a certain style for annotations
anmyachev Apr 23, 2019
67a0c40
added 'Scalar' and 'DatetimeScalar' unions
anmyachev Apr 24, 2019
1e0d953
added annotations for some arguments; changed formatting
anmyachev Apr 26, 2019
1942bbe
using 'assert_almost_equal' now
anmyachev Apr 28, 2019
97e4548
rerun CI tests
anmyachev Jun 12, 2019
342d7d0
fixed problems found by review
anmyachev Jun 12, 2019
12f9853
replaced '# noqa' statement for flake8 linter
anmyachev Jun 14, 2019
62e75f8
removed 'Optional' using
anmyachev Jun 14, 2019
71ca9be
using TypeVar for 'DatetimeScalar' definition
anmyachev Jun 14, 2019
c35e124
fixed isort error
anmyachev Jun 14, 2019
d3412e2
fixed bug: 'UTC' and 'Etc/GMT' should be the same
anmyachev Jun 16, 2019
3141cb6
added comment about timezones
anmyachev Jun 16, 2019
ca200cd
removed 'UTC_EQ_STR', 'UTC_EQ'
anmyachev Jun 16, 2019
1cc469e
removing extra stuff
anmyachev Jul 2, 2019
137395f
fixed mypy errors
anmyachev Jul 2, 2019
2d8921b
renamed arg in '_box_as_indexlike' func: tz -> utc
anmyachev Jul 3, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
rollback of a certain style for annotations
  • Loading branch information
@anmyachev
anmyachev committed Jul 2, 2019
commit c72a561b04a09b861266c3b3d0cd264f9888ef57
22 changes: 11 additions & 11 deletions pandas/core/tools/datetimes.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,16 @@
ensure_object, is_datetime64_dtype, is_datetime64_ns_dtype,
is_datetime64tz_dtype, is_float, is_integer, is_integer_dtype,
is_list_like, is_numeric_dtype, is_scalar)
from pandas.core.dtypes.generic import ABCDataFrame, ABCIndexClass, ABCSeries
from pandas.core.dtypes.generic import (
ABCDataFrame, ABCDatetimeIndex, ABCIndex, ABCIndexClass, ABCSeries)
from pandas.core.dtypes.missing import notna

from pandas._typing import (
ArrayLike, IndexLike, IndexLikeOrNdarray, SeriesLike, Union)
from pandas._typing import ArrayLike, Union
from pandas.core import algorithms

# annotations
IntFltStrDateLisTuplArrSer = Union[int, float, str, list, tuple,
datetime, ArrayLike, SeriesLike]
DatetimeScalarOrArrayConvertible = Union[int, float, str, list, tuple,
datetime, ArrayLike, ABCSeries]


def _guess_datetime_format_for_array(arr, **kwargs):
Expand Down Expand Up @@ -66,7 +66,8 @@ def _maybe_cache(arg, format, cache, convert_listlike):
return cache_array


def _box_as_indexlike(dt_array: ArrayLike, tz=None, name=None) -> IndexLike:
def _box_as_indexlike(dt_array: ArrayLike,
tz=None, name=None) -> Union[ABCIndex, ABCDatetimeIndex]:
"""
Properly boxes the ndarray of datetimes to DatetimeIndex
if it is possible or to generic Index instead
Expand All @@ -92,9 +93,10 @@ def _box_as_indexlike(dt_array: ArrayLike, tz=None, name=None) -> IndexLike:
return Index(dt_array, name=name)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to apply tz to this result if Index(dt_array, name=name) returns a DatetimeIndex

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jreback @mroeschke After a close look at the Index constructor(pandas/core/indexes/base.py), I had a question about dtype default value. The documentation says that it should be object, but actually None is used. What case is correct?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its default is None, however it will attempt to infer the type of thing passed, so you may need to explicity pass dtype='object' to force it to object here.



def _convert_and_box_cache(arg: IntFltStrDateLisTuplArrSer,
cache_array: SeriesLike,
box: bool, name=None) -> IndexLikeOrNdarray:
def _convert_and_box_cache(arg: DatetimeScalarOrArrayConvertible,
cache_array: ABCSeries,
box: bool, name=None) -> Union[ABCIndex,
np.ndarray]:
"""
Convert array of dates with a cache and box the result

Expand All @@ -111,8 +113,6 @@ def _convert_and_box_cache(arg: IntFltStrDateLisTuplArrSer,
Returns
-------
result : datetime of converted dates
Returns:

- Index-like if box=True
- ndarray if box=False
"""
Expand Down
0