8000 BUG: Construct Timestamp with tz correctly near DST border by mroeschke · Pull Request #21407 · pandas-dev/pandas · GitHub
[go: up one dir, main page]

Skip to content

BUG: Construct Timestamp with tz correctly near DST border #21407

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jun 13, 2018
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add whatsnew and more comments
  • Loading branch information
mroeschke committed Jun 10, 2018
commit f4eb827790f356dec7a7e382ee0642126a2403ed
4 changes: 4 additions & 0 deletions doc/source/whatsnew/v0.23.1.txt
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,10 @@ Bug Fixes
- Bug in :func:`concat` where error was raised in concatenating :class:`Series` with numpy scalar and tuple names (:issue:`21015`)
- Bug in :func:`concat` warning message providing the wrong guidance for future behavior (:issue:`21101`)

**Timezones**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to 0.23.2

- Bug in :class:`Timestamp` and :class:`DatetimeIndex` where passing a :class:`Timestamp` localized after a DST transition would return a datetime before the DST transition (:issue:`20854`)
- Bug in comparing :class:`DataFrame`s with tz-aware :class:`DatetimeIndex` columns with a DST transition that raised a ``KeyError`` (:issue:`19970`)

**Other**

- Tab completion on :class:`Index` in IPython no longer outputs deprecation warnings (:issue:`21125`)
Expand Down
7 changes: 5 additions & 2 deletions pandas/_libs/tslibs/conversion.pyx
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# -*- coding: utf-8 -*-
# cython: profile=False

from warnings import catch_warnings

cimport cython
from cython cimport Py_ssize_t

Expand Down Expand Up @@ -347,13 +349,14 @@ cdef _TSObject convert_datetime_to_tsobject(datetime ts, object tz,
if tz is not None:
tz = maybe_get_tz(tz)

# sort of a temporary hack
if ts.tzinfo is not None:
if hasattr(tz, 'normalize') and hasattr(ts.tzinfo, '_utcoffset'):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check might be superfluous since datetimes natively have the astimezone method. Then this whole block might be able to be simplified.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, you can try that and see if it works. maybe add a comment anyhow

# tz.localize does not correctly localize Timestamps near DST
# but correctly localizes datetimes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC tz.localize (or possibly tz.normalize if thats relevant) operates by calling ts methods. Is this potentially something we can fix "higher up" in Timestamp?

if hasattr(ts, 'to_pydatetime'):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Elsewhere we've used not PyDatetimeCheckExact(ts) as a standin for isinstance(ts, pd.Timestamp). This might also be use case for the nanos kwarg.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jbrockmendel. Seems clearer than trying to get Timestamp to quack

nanos += ts.nanosecond
ts = ts.to_pydatetime()
with catch_warnings(record=True):
ts = ts.to_pydatetime()
ts = tz.normalize(ts)
obj.value = pydatetime_to_dt64(ts, &obj.dts)
obj.tzinfo = ts.tzinfo
Expand Down
0