8000 BUG: Construct Timestamp with tz correctly near DST border by mroeschke · Pull Request #21407 · pandas-dev/pandas · GitHub
[go: up one dir, main page]
Skip to content

BUG: Construct Timestamp with tz correctly near DST border #21407

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jun 13, 2018
Merged

BUG: Construct Timestamp with tz correctly near DST border #21407

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
c5566d5
BUG: Correctly localize Timestamp near DST
mroeschke Jun 9, 2018
5d207ea
fix #19970
mroeschke Jun 9, 2018
f4eb827
Add whatsnew and more comments
mroeschke Jun 10, 2018
c8dabe1
pep8
mroeschke Jun 10, 2018
795cc39
Use pydatetime_checkexact
mroeschke Jun 11, 2018
f171f9f
Merge remote-tracking branch 'upstream/master' into correct_normalize
mroeschke Jun 12, 2018
7e183a8
use astimezone instead of normalize
mroeschke Jun 12, 2018
3dd00d7
Merge remote-tracking branch 'upstream/master' into correct_normalize
mroeschke Jun 13, 2018
ae9dc7b
move whatsnew and simplify logic
mroeschke Jun 13, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
BUG: Correctly localize Timestamp near DST
  • Loading branch information
@mroeschke
mroeschke committed Jun 9, 2018
commit c5566d540799ab6d9b6b146e0584350d1e4fa46f
4 changes: 4 additions & 0 deletions pandas/_libs/tslibs/conversion.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -350,6 +350,10 @@ cdef _TSObject convert_datetime_to_tsobject(datetime ts, object tz,
# sort of a temporary hack
if ts.tzinfo is not None:
if hasattr(tz, 'normalize') and hasattr(ts.tzinfo, '_utcoffset'):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check might be superfluous since datetimes natively have the astimezone method. Then this whole block might be able to be simplified.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, you can try that and see if it works. maybe add a comment anyhow

# tz.localize does not correctly localize Timestamps near DST
if hasattr(ts, 'to_pydatetime'):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Elsewhere we've used not PyDatetimeCheckExact(ts) as a standin for isinstance(ts, pd.Timestamp). This might also be use case for the nanos kwarg.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jbrockmendel. Seems clearer than trying to get Timestamp to quack

nanos += ts.nanosecond
ts = ts.to_pydatetime()
ts = tz.normalize(ts)
obj.value = pydatetime_to_dt64(ts, &obj.dts)
obj.tzinfo = ts.tzinfo
Expand Down
9 changes: 9 additions & 0 deletions pandas/tests/indexes/datetimes/test_construction.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -469,6 +469,15 @@ def test_constructor_with_non_normalized_pytz(self, tz):
result = DatetimeIndex(['2010'], tz=non_norm_tz)
assert pytz.timezone(tz) is result.tz

def test_constructor_timestamp_near_dst(self):
# GH 20854
ts = [Timestamp('2016-10-30 03:00:00+0300', tz='Europe/Helsinki'),
Timestamp('2016-10-30 03:00:00+0200', tz='Europe/Helsinki')]
result = DatetimeIndex(ts)
expected = DatetimeIndex([ts[0].to_pydatetime(),
ts[1].to_pydatetime()])
tm.assert_index_equal(result, expected)


class TestTimeSeries(object):

Expand Down
14 changes: 14 additions & 0 deletions pandas/tests/indexes/datetimes/test_date_range.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,20 @@ def test_wom_len(self, periods):
res = date_range(start='20110101', periods=periods, freq='WOM-1MON')
assert len(res) == periods

def test_construct_over_dst(self):
# GH 20854
pre_dst = Timestamp('2010-11-07 01:00:00').tz_localize('US/Pacific',
ambiguous=True)
pst_dst = Timestamp('2010-11-07 01:00:00').tz_localize('US/Pacific',
ambiguous=False)
expect_data = [Timestamp('2010-11-07 00:00:00', tz='US/Pacific'),
pre_dst,
pst_dst]
expected = DatetimeIndex(expect_data)
result = date_range(start='2010-11-7', periods=3,
freq='H', tz='US/Pacific')
tm.assert_index_equal(result, expected)


class TestGenRangeGeneration(object):

Expand Down
8 changes: 8 additions & 0 deletions pandas/tests/scalar/timestamp/test_timestamp.py
View file
Open in desktop
4F37
Original file line number Diff line number Diff line change
Expand Up @@ -528,6 +528,14 @@ def test_disallow_setting_tz(self, tz):
with pytest.raises(AttributeError):
ts.tz = tz

@pytest.mark.parametrize('offset', ['+0300', '+0200'])
def test_construct_timestamp_near_dst(self, offset):
# GH 20854
expected = Timestamp('2016-10-30 03:00:00{}'.format(offset),
tz='Europe/Helsinki')
result = Timestamp(expected, tz='Europe/Helsinki')
assert result == expected


class TestTimestamp(object):

Expand Down
0