10000 PERF: Use is_utc check to improve performance of dateutil UTC in DatetimeIndex methods by mroeschke · Pull Request #23772 · pandas-dev/pandas · GitHub
[go: up one dir, main page]
Skip to content

PERF: Use is_utc check to improve performance of dateutil UTC in DatetimeIndex methods #23772

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 9 commits into from
Nov 18, 2018
Merged

PERF: Use is_utc check to improve performance of dateutil UTC in DatetimeIndex methods #23772

merged 9 commits into from
Nov 18, 2018

Conversation

mroeschke
Copy link
Member
  • passes git diff upstream/master -u -- "*.py" | flake8 --diff
  • whatsnew entry

Benchmarks from timeseries.py and timestamp.py

       before           after         ratio
     [91d1c503]       [9cc84951]
-      32.6±0.7μs       6.89±0.2μs     0.21  timestamp.TimestampOps.time_normalize(tzutc())

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.

       before           after         ratio
     [91d1c503]       [9cc84951]
-      4.72±0.1μs      4.26±0.06μs     0.90  timeseries.DatetimeIndex.time_get('dst')
-      5.09±0.4ms      2.79±0.02ms     0.55  timeseries.DatetimeAccessor.time_dt_accessor_normalize(tzutc())
-        297±20μs       42.9±0.2μs     0.14  timeseries.TzLocalize.time_infer_dst(tzutc())

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.

@pep8speaks
Copy link

Hello @mroeschke! Thanks for submitting the PR.

@jreback jreback added Performance Memory or execution speed performance Timezones Timezone data dtype labels Nov 18, 2018
@jreback jreback added this to the 0.24.0 milestone Nov 18, 2018
@jreback
Copy link
Contributor
jreback commented Nov 18, 2018

lgtm. I bet this actually fixed a bug where a dateutil UTC value was not treated correctly?

@jbrockmendel
Copy link
Member

Also datetime.timezones.utc in py3

@codecov
Copy link
codecov bot commented Nov 18, 2018 •
edited
Loading

Codecov Report

Merging #23772 into master will decrease coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #23772      +/-   ##
==========================================
- Coverage   92.24%   92.24%   -0.01%     
==========================================
  Files         161      161              
  Lines       51433    51431       -2     
==========================================
- Hits        47446    47444       -2     
  Misses       3987     3987
Flag Coverage Δ
#multiple 90.64% <100%> (-0.01%) ⬇️
#single 42.28% <14.28%> (-0.01%) ⬇️
Impacted Files Coverage Δ
pandas/core/indexes/datetimes.py 96.2% <100%> (-0.01%) ⬇️
pandas/core/arrays/datetimes.py 98.5% <100%> (-0.01%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 91d1c50...ca60770. Read the comment docs.

@jreback jreback merged commit 84cc0fb into pandas-dev:master Nov 18, 2018
@jreback
Copy link
Contributor
jreback commented Nov 18, 2018

thanks @mroeschke

as above, if you find some issues that now are closable because of the better checking (mine and @jbrockmendel comments). pls issue a PR with tests.

@mroeschke mroeschke deleted the dateutil_tz_performance branch November 18, 2018 23:27
thoo added a commit to thoo/pandas that referenced this pull request Nov 19, 2018
@thoo
Merge remote-tracking branch 'upstream/master' into io_csv_docstring_…
7732fbd 
…fixed

* upstream/master: (46 commits)
  DEPS: bump xlrd min version to 1.0.0 (pandas-dev#23774)
  BUG: Don't warn if default conflicts with dialect (pandas-dev#23775)
  BUG: Fixing memory leaks in read_csv (pandas-dev#23072)
  TST: Extend datetime64 arith tests to array classes, fix several broken cases (pandas-dev#23771)
  STYLE: Specify bare exceptions in pandas/tests (pandas-dev#23370)
  ENH: between_time, at_time accept axis parameter (pandas-dev#21799)
  PERF: Use is_utc check to improve performance of dateutil UTC in DatetimeIndex methods (pandas-dev#23772)
  CLN: io/formats/html.py: refactor (pandas-dev#22726)
  API: Make Categorical.searchsorted returns a scalar when supplied a scalar (pandas-dev#23466)
  TST: Add test case for GH14080 for overflow exception (pandas-dev#23762)
  BUG: Don't extract header names if none specified (pandas-dev#23703)
  BUG: Index.str.partition not nan-safe (pandas-dev#23558) (pandas-dev#23618)
  DEPR: tz_convert in the Timestamp constructor (pandas-dev#23621)
  PERF: Datetime/Timestamp.normalize for timezone naive datetimes (pandas-dev#23634)
  TST: Use new arithmetic fixtures, parametrize many more tests (pandas-dev#23757)
  REF/TST: Add more pytest idiom to parsers tests (pandas-dev#23761)
  DOC: Add ignore-deprecate argument to validate_docstrings.py (pandas-dev#23650)
  ENH: update pandas-gbq to 0.8.0, adds credentials arg (pandas-dev#23662)
  DOC: Improve error message to show correct order (pandas-dev#23652)
  ENH: Improve error message for empty object array (pandas-dev#23718)
  ...
Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019
@mroeschke @Pingviinituutti
PERF: Use is_utc check to improve performance of dateutil UTC in Date…
1591837 
…timeIndex methods (pandas-dev#23772)
Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019
@mroeschke @Pingviinituutti
PERF: Use is_utc check to improve performance of dateutil UTC in Date…
55ee9a0 
…timeIndex methods (pandas-dev#23772)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Performance Memory or execution speed performance Timezones Timezone data dtype
Projects
None yet
Milestone
0.24.0
Development

Successfully merging this pull request may close these issues.
4 participants
@mroeschke @pep8speaks @jreback @jbrockmendel
0