8000 Optimize `ffill`, `bfill` with dask when `limit` is specified by josephnowak · Pull Request #9771 · pydata/xarray · GitHub
[go: up one dir, main page]
Skip to content

Optimize ffill, bfill with dask when limit is specified #9771

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Nov 14, 2024
Merged

Optimize ffill, bfill with dask when limit is specified #9771

Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
f4fb252
Reduce the number of tasks when the limit parameter is set on the pus…
josephnowak Nov 12, 2024
40232dc
Reduce the number of tasks when the limit parameter is set on the pus…
josephnowak Nov 12, 2024
fde767b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 12, 2024
0d61d74
Update xarray/core/dask_array_ops.py
josephnowak Nov 12, 2024
b3d3307
Merge branch 'main' into feature/reduce-tasks-on-push-method-with-limit
josephnowak Nov 12, 2024
8d99dd0
Use last instead of creating a custom function, and add a keepdims pa…
josephnowak Nov 12, 2024
4fc115c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 12, 2024
297afdd
Remove the keepdims on the last and first method and use the nanlast …
josephnowak Nov 12, 2024
656c508
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 12, 2024
100369e
Include the optimization of ffill and bfill on the whats-new.rst
josephnowak Nov 13, 2024
b528a4b
Use map_overlap when the n is smaller than all the chunks
josephnowak Nov 14, 2024
6ae5e1f
Avoid creating a numpy array to check if all the chunks are bigger th…
josephnowak Nov 14, 2024
aad5ecb
Updating the whats-new.rst
josephnowak Nov 14, 2024
61d6574
Merge branch 'main' into feature/reduce-tasks-on-push-method-with-limit
dcherian Nov 14, 2024
c30ed51
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Updating the whats-new.rst
  • Loading branch information
@josephnowak
josephnowak committed Nov 14, 2024
commit aad5ecbbefbd742fb831a9bcbdaeb2634a379d72
6 changes: 4 additions & 2 deletions doc/whats-new.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,10 @@ New Features
- Support lazy grouping by dask arrays, and allow specifying ordered groups with ``UniqueGrouper(labels=["a", "b", "c"])``
(:issue:`2852`, :issue:`757`).
By `Deepak Cherian <https://github.com/dcherian>`_.
- Optimize ffill, bfill with dask when limit is specified and use the blelloch algorithm by default for both methods
(:pull:`9771`)
- Optimize ffill, bfill with dask when limit is specified
(:pull:`9771`).
By `Joseph Nowak <https://github.com/josephnowak>`_, and
`Patrick Hoefler <https://github.com/phofl>`.

Breaking changes
~~~~~~~~~~~~~~~~
Expand Down
0