-
-
Notifications
You must be signed in to change notification settings - Fork 4.2k
[Utils] Create exponential backoff utility class #12264
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
06bdefc
to
11b5478
Compare
11b5478
to
6a8b01f
Compare
LocalStack Community integration with Pro 2 files ±0 2 suites ±0 1h 50m 24s ⏱️ - 2m 14s Results for commit f17c9eb. ± Comparison against base commit a79d200. This pull request skips 1 test.
♻️ This comment has been updated with latest results. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice implementation!
Utility in itself is easy to understand, it would be great to see a companion example PR with its real-world usage, to understand the reason for this addition and how it fits the solution (e.g. to understand why we write utility ourselves and don't use some ready solution).
@tiurin Thanks for the review!
I've linked a companion PR showing its usage 🙂
From what I've read, most large + well maintained external retry libraries provide us with retry function decorators (see https://tenacity.readthedocs.io/en/latest/ or https://github.com/litl/backoff) which would be far more difficult to integrate into our current implementations than the class-based approach. |
Thank you! ❤️
Indeed, in linked PR case of a loop with a complex exit condition a separate backoff mechanism is better suited! 👍 It's not too easy to wrap something with a simple retry there, nor needed probably. For the reference, we also have our own collection of
I do agree a separate class that you suggest is easier to adjust to different use cases as its only job is to keep track of numbers. 👍 |
Good reference here. However... those functions wont allow us much flexability and could cause blocking waits + shutdowns since the Also, I prefer the utility-based approach since it makes no assumptions about where the backoff is going to be called from. That is, the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work @gregfurman! I just have one last validation comment, leaving it up to you to address in this or in comanion PR as you see fit. 👍
Motivation
This PR creates a re-usable component for calculating backoff durations. This was inspired by the exponential backoff algorithm from Google's HTTP Client Library for Java.
Changes
ExponentialBackoff
utility class whosenext_backoff
method calculates and returns the next duration that should be waited for.Usage
The below shows this class being used with a maximum of 3 attempts (with no random sampling between intervals):