[ESM] Dynamically set SQS message override count #12233
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Currently, event hooks used to override the number of SQS messages to delete/receive use static parameters from the
SqsPoller
class to determine override values. Since boto clients in the same session can be re-used, this approach could possibly cause data races if multiple clients register against the same event source with differentBatchSize
parameters.Instead, this PR introduces event hooks that allow the override parameter to be dynamically passed in to each boto call (and injected into the headers) instead of trying to retrieve this information from the static ESM values.
Changes
SqsPoller.sqs_queue_parameters
.sqs_override_max_message_count
parameter toReceiveMessage
andDeleteMessageBatch
requests to allow overriding message count on a per-boto-request basis.