8000 Messenger retry_strategy can lead to memory being exhausted · Issue #34096 · symfony/symfony · GitHub
[go: up one dir, main page]
Skip to content

Messenger retry_strategy can lead to memory being exhausted #34096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Jontsa opened this issue Oct 24, 2019 · 1 comment
Closed

Messenger retry_strategy can lead to memory being exhausted #34096

Jontsa opened this issue Oct 24, 2019 · 1 comment
Labels
Bug Messenger Status: Needs Review

Comments

@Jontsa
Copy link
Contributor
Jontsa commented Oct 24, 2019

Symfony version(s) affected: 4.3.5

Description
When using retry_strategy with Symfony messenger, if message handler keeps failing with an exception, the envelope gets filled with RedeliveryStamps. After enough re-tries, the PhpSerializer will start throwing "Allowed memory size of X bytes exhausted".

We are using messenger to ensure successful HTTP callback delivery. Notifications are queued and message handler sends the HTTP notification. If HTTP status code is not 200, Guzzle client will throw an exception, message is re-queued and tried again later. We encountered this issue when a network problem was blocking some notifications for a longer period of time.

How to reproduce

Example project is available here: https://github.com/Jontsa/messenger-retrystrategy-bug-example

You setup a symfony project with messenger and transport like doctrine. Create a message and message handler which will always throw exception. You can now queue one message and using messenger:consume -vvv see how the message size increases at each retry.

You can try and set a lower memory_limit:

$ /usr/bin/php -d memory_limit=10M bin/console messenger:consume -v --limit=100

 [OK] Consuming messages from transports "async".                                                                       

 // The worker will automatically exit once it has processed 100 messages or received a stop signal via the             
 // messenger:stop-workers command.                                                                                     

 // Quit the worker with CONTROL-C.                                                                                     

10:24:12 ERROR     [messenger] Error thrown while handling message App\NotificationMessage. Dispatching for retry #1 using 1000 ms delay. Error: "I failed, please requeue..." ["message" => App\NotificationMessage^ { …},"class" => "App\NotificationMessage","retryCount" => 1,"delay" => 1000,"error" => "I failed, please requeue...","exception" => Symfony\Component\Messenger\Exception\HandlerFailedException^ { …}]
10:24:13 ERROR     [messenger] Error thrown while handling message App\NotificationMessage. Dispatching for retry #2 using 1000 ms delay. Error: "I failed, please requeue..." ["message" => App\NotificationMessage^ { …},"class" => "App\NotificationMessage","retryCount" => 2,"delay" => 1000,"error" => "I failed, please requeue...","exception" => Symfony\Component\Messenger\Exception\HandlerFailedException^ { …}]
10:24:14 ERROR     [messenger] Error thrown while handling message App\NotificationMessage. Dispatching for retry #3 using 1000 ms delay. Error: "I failed, please requeue..." ["message" => App\NotificationMessage^ { …},"class" => "App\NotificationMessage","retryCount" => 3,"delay" => 1000,"error" => "I failed, please requeue...","exception" => Symfony\Component\Messenger\Exception\HandlerFailedException^ { …}]
PHP Fatal error:  Allowed memory size of 10485760 bytes exhausted (tried to allocate 20480 bytes) in /home/jontsa/Code/testing/messenger-retrystrategy-bug-example/vendor/symfony/messenger/Transport/Serialization/PhpSerializer.php on line 67
PHP Fatal error:  Allowed memory size of 10485760 bytes exhausted (tried to allocate 20480 bytes) in /home/jontsa/Code/testing/messenger-retrystrategy-bug-example/vendor/symfony/debug/ErrorHandler.php on line 620

Possible Solution

Additional context
@Tobion
Copy link
Contributor
Tobion commented Oct 24, 2019 •
edited
Loading

Thanks for the detailed report. It's already fixed by #34082

@Tobion Tobion closed this as completed Oct 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Messenger Status: Needs Review
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Tobion @xabbuh @carsonbot @Jontsa
0