@@ -486,23 +486,30 @@ Deploying to Production
486
486
487
487
On production, there are a few important things to think about:
488
488
489
- **Use Supervisor to keep your worker(s) running **
489
+ **Use a Process Manager like Supervisor or systemd to keep your worker(s) running **
490
490
You'll want one or more "workers" running at all times. To do that, use a
491
- process control system like :ref: `Supervisor <messenger-supervisor >`.
491
+ process control system like :ref: `Supervisor <messenger-supervisor >`
492
+ or :ref: `systemd <messenger-systemd >`.
492
493
493
494
**Don't Let Workers Run Forever **
494
495
Some services (like Doctrine's ``EntityManager ``) will consume more memory
495
496
over time. So, instead of allowing your worker to run forever, use a flag
496
497
like ``messenger:consume --limit=10 `` to tell your worker to only handle 10
497
- messages before exiting (then Supervisor will create a new process). There
498
+ messages before exiting (then the process manager will create a new process). There
498
499
are also other options like ``--memory-limit=128M `` and ``--time-limit=3600 ``.
499
500
501
+ **Stopping Workers That Encounter Errors **
502
+ If a worker dependency like your database server is down, or timeout is reached,
503
+ you can try to add :ref: `reconnect logic <middleware-doctrine >`, or just quit
504
+ the worker if it receives too many errors with the ``--failure-limit `` option of
505
+ the ``messenger:consume `` command.
506
+
500
507
**Restart Workers on Deploy **
501
508
Each time you deploy, you'll need to restart all your worker processes so
502
509
that they see the newly deployed code. To do this, run ``messenger:stop-workers ``
503
510
on deployment. This will signal to each worker that it should finish the message
504
- it's currently handling and should shut down gracefully. Then, Supervisor will create
505
- new worker processes. The command uses the :ref: `app <cache-configuration-with-frameworkbundle >`
511
+ it's currently handling and should shut down gracefully. Then, the process manager
512
+ will create new worker processes. The command uses the :ref: `app <cache-configuration-with-frameworkbundle >`
506
513
cache internally - so make sure this is configured to use an adapter you like.
507
514
508
515
**Use the Same Cache Between Deploys **
@@ -665,11 +672,25 @@ times:
665
672
startsecs =0
666
673
autostart =true
667
674
autorestart =true
675
+ startretries =10
668
676
process_name =%(program_name)s_%(process_num)02d
669
677
670
678
Change the ``async `` argument to use the name of your transport (or transports)
671
679
and ``user `` to the Unix user on your server.
672
680
681
+ .. caution ::
682
+
683
+ During a deployment, something might be unavailable (e.g. the
684
+ database) causing the consumer to fail to start. In this situation,
685
+ Supervisor will try ``startretries `` number of times to restart the
686
+ command. Make sure to change this setting to avoid getting the command
687
+ in a FATAL state, which will never restart again.
688
+
689
+ Each restart, Supervisor increases the delay by 1 second. For instance, if
690
+ the value is ``10 ``, it will wait 1 sec, 2 sec, 3 sec, etc. This gives the
691
+ service a total of 55 seconds to become available again. Increase the
692
+ ``startretries `` setting to cover the maximum expected downtime.
693
+
673
694
If you use the Redis Transport, note that each worker needs a unique consumer
674
695
name to avoid the same message being handled by multiple workers. One way to
675
696
achieve this is to set an environment variable in the Supervisor configuration
@@ -692,7 +713,7 @@ Next, tell Supervisor to read your config and start your workers:
692
713
See the `Supervisor docs `_ for more details.
693
714
694
715
Graceful Shutdown
695
- ~~~~~~~~~~~~~~~~~
716
+ .................
696
717
697
718
If you install the `PCNTL `_ PHP extension in your project, workers will handle
698
719
the ``SIGTERM `` POSIX signal to finish processing their current message before
@@ -708,6 +729,88 @@ of the desired grace period in seconds) in order to perform a graceful shutdown:
708
729
[program:x]
709
730
stopwaitsecs =20
710
731
732
+ .. _messenger-systemd :
733
+
734
+ Systemd Configuration
735
+ ~~~~~~~~~~~~~~~~~~~~~
736
+
737
+ While Supervisor is a great tool, it has the disadvantage that you need system
738
+ access to run it. Systemd has become the standard on most Linux distributions,
739
+ and has a good alternative called *user services *.
740
+
741
+ Systemd user service configuration files typically live in a ``~/.config/systemd/user ``
742
+ directory. For example, you can create a new ``messenger-worker.service `` file. Or a
743
+ ``messenger-worker@.service `` file if you want more instances running at the same time:
744
+
745
+ .. code-block :: ini
746
+
747
+ [Unit]
748
+ Description =Symfony messenger-consume %i
749
+
750
+ [Service]
751
+ ExecStart =php /path/to/your/app/bin/console messenger:consume async --time-limit =3600
752
+ Restart =always
753
+ RestartSec =30
754
+
755
+ [Install]
756
+ WantedBy =default.target
757
+
758
+ Now, tell systemd to enable and start one worker:
759
+
760
+ .. code-block :: terminal
761
+
E377
762
+ $ systemctl --user enable messenger-worker@1.service
763
+ $ systemctl --user start messenger-worker@1.service
764
+
765
+ # to enable and start 20 workers
766
+ $ systemctl --user enable messenger-worker@{1..20}.service
767
+ $ systemctl --user start messenger-worker@{1..20}.service
768
+
769
+ If you change your service config file, you need to reload the daemon:
770
+
771
+ .. code-block :: terminal
772
+
773
+ $ systemctl --user daemon-reload
774
+
775
+ To restart all your consumers:
776
+
777
+ .. code-block :: terminal
778
+
779
+ $ systemctl --user restart messenger-consume@*.service
780
+
781
+ The systemd user instance is only started after the first login of the
782
+ particular user. Consumer often need to start on system boot instead.
783
+ Enable lingering on the user to activate that behavior:
784
+
785
+ .. code-block :: terminal
786
+
787
+ $ loginctl enable-linger <your-username>
788
+
789
+ Logs are managed by journald and can be worked with using the journalctl
790
+ command:
791
+
792
+ .. code-block :: terminal
793
+
794
+ # follow logs of consumer nr 11
795
+ $ journalctl -f --user-unit messenger-consume@11.service
796
+
797
+ # follow logs of all consumers
798
+ $ journalctl -f --user-unit messenger-consume@*
799
+
800
+ # follow all logs from your user services
801
+ $ journalctl -f _UID=$UID
802
+
803
+ See the `systemd docs `_ for more details.
804
+
805
+ .. note ::
806
+
807
+ You either need elevated privileges for the ``journalctl `` command, or add
808
+ your user to the systemd-journal group:
809
+
810
+ .. code-block :: terminal
811
+
812
+ $ sudo usermod -a -G systemd-journal <your-username>
813
+
711
814
Stateless Worker
712
815
~~~~~~~~~~~~~~~~
713
816
@@ -2100,6 +2203,8 @@ middleware and *only* include your own:
2100
2203
If a middleware service is abstract, a different instance of the service will
2101
2204
be created per bus.
2102
2205
2206
+ .. _middleware-doctrine :
2207
+
2103
2208
Middleware for Doctrine
2104
2209
~~~~~~~~~~~~~~~~~~~~~~~
2105
2210
@@ -2283,6 +2388,7 @@ Learn more
2283
2388
.. _`streams` : https://redis.io/topics/streams-intro
2284
2389
.. _`Supervisor docs` : http://supervisord.org/
2285
2390
.. _`PCNTL` : https://www.php.net/manual/book.pcntl.php
2391
+ .. _`systemd docs` : https://www.freedesktop.org/wiki/Software/systemd/
2286
2392
.. _`SymfonyCasts' message serializer tutorial` : https://symfonycasts.com/screencast/messenger/transport-serializer
2287
2393
.. _`Long polling` : https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-short-and-long-polling.html
2288
2394
.. _`Visibility Timeout` : https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html
0 commit comments