10000 [Messenger][4.3] Tracker for changes · Issue #11236 · symfony/symfony-docs · GitHub
[go: up one dir, main page]
Skip to content

[Messenger][4.3] Tracker for changes #11236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
25 tasks done
weaverryan opened this issue Mar 28, 2019 · 1 comment
Closed
25 tasks done

[Messenger][4.3] Tracker for changes #11236

weaverryan opened this issue Mar 28, 2019 · 1 comment
Labels
Messenger
Milestone
4.3

Comments

@weaverryan
Copy link
Member
weaverryan commented Mar 28, 2019 •
edited
Loading

A lot of new things have happened in Messenger for 4.3. This is a meta issue to try to track then:
@weaverryan weaverryan added this to the 4.3 milestone Mar 28, 2019
This was referenced Mar 28, 2019
Merged
Merged
Merged
Closed
Closed
symfony-splitter pushed a commit to symfony/messenger that referenced this issue Mar 30, 2019
@fabpot
feature #30759 [Messenger] Adding the "sync" transport to call handle…
affa4ba 
…rs synchronously (weaverryan)

This PR was merged into the 4.3-dev branch.

Discussion
----------

[Messenger] Adding the "sync" transport to call handlers synchronously

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | none
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

This adds a `sync://` transport that just calls the handlers immediately. Why? This allows you to route your messages to some "async" transport. But then, when developing locally or running your tests, you can choose to run them synchronously instead:

```yml
# config/packages/messenger.yaml
framework:
    messenger:
        transports:
            async: '%env(MESSENGER_TRANSPORT_DSN)%'
        routing:
            'App\Message\SmsNotification': async
            'App\Message\OtherMessage': async
```

```
# .env
# by default, handle this sync
MESSENGER_TRANSPORT_DSN=sync://
```

```
# .env.local on production (or set this via real env vars)
# on production, use amqp
MESSENGER_TRANSPORT_DSN=amqp://.......
```

Cheers!

Commits
-------

3da5a438aa Adding the "sync" transport to call handlers synchronously
fabpot added a commit to symfony/symfony that referenced this issue Mar 30, 2019
@fabpot
feature #30759 [Messenger] Adding the "sync" transport to call handle…
6ffad7b 
…rs synchronously (weaverryan)

This PR was merged into the 4.3-dev branch.

Discussion
----------

[Messenger] Adding the "sync" transport to call handlers synchronously

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | none
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

This adds a `sync://` transport that just calls the handlers immediately. Why? This allows you to route your messages to some "async" transport. But then, when developing locally or running your tests, you can choose to run them synchronously instead:

```yml
# config/packages/messenger.yaml
framework:
    messenger:
        transports:
            async: '%env(MESSENGER_TRANSPORT_DSN)%'
        routing:
            'App\Message\SmsNotification': async
            'App\Message\OtherMessage': async
```

```
# .env
# by default, handle this sync
MESSENGER_TRANSPORT_DSN=sync://
```

```
# .env.local on production (or set this via real env vars)
# on production, use amqp
MESSENGER_TRANSPORT_DSN=amqp://.......
```

Cheers!

Commits
-------

3da5a43 Adding the "sync" transport to call handlers synchronously
symfony-splitter pushed a commit to symfony/messenger that referenced this issue Mar 31, 2019
@fabpot
feature #30754 [Messenger] New messenger:stop-workers Command (weaver…
f8f3110 
…ryan)

This PR was squashed before being merged into the 4.3-dev branch (closes #30754).

Discussion
----------

[Messenger] New messenger:stop-workers Command

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | Kinda of #29451
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

o/ me again.

This requires and is built on top of #30708

When you deploy, all workers need to be stopped and restarted. That's not currently possible, unless you manually track the pids and send a SIGTERM signal. We can make that much easier :).

Now run:

```
bin/console messenger:stop-workers
```

And it will signal to all workers (even if they're distributed on other servers) that they should stop, once they finish processing their current message. This is done via a key in `cache.app`.

Cheers!

Commits
-------

58971627f5 [Messenger] New messenger:stop-workers Command
fabpot added a commit to symfony/symfony that referenced this issue Mar 31, 2019
@fabpot
feature #30754 [Messenger] New messenger:stop-workers Command (weaver…
343d28e 
…ryan)

This PR was squashed before being merged into the 4.3-dev branch (closes #30754).

Discussion
----------

[Messenger] New messenger:stop-workers Command

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | Kinda of #29451
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

o/ me again.

This requires and is built on top of #30708

When you deploy, all workers need to be stopped and restarted. That's not currently possible, unless you manually track the pids and send a SIGTERM signal. We can make that much easier :).

Now run:

```
bin/console messenger:stop-workers
```

And it will signal to all workers (even if they're distributed on other servers) that they should stop, once they finish processing their current message. This is done via a key in `cache.app`.

Cheers!

Commits
-------

5897162 [Messenger] New messenger:stop-workers Command
@javiereguiluz javiereguiluz mentioned this issue Apr 1, 2019
Merged
OskarStark added a commit to OskarStark/symfony-docs that referenced this issue Apr 1, 2019
@OskarStark
minor symfony#11262 [Messenger] Rename the messenger:consume-messages…
b4fcfe4 
… command (javiereguiluz)

This PR was merged into the master branch.

Discussion
----------

[Messenger] Rename the messenger:consume-messages command

This fixes one of the bullet points of symfony#11236.

Commits
-------

52a7157 [Messenger] Rename the messenger:consume-messages command
symfony-splitter pushed a commit to symfony/messenger that referenced this issue Apr 3, 2019
@fabpot
feature #30757 [Messenger] Adding MessageCountAwareInterface to get t…
3ad3acb 
…ransport message count (weaverryan)

This PR was squashed before being merged into the 4.3-dev branch (closes #30757).

Discussion
----------

[Messenger] Adding MessageCountAwareInterface to get transport message count

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | none
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

This adds a new optional interface that receivers should implement to give an approximate number of the messages "waiting" to be handled. Why? Because, with this, you could design a system that dynamically adds/removes worker processes if a specific transport is getting slammed and needs help. Creating that system could be something we discuss for core later, but this at least makes it possible - and means it could be implemented by the user or in a bundle... which I might do if we don't get it in core ;).

Commits
-------

fc5b0cf570 [Messenger] Adding MessageCountAwareInterface to get transport message count
fabpot added a commit to symfony/symfony that referenced this issue Apr 3, 2019
@fabpot
feature #30757 [Messenger] Adding MessageCountAwareInterface to get t…
574097f 
…ransport message count (weaverryan)

This PR was squashed before being merged into the 4.3-dev branch (closes #30757).

Discussion
----------

[Messenger] Adding MessageCountAwareInterface to get transport message count

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | none
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

This adds a new optional interface that receivers should implement to give an approximate number of the messages "waiting" to be handled. Why? Because, with this, you could design a system that dynamically adds/removes worker processes if a specific transport is getting slammed and needs help. Creating that system could be something we discuss for core later, but this at least makes it possible - and means it could be implemented by the user or in a bundle... which I might do if we don't get it in core ;).

Commits
-------

fc5b0cf [Messenger] Adding MessageCountAwareInterface to get transport message count
@G15N G15N mentioned this issue Apr 6, 2019
Merged
symfony-splitter pushed a commit to symfony/messenger that referenced this issue Apr 6, 2019
@sroze
feature #30008 [messenger] Adds a stamp to provide a routing key on m…
e24db2c 
…essage publishing (G15N, sroze)

This PR was merged into the 4.3-dev branch.

Discussion
----------

[messenger] Adds a stamp to provide a routing key on message publishing

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #29950
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

Adds a stamp allowing to set a `routing_key` at `MessageBus::dispatch()` level.

```php
$message = (new Envelope('message'))->with(new RoutingKeyStamp('routing_key'));
$bus->dispatch($message);
```

Commits
-------

a515635f18 Simply code and rename "configuration" to "options"
3151b54b7a [messenger] AMQP configurable routing key & multiple queues
sroze added a commit to symfony/symfony that referenced this issue Apr 6, 2019
@sroze
feature #30008 [messenger] Adds a stamp to provide a routing key on m…
e6eb43b 
…essage publishing (G15N, sroze)

This PR was merged into the 4.3-dev branch.

Discussion
----------

[messenger] Adds a stamp to provide a routing key on message publishing

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #29950
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

Adds a stamp allowing to set a `routing_key` at `MessageBus::dispatch()` level.

```php
$message = (new Envelope('message'))->with(new RoutingKeyStamp('routing_key'));
$bus->dispatch($message);
```

Commits
-------

a515635 Simply code and rename "configuration" to "options"
3151b54 [messenger] AMQP configurable routing key & multiple queues
@weaverryan weaverryan mentioned this issue Apr 7, 2019
Merged
@sroze sroze mentioned this issue Apr 8, 2019
Merged
This was referenced Apr 15, 2019
Closed
Closed
symfony-splitter pushed a commit to symfony/messenger that referenced this issue Apr 28, 2019
@sroze
feature #30958 [Messenger] Allows to register handlers on a specific …
f0b3421 
…transport (sroze)

This PR was merged into the 4.3-dev branch.

Discussion
----------

[Messenger] Allows to register handlers on a specific transport

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #30110
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

With the #30008 pull-request in, we can now do the following:
```yaml
framework:
    messenger:
        transports:
            events:
                dsn: amqp://guest:guest@127.0.0.1/%2f/events
                options:
                    queues:
                        3rdparty:
                            binding_keys: [ 3rdparty ]
                        projection:
                            binding_keys: [ projection ]

            events_3rdparty: amqp://guest:guest@127.0.0.1/%2f/events?queues[3rdparty]
            events_projection: amqp://guest:guest@127.0.0.1/%2f/events?queues[projection]

        routing:
            'App\Message\RegisterBet': events
```

This will bind two queues to the `events` exchange, fantastic:
<img width="325" alt="Screenshot 2019-04-07 at 10 26 27" src="https://user-images.githubusercontent.com/804625/55680861-af373580-591f-11e9-8f1e-2d3b6ddba2fd.png">

---

Now, in this setup, the message will be duplicated within the `3rdparty` & `projection` queues. If you just run the consumer for each transport, it will consume the message and call all the handlers. You can't do different things based on which queue you have consumed the message. **This pull-request adds the following feature:**

```php
class RegisterBetHandler implements MessageSubscriberInterface
{
    public function __invoke(RegisterBet $message)
    {
        // Do something only when the message comes from the `events_projection` transport...
    }

    /**
     * {@inheritdoc}
     */
    public static function getHandledMessages(): iterable
    {
        yield RegisterBet::class => [
            'from_transport' => 'events_projection',
        ];
    }
}
```

---

In the debugger, it looks like this:
<img width="649" alt="Screenshot 2019-04-07 at 10 29 55" src="https://user-images.githubusercontent.com/804625/55680892-1d7bf800-5920-11e9-80f7-853f0201c6d8.png">

Commits
-------

f0b2acd67d Allows to register handlers on a specific transport (and get rid of this handler alias)
sroze added a commit to symfony/symfony that referenced this issue Apr 28, 2019
@sroze
feature #30958 [Messenger] Allows to register handlers on a specific …
584c21b 
…transport (sroze)

This PR was merged into the 4.3-dev branch.

Discussion
----------

[Messenger] Allows to register handlers on a specific transport

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #30110
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

With the #30008 pull-request in, we can now do the following:
```yaml
framework:
    messenger:
        transports:
            events:
                dsn: amqp://guest:guest@127.0.0.1/%2f/events
                options:
                    queues:
                        3rdparty:
                            binding_keys: [ 3rdparty ]
                        projection:
                            binding_keys: [ projection ]

            events_3rdparty: amqp://guest:guest@127.0.0.1/%2f/events?queues[3rdparty]
            events_projection: amqp://guest:guest@127.0.0.1/%2f/events?queues[projection]

        routing:
            'App\Message\RegisterBet': events
```

This will bind two queues to the `events` exchange, fantastic:
<img width="325" alt="Screenshot 2019-04-07 at 10 26 27" src="https://user-images.githubusercontent.com/804625/55680861-af373580-591f-11e9-8f1e-2d3b6ddba2fd.png">

---

Now, in this setup, the message will be duplicated within the `3rdparty` & `projection` queues. If you just run the consumer for each transport, it will consume the message and call all the handlers. You can't do different things based on which queue you have consumed the message. **This pull-request adds the following feature:**

```php
class RegisterBetHandler implements MessageSubscriberInterface
{
    public function __invoke(RegisterBet $message)
    {
        // Do something only when the message comes from the `events_projection` transport...
    }

    /**
     * {@inheritdoc}
     */
    public static function getHandledMessages(): iterable
    {
        yield RegisterBet::class => [
            'from_transport' => 'events_projection',
        ];
    }
}
```

---

In the debugger, it looks like this:
<img width="649" alt="Screenshot 2019-04-07 at 10 29 55" src="https://user-images.githubusercontent.com/804625/55680892-1d7bf800-5920-11e9-80f7-853f0201c6d8.png">

Commits
-------

f0b2acd Allows to register handlers on a specific transport (and get rid of this handler alias)
fabpot added a commit to symfony/symfony that referenced this issue May 1, 2019
@fabpot
feature #30970 [Messenger] Adding failure transport support (weaverryan)
fdeb93e 
This PR was squashed before being merged into the 4.3-dev branch (closes #30970).

Discussion
----------

[Messenger] Adding failure transport support

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | yes
| New feature?  | yes
| BC breaks?    | yes
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #31231
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

This adds "failure" transport support for messenger, so that messages that fail on *all* their retries can be collected in one spot and retried later if wanted:

```yml
framework:
    messenger:
        failure_transport: failed

        transports:
            async:
                dsn: 'amqp://'
            failed:
                dsn: 'doctrine://default?queue_name=failed'

        routing:
            'App\Message\SmsNotification': async
```

In this setup, `SmsNotification` would be retried 3 times on the `async` transport (current behavior) and then finally sent to the `failed` transport. The `failed` transport can be consumed like a normal transport, but should usually be handled & consumed by one of the new commands:

**> bin/console messenger:failed:show**
<img width="861" alt="Screen Shot 2019-04-10 at 3 15 45 PM" src="https://user-images.githubusercontent.com/121003/55917329-ddc54280-5ba3-11e9-878c-af3c653643de.png">

**> bin/console messenger:failed:show 217**
<img width="804" alt="Screen Shot 2019-04-10 at 3 15 55 PM" src="https://user-images.githubusercontent.com/121003/55917360-f33a6c80-5ba3-11e9-9f12-a8c57a9a7a4b.png">

**> bin/console messenger:failed:purge 217**
<img width="835" alt="Screen Shot 2019-04-10 at 3 16 07 PM" src="https://user-images.githubusercontent.com/121003/55917383-ff262e80-5ba3-11e9-9720-e24176b834f7.png">

**> bin/console messenger:failed:retry 217**
<img width="737" alt="Screen Shot 2019-04-10 at 3 16 29 PM" src="https://user-images.githubusercontent.com/121003/55917396-09482d00-5ba4-11e9-8d51-0bbe2b4ffc14.png">

**> bin/console messenger:failed:retry 218 -vv**
<img width="1011" alt="Screen Shot 2019-04-10 at 3 20 39 PM" src="https://user-images.githubusercontent.com/121003/55917503-6512b600-5ba4-11e9-9365-4ac87d858541.png">

**Note** (This screenshot is ugly - need to make the dump of the message and the exception more attractive)

Or you can run `bin/console messenger:failed:retry` without any argument, and it will consume the failed messages one-by-one and ask you if you want to retry/handle each. By passing

Cheers!

Commits
-------

36487e5 [Messenger] Adding failure transport support
This was referenced May 6, 2019
Closed
Merged
@Tobion Tobion mentioned this issue May 10, 2019
Merged
fabpot added a commit to symfony/symfony that referenced this issue May 11, 2019
@fabpot
bug #31425 [Messenger] On failure retry, make message appear received…
7800396 
… from original sender (weaverryan)

This PR was squashed before being merged into the 4.3 branch (closes #31425).

Discussion
----------

[Messenger] On failure retry, make message appear received from original sender

| Q             | A
| ------------- | ---
| Branch?       | master (4.3)
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #31413
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

Fixes a bug when using the transport-based handler config from #30958. This also adds a pretty robust integration test that dispatches a complex message with transport-based handler config, failures, failure transport, etc - and verifies the correct behavior.

Cheers!

Commits
-------

80b5df2 [Messenger] On failure retry, make message appear received from original sender
symfony-splitter pushed a commit to symfony/messenger 10000 that referenced this issue May 11, 2019
@fabpot
bug #31425 [Messenger] On failure retry, make message appear received…
fb67380 
… from original sender (weaverryan)

This PR was squashed before being merged into the 4.3 branch (closes #31425).

Discussion
----------

[Messenger] On failure retry, make message appear received from original sender

| Q             | A
| ------------- | ---
| Branch?       | master (4.3)
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #31413
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

Fixes a bug when using the transport-based handler config from #30958. This also adds a pretty robust integration test that dispatches a complex message with transport-based handler config, failures, failure transport, etc - and verifies the correct behavior.

Cheers!

Commits
-------

80b5df270f [Messenger] On failure retry, make message appear received from original sender
symfony-splitter pushed a commit to symfony/framework-bundle that referenced this issue May 11, 2019
@fabpot
bug #31425 [Messenger] On failure retry, make message appear received…
af51393 
… from original sender (weaverryan)

This PR was squashed before being merged into the 4.3 branch (closes #31425).

Discussion
----------

[Messenger] On failure retry, make message appear received from original sender

| Q             | A
| ------------- | ---
| Branch?       | master (4.3)
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #31413
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

Fixes a bug when using the transport-based handler config from #30958. This also adds a pretty robust integration test that dispatches a complex message with transport-based handler config, failures, failure transport, etc - and verifies the correct behavior.

Cheers!

Commits
-------

80b5df270f [Messenger] On failure retry, make message appear received from original sender
fabpot added a commit to symfony/symfony that referenced this issue May 11, 2019
@fabpot
feature #31454 [Messenger] remove send_and_handle which can be achiev…
22ffa5f 
…ed with SyncTransport (Tobion)

This PR was merged into the 4.3 branch.

Discussion
----------

[Messenger] remove send_and_handle which can be achieved with SyncTransport

| Q             | A
| ------------- | ---
| Branch?       | 4.3
| Bug fix?      | no
| New feature?  | yes/no <!-- please update src/**/CHANGELOG.md files -->
| BC breaks?    | yes     <!-- see https://symfony.com/bc -->
| Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tests pass?   | yes    <!-- please add some, will be required by reviewers -->
| Fixed tickets |
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

The send_and_handle option is pretty awkward and we don't need it anymore because the same thing can be achieved with the SyncTransport from #30759

So the following example from the doc in https://symfony.com/doc/current/messenger.html#routing
```yaml
framework:
    messenger:
        routing:
            'My\Message\ThatIsGoingToBeSentAndHandledLocally':
                 senders: [amqp]
                 send_and_handle: true
```
is the same as
```yaml
framework:
    messenger:
        routing:
            'My\Message\ThatIsGoingToBeSentAndHandledLocally':
                 senders: [amqp, sync]
```

#31401 (review)

Commits
-------

4552b7f [Messenger] remove send_and_handle option which can be achieved with SyncTransport
symfony-splitter pushed a commit to symfony/messenger that referenced this issue May 11, 2019
@fabpot
feature #31454 [Messenger] remove send_and_handle which can be achiev…
cdc4635 
…ed with SyncTransport (Tobion)

This PR was merged into the 4.3 branch.

Discussion
----------

[Messenger] remove send_and_handle which can be achieved with SyncTransport

| Q             | A
| ------------- | ---
| Branch?       | 4.3
| Bug fix?      | no
| New feature?  | yes/no <!-- please update src/**/CHANGELOG.md files -->
| BC breaks?    | yes     <!-- see https://symfony.com/bc -->
| Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tests pass?   | yes    <!-- please add some, will be required by reviewers -->
| Fixed tickets |
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

The send_and_handle option is pretty awkward and we don't need it anymore because the same thing can be achieved with the SyncTransport from #30759

So the following example from the doc in https://symfony.com/doc/current/messenger.html#routing
```yaml
framework:
    messenger:
        routing:
            'My\Message\ThatIsGoingToBeSentAndHandledLocally':
                 senders: [amqp]
                 send_and_handle: true
```
is the same as
```yaml
framework:
    messenger:
        routing:
            'My\Message\ThatIsGoingToBeSentAndHandledLocally':
                 senders: [amqp, sync]
```

symfony/symfony#31401 (review)

Commits
-------

4552b7f5b2 [Messenger] remove send_and_handle option which can be achieved with SyncTransport
symfony-splitter pushed a commit to symfony/framework-bundle that referenced this issue May 11, 2019
@fabpot
feature #31454 [Messenger] remove send_and_handle which can be achiev…
c16743c 
…ed with SyncTransport (Tobion)

This PR was merged into the 4.3 branch.

Discussion
----------

[Messenger] remove send_and_handle which can be achieved with SyncTransport

| Q             | A
| ------------- | ---
| Branch?       | 4.3
| Bug fix?      | no
| New feature?  | yes/no <!-- please update src/**/CHANGELOG.md files -->
| BC breaks?    | yes     <!-- see https://symfony.com/bc -->
| Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tests pass?   | yes    <!-- please add some, will be required by reviewers -->
| Fixed tickets |
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

The send_and_handle option is pretty awkward and we don't need it anymore because the same thing can be achieved with the SyncTransport from #30759

So the following example from the doc in https://symfony.com/doc/current/messenger.html#routing
```yaml
framework:
    messenger:
        routing:
            'My\Message\ThatIsGoingToBeSentAndHandledLocally':
                 senders: [amqp]
                 send_and_handle: true
```
is the same as
```yaml
framework:
    messenger:
        routing:
            'My\Message\ThatIsGoingToBeSentAndHandledLocally':
                 senders: [amqp, sync]
```

symfony/symfony#31401 (review)

Commits
-------

4552b7f5b2 [Messenger] remove send_and_handle option which can be achieved with SyncTransport
@Tobion
Copy link
Contributor
Tobion commented May 11, 2019 •
edited by weaverryan
Loading

- [X] send_and_handle messenger routing got removed in favor of SyncTransport symfony/symfony#31454

(note from Ryan): I moved this up into the main list.

@weaverryan weaverryan mentioned this issue May 26, 2019
Merged
symfony-splitter pushed a commit to symfony/messenger that referenced this issue Jan 28, 2020
@fabpot
feature #30759 [Messenger] Adding the "sync" transport to call handle…
2ac7b9c 
…rs synchronously (weaverryan)

This PR was merged into the 4.3-dev branch.

Discussion
----------

[Messenger] Adding the "sync" transport to call handlers synchronously

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | none
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

This adds a `sync://` transport that just calls the handlers immediately. Why? This allows you to route your messages to some "async" transport. But then, when developing locally or running your tests, you can choose to run them synchronously instead:

```yml
# config/packages/messenger.yaml
framework:
    messenger:
        transports:
            async: '%env(MESSENGER_TRANSPORT_DSN)%'
        routing:
            'App\Message\SmsNotification': async
            'App\Message\OtherMessage': async
```

```
# .env
# by default, handle this sync
MESSENGER_TRANSPORT_DSN=sync://
```

```
# .env.local on production (or set this via real env vars)
# on production, use amqp
MESSENGER_TRANSPORT_DSN=amqp://.......
```

Cheers!

Commits
-------

3da5a438aa Adding the "sync" transport to call handlers synchronously
symfony-splitter pushed a commit to symfony/messenger that referenced this issue Jan 28, 2020
@fabpot
feature #30754 [Messenger] New messenger:stop-workers Command (weaver…
1fc6038 
…ryan)

This PR was squashed before being merged into the 4.3-dev branch (closes #30754).

Discussion
----------

[Messenger] New messenger:stop-workers Command

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | Kinda of #29451
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

o/ me again.

This requires and is built on top of #30708

When you deploy, all workers need to be stopped and restarted. That's not currently possible, unless you manually track the pids and send a SIGTERM signal. We can make that much easier :).

Now run:

```
bin/console messenger:stop-workers
```

And it will signal to all workers (even if they're distributed on other servers) that they should stop, once they finish processing their current message. This is done via a key in `cache.app`.

Cheers!

Commits
-------

58971627f5 [Messenger] New messenger:stop-workers Command
symfony-splitter pushed a commit to symfony/messenger that referenced this issue Jan 28, 2020
@fabpot
feature #30757 [Messenger] Adding MessageCountAwareInterface to get t…
bc57b8c 
…ransport message count (weaverryan)

This PR was squashed before being merged into the 4.3-dev branch (closes #30757).

Discussion
----------

[Messenger] Adding MessageCountAwareInterface to get transport message count

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | none
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

This adds a new optional interface that receivers should implement to give an approximate number of the messages "waiting" to be handled. Why? Because, with this, you could design a system that dynamically adds/removes worker processes if a specific transport is getting slammed and needs help. Creating that system could be something we discuss for core later, but this at least makes it possible - and means it could be implemented by the user or in a bundle... which I might do if we don't get it in core ;).

Commits
-------

fc5b0cf570 [Messenger] Adding MessageCountAwareInterface to get transport message count
symfony-splitter pushed a commit to symfony/messenger that referenced this issue Jan 28, 2020
@sroze
feature #30008 [messenger] Adds a stamp to provide a routing key on m…
a67e49c 
…essage publishing (G15N, sroze)

This PR was merged into the 4.3-dev branch.

Discussion
----------

[messenger] Adds a stamp to provide a routing key on message publishing

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #29950
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

Adds a stamp allowing to set a `routing_key` at `MessageBus::dispatch()` level.

```php
$message = (new Envelope('message'))->with(new RoutingKeyStamp('routing_key'));
$bus->dispatch($message);
```

Commits
-------

a515635f18 Simply code and rename "configuration" to "options"
3151b54b7a [messenger] AMQP configurable routing key & multiple queues
symfony-splitter pushed a commit to symfony/messenger that referenced this issue Jan 28, 2020
@sroze
feature #30958 [Messenger] Allows to register handlers on a specific …
36273d4 
…transport (sroze)

This PR was merged into the 4.3-dev branch.

Discussion
----------

[Messenger] Allows to register handlers on a specific transport

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #30110
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

With the #30008 pull-request in, we can now do the following:
```yaml
framework:
    messenger:
        transports:
            events:
                dsn: amqp://guest:guest@127.0.0.1/%2f/events
                options:
                    queues:
                        3rdparty:
                            binding_keys: [ 3rdparty ]
                        projection:
                            binding_keys: [ projection ]

            events_3rdparty: amqp://guest:guest@127.0.0.1/%2f/events?queues[3rdparty]
            events_projection: amqp://guest:guest@127.0.0.1/%2f/events?queues[projection]

        routing:
            'App\Message\RegisterBet': events
```

This will bind two queues to the `events` exchange, fantastic:
<img width="325" alt="Screenshot 2019-04-07 at 10 26 27" src="https://user-images.githubusercontent.com/804625/55680861-af373580-591f-11e9-8f1e-2d3b6ddba2fd.png">

---

Now, in this setup, the message will be duplicated within the `3rdparty` & `projection` queues. If you just run the consumer for each transport, it will consume the message and call all the handlers. You can't do different things based on which queue you have consumed the message. **This pull-request adds the following feature:**

```php
class RegisterBetHandler implements MessageSubscriberInterface
{
    public function __invoke(RegisterBet $message)
    {
        // Do something only when the message comes from the `events_projection` transport...
    }

    /**
     * {@inheritdoc}
     */
    public static function getHandledMessages(): iterable
    {
        yield RegisterBet::class => [
            'from_transport' => 'events_projection',
        ];
    }
}
```

---

In the debugger, it looks like this:
<img width="649" alt="Screenshot 2019-04-07 at 10 29 55" src="https://user-images.githubusercontent.com/804625/55680892-1d7bf800-5920-11e9-80f7-853f0201c6d8.png">

Commits
-------

f0b2acd67d Allows to register handlers on a specific transport (and get rid of this handler alias)
symfony-splitter pushed a commit to symfony/messenger that referenced this issue Jan 28, 2020
@fabpot
bug #31425 [Messenger] On failure retry, make message appear received…
9a75b92 
… from original sender (weaverryan)

This PR was squashed before being merged into the 4.3 branch (closes #31425).

Discussion
----------

[Messenger] On failure retry, make message appear received from original sender

| Q             | A
| ------------- | ---
| Branch?       | master (4.3)
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #31413
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

Fixes a bug when using the transport-based handler config from #30958. This also adds a pretty robust integration test that dispatches a complex message with transport-based handler config, failures, failure transport, etc - and verifies the correct behavior.

Cheers!

Commits
-------

80b5df270f [Messenger] On failure retry, make message appear received from original sender
symfony-splitter pushed a commit to symfony/messenger that referenced this issue Jan 28, 2020
@fabpot
feature #31454 [Messenger] remove send_and_handle which can be achiev…
01a1678 
…ed with SyncTransport (Tobion)

This PR was merged into the 4.3 branch.

Discussion
----------

[Messenger] remove send_and_handle which can be achieved with SyncTransport

| Q             | A
| ------------- | ---
| Branch?       | 4.3
| Bug fix?      | no
| New feature?  | yes/no <!-- please update src/**/CHANGELOG.md files -->
| BC breaks?    | yes     <!-- see https://symfony.com/bc -->
| Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tests pass?   | yes    <!-- please add some, will be required by reviewers -->
| Fixed tickets |
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

The send_and_handle option is pretty awkward and we don't need it anymore because the same thing can be achieved with the SyncTransport from #30759

So the following example from the doc in https://symfony.com/doc/current/messenger.html#routing
```yaml
framework:
    messenger:
        routing:
            'My\Message\ThatIsGoingToBeSentAndHandledLocally':
                 senders: [amqp]
                 send_and_handle: true
```
is the same as
```yaml
framework:
    messenger:
        routing:
            'My\Message\ThatIsGoingToBeSentAndHandledLocally':
                 senders: [amqp, sync]
```

symfony/symfony#31401 (review)

Commits
-------

4552b7f5b2 [Messenger] remove send_and_handle option which can be achieved with SyncTransport
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Messenger
Projects
None yet
Milestone
4.3
Development

No branches or pull requests
2 participants
@weaverryan @Tobion
0