8000 [Messenger] Allows to register handlers on a specific transport by sroze · Pull Request #30958 · symfony/symfony · GitHub
[go: up one dir, main page]

Skip to content

[Messenger] Allows to register handlers on a specific transport #30958

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 28, 2019

Conversation

sroze
Copy link
Contributor
@sroze sroze commented Apr 7, 2019
Q A
Branch? master
Bug fix? no
New feature? yes
BC breaks? no
Deprecations? no
Tests pass? yes
Fixed tickets #30110
License MIT
Doc PR symfony/symfony-docs#11236

With the #30008 pull-request in, we can now do the following:

framework:
    messenger:
        transports:
            events:
                dsn: amqp://guest:guest@127.0.0.1/%2f/events
                options:
                    queues:
                        3rdparty:
                            binding_keys: [ 3rdparty ]
                        projection:
                            binding_keys: [ projection ]

            events_3rdparty: amqp://guest:guest@127.0.0.1/%2f/events?queues[3rdparty]
            events_projection: amqp://guest:guest@127.0.0.1/%2f/events?queues[projection]

        routing:
            'App\Message\RegisterBet': events

This will bind two queues to the events exchange, fantastic:
Screenshot 2019-04-07 at 10 26 27


Now, in this setup, the message will be duplicated within the 3rdparty & projection queues. If you just run the consumer for each transport, it will consume the message and call all the handlers. You can't do different things based on which queue you have consumed the message. This pull-request adds the following feature:

class RegisterBetHandler implements MessageSubscriberInterface
{
    public function __invoke(RegisterBet $message)
    {
        // Do something only when the message comes from the `events_projection` transport...
    }

    /**
     * {@inheritdoc}
     */
    public static function getHandledMessages(): iterable
    {
        yield RegisterBet::class => [
            'from_transport' => 'events_projection',
        ];
    }
}

In the debugger, it looks like this:
Screenshot 2019-04-07 at 10 29 55

@Koc
Copy link
Contributor
Koc commented Apr 7, 2019

I see some drawbacks with current configuration:

  • All this queues[3rdparty][binding_keys][]=3rdparty&queues[projection][binding_keys][]=projection dsn strings is quite hard to understand and write. And dsn string became longer and longer on each new queue.
  • You should create +1 transport for consuming messages from each queue (events_3rdparty, events_projection)
  • Handler (code level) knows about transport name (config level) - it similar to breaking dependency inversion principe but for config

Please consider move out queue name from dsn and extend routing configuration:

framework:
    messenger:
        transports:
            events: amqp://guest:guest@127.0.0.1/%2f/events
        routing:
            App\Message\RegisterBet:
                -   transport: events
                    queue: events_projection
                    handler: App\MessageHandler\RegisterBetHandler
                -   transport: events
                    queue: events_3rdparty
                    handler: App\MessageHandler\SomethingOnBetRegister

Handler can be omited for cases when we just pulishing messages and don't comsuming they through Messenger.

This kind of config is super easy to understand. We have configuration of message-transport/queue-handler in single place.

Of course messenger:consume command should be updated to use handler name as argument messenger:consume 'App\MessageHandler\RegisterBetHandler'

@nicolas-grekas nicolas-grekas added this to the next milestone Apr 7, 2019
@sroze sroze force-pushed the messenger-locator-on-transport branch 2 times, most recently from 106be1d to 1de1cb2 Compare April 7, 2019 12:00
@sroze
Copy link
Contributor Author
sroze commented Apr 7, 2019

@Koc I don't see your point as being against this proposed implementation here. The main point is to create this HandlerDescriptor, which allows us to have additional metadata on a specific handler (following a very long discussion with @webmozart).

In the example given, the handler indeed knows about the transport (it's already knowing about things like the bus and its own priority when using the MessageSubscriberInterface). But it allows us to harvest these options (HandlerDescriptor's options) from outside now, which can indeed be a YAML configuration.

Regarding the overhead in terms of configuring the exchange and the queue bindings, you could also use the options if you don't want it in the DSN and might challenge would here be that actually you'd be better in creating this setup by yourself in RabbitMq. Having to create multiple transports actually simplifies the configuration because we don't need to introduce this extra concept queue that you suggested above.

@sroze sroze force-pushed the messenger-locator-on-transport branch from 1de1cb2 to af34fbb Compare April 7, 2019 12:42
@sroze
Copy link
Contributor Author
sroze commented Apr 7, 2019

(requires CHANGELOG when a few 👍 will be around 😉 )

@sroze
Copy link
Contributor Author
sroze commented Apr 7, 2019

Open question: should we only have handlerName in HandlerDescriptor instead of directly the handler so it can later be serialized if needed?

@weaverryan
Copy link
Member

Makes sense to me - we do this with the bus name stamp and sender name for retry.

Copy link
Member
@nicolas-grekas nicolas-grekas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a few minor things + rebase needed

@sroze sroze changed the title [Messenger] Allows to register handlers on a specific transport (and get rid of this handler alias) [Messenger] Allows to register handlers on a specific transport Apr 22, 2019
@sroze sroze force-pushed the messenger-locator-on-transport branch from af34fbb to 83758b6 Compare April 22, 2019 16:19
@sroze
Copy link
Contributor Author
sroze commented Apr 22, 2019

@nicolas-grekas @weaverryan updated 👍

@sroze sroze force-pushed the messenger-locator-on-transport branch from 83758b6 to cc6f13d Compare April 22, 2019 16:24
Copy link
Member
@weaverryan weaverryan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me - and I tested it locally.

There's just one piece that bothers me:

    public static function getHandledMessages(): iterable
    {
        yield RegisterBet::class => [
            'transport' => 'events_projection',
        ];
    }

The meaning of transport is this:

This handler will only be called when the message is received from the events_projection transport

But it's just as reasonable to think it means this:

This handler will be sent to the events_projection transport

But, the only possibly better key I can think of is: 'only_handle_from_transport' => 'events_projection',

@chalasr
Copy link
Member
chalasr commented Apr 26, 2019

But, the only possibly better key I can think of is: 'only_handle_from_transport' => 'events_projection',

Maybe from_transport is enough to avoid the confusion?

@weaverryan
Copy link
Member

Hmm, I do kinda like from_transport

@sroze
Copy link
Contributor Author
sroze commented Apr 28, 2019

Happy with from_transport as well. Let me change this 👍

@sroze sroze force-pushed the messenger-locator-on-transport branch from cc6f13d to 271c12b Compare April 28, 2019 15:12
@sroze
Copy link
Contributor Author
sroze commented Apr 28, 2019

This PR should be ready :)

Copy link
Member
@chalasr chalasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 with minor comment

@sroze sroze force-pushed the messenger-locator-on-transport branch from 271c12b to f0b2acd Compare April 28, 2019 15:17
@sroze sroze merged commit f0b2acd into symfony:master Apr 28, 2019
sroze added a commit that referenced this pull request Apr 28, 2019
…transport (sroze)

This PR was merged into the 4.3-dev branch.

Discussion
----------

[Messenger] Allows to register handlers on a specific transport

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #30110
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

With the #30008 pull-request in, we can now do the following:
```yaml
framework:
    messenger:
        transports:
            events:
                dsn: amqp://guest:guest@127.0.0.1/%2f/events
                options:
                    queues:
                        3rdparty:
                            binding_keys: [ 3rdparty ]
                        projection:
                            binding_keys: [ projection ]

            events_3rdparty: amqp://guest:guest@127.0.0.1/%2f/events?queues[3rdparty]
            events_projection: amqp://guest:guest@127.0.0.1/%2f/events?queues[projection]

        routing:
            'App\Message\RegisterBet': events
```

This will bind two queues to the `events` exchange, fantastic:
<img width="325" alt="Screenshot 2019-04-07 at 10 26 27" src="https://user-images.githubusercontent.com/804625/55680861-af373580-591f-11e9-8f1e-2d3b6ddba2fd.png">

---

Now, in this setup, the message will be duplicated within the `3rdparty` & `projection` queues. If you just run the consumer for each transport, it will consume the message and call all the handlers. You can't do different things based on which queue you have consumed the message. **This pull-request adds the following feature:**

```php
class RegisterBetHandler implements MessageSubscriberInterface
{
    public function __invoke(RegisterBet $message)
    {
        // Do something only when the message comes from the `events_projection` transport...
    }

    /**
     * {@inheritdoc}
     */
    public static function getHandledMessages(): iterable
    {
        yield RegisterBet::class => [
            'from_transport' => 'events_projection',
        ];
    }
}
```

---

In the debugger, it looks like this:
<img width="649" alt="Screenshot 2019-04-07 at 10 29 55" src="https://user-images.githubusercontent.com/804625/55680892-1d7bf800-5920-11e9-80f7-853f0201c6d8.png">

Commits
-------

f0b2acd Allows to register handlers on a specific transport (and get rid of this handler alias)
@nicolas-grekas nicolas-grekas modified the milestones: next, 4.3 Apr 30, 2019
@fabpot fabpot mentioned this pull request May 9, 2019
fabpot added a commit that referenced this pull request May 11, 2019
… from original sender (weaverryan)

This PR was squashed before being merged into the 4.3 branch (closes #31425).

Discussion
----------

[Messenger] On failure retry, make message appear received from original sender

| Q             | A
| ------------- | ---
| Branch?       | master (4.3)
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #31413
| License       | MIT
| Doc PR        | symfony/symfony-docs#11236

Fixes a bug when using the transport-based handler config from #30958. This also adds a pretty robust integration test that dispatches a complex message with transport-based handler config, failures, failure transport, etc - and verifies the correct behavior.

Cheers!

Commits
-------

80b5df2 [Messenger] On failure retry, make message appear received from original sender
@Tobion
Copy link
Contributor
Tobion commented May 11, 2019

The from_transport is not documented in the MessageSubscriberInterface.

I still can't believe we use arbitrary array values in the suscriber instead of a value object that is self-documenting and can have types. There is even a HandlerDescriptor now but only for internal use...

fabpot added a commit that referenced this pull request May 13, 2019
…(sroze)

This PR was merged into the 4.3 branch.

Discussion
----------

[Messenger] Add `from_transport` in subscriber's phpdoc

| Q             | A
| ------------- | ---
| Branch?       | 4.3
| Bug fix?      | yes
| New feature?  | ø
| BC breaks?    | ø
| Deprecations? | ø
| Tests pass?   | yes
| Fixed tickets | #30958
| License       | MIT
| Doc PR        | ø

Add the `from_transport` to the subscriber's phpdoc as it was missed from the original PR.

Commits
-------

6a542cd Add transport in subscriber's phpdoc
@sroze sroze deleted the messenger-locator-on-transport branch June 2, 2019 15:49
javiereguiluz added a commit to symfony/symfony-docs that referenced this pull request Oct 2, 2019
This PR was merged into the 4.3 branch.

Discussion
----------

Remove info about the HandlersLocator alias

It seems that the option to provide an alias to the `HandlersLocator` was removed in symfony/symfony#30958 , but there are still some mentions of it in the documentation.

Commits
-------

fe8dc43 Remove info about the HandlersLocator alias
@tony-stark-eth
Copy link

Please add the wonderful example from @sroze to the documentation on https://symfony.com/doc/current/messenger.html as a how to, took me some time to find this pull request. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants
0