[Messenger] Provide a deduplication strategy to Queue #54126
Comments
I tried doing this with a Middleware and it requires more testing but this might to work with something like class LockMiddleware implements MiddlewareInterface
{
public function __construct(
private readonly LockFactory $lockFactory,
) {
}
public function handle(Envelope $envelope, StackInterface $stack): Envelope
{
$message = $envelope->getMessage();
// If we're trying to dispatch a lockable message.
if ($message inst
8000
anceof LockableMessage && null === $envelope->last(ReceivedStamp::class)) {
$key = $message->getKey();
if (null !== $key) {
// The acquire call must be done before stamping the message
// in order to have the full state of the key in the stamp.
$canAcquire = $this->lockFactory->createLockFromKey($key, autoRelease: false)->acquire();
$envelope = $envelope->with(new LockStamp($key));
if (!$canAcquire) {
return $envelope;
}
}
}
try {
$envelope = $stack->next()->handle($envelope, $stack);
} finally {
// If we've received a lockable message, we're releasing it.
if (null !== $envelope->last(ReceivedStamp::class)) {
$stamp = $envelope->last(LockStamp::class);
if ($stamp instanceof LockStamp) {
$key = $stamp->key;
$this->lockFactory->createLockFromKey($key, autoRelease: false)->release();
}
}
}
return $envelope;
}
} |
|
many queues like sqs support deduplication out of the box, i don't think it is something symfony should do (read care about). |
The deduplication provided by sqs is different from the one I proposed in the issue
It means even after a message is handled, you can't add a new message to the queue during the 5 minute deduplication-interval. What I'm looking for is a deduplication only when the message is in queue. As soon as the message is received/deleted, it's possible to add another message.
If it's something which can be fully implemented with only a Symfony Middleware and the Lock component, with full queue-agnostic implementation ; it could be great having the feature on Symfony side. |
Description
It would be useful to provide an easy way to handle duplicate message in queue.
I might be wrong but I found nothing implemented yet, just a discussion here #49215
The idea would be to propose something like an interface Message could implement
interface DeduplicatedMessageInterfaceand if they doing so, when I try to publish such a message, it checks if the queue already has the same message with the same body. If so, the new message is not added to the queue.
I assume something could be possible with Symfony lock for instance:
__invokemethod.But maybe a better implementation exists ?
Example
No response
The text was updated successfully, but these errors were encountered: