-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
enhancementNew feature or requestNew feature or request
Description
📘 Context
Currently, when the system encounters a message it cannot process (e.g., unknown trigger), it enters an infinite reprocessing loop due to NACK with requeue. This causes:
- High redelivery rate (200+ messages/second)
- Blocking of other messages in the queue (
prefetch=1) - Unnecessary resource consumption
- Inability to process valid messages
🐞 Problem Identified
During an accidental deployment of development code to production, voting-power-changed messages were generated. When production code was restored, these messages got stuck in the queue causing an infinite loop, as the dispatcher doesn't have a handler for this trigger type.
🔧 Tasks
-
Update RabbitMQConsumer to automatically create DLQ
- Modify
consumer.tsto configure DLQ for all queues - Configure queue with
x-dead-letter-exchangeandx-dead-letter-routing-key - Automatically create DLQ with
.dlqsuffix - Set 7-day TTL for messages in DLQ
- Modify
-
Implement retry logic with counter
- Add
x-retry-countheader to messages - After 3 attempts, use
channel.reject(msg, false)to send to DLQ - Include error information in message header
- Add
-
Add default handler for unknown triggers
- Warning log for unregistered trigger
- Direct
ACKafter logging (don't send to DLQ)
-
Create monitoring script
- Script to check messages in DLQs
- Prepare structure for
n8nintegration
-
Tests
- Unit test for retry behavior
- Integration test with unknown trigger message
- Verify messages go to DLQ after 3 attempts
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request