-
Notifications
You must be signed in to change notification settings - Fork 4k
Shovel bugfix: handle cluster alarms in AMQP10 and local shovels #14873
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
When an alarm is set in the destination cluster, the AMQP10 shovel receives a remote_incoming_window_exceeded error. It must handle it and retry the send later on, until the alarm is cleared
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test failures are reproducible.
Local shovels must stop publishing when alarms are set in the destination cluster. Messages are stored in memory and sent when the alarm clears, the same way it is done for AMQP091
9bf2fab to
f038115
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the record, since we depend on queue:len/1: modern Erlang/OTP's queue module still optimizes for memory footprint and queue:len/1 is an O(n) operation. It is documented right at the top of the module.
That's why RabbitMQ has had an alternative lqueue module since Sep 2011: it keeps the length around and makes queue:len/1 a constant time operation.
I will try switching these changes to use lqueue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's worry about module swapping in a separate PR.
Shovel bugfix: handle cluster alarms in AMQP10 and local shovels (backport #14873)
AMQP10 needs to handle the
remote_income_window_exceedederror when an alarm is triggered.Local shovels need to stop publishing and store the pending deliveries when an alarm is triggered. When all alarms are cleared, it must publish the pending deliveries.