What happens on TaskRejectedException in an rqueue message listener?

[damien-swarm]

I'm seeing a bunch of warnings like this:

2021-10-26 09:43:20.185  WARN                  c.g.s.r.l.RqueueMessagePoller            : [msg-delivery-retry] Execution failed Msg: RqueueMessage(id=6fe0fa8e-47d6-4cbf-a4d5-ce4cc1bd9cc5, queueName=msg-delivery-retry, message={"message":"MY_MESSAGE"}, retryCount=null, queuedTime=5718680906046356, processAt=1635240437975, reEnqueuedAt=null, failureCount=0, sourceQueueFailureCount=0, sourceQueueName=null, period=0)
org.springframework.core.task.TaskRejectedException: Executor [java.util.concurrent.ThreadPoolExecutor@e17ac46[Running, pool size = 5, active threads = 5, queued tasks = 0, completed tasks = 47476]] did not accept task: com.github.sonus21.rqueue.listener.RqueueExecutor@33a8e62f

I think it's because I'm accepting >5 tasks from this msg-delivery-retry queue at the same time, so rqueue is basically throttling what's coming through that queue. Is that correct?

This is actually desirable behavior for me, just want to make sure I'm understanding what happened, and that rqueue isn't just dropping messages over the thread limit, or something.

EDIT: I also have simpleRqueueListenerContainerFactory.setMaxNumWorkers(3) in my config; what else uses this ThreadPoolExecutor?

Also (unrelated), I realized I probably want something more similar to #43 rather than hoping the threadpool throttles me, where I want to listen to things in this queue sequentially per partition, but concurrently across different partitions; I'll try out your groupId suggestion there. (It's unfortunate that only a static set of partitions is supported.)

[sonus21]

These tasks would be retried later, generally this error should not come. You should increase MaxNumWorkers, also can you try the latest release 2.10.1-RELEASE?

[sonus21]

@damien-swarm did you get a chance to validate this issue on 2.10.1-RELEASE? This error message should not come.

[damien-swarm]

@sonus21 thanks for your answer. I upgraded the library but haven't deployed to production yet; I'll let you know if the warnings go away after that.

[damien-swarm]

I don't see the warnings anymore for now, though it also may be due to other changes I made. :)

I increased MaxNumWorkers, partitioned the queue as per your recommendation in #43, and set it to 1 concurrent listener per partition. I guess I'll have to wait for our application to get more load across different partitions before I see if this comes up again.