Skip to content

Commit 4718c32

Browse files
garyrussellartembilan
authored andcommitted
Doc: Partition Assignment
See: https://stackoverflow.com/questions/52077027/kafkalistener-concurrency-multiple-topics/52078148#52078148 **cherry-pick to 2.1.x, 1.3.x** (cherry picked from commit 9f828a8)
1 parent e7a1206 commit 4718c32

File tree

1 file changed

+19
-1
lines changed

1 file changed

+19
-1
lines changed

src/reference/asciidoc/kafka.adoc

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -495,7 +495,25 @@ public ConcurrentMessageListenerContainer(ConsumerFactory<K, V> consumerFactory,
495495

496496
It also has a property `concurrency`, e.g. `container.setConcurrency(3)` will create 3 `KafkaMessageListenerContainer` s.
497497

498-
For the first constructor, kafka will distribute the partitions across the consumers.
498+
For the first constructor, kafka will distribute the partitions across the consumers using its group management capabilities.
499+
500+
[IMPORTANT]
501+
====
502+
When listening to multiple topics, the default partition distribution may not be what you expect.
503+
For example, if you have 3 topics with 5 partitions each and you want to use `concurrency=15` you will only see 5 active consumers, each assigned one partition from each topic, with the other 10 consumers being idle.
504+
This is because the default Kafka `PartitionAssignor` is the `RangeAssignor` (see its javadocs).
505+
For this scenario, you may want to consider using the `RoundRobinAssignor` instead, which will distribute the partitions across all of the consumers.
506+
Then, each consumer will be assigned one topic/partition.
507+
To change the `PartitionAssignor`, set the `partition.assignment.strategy` consumer property (`ConsumerConfigs.PARTITION_ASSIGNMENT_STRATEGY_CONFIG`) in the properties provided to the `DefaultKafkaConsumerFactory`.
508+
509+
When using Spring Boot:
510+
511+
[source]
512+
----
513+
spring.kafka.consumer.properties.partition.assignment.strategy=org.apache.kafka.clients.consumer.RoundRobinAssignor
514+
----
515+
====
516+
499517
For the second constructor, the `ConcurrentMessageListenerContainer` distributes the `TopicPartition` s across the
500518
delegate `KafkaMessageListenerContainer` s.
501519

0 commit comments

Comments
 (0)