Skip to content

Commit db84b14

Browse files
garyrussellartembilan
authored andcommitted
Doc: Partition Assignment
See: https://stackoverflow.com/questions/52077027/kafkalistener-concurrency-multiple-topics/52078148#52078148 **cherry-pick to 2.1.x, 1.3.x** (cherry picked from commit 9f828a8)
1 parent 8b5c08a commit db84b14

File tree

1 file changed

+19
-1
lines changed

1 file changed

+19
-1
lines changed

src/reference/asciidoc/kafka.adoc

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -635,7 +635,25 @@ public ConcurrentMessageListenerContainer(ConsumerFactory<K, V> consumerFactory,
635635

636636
It also has a property `concurrency`, e.g. `container.setConcurrency(3)` will create 3 `KafkaMessageListenerContainer` s.
637637

638-
For the first constructor, kafka will distribute the partitions across the consumers.
638+
For the first constructor, kafka will distribute the partitions across the consumers using its group management capabilities.
639+
640+
[IMPORTANT]
641+
====
642+
When listening to multiple topics, the default partition distribution may not be what you expect.
643+
For example, if you have 3 topics with 5 partitions each and you want to use `concurrency=15` you will only see 5 active consumers, each assigned one partition from each topic, with the other 10 consumers being idle.
644+
This is because the default Kafka `PartitionAssignor` is the `RangeAssignor` (see its javadocs).
645+
For this scenario, you may want to consider using the `RoundRobinAssignor` instead, which will distribute the partitions across all of the consumers.
646+
Then, each consumer will be assigned one topic/partition.
647+
To change the `PartitionAssignor`, set the `partition.assignment.strategy` consumer property (`ConsumerConfigs.PARTITION_ASSIGNMENT_STRATEGY_CONFIG`) in the properties provided to the `DefaultKafkaConsumerFactory`.
648+
649+
When using Spring Boot:
650+
651+
[source]
652+
----
653+
spring.kafka.consumer.properties.partition.assignment.strategy=org.apache.kafka.clients.consumer.RoundRobinAssignor
654+
----
655+
====
656+
639657
For the second constructor, the `ConcurrentMessageListenerContainer` distributes the `TopicPartition` s across the
640658
delegate `KafkaMessageListenerContainer` s.
641659

0 commit comments

Comments
 (0)