SpringSpring Kafka Configuration
What is the difference between concurrency and partitions in Spring Kafka's listener container?
Reference answer
Concurrency, set on the container factory or overridden by the concurrency attribute of @KafkaListener, is the number of consumers the container creates: ConcurrentMessageListenerContainer delegates to that many KafkaMessageListenerContainer instances, each with its own KafkaConsumer. Partitions belong to the topic. With topics or a topic pattern, Kafka's group management distributes the partitions among those consumers, so concurrency=3 over 6 partitions gives each consumer some of the partitions (two each if the assignor spreads them evenly). With explicit TopicPartitionOffsets, a concurrency above the number of partitions is adjusted down so each container gets one partition; the same documented example of 6 partitions and concurrency=3 gives each container two. With group management, a consumer that receives no partition sits idle: three topics of five partitions each with concurrency=15 and the default RangeAssignor leave only five active consumers, and RoundRobinAssignor is one way to spread them. Concurrency is per application instance; to add consumers across JVMs, run more instances of the application, and a common practice is not to exceed the partition count because extra consumers would be idle.
Expected key concepts: concurrency, ConcurrentMessageListenerContainer, KafkaMessageListenerContainer, KafkaConsumer, partitions, idle, RangeAssignor, RoundRobinAssignor, instances
How did that feel?
Your own verdict, not the score. It shows up next to this question in the list, so you can come back to the shaky ones.