Skip to content
Kafka Is Fun

SpringSpring Kafka Configuration

How do you control the number of messages processed per @KafkaListener invocation?

Reference answer

Two separate settings are involved. First, the upper bound of records returned by one poll is max.poll.records (Spring Boot: spring.kafka.consumer.max-poll-records; Kafka default 500), 'the maximum number of records returned in a single call to poll()'. Second, the listener type decides how those records are handed to your method. A record listener (Boot: spring.kafka.listener.type=SINGLE) is called once per record. A batch listener (spring.kafka.listener.type=BATCH, factory.setBatchListener(true), or batch=true on the @KafkaListener annotation since Spring Kafka 2.8) receives the records of a poll in a List, for example List<ConsumerRecord<K,V>> or List<String>, so a batch holds at most max.poll.records records and can be smaller. Do not confuse this with the container AckMode values RECORD and BATCH, which control when offsets are committed (default BATCH: after all records from the poll are processed). fetch.min.bytes (default 1) and fetch.max.wait.ms (default 500) control how the broker answers fetch requests; max.poll.records does not change fetch behaviour, because records are cached from each fetch and returned incrementally, so these settings influence but do not set the batch size. Keep the time to process one poll within max.poll.interval.ms (default 300000), or the consumer is removed from the group.

Expected key concepts: max-poll-records, max.poll.records, spring.kafka.listener.type, batch listener, AckMode, fetch.min.bytes, fetch.max.wait.ms, batchListener, max.poll.interval.ms, poll

How did that feel?

Your own verdict, not the score. It shows up next to this question in the list, so you can come back to the shaky ones.