Skip to content
Kafka Is Fun

SpringSpring Kafka Advanced

How do you implement a batch @KafkaListener in Spring Kafka?

Reference answer

Set spring.kafka.listener.type=batch (the default is single) in Spring Boot, or set the batchListener property (setBatchListener(true)) on the container factory, or use the batch attribute on @KafkaListener. The method can accept List<V>, List<Message<?>>, List<ConsumerRecord<K,V>>, or the whole ConsumerRecords object. The batch is what consumer.poll() returned, so max-poll-records (spring.kafka.consumer.max-poll-records, the Kafka max.poll.records setting, default 500) caps its size. For manual commits use a manual ack mode, inject Acknowledgment and call acknowledge() after processing the batch; with MANUAL_IMMEDIATE and a List parameter you can also call acknowledge(index) to commit part of the batch. To handle one bad record, throw BatchListenerFailedException identifying the record (or its index): DefaultErrorHandler commits the records before it, retries the failed record and those after it, and when retries are exhausted can publish only the failed record to a DLT. Any other exception makes the whole batch be retried. Non-blocking retries with @RetryableTopic are not supported for batch listeners.

Expected key concepts: spring.kafka.listener.type, List<ConsumerRecord<K,V>>, max-poll-records, Acknowledgment, acknowledge, BatchListenerFailedException, DefaultErrorHandler, consumer.poll, batchListener, DLT

How did that feel?

Your own verdict, not the score. It shows up next to this question in the list, so you can come back to the shaky ones.