Skip to content
Kafka Is Fun

SpringSpring Kafka Configuration

How do you configure Kafka transactions in Spring Kafka?

Reference answer

Set spring.kafka.producer.transaction-id-prefix (it must be unique per application instance); once it is defined, Spring Boot configures a KafkaTransactionManager and associates it with the listener container factory. KafkaTransactionManager is a Spring PlatformTransactionManager, so @Transactional or a TransactionTemplate starts a Kafka transaction and KafkaTemplate sends inside it take part. With @Transactional on a method that also updates a database, the database transaction commits first and the Kafka transaction after it; nested @Transactional methods reverse that order. Or use kafkaTemplate.executeInTransaction(t -> {...}): it commits if the callback returns normally and rolls back on an exception. For consume-process-produce, give the listener container a KafkaAwareTransactionManager: it starts a transaction before invoking the listener, sends the offsets to the transaction with sendOffsetsToTransaction, and rolls back on an exception, so the read-process-write sequence is completed exactly once (the read and process have at-least-once semantics). ChainedKafkaTransactionManager is deprecated since 2.7; use a KafkaTransactionManager in the container and @Transactional on the listener for the other transaction. Since 3.0 only EOSMode.V2 is supported (brokers 2.5 or later); fencing zombies relies on consumer metadata sent with the offsets, and the generated transactional.id is the prefix plus a number.

Expected key concepts: transaction-id-prefix, KafkaTransactionManager, @Transactional, executeInTransaction, KafkaAwareTransactionManager, sendOffsetsToTransaction, ChainedKafkaTransactionManager, EOSMode.V2, zombie

How did that feel?

Your own verdict, not the score. It shows up next to this question in the list, so you can come back to the shaky ones.