Overview
When a produce request fails with a transient error, the producer retries until it succeeds, fails with a non-transient error, or delivery.timeout.ms (120000 ms by default) expires; Apache recommends leaving retries at its default and tuning delivery.timeout.ms instead. Without idempotence, retries can write duplicates, because the network can fail after the broker has committed a message. With enable.idempotence=true the broker assigns each producer an ID and deduplicates messages using a sequence number the producer sends with every message. The guarantee holds within a single producer session, and it cannot de-duplicate re-sends made by your own application code.
Key configuration
enable.idempotence=true — the default since Kafka 3.0
retries=2147483647 — default; retrying stops at delivery.timeout.ms
retry.backoff.ms=100 — initial backoff, growing exponentially up to retry.backoff.max.ms (default 1000)
max.in.flight.requests.per.connection=5Production tip
Check yourself
With enable.idempotence=true, how does the broker recognise a duplicate caused by a retry?
Sources
- Producer configs (4.2 documentation) (opens in a new tab)
- KafkaProducer Javadoc (4.2) (opens in a new tab)
- Design (4.2 documentation) (opens in a new tab)
Checked against Kafka 4.2 documentation on Oct 2026. Draft: not yet reviewed by a Kafka specialist. Releases move on, so confirm against the current documentation before you act on a number.