When a ProduceRequest fails with a retryable error (LEADER_NOT_AVAILABLE, NOT_LEADER_OR_FOLLOWER), the producer retries up to retries times with retry.backoff.ms delay. Without idempotency, retries can cause duplicates — the network can fail AFTER the broker commits the message. enable.idempotence=true assigns each producer a PID and a monotonic sequence number per partition, letting the broker deduplicate within a single session.
Key configuration
enable.idempotence=true — strongly recommended for all critical data
retries=2147483647 — Integer.MAX_VALUE (effective 'retry forever')
retry.backoff.ms=100 — wait before retry
max.in.flight.requests.per.connection=5