Skip to content
Kafka Is Fun

IntermediateInternals

How does a Kafka producer handle a NOT_LEADER_OR_FOLLOWER error?

Reference answer

When the producer sends to a broker that is no longer the leader for a partition (its metadata is stale), the broker returns NOT_LEADER_OR_FOLLOWER, an invalid-metadata error that is retriable. The producer's Sender thread then: 1) requests a metadata update (the produce response can also carry the current leader id and epoch, which the producer uses to refresh its cached leader info); 2) re-enqueues the batch and retries it after the retry backoff, as long as the retry limits allow (retries and delivery.timeout.ms, default 120000 ms). The backoff starts at retry.backoff.ms (default 100) and grows exponentially up to retry.backoff.max.ms (default 1000). With enable.idempotence=true (the default) each batch carries a producer ID and sequence number, so if the original request actually succeeded the broker detects the duplicate and the retry does not create one. This automatic recovery makes producers resilient to leader elections; the application only sees an error if the delivery timeout expires or the retries run out.

Expected key concepts: NOT_LEADER_OR_FOLLOWER, stale, metadata update, retry, retry.backoff.ms, delivery.timeout.ms, enable.idempotence, sequence number, duplicate, leader election

How did that feel?

Your own verdict, not the score. It shows up next to this question in the list, so you can come back to the shaky ones.