Skip to content
Kafka Is Fun

IntermediateInternals

What is Kafka's inter-broker replication protocol?

Reference answer

Followers replicate by consuming from the leader just as a normal Kafka consumer would, fetching messages with fetch requests and applying them to their own log, which lets them batch log entries. Each follower has its own log end offset (LEO). Fetch sizes are bounded by replica.fetch.max.bytes (per partition, default 1 MiB; not an absolute maximum, because the first record batch is always returned) and replica.fetch.response.max.bytes (the whole response, default 10 MiB), and num.replica.fetchers (default 1) sets the number of fetcher threads per source broker. The leader maintains the ISR: a follower that has not sent fetch requests, or has not consumed up to the leader's LEO, for replica.lag.time.max.ms (default 30 seconds) is removed from it. A message is committed once all replicas in the ISR have it, and only committed messages are given to consumers.

Expected key concepts: follower, leader, fetch requests, LEO, ISR, replica.fetch.max.bytes, replica.lag.time.max.ms, committed, num.replica.fetchers, replica.fetch.response.max.bytes

How did that feel?

Your own verdict, not the score. It shows up next to this question in the list, so you can come back to the shaky ones.