Replication & Durability
Leaders, ISR, High Watermark, LEO — the full replication protocol explained.
- 1Leader Election ProtocolEvery partition has exactly one leader at a time. The leader handles all produce and fetch requests. Followers replicate…
- 2ISR: In-Sync ReplicasThe ISR is the set of replicas that are 'caught up' with the leader. Caught up means the replica's last fetch was within…
- 3High Watermark (HWM) & LEOThe Log End Offset (LEO) is the next offset to be written — it advances on every leader append. The High Watermark (HWM)…
- 4Preferred Leader ElectionEach partition has a 'preferred' leader — the first replica in its replica assignment list. After broker restarts, leade…
- 5Replica Recovery ProcessWhen a failed broker restarts, it truncates its local log to the last known HWM (removing potentially divergent messages…
Knowledge check
Q1.What is the High Watermark?
Q2.When is a replica removed from the ISR?
Q3.What happens when a broker restarts after failure?