What is the difference between LEO and HWM? Why does it matter for consumers?

LEO (log end offset) is the offset of the next message to be appended to a replica's log; every replica has its own LEO, and the leader's advances as soon as it appends. The high watermark (HWM) marks what is committed, meaning replicated to all in-sync replicas: the leader sets it to the smallest LEO among the in-sync replicas (it only advances while the ISR size is at least min.insync.replicas) and passes it on to followers in fetch responses. For a default (read_uncommitted) consumer the end offset is the HWM, the offset of the last successfully replicated message plus one, so consumers see only committed messages; read_committed consumers are limited further to the last stable offset, which is never past the HWM. The gap between LEO and HWM is data that is not yet committed and can be lost if the leader fails (a committed message is not lost as long as one in-sync replica survives), so hiding it means consumers never read data that a leader change could remove.

Expected key concepts: LEO, HWM, log end offset, high watermark, committed, in-sync replicas, ISR, min.insync.replicas, last stable offset, read_committed, leader