What is the difference between LEO and HWM? Why does it matter for consumers?

LEO (Log End Offset) is the offset of the next message to be written — it advances immediately when the leader appends to its local log. HWM (High Watermark) is the highest offset fully replicated to all ISR replicas — it advances only after all ISR members acknowledge the write. Consumers can only read messages up to the HWM. The gap between LEO and HWM represents uncommitted messages that might be lost if the leader fails before replication completes. Reading past the HWM could expose consumers to data that gets truncated during leader recovery.

Expected key concepts: LEO, HWM, log end offset, high watermark, uncommitted, replicated, ISR, truncated, leader recovery, consumers