What is Kafka's log segment and why does it matter?

A Kafka partition is stored as a set of segment files, each with a base offset in the filename (e.g., 00000000000000000000.log). New messages go to the active segment. When a segment reaches log.segment.bytes (default 1GB) or log.roll.ms, it's rolled to a new segment. The active segment is never eligible for retention deletion — only closed segments are. Log compaction and retention operate on closed segments. This means if you have a very active partition with one large active segment, you might use more disk than your retention policy suggests.

Expected key concepts: segment files, base offset, active segment, log.segment.bytes, log.roll.ms, closed segments, retention, compaction, disk usage