What is Kafka's log segment and why does it matter?

A Kafka partition's log is stored as a set of segment files. Each file is named with its base offset, the offset of the first message it contains, so the first file is 00000000000000000000.log. Appends always go to the last file, the active segment, which is rolled over to a new file when it reaches log.segment.bytes (segment.bytes on the topic, default 1 GiB) or when the time limit passes (segment.ms on the topic, default 7 days; on the broker log.roll.ms, with log.roll.hours of 168 as the secondary setting). Data is deleted one log segment at a time, based on time or size (size-based retention is disabled by default), so retention works on whole segments: a segment is removed only once it is eligible, which means records can stay on disk longer than the retention setting alone suggests. Log compaction never compacts the active segment, even if all its messages are older than the minimum compaction lag.

Expected key concepts: segment files, base offset, active segment, log.segment.bytes, segment.ms, log.roll.ms, retention, compaction, whole segments, rolled