How does Kafka's compaction work internally? What is the cleaner thread?

The Log Cleaner thread runs in the background and compacts closed segments. It maintains a SkipList (OffsetMap) in memory to track the latest offset for each key seen so far. It reads segments in order, adding key→offset entries. Then it rewrites segments, keeping only messages whose offset matches the latest for their key. Tombstones (null value) are retained for a configurable period (delete.retention.ms) to propagate deletions to consumers, then cleaned. The cleaner runs when the 'dirty ratio' (unclean data / total) exceeds min.cleanable.dirty.ratio.

Expected key concepts: Log Cleaner thread, SkipList, OffsetMap, latest offset, rewrite segments, tombstones, delete.retention.ms, dirty ratio, min.cleanable.dirty.ratio, background thread