How does Kafka's compaction work internally? What is the cleaner thread?

Log compaction is handled by the log cleaner, a pool of background threads (log.cleaner.threads, default 1) that recopy log segment files. For a topic with cleanup.policy=compact, the cleaner chooses the log with the highest ratio of log head (the not-yet-cleaned part) to log tail. It builds a succinct summary of the last offset for each key in the head, essentially a space-compact hash table (the offset map; its memory is bounded by log.cleaner.dedupe.buffer.size, default 128 MiB). It then recopies the log from beginning to end, removing keys that have a later occurrence. Offsets of retained messages never change. A record with a key and a null value is a tombstone (a delete marker); it is kept for delete.retention.ms (default 24 hours, so consumers that lag less than that still see the delete) and is then removed. A log is only cleaned once its dirty ratio exceeds min.cleanable.dirty.ratio (default 0.5); min.compaction.lag.ms and max.compaction.lag.ms can delay or force compaction.

Expected key concepts: log cleaner, background threads, offset map, hash table, last offset for each key, recopy, head, tombstone, delete.retention.ms, min.cleanable.dirty.ratio, cleanup.policy=compact