Log compaction is an alternative to deletion. Instead of time/size-based deletion, Kafka keeps only the LATEST message per key. This turns Kafka into a key-value store with complete change history. Deleted keys are represented by tombstone messages (null value) which are eventually removed. This is used by Kafka Connect for CDC and Kafka Streams for KTables.
Key configuration
log.cleanup.policy=compact — enable compaction
min.cleanable.dirty.ratio=0.5 — trigger compaction when 50% is dirty
delete.retention.ms=86400000 — tombstone retention (1 day)