What is Kafka's data retention and how is it configured?

Kafka keeps events for as long as you configure, not until they are read: events are not deleted after consumption. Under the default cleanup.policy=delete, old segments are discarded when a limit is reached: retention.ms (default 604800000, 7 days; -1 means no time limit) and retention.bytes (default -1, no size limit; the limit applies per partition, so multiply by the partition count for the topic total). Deletion works on whole log segments, not individual messages. These are topic settings that override broker defaults: log.retention.ms, log.retention.minutes or log.retention.hours (default 168) and log.retention.bytes. Setting cleanup.policy=compact enables log compaction instead, which keeps at least the last value for each key; both can be combined as compact,delete.

Expected key concepts: retention.ms, retention.bytes, 7 days, segments, cleanup.policy, delete, compact, log.retention.hours, per partition, time