Skip to content
Kafka Is FunProgress

ScenarioProduction Scenarios

How would you implement an audit log system using Kafka?

Reference answer

Design: 1) Dedicated audit topic: audit.events (key=entityId for compaction, value=AuditEvent Avro). 2) High replication: RF=5, minISR=3 for extra durability. 3) Retention: indefinite or very long (1 year+) via retention.ms=-1 with retention.bytes cap, or log compaction to keep latest state per entity. 4) Producers: all services write audit events synchronously with acks=all + enable.idempotence=true (never lose audit events). 5) Separate consumer groups per audit consumer (compliance, analytics, SIEM). 6) Protect topic with Kafka ACLs — only audit producers can write. 7) Consider Kafka KRaft for metadata durability. 8) Use immutable event schema (never modify, only add fields).

Expected key concepts: audit topic, RF=5, minISR=3, retention=-1, compaction, acks=all, enable.idempotence, ACLs, immutable, compliance, SIEM, Avro, separate consumer groups