Production Architecture
Sizing, monitoring, security, operations — running Kafka reliably at scale.
- 1Cluster SizingThe key inputs for cluster sizing: throughput (MB/s producer), replication factor (× RF for replica traffic), retention …
- 2Essential Monitoring MetricsCritical JMX metrics every Kafka team must monitor: • UnderReplicatedPartitions > 0 → replication problem • ActiveContro…
- 3Security: Authentication & AuthorizationKafka supports SASL (PLAIN, SCRAM-SHA-256, GSSAPI/Kerberos, OAUTHBEARER) for authentication and TLS for encryption in tr…
- 4Topic Design Best PracticesName topics descriptively: {domain}.{entity}.{event-type} e.g. orders.order.created. Avoid broad topics with mixed schem…
- 5Operational Runbook EssentialsKey operational procedures: Rolling restart (upgrade brokers one at a time, wait for ISR to recover between each). Prefe…
Knowledge check
Q1.What does UnderReplicatedPartitions > 0 indicate?
Q2.What is the recommended approach to Kafka security in production?
Q3.During a rolling restart, what should you verify before restarting the next broker?