Production Architecture
Sizing, monitoring, security, operations — running Kafka reliably at scale.
0 of 6 lessons complete
- 1Cluster SizingSizing starts from your own numbers: producer throughput, the replication factor (each message is stored on RF brokers, …
- 2Essential Monitoring MetricsKafka exposes metrics through JMX. Ones worth watching, with the exact names from the monitoring page: • kafka.server:ty…
- 3Security: Authentication & AuthorizationSecurity is optional in Kafka. For authentication it supports SSL client certificates and SASL, with the mechanisms GSSA…
- 4Topic Design Best PracticesChoose a naming convention early. Kafka cannot rename a topic (kafka-topics.sh --alter changes partitions and replica as…
- 5Operational Runbook EssentialsRolling restart: the upgrade documentation says to upgrade the brokers one at a time, shutting each down, updating the c…
- 6Dead Letter Queues & TriageA dead letter queue (DLQ) is a topic where records go when a pipeline has given up on processing them. Kafka Connect sin…
Knowledge check
Q1.What does UnderReplicatedPartitions > 0 indicate?
Q2.Which mechanisms does Kafka offer for securing a production cluster?
Q3.During a rolling restart, what is a sensible check before restarting the next broker?
Q4.A Connect sink connector's dead letter queue topic suddenly has 25,000 new records. What is a sensible first move?