ScenarioProduction Scenarios
What's your strategy for monitoring Kafka in production with 100+ topics?
Reference answer
Monitoring stack: 1) JMX Exporter → Prometheus → Grafana for all broker metrics. 2) Kafka Exporter for consumer lag metrics. 3) Alert rules: URP > 0 (PagerDuty), OfflinePartitions > 0 (critical), consumer lag growing > N min (per SLA), disk > 70%, RequestHandlerIdle < 30%, BytesIn/Out anomaly detection. 4) Per-topic dashboards: BytesIn, consumer lag per group, URP. 5) Use Confluent Control Center or custom Grafana for overview. 6) Implement dead man's switch: if audit topic has no writes for N seconds, alert. 7) Track topic creation/deletion via __cluster_metadata changes. 8) Monthly capacity planning review.
Expected key concepts: JMX Exporter, Prometheus, Grafana, Kafka Exporter, consumer lag, URP, OfflinePartitions, disk 70%, RequestHandlerIdle, dead man's switch, capacity planning, Control Center