ScenarioProduction Scenarios
A Kafka producer is experiencing high p99 latency. List 5 potential causes and fixes.
Reference answer
1) Metadata fetch: producer lacks cached metadata → pre-warm by calling partitionsFor() at startup. 2) Batch not full: linger.ms too low → increase linger.ms. 3) Buffer full: buffer.memory exhausted → increase buffer.memory, reduce send rate, or profile slow paths. 4) Broker request handler saturated: RequestHandlerAvgIdlePercent < 30% → add broker capacity. 5) Follower replica lag: acks=all waiting for slow ISR member → check replica lag, check broker disk/network. 6) GC pause on producer JVM: Stop-the-World GC → tune G1GC/ZGC, reduce heap, reduce long-lived objects. 7) Network latency between producer DC and broker DC → co-locate or use async production.
Expected key concepts: metadata fetch, linger.ms, buffer.memory, RequestHandlerAvgIdlePercent, follower replica lag, ISR, GC pause, G1GC, ZGC, network latency, co-locate