Skip to content
Kafka Is FunProgress

IntermediatePerformance

How do you prevent consumer lag from growing under high load?

Reference answer

Strategies: 1) Scale consumers (add up to partition count). 2) Increase max.poll.records to process more per batch. 3) Optimize message processing — profile the consumer logic for bottlenecks. 4) Use async processing within consumer (careful with offset commits). 5) Add partitions to enable more consumer parallelism. 6) Use multiple consumers per JVM if processing is I/O-bound. 7) Parallelize within a consumer using thread pools (with careful partition-level offset tracking). 8) For batch consumers: increase fetch.max.bytes and session timeouts appropriately.

Expected key concepts: scale consumers, max.poll.records, async processing, offset commits, partitions, parallel, thread pools, fetch.max.bytes, bottleneck