IntermediateInternals
What is the difference between Kafka's request queue and the Purgatory?
Reference answer
The request queue holds incoming ProduceRequests waiting for handler threads to process them. RequestHandlerAvgIdlePercent < 30% indicates the queue is saturated. The Purgatory (Kafka's deferred processing subsystem using a TimingWheel) holds requests waiting for remote conditions: acks=all ProduceRequests wait here until all ISR replicas fetch the data, and FetchRequests with fetch.min.bytes wait here until enough data accumulates. If a request doesn't complete within request.timeout.ms, it's expired from Purgatory and an error is returned.
Expected key concepts: request queue, Purgatory, TimingWheel, ProduceRequests, ISR replicas, FetchRequests, fetch.min.bytes, request.timeout.ms, handler threads