Skip to content
Kafka Is Fun

AdvancedInternals Deep Dive

Explain the Kafka consumer's heartbeat threading model and how it relates to max.poll.interval.ms.

Reference answer

With the classic group protocol, a consumer sends heartbeats to the Group Coordinator from a background thread at heartbeat.interval.ms (default 3000 ms), while the application thread calls poll(), which fetches records and handles rebalancing (KIP-62). The coordinator removes a member whose heartbeats stop for session.timeout.ms (default 45000 ms). max.poll.interval.ms (default 300000 ms) is a separate, locally enforced limit on the delay between calls to poll(): if it is exceeded, the consumer stops sending heartbeats and sends an explicit LeaveGroup request, so the group rebalances and reassigns its partitions even though heartbeats were healthy. This guards against a livelock, where a consumer keeps heartbeating but makes no progress on its records. The consumer rejoins the group on its next poll(). With the newer group.protocol=consumer, heartbeat.interval.ms and session.timeout.ms are not used; the broker-side group.consumer.* settings control them.

Expected key concepts: background thread, Group Coordinator, heartbeat.interval.ms, poll(), max.poll.interval.ms, session.timeout.ms, LeaveGroup, livelock, locally enforced, rejoin

How did that feel?

Your own verdict, not the score. It shows up next to this question in the list, so you can come back to the shaky ones.