IntermediateInternals
What is the difference between Kafka's Fetch API and consumer.poll()?
Reference answer
The Fetch API is the low-level broker RPC: a FetchRequest names the topic-partitions and the offset to start from, plus limits such as the minimum bytes (fetch.min.bytes, default 1) and how long the broker may wait for them (fetch.max.wait.ms, default 500), and the broker answers with a chunk of log from that position. consumer.poll() is the high-level client API built on top of it: it joins the group and takes part in partition assignment, rebalances only happen inside poll, it issues the fetches, and it returns a ConsumerRecords batch; with enable.auto.commit the offsets are committed periodically. Underneath the covers the consumer also sends heartbeats to show it is alive. The Javadoc says poll is designed to ensure liveness: to stay in the group you must keep calling it. If poll is not called again within max.poll.interval.ms (default 300000), the client is considered failed, proactively leaves the group, and the group rebalances.
Expected key concepts: Fetch API, FetchRequest, fetch.min.bytes, consumer.poll, partition assignment, rebalances, heartbeats, ConsumerRecords, max.poll.interval.ms, leaves the group
How did that feel?
Your own verdict, not the score. It shows up next to this question in the list, so you can come back to the shaky ones.