Consumer Deep Dive
Consumer groups, rebalancing protocols, offset management, and consumer lag mastery.
- 1Consumer Groups & Partition AssignmentA consumer group is a set of consumers that jointly consume a topic. Each partition is assigned to exactly one consumer …
- 2The Rebalance ProtocolWhen group membership changes (consumer joins/leaves/crashes), the Group Coordinator broker triggers a rebalance. In the…
- 3Offset ManagementConsumer offsets are committed to the __consumer_offsets internal Kafka topic. enable.auto.commit=true commits offsets p…
- 4Consumer Lag & MonitoringConsumer lag = (partition LEO) - (consumer committed offset). Zero lag means the consumer is caught up. Growing lag mean…
- 5Cooperative & Static MembershipStatic membership (group.instance.id) lets a consumer rejoin its group with the same partition assignments after a resta…
Knowledge check
Q1.What is consumer lag?
Q2.What does auto.offset.reset=earliest do for a new consumer group?
Q3.Why is group.instance.id useful in Kubernetes?