What causes consumer rebalancing and how can you minimize it?

With the classic group protocol, a rebalance is triggered when a new member joins, when a member shuts down or fails (it stops heartbeating for session.timeout.ms, or does not call poll() within max.poll.interval.ms so the client leaves the group), when the number of partitions of a subscribed topic changes, and when a subscribed topic is created or deleted (relevant for pattern subscriptions). Ways to reduce them: 1) Static membership: set a unique group.instance.id per consumer, so a restarting member (for example after a pod restart) that rejoins in time keeps its assignment without a rebalance; pair it with a larger session.timeout.ms. 2) CooperativeStickyAssignor, which is in the default partition.assignment.strategy [RangeAssignor, CooperativeStickyAssignor] (Range is used until you change the order), uses cooperative rebalancing and only revokes the partitions that must move. 3) Keep session.timeout.ms (default 45000) longer than pauses such as long GC. 4) Keep max.poll.interval.ms (default 300000) longer than your worst-case processing time between polls, or lower max.poll.records (default 500). 5) heartbeat.interval.ms (default 3000) must be lower than session.timeout.ms and is recommended to be no more than a third of it. 6) With group.protocol=consumer (KIP-848, generally available in Kafka 4.0) the coordinator computes the assignment and members reconcile incrementally, so a consumer whose assignment does not change is not disturbed.

Expected key concepts: static membership, group.instance.id, CooperativeStickyAssignor, cooperative rebalancing, session.timeout.ms, max.poll.interval.ms, max.poll.records, heartbeat.interval.ms, group.protocol, GC