Share Groups: Queues for Kafka (KIP-932)

Overview

Share groups let several consumers read the same partitions at the same time and acknowledge each record on its own, which gives queue-like consumption on ordinary topics. Kafka does not gain a queue: a share group is a new group type next to consumer groups. In a consumer group a partition belongs to one consumer, so parallelism stops at the partition count. In a share group the consumers can outnumber the partitions. A fetched record is acquired for a limited time (30 seconds by default); the consumer accepts it, releases it for another try, or rejects it, and if it does nothing the lock simply expires. Kafka counts delivery attempts, and once a record reaches the limit (5 by default) it is archived instead of being redelivered forever. Delivery is at-least-once, the count is approximate, and ordering is only guaranteed inside one batch, so redelivered records can arrive out of order. That is why the upgrade notes recommend share groups when records are handled one at a time rather than as an ordered stream. Share groups were early access in 4.0, a preview in 4.1 and are production-ready in 4.2.

Analogy

Key configuration

Using share groups (4.2)properties

# Enabling (4.1 upgrade notes): upgrade the cluster to share.version=1 with kafka-features.sh

# Client: a new class, KafkaShareConsumer
# Per-group setting, default 30s: how long an acquired record is locked
share.record.lock.duration.ms
# Broker: cap on in-flight (acquired) records per share-partition
group.share.partition.max.record.locks

# Internal topic, created on first use, 3 replicas by default.
# With fewer than 3 brokers set these to 1 before first use:
share.coordinator.state.topic.replication.factor=1
share.coordinator.state.topic.min.isr=1

# Tools: kafka-console-share-consumer.sh, kafka-share-consumer-perf-test.sh, kafka-share-groups.sh

Production tip

Check yourself

Why are share groups a poor fit for processing events in strict order?

Sources

Checked against Kafka 4.3 documentation on Oct 2026. Draft: not yet reviewed by a Kafka specialist. Releases move on, so confirm against the current documentation before you act on a number.