Skip to content
Kafka Is Fun

SpringSpring Kafka Advanced

How do you configure Spring Kafka consumer groups for multi-tenancy?

Reference answer

Kafka delivers each message to one process in each consumer group, so separate consumer groups give separate consumers independent progress. A common naming convention for tenant isolation is group.id = tenant-{tenantId}-{service-name}, set per listener with the groupId attribute. For tenants that have their own topics, a @KafkaListener can subscribe by regular expression with topicPattern, for example @KafkaListener(topicPattern = 'tenant-.*-orders'), which maps to the consumer's subscribe(Pattern); matching topics are checked periodically against the topics existing at the time of check (governed by metadata.max.age.ms), so a new tenant topic is picked up on a later metadata refresh. missingTopicsFatal does not apply to pattern subscriptions. Another approach is one shared topic where the listener routes each record to tenant-specific processing, for example by key or header. For complete isolation with a separate cluster per tenant, define multiple ConsumerFactory/ProducerFactory beans with tenant-specific configuration and multiple KafkaListenerContainerFactory beans, then select one with the containerFactory attribute.

Expected key concepts: group.id, tenant isolation, topicPattern, metadata.max.age.ms, KafkaListenerContainerFactory, containerFactory, ProducerFactory, ConsumerFactory

How did that feel?

Your own verdict, not the score. It shows up next to this question in the list, so you can come back to the shaky ones.