Skip to content
Kafka Is FunProgress

ScenarioProduction Scenarios

How do you safely add a new topic and deploy a new consumer without downtime?

Reference answer

Strategy: 1) Create topic first (kafka-topics.sh or KafkaAdmin NewTopic bean). 2) Deploy producer updates — they start writing to new topic. 3) Deploy consumer with appropriate auto-offset-reset=earliest (to read all messages from start). 4) Verify consumer lag reaches 0. 5) Monitor for errors. For zero-lag consumer start with new topic: if producer writes before consumer starts, consumer will replay from start with earliest. If you want consumers to only read live messages: deploy consumer first (it will subscribe with latest), then producer. Use @KafkaListener with missing-topics-fatal=false so service starts even if topic not yet created.

Expected key concepts: kafka-topics.sh, KafkaAdmin, auto-offset-reset, earliest, latest, zero-lag, consumer lag, deploy producer first, deploy consumer first, missing-topics-fatal