ScenarioProduction Scenarios
How do you migrate a Kafka topic to a new cluster with zero data loss?
Reference answer
Migration procedure: 1) Set up Kafka MirrorMaker 2 (MM2) to replicate source topic to destination cluster. 2) Wait for lag to reach 0 on destination. 3) Pause producers from writing to source. 4) Wait for destination lag to hit 0 one more time. 5) Update consumer groups to use destination cluster (commit matching offsets — MM2 provides offset translation). 6) Update producers to write to destination. 7) Verify everything is working. 8) Decommission MM2 and source topic. Key: MM2 handles offset translation automatically. The switch window (step 3-6) determines RTO — keep it short.
Expected key concepts: MirrorMaker 2, MM2, replicate, source, destination, pause producers, lag=0, offset translation, consumer groups, switch window, RTO, zero data loss