CriticalConsumer Failure
Consumer Offset Desync / Message Loss
Symptoms
- Fewer records are processed than were written: some seem to be missing
- After a restart a consumer begins at an offset you did not expect
- Gaps in business metrics
Detection
Compare the number of records on the topic with the number processed downstream. kafka-consumer-groups.sh --describe shows the committed offset against the log end offset for each partition.
Challenge mode: diagnose the root cause from the symptoms before reading the answer.
Root cause
The consumer's committed offset moved past records it had not finished processing. Saving the position before processing gives at-most-once delivery: after a crash the unprocessed records are not read again. With enable.auto.commit=true the offset is committed in the background, and if the application does not process everything poll() returned before the next poll, the committed offset can get ahead of what was consumed and records are missed. Committing in a finally block, or before asynchronous work completes, has the same effect.
Immediate actions
- Stop the consumer so the offset stops advancing
- Work out the last offset that was processed, not just consumed
- Reset the group to it: kafka-consumer-groups.sh --reset-offsets prints a dry run by default and applies the change only with --execute, and the group must have no active members
- Fix the commit logic before restarting the consumer
Permanent fix
Turn off auto-commit and commit only after processing has succeeded (commitSync or commitAsync). In Spring Kafka the container sets enable.auto.commit to false unless you set it, and the manual acknowledgement modes (MANUAL, MANUAL_IMMEDIATE) let the listener acknowledge after processing.
Prevention
Processing first and saving the position after gives at-least-once delivery, so a record may be read twice after a failure: make processing tolerate that.
Reproduce it in the simulator
In the Consumer tab, turn auto-commit off for a group, send a message and see it read but not committed, then Crash the consumer that read it: the new owner starts from the committed offset. The guided scenario "Crash before committing" walks through it.
Open the simulator to try it.
The interview angle
What decides whether a failure loses records or repeats them is the order of processing and committing: commit first and a crash loses records (at-most-once), process first and a crash repeats them (at-least-once).
Sources
- Design (4.2 documentation) (opens in a new tab)
- Consumer configs (4.2 documentation) (opens in a new tab)
- KafkaConsumer Javadoc (4.2) (opens in a new tab)
- Basic Kafka operations (4.2 documentation) (opens in a new tab)
- ConsumerGroupCommand.java (4.2 source) (opens in a new tab)
- Message listener containers (Spring for Apache Kafka) (opens in a new tab)
Checked against Kafka 4.2 documentation on 11 October 2026. Draft: not yet reviewed by a Kafka specialist, and releases move on, so confirm against the current documentation before you act on it.