Skip to content
Kafka Is Fun

HighData Quality

Schema Incompatibility

Symptoms

  • Consumers throw RecordDeserializationException (a subclass of SerializationException) when polling
  • Schema Registry answers a registration with 409 Conflict (incompatible schema)
  • A consumer that cannot deserialize a record keeps failing on it, so its lag grows

Detection

Schema Registry: GET /subjects/{subject}/versions lists the registered versions, and GET /config/{subject} returns the subject's compatibility level (an error means no subject-level setting; GET /config returns the global one). In the consumer, look for RecordDeserializationException.

Challenge mode: diagnose the root cause from the symptoms before reading the answer.

Root cause

A producer started writing a new schema version that readers cannot handle. Schema Registry compares each new version against the previous one using the subject's compatibility type (BACKWARD by default) and refuses an incompatible one, but a producer that bypasses the check, or a compatibility type that does not match the order in which you upgrade, can still leave consumers unable to read what is on the topic.

Immediate actions

  1. Roll back the producer to the last schema consumers can read
  2. Check the compatibility level: under BACKWARD upgrade consumers before producers, under FORWARD upgrade producers first, under FULL either order
  3. In a Spring application, ErrorHandlingDeserializer turns a deserialization failure into a null value plus an exception header, so the consumer is not stuck on the record; the error handler and dead-letter publishing then deal with it

Permanent fix

Choose the compatibility type that matches your upgrade order. BACKWARD (the default) means consumers using the new schema can read data written with the previous schema, so upgrade consumers first. Under it you can add a field with a default value or remove a field, but a new field without a default is incompatible.

Prevention

Keep compatibility enforcement on. Remember BACKWARD does not promise that consumers still on an older schema can read new data: that is FORWARD, and FULL gives both directions. Test schema changes against real consumers.

Reproduce it in the simulator

This failure is at the application layer, not in the simulator. The Spring Boot topic on Avro and Schema Registry shows the configuration.

Open the simulator to try it.

The interview angle

BACKWARD compatibility means consumers using the new schema can read data produced with the previous schema, so you deploy consumers first. It says nothing about older consumers reading new data; that is FORWARD, and both directions is FULL.

Sources

Checked against Kafka 4.2 documentation on 11 October 2026. Draft: not yet reviewed by a Kafka specialist, and releases move on, so confirm against the current documentation before you act on it.