How does Kafka handle message schema evolution, and what do teams do about it?

Apache Kafka itself does not manage schemas: a record's key and value are byte arrays, and client-side Serializer and Deserializer classes (key.serializer, value.serializer and their deserializer counterparts, or Kafka Connect converters) turn objects into bytes and back. Brokers do not check schema compatibility, so evolution is the responsibility of the applications and their tooling. A common approach is to use a format such as Avro, Protobuf or JSON Schema together with a schema registry that enforces compatibility rules. The usual modes are backward compatibility (readers on the new schema can read data written with the old one), forward compatibility (readers on the old schema can read new data) and full compatibility (both); these are conventions of schema tools, not something Apache Kafka defines. Common practices: add only optional fields with defaults, avoid removing or renaming fields, and for a breaking change publish to a new versioned topic and migrate the consumers. Because the log is retained, consumers can meet old and new versions of a record at the same time, so readers must tolerate both.

Expected key concepts: byte arrays, Serializer, Deserializer, converters, schema registry, backward compatibility, forward compatibility, optional fields, defaults, versioned topic