Skip to content
Kafka Is Fun

BeginnerCore Concepts

What is a Kafka record/message structure?

Reference answer

Conceptually a Kafka record (event, message) has a key, a value, a timestamp and optional metadata headers. In the on-disk and wire format, records are written in batches, and each record holds a timestamp delta, an offset delta, a key (byte array), a value (byte array) and a list of headers, each header being a key and a byte-array value. The key and value are just bytes: clients convert objects to and from bytes with a Serializer and a Deserializer. The timestamp is the producer's create time by default (message.timestamp.type=CreateTime) or the broker's append time (LogAppendTime). The topic, partition and offset are not part of the payload but identify where a consumed record lives: the partition within the topic, and the offset as its position in that partition. The key is also what the default partitioner uses to choose a partition.

Expected key concepts: key, value, timestamp, headers, topic, partition, offset, bytes, Serializer, CreateTime, LogAppendTime

How did that feel?

Your own verdict, not the score. It shows up next to this question in the list, so you can come back to the shaky ones.