Overview
A partition's directory holds segment files named by the base offset of the first record they contain, for example 00000000000000000000.log. Each segment has a .log file with the records, an .index file (a sparse offset-to-file-position index) and a .timeindex file (a sparse timestamp-to-offset index). A .txnindex file for aborted transactions can also exist, and producer-state .snapshot files sit in the same directory. To read offset N, Kafka finds the segment whose base offset is at or below N, then binary-searches that segment's sparse index for the greatest entry at or below N and reads from that file position.
Key configuration
log.index.interval.bytes=4096 — roughly one index entry per 4096 bytes (topic: index.interval.bytes)
log.index.size.max.bytes=10485760 — maximum index file size (topic: segment.index.bytes)Production tip
Check yourself
A consumer requests offset N. How does Kafka find it in a segment?
Sources
- Log implementation (4.2 documentation) (opens in a new tab)
- LogFileUtils.java (4.2 source) (opens in a new tab)
- OffsetIndex.java (4.2 source) (opens in a new tab)
- Topic configs (4.2 documentation) (opens in a new tab)
Checked against Kafka 4.2 documentation on Oct 2026. Draft: not yet reviewed by a Kafka specialist. Releases move on, so confirm against the current documentation before you act on a number.