What is the OS page cache and why does it matter for Kafka performance?

The OS page cache is memory the operating system uses to cache file contents. Kafka's design relies on the filesystem and the page cache rather than keeping its own in-process cache, because an in-process cache would duplicate data already held in the page cache and add garbage-collection overhead for the JVM heap. The page cache also stays warm even if the Kafka service is restarted. Recently written data that consumers read can therefore be served from memory instead of disk, and with sendfile the OS can send data from the page cache to the network directly. A common approach is to leave most of the machine's RAM to the OS page cache instead of giving the JVM a very large heap.

Expected key concepts: page cache, RAM, JVM heap, garbage-collection, sendfile, filesystem, warm, restarted, in-process cache, file contents