What is the impact of increasing Kafka's JVM heap size?

A larger JVM heap does not make Kafka faster in most cases. Kafka deliberately relies on the filesystem and the OS page cache instead of an in-process cache: the design notes say Java object overhead is high and garbage collection becomes increasingly fiddly and slow as in-heap data increases, whereas the page cache holds a compact byte structure and stays warm across broker restarts. OS memory left free is therefore used for caching log data. The Apache docs show typical broker arguments with a modest 6 GB heap (-Xmx6g -Xms6g) and G1 (-XX:+UseG1GC with -XX:MaxGCPauseMillis=20), and report a 90th percentile GC pause of about 21 ms on one of LinkedIn's busiest clusters. GC pauses matter because a broker that stalls can fall behind: a follower that cannot catch up within replica.lag.time.max.ms (default 30000) is removed from the ISR. The docs recommend monitoring GC time, and a common approach is to keep pauses well below that limit.

Expected key concepts: JVM heap, GC, page cache, OS memory, G1, MaxGCPauseMillis, 6 GB heap, ISR, replica.lag.time.max.ms, monitoring GC time