How does Kafka achieve low latency with acks=all?

With acks=all the leader waits until all in-sync replicas have the record, yet latency stays low because of several design choices: 1) Appends are sequential disk writes into the OS page cache; Kafka does not fsync every write, since durability comes from replication, and the docs recommend the default flush settings, which disable application fsync. 2) Batching and pipelining: producers group records into batches and can keep several requests in flight per connection (max.in.flight.requests.per.connection, default 5). 3) Followers replicate by fetching from the leader like a normal consumer, so they can batch the log entries they apply. 4) Produce requests waiting for acknowledgements are held in a purgatory, backed by a hierarchical TimingWheel, and are completed when the replicas catch up or the timeout expires. 5) Reads of recent data come from the page cache, and sendfile zero-copy avoids extra copies. The design docs say that waiting for a message to be committed can take on the order of 10 ms; the actual latency depends on the network round trip to the followers, the disks and the configuration, so measure it rather than assume a figure.

Expected key concepts: sequential disk writes, page cache, fsync, batching, pipelining, in flight, fetching, Purgatory, TimingWheel, zero-copy, network round trip