Producers batch messages before sending to reduce network overhead and improve throughput. linger.ms controls how long to wait for more messages to fill a batch. batch.size sets the max batch bytes. Kafka supports GZIP, Snappy, LZ4, and ZSTD compression applied per-batch. Compression happens at the producer, is stored compressed, and decompressed at the consumer — saving both network and disk.
Key configuration
linger.ms=5 — wait 5ms to fill batches
batch.size=16384 — 16KB batch (default)
compression.type=lz4 — fast compression