Producer Deep Dive
Batching, compression, partitioning strategies, retries, and idempotency — from first principles.
0 of 5 lessons complete
- 1Producer Lifecycle & Batchingsend() is asynchronous. Once metadata is available and the record has been serialized and assigned a partition, the reco…
- 2Partitioning StrategiesWith a key, the producer picks the partition from a murmur2 hash of the serialized key modulo the partition count. Witho…
- 3Retries & IdempotencyWhen a produce request fails with a transient error, the producer retries until it succeeds, fails with a non-transient …
- 4Producer TransactionsKafka transactions allow atomic writes across multiple partitions and topics. A transactional producer sets transactiona…
- 5Delivery SemanticsKafka lets you choose. At-most-once: disable producer retries and commit consumer offsets before processing a batch, so …
Knowledge check
Q1.What does enable.idempotence=true prevent?
Q2.What does linger.ms control?
Q3.Without a message key, how does Kafka 2.4+ choose a partition?