How does Kafka's design deal with failing nodes and the cost of the network?

Kafka's design assumes a fail/recover model: nodes can suddenly stop working and later recover; it does not handle Byzantine failures where nodes return arbitrary or malicious responses. Each partition is replicated (Kafka is meant to be used with replication by default), with a leader and followers, and when a leader fails the controller elects a new leader, normally from the in-sync replicas, so Kafka stays available after a short fail-over period. Producers choose the trade-off with acks=all and min.insync.replicas, which trades consistency against availability. For network cost, the protocol groups messages into batches (message sets) to amortize the overhead of the network round trip, the producer can accumulate messages up to a fixed number or time bound (linger.ms) to trade a little latency for throughput, and a batch can be compressed as a unit, stays compressed in the log and is decompressed by the consumer. Security also has a cost: sendfile is not used when SSL is enabled.

Expected key concepts: fail/recover, Byzantine, replication, in-sync replicas, acks=all, min.insync.replicas, linger.ms, sendfile