Why can a handful of failures flood a dead letter queue with thousands of messages?

By default a Kafka Connect connector fails fast: any error in conversion or in a transformation fails the task. When you set errors.tolerance=all, the connector skips the problematic records instead, and if errors.deadletterqueue.topic.name is set, each failed record is written to that dead letter queue topic (DLQ settings apply to sink connectors). So one root cause, such as a schema change, a misconfigured converter or a broken transform, is not one failure: it hits every record that goes through the same step, and every one of those records becomes a DLQ record. A large DLQ can therefore reflect only a few causes. A common approach is not to read everything: sample some records, enable the context headers (errors.deadletterqueue.context.headers.enable=true) so each record carries the exception class and message, group by exception class, and fix the cause.

Expected key concepts: dead letter queue, DLQ, errors.tolerance, errors.deadletterqueue.topic.name, context headers, exception class, root cause, Kafka Connect, sink connectors, fails fast