How do you implement exactly-once delivery in a Kafka-to-database pipeline?

Kafka's exactly-once features (idempotent producer, transactions, read_committed consumers) cover reading and writing within Kafka. For another destination system the Apache docs say exactly-once generally requires cooperation with that system, and they recommend letting the consumer store its offset in the same place as its output, because many output systems do not support two-phase commit. Options: 1) Offsets in the database: set enable.auto.commit=false, write the results and the topic, partition and offset in one database transaction, and on startup or rebalance seek to the stored offsets. 2) Idempotent writes: make a repeated write harmless, for example an upsert keyed by a business key, an event ID, or topic plus partition plus offset, so at-least-once redelivery has no visible effect. 3) If the pipeline is consume-transform-produce within Kafka, use transactions with isolation.level=read_committed (or Kafka Streams with exactly_once_v2). 4) In Spring, a Kafka transaction can be synchronized with a database transaction using @Transactional, but these are two separate commits; if the second fails after the first succeeded the exception is thrown and you must compensate. ChainedKafkaTransactionManager has been deprecated since Spring Kafka 2.7. 5) For the opposite direction (database to Kafka), a common approach is the outbox pattern: write the event to an outbox table in the same database transaction and publish it with change data capture such as Debezium; consumers still need to tolerate duplicates.

Expected key concepts: exactly-once, enable.auto.commit, upsert, idempotent, read_committed, two-phase commit, @Transactional, ChainedKafkaTransactionManager, outbox pattern, Debezium