What is the __transaction_state topic and how does it work?

__transaction_state is an internal compacted topic storing the state of all active transactions. The Transaction Coordinator (a role on one broker) is authoritative for each transactional.id, determined by hash(transactional.id) % num.transaction.state.log.partitions. When beginTransaction() is called, a record is written to __transaction_state. Each sent message's batch header includes the PID, epoch, and sequence. On commitTransaction(), a commit marker is appended to each affected partition AND a final commit record is written to __transaction_state. Crash recovery replays this log to complete or abort incomplete transactions.

Expected key concepts: __transaction_state, Transaction Coordinator, transactional.id, PID, epoch, sequence, commit marker, crash recovery, replay, abort, incomplete transactions