How would you handle a Kafka cluster with millions of partitions?

Apache's docs give no partition-count limit or guaranteed ceiling, so treat millions of partitions as a target to load test, not a promise. What the sources do say: KIP-500 moved metadata into a Raft-based quorum (KRaft) specifically to support more partitions with less CPU use, and the follower controllers serve as hot standbys, so controller failover does not need a lengthy metadata reload. ZooKeeper mode was removed in Kafka 4.0, so KRaft is the only option on 4.x. Practical steps: 1) Run dedicated controller nodes (process.roles=controller), typically 3 or 5; the docs say combined mode should be avoided in critical deployments. 2) Size the controllers' memory and metadata log disk; the docs say 5 GB of memory and 5 GB of disk is sufficient for a typical cluster, so verify that for your size. 3) Monitor ActiveControllerCount, GlobalPartitionCount and the per-broker PartitionCount (which should be mostly even across brokers). 4) Avoid creating partitions you do not need and delete unused topics, since every partition adds metadata and per-partition overhead. 5) Load test at the target size before relying on it.

Expected key concepts: KRaft, KIP-500, process.roles, hot standby, controller failover, ActiveControllerCount, GlobalPartitionCount, PartitionCount, unused topics, load test