What are the main JMX metrics to monitor for a healthy Kafka cluster?

Essential JMX metrics from the Apache monitoring docs: kafka.server:type=ReplicaManager,name=UnderReplicatedPartitions (should be 0); kafka.controller:type=KafkaController,name=ActiveControllerCount (only one broker in the cluster should report 1); kafka.controller:type=KafkaController,name=OfflinePartitionsCount (a nonzero value means partitions are offline, so it should normally be 0); kafka.server:type=KafkaRequestHandlerPool,name=RequestHandlerAvgIdlePercent and kafka.network:type=SocketServer,name=NetworkProcessorAvgIdlePercent (between 0 and 1, ideally above 0.3); kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec and BytesOutPerSec (throughput); kafka.log:type=Log,name=LogEndOffset (the last offset of each partition). On the consumer side, records-lag-max (consumer-fetch-manager-metrics) is the maximum lag, in records, for any partition of that consumer; on the producer side, buffer-available-bytes shows unused buffer memory. All of these are exposed over JMX. The docs also recommend monitoring GC time and CPU, and watching consumer max lag against a threshold.

Expected key concepts: UnderReplicatedPartitions, ActiveControllerCount, OfflinePartitionsCount, RequestHandlerAvgIdlePercent, NetworkProcessorAvgIdlePercent, BytesInPerSec, LogEndOffset, records-lag-max, buffer-available-bytes, JMX