Ensuring high availability when a Kafka broker fails unexpectedly comes down to configuring replication and acknowledgment settings ahead of time so the cluster can automatically fail over without data loss.
Key Points: • Set the replication factor to at least 3 across topics so the cluster can tolerate the loss of one or even two brokers. • Configure min.insync.replicas (commonly 2 with replication factor 3) so writes are only acknowledged once a safe number of replicas have them. • Use acks=all on producers to ensure durability guarantees are actually enforced end-to-end. • Rely on Kafka's controller to automatically elect a new leader from the in-sync replicas for any partitions the failed broker was leading. • Monitor for under-replicated partitions and broker health so degraded redundancy is caught and fixed before a second failure causes real unavailability.
Example: With replication factor 3 and min.insync.replicas=2, if one broker in the cluster crashes unexpectedly, the controller promotes an in-sync replica on another broker to leader within seconds, and producers using acks=all continue writing without any data loss.
Interview Tip: A concise interview answer is:
"High availability comes from planning ahead — replication factor of at least 3, min.insync.replicas set to enforce durability, and acks=all on producers — so that when a broker does fail unexpectedly, the controller can automatically elect a new leader from the in-sync replicas with no manual intervention and no data loss."