When a Kafka partition leader fails, Kafka detects the failure and promotes one of the partition's in-sync replicas to become the new leader so reads and writes can continue with minimal disruption.
Key Points: • Only replicas in the in-sync replica (ISR) set — those fully caught up with the old leader — are eligible for election, which prevents promoting a replica with stale data. • The controller broker (coordinated historically via ZooKeeper, or via the KRaft controller quorum in newer Kafka versions) detects the leader's failure and performs the election. • Producers and consumers automatically discover the new leader through metadata refreshes and redirect their requests to it. • If no replica is in the ISR when the leader fails, Kafka either waits (safer) or, if unclean.leader.election.enable=true, elects an out-of-sync replica and risks data loss.
Example: If the broker hosting the leader for partition 2 of the "payments" topic crashes, a follower already in the ISR is promoted to leader within seconds, and producers writing to that partition experience only a brief retryable error before continuing normally.
Interview Tip: A concise interview answer is:
"When a partition leader fails, the controller elects a new leader from the in-sync replicas, and clients transparently discover and switch to it; with unclean leader election disabled, Kafka favors consistency and only promotes replicas that were fully caught up, even if that means a brief unavailability."