ZooKeeper is critical to a traditional Kafka deployment because it's the shared source of truth for cluster metadata, coordinating which brokers exist, which broker leads each partition, and how the cluster reacts to changes.
Key Points: • It maintains the authoritative list of live brokers, detecting failures quickly via session timeouts on ephemeral nodes. • It coordinates leader election for both the controller broker and individual partitions, which is essential for the cluster to recover from failures. • It stores topic configuration and partition assignment metadata that all brokers rely on to behave consistently. • Without a healthy ZooKeeper, the cluster can't safely add brokers, change partition leadership, or apply configuration changes. • Its criticality — and the operational burden of running a separate ZooKeeper ensemble — is the main motivation behind Kafka's move to KRaft mode, where the cluster manages its own metadata internally.
Example: If ZooKeeper's quorum loses majority (e.g., two of three nodes go down), Kafka's controller can no longer safely coordinate leader elections, so any subsequent broker failure would leave partitions without a new leader until ZooKeeper's quorum is restored.
Interview Tip: A concise interview answer is:
"ZooKeeper is critical because it's the coordination layer that keeps every broker's view of the cluster consistent — broker membership, partition leadership, and topic metadata all flow through it — which is exactly why running it reliably (or migrating to KRaft mode) matters so much for cluster stability."