In a traditional Kafka deployment, ZooKeeper is the external coordination service that stores and manages the cluster's metadata, including broker membership, topic configuration, and partition leadership.
Key Points: • ZooKeeper tracks which brokers are alive and part of the cluster, using ephemeral nodes that disappear if a broker disconnects. • It stores topic-level metadata such as partition counts, replica assignments, and configuration. • It's involved in electing the controller broker, which in turn uses ZooKeeper to coordinate partition leader elections. • It manages ACLs and quotas in some configurations, depending on how security is set up. • Newer Kafka versions replace ZooKeeper with KRaft, an internal Raft-based controller quorum, removing this external dependency entirely.
Example: When a new broker joins the cluster, it registers itself in ZooKeeper, and the controller broker (itself elected via ZooKeeper) uses that information to assign it partitions and include it in future leader elections.
Interview Tip: A concise interview answer is:
"ZooKeeper acts as Kafka's coordination and metadata store in traditional deployments, tracking broker membership, topic configuration, and driving controller and partition leader elections, though newer Kafka versions have moved to KRaft mode, which replaces ZooKeeper with a built-in Raft-based metadata quorum."