Kafka's architecture is made up of a small set of cooperating components — producers, consumers, brokers, topics/partitions, and a metadata layer — that together let it store and move large volumes of data reliably.
Key Points: • Producers are client applications that publish records to topics. • Consumers are client applications that subscribe to topics and read records, often organized into consumer groups. • Brokers are the Kafka servers that store partition data on disk and serve client requests; a cluster is made up of many brokers. • Topics are named logs that organize records, and each topic is split into partitions distributed across brokers for scalability. • ZooKeeper (or the KRaft controller quorum in newer versions) manages cluster metadata such as broker membership and partition leadership.
Example: In a typical setup, a web application's producer publishes "page-view" events to a topic with several partitions spread across three brokers, while a separate analytics service's consumer group reads from that same topic to build real-time dashboards.
Interview Tip: A concise interview answer is:
"Kafka's architecture consists of producers that publish data, brokers that store it in partitioned topics, consumers that read it, and a metadata layer — historically ZooKeeper, now KRaft in newer versions — that tracks broker and partition state, all working together to move large volumes of data reliably."