Apache Kafka is a distributed event streaming platform used to publish, store, and process continuous streams of data records in real time, at high throughput and scale.
Key Points: • Kafka stores records durably in partitioned, ordered logs called topics, which multiple producers and consumers can use independently. • It's designed for horizontal scalability, running as a cluster of brokers that spread data and load across machines. • Producers write records to topics and consumers read from them, often organized into consumer groups for parallel processing. • Kafka is widely used for real-time data pipelines, event-driven architectures, log aggregation, and stream processing.
Example: An online retailer might use Kafka to capture every "item added to cart" event in real time, feeding both a live inventory system and a recommendation engine from the same stream of events.
Interview Tip: A concise interview answer is:
"Apache Kafka is a distributed event streaming platform that lets applications publish and subscribe to continuous streams of data through durable, partitioned topics, and it's built to handle very high throughput with strong durability and horizontal scalability, which is why it's a common backbone for real-time data pipelines."