Kafka topics are partitioned by splitting each topic's log into multiple independent, ordered segments that can be distributed across different brokers and processed in parallel.
Key Points: • Partitioning lets a topic's total throughput scale beyond what a single broker or disk could handle, since partitions are spread across the cluster. • Multiple consumers in a group can each own different partitions, allowing parallel consumption of the same topic. • Partitions are the unit of ordering guarantees — order is preserved within a partition but not across the whole topic. • The partition count sets an upper bound on how many consumers in a single group can actively process data at once.
Example: A topic with 12 partitions spread across 4 brokers lets up to 12 consumers in the same group each process a dedicated slice of the traffic simultaneously, which would be impossible if the topic were a single unpartitioned log.
Interview Tip: A concise interview answer is:
"Topics are partitioned by splitting their log into multiple ordered segments distributed across brokers, which is important because it's what allows Kafka to scale throughput horizontally and lets multiple consumers in a group process the topic in parallel, at the cost of only guaranteeing order within each partition."