A Kafka consumer group rebalance is the process of redistributing topic partitions among the group's consumers whenever membership changes, and it can take a long time because every consumer must stop processing, commit offsets, and wait for a new partition assignment to be computed and applied.
Key Points: • The group coordinator must detect a consumer joining or leaving, which depends on session.timeout.ms and heartbeat.interval.ms — overly conservative values delay detection. • A slow consumer that takes too long between poll() calls can exceed max.poll.interval.ms and be kicked out, triggering yet another rebalance. • The default "stop-the-world" eager rebalance protocol revokes all partitions from all consumers before reassigning them, causing a full pause even for consumers unaffected by the change. • Offset commit synchronization and rejoin round-trips with the coordinator add latency, especially with many partitions or consumers.
Example: If session.timeout.ms is set too high, Kafka waits unnecessarily long to notice a crashed consumer before starting the rebalance, so lowering it (with a matching heartbeat interval) speeds up failure detection without causing false positives.
Interview Tip: A concise interview answer is:
"Rebalances take time mainly because of coordinator detection delay and the stop-the-world nature of the default protocol, so I reduce it by tuning session.timeout.ms and max.poll.interval.ms appropriately, keeping poll loops fast, and switching to the cooperative sticky assignor so only changed partitions are reassigned instead of all of them."