Consumer lag is the gap between the latest offset produced to a partition and the offset a consumer has processed, and closing that gap requires diagnosing whether the limitation is parallelism, application efficiency, or configuration.
Key Points: • Scale out consumers within the group, up to the number of partitions available, to process more partitions in parallel. • Review and optimize the consumer's processing code, since a slow downstream dependency (database, external API) is a frequent hidden bottleneck. • Increase the topic's partition count if the current number of partitions is the ceiling on how many consumers can usefully run in parallel. • Tune fetch.min.bytes and fetch.max.wait.ms to improve batching efficiency between broker and consumer. • Confirm consumers have adequate CPU, memory, and network bandwidth, since resource starvation can throttle processing regardless of code or config.
Example: A consumer group falling behind on a high-volume clickstream topic might diagnose the root cause as a synchronous downstream call per record, and fix it by batching writes to the downstream system, which recovers throughput faster than simply adding more consumer instances.
Interview Tip: A concise interview answer is:
"I'd first measure whether lag is caused by insufficient parallelism, slow processing logic, or a resource bottleneck, and address it in that order — scaling consumers up to the partition count, optimizing the hot path in the code, adding partitions if needed, and tuning fetch settings to improve batching."