What strategies would you use if your Kafka messages are larger than the default size limit (1 MB) and causing performance issues?

When Kafka messages exceed the default 1 MB limit and hurt performance, the two main strategies are raising the configured message size limits or redesigning the producer to split large payloads into smaller chunks.

Key Points: • Increase message.max.bytes on the broker/topic and max.request.size on the producer (and fetch.message.max.bytes / max.partition.fetch.bytes on consumers) to actually allow larger messages end-to-end. • Raising the limit everywhere increases memory pressure and can hurt overall throughput, so it should be a deliberate, tested trade-off, not a default. • Alternatively, split large payloads into smaller chunks at the producer, tag them with a sequence identifier, and reassemble them on the consumer side. • A common pattern is the "claim check": store the large payload in external storage (e.g. S3) and publish only a reference/key on the Kafka topic.

Example: A service producing large image-processing payloads might switch to storing the actual file in object storage and publishing just the object's URL and metadata on the Kafka topic, keeping messages small while still giving consumers access to the full data.

Code Example:

# broker/topic config
message.max.bytes=5242880

# producer config
max.request.size=5242880

# consumer config
max.partition.fetch.bytes=5242880

Interview Tip: A concise interview answer is:

"I'd first ask whether the payload really needs to be in Kafka at all — often the better fix is a claim-check pattern where the large data goes to external storage and only a reference goes on the topic; if the message genuinely needs to stay in Kafka, I raise message.max.bytes and matching producer/consumer limits, understanding the throughput trade-off."