Your application requires exactly-once processing semantics. How do you configure Kafka to achieve this?

Achieving exactly-once processing semantics in Kafka requires combining an idempotent producer, transactional writes, and offset commits that are tied to the same transaction as the processing output.

Key Points: • Set enable.idempotence=true on the producer so retried sends can't create duplicate records on the broker. • Assign the producer a transactional.id and wrap related writes in beginTransaction()/commitTransaction() so a group of writes either all succeed or all fail together. • Use the transactional producer's sendOffsetsToTransaction() to commit consumer offsets as part of the same transaction as the produced output, tying "read" and "write" together atomically. • For stream processing specifically, Kafka Streams offers this out of the box via processing.guarantee=exactly_once_v2, without needing to hand-roll the transactional API. • Consumers reading transactional output should set isolation.level=read_committed so they only see records from committed transactions.

Example: A service that reads payment events, computes a balance update, and writes the result back to another topic can wrap the read-process-write cycle in a Kafka transaction so that if the process crashes mid-way, the partial output is never visible to downstream consumers and the offset isn't committed either.

Code Example:

Properties props = new Properties();
props.put("enable.idempotence", "true");
props.put("transactional.id", "balance-updater-1");

KafkaProducer<String, String> producer = new KafkaProducer<>(props);
producer.initTransactions();

try {
    producer.beginTransaction();
    producer.send(new ProducerRecord<>("balances", key, updatedBalance));
    producer.sendOffsetsToTransaction(offsets, consumerGroupId);
    producer.commitTransaction();
} catch (Exception e) {
    producer.abortTransaction();
}

Interview Tip: A concise interview answer is:

"I'd enable idempotence on the producer, use a transactional producer with a transactional.id to wrap the produce and offset commit in one atomic transaction via sendOffsetsToTransaction, and set consumers to read_committed isolation, or simply use Kafka Streams' exactly_once_v2 guarantee if the workload is a stream processing job."