Top 10 Kafka Interview Questions for Senior Java Developers

1. Explain Kafka’s Exactly-Once Semantics

Q: How does Kafka achieve exactly-once processing? A: Idempotent producers (enable.idempotence=true) + Transactions (transactional.id). Producer assigns PID/epoch, broker deduplicates based on sequence numbers. Consumers use isolation.level=read_committed to see only committed records.

producer.initTransactions(); producer.beginTransaction(); producer.send(record1); producer.send(record2); producer.commitTransaction();

2. Consumer Rebalancing & Strategies

Q: What happens during consumer rebalance? How to optimize? A: Range (even partition distribution), RoundRobin (merge all assignments), CooperativeSticky (minimize movement). Use session.timeout.ms=30s, max.poll.interval.ms=5m, partition.assignment.strategy=cooperative-sticky. Static membership (static.member.group.id) avoids rebalances.

3. ISR vs MinISR

Q: Difference between ISR and min.insync.replicas? A: ISR = In-Sync Replicas (current healthy followers). min.insync.replicas=2 means producer requires 2+ ISR (acks=all) for write success. If ISR < minISR → NOT_ENOUGH_REPLICAS error. Leader election only from ISR.

4. Spring Kafka @Transactional

Q: How does Spring Kafka handle transactions? A: KafkaTransactionManager + @Transactional coordinates producer sends + consumer commits. Listener rollback on exceptions. Isolation: isolation.level=read_committed.

@KafkaListener(topics=”orders”) @Transactional public void process(Order order) { // Produces to “shipments” + commits offset atomically }

5. Lag Monitoring & Backpressure

Q: How to detect/handle consumer lag? A: Burrow + Kafka Lag Exporter → Prometheus → Grafana. Max lag threshold → scale consumers or pause producers. KStreams: max.poll.records=100, fetch.max.wait.ms=500. Dead Letter Topics for poison messages.

6. Schema Evolution

Q: How to handle schema changes in production? A: Confluent Schema Registry + Avro/Protobuf. Basic compatibility (forward/backward), Full transitive. Code:

*@Bean public ProducerFactory<String, Order> producerFactory() { Map<String, Object> config = new HashMap<>(); config.put(“value.serializer”, KafkaAvroSerializer.class.getName()); config.put(“schema.registry.url”, “http://localhost:8081"); return new DefaultKafkaProducerFactory<>(config); }*

7. Partitioning Strategy

Q: How to choose partition key for order processing? A: customerId (guarantees ordering per customer), Composite key (customerId_orderType). Sticky partitioning for low-latency. Custom partitioner for complex business logic.

public int partition(String topic, Order order, byte[] keyBytes, byte[] valueBytes, int numPartitions) { return Math.abs(order.getCustomerId().hashCode()) % numPartitions; }

8. KRaft vs ZooKeeper

Q: Why migrate from ZooKeeper to KRaft? A: KRaft = Kafka Raft (Kafka-native metadata). No external ZooKeeper → 50% faster controller election, simpler ops, smaller clusters. Backward compatible till Kafka 4.0. Production ready since 3.3+.

9. Compaction & Log Cleanup

Q: When to use cleanup.policy=compact vs delete? A: compact: Key-based retention (latest value per key). Use cases: KTables, CDC, config topics. delete: Time/size-based (normal topics). Log segments → size=1GB, segment.ms=7days.

10. Your_app Production Config

Q: Kafka config for 10k RPS banking platform? A: Producer: batch.size=64KB, linger.ms=10, compression=lz4, acks=all, retries=Integer.MAX Consumer: max.poll.records=500, fetch.min.bytes=1MB, session.timeout=30s, enable.auto.commit=false Topic: replication=3, min.insync=2, partitions=20/customer shard, retention=7d Monitoring: JMX → Prometheus → Grafana (lag, under-replicated, ISR shrink)

99.99% durability = acks=all + min.insync.replicas=2 + replication.factor=3.