Kafka ISR Explained: The Hidden Mechanism Behind Write Durability

A deep dive into one of the most important — and most overlooked — concepts in Kafka reliability.

The question nobody asks… until things break

When people start learning Kafka, they usually get comfortable with replication pretty quickly. Leader holds the data. Followers copy it. If the leader dies, a follower takes over.

Seems simple enough, right?

But then… someone asks a question:

Which follower gets elected? And what happens if that follower was behind?

That’s exactly where the ISR comes in — and honestly, this is where most people get it wrong.

What is an ISR?

ISR stands for In-Sync Replica.

It’s not just “any follower.” It’s a specific set of replicas that Kafka considers fully caught up with the leader.

Think of it this way.

You have a partition with one leader and two followers. At any point in time, Kafka is continuously tracking how each follower is behaving.

If a follower is keeping up — replicating messages fast enough — it stays in the ISR.

If it falls too far behind, Kafka removes it from the ISR.

So the ISR is not fixed. It’s a dynamic list. It shrinks and grows depending on how replicas behave.

How does Kafka decide who is “in sync”?

Kafka uses a configuration called replica.lag.time.max.ms. Default is 30 seconds.

If a follower hasn’t sent a fetch request to the leader within that time window, Kafka considers it out of sync and removes it from the ISR.

And here’s something important — it’s not just about how far a follower is behind.

It’s about whether it is actively trying to keep up.

A follower that is slightly behind but continuously fetching data stays in the ISR. A follower that goes silent gets removed.

When the lagging broker catches up again, Kafka adds it back to the ISR automatically.

Why does the ISR exist at all?

Here’s the real reason it matters.

In Kafka, a write is not considered “committed” until all replicas in the ISR have acknowledged it.

Not all replicas — only the ISR.

This is a critical distinction — and this is where things get interesting.

If a follower is slow or down, it leaves the ISR. That means the leader no longer waits for it. The system keeps moving.

But there’s a trade-off.

If the ISR shrinks to just one broker — the leader itself — then a “committed” write exists on just one machine.

And if that machine fails before others catch up… you lose data.

I’ve seen teams assume replication alone guarantees safety — it doesn’t.

And this is exactly why ISR becomes the beating heart of write durability in Kafka.

The acks setting — where ISR becomes real

The ISR directly connects to how you configure your producer.

The acks setting controls what the producer considers a successful write.

There are three options.

acks=0 — the producer doesn't wait for any acknowledgement at all. Maximum throughput, zero durability. Fire and forget.

acks=1 — the producer waits for the leader alone to confirm. Fast — but if the leader crashes before replication, the message is lost.

acks=all (or acks=-1) — the producer waits for all ISR replicas to confirm. This is real durability. The message is acknowledged only when every in-sync replica has it.

**min.insync.replicas — the safety net**

Now here’s where things get a bit tricky.

acks=all alone is not enough.

If the ISR shrinks to just the leader, acks=all still succeeds.

Why? Because the leader is the only ISR member — it just acknowledges itself.

So technically the write is “committed”… but only on one machine.

That’s risky.

This is where min.insync.replicas comes in.

It tells Kafka:

Do not accept writes unless at least N replicas are in the ISR.

For example:

min.insync.replicas = 2

If ISR drops to just 1 broker, the producer will receive a NotEnoughReplicasException.The write is rejected — instead of silently reducing durability.

This is the combination that actually protects you:

acks=all + min.insync.replicas=2

Together, they mean: A write is only accepted when at least 2 in-sync replicas hold the data.

What happens during a leader election?

When a leader broker goes down, Kafka elects a new leader.

But here’s the key point: Only replicas in the ISR are eligible.

Not all replicas.

A follower that fell out of the ISR is not considered — because it may be missing data.

If ISR is healthy, election is clean, and no data is lost.

But if ISR shrinks to zero… Kafka has a tough decision to make.

By default, it waits for an ISR replica to come back.

But if you enable: unclean.leader.election.enable = true

Kafka may elect an out-of-sync replica.

This brings the system back online — but you accept data loss.

It’s a classic trade-off: availability vs consistency.

The recommended production configuration

For most production systems where data loss is not acceptable, this is the combination to use:

replication.factor = 3
min.insync.replicas = 2
acks = all
unclean.leader.election.enable = false

What this gives you:

• At least 2 replicas must acknowledge a write
• One broker can fail without data loss
• No out-of-sync replica can become a leader.

The cost is slightly higher write latency — the producer must wait for 2 brokers to respond instead of 1. In practice, for most systems, this is an acceptable trade-off.

Putting it all together

The ISR is not a background concept.

It’s the mechanism that connects everything:

• replication
• producer acknowledgements
• leader election

Replication gives you copies.

ISR tells you which copies are actually trustworthy.

acks=all ensures writes land on those trustworthy copies.

min.insync.replicas ensures there are enough of them.

Leader election ensures only safe replicas take over.

Remove any one of these — and your durability guarantee has a hole in it.

Final thoughts

Most Kafka beginners think:

replication = reliability

But that’s only half the story.

Real reliability in Kafka comes from:

• knowing which replicas are truly in sync
• ensuring writes reach enough of them
• and making sure only those replicas can become leaders

The ISR is what makes all of that precise.

Once this clicks, you stop thinking of Kafka as just:

“it has replicas so it’s safe”

…and start thinking about what your actual durability guarantee is — and whether your configuration truly delivers it.