Kafka Consumer Offsets — Visually Explained

What every developer must know before touching an offset reset — and the production incident that taught us why.

One of our production releases accidentally flooded a downstream system with 7 days of old Kafka data.

The root cause?

During a refactor, the consumer group ID was changed. Kafka treated it as a brand-new consumer group and started reading from the earliest offsets in the topic.

The downstream system wasn’t designed to handle that load. We had to quickly reset offsets to latest and later republish the required data in a controlled way.

That incident taught us an important lesson: if you don’t understand how Kafka offsets work, production will eventually teach you — and it won’t be gentle. Misunderstand them and you’re looking at duplicate processing, unexpected replays, spiraling lag, or silent data loss.

In this article, we’ll break down Kafka consumer offsets visually, from first principles to reset strategies, so you’re prepared before production gets the chance.

What is an Offset ?

Before we can understand what went wrong in our incident, we need to build a shared vocabulary. And it starts with a number most developers think they understand — until production proves otherwise

People think there is only one offset. There is not.

Kafka consumers usually deal with three different offset positions at the same time. And once you see them visually, most Kafka behavior suddenly starts making sense.

Kafka stores messages inside topics. And each topic is divided into smaller units called partitions.

A Kafka partition is just an ordered sequence of messages.

Every new message gets a sequential number when it arrives. That number is called the offset.

An offset is simply a message’s position number in the partition — nothing more, nothing less.

Topic, Partition and Offsets

The three offset positions

A Kafka consumer usually tracks three different positions at the same time.

There are three offset positions every Kafka developer needs to keep in their head.

The Log-End Offset is where the partition currently stands — the position of the latest message written to it. It keeps moving forward as producers write new messages.

The Current Offset is where your consumer is reading right now in this session. It advances as your consumer polls and processes messages — but it exists only in memory.

The Committed Offset is the one that actually matters. It’s where your consumer last said “I’ve safely processed everything up to here.” Think of it as a checkpoint — the position Kafka remembers on your behalf, even if your consumer crashes and restarts.

3 offsets

Consumer Groups and Offset Tracking

At this point, one question naturally comes up: Where does Kafka actually store committed offsets?

The answer surprises many people the first time they learn it. Kafka stores consumer offsets inside a special internal topic called:

__consumer_offsets

Yes — offsets are themselves stored in Kafka.

Every time a consumer commits an offset, Kafka writes a small metadata record into this internal topic __consumer_offsets.

That record contains things like: topic, partition, committed offset keyed by consumer group ID.

This is how Kafka remembers where a consumer group last stopped.

If you have ever needed to track what the producer last wrote — rather than what the consumer last read — that is a different problem entirely. I covered the producer checkpoint pattern in an earlier article: [Reading the Last Written Offset in Kafka — A Producer Checkpoint Pattern]

The group coordinator

But Kafka does not let consumers directly coordinate all of this themselves.

That responsibility belongs to something called the: Group Coordinator

The group coordinator is responsible for partition assignment, consumer membership, heartbeats, rebalancing, and tracking committed offsets. Think of it as the manager for your consumer group.

What happens when a new consumer group ID is introduced

This is exactly what happened to us.

During a refactor, our team renamed the consumer group ID from order-processor-v1 to order-processor-v2. It felt like a minor configuration change.

One line in a properties file. But to Kafka, order-processor-v2 was a completely unknown entity. There was no entry for it anywhere in __consumer_offsets. So when the new consumer started up and asked the group coordinator “where did I last commit?”, the coordinator had no answer. In that situation, Kafka falls back to a configuration property called auto.offset.reset. In our case it was set to earliest — which means Kafka instructed our brand new consumer group to start reading from the very beginning of the topic. Seven days of order events, replayed in full, straight into a downstream system that had no idea what was coming. The diagram below shows what changed — and more importantly, what didn’t. The partition, the data, the topic — all identical. The only thing that changed was the group ID. That was enough.

Production Issue due to Kafka Consumer Group ID change

The lesson is not that renaming group IDs is wrong. Sometimes it is the right call. The lesson is that when you introduce a new group ID, you must immediately set its starting offset deliberately — before the consumer ever starts reading. Which brings us to offset reset.

Offset Reset Strategies

Once we understood how consumer groups and committed offsets worked, the recovery path became much clearer. We did not need to change the topic.

We needed to change where the consumer group would resume reading from. That is exactly what offset reset does.

Before running any reset command, remember one critical rule:

Your consumer group must be completely stopped.

No active consumers. No running instances. No heartbeats.

Kafka will reject offset resets for active consumer groups.

Reset to earliest

kafka-consumer-groups.sh \
 --bootstrap-server localhost:9092 \
 --group order-processor-v1 \
 --topic orders \
 --reset-offsets \
 --to-earliest \
 --execute

This resets the consumer group to the beginning of the topic.

Use this when you want:

• full replay
• state rebuilds
• reprocessing historical data
• backfills

This is also the most dangerous reset to run accidentally in production.

It is effectively: “Read everything again.”

Reset to latest

kafka-consumer-groups.sh \
 --bootstrap-server localhost:9092 \
 --group order-processor-v1 \
 --topic orders \
 --reset-offsets \
 --to-latest \
 --execute

This moves the consumer group to the latest available offset.

The consumer skips old messages and starts reading only new incoming data.

This is what we used during our incident to immediately stop replaying 7 days of old events.

Very useful during:

• replay storms
• accidental reprocessing
• recovery stabilization

Reset to a specific datetime

kafka-consumer-groups.sh \
  --bootstrap-server localhost:9092 \
  --group order-processor-v1 \
  --topic orders \
  --reset-offsets \
  --to-datetime 2026-05-20T10:00:00.000 \
  --execute

This is one of the most underrated reset strategies.

Kafka finds the first offset whose timestamp matches the provided datetime.

Useful when:

• replay should start from a known incident window partial historical recovery is needed
• you know when the problem started, but not the exact offset

Operationally, this is often safer than replaying everything from earliest.

Shift offsets forward or backward

kafka-consumer-groups.sh \
  --bootstrap-server localhost:9092 \
  --group order-processor-v1 \
  --topic orders \
  --reset-offsets \
  --shift-by -100 \
  --execute

This moves offsets relative to their current position.

Negative values move backward. Positive values move forward.

Useful for:

• small replay corrections skipping problematic records
• surgical recovery scenarios

Also easy to misuse. Always preview the result before executing large shifts.

Reset to an exact offset

kafka-consumer-groups.sh \
  --bootstrap-server localhost:9092 \
  --group order-processor-v1 \
  --topic orders:0 \
  --reset-offsets \
  --to-offset 892000 \
  --execute

This gives you complete control. Kafka positions the consumer group exactly at the provided offset.

Useful when:

• investigation identified the precise recovery point
• replay boundaries are strict
• production recovery must be tightly controlled

This is the most precise reset strategy — but also the one that requires the most operational confidence.

Always preview before executing

One final habit is worth developing early:

Use:

--dry-run

before:

--execute

A dry run shows exactly what Kafka plans to change without modifying offsets.

In production systems, this small step can prevent very large mistakes

Kafka offsets look deceptively simple at first.

Just numbers attached to messages.

But in production, offsets define:

• where recovery begins
• what gets replayed
• what gets skipped
• and sometimes, whether downstream systems survive a deployment at all.

That is exactly what our incident taught us.

A single consumer group ID change completely changed where Kafka believed the application should start reading from.

Kafka was not broken. It was doing exactly what we had configured it to do.

Once you understand:

• committed offsets
• consumer groups
• __consumer_offsets
• and reset strategies

Kafka behavior stops feeling mysterious. And starts becoming predictable.

If you found this useful, follow me on Medium for more articles on Kafka, Java, and system design from production experience.