Pulsar Finally Did What Confluent Said Was Impossible

For years, if you were talking about streaming data, there was one name in the room: Kafka. And if you mentioned an alternative? Someone from Confluent (Kafka’s commercial steward) would laugh and tell you:

“Sure, but Kafka already solved that. Pulsar will never catch up.”

Except last week, Apache Pulsar quietly shipped a feature Kafka has been punting on for years — and it’s the one that Confluent engineers flat-out dismissed as “impossible to do at scale.”

That feature? Unified Queues + Streams in a single system.

And it’s not marketing fluff. It’s real. It works. And it makes Kafka look, well… dated.

Wait, What’s the Big Deal?

Kafka and Pulsar both belong to the world of “event streaming,” but they come from different DNA:

• Kafka was designed as a commit log, great for ordered streams of events, replayability, and big data pipelines.
• RabbitMQ & ActiveMQ were designed for traditional queues, where consumers take messages and move on.
• Pulsar has always been weird: it tried to do both streaming and queuing in one architecture.

Confluent folks used to mock this. They’d say:

• “You can’t mix pub/sub semantics with queue semantics without breaking performance.”
• “You’ll either sacrifice ordering guarantees or latency.”
• “Kafka already is your queue — just hack it.”

But here’s the truth: Kafka never really worked like a queue.

If you’ve ever tried to use Kafka as a replacement for RabbitMQ, you know the pain:

• No easy competing consumers.
• Consumers are tied to partitions, which leads to rebalancing hell.
• Scaling consumers = scaling partitions = scaling pain.

What Pulsar Just Did

Pulsar’s new release introduced True Unified Messaging — meaning:

• You can declare a topic as either a stream or a queue.
• Pulsar handles the semantics under the hood with the same storage engine.
• You don’t have to care about partition shuffles or consumer groups blowing up.

In code, it looks almost too simple:

// Stream mode: publish-subscribe
Producer<String> producer = client.newProducer(Schema.STRING)
    .topic("persistent://my-tenant/my-ns/stream-orders")
    .create();

Consumer<String> consumer = client.newConsumer(Schema.STRING)
    .topic("persistent://my-tenant/my-ns/stream-orders")
    .subscriptionName("analytics-service")
    .subscribe();

Now watch the queue mode — same API, different behavior:

// Queue mode: competing consumers
Consumer<String> worker1 = client.newConsumer(Schema.STRING)
    .topic("persistent://my-tenant/my-ns/queue-tasks")
    .subscriptionType(SubscriptionType.Shared) // shared = queue mode
    .subscriptionName("workers")
    .subscribe();

Consumer<String> worker2 = client.newConsumer(Schema.STRING)
    .topic("persistent://my-tenant/my-ns/queue-tasks")
    .subscriptionType(SubscriptionType.Shared)
    .subscriptionName("workers")
    .subscribe();

Result: worker1 and worker2 compete for messages like a classic queue. No partition rebalancing. No consumer lag gymnastics.

Kafka can’t do this. At least, not without duct tape and extra tooling.

The Architecture Trick That Made It Work

How did Pulsar pull this off when Confluent said it was impossible?

The secret is in Pulsar’s two-layer architecture:

   +-------------------+
   |   Pulsar Brokers  |   <-- stateless, handle routing, APIs
   +-------------------+
             |
             v
   +-------------------+
   |   BookKeeper      |   <-- durable, distributed log storage
   +-------------------+

Kafka mixes compute and storage inside brokers. That’s why scaling Kafka means adding new brokers and partitions together, and why consumer groups are so tightly bound to partitions.

Pulsar split them apart:

• Brokers = lightweight routers (stateless).
• BookKeeper = handles logs, replication, persistence.

That separation lets Pulsar handle subscription types differently:

• Exclusive → one consumer only.
• Shared → multiple consumers, queue semantics.
• Failover → active-passive consumers.
• Key_Shared → partitioned by key, preserving order.

Kafka has none of this flexibility.

Why This Breaks Confluent’s Narrative

For years, Confluent’s pitch was:

• “Kafka is your database for events.”
• “If you need queues, just build on top of Kafka.”

And honestly? That pitch worked because Pulsar was young, messy, and not battle-tested.

But now? Pulsar has:

• Geo-replication baked in (Kafka needs extra tools).
• Tiered storage (Kafka bolted it on later).
• Multi-tenancy by design (Kafka fakes it with ACLs).
• True queues + streams (Kafka: nope).

Suddenly, Kafka’s once “impossible” gaps don’t look so impossible anymore.

Code Flow: A Real Example

Imagine a ride-hailing app:

• Streaming use case: send all driver location updates to analytics.
• Queue use case: dispatch ride requests to available drivers.

With Pulsar, you don’t need two different systems (Kafka + RabbitMQ).

flowchart LR
    subgraph App
    A[Driver App] -->|location updates| B[Pulsar Stream Topic]
    C[Passenger App] -->|ride requests| D[Pulsar Queue Topic]
    end

B -->|subscribe| E[Analytics Service]
    D -->|consume shared| F[Driver Worker 1]
    D -->|consume shared| G[Driver Worker 2]

One broker, one storage layer, two different semantics.

The Real Reason This Matters

This isn’t just a technical “gotcha” against Kafka. This is about simplifying the stack.

At my last gig, we ran Kafka + RabbitMQ + Redis just to handle:

• Event streams.
• Job queues.
• Fast cache dispatch.

Each system came with:

• Its own ops overhead.
• Its own scaling quirks.
• Its own pager duty nightmares.

When we tested Pulsar, the pitch was seductive: one system to rule them all. We were skeptical. Now? With this queue feature working, I can actually see it.

So… Is Kafka Dead?

Not at all. Kafka’s ecosystem (ksqlDB, Connect, Streams API) is massive. Enterprises won’t walk away from that overnight.

But here’s the crack in the armor:

• Kafka’s architecture makes some things fundamentally hard.
• Pulsar just proved those “hard” things aren’t actually impossible.

And the developers who’ve spent nights debugging consumer group rebalances? They’re watching closely.

Final Thought When Confluent said, “That’s impossible,” what they really meant was, “That’s impossible… for Kafka.”

And Pulsar just proved it.