How to Solve the Dual-Write Problem in Microservices Using the Transactional Outbox Pattern

One of the most common challenges in microservice architectures is the dual-write problem: when a service needs to update its database and publish an event to a message broker, there is no guarantee that both operations happen atomically.

Example

Consider a microservices application with an Auth service (responsible for user registration, authentication, and authorization) and a Notification service (responsible for sending notifications such as emails).

When a user registers, the Auth service must save the user’s data and publish a “UserRegistered” event so the Notification service can send an OTP verification email.

This creates a problem regardless of the order of operations:

• If the Auth service saves the user first and then crashes before publishing the event, the user is successfully created, but the OTP email is never sent.
• If the Auth service publishes the event first and then crashes before committing the database transaction, the Notification service may process an event that references data that was never committed.

In both cases, the database state and the published events become inconsistent.

Solution: Transactional Outbox Pattern

Instead of treating the database write and event publishing as two separate operations, we introduce an Outbox table in the same database.

When a user registers, the Auth service writes both the user record and an outbox record containing the event within the same database transaction. Because both writes occur in a single ACID transaction, the database guarantees that they either both commit or both roll back.

A separate message relay then reads events from the outbox table, publishes them to the message broker, and marks them as processed (or deletes/archives them).

The relay itself can be implemented in different ways, such as periodically polling the outbox table or using Change Data Capture (CDC) tools like Debezium to stream newly committed events.

Implementation Notes

At-least-once delivery

Publishing the event and marking the outbox record as processed cannot be done atomically. If the relay publishes an event but crashes before updating the outbox record, the same event will be published again after the relay restarts.

For this reason, consumers should be idempotent. Give every outbox record a unique message ID, and have consuming services keep track of processed IDs so duplicate events can be safely ignored.

Ordering

Events related to the same aggregate (for example, a user) should be published in the order they were created. The relay should read outbox records in sequence, and related events should be routed using a consistent partition or routing key (such as the aggregate ID) so the message broker preserves their ordering.

Without a consistent routing strategy, events may still be processed out of order when multiple partitions or consumer instances are involved.

Outbox maintenance

Processed events should be periodically archived or deleted to prevent the outbox table from growing indefinitely. It’s also a good practice to index the fields used by the relay (such as processing status and creation sequence) to ensure efficient scanning.

Conclusion

The Transactional Outbox pattern solves the dual-write problem by guaranteeing the atomic persistence of both business data and events without requiring distributed transactions.

While it introduces additional infrastructure such as a message relay, requires idempotent consumers, and demands careful consideration of event ordering, it is one of the most widely adopted patterns for reliably publishing events in event-driven microservice architectures.