Transactional Outbox Pattern with Spring Boot, Kafka and PostgreSQL

In an event-driven microservice architecture, a service often needs to perform two operations:

• Save business data to a database
• Publish an event to a message broker

For example, Order Service may save a new order to PostgreSQL and then publish an OrderCreatedEvent to Apache Kafka.

These operations use two different systems.

A PostgreSQL transaction cannot automatically include a Kafka message, and a Kafka transaction cannot automatically control a PostgreSQL commit.

This creates a dual-write problem.

Consider the following flow:

Save Order
    ↓
Commit Database Transaction
    ↓
Publish OrderCreatedEvent

What happens if the database transaction succeeds but the application stops before publishing the event?

The order exists in the database, but downstream services never receive the event.

The system becomes inconsistent.

This is where the Transactional Outbox Pattern becomes useful.

In this article, we will implement a polling-based Transactional Outbox Pattern using Spring Boot, Apache Kafka, PostgreSQL, and Docker Compose.

The complete source code is available on GitHub:

Spring Boot Kafka Transactional Outbox Pattern Demo

What Is the Transactional Outbox Pattern?

The Transactional Outbox Pattern stores business data and event data in the same database transaction.

Instead of publishing an event directly to Kafka, the application writes the event to an Outbox table.

Database Transaction
    ├── Insert Order
    └── Insert Outbox Event

Because both records are stored in the same PostgreSQL transaction, they either succeed together or fail together.

A separate component reads pending records from the Outbox table and publishes them to Kafka.

Client
  |
  v
Order Service
  |
  | Single PostgreSQL Transaction
  |
  +-----------> orders
  |
  +-----------> outbox_events
                     |
                     v
              Outbox Publisher
                     |
                     v
                   Kafka
                     |
                     v
             Inventory Service

If Kafka is unavailable, the Outbox event remains in PostgreSQL.

The publisher can retry it when Kafka becomes available again.

According to Spring’s explanation of the Outbox Pattern, the main objective is to coordinate database changes and message publication without using distributed two-phase commits.

Project Scenario

The project contains two Spring Boot services:

• Order Service
• Inventory Service

Order Service accepts an order request.

It stores:

• The new order in the orders table
• An OrderCreatedEvent in the outbox_events table

Both operations are executed inside the same database transaction.

The Outbox Publisher periodically checks the Outbox table.

When it finds a pending event, it publishes the serialized event to Kafka.

Inventory Service consumes the event and creates an inventory record in its own PostgreSQL database.

The complete flow is:

POST /orders
     |
     v
Order Service
     |
     | One Local Transaction
     |
     +----> orders
     |
     +----> outbox_events: NEW
                    |
                    v
            Outbox Publisher
                    |
                    v
        Kafka: outbox.order-created
                    |
                    v
           Inventory Service
                    |
                    v
       inventory_records: RESERVED

After Kafka acknowledges the message, the Outbox event status changes from NEW to PUBLISHED.

Technologies

The project uses:

• Java 17
• Spring Boot 3.5
• Spring Data JPA
• Spring Kafka
• Apache Kafka 4.1
• PostgreSQL 17
• Docker Compose
• Kafka UI

Kafka runs in KRaft mode, so ZooKeeper is not required.

Project Structure

The project is implemented as a Maven multi-module application.

spring-boot-kafka-transactional-outbox-pattern-demo
├── outbox-messages
├── order-service
├── inventory-service
├── docker-compose.yml
└── pom.xml

The root pom.xml defines the modules:

<modules>
    <module>outbox-messages</module>
    <module>order-service</module>
    <module>inventory-service</module>
</modules>

outbox-messages

This module contains the event contract and Kafka topic name shared by the services.

order-service

This service:

• Creates orders
• Stores Outbox events
• Publishes pending events to Kafka
• Retries failed event publications

inventory-service

This service:

• Consumes OrderCreatedEvent
• Creates an inventory reservation record
• Stores the result in a separate PostgreSQL database

Defining the Kafka Topic

The project uses a single Kafka topic:

public final class OutboxTopics {

    public static final String ORDER_CREATED =
            "outbox.order-created";

    private OutboxTopics() {
    }
}

Keeping topic names in a shared class prevents services from using different topic names accidentally.

Creating the Event Contract

The event contains an eventId in addition to the order information.

public record OrderCreatedEvent(
        UUID eventId,
        UUID orderId,
        String productId,
        int quantity,
        BigDecimal amount,
        Instant occurredAt
) {
}

The eventId uniquely identifies the event.

This identifier is important because the Transactional Outbox Pattern normally provides at-least-once delivery.

The same event may therefore be delivered more than once.

A production consumer can use eventId to detect duplicate messages.

Order Entity

The Order Service stores orders in its own PostgreSQL database.

@Entity
@Table(name = "orders")
public class OrderEntity {

    @Id
    private UUID id;

    private String productId;

    private int quantity;

    private BigDecimal amount;

    @Enumerated(EnumType.STRING)
    private OrderStatus status;

    private Instant createdAt;

    protected OrderEntity() {
    }

    public OrderEntity(
            UUID id,
            String productId,
            int quantity,
            BigDecimal amount
    ) {
        this.id = id;
        this.productId = productId;
        this.quantity = quantity;
        this.amount = amount;
        this.status = OrderStatus.CREATED;
        this.createdAt = Instant.now();
    }
}

The demo uses a simple order status:

public enum OrderStatus {
    CREATED
}

Outbox Event Entity

The Outbox table stores the serialized event and the information required to publish it.

@Entity
@Table(name = "outbox_events")
public class OutboxEvent {

    @Id
    private UUID id;

    private String aggregateType;

    private UUID aggregateId;

    private String eventType;

    private String topic;

    @Lob
    @Column(columnDefinition = "TEXT", nullable = false)
    private String payload;

    @Enumerated(EnumType.STRING)
    private OutboxStatus status;

    private int retryCount;

    private Instant createdAt;

    private Instant publishedAt;

    @Column(length = 1000)
    private String lastError;

    @Version
    private long version;
}

The most important fields are:

• id: Unique event identifier
• aggregateId: Order identifier
• eventType: Type of the stored event
• topic: Kafka destination topic
• payload: Serialized JSON event
• status: Current publication status
• retryCount: Number of failed attempts
• lastError: Latest publication error
• publishedAt: Successful publication time

The event statuses are:

public enum OutboxStatus {
    NEW,
    PUBLISHED,
    FAILED
}

Their meanings are:

NEW        Event is waiting to be published
PUBLISHED  Kafka acknowledged the message
FAILED     The latest attempt failed and will be retried

Saving the Order and Outbox Event Together

The most important part of the implementation is the transaction boundary.

@Service
public class OrderApplicationService {

    private final OrderRepository orderRepository;
    private final OutboxEventRepository outboxRepository;
    private final ObjectMapper objectMapper;

    public OrderApplicationService(
            OrderRepository orderRepository,
            OutboxEventRepository outboxRepository,
            ObjectMapper objectMapper
    ) {
        this.orderRepository = orderRepository;
        this.outboxRepository = outboxRepository;
        this.objectMapper = objectMapper;
    }

    @Transactional
    public OrderEntity createOrder(
            String productId,
            int quantity,
            BigDecimal amount
    ) {
        UUID orderId = UUID.randomUUID();
        UUID eventId = UUID.randomUUID();
        Instant now = Instant.now();

        OrderEntity order = orderRepository.save(
                new OrderEntity(
                        orderId,
                        productId,
                        quantity,
                        amount
                )
        );

        OrderCreatedEvent event = new OrderCreatedEvent(
                eventId,
                orderId,
                productId,
                quantity,
                amount,
                now
        );

        try {
            String payload =
                    objectMapper.writeValueAsString(event);

            outboxRepository.save(
                    new OutboxEvent(
                            eventId,
                            orderId,
                            OrderCreatedEvent.class
                                    .getSimpleName(),
                            OutboxTopics.ORDER_CREATED,
                            payload
                    )
            );
        } catch (JsonProcessingException exception) {
            throw new IllegalStateException(
                    "Could not serialize the order event",
                    exception
            );
        }

        return order;
    }
}

The method does not call Kafka.

It only writes to PostgreSQL.

Because the method is annotated with @Transactional, inserting the order and inserting the Outbox event belong to the same local transaction.

If event serialization or the Outbox insert fails, the order transaction is also rolled back.

The database therefore cannot contain a successfully created order without its corresponding Outbox event.

Creating the Order Endpoint

The controller accepts a product ID, quantity, and amount.

@RestController
@RequestMapping("/orders")
public class OrderController {

    private final OrderApplicationService service;
    private final OrderRepository repository;

    public OrderController(
            OrderApplicationService service,
            OrderRepository repository
    ) {
        this.service = service;
        this.repository = repository;
    }

    @PostMapping
    @ResponseStatus(HttpStatus.CREATED)
    public OrderEntity create(
            @Valid @RequestBody CreateOrderRequest request
    ) {
        return service.createOrder(
                request.productId(),
                request.quantity(),
                request.amount()
        );
    }

    @GetMapping
    public List<OrderEntity> list() {
        return repository.findAll();
    }

    public record CreateOrderRequest(
            @NotBlank String productId,
            @Min(1) int quantity,
            @NotNull @DecimalMin("0.01")
            BigDecimal amount
    ) {
    }
}

The request is validated before the transaction begins.

Finding Pending Outbox Events

The Outbox repository retrieves both new and previously failed events.

public interface OutboxEventRepository
        extends JpaRepository<OutboxEvent, UUID> {

    List<OutboxEvent>
    findTop50ByStatusInOrderByCreatedAtAsc(
            Collection<OutboxStatus> statuses
    );
}

The publisher processes a maximum of 50 events during each polling cycle.

Both statuses are included:

List.of(
    OutboxStatus.NEW,
    OutboxStatus.FAILED
)

This allows previously failed events to be retried automatically.

Publishing Events to Kafka

The Outbox Publisher runs periodically using Spring’s @Scheduled annotation.

@Component
public class OutboxPublisher {

    private static final Logger log =
            LoggerFactory.getLogger(
                    OutboxPublisher.class
            );

    private static final List<OutboxStatus>
            PENDING_STATUSES = List.of(
                    OutboxStatus.NEW,
                    OutboxStatus.FAILED
            );

    private final OutboxEventRepository repository;
    private final KafkaTemplate<String, String>
            kafkaTemplate;

    public OutboxPublisher(
            OutboxEventRepository repository,
            KafkaTemplate<String, String> kafkaTemplate
    ) {
        this.repository = repository;
        this.kafkaTemplate = kafkaTemplate;
    }

    @Scheduled(
        fixedDelayString =
            "${outbox.publisher.fixed-delay:3000}"
    )
    @Transactional
    public void publishPendingEvents() {
        repository
                .findTop50ByStatusInOrderByCreatedAtAsc(
                        PENDING_STATUSES
                )
                .forEach(this::publish);
    }
}

The publisher sends the stored JSON payload to the topic recorded in the Outbox row.

private void publish(OutboxEvent event) {
    try {
        kafkaTemplate.send(
                event.getTopic(),
                event.getAggregateId().toString(),
                event.getPayload()
        ).get(10, TimeUnit.SECONDS);

        event.markPublished();

        log.info(
                "Published outbox event {} for order {}",
                event.getId(),
                event.getAggregateId()
        );
    } catch (Exception exception) {
        event.markFailed(exception);

        log.warn(
                "Could not publish outbox event {}; " +
                "it will be retried",
                event.getId()
        );
    }
}

The order ID is used as the Kafka message key.

When Kafka acknowledges the message, the event is marked as PUBLISHED.

public void markPublished() {
    this.status = OutboxStatus.PUBLISHED;
    this.publishedAt = Instant.now();
    this.lastError = null;
}

If publication fails, the event is not deleted.

public void markFailed(Throwable exception) {
    this.status = OutboxStatus.FAILED;
    this.retryCount++;
    this.lastError = exception.getMessage();
}

The next polling cycle selects the failed event again.

Order Service Configuration

The Order Service uses PostgreSQL and a String-based Kafka producer.

server:
  port: 8080

spring:
  application:
    name: order-service

  datasource:
    url: ${DB_URL:jdbc:postgresql://localhost:5433/orders}
    username: ${DB_USERNAME:orders}
    password: ${DB_PASSWORD:orders}

  jpa:
    hibernate:
      ddl-auto: update
    open-in-view: false

  kafka:
    bootstrap-servers:
      ${KAFKA_BOOTSTRAP_SERVERS:localhost:9092}

    producer:
      key-serializer:
        org.apache.kafka.common.serialization.StringSerializer
      value-serializer:
        org.apache.kafka.common.serialization.StringSerializer

      properties:
        acks: all
        max.block.ms: 5000
        request.timeout.ms: 5000
        delivery.timeout.ms: 10000

outbox:
  publisher:
    fixed-delay:
      ${OUTBOX_PUBLISHER_DELAY:3000}

The publisher runs approximately every three seconds.

The value is configurable using the OUTBOX_PUBLISHER_DELAY environment variable.

Consuming the Event

Inventory Service consumes the JSON payload as a String and converts it to OrderCreatedEvent.

@Component
public class OrderCreatedListener {

    private static final Logger log =
            LoggerFactory.getLogger(
                    OrderCreatedListener.class
            );

    private final InventoryRepository repository;
    private final ObjectMapper objectMapper;

    public OrderCreatedListener(
            InventoryRepository repository,
            ObjectMapper objectMapper
    ) {
        this.repository = repository;
        this.objectMapper = objectMapper;
    }

    @KafkaListener(
            topics = OutboxTopics.ORDER_CREATED
    )
    @Transactional
    public void onOrderCreated(String payload)
            throws JsonProcessingException {

        OrderCreatedEvent event =
                objectMapper.readValue(
                        payload,
                        OrderCreatedEvent.class
                );

        if (repository.existsById(event.orderId())) {
            log.info(
                    "Order {} was already processed",
                    event.orderId()
            );
            return;
        }

        repository.save(
                new InventoryRecord(
                        event.orderId(),
                        event.eventId(),
                        event.productId(),
                        event.quantity()
                )
        );

        log.info(
                "Reserved inventory for order {} " +
                "from outbox event {}",
                event.orderId(),
                event.eventId()
        );
    }
}

The demo includes a simple duplicate check based on orderId.

If the order was already processed, Inventory Service does not create another record.

For a production system, this should normally be implemented using a durable Idempotent Consumer or Inbox Pattern based on eventId.

Docker Compose Environment

The complete environment contains six containers:

• Order Service
• Inventory Service
• Order PostgreSQL
• Inventory PostgreSQL
• Apache Kafka
• Kafka UI

The services are available at:

Order Service      http://localhost:8090
Inventory Service  http://localhost:8091
Kafka UI           http://localhost:8088
Kafka              localhost:9092
Order Database     localhost:5433
Inventory Database localhost:5434

Order Service intentionally depends only on Order PostgreSQL.

It does not require Kafka to be available before starting.

This is important for the failure demonstration because orders should still be accepted while Kafka is temporarily unavailable.

Running the Project

First, clone the repository:

git clone \
https://github.com/erkandemirel/spring-boot-kafka-transactional-outbox-pattern-demo.git

Open the project directory:

cd spring-boot-kafka-transactional-outbox-pattern-demo

Build all Maven modules:

mvn clean package

Start the complete environment:

docker compose up --build

Check the running containers:

docker compose ps

Testing the Normal Flow

Create an order:

curl -X POST http://localhost:8090/orders \
  -H "Content-Type: application/json" \
  -d '{
    "productId": "product-100",
    "quantity": 2,
    "amount": 249.90
  }'

The response contains the created order:

{
  "id": "02972e1b-e6d5-4178-8042-04703cd07e5c",
  "productId": "product-100",
  "quantity": 2,
  "amount": 249.90,
  "status": "CREATED"
}

Immediately after the request, the database contains:

orders
└── Order status: CREATED

outbox_events
└── Outbox status: NEW

Wait a few seconds for the publisher.

List the Outbox events:

curl http://localhost:8090/outbox-events

The status should become:

PUBLISHED

List the inventory records:

curl http://localhost:8091/inventory-records

The final state is:

Order              CREATED
Outbox Event       PUBLISHED
Inventory Record   RESERVED

The Kafka topic can be inspected using Kafka UI:

http://localhost:8088

The topic name is:

outbox.order-created

Testing Kafka Failure

The most important test is creating an order while Kafka is unavailable.

Stop only the Kafka container:

docker compose stop kafka

Order Service and PostgreSQL remain available.

Create another order:

curl -X POST http://localhost:8090/orders \
  -H "Content-Type: application/json" \
  -d '{
    "productId": "product-offline",
    "quantity": 3,
    "amount": 399.00
  }'

The request still succeeds.

Check the orders:

curl http://localhost:8090/orders

The order exists because creating an order does not require a direct Kafka call.

Now check the Outbox table:

curl http://localhost:8090/outbox-events

The latest event has the following state:

status: FAILED
retryCount: 1
lastError: Kafka connection error

The event has not been lost.

Its complete JSON payload remains in PostgreSQL.

Testing Automatic Recovery

Start Kafka again:

docker compose start kafka

After Kafka becomes available, the Outbox Publisher selects the failed event again.

The state changes from:

FAILED → PUBLISHED

Check the events:

curl http://localhost:8090/outbox-events

Then check Inventory Service:

curl http://localhost:8091/inventory-records

The inventory record should now exist:

Inventory status: RESERVED

The complete recovery flow is:

Kafka Unavailable
       |
       v
Order + Outbox Event Saved
       |
       v
Outbox Status: FAILED
       |
       v
Kafka Started
       |
       v
Publisher Retries Event
       |
       v
Outbox Status: PUBLISHED
       |
       v
Inventory Status: RESERVED

This demonstrates the main benefit of the Transactional Outbox Pattern.

The event does not need to be recreated from the order because the original event payload was stored safely during the original database transaction.

Inspecting PostgreSQL

The Order database can be opened from pgAdmin using:

Host: localhost
Port: 5433
Database: orders
Username: orders
Password: orders

The database contains:

orders
outbox_events

The Inventory database connection is:

Host: localhost
Port: 5434
Database: inventory
Username: inventory
Password: inventory

It contains:

inventory_records

The Outbox table allows us to inspect:

• Pending events
• Successfully published events
• Failed events
• Retry counts
• Publication timestamps
• Latest errors
• Serialized event payloads

Delivery Semantics

This implementation provides at-least-once delivery.

Consider the following situation:

Publish Event to Kafka
        ↓
Kafka Acknowledges Event
        ↓
Application Stops
        ↓
Outbox Status Was Not Updated

Kafka contains the event, but the Outbox row may still have the NEW or FAILED status.

After restarting, the publisher sends the event again.

This means that the Transactional Outbox Pattern prevents lost events, but it does not automatically prevent duplicate events.

Consumers must be idempotent.

An idempotent consumer should produce the same result when it receives the same event multiple times.

For example, Inventory Service should not reserve the same inventory twice.

Polling Publisher Advantages

The polling approach is simple to understand and implement.

Its advantages include:

• No additional infrastructure is required
• Events remain visible in PostgreSQL
• Failed events can be retried
• Operational state is easy to inspect
• It works with standard Spring Data JPA
• The polling interval can be configured

For a demonstration project or a moderate event volume, polling provides a clear implementation of the pattern.

Polling Publisher Challenges

The polling approach also introduces several considerations:

• Frequent polling increases database activity
• A long polling interval increases event latency
• Multiple publisher instances may select the same rows
• Published records must eventually be archived or deleted
• Failed events need backoff and alerting
• Duplicate delivery must be handled by consumers
• Large event payloads increase Outbox table size

Another implementation option is Change Data Capture.

A tool such as Debezium can read committed changes from the PostgreSQL transaction log and publish Outbox events to Kafka.

That approach can be explored in a separate project.

Production Considerations

This project intentionally focuses on the core Transactional Outbox flow.

A production-ready implementation should also consider the following patterns and features.

Idempotent Consumer or Inbox Pattern

Consumers should persist processed eventId values.

Before applying a business operation, the consumer should check whether the event was previously processed.

The event processing result and processed-event record should be committed in the same local database transaction.

Safe Event Claiming

When multiple Outbox Publisher instances run simultaneously, they may select the same event.

Rows should be claimed safely using a mechanism such as:

SELECT ...
FOR UPDATE SKIP LOCKED

Another option is assigning a temporary processing status and lease expiration time.

Retry Backoff

Failed events should not be retried continuously without delay.

A production implementation should include:

• Exponential backoff
• Maximum retry limits
• Next attempt timestamp
• Failure metrics
• Operational alerts

Outbox Cleanup

Published events should eventually be archived or deleted.

Cleanup should be performed in batches to avoid creating large database transactions.

Database Migrations

The demo uses Hibernate automatic schema updates.

Production applications should normally manage database schemas with tools such as Flyway or Liquibase.

Observability

Useful Outbox metrics include:

• Number of NEW events
• Number of FAILED events
• Oldest unpublished event age
• Publication success rate
• Retry count
• Publishing duration

The eventId and aggregateId should also be included in logs and distributed traces.

Event Schema Versioning

Event structures change over time.

Events should include schema version information, and producers should avoid publishing internal JPA entities directly.

Change Data Capture

For systems with higher event volumes, Change Data Capture can publish Outbox rows without application-level polling.

Debezium is commonly used to capture committed PostgreSQL changes and send them to Kafka.

Transactional Outbox and Saga Pattern

The Transactional Outbox Pattern and Saga Pattern solve different problems.

Saga Pattern manages a distributed business transaction across multiple services.

Transactional Outbox Pattern ensures that a service’s committed database change has a corresponding event available for publication.

They are often used together.

For example:

Order Service
    |
    | Save Order + OrderCreatedEvent
    | in one local transaction
    v
Transactional Outbox
    |
    v
Kafka
    |
    v
Saga Participants

In the previous Saga project, Order Service saved an order and published an event directly.

Adding the Transactional Outbox Pattern closes the consistency gap between those two operations.

Saga manages business consistency across services.

Transactional Outbox manages reliable event publication inside each service.

Conclusion

The Transactional Outbox Pattern solves the dual-write problem that appears when a microservice needs to update its database and publish an event.

In this example:

• Order Service stores an order.
• The same transaction stores an Outbox event.
• A scheduled publisher reads pending events.
• Kafka receives the serialized event.
• Inventory Service consumes the event.
• Failed publications remain in PostgreSQL.
• Events are retried after Kafka recovers.

The pattern does not create a distributed transaction between PostgreSQL and Kafka.

Instead, it guarantees that the information required to publish the event is committed together with the business data.

This prevents database changes from becoming invisible to downstream services.

The complete source code is available on GitHub:

Spring Boot Kafka Transactional Outbox Pattern Demo

Thanks for reading.

Happy coding!