Data consistency patterns comparisons, 2PC vs SAGA

Hello there, in this article I will try to explain data consistency and the two popular patterns used to achieve it and their differences.

What is data consistency meaning?

Data consistency is the effort to ensure that all the data that has been fragmented across distributed systems is seamless, consistent and complete when it is reassembled. This also applies to the consistency of the same data distributed across multiple systems.

Why we need to data consistency?

Applications in different systems that use the same data for a critical operation in their own system’s decision-making process can cause critical errors because the data they use is inconsistent.

In some cases, data consistency can be ignored when critical operations are not being performed and the data does not need to be consistent at that time.

How to avoid data inconsistency?

According to the CAP theorem.), in distributed systems you must sacrifice some system features if you want to ensure data consistency. After analyzing your system, if data consistency is a priority for you, there are two main types of patterns used by the community to ensure it.

Both patterns have their own advantages and disadvantages and are the optimal choice for different scenarios.

Two-Phase Commit (2PC)

As the name suggests, 2PC is built in two phases. The first is the preparation of the data on each system. The purpose of this preparation is to verify that the distributed data is correct on each system. Then, in a second step, the data that has been confirmed to be correct on each system is processed. The first of these phases is called the prepare phase, and the second is called the commit phase.

Preparing phase

During the prepare phase, a coordinator waits for feedback from each system to which the data is being distributed. If the systems report that there are no problems, the second phase begins. However, if any system returns an error message, the process is aborted. In this way, the data is processed atomically in all systems at the same time.

Commit Phase

After receiving positive feedback from all participants, the coordinator contacts each participant to make the changes permanent. As a result of this connection, the participants approve the action and send this feedback to the coordinator. If there is an error between the feedbacks, or if the expected feedback is not received (timeout), the coordinator decides what to do next. It must either rollback or trigger a retry mechanism. In this case, if an error occurs, it will eventually have to rollback.

Why you should use?

You can use the 2PC pattern when you want your system to behave atomically. You can also use it in distributed systems in scenarios where you want to write and retrieve data simultaneously in batches. However, because the operations are performed in batches, the system is locked down during these operations. While this is a plus in terms of reliability, it can be a minus in terms of scalability and performance. As the system grows, complexity increases.

SAGA

The SAGA pattern helps to process data sequentially in distributed systems. With this pattern, there is no blocking in the whole system, each system checks the trigger status of the next system and triggers it. This is done using the asynchronous messaging pattern. If an error occurs, the last triggered system — the one that received the error — backtracks and creates a new chain that rolls back the improvements made by the previous systems. Depending on the number of systems, the approaches to using SAGA may also change. Saga Orchestration and Saga Choreography are the two most popular approaches.

SAGA Choreography

The SAGA pattern is the first method that comes to mind. Systems process data sequentially. The first system sends a “go ahead” signal to the next system after it has processed the data. The next system that receives this message does its own processing on the data and sends it to the next system, just like the message sent by the previous system. The flow continues in this manner. If there is an error, the flow is reversed and the message becomes a rollback message. It is preferred when the number of systems is small. It is difficult to test and debug the entire system.

SAGA Orchestration

In the Saga orchestration pattern, sequential operations are still performed, but under the control of an orchestrator. When the orchestrator is alerted to a process, it begins alerting systems in sequence. After the first system is alerted and provides feedback, the orchestrator moves on to the next system. The orchestrator can keep a history of the process by writing the state of the system to which it sent the message. If an error occurs, the orchestrator sends a rollback message to the systems to which it sent messages until the error occurred. Each system does its own job and does not need to send messages to any other system except the orchestrator. Because the system is managed by the orchestrator, it is easier to test. Systems can work in isolation, they do not need to know about each other (“Separation of Concerns”). The orchestrator can call the systems, but the systems should not call the orchestrator.

Why you should use?

It can be preferred for a more scalable and performant system. It is also an alternative that can be used as the number of systems increases. Although it provides an atomic structure, it should not be preferred in critical situations that require consistency when reading data, since it uses an asynchronous messaging pattern. As the number of services increases, debugging and testing becomes more complex. Rollback can be solved with easier, simpler, and more applicable approaches.

In this article I have tried to explain data consistency and the two main patterns used to ensure data consistency, I hope it was useful, thank you for reading.

Resources

• https://microservices.io/patterns/data/saga.html
• https://martinfowler.com/articles/patterns-of-distributed-systems/two-phase-commit.html
• https://www.baeldung.com/cs/two-phase-commit-vs-saga-pattern